From 7a90af89562def0ba36622cc26e435b0b274cc58 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 17:17:15 -0700 Subject: [PATCH 001/493] docs: plan managed provider environment and recovery fixes --- AGENTS.md | 2 + ...2026-10-05-managed-provider-environment.md | 295 ++++++++++++++++++ 2 files changed, 297 insertions(+) create mode 100644 docs/plans/2026-10-05-managed-provider-environment.md diff --git a/AGENTS.md b/AGENTS.md index 7c0dd240c..a38543117 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -325,4 +325,6 @@ All components **must** be accessible for browser-use automation and WCAG compli ## Subagent polling policy +Managed provider environment and recovery repair: [implementation plan](docs/plans/2026-10-05-managed-provider-environment.md). + When idle-waiting on subagents, always call `wait_agent` with `timeout_ms=3600000` (the 1-hour maximum). `wait_agent` wakes immediately on any subagent message or completion, so long timeouts add no latency — but every wait return (including a no-op timeout) costs a full model turn that re-sends the entire context. Never poll on shorter timeouts. Have workers `send_message` to `/root` at milestones so the parent wakes exactly when there is news. diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md new file mode 100644 index 000000000..3aa0c4cc8 --- /dev/null +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -0,0 +1,295 @@ +# Managed Provider Environment and Recovery Implementation Plan + +> **For agentic workers:** Execute this plan task by task with a fresh +> implementer and a specification-plus-quality review after every task. Track +> progress with the checkbox steps below. + +## User Request + +### Requested result +Fix all managed provider environment defects tracked in freshell#bdpt so OpenCode starts and is usable with its existing configuration, and supported coding providers retain executable skills, resources, plugin dependencies, and resolvable configuration file references. Fix the downstream recovery defects recorded in the same kata: unsupported checkpoint-absence decisions and stale running terminal status with repeated warnings. + +### Explicit constraints +- Fix both the controller staging and session-host copying stages with consistent rules, meaningful resource bounds, actionable errors, and structured logs. +- Preserve executable permissions, supporting skill resources, and plugin dependencies; do not silently truncate environments or disable configured capabilities. +- Resolve absolute and relative configured file references, including OpenCode {file:...}; transport credential references through existing OneCLI/managed secret mechanisms without persisting or logging secret contents. +- Distinguish unrecorded identity/checkpoint metadata from confirmed provider-store absence; incomplete evidence must block recovery with a clear reason and preserve data, without silently replacing dispatched conversations. +- Propagate final recovery outcomes to accurate terminal status, retain output and recovery information, and log structured state transitions instead of repeating unchanged failures on every poll; preserve successful recovery and reattachment. +- Cover startup failure before identity capture, metadata-only negatives, confirmed absence, final Lost/stopped and blocked outcomes, retained output, bounded logging, and successful recovery with behavioral regression, integration/smoke, and affected browser e2e tests; destructive coverage runs only in the documented disposable sandbox. +- Use behavioral red/green/refactor tests, complete environment integration/smoke coverage, and affected browser e2e tests on the configured backend; exercise executables, resources, dependency loading, file resolution, limits, and failure cleanup using synthetic fixtures. +- Follow the-usual in a dedicated worktree; preserve unrelated work, keep changes in focused commits, and track the work in freshell#bdpt. +- Do not open a PR without explicit approval, and do not restart or deploy the live self-hosted Freshell server without the user's explicit APPROVED. +- After an approved supported deployment, recheck the original OpenCode failure through Freshell MCP without inspecting secret contents. + +### Accepted tradeoffs and residuals +- The separate POST /api/fresh-agent/checkpoints HTTP 500 concerns code-rewind snapshots and is excluded; its cause remains undiagnosed. +- Darkforge installation cleanup and installer repair are complete separately and are excluded from this Freshell change. +- Live deployment and its final MCP check remain pending separate restart approval. + +**Goal:** A managed OpenCode pane starts with its existing supported configuration, every supported provider can execute its configured skills and resources, and failed managed launches recover conservatively with accurate status and retained diagnostics. + +**Architecture:** Put bounded resource copying in the existing platform crate and use it at controller staging, initial projection, and resume refresh. Resolve known configuration references at their original source, transport potentially sensitive file references through existing managed secret mounts, and rewrite only run-owned copies. Carry inspected negative evidence and typed recovery outcomes through the existing recovery and terminal interfaces; retain output through the existing spool and expose blocked/final state through the existing inventory summary. + +**Tech Stack:** Rust 1.96, Axum/Tokio, existing runtime protocol and Docker supervisor, React/Redux, pnpm 10.34.5, Vitest, Playwright, Cargo integration tests. + +## Global Constraints + +1. The clean `origin/main` baseline passed before this feature worktree was created at `d0cc7cb5e038811be0fb3ff5f4fc5812d2723228`. Author all changes on `the-usual/managed-provider-environment` in `.worktrees/managed-provider-environment`; this agent plan is linked from `AGENTS.md`. Preserve unrelated work and keep workflow reports outside the tracked tree. +2. Run pnpm frozen installs, existing coordinated commands, configured cloud Vitest/browser backends, and `GCLOUD_ROBOT_REQUIRE=1` for broad agent gates. Destructive process/restart/corruption tests run only in the documented disposable sandbox. The live server, real user configuration, credential contents, and unrelated agent work are outside this implementation's mutation scope. +3. All four providers use one copy policy. Defaults are 100,000 entries, 512 MiB aggregate regular-file bytes, 128 MiB per regular file, and depth 64, with checked arithmetic. Tests inject smaller bounds; no separate controller/host environment knobs or lower hidden ownership limit. Secrets retain their existing stricter OneCLI limits; referenced configuration files use an explicit bounded file profile. +4. A selected missing, unreadable, cyclic, or unsupported resource is an actionable error, never an omitted configured capability. Optional provider roots absent before selection remain optional. Exclude concrete native state/cache/history and actual credential paths, not arbitrary basenames within dependencies. +5. Preserve executable bits while keeping files private: `0600 | (source_mode & 0111)`. Resume helper access uses `0640 | (source_mode & 0111)` and directories 0750, then final projection restores the private form. No setuid/setgid/sticky preservation. Provider-owned state is excluded from projection authority. +6. Never expand secret file contents into staged configuration, registry/SQLite records, grant manifests, Docker JSON, logs, or test receipts. Persist only validated reference metadata. Known configuration syntax and declaring origin control resolution; never rewrite arbitrary strings that look like paths. +7. Metadata absence is unknown evidence. Loss requires actual authoritative native/checkpoint store inspection for every required recovery path, with typed path evidence. Legacy negatives without inspected evidence cannot authorize loss or cleanup. Proven pristine seed recovery and exact existing identities remain supported. +8. Blocked status uses additive `runtimeStatus: blocked` and the existing `recoverySummary`. Keep the internal facade attachable/replayable with its terminal/session identity. Every user-facing process listing, including `/api/terminals`, MCP `list-terminals`, CLI `list-terminals` and client inventory, must distinguish blocked from running even if the internal facade remains Running for attachability. Lost/Stopped set facade Exited only after retained final output drains; no new recovery UI or unrelated protocol framework. +9. Live deployment follows `docs/development/managed-runtime-rollout.md`, requires the user's separate `APPROVED`, and includes the managed-feature server, supervisor, session host and correct image. No PR is opened before explicit branch approval. The post-deployment MCP check remains a separate authorized operation. + +## Load-bearing validation before implementation + +The parent performs Stage 2 validation of the following facts and revises this draft before executing a task whose inputs change. Validation uses disposable synthetic homes and public pinned provider sources, never real credential contents. + +1. Verify the pinned providers' actual path-bearing forms. OpenCode 1.18.21 first performs one env substitution pass, then resolves file tokens; missing/empty variables become empty strings, comment-prefixed file tokens are skipped, local file origins are their declaring parent and inline origin is instance cwd. The follow-up `reports/plan-env-file-resolution.md` supplies a concrete pre-mount transport phase and shared-parser design for nested env-dependent file paths and env-introduced file tokens. Validate the synthetic original request/server environment ordering, loader-source-specific authEnv precedence, original HOME/runtime CA mapping, raw JSONC source spans, and changed-grant detection described in Task 2. For Claude/Codex/Amplifier, record exact supported fields already consumed by their pinned implementations and the existing resource roots they load. Do not invent a universal configuration schema. Task 2's syntax adapters require that concrete supported-field table before execution. +2. Prove resource symlink closure supports external linked skills, `.bin`, and package-store dependency cycles without recursing forever, losing imports, or exposing unrelated HOME. Verify bounds against a synthetic dependency tree exceeding the former 4096 entries. The controller and host both consume the shared implementation. +3. Prove project config overlays are the files OpenCode actually loads from the original mounted workspace, including ancestor discovery, nested cwd, inline and TUI references; the unused HOME project copies are insufficient. Validate overlays in both creation and adoption mount proofs. +4. Confirm native/checkpoint shared-store policies from the adapters, independent fixture checkpoint inspection, retained spool availability after stop/cleanup, and multi-page final output draining. Audit all `status == Running`, client running-count/clear-dead consumers, server `/api/terminals` directory and paged listings (`crates/freshell-server/src/terminals.rs`), MCP `list-terminals` (`tools/freshell-mcp/freshell-tool.ts:678`) and CLI `list-terminals` (`tools/freshell-cli/index.ts:945`) for the chosen blocked-facade policy. These tools currently return the API projection unchanged: correcting only browser `recoverySummary` leaves their top-level status falsely running. Keep attachability internal while projecting process state distinctly in every listing. +5. Validate the actual configured-cloud managed browser path. Current cloud image has no managed-feature server/Docker rig and excludes managed qualification specs. The proposed narrow path is a run-owned local managed fixture server plus an ephemeral Cloudflare Quick Tunnel and allowlisted external-target forwarding to Cloud Run. It must prove Cloud Run health/WebSocket reachability, exact candidate provenance, real controller→session-host preparation, synthetic-only data, and owned cleanup. Download/run a task-local tunnel binary; do not modify DNS, system services, production servers, or permanent routing. Root owns the feasibility audit and updated report. + +**UNRESOLVED COVERAGE GAP until item 5 passes:** A configured-cloud browser must create an actual managed OpenCode CLI pane whose child executes both production preparation stages with representative configuration, executable resources, dependency-loaded plugins, and configured file references. Recovery UI fixtures, an ordinary CLI fixture, local-only skipped qualification, and split Cargo preparation tests are useful but cannot clear this complete requirement. If no approved usable topology is established, retain FAILED/non-converged for this gate; do not silently substitute a local backend. + +**Env-before-file transport decision:** Production preparation transiently resolves existing bounded OneCLI environment grants before mount planning, retaining only canonical referenced-file metadata and unexpanded templates. Session-host secret resolution uses the same pure decoder, installs transported files before provider preparation, and computes any generated substitutions in the child-only environment. The concrete Task 2 design replaces the earlier unsupported-runtime-reference fallback. Stage 2 validates synthetic execution and source/grant freshness; no explorer reads real secrets and no actual grant/environment map is persisted or logged. Ordinary supported env-before-file references must work, not merely return a clear unsupported error. + +## File responsibilities and interfaces + +1. `crates/freshell-platform/src/provider_environment.rs` (new), exported by `lib.rs`: common deterministic resource planning/copying, bounds, executable modes, and sanitized copy errors. `provider_environment_grants.rs` (new) holds the existing pure bounded OneCLI decoder shared by controller and host; existing `opencode_config.rs` holds the small shared pinned-order string/token transformer. No provider protocol dependency; wrappers map finite protocol profiles locally. +2. `crates/freshell-server/src/managed_provider_bootstrap.rs`: select supported resource roots and exact credential exclusions; return discovery errors instead of silently dropping configured selections. +3. `crates/freshell-server/src/managed_provider_references.rs` (new), registered in `main.rs`: source-origin aware known-form reference planning, resource closure selection, and staged-copy rewrites. No file-content secret expansion. +4. `crates/freshell-server/src/managed_mcp_capability.rs`: transactional grant/stage lifecycle, reference metadata, inline/TUI retention and reissue; structured preparation events. +5. `crates/freshell-runtime-protocol/src/lib.rs`: additive configuration-file reference metadata and explicit inspected recovery rows, preserving old defaults. `protocol/server_messages.rs` and shared TS types: additive blocked inventory process status only. +6. `crates/freshell-supervisor/src/backend/docker.rs` and `backend.rs`: validate/mount exact rewritten project-config overlays and dynamic file references; creation/adoption parity. +7. `crates/freshell-session-host/src/main.rs`, `provider_secret_resolution.rs`, and `providers/mod.rs`: common copying, transactional provider-owned refresh, executable permissions/ownership, process-local referenced files and origin-correct child preparation. +8. `crates/freshell-agent-runtime/src/lib.rs`, supervisor `checkpoints.rs`, `recovery.rs` and `loss_report_tests.rs`: emit/consume inspected recovery evidence and block metadata-only decisions, including offline recovery. +9. `crates/freshell-supervisor/src/service.rs`, `registry.rs`, session-host `output_journal.rs`: narrowly expose exact owned incarnation spool reads after recovery ends. Share the existing parser in platform if necessary; no new output storage system. +10. Server `managed_runtime.rs`, `terminals.rs`, terminal `registry.rs`, `managed_output_tests.rs`, WS `terminal.rs`: atomic typed outcomes, output/lifecycle propagation, final-head drain ordering, transition-deduplicated logs, and accurate user-facing terminal-directory process projection. +11. `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`, existing inventory consumers: process status correctness and blocked identity/output retention through existing summary projection. +12. Existing parity fixture/tests, process refresh integration, runtime loss tests, managed recovery unit/browser tests, cloud wrapper and a dedicated managed environment browser spec: behavioral proof. `docs/development/managed-runtime.md`: agent-facing copy/reference/recovery guarantees and actionable troubleshooting; README only if actual user-facing setup changes. + +--- + +### Task 1: One bounded executable resource copier used at both stages + +**Files:** +- Create: `crates/freshell-platform/src/provider_environment.rs` +- Modify: `crates/freshell-platform/src/lib.rs`; `crates/freshell-server/src/managed_provider_bootstrap.rs:65`; `crates/freshell-server/src/managed_mcp_capability.rs:445`; `crates/freshell-session-host/src/main.rs:1130,1506,1810` +- Test: common-module tests, existing `managed_provider_bootstrap::tests`, `managed_mcp_capability::tests`, and host `provider_config` tests. + +**Interfaces:** + +```rust +pub struct CopyLimits { + pub max_entries: usize, pub max_total_bytes: u64, + pub max_file_bytes: u64, pub max_depth: usize, +} +impl Default for CopyLimits { + fn default() -> Self { + Self { max_entries: 100_000, max_total_bytes: 512 * 1024 * 1024, + max_file_bytes: 128 * 1024 * 1024, max_depth: 64 } + } +} +pub struct CopySelection { pub source: std::path::PathBuf, + pub destination_relative: std::path::PathBuf } +pub struct CopyReport { pub entries: usize, pub bytes: u64, + pub projected_paths: Vec } +pub fn copy_environment(selections: &[CopySelection], destination_root: &std::path::Path, + excluded_canonical_sources: &[std::path::PathBuf], limits: CopyLimits) + -> Result; +pub fn private_file_mode(source_mode: u32) -> u32 { 0o600 | (source_mode & 0o111) } +pub fn provider_read_stage_mode(source_mode: u32) -> u32 { 0o640 | (source_mode & 0o111) } +``` + +`CopyError` has safe `relative_path`, operation, limit kind, observed and permitted values, and a corrective message. It contains no bytes/config fragments. The caller supplies stage/provider fields when logging. Selection destination components must be normal bounded relative components, with no traversal. + +- [ ] **Step 1: Add behavioral tests.** Create a disposable selection containing `SKILL.md`, `scripts/run` mode 0755 and then 0700, `references/data.json`, and `assets/template.txt`. Script body is `#!/bin/sh\nset -eu\ncd "$(dirname "$0")/.."\ncat references/data.json assets/template.txt\n`. Copy twice (controller root then provider HOME) and execute the final script with `Command::new`; assert exact nonce output and success. Create an ES module plugin importing `fixture-dependency` whose package reads its harmless `auth.json`; execute Node against the copied plugin and assert its exported nonce. Include actual relative `.pnpm` and `.bin` links, a transitive circular package link, and an external linked skill; assert runnable resource/dependency closure. Add missing/dangling/unsupported-node errors and exact/beyond tests for all four injected limits, checking observed error fields and no caller-published stage. Assert canonical credential source aliases are excluded but ordinary package `auth.json` survives. + +```rust +#[cfg(unix)] +#[test] +fn copied_skill_executes_sibling_resources_after_two_stages() { + use std::{fs, os::unix::fs::PermissionsExt, process::Command}; + use super::{copy_environment, CopyLimits, CopySelection}; + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source/skill"); + fs::create_dir_all(source.join("scripts")).unwrap(); + fs::create_dir_all(source.join("references")).unwrap(); + fs::create_dir_all(source.join("assets")).unwrap(); + fs::write(source.join("SKILL.md"), "fixture skill").unwrap(); + fs::write(source.join("references/data.json"), "resource-nonce\n").unwrap(); + fs::write(source.join("assets/template.txt"), "asset-nonce\n").unwrap(); + fs::write(source.join("scripts/run"), + "#!/bin/sh\nset -eu\ncd \"$(dirname \"$0\")/..\"\ncat references/data.json assets/template.txt\n").unwrap(); + for mode in [0o755, 0o700] { + fs::set_permissions(source.join("scripts/run"), fs::Permissions::from_mode(mode)).unwrap(); + let stage = root.path().join(format!("stage-{mode}")); + copy_environment(&[CopySelection { source: source.clone(), + destination_relative: "skills/fixture".into() }], &stage, &[], CopyLimits::default()).unwrap(); + let home = root.path().join(format!("home-{mode}")); + copy_environment(&[CopySelection { source: stage.join("skills"), + destination_relative: "skills".into() }], &home, &[], CopyLimits::default()).unwrap(); + let output = Command::new(home.join("skills/fixture/scripts/run")).output().unwrap(); + assert!(output.status.success(), "{:?}", output); + assert_eq!(output.stdout, b"resource-nonce\nasset-nonce\n"); + } +} +``` + +- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`. Expected missing implementation, then behavioral failure when the test is initially wired through current copying; record the old two-stage script execution PermissionDenied and missing dependency result, rather than treating initial compile setup as the required red evidence. Run matching controller/host tests against old code for the intended omissions before replacing it. +- [ ] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Preseed canonical source→projected destination mappings for all selected roots before walking any tree, then reuse those mappings for nested targets; reject genuinely conflicting destination selections before writing. Preserve symbolic links with rewritten relative targets and enqueue their target closure under private `.freshell/resources/` when outside selected roots. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. + Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure directory as a normal `.freshell/resources` resource selection in `ProviderLaunchContext`, so the host copies controller-added closure instead of preserving links to an unprojected target. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the accepted report's exact paths/parents. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. +- [ ] **Step 4: Green.** Repeat focused platform/controller/host commands; expected executable and Node resource/dependency assertions pass with correct boundary errors. +- [ ] **Step 5: Refactor.** Remove the two obsolete private copier policies and basename-based secret filtering; keep provider selection and ownership at callers and filesystem behavior in the common module. Avoid adding environment policy configuration. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`. Expected all pass. These cover the shared policy and every existing caller. +- [ ] **Step 7: Commit.** `git add crates/freshell-platform/src/provider_environment.rs crates/freshell-platform/src/lib.rs crates/freshell-server/src/managed_provider_bootstrap.rs crates/freshell-server/src/managed_mcp_capability.rs crates/freshell-session-host/src/main.rs` then `git commit -m "fix: preserve bounded managed provider resources and executable modes"`. + +### Task 2: Origin-correct configuration references through managed file grants + +**Files:** +- Create: `crates/freshell-server/src/managed_provider_references.rs` +- Create: `crates/freshell-platform/src/provider_environment_grants.rs` +- Modify: `crates/freshell-platform/src/lib.rs`, `crates/freshell-platform/src/opencode_config.rs`, `crates/freshell-platform/Cargo.toml` only if moving existing URL validation requires its existing dependency there. +- Modify: server `main.rs`, `managed_provider_bootstrap.rs`, `managed_mcp_capability.rs`, `managed_runtime.rs`, `fresh_agent_proxy.rs`; protocol `lib.rs`; supervisor `backend/docker.rs`, `backend.rs`; host `provider_secret_resolution.rs`, `main.rs`, `providers/mod.rs`. +- Test: planner tests, protocol validation tests, managed capability/reissue tests, host secret-resolution tests, Docker create/adoption validation tests. + +**Interfaces:** `ReferencePlan` owns rewritten config templates plus bounded resource selections and `ProviderSecretReference` metadata; it stores no substituted referenced bytes. It consumes validated provider, original HOME, workspace root, project cwd, declaring source and child-independent known configuration syntax. Add finite `ClaudeConfigurationFile`, `CodexConfigurationFile`, `OpencodeConfigurationFile`, and `AmplifierConfigurationFile` profiles and defaulted `provider_relative_path: Option` on `ProviderSecretReference`; old profiles require None and keep their static destinations. New destinations are normal relative paths in `.freshell/references/`, maximum 128 unique references, no other provider or duplicate destination. `ResolvedProviderSecrets.provider_files` becomes `Vec<(String, Vec)>`. Add exact staged `provider_config_overrides` source/original-workspace destination pairs to `TerminalMounts`; do not publish raw secret contents. + +```rust +// freshell-platform::provider_environment_grants: no runtime-protocol import. +pub enum EnvironmentProfile { Claude, Codex, Opencode, Amplifier, + AmplifierLunarouteGlm53 } +pub fn resolve_environment_profile(profile: EnvironmentProfile, raw: &str) + -> Result, GrantError>; +pub fn read_environment_grant(path: &std::path::Path, profile: EnvironmentProfile) + -> Result, GrantError>; + +// Controller boundary: callback values are transient, result is metadata/template only. +pub struct FileSourceOrigin<'a> { + pub declaring_config: &'a std::path::Path, + pub cwd: &'a std::path::Path, + pub original_home: &'a std::path::Path, +} +pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, + env: &dyn Fn(&str) -> Option) + -> Result; +``` + +`GrantError` and `ReferenceError` retain classified safe operation/profile/field/source-ordinal data and never include raw grant lines, expanded values or referenced bytes. `ResolvedFileReferencePlan` contains canonical sources, declaring source locations/ordinals, runtime destinations, referenced variable names, generated-placeholder recipes and unexpanded templates only; it has no transient expanded-text/map field and no derived Debug exposing private input. Controller and host translate existing finite protocol profiles to `EnvironmentProfile` in their wrappers. + +- [ ] **Step 1: Add failing consumers.** Table-drive absolute, relative, `~/`, spaces, in-root symlinks and external references for each verified field form from Stage 2. OpenCode cases cover global JSON/JSONC, original-workspace project config at nested cwd and ancestor, TUI, inline instance-cwd origin, nested `{file:{env:NODE_EXTRA_CA_CERTS}}`, one-pass env substitutions that introduce complete file tokens plus secret prefix/suffix text, original ambient/request path variables, missing/empty-variable semantics, comment-prefixed tokens, runtime CA destination→source mapping, loader-source-specific authEnv precedence, local path/file-URL plugins and instruction resources. More than four valid file references must work for terminal and fresh launch specs; duplicate destination, excessive per-profile/aggregate counts and cross-provider reference must fail before mounts. Change a synthetic grant's path between planning and spawn: dispatch is prevented with changed-input retry, previous usable stage survives, no different file is silently consumed. + The child executes the real supported configuration resolver where available and otherwise the pinned resolver-equivalent synthetic provider fixture; report the actual file-consumption nonce/digest and native trim/escape behavior, never the referenced bytes. Distinguish same relative spelling in two declaring directories. Copy original config bytes beforehand and assert they remain unchanged afterward. Missing/unreadable/dangling/cyclic referenced targets must prevent launch with an actionable named field/path, no capability omission. Sentinel credential and generated-placeholder cases assert absence from staged templates, durable registry/JSON, grant metadata, Docker inspect and logs while child consumption succeeds. Test initial launch, resume/reissue and both terminal/fresh routes using synthetic grants only. Pure decoder tests must preserve current interpolation, provider allowlists, proxy validation and unknown/legacy-profile rejection, and detect actual 128 KiB+1 read growth. +- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_references`; existing stage/host consumer tests must fail because original absolute/relative references are unresolved in runtime. A compile-only missing module is setup evidence, not behavioral red. +- [ ] **Step 3: Apply precise implementation patches.** Extract `provider_secret_resolution.rs:88-432`'s existing pure keys/env parsing, interpolation and provider/proxy validation into `provider_environment_grants.rs`; do not create a second permissive parser. Keep auth/CA-file profiles out of environment decoding and map protocol profiles in wrappers. `read_environment_grant` verifies the canonical regular private grant and reads at most 128 KiB+1 through a bounded reader before decoding, rejecting source growth; mounted host bytes use the same pure decoder. Add platform URL dependency only if the existing validator requires it. Do not instrument raw maps/values. + Immediately before capability stage/publication and final mount construction, controller builds a transient original host/provider env lookup with request overrides for terminal launches, current server launch env for fresh launches, then approved OneCLI child env overlay matching direct child precedence. Use original host HOME/cwd/config origins. Select only file-expression variable needs for reference planning; never expand the durable environment allowlist in `managed_runtime.rs:714`. Apply authEnv precedence only for the actual pinned loader sources receiving it; share a bounded existing-auth-record resolver if those source paths need auth overlay, rather than applying authEnv to inline config globally. Drop decoded maps after obtaining canonical source metadata and unexpanded templates; controller never reads referenced payloads. + Add a small shared token/string transformer to existing `opencode_config.rs` with the pinned single-env-pass→file-scan order. Recognize an outer file expression containing complete nested env tokens so the inner brace cannot terminate the outer match. Expand only its path transiently; missing/empty env is empty text as upstream, then resolve absolute/relative/`~/` against original origin. Canonicalize/check source metadata and replace the whole outer expression with `{file:/home/freshell/provider/.freshell/references/}` in staged copies; preserve unrelated env tokens. Do not try to canonicalize the runtime HOME on the controller: map a known runtime auth/CA file destination back to its existing typed source reference and reuse its grant/destination. Source errors identify original field/ordinal and variable names, not raw expanded credential values. + For an env value introducing file-token text or secret prefix/suffix around such tokens, use a bounded private source-span/ordinal recipe beside the already private staged source. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded original source span, variable names, approved canonical file selections and runtime destinations. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret env before spawn. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. + Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/128 MiB combined and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. Write referenced files into the existing tmpfs-backed temporary auth directory with private modes and provider ownership, and install dynamic provider HOME links before OpenCode configuration validation/provider spawn. Validate destination containment, uniqueness and profile match before any spawn. + For ordinary configured resource/plugin paths, copy complete package/resource closure via Task 1 and rewrite only recognized field nodes to the corresponding projected paths, preserving adjacent imports. For project files, write reference-only rewritten copies inside the new grant stage and mount exact files read-only over their original workspace locations. Validate canonical source stage root and canonical destination beneath workspace, reject missing/unsafe overlays, and include identical expected mount records in adoption proof. Inline origin is instance cwd and TUI/local origins follow their declaring sources, not private stage paths. + Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. At session-host preparation, re-evaluate referenced path expressions from freshly decoded mounted grants and source-origin recipe; compare normalized original-origin source path metadata to the planned selection and reuse its approved canonical mount. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. +- [ ] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. +- [ ] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. +- [ ] **Step 7: Commit.** Stage the exact production/test files in this task and `git commit -m "fix: resolve managed provider file references at their source origins"`. + +### Task 3: Transactional provider projection and complete environment smoke + +**Files:** +- Modify: host `main.rs:1281,1532,1704`; server `managed_mcp_capability.rs:637`; `crates/freshell-session-host/tests/provider_config_refresh.rs`; `test/fixtures/providers/parity-provider.mjs`; `test/integration/server/managed-provider-parity.test.ts`; its fixture-building helpers under `test/integration/server/`. + +**Interfaces:** Consumes `copy_environment`, `CopyReport`, `ReferencePlan` and existing `.freshell-config-projection.json`. Produces an all-or-error installed projection and unchanged previously working environment on failed refresh. Only prior manifest-owned paths may be replaced/removed. + +- [ ] **Step 1: Add failing process tests.** Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. +- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked --test provider_config_refresh`. Expected current stale-delete-before-copy path loses prior usable projection on injected failure, or resets executable mode. +- [ ] **Step 3: Apply precise transaction patch.** Stage the complete desired projection and manifest on the same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. Initial launch uses the same staged install before provider spawn. Preserve projected permissions/parent ownership and existing setpriv privilege boundary; do not add DAC_OVERRIDE/FOWNER. + Controller keeps temp-root→rename→capability publication ordering and revokes only new capability/resources on failure. Run-owned ephemeral references are cleaned with their grant; old grant/root/volume remain on failed replacement. Preparation error becomes explicit pre-spawn failure, never synthetic absence of identity. Log transaction stage/start/commit/rollback/failure once with safe counters and paths. +- [ ] **Step 4: Green.** Repeat refresh and controller fault cases; expected previous environment survives every precommit failure and complete new environment runs after success. +- [ ] **Step 5: Refactor.** Consolidate initial and exact-resume projection into one transaction helper; manifest authority remains limited to copied files. Share fixture setup for executable/resource/dependency cases without duplicating policy implementations in JS. +- [ ] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. +- [ ] **Step 7: Commit.** Stage listed files and `git commit -m "fix: publish managed provider refreshes transactionally"`. + +### Task 4: Inspected negative recovery evidence and conservative startup failures + +**Files:** +- Modify: protocol `lib.rs:321`; agent runtime `lib.rs:256,824`; host `main.rs:651`, `providers/mod.rs:432`; supervisor `checkpoints.rs:15,58`, `recovery.rs:1260,1876,1928`; tests in those modules and `loss_report_tests.rs`. + +**Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPathKind`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. + +- [ ] **Step 1: Add failing decisions.** Host dispatched-journal/no resume identity returns Blocked and preserves data. Supervisor zero checkpoint revision is blocked. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. +- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. +- [ ] **Step 3: Apply exact decision replacements.** Host `ProbeRecovery` with None resume plus false `never_dispatched` returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero returns Blocked/unknown. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. + Provider adapters emit NativeResume negative rows only after real metadata/read-dir/SQLite queries. Emit a CheckpointRestore row only when that adapter's declared recovery policy proves the inspected same store covers checkpoints, or when the independent checkpoint store was actually inspected. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline missing spec/identity after dispatch uses the same blocked classification. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. +- [ ] **Step 4: Green.** All metadata-only paths block and retain data, confirmed inspected absence still authorizes the correct final result, independently present fixture checkpoint restores. +- [ ] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification and current pristine-journal rules. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected tests run/pass, no unintended new conversation or data cleanup. +- [ ] **Step 7: Commit.** Stage this task's exact files and `git commit -m "fix: require inspected evidence before declaring recovery unavailable"`. + +### Task 5: Typed final recovery status, retained spool and bounded transitions + +**Files:** +- Modify: server `managed_runtime.rs:22,77,357,441`, `terminals.rs:396,731`; supervisor `service.rs:1555`, `registry.rs:681`; host `output_journal.rs`; terminal `registry.rs:753,860,4733,4878`, `managed_output_tests.rs`; WS `terminal.rs:577,8488`; protocol `server_messages.rs:408`. +- Create only if extraction is needed: `crates/freshell-platform/src/terminal_spool.rs`, exported in `lib.rs`, containing the existing bounded spool parser shared by host/supervisor. + +**Interfaces:** + +```rust +pub enum ManagedOutputLifecycle { + Running, Recovering, + Blocked { reason: String, attempt_id: Option, incident_id: Option }, + Ended { outcome: ManagedTerminalEndOutcome, exit_code: i64, + incident_id: Option, final_head: u64 }, +} +pub enum ManagedTerminalEndOutcome { Lost, Stopped } +// ManagedOutputRead adds lifecycle: Option. +``` + +`None` means no lifecycle observation, not Running. Registry stores the current managed lifecycle independently of attachable facade identity. One controller mutex owns each soul's `Recovering`, `Finished(RecoveryResult)` or classified `Failed` slot; retain complete typed result and observed exit/epoch, not strings. Supervisor retained-output resolver is read-only, exact owned incarnation, never authority for input/replacement/cleanup. + +- [ ] **Step 1: Add failing output/status behaviors.** Starting from running and recovering, drive final Lost, Stopped and Blocked outcomes; inventory distinguishes blocked and exited, retains terminal ID/native identity/reason/incident and output. Invoke actual `/api/terminals` plain and paged listings plus MCP/CLI `list-terminals` against the managed facade: returned user-facing process status must be blocked, never running, while attachment/replay still succeeds and input is refused. Drive a successful retry and assert those same listings return running again. Deliver >64 KiB final tail in multiple pages, recover/stop between pages, reconnect two readers and verify identical retained window with exactly one terminal.exit after final head. Final output survives supervisor cleanup and web/controller reattachment using retained spool. Drive 20 identical failed polls across two attached sockets; expect one structured transition and no repeated output-refresh warning, then changed failure cause gives one new event and later successful retry gives a recovered event. Success Reattached/Replaced/AlreadyLive restores Running/input and exact identity with existing incarnation reset behavior. +- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime`; `mise exec rust@1.96 -- cargo test -p freshell-terminal --locked managed_output`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked managed`. Expected cached final errors suppress retained output, leave Running, and repeat warning events. Destructive spool cleanup/stop cases run in the documented sandbox. +- [ ] **Step 3: Apply precise lifecycle/read patch.** Replace separate in-flight/blocked locks with atomic per-soul typed slots and one state-publication function for background/manual recovery. Preserve observed exit code/epoch before scheduling. Settled Lost/Stopped reads return successful `Ended` observations; settled Blocked or classified preparation/identity failure returns `Blocked` plus retained output, never a cached Err before output access. Success clears failure and reports Running. Keep one recovery task per soul and existing timeout/backoff; read path never sleeps or starts/stops synchronously. + Extend supervisor terminal-output resolution from active-only handle to exact retained owned handle when state is Recovering/Blocked/Lost/Stopped. Query host if still accessible; otherwise read existing current/previous incarnation spool files, validating ownership/path/epoch/incarnation and parsing within two existing rotation limits. Preserve 64 KiB transport pages, UTF-8 framing, retained start/head, reset/truncation flags and real errors. Freeze final head after the provider/host's final flush has settled; never freeze before a live final writer can append. Hydrate controller outcome/lifecycle from authoritative supervisor inventory when reconnecting after a web/controller restart, rather than relying on the in-memory finished map. Do not replace missing/unreadable spool with empty success. + WS ordering is: ingest complete page into replay; associate native identity as applicable; publish runtimeStatus/recovery lifecycle; call `finish_managed_exit` last only when delivered sequence reaches final head. Final outcome is visible while tail drains; paced replay defers terminal.exit as today. Never clear replay/terminal identity for blocked/final outcomes, and permit refresh/manual retry on blocked facades while refusing provider input. Lost/Stopped use actual exit code or explicit existing fallback, not fabricated running. + Emit `managed_runtime.recovery_transition` only on changed state/classified cause with terminal/soul/incarnation/attempt/incident/from/to/outcome/cause fields. A controller/registry-shared transport-error fingerprint suppresses repeated unchanged output warnings across sockets and emits one recovered/change event. Do not log full probe/raw credential-bearing error strings. Facade inventory exposes additive RuntimeStatus::Blocked and existing recovery summary; final Exited follows current lifecycle handling. Server `terminals.rs::directory_items` must project managed process state from runtime status/recovery outcome before its internal facade `TerminalRunStatus` fallback. A blocked facade's user-facing `status` is blocked in plain and paged directory output, so unchanged MCP/CLI list pass-throughs also report blocked. Extend the actual directory item schema/types to permit that status; preserve ordinary terminal statuses and internal attachment/lifetime checks. If consumer rendering transforms statuses, audit and correct that transformation too. +- [ ] **Step 4: Green.** Repeat focused tests; expected full final tail, accurate inventory/status, one exit and bounded transitions; successful retry behavior remains green. +- [ ] **Step 5: Refactor.** Consolidate final/manual/background result publication, extract only existing spool parsing if needed, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. +- [ ] **Step 6: Impact verification.** Run the three focused Cargo commands from Step 2 plus `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked retained_output"` after new tests use that filter, and `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected all run/pass. Include existing `managed_runtime_api_stop_tests` with `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime_api_stop` to protect explicit stop. +- [ ] **Step 7: Commit.** Stage listed lifecycle/output files and `git commit -m "fix: retain managed terminal output and publish final recovery outcomes"`. + +### Task 6: Browser status/reattachment and complete configured-cloud managed environment proof + +**Files:** +- Modify: `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`; actual running-status inventory consumers identified in Stage 2; `test/unit/lib/managed-runtime-recovery.test.ts`; `test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`; `scripts/e2e-cloud.sh`; `test/e2e-browser/helpers/external-target.ts` only if receipt/ownership validation is needed. +- Create: `test/unit/server/e2e-cloud-external-target.test.ts`, using fake gcloud executable dispatch to inspect the actual generated cloud env file and dispatched process inputs; this is not a source-string/config-text test. +- Create: `test/e2e-browser/specs/managed-provider-environment-external.spec.ts`; run-owned fixture/tunnel orchestration under `scripts/testing/managed-provider-environment-target.ts` if the topology validation succeeds. +- Modify: `docs/development/managed-runtime.md`; `AGENTS.md` plan/runbook links; README only for actual changed user setup. + +**Interfaces:** Existing inventory `runtimeStatus` admits `blocked` and the existing Rust exited process state where final tail is still draining; pane recovery projection uses `error` for Blocked/Lost, `exited` for Stopped, `recovering` only for active recovery, Running only after verified live recovery. Retain recoverySummary reason/attempt/incident and existing retry controls. External browser consumes `FRESHELL_E2E_TARGET_URL`, `FRESHELL_E2E_TARGET_WS_URL`, `FRESHELL_E2E_TARGET_TOKEN`, optional `FRESHELL_E2E_TARGET_TIMEOUT_MS`, and the narrow nonsecret `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE` JSON parameter through allowlisted cloud forwarding. That parameter contains candidate SHA, run ID, remote workspace path and expected nonce only; ownership/cleanup receipt remains on the target host. Do not forward TARGET_HOME: cloud `/app` is a different filesystem. + +- [ ] **Step 1: Add failing UI and real managed browser tests.** Extend unit cases from running and recovering through Blocked/Lost/Stopped, asserting exact projected status and retained summary/identity. Extend cloud-compatible contextual UI spec to show actual shell startup output, final blocked/lost/stopped status, existing retry controls, retained output and successful replacement attach; assert no silent extra `terminal.create`. These tests cover UI rendering/state and remain explicitly narrower than complete managed startup. + The external managed environment spec verifies actual `/api/server-info` candidate and active managed runtime against fixture metadata, creates a new OpenCode CLI pane through the real picker against the scratch managed target, and asserts terminal output from executable skill/resources, a dependency-loaded plugin and resolved absolute/relative config-file references with per-run nonce. Target uses representative synthetic supported configuration and synthetic managed grants; actual controller stage and session host child execute unchanged production route. Missing selected reference and startup-before-identity cases show actionable retained diagnostics and blocked outcome; no hidden conversation replacement. Browser reconnect checks terminal/session identity and retained output. Assert target stage/child receipt matches the created terminal ID and candidate so an ordinary fixture or stale server cannot pass. Fake-gcloud behavioral tests invoke the wrapper with URL/token pairs, consume the actual --env-vars-file, and verify what the fake job receives; missing URL/token pair errors before dispatch, secret token never appears in stdout, and unrelated host variables are absent. +- [ ] **Step 2: Red.** Run `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:cloud test/unit/lib/managed-runtime-recovery.test.ts`; `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`. Expected existing blocked projection remains recovering and running→final cases fail. The full managed external spec must fail on old candidate due to actual resource permissions/reference failure, not absent topology or zero matches. Establish the topology first as Stage 2 evidence. +- [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Update running-count/filter consumers to treat blocked as a retained failed process rather than normal running; no new UI card or second recovery protocol. + After topology proof, orchestrator allocates an owned `ManagedRuntimeBrowserRig`, synthetic input home/grants, run-specific ports and logs, and a task-local Quick Tunnel with recorded PID/cwd. Download the official task-local Linux binary and execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. + Extend the cloud env-file writer with only the exact allowlisted parameters above. Require URL/token together; serialize values as JSON strings valid in YAML, write a private temporary file and retain existing cleanup. Keep secret token out of command echo/receipts/logs. Do not forward arbitrary ambient vars. New spec uses existing ExternalServer fixture; fixture setup occurs on target host before the cloud browser, not ignored `setupHome` options. Retain no destructive runtime cases in the unconstrained cloud fixture; actual kill/restart/corruption acceptance remains in the documented disposable sandbox. Quick Tunnel has HTTP/WebSocket support, no SSE, 200 concurrent-request limit and no uptime guarantee; select one spec/run and report preflight/transport failure explicitly without switching backend. + Record copy bounds, exact unsupported-reference errors, credential reference lifecycle, unknown-vs-absent recovery and retained-output guarantees in agent docs. Do not add documentation-text tests or homepage changes for this repair. +- [ ] **Step 4: Green.** Run `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts --config config/vitest/vitest.config.ts`; repeat cloud unit/contextual UI tests. Start the run-owned driver with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts`; it invokes `pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-provider-environment-external.spec.ts` on configured cloud with its private target handoff. Expected nonzero tests pass and actual managed candidate/stage/child observations prove both preparation stages. No local-only qualification or UI mock substitutes. +- [ ] **Step 5: Refactor.** Reuse external target/owned rig and existing recovery UI helpers. Consolidate allowed cloud env forwarding without adding permanent tunnel deployment, remote Docker protocol or generic provider framework. +- [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. This task step does not run a broad suite; the single end-of-execution broad gate follows every task commit and review below. No interim broad gate unless the actual changed interface/impacted set requires one and that reason is recorded. +- [ ] **Step 7: Commit.** Stage exact TS/UI/tests/cloud-helper/docs files changed by this task and `git commit -m "test: verify managed provider environments and recovery in the browser"`. + +## Acceptance and completion boundary + +1. Map freshell#bdpt's exact latest 11 criteria to the executed task/test evidence in the parent run record; the user's requested provider and recovery additions remain mandatory. Do not close the kata or claim full completion with unresolved schema/reference or cloud managed browser coverage. +2. All code tasks require recorded behavioral red, green, refactor and affected-test evidence. Synthetic files/commands are intentional fixtures; production preparation, grants, ownership, terminal transport and browser lifecycle are real. No static prose/config hash tests qualify. +3. After all six tasks' commits and specification-plus-quality reviews, finish the worktree clean and record the final committed HEAD. Run the separate required no-write typecheck `pnpm run typecheck:client`, then the single final coordinated broad gate `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed provider environment and recovery final committed HEAD acceptance" pnpm run test`. These commands are on the final committed candidate, not a precommit Task 6 tree; preserve separately needed focused managed-feature, integration, browser and sandbox evidence from the tasks. Record command/results and verified HEAD, then push the feature branch with required prepush checks. A subsequent source/review fix requires a new focused commit and appropriate renewed verification of that changed final HEAD; do not rerun an unchanged broad gate merely because a task boundary passed. Stop before PR creation until explicit approval. Do not deploy/restart the live server while finishing the branch. +4. Once separately approved, follow the managed rollout runbook, preserve registry/runtime/image/secrets roots, and recheck original OpenCode failure via Freshell MCP. Assert usable startup and existing configuration without reading/logging secret contents. This operational residual is accepted pending approval, whereas missing implementation or configured-cloud test evidence is not. + +## Draft self-review + +The dispatcher-authored User Request is copied once unchanged. Tasks cover both copy stages, executable/resource/dependency closure, actual supported reference transport, transactions, conservative recovery evidence, typed final status, retained multi-page diagnostics, bounded logs and successful retry. Env-before-file now has a concrete bounded pre-mount OneCLI resolution phase, shared decoder/transformer, child-only generated substitution and freshness checks; both terminal/fresh four-reference ceilings are explicitly replaced. Exact source-form verification, synthetic sequencing/freshness proof and actual managed cloud browser topology remain visible Stage 2 gates; the plan does not claim those assumptions are already validated. Paths and known interfaces derive from the supplied exploration reports at base `d0cc7cb5e038811be0fb3ff5f4fc5812d2723228`. No repo mutation, test, worktree creation or commit was performed by this plan writer. From 8ceaa0166d16c566aeabe5ee33ccb1c35ce1e5c5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 19:54:14 -0700 Subject: [PATCH 002/493] docs: incorporate validated provider recovery and checkpoint design --- ...2026-10-05-managed-provider-environment.md | 136 ++++++++++++------ 1 file changed, 90 insertions(+), 46 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 3aa0c4cc8..b727e143d 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -7,7 +7,7 @@ ## User Request ### Requested result -Fix all managed provider environment defects tracked in freshell#bdpt so OpenCode starts and is usable with its existing configuration, and supported coding providers retain executable skills, resources, plugin dependencies, and resolvable configuration file references. Fix the downstream recovery defects recorded in the same kata: unsupported checkpoint-absence decisions and stale running terminal status with repeated warnings. +Fix all managed provider environment defects tracked in freshell#bdpt so OpenCode starts and is usable with its existing configuration, and supported coding providers retain executable skills, resources, plugin dependencies, and resolvable configuration file references. Fix the downstream recovery defects recorded in the same kata: unsupported checkpoint-absence decisions and stale running terminal status with repeated warnings. Fix automatic code-rewind checkpoint workspace selection, complete-readable snapshot enforcement, failure observability and per-turn rewind availability while keeping message sending usable. ### Explicit constraints - Fix both the controller staging and session-host copying stages with consistent rules, meaningful resource bounds, actionable errors, and structured logs. @@ -17,18 +17,23 @@ Fix all managed provider environment defects tracked in freshell#bdpt so OpenCod - Propagate final recovery outcomes to accurate terminal status, retain output and recovery information, and log structured state transitions instead of repeating unchanged failures on every poll; preserve successful recovery and reattachment. - Cover startup failure before identity capture, metadata-only negatives, confirmed absence, final Lost/stopped and blocked outcomes, retained output, bounded logging, and successful recovery with behavioral regression, integration/smoke, and affected browser e2e tests; destructive coverage runs only in the documented disposable sandbox. - Use behavioral red/green/refactor tests, complete environment integration/smoke coverage, and affected browser e2e tests on the configured backend; exercise executables, resources, dependency loading, file resolution, limits, and failure cleanup using synthetic fixtures. +- Choose code-rewind snapshot scope from the resolved project/workspace, independently of launch cwd; do not automatically snapshot an account home without a defined scope. Preserve valid explicitly selected non-Git workspaces. +- Require complete readable code snapshots without changing file permissions or silently omitting unreadable paths; failed snapshots cannot become rewind targets. Message sending remains usable when its pre-turn checkpoint fails, with accurate per-turn rewind availability and a clear client reason. +- Log structured code-checkpoint failures with request correlation, stage/operation, workspace, offending path when known and Git/filesystem cause, keeping credentials and outgoing message text out. Cover permission-restricted synthetic failures, home versus explicit scope, non-Git scope and successful complete snapshot/restore through behavior, integration/smoke and genuinely running configured-backend browser tests; existing skipped specs do not count. - Follow the-usual in a dedicated worktree; preserve unrelated work, keep changes in focused commits, and track the work in freshell#bdpt. - Do not open a PR without explicit approval, and do not restart or deploy the live self-hosted Freshell server without the user's explicit APPROVED. - After an approved supported deployment, recheck the original OpenCode failure through Freshell MCP without inspecting secret contents. ### Accepted tradeoffs and residuals -- The separate POST /api/fresh-agent/checkpoints HTTP 500 concerns code-rewind snapshots and is excluded; its cause remains undiagnosed. +- Code-rewind snapshots remain distinct from provider recovery evidence; both are now in scope under the kata's 15 criteria. - Darkforge installation cleanup and installer repair are complete separately and are excluded from this Freshell change. - Live deployment and its final MCP check remain pending separate restart approval. -**Goal:** A managed OpenCode pane starts with its existing supported configuration, every supported provider can execute its configured skills and resources, and failed managed launches recover conservatively with accurate status and retained diagnostics. +**Goal:** A managed OpenCode pane starts with its existing supported configuration, every supported provider can execute its configured skills and resources, failed managed launches recover conservatively with accurate status and retained diagnostics, and code-rewind snapshots capture only complete selected workspaces with accurate per-turn availability. -**Architecture:** Put bounded resource copying in the existing platform crate and use it at controller staging, initial projection, and resume refresh. Resolve known configuration references at their original source, transport potentially sensitive file references through existing managed secret mounts, and rewrite only run-owned copies. Carry inspected negative evidence and typed recovery outcomes through the existing recovery and terminal interfaces; retain output through the existing spool and expose blocked/final state through the existing inventory summary. +**Architecture:** Put bounded resource copying in the existing platform crate and use it at controller staging, initial projection, and resume refresh. Resolve known configuration references at their original source, transport potentially sensitive file references through existing managed secret mounts, and rewrite only run-owned copies. Carry inspected per-path negative evidence and typed recovery outcomes through the existing recovery and terminal interfaces. Persist the actual stream/incarnation descriptor before output begins and its coherent completion after writer settlement, using the existing spool. Keep retained managed-terminal membership independent of process Running status through API, WebSocket, cold adoption and client reconciliation. + +Code-rewind uses the existing shadow Git store with isolated request indexes and publish-last references. Resolve snapshot scope independently from launch cwd, finish each pre-turn capture before dispatch, and keep failed capture from blocking sends or becoming a valid rewind target. **Tech Stack:** Rust 1.96, Axum/Tokio, existing runtime protocol and Docker supervisor, React/Redux, pnpm 10.34.5, Vitest, Playwright, Cargo integration tests. @@ -36,17 +41,19 @@ Fix all managed provider environment defects tracked in freshell#bdpt so OpenCod 1. The clean `origin/main` baseline passed before this feature worktree was created at `d0cc7cb5e038811be0fb3ff5f4fc5812d2723228`. Author all changes on `the-usual/managed-provider-environment` in `.worktrees/managed-provider-environment`; this agent plan is linked from `AGENTS.md`. Preserve unrelated work and keep workflow reports outside the tracked tree. 2. Run pnpm frozen installs, existing coordinated commands, configured cloud Vitest/browser backends, and `GCLOUD_ROBOT_REQUIRE=1` for broad agent gates. Destructive process/restart/corruption tests run only in the documented disposable sandbox. The live server, real user configuration, credential contents, and unrelated agent work are outside this implementation's mutation scope. -3. All four providers use one copy policy. Defaults are 100,000 entries, 512 MiB aggregate regular-file bytes, 128 MiB per regular file, and depth 64, with checked arithmetic. Tests inject smaller bounds; no separate controller/host environment knobs or lower hidden ownership limit. Secrets retain their existing stricter OneCLI limits; referenced configuration files use an explicit bounded file profile. +3. All four providers use one copy policy. Defaults are 100,000 entries, 512 MiB aggregate regular-file bytes, 128 MiB per regular file, and depth 64, with checked arithmetic. Tests inject smaller bounds; no separate controller/host environment knobs or lower hidden ownership limit. Secrets retain existing stricter OneCLI limits. Configuration references have an explicit32MiB aggregate/16MiB per-source/128-source/256-alias profile; resource manifests/path/symlink metadata have a shared64MiB stored-byte budget and remain on stage/provider filesystems, not the1MiB callback. Original/effective config-source views are bounded16MiB each/64MiB aggregate simultaneous bytes, event metadata8192events/8MiB; check expansion before allocation. Final child argv/environment has an explicit1MiB aggregate accounting limit including executable/interpreters/pointers and every complete string's target32-page bound, with actual child soft stack at least8MiB. Generated and selected path inputs together allow128 bounded names (128bytes),64KiB/value and256KiB total, with control/provenance/auth collision checks. These shared profiles fail clearly instead of silently lowering effective capacity. 4. A selected missing, unreadable, cyclic, or unsupported resource is an actionable error, never an omitted configured capability. Optional provider roots absent before selection remain optional. Exclude concrete native state/cache/history and actual credential paths, not arbitrary basenames within dependencies. -5. Preserve executable bits while keeping files private: `0600 | (source_mode & 0111)`. Resume helper access uses `0640 | (source_mode & 0111)` and directories 0750, then final projection restores the private form. No setuid/setgid/sticky preservation. Provider-owned state is excluded from projection authority. +5. Preserve executable bits in provider-owned private files as `0600 | (source_mode & 0111)`, directories 0700. All controller-owned selected transport inodes use mapped group 0, files `0640 | (source_mode & 0111)` and directories 0750, including ephemeral ordinary selections; physical ownership walks never follow links. Outer controller/control ancestors stay 0700. The exact read-only input bind under `/run/freshell-private/user-provider` uses a trusted 0:0 private parent 0710, while separate control/grant files remain private. Normalize or verify the rootless-daemon-mapped group, fail clearly if unavailable, and never modify original sources. No extra capabilities or setuid/setgid/sticky preservation; rootful support is not added. 6. Never expand secret file contents into staged configuration, registry/SQLite records, grant manifests, Docker JSON, logs, or test receipts. Persist only validated reference metadata. Known configuration syntax and declaring origin control resolution; never rewrite arbitrary strings that look like paths. 7. Metadata absence is unknown evidence. Loss requires actual authoritative native/checkpoint store inspection for every required recovery path, with typed path evidence. Legacy negatives without inspected evidence cannot authorize loss or cleanup. Proven pristine seed recovery and exact existing identities remain supported. -8. Blocked status uses additive `runtimeStatus: blocked` and the existing `recoverySummary`. Keep the internal facade attachable/replayable with its terminal/session identity. Every user-facing process listing, including `/api/terminals`, MCP `list-terminals`, CLI `list-terminals` and client inventory, must distinguish blocked from running even if the internal facade remains Running for attachability. Lost/Stopped set facade Exited only after retained final output drains; no new recovery UI or unrelated protocol framework. +8. Blocked status uses additive `runtimeStatus: blocked` and the existing `recoverySummary`. Retained managed membership is independent of whether a provider process runs. Preserve the exact terminal/session handle in API plain/paged listings, MCP/CLI output, inventory, cold reconnect, compatibility create/attach and sidebar navigation. A retained managed handle dominates ordinary dead-terminal clearing or respawn; read-only replay remains available while input and automatic recreation are blocked. Final exit follows settled retained-output drain and keeps identity/replay. Manual retry updates controller typed state and supervisor state; no new recovery UI or unrelated protocol framework. 9. Live deployment follows `docs/development/managed-runtime-rollout.md`, requires the user's separate `APPROVED`, and includes the managed-feature server, supervisor, session host and correct image. No PR is opened before explicit branch approval. The post-deployment MCP check remains a separate authorized operation. +10. Code-rewind preserves the existing Git-ignore code scope and source permissions. Unreadable eligible paths, failed metadata/ref publication or unavailable scope cannot produce a valid target. Snapshot attempts settle before provider dispatch with a bounded server-enforced deadline; failure keeps sends usable and a durable per-turn reason. Captured workspace belongs to the turn and is not redirected by later cwd changes. Keep code-rewind separate from native provider recovery/loss evidence. + ## Load-bearing validation before implementation -The parent performs Stage 2 validation of the following facts and revises this draft before executing a task whose inputs change. Validation uses disposable synthetic homes and public pinned provider sources, never real credential contents. +Stage2 reviewed18 load-bearing claims. Required corrections are incorporated below; source-level contracts establish the chosen architecture and safe failure boundaries, while actual implementation/consumer/near-bound/browser proofs remain Stage4 acceptance gates. Root-owned synthetic cloudHTTP/WSS, Git capture/publication and pinnedSDK UUID probes passed with exact cleanup; none substitutes for managed candidate coverage. Validation used public pinned sources and synthetic fixtures, never real credential contents. 1. Verify the pinned providers' actual path-bearing forms. OpenCode 1.18.21 first performs one env substitution pass, then resolves file tokens; missing/empty variables become empty strings, comment-prefixed file tokens are skipped, local file origins are their declaring parent and inline origin is instance cwd. The follow-up `reports/plan-env-file-resolution.md` supplies a concrete pre-mount transport phase and shared-parser design for nested env-dependent file paths and env-introduced file tokens. Validate the synthetic original request/server environment ordering, loader-source-specific authEnv precedence, original HOME/runtime CA mapping, raw JSONC source spans, and changed-grant detection described in Task 2. For Claude/Codex/Amplifier, record exact supported fields already consumed by their pinned implementations and the existing resource roots they load. Do not invent a universal configuration schema. Task 2's syntax adapters require that concrete supported-field table before execution. 2. Prove resource symlink closure supports external linked skills, `.bin`, and package-store dependency cycles without recursing forever, losing imports, or exposing unrelated HOME. Verify bounds against a synthetic dependency tree exceeding the former 4096 entries. The controller and host both consume the shared implementation. @@ -56,7 +63,7 @@ The parent performs Stage 2 validation of the following facts and revises this d **UNRESOLVED COVERAGE GAP until item 5 passes:** A configured-cloud browser must create an actual managed OpenCode CLI pane whose child executes both production preparation stages with representative configuration, executable resources, dependency-loaded plugins, and configured file references. Recovery UI fixtures, an ordinary CLI fixture, local-only skipped qualification, and split Cargo preparation tests are useful but cannot clear this complete requirement. If no approved usable topology is established, retain FAILED/non-converged for this gate; do not silently substitute a local backend. -**Env-before-file transport decision:** Production preparation transiently resolves existing bounded OneCLI environment grants before mount planning, retaining only canonical referenced-file metadata and unexpanded templates. Session-host secret resolution uses the same pure decoder, installs transported files before provider preparation, and computes any generated substitutions in the child-only environment. The concrete Task 2 design replaces the earlier unsupported-runtime-reference fallback. Stage 2 validates synthetic execution and source/grant freshness; no explorer reads real secrets and no actual grant/environment map is persisted or logged. Ordinary supported env-before-file references must work, not merely return a clear unsupported error. +**Env-before-file transport decision:** Production preparation transiently resolves existing bounded OneCLI environment grants before mount planning, retaining only canonical referenced-file metadata and unexpanded templates. Session-host secret resolution uses the same pure decoder, installs transported files before provider preparation, and computes any generated substitutions in the child-only environment. The concrete Task 2 design replaces the earlier unsupported-runtime-reference fallback. Stage2 selected the source-supported mechanisms and bounded freshness contract; Stage4 validates actual synthetic execution and source/grant comparison; no explorer reads real secrets and no actual grant/environment map is persisted or logged. Ordinary supported env-before-file references must work, not merely return a clear unsupported error. ## File responsibilities and interfaces @@ -71,7 +78,8 @@ The parent performs Stage 2 validation of the following facts and revises this d 9. `crates/freshell-supervisor/src/service.rs`, `registry.rs`, session-host `output_journal.rs`: narrowly expose exact owned incarnation spool reads after recovery ends. Share the existing parser in platform if necessary; no new output storage system. 10. Server `managed_runtime.rs`, `terminals.rs`, terminal `registry.rs`, `managed_output_tests.rs`, WS `terminal.rs`: atomic typed outcomes, output/lifecycle propagation, final-head drain ordering, transition-deduplicated logs, and accurate user-facing terminal-directory process projection. 11. `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`, existing inventory consumers: process status correctness and blocked identity/output retention through existing summary projection. -12. Existing parity fixture/tests, process refresh integration, runtime loss tests, managed recovery unit/browser tests, cloud wrapper and a dedicated managed environment browser spec: behavioral proof. `docs/development/managed-runtime.md`: agent-facing copy/reference/recovery guarantees and actionable troubleshooting; README only if actual user-facing setup changes. +12. Code-rewind server `checkpoints.rs`, existing scope/session projections, FreshAgentView/send/checkpoint helpers and shared turn actions: explicit snapshot scope, complete readable capture with isolated index/publish-last refs, typed correlated failures and persisted per-turn rewind availability. +13. Existing parity fixture/tests, process refresh integration, runtime loss tests, managed recovery unit/browser tests, cloud wrapper and a dedicated managed environment browser spec: behavioral proof. `docs/development/managed-runtime.md`: agent-facing copy/reference/recovery guarantees and actionable troubleshooting; README only if actual user-facing setup changes. --- @@ -142,8 +150,8 @@ fn copied_skill_executes_sibling_resources_after_two_stages() { ``` - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`. Expected missing implementation, then behavioral failure when the test is initially wired through current copying; record the old two-stage script execution PermissionDenied and missing dependency result, rather than treating initial compile setup as the required red evidence. Run matching controller/host tests against old code for the intended omissions before replacing it. -- [ ] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Preseed canonical source→projected destination mappings for all selected roots before walking any tree, then reuse those mappings for nested targets; reject genuinely conflicting destination selections before writing. Preserve symbolic links with rewritten relative targets and enqueue their target closure under private `.freshell/resources/` when outside selected roots. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. - Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure directory as a normal `.freshell/resources` resource selection in `ProviderLaunchContext`, so the host copies controller-added closure instead of preserving links to an unprojected target. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the accepted report's exact paths/parents. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. +- [ ] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Select complete skill/plugin/package resource units before mapping. External plugin entry files require their containing resource/package root, adjacent imports/assets and applicable package metadata; canonical target leaves alone are insufficient. Include existing installed node_modules lookup context and connecting ancestor skeleton without copying unrelated ancestor trees. Prefer one bounded complete installation node_modules unit for pnpm store/hoisted/.bin topology, preserving ancestor lookup order and package scope. Preplan coherent physical mappings for overlapping parent/child units, reuse one physical canonical target and preserve additional logical resource destinations as links; overlap itself is not an error. Reject incompatible destination/type claims before writing. Preserve symbolic links with rewritten relative targets and enqueue the required complete resource/package and installed lookup context under private `.freshell/resources` when outside selected roots, preserving relative ancestor/sibling hierarchy rather than flattening one canonical leaf per private ID. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. + Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure/context directory and required aliases as normal `.freshell/resources` selections in ProviderLaunchContext; transport the complete graph once and preserve its relative topology identically at the second stage, without independent package rediscovery or renesting. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the accepted report's exact paths/parents. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. - [ ] **Step 4: Green.** Repeat focused platform/controller/host commands; expected executable and Node resource/dependency assertions pass with correct boundary errors. - [ ] **Step 5: Refactor.** Remove the two obsolete private copier policies and basename-based secret filtering; keep provider selection and ownership at callers and filesystem behavior in the common module. Avoid adding environment policy configuration. - [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`. Expected all pass. These cover the shared policy and every existing caller. @@ -156,9 +164,10 @@ fn copied_skill_executes_sibling_resources_after_two_stages() { - Create: `crates/freshell-platform/src/provider_environment_grants.rs` - Modify: `crates/freshell-platform/src/lib.rs`, `crates/freshell-platform/src/opencode_config.rs`, `crates/freshell-platform/Cargo.toml` only if moving existing URL validation requires its existing dependency there. - Modify: server `main.rs`, `managed_provider_bootstrap.rs`, `managed_mcp_capability.rs`, `managed_runtime.rs`, `fresh_agent_proxy.rs`; protocol `lib.rs`; supervisor `backend/docker.rs`, `backend.rs`; host `provider_secret_resolution.rs`, `main.rs`, `providers/mod.rs`. -- Test: planner tests, protocol validation tests, managed capability/reissue tests, host secret-resolution tests, Docker create/adoption validation tests. +- Modify: supervisor `service.rs`, callback control protocol and controller `managed_mcp_capability.rs` for complete prepared context, callback-safe attempt-bound native-auth lookup and idempotent generation commit/abort; extend the narrow existing `backend/native_history.rs` owned-helper lifecycle in a dedicated auth helper module and fixed host subcommand, plus exact disposable broker helper allowance. +- Test: planner tests, protocol validation tests, managed capability/reissue/lost-response and startup-orphan reconciliation tests, host secret-resolution tests, Docker create/adoption validation tests. -**Interfaces:** `ReferencePlan` owns rewritten config templates plus bounded resource selections and `ProviderSecretReference` metadata; it stores no substituted referenced bytes. It consumes validated provider, original HOME, workspace root, project cwd, declaring source and child-independent known configuration syntax. Add finite `ClaudeConfigurationFile`, `CodexConfigurationFile`, `OpencodeConfigurationFile`, and `AmplifierConfigurationFile` profiles and defaulted `provider_relative_path: Option` on `ProviderSecretReference`; old profiles require None and keep their static destinations. New destinations are normal relative paths in `.freshell/references/`, maximum 128 unique references, no other provider or duplicate destination. `ResolvedProviderSecrets.provider_files` becomes `Vec<(String, Vec)>`. Add exact staged `provider_config_overrides` source/original-workspace destination pairs to `TerminalMounts`; do not publish raw secret contents. +**Interfaces:** `ReferencePlan` owns rewritten config templates plus bounded resource selections and `ProviderSecretReference` metadata; it stores no substituted referenced bytes. It consumes validated provider, original HOME, workspace root, project cwd, declaring source and child-independent known configuration syntax. Add finite `ClaudeConfigurationFile`, `CodexConfigurationFile`, `OpencodeConfigurationFile`, and `AmplifierConfigurationFile` profiles and defaulted `provider_relative_path: Option` on `ProviderSecretReference`; old profiles require None and keep their static destinations. New destinations are normal relative paths in `.freshell/references/`, maximum 128 unique references, no other provider or duplicate destination. Configuration payload publication is indexed, bounded and chunk-streamed through private IPC, not a Vec of complete large file payloads. Retain legacy small provider-file handling separately. Reference metadata includes finite indexed source IDs and exact approved alias destinations (maximum256 across the plan), with logical/effective destination topology. TerminalMounts additionally carries exact config binds and writable external resource roots backed by physical subdirectories of the owned soul volume. All terminal/fresh create, prepared replacement, digest and cold-adoption paths share these descriptors; no payload values enter them. ```rust // freshell-platform::provider_environment_grants: no runtime-protocol import. @@ -183,15 +192,15 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, `GrantError` and `ReferenceError` retain classified safe operation/profile/field/source-ordinal data and never include raw grant lines, expanded values or referenced bytes. `ResolvedFileReferencePlan` contains canonical sources, declaring source locations/ordinals, runtime destinations, referenced variable names, generated-placeholder recipes and unexpanded templates only; it has no transient expanded-text/map field and no derived Debug exposing private input. Controller and host translate existing finite protocol profiles to `EnvironmentProfile` in their wrappers. - [ ] **Step 1: Add failing consumers.** Table-drive absolute, relative, `~/`, spaces, in-root symlinks and external references for each verified field form from Stage 2. OpenCode cases cover global JSON/JSONC, original-workspace project config at nested cwd and ancestor, TUI, inline instance-cwd origin, nested `{file:{env:NODE_EXTRA_CA_CERTS}}`, one-pass env substitutions that introduce complete file tokens plus secret prefix/suffix text, original ambient/request path variables, missing/empty-variable semantics, comment-prefixed tokens, runtime CA destination→source mapping, loader-source-specific authEnv precedence, local path/file-URL plugins and instruction resources. More than four valid file references must work for terminal and fresh launch specs; duplicate destination, excessive per-profile/aggregate counts and cross-provider reference must fail before mounts. Change a synthetic grant's path between planning and spawn: dispatch is prevented with changed-input retry, previous usable stage survives, no different file is silently consumed. - The child executes the real supported configuration resolver where available and otherwise the pinned resolver-equivalent synthetic provider fixture; report the actual file-consumption nonce/digest and native trim/escape behavior, never the referenced bytes. Distinguish same relative spelling in two declaring directories. Copy original config bytes beforehand and assert they remain unchanged afterward. Missing/unreadable/dangling/cyclic referenced targets must prevent launch with an actionable named field/path, no capability omission. Sentinel credential and generated-placeholder cases assert absence from staged templates, durable registry/JSON, grant metadata, Docker inspect and logs while child consumption succeeds. Test initial launch, resume/reissue and both terminal/fresh routes using synthetic grants only. Pure decoder tests must preserve current interpolation, provider allowlists, proxy validation and unknown/legacy-profile rejection, and detect actual 128 KiB+1 read growth. + The child executes the real supported configuration resolver where available and otherwise the pinned resolver-equivalent synthetic provider fixture; report the actual file-consumption nonce/digest and native trim/escape behavior, never the referenced bytes. Distinguish same relative spelling in two declaring directories. Copy original config bytes beforehand and assert they remain unchanged afterward. Missing/unreadable/dangling/cyclic referenced targets must prevent launch with an actionable named field/path, no capability omission. Sentinel credential and generated-placeholder cases assert absence from staged templates, durable registry/JSON, grant metadata, Docker inspect and logs while child consumption succeeds. Test initial launch, resume/reissue and both terminal/fresh routes using synthetic grants only. Cover exact native auth fallback and typed-grant precedence, new absent versus unexpected/missing retained volumes, callback nested lookup without deadlock, duplicate auth-key order, same-path changed reference selection, timeout/lost helper acknowledgement and exact cleanup. Capture sentinel-bearing Docker logs/create bodies, IPC prepared metadata, registry/caches/receipts and staged ordinary roots: auth values and unvalidated candidate expressions must be absent. Pure decoder tests must preserve current interpolation, provider allowlists, proxy validation and unknown/legacy-profile rejection, and detect actual 128 KiB+1 read growth. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_references`; existing stage/host consumer tests must fail because original absolute/relative references are unresolved in runtime. A compile-only missing module is setup evidence, not behavioral red. - [ ] **Step 3: Apply precise implementation patches.** Extract `provider_secret_resolution.rs:88-432`'s existing pure keys/env parsing, interpolation and provider/proxy validation into `provider_environment_grants.rs`; do not create a second permissive parser. Keep auth/CA-file profiles out of environment decoding and map protocol profiles in wrappers. `read_environment_grant` verifies the canonical regular private grant and reads at most 128 KiB+1 through a bounded reader before decoding, rejecting source growth; mounted host bytes use the same pure decoder. Add platform URL dependency only if the existing validator requires it. Do not instrument raw maps/values. - Immediately before capability stage/publication and final mount construction, controller builds a transient original host/provider env lookup with request overrides for terminal launches, current server launch env for fresh launches, then approved OneCLI child env overlay matching direct child precedence. Use original host HOME/cwd/config origins. Select only file-expression variable needs for reference planning; never expand the durable environment allowlist in `managed_runtime.rs:714`. Apply authEnv precedence only for the actual pinned loader sources receiving it; share a bounded existing-auth-record resolver if those source paths need auth overlay, rather than applying authEnv to inline config globally. Drop decoded maps after obtaining canonical source metadata and unexpanded templates; controller never reads referenced payloads. + Immediately before capability stage/publication and final mount construction, controller builds a transient original host/provider env lookup with request overrides for terminal launches, current server launch env for fresh launches, then approved OneCLI child env overlay matching direct child precedence. Use original host HOME/cwd/config origins. Select only file-expression variable needs for reference planning; never expand the durable environment allowlist in `managed_runtime.rs:714`. Apply well-known authEnv only to pinned local loader sources; inline/TUI receive no overlay. Prefer the selected typed OpencodeOnecliAuthFile, read through a bounded128KiB private reader and shared order-preserving auth decoder; do not substitute host user's auth or add generic OPENCODE_AUTH_CONTENT forwarding. Without a grant, perform a finite selected-key lookup from the exact registry-owned quiescent soul volume before mount planning. First-launch empty auth needs explicit installation/soul intent authority plus confirmed expected-volume absence, checked again at launch; missing native ID, unavailable daemon or unexpected volume is not new-absence proof. Use one fixed provider-UID cold helper and existing owned create/start/wait/delete pattern, with only the exact read-only soul volume and a single-use private exchange-directory socket. Auth reply is bounded memory-only, has no value-printing Debug/cache/durable/log path, and Docker logging is disabled; fixed sanitized failures only. Accept/framed exchange and helper supervision run concurrently within one aligned operation/callback deadline. Keep socket path within sun_path bounds and mapped-group directory0710/socket0660 access. For replacement callback, invoke an attempt-bound lookup under the recovery ownership already held, never reacquire the same soul lock or await IPC with capability-store/registry locks held. No live worker is stopped for a lookup. Host recomputes validated reference/mount metadata from actual selected auth/grant bytes before dispatch; same grant/path with changed reference target triggers bounded replan/failure, while unrelated auth changes need not. Retain native auth unless matching typed grant replaces it. Drop decoded maps after obtaining canonical source metadata and unexpanded templates; controller reads only approved payloads through bounded transient managed-secret publication, never ordinary-stage expansion. Add a small shared token/string transformer to existing `opencode_config.rs` with the pinned single-env-pass→file-scan order. Recognize an outer file expression containing complete nested env tokens so the inner brace cannot terminate the outer match. Expand only its path transiently; missing/empty env is empty text as upstream, then resolve absolute/relative/`~/` against original origin. Canonicalize/check source metadata and replace the whole outer expression with `{file:/home/freshell/provider/.freshell/references/}` in staged copies; preserve unrelated env tokens. Do not try to canonicalize the runtime HOME on the controller: map a known runtime auth/CA file destination back to its existing typed source reference and reuse its grant/destination. Source errors identify original field/ordinal and variable names, not raw expanded credential values. - For an env value introducing file-token text or secret prefix/suffix around such tokens, use a bounded private source-span/ordinal recipe beside the already private staged source. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded original source span, variable names, approved canonical file selections and runtime destinations. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret env before spawn. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. - Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/128 MiB combined and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. Write referenced files into the existing tmpfs-backed temporary auth directory with private modes and provider ownership, and install dynamic provider HOME links before OpenCode configuration validation/provider spawn. Validate destination containment, uniqueness and profile match before any spawn. - For ordinary configured resource/plugin paths, copy complete package/resource closure via Task 1 and rewrite only recognized field nodes to the corresponding projected paths, preserving adjacent imports. For project files, write reference-only rewritten copies inside the new grant stage and mount exact files read-only over their original workspace locations. Validate canonical source stage root and canonical destination beneath workspace, reject missing/unsafe overlays, and include identical expected mount records in adoption proof. Inline origin is instance cwd and TUI/local origins follow their declaring sources, not private stage paths. - Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. At session-host preparation, re-evaluate referenced path expressions from freshly decoded mounted grants and source-origin recipe; compare normalized original-origin source path metadata to the planned selection and reuse its approved canonical mount. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. + Keep exact accepted original sources and ordered logical occurrence metadata. Produce each lane's current unexpanded normalization/owned edits before making recipes: raw main documents retain comments/escapes, inherited inline normalizes after MCP edits, selected rebind TUI normalizes after its existing plugin edit, ordinary TUI retains raw form. Plan one transient document-level env pass followed by the pinned snapshotted file-match pass with full line-prefix context, including relevant key lexemes and partial inserted tokens; decoded strings and original offsets across later serialization are insufficient. Source-view generation/offsets belong to the effective unexpanded document. For env-introduced file tokens or secret prefix/suffix, use a bounded private document-context/source-event recipe beside that source; a contained raw lexeme placeholder is allowed only when the document-level match proves containment. Preserve the single pass, raw escapes, comment-prefix rule and introduced env tokens remaining literal. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded effective source/span with lane/view identity and full-context event metadata, variable provenance, approved selections and runtime destinations; never expanded values. Cross-lexeme/outside-lexeme accepted occurrences require a bounded whole-effective-document child-only recipe: validate/perform owned edits on retained accepted source first, then emit a native env wrapper. Its expanded document exists only in the child secret value and native parser input; wrapper consumption is a mandatory actual-provider fixture, not assumed proof. Original-form alias families may instead preserve native substitution, per the selected C05 representation. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret and source-specific auth lookup before spawn; auth must not shadow a generated name. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. + Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/32 MiB combined (capacity validation may revise this explicit profile before implementation) and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. Publish indexed files into one attempt-owned local-driver tmpfs volume using fixed nonsecret options type=tmpfs/device=tmpfs/o=size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec. The provider-UID publisher creates normal files/files-ordinal components with0600, receives at most64KiB raw chunks with one outstanding acknowledgement and encoded frames below1MiB, and logs no payload. Its exact exchange-directory socket is private and supervised concurrently; Docker logging isnone. Typed VolumeOptions.Subpath regular-file mounts with NoCopy=true/ReadOnly=true expose indexed sources at approved exact aliases or private indexed destinations. Keep publisher and workload active mount references overlapping: population acknowledgement returns before workload creation; supervisor retains publisher until successful mount/startup GrantAccepted and then releases it. A stopped volume object retains no tmpfs content; cold last-unmount requires authorized repopulation. Allow at most one active and one pending64MiB volume per soul; abort/uncertain-delivery/startup cleanup uses exact preparation ownership and cannot accumulate generations. Legacy small auth publication remains separate. Use existing child-secret map for finite selected original-form file-path inputs only, never ambientenv forwarding. Validate every logical/effective destination, uniqueness/content identity, profile and reserved mount collision before spawn. Workspace file/root mountpoints and their parents must already exist; Docker must not create host-workspace entries. Outside-root mountpoints may be constructed in the container image; provider-owned volume placeholders require explicit precreation. + For ordinary configured resource/plugin paths, copy complete package/resource closure via Task 1 and rewrite only recognized field nodes to the corresponding projected paths, preserving adjacent imports. Include complete selected implicit external config/skill/plugin roots, not just config leaves. Precreate descriptor-owned physical .freshell/provider-roots/ subdirectories/placeholders in the exact soul volume before main mounting, under initial ownership or retained provider UID as appropriate. Mount only selected subpaths at required discovery paths with NoCopy=true and writable provider behavior. Populate via canonical HOME paths with the common provider-owned transaction; keep mounted root inodes stable and replace only owned leaves. Private reference placeholders are explicit exclusions, never copied bytes. Adopt compares HostConfig.Mounts subpath/NoCopy plus actual mount type/name/source/destination/RW and exact owned volume options; flattened Mounts listing has no subpath field. Keep ordered main/TUI logical source occurrences separately from canonical read identities. Ordinary independent source destinations can use exact run-owned config binds, including discovered outside-workspace ancestors at their original logical filenames. Shared workspace-symlink families retain origin-dependent relative references and expose each selected target through exact private aliases; only source-independent absolute/HOME-sensitive collisions get identical selective rewrites to distinct indexed destinations. A shared child-only document recipe requires compatible lane evaluation; differing source-specific auth is not implicitly flattened. Keep physical HOME for all supported workspace topologies; the considered logical-HOME simplification is not selected because default-HOME workspaces can be masked or mutated. Preserve native original-form evaluation where exact target aliases are injective; generated shared recipes may not flatten differing auth lanes. Validate representability/collisions before dispatch, retain original data and return classified preparation conflict rather than wrong payload or disabled capability. Do not add parser/document machinery solely to certify arbitrary env-injected JSON programs outside the requested ordinary field/reference matrix; actual required matrix/consumer fixtures remain acceptance gates. Never deduplicate config occurrences by inode or attempt independent binds through symlinks that collapse to one destination. The same authoritative ordered source plan drives the preparer and native discovery. Inline origin is instance cwd and TUI/local origins follow their declaring sources, not private stage paths. + Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. Extend the existing authenticated private capability callback with one complete prepared metadata context for both terminal and fresh launches: selected roots, references, unexpanded recipes/origins, exact config mounts, new grant and prior/pending ownership IDs. No auth lookup values enter this reply. Consume it before supervisor registry preparation and immutable Docker mount publication; stop cloning stale references or separately rescanning roots. Replace immediate old-grant revocation with idempotent commit/abort. Retire prior inputs only after Host GrantAccepted, relevant registry commit and required exact-native verification; MCP activation is too early. Startup orphan cleanup protects pending/prior generations and reconciles uncertain delivery/dispatch. A lost response cannot authorize revoking both generations or redispatching a conversation. Partial new-stage/capability failures remove only this attempt's owned new inputs. Reference identity is the approved preparation snapshot: ordered source occurrence/lane/logical origin, controller-validated selection and canonical read/bind source, finite validated logical/runtime aliases, typed profile/destination and immutable candidate generation. At host preparation, freshly available auth/grant-derived candidates map only through that occurrence-specific snapshot to an authorized mounted source. Host cannot detect an unmounted original symlink retarget or canonicalize the controller tree. Controller-only ambient path inputs already consumed at planning are marked frozen selections, not fictitiously re-evaluated. Unknown/new/ambiguous aliases cause bounded controller replan before dispatch; stable and newly validated legitimate aliases must work. Use at most two pre-dispatch replan retries, preserving old inputs and recording classified failure on exhaustion. Selection identity is not inode/latest-content identity: actual bounded read consumes the mounted generation. Reuse that same read for comparison, auth publication and generated child values; no reread may silently select another plan. Next preparation resolves later original path changes. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. - [ ] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. - [ ] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. - [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. @@ -206,11 +215,11 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, - [ ] **Step 1: Add failing process tests.** Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked --test provider_config_refresh`. Expected current stale-delete-before-copy path loses prior usable projection on injected failure, or resets executable mode. -- [ ] **Step 3: Apply precise transaction patch.** Stage the complete desired projection and manifest on the same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. Initial launch uses the same staged install before provider spawn. Preserve projected permissions/parent ownership and existing setpriv privilege boundary; do not add DAC_OVERRIDE/FOWNER. - Controller keeps temp-root→rename→capability publication ordering and revokes only new capability/resources on failure. Run-owned ephemeral references are cleaned with their grant; old grant/root/volume remain on failed replacement. Preparation error becomes explicit pre-spawn failure, never synthetic absence of identity. Log transaction stage/start/commit/rollback/failure once with safe counters and paths. +- [ ] **Step 3: Apply precise transaction patch.** The provider UID65534/GID0 reads the exact approved read-only controller stage and creates private projection/backup inodes on its own HOME volume; it never renames or chmods transport inodes and never stages the full ordinary tree in tmpfs. Stage the complete desired projection and manifest on that same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. For terminal and fresh initial launch, finish all host-required stub/bootstrap and Amplifier work first, transfer HOME ownership last, then install selected auth links and run provider-owned preparation. No later trusted-host HOME traversal. Exact resume skips host bootstrap/ownership reclamation. Change the private-parent Docker tmpfs producer, host initial/resume reset and disposable broker validator together to 0710. Use physical lchown ownership walks and preserve ordinary execute bits. Preserve existing setpriv boundary; do not add DAC_OVERRIDE/FOWNER. + Native auth destinations survive when no corresponding typed auth-file grant is selected; CA/dynamic-file grants do not authorize auth deletion. Publish selected auth replacements before configuration/provider consumers, preserve old native destination on failure, and exclude native auth from ordinary projection manifest cleanup. Controller keeps temp-root→rename→capability publication ordering and revokes only new capability/resources on failure. Run-owned ephemeral references are cleaned with their grant; old grant/root/volume remain on failed replacement. Preparation error becomes explicit pre-spawn failure, never synthetic absence of identity. Log transaction stage/start/commit/rollback/failure once with safe counters and paths. - [ ] **Step 4: Green.** Repeat refresh and controller fault cases; expected previous environment survives every precommit failure and complete new environment runs after success. - [ ] **Step 5: Refactor.** Consolidate initial and exact-resume projection into one transaction helper; manifest authority remains limited to copied files. Share fixture setup for executable/resource/dependency cases without duplicating policy implementations in JS. -- [ ] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. +- [ ] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. Node fixtures alone do not certify OpenCode's Bun-compiled plugin loader or fresh Claude's bundled2.1.237 versus terminal2.1.263; Task6 actual pinned managed provider proof and dedicated fresh-path smoke must load the selected resources without reinstall masking omitted dependencies. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. - [ ] **Step 7: Commit.** Stage listed files and `git commit -m "fix: publish managed provider refreshes transactionally"`. ### Task 4: Inspected negative recovery evidence and conservative startup failures @@ -220,13 +229,13 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPathKind`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. -- [ ] **Step 1: Add failing decisions.** Host dispatched-journal/no resume identity returns Blocked and preserves data. Supervisor zero checkpoint revision is blocked. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. +- [ ] **Step 1: Add failing decisions.** Host dispatched-journal/no resume identity returns Blocked and preserves data. Supervisor zero checkpoint revision is blocked. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. - [ ] **Step 3: Apply exact decision replacements.** Host `ProbeRecovery` with None resume plus false `never_dispatched` returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero returns Blocked/unknown. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. - Provider adapters emit NativeResume negative rows only after real metadata/read-dir/SQLite queries. Emit a CheckpointRestore row only when that adapter's declared recovery policy proves the inspected same store covers checkpoints, or when the independent checkpoint store was actually inspected. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline missing spec/identity after dispatch uses the same blocked classification. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. + Provider adapters emit NativeResume negatives only after complete inspection for effective identity, roots and accepted artifacts; a declaration or same executable invocation is insufficient. OpenCode exact SQLite no-match can cover both paths only when effective DB is established, including accepted data-root/DB/channel overrides. Claude/Codex scans that skip unreadable, deep, symlinked, oversized or database-selected candidates cannot certify absence: preserve exact positive recovery and block unknown negatives until complete authority exists. Amplifier inspection includes accepted backup transcript/metadata. The NativeSession fixture requires separate complete checkpoint inspection; oversize skipped candidate is Unknown. CheckpointRestore negatives require demonstrably complete shared or independent inspection. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline missing spec/identity after dispatch uses the same blocked classification. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. - [ ] **Step 4: Green.** All metadata-only paths block and retain data, confirmed inspected absence still authorizes the correct final result, independently present fixture checkpoint restores. -- [ ] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification and current pristine-journal rules. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected tests run/pass, no unintended new conversation or data cleanup. +- [ ] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification, pre-IPC input journals, queued-only replay and actor-native identity adoption. Input-row/native-ID absence alone is not pristine proof: consult durable execution-grant/actor evidence and record initialization intent before provider creation. Ambiguous startup before ID capture blocks; seed replacement needs actual no-dispatch/zero-turn proof. Preserve existing five-successful-replacements-per-hour budget and make final retry-budget Blocked sticky across automatic requests/cold restart until ManualRetry. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. - [ ] **Step 7: Commit.** Stage this task's exact files and `git commit -m "fix: require inspected evidence before declaring recovery unavailable"`. ### Task 5: Typed final recovery status, retained spool and bounded transitions @@ -237,24 +246,16 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Interfaces:** -```rust -pub enum ManagedOutputLifecycle { - Running, Recovering, - Blocked { reason: String, attempt_id: Option, incident_id: Option }, - Ended { outcome: ManagedTerminalEndOutcome, exit_code: i64, - incident_id: Option, final_head: u64 }, -} -pub enum ManagedTerminalEndOutcome { Lost, Stopped } -// ManagedOutputRead adds lifecycle: Option. -``` +Managed output observations distinguish Running, active Recovering, Blocked with reason/attempt/incident, and ended Lost/Stopped with settled completion. Completion carries exact epoch/final head and known exit code or explicit unavailable result. Recovery outcome and writer-completion status remain separate when cleanup is uncertain. + -`None` means no lifecycle observation, not Running. Registry stores the current managed lifecycle independently of attachable facade identity. One controller mutex owns each soul's `Recovering`, `Finished(RecoveryResult)` or classified `Failed` slot; retain complete typed result and observed exit/epoch, not strings. Supervisor retained-output resolver is read-only, exact owned incarnation, never authority for input/replacement/cleanup. +`None` means no lifecycle observation, not Running. Registry stores the current managed lifecycle independently of attachable facade identity. One controller mutex owns each soul's Recovering, Finished(RecoveryResult) or classified Failed slot; retain full typed result, exit/epoch and durable generation. Background and HTTP manual recovery publish through one state transition; successful retry resumes actual output/input. Supervisor retained-output resolver is read-only, exact owned incarnation, never authority for input/replacement/cleanup. - [ ] **Step 1: Add failing output/status behaviors.** Starting from running and recovering, drive final Lost, Stopped and Blocked outcomes; inventory distinguishes blocked and exited, retains terminal ID/native identity/reason/incident and output. Invoke actual `/api/terminals` plain and paged listings plus MCP/CLI `list-terminals` against the managed facade: returned user-facing process status must be blocked, never running, while attachment/replay still succeeds and input is refused. Drive a successful retry and assert those same listings return running again. Deliver >64 KiB final tail in multiple pages, recover/stop between pages, reconnect two readers and verify identical retained window with exactly one terminal.exit after final head. Final output survives supervisor cleanup and web/controller reattachment using retained spool. Drive 20 identical failed polls across two attached sockets; expect one structured transition and no repeated output-refresh warning, then changed failure cause gives one new event and later successful retry gives a recovered event. Success Reattached/Replaced/AlreadyLive restores Running/input and exact identity with existing incarnation reset behavior. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime`; `mise exec rust@1.96 -- cargo test -p freshell-terminal --locked managed_output`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked managed`. Expected cached final errors suppress retained output, leave Running, and repeat warning events. Destructive spool cleanup/stop cases run in the documented sandbox. - [ ] **Step 3: Apply precise lifecycle/read patch.** Replace separate in-flight/blocked locks with atomic per-soul typed slots and one state-publication function for background/manual recovery. Preserve observed exit code/epoch before scheduling. Settled Lost/Stopped reads return successful `Ended` observations; settled Blocked or classified preparation/identity failure returns `Blocked` plus retained output, never a cached Err before output access. Success clears failure and reports Running. Keep one recovery task per soul and existing timeout/backoff; read path never sleeps or starts/stops synchronously. - Extend supervisor terminal-output resolution from active-only handle to exact retained owned handle when state is Recovering/Blocked/Lost/Stopped. Query host if still accessible; otherwise read existing current/previous incarnation spool files, validating ownership/path/epoch/incarnation and parsing within two existing rotation limits. Preserve 64 KiB transport pages, UTF-8 framing, retained start/head, reset/truncation flags and real errors. Freeze final head after the provider/host's final flush has settled; never freeze before a live final writer can append. Hydrate controller outcome/lifecycle from authoritative supervisor inventory when reconnecting after a web/controller restart, rather than relying on the in-memory finished map. Do not replace missing/unreadable spool with empty success. - WS ordering is: ingest complete page into replay; associate native identity as applicable; publish runtimeStatus/recovery lifecycle; call `finish_managed_exit` last only when delivered sequence reaches final head. Final outcome is visible while tail drains; paced replay defers terminal.exit as today. Never clear replay/terminal identity for blocked/final outcomes, and permit refresh/manual retry on blocked facades while refusing provider input. Lost/Stopped use actual exit code or explicit existing fallback, not fabricated running. + Extend supervisor terminal-output resolution from active-only handle to exact retained owned handle when state is Recovering/Blocked/Lost/Stopped. Persist actual output epoch (including host boot identity), terminal ID and incarnation before the output writer starts, preserving that association across host restart. Distinguish prior committed and failed replacement streams; source prefix is not actual epoch. Reuse existing two 32 MiB spool segments and bounded torn/truncated-frame handling. Query host if still accessible; otherwise read existing current/previous incarnation spool files, validating ownership/path/epoch/incarnation and parsing within two existing rotation limits. Preserve 64 KiB transport pages, UTF-8 framing, retained start/head, reset/truncation flags and real errors. Persist coherent completion keyed by terminal/incarnation/actual epoch after writer settlement. Normal EOF flush and existing successful host Stop reader join settle output; after abrupt removal, exact backend VerifiedEmpty settles the writer before spool scanning/freezing. Include retained bounds and known exit code or explicit unavailable status. Never pair stale sampled head with later exited flag. Lost with CleanupTerminationUnconfirmed or BackendUnavailable is a recovery outcome with unresolved writer completion, never fabricated stopped/empty output. Hydrate controller outcome/lifecycle from authoritative supervisor inventory when reconnecting after a web/controller restart, rather than relying on the in-memory finished map. Do not replace missing/unreadable spool with empty success. + WS ordering is: ingest complete page into replay; associate native identity as applicable; publish runtimeStatus/recovery lifecycle; call `finish_managed_exit` last only when delivered sequence reaches final head. Final outcome is visible while tail drains; paced replay defers terminal.exit as today. Never clear replay/terminal identity for blocked/final outcomes, and permit refresh/manual retry on blocked facades while refusing provider input. Lost/Stopped use known exit code or explicit unavailable completion semantics; unknown cleanup does not complete a stream. Managed terminal.exit retains client handle, stream and restore keys; ordinary exits retain existing behavior. Emit `managed_runtime.recovery_transition` only on changed state/classified cause with terminal/soul/incarnation/attempt/incident/from/to/outcome/cause fields. A controller/registry-shared transport-error fingerprint suppresses repeated unchanged output warnings across sockets and emits one recovered/change event. Do not log full probe/raw credential-bearing error strings. Facade inventory exposes additive RuntimeStatus::Blocked and existing recovery summary; final Exited follows current lifecycle handling. Server `terminals.rs::directory_items` must project managed process state from runtime status/recovery outcome before its internal facade `TerminalRunStatus` fallback. A blocked facade's user-facing `status` is blocked in plain and paged directory output, so unchanged MCP/CLI list pass-throughs also report blocked. Extend the actual directory item schema/types to permit that status; preserve ordinary terminal statuses and internal attachment/lifetime checks. If consumer rendering transforms statuses, audit and correct that transformation too. - [ ] **Step 4: Green.** Repeat focused tests; expected full final tail, accurate inventory/status, one exit and bounded transitions; successful retry behavior remains green. - [ ] **Step 5: Refactor.** Consolidate final/manual/background result publication, extract only existing spool parsing if needed, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. @@ -269,13 +270,13 @@ pub enum ManagedTerminalEndOutcome { Lost, Stopped } - Create: `test/e2e-browser/specs/managed-provider-environment-external.spec.ts`; run-owned fixture/tunnel orchestration under `scripts/testing/managed-provider-environment-target.ts` if the topology validation succeeds. - Modify: `docs/development/managed-runtime.md`; `AGENTS.md` plan/runbook links; README only for actual changed user setup. -**Interfaces:** Existing inventory `runtimeStatus` admits `blocked` and the existing Rust exited process state where final tail is still draining; pane recovery projection uses `error` for Blocked/Lost, `exited` for Stopped, `recovering` only for active recovery, Running only after verified live recovery. Retain recoverySummary reason/attempt/incident and existing retry controls. External browser consumes `FRESHELL_E2E_TARGET_URL`, `FRESHELL_E2E_TARGET_WS_URL`, `FRESHELL_E2E_TARGET_TOKEN`, optional `FRESHELL_E2E_TARGET_TIMEOUT_MS`, and the narrow nonsecret `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE` JSON parameter through allowlisted cloud forwarding. That parameter contains candidate SHA, run ID, remote workspace path and expected nonce only; ownership/cleanup receipt remains on the target host. Do not forward TARGET_HOME: cloud `/app` is a different filesystem. +**Interfaces:** Existing inventory `runtimeStatus` admits `blocked` and the existing Rust exited process state where final tail is still draining; pane recovery projection uses `error` for Blocked/Lost, `exited` for Stopped, `recovering` only for active recovery, Running only after verified live recovery. Retain recoverySummary reason/attempt/incident and existing retry controls. External browser consumes `FRESHELL_E2E_TARGET_URL`, `FRESHELL_E2E_TARGET_WS_URL`, `FRESHELL_E2E_TARGET_TOKEN`, optional `FRESHELL_E2E_TARGET_TIMEOUT_MS`, and the narrow nonsecret `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE` JSON parameter through allowlisted cloud forwarding. That parameter contains candidate SHA, run ID, remote workspace path and expected nonce only; ownership/cleanup receipt remains on the target host. Do not forward TARGET_HOME: cloud `/app` is a different filesystem. Candidate provenance comes from authenticated `/api/server-info` commit/buildDirty; actual managed readiness comes from `/api/runtime/readiness`, negotiated ready.managedRuntimeV1 and the created terminal/soul/incarnation inventory, not an invented server-info flag. - [ ] **Step 1: Add failing UI and real managed browser tests.** Extend unit cases from running and recovering through Blocked/Lost/Stopped, asserting exact projected status and retained summary/identity. Extend cloud-compatible contextual UI spec to show actual shell startup output, final blocked/lost/stopped status, existing retry controls, retained output and successful replacement attach; assert no silent extra `terminal.create`. These tests cover UI rendering/state and remain explicitly narrower than complete managed startup. The external managed environment spec verifies actual `/api/server-info` candidate and active managed runtime against fixture metadata, creates a new OpenCode CLI pane through the real picker against the scratch managed target, and asserts terminal output from executable skill/resources, a dependency-loaded plugin and resolved absolute/relative config-file references with per-run nonce. Target uses representative synthetic supported configuration and synthetic managed grants; actual controller stage and session host child execute unchanged production route. Missing selected reference and startup-before-identity cases show actionable retained diagnostics and blocked outcome; no hidden conversation replacement. Browser reconnect checks terminal/session identity and retained output. Assert target stage/child receipt matches the created terminal ID and candidate so an ordinary fixture or stale server cannot pass. Fake-gcloud behavioral tests invoke the wrapper with URL/token pairs, consume the actual --env-vars-file, and verify what the fake job receives; missing URL/token pair errors before dispatch, secret token never appears in stdout, and unrelated host variables are absent. - [ ] **Step 2: Red.** Run `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:cloud test/unit/lib/managed-runtime-recovery.test.ts`; `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`. Expected existing blocked projection remains recovering and running→final cases fail. The full managed external spec must fail on old candidate due to actual resource permissions/reference failure, not absent topology or zero matches. Establish the topology first as Stage 2 evidence. -- [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Update running-count/filter consumers to treat blocked as a retained failed process rather than normal running; no new UI card or second recovery protocol. - After topology proof, orchestrator allocates an owned `ManagedRuntimeBrowserRig`, synthetic input home/grants, run-specific ports and logs, and a task-local Quick Tunnel with recorded PID/cwd. Download the official task-local Linux binary and execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. +- [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Separate running counts from retained managed membership in App census, panesSlice clearDeadTerminals, TerminalView attach/exit and sidebar/background lookup, including cold-load ordering and compatibility fallback. Read-only same-handle replay attach is distinct from input/create/recovery permission. Server cold lookup and WS reconcile adopt retained Stopped/Failed handles instead of ordinary respawn; attach cannot overwrite authoritative blocked status. Sidebar click uses retained owned target, including provisional pre-native identity. Reuse managed lifetime claims/reaper behavior; no new UI card or recovery protocol. + The owned generic Cloud Run HTTP/WSS preflight establishes reachability, while full candidate/browser proof remains required. Orchestrator allocates an owned `ManagedRuntimeBrowserRig`, explicit synthetic provider/workspace/bootstrap roots for both broker and supervisor, run-specific ports/logs, and a task-local Quick Tunnel with recorded PID/cwd. RuntimeHarness currently hardcodes real os.homedir() provider roots and ambient bootstrap grants; add narrow fixture inputs rather than assuming webHomeDir isolates them. Existing callers retain their defaults; this run never selects ambient real roots/grants. Download the official task-local Linux binary and execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. Extend the cloud env-file writer with only the exact allowlisted parameters above. Require URL/token together; serialize values as JSON strings valid in YAML, write a private temporary file and retain existing cleanup. Keep secret token out of command echo/receipts/logs. Do not forward arbitrary ambient vars. New spec uses existing ExternalServer fixture; fixture setup occurs on target host before the cloud browser, not ignored `setupHome` options. Retain no destructive runtime cases in the unconstrained cloud fixture; actual kill/restart/corruption acceptance remains in the documented disposable sandbox. Quick Tunnel has HTTP/WebSocket support, no SSE, 200 concurrent-request limit and no uptime guarantee; select one spec/run and report preflight/transport failure explicitly without switching backend. Record copy bounds, exact unsupported-reference errors, credential reference lifecycle, unknown-vs-absent recovery and retained-output guarantees in agent docs. Do not add documentation-text tests or homepage changes for this repair. - [ ] **Step 4: Green.** Run `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts --config config/vitest/vitest.config.ts`; repeat cloud unit/contextual UI tests. Start the run-owned driver with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts`; it invokes `pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-provider-environment-external.spec.ts` on configured cloud with its private target handoff. Expected nonzero tests pass and actual managed candidate/stage/child observations prove both preparation stages. No local-only qualification or UI mock substitutes. @@ -283,13 +284,56 @@ pub enum ManagedTerminalEndOutcome { Lost, Stopped } - [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. This task step does not run a broad suite; the single end-of-execution broad gate follows every task commit and review below. No interim broad gate unless the actual changed interface/impacted set requires one and that reason is recorded. - [ ] **Step 7: Commit.** Stage exact TS/UI/tests/cloud-helper/docs files changed by this task and `git commit -m "test: verify managed provider environments and recovery in the browser"`. +### Task 7: Resolve code-rewind workspace, publish complete snapshots, and expose the result per turn + +**Criteria:** 12–15. **Dependencies:** finish the planning/assumption gate; coordinate shared fresh-agent/server files with Tasks 5–6. This task does not use code-rewind results as provider-store recovery evidence, does not change launch cwd, and does not restart production. Run-owned browser target/tunnel support from Task 6 may be reused; it must remain synthetic and isolated. + +**Files:** server `checkpoints.rs` and shared checkpoint service/state; WS send protocol and narrow injected trait plus managed/local barrier/admission paths; exact provider producer/history linkage files identified by V18; fixed sandbox entrypoint/wrapper fixture mode and owned bridge target driver; `crates/freshell-server/src/checkpoints.rs`; a small sibling checkpoint-scope helper only if it makes the route simpler; checkpoint route wiring in `main.rs` only if existing managed/session resolution is required; `src/lib/fresh-agent-checkpoints.ts`; `src/components/fresh-agent/FreshAgentView.tsx`, `FreshAgentTranscript.tsx`, `FreshAgentTurnActions.tsx`; relevant pane normalization/persistence in `src/store/paneTypes.ts` and `panesSlice.ts`; current checkpoint/component tests; one focused Rust black-box integration test; one new cloud-legal browser spec and its bounded synthetic fixture. Use existing fixture/support code where it actually supplies the needed behavior. No general snapshot framework, environment copier reuse, broad directory exclusions, chmod repair, or real-account fixture. + +- [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Verify these tests fail for the intended current behaviors and record red output. + +- [ ] **Step 2: Resolve checkpoint scope independently from provider launch/routing cwd.** Give the checkpoint request additive explicit workspace information without reinterpreting `initialCwd` as proof of project intent. Resolve in this order: a valid deliberately explicit checkpoint workspace; revalidated canonical session checkout evidence; an independently resolved error-aware Git checkout from retained selected/session cwd (do not reuse presentation git_meta fallback as proof); then a valid deliberately selected non-Git directory. DirectoryPicker confirmation supplies starting-directory provenance, not an automatic explicit workspace override. checkoutPath and managed workspace_path may themselves be cwd fallbacks, so independently verify Git evidence rather than trusting field names or repository grouping. Missing/conflicting inferred provenance remains unavailable; explicit non-Git scope stays valid. Do not change provider routing cwd. Use the existing directory-selection/resume creation sites to carry selected scope provenance where it is actually known; default/home cwd alone supplies no such provenance. Avoid a new UI flow when current project/directory selection already gives the answer. + + Make the server the final filesystem resolver: canonicalize an existing directory, distinguish inaccessible/not-directory from absent, and return the resolved workspace on successful create. Account-home detection must cover the relevant account, not only `CheckpointsApiState.home` (the reproduced failure involves a different UID's home). Use OS account-home information and known managed home paths on Linux; ensure the cross-platform fallback does not disable valid Windows project scopes. Automatically inferred scopes equal to a recognized account home are unavailable even when a dotfiles Git repository happens to exist there. A genuine explicit non-Git scope remains supported. Never search arbitrary subdirectories of a home for a project or silently broaden a selected subtree to another checkout. If root evidence conflicts or is missing, return a typed unavailable reason and keep sending usable. + + Capture canonical workspace and immutable scope/storage locator with the attempt/checkpoint, using it for list/metadata/restore without rerunning new-root discovery. Legacy shadow repos keep dual addressing by original stored/legacy scope string/storage key and validated original work-tree destination; do not rehash a historical nested/symlink cwd as today's checkout root or scan homes to find it. A subsequent pane cwd or session-directory update must not redirect an older turn's restore. Preserve existing valid shadow repositories by resolving existing entries with their original stored/legacy scope; no bulk migration or deletion is needed. + +- [ ] **Step 3: Require complete readable eligible code and publish only successful targets.** Keep the existing Git-ignore semantics as the defined code scope; `.git` administrative state stays outside it. Before publishing, explicitly traverse eligible directories with error-aware filesystem operations and verify readable eligible regular files. Determine ignores with Git's existing rules, not a new hard-coded list. Do not use `Path::exists`, an ignored `read_dir` error, a process UID of zero, or success from `git ls-files` alone as proof of readability. Symlinks keep Git's link semantics and must not cause traversal into unrelated trees. Reject unsupported eligible nested-repository/gitlink capture with an actionable path/cause unless the bounded validation proves an equally simple way to capture its complete code tree; never advertise only the gitlink as a complete snapshot. The existing policy excludes ignored resources intentionally; it does not authorize omitting an unreadable eligible path. + + Serialize create/metadata/restore operations for one canonical snapshot workspace with a shared route-state keyed mutex; separate workspaces remain independent. Stage with a request-local `GIT_INDEX_FILE`, initialized from the previous published tree (or empty for the first checkpoint), so failed/partial `git add` cannot contaminate a later snapshot. Build a tree and commit object without moving the published branch; persist complete checkpoint/request metadata atomically; then move the published ref as the final success step. `commit-tree` plus an explicit publish-last `update-ref` is preferable to moving HEAD and trying to undo after failure. Preserve original Git index/HEAD and existing file permissions. Clean only owned temporary index/metadata files on failure. An orphan unpublished Git object/metadata entry must never appear in list or be accepted by restore. + + List and restore operate only on published complete checkpoints. Distinguish an actually empty repository from Git-log/storage errors instead of swallowing all errors. A metadata-write or publish failure fails the request, leaves the prior published head usable and supplies no new valid target. Revalidate readability when capture operations report changes/errors; report incompleteness rather than converting it into a valid rewind target. Do not claim transaction-like consistency against arbitrary independent writers; the required ordering guarantee is that this pane's provider turn has not begun before its successful snapshot finishes. + +- [ ] **Step 4: Own capture and provider-send release in the server.** Add optional modern checkpoint intent and stable logical attempt ID to the existing freshAgent.send frame, with frozen provider/session/scope and observed fence. Local echo/pending state starts immediately; the client sends one command, without an independent automatic checkpoint POST or browser timer releasing a second send. Inject a narrow checkpoint-service trait into the WS crate from server main/checkpoint state; intercept before both managed gateway and local provider-send dispatch and run one owned asynchronous barrier-and-forward job so the receive loop stays responsive. Direct REST capture/list/metadata/restore use the same service/storage/locks. + + Capture budget begins before workspace-lock acquisition. The service owns all Git children/publication work; timeout requests cancellation and waits for actual owned termination/settlement before releasing a send. Dropping a future or aborting browser HTTP is insufficient. Reconcile timeout racing update-ref under the workspace lock: either complete publication already succeeded while send was held, so retain Available, or it did not, so return Unavailable preserving prior targets. No remaining work may publish an attempt after terminal settlement/provider-send release. Do not promise absolute-clock/ref atomicity. Add a scoped owned Git runner with retained children/stdout/stderr tasks, explicit POSIX process-group termination/reaping/group settlement and supported Windows JobObject ownership before child execution, followed by direct wait and actual-job/process-count settlement. A dropped future, taskkill example or kill_on_drop alone is insufficient. Reconcile ref/complete metadata before terminalizing ambiguous publication. Destructive tests stay in the sandbox; preserve supported Windows capture rather than disabling its timeout behavior. + + Preserve the immutable send and validate captured conversation/session ownership and observed epoch/generation again after capture for every managed/local provider. Existing managed proxy drops observed fences and Claude does not enforce them, so implement the narrow post-delay admission rather than assuming it exists. Never reread latest pane/session/fence to retarget the request. A rejected stale/closed conversation is a send rejection distinct from snapshot failure. Within the process duplicate transport requests join the same barrier-and-forward job; known-rejected retries in the same canonical conversation may alias their new transport ID to the same logical capture result, never recapture that attempt. Ambiguous acceptance follows existing no-redispatch rules. A different conversation gets a separate logical attempt. Queue draining uses the same service immediately before forwarding only when capture has not settled. + + Persist one minimal positive/negative outcome independently from ephemeral metadata-patch bookkeeping: schema/version, logical attempt and transport aliases, canonical conversation, captured workspace, Available checkpoint ID or Unavailable code/reason, and native/display user-row join when established. No new prompt-text storage. Successful target authority remains complete server publication; pane persistence retains pending/current UI state and failed attempts. Metadata/outcome storage failure yields Unknown/Unavailable, never label-derived success. Restore uses captured scope and actual published target. + + Bind outcomes using real provider IDs, not optimistic echo: Codex native accepted turn ID already exists, but mixed history projects the user as turn-N:row-0 rather than turn-N; retain explicit native turn identity in the strict shared row schema and bind actual user display rows without suffix guessing or rewriting historical IDs. Claude's pinnedSDK0.3.237/bundle2.1.237 synthetic probe confirmed supplied UUID in real user replay and persisted transcript: pass a valid per-message UUID through sidecar SDKUserMessage, enable replay-user-messages, relay the actual user replay/native identity with request correlation. OpenCode1.18.21 exact source supports prompt.messageID persisted as info.id: carry a valid native message ID through prompt construction/compact queue and bind observed native history. Accepted queue acknowledgement alone is not persisted-user proof. Preserve session materialization aliases. A generated ID is usable only if actual native history carries that same identity; otherwise linkage remains unavailable and is not claimed complete. Modern attempted rows never fall back by text/ordinal; retain label fallback only for demonstrably legacy rows. Durable modern/version evidence or a safely established feature boundary survives failed result writes/store loss so absence cannot become legacy success. If turn binding/result persistence is unavailable, that row remains explicitly unknown/unavailable; another provider's success is not universal coverage. Desktop/touch/context menus share the same availability: pending/failed/unavailable disables rewind with a readable reason, success enables its exact turn. Snapshot failure still forwards one usable message after settlement and retains its reason after notices/reload; client timeout/disconnect cannot release another send. + +- [ ] **Step 5: Structured failures and safe client details.** Replace create-path error strings with a small typed checkpoint failure carrying stage/operation, stable code, resolved workspace, optional offending path, underlying filesystem kind/errno or Git exit status and bounded cause. Emit one warning for expected scope/readability unavailability and one error for unexpected Git/storage failure, with an event such as `code_checkpoint.failed`; include the send correlation as `send_request_id`, and preserve the HTTP span's `request_id`. When known, include filesystem path directly from traversal; Git failures with no reliably extracted path report an explicit operation and bounded stderr cause. Do not parse stderr as authority about complete storage. Do not log labels, commit argv, request body, outgoing text, raw configuration, or file contents. Exercise failure logs through the real HTTP request/logger, assert correlation and meaningful cause, and assert synthetic outgoing-message/credential markers are absent. Keep the client's human reason separate from internal failure context; response JSON includes code/details needed by per-turn availability while retaining normal API error compatibility. + +- [ ] **Step 6: Complete route smoke with actual restricted identity.** Add one fixed checkpoint-fixture initialization mode in the documented sandbox wrapper/entrypoint trusted root phase: create only run-owned synthetic tmpfs fixture roots with server UID B-owned writable project/storage/home parents and root/UID A-owned0600 file/0700 directory, then permanently drop to B with no DAC capability. Existing --runtime-suite is networkless/single-UID and cannot arrange this unchanged. B can rename those prepared private entries through its own writable parent to vary eligibility without chmod/read or privileged control service. Launch the real compiled checkpoint server under B; loopback HTTP smoke needs no external network. Create only synthetic accessible project/code paths plus one A-owned 0600 file and one non-traversable A-owned directory. Verify B cannot read them by an actual open/read operation before issuing the route; test fails if B is root or has DAC override capability. Point server storage/home at a B-writable synthetic directory. Do not chmod the private source to make capture pass. + + Exercise HTTP create/list/metadata/restore: first complete non-Git/Git snapshots restore all eligible fixture files, executable bits and symlink semantics; after adding the inaccessible eligible file/directory a create fails with workspace/path/cause and list retains only prior successes; a guessed/unpublished failed commit cannot restore; repeated reads do not erase or duplicate result state; fixing the synthetic fixture deliberately and issuing a new request succeeds. Include an injected real storage-write failure after Git object creation to prove publication-last, not merely staging failure. Verify original repository state and source permissions remain unchanged. Include explicit project while account-home launch is supplied and no-scope home refusal without reading home secrets. Capture JSONL events and assert send/HTTP correlations, stage, cause and no message/secret marker. Kill/restart portions, if any, execute only inside `scripts/sandbox-test.sh` and affect recorded owned PIDs. + +- [ ] **Step 7: Actually running configured-cloud browser proof.** Add a focused spec outside `CLOUD_SKIP_SPECS` rather than counting the existing skipped rewind test. Reuse Task6's owned external-target/tunnel handoff with a separate checkpoint target container using the same fixed trusted initialization and restricted UID B. It uses an ordinary dedicated bridge and only an ephemeral host-loopback published port, read-only artifacts/root, synthetic writable tmpfs, no host network/user HOME/corpus/Docker socket. The networkless destructive runtime-suite container is not browser reachable. The target driver owns exact container/tunnel names, observations and cleanup; private fixture files remain inside that target, not Cloud Run. Seed fixture paths and collect disk/log observations through the existing test-owned target harness/control channel, not a production testing endpoint. The permission-failure server must have the restricted identity from Step 6. No provider network/model dependency is required: select the direct Codex app-server fixture because actual Codex acceptance supplies a native turn ID. Reuse the shared direct fake-app-server.mjs behavior.recordTurns mode, whose turn/start and thread/read already persist matching distinct native turns/user rows; extend only missing fixture behavior and exercise the production Codex adapter/projector's native-to-display identity. Do not inject display IDs into UI or reuse assistant-only/static-ID history. Validate the chosen transport through actual fresh-agent send/accepted/snapshot behavior before relying on it. + + Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately cover home launch with explicit project scope and home launch without scope; the former snapshots only project code, the latter sends successfully with a clear unavailable reason and no home checkpoint. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. + +- [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the required broad gate once Tasks 1–6 join. Validate no foreign files or permission changes entered the diff. Commit Task 7 atomically with descriptive test evidence. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. + Focused commands: `mise exec rust@1.96 -- cargo test -p freshell-server --locked checkpoints`; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx --config config/vitest/vitest.config.ts`. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-server --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs via the Task6 owned external-target driver with configured cloud selection; expected positive executed/pass count, real accepted turns and successful/failed/unavailable results, no skipped/zero-test coverage. + ## Acceptance and completion boundary -1. Map freshell#bdpt's exact latest 11 criteria to the executed task/test evidence in the parent run record; the user's requested provider and recovery additions remain mandatory. Do not close the kata or claim full completion with unresolved schema/reference or cloud managed browser coverage. +1. Map freshell#bdpt's exact latest 15 criteria to the executed task/test evidence in the parent run record; the user's requested provider and recovery additions remain mandatory. Do not close the kata or claim full completion with unresolved schema/reference or cloud managed browser coverage. 2. All code tasks require recorded behavioral red, green, refactor and affected-test evidence. Synthetic files/commands are intentional fixtures; production preparation, grants, ownership, terminal transport and browser lifecycle are real. No static prose/config hash tests qualify. 3. After all six tasks' commits and specification-plus-quality reviews, finish the worktree clean and record the final committed HEAD. Run the separate required no-write typecheck `pnpm run typecheck:client`, then the single final coordinated broad gate `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed provider environment and recovery final committed HEAD acceptance" pnpm run test`. These commands are on the final committed candidate, not a precommit Task 6 tree; preserve separately needed focused managed-feature, integration, browser and sandbox evidence from the tasks. Record command/results and verified HEAD, then push the feature branch with required prepush checks. A subsequent source/review fix requires a new focused commit and appropriate renewed verification of that changed final HEAD; do not rerun an unchanged broad gate merely because a task boundary passed. Stop before PR creation until explicit approval. Do not deploy/restart the live server while finishing the branch. 4. Once separately approved, follow the managed rollout runbook, preserve registry/runtime/image/secrets roots, and recheck original OpenCode failure via Freshell MCP. Assert usable startup and existing configuration without reading/logging secret contents. This operational residual is accepted pending approval, whereas missing implementation or configured-cloud test evidence is not. -## Draft self-review +## Dispatcher self-review -The dispatcher-authored User Request is copied once unchanged. Tasks cover both copy stages, executable/resource/dependency closure, actual supported reference transport, transactions, conservative recovery evidence, typed final status, retained multi-page diagnostics, bounded logs and successful retry. Env-before-file now has a concrete bounded pre-mount OneCLI resolution phase, shared decoder/transformer, child-only generated substitution and freshness checks; both terminal/fresh four-reference ceilings are explicitly replaced. Exact source-form verification, synthetic sequencing/freshness proof and actual managed cloud browser topology remain visible Stage 2 gates; the plan does not claim those assumptions are already validated. Paths and known interfaces derive from the supplied exploration reports at base `d0cc7cb5e038811be0fb3ff5f4fc5812d2723228`. No repo mutation, test, worktree creation or commit was performed by this plan writer. +The exact sanitized User Request occurs once and includes all15current kata criteria. Seven tasks retain behavioral red/green/refactor, focused commits/reviews, actual managed environment smoke, genuinely configured-cloud browser proof, conservative recovery/store decisions, durable retained diagnostics and code-rewind scope/completeness/turn outcomes. Source/ownership/protocol corrections are reflected in task inputs and acceptance; empirical claims are not certified from planned code. Environment copy, secrets and callback metadata have explicit separate profiles, with one policy across both stages and no lower hidden ownership limit. Code-rewind uses server-owned send/capture settlement, real provider identity and fixed restricted fixtures rather than browser timing or fake display IDs. Original workspace/native data and conversation semantics remain protected. Final whole-suite runs on committed final HEAD; PR/live rollout/MCP boundaries remain separately approved. Ordinary provider matrix requirements cannot be replaced by arbitrary-program certification or an unsupported-error workaround for a required reference. Reports remain outside tracked source; plan/code snippets are drafts, not implementation evidence. From c93c37afde8c68e0f7ead0f343804aba2ea5dce6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 19:56:42 -0700 Subject: [PATCH 003/493] docs: bound prepared metadata to private source descriptors --- docs/plans/2026-10-05-managed-provider-environment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index b727e143d..233e7a2ae 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -200,7 +200,7 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, Keep exact accepted original sources and ordered logical occurrence metadata. Produce each lane's current unexpanded normalization/owned edits before making recipes: raw main documents retain comments/escapes, inherited inline normalizes after MCP edits, selected rebind TUI normalizes after its existing plugin edit, ordinary TUI retains raw form. Plan one transient document-level env pass followed by the pinned snapshotted file-match pass with full line-prefix context, including relevant key lexemes and partial inserted tokens; decoded strings and original offsets across later serialization are insufficient. Source-view generation/offsets belong to the effective unexpanded document. For env-introduced file tokens or secret prefix/suffix, use a bounded private document-context/source-event recipe beside that source; a contained raw lexeme placeholder is allowed only when the document-level match proves containment. Preserve the single pass, raw escapes, comment-prefix rule and introduced env tokens remaining literal. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded effective source/span with lane/view identity and full-context event metadata, variable provenance, approved selections and runtime destinations; never expanded values. Cross-lexeme/outside-lexeme accepted occurrences require a bounded whole-effective-document child-only recipe: validate/perform owned edits on retained accepted source first, then emit a native env wrapper. Its expanded document exists only in the child secret value and native parser input; wrapper consumption is a mandatory actual-provider fixture, not assumed proof. Original-form alias families may instead preserve native substitution, per the selected C05 representation. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret and source-specific auth lookup before spawn; auth must not shadow a generated name. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/32 MiB combined (capacity validation may revise this explicit profile before implementation) and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. Publish indexed files into one attempt-owned local-driver tmpfs volume using fixed nonsecret options type=tmpfs/device=tmpfs/o=size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec. The provider-UID publisher creates normal files/files-ordinal components with0600, receives at most64KiB raw chunks with one outstanding acknowledgement and encoded frames below1MiB, and logs no payload. Its exact exchange-directory socket is private and supervised concurrently; Docker logging isnone. Typed VolumeOptions.Subpath regular-file mounts with NoCopy=true/ReadOnly=true expose indexed sources at approved exact aliases or private indexed destinations. Keep publisher and workload active mount references overlapping: population acknowledgement returns before workload creation; supervisor retains publisher until successful mount/startup GrantAccepted and then releases it. A stopped volume object retains no tmpfs content; cold last-unmount requires authorized repopulation. Allow at most one active and one pending64MiB volume per soul; abort/uncertain-delivery/startup cleanup uses exact preparation ownership and cannot accumulate generations. Legacy small auth publication remains separate. Use existing child-secret map for finite selected original-form file-path inputs only, never ambientenv forwarding. Validate every logical/effective destination, uniqueness/content identity, profile and reserved mount collision before spawn. Workspace file/root mountpoints and their parents must already exist; Docker must not create host-workspace entries. Outside-root mountpoints may be constructed in the container image; provider-owned volume placeholders require explicit precreation. For ordinary configured resource/plugin paths, copy complete package/resource closure via Task 1 and rewrite only recognized field nodes to the corresponding projected paths, preserving adjacent imports. Include complete selected implicit external config/skill/plugin roots, not just config leaves. Precreate descriptor-owned physical .freshell/provider-roots/ subdirectories/placeholders in the exact soul volume before main mounting, under initial ownership or retained provider UID as appropriate. Mount only selected subpaths at required discovery paths with NoCopy=true and writable provider behavior. Populate via canonical HOME paths with the common provider-owned transaction; keep mounted root inodes stable and replace only owned leaves. Private reference placeholders are explicit exclusions, never copied bytes. Adopt compares HostConfig.Mounts subpath/NoCopy plus actual mount type/name/source/destination/RW and exact owned volume options; flattened Mounts listing has no subpath field. Keep ordered main/TUI logical source occurrences separately from canonical read identities. Ordinary independent source destinations can use exact run-owned config binds, including discovered outside-workspace ancestors at their original logical filenames. Shared workspace-symlink families retain origin-dependent relative references and expose each selected target through exact private aliases; only source-independent absolute/HOME-sensitive collisions get identical selective rewrites to distinct indexed destinations. A shared child-only document recipe requires compatible lane evaluation; differing source-specific auth is not implicitly flattened. Keep physical HOME for all supported workspace topologies; the considered logical-HOME simplification is not selected because default-HOME workspaces can be masked or mutated. Preserve native original-form evaluation where exact target aliases are injective; generated shared recipes may not flatten differing auth lanes. Validate representability/collisions before dispatch, retain original data and return classified preparation conflict rather than wrong payload or disabled capability. Do not add parser/document machinery solely to certify arbitrary env-injected JSON programs outside the requested ordinary field/reference matrix; actual required matrix/consumer fixtures remain acceptance gates. Never deduplicate config occurrences by inode or attempt independent binds through symlinks that collapse to one destination. The same authoritative ordered source plan drives the preparer and native discovery. Inline origin is instance cwd and TUI/local origins follow their declaring sources, not private stage paths. - Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. Extend the existing authenticated private capability callback with one complete prepared metadata context for both terminal and fresh launches: selected roots, references, unexpanded recipes/origins, exact config mounts, new grant and prior/pending ownership IDs. No auth lookup values enter this reply. Consume it before supervisor registry preparation and immutable Docker mount publication; stop cloning stale references or separately rescanning roots. Replace immediate old-grant revocation with idempotent commit/abort. Retire prior inputs only after Host GrantAccepted, relevant registry commit and required exact-native verification; MCP activation is too early. Startup orphan cleanup protects pending/prior generations and reconciles uncertain delivery/dispatch. A lost response cannot authorize revoking both generations or redispatching a conversation. Partial new-stage/capability failures remove only this attempt's owned new inputs. Reference identity is the approved preparation snapshot: ordered source occurrence/lane/logical origin, controller-validated selection and canonical read/bind source, finite validated logical/runtime aliases, typed profile/destination and immutable candidate generation. At host preparation, freshly available auth/grant-derived candidates map only through that occurrence-specific snapshot to an authorized mounted source. Host cannot detect an unmounted original symlink retarget or canonicalize the controller tree. Controller-only ambient path inputs already consumed at planning are marked frozen selections, not fictitiously re-evaluated. Unknown/new/ambiguous aliases cause bounded controller replan before dispatch; stable and newly validated legitimate aliases must work. Use at most two pre-dispatch replan retries, preserving old inputs and recording classified failure on exhaustion. Selection identity is not inode/latest-content identity: actual bounded read consumes the mounted generation. Reuse that same read for comparison, auth publication and generated child values; no reread may silently select another plan. Next preparation resolves later original path changes. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. + Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. Extend the existing authenticated private capability callback with one complete prepared metadata context for both terminal and fresh launches: selected roots, references, unexpanded recipes/origins, exact config mounts, new grant and prior/pending ownership IDs. No auth lookup values enter this reply. Source/recipe descriptors reference immutable private staged files, view generation and offsets; no raw source/template snippets or100,000-entry manifests enter callback/launch/registry metadata. The shared1MiB control-frame ceiling is checked by bounded serialization before publication; oversized metadata is an explicit profile-limit error. Complete manifests stay on their stage/provider filesystem under the shared metadata budget. Consume it before supervisor registry preparation and immutable Docker mount publication; stop cloning stale references or separately rescanning roots. Replace immediate old-grant revocation with idempotent commit/abort. Retire prior inputs only after Host GrantAccepted, relevant registry commit and required exact-native verification; MCP activation is too early. Startup orphan cleanup protects pending/prior generations and reconciles uncertain delivery/dispatch. A lost response cannot authorize revoking both generations or redispatching a conversation. Partial new-stage/capability failures remove only this attempt's owned new inputs. Reference identity is the approved preparation snapshot: ordered source occurrence/lane/logical origin, controller-validated selection and canonical read/bind source, finite validated logical/runtime aliases, typed profile/destination and immutable candidate generation. At host preparation, freshly available auth/grant-derived candidates map only through that occurrence-specific snapshot to an authorized mounted source. Host cannot detect an unmounted original symlink retarget or canonicalize the controller tree. Controller-only ambient path inputs already consumed at planning are marked frozen selections, not fictitiously re-evaluated. Unknown/new/ambiguous aliases cause bounded controller replan before dispatch; stable and newly validated legitimate aliases must work. Use at most two pre-dispatch replan retries, preserving old inputs and recording classified failure on exhaustion. Selection identity is not inode/latest-content identity: actual bounded read consumes the mounted generation. Reuse that same read for comparison, auth publication and generated child values; no reread may silently select another plan. Next preparation resolves later original path changes. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. - [ ] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. - [ ] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. - [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. From f215aca0985d5820daff8bf12261a1124cee4cf1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 20:40:34 -0700 Subject: [PATCH 004/493] docs: repair managed provider plan decisions and verification gates --- ...2026-10-05-managed-provider-environment.md | 116 +++++++++++++----- 1 file changed, 82 insertions(+), 34 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 233e7a2ae..9f0281d02 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -41,7 +41,7 @@ Code-rewind uses the existing shadow Git store with isolated request indexes and 1. The clean `origin/main` baseline passed before this feature worktree was created at `d0cc7cb5e038811be0fb3ff5f4fc5812d2723228`. Author all changes on `the-usual/managed-provider-environment` in `.worktrees/managed-provider-environment`; this agent plan is linked from `AGENTS.md`. Preserve unrelated work and keep workflow reports outside the tracked tree. 2. Run pnpm frozen installs, existing coordinated commands, configured cloud Vitest/browser backends, and `GCLOUD_ROBOT_REQUIRE=1` for broad agent gates. Destructive process/restart/corruption tests run only in the documented disposable sandbox. The live server, real user configuration, credential contents, and unrelated agent work are outside this implementation's mutation scope. -3. All four providers use one copy policy. Defaults are 100,000 entries, 512 MiB aggregate regular-file bytes, 128 MiB per regular file, and depth 64, with checked arithmetic. Tests inject smaller bounds; no separate controller/host environment knobs or lower hidden ownership limit. Secrets retain existing stricter OneCLI limits. Configuration references have an explicit32MiB aggregate/16MiB per-source/128-source/256-alias profile; resource manifests/path/symlink metadata have a shared64MiB stored-byte budget and remain on stage/provider filesystems, not the1MiB callback. Original/effective config-source views are bounded16MiB each/64MiB aggregate simultaneous bytes, event metadata8192events/8MiB; check expansion before allocation. Final child argv/environment has an explicit1MiB aggregate accounting limit including executable/interpreters/pointers and every complete string's target32-page bound, with actual child soft stack at least8MiB. Generated and selected path inputs together allow128 bounded names (128bytes),64KiB/value and256KiB total, with control/provenance/auth collision checks. These shared profiles fail clearly instead of silently lowering effective capacity. +3. All four providers use one copy policy. Defaults are 100,000 entries, 512 MiB aggregate regular-file bytes, 128 MiB per regular file, and depth 64, with checked arithmetic. Tests inject smaller bounds; no separate controller/host environment knobs or lower hidden ownership limit. Secrets retain existing stricter OneCLI limits. Configuration references have an explicit 32 MiB aggregate/16 MiB per-source/128-source/256-alias profile; resource manifests/path/symlink metadata have a shared 64 MiB stored-byte budget and remain on stage/provider filesystems, not the 1 MiB callback. Original/effective config-source views are bounded 16 MiB each/64 MiB aggregate simultaneous bytes, event metadata 8192 events/8 MiB; check expansion before allocation. Final child argv/environment has an explicit 1 MiB aggregate accounting limit including executable/interpreters/pointers and every complete string's target 32-page bound, with actual child soft stack at least 8 MiB. Generated and selected path inputs together allow 128 bounded names (128 bytes), 64 KiB/value and 256 KiB total, with control/provenance/auth collision checks. These shared profiles fail clearly instead of silently lowering effective capacity. 4. A selected missing, unreadable, cyclic, or unsupported resource is an actionable error, never an omitted configured capability. Optional provider roots absent before selection remain optional. Exclude concrete native state/cache/history and actual credential paths, not arbitrary basenames within dependencies. 5. Preserve executable bits in provider-owned private files as `0600 | (source_mode & 0111)`, directories 0700. All controller-owned selected transport inodes use mapped group 0, files `0640 | (source_mode & 0111)` and directories 0750, including ephemeral ordinary selections; physical ownership walks never follow links. Outer controller/control ancestors stay 0700. The exact read-only input bind under `/run/freshell-private/user-provider` uses a trusted 0:0 private parent 0710, while separate control/grant files remain private. Normalize or verify the rootless-daemon-mapped group, fail clearly if unavailable, and never modify original sources. No extra capabilities or setuid/setgid/sticky preservation; rootful support is not added. 6. Never expand secret file contents into staged configuration, registry/SQLite records, grant manifests, Docker JSON, logs, or test receipts. Persist only validated reference metadata. Known configuration syntax and declaring origin control resolution; never rewrite arbitrary strings that look like paths. @@ -53,9 +53,9 @@ Code-rewind uses the existing shadow Git store with isolated request indexes and ## Load-bearing validation before implementation -Stage2 reviewed18 load-bearing claims. Required corrections are incorporated below; source-level contracts establish the chosen architecture and safe failure boundaries, while actual implementation/consumer/near-bound/browser proofs remain Stage4 acceptance gates. Root-owned synthetic cloudHTTP/WSS, Git capture/publication and pinnedSDK UUID probes passed with exact cleanup; none substitutes for managed candidate coverage. Validation used public pinned sources and synthetic fixtures, never real credential contents. +Stage 2 reviewed 18 load-bearing claims. Required corrections are incorporated below; source-level contracts establish the chosen architecture and safe failure boundaries, while actual implementation, consumer, near-bound and browser proofs remain Stage 4 acceptance gates. Root-owned synthetic cloud HTTP/WSS, Git capture/publication and pinned SDK UUID probes passed with exact cleanup; none substitutes for managed candidate coverage. Validation used public pinned sources and synthetic fixtures, never real credential contents. -1. Verify the pinned providers' actual path-bearing forms. OpenCode 1.18.21 first performs one env substitution pass, then resolves file tokens; missing/empty variables become empty strings, comment-prefixed file tokens are skipped, local file origins are their declaring parent and inline origin is instance cwd. The follow-up `reports/plan-env-file-resolution.md` supplies a concrete pre-mount transport phase and shared-parser design for nested env-dependent file paths and env-introduced file tokens. Validate the synthetic original request/server environment ordering, loader-source-specific authEnv precedence, original HOME/runtime CA mapping, raw JSONC source spans, and changed-grant detection described in Task 2. For Claude/Codex/Amplifier, record exact supported fields already consumed by their pinned implementations and the existing resource roots they load. Do not invent a universal configuration schema. Task 2's syntax adapters require that concrete supported-field table before execution. +1. Use the finite supported-field/origin matrix below for Task 2's syntax adapters and consumer fixtures. OpenCode 1.18.21 first performs one env substitution pass, then resolves file tokens; missing/empty variables become empty strings, comment-prefixed file tokens are skipped, local file origins are their declaring parent and inline origin is instance cwd. Validate the synthetic original request/server environment ordering, loader-source-specific authEnv precedence, original HOME/runtime CA mapping, raw JSONC source spans, and changed-grant detection described in Task 2. The matrix selects concrete input forms; closed-source/version-limited rows retain actual pinned-consumer acceptance rather than an unsupported-error fallback. Do not invent a universal configuration schema. 2. Prove resource symlink closure supports external linked skills, `.bin`, and package-store dependency cycles without recursing forever, losing imports, or exposing unrelated HOME. Verify bounds against a synthetic dependency tree exceeding the former 4096 entries. The controller and host both consume the shared implementation. 3. Prove project config overlays are the files OpenCode actually loads from the original mounted workspace, including ancestor discovery, nested cwd, inline and TUI references; the unused HOME project copies are insufficient. Validate overlays in both creation and adoption mount proofs. 4. Confirm native/checkpoint shared-store policies from the adapters, independent fixture checkpoint inspection, retained spool availability after stop/cleanup, and multi-page final output draining. Audit all `status == Running`, client running-count/clear-dead consumers, server `/api/terminals` directory and paged listings (`crates/freshell-server/src/terminals.rs`), MCP `list-terminals` (`tools/freshell-mcp/freshell-tool.ts:678`) and CLI `list-terminals` (`tools/freshell-cli/index.ts:945`) for the chosen blocked-facade policy. These tools currently return the API projection unchanged: correcting only browser `recoverySummary` leaves their top-level status falsely running. Keep attachability internal while projecting process state distinctly in every listing. @@ -63,7 +63,48 @@ Stage2 reviewed18 load-bearing claims. Required corrections are incorporated bel **UNRESOLVED COVERAGE GAP until item 5 passes:** A configured-cloud browser must create an actual managed OpenCode CLI pane whose child executes both production preparation stages with representative configuration, executable resources, dependency-loaded plugins, and configured file references. Recovery UI fixtures, an ordinary CLI fixture, local-only skipped qualification, and split Cargo preparation tests are useful but cannot clear this complete requirement. If no approved usable topology is established, retain FAILED/non-converged for this gate; do not silently substitute a local backend. -**Env-before-file transport decision:** Production preparation transiently resolves existing bounded OneCLI environment grants before mount planning, retaining only canonical referenced-file metadata and unexpanded templates. Session-host secret resolution uses the same pure decoder, installs transported files before provider preparation, and computes any generated substitutions in the child-only environment. The concrete Task 2 design replaces the earlier unsupported-runtime-reference fallback. Stage2 selected the source-supported mechanisms and bounded freshness contract; Stage4 validates actual synthetic execution and source/grant comparison; no explorer reads real secrets and no actual grant/environment map is persisted or logged. Ordinary supported env-before-file references must work, not merely return a clear unsupported error. +**Env-before-file transport decision:** Production preparation transiently resolves existing bounded OneCLI environment grants before mount planning, retaining only canonical referenced-file metadata and unexpanded templates. Session-host secret resolution uses the same pure decoder, installs transported files before provider preparation, and computes any generated substitutions in the child-only environment. The concrete Task 2 design replaces the earlier unsupported-runtime-reference fallback. Stage 2 selected the source-supported mechanisms and bounded freshness contract; Stage 4 validates actual synthetic execution and source/grant comparison; no explorer reads real secrets and no actual grant/environment map is persisted or logged. Ordinary supported env-before-file references must work, not merely return a clear unsupported error. + +### Finite provider field, reference and origin matrix + +R means an ordinary selected complete resource/configuration; S means private referenced payload transported through managed secrets; C means command/argv preserved as a command, without shell or arbitrary argument interpretation. Preserve remote/package identities for native provider resolution. Source documents remain private and their contents are never logged. Apply shared selection/closure and reference budgets across all rows, not a fresh budget per row. Each row requires terminal and supported fresh-path behavioral consumer fixtures; pin-limited Claude rows require the actual selected executables. + +| Provider/source | Exact selected fields or resource forms | Original origin and required handling | +|---|---|---| +| OpenCode 1.18.21 main config | Global `config.json`, `opencode.json{,c}`; explicit `OPENCODE_CONFIG`; cwd/ancestor `opencode.json{,c}`; discovered `.opencode`, HOME `.opencode`, `OPENCODE_CONFIG_DIR`; inline `OPENCODE_CONFIG_CONTENT` | Ordered native discovery: global, explicit file, root-first project, directory sources, inline. Each local file has its logical declaring parent; inline uses instance cwd. Preserve explicit source selection through managed preparation. R documents; local main loaders receive their own native authEnv overlay, inline does not. | +| OpenCode file substitution | Native `{file:...}` after the single `{env:...}` pass, including nested path variables and introduced complete tokens | S payloads. Absolute path as selected; `~/` from original HOME; relative from logical declaring parent, inline from instance cwd. Preserve trim/escaping and comment-prefix exclusion, source occurrence and main/TUI auth distinctions. | +| OpenCode plugins, resources, directories | Main/TUI `plugin[]` string or `[specifier, options]` local path/file URL; discovered `plugin`/`plugins`; `agent`/`agents`, `command`/`commands`, `mode`/`modes`; `skill`/`skills`, HOME/project `.claude/skills` and `.agents/skills`; selected package metadata/locks, installed `node_modules` | R complete skill/plugin/package/dependency units. Local plugin spec from declaring config parent. Auto resources from each discovered config directory. Preserve installed Bun/Node resolution and writable selected config roots; network reinstall cannot prove transported dependency completeness. | +| OpenCode instruction/skill selection | `instructions[]`, global `AGENTS.md`, HOME `.claude/CLAUDE.md`, project `AGENTS.md`/`CLAUDE.md`/`CONTEXT.md`; `skills.paths[]`, `skills.urls[]` | R file/glob instructions: `~/` HOME, absolute at its parent, relative searched upward from instance cwd. Skill paths use instance cwd for relative forms; preserve whole selected directory. URLs stay URLs. Respect native disable flags and first-applicable discovery. | +| OpenCode TUI/commands | Global/explicit `OPENCODE_TUI_CONFIG`, root-first ancestor and `.opencode` `tui.json{,c}`; `attention.sounds.*`; `mcp..cwd`; MCP/formatter/LSP `command[]` | TUI relative files/sounds/plugin paths use declaring TUI parent; no authEnv. TUI ancestor search has no worktree stop. MCP cwd is workspace-relative R directory. Commands are C, not file schemas; document-level native substitution still applies. | +| Codex 0.147.0 config layers | `$CODEX_HOME/config.toml`, selected `.config.toml`, cwd `config.toml`, ancestor `.codex/config.toml`; `model_instructions_file`, `experimental_compact_prompt_file`, `model_catalog_json`, `agents..config_file` | R typed input files resolve from their declaring layer parent with native HOME expansion; recursively loaded role config retains its own source parent. Preserve trust gating and layer precedence; never treat cwd fallback as config origin. | +| Codex skills/plugins | `skills.config[].path`; layer `skills`, legacy `$CODEX_HOME/skills`, HOME/cwd-to-project `.agents/skills`; selected `plugins/cache`, `plugins/data`; manifest path forms `skills`, `mcpServers`, `hooks`, `apps` | Path is an enable/disable selector, not a new discovery root. R complete discovered skill trees and selected installed plugin roots. Plugin paths from plugin root; retain inline MCP/hook forms and dependency context. | +| Codex commands | `mcp_servers..command`, `.args`, `.cwd`, `notify[]`, hook commands | Commands/argv are C. MCP cwd follows its pinned `LegacyAppPathString` loader semantics, resolved from the launch working directory when relative, rather than pretending it is a typed per-layer input-file path. Preserve actual native command semantics with fixture execution; do not rewrite arbitrary args. | +| Amplifier app `1873aa9`, foundation `0d5c520` | HOME/cwd `.amplifier/settings.yaml`, `settings.local.yaml`, session settings; `bundle.active`, `bundle.app[]`, `bundle.added`, `sources.bundles`, `sources.modules` | Preserve global → project → local → session order. Bundle names remain names; select R local `file://`, `./`, `../`, absolute bundle/module sources by pinned resolver with top-level cwd base. Project/user `.amplifier/bundles` discovery follows registered → project → user → image-packaged order. | +| Amplifier bundle resources | `includes[]`, `namespace_root`, `context.include[]`, direct `context.`, relative module `source`; selected `.amplifier/skills`, `agents`, local bundle/module resource/dependency trees | R complete selected bundle/module trees. Direct context paths from bundle base; namespace references use the retained namespace-to-source-base map, not the settings file parent. Follow finite local include graph under common cycle/budget policy. `keys.env` stays existing S OneCLI transport. Dynamic remote modules' arbitrary config is not a universal language to evaluate. | +| Claude terminal 2.1.263 / fresh SDK 0.3.237 | User/project/local settings and SDK `settingSources`; `CLAUDE.md`, `CLAUDE.local.md`, `.claude/rules`, native `@path` memory imports | R sources at actual selected HOME/project roots; imports from declaring Markdown parent (absolute also supported), recursively preserving native code-span/fence exclusions. Preserve original-form imports where possible; pinned fixture decides native support rather than a prose regex. | +| Claude skills/plugins | HOME/project/nested `.claude/skills`, `.claude/commands`; SDK `plugins[]` local paths, CLI `--plugin-dir`; `.claude-plugin/plugin.json` supported path forms `skills`, `commands`, `agents`, `hooks`, `mcpServers`, `lspServers`, `outputStyles` | R complete selected root/dependencies; manifest paths from plugin root; inline hooks/MCP stay inline. C commands and native `${CLAUDE_PLUGIN_ROOT}`/`${CLAUDE_PLUGIN_DATA}` substitution remain native. Exact pin tests gate version-limited fields; preserve configured capabilities through complete roots, not invented current-version fields. | + +State, session histories, cache-only native data, output/log locations and enterprise remote policy roots are not added as ordinary input-file selections. Actual canonical credential files are excluded from R copying, including aliases. Encountered OpenCode legacy extensionless TOML `config` is preserved and exercised through its native migration loader if selected; it is not parsed by a new general TOML/JSON adapter. Full immutable provenance and field-level evidence: `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/load-bearing-validator-f01-f02-f05.md`; env-order details: `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/plan-env-file-resolution.md`. + +### Selected C05 projection and exact initial ownership decisions + +| Source topology | Selected representation | +|---|---| +| Independent logical/effective config destinations | Exact run-owned rewritten config bind; native source origin remains recorded. Existing inline/TUI owned edits apply before recipes. | +| Shared workspace symlink source loaded from two origins | Original-form family: keep logical links/source bytes and relative spellings. Example `repo/opencode.json` and `repo/sub/opencode.json` → `repo/shared.json`: `./rules`/`./plugin.js` retain the two logical meanings. Never bind two differing rewritten copies over the one effective inode. | +| Sensitive file targets for that family | Existing HOME indexed link when sufficient; exact indexed tmpfs regular-file subpath aliases for outside-HOME or distinct effective destinations. Share an alias only for the same approved canonical source. Carry only selected original-form file-path inputs in the existing child-secret map; preserve each loader's authEnv evaluation. | +| Selected external config/skill/plugin root needing native writes | Exact provider-writable subdirectory of owned soul volume `.freshell/provider-roots/`, precreated before mount, populated by the common provider-owned transaction. Keep root inode stable, private-file placeholders excluded, provider siblings intact. No parent-tree masking. | +| Existing workspace topology | Require mountpoints/parents already present; resolve logical/effective symlink paths, detect reserved/ancestor collisions and recheck before publication/adoption. No host mkdir, link edit, source chmod or origin-dependent last-mount-wins. | + +The new indexed file-subpath publication is narrowly necessary for required exact outside aliases/shared-origin sources on the read-only container root: existing `/tmp/freshell-provider-auth` writes plus HOME links cannot place a readable file at an outside absolute destination, and a private socket is IPC rather than a pathname the native `{file:...}` loader can open. Rewriting the one shared source cannot encode two origin/auth meanings; making links at those paths would require rootfs or host-workspace writes. Use existing managed-grant authorization, profiles, framing and preparation ownership with this one bounded ephemeral mount extension; ordinary independent HOME references and legacy auth keep the smaller existing path. Do not create a general secret service or overlay framework. Source-supported subpath/lifetime contract and C05 detail: `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/load-bearing-validator-c05-final-subpaths.md` and `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/load-bearing-strategist-final-corrections.md`. + +Replace the blanket Amplifier tree transfer with recorded exact host-created paths plus their exact parent directories. For an initial `amplifier_stub::ensure_session` creation, these are `.amplifier/projects//sessions//{metadata.json,transcript.jsonl,events.jsonl}` and the newly created parent chain through `.amplifier` to HOME. An adopted existing stub grants no recursive ownership authority. Bootstrap contributes only each validated `ProviderBootstrapFile.provider_relative_path` and its created parent chain; any other host-created stub/config leaf is added by its creating operation. Shared copier selections are populated after the HOME-last handoff as provider UID and need no trusted recursive ownership walk. Deduplicate/charge all created paths against the shared 100,000-entry policy, inspect with `symlink_metadata`, use physical `lchown`, transfer deepest-first and HOME last, with no traversal of pre-existing provider state or hidden 10,000-entry cap. Tests include >10,000 selected entries, adopted siblings and a link to an unrelated target whose owner remains unchanged. + +### Named consumer and turn-identity seams + +Task 6's concrete UI consumers are `src/App.tsx` (inventory census and running counts), `src/store/panesSlice.ts` (`clearDeadTerminals`, cold normalization), `src/components/TerminalView.tsx` (create/attach/exit/compatibility ordering), `src/components/Sidebar.tsx`, `src/store/selectors/sidebarSelectors.ts`, and `src/lib/session-utils.ts` (`liveTerminalRowIdentity`, background/session lookup). Correct only affected running-versus-retained decisions; process counts use verified running state while retained handles remain reachable. Directory schema/projection and MCP/CLI pass-through coverage remain Task 5 obligations. + +Task 7's exact producer/projection inputs are `crates/freshell-protocol/src/client_messages.rs`, `server_messages.rs`, `shared/ws-protocol.ts`, `shared/fresh-agent-contract.ts`; `crates/freshell-server/src/fresh_agent_proxy.rs` and `fresh_agent_proxy_tests.rs`; `crates/freshell-ws/src/hosted_fresh_agent.rs` and local dispatch in `terminal.rs`; session-host `providers/fresh_agent.rs`; runtime protocol `fresh_agent_ops.rs`; freshagent `codex.rs`, `claude.rs`, `claude_snapshot.rs`, `opencode_ws.rs`; OpenCode `serve.rs`, `lib.rs`; Claude sidecar `index.mjs`; agent-runtime `snapshot_projection.rs` only where hosted projection must retain identity. Codex accepted native turn ID joins a role-qualified actual user row carrying additive `nativeTurnId`; display `turn-N:row-0` remains unchanged, ambiguous multiple user rows stay unavailable. Claude sends a valid UUID with logical/request correlation through SDKUserMessage, enables replay-user-messages and binds only actual replay/persisted UUID. OpenCode carries its valid `messageID` through prompt body and compact queue, then binds observed persisted `info.id`; queue acknowledgement is pending only. All retain canonical materialization aliases, never text/ordinal joins for modern attempts. Default fresh Claude uses SDK-bundled 2.1.237, separate from terminal 2.1.263; preserve an explicit approved `CLAUDE_CMD` override. Evidence: `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/load-bearing-validator-v18-checkpoint-turns-fixture.md`, `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-identity-observation.md`, and `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`. These name implementation decisions; runtime identity/consumer tests remain mandatory. ## File responsibilities and interfaces @@ -151,7 +192,7 @@ fn copied_skill_executes_sibling_resources_after_two_stages() { - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`. Expected missing implementation, then behavioral failure when the test is initially wired through current copying; record the old two-stage script execution PermissionDenied and missing dependency result, rather than treating initial compile setup as the required red evidence. Run matching controller/host tests against old code for the intended omissions before replacing it. - [ ] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Select complete skill/plugin/package resource units before mapping. External plugin entry files require their containing resource/package root, adjacent imports/assets and applicable package metadata; canonical target leaves alone are insufficient. Include existing installed node_modules lookup context and connecting ancestor skeleton without copying unrelated ancestor trees. Prefer one bounded complete installation node_modules unit for pnpm store/hoisted/.bin topology, preserving ancestor lookup order and package scope. Preplan coherent physical mappings for overlapping parent/child units, reuse one physical canonical target and preserve additional logical resource destinations as links; overlap itself is not an error. Reject incompatible destination/type claims before writing. Preserve symbolic links with rewritten relative targets and enqueue the required complete resource/package and installed lookup context under private `.freshell/resources` when outside selected roots, preserving relative ancestor/sibling hierarchy rather than flattening one canonical leaf per private ID. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. - Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure/context directory and required aliases as normal `.freshell/resources` selections in ProviderLaunchContext; transport the complete graph once and preserve its relative topology identically at the second stage, without independent package rediscovery or renesting. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the accepted report's exact paths/parents. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. + Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure/context directory and required aliases as normal `.freshell/resources` selections in ProviderLaunchContext; transport the complete graph once and preserve its relative topology identically at the second stage, without independent package rediscovery or renesting. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the exact created leaf/parent ledger in Selected C05 projection and exact initial ownership decisions; carry it from the creating operations, not a fresh recursive HOME walk. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. - [ ] **Step 4: Green.** Repeat focused platform/controller/host commands; expected executable and Node resource/dependency assertions pass with correct boundary errors. - [ ] **Step 5: Refactor.** Remove the two obsolete private copier policies and basename-based secret filtering; keep provider selection and ownership at callers and filesystem behavior in the common module. Avoid adding environment policy configuration. - [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`. Expected all pass. These cover the shared policy and every existing caller. @@ -167,7 +208,7 @@ fn copied_skill_executes_sibling_resources_after_two_stages() { - Modify: supervisor `service.rs`, callback control protocol and controller `managed_mcp_capability.rs` for complete prepared context, callback-safe attempt-bound native-auth lookup and idempotent generation commit/abort; extend the narrow existing `backend/native_history.rs` owned-helper lifecycle in a dedicated auth helper module and fixed host subcommand, plus exact disposable broker helper allowance. - Test: planner tests, protocol validation tests, managed capability/reissue/lost-response and startup-orphan reconciliation tests, host secret-resolution tests, Docker create/adoption validation tests. -**Interfaces:** `ReferencePlan` owns rewritten config templates plus bounded resource selections and `ProviderSecretReference` metadata; it stores no substituted referenced bytes. It consumes validated provider, original HOME, workspace root, project cwd, declaring source and child-independent known configuration syntax. Add finite `ClaudeConfigurationFile`, `CodexConfigurationFile`, `OpencodeConfigurationFile`, and `AmplifierConfigurationFile` profiles and defaulted `provider_relative_path: Option` on `ProviderSecretReference`; old profiles require None and keep their static destinations. New destinations are normal relative paths in `.freshell/references/`, maximum 128 unique references, no other provider or duplicate destination. Configuration payload publication is indexed, bounded and chunk-streamed through private IPC, not a Vec of complete large file payloads. Retain legacy small provider-file handling separately. Reference metadata includes finite indexed source IDs and exact approved alias destinations (maximum256 across the plan), with logical/effective destination topology. TerminalMounts additionally carries exact config binds and writable external resource roots backed by physical subdirectories of the owned soul volume. All terminal/fresh create, prepared replacement, digest and cold-adoption paths share these descriptors; no payload values enter them. +**Interfaces:** `ReferencePlan` owns rewritten config templates plus bounded resource selections and `ProviderSecretReference` metadata; it stores no substituted referenced bytes. It consumes validated provider, original HOME, workspace root, project cwd, declaring source and child-independent known configuration syntax. Add finite `ClaudeConfigurationFile`, `CodexConfigurationFile`, `OpencodeConfigurationFile`, and `AmplifierConfigurationFile` profiles and defaulted `provider_relative_path: Option` on `ProviderSecretReference`; old profiles require None and keep their static destinations. New destinations are normal relative paths in `.freshell/references/`, maximum 128 unique references, no other provider or duplicate destination. Configuration payload publication is indexed, bounded and chunk-streamed through private IPC, not a Vec of complete large file payloads. Retain legacy small provider-file handling separately. Reference metadata includes finite indexed source IDs and exact approved alias destinations (maximum 256 across the plan), with logical/effective destination topology. TerminalMounts additionally carries exact config binds and writable external resource roots backed by physical subdirectories of the owned soul volume. All terminal/fresh create, prepared replacement, digest and cold-adoption paths share these descriptors; no payload values enter them. ```rust // freshell-platform::provider_environment_grants: no runtime-protocol import. @@ -191,16 +232,16 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, `GrantError` and `ReferenceError` retain classified safe operation/profile/field/source-ordinal data and never include raw grant lines, expanded values or referenced bytes. `ResolvedFileReferencePlan` contains canonical sources, declaring source locations/ordinals, runtime destinations, referenced variable names, generated-placeholder recipes and unexpanded templates only; it has no transient expanded-text/map field and no derived Debug exposing private input. Controller and host translate existing finite protocol profiles to `EnvironmentProfile` in their wrappers. -- [ ] **Step 1: Add failing consumers.** Table-drive absolute, relative, `~/`, spaces, in-root symlinks and external references for each verified field form from Stage 2. OpenCode cases cover global JSON/JSONC, original-workspace project config at nested cwd and ancestor, TUI, inline instance-cwd origin, nested `{file:{env:NODE_EXTRA_CA_CERTS}}`, one-pass env substitutions that introduce complete file tokens plus secret prefix/suffix text, original ambient/request path variables, missing/empty-variable semantics, comment-prefixed tokens, runtime CA destination→source mapping, loader-source-specific authEnv precedence, local path/file-URL plugins and instruction resources. More than four valid file references must work for terminal and fresh launch specs; duplicate destination, excessive per-profile/aggregate counts and cross-provider reference must fail before mounts. Change a synthetic grant's path between planning and spawn: dispatch is prevented with changed-input retry, previous usable stage survives, no different file is silently consumed. +- [ ] **Step 1: Add failing consumers.** Table-drive absolute, relative, `~/`, spaces, in-root symlinks and external references for each selected field/origin row in the inline finite provider matrix. OpenCode cases cover global JSON/JSONC, original-workspace project config at nested cwd and ancestor, TUI, inline instance-cwd origin, nested `{file:{env:NODE_EXTRA_CA_CERTS}}`, one-pass env substitutions that introduce complete file tokens plus secret prefix/suffix text, original ambient/request path variables, missing/empty-variable semantics, comment-prefixed tokens, runtime CA destination→source mapping, loader-source-specific authEnv precedence, local path/file-URL plugins and instruction resources. More than four valid file references must work for terminal and fresh launch specs; duplicate destination, excessive per-profile/aggregate counts and cross-provider reference must fail before mounts. Change a synthetic grant's path between planning and spawn: dispatch is prevented with changed-input retry, previous usable stage survives, no different file is silently consumed. The child executes the real supported configuration resolver where available and otherwise the pinned resolver-equivalent synthetic provider fixture; report the actual file-consumption nonce/digest and native trim/escape behavior, never the referenced bytes. Distinguish same relative spelling in two declaring directories. Copy original config bytes beforehand and assert they remain unchanged afterward. Missing/unreadable/dangling/cyclic referenced targets must prevent launch with an actionable named field/path, no capability omission. Sentinel credential and generated-placeholder cases assert absence from staged templates, durable registry/JSON, grant metadata, Docker inspect and logs while child consumption succeeds. Test initial launch, resume/reissue and both terminal/fresh routes using synthetic grants only. Cover exact native auth fallback and typed-grant precedence, new absent versus unexpected/missing retained volumes, callback nested lookup without deadlock, duplicate auth-key order, same-path changed reference selection, timeout/lost helper acknowledgement and exact cleanup. Capture sentinel-bearing Docker logs/create bodies, IPC prepared metadata, registry/caches/receipts and staged ordinary roots: auth values and unvalidated candidate expressions must be absent. Pure decoder tests must preserve current interpolation, provider allowlists, proxy validation and unknown/legacy-profile rejection, and detect actual 128 KiB+1 read growth. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_references`; existing stage/host consumer tests must fail because original absolute/relative references are unresolved in runtime. A compile-only missing module is setup evidence, not behavioral red. - [ ] **Step 3: Apply precise implementation patches.** Extract `provider_secret_resolution.rs:88-432`'s existing pure keys/env parsing, interpolation and provider/proxy validation into `provider_environment_grants.rs`; do not create a second permissive parser. Keep auth/CA-file profiles out of environment decoding and map protocol profiles in wrappers. `read_environment_grant` verifies the canonical regular private grant and reads at most 128 KiB+1 through a bounded reader before decoding, rejecting source growth; mounted host bytes use the same pure decoder. Add platform URL dependency only if the existing validator requires it. Do not instrument raw maps/values. - Immediately before capability stage/publication and final mount construction, controller builds a transient original host/provider env lookup with request overrides for terminal launches, current server launch env for fresh launches, then approved OneCLI child env overlay matching direct child precedence. Use original host HOME/cwd/config origins. Select only file-expression variable needs for reference planning; never expand the durable environment allowlist in `managed_runtime.rs:714`. Apply well-known authEnv only to pinned local loader sources; inline/TUI receive no overlay. Prefer the selected typed OpencodeOnecliAuthFile, read through a bounded128KiB private reader and shared order-preserving auth decoder; do not substitute host user's auth or add generic OPENCODE_AUTH_CONTENT forwarding. Without a grant, perform a finite selected-key lookup from the exact registry-owned quiescent soul volume before mount planning. First-launch empty auth needs explicit installation/soul intent authority plus confirmed expected-volume absence, checked again at launch; missing native ID, unavailable daemon or unexpected volume is not new-absence proof. Use one fixed provider-UID cold helper and existing owned create/start/wait/delete pattern, with only the exact read-only soul volume and a single-use private exchange-directory socket. Auth reply is bounded memory-only, has no value-printing Debug/cache/durable/log path, and Docker logging is disabled; fixed sanitized failures only. Accept/framed exchange and helper supervision run concurrently within one aligned operation/callback deadline. Keep socket path within sun_path bounds and mapped-group directory0710/socket0660 access. For replacement callback, invoke an attempt-bound lookup under the recovery ownership already held, never reacquire the same soul lock or await IPC with capability-store/registry locks held. No live worker is stopped for a lookup. Host recomputes validated reference/mount metadata from actual selected auth/grant bytes before dispatch; same grant/path with changed reference target triggers bounded replan/failure, while unrelated auth changes need not. Retain native auth unless matching typed grant replaces it. Drop decoded maps after obtaining canonical source metadata and unexpanded templates; controller reads only approved payloads through bounded transient managed-secret publication, never ordinary-stage expansion. + Immediately before capability stage/publication and final mount construction, controller builds a transient original host/provider env lookup with request overrides for terminal launches, current server launch env for fresh launches, then approved OneCLI child env overlay matching direct child precedence. Use original host HOME/cwd/config origins. Select only file-expression variable needs for reference planning; never expand the durable environment allowlist in `managed_runtime.rs:714`. Apply well-known authEnv only to pinned local loader sources; inline/TUI receive no overlay. Prefer the selected typed OpencodeOnecliAuthFile, read through a bounded 128 KiB private reader and shared order-preserving auth decoder; do not substitute host user's auth or add generic OPENCODE_AUTH_CONTENT forwarding. Without a grant, perform a finite selected-key lookup from the exact registry-owned quiescent soul volume before mount planning. First-launch empty auth needs explicit installation/soul intent authority plus confirmed expected-volume absence, checked again at launch; missing native ID, unavailable daemon or unexpected volume is not new-absence proof. Use one fixed provider-UID cold helper and existing owned create/start/wait/delete pattern, with only the exact read-only soul volume and a single-use private exchange-directory socket. Auth reply is bounded memory-only, has no value-printing Debug/cache/durable/log path, and Docker logging is disabled; fixed sanitized failures only. Accept/framed exchange and helper supervision run concurrently within one aligned operation/callback deadline. Keep socket path within sun_path bounds and mapped-group directory 0710/socket 0660 access. For replacement callback, invoke an attempt-bound lookup under the recovery ownership already held, never reacquire the same soul lock or await IPC with capability-store/registry locks held. No live worker is stopped for a lookup. Host recomputes validated reference/mount metadata from actual selected auth/grant bytes before dispatch; same grant/path with changed reference target triggers bounded replan/failure, while unrelated auth changes need not. Retain native auth unless matching typed grant replaces it. Drop decoded maps after obtaining canonical source metadata and unexpanded templates; controller reads only approved payloads through bounded transient managed-secret publication, never ordinary-stage expansion. Add a small shared token/string transformer to existing `opencode_config.rs` with the pinned single-env-pass→file-scan order. Recognize an outer file expression containing complete nested env tokens so the inner brace cannot terminate the outer match. Expand only its path transiently; missing/empty env is empty text as upstream, then resolve absolute/relative/`~/` against original origin. Canonicalize/check source metadata and replace the whole outer expression with `{file:/home/freshell/provider/.freshell/references/}` in staged copies; preserve unrelated env tokens. Do not try to canonicalize the runtime HOME on the controller: map a known runtime auth/CA file destination back to its existing typed source reference and reuse its grant/destination. Source errors identify original field/ordinal and variable names, not raw expanded credential values. - Keep exact accepted original sources and ordered logical occurrence metadata. Produce each lane's current unexpanded normalization/owned edits before making recipes: raw main documents retain comments/escapes, inherited inline normalizes after MCP edits, selected rebind TUI normalizes after its existing plugin edit, ordinary TUI retains raw form. Plan one transient document-level env pass followed by the pinned snapshotted file-match pass with full line-prefix context, including relevant key lexemes and partial inserted tokens; decoded strings and original offsets across later serialization are insufficient. Source-view generation/offsets belong to the effective unexpanded document. For env-introduced file tokens or secret prefix/suffix, use a bounded private document-context/source-event recipe beside that source; a contained raw lexeme placeholder is allowed only when the document-level match proves containment. Preserve the single pass, raw escapes, comment-prefix rule and introduced env tokens remaining literal. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded effective source/span with lane/view identity and full-context event metadata, variable provenance, approved selections and runtime destinations; never expanded values. Cross-lexeme/outside-lexeme accepted occurrences require a bounded whole-effective-document child-only recipe: validate/perform owned edits on retained accepted source first, then emit a native env wrapper. Its expanded document exists only in the child secret value and native parser input; wrapper consumption is a mandatory actual-provider fixture, not assumed proof. Original-form alias families may instead preserve native substitution, per the selected C05 representation. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret and source-specific auth lookup before spawn; auth must not shadow a generated name. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. - Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/32 MiB combined (capacity validation may revise this explicit profile before implementation) and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. Publish indexed files into one attempt-owned local-driver tmpfs volume using fixed nonsecret options type=tmpfs/device=tmpfs/o=size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec. The provider-UID publisher creates normal files/files-ordinal components with0600, receives at most64KiB raw chunks with one outstanding acknowledgement and encoded frames below1MiB, and logs no payload. Its exact exchange-directory socket is private and supervised concurrently; Docker logging isnone. Typed VolumeOptions.Subpath regular-file mounts with NoCopy=true/ReadOnly=true expose indexed sources at approved exact aliases or private indexed destinations. Keep publisher and workload active mount references overlapping: population acknowledgement returns before workload creation; supervisor retains publisher until successful mount/startup GrantAccepted and then releases it. A stopped volume object retains no tmpfs content; cold last-unmount requires authorized repopulation. Allow at most one active and one pending64MiB volume per soul; abort/uncertain-delivery/startup cleanup uses exact preparation ownership and cannot accumulate generations. Legacy small auth publication remains separate. Use existing child-secret map for finite selected original-form file-path inputs only, never ambientenv forwarding. Validate every logical/effective destination, uniqueness/content identity, profile and reserved mount collision before spawn. Workspace file/root mountpoints and their parents must already exist; Docker must not create host-workspace entries. Outside-root mountpoints may be constructed in the container image; provider-owned volume placeholders require explicit precreation. + Keep exact accepted original sources and ordered logical occurrence metadata. Produce each lane's current unexpanded normalization/owned edits before making recipes: raw main documents retain comments/escapes, inherited inline normalizes after MCP edits, selected rebind TUI normalizes after its existing plugin edit, ordinary TUI retains raw form. Plan one transient document-level env pass followed by the pinned snapshotted file-match pass with full line-prefix context, including relevant key lexemes and partial inserted tokens; decoded strings and original offsets across later serialization are insufficient. Source-view generation/offsets belong to the effective unexpanded document. For env-introduced file tokens or secret prefix/suffix, use a bounded private document-context/source-event recipe beside that source; a contained raw lexeme placeholder is allowed only when the document-level match proves containment. Preserve the single pass, raw escapes, comment-prefix rule and introduced env tokens remaining literal. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded effective source/span with lane/view identity and full-context event metadata, variable provenance, approved selections and runtime destinations; never expanded values. Cross-lexeme/outside-lexeme accepted occurrences require a bounded whole-effective-document child-only recipe: validate/perform owned edits on retained accepted source first, then emit a native env wrapper. Its expanded document exists only in the child secret value and native parser input; wrapper consumption is a mandatory actual-provider fixture, not assumed proof. Original-form alias families may instead preserve native substitution, per the inline Selected C05 projection table. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret and source-specific auth lookup before spawn; auth must not shadow a generated name. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. + Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/32 MiB combined (capacity validation may revise this explicit profile before implementation) and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. For the exact outside/shared-origin aliases identified in the C05 table, publish indexed files into one attempt-owned local-driver tmpfs volume using fixed nonsecret options type=tmpfs/device=tmpfs/o=size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec. The provider-UID publisher creates normal files/files-ordinal components with 0600, receives at most 64 KiB raw chunks with one outstanding acknowledgement and encoded frames below 1 MiB, and logs no payload. Its exact exchange-directory socket is private and supervised concurrently; Docker logging is none. Typed VolumeOptions.Subpath regular-file mounts with NoCopy=true/ReadOnly=true expose indexed sources at approved exact aliases or private indexed destinations. Keep publisher and workload active mount references overlapping: population acknowledgement returns before workload creation; supervisor retains publisher until successful mount/startup GrantAccepted and then releases it. A stopped volume object retains no tmpfs content; cold last-unmount requires authorized repopulation. Allow at most one active and one pending 64 MiB volume per soul; abort/uncertain-delivery/startup cleanup uses exact preparation ownership and cannot accumulate generations. Legacy small auth publication remains separate. Use existing child-secret map for finite selected original-form file-path inputs only, never ambient env forwarding. Validate every logical/effective destination, uniqueness/content identity, profile and reserved mount collision before spawn. Workspace file/root mountpoints and their parents must already exist; Docker must not create host-workspace entries. Outside-root mountpoints may be constructed in the container image; provider-owned volume placeholders require explicit precreation. For ordinary configured resource/plugin paths, copy complete package/resource closure via Task 1 and rewrite only recognized field nodes to the corresponding projected paths, preserving adjacent imports. Include complete selected implicit external config/skill/plugin roots, not just config leaves. Precreate descriptor-owned physical .freshell/provider-roots/ subdirectories/placeholders in the exact soul volume before main mounting, under initial ownership or retained provider UID as appropriate. Mount only selected subpaths at required discovery paths with NoCopy=true and writable provider behavior. Populate via canonical HOME paths with the common provider-owned transaction; keep mounted root inodes stable and replace only owned leaves. Private reference placeholders are explicit exclusions, never copied bytes. Adopt compares HostConfig.Mounts subpath/NoCopy plus actual mount type/name/source/destination/RW and exact owned volume options; flattened Mounts listing has no subpath field. Keep ordered main/TUI logical source occurrences separately from canonical read identities. Ordinary independent source destinations can use exact run-owned config binds, including discovered outside-workspace ancestors at their original logical filenames. Shared workspace-symlink families retain origin-dependent relative references and expose each selected target through exact private aliases; only source-independent absolute/HOME-sensitive collisions get identical selective rewrites to distinct indexed destinations. A shared child-only document recipe requires compatible lane evaluation; differing source-specific auth is not implicitly flattened. Keep physical HOME for all supported workspace topologies; the considered logical-HOME simplification is not selected because default-HOME workspaces can be masked or mutated. Preserve native original-form evaluation where exact target aliases are injective; generated shared recipes may not flatten differing auth lanes. Validate representability/collisions before dispatch, retain original data and return classified preparation conflict rather than wrong payload or disabled capability. Do not add parser/document machinery solely to certify arbitrary env-injected JSON programs outside the requested ordinary field/reference matrix; actual required matrix/consumer fixtures remain acceptance gates. Never deduplicate config occurrences by inode or attempt independent binds through symlinks that collapse to one destination. The same authoritative ordered source plan drives the preparer and native discovery. Inline origin is instance cwd and TUI/local origins follow their declaring sources, not private stage paths. - Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. Extend the existing authenticated private capability callback with one complete prepared metadata context for both terminal and fresh launches: selected roots, references, unexpanded recipes/origins, exact config mounts, new grant and prior/pending ownership IDs. No auth lookup values enter this reply. Source/recipe descriptors reference immutable private staged files, view generation and offsets; no raw source/template snippets or100,000-entry manifests enter callback/launch/registry metadata. The shared1MiB control-frame ceiling is checked by bounded serialization before publication; oversized metadata is an explicit profile-limit error. Complete manifests stay on their stage/provider filesystem under the shared metadata budget. Consume it before supervisor registry preparation and immutable Docker mount publication; stop cloning stale references or separately rescanning roots. Replace immediate old-grant revocation with idempotent commit/abort. Retire prior inputs only after Host GrantAccepted, relevant registry commit and required exact-native verification; MCP activation is too early. Startup orphan cleanup protects pending/prior generations and reconciles uncertain delivery/dispatch. A lost response cannot authorize revoking both generations or redispatching a conversation. Partial new-stage/capability failures remove only this attempt's owned new inputs. Reference identity is the approved preparation snapshot: ordered source occurrence/lane/logical origin, controller-validated selection and canonical read/bind source, finite validated logical/runtime aliases, typed profile/destination and immutable candidate generation. At host preparation, freshly available auth/grant-derived candidates map only through that occurrence-specific snapshot to an authorized mounted source. Host cannot detect an unmounted original symlink retarget or canonicalize the controller tree. Controller-only ambient path inputs already consumed at planning are marked frozen selections, not fictitiously re-evaluated. Unknown/new/ambiguous aliases cause bounded controller replan before dispatch; stable and newly validated legitimate aliases must work. Use at most two pre-dispatch replan retries, preserving old inputs and recording classified failure on exhaustion. Selection identity is not inode/latest-content identity: actual bounded read consumes the mounted generation. Reuse that same read for comparison, auth publication and generated child values; no reread may silently select another plan. Next preparation resolves later original path changes. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. + Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. Extend the existing authenticated private capability callback with one complete prepared metadata context for both terminal and fresh launches: selected roots, references, unexpanded recipes/origins, exact config mounts, new grant and prior/pending ownership IDs. No auth lookup values enter this reply. Source/recipe descriptors reference immutable private staged files, view generation and offsets; no raw source/template snippets or 100,000-entry manifests enter callback/launch/registry metadata. The shared 1 MiB control-frame ceiling is checked by bounded serialization before publication; oversized metadata is an explicit profile-limit error. Complete manifests stay on their stage/provider filesystem under the shared metadata budget. Consume it before supervisor registry preparation and immutable Docker mount publication; stop cloning stale references or separately rescanning roots. Replace immediate old-grant revocation with idempotent commit/abort. Retire prior inputs only after Host GrantAccepted, relevant registry commit and required exact-native verification; MCP activation is too early. Startup orphan cleanup protects pending/prior generations and reconciles uncertain delivery/dispatch. A lost response cannot authorize revoking both generations or redispatching a conversation. Partial new-stage/capability failures remove only this attempt's owned new inputs. Reference identity is the approved preparation snapshot: ordered source occurrence/lane/logical origin, controller-validated selection and canonical read/bind source, finite validated logical/runtime aliases, typed profile/destination and immutable candidate generation. At host preparation, freshly available auth/grant-derived candidates map only through that occurrence-specific snapshot to an authorized mounted source. Host cannot detect an unmounted original symlink retarget or canonicalize the controller tree. Controller-only ambient path inputs already consumed at planning are marked frozen selections, not fictitiously re-evaluated. Unknown/new/ambiguous aliases cause bounded controller replan before dispatch; stable and newly validated legitimate aliases must work. Use at most two pre-dispatch replan retries, preserving old inputs and recording classified failure on exhaustion. Selection identity is not inode/latest-content identity: actual bounded read consumes the mounted generation. Reuse that same read for comparison, auth publication and generated child values; no reread may silently select another plan. Next preparation resolves later original path changes. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. - [ ] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. - [ ] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. - [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. @@ -210,24 +251,25 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Files:** - Modify: host `main.rs:1281,1532,1704`; server `managed_mcp_capability.rs:637`; `crates/freshell-session-host/tests/provider_config_refresh.rs`; `test/fixtures/providers/parity-provider.mjs`; `test/integration/server/managed-provider-parity.test.ts`; its fixture-building helpers under `test/integration/server/`. +- Modify: `crates/freshell-supervisor/src/backend.rs` (private-parent tmpfs producer); `scripts/testing/runtime-test-broker.ts` (exact 0710 validator); `test/unit/tooling/testing/runtime-test-broker.test.ts` (behavioral create/adopt validation cases). **Interfaces:** Consumes `copy_environment`, `CopyReport`, `ReferencePlan` and existing `.freshell-config-projection.json`. Produces an all-or-error installed projection and unchanged previously working environment on failed refresh. Only prior manifest-owned paths may be replaced/removed. -- [ ] **Step 1: Add failing process tests.** Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. +- [ ] **Step 1: Add failing process tests.** Add backend `provider_private_parent_mount` production/adoption cases (so Task 3's `provider` selector actually runs them) and broker dispatch tests that accept the exact 0710 private-parent topology and reject wrong modes/extra authority; test through validator/dispatch behavior, not a source-text assertion. Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked --test provider_config_refresh`. Expected current stale-delete-before-copy path loses prior usable projection on injected failure, or resets executable mode. -- [ ] **Step 3: Apply precise transaction patch.** The provider UID65534/GID0 reads the exact approved read-only controller stage and creates private projection/backup inodes on its own HOME volume; it never renames or chmods transport inodes and never stages the full ordinary tree in tmpfs. Stage the complete desired projection and manifest on that same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. For terminal and fresh initial launch, finish all host-required stub/bootstrap and Amplifier work first, transfer HOME ownership last, then install selected auth links and run provider-owned preparation. No later trusted-host HOME traversal. Exact resume skips host bootstrap/ownership reclamation. Change the private-parent Docker tmpfs producer, host initial/resume reset and disposable broker validator together to 0710. Use physical lchown ownership walks and preserve ordinary execute bits. Preserve existing setpriv boundary; do not add DAC_OVERRIDE/FOWNER. +- [ ] **Step 3: Apply precise transaction patch.** The provider UID 65534/GID 0 reads the exact approved read-only controller stage and creates private projection/backup inodes on its own HOME volume; it never renames or chmods transport inodes and never stages the full ordinary tree in tmpfs. Stage the complete desired projection and manifest on that same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. For terminal and fresh initial launch, finish all host-required stub/bootstrap and Amplifier work first, transfer HOME ownership last, then install selected auth links and run provider-owned preparation. No later trusted-host HOME traversal. Exact resume skips host bootstrap/ownership reclamation. Change the private-parent Docker tmpfs producer, host initial/resume reset and disposable broker validator together to 0710. Use physical lchown ownership walks and preserve ordinary execute bits. Preserve existing setpriv boundary; do not add DAC_OVERRIDE/FOWNER. Native auth destinations survive when no corresponding typed auth-file grant is selected; CA/dynamic-file grants do not authorize auth deletion. Publish selected auth replacements before configuration/provider consumers, preserve old native destination on failure, and exclude native auth from ordinary projection manifest cleanup. Controller keeps temp-root→rename→capability publication ordering and revokes only new capability/resources on failure. Run-owned ephemeral references are cleaned with their grant; old grant/root/volume remain on failed replacement. Preparation error becomes explicit pre-spawn failure, never synthetic absence of identity. Log transaction stage/start/commit/rollback/failure once with safe counters and paths. - [ ] **Step 4: Green.** Repeat refresh and controller fault cases; expected previous environment survives every precommit failure and complete new environment runs after success. - [ ] **Step 5: Refactor.** Consolidate initial and exact-resume projection into one transaction helper; manifest authority remains limited to copied files. Share fixture setup for executable/resource/dependency cases without duplicating policy implementations in JS. -- [ ] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. Node fixtures alone do not certify OpenCode's Bun-compiled plugin loader or fresh Claude's bundled2.1.237 versus terminal2.1.263; Task6 actual pinned managed provider proof and dedicated fresh-path smoke must load the selected resources without reinstall masking omitted dependencies. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. +- [ ] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`; `pnpm run test:vitest run test/unit/tooling/testing/runtime-test-broker.test.ts --config config/vitest/vitest.config.ts`. Require positive counts and exact producer/broker/host 0710 agreement in the actual child smoke. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. Node fixtures alone do not certify OpenCode's Bun-compiled plugin loader or fresh Claude's bundled 2.1.237 versus terminal 2.1.263; Task 6 actual pinned managed provider proof and dedicated fresh-path smoke must load the selected resources without reinstall masking omitted dependencies. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. - [ ] **Step 7: Commit.** Stage listed files and `git commit -m "fix: publish managed provider refreshes transactionally"`. ### Task 4: Inspected negative recovery evidence and conservative startup failures **Files:** -- Modify: protocol `lib.rs:321`; agent runtime `lib.rs:256,824`; host `main.rs:651`, `providers/mod.rs:432`; supervisor `checkpoints.rs:15,58`, `recovery.rs:1260,1876,1928`; tests in those modules and `loss_report_tests.rs`. +- Modify: protocol `lib.rs:321`; agent runtime `lib.rs:256,824`; host `main.rs` (`ProbeRecovery`, currently line 662), `providers/mod.rs:432`; supervisor `checkpoints.rs:15,58`, `recovery.rs:1260,1876,1928`; tests in those modules and `loss_report_tests.rs`. -**Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPathKind`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. +**Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPath`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. - [ ] **Step 1: Add failing decisions.** Host dispatched-journal/no resume identity returns Blocked and preserves data. Supervisor zero checkpoint revision is blocked. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. @@ -259,36 +301,41 @@ Managed output observations distinguish Running, active Recovering, Blocked with Emit `managed_runtime.recovery_transition` only on changed state/classified cause with terminal/soul/incarnation/attempt/incident/from/to/outcome/cause fields. A controller/registry-shared transport-error fingerprint suppresses repeated unchanged output warnings across sockets and emits one recovered/change event. Do not log full probe/raw credential-bearing error strings. Facade inventory exposes additive RuntimeStatus::Blocked and existing recovery summary; final Exited follows current lifecycle handling. Server `terminals.rs::directory_items` must project managed process state from runtime status/recovery outcome before its internal facade `TerminalRunStatus` fallback. A blocked facade's user-facing `status` is blocked in plain and paged directory output, so unchanged MCP/CLI list pass-throughs also report blocked. Extend the actual directory item schema/types to permit that status; preserve ordinary terminal statuses and internal attachment/lifetime checks. If consumer rendering transforms statuses, audit and correct that transformation too. - [ ] **Step 4: Green.** Repeat focused tests; expected full final tail, accurate inventory/status, one exit and bounded transitions; successful retry behavior remains green. - [ ] **Step 5: Refactor.** Consolidate final/manual/background result publication, extract only existing spool parsing if needed, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. -- [ ] **Step 6: Impact verification.** Run the three focused Cargo commands from Step 2 plus `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked retained_output"` after new tests use that filter, and `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected all run/pass. Include existing `managed_runtime_api_stop_tests` with `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime_api_stop` to protect explicit stop. +- [ ] **Step 6: Impact verification.** Run the three focused Cargo commands from Step 2 plus `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked retained_output"` after new tests use that filter, and `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected all run/pass. Include existing `managed_runtime_api::stop_contract_tests` with `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime_api::stop_contract_tests` and record a positive executed test count; a zero-test filter fails the explicit-stop verification. - [ ] **Step 7: Commit.** Stage listed lifecycle/output files and `git commit -m "fix: retain managed terminal output and publish final recovery outcomes"`. ### Task 6: Browser status/reattachment and complete configured-cloud managed environment proof **Files:** -- Modify: `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`; actual running-status inventory consumers identified in Stage 2; `test/unit/lib/managed-runtime-recovery.test.ts`; `test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`; `scripts/e2e-cloud.sh`; `test/e2e-browser/helpers/external-target.ts` only if receipt/ownership validation is needed. +- Modify: `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`; `src/App.tsx`, `src/store/panesSlice.ts`, `src/components/TerminalView.tsx`, `src/components/Sidebar.tsx`, `src/store/selectors/sidebarSelectors.ts`, `src/lib/session-utils.ts`; `test/unit/lib/managed-runtime-recovery.test.ts`; `test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`; `scripts/e2e-cloud.sh`; `test/e2e-browser/helpers/external-target.ts` only if receipt/ownership validation is needed. - Create: `test/unit/server/e2e-cloud-external-target.test.ts`, using fake gcloud executable dispatch to inspect the actual generated cloud env file and dispatched process inputs; this is not a source-string/config-text test. - Create: `test/e2e-browser/specs/managed-provider-environment-external.spec.ts`; run-owned fixture/tunnel orchestration under `scripts/testing/managed-provider-environment-target.ts` if the topology validation succeeds. +- Modify: `test/e2e-browser/playwright.config.ts`, `playwright.cloud.config.ts` for the positively selected `chromium-external-target` project and ordinary discovery exclusion of the two external-target specs; this is separate from `CLOUD_SKIP_SPECS`. - Modify: `docs/development/managed-runtime.md`; `AGENTS.md` plan/runbook links; README only for actual changed user setup. **Interfaces:** Existing inventory `runtimeStatus` admits `blocked` and the existing Rust exited process state where final tail is still draining; pane recovery projection uses `error` for Blocked/Lost, `exited` for Stopped, `recovering` only for active recovery, Running only after verified live recovery. Retain recoverySummary reason/attempt/incident and existing retry controls. External browser consumes `FRESHELL_E2E_TARGET_URL`, `FRESHELL_E2E_TARGET_WS_URL`, `FRESHELL_E2E_TARGET_TOKEN`, optional `FRESHELL_E2E_TARGET_TIMEOUT_MS`, and the narrow nonsecret `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE` JSON parameter through allowlisted cloud forwarding. That parameter contains candidate SHA, run ID, remote workspace path and expected nonce only; ownership/cleanup receipt remains on the target host. Do not forward TARGET_HOME: cloud `/app` is a different filesystem. Candidate provenance comes from authenticated `/api/server-info` commit/buildDirty; actual managed readiness comes from `/api/runtime/readiness`, negotiated ready.managedRuntimeV1 and the created terminal/soul/incarnation inventory, not an invented server-info flag. +**External-target selection contract (also Task 7):** Ordinary Chromium/Firefox/WebKit projects exclude exactly `managed-provider-environment-external.spec.ts` and `code-checkpoint-workspace-external.spec.ts` in shared match-all discovery. Do not add them to `CLOUD_SKIP_SPECS`, discover them and call `test.skip`, or count absence from the ordinary broad suite as coverage. Add the dedicated `chromium-external-target` project only when `FRESHELL_E2E_EXTERNAL_FIXTURE_KIND` is the closed value `managed-environment` or `code-checkpoint` and its corresponding validated fixture JSON plus target URL/token are paired. Its `testMatch` selects only that kind's named spec and does not inherit the ordinary external-spec ignore. Explicit selection without a complete matching fixture is an error before dispatch/config use; its driver cannot fall back to an owned ordinary server or an ambient external/live target. Task 7's `FRESHELL_CODE_CHECKPOINT_FIXTURE` JSON contains candidate SHA, run ID, synthetic workspace/home/non-Git fixture paths, restricted UID and expected nonce only. Forward only these two new names and the kind through the cloud env-file allowlist. The wrapper's shard/spec discovery and job invocation must use the same positively selected project/fixture contract, otherwise zero discovered tests fail. Extend fake-gcloud dispatch tests and Playwright discovery tests by executing configuration/list selection: ordinary cloud lists neither external spec, a correctly paired kind lists/runs its exact nonzero spec, and missing/mismatched pairs fail. Record actual pass counts separately from the final ordinary broad receipt. + - [ ] **Step 1: Add failing UI and real managed browser tests.** Extend unit cases from running and recovering through Blocked/Lost/Stopped, asserting exact projected status and retained summary/identity. Extend cloud-compatible contextual UI spec to show actual shell startup output, final blocked/lost/stopped status, existing retry controls, retained output and successful replacement attach; assert no silent extra `terminal.create`. These tests cover UI rendering/state and remain explicitly narrower than complete managed startup. The external managed environment spec verifies actual `/api/server-info` candidate and active managed runtime against fixture metadata, creates a new OpenCode CLI pane through the real picker against the scratch managed target, and asserts terminal output from executable skill/resources, a dependency-loaded plugin and resolved absolute/relative config-file references with per-run nonce. Target uses representative synthetic supported configuration and synthetic managed grants; actual controller stage and session host child execute unchanged production route. Missing selected reference and startup-before-identity cases show actionable retained diagnostics and blocked outcome; no hidden conversation replacement. Browser reconnect checks terminal/session identity and retained output. Assert target stage/child receipt matches the created terminal ID and candidate so an ordinary fixture or stale server cannot pass. Fake-gcloud behavioral tests invoke the wrapper with URL/token pairs, consume the actual --env-vars-file, and verify what the fake job receives; missing URL/token pair errors before dispatch, secret token never appears in stdout, and unrelated host variables are absent. - [ ] **Step 2: Red.** Run `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:cloud test/unit/lib/managed-runtime-recovery.test.ts`; `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`. Expected existing blocked projection remains recovering and running→final cases fail. The full managed external spec must fail on old candidate due to actual resource permissions/reference failure, not absent topology or zero matches. Establish the topology first as Stage 2 evidence. - [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Separate running counts from retained managed membership in App census, panesSlice clearDeadTerminals, TerminalView attach/exit and sidebar/background lookup, including cold-load ordering and compatibility fallback. Read-only same-handle replay attach is distinct from input/create/recovery permission. Server cold lookup and WS reconcile adopt retained Stopped/Failed handles instead of ordinary respawn; attach cannot overwrite authoritative blocked status. Sidebar click uses retained owned target, including provisional pre-native identity. Reuse managed lifetime claims/reaper behavior; no new UI card or recovery protocol. The owned generic Cloud Run HTTP/WSS preflight establishes reachability, while full candidate/browser proof remains required. Orchestrator allocates an owned `ManagedRuntimeBrowserRig`, explicit synthetic provider/workspace/bootstrap roots for both broker and supervisor, run-specific ports/logs, and a task-local Quick Tunnel with recorded PID/cwd. RuntimeHarness currently hardcodes real os.homedir() provider roots and ambient bootstrap grants; add narrow fixture inputs rather than assuming webHomeDir isolates them. Existing callers retain their defaults; this run never selects ambient real roots/grants. Download the official task-local Linux binary and execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. - Extend the cloud env-file writer with only the exact allowlisted parameters above. Require URL/token together; serialize values as JSON strings valid in YAML, write a private temporary file and retain existing cleanup. Keep secret token out of command echo/receipts/logs. Do not forward arbitrary ambient vars. New spec uses existing ExternalServer fixture; fixture setup occurs on target host before the cloud browser, not ignored `setupHome` options. Retain no destructive runtime cases in the unconstrained cloud fixture; actual kill/restart/corruption acceptance remains in the documented disposable sandbox. Quick Tunnel has HTTP/WebSocket support, no SSE, 200 concurrent-request limit and no uptime guarantee; select one spec/run and report preflight/transport failure explicitly without switching backend. + Extend the cloud env-file writer with only the exact allowlisted parameters above and enforce the External-target selection contract. Require URL/token together; serialize values as JSON strings valid in YAML, write a private temporary file and retain existing cleanup. Keep secret token out of command echo/receipts/logs. Do not forward arbitrary ambient vars. New spec uses existing ExternalServer fixture and fails if its matching paired fixture is absent; fixture setup occurs on target host before the cloud browser, not ignored `setupHome` options. Retain no destructive runtime cases in the unconstrained cloud fixture; actual kill/restart/corruption acceptance remains in the documented disposable sandbox. Quick Tunnel has HTTP/WebSocket support, no SSE, 200 concurrent-request limit and no uptime guarantee; select one spec/run and report preflight/transport failure explicitly without switching backend. Record copy bounds, exact unsupported-reference errors, credential reference lifecycle, unknown-vs-absent recovery and retained-output guarantees in agent docs. Do not add documentation-text tests or homepage changes for this repair. -- [ ] **Step 4: Green.** Run `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts --config config/vitest/vitest.config.ts`; repeat cloud unit/contextual UI tests. Start the run-owned driver with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts`; it invokes `pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-provider-environment-external.spec.ts` on configured cloud with its private target handoff. Expected nonzero tests pass and actual managed candidate/stage/child observations prove both preparation stages. No local-only qualification or UI mock substitutes. +- [ ] **Step 4: Green.** Run `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts --config config/vitest/vitest.config.ts`; repeat cloud unit/contextual UI tests. Start the run-owned driver with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts`; after its owned target/handoff passes validation it sets the matching `managed-environment` kind and invokes `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/managed-provider-environment-external.spec.ts` on configured cloud. Expected nonzero tests pass and actual managed candidate/stage/child observations prove both preparation stages. No local-only qualification or UI mock substitutes. - [ ] **Step 5: Refactor.** Reuse external target/owned rig and existing recovery UI helpers. Consolidate allowed cloud env forwarding without adding permanent tunnel deployment, remote Docker protocol or generic provider framework. -- [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. This task step does not run a broad suite; the single end-of-execution broad gate follows every task commit and review below. No interim broad gate unless the actual changed interface/impacted set requires one and that reason is recorded. +- [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. At each passed task/review boundary, apply the periodic broad-gate rule in Acceptance below; the final exact-HEAD broad gate still follows all seven task commits/reviews. An additional earlier broad gate needs a recorded impacted-interface reason. - [ ] **Step 7: Commit.** Stage exact TS/UI/tests/cloud-helper/docs files changed by this task and `git commit -m "test: verify managed provider environments and recovery in the browser"`. ### Task 7: Resolve code-rewind workspace, publish complete snapshots, and expose the result per turn **Criteria:** 12–15. **Dependencies:** finish the planning/assumption gate; coordinate shared fresh-agent/server files with Tasks 5–6. This task does not use code-rewind results as provider-store recovery evidence, does not change launch cwd, and does not restart production. Run-owned browser target/tunnel support from Task 6 may be reused; it must remain synthetic and isolated. -**Files:** server `checkpoints.rs` and shared checkpoint service/state; WS send protocol and narrow injected trait plus managed/local barrier/admission paths; exact provider producer/history linkage files identified by V18; fixed sandbox entrypoint/wrapper fixture mode and owned bridge target driver; `crates/freshell-server/src/checkpoints.rs`; a small sibling checkpoint-scope helper only if it makes the route simpler; checkpoint route wiring in `main.rs` only if existing managed/session resolution is required; `src/lib/fresh-agent-checkpoints.ts`; `src/components/fresh-agent/FreshAgentView.tsx`, `FreshAgentTranscript.tsx`, `FreshAgentTurnActions.tsx`; relevant pane normalization/persistence in `src/store/paneTypes.ts` and `panesSlice.ts`; current checkpoint/component tests; one focused Rust black-box integration test; one new cloud-legal browser spec and its bounded synthetic fixture. Use existing fixture/support code where it actually supplies the needed behavior. No general snapshot framework, environment copier reuse, broad directory exclusions, chmod repair, or real-account fixture. +**Files:** server `checkpoints.rs` and shared checkpoint service/state; WS send protocol and narrow injected trait plus managed/local barrier/admission paths; the exact producer/projection files named in Named consumer and turn-identity seams above; fixed sandbox entrypoint/wrapper fixture mode and owned bridge target driver; `crates/freshell-server/src/checkpoints.rs`; a small sibling checkpoint-scope helper only if it makes the route simpler; checkpoint route wiring in `main.rs` only if existing managed/session resolution is required; `src/lib/fresh-agent-checkpoints.ts`; `src/components/fresh-agent/FreshAgentView.tsx`, `FreshAgentTranscript.tsx`, `FreshAgentTurnActions.tsx`; relevant pane normalization/persistence in `src/store/paneTypes.ts` and `panesSlice.ts`; current checkpoint/component tests; one focused Rust black-box integration test; one new cloud-legal browser spec and its bounded synthetic fixture. Use existing fixture/support code where it actually supplies the needed behavior. No general snapshot framework, environment copier reuse, broad directory exclusions, chmod repair, or real-account fixture. + +**Named identity regression tests:** Add `test/unit/claude-sidecar/user-message-replay.test.ts` to execute the real sidecar queue/SDK-input handler with an injected synthetic SDK transport and verify `claude_checkpoint_uuid_replay_binds_persisted_user`: the supplied valid UUID/request correlation survives actual user replay, mismatched/absent replay stays unknown, and unrelated assistant/result events cannot bind it. Add `test/integration/server/claude-checkpoint-uuid-replay.test.ts` to run the pinned SDK/bundled CLI with an isolated synthetic local provider endpoint, no model-network or real grants, and assert the same UUID in both replay and persisted user transcript. Reuse the synthetic probe arrangement recorded at `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`; a mock-input-only test is insufficient for this integration gate. Add `crates/freshell-opencode/tests/checkpoint_message_id.rs` with `opencode_checkpoint_message_id_persists_after_prompt`: execute the real prompt builder against pinned OpenCode 1.18.21 in an isolated synthetic store and local model endpoint, then read native persisted history and assert `messageID == info.id`. Separately add `checkpoint_turn_identity::opencode_checkpoint_message_id_survives_compact_queue` in `crates/freshell-freshagent/src/opencode_ws.rs`, where the compact queue actually lives: execute queue/drain/prompt and bind the observed persisted user row, rather than a queue acknowledgement or rewritten fixture ID. Freshagent tests named `checkpoint_turn_identity` also cover the production Codex native/display user join, modern duplicate text, aliases, unknown results and queue handling. Server tests named `managed_checkpoint_send_barrier` cover feature-gated post-capture admission, stale fences, one dispatch after failure and settlement/duplicate joining. Each named selector must execute a positive test count after Task 7's changes. - [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Verify these tests fail for the intended current behaviors and record red output. @@ -296,44 +343,45 @@ Managed output observations distinguish Running, active Recovering, Blocked with Make the server the final filesystem resolver: canonicalize an existing directory, distinguish inaccessible/not-directory from absent, and return the resolved workspace on successful create. Account-home detection must cover the relevant account, not only `CheckpointsApiState.home` (the reproduced failure involves a different UID's home). Use OS account-home information and known managed home paths on Linux; ensure the cross-platform fallback does not disable valid Windows project scopes. Automatically inferred scopes equal to a recognized account home are unavailable even when a dotfiles Git repository happens to exist there. A genuine explicit non-Git scope remains supported. Never search arbitrary subdirectories of a home for a project or silently broaden a selected subtree to another checkout. If root evidence conflicts or is missing, return a typed unavailable reason and keep sending usable. - Capture canonical workspace and immutable scope/storage locator with the attempt/checkpoint, using it for list/metadata/restore without rerunning new-root discovery. Legacy shadow repos keep dual addressing by original stored/legacy scope string/storage key and validated original work-tree destination; do not rehash a historical nested/symlink cwd as today's checkout root or scan homes to find it. A subsequent pane cwd or session-directory update must not redirect an older turn's restore. Preserve existing valid shadow repositories by resolving existing entries with their original stored/legacy scope; no bulk migration or deletion is needed. + Capture canonical workspace and immutable scope/storage locator with the attempt/checkpoint, using it for list/metadata/restore without rerunning new-root discovery. Legacy shadow repos keep dual addressing by original stored/legacy scope string/storage key and validated original work-tree destination; do not rehash a historical nested/symlink cwd as today's checkout root or scan homes to find it. A subsequent pane cwd or session-directory update must not redirect an older turn's restore. Preserve existing valid shadow repositories by resolving existing entries with their original stored/legacy scope; no bulk migration or deletion is needed. For a pre-change non-Git pane lacking picker provenance, accept only an actual prior valid published checkpoint record already linked to this pane/session by its persisted checkpoint ID/locator or existing checkpoint metadata. Revalidate its original scope/storage locator and work-tree destination to the same canonical non-home directory, preserving the existing association without demanding newly added provenance fields from that historical record. This record is legacy scope evidence for new turns, not permission to infer scope from `initialCwd`, labels or an account home. A missing/malformed/ambiguous record remains unavailable. Add a behavioral old-pane fixture with a real prior complete non-Git checkpoint and verify a new turn captures/restores that same scope; a home-only launch fixture without such a record must still refuse automatic capture. - [ ] **Step 3: Require complete readable eligible code and publish only successful targets.** Keep the existing Git-ignore semantics as the defined code scope; `.git` administrative state stays outside it. Before publishing, explicitly traverse eligible directories with error-aware filesystem operations and verify readable eligible regular files. Determine ignores with Git's existing rules, not a new hard-coded list. Do not use `Path::exists`, an ignored `read_dir` error, a process UID of zero, or success from `git ls-files` alone as proof of readability. Symlinks keep Git's link semantics and must not cause traversal into unrelated trees. Reject unsupported eligible nested-repository/gitlink capture with an actionable path/cause unless the bounded validation proves an equally simple way to capture its complete code tree; never advertise only the gitlink as a complete snapshot. The existing policy excludes ignored resources intentionally; it does not authorize omitting an unreadable eligible path. - Serialize create/metadata/restore operations for one canonical snapshot workspace with a shared route-state keyed mutex; separate workspaces remain independent. Stage with a request-local `GIT_INDEX_FILE`, initialized from the previous published tree (or empty for the first checkpoint), so failed/partial `git add` cannot contaminate a later snapshot. Build a tree and commit object without moving the published branch; persist complete checkpoint/request metadata atomically; then move the published ref as the final success step. `commit-tree` plus an explicit publish-last `update-ref` is preferable to moving HEAD and trying to undo after failure. Preserve original Git index/HEAD and existing file permissions. Clean only owned temporary index/metadata files on failure. An orphan unpublished Git object/metadata entry must never appear in list or be accepted by restore. + Serialize create/metadata/restore operations for one canonical snapshot workspace with a shared route-state keyed mutex; separate workspaces remain independent. Stage with a request-local `GIT_INDEX_FILE`: copy the last successfully published stat-bearing index into it when its recorded published-tree ID matches the current published ref; otherwise reconstruct from that tree (empty for first capture). Never initialize every ordinary repeat by `read-tree`, which discards the stat cache and rehashes all files. Git refresh/add checks changed/racy entries; the explicit full readability pass still opens every eligible regular file for reading and enumerates every eligible directory on each capture, so cache hits never prove access. Install the new disposable index cache only after successful complete publication; cache-install failure invalidates/logs the cache without changing availability of the complete published target. Corrupt/stale/missing cache falls back to tree initialization, failed partial indexes are deleted. This bounds ordinary repeated work to complete traversal/access validation plus Git's changed-file hashing, rather than hashing all unchanged bytes. A behavioral instrumented fixture proves an unchanged second capture avoids whole-tree rehashing, edits are captured/restored, and permission loss after a cached success still fails. Build a tree and commit object without moving the published branch; persist complete checkpoint/request metadata atomically; then move the published ref as the final success step. `commit-tree` plus an explicit publish-last `update-ref` is preferable to moving HEAD and trying to undo after failure. Preserve original Git index/HEAD and existing file permissions. Clean only owned temporary index/metadata files on failure. An orphan unpublished Git object/metadata entry must never appear in list or be accepted by restore. List and restore operate only on published complete checkpoints. Distinguish an actually empty repository from Git-log/storage errors instead of swallowing all errors. A metadata-write or publish failure fails the request, leaves the prior published head usable and supplies no new valid target. Revalidate readability when capture operations report changes/errors; report incompleteness rather than converting it into a valid rewind target. Do not claim transaction-like consistency against arbitrary independent writers; the required ordering guarantee is that this pane's provider turn has not begun before its successful snapshot finishes. - [ ] **Step 4: Own capture and provider-send release in the server.** Add optional modern checkpoint intent and stable logical attempt ID to the existing freshAgent.send frame, with frozen provider/session/scope and observed fence. Local echo/pending state starts immediately; the client sends one command, without an independent automatic checkpoint POST or browser timer releasing a second send. Inject a narrow checkpoint-service trait into the WS crate from server main/checkpoint state; intercept before both managed gateway and local provider-send dispatch and run one owned asynchronous barrier-and-forward job so the receive loop stays responsive. Direct REST capture/list/metadata/restore use the same service/storage/locks. - Capture budget begins before workspace-lock acquisition. The service owns all Git children/publication work; timeout requests cancellation and waits for actual owned termination/settlement before releasing a send. Dropping a future or aborting browser HTTP is insufficient. Reconcile timeout racing update-ref under the workspace lock: either complete publication already succeeded while send was held, so retain Available, or it did not, so return Unavailable preserving prior targets. No remaining work may publish an attempt after terminal settlement/provider-send release. Do not promise absolute-clock/ref atomicity. Add a scoped owned Git runner with retained children/stdout/stderr tasks, explicit POSIX process-group termination/reaping/group settlement and supported Windows JobObject ownership before child execution, followed by direct wait and actual-job/process-count settlement. A dropped future, taskkill example or kill_on_drop alone is insufficient. Reconcile ref/complete metadata before terminalizing ambiguous publication. Destructive tests stay in the sandbox; preserve supported Windows capture rather than disabling its timeout behavior. + The finite production capture budget is 15 seconds, beginning before workspace-lock acquisition and covering scope/readability traversal, Git, metadata and publication; tests inject a shorter budget through the checkpoint service, with no browser timer or new environment knob. Budget exhaustion produces `capture_timeout` and a clear per-turn reason. Cancellation settlement may outlast that deadline only while owned children are being terminated/reaped; provider dispatch waits for actual settlement. Record stage timings and eligible-file/changed-file counts without names on success; an ordinary unchanged-repeat fixture and a bounded many-file fixture must complete within this default on the supported synthetic lane, or tune the single value from measured evidence before completion while retaining a documented finite value. The service owns all Git children/publication work; timeout requests cancellation and waits for actual owned termination/settlement before releasing a send. Dropping a future or aborting browser HTTP is insufficient. Reconcile timeout racing update-ref under the workspace lock: either complete publication already succeeded while send was held, so retain Available, or it did not, so return Unavailable preserving prior targets. No remaining work may publish an attempt after terminal settlement/provider-send release. Do not promise absolute-clock/ref atomicity. Add a scoped owned Git runner with retained children/stdout/stderr tasks, explicit POSIX process-group termination/reaping/group settlement and supported Windows JobObject ownership before child execution, followed by direct wait and actual-job/process-count settlement. A dropped future, taskkill example or kill_on_drop alone is insufficient. Reconcile ref/complete metadata before terminalizing ambiguous publication. Destructive tests stay in the sandbox; preserve supported Windows capture rather than disabling its timeout behavior. Preserve the immutable send and validate captured conversation/session ownership and observed epoch/generation again after capture for every managed/local provider. Existing managed proxy drops observed fences and Claude does not enforce them, so implement the narrow post-delay admission rather than assuming it exists. Never reread latest pane/session/fence to retarget the request. A rejected stale/closed conversation is a send rejection distinct from snapshot failure. Within the process duplicate transport requests join the same barrier-and-forward job; known-rejected retries in the same canonical conversation may alias their new transport ID to the same logical capture result, never recapture that attempt. Ambiguous acceptance follows existing no-redispatch rules. A different conversation gets a separate logical attempt. Queue draining uses the same service immediately before forwarding only when capture has not settled. Persist one minimal positive/negative outcome independently from ephemeral metadata-patch bookkeeping: schema/version, logical attempt and transport aliases, canonical conversation, captured workspace, Available checkpoint ID or Unavailable code/reason, and native/display user-row join when established. No new prompt-text storage. Successful target authority remains complete server publication; pane persistence retains pending/current UI state and failed attempts. Metadata/outcome storage failure yields Unknown/Unavailable, never label-derived success. Restore uses captured scope and actual published target. - Bind outcomes using real provider IDs, not optimistic echo: Codex native accepted turn ID already exists, but mixed history projects the user as turn-N:row-0 rather than turn-N; retain explicit native turn identity in the strict shared row schema and bind actual user display rows without suffix guessing or rewriting historical IDs. Claude's pinnedSDK0.3.237/bundle2.1.237 synthetic probe confirmed supplied UUID in real user replay and persisted transcript: pass a valid per-message UUID through sidecar SDKUserMessage, enable replay-user-messages, relay the actual user replay/native identity with request correlation. OpenCode1.18.21 exact source supports prompt.messageID persisted as info.id: carry a valid native message ID through prompt construction/compact queue and bind observed native history. Accepted queue acknowledgement alone is not persisted-user proof. Preserve session materialization aliases. A generated ID is usable only if actual native history carries that same identity; otherwise linkage remains unavailable and is not claimed complete. Modern attempted rows never fall back by text/ordinal; retain label fallback only for demonstrably legacy rows. Durable modern/version evidence or a safely established feature boundary survives failed result writes/store loss so absence cannot become legacy success. If turn binding/result persistence is unavailable, that row remains explicitly unknown/unavailable; another provider's success is not universal coverage. Desktop/touch/context menus share the same availability: pending/failed/unavailable disables rewind with a readable reason, success enables its exact turn. Snapshot failure still forwards one usable message after settlement and retains its reason after notices/reload; client timeout/disconnect cannot release another send. + Bind outcomes using real provider IDs, not optimistic echo: Codex native accepted turn ID already exists, but mixed history projects the user as turn-N:row-0 rather than turn-N; retain explicit native turn identity in the strict shared row schema and bind actual user display rows without suffix guessing or rewriting historical IDs. Claude's pinned SDK 0.3.237/bundle 2.1.237 synthetic probe confirmed supplied UUID in real user replay and persisted transcript: pass a valid per-message UUID through sidecar SDKUserMessage, enable replay-user-messages, relay the actual user replay/native identity with request correlation. OpenCode 1.18.21 exact source supports prompt.messageID persisted as info.id: carry a valid native message ID through prompt construction/compact queue and bind observed native history. Accepted queue acknowledgement alone is not persisted-user proof. Preserve session materialization aliases. A generated ID is usable only if actual native history carries that same identity; otherwise linkage remains unavailable and is not claimed complete. Modern attempted rows never fall back by text/ordinal; retain label fallback only for demonstrably legacy rows. Durable modern/version evidence or a safely established feature boundary survives failed result writes/store loss so absence cannot become legacy success. If turn binding/result persistence is unavailable, that row remains explicitly unknown/unavailable; another provider's success is not universal coverage. Desktop/touch/context menus share the same availability: pending/failed/unavailable disables rewind with a readable reason, success enables its exact turn. Snapshot failure still forwards one usable message after settlement and retains its reason after notices/reload; client timeout/disconnect cannot release another send. - [ ] **Step 5: Structured failures and safe client details.** Replace create-path error strings with a small typed checkpoint failure carrying stage/operation, stable code, resolved workspace, optional offending path, underlying filesystem kind/errno or Git exit status and bounded cause. Emit one warning for expected scope/readability unavailability and one error for unexpected Git/storage failure, with an event such as `code_checkpoint.failed`; include the send correlation as `send_request_id`, and preserve the HTTP span's `request_id`. When known, include filesystem path directly from traversal; Git failures with no reliably extracted path report an explicit operation and bounded stderr cause. Do not parse stderr as authority about complete storage. Do not log labels, commit argv, request body, outgoing text, raw configuration, or file contents. Exercise failure logs through the real HTTP request/logger, assert correlation and meaningful cause, and assert synthetic outgoing-message/credential markers are absent. Keep the client's human reason separate from internal failure context; response JSON includes code/details needed by per-turn availability while retaining normal API error compatibility. -- [ ] **Step 6: Complete route smoke with actual restricted identity.** Add one fixed checkpoint-fixture initialization mode in the documented sandbox wrapper/entrypoint trusted root phase: create only run-owned synthetic tmpfs fixture roots with server UID B-owned writable project/storage/home parents and root/UID A-owned0600 file/0700 directory, then permanently drop to B with no DAC capability. Existing --runtime-suite is networkless/single-UID and cannot arrange this unchanged. B can rename those prepared private entries through its own writable parent to vary eligibility without chmod/read or privileged control service. Launch the real compiled checkpoint server under B; loopback HTTP smoke needs no external network. Create only synthetic accessible project/code paths plus one A-owned 0600 file and one non-traversable A-owned directory. Verify B cannot read them by an actual open/read operation before issuing the route; test fails if B is root or has DAC override capability. Point server storage/home at a B-writable synthetic directory. Do not chmod the private source to make capture pass. +- [ ] **Step 6: Complete route smoke with actual restricted identity.** Add the closed flag `--checkpoint-fixture` to `scripts/sandbox-test.sh`, permitted only with `--runtime-suite` and never `--corpus`. It sets only `FRESHELL_SANDBOX_CHECKPOINT_FIXTURE=1` for the container. In `docker/sandbox/entrypoint.sh`, before its final `exec gosu sandbox "$@"`, handle that exact flag by fixed initialization of `/tmp/freshell-checkpoint-fixture` on the existing synthetic tmpfs (no arbitrary path or command argument): create only run-owned synthetic tmpfs fixture roots with server UID B-owned writable project/storage/home parents and root/UID A-owned 0600 file/0700 directory, then permanently drop to B (the existing sandbox UID) with no DAC capability. Set the nonsecret fixture-root locator for the unprivileged tests; root initialization accepts no test-supplied ownership script or command. The separate browser target calls this same fixed initializer before its restricted server launch; fixture manipulation/observations run only as B. There is no privileged mutation endpoint. Existing --runtime-suite is networkless/single-UID and cannot arrange this unchanged. B can rename those prepared private entries through its own writable parent to vary eligibility without chmod/read or privileged control service. Launch the real compiled checkpoint server under B; loopback HTTP smoke needs no external network. Create only synthetic accessible project/code paths plus one A-owned 0600 file and one non-traversable A-owned directory. Verify B cannot read them by an actual open/read operation before issuing the route; test fails if B is root or has DAC override capability. Point server storage/home at a B-writable synthetic directory. Do not chmod the private source to make capture pass. Exercise HTTP create/list/metadata/restore: first complete non-Git/Git snapshots restore all eligible fixture files, executable bits and symlink semantics; after adding the inaccessible eligible file/directory a create fails with workspace/path/cause and list retains only prior successes; a guessed/unpublished failed commit cannot restore; repeated reads do not erase or duplicate result state; fixing the synthetic fixture deliberately and issuing a new request succeeds. Include an injected real storage-write failure after Git object creation to prove publication-last, not merely staging failure. Verify original repository state and source permissions remain unchanged. Include explicit project while account-home launch is supplied and no-scope home refusal without reading home secrets. Capture JSONL events and assert send/HTTP correlations, stage, cause and no message/secret marker. Kill/restart portions, if any, execute only inside `scripts/sandbox-test.sh` and affect recorded owned PIDs. -- [ ] **Step 7: Actually running configured-cloud browser proof.** Add a focused spec outside `CLOUD_SKIP_SPECS` rather than counting the existing skipped rewind test. Reuse Task6's owned external-target/tunnel handoff with a separate checkpoint target container using the same fixed trusted initialization and restricted UID B. It uses an ordinary dedicated bridge and only an ephemeral host-loopback published port, read-only artifacts/root, synthetic writable tmpfs, no host network/user HOME/corpus/Docker socket. The networkless destructive runtime-suite container is not browser reachable. The target driver owns exact container/tunnel names, observations and cleanup; private fixture files remain inside that target, not Cloud Run. Seed fixture paths and collect disk/log observations through the existing test-owned target harness/control channel, not a production testing endpoint. The permission-failure server must have the restricted identity from Step 6. No provider network/model dependency is required: select the direct Codex app-server fixture because actual Codex acceptance supplies a native turn ID. Reuse the shared direct fake-app-server.mjs behavior.recordTurns mode, whose turn/start and thread/read already persist matching distinct native turns/user rows; extend only missing fixture behavior and exercise the production Codex adapter/projector's native-to-display identity. Do not inject display IDs into UI or reuse assistant-only/static-ID history. Validate the chosen transport through actual fresh-agent send/accepted/snapshot behavior before relying on it. +- [ ] **Step 7: Actually running configured-cloud browser proof.** Add a focused spec outside `CLOUD_SKIP_SPECS` rather than counting the existing skipped rewind test. Reuse Task 6's owned external-target/tunnel handoff with a separate checkpoint target container using the same fixed trusted initialization and restricted UID B. It uses an ordinary dedicated bridge and only an ephemeral host-loopback published port, read-only artifacts/root, synthetic writable tmpfs, no host network/user HOME/corpus/Docker socket. The networkless destructive runtime-suite container is not browser reachable. The target driver owns exact container/tunnel names, observations and cleanup; private fixture files remain inside that target, not Cloud Run. Seed fixture paths and collect disk/log observations through the existing test-owned target harness/control channel, not a production testing endpoint. The permission-failure server must have the restricted identity from Step 6. No provider network/model dependency is required: select the direct Codex app-server fixture because actual Codex acceptance supplies a native turn ID. Reuse the shared direct fake-app-server.mjs behavior.recordTurns mode, whose turn/start and thread/read already persist matching distinct native turns/user rows; extend only missing fixture behavior and exercise the production Codex adapter/projector's native-to-display identity. Do not inject display IDs into UI or reuse assistant-only/static-ID history. Validate the chosen transport through actual fresh-agent send/accepted/snapshot behavior before relying on it. - Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately cover home launch with explicit project scope and home launch without scope; the former snapshots only project code, the latter sends successfully with a clear unavailable reason and no home checkpoint. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. + Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately create a pane using the actual DirectoryPicker confirmation of a synthetic non-Git directory (no `.git`), send an accepted user turn, verify its resolved scope/exact row has enabled rewind, mutate at least two eligible files, restore that row through UI confirmation and verify both original contents and the unchanged non-Git scope. This tests picker provenance through client send, server resolution, publication and restore; API-only seeding is insufficient. Also cover home launch with explicit project scope and home launch without scope; the former snapshots only project code, the latter sends successfully with a clear unavailable reason and no home checkpoint. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. -- [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the required broad gate once Tasks 1–6 join. Validate no foreign files or permission changes entered the diff. Commit Task 7 atomically with descriptive test evidence. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. - Focused commands: `mise exec rust@1.96 -- cargo test -p freshell-server --locked checkpoints`; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx --config config/vitest/vitest.config.ts`. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-server --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs via the Task6 owned external-target driver with configured cloud selection; expected positive executed/pass count, real accepted turns and successful/failed/unavailable results, no skipped/zero-test coverage. +- [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the periodic task-boundary gate and final exact-HEAD broad gate after all Tasks 1–7 join. Validate no foreign files or permission changes entered the diff. Commit Task 7 atomically with descriptive test evidence. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. + Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id opencode_checkpoint_message_id_persists_after_prompt`; `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/server/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx --config config/vitest/vitest.config.ts`. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, both home cases and actual picker-confirmed non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. ## Acceptance and completion boundary 1. Map freshell#bdpt's exact latest 15 criteria to the executed task/test evidence in the parent run record; the user's requested provider and recovery additions remain mandatory. Do not close the kata or claim full completion with unresolved schema/reference or cloud managed browser coverage. 2. All code tasks require recorded behavioral red, green, refactor and affected-test evidence. Synthetic files/commands are intentional fixtures; production preparation, grants, ownership, terminal transport and browser lifecycle are real. No static prose/config hash tests qualify. -3. After all six tasks' commits and specification-plus-quality reviews, finish the worktree clean and record the final committed HEAD. Run the separate required no-write typecheck `pnpm run typecheck:client`, then the single final coordinated broad gate `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed provider environment and recovery final committed HEAD acceptance" pnpm run test`. These commands are on the final committed candidate, not a precommit Task 6 tree; preserve separately needed focused managed-feature, integration, browser and sandbox evidence from the tasks. Record command/results and verified HEAD, then push the feature branch with required prepush checks. A subsequent source/review fix requires a new focused commit and appropriate renewed verification of that changed final HEAD; do not rerun an unchanged broad gate merely because a task boundary passed. Stop before PR creation until explicit approval. Do not deploy/restart the live server while finishing the branch. -4. Once separately approved, follow the managed rollout runbook, preserve registry/runtime/image/secrets roots, and recheck original OpenCode failure via Freshell MCP. Assert usable startup and existing configuration without reading/logging secret contents. This operational residual is accepted pending approval, whereas missing implementation or configured-cloud test evidence is not. +3. Periodic broad gate: record the execution-start commit and most recent passing broad-gate commit. At every passed task/specification-plus-quality-review boundary, count non-plan commits since execution start or the last broad gate (whichever is later). If the count is at least 10, run `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed environment passed-task periodic acceptance" pnpm run test` on that committed boundary before starting another task; record HEAD/result and reset the count only after a pass. Plan/report-only commits do not increment it. This rule preserves the-usual periodic gate and does not replace final acceptance. +4. After all seven tasks' commits and specification-plus-quality reviews, finish the worktree clean and record the final committed HEAD. On that exact HEAD, after Task 7, run the feature-enabled server build and `checkpoints`, `managed_checkpoint_send_barrier`, `managed_runtime` and `managed_provider` tests, plus `managed_runtime_api::stop_contract_tests` with a nonzero count. Run the named SDK UUID replay/persisted-transcript and OpenCode prompt/compact-queue `messageID` persistence commands from Task 7; record their real executed/passed counts and candidate HEAD. Default Cargo/CI does not enable `managed-runtime-v1`, so default-suite success is insufficient for these managed paths. Preserve actual paired configured-cloud environment/checkpoint browser receipts after their final affecting source changes. Then run the separate required no-write typecheck `pnpm run typecheck:client`, followed by the final coordinated broad gate `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed provider environment and recovery final committed HEAD acceptance" pnpm run test`. These commands are on the final committed candidate, not a precommit Task 6 or Task 7 tree; preserve separately needed focused managed-feature, integration, browser and sandbox evidence from the tasks. Record command/results and verified HEAD, then push the feature branch with required prepush checks. A subsequent source/review fix requires a new focused commit and appropriate renewed verification of that changed final HEAD; do not rerun an unchanged broad gate merely because a task boundary passed. Stop before PR creation until explicit approval. Do not deploy/restart the live server while finishing the branch. +5. Once separately approved, follow the managed rollout runbook, preserve registry/runtime/image/secrets roots, and recheck original OpenCode failure via Freshell MCP. Assert usable startup and existing configuration without reading/logging secret contents. This operational residual is accepted pending approval, whereas missing implementation or configured-cloud test evidence is not. ## Dispatcher self-review -The exact sanitized User Request occurs once and includes all15current kata criteria. Seven tasks retain behavioral red/green/refactor, focused commits/reviews, actual managed environment smoke, genuinely configured-cloud browser proof, conservative recovery/store decisions, durable retained diagnostics and code-rewind scope/completeness/turn outcomes. Source/ownership/protocol corrections are reflected in task inputs and acceptance; empirical claims are not certified from planned code. Environment copy, secrets and callback metadata have explicit separate profiles, with one policy across both stages and no lower hidden ownership limit. Code-rewind uses server-owned send/capture settlement, real provider identity and fixed restricted fixtures rather than browser timing or fake display IDs. Original workspace/native data and conversation semantics remain protected. Final whole-suite runs on committed final HEAD; PR/live rollout/MCP boundaries remain separately approved. Ordinary provider matrix requirements cannot be replaced by arbitrary-program certification or an unsupported-error workaround for a required reference. Reports remain outside tracked source; plan/code snippets are drafts, not implementation evidence. +The exact sanitized User Request occurs once and includes all 15 current kata criteria. Seven tasks retain behavioral red/green/refactor, focused commits/reviews, actual managed environment smoke, genuinely configured-cloud browser proof, conservative recovery/store decisions, durable retained diagnostics and code-rewind scope/completeness/turn outcomes. Source/ownership/protocol corrections are reflected in task inputs and acceptance; empirical claims are not certified from planned code. Environment copy, secrets and callback metadata have explicit separate profiles, with one policy across both stages and no lower hidden ownership limit. Code-rewind uses server-owned send/capture settlement, real provider identity and fixed restricted fixtures rather than browser timing or fake display IDs. Original workspace/native data and conversation semantics remain protected. The inline provider/projection/ownership/consumer/identity decisions are self-contained; supporting reports have absolute locators. Task 7 and final acceptance explicitly compile/run managed-runtime-v1 and named SDK/OpenCode identity regressions. Configured-cloud browser coverage includes picker-confirmed non-Git capture/restore with paired external-target discovery, and ordinary broad discovery does not count those separate receipts. Restricted initialization is a closed pre-gosu flag with no privileged mutation endpoint. Periodic broad gates run at passed task boundaries after at least 10 non-plan commits; the final whole-suite runs on committed final HEAD; PR/live rollout/MCP boundaries remain separately approved. Ordinary provider matrix requirements cannot be replaced by arbitrary-program certification or an unsupported-error workaround for a required reference. Reports remain outside tracked source; plan/code snippets are drafts, not implementation evidence. From e1a6d18dbda41bbd066715b65b1be0ca0a4c436a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 21:08:48 -0700 Subject: [PATCH 005/493] docs: resolve second managed environment plan review findings --- ...2026-10-05-managed-provider-environment.md | 50 +++++++++++++------ 1 file changed, 34 insertions(+), 16 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 9f0281d02..35fb458c9 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -102,7 +102,7 @@ Replace the blanket Amplifier tree transfer with recorded exact host-created pat ### Named consumer and turn-identity seams -Task 6's concrete UI consumers are `src/App.tsx` (inventory census and running counts), `src/store/panesSlice.ts` (`clearDeadTerminals`, cold normalization), `src/components/TerminalView.tsx` (create/attach/exit/compatibility ordering), `src/components/Sidebar.tsx`, `src/store/selectors/sidebarSelectors.ts`, and `src/lib/session-utils.ts` (`liveTerminalRowIdentity`, background/session lookup). Correct only affected running-versus-retained decisions; process counts use verified running state while retained handles remain reachable. Directory schema/projection and MCP/CLI pass-through coverage remain Task 5 obligations. +Task 6's concrete UI consumers are `src/App.tsx` (inventory census and running counts), `src/store/panesSlice.ts` (`clearDeadTerminals`, cold normalization), `src/components/TerminalView.tsx` (create/attach/exit/compatibility ordering), `src/components/Sidebar.tsx`, `src/components/OverviewView.tsx` (terminal row partitions and rename/kill controls), `src/components/BackgroundSessions.tsx` (detached row filtering and attach/kill controls), `src/store/selectors/sidebarSelectors.ts`, and `src/lib/session-utils.ts` (`liveTerminalRowIdentity`, background/session lookup). API/directory consumers include `src/lib/api.ts`, `src/store/terminalDirectoryThunks.ts`, `src/store/terminalDirectorySlice.ts`, and their actual row types in `src/store/types.ts`. Correct only affected running-versus-retained decisions; process counts use verified running state while retained handles remain reachable. Directory schema/projection and MCP/CLI pass-through coverage remain Task 5 obligations. Blocked rows must remain visible in Overview and detached Background Sessions with their reason and same retained replay handle; existing supported rename, attach/replay and fenced kill controls retain their behavior, and blocked rows do not count as running. Task 7's exact producer/projection inputs are `crates/freshell-protocol/src/client_messages.rs`, `server_messages.rs`, `shared/ws-protocol.ts`, `shared/fresh-agent-contract.ts`; `crates/freshell-server/src/fresh_agent_proxy.rs` and `fresh_agent_proxy_tests.rs`; `crates/freshell-ws/src/hosted_fresh_agent.rs` and local dispatch in `terminal.rs`; session-host `providers/fresh_agent.rs`; runtime protocol `fresh_agent_ops.rs`; freshagent `codex.rs`, `claude.rs`, `claude_snapshot.rs`, `opencode_ws.rs`; OpenCode `serve.rs`, `lib.rs`; Claude sidecar `index.mjs`; agent-runtime `snapshot_projection.rs` only where hosted projection must retain identity. Codex accepted native turn ID joins a role-qualified actual user row carrying additive `nativeTurnId`; display `turn-N:row-0` remains unchanged, ambiguous multiple user rows stay unavailable. Claude sends a valid UUID with logical/request correlation through SDKUserMessage, enables replay-user-messages and binds only actual replay/persisted UUID. OpenCode carries its valid `messageID` through prompt body and compact queue, then binds observed persisted `info.id`; queue acknowledgement is pending only. All retain canonical materialization aliases, never text/ordinal joins for modern attempts. Default fresh Claude uses SDK-bundled 2.1.237, separate from terminal 2.1.263; preserve an explicit approved `CLAUDE_CMD` override. Evidence: `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/load-bearing-validator-v18-checkpoint-turns-fixture.md`, `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-identity-observation.md`, and `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`. These name implementation decisions; runtime identity/consumer tests remain mandatory. @@ -128,8 +128,8 @@ Task 7's exact producer/projection inputs are `crates/freshell-protocol/src/clie **Files:** - Create: `crates/freshell-platform/src/provider_environment.rs` -- Modify: `crates/freshell-platform/src/lib.rs`; `crates/freshell-server/src/managed_provider_bootstrap.rs:65`; `crates/freshell-server/src/managed_mcp_capability.rs:445`; `crates/freshell-session-host/src/main.rs:1130,1506,1810` -- Test: common-module tests, existing `managed_provider_bootstrap::tests`, `managed_mcp_capability::tests`, and host `provider_config` tests. +- Modify: `crates/freshell-platform/src/lib.rs`; `crates/freshell-server/src/managed_provider_bootstrap.rs:65`; `crates/freshell-server/src/managed_mcp_capability.rs:445`; `crates/freshell-session-host/src/main.rs:1130,1506,1810`; `crates/freshell-session-host/src/providers/mod.rs` (`prepare_provider_state_before_bootstrap`); `crates/freshell-sessions/src/amplifier_stub.rs` (`ensure_session`, `EnsuredSession` and its creating helpers). +- Test: common-module tests, existing `managed_provider_bootstrap::tests`, `managed_mcp_capability::tests`, host `provider_config` tests, and new behavioral modules `amplifier_stub::tests::created_path_ledger` and host `tests::created_path_ownership` in those actual source files. **Interfaces:** @@ -191,12 +191,14 @@ fn copied_skill_executes_sibling_resources_after_two_stages() { ``` - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`. Expected missing implementation, then behavioral failure when the test is initially wired through current copying; record the old two-stage script execution PermissionDenied and missing dependency result, rather than treating initial compile setup as the required red evidence. Run matching controller/host tests against old code for the intended omissions before replacing it. + Add `created_path_ledger` tests calling the real `ensure_session` creation/adoption operations and asserting their returned exact paths against actual filesystem creation: new parents/leaves are recorded, pre-existing parents and adopted stubs/siblings are absent, partial-create rollback records no surviving authority. Add `created_path_ownership` tests invoking the production initial-preparation/ledger-transfer helper with >10,000 selected entries (including actual bootstrap-created leaves), an adopted sibling and a selected symlink to an unrelated target. Assert all selected entries remain present/runnable, only creator-recorded physical inodes are handed off, and sibling/target ownership and contents are unchanged. Observe actual creation and physical `lchown` operations/metadata, never source text; the same-UID unit fixture can capture invoked ownership operations, while Task 3's real provider-UID child smoke verifies the resulting transfer. Record behavioral failures against the old 10,000-cap/traversal path before replacing it. Run `mise exec rust@1.96 -- cargo test -p freshell-sessions --locked created_path_ledger` and `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked created_path_ownership`; require at least three executed tests in each named module, with no skipped cases or zero-match success. - [ ] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Select complete skill/plugin/package resource units before mapping. External plugin entry files require their containing resource/package root, adjacent imports/assets and applicable package metadata; canonical target leaves alone are insufficient. Include existing installed node_modules lookup context and connecting ancestor skeleton without copying unrelated ancestor trees. Prefer one bounded complete installation node_modules unit for pnpm store/hoisted/.bin topology, preserving ancestor lookup order and package scope. Preplan coherent physical mappings for overlapping parent/child units, reuse one physical canonical target and preserve additional logical resource destinations as links; overlap itself is not an error. Reject incompatible destination/type claims before writing. Preserve symbolic links with rewritten relative targets and enqueue the required complete resource/package and installed lookup context under private `.freshell/resources` when outside selected roots, preserving relative ancestor/sibling hierarchy rather than flattening one canonical leaf per private ID. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure/context directory and required aliases as normal `.freshell/resources` selections in ProviderLaunchContext; transport the complete graph once and preserve its relative topology identically at the second stage, without independent package rediscovery or renesting. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the exact created leaf/parent ledger in Selected C05 projection and exact initial ownership decisions; carry it from the creating operations, not a fresh recursive HOME walk. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. + Extend `EnsuredSession` with the exact paths its creating operations actually created, not a directory listing after creation. Preserve all existing identity/adoption fields and update its constructors/callers. Return the bounded ledger from `providers::prepare_provider_state_before_bootstrap` into host initial preparation and combine it with the bootstrap writer's created leaves/parents under the one shared entry budget. Adopted sessions contribute no recursive authority. The ownership helper consumes that ledger with physical `symlink_metadata`/`lchown`, deduplicates, transfers deepest-first and HOME last; no fresh Amplifier traversal. Include these creator/callsite/helper changes in Task 1 before Task 3 uses them. - [ ] **Step 4: Green.** Repeat focused platform/controller/host commands; expected executable and Node resource/dependency assertions pass with correct boundary errors. - [ ] **Step 5: Refactor.** Remove the two obsolete private copier policies and basename-based secret filtering; keep provider selection and ownership at callers and filesystem behavior in the common module. Avoid adding environment policy configuration. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`. Expected all pass. These cover the shared policy and every existing caller. -- [ ] **Step 7: Commit.** `git add crates/freshell-platform/src/provider_environment.rs crates/freshell-platform/src/lib.rs crates/freshell-server/src/managed_provider_bootstrap.rs crates/freshell-server/src/managed_mcp_capability.rs crates/freshell-session-host/src/main.rs` then `git commit -m "fix: preserve bounded managed provider resources and executable modes"`. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`; the two `created_path_ledger`/`created_path_ownership` commands from Step 2. Expected all pass with positive counts, including at least three tests per new ledger/ownership module. These cover the shared policy, actual Amplifier creator/callsite and every existing caller. +- [ ] **Step 7: Commit.** `git add crates/freshell-platform/src/provider_environment.rs crates/freshell-platform/src/lib.rs crates/freshell-server/src/managed_provider_bootstrap.rs crates/freshell-server/src/managed_mcp_capability.rs crates/freshell-session-host/src/main.rs crates/freshell-session-host/src/providers/mod.rs crates/freshell-sessions/src/amplifier_stub.rs` then `git commit -m "fix: preserve bounded managed provider resources and executable modes"`. ### Task 2: Origin-correct configuration references through managed file grants @@ -245,7 +247,7 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, - [ ] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. - [ ] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. - [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. -- [ ] **Step 7: Commit.** Stage the exact production/test files in this task and `git commit -m "fix: resolve managed provider file references at their source origins"`. +- [ ] **Step 7: Focused compiling checkpoint commits.** Record Task 2's start commit. Commit coherent, compiling changes with their behavioral tests at major checkpoints, for example shared bounded decoder/planner, source-correct grant/mount preparation, then generation commit/abort and host refresh integration. Each commit must include the necessary protocol/callsite changes for its logical unit and pass its affected checks; do not force this entire task into one commit or commit broken intermediate wiring. Stage only that checkpoint's exact files and use a descriptive message such as `fix: resolve managed provider file references at their source origins`. Record the final Task 2 commit; the fresh task specification-plus-quality review covers the complete start..end range, including every intermediate checkpoint. ### Task 3: Transactional provider projection and complete environment smoke @@ -255,7 +257,7 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Interfaces:** Consumes `copy_environment`, `CopyReport`, `ReferencePlan` and existing `.freshell-config-projection.json`. Produces an all-or-error installed projection and unchanged previously working environment on failed refresh. Only prior manifest-owned paths may be replaced/removed. -- [ ] **Step 1: Add failing process tests.** Add backend `provider_private_parent_mount` production/adoption cases (so Task 3's `provider` selector actually runs them) and broker dispatch tests that accept the exact 0710 private-parent topology and reject wrong modes/extra authority; test through validator/dispatch behavior, not a source-text assertion. Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. +- [ ] **Step 1: Add failing process tests.** Add backend `provider_private_parent_mount` production/adoption cases (so Task 3's `provider` selector actually runs them) and broker dispatch tests that accept the exact 0710 private-parent topology and reject wrong modes/extra authority; test through validator/dispatch behavior, not a source-text assertion. Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Include Task 1's >10,000-entry initial preparation fixture, new/adopted Amplifier stubs, provider-created sibling and unrelated symlink target in the actual production initial-preparation/ownership/child smoke; assert complete selected resources run as the provider UID and only creator-recorded inodes changed ownership, with adopted sibling/target owner/content unchanged. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked --test provider_config_refresh`. Expected current stale-delete-before-copy path loses prior usable projection on injected failure, or resets executable mode. - [ ] **Step 3: Apply precise transaction patch.** The provider UID 65534/GID 0 reads the exact approved read-only controller stage and creates private projection/backup inodes on its own HOME volume; it never renames or chmods transport inodes and never stages the full ordinary tree in tmpfs. Stage the complete desired projection and manifest on that same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. For terminal and fresh initial launch, finish all host-required stub/bootstrap and Amplifier work first, transfer HOME ownership last, then install selected auth links and run provider-owned preparation. No later trusted-host HOME traversal. Exact resume skips host bootstrap/ownership reclamation. Change the private-parent Docker tmpfs producer, host initial/resume reset and disposable broker validator together to 0710. Use physical lchown ownership walks and preserve ordinary execute bits. Preserve existing setpriv boundary; do not add DAC_OVERRIDE/FOWNER. Native auth destinations survive when no corresponding typed auth-file grant is selected; CA/dynamic-file grants do not authorize auth deletion. Publish selected auth replacements before configuration/provider consumers, preserve old native destination on failure, and exclude native auth from ordinary projection manifest cleanup. Controller keeps temp-root→rename→capability publication ordering and revokes only new capability/resources on failure. Run-owned ephemeral references are cleaned with their grant; old grant/root/volume remain on failed replacement. Preparation error becomes explicit pre-spawn failure, never synthetic absence of identity. Log transaction stage/start/commit/rollback/failure once with safe counters and paths. @@ -268,16 +270,18 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Files:** - Modify: protocol `lib.rs:321`; agent runtime `lib.rs:256,824`; host `main.rs` (`ProbeRecovery`, currently line 662), `providers/mod.rs:432`; supervisor `checkpoints.rs:15,58`, `recovery.rs:1260,1876,1928`; tests in those modules and `loss_report_tests.rs`. +- Add actual host module `tests::probe_recovery` in `crates/freshell-session-host/src/main.rs` and supervisor module `checkpoints::tests` in `crates/freshell-supervisor/src/checkpoints.rs`. Extract only the existing probe decision into a callable helper if needed to exercise its production path directly. **Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPath`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. - [ ] **Step 1: Add failing decisions.** Host dispatched-journal/no resume identity returns Blocked and preserves data. Supervisor zero checkpoint revision is blocked. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. + Name host cases `probe_recovery::{dispatched_without_identity_blocks_and_preserves_data, startup_before_identity_retains_diagnostic, pristine_seed_still_recovers}` and execute actual ProbeRecovery decision/fixture behavior. Name supervisor cases `checkpoints::tests::{zero_revision_blocks, reason_text_is_not_inspection, legacy_negative_blocks, conflicting_rows_block, unreadable_store_blocks, independent_present_checkpoint_restores, inspected_shared_store_absence_remains_definitive}`. The last cases call real probe/refinement helpers with inspected fixture results; destructive process/corruption setup remains in sandbox. Before production changes, add these tests using the current interfaces, record behavioral RED, then extend them for additive evidence. Missing-module compilation and zero matches are setup failures, never RED evidence. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. - [ ] **Step 3: Apply exact decision replacements.** Host `ProbeRecovery` with None resume plus false `never_dispatched` returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero returns Blocked/unknown. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. Provider adapters emit NativeResume negatives only after complete inspection for effective identity, roots and accepted artifacts; a declaration or same executable invocation is insufficient. OpenCode exact SQLite no-match can cover both paths only when effective DB is established, including accepted data-root/DB/channel overrides. Claude/Codex scans that skip unreadable, deep, symlinked, oversized or database-selected candidates cannot certify absence: preserve exact positive recovery and block unknown negatives until complete authority exists. Amplifier inspection includes accepted backup transcript/metadata. The NativeSession fixture requires separate complete checkpoint inspection; oversize skipped candidate is Unknown. CheckpointRestore negatives require demonstrably complete shared or independent inspection. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline missing spec/identity after dispatch uses the same blocked classification. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. - [ ] **Step 4: Green.** All metadata-only paths block and retain data, confirmed inspected absence still authorizes the correct final result, independently present fixture checkpoint restores. - [ ] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification, pre-IPC input journals, queued-only replay and actor-native identity adoption. Input-row/native-ID absence alone is not pristine proof: consult durable execution-grant/actor evidence and record initialization intent before provider creation. Ambiguous startup before ID capture blocks; seed replacement needs actual no-dispatch/zero-turn proof. Preserve existing five-successful-replacements-per-hour budget and make final retry-budget Blocked sticky across automatic requests/cold restart until ManualRetry. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Record actual executed/passed counts: host `probe_recovery` must include at least the three named tests, supervisor `checkpoint` at least the seven named `checkpoints::tests` tests, and every other selector a positive count; zero matches or skipped required cases fail the gate. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. - [ ] **Step 7: Commit.** Stage this task's exact files and `git commit -m "fix: require inspected evidence before declaring recovery unavailable"`. ### Task 5: Typed final recovery status, retained spool and bounded transitions @@ -307,7 +311,7 @@ Managed output observations distinguish Running, active Recovering, Blocked with ### Task 6: Browser status/reattachment and complete configured-cloud managed environment proof **Files:** -- Modify: `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`; `src/App.tsx`, `src/store/panesSlice.ts`, `src/components/TerminalView.tsx`, `src/components/Sidebar.tsx`, `src/store/selectors/sidebarSelectors.ts`, `src/lib/session-utils.ts`; `test/unit/lib/managed-runtime-recovery.test.ts`; `test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`; `scripts/e2e-cloud.sh`; `test/e2e-browser/helpers/external-target.ts` only if receipt/ownership validation is needed. +- Modify: `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`; `src/App.tsx`, `src/store/panesSlice.ts`, `src/components/TerminalView.tsx`, `src/components/Sidebar.tsx`, `src/components/OverviewView.tsx`, `src/components/BackgroundSessions.tsx`, `src/store/selectors/sidebarSelectors.ts`, `src/lib/session-utils.ts`, `src/lib/api.ts`, `src/store/terminalDirectoryThunks.ts`, `src/store/terminalDirectorySlice.ts`; `test/unit/lib/managed-runtime-recovery.test.ts`, `test/unit/client/components/OverviewView.test.tsx`, `test/unit/client/components/BackgroundSessions.test.tsx`, `test/unit/client/store/terminalDirectoryThunks.test.ts`, `test/unit/client/store/terminalDirectorySlice.test.ts`; `test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`; `scripts/e2e-cloud.sh`; `test/e2e-browser/helpers/external-target.ts` only if receipt/ownership validation is needed. - Create: `test/unit/server/e2e-cloud-external-target.test.ts`, using fake gcloud executable dispatch to inspect the actual generated cloud env file and dispatched process inputs; this is not a source-string/config-text test. - Create: `test/e2e-browser/specs/managed-provider-environment-external.spec.ts`; run-owned fixture/tunnel orchestration under `scripts/testing/managed-provider-environment-target.ts` if the topology validation succeeds. - Modify: `test/e2e-browser/playwright.config.ts`, `playwright.cloud.config.ts` for the positively selected `chromium-external-target` project and ordinary discovery exclusion of the two external-target specs; this is separate from `CLOUD_SKIP_SPECS`. @@ -318,15 +322,16 @@ Managed output observations distinguish Running, active Recovering, Blocked with **External-target selection contract (also Task 7):** Ordinary Chromium/Firefox/WebKit projects exclude exactly `managed-provider-environment-external.spec.ts` and `code-checkpoint-workspace-external.spec.ts` in shared match-all discovery. Do not add them to `CLOUD_SKIP_SPECS`, discover them and call `test.skip`, or count absence from the ordinary broad suite as coverage. Add the dedicated `chromium-external-target` project only when `FRESHELL_E2E_EXTERNAL_FIXTURE_KIND` is the closed value `managed-environment` or `code-checkpoint` and its corresponding validated fixture JSON plus target URL/token are paired. Its `testMatch` selects only that kind's named spec and does not inherit the ordinary external-spec ignore. Explicit selection without a complete matching fixture is an error before dispatch/config use; its driver cannot fall back to an owned ordinary server or an ambient external/live target. Task 7's `FRESHELL_CODE_CHECKPOINT_FIXTURE` JSON contains candidate SHA, run ID, synthetic workspace/home/non-Git fixture paths, restricted UID and expected nonce only. Forward only these two new names and the kind through the cloud env-file allowlist. The wrapper's shard/spec discovery and job invocation must use the same positively selected project/fixture contract, otherwise zero discovered tests fail. Extend fake-gcloud dispatch tests and Playwright discovery tests by executing configuration/list selection: ordinary cloud lists neither external spec, a correctly paired kind lists/runs its exact nonzero spec, and missing/mismatched pairs fail. Record actual pass counts separately from the final ordinary broad receipt. - [ ] **Step 1: Add failing UI and real managed browser tests.** Extend unit cases from running and recovering through Blocked/Lost/Stopped, asserting exact projected status and retained summary/identity. Extend cloud-compatible contextual UI spec to show actual shell startup output, final blocked/lost/stopped status, existing retry controls, retained output and successful replacement attach; assert no silent extra `terminal.create`. These tests cover UI rendering/state and remain explicitly narrower than complete managed startup. + Feed blocked plain/paged directory rows through the real API/thunk/store flow, render Overview and Background Sessions, and assert the same terminal remains visible with its blocked reason/recovery data. Exercise existing Overview rename/fenced kill and Background attach/fenced kill controls as applicable; replay opens the retained output with input refused, rather than creating a replacement or fabricating running state. Assert counts exclude blocked processes and successful retry changes that same row to running. The contextual browser spec visits these actual surfaces and opens retained replay; it must preserve final diagnostics and existing controls rather than merely asserting an inventory object. The external managed environment spec verifies actual `/api/server-info` candidate and active managed runtime against fixture metadata, creates a new OpenCode CLI pane through the real picker against the scratch managed target, and asserts terminal output from executable skill/resources, a dependency-loaded plugin and resolved absolute/relative config-file references with per-run nonce. Target uses representative synthetic supported configuration and synthetic managed grants; actual controller stage and session host child execute unchanged production route. Missing selected reference and startup-before-identity cases show actionable retained diagnostics and blocked outcome; no hidden conversation replacement. Browser reconnect checks terminal/session identity and retained output. Assert target stage/child receipt matches the created terminal ID and candidate so an ordinary fixture or stale server cannot pass. Fake-gcloud behavioral tests invoke the wrapper with URL/token pairs, consume the actual --env-vars-file, and verify what the fake job receives; missing URL/token pair errors before dispatch, secret token never appears in stdout, and unrelated host variables are absent. -- [ ] **Step 2: Red.** Run `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:cloud test/unit/lib/managed-runtime-recovery.test.ts`; `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts`. Expected existing blocked projection remains recovering and running→final cases fail. The full managed external spec must fail on old candidate due to actual resource permissions/reference failure, not absent topology or zero matches. Establish the topology first as Stage 2 evidence. -- [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Separate running counts from retained managed membership in App census, panesSlice clearDeadTerminals, TerminalView attach/exit and sidebar/background lookup, including cold-load ordering and compatibility fallback. Read-only same-handle replay attach is distinct from input/create/recovery permission. Server cold lookup and WS reconcile adopt retained Stopped/Failed handles instead of ordinary respawn; attach cannot overwrite authoritative blocked status. Sidebar click uses retained owned target, including provisional pre-native identity. Reuse managed lifetime claims/reaper behavior; no new UI card or recovery protocol. +- [ ] **Step 2: Honest per-behavior Red and browser acceptance sequencing.** Run `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:cloud test/unit/lib/managed-runtime-recovery.test.ts`; the new affected component/directory tests through `test:vitest`; and `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts` before the corresponding UI changes. Record actual absent-UI/blocked-row failures on the current candidate where they exist. Tasks 1–5 already require behavioral RED before their respective production changes; preserve those receipts as the environment/recovery regression evidence. If backend work has already fixed a browser-observed behavior, record its passing baseline honestly and do not manufacture a failure or require a later old-candidate managed build. Stage 2's completed generic Cloud Run authenticated HTTP/WSS proof establishes transport feasibility only. Assemble the complete managed synthetic fixture as its assigned code becomes available, and require the genuinely executed full configured-cloud managed browser acceptance in Step 4 and final gates; missing topology, skips and zero matches never count as behavioral RED or acceptance. +- [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Separate running counts from retained managed membership in App census, panesSlice clearDeadTerminals, TerminalView attach/exit, Overview row partitions, detached Background Sessions rows and sidebar/background lookup, including cold-load ordering and compatibility fallback. Carry blocked status/summary through API directory thunks/store unchanged; render the existing rows and controls with accurate status, retained replay and recovery reason instead of filtering them out or relabeling their attach as running. Read-only same-handle replay attach is distinct from input/create/recovery permission. Server cold lookup and WS reconcile adopt retained Stopped/Failed handles instead of ordinary respawn; attach cannot overwrite authoritative blocked status. Sidebar click uses retained owned target, including provisional pre-native identity. Reuse managed lifetime claims/reaper behavior; no new UI card or recovery protocol. The owned generic Cloud Run HTTP/WSS preflight establishes reachability, while full candidate/browser proof remains required. Orchestrator allocates an owned `ManagedRuntimeBrowserRig`, explicit synthetic provider/workspace/bootstrap roots for both broker and supervisor, run-specific ports/logs, and a task-local Quick Tunnel with recorded PID/cwd. RuntimeHarness currently hardcodes real os.homedir() provider roots and ambient bootstrap grants; add narrow fixture inputs rather than assuming webHomeDir isolates them. Existing callers retain their defaults; this run never selects ambient real roots/grants. Download the official task-local Linux binary and execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. Extend the cloud env-file writer with only the exact allowlisted parameters above and enforce the External-target selection contract. Require URL/token together; serialize values as JSON strings valid in YAML, write a private temporary file and retain existing cleanup. Keep secret token out of command echo/receipts/logs. Do not forward arbitrary ambient vars. New spec uses existing ExternalServer fixture and fails if its matching paired fixture is absent; fixture setup occurs on target host before the cloud browser, not ignored `setupHome` options. Retain no destructive runtime cases in the unconstrained cloud fixture; actual kill/restart/corruption acceptance remains in the documented disposable sandbox. Quick Tunnel has HTTP/WebSocket support, no SSE, 200 concurrent-request limit and no uptime guarantee; select one spec/run and report preflight/transport failure explicitly without switching backend. Record copy bounds, exact unsupported-reference errors, credential reference lifecycle, unknown-vs-absent recovery and retained-output guarantees in agent docs. Do not add documentation-text tests or homepage changes for this repair. - [ ] **Step 4: Green.** Run `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts --config config/vitest/vitest.config.ts`; repeat cloud unit/contextual UI tests. Start the run-owned driver with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts`; after its owned target/handoff passes validation it sets the matching `managed-environment` kind and invokes `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/managed-provider-environment-external.spec.ts` on configured cloud. Expected nonzero tests pass and actual managed candidate/stage/child observations prove both preparation stages. No local-only qualification or UI mock substitutes. - [ ] **Step 5: Refactor.** Reuse external target/owned rig and existing recovery UI helpers. Consolidate allowed cloud env forwarding without adding permanent tunnel deployment, remote Docker protocol or generic provider framework. -- [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. At each passed task/review boundary, apply the periodic broad-gate rule in Acceptance below; the final exact-HEAD broad gate still follows all seven task commits/reviews. An additional earlier broad gate needs a recorded impacted-interface reason. +- [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts test/unit/client/components/OverviewView.test.tsx test/unit/client/components/BackgroundSessions.test.tsx test/unit/client/store/terminalDirectoryThunks.test.ts test/unit/client/store/terminalDirectorySlice.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. At each passed task/review boundary, apply the periodic broad-gate rule in Acceptance below; the final exact-HEAD broad gate still follows all seven task commits/reviews. An additional earlier broad gate needs a recorded impacted-interface reason. - [ ] **Step 7: Commit.** Stage exact TS/UI/tests/cloud-helper/docs files changed by this task and `git commit -m "test: verify managed provider environments and recovery in the browser"`. ### Task 7: Resolve code-rewind workspace, publish complete snapshots, and expose the result per turn @@ -334,10 +339,17 @@ Managed output observations distinguish Running, active Recovering, Blocked with **Criteria:** 12–15. **Dependencies:** finish the planning/assumption gate; coordinate shared fresh-agent/server files with Tasks 5–6. This task does not use code-rewind results as provider-store recovery evidence, does not change launch cwd, and does not restart production. Run-owned browser target/tunnel support from Task 6 may be reused; it must remain synthetic and isolated. **Files:** server `checkpoints.rs` and shared checkpoint service/state; WS send protocol and narrow injected trait plus managed/local barrier/admission paths; the exact producer/projection files named in Named consumer and turn-identity seams above; fixed sandbox entrypoint/wrapper fixture mode and owned bridge target driver; `crates/freshell-server/src/checkpoints.rs`; a small sibling checkpoint-scope helper only if it makes the route simpler; checkpoint route wiring in `main.rs` only if existing managed/session resolution is required; `src/lib/fresh-agent-checkpoints.ts`; `src/components/fresh-agent/FreshAgentView.tsx`, `FreshAgentTranscript.tsx`, `FreshAgentTurnActions.tsx`; relevant pane normalization/persistence in `src/store/paneTypes.ts` and `panesSlice.ts`; current checkpoint/component tests; one focused Rust black-box integration test; one new cloud-legal browser spec and its bounded synthetic fixture. Use existing fixture/support code where it actually supplies the needed behavior. No general snapshot framework, environment copier reuse, broad directory exclusions, chmod repair, or real-account fixture. + The OpenCode lane also owns `crates/freshell-opencode/Cargo.toml`, `crates/freshell-opencode/tests/checkpoint_message_id.rs`, `crates/freshell-opencode/tests/checkpoint_message_id_contract.rs`, and new narrow runner `scripts/testing/opencode-checkpoint-contract.ts`; these exact files enter the corresponding native-identity/checkpoint commit. -**Named identity regression tests:** Add `test/unit/claude-sidecar/user-message-replay.test.ts` to execute the real sidecar queue/SDK-input handler with an injected synthetic SDK transport and verify `claude_checkpoint_uuid_replay_binds_persisted_user`: the supplied valid UUID/request correlation survives actual user replay, mismatched/absent replay stays unknown, and unrelated assistant/result events cannot bind it. Add `test/integration/server/claude-checkpoint-uuid-replay.test.ts` to run the pinned SDK/bundled CLI with an isolated synthetic local provider endpoint, no model-network or real grants, and assert the same UUID in both replay and persisted user transcript. Reuse the synthetic probe arrangement recorded at `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`; a mock-input-only test is insufficient for this integration gate. Add `crates/freshell-opencode/tests/checkpoint_message_id.rs` with `opencode_checkpoint_message_id_persists_after_prompt`: execute the real prompt builder against pinned OpenCode 1.18.21 in an isolated synthetic store and local model endpoint, then read native persisted history and assert `messageID == info.id`. Separately add `checkpoint_turn_identity::opencode_checkpoint_message_id_survives_compact_queue` in `crates/freshell-freshagent/src/opencode_ws.rs`, where the compact queue actually lives: execute queue/drain/prompt and bind the observed persisted user row, rather than a queue acknowledgement or rewritten fixture ID. Freshagent tests named `checkpoint_turn_identity` also cover the production Codex native/display user join, modern duplicate text, aliases, unknown results and queue handling. Server tests named `managed_checkpoint_send_barrier` cover feature-gated post-capture admission, stale fences, one dispatch after failure and settlement/duplicate joining. Each named selector must execute a positive test count after Task 7's changes. +**Pinned OpenCode contract provisioning and gate:** In `crates/freshell-opencode/Cargo.toml`, add default-off feature `real-provider-contract-tests = ["real-transport"]` and explicit `[[test]]` target `name = "checkpoint_message_id_contract"`, `path = "tests/checkpoint_message_id_contract.rs"`, `required-features = ["real-provider-contract-tests"]`. No crate/production dependency enables this feature; default `cargo test --workspace --exclude freshell-tauri --locked`, crate prepush tests and default fake `checkpoint_message_id` remain binary-independent. Do not use `#[ignore]`, ambient executable discovery, a missing-binary return or a silent skip. -- [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Verify these tests fail for the intended current behaviors and record red output. + The opt-in command is `pnpm exec tsx scripts/testing/opencode-checkpoint-contract.ts`. This runner creates a mode-0700 run-owned temporary root and an independent tool project outside the repository/workspace, with exact `packageManager: pnpm@10.34.5`, standalone `pnpm-workspace.yaml` and fixture-local `allowBuilds` permitting only `opencode-ai`. It provisions `opencode-ai@1.18.21` with `pnpm add --save-exact opencode-ai@1.18.21` from that tool directory, retaining its fixture-local manifest/lock and package version/integrity receipt; a subsequent install uses `pnpm install --frozen-lockfile`. Optional platform packages and the normal package install script must complete so the real binary is usable. Never change root locks/dependencies, install globally, reuse user provider configuration or fall back to a different version. The runner selects only `/tool/node_modules/.bin/opencode`, executes its `--version` under synthetic HOME/XDG/config/data roots, and fails if installation fails, the binary is absent or the reported version is not exactly 1.18.21. Install traffic is package provisioning only; the contract uses a loopback synthetic model endpoint with no real provider/model grants. + + Then the runner supplies the absolute selected binary via `FRESHELL_OPENCODE_CONTRACT_BIN` and its owned fixture locator to `mise exec rust@1.96 -- cargo test -p freshell-opencode --features real-provider-contract-tests --locked --test checkpoint_message_id_contract opencode_checkpoint_message_id_persists_after_prompt -- --exact --nocapture`. The test independently validates the selected binary/version, starts the real production serve/prompt path with isolated synthetic configuration and local model server, and reads actual native persisted history. Use a 120-second whole-contract execution deadline (including real serve startup, synthetic prompt completion and history observation) and a 15-second owned-process cleanup deadline; timeout is a failing result, never a skip. Bound package provisioning separately to five minutes and fail rather than using an ambient binary. Own fixture PIDs and finish termination/reaping in `finally`, preserving only a private nonsecret command/version/integrity/pass-count receipt outside the tracked tree. Gate requires exactly one executed/passed contract test and zero failures/ignored tests; zero matches or missing/wrong binary is a failure. This dedicated lane remains mandatory before readiness and on final committed candidate HEAD, alongside the default fake tests. It does not imply enabling real-provider tests in the default CI/prepush lane. + +**Named identity regression tests:** Add `test/unit/claude-sidecar/user-message-replay.test.ts` to execute the real sidecar queue/SDK-input handler with an injected synthetic SDK transport and verify `claude_checkpoint_uuid_replay_binds_persisted_user`: the supplied valid UUID/request correlation survives actual user replay, mismatched/absent replay stays unknown, and unrelated assistant/result events cannot bind it. Add `test/integration/server/claude-checkpoint-uuid-replay.test.ts` to run the pinned SDK/bundled CLI with an isolated synthetic local provider endpoint, no model-network or real grants, and assert the same UUID in both replay and persisted user transcript. Reuse the synthetic probe arrangement recorded at `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`; a mock-input-only test is insufficient for this integration gate. Add default fake-only `crates/freshell-opencode/tests/checkpoint_message_id.rs` with `opencode_checkpoint_message_id_in_prompt_body` and `opencode_checkpoint_message_id_binds_observed_native_row`: execute the production prompt/transport path with injected HTTP/history fakes and assert its outgoing native ID and observed user-row binding without any provider binary. Add separate opt-in `crates/freshell-opencode/tests/checkpoint_message_id_contract.rs` with `opencode_checkpoint_message_id_persists_after_prompt`: execute that production path against pinned OpenCode 1.18.21 in an isolated synthetic store and local model endpoint, then read native persisted history and assert `messageID == info.id`. The fake target guards production behavior in default Cargo/CI/prepush; only the actual pinned contract target proves native persistence. Separately add `checkpoint_turn_identity::opencode_checkpoint_message_id_survives_compact_queue` in `crates/freshell-freshagent/src/opencode_ws.rs`, where the compact queue actually lives: execute queue/drain/prompt and bind the observed persisted user row, rather than a queue acknowledgement or rewritten fixture ID. Freshagent tests named `checkpoint_turn_identity` also cover the production Codex native/display user join, modern duplicate text, aliases, unknown results and queue handling. Server tests named `managed_checkpoint_send_barrier` cover feature-gated post-capture admission, stale fences, one dispatch after failure and settlement/duplicate joining. Each named selector must execute a positive test count after Task 7's changes. + +- [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Add the six timeout/cooldown service cases named in Step 4 before implementing that behavior: their controllable runner/clock must expose repeated full-wait behavior as RED, not just a missing field/module compile failure. Verify these tests fail for the intended current behaviors and record red output. - [ ] **Step 2: Resolve checkpoint scope independently from provider launch/routing cwd.** Give the checkpoint request additive explicit workspace information without reinterpreting `initialCwd` as proof of project intent. Resolve in this order: a valid deliberately explicit checkpoint workspace; revalidated canonical session checkout evidence; an independently resolved error-aware Git checkout from retained selected/session cwd (do not reuse presentation git_meta fallback as proof); then a valid deliberately selected non-Git directory. DirectoryPicker confirmation supplies starting-directory provenance, not an automatic explicit workspace override. checkoutPath and managed workspace_path may themselves be cwd fallbacks, so independently verify Git evidence rather than trusting field names or repository grouping. Missing/conflicting inferred provenance remains unavailable; explicit non-Git scope stays valid. Do not change provider routing cwd. Use the existing directory-selection/resume creation sites to carry selected scope provenance where it is actually known; default/home cwd alone supplies no such provenance. Avoid a new UI flow when current project/directory selection already gives the answer. @@ -355,6 +367,10 @@ Managed output observations distinguish Running, active Recovering, Blocked with The finite production capture budget is 15 seconds, beginning before workspace-lock acquisition and covering scope/readability traversal, Git, metadata and publication; tests inject a shorter budget through the checkpoint service, with no browser timer or new environment knob. Budget exhaustion produces `capture_timeout` and a clear per-turn reason. Cancellation settlement may outlast that deadline only while owned children are being terminated/reaped; provider dispatch waits for actual settlement. Record stage timings and eligible-file/changed-file counts without names on success; an ordinary unchanged-repeat fixture and a bounded many-file fixture must complete within this default on the supported synthetic lane, or tune the single value from measured evidence before completion while retaining a documented finite value. The service owns all Git children/publication work; timeout requests cancellation and waits for actual owned termination/settlement before releasing a send. Dropping a future or aborting browser HTTP is insufficient. Reconcile timeout racing update-ref under the workspace lock: either complete publication already succeeded while send was held, so retain Available, or it did not, so return Unavailable preserving prior targets. No remaining work may publish an attempt after terminal settlement/provider-send release. Do not promise absolute-clock/ref atomicity. Add a scoped owned Git runner with retained children/stdout/stderr tasks, explicit POSIX process-group termination/reaping/group settlement and supported Windows JobObject ownership before child execution, followed by direct wait and actual-job/process-count settlement. A dropped future, taskkill example or kill_on_drop alone is insufficient. Reconcile ref/complete metadata before terminalizing ambiguous publication. Destructive tests stay in the sandbox; preserve supported Windows capture rather than disabling its timeout behavior. + After an automatic capture actually settles as Unavailable `capture_timeout`, remember only a transient per-canonical-workspace timeout deadline/cause for 60 seconds, starting at settlement. During this cooldown, new automatic pre-turn attempts return immediate durable per-turn Unavailable `capture_timeout_cooldown` with a clear remaining retry delay, then forward their message normally; they do not reacquire a long capture lock, start Git, reuse a failed index or claim a previous turn's target. Only timeout outcomes establish this cooldown; successful complete capture clears it. Existing explicit checkpoint create/retry through the REST capture service bypasses cooldown and attempts a real complete capture, and a changed canonical scope/storage locator has its own attempt immediately. At expiry the next automatic turn retries within the same 15-second budget. A same-scope repair during cooldown does not silently mark any turn Available: an explicit retry or post-expiry capture must actually publish a complete checkpoint first. An explicit retry that times out renews the timeout deadline; a successful retry clears it for future turns, leaving earlier failed turns unavailable. Keep this state only in existing checkpoint route/service state; prune expired entries and bound live entries to 128 (evict oldest expired/oldest timeout entry), with no persistent cache framework, new user preferences or partial snapshot publication. + + Add deterministic production-service tests named `checkpoints::tests::{capture_timeout_establishes_cooldown, cooldown_sends_immediately_without_capture, cooldown_scope_change_attempts_capture, explicit_capture_bypasses_cooldown, repaired_scope_needs_published_success, cooldown_expiry_retries}` using an injected monotonic clock/budget and controllable owned capture runner. Assert actual runner invocation/settlement, one dispatch for each send, immediate later unavailable responses without a second full wait, continued earlier target validity, no failed target publication, changed-scope bypass, successful explicit retry and post-expiry retry. Include same-scope repair while cooldown remains active so it cannot fabricate availability. Run the `checkpoints` server selector with all six executed/passed cases and exercise send/row behavior through the managed/local barrier tests; cancellation/process-kill variants stay in the disposable sandbox. The browser verifies accepted later sends with the timeout reason and accurate disabled rewind where that synthetic scenario is selected; never wait a full repeated timeout merely to make sends usable. + Preserve the immutable send and validate captured conversation/session ownership and observed epoch/generation again after capture for every managed/local provider. Existing managed proxy drops observed fences and Claude does not enforce them, so implement the narrow post-delay admission rather than assuming it exists. Never reread latest pane/session/fence to retarget the request. A rejected stale/closed conversation is a send rejection distinct from snapshot failure. Within the process duplicate transport requests join the same barrier-and-forward job; known-rejected retries in the same canonical conversation may alias their new transport ID to the same logical capture result, never recapture that attempt. Ambiguous acceptance follows existing no-redispatch rules. A different conversation gets a separate logical attempt. Queue draining uses the same service immediately before forwarding only when capture has not settled. Persist one minimal positive/negative outcome independently from ephemeral metadata-patch bookkeeping: schema/version, logical attempt and transport aliases, canonical conversation, captured workspace, Available checkpoint ID or Unavailable code/reason, and native/display user-row join when established. No new prompt-text storage. Successful target authority remains complete server publication; pane persistence retains pending/current UI state and failed attempts. Metadata/outcome storage failure yields Unknown/Unavailable, never label-derived success. Restore uses captured scope and actual published target. @@ -371,8 +387,8 @@ Managed output observations distinguish Running, active Recovering, Blocked with Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately create a pane using the actual DirectoryPicker confirmation of a synthetic non-Git directory (no `.git`), send an accepted user turn, verify its resolved scope/exact row has enabled rewind, mutate at least two eligible files, restore that row through UI confirmation and verify both original contents and the unchanged non-Git scope. This tests picker provenance through client send, server resolution, publication and restore; API-only seeding is insufficient. Also cover home launch with explicit project scope and home launch without scope; the former snapshots only project code, the latter sends successfully with a clear unavailable reason and no home checkpoint. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. -- [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the periodic task-boundary gate and final exact-HEAD broad gate after all Tasks 1–7 join. Validate no foreign files or permission changes entered the diff. Commit Task 7 atomically with descriptive test evidence. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. - Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id opencode_checkpoint_message_id_persists_after_prompt`; `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/server/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx --config config/vitest/vitest.config.ts`. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, both home cases and actual picker-confirmed non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. +- [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the periodic task-boundary gate and final exact-HEAD broad gate after all Tasks 1–7 join. Validate no foreign files or permission changes entered the diff. Record Task 7 start/end commits and commit focused, compiling logical checkpoints with their behavioral tests, for example scope/complete publication, settled send barrier and native identity, then per-turn UI/restricted fixtures/browser acceptance. Include all required interface/callsite wiring in each checkpoint; do not commit broken intermediates or force this whole task into one commit. The fresh task specification-plus-quality review covers the entire start..end range and descriptive test evidence for every checkpoint. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. + Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id` (at least two fake-only tests); `pnpm exec tsx scripts/testing/opencode-checkpoint-contract.ts` (exactly one real pinned persistence test, using the required-features command above); `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/server/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx --config config/vitest/vitest.config.ts`. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, both home cases and actual picker-confirmed non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. ## Acceptance and completion boundary @@ -385,3 +401,5 @@ Managed output observations distinguish Running, active Recovering, Blocked with ## Dispatcher self-review The exact sanitized User Request occurs once and includes all 15 current kata criteria. Seven tasks retain behavioral red/green/refactor, focused commits/reviews, actual managed environment smoke, genuinely configured-cloud browser proof, conservative recovery/store decisions, durable retained diagnostics and code-rewind scope/completeness/turn outcomes. Source/ownership/protocol corrections are reflected in task inputs and acceptance; empirical claims are not certified from planned code. Environment copy, secrets and callback metadata have explicit separate profiles, with one policy across both stages and no lower hidden ownership limit. Code-rewind uses server-owned send/capture settlement, real provider identity and fixed restricted fixtures rather than browser timing or fake display IDs. Original workspace/native data and conversation semantics remain protected. The inline provider/projection/ownership/consumer/identity decisions are self-contained; supporting reports have absolute locators. Task 7 and final acceptance explicitly compile/run managed-runtime-v1 and named SDK/OpenCode identity regressions. Configured-cloud browser coverage includes picker-confirmed non-Git capture/restore with paired external-target discovery, and ordinary broad discovery does not count those separate receipts. Restricted initialization is a closed pre-gosu flag with no privileged mutation endpoint. Periodic broad gates run at passed task boundaries after at least 10 non-plan commits; the final whole-suite runs on committed final HEAD; PR/live rollout/MCP boundaries remain separately approved. Ordinary provider matrix requirements cannot be replaced by arbitrary-program certification or an unsupported-error workaround for a required reference. Reports remain outside tracked source; plan/code snippets are drafts, not implementation evidence. + +Round 2 remediation defines a fake-only default OpenCode target and a separately provisioned required-features pinned persistence lane, actual positive recovery selectors, the Amplifier creator-to-host ledger and real ownership smoke, Overview/Background/API-directory consumers, and a finite workspace timeout cooldown with accurate unavailable turns. Task 2 and Task 7 commit at compiling logical checkpoints and their task reviews cover complete start..end ranges. Behavioral RED precedes the relevant changes; already-green browser baselines stay honest, and full configured-cloud managed acceptance remains mandatory. Root dispositions for round 2 findings 6/7/10 remain OPTIONAL and finding 8 remains UNSUBSTANTIATED; this repair adds no speculative fixture-isolation/redaction work, removes no validated native-auth/ephemeral-projection mechanism, and does not move the AGENTS link. From dbcc7c2f2699ab113a58147d23bab666c42af6d0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 21:53:18 -0700 Subject: [PATCH 006/493] docs: settle final managed environment plan findings --- ...2026-10-05-managed-provider-environment.md | 61 +++++++++++-------- 1 file changed, 34 insertions(+), 27 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 35fb458c9..e05c67a83 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -45,7 +45,7 @@ Code-rewind uses the existing shadow Git store with isolated request indexes and 4. A selected missing, unreadable, cyclic, or unsupported resource is an actionable error, never an omitted configured capability. Optional provider roots absent before selection remain optional. Exclude concrete native state/cache/history and actual credential paths, not arbitrary basenames within dependencies. 5. Preserve executable bits in provider-owned private files as `0600 | (source_mode & 0111)`, directories 0700. All controller-owned selected transport inodes use mapped group 0, files `0640 | (source_mode & 0111)` and directories 0750, including ephemeral ordinary selections; physical ownership walks never follow links. Outer controller/control ancestors stay 0700. The exact read-only input bind under `/run/freshell-private/user-provider` uses a trusted 0:0 private parent 0710, while separate control/grant files remain private. Normalize or verify the rootless-daemon-mapped group, fail clearly if unavailable, and never modify original sources. No extra capabilities or setuid/setgid/sticky preservation; rootful support is not added. 6. Never expand secret file contents into staged configuration, registry/SQLite records, grant manifests, Docker JSON, logs, or test receipts. Persist only validated reference metadata. Known configuration syntax and declaring origin control resolution; never rewrite arbitrary strings that look like paths. -7. Metadata absence is unknown evidence. Loss requires actual authoritative native/checkpoint store inspection for every required recovery path, with typed path evidence. Legacy negatives without inspected evidence cannot authorize loss or cleanup. Proven pristine seed recovery and exact existing identities remain supported. +7. Metadata absence is unknown evidence. Loss requires actual authoritative native/checkpoint store inspection for every required recovery path, with typed path evidence. Legacy negatives without inspected evidence cannot authorize coding-provider loss or cleanup. Shell has no required NativeResume/CheckpointRestore paths: preserve verified terminal end after dispatch and its never-dispatched-only pristine policy without inventing store absence. Proven pristine seed recovery and exact existing identities remain supported. 8. Blocked status uses additive `runtimeStatus: blocked` and the existing `recoverySummary`. Retained managed membership is independent of whether a provider process runs. Preserve the exact terminal/session handle in API plain/paged listings, MCP/CLI output, inventory, cold reconnect, compatibility create/attach and sidebar navigation. A retained managed handle dominates ordinary dead-terminal clearing or respawn; read-only replay remains available while input and automatic recreation are blocked. Final exit follows settled retained-output drain and keeps identity/replay. Manual retry updates controller typed state and supervisor state; no new recovery UI or unrelated protocol framework. 9. Live deployment follows `docs/development/managed-runtime-rollout.md`, requires the user's separate `APPROVED`, and includes the managed-feature server, supervisor, session host and correct image. No PR is opened before explicit branch approval. The post-deployment MCP check remains a separate authorized operation. @@ -116,7 +116,7 @@ Task 7's exact producer/projection inputs are `crates/freshell-protocol/src/clie 6. `crates/freshell-supervisor/src/backend/docker.rs` and `backend.rs`: validate/mount exact rewritten project-config overlays and dynamic file references; creation/adoption parity. 7. `crates/freshell-session-host/src/main.rs`, `provider_secret_resolution.rs`, and `providers/mod.rs`: common copying, transactional provider-owned refresh, executable permissions/ownership, process-local referenced files and origin-correct child preparation. 8. `crates/freshell-agent-runtime/src/lib.rs`, supervisor `checkpoints.rs`, `recovery.rs` and `loss_report_tests.rs`: emit/consume inspected recovery evidence and block metadata-only decisions, including offline recovery. -9. `crates/freshell-supervisor/src/service.rs`, `registry.rs`, session-host `output_journal.rs`: narrowly expose exact owned incarnation spool reads after recovery ends. Share the existing parser in platform if necessary; no new output storage system. +9. `crates/freshell-supervisor/src/service.rs`, `registry.rs`, session-host `output_journal.rs`: narrowly expose exact owned incarnation spool reads after recovery ends. Share bounded spool descriptors/frame codec in the already-shared runtime protocol; host/supervisor keep separate local filesystem adapters and add no platform dependency or output store. 10. Server `managed_runtime.rs`, `terminals.rs`, terminal `registry.rs`, `managed_output_tests.rs`, WS `terminal.rs`: atomic typed outcomes, output/lifecycle propagation, final-head drain ordering, transition-deduplicated logs, and accurate user-facing terminal-directory process projection. 11. `shared/managed-runtime.ts`, `shared/ws-protocol.ts`, `src/store/types.ts`, `src/lib/recovery/managed-runtime-recovery.ts`, existing inventory consumers: process status correctness and blocked identity/output retention through existing summary projection. 12. Code-rewind server `checkpoints.rs`, existing scope/session projections, FreshAgentView/send/checkpoint helpers and shared turn actions: explicit snapshot scope, complete readable capture with isolated index/publish-last refs, typed correlated failures and persisted per-turn rewind availability. @@ -274,21 +274,21 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPath`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. -- [ ] **Step 1: Add failing decisions.** Host dispatched-journal/no resume identity returns Blocked and preserves data. Supervisor zero checkpoint revision is blocked. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. - Name host cases `probe_recovery::{dispatched_without_identity_blocks_and_preserves_data, startup_before_identity_retains_diagnostic, pristine_seed_still_recovers}` and execute actual ProbeRecovery decision/fixture behavior. Name supervisor cases `checkpoints::tests::{zero_revision_blocks, reason_text_is_not_inspection, legacy_negative_blocks, conflicting_rows_block, unreadable_store_blocks, independent_present_checkpoint_restores, inspected_shared_store_absence_remains_definitive}`. The last cases call real probe/refinement helpers with inspected fixture results; destructive process/corruption setup remains in sandbox. Before production changes, add these tests using the current interfaces, record behavioral RED, then extend them for additive evidence. Missing-module compilation and zero matches are setup failures, never RED evidence. +- [ ] **Step 1: Add failing decisions.** For coding providers with required NativeResume/CheckpointRestore paths, host dispatched-journal/no resume identity returns Blocked and preserves data; supervisor zero checkpoint revision is unknown/blocked. Managed shell has only Reattach and PristineSeed: after verified actual process end and dispatched input, online/offline recovery must reach `NonResumableTerminalEnded`, retain output and never fabricate native/checkpoint evidence. Uncertain process termination remains blocked/unsettled; proven never-dispatched shell still uses its existing pristine-seed-only policy. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. + Name host cases `probe_recovery::{dispatched_without_identity_blocks_and_preserves_data, startup_before_identity_retains_diagnostic, pristine_seed_still_recovers, shell_without_native_paths_uses_terminal_end_policy}` and execute actual ProbeRecovery decision/fixture behavior. Name supervisor cases `checkpoints::tests::{zero_revision_blocks, reason_text_is_not_inspection, legacy_negative_blocks, conflicting_rows_block, unreadable_store_blocks, independent_present_checkpoint_restores, inspected_shared_store_absence_remains_definitive}`. The last cases call real probe/refinement helpers with inspected fixture results; destructive process/corruption setup remains in sandbox. Add supervisor production recovery/loss cases `recovery::tests::{shell_dispatched_verified_end_online, shell_dispatched_verified_end_offline, shell_uncertain_end_blocks, shell_never_dispatched_preserves_pristine_seed}`; exact positive end cases assert the actual `NonResumableTerminalEnded` certificate, no replacement and retained output. Before production changes, add these tests using the current interfaces, record behavioral RED, then extend them for additive evidence. Missing-module compilation and zero matches are setup failures, never RED evidence. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. -- [ ] **Step 3: Apply exact decision replacements.** Host `ProbeRecovery` with None resume plus false `never_dispatched` returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero returns Blocked/unknown. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. - Provider adapters emit NativeResume negatives only after complete inspection for effective identity, roots and accepted artifacts; a declaration or same executable invocation is insufficient. OpenCode exact SQLite no-match can cover both paths only when effective DB is established, including accepted data-root/DB/channel overrides. Claude/Codex scans that skip unreadable, deep, symlinked, oversized or database-selected candidates cannot certify absence: preserve exact positive recovery and block unknown negatives until complete authority exists. Amplifier inspection includes accepted backup transcript/metadata. The NativeSession fixture requires separate complete checkpoint inspection; oversize skipped candidate is Unknown. CheckpointRestore negatives require demonstrably complete shared or independent inspection. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline missing spec/identity after dispatch uses the same blocked classification. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. +- [ ] **Step 3: Apply exact decision replacements.** Apply unknown conversions only to declared coding-provider recovery paths. Host `ProbeRecovery` with None resume plus false `never_dispatched` for a coding provider returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero for required CheckpointRestore is Blocked/unknown. Resolve provider/capability from the existing authoritative incarnation/context, not absence of a resume spec. For shell, bypass nonexistent NativeResume/CheckpointRestore probes in both online and offline recovery; certify unavailable Reattach only after verified process end/VerifiedEmpty and unavailable PristineSeed only from durable dispatched-input evidence, then use existing strict loss certification to produce `NonResumableTerminalEnded`. Keep exact inventory/cleanup validation; no fake Missing store row, weakened proof or identity requirement for a path shell does not declare. A live shell reattaches; uncertain process/backend termination remains Blocked with writer completion unresolved. Never-dispatched shell retains actual pristine proof. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. + Provider adapters emit NativeResume negatives only after complete inspection for effective identity, roots and accepted artifacts; a declaration or same executable invocation is insufficient. OpenCode exact SQLite no-match can cover both paths only when effective DB is established, including accepted data-root/DB/channel overrides. Claude/Codex scans that skip unreadable, deep, symlinked, oversized or database-selected candidates cannot certify absence: preserve exact positive recovery and block unknown negatives until complete authority exists. Amplifier inspection includes accepted backup transcript/metadata. The NativeSession fixture requires separate complete checkpoint inspection; oversize skipped candidate is Unknown. CheckpointRestore negatives require demonstrably complete shared or independent inspection. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline coding-provider missing spec/identity after dispatch uses the same blocked classification; shell follows the capability-specific verified terminal-end policy above. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. - [ ] **Step 4: Green.** All metadata-only paths block and retain data, confirmed inspected absence still authorizes the correct final result, independently present fixture checkpoint restores. - [ ] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification, pre-IPC input journals, queued-only replay and actor-native identity adoption. Input-row/native-ID absence alone is not pristine proof: consult durable execution-grant/actor evidence and record initialization intent before provider creation. Ambiguous startup before ID capture blocks; seed replacement needs actual no-dispatch/zero-turn proof. Preserve existing five-successful-replacements-per-hour budget and make final retry-budget Blocked sticky across automatic requests/cold restart until ManualRetry. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Record actual executed/passed counts: host `probe_recovery` must include at least the three named tests, supervisor `checkpoint` at least the seven named `checkpoints::tests` tests, and every other selector a positive count; zero matches or skipped required cases fail the gate. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. +- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Record actual executed/passed counts: host `probe_recovery` must include at least the four named tests, supervisor `checkpoint` at least the seven named `checkpoints::tests` tests, and sandbox supervisor `recovery` must execute all four named shell cases; every other selector needs a positive count; zero matches or skipped required cases fail the gate. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. - [ ] **Step 7: Commit.** Stage this task's exact files and `git commit -m "fix: require inspected evidence before declaring recovery unavailable"`. ### Task 5: Typed final recovery status, retained spool and bounded transitions **Files:** - Modify: server `managed_runtime.rs:22,77,357,441`, `terminals.rs:396,731`; supervisor `service.rs:1555`, `registry.rs:681`; host `output_journal.rs`; terminal `registry.rs:753,860,4733,4878`, `managed_output_tests.rs`; WS `terminal.rs:577,8488`; protocol `server_messages.rs:408`. -- Create only if extraction is needed: `crates/freshell-platform/src/terminal_spool.rs`, exported in `lib.rs`, containing the existing bounded spool parser shared by host/supervisor. +- Modify: `crates/freshell-runtime-protocol/src/lib.rs` exports; add `crates/freshell-runtime-protocol/src/terminal_spool.rs` for exact spool descriptors and the bounded frame codec. Host `output_journal.rs` and supervisor `service.rs` retain their local filesystem adapters; both already depend on runtime protocol, so no Cargo dependency widening. **Interfaces:** @@ -297,14 +297,14 @@ Managed output observations distinguish Running, active Recovering, Blocked with `None` means no lifecycle observation, not Running. Registry stores the current managed lifecycle independently of attachable facade identity. One controller mutex owns each soul's Recovering, Finished(RecoveryResult) or classified Failed slot; retain full typed result, exit/epoch and durable generation. Background and HTTP manual recovery publish through one state transition; successful retry resumes actual output/input. Supervisor retained-output resolver is read-only, exact owned incarnation, never authority for input/replacement/cleanup. -- [ ] **Step 1: Add failing output/status behaviors.** Starting from running and recovering, drive final Lost, Stopped and Blocked outcomes; inventory distinguishes blocked and exited, retains terminal ID/native identity/reason/incident and output. Invoke actual `/api/terminals` plain and paged listings plus MCP/CLI `list-terminals` against the managed facade: returned user-facing process status must be blocked, never running, while attachment/replay still succeeds and input is refused. Drive a successful retry and assert those same listings return running again. Deliver >64 KiB final tail in multiple pages, recover/stop between pages, reconnect two readers and verify identical retained window with exactly one terminal.exit after final head. Final output survives supervisor cleanup and web/controller reattachment using retained spool. Drive 20 identical failed polls across two attached sockets; expect one structured transition and no repeated output-refresh warning, then changed failure cause gives one new event and later successful retry gives a recovered event. Success Reattached/Replaced/AlreadyLive restores Running/input and exact identity with existing incarnation reset behavior. +- [ ] **Step 1: Add failing output/status behaviors.** Starting from running and recovering, drive final Lost, Stopped, Blocked and shell `NonResumableTerminalEnded` outcomes; inventory distinguishes blocked and exited, retains terminal ID/native identity/reason/incident and output. Invoke actual `/api/terminals` plain and paged listings plus MCP/CLI `list-terminals` against the managed facade: for the Blocked cases, returned user-facing process status must be blocked, never running, while attachment/replay still succeeds and input is refused; verified ended shell/Lost/Stopped cases report exited after settled completion. Drive a successful retry and assert those same listings return running again. Deliver >64 KiB final tail in multiple pages, recover/stop between pages, reconnect two readers and verify identical retained window with exactly one terminal.exit after final head. Add named `managed_runtime::tests::shell_terminal_ended_facade_retains_output` and terminal `managed_output_shell_terminal_ended_final_tail`: exercise actual ended-shell facade/inventory, drain retained output once to final head, refuse input, preserve identity and show exited status through plain/paged/MCP/CLI listings. Task 5 managed-runtime/managed-output selectors must execute these tests positively. Final output survives supervisor cleanup and web/controller reattachment using retained spool. Drive 20 identical failed polls across two attached sockets; expect one structured transition and no repeated output-refresh warning, then changed failure cause gives one new event and later successful retry gives a recovered event. Success Reattached/Replaced/AlreadyLive restores Running/input and exact identity with existing incarnation reset behavior. - [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime`; `mise exec rust@1.96 -- cargo test -p freshell-terminal --locked managed_output`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked managed`. Expected cached final errors suppress retained output, leave Running, and repeat warning events. Destructive spool cleanup/stop cases run in the documented sandbox. -- [ ] **Step 3: Apply precise lifecycle/read patch.** Replace separate in-flight/blocked locks with atomic per-soul typed slots and one state-publication function for background/manual recovery. Preserve observed exit code/epoch before scheduling. Settled Lost/Stopped reads return successful `Ended` observations; settled Blocked or classified preparation/identity failure returns `Blocked` plus retained output, never a cached Err before output access. Success clears failure and reports Running. Keep one recovery task per soul and existing timeout/backoff; read path never sleeps or starts/stops synchronously. - Extend supervisor terminal-output resolution from active-only handle to exact retained owned handle when state is Recovering/Blocked/Lost/Stopped. Persist actual output epoch (including host boot identity), terminal ID and incarnation before the output writer starts, preserving that association across host restart. Distinguish prior committed and failed replacement streams; source prefix is not actual epoch. Reuse existing two 32 MiB spool segments and bounded torn/truncated-frame handling. Query host if still accessible; otherwise read existing current/previous incarnation spool files, validating ownership/path/epoch/incarnation and parsing within two existing rotation limits. Preserve 64 KiB transport pages, UTF-8 framing, retained start/head, reset/truncation flags and real errors. Persist coherent completion keyed by terminal/incarnation/actual epoch after writer settlement. Normal EOF flush and existing successful host Stop reader join settle output; after abrupt removal, exact backend VerifiedEmpty settles the writer before spool scanning/freezing. Include retained bounds and known exit code or explicit unavailable status. Never pair stale sampled head with later exited flag. Lost with CleanupTerminationUnconfirmed or BackendUnavailable is a recovery outcome with unresolved writer completion, never fabricated stopped/empty output. Hydrate controller outcome/lifecycle from authoritative supervisor inventory when reconnecting after a web/controller restart, rather than relying on the in-memory finished map. Do not replace missing/unreadable spool with empty success. - WS ordering is: ingest complete page into replay; associate native identity as applicable; publish runtimeStatus/recovery lifecycle; call `finish_managed_exit` last only when delivered sequence reaches final head. Final outcome is visible while tail drains; paced replay defers terminal.exit as today. Never clear replay/terminal identity for blocked/final outcomes, and permit refresh/manual retry on blocked facades while refusing provider input. Lost/Stopped use known exit code or explicit unavailable completion semantics; unknown cleanup does not complete a stream. Managed terminal.exit retains client handle, stream and restore keys; ordinary exits retain existing behavior. +- [ ] **Step 3: Apply precise lifecycle/read patch.** Replace separate in-flight/blocked locks with atomic per-soul typed slots and one state-publication function for background/manual recovery. Preserve observed exit code/epoch before scheduling. Settled Lost/Stopped and shell `NonResumableTerminalEnded` reads return successful `Ended` observations; settled Blocked or classified preparation/identity failure returns `Blocked` plus retained output, never a cached Err before output access. Success clears failure and reports Running. Keep one recovery task per soul and existing timeout/backoff; read path never sleeps or starts/stops synchronously. + Extend supervisor terminal-output resolution from active-only handle to exact retained owned handle when state is Recovering/Blocked/Lost/Stopped or verified shell terminal-ended. Persist actual output epoch (including host boot identity), terminal ID and incarnation before the output writer starts, preserving that association across host restart. Distinguish prior committed and failed replacement streams; source prefix is not actual epoch. Reuse existing two 32 MiB spool segments and bounded torn/truncated-frame handling. Query host if still accessible; otherwise read existing current/previous incarnation spool files, validating ownership/path/epoch/incarnation and parsing within two existing rotation limits. Preserve 64 KiB transport pages, UTF-8 framing, retained start/head, reset/truncation flags and real errors. Persist coherent completion keyed by terminal/incarnation/actual epoch after writer settlement. Normal EOF flush and existing successful host Stop reader join settle output; after abrupt removal, exact backend VerifiedEmpty settles the writer before spool scanning/freezing. Include retained bounds and known exit code or explicit unavailable status. Never pair stale sampled head with later exited flag. Lost with CleanupTerminationUnconfirmed or BackendUnavailable is a recovery outcome with unresolved writer completion, never fabricated stopped/empty output. Hydrate controller outcome/lifecycle from authoritative supervisor inventory when reconnecting after a web/controller restart, rather than relying on the in-memory finished map. Do not replace missing/unreadable spool with empty success. + WS ordering is: ingest complete page into replay; associate native identity as applicable; publish runtimeStatus/recovery lifecycle; call `finish_managed_exit` last only when delivered sequence reaches final head. Final outcome is visible while tail drains; paced replay defers terminal.exit as today. Never clear replay/terminal identity for blocked/final outcomes, and permit refresh/manual retry on blocked facades while refusing provider input. Lost/Stopped/shell terminal-ended use known exit code or explicit unavailable completion semantics; unknown cleanup does not complete a stream. Managed terminal.exit retains client handle, stream and restore keys; ordinary exits retain existing behavior. Emit `managed_runtime.recovery_transition` only on changed state/classified cause with terminal/soul/incarnation/attempt/incident/from/to/outcome/cause fields. A controller/registry-shared transport-error fingerprint suppresses repeated unchanged output warnings across sockets and emits one recovered/change event. Do not log full probe/raw credential-bearing error strings. Facade inventory exposes additive RuntimeStatus::Blocked and existing recovery summary; final Exited follows current lifecycle handling. Server `terminals.rs::directory_items` must project managed process state from runtime status/recovery outcome before its internal facade `TerminalRunStatus` fallback. A blocked facade's user-facing `status` is blocked in plain and paged directory output, so unchanged MCP/CLI list pass-throughs also report blocked. Extend the actual directory item schema/types to permit that status; preserve ordinary terminal statuses and internal attachment/lifetime checks. If consumer rendering transforms statuses, audit and correct that transformation too. - [ ] **Step 4: Green.** Repeat focused tests; expected full final tail, accurate inventory/status, one exit and bounded transitions; successful retry behavior remains green. -- [ ] **Step 5: Refactor.** Consolidate final/manual/background result publication, extract only existing spool parsing if needed, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. +- [ ] **Step 5: Refactor.** Consolidate final/manual/background result publication, keep shared spool descriptor/frame codec in runtime protocol and local readers in their owning crates, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. - [ ] **Step 6: Impact verification.** Run the three focused Cargo commands from Step 2 plus `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked retained_output"` after new tests use that filter, and `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected all run/pass. Include existing `managed_runtime_api::stop_contract_tests` with `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime_api::stop_contract_tests` and record a positive executed test count; a zero-test filter fails the explicit-stop verification. - [ ] **Step 7: Commit.** Stage listed lifecycle/output files and `git commit -m "fix: retain managed terminal output and publish final recovery outcomes"`. @@ -315,6 +315,7 @@ Managed output observations distinguish Running, active Recovering, Blocked with - Create: `test/unit/server/e2e-cloud-external-target.test.ts`, using fake gcloud executable dispatch to inspect the actual generated cloud env file and dispatched process inputs; this is not a source-string/config-text test. - Create: `test/e2e-browser/specs/managed-provider-environment-external.spec.ts`; run-owned fixture/tunnel orchestration under `scripts/testing/managed-provider-environment-target.ts` if the topology validation succeeds. - Modify: `test/e2e-browser/playwright.config.ts`, `playwright.cloud.config.ts` for the positively selected `chromium-external-target` project and ordinary discovery exclusion of the two external-target specs; this is separate from `CLOUD_SKIP_SPECS`. +- Modify: `test/e2e-browser/helpers/managed-runtime.ts`, `scripts/testing/runtime-sandbox.ts` for closed synthetic roots and fixture activation; `crates/freshell-server/src/main.rs` for scratch-only registry construction; `crates/freshell-terminal/src/registry.rs` for ordinary-PTY creation refusal; real endpoint rejection tests in `crates/freshell-server/src/terminals.rs` and `crates/freshell-ws/src/terminal.rs`. - Modify: `docs/development/managed-runtime.md`; `AGENTS.md` plan/runbook links; README only for actual changed user setup. **Interfaces:** Existing inventory `runtimeStatus` admits `blocked` and the existing Rust exited process state where final tail is still draining; pane recovery projection uses `error` for Blocked/Lost, `exited` for Stopped, `recovering` only for active recovery, Running only after verified live recovery. Retain recoverySummary reason/attempt/incident and existing retry controls. External browser consumes `FRESHELL_E2E_TARGET_URL`, `FRESHELL_E2E_TARGET_WS_URL`, `FRESHELL_E2E_TARGET_TOKEN`, optional `FRESHELL_E2E_TARGET_TIMEOUT_MS`, and the narrow nonsecret `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE` JSON parameter through allowlisted cloud forwarding. That parameter contains candidate SHA, run ID, remote workspace path and expected nonce only; ownership/cleanup receipt remains on the target host. Do not forward TARGET_HOME: cloud `/app` is a different filesystem. Candidate provenance comes from authenticated `/api/server-info` commit/buildDirty; actual managed readiness comes from `/api/runtime/readiness`, negotiated ready.managedRuntimeV1 and the created terminal/soul/incarnation inventory, not an invented server-info flag. @@ -326,19 +327,19 @@ Managed output observations distinguish Running, active Recovering, Blocked with The external managed environment spec verifies actual `/api/server-info` candidate and active managed runtime against fixture metadata, creates a new OpenCode CLI pane through the real picker against the scratch managed target, and asserts terminal output from executable skill/resources, a dependency-loaded plugin and resolved absolute/relative config-file references with per-run nonce. Target uses representative synthetic supported configuration and synthetic managed grants; actual controller stage and session host child execute unchanged production route. Missing selected reference and startup-before-identity cases show actionable retained diagnostics and blocked outcome; no hidden conversation replacement. Browser reconnect checks terminal/session identity and retained output. Assert target stage/child receipt matches the created terminal ID and candidate so an ordinary fixture or stale server cannot pass. Fake-gcloud behavioral tests invoke the wrapper with URL/token pairs, consume the actual --env-vars-file, and verify what the fake job receives; missing URL/token pair errors before dispatch, secret token never appears in stdout, and unrelated host variables are absent. - [ ] **Step 2: Honest per-behavior Red and browser acceptance sequencing.** Run `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:cloud test/unit/lib/managed-runtime-recovery.test.ts`; the new affected component/directory tests through `test:vitest`; and `GCLOUD_ROBOT_REQUIRE=1 pnpm run test:e2e --project=chromium test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts` before the corresponding UI changes. Record actual absent-UI/blocked-row failures on the current candidate where they exist. Tasks 1–5 already require behavioral RED before their respective production changes; preserve those receipts as the environment/recovery regression evidence. If backend work has already fixed a browser-observed behavior, record its passing baseline honestly and do not manufacture a failure or require a later old-candidate managed build. Stage 2's completed generic Cloud Run authenticated HTTP/WSS proof establishes transport feasibility only. Assemble the complete managed synthetic fixture as its assigned code becomes available, and require the genuinely executed full configured-cloud managed browser acceptance in Step 4 and final gates; missing topology, skips and zero matches never count as behavioral RED or acceptance. - [ ] **Step 3: Apply precise browser/infrastructure patches.** Update additive inventory status types and reducer's terminalStatus mapping; preserve TerminalView's existing cross-channel ordering guard and authoritative summary reconciliation. Separate running counts from retained managed membership in App census, panesSlice clearDeadTerminals, TerminalView attach/exit, Overview row partitions, detached Background Sessions rows and sidebar/background lookup, including cold-load ordering and compatibility fallback. Carry blocked status/summary through API directory thunks/store unchanged; render the existing rows and controls with accurate status, retained replay and recovery reason instead of filtering them out or relabeling their attach as running. Read-only same-handle replay attach is distinct from input/create/recovery permission. Server cold lookup and WS reconcile adopt retained Stopped/Failed handles instead of ordinary respawn; attach cannot overwrite authoritative blocked status. Sidebar click uses retained owned target, including provisional pre-native identity. Reuse managed lifetime claims/reaper behavior; no new UI card or recovery protocol. - The owned generic Cloud Run HTTP/WSS preflight establishes reachability, while full candidate/browser proof remains required. Orchestrator allocates an owned `ManagedRuntimeBrowserRig`, explicit synthetic provider/workspace/bootstrap roots for both broker and supervisor, run-specific ports/logs, and a task-local Quick Tunnel with recorded PID/cwd. RuntimeHarness currently hardcodes real os.homedir() provider roots and ambient bootstrap grants; add narrow fixture inputs rather than assuming webHomeDir isolates them. Existing callers retain their defaults; this run never selects ambient real roots/grants. Download the official task-local Linux binary and execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. + The owned generic Cloud Run HTTP/WSS preflight establishes reachability, while full candidate/browser proof remains required. Orchestrator allocates an owned `ManagedRuntimeBrowserRig`, explicit synthetic provider/workspace/bootstrap roots for both broker and supervisor, run-specific ports/logs, and a task-local Quick Tunnel with recorded PID/cwd. RuntimeHarness currently hardcodes real os.homedir() provider roots and ambient bootstrap grants; add narrow fixture inputs rather than assuming webHomeDir isolates them. Existing callers retain their defaults; this run never selects ambient real roots/grants. Before exposing the tunnel, the owned rig activates one closed scratch-only `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY=1` server-construction input, supplied only by its validated synthetic setup. It constructs the terminal registry with ordinary host PTY creation disabled at the shared `TerminalRegistry::create` spawn boundary; managed facade registration remains enabled. All WS clients (including legacy/non-negotiated), HTTP/CLI/MCP create paths and error fallbacks therefore receive an explicit refusal instead of spawning any ordinary host PTY. This fixture flag is not a general product permission policy; ordinary launches retain existing construction defaults. Add actual endpoint tests `managed_fixture_denies_host_pty::{legacy_ws, non_negotiated_ws, http_cli_fallback, managed_coding_still_works}` across real WS/server test modules, plus a shared-registry refusal test. Exercise real authenticated requests, assert no child PID/ordinary terminal appears and a negotiated managed coding pane still runs its synthetic configuration. Require positive executed counts via `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_fixture_denies_host_pty`, `mise exec rust@1.96 -- cargo test -p freshell-ws --locked managed_fixture_denies_host_pty` and `mise exec rust@1.96 -- cargo test -p freshell-terminal --locked fixture_denies_host_pty` before tunnel publication (the endpoint commands together run all four named cases; each selector executes at least one), and repeat rejection checks against the actual owned scratch target. Pin the official task-local Linux `cloudflared` binary to `2026.10.0`, with SHA256 `d33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db`: download only that exact release asset into the private run-owned tool directory, verify checksum before execution and version exactly, or reuse an existing equivalent only after both checks. Never install globally or download latest. Then execute `cloudflared tunnel --url http://127.0.0.1:` with HOME/config isolated from existing installations; record the generated HTTPS hostname without token. It starts the scratch target, verifies local health/WebSocket and tunnel health, writes a private receipt/target handoff, launches the configured cloud browser test, and cleans only exact owned processes/containers/files in finally after confirming identities. The tunnel exposes only the synthetic authenticated scratch server and ephemeral URL after the common ordinary-host-PTY deny checks pass. The existing authorized synthetic configured-cloud test requires no new approval; additional general isolation/private-trace work retains its existing OPTIONAL disposition. It never uses the live 3001 server or real user credentials. Preserve local workload MCP callback routing; browser alone uses the public hostname. If Host/Origin policy requires adjustment, allow only this generated origin via scratch `EXTRA_ALLOWED_ORIGINS`, never production configuration. Extend the cloud env-file writer with only the exact allowlisted parameters above and enforce the External-target selection contract. Require URL/token together; serialize values as JSON strings valid in YAML, write a private temporary file and retain existing cleanup. Keep secret token out of command echo/receipts/logs. Do not forward arbitrary ambient vars. New spec uses existing ExternalServer fixture and fails if its matching paired fixture is absent; fixture setup occurs on target host before the cloud browser, not ignored `setupHome` options. Retain no destructive runtime cases in the unconstrained cloud fixture; actual kill/restart/corruption acceptance remains in the documented disposable sandbox. Quick Tunnel has HTTP/WebSocket support, no SSE, 200 concurrent-request limit and no uptime guarantee; select one spec/run and report preflight/transport failure explicitly without switching backend. Record copy bounds, exact unsupported-reference errors, credential reference lifecycle, unknown-vs-absent recovery and retained-output guarantees in agent docs. Do not add documentation-text tests or homepage changes for this repair. - [ ] **Step 4: Green.** Run `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts --config config/vitest/vitest.config.ts`; repeat cloud unit/contextual UI tests. Start the run-owned driver with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts`; after its owned target/handoff passes validation it sets the matching `managed-environment` kind and invokes `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/managed-provider-environment-external.spec.ts` on configured cloud. Expected nonzero tests pass and actual managed candidate/stage/child observations prove both preparation stages. No local-only qualification or UI mock substitutes. - [ ] **Step 5: Refactor.** Reuse external target/owned rig and existing recovery UI helpers. Consolidate allowed cloud env forwarding without adding permanent tunnel deployment, remote Docker protocol or generic provider framework. - [ ] **Step 6: Focused impacted-test verification.** Run `pnpm run typecheck:client`; `pnpm run test:vitest run test/unit/server/e2e-cloud-external-target.test.ts test/unit/lib/managed-runtime-recovery.test.ts test/unit/client/components/OverviewView.test.tsx test/unit/client/components/BackgroundSessions.test.tsx test/unit/client/store/terminalDirectoryThunks.test.ts test/unit/client/store/terminalDirectorySlice.test.ts --config config/vitest/vitest.config.ts`; the four-provider positive managed parity command from Task 3; both affected browser commands from Step 4; existing affected OpenCode/recovery specs positively selected on the configured backend. Confirm every spec executes rather than cloud-skip/no-match. Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime` and `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider` because default cloud binary omits that feature; affected supervisor/runtime destructive selectors stay in disposable sandbox. Include the behavioral terminal-directory/MCP/CLI listing tests from Task 5, not only browser summary tests. Capture command/result/actual spec counts, JSON transition evidence and nonsecret environment receipts. At each passed task/review boundary, apply the periodic broad-gate rule in Acceptance below; the final exact-HEAD broad gate still follows all seven task commits/reviews. An additional earlier broad gate needs a recorded impacted-interface reason. -- [ ] **Step 7: Commit.** Stage exact TS/UI/tests/cloud-helper/docs files changed by this task and `git commit -m "test: verify managed provider environments and recovery in the browser"`. +- [ ] **Step 7: Focused compiling commits.** Record Task 6 start/end commits and commit compiling logical changes with their behavioral tests: `fix: preserve retained managed terminal status in browser views` for production UI/API/store changes; `test: constrain synthetic managed browser targets and cloud handoff` for registry fixture denial, rig/tunnel/cloud wrapper/discovery and acceptance specs; then `docs: document managed environment and recovery guarantees` when a separate documentation checkpoint is appropriate. Keep complete necessary interface/callsite wiring in each commit and use accurate fix/test titles if the grouping differs. Do not force UI behavior, harness/cloud and docs into a single test-labelled commit. The fresh task specification-plus-quality review covers the entire Task 6 start..end range and all checkpoint evidence. ### Task 7: Resolve code-rewind workspace, publish complete snapshots, and expose the result per turn **Criteria:** 12–15. **Dependencies:** finish the planning/assumption gate; coordinate shared fresh-agent/server files with Tasks 5–6. This task does not use code-rewind results as provider-store recovery evidence, does not change launch cwd, and does not restart production. Run-owned browser target/tunnel support from Task 6 may be reused; it must remain synthetic and isolated. -**Files:** server `checkpoints.rs` and shared checkpoint service/state; WS send protocol and narrow injected trait plus managed/local barrier/admission paths; the exact producer/projection files named in Named consumer and turn-identity seams above; fixed sandbox entrypoint/wrapper fixture mode and owned bridge target driver; `crates/freshell-server/src/checkpoints.rs`; a small sibling checkpoint-scope helper only if it makes the route simpler; checkpoint route wiring in `main.rs` only if existing managed/session resolution is required; `src/lib/fresh-agent-checkpoints.ts`; `src/components/fresh-agent/FreshAgentView.tsx`, `FreshAgentTranscript.tsx`, `FreshAgentTurnActions.tsx`; relevant pane normalization/persistence in `src/store/paneTypes.ts` and `panesSlice.ts`; current checkpoint/component tests; one focused Rust black-box integration test; one new cloud-legal browser spec and its bounded synthetic fixture. Use existing fixture/support code where it actually supplies the needed behavior. No general snapshot framework, environment copier reuse, broad directory exclusions, chmod repair, or real-account fixture. +**Files:** server `checkpoints.rs` and shared checkpoint service/state; WS send protocol and narrow injected trait plus managed/local barrier/admission paths; the exact producer/projection files named in Named consumer and turn-identity seams above; fixed sandbox entrypoint/wrapper fixture mode and owned bridge target driver; `crates/freshell-server/src/checkpoints.rs`; a small sibling checkpoint-scope helper only if it makes the route simpler; checkpoint route wiring in `main.rs` only if existing managed/session resolution is required; `src/lib/fresh-agent-checkpoints.ts`; `src/components/fresh-agent/FreshAgentView.tsx`, `FreshAgentTranscript.tsx`, `FreshAgentTurnActions.tsx`; existing directory confirmation in `src/components/panes/DirectoryPicker.tsx`, `PaneContainer.tsx` and `crates/freshell-server/src/files.rs` (`validate_dir`); resume/hydration producers `src/lib/session-type-utils.ts`, `pane-reconcile.ts`, `tab-registry-open.ts` and `src/store/tabsSlice.ts`; relevant pane normalization/persistence in `src/store/paneTypes.ts`, `panesSlice.ts` and `src/store/persistMiddleware.ts`; current checkpoint/component tests including `test/unit/client/components/panes/{DirectoryPicker,PaneContainer,PaneContainer.createContent}.test.tsx`, pane normalization/persistence and resume producer tests; one focused Rust black-box integration test; one new cloud-legal browser spec and its bounded synthetic fixture. Use existing fixture/support code where it actually supplies the needed behavior. No general snapshot framework, environment copier reuse, broad directory exclusions, chmod repair, or real-account fixture. The OpenCode lane also owns `crates/freshell-opencode/Cargo.toml`, `crates/freshell-opencode/tests/checkpoint_message_id.rs`, `crates/freshell-opencode/tests/checkpoint_message_id_contract.rs`, and new narrow runner `scripts/testing/opencode-checkpoint-contract.ts`; these exact files enter the corresponding native-identity/checkpoint commit. **Pinned OpenCode contract provisioning and gate:** In `crates/freshell-opencode/Cargo.toml`, add default-off feature `real-provider-contract-tests = ["real-transport"]` and explicit `[[test]]` target `name = "checkpoint_message_id_contract"`, `path = "tests/checkpoint_message_id_contract.rs"`, `required-features = ["real-provider-contract-tests"]`. No crate/production dependency enables this feature; default `cargo test --workspace --exclude freshell-tauri --locked`, crate prepush tests and default fake `checkpoint_message_id` remain binary-independent. Do not use `#[ignore]`, ambient executable discovery, a missing-binary return or a silent skip. @@ -349,15 +350,19 @@ Managed output observations distinguish Running, active Recovering, Blocked with **Named identity regression tests:** Add `test/unit/claude-sidecar/user-message-replay.test.ts` to execute the real sidecar queue/SDK-input handler with an injected synthetic SDK transport and verify `claude_checkpoint_uuid_replay_binds_persisted_user`: the supplied valid UUID/request correlation survives actual user replay, mismatched/absent replay stays unknown, and unrelated assistant/result events cannot bind it. Add `test/integration/server/claude-checkpoint-uuid-replay.test.ts` to run the pinned SDK/bundled CLI with an isolated synthetic local provider endpoint, no model-network or real grants, and assert the same UUID in both replay and persisted user transcript. Reuse the synthetic probe arrangement recorded at `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`; a mock-input-only test is insufficient for this integration gate. Add default fake-only `crates/freshell-opencode/tests/checkpoint_message_id.rs` with `opencode_checkpoint_message_id_in_prompt_body` and `opencode_checkpoint_message_id_binds_observed_native_row`: execute the production prompt/transport path with injected HTTP/history fakes and assert its outgoing native ID and observed user-row binding without any provider binary. Add separate opt-in `crates/freshell-opencode/tests/checkpoint_message_id_contract.rs` with `opencode_checkpoint_message_id_persists_after_prompt`: execute that production path against pinned OpenCode 1.18.21 in an isolated synthetic store and local model endpoint, then read native persisted history and assert `messageID == info.id`. The fake target guards production behavior in default Cargo/CI/prepush; only the actual pinned contract target proves native persistence. Separately add `checkpoint_turn_identity::opencode_checkpoint_message_id_survives_compact_queue` in `crates/freshell-freshagent/src/opencode_ws.rs`, where the compact queue actually lives: execute queue/drain/prompt and bind the observed persisted user row, rather than a queue acknowledgement or rewritten fixture ID. Freshagent tests named `checkpoint_turn_identity` also cover the production Codex native/display user join, modern duplicate text, aliases, unknown results and queue handling. Server tests named `managed_checkpoint_send_barrier` cover feature-gated post-capture admission, stale fences, one dispatch after failure and settlement/duplicate joining. Each named selector must execute a positive test count after Task 7's changes. -- [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Add the six timeout/cooldown service cases named in Step 4 before implementing that behavior: their controllable runner/clock must expose repeated full-wait behavior as RED, not just a missing field/module compile failure. Verify these tests fail for the intended current behaviors and record red output. +- [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add the named picker producer/normalization/route tests from Step 2, including actual validate-dir HOME/non-home results and mkdir-confirm reuse. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Add the ten timeout/cooldown/lock-budget service cases named in Step 4 before implementing that behavior: their controllable runner/clock must expose repeated full-wait behavior as RED, not just a missing field/module compile failure. Verify these tests fail for the intended current behaviors and record red output. -- [ ] **Step 2: Resolve checkpoint scope independently from provider launch/routing cwd.** Give the checkpoint request additive explicit workspace information without reinterpreting `initialCwd` as proof of project intent. Resolve in this order: a valid deliberately explicit checkpoint workspace; revalidated canonical session checkout evidence; an independently resolved error-aware Git checkout from retained selected/session cwd (do not reuse presentation git_meta fallback as proof); then a valid deliberately selected non-Git directory. DirectoryPicker confirmation supplies starting-directory provenance, not an automatic explicit workspace override. checkoutPath and managed workspace_path may themselves be cwd fallbacks, so independently verify Git evidence rather than trusting field names or repository grouping. Missing/conflicting inferred provenance remains unavailable; explicit non-Git scope stays valid. Do not change provider routing cwd. Use the existing directory-selection/resume creation sites to carry selected scope provenance where it is actually known; default/home cwd alone supplies no such provenance. Avoid a new UI flow when current project/directory selection already gives the answer. +- [ ] **Step 2: Resolve checkpoint scope independently from provider launch/routing cwd.** Use the existing DirectoryPicker confirmation as the real explicit-scope producer. Add persisted `FreshAgentPaneContent.checkpointWorkspace?: { path: string; provenance: "directory-picker" | "resolved-project" | "legacy-checkpoint" }`, normalized as a separate field from `initialCwd`. Extend the existing authenticated `/api/files/validate-dir` result with additive `checkpointWorkspaceEligible: boolean` using the same error-aware account-home classification as the checkpoint resolver. `DirectoryPicker` retains this result until actual confirmation and passes `(resolvedCwd, eligibleWorkspace?)` to `PaneContainer.handleDirectoryConfirm`; the existing mkdir-confirm path reuses validate-dir after creation. A confirmed eligible non-account-home project/non-Git directory populates both starting cwd and this independent workspace record; confirmed account HOME or unresolved eligibility is launch-only and never qualifies as explicit non-Git scope. No extra dialog or new endpoint is added, and the server revalidates at capture instead of trusting stale eligibility. Default/provider cwd, settings fallback and an unconfirmed picker value do not create this record. Existing resume/hydration creation in `session-type-utils::buildResumeContent`, `pane-reconcile`, `tab-registry-open` and tabs/panes reducers preserves an already carried scope record; when independently resolved canonical project checkout evidence or a validated prior checkpoint locator actually exists, carry it as its own declared provenance, never fabricating picker intent from cwd. Missing evidence remains absent and the server can still resolve Git checkout evidence by the next tier. Keep the record immutable for each logical attempt/checkpoint; later launch/routing cwd changes do not change that attempt or older restore destinations. No new picker/modal or independent home/project selection flow is needed. - Make the server the final filesystem resolver: canonicalize an existing directory, distinguish inaccessible/not-directory from absent, and return the resolved workspace on successful create. Account-home detection must cover the relevant account, not only `CheckpointsApiState.home` (the reproduced failure involves a different UID's home). Use OS account-home information and known managed home paths on Linux; ensure the cross-platform fallback does not disable valid Windows project scopes. Automatically inferred scopes equal to a recognized account home are unavailable even when a dotfiles Git repository happens to exist there. A genuine explicit non-Git scope remains supported. Never search arbitrary subdirectories of a home for a project or silently broaden a selected subtree to another checkout. If root evidence conflicts or is missing, return a typed unavailable reason and keep sending usable. + Define the additive shared `CheckpointWorkspaceIntent` shape in `shared/fresh-agent-contract.ts`, reference it in `shared/ws-protocol.ts` modern `freshAgent.send.checkpointIntent.workspace`, and carry the same optional workspace intent in REST checkpoint create. Mirror it in `crates/freshell-protocol/src/client_messages.rs` and the injected server checkpoint-service request; `FreshAgentView` freezes the pane record into its send intent and checkpoint helpers/persistence carry the resulting canonical scope/storage locator. Existing managed proxy/WS local admission paths forward the intent unchanged to the server service before provider dispatch; provider launch still uses its separate `cwd`. The server treats provenance as a scope candidate, validates conversation ownership and performs final filesystem/account-home/Git resolution; the client record never certifies readability or publication. Name component/route tests `picker_workspace_survives_launch_cwd_change`, `picker_home_is_launch_only`, `explicit_non_git_picker_scope_restores`, and `resolved_project_scope_independent_of_routing_cwd`, exercising actual producer/consumer routes rather than field shape alone. Resume/cold-load tests retain valid stored scope and decline missing/ambiguous provenance. + + Resolve in this order: valid separately carried picker/project workspace record; revalidated canonical session checkout evidence; independently resolved error-aware Git checkout from retained selected/session cwd (do not reuse presentation git_meta fallback as proof); then validated legacy published non-Git scope evidence described below. checkoutPath and managed workspace_path may themselves be cwd fallbacks, so independently verify Git evidence rather than trusting names/grouping. Git selection resolves a nested selected cwd to its actual checkout root; explicit non-Git directories remain their selected canonical scope. Missing/conflicting inferred provenance remains unavailable. Recognized account home is unavailable even when picker-confirmed or a dotfiles Git root; never relabel it as deliberate non-Git scope. Unit/route tests can supply a separately resolved project with account-home launch/routing cwd to prove independence. Browser tests use two reachable real UI flows: HOME selection with no snapshot scope, and picker-confirmed project/non-Git selection with explicit scope; they do not claim unsupported same-pane independent home/project selection or use API-only browser seeding. + + Make the server the final filesystem resolver: canonicalize an existing directory, distinguish inaccessible/not-directory from absent, and return the resolved workspace on successful create. Account-home detection must cover the relevant account, not only `CheckpointsApiState.home` (the reproduced failure involves a different UID's home). Use OS account-home information and known managed home paths on Linux; ensure the cross-platform fallback does not disable valid Windows project scopes. Automatically inferred scopes equal to a recognized account home are unavailable even when a dotfiles Git repository happens to exist there. A genuine explicit non-Git non-home scope remains supported. Never search arbitrary subdirectories of a home for a project or silently broaden a selected subtree to another checkout. If root evidence conflicts or is missing, return a typed unavailable reason and keep sending usable. Capture canonical workspace and immutable scope/storage locator with the attempt/checkpoint, using it for list/metadata/restore without rerunning new-root discovery. Legacy shadow repos keep dual addressing by original stored/legacy scope string/storage key and validated original work-tree destination; do not rehash a historical nested/symlink cwd as today's checkout root or scan homes to find it. A subsequent pane cwd or session-directory update must not redirect an older turn's restore. Preserve existing valid shadow repositories by resolving existing entries with their original stored/legacy scope; no bulk migration or deletion is needed. For a pre-change non-Git pane lacking picker provenance, accept only an actual prior valid published checkpoint record already linked to this pane/session by its persisted checkpoint ID/locator or existing checkpoint metadata. Revalidate its original scope/storage locator and work-tree destination to the same canonical non-home directory, preserving the existing association without demanding newly added provenance fields from that historical record. This record is legacy scope evidence for new turns, not permission to infer scope from `initialCwd`, labels or an account home. A missing/malformed/ambiguous record remains unavailable. Add a behavioral old-pane fixture with a real prior complete non-Git checkpoint and verify a new turn captures/restores that same scope; a home-only launch fixture without such a record must still refuse automatic capture. -- [ ] **Step 3: Require complete readable eligible code and publish only successful targets.** Keep the existing Git-ignore semantics as the defined code scope; `.git` administrative state stays outside it. Before publishing, explicitly traverse eligible directories with error-aware filesystem operations and verify readable eligible regular files. Determine ignores with Git's existing rules, not a new hard-coded list. Do not use `Path::exists`, an ignored `read_dir` error, a process UID of zero, or success from `git ls-files` alone as proof of readability. Symlinks keep Git's link semantics and must not cause traversal into unrelated trees. Reject unsupported eligible nested-repository/gitlink capture with an actionable path/cause unless the bounded validation proves an equally simple way to capture its complete code tree; never advertise only the gitlink as a complete snapshot. The existing policy excludes ignored resources intentionally; it does not authorize omitting an unreadable eligible path. +- [ ] **Step 3: Require complete readable eligible code and publish only successful targets.** Keep the existing Git-ignore semantics as the defined code scope; `.git` administrative state stays outside it. Before publishing, explicitly traverse eligible directories with error-aware filesystem operations and verify readable eligible regular files. Determine ignores with Git's existing rules, not a new hard-coded list. Do not use `Path::exists`, an ignored `read_dir` error, a process UID of zero, or success from `git ls-files` alone as proof of readability. Symlinks keep Git's link semantics and must not cause traversal into unrelated trees. Capture eligible nested repositories/submodules as prefixed ordinary working-file blobs/symlinks, never as a gitlink-only tree. After outer eligibility is established, enumerate each nested working tree with its existing tracked-plus-untracked Git rules (`ls-files -z --cached --others --exclude-standard`): retain tracked files even if now ignored, exclude ignored untracked files, and reapply the same rule recursively to eligible nested repositories. Error-aware traversal/open checks remain mandatory independently of successful Git enumeration, including eligible directories whose contents Git enumeration cannot inspect. An outer eligible recorded gitlink must have a readable populated working directory; missing/unreadable nested code fails complete capture with its path/cause, rather than inventing a complete empty tree. An intentionally ignored untracked nested directory remains excluded by the existing scope rules. Exclude every nested `.git` file/directory and preserve original nested HEAD/index/admin state. Use the existing request-local shadow index and normal `update-index --add --remove -z --stdin` on the exact enumerated eligible working-file paths to remove any gitlink and install prefixed ordinary file entries with Git stat caching; feed deleted formerly captured paths for removal and do not run blanket `git add` that recreates gitlinks. Use `hash-object`/`update-index --cacheinfo` only if an existing index transition needs it; no packaging framework or submodule fetch. Restore those captured paths using shadow-tree file checkout, preserving unrelated later files and nested Git identity. Maintain the same readability/stat-cache/deadline/publication rules for nested files. A small owned Stage 3 Git probe at `/tmp/freshell-plan-nested-probe-74l5vfrk` restored six prefixed selected entries (tracked-ignored and untracked eligible included, ignored untracked/admin excluded) with executable/symlink/native HEAD preserved; A follow-up direct `update-index`/refresh/repeat probe produced the identical complete tree with a stat-bearing normal index; this grounds the choice but is not implementation acceptance. Add real `checkpoints::tests::{nested_clone_non_git_scope_complete_restore, submodule_working_edits_complete_restore, nested_tracked_ignore_and_untracked_rules, nested_unreadable_eligible_path_rejects_publication}` plus restricted-UID route smoke for unreadable nested code. Assert full bytes/edits/restore and no gitlink-only target; existing legacy targets keep immutable original addressing and are not retroactively renamed or claimed complete. The existing policy excludes ignored resources intentionally; it does not authorize omitting an unreadable eligible path. Serialize create/metadata/restore operations for one canonical snapshot workspace with a shared route-state keyed mutex; separate workspaces remain independent. Stage with a request-local `GIT_INDEX_FILE`: copy the last successfully published stat-bearing index into it when its recorded published-tree ID matches the current published ref; otherwise reconstruct from that tree (empty for first capture). Never initialize every ordinary repeat by `read-tree`, which discards the stat cache and rehashes all files. Git refresh/add checks changed/racy entries; the explicit full readability pass still opens every eligible regular file for reading and enumerates every eligible directory on each capture, so cache hits never prove access. Install the new disposable index cache only after successful complete publication; cache-install failure invalidates/logs the cache without changing availability of the complete published target. Corrupt/stale/missing cache falls back to tree initialization, failed partial indexes are deleted. This bounds ordinary repeated work to complete traversal/access validation plus Git's changed-file hashing, rather than hashing all unchanged bytes. A behavioral instrumented fixture proves an unchanged second capture avoids whole-tree rehashing, edits are captured/restored, and permission loss after a cached success still fails. Build a tree and commit object without moving the published branch; persist complete checkpoint/request metadata atomically; then move the published ref as the final success step. `commit-tree` plus an explicit publish-last `update-ref` is preferable to moving HEAD and trying to undo after failure. Preserve original Git index/HEAD and existing file permissions. Clean only owned temporary index/metadata files on failure. An orphan unpublished Git object/metadata entry must never appear in list or be accepted by restore. @@ -365,11 +370,11 @@ Managed output observations distinguish Running, active Recovering, Blocked with - [ ] **Step 4: Own capture and provider-send release in the server.** Add optional modern checkpoint intent and stable logical attempt ID to the existing freshAgent.send frame, with frozen provider/session/scope and observed fence. Local echo/pending state starts immediately; the client sends one command, without an independent automatic checkpoint POST or browser timer releasing a second send. Inject a narrow checkpoint-service trait into the WS crate from server main/checkpoint state; intercept before both managed gateway and local provider-send dispatch and run one owned asynchronous barrier-and-forward job so the receive loop stays responsive. Direct REST capture/list/metadata/restore use the same service/storage/locks. - The finite production capture budget is 15 seconds, beginning before workspace-lock acquisition and covering scope/readability traversal, Git, metadata and publication; tests inject a shorter budget through the checkpoint service, with no browser timer or new environment knob. Budget exhaustion produces `capture_timeout` and a clear per-turn reason. Cancellation settlement may outlast that deadline only while owned children are being terminated/reaped; provider dispatch waits for actual settlement. Record stage timings and eligible-file/changed-file counts without names on success; an ordinary unchanged-repeat fixture and a bounded many-file fixture must complete within this default on the supported synthetic lane, or tune the single value from measured evidence before completion while retaining a documented finite value. The service owns all Git children/publication work; timeout requests cancellation and waits for actual owned termination/settlement before releasing a send. Dropping a future or aborting browser HTTP is insufficient. Reconcile timeout racing update-ref under the workspace lock: either complete publication already succeeded while send was held, so retain Available, or it did not, so return Unavailable preserving prior targets. No remaining work may publish an attempt after terminal settlement/provider-send release. Do not promise absolute-clock/ref atomicity. Add a scoped owned Git runner with retained children/stdout/stderr tasks, explicit POSIX process-group termination/reaping/group settlement and supported Windows JobObject ownership before child execution, followed by direct wait and actual-job/process-count settlement. A dropped future, taskkill example or kill_on_drop alone is insufficient. Reconcile ref/complete metadata before terminalizing ambiguous publication. Destructive tests stay in the sandbox; preserve supported Windows capture rather than disabling its timeout behavior. + Automatic pre-turn capture has a finite 15-second budget, beginning before workspace-lock acquisition and covering scope/readability traversal, Git, metadata and publication. Existing explicit REST capture/retry uses a longer finite 120-second whole-attempt budget, including lock wait, to bootstrap a complete first index for ordinary larger workspaces; no indefinite manual job or user preference. Tests inject proportionally shorter independent budgets through the production checkpoint service, with no browser timer or new environment knob. Distinguish `workspace_lock_timeout` before capture work starts from actual `capture_timeout` after the workspace lock is held. Budget exhaustion produces `capture_timeout` for actual capture work or `workspace_lock_timeout` while waiting, with a clear per-turn reason. Cancellation settlement may outlast that deadline only while owned children are being terminated/reaped; provider dispatch waits for actual settlement. Record stage timings and eligible-file/changed-file counts without names on success; an ordinary unchanged-repeat fixture and a bounded many-file fixture must complete within the automatic budget on the supported synthetic lane. A larger complete-first-index fixture exercises the explicit bootstrap budget and subsequent fast automatic capture; retain the documented finite 15/120-second values unless measured evidence supports a bounded adjustment before completion. The service owns all Git children/publication work; timeout requests cancellation and waits for actual owned termination/settlement before releasing a send. Dropping a future or aborting browser HTTP is insufficient. Reconcile timeout racing update-ref under the workspace lock: either complete publication already succeeded while send was held, so retain Available, or it did not, so return Unavailable preserving prior targets. No remaining work may publish an attempt after terminal settlement/provider-send release. Do not promise absolute-clock/ref atomicity. Add a scoped owned Git runner with retained children/stdout/stderr tasks, explicit POSIX process-group termination/reaping/group settlement and supported Windows JobObject ownership before child execution, followed by direct wait and actual-job/process-count settlement. A dropped future, taskkill example or kill_on_drop alone is insufficient. Reconcile ref/complete metadata before terminalizing ambiguous publication. Destructive tests stay in the sandbox; preserve supported Windows capture rather than disabling its timeout behavior. - After an automatic capture actually settles as Unavailable `capture_timeout`, remember only a transient per-canonical-workspace timeout deadline/cause for 60 seconds, starting at settlement. During this cooldown, new automatic pre-turn attempts return immediate durable per-turn Unavailable `capture_timeout_cooldown` with a clear remaining retry delay, then forward their message normally; they do not reacquire a long capture lock, start Git, reuse a failed index or claim a previous turn's target. Only timeout outcomes establish this cooldown; successful complete capture clears it. Existing explicit checkpoint create/retry through the REST capture service bypasses cooldown and attempts a real complete capture, and a changed canonical scope/storage locator has its own attempt immediately. At expiry the next automatic turn retries within the same 15-second budget. A same-scope repair during cooldown does not silently mark any turn Available: an explicit retry or post-expiry capture must actually publish a complete checkpoint first. An explicit retry that times out renews the timeout deadline; a successful retry clears it for future turns, leaving earlier failed turns unavailable. Keep this state only in existing checkpoint route/service state; prune expired entries and bound live entries to 128 (evict oldest expired/oldest timeout entry), with no persistent cache framework, new user preferences or partial snapshot publication. + After an actual capture-work timeout settles as Unavailable `capture_timeout`, remember only a transient per-canonical-workspace timeout deadline/cause for 60 seconds, starting at settlement. During this cooldown, new automatic pre-turn attempts return immediate durable per-turn Unavailable `capture_timeout_cooldown` with a clear remaining retry delay, then forward their message normally; they do not reacquire a long capture lock, start Git, reuse a failed index or claim a previous turn's target. Only actual capture-work timeout outcomes establish this cooldown; a lock-acquisition timeout is `workspace_lock_timeout`, releases only that waiter and never establishes/renews workspace cooldown or cancels the holder; successful complete capture clears it. Existing explicit checkpoint create/retry through the REST capture service bypasses cooldown and attempts a real complete capture within its 120-second budget, and a changed canonical scope/storage locator has its own attempt immediately. At expiry the next automatic turn retries within the same 15-second budget. A same-scope repair during cooldown does not silently mark any turn Available: an explicit retry or post-expiry capture must actually publish a complete checkpoint first. An explicit retry whose actual capture work times out renews the timeout deadline; an explicit lock-wait timeout does not; a successful retry clears it for future turns, leaving earlier failed turns unavailable. Keep this state only in existing checkpoint route/service state; prune expired entries and bound live entries to 128 (evict oldest expired/oldest timeout entry), with no persistent cache framework, new user preferences or partial snapshot publication. - Add deterministic production-service tests named `checkpoints::tests::{capture_timeout_establishes_cooldown, cooldown_sends_immediately_without_capture, cooldown_scope_change_attempts_capture, explicit_capture_bypasses_cooldown, repaired_scope_needs_published_success, cooldown_expiry_retries}` using an injected monotonic clock/budget and controllable owned capture runner. Assert actual runner invocation/settlement, one dispatch for each send, immediate later unavailable responses without a second full wait, continued earlier target validity, no failed target publication, changed-scope bypass, successful explicit retry and post-expiry retry. Include same-scope repair while cooldown remains active so it cannot fabricate availability. Run the `checkpoints` server selector with all six executed/passed cases and exercise send/row behavior through the managed/local barrier tests; cancellation/process-kill variants stay in the disposable sandbox. The browser verifies accepted later sends with the timeout reason and accurate disabled rewind where that synthetic scenario is selected; never wait a full repeated timeout merely to make sends usable. + Add deterministic production-service tests named `checkpoints::tests::{capture_timeout_establishes_cooldown, cooldown_sends_immediately_without_capture, cooldown_scope_change_attempts_capture, explicit_capture_bypasses_cooldown, repaired_scope_needs_published_success, cooldown_expiry_retries, explicit_longer_budget_bootstraps_complete_index, lock_wait_timeout_does_not_establish_cooldown, explicit_lock_wait_timeout_does_not_renew_cooldown, healthy_holder_and_next_pane_capture_after_waiter_timeout}` using an injected monotonic clock/budget and controllable owned capture runner. Assert actual runner invocation/settlement, one dispatch for each send, immediate later unavailable responses without a second full wait, continued earlier target validity, no failed target publication, changed-scope bypass, successful explicit retry and post-expiry retry. Include same-scope repair while cooldown remains active so it cannot fabricate availability. For the bootstrap case, exercise the production checkpoint service with its actual 15-second automatic and 120-second explicit budgets in a controlled synthetic integration fixture: an owned cancellable 20-second first-index delay during lock-held capture makes automatic capture time out without publication; explicit capture with that same delay then performs complete real Git capture and installs its index; a subsequent unchanged automatic capture without bootstrap delay succeeds within 15 seconds. Verify actual publication/file restore and no partial-index reuse, not just configuration values or mocked Available results. Other deterministic cases may inject shorter proportional budgets and a monotonic clock. The two-pane cases hold a healthy capture lock while a waiter reaches its own deadline, then let the holder publish and a subsequent pane capture succeed; assert the timed-out waiter never suppresses healthy work or renews an existing cooldown. Run the `checkpoints` server selector with all ten executed/passed cases and exercise send/row behavior through the managed/local barrier tests; cancellation/process-kill variants stay in the disposable sandbox. The browser verifies accepted later sends with the timeout reason and accurate disabled rewind where that synthetic scenario is selected; never wait a full repeated timeout merely to make sends usable. Preserve the immutable send and validate captured conversation/session ownership and observed epoch/generation again after capture for every managed/local provider. Existing managed proxy drops observed fences and Claude does not enforce them, so implement the narrow post-delay admission rather than assuming it exists. Never reread latest pane/session/fence to retarget the request. A rejected stale/closed conversation is a send rejection distinct from snapshot failure. Within the process duplicate transport requests join the same barrier-and-forward job; known-rejected retries in the same canonical conversation may alias their new transport ID to the same logical capture result, never recapture that attempt. Ambiguous acceptance follows existing no-redispatch rules. A different conversation gets a separate logical attempt. Queue draining uses the same service immediately before forwarding only when capture has not settled. @@ -379,16 +384,16 @@ Managed output observations distinguish Running, active Recovering, Blocked with - [ ] **Step 5: Structured failures and safe client details.** Replace create-path error strings with a small typed checkpoint failure carrying stage/operation, stable code, resolved workspace, optional offending path, underlying filesystem kind/errno or Git exit status and bounded cause. Emit one warning for expected scope/readability unavailability and one error for unexpected Git/storage failure, with an event such as `code_checkpoint.failed`; include the send correlation as `send_request_id`, and preserve the HTTP span's `request_id`. When known, include filesystem path directly from traversal; Git failures with no reliably extracted path report an explicit operation and bounded stderr cause. Do not parse stderr as authority about complete storage. Do not log labels, commit argv, request body, outgoing text, raw configuration, or file contents. Exercise failure logs through the real HTTP request/logger, assert correlation and meaningful cause, and assert synthetic outgoing-message/credential markers are absent. Keep the client's human reason separate from internal failure context; response JSON includes code/details needed by per-turn availability while retaining normal API error compatibility. -- [ ] **Step 6: Complete route smoke with actual restricted identity.** Add the closed flag `--checkpoint-fixture` to `scripts/sandbox-test.sh`, permitted only with `--runtime-suite` and never `--corpus`. It sets only `FRESHELL_SANDBOX_CHECKPOINT_FIXTURE=1` for the container. In `docker/sandbox/entrypoint.sh`, before its final `exec gosu sandbox "$@"`, handle that exact flag by fixed initialization of `/tmp/freshell-checkpoint-fixture` on the existing synthetic tmpfs (no arbitrary path or command argument): create only run-owned synthetic tmpfs fixture roots with server UID B-owned writable project/storage/home parents and root/UID A-owned 0600 file/0700 directory, then permanently drop to B (the existing sandbox UID) with no DAC capability. Set the nonsecret fixture-root locator for the unprivileged tests; root initialization accepts no test-supplied ownership script or command. The separate browser target calls this same fixed initializer before its restricted server launch; fixture manipulation/observations run only as B. There is no privileged mutation endpoint. Existing --runtime-suite is networkless/single-UID and cannot arrange this unchanged. B can rename those prepared private entries through its own writable parent to vary eligibility without chmod/read or privileged control service. Launch the real compiled checkpoint server under B; loopback HTTP smoke needs no external network. Create only synthetic accessible project/code paths plus one A-owned 0600 file and one non-traversable A-owned directory. Verify B cannot read them by an actual open/read operation before issuing the route; test fails if B is root or has DAC override capability. Point server storage/home at a B-writable synthetic directory. Do not chmod the private source to make capture pass. +- [ ] **Step 6: Complete route smoke with actual restricted identity.** Add the closed flag `--checkpoint-fixture` to `scripts/sandbox-test.sh`, permitted only with `--runtime-suite` and never `--corpus`. It sets only `FRESHELL_SANDBOX_CHECKPOINT_FIXTURE=1` for the container. In `docker/sandbox/entrypoint.sh`, before its final `exec gosu sandbox "$@"`, handle that exact flag by fixed initialization of `/tmp/freshell-checkpoint-fixture` on the existing synthetic tmpfs (no arbitrary path or command argument): create only run-owned synthetic tmpfs fixture roots with server UID B-owned writable project/storage/home parents and root/UID A-owned 0600 file/0700 directory, then permanently drop to B (the existing sandbox UID) with no DAC capability. Set the nonsecret fixture-root locator for the unprivileged tests; root initialization accepts no test-supplied ownership script or command. The separate browser target calls this same fixed initializer before its restricted server launch; fixture manipulation/observations run only as B. There is no privileged mutation endpoint. Existing --runtime-suite is networkless/single-UID and cannot arrange this unchanged. B renames each prepared A-owned private file/0700 directory only between eligible and explicitly ignored names within the same B-owned writable nonsticky parent. Fixed parent `.gitignore` patterns such as `private-file.ignored` and `private-dir.ignored/` are installed during setup; tests do not change ignore patterns to hide a failure. Never move the A-owned directory across parents (which requires directory write permission), alter its UID/mode, chmod/read it or use a privileged control service. Prove fixture setup by actual B open/traverse denial, same-parent rename out/in and unchanged UID/mode before invoking checkpoint routes; each rename toggles only intentional eligibility. Launch the real compiled checkpoint server under B; loopback HTTP smoke needs no external network. Create only synthetic accessible project/code paths plus one A-owned 0600 file and one non-traversable A-owned directory. Verify B cannot read them by an actual open/read operation before issuing the route; test fails if B is root or has DAC override capability. Point server storage/home at a B-writable synthetic directory. Do not chmod the private source to make capture pass. - Exercise HTTP create/list/metadata/restore: first complete non-Git/Git snapshots restore all eligible fixture files, executable bits and symlink semantics; after adding the inaccessible eligible file/directory a create fails with workspace/path/cause and list retains only prior successes; a guessed/unpublished failed commit cannot restore; repeated reads do not erase or duplicate result state; fixing the synthetic fixture deliberately and issuing a new request succeeds. Include an injected real storage-write failure after Git object creation to prove publication-last, not merely staging failure. Verify original repository state and source permissions remain unchanged. Include explicit project while account-home launch is supplied and no-scope home refusal without reading home secrets. Capture JSONL events and assert send/HTTP correlations, stage, cause and no message/secret marker. Kill/restart portions, if any, execute only inside `scripts/sandbox-test.sh` and affect recorded owned PIDs. + Exercise HTTP create/list/metadata/restore: first complete non-Git/Git snapshots restore all eligible fixture files, executable bits and symlink semantics; after adding the inaccessible eligible file/directory a create fails with workspace/path/cause and list retains only prior successes; a guessed/unpublished failed commit cannot restore; repeated reads do not erase or duplicate result state; fixing the synthetic fixture deliberately and issuing a new request succeeds. Include an injected real storage-write failure after Git object creation to prove publication-last, not merely staging failure. Verify original repository state and source permissions remain unchanged. Route smoke includes independently carried explicit project scope while account-home launch/routing cwd is supplied, and no-scope home refusal without reading home secrets; browser scope proof uses the reachable picker flows below. Capture JSONL events and assert send/HTTP correlations, stage, cause and no message/secret marker. Kill/restart portions, if any, execute only inside `scripts/sandbox-test.sh` and affect recorded owned PIDs. - [ ] **Step 7: Actually running configured-cloud browser proof.** Add a focused spec outside `CLOUD_SKIP_SPECS` rather than counting the existing skipped rewind test. Reuse Task 6's owned external-target/tunnel handoff with a separate checkpoint target container using the same fixed trusted initialization and restricted UID B. It uses an ordinary dedicated bridge and only an ephemeral host-loopback published port, read-only artifacts/root, synthetic writable tmpfs, no host network/user HOME/corpus/Docker socket. The networkless destructive runtime-suite container is not browser reachable. The target driver owns exact container/tunnel names, observations and cleanup; private fixture files remain inside that target, not Cloud Run. Seed fixture paths and collect disk/log observations through the existing test-owned target harness/control channel, not a production testing endpoint. The permission-failure server must have the restricted identity from Step 6. No provider network/model dependency is required: select the direct Codex app-server fixture because actual Codex acceptance supplies a native turn ID. Reuse the shared direct fake-app-server.mjs behavior.recordTurns mode, whose turn/start and thread/read already persist matching distinct native turns/user rows; extend only missing fixture behavior and exercise the production Codex adapter/projector's native-to-display identity. Do not inject display IDs into UI or reuse assistant-only/static-ID history. Validate the chosen transport through actual fresh-agent send/accepted/snapshot behavior before relying on it. - Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately create a pane using the actual DirectoryPicker confirmation of a synthetic non-Git directory (no `.git`), send an accepted user turn, verify its resolved scope/exact row has enabled rewind, mutate at least two eligible files, restore that row through UI confirmation and verify both original contents and the unchanged non-Git scope. This tests picker provenance through client send, server resolution, publication and restore; API-only seeding is insufficient. Also cover home launch with explicit project scope and home launch without scope; the former snapshots only project code, the latter sends successfully with a clear unavailable reason and no home checkpoint. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. + Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately create a pane using the actual DirectoryPicker confirmation of a synthetic non-Git directory (no `.git`), send an accepted user turn, verify its resolved scope/exact row has enabled rewind, mutate at least two eligible files, restore that row through UI confirmation and verify both original contents and the unchanged non-Git scope. This tests picker provenance through client send, server resolution, publication and restore; API-only seeding is insufficient. Separately create a HOME-launched pane through the actual DirectoryPicker: account-home confirmation sets launch cwd only, sends succeed with a clear unavailable reason and no home checkpoint. The project and non-Git panes are created by picker confirmation of their own directories, carry the separate workspace record and snapshot/restore only that selected code. Assert actual persisted pane intent and resolved server result; do not claim a same-pane HOME launch plus independent project selection that the current UI cannot express. Component/route/service tests above cover independence from a later different launch/routing cwd. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. - [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the periodic task-boundary gate and final exact-HEAD broad gate after all Tasks 1–7 join. Validate no foreign files or permission changes entered the diff. Record Task 7 start/end commits and commit focused, compiling logical checkpoints with their behavioral tests, for example scope/complete publication, settled send barrier and native identity, then per-turn UI/restricted fixtures/browser acceptance. Include all required interface/callsite wiring in each checkpoint; do not commit broken intermediates or force this whole task into one commit. The fresh task specification-plus-quality review covers the entire start..end range and descriptive test evidence for every checkpoint. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. - Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id` (at least two fake-only tests); `pnpm exec tsx scripts/testing/opencode-checkpoint-contract.ts` (exactly one real pinned persistence test, using the required-features command above); `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/server/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx --config config/vitest/vitest.config.ts`. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, both home cases and actual picker-confirmed non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. + Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id` (at least two fake-only tests); `pnpm exec tsx scripts/testing/opencode-checkpoint-contract.ts` (exactly one real pinned persistence test, using the required-features command above); `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/server/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx test/unit/client/components/panes/DirectoryPicker.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/store/panesSlice.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/pane-reconcile.fresh-agent.test.ts test/unit/client/lib/tab-registry-open.test.ts --config config/vitest/vitest.config.ts`. Require positive named scope/producer cases and actual server `files::tests` validate-dir eligibility behavior in addition to the `checkpoints` gate. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, picker-confirmed HOME with no scope and separate project/non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. ## Acceptance and completion boundary @@ -403,3 +408,5 @@ Managed output observations distinguish Running, active Recovering, Blocked with The exact sanitized User Request occurs once and includes all 15 current kata criteria. Seven tasks retain behavioral red/green/refactor, focused commits/reviews, actual managed environment smoke, genuinely configured-cloud browser proof, conservative recovery/store decisions, durable retained diagnostics and code-rewind scope/completeness/turn outcomes. Source/ownership/protocol corrections are reflected in task inputs and acceptance; empirical claims are not certified from planned code. Environment copy, secrets and callback metadata have explicit separate profiles, with one policy across both stages and no lower hidden ownership limit. Code-rewind uses server-owned send/capture settlement, real provider identity and fixed restricted fixtures rather than browser timing or fake display IDs. Original workspace/native data and conversation semantics remain protected. The inline provider/projection/ownership/consumer/identity decisions are self-contained; supporting reports have absolute locators. Task 7 and final acceptance explicitly compile/run managed-runtime-v1 and named SDK/OpenCode identity regressions. Configured-cloud browser coverage includes picker-confirmed non-Git capture/restore with paired external-target discovery, and ordinary broad discovery does not count those separate receipts. Restricted initialization is a closed pre-gosu flag with no privileged mutation endpoint. Periodic broad gates run at passed task boundaries after at least 10 non-plan commits; the final whole-suite runs on committed final HEAD; PR/live rollout/MCP boundaries remain separately approved. Ordinary provider matrix requirements cannot be replaced by arbitrary-program certification or an unsupported-error workaround for a required reference. Reports remain outside tracked source; plan/code snippets are drafts, not implementation evidence. Round 2 remediation defines a fake-only default OpenCode target and a separately provisioned required-features pinned persistence lane, actual positive recovery selectors, the Amplifier creator-to-host ledger and real ownership smoke, Overview/Background/API-directory consumers, and a finite workspace timeout cooldown with accurate unavailable turns. Task 2 and Task 7 commit at compiling logical checkpoints and their task reviews cover complete start..end ranges. Behavioral RED precedes the relevant changes; already-green browser baselines stay honest, and full configured-cloud managed acceptance remains mandatory. Root dispositions for round 2 findings 6/7/10 remain OPTIONAL and finding 8 remains UNSUBSTANTIATED; this repair adds no speculative fixture-isolation/redaction work, removes no validated native-auth/ephemeral-projection mechanism, and does not move the AGENTS link. + +Round 3 final remediation preserves shell capability-specific terminal-end certification; supplies picker-owned separate snapshot scope and reachable HOME/project/non-Git browser cases; allows focused Task 6 commits; denies every ordinary host PTY path in the owned tunneled fixture and pins/checks cloudflared. Automatic and explicit capture have separate 15/120-second budgets, capture-only cooldown excludes lock wait, eligible nested code is stored as complete prefixed files, protocol owns spool descriptors/codec, and restricted private entries change eligibility only by same-parent rename with unchanged UID/mode. Named behavioral tests and positive-count gates cover these choices. Finding 5a remains OPTIONAL (safe settlement retained; no abandoned-worker subsystem), finding 7 remains UNSUBSTANTIATED (validated native-auth/ephemeral alias mechanisms retained), and earlier scope dispositions remain unchanged. No fourth review or implementation claim is made by this plan repair. From c4c55c1a2d129da5bffff56e9b1667586b142520 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 5 Oct 2026 23:41:59 -0700 Subject: [PATCH 007/493] fix: preserve bounded managed provider resources and executable modes --- crates/freshell-platform/src/lib.rs | 1 + .../src/provider_environment.rs | 1351 +++++++++++++++++ .../freshell-server/src/fresh_agent_proxy.rs | 5 +- .../src/managed_mcp_capability.rs | 475 ++++-- .../src/managed_provider_bootstrap.rs | 314 +++- crates/freshell-server/src/managed_runtime.rs | 2 +- crates/freshell-session-host/src/main.rs | 1126 ++++++++------ .../src/providers/mod.rs | 6 +- .../freshell-sessions/src/amplifier_stub.rs | 241 ++- 9 files changed, 2840 insertions(+), 681 deletions(-) create mode 100644 crates/freshell-platform/src/provider_environment.rs diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 8691bc2fc..f020f4192 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -61,6 +61,7 @@ pub mod mcp_inject; pub mod opencode_config; pub mod opencode_plugin; pub mod path; +pub mod provider_environment; pub mod resume_gate; pub mod spawn; diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs new file mode 100644 index 000000000..42a1a3171 --- /dev/null +++ b/crates/freshell-platform/src/provider_environment.rs @@ -0,0 +1,1351 @@ +//! Bounded resource projection shared by controller staging and provider HOME copying. + +use std::collections::{BTreeMap, BTreeSet, VecDeque}; +use std::fs::{self, File, OpenOptions}; +use std::io::{Read, Write}; +use std::path::{Component, Path, PathBuf}; +use std::sync::atomic::{AtomicU64, Ordering}; + +#[derive(Clone, Copy, Debug)] +pub struct CopyLimits { + pub max_entries: usize, + pub max_total_bytes: u64, + pub max_file_bytes: u64, + pub max_depth: usize, +} +impl Default for CopyLimits { + fn default() -> Self { + Self { + max_entries: 100_000, + max_total_bytes: 512 * 1024 * 1024, + max_file_bytes: 128 * 1024 * 1024, + max_depth: 64, + } + } +} +#[derive(Debug, Clone)] +pub struct CopySelection { + pub source: PathBuf, + pub destination_relative: PathBuf, +} +#[derive(Debug)] +pub struct CopyReport { + pub entries: usize, + pub bytes: u64, + pub projected_paths: Vec, +} +#[derive(Debug)] +pub struct CopyError { + pub relative_path: PathBuf, + pub operation: &'static str, + pub limit_kind: Option<&'static str>, + pub observed: Option, + pub permitted: Option, + pub corrective_message: String, +} +impl std::fmt::Display for CopyError { + fn fmt(&self, out: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + out, + "{} {}: {}", + self.operation, + self.relative_path.display(), + self.corrective_message + )?; + if let (Some(kind), Some(observed), Some(permitted)) = + (self.limit_kind, self.observed, self.permitted) + { + write!(out, " ({kind}: observed {observed}, permitted {permitted})")?; + } + Ok(()) + } +} +impl std::error::Error for CopyError {} +fn error(path: &Path, operation: &'static str, message: impl Into) -> CopyError { + CopyError { + relative_path: if valid_relative(path) || path.as_os_str().is_empty() { + path.to_owned() + } else { + "invalid-selection".into() + }, + operation, + limit_kind: None, + observed: None, + permitted: None, + corrective_message: message.into(), + } +} +fn io_error(path: &Path, operation: &'static str, cause: std::io::Error) -> CopyError { + // IO error kinds/codes are useful; never include source bytes or arbitrary IO text. + error( + path, + operation, + format!( + "selected resource is unavailable ({:?}, OS {:?}); repair the named resource and retry", + cause.kind(), + cause.raw_os_error() + ), + ) +} +fn bound(path: &Path, kind: &'static str, observed: u64, permitted: u64) -> Result<(), CopyError> { + if observed <= permitted { + return Ok(()); + } + Err(CopyError { + relative_path: if valid_relative(path) || path.as_os_str().is_empty() { + path.to_owned() + } else { + "invalid-selection".into() + }, + operation: "plan", + limit_kind: Some(kind), + observed: Some(observed), + permitted: Some(permitted), + corrective_message: + "reduce unrelated data in the selected resource or repair its topology and retry".into(), + }) +} +pub fn private_file_mode(source_mode: u32) -> u32 { + 0o600 | (source_mode & 0o111) +} +pub fn provider_read_stage_mode(source_mode: u32) -> u32 { + 0o640 | (source_mode & 0o111) +} +fn valid_relative(path: &Path) -> bool { + !path.as_os_str().is_empty() && path.components().all(|part| matches!(part,Component::Normal(name) if name.len()<=255 && !name.to_string_lossy().chars().any(char::is_control))) +} +#[derive(Clone, Debug, PartialEq, Eq)] +enum Node { + Directory(PathBuf), + File(PathBuf, u32, u64), + Link(PathBuf), +} +struct Plan { + nodes: BTreeMap, + mappings: BTreeMap, + pending: VecDeque<(PathBuf, PathBuf, usize)>, + exclusions: Vec, + counted_files: BTreeSet, + bytes: u64, + metadata_bytes: u64, + limits: CopyLimits, + account_home: Option, +} +const MAX_METADATA_BYTES: u64 = 64 * 1024 * 1024; +impl Plan { + fn excluded(&self, source: &Path) -> bool { + self.exclusions + .iter() + .any(|path| source == path || source.starts_with(path)) + } + fn resolve_destination(&self, path: &Path, include_leaf: bool) -> Result { + let mut current = path.to_owned(); + // Every redirect uses an already planned link; more redirects than + // entries proves a cyclic logical destination rather than a package cycle. + for _ in 0..=self.nodes.len() { + let link = current + .ancestors() + .skip(usize::from(!include_leaf)) + .find_map(|ancestor| match self.nodes.get(ancestor) { + Some(Node::Link(target)) => Some((ancestor.to_owned(), target.clone())), + _ => None, + }); + let Some((ancestor, target)) = link else { + return Ok(current); + }; + current = target.join(current.strip_prefix(&ancestor).unwrap()); + } + Err(error( + path, + "plan", + "cyclic logical resource destinations; repair the selected aliases", + )) + } + fn mapped(&self, source: &Path) -> Result, CopyError> { + for root in source.ancestors() { + if let Some(destination) = self.mappings.get(root) { + return self + .resolve_destination( + &destination.join(source.strip_prefix(root).unwrap()), + true, + ) + .map(Some); + } + } + Ok(None) + } + fn claim(&mut self, path: PathBuf, node: Node) -> Result<(), CopyError> { + let path = self.resolve_destination(&path, false)?; + let node = if let Node::Link(target) = node { + let target = self.resolve_destination(&target, true)?; + // A parent alias already exposes the physical child. Redundant + // selections add no inode or duplicate byte charge. + if target == path { + return Ok(()); + } + Node::Link(target) + } else { + node + }; + if let Some(existing) = self.nodes.get(&path) { + if existing == &node + || matches!((existing,&node),(Node::Directory(a),Node::Directory(b)) if a.as_os_str().is_empty() || b.as_os_str().is_empty()) + { + return Ok(()); + } + return Err(error( + &path, + "plan", + "incompatible selected destinations; select one coherent resource for this path", + )); + } + if let Some(parent) = path + .parent() + .filter(|parent| !parent.as_os_str().is_empty()) + { + self.claim(parent.to_owned(), Node::Directory(PathBuf::new()))?; + } + let entries = (self.nodes.len() as u64) + .checked_add(1) + .ok_or_else(|| error(&path, "plan", "entry accounting overflow"))?; + bound(&path, "entries", entries, self.limits.max_entries as u64)?; + let source_bytes = match &node { + Node::Directory(source) | Node::File(source, ..) | Node::Link(source) => { + source.as_os_str().len() as u64 + } + }; + self.metadata_bytes = self + .metadata_bytes + .checked_add(path.as_os_str().len() as u64) + .and_then(|bytes| bytes.checked_add(source_bytes)) + .ok_or_else(|| error(&path, "plan", "resource metadata accounting overflow"))?; + bound( + &path, + "metadata_bytes", + self.metadata_bytes, + MAX_METADATA_BYTES, + )?; + self.nodes.insert(path, node); + Ok(()) + } + fn add_mapping(&mut self, source: PathBuf, destination: PathBuf) -> Result<(), CopyError> { + self.charge_metadata( + &destination, + source.as_os_str().len() as u64 + destination.as_os_str().len() as u64, + )?; + self.mappings.insert(source, destination); + Ok(()) + } + fn charge_metadata(&mut self, path: &Path, bytes: u64) -> Result<(), CopyError> { + let observed = self + .metadata_bytes + .checked_add(bytes) + .ok_or_else(|| error(path, "plan", "resource metadata accounting overflow"))?; + bound(path, "metadata_bytes", observed, MAX_METADATA_BYTES)?; + self.metadata_bytes = observed; + Ok(()) + } + fn enqueue(&mut self, source: PathBuf, path: PathBuf, depth: usize) -> Result<(), CopyError> { + // Reserve before queue allocation. Excluded aliases never consume the + // selected-resource entry budget or cause their bytes to be read. + let canonical = + fs::canonicalize(&source).map_err(|cause| io_error(&path, "resolve", cause))?; + if self.excluded(&canonical) { + return Ok(()); + } + bound( + &path, + "entries", + self.nodes + .len() + .checked_add(self.pending.len()) + .and_then(|count| count.checked_add(1)) + .ok_or_else(|| error(&path, "plan", "entry accounting overflow"))? + as u64, + self.limits.max_entries as u64, + )?; + self.charge_metadata( + &path, + source.as_os_str().len() as u64 + path.as_os_str().len() as u64, + )?; + self.pending.push_back((source, path, depth)); + Ok(()) + } + fn directory_mapping( + &mut self, + source: &Path, + destination: &Path, + depth: usize, + ) -> Result<(), CopyError> { + if let Some(mapped) = self.mapped(source)? { + if mapped != destination { + self.claim(destination.to_owned(), Node::Link(mapped))?; + } + return Ok(()); + } + self.add_mapping(source.to_owned(), destination.to_owned())?; + self.enqueue(source.to_owned(), destination.to_owned(), depth)?; + Ok(()) + } + fn external_resource( + &mut self, + target: &Path, + path: &Path, + depth: usize, + ) -> Result { + if let Some(mapped) = self.mapped(target)? { + return Ok(mapped); + } + let metadata = + fs::metadata(target).map_err(|cause| io_error(path, "inspect_link_target", cause))?; + let mut resource = target.to_owned(); + if metadata.is_file() { + let parent = target.parent().ok_or_else(|| { + error(path, "plan", "linked resource has no containing directory") + })?; + if matches!( + target.extension().and_then(|ext| ext.to_str()), + Some("js" | "mjs" | "cjs" | "ts" | "tsx") + ) { + resource = parent.to_owned(); + } + for ancestor in parent.ancestors() { + if self.account_home.as_deref() == Some(ancestor) { + break; + } + if ancestor.join("package.json").is_file() || ancestor.join("SKILL.md").is_file() { + resource = ancestor.to_owned(); + break; + } + // Do not select arbitrary account HOME ancestors as resource units. + if ancestor + .file_name() + .is_some_and(|name| name == "node_modules") + { + break; + } + } + } + if resource == Path::new("/") || self.account_home.as_deref() == Some(resource.as_path()) { + return Err(error(path,"select_resource_scope","an external resource cannot infer the entire account home or filesystem root; select a complete resource/package directory with its own scope")); + } + let destination = Path::new(".freshell/resources/tree") + .join(resource.strip_prefix(Path::new("/")).unwrap_or(&resource)); + if metadata.is_file() && resource == target { + self.add_mapping(target.to_owned(), destination.clone())?; + self.enqueue(target.to_owned(), destination, depth)?; + } else { + self.directory_mapping(&resource, &destination, depth)?; + } + // Preserve installed lookup order by copying bounded node_modules units, + // and package-scope metadata, with just the connecting ancestor skeleton. + for ancestor in resource.parent().into_iter().flat_map(Path::ancestors) { + if ancestor.as_os_str().is_empty() || ancestor == Path::new("/") { + break; + } + let modules = ancestor.join("node_modules"); + if modules.is_dir() { + let canonical = fs::canonicalize(&modules) + .map_err(|cause| io_error(path, "resolve_lookup_context", cause))?; + let destination = Path::new(".freshell/resources/tree") + .join(modules.strip_prefix("/").unwrap_or(&modules)); + self.directory_mapping(&canonical, &destination, depth)?; + } + let package = ancestor.join("package.json"); + if package.is_file() { + let canonical = fs::canonicalize(&package) + .map_err(|cause| io_error(path, "resolve_package_scope", cause))?; + let destination = Path::new(".freshell/resources/tree") + .join(package.strip_prefix("/").unwrap_or(&package)); + if let Some(mapped) = self.mapped(&canonical)? { + if mapped != destination { + self.claim(destination, Node::Link(mapped))?; + } + } else { + self.add_mapping(canonical.clone(), destination.clone())?; + self.enqueue(canonical, destination, depth)?; + } + } + if self.account_home.as_deref() == Some(ancestor) { + break; + } + } + self.mapped(target)?.ok_or_else(|| { + error( + path, + "plan", + "linked resource has no coherent physical mapping", + ) + }) + } + fn select_linked_ancestors( + &mut self, + selection: &CopySelection, + selections: &[&CopySelection], + canonical: &Path, + ) -> Result<(), CopyError> { + for parent in selections { + let Ok(tail) = selection + .destination_relative + .strip_prefix(&parent.destination_relative) + else { + continue; + }; + if tail.as_os_str().is_empty() { + continue; + } + if !fs::metadata(&parent.source).is_ok_and(|metadata| metadata.is_dir()) { + continue; + } + let mut source = parent.source.clone(); + let mut destination = parent.destination_relative.clone(); + let parts = std::iter::once(None).chain(tail.components().map(Some)); + for part in parts { + if let Some(part) = part { + source.push(part.as_os_str()); + destination.push(part.as_os_str()); + } + let Ok(metadata) = fs::symlink_metadata(&source) else { + break; + }; + if !metadata.file_type().is_symlink() { + continue; + } + let target = fs::canonicalize(&source) + .map_err(|cause| io_error(&destination, "resolve_selected_alias", cause))?; + if destination == selection.destination_relative && target != canonical { + continue; + } + if !fs::metadata(&target).is_ok_and(|metadata| metadata.is_dir()) { + continue; + } + if self.excluded(&target) { + return Err(error( + &destination, + "select", + "selected directory alias is excluded native state or a credential source", + )); + } + let mapped = self.external_resource(&target, &destination, 0)?; + self.claim(destination.clone(), Node::Link(mapped))?; + } + } + Ok(()) + } + fn walk(&mut self, source: PathBuf, path: PathBuf, depth: usize) -> Result<(), CopyError> { + let path = self.resolve_destination(&path, false)?; + bound(&path, "depth", depth as u64, self.limits.max_depth as u64)?; + let metadata = + fs::symlink_metadata(&source).map_err(|cause| io_error(&path, "inspect", cause))?; + let canonical = + fs::canonicalize(&source).map_err(|cause| io_error(&path, "resolve", cause))?; + if self.excluded(&canonical) { + return Ok(()); + } + if !metadata.file_type().is_symlink() { + if let Some(mapped) = self.mapped(&canonical)? { + if mapped != path { + return self.claim(path, Node::Link(mapped)); + } + } + } + if metadata.file_type().is_symlink() { + let target = self.external_resource(&canonical, &path, depth)?; + return self.claim(path, Node::Link(target)); + } + if metadata.is_dir() { + self.claim(path.clone(), Node::Directory(canonical))?; + let pending_start = self.pending.len(); + for entry in + fs::read_dir(&source).map_err(|cause| io_error(&path, "read_directory", cause))? + { + let entry = entry.map_err(|cause| io_error(&path, "read_directory", cause))?; + let child = path.join(entry.file_name()); + if !valid_relative(&child) { + return Err(error( + &child, + "plan", + "resource name is not a bounded normal relative path", + )); + } + self.enqueue( + entry.path(), + child, + depth + .checked_add(1) + .ok_or_else(|| error(&path, "plan", "depth accounting overflow"))?, + )?; + } + self.pending.make_contiguous()[pending_start..].sort_by(|a, b| a.1.cmp(&b.1)); + } else if metadata.is_file() { + bound( + &path, + "file_bytes", + metadata.len(), + self.limits.max_file_bytes, + )?; + if self.counted_files.insert(canonical.clone()) { + self.bytes = self + .bytes + .checked_add(metadata.len()) + .ok_or_else(|| error(&path, "plan", "regular-file byte accounting overflow"))?; + bound( + &path, + "total_bytes", + self.bytes, + self.limits.max_total_bytes, + )?; + } + #[cfg(unix)] + let mode = { + use std::os::unix::fs::PermissionsExt; + metadata.permissions().mode() + }; + #[cfg(not(unix))] + let mode = 0; + self.claim(path, Node::File(canonical, mode, metadata.len()))?; + } else { + return Err(error(&path,"inspect","selected resource must be a regular file, directory or resolvable symbolic link; replace the unsupported node")); + } + Ok(()) + } +} +fn relative_link(from: &Path, to: &Path) -> PathBuf { + let from = from.components().collect::>(); + let to = to.components().collect::>(); + let common = from.iter().zip(&to).take_while(|(a, b)| a == b).count(); + let mut result = PathBuf::new(); + for _ in common..from.len() { + result.push(".."); + } + for part in &to[common..] { + result.push(part.as_os_str()); + } + if result.as_os_str().is_empty() { + result.push("."); + } + result +} +fn mode(path: &Path, permissions: u32, relative: &Path) -> Result<(), CopyError> { + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions(path, fs::Permissions::from_mode(permissions)) + .map_err(|cause| io_error(relative, "set_private_mode", cause))?; + } + #[cfg(not(unix))] + let _ = (path, permissions, relative); + Ok(()) +} +struct TemporaryProjection(PathBuf); +impl Drop for TemporaryProjection { + fn drop(&mut self) { + let _ = fs::remove_dir_all(&self.0); + } +} +static NEXT_PROJECTION: AtomicU64 = AtomicU64::new(0); + +/// Plans the complete physical graph first; rejected selections cannot publish +/// partial environments. Only selected resources and their dependency closure +/// are read. Originals, credentials and provider-owned siblings remain intact. +pub fn copy_environment( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + limits: CopyLimits, +) -> Result { + let account_home = std::env::var_os("HOME").and_then(|home| fs::canonicalize(home).ok()); + copy_environment_with_account_home( + selections, + destination_root, + excluded_canonical_sources, + limits, + account_home, + ) +} +fn copy_environment_with_account_home( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + limits: CopyLimits, + account_home: Option, +) -> Result { + let mut plan = Plan { + nodes: BTreeMap::new(), + mappings: BTreeMap::new(), + pending: VecDeque::new(), + exclusions: excluded_canonical_sources.to_vec(), + counted_files: BTreeSet::new(), + bytes: 0, + metadata_bytes: 0, + limits, + account_home, + }; + let mut selections = selections.iter().collect::>(); + selections.sort_by_key(|selection| { + ( + selection.source.components().count(), + selection.source.clone(), + selection.destination_relative.clone(), + ) + }); + for selection in &selections { + if !valid_relative(&selection.destination_relative) { + return Err(error( + &selection.destination_relative, + "select", + "destination must contain bounded normal relative components without traversal", + )); + } + } + for selection in &selections { + let path = &selection.destination_relative; + if !valid_relative(path) { + return Err(error( + path, + "select", + "destination must contain bounded normal relative components without traversal", + )); + } + let metadata = fs::symlink_metadata(&selection.source) + .map_err(|cause| io_error(path, "select", cause))?; + let canonical = fs::canonicalize(&selection.source) + .map_err(|cause| io_error(path, "resolve_selection", cause))?; + if plan.excluded(&canonical) { + return Err(error(path,"select","selected resource is excluded native state or a credential source; keep native state in its store and use managed references for credentials")); + } + plan.select_linked_ancestors(selection, &selections, &canonical)?; + if metadata.file_type().is_symlink() { + plan.enqueue(selection.source.clone(), path.clone(), 0)?; + } else if let Some(mapped) = plan.mapped(&canonical)? { + if mapped != *path { + plan.claim(path.clone(), Node::Link(mapped))?; + } + } else { + plan.add_mapping(canonical, path.clone())?; + plan.enqueue(selection.source.clone(), path.clone(), 0)?; + } + } + while let Some((source, path, depth)) = plan.pending.pop_front() { + plan.metadata_bytes -= source.as_os_str().len() as u64 + path.as_os_str().len() as u64; + plan.walk(source, path, depth)?; + } + // Ensure physical targets and incompatible parent claims have been settled. + for (path, node) in &plan.nodes { + if let Node::Link(target) = node { + if !plan.nodes.contains_key(target) { + return Err(error( + path, + "plan", + "selected link has an excluded or unavailable target; repair the resource", + )); + } + } + } + let parent = destination_root + .parent() + .ok_or_else(|| error(Path::new(""), "stage", "destination has no parent"))?; + fs::create_dir_all(parent) + .map_err(|cause| io_error(Path::new(""), "create_stage_parent", cause))?; + let temporary = TemporaryProjection(parent.join(format!( + ".freshell-copy-{}-{}", + std::process::id(), + NEXT_PROJECTION.fetch_add(1, Ordering::Relaxed) + ))); + fs::create_dir(&temporary.0).map_err(|cause| io_error(Path::new(""), "create_stage", cause))?; + mode(&temporary.0, 0o700, Path::new(""))?; + let mut actual_bytes = 0u64; + for (relative, node) in &plan.nodes { + let destination = temporary.0.join(relative); + match node { + Node::Directory(_) => { + fs::create_dir(&destination) + .map_err(|cause| io_error(relative, "create_directory", cause))?; + mode(&destination, 0o700, relative)?; + } + Node::File(source, source_mode, expected) => { + let mut input = + File::open(source).map_err(|cause| io_error(relative, "open", cause))?; + let mut options = OpenOptions::new(); + options.create_new(true).write(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600); + } + let mut output = options + .open(&destination) + .map_err(|cause| io_error(relative, "create_file", cause))?; + let mut buffer = [0u8; 64 * 1024]; + let mut file_bytes = 0u64; + loop { + let read = input + .read(&mut buffer) + .map_err(|cause| io_error(relative, "read", cause))?; + if read == 0 { + break; + } + file_bytes = file_bytes + .checked_add(read as u64) + .ok_or_else(|| error(relative, "read", "file byte accounting overflow"))?; + actual_bytes = actual_bytes + .checked_add(read as u64) + .ok_or_else(|| error(relative, "read", "total byte accounting overflow"))?; + bound(relative, "file_bytes", file_bytes, limits.max_file_bytes)?; + bound( + relative, + "total_bytes", + actual_bytes, + limits.max_total_bytes, + )?; + output + .write_all(&buffer[..read]) + .map_err(|cause| io_error(relative, "write", cause))?; + } + if file_bytes != *expected { + return Err(error( + relative, + "read", + "resource changed while being copied; retry with a stable installation", + )); + } + mode(&destination, private_file_mode(*source_mode), relative)?; + } + Node::Link(target) => { + #[cfg(unix)] + std::os::unix::fs::symlink( + relative_link(relative.parent().unwrap_or(Path::new("")), target), + &destination, + ) + .map_err(|cause| io_error(relative, "create_link", cause))?; + #[cfg(not(unix))] + return Err(error( + relative, + "create_link", + "symbolic resource projection requires Unix", + )); + } + } + } + if !destination_root.exists() { + fs::rename(&temporary.0, destination_root) + .map_err(|cause| io_error(Path::new(""), "publish", cause))?; + } else { + let root_metadata = fs::symlink_metadata(destination_root) + .map_err(|cause| io_error(Path::new(""), "inspect_destination", cause))?; + if !root_metadata.is_dir() || root_metadata.file_type().is_symlink() { + return Err(error( + Path::new(""), + "publish", + "destination root must be a physical directory", + )); + } + for (relative, node) in &plan.nodes { + for parent in relative + .parent() + .into_iter() + .flat_map(Path::ancestors) + .filter(|path| !path.as_os_str().is_empty()) + { + match fs::symlink_metadata(destination_root.join(parent)) { + Ok(metadata) if !metadata.is_dir() || metadata.file_type().is_symlink() => { + return Err(error( + relative, + "inspect_destination", + "selected destination parent must be a physical directory", + )) + } + Err(cause) if cause.kind() != std::io::ErrorKind::NotFound => { + return Err(io_error(relative, "inspect_destination", cause)) + } + _ => {} + } + } + match fs::symlink_metadata(destination_root.join(relative)) { + Ok(metadata) + if matches!(node, Node::Directory(_)) + && (!metadata.is_dir() || metadata.file_type().is_symlink()) => + { + return Err(error( + relative, + "inspect_destination", + "existing destination type conflicts with selected resource", + )) + } + Ok(metadata) if !matches!(node, Node::Directory(_)) && metadata.is_dir() => { + return Err(error( + relative, + "inspect_destination", + "existing directory conflicts with selected resource leaf", + )) + } + Err(cause) if cause.kind() != std::io::ErrorKind::NotFound => { + return Err(io_error(relative, "inspect_destination", cause)) + } + _ => {} + } + } + for (relative, node) in &plan.nodes { + let destination = destination_root.join(relative); + match fs::symlink_metadata(&destination) { + Ok(metadata) + if matches!(node, Node::Directory(_)) + && metadata.is_dir() + && !metadata.file_type().is_symlink() => {} + Ok(metadata) if metadata.is_dir() || (matches!(node, Node::Directory(_))) => { + return Err(error( + relative, + "publish", + "existing destination type conflicts with selected resource", + )) + } + Ok(_) => { + fs::remove_file(&destination) + .map_err(|cause| io_error(relative, "replace_leaf", cause))?; + } + Err(cause) if cause.kind() == std::io::ErrorKind::NotFound => {} + Err(cause) => return Err(io_error(relative, "inspect_destination", cause)), + } + if matches!(node, Node::Directory(_)) { + if !destination.exists() { + fs::create_dir(&destination) + .map_err(|cause| io_error(relative, "publish_directory", cause))?; + } + mode(&destination, 0o700, relative)?; + } else { + fs::rename(temporary.0.join(relative), destination) + .map_err(|cause| io_error(relative, "publish_leaf", cause))?; + } + } + } + let entries = plan.nodes.len(); + Ok(CopyReport { + entries, + bytes: actual_bytes, + projected_paths: plan.nodes.into_keys().collect(), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + #[cfg(unix)] + use std::os::unix::fs::{symlink, PermissionsExt}; + use std::{fs, path::Path, process::Command}; + + fn selection(source: &Path, destination: &str) -> CopySelection { + CopySelection { + source: source.to_owned(), + destination_relative: destination.into(), + } + } + + #[cfg(unix)] + #[test] + fn copied_skill_executes_sibling_resources_after_two_stages() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source/skill"); + fs::create_dir_all(source.join("scripts")).unwrap(); + fs::create_dir_all(source.join("references")).unwrap(); + fs::create_dir_all(source.join("assets")).unwrap(); + fs::write(source.join("SKILL.md"), "fixture skill").unwrap(); + fs::write(source.join("references/data.json"), "resource-nonce\n").unwrap(); + fs::write(source.join("assets/template.txt"), "asset-nonce\n").unwrap(); + fs::write(source.join("scripts/run"), "#!/bin/sh\nset -eu\ncd \"$(dirname \"$0\")/..\"\ncat references/data.json assets/template.txt\n").unwrap(); + for mode in [0o755, 0o700] { + fs::set_permissions(source.join("scripts/run"), fs::Permissions::from_mode(mode)) + .unwrap(); + let stage = root.path().join(format!("stage-{mode}")); + copy_environment( + &[selection(&source, "skills/fixture")], + &stage, + &[], + CopyLimits::default(), + ) + .unwrap(); + let home = root.path().join(format!("home-{mode}")); + copy_environment( + &[selection(&stage.join("skills"), "skills")], + &home, + &[], + CopyLimits::default(), + ) + .unwrap(); + let output = Command::new(home.join("skills/fixture/scripts/run")) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"resource-nonce\nasset-nonce\n"); + assert_eq!( + fs::metadata(source.join("scripts/run")) + .unwrap() + .permissions() + .mode() + & 0o7777, + mode, + "copying never changes the original executable" + ); + assert_eq!( + fs::metadata(home.join("skills/fixture/scripts/run")) + .unwrap() + .permissions() + .mode() + & 0o7777, + private_file_mode(mode) + ); + } + } + + #[cfg(unix)] + #[test] + fn copied_pnpm_plugin_external_resources_and_circular_dependencies_execute() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + let package = + source.join("node_modules/.pnpm/fixture-dependency@1/node_modules/fixture-dependency"); + fs::create_dir_all(&package).unwrap(); + fs::create_dir_all(source.join("node_modules/.bin")).unwrap(); + fs::create_dir_all(source.join("plugins")).unwrap(); + fs::write(source.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write( + package.join("package.json"), + r#"{"name":"fixture-dependency","type":"module","exports":"./index.js"}"#, + ) + .unwrap(); + fs::write(package.join("auth.json"), r#"{"nonce":"dependency-nonce"}"#).unwrap(); + fs::write(package.join("index.js"), "import fs from 'node:fs'; export default JSON.parse(fs.readFileSync(new URL('./auth.json', import.meta.url))).nonce;\n").unwrap(); + fs::write(package.join("bin"), "#!/bin/sh\nprintf bin-nonce\n").unwrap(); + fs::set_permissions(package.join("bin"), fs::Permissions::from_mode(0o755)).unwrap(); + symlink( + ".pnpm/fixture-dependency@1/node_modules/fixture-dependency", + source.join("node_modules/fixture-dependency"), + ) + .unwrap(); + symlink( + "../fixture-dependency/bin", + source.join("node_modules/.bin/fixture"), + ) + .unwrap(); + symlink("..", package.join("node_modules")).unwrap(); + fs::write( + source.join("plugins/main.js"), + "import nonce from 'fixture-dependency'; console.log(nonce);\n", + ) + .unwrap(); + let external = root.path().join("external/skill"); + fs::create_dir_all(external.join("assets")).unwrap(); + fs::write(external.join("SKILL.md"), "fixture").unwrap(); + fs::write(external.join("assets/data"), "external-nonce").unwrap(); + symlink(&external, source.join("linked-skill")).unwrap(); + let selections = [ + selection(&source.join("plugins"), "plugins"), + selection(&source.join("node_modules"), "node_modules"), + selection(&source.join("package.json"), "package.json"), + selection(&source.join("linked-skill"), "skills/linked"), + ]; + let stage = root.path().join("stage"); + copy_environment(&selections, &stage, &[], CopyLimits::default()).unwrap(); + let home = root.path().join("home"); + let transported = fs::read_dir(&stage) + .unwrap() + .map(|entry| { + let entry = entry.unwrap(); + CopySelection { + source: entry.path(), + destination_relative: entry.file_name().into(), + } + }) + .collect::>(); + copy_environment(&transported, &home, &[], CopyLimits::default()).unwrap(); + let output = Command::new("node") + .arg(home.join("plugins/main.js")) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"dependency-nonce\n"); + let output = Command::new(home.join("node_modules/.bin/fixture")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"bin-nonce"); + assert_eq!( + fs::read(home.join("skills/linked/assets/data")).unwrap(), + b"external-nonce" + ); + assert!( + fs::symlink_metadata(home.join("node_modules/fixture-dependency")) + .unwrap() + .file_type() + .is_symlink() + ); + } + + #[cfg(unix)] + #[test] + fn external_plugin_entry_preserves_adjacent_imports_and_selected_lookup_context() { + let root = tempfile::tempdir().unwrap(); + let install = root.path().join("installation"); + let suite = install.join("suite"); + let selected = root.path().join("selected"); + fs::create_dir_all(suite.join("plugins")).unwrap(); + fs::create_dir_all(install.join("node_modules/fixture-dependency")).unwrap(); + fs::create_dir(&selected).unwrap(); + fs::create_dir_all(suite.join("node_modules/fixture-local")).unwrap(); + fs::write( + suite.join("node_modules/fixture-local/package.json"), + r#"{"main":"index.js"}"#, + ) + .unwrap(); + fs::write( + suite.join("node_modules/fixture-local/index.js"), + "module.exports='local-dependency';", + ) + .unwrap(); + fs::write(suite.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write( + suite.join("plugins/helper.js"), + "export default 'adjacent-nonce';", + ) + .unwrap(); + fs::write(suite.join("plugins/main.js"),"import local from './helper.js'; import dependency from 'fixture-dependency'; import scoped from 'fixture-local'; console.log(local + ':' + dependency + ':' + scoped);").unwrap(); + fs::write( + install.join("node_modules/fixture-dependency/package.json"), + r#"{"main":"index.js"}"#, + ) + .unwrap(); + fs::write( + install.join("node_modules/fixture-dependency/index.js"), + "module.exports=require('./auth.json').nonce;", + ) + .unwrap(); + fs::write( + install.join("node_modules/fixture-dependency/auth.json"), + r#"{"nonce":"dependency-nonce"}"#, + ) + .unwrap(); + symlink(suite.join("plugins/main.js"), selected.join("main.js")).unwrap(); + let stage = root.path().join("stage"); + copy_environment( + &[ + selection(&selected, "plugin"), + selection(&install.join("node_modules"), "node_modules"), + selection(&suite.join("node_modules"), "local-node_modules"), + ], + &stage, + &[], + Default::default(), + ) + .unwrap(); + let home = root.path().join("home"); + copy_environment( + &[ + selection(&stage.join("plugin"), ".config/opencode/plugin"), + selection(&stage.join("node_modules"), ".config/opencode/node_modules"), + selection( + &stage.join("local-node_modules"), + ".config/opencode/local-node_modules", + ), + selection(&stage.join(".freshell/resources"), ".freshell/resources"), + ], + &home, + &[], + Default::default(), + ) + .unwrap(); + let output = Command::new("node") + .arg(home.join(".config/opencode/plugin/main.js")) + .output() + .unwrap(); + assert!( + output.status.success(), + "external package context: {output:?}" + ); + assert_eq!( + output.stdout, + b"adjacent-nonce:dependency-nonce:local-dependency\n" + ); + } + + #[test] + fn all_four_limits_accept_exact_and_reject_beyond_before_publication() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + fs::create_dir_all(source.join("nested")).unwrap(); + fs::write(source.join("nested/data"), b"1234").unwrap(); + let selections = [selection(&source, "resource")]; + let exact = CopyLimits { + max_entries: 3, + max_total_bytes: 4, + max_file_bytes: 4, + max_depth: 2, + }; + let report = copy_environment(&selections, &root.path().join("exact"), &[], exact).unwrap(); + assert_eq!((report.entries, report.bytes), (3, 4)); + for (kind, limits, observed, permitted) in [ + ( + "entries", + CopyLimits { + max_entries: 2, + ..exact + }, + 3, + 2, + ), + ( + "total_bytes", + CopyLimits { + max_total_bytes: 3, + ..exact + }, + 4, + 3, + ), + ( + "file_bytes", + CopyLimits { + max_file_bytes: 3, + ..exact + }, + 4, + 3, + ), + ( + "depth", + CopyLimits { + max_depth: 1, + ..exact + }, + 2, + 1, + ), + ] { + let destination = root.path().join(kind); + let error = copy_environment(&selections, &destination, &[], limits).unwrap_err(); + assert_eq!(error.limit_kind, Some(kind)); + assert_eq!(error.observed, Some(observed)); + assert_eq!(error.permitted, Some(permitted)); + assert!(!error.relative_path.is_absolute()); + assert!(!destination.exists()); + } + } + + #[cfg(unix)] + #[test] + fn concrete_credential_identities_exclude_aliases_but_keep_package_auth() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + fs::create_dir_all(source.join("package")).unwrap(); + fs::write(source.join("auth.json"), "credential-marker").unwrap(); + fs::write(source.join("package/auth.json"), "ordinary-resource").unwrap(); + symlink("../auth.json", source.join("package/alias")).unwrap(); + let destination = root.path().join("destination"); + copy_environment( + &[selection(&source, "config")], + &destination, + &[fs::canonicalize(source.join("auth.json")).unwrap()], + CopyLimits::default(), + ) + .unwrap(); + assert!(!destination.join("config/auth.json").exists()); + assert!(!destination.join("config/package/alias").exists()); + assert_eq!( + fs::read(destination.join("config/package/auth.json")).unwrap(), + b"ordinary-resource" + ); + } + + #[cfg(unix)] + #[test] + fn automatic_external_package_scope_never_selects_account_home() { + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let selected = root.path().join("selected"); + fs::create_dir(&home).unwrap(); + fs::create_dir(&selected).unwrap(); + fs::write(home.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write(home.join("standalone.js"), "export default 'nonce';").unwrap(); + fs::write(home.join("account-data"), "account-nonce").unwrap(); + symlink(home.join("standalone.js"), selected.join("main.js")).unwrap(); + let destination = root.path().join("destination"); + let result = copy_environment_with_account_home( + &[selection(&selected, "plugins")], + &destination, + &[], + Default::default(), + Some(home.clone()), + ); + let error = + result.expect_err("a plugin file does not authorize copying the entire account home"); + assert!(error.corrective_message.contains("account home")); + assert!(!destination.exists()); + assert_eq!( + fs::read(home.join("account-data")).unwrap(), + b"account-nonce" + ); + let scoped = home.join("scoped"); + fs::create_dir(&scoped).unwrap(); + fs::write(scoped.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write(scoped.join("main.js"), "export default 'scoped-nonce';").unwrap(); + symlink(scoped.join("main.js"), selected.join("scoped.js")).unwrap(); + fs::remove_file(selected.join("main.js")).unwrap(); + copy_environment_with_account_home( + &[selection(&selected, "plugins")], + &destination, + &[], + Default::default(), + Some(home), + ) + .unwrap(); + assert!(!destination.join("account-data").exists()); + assert_eq!( + fs::read(destination.join("plugins/scoped.js")).unwrap(), + b"export default 'scoped-nonce';" + ); + } + + #[cfg(unix)] + #[test] + fn external_regular_link_uses_the_same_accounted_resource_budget() { + let root = tempfile::tempdir().unwrap(); + let external = root.path().join("external.txt"); + fs::write(&external, "external-nonce").unwrap(); + let source = root.path().join("selected-link"); + symlink(&external, &source).unwrap(); + let destination = root.path().join("destination"); + let report = copy_environment( + &[selection(&source, "alias")], + &destination, + &[], + Default::default(), + ) + .unwrap(); + assert_eq!(report.bytes, 14); + assert!(fs::symlink_metadata(destination.join("alias")) + .unwrap() + .file_type() + .is_symlink()); + assert_eq!( + fs::read(destination.join("alias")).unwrap(), + b"external-nonce" + ); + let failed = root.path().join("failed"); + let error = copy_environment( + &[selection(&source, "alias")], + &failed, + &[], + CopyLimits { + max_total_bytes: 13, + ..Default::default() + }, + ) + .unwrap_err(); + assert_eq!(error.limit_kind, Some("total_bytes")); + assert_eq!(error.observed, Some(14)); + assert!(!failed.exists()); + } + + #[cfg(unix)] + #[test] + fn missing_dangling_and_unsupported_resources_fail_without_publication() { + use std::os::unix::net::UnixListener; + let root = tempfile::tempdir().unwrap(); + symlink("missing", root.path().join("dangling")).unwrap(); + let _socket = UnixListener::bind(root.path().join("socket")).unwrap(); + symlink("cycle-b", root.path().join("cycle-a")).unwrap(); + symlink("cycle-a", root.path().join("cycle-b")).unwrap(); + for name in ["missing", "dangling", "socket", "cycle-a"] { + let destination = root.path().join(format!("out-{name}")); + let error = copy_environment( + &[selection(&root.path().join(name), "selected")], + &destination, + &[], + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(error.relative_path, Path::new("selected")); + assert!(!error.corrective_message.is_empty()); + assert!(!destination.exists()); + } + } + + #[cfg(unix)] + #[test] + fn overlapping_selections_share_physical_resources_without_double_byte_charge() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + fs::create_dir_all(source.join("skill")).unwrap(); + fs::write(source.join("skill/data"), "nonce").unwrap(); + let destination = root.path().join("out"); + let report = copy_environment( + &[ + selection(&source, "all"), + selection(&source.join("skill"), "skill"), + selection(&source.join("skill/data"), "skill/data"), + ], + &destination, + &[], + CopyLimits::default(), + ) + .unwrap(); + assert_eq!(report.bytes, 5); + assert!(fs::symlink_metadata(destination.join("skill")) + .unwrap() + .file_type() + .is_symlink()); + assert_eq!(fs::read(destination.join("skill/data")).unwrap(), b"nonce"); + } + + #[cfg(unix)] + #[test] + fn overlapping_root_and_external_canonical_unit_preserve_the_logical_link() { + let root = tempfile::tempdir().unwrap(); + let parent = root.path().join("source-a"); + let external = root.path().join("source-b"); + fs::create_dir(&parent).unwrap(); + fs::create_dir(&external).unwrap(); + fs::write(external.join("SKILL.md"), "fixture").unwrap(); + fs::write(external.join("data"), "nonce").unwrap(); + symlink(&external, parent.join("linked")).unwrap(); + let destination = root.path().join("destination"); + let report = copy_environment( + &[ + selection(&parent, "skills"), + selection(&external, "skills/linked"), + selection(&external.join("data"), "skills/linked/data"), + ], + &destination, + &[], + Default::default(), + ) + .unwrap(); + assert!(fs::symlink_metadata(destination.join("skills/linked")) + .unwrap() + .file_type() + .is_symlink()); + assert_eq!( + fs::read(destination.join("skills/linked/data")).unwrap(), + b"nonce" + ); + assert_eq!(report.bytes, 12); + } + + #[test] + fn unsafe_and_incompatible_destinations_fail_before_publication() { + let root = tempfile::tempdir().unwrap(); + fs::write(root.path().join("one"), "one").unwrap(); + fs::write(root.path().join("two"), "two").unwrap(); + for selections in [ + vec![selection(&root.path().join("one"), "../escape")], + vec![ + selection(&root.path().join("one"), "same"), + selection(&root.path().join("two"), "same"), + ], + ] { + let destination = root.path().join("out"); + assert!( + copy_environment(&selections, &destination, &[], CopyLimits::default()).is_err() + ); + assert!(!destination.exists()); + } + } +} diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 25c281b2d..4e23748b5 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -469,7 +469,10 @@ impl HostedFreshAgentProxy { &workspace, &cwd, Some(mcp_capability.clone()), - ); + ) + .map_err(|error| { + tracing::warn!(event="fresh_agent.create.provider_environment_failed", %error); + })?; // Fresh agents use the direct provider transports, which currently do // not inject Freshell's terminal MCP server. Retain ordinary provider // MCP/config files, plugins, and the staged user root; only remove diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index dcd4e937a..9a9c0b7fc 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -323,7 +323,7 @@ fn provider_source_references( Path::new("/"), home, None, - ) + )? .ok_or("managed provider config selectors are unavailable")?; Ok(context.config) } @@ -418,19 +418,6 @@ pub(crate) fn approved_tui_source( }) } -fn is_secret_component(name: &str) -> bool { - matches!( - name.to_ascii_lowercase().as_str(), - ".credentials.json" - | "auth.json" - | "keys.env" - | ".env" - | "credentials" - | "secrets" - | "token.json" - ) -} - fn private_directory(path: &Path) -> Result<(), String> { std::fs::create_dir_all(path).map_err(|error| error.to_string())?; #[cfg(unix)] @@ -442,72 +429,87 @@ fn private_directory(path: &Path) -> Result<(), String> { Ok(()) } +#[cfg(test)] fn copy_provider_source( root: &Path, source: &Path, destination: &Path, - depth: usize, + _depth: usize, entries: &mut usize, ) -> Result<(), String> { - if depth > 32 || *entries >= 4096 { - return Err("managed provider config copy exceeds bounds".into()); + use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; + let report = copy_environment( + &[CopySelection { + source: source.to_owned(), + destination_relative: destination + .file_name() + .ok_or("destination has no name")? + .into(), + }], + destination.parent().ok_or("destination has no parent")?, + &provider_credential_sources(root), + CopyLimits::default(), + ) + .map_err(|error| error.to_string())?; + *entries += report.entries; + Ok(()) +} + +fn provider_credential_sources(root: &Path) -> Vec { + [ + ".credentials.json", + "auth.json", + "keys.env", + ".env", + "credentials", + "secrets", + "token.json", + ] + .into_iter() + .filter_map(|name| std::fs::canonicalize(root.join(name)).ok()) + .collect() +} + +#[cfg(unix)] +fn allow_controller_stage_read(path: &Path) -> Result<(), String> { + use std::os::unix::{ffi::OsStrExt, fs::PermissionsExt}; + let metadata = std::fs::symlink_metadata(path).map_err(|error| error.to_string())?; + let name = std::ffi::CString::new(path.as_os_str().as_bytes()) + .map_err(|_| "stage path contains NUL")?; + // Rootless Docker maps its namespace group 0 to the controller's primary + // group. Physical lchown never follows resource aliases. + let gid = unsafe { libc::getegid() }; + if unsafe { libc::lchown(name.as_ptr(), libc::geteuid(), gid) } != 0 { + return Err(format!( + "normalize mapped group 0 for provider stage: {}", + std::io::Error::last_os_error() + )); } - *entries += 1; - let Some(name) = source.file_name().and_then(|name| name.to_str()) else { - return Ok(()); - }; - if is_secret_component(name) { + if metadata.file_type().is_symlink() { return Ok(()); } - let canonical = match std::fs::canonicalize(source) { - Ok(canonical) => canonical, - Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), - Err(error) => return Err(format!("managed provider config source: {error}")), + let mode = if metadata.is_dir() { + 0o750 + } else if metadata.is_file() { + freshell_platform::provider_environment::provider_read_stage_mode( + metadata.permissions().mode(), + ) + } else { + return Err("provider stage contains unsupported inode".into()); }; - if !canonical.starts_with(root) { - tracing::warn!("managed_mcp.provider_config_symlink_escape_skipped"); - return Ok(()); - } - if canonical - .strip_prefix(root) - .map_err(|error| error.to_string())? - .components() - .any(|component| { - component - .as_os_str() - .to_str() - .is_some_and(is_secret_component) - }) - { - return Ok(()); - } - let metadata = std::fs::metadata(&canonical).map_err(|error| error.to_string())?; + std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode)) + .map_err(|error| error.to_string())?; if metadata.is_dir() { - private_directory(destination)?; - for entry in std::fs::read_dir(&canonical).map_err(|error| error.to_string())? { - let entry = entry.map_err(|error| error.to_string())?; - copy_provider_source( - root, - &entry.path(), - &destination.join(entry.file_name()), - depth + 1, - entries, - )?; - } - } else if metadata.is_file() { - if metadata.len() > 16 * 1024 * 1024 { - return Err("managed provider config file exceeds bounds".into()); - } - std::fs::copy(&canonical, destination).map_err(|error| error.to_string())?; - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - std::fs::set_permissions(destination, std::fs::Permissions::from_mode(0o600)) - .map_err(|error| error.to_string())?; + for entry in std::fs::read_dir(path).map_err(|error| error.to_string())? { + allow_controller_stage_read(&entry.map_err(|error| error.to_string())?.path())?; } } Ok(()) } +#[cfg(not(unix))] +fn allow_controller_stage_read(_path: &Path) -> Result<(), String> { + Err("managed provider stage requires rootless Unix ownership".into()) +} fn staged_provider_root_path(directory: &Path, grant_id: &str) -> Result { grant_path(directory, grant_id)?; @@ -674,34 +676,58 @@ fn stage_provider_root( Err(error) if error.kind() == std::io::ErrorKind::NotFound => None, Err(error) => return Err(error.to_string()), }; - let mut entries = 0; + use freshell_platform::provider_environment::{ + copy_environment, CopyLimits, CopySelection, + }; + let resource_root = temporary.join(".freshell/resources"); + private_directory(&resource_root)?; + let mut selections = vec![CopySelection { + source: resource_root, + destination_relative: ".freshell/resources".into(), + }]; for reference in references { - if reference.root != ProviderConfigRoot::UserProvider - || reference.relative_path.contains('/') - || reference.relative_path.contains('\\') + if reference.root != ProviderConfigRoot::UserProvider { + return Err("managed provider selector is unsafe".into()); + } + if reference.relative_path == ".freshell/resources" + && reference.provider_relative_path == ".freshell/resources" + { + continue; + } + if provider == "codex" + && reference.relative_path == "freshell-discovered-agents" + && reference.provider_relative_path == ".codex/skills/freshell-discovered-agents" + { + selections.push(CopySelection { + source: home.join(".agents/skills"), + destination_relative: reference.relative_path.clone().into(), + }); + continue; + } + if reference.relative_path.contains(['/', '\\']) || reference.relative_path.is_empty() || reference.relative_path == "." || reference.relative_path == ".." - || is_secret_component(&reference.relative_path) || reference.provider_relative_path != format!("{provider_dir}/{}", reference.relative_path) { return Err("managed provider selector is unsafe".into()); } - let Some(root) = canonical_root.as_ref() else { + if canonical_root.is_none() { return Err("referenced managed provider root is unavailable".into()); - }; - copy_provider_source( - root, - &source.join(&reference.relative_path), - &temporary.join(&reference.relative_path), - 0, - &mut entries, - )?; + } + selections.push(CopySelection { + source: source.join(&reference.relative_path), + destination_relative: reference.relative_path.clone().into(), + }); } if provider == "opencode" { let ephemeral = temporary.join("ephemeral"); private_directory(&ephemeral)?; + selections.push(CopySelection { + source: ephemeral.clone(), + destination_relative: "ephemeral".into(), + }); if let Some(prior_stage) = prior_stage { let source = prior_stage.join("ephemeral").join("inline-config.json"); if source.is_file() { @@ -754,6 +780,58 @@ fn stage_provider_root( } } } + let started = std::time::Instant::now(); + tracing::info!( + event = "provider.environment.copy.started", + stage = "controller", + provider, + selections = selections.len() + ); + let mut excluded = provider_credential_sources(&source); + for path in [ + ".claude/.credentials.json", + ".codex/auth.json", + ".local/share/opencode/auth.json", + ".config/opencode/auth.json", + ".amplifier/keys.env", + ".amplifier/.env", + // Native conversations/history/logs remain in their own stores; + // an alias cannot turn them into selected package resources. + ".claude/projects", + ".claude/history.jsonl", + ".claude/debug", + ".codex/sessions", + ".codex/archived_sessions", + ".codex/history.jsonl", + ".codex/log", + ".amplifier/projects", + ".local/share/opencode/storage", + ".local/share/opencode/opencode.db", + ".local/share/opencode/opencode.db-wal", + ".local/share/opencode/opencode.db-shm", + ] { + if let Ok(canonical) = std::fs::canonicalize(home.join(path)) { + excluded.push(canonical); + } + } + excluded.extend( + crate::managed_provider_bootstrap::named_provider_onecli_references(provider)? + .into_iter() + .map(|reference| PathBuf::from(reference.source_path)), + ); + let report=copy_environment(&selections,&temporary,&excluded,CopyLimits::default()).map_err(|error| { + tracing::error!(event="provider.environment.copy.failed",stage="controller",provider,operation=error.operation,relative_path=%error.relative_path.display(),limit_kind=error.limit_kind,observed=error.observed,permitted=error.permitted,elapsed_ms=started.elapsed().as_millis() as u64,corrective_message=%error.corrective_message); + error.to_string() + })?; + tracing::info!( + event = "provider.environment.copy.completed", + stage = "controller", + provider, + entries = report.entries, + bytes = report.bytes, + elapsed_ms = started.elapsed().as_millis() as u64 + ); + allow_controller_stage_read(&temporary)?; std::fs::rename(&temporary, &staged).map_err(|error| error.to_string())?; Ok(staged.clone()) })(); @@ -822,12 +900,32 @@ fn write_private_bytes(path: &Path, bytes: &[u8]) -> Result<(), String> { } fn copy_ephemeral_file(source: &Path, destination: &Path) -> Result<(), String> { + use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; let metadata = std::fs::metadata(source).map_err(|error| error.to_string())?; - if !metadata.is_file() || metadata.len() > 64 * 1024 { - return Err("managed OpenCode config file is invalid".into()); + // Configuration-source views have their explicit 16 MiB profile; resource + // copying retains the shared 128 MiB per-file policy, with no hidden cap. + if !metadata.is_file() || metadata.len() > 16 * 1024 * 1024 { + return Err(format!("selected OpenCode configuration source exceeds 16 MiB or is not a regular file; observed {} bytes",metadata.len())); } - let bytes = std::fs::read(source).map_err(|error| error.to_string())?; - write_private_bytes(destination, &bytes) + let report = copy_environment( + &[CopySelection { + source: source.to_owned(), + destination_relative: destination + .file_name() + .ok_or("config destination has no name")? + .into(), + }], + destination + .parent() + .ok_or("config destination has no parent")?, + &[], + CopyLimits::default(), + ) + .map_err(|error| error.to_string())?; + if report.bytes > 16 * 1024 * 1024 { + return Err("OpenCode configuration source grew beyond its explicit 16 MiB profile; retry with a stable source".into()); + } + Ok(()) } #[derive(Serialize)] @@ -1224,6 +1322,207 @@ pub(crate) async fn scoped_auth_middleware( mod tests { use super::*; + #[cfg(unix)] + #[test] + fn managed_provider_resource_script_and_dependency_execute() { + use std::{os::unix::fs::PermissionsExt, process::Command}; + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + std::fs::create_dir_all(source.join("scripts")).unwrap(); + std::fs::create_dir_all(source.join("node_modules/fixture-dependency")).unwrap(); + std::fs::write(source.join("data"), "resource-nonce\n").unwrap(); + std::fs::write( + source.join("scripts/run"), + "#!/bin/sh\ncat \"$(dirname \"$0\")/../data\"\n", + ) + .unwrap(); + std::fs::set_permissions( + source.join("scripts/run"), + std::fs::Permissions::from_mode(0o700), + ) + .unwrap(); + std::fs::write( + source.join("node_modules/fixture-dependency/package.json"), + r#"{"main":"index.js"}"#, + ) + .unwrap(); + std::fs::write( + source.join("node_modules/fixture-dependency/auth.json"), + r#"{"nonce":"dependency-nonce"}"#, + ) + .unwrap(); + std::fs::write( + source.join("node_modules/fixture-dependency/index.js"), + "module.exports=require('./auth.json').nonce", + ) + .unwrap(); + std::fs::write( + source.join("plugin.js"), + "console.log(require('fixture-dependency'))", + ) + .unwrap(); + let stage = root.path().join("stage"); + let home = root.path().join("home"); + copy_provider_source(&source, &source, &stage, 0, &mut 0).unwrap(); + copy_provider_source(&stage, &stage, &home, 0, &mut 0).unwrap(); + let plugin = Command::new("node") + .arg(home.join("plugin.js")) + .output() + .unwrap(); + assert!(plugin.status.success(), "dependency lookup: {plugin:?}"); + assert_eq!(plugin.stdout, b"dependency-nonce\n"); + let script = Command::new(home.join("scripts/run")) + .output() + .expect("two-stage executable must retain execute permission"); + assert!(script.status.success(), "{script:?}"); + assert_eq!(script.stdout, b"resource-nonce\n"); + } + + #[cfg(unix)] + #[test] + fn managed_provider_two_stage_script_retains_execute_permission() { + use std::{os::unix::fs::PermissionsExt, process::Command}; + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + std::fs::create_dir(&source).unwrap(); + std::fs::write(source.join("run"), "#!/bin/sh\nprintf resource-nonce\n").unwrap(); + std::fs::set_permissions(source.join("run"), std::fs::Permissions::from_mode(0o755)) + .unwrap(); + let stage = root.path().join("stage"); + let home = root.path().join("home"); + copy_provider_source(&source, &source, &stage, 0, &mut 0).unwrap(); + copy_provider_source(&stage, &stage, &home, 0, &mut 0).unwrap(); + assert!(Command::new(home.join("run")) + .output() + .expect("two-stage script spawn") + .status + .success()); + } + + #[cfg(unix)] + #[test] + fn failed_selected_resource_copy_never_publishes_or_leaves_a_stage() { + use freshell_runtime_protocol::{ProviderConfigReference, ProviderConfigRoot}; + use std::os::unix::{fs::symlink, net::UnixListener}; + let control = tempfile::tempdir().unwrap(); + let capabilities = control.path().join("capabilities"); + std::fs::create_dir(&capabilities).unwrap(); + let home = tempfile::tempdir().unwrap(); + let source = home.path().join(".claude"); + std::fs::create_dir(&source).unwrap(); + symlink("missing", source.join("dangling")).unwrap(); + let _socket = UnixListener::bind(source.join("socket")).unwrap(); + for name in ["missing", "dangling", "socket"] { + let references = [ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: name.into(), + provider_relative_path: format!(".claude/{name}"), + format: "directory".into(), + }]; + assert!(stage_provider_root( + &capabilities, + &format!("grant-{name}"), + "claude", + home.path(), + &references, + OpencodeEphemeralInput::default(), + None + ) + .is_err()); + assert!( + !staged_provider_root_path(&capabilities, &format!("grant-{name}")) + .unwrap() + .exists() + ); + assert_eq!( + std::fs::read_dir(control.path().join("provider-roots")) + .unwrap() + .count(), + 0 + ); + } + } + #[cfg(unix)] + #[test] + fn native_state_aliases_are_excluded_without_filtering_dependency_basenames() { + use freshell_runtime_protocol::{ProviderConfigReference, ProviderConfigRoot}; + use std::os::unix::fs::symlink; + let control = tempfile::tempdir().unwrap(); + let capabilities = control.path().join("capabilities"); + std::fs::create_dir(&capabilities).unwrap(); + let home = tempfile::tempdir().unwrap(); + let plugins = home.path().join(".claude/plugins"); + let native = home.path().join(".claude/projects"); + std::fs::create_dir_all(&plugins).unwrap(); + std::fs::create_dir_all(&native).unwrap(); + std::fs::write(native.join("conversation.jsonl"), "native-history-nonce").unwrap(); + std::fs::write(plugins.join("history.jsonl"), "ordinary-resource-nonce").unwrap(); + symlink( + native.join("conversation.jsonl"), + plugins.join("native-alias"), + ) + .unwrap(); + let refs = [ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "plugins".into(), + provider_relative_path: ".claude/plugins".into(), + format: "directory".into(), + }]; + let stage = stage_provider_root( + &capabilities, + "grant-native-state", + "claude", + home.path(), + &refs, + Default::default(), + None, + ) + .unwrap(); + assert!(!stage.join("plugins/native-alias").exists()); + assert_eq!( + std::fs::read(stage.join("plugins/history.jsonl")).unwrap(), + b"ordinary-resource-nonce" + ); + assert_eq!( + std::fs::read(native.join("conversation.jsonl")).unwrap(), + b"native-history-nonce" + ); + } + + #[cfg(unix)] + #[test] + fn controller_stage_preserves_executables_and_ordinary_ephemeral_modes() { + use std::os::unix::fs::{symlink, PermissionsExt}; + let root = tempfile::tempdir().unwrap(); + let stage = root.path().join("stage"); + std::fs::create_dir(&stage).unwrap(); + let source = root.path().join("run"); + std::fs::write(&source, "#!/bin/sh\nprintf nonce").unwrap(); + std::fs::set_permissions(&source, std::fs::Permissions::from_mode(0o700)).unwrap(); + copy_ephemeral_file(&source, &stage.join("run")).unwrap(); + let target = root.path().join("outside"); + std::fs::write(&target, "sentinel").unwrap(); + symlink(&target, stage.join("alias")).unwrap(); + let before = std::fs::metadata(&target).unwrap().permissions().mode(); + allow_controller_stage_read(&stage).unwrap(); + assert_eq!( + std::fs::metadata(stage.join("run")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o740 + ); + assert_eq!( + std::fs::metadata(&source).unwrap().permissions().mode() & 0o777, + 0o700 + ); + assert_eq!( + std::fs::metadata(&target).unwrap().permissions().mode(), + before + ); + } + #[test] fn managed_mcp_stages_only_approved_provider_config() { use freshell_runtime_protocol::{ProviderConfigReference, ProviderConfigRoot}; @@ -1272,9 +1571,15 @@ mod tests { std::fs::read_to_string(staged.join("plugins/inside.txt")).unwrap(), "current plugin" ); - assert!(!staged.join("plugins/auth.json").exists()); + assert_eq!( + std::fs::read_to_string(staged.join("plugins/auth.json")).unwrap(), + "secret" + ); assert!(!staged.join("plugins/alias-secret.json").exists()); - assert!(!staged.join("plugins/outside.txt").exists()); + assert_eq!( + std::fs::read_to_string(staged.join("plugins/outside.txt")).unwrap(), + "outside" + ); let store = CapabilityStore::default(); let reference = store .issue( diff --git a/crates/freshell-server/src/managed_provider_bootstrap.rs b/crates/freshell-server/src/managed_provider_bootstrap.rs index b709b373e..0a4a4637b 100644 --- a/crates/freshell-server/src/managed_provider_bootstrap.rs +++ b/crates/freshell-server/src/managed_provider_bootstrap.rs @@ -18,7 +18,15 @@ pub fn provider_launch_context_for_managed( workspace: &Path, mcp_capability: Option, ) -> Option { - provider_launch_context_for_managed_at(provider, workspace, workspace, mcp_capability) + let home = tempfile::tempdir().unwrap(); + provider_launch_context_from_home_at( + provider, + workspace, + workspace, + home.path(), + mcp_capability, + ) + .expect("provider discovery") } pub fn provider_launch_context_for_managed_at( @@ -26,7 +34,7 @@ pub fn provider_launch_context_for_managed_at( workspace: &Path, project_dir: &Path, mcp_capability: Option, -) -> Option { +) -> Result, String> { let home = std::env::var_os("HOME") .map(PathBuf::from) .unwrap_or_default(); @@ -38,7 +46,7 @@ pub(crate) fn provider_launch_context_from_home( workspace: &Path, home: &Path, mcp_capability: Option, -) -> Option { +) -> Result, String> { provider_launch_context_from_home_at(provider, workspace, workspace, home, mcp_capability) } @@ -48,7 +56,7 @@ fn provider_launch_context_from_home_at( project_dir: &Path, home: &Path, mcp_capability: Option, -) -> Option { +) -> Result, String> { let xdg_config_home = std::env::var_os("XDG_CONFIG_HOME") .filter(|value| !value.is_empty()) .map(PathBuf::from); @@ -69,7 +77,7 @@ fn provider_launch_context_from_home_at_with_xdg( home: &Path, xdg_config_home: Option<&Path>, mcp_capability: Option, -) -> Option { +) -> Result, String> { let mcp_enabled = mcp_capability.is_some(); let (root, entries, preparation) = match provider { "claude" => ( @@ -97,12 +105,12 @@ fn provider_launch_context_from_home_at_with_xdg( &["config.toml", "AGENTS.md", "skills", "rules"][..], ProviderPreparation::Codex { tui_args: if mcp_enabled { - managed_codex_mcp_args()? + managed_codex_mcp_args().ok_or("managed Codex MCP preparation unavailable")? } else { Vec::new() }, sidecar_args: if mcp_enabled { - managed_codex_mcp_args()? + managed_codex_mcp_args().ok_or("managed Codex MCP preparation unavailable")? } else { Vec::new() }, @@ -118,6 +126,13 @@ fn provider_launch_context_from_home_at_with_xdg( "agents", "commands", "skills", + "node_modules", + "package.json", + "package-lock.json", + "pnpm-lock.yaml", + "yarn.lock", + "bun.lock", + "bun.lockb", ][..], ProviderPreparation::Opencode { project_config: Vec::new(), @@ -134,28 +149,35 @@ fn provider_launch_context_from_home_at_with_xdg( resume_args: Vec::new(), }, ), - _ => return None, + _ => return Ok(None), }; let preparation = match preparation { ProviderPreparation::Opencode { tui_config, .. } => { - let project_relative = project_dir.strip_prefix(workspace).ok()?; - let project_config = [ + let project_relative = project_dir + .strip_prefix(workspace) + .map_err(|_| "managed project is outside workspace")?; + let mut project_config = Vec::new(); + for entry in [ "opencode.json", "opencode.jsonc", ".opencode/opencode.json", ".opencode/opencode.jsonc", - ] - .into_iter() - .filter_map(|entry| { + ] { let relative_path = project_relative.join(entry); let relative_path = relative_path - .to_str()? + .to_str() + .ok_or("managed config path is not UTF-8")? .replace(std::path::MAIN_SEPARATOR, "/"); + if !optional_resource_exists(&workspace.join(&relative_path))? { + continue; + } let source = approved_config_path(workspace, &relative_path)?; if !source.is_file() { - return None; + return Err(format!( + "selected project configuration {relative_path} is not a regular file" + )); } - Some(ProviderConfigReference { + project_config.push(ProviderConfigReference { root: ProviderConfigRoot::Workspace, relative_path: relative_path.clone(), provider_relative_path: format!(".config/opencode/project/{relative_path}"), @@ -165,9 +187,8 @@ fn provider_launch_context_from_home_at_with_xdg( "json" } .into(), - }) - }) - .collect(); + }); + } ProviderPreparation::Opencode { project_config, tui_config, @@ -189,9 +210,16 @@ fn provider_launch_context_from_home_at_with_xdg( .iter() .map(|entry| (*entry).to_owned()) .collect::>(); - if let Ok(discovered) = std::fs::read_dir(&provider_root) { - for item in discovered.flatten() { - let name = item.file_name().to_string_lossy().into_owned(); + if optional_resource_exists(&provider_root)? { + for item in std::fs::read_dir(&provider_root) + .map_err(|error| format!("inspect provider root: {:?}", error.kind()))? + { + let item = + item.map_err(|error| format!("inspect provider entry: {:?}", error.kind()))?; + let name = item + .file_name() + .into_string() + .map_err(|_| "provider resource name is not UTF-8")?; let extension = Path::new(&name) .extension() .and_then(|value| value.to_str()); @@ -199,7 +227,7 @@ fn provider_launch_context_from_home_at_with_xdg( extension, Some("json" | "jsonc" | "toml" | "yaml" | "yml" | "md") ) && !matches!( - name.to_ascii_lowercase().as_str(), + name.as_str(), ".credentials.json" | "auth.json" | "keys.env" @@ -212,36 +240,62 @@ fn provider_launch_context_from_home_at_with_xdg( } } } - let config = selected - .into_iter() - .filter_map(|entry| { - let path = approved_config_path(&provider_root, &entry)?; - let format = if path.is_dir() { - "directory" - } else if path.is_file() { - match path.extension().and_then(|value| value.to_str()) { - Some("json") => "json", - Some("jsonc") => "jsonc", - Some("toml") => "toml", - Some("yaml" | "yml") => "yaml", - _ => "text", - } - } else { - return None; - }; - Some(ProviderConfigReference { - root: ProviderConfigRoot::UserProvider, - relative_path: entry.clone(), - provider_relative_path: format!("{root}/{entry}"), - format: format.into(), - }) - }) - .collect(); - Some(ProviderLaunchContext { + let mut config = Vec::new(); + for entry in selected { + if !optional_resource_exists(&provider_root.join(&entry))? { + continue; + } + let path = approved_config_path(&provider_root, &entry)?; + let format = if path.is_dir() { + "directory" + } else if path.is_file() { + match path.extension().and_then(|value| value.to_str()) { + Some("json") => "json", + Some("jsonc") => "jsonc", + Some("toml") => "toml", + Some("yaml" | "yml") => "yaml", + _ => "text", + } + } else { + return Err(format!( + "selected provider resource {entry} is an unsupported filesystem node" + )); + }; + config.push(ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: entry.clone(), + provider_relative_path: format!("{root}/{entry}"), + format: format.into(), + }); + } + if provider == "codex" && optional_resource_exists(&home.join(".agents/skills"))? { + approved_config_path(home, ".agents/skills")?; + config.push(ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "freshell-discovered-agents".into(), + provider_relative_path: ".codex/skills/freshell-discovered-agents".into(), + format: "directory".into(), + }); + } + // One normal transport selection preserves the shared resource closure + // identically in the host stage, without dependency rediscovery. + config.push(ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: ".freshell/resources".into(), + provider_relative_path: ".freshell/resources".into(), + format: "directory".into(), + }); + if config.len() > 128 { + return Err( + "managed provider selections exceed 128; reduce unrelated top-level configurations" + .into(), + ); + } + Ok(Some(ProviderLaunchContext { preparation, mcp_capability, config, - }) + })) } struct ManagedImageMcpRuntime; @@ -280,21 +334,26 @@ fn managed_codex_mcp_args() -> Option> { Some(renderings.tui.args) } -/// Ordinary symlinked configs are included only when each resolved component -/// remains in the approved provider or workspace root. -fn approved_config_path(root: &Path, relative: &str) -> Option { - let canonical_root = std::fs::canonicalize(root).ok()?; - let mut current = canonical_root.clone(); - for component in Path::new(relative).components() { - if !matches!(component, std::path::Component::Normal(_)) { - return None; - } - current = std::fs::canonicalize(current.join(component.as_os_str())).ok()?; - if !current.starts_with(&canonical_root) { - return None; - } +/// Absence is optional only before a provider root/resource is selected. +fn optional_resource_exists(path: &Path) -> Result { + match std::fs::symlink_metadata(path) { + Ok(_) => Ok(true), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(error) => Err(format!( + "inspect selected provider resource: {:?}; repair permissions and retry", + error.kind() + )), + } +} +fn approved_config_path(root: &Path, relative: &str) -> Result { + if relative.is_empty() + || Path::new(relative) + .components() + .any(|component| !matches!(component, std::path::Component::Normal(_))) + { + return Err("managed provider resource selector is unsafe".into()); } - Some(current) + std::fs::canonicalize(root.join(relative)).map_err(|error| format!("resolve selected provider resource {relative}: {:?}; repair the selected resource or link and retry",error.kind())) } pub fn named_provider_onecli_references( @@ -474,6 +533,106 @@ mod tests { assert!(tui_args.iter().any(|arg| arg.contains("freshell-mcp"))); } + #[test] + fn opencode_selects_complete_installed_dependency_and_package_forms() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let provider = home.path().join(".config/opencode"); + fs::create_dir_all(provider.join("node_modules/package")).unwrap(); + fs::write(provider.join("node_modules/package/auth.json"), "ordinary").unwrap(); + for name in [ + "package.json", + "package-lock.json", + "pnpm-lock.yaml", + "yarn.lock", + "bun.lock", + "bun.lockb", + ] { + fs::write(provider.join(name), "fixture").unwrap(); + } + let context = provider_launch_context_from_home_at_with_xdg( + "opencode", + workspace.path(), + workspace.path(), + home.path(), + None, + None, + ) + .unwrap() + .unwrap(); + for name in [ + "node_modules", + "package.json", + "package-lock.json", + "pnpm-lock.yaml", + "yarn.lock", + "bun.lock", + "bun.lockb", + ] { + assert!( + context + .config + .iter() + .any(|reference| reference.relative_path == name), + "missing resource selection {name}" + ); + } + context.validate("opencode").unwrap(); + } + #[test] + fn codex_selects_discovered_agents_skills_without_native_state() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + fs::create_dir_all(home.path().join(".agents/skills/fixture/scripts")).unwrap(); + fs::write( + home.path().join(".agents/skills/fixture/SKILL.md"), + "fixture", + ) + .unwrap(); + fs::create_dir_all(home.path().join(".codex/sessions")).unwrap(); + let context = provider_launch_context_from_home_at_with_xdg( + "codex", + workspace.path(), + workspace.path(), + home.path(), + None, + None, + ) + .unwrap() + .unwrap(); + assert!(context + .config + .iter() + .any( + |reference| reference.relative_path == "freshell-discovered-agents" + && reference + .provider_relative_path + .starts_with(".codex/skills/") + )); + assert!(!context + .config + .iter() + .any(|reference| reference.relative_path == "sessions")); + context.validate("codex").unwrap(); + } + #[test] + fn selected_dangling_provider_resource_is_an_actionable_error() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + fs::create_dir(home.path().join(".claude")).unwrap(); + std::os::unix::fs::symlink("missing", home.path().join(".claude/skills")).unwrap(); + let error = provider_launch_context_from_home_at_with_xdg( + "claude", + workspace.path(), + workspace.path(), + home.path(), + None, + None, + ) + .unwrap_err(); + assert!(error.contains("skills") && error.contains("repair")); + } + #[test] fn replacement_context_discovers_new_top_level_provider_config() { let home = tempfile::tempdir().unwrap(); @@ -481,6 +640,7 @@ mod tests { fs::create_dir_all(home.path().join(".claude")).unwrap(); let first = provider_launch_context_from_home("claude", workspace.path(), home.path(), None) + .unwrap() .unwrap(); assert!(!first .config @@ -490,6 +650,7 @@ mod tests { fs::write(home.path().join(".claude/keybindings.json"), "{}").unwrap(); let replacement = provider_launch_context_from_home("claude", workspace.path(), home.path(), None) + .unwrap() .unwrap(); assert!(replacement .config @@ -556,6 +717,7 @@ mod tests { home.path(), None, ) + .unwrap() .unwrap(); let ProviderPreparation::Opencode { project_config, .. } = context.preparation else { panic!("expected OpenCode preparation"); @@ -581,6 +743,7 @@ mod tests { .unwrap(); let context = provider_launch_context_from_home("opencode", workspace.path(), home.path(), None) + .unwrap() .unwrap(); assert!(context.config.iter().any( |reference| reference.relative_path == "plugin" && reference.format == "directory" @@ -607,6 +770,7 @@ mod tests { Some(xdg.path()), None, ) + .unwrap() .unwrap(); assert!(context.config.iter().any(|reference| { @@ -615,11 +779,21 @@ mod tests { assert!(context.config.iter().any(|reference| { reference.relative_path == "plugin" && reference.format == "directory" })); - assert!(context.config.iter().all(|reference| { - reference + assert_eq!( + context + .config + .iter() + .filter(|reference| reference.relative_path == ".freshell/resources") + .count(), + 1 + ); + assert!(context + .config + .iter() + .filter(|reference| reference.relative_path != ".freshell/resources") + .all(|reference| reference .provider_relative_path - .starts_with(".config/opencode/") - })); + .starts_with(".config/opencode/"))); assert!(!context .config .iter() @@ -628,7 +802,7 @@ mod tests { } #[test] - fn opencode_context_accepts_in_root_links_and_omits_escaping_links() { + fn opencode_context_retains_selected_external_and_in_root_links() { use std::os::unix::fs::symlink; let root = tempfile::tempdir().unwrap(); let workspace = root.path().join("workspace"); @@ -650,7 +824,7 @@ mod tests { assert!(project_config .iter() .any(|entry| entry.relative_path == "opencode.json")); - assert!(!project_config + assert!(project_config .iter() .any(|entry| entry.relative_path == ".opencode/opencode.json")); } diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 138aa6c3e..7cd0687bf 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -228,7 +228,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { &workspace, &cwd, mcp_capability.clone(), - ); + )?; if let Some(context) = provider_launch_context.as_mut() { if let freshell_runtime_protocol::ProviderPreparation::Opencode { inline_config, diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 9b75c258d..d79669582 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -958,28 +958,28 @@ async fn grant_execution( launch.cwd = resume.cwd.clone(); } if !exact_resume { - prepare_provider_context( - launch.provider_launch_context.as_ref(), - launch.provider.as_str(), - &launch.workspace_path, + prepare_provider_bootstrap_files( + &launch.provider_bootstrap_files, launch.run_as_uid, launch.run_as_gid, ) .map_err(|error| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, - format!("prepare hosted provider config: {error}"), + format!("prepare hosted fresh-agent provider state failed: {error}"), ) })?; - prepare_provider_bootstrap_files( - &launch.provider_bootstrap_files, + prepare_provider_context( + launch.provider_launch_context.as_ref(), + launch.provider.as_str(), + &launch.workspace_path, launch.run_as_uid, launch.run_as_gid, ) .map_err(|error| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, - format!("prepare hosted fresh-agent provider state failed: {error}"), + format!("prepare hosted provider config: {error}"), ) })?; } else { @@ -1071,103 +1071,123 @@ fn prepare_provider_state_for_fresh_launch( ) -> Result<(), RuntimeError> { prepare_provider_state_before_bootstrap_with( terminal, - providers::prepare_provider_state_before_bootstrap, - transfer_host_created_provider_state, + |terminal| { + let mut created = providers::prepare_provider_state_before_bootstrap(terminal)?; + match write_provider_bootstrap_files( + &terminal.provider_bootstrap_files, + Path::new("/home/freshell/provider"), + Path::new("/run/freshell-bootstrap"), + ) { + Ok(bootstrap) => created.extend(bootstrap), + Err(error) => { + rollback_created_provider_paths(&created); + return Err(error); + } + } + Ok(created) + }, + |terminal, created| { + transfer_created_paths( + Path::new("/home/freshell/provider"), + created, + terminal.run_as_uid, + terminal.run_as_gid, + ) + }, |terminal| { prepare_provider_context_for_fresh_launch(terminal, &ProviderConfigPaths::default()) }, - )?; - prepare_provider_bootstrap_files( - &terminal.provider_bootstrap_files, - terminal.run_as_uid, - terminal.run_as_gid, ) - .map_err(|error| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - format!("prepare provider bootstrap: {error}"), - ) - }) } fn prepare_provider_state_before_bootstrap_with( terminal: &mut TerminalLaunchSpec, - prepare_state: impl FnOnce(&mut TerminalLaunchSpec) -> Result<(), String>, - transfer_state: impl FnOnce(&TerminalLaunchSpec) -> Result<(), String>, + prepare_state: impl FnOnce(&mut TerminalLaunchSpec) -> Result, String>, + transfer_state: impl FnOnce(&TerminalLaunchSpec, &[PathBuf]) -> Result<(), String>, prepare_context: impl FnOnce(&TerminalLaunchSpec) -> Result<(), String>, ) -> Result<(), RuntimeError> { - prepare_state(terminal).map_err(|error| { + let created = prepare_state(terminal).map_err(|error| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, format!("prepare managed provider state: {error}"), ) })?; - prepare_context(terminal).map_err(|error| { + transfer_state(terminal, &created).map_err(|error| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, - format!("prepare provider config: {error}"), + format!("transfer managed provider state: {error}"), ) })?; - transfer_state(terminal).map_err(|error| { + prepare_context(terminal).map_err(|error| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, - format!("transfer managed provider state: {error}"), + format!("prepare provider config: {error}"), ) }) } -fn transfer_host_created_provider_state(terminal: &TerminalLaunchSpec) -> Result<(), String> { - if terminal.mode != "amplifier" { - return Ok(()); - } - let root = Path::new("/home/freshell/provider/.amplifier"); - if !root.exists() { - return Ok(()); +fn transfer_created_paths( + home: &Path, + created: &[PathBuf], + uid: u32, + gid: u32, +) -> Result<(), String> { + let mut paths = created + .iter() + .cloned() + .collect::>(); + if paths.len() > freshell_platform::provider_environment::CopyLimits::default().max_entries { + return Err(format!( + "created provider paths exceed shared entry bound: observed {}, permitted 100000", + paths.len() + )); } - transfer_owned_tree(root, terminal.run_as_uid, terminal.run_as_gid, 10_000) -} - -fn transfer_owned_tree(root: &Path, uid: u32, gid: u32, max_entries: usize) -> Result<(), String> { - let root = std::fs::canonicalize(root) - .map_err(|error| format!("canonicalize {}: {error}", root.display()))?; - let mut stack = vec![root.clone()]; - let mut paths = Vec::new(); - while let Some(path) = stack.pop() { - if paths.len() >= max_entries { - return Err(format!( - "provider state tree exceeded the {max_entries}-entry ownership bound" - )); - } - let metadata = std::fs::symlink_metadata(&path) - .map_err(|error| format!("inspect {}: {error}", path.display()))?; - if metadata.file_type().is_symlink() { - return Err(format!( - "provider state ownership transfer refuses symlink {}", - path.display() - )); - } - if !path.starts_with(&root) { - return Err(format!( - "provider state ownership transfer escaped {}", - root.display() - )); - } - paths.push(path.clone()); - if metadata.is_dir() { - for entry in std::fs::read_dir(&path) - .map_err(|error| format!("read {}: {error}", path.display()))? + for path in &paths { + if !path.starts_with(home) + || path + .strip_prefix(home) + .map_err(|error| error.to_string())? + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("created provider ownership path is outside the physical HOME".into()); + } + std::fs::symlink_metadata(path).map_err(|error| { + format!("inspect created provider path {}: {error}", path.display()) + })?; + // The leaf may be a selected link, but no traversed parent may be one. + for parent in path + .parent() + .into_iter() + .flat_map(Path::ancestors) + .take_while(|parent| parent.starts_with(home)) + { + if std::fs::symlink_metadata(parent) + .map_err(|error| error.to_string())? + .file_type() + .is_symlink() { - let entry = entry.map_err(|error| error.to_string())?; - stack.push(entry.path()); + return Err("created ownership path has a symbolic-link parent".into()); } } } - // Children first, so the trusted host never loses traversal rights before - // all exact host-created files have been transferred. - paths.sort_by_key(|path| std::cmp::Reverse(path.components().count())); - for path in paths { - set_owner(&path, uid, gid)?; - } + let started = std::time::Instant::now(); + tracing::info!( + event = "provider.environment.ownership.started", + stage = "session_host", + entries = paths.len() + ); + let mut ordered = std::mem::take(&mut paths).into_iter().collect::>(); + ordered.sort_by_key(|path| std::cmp::Reverse(path.components().count())); + for path in ordered { + set_owner(&path,uid,gid).inspect_err(|error| {tracing::error!(event="provider.environment.ownership.failed",stage="session_host",relative_path=%path.strip_prefix(home).unwrap_or(Path::new("")).display(),elapsed_ms=started.elapsed().as_millis() as u64,operation="lchown",target_uid=uid,target_gid=gid,%error);})?; + } + tracing::info!( + event = "provider.environment.ownership.completed", + stage = "session_host", + entries = created.len(), + elapsed_ms = started.elapsed().as_millis() as u64 + ); Ok(()) } @@ -1176,50 +1196,96 @@ fn prepare_provider_bootstrap_files( run_as_uid: u32, run_as_gid: u32, ) -> Result<(), String> { - let home = PathBuf::from("/home/freshell/provider"); - if home != Path::new("/home/freshell/provider") { - return Err("managed provider HOME must be /home/freshell/provider".into()); + let home = Path::new("/home/freshell/provider"); + let created = + write_provider_bootstrap_files(files, home, Path::new("/run/freshell-bootstrap"))?; + transfer_created_paths(home, &created, run_as_uid, run_as_gid) +} + +fn rollback_created_provider_paths(paths: &[PathBuf]) { + let mut paths = paths.iter().collect::>(); + paths.sort_by_key(|path| std::cmp::Reverse(path.components().count())); + for path in paths { + if let Ok(metadata) = std::fs::symlink_metadata(path) { + if metadata.is_dir() { + let _ = std::fs::remove_dir(path); + } else { + let _ = std::fs::remove_file(path); + } + } + } +} +struct BootstrapCreatedPaths { + paths: std::collections::BTreeSet, +} +impl BootstrapCreatedPaths { + fn reserve(&self, path: &Path) -> Result<(), String> { + let observed = self + .paths + .len() + .checked_add(if self.paths.contains(path) { 1 } else { 2 }) + .ok_or("created path accounting overflow")?; + if observed > freshell_platform::provider_environment::CopyLimits::default().max_entries { + return Err(format!("created provider paths exceed shared entry bound: observed {observed}, permitted 100000")); + } + Ok(()) } - if !home.is_dir() { - return Err("managed provider HOME volume is not mounted".into()); +} +impl Drop for BootstrapCreatedPaths { + fn drop(&mut self) { + rollback_created_provider_paths(&self.paths.iter().cloned().collect::>()); } +} - // Keep directories root-owned until every bootstrap file is durable. The - // trusted host deliberately lacks DAC_OVERRIDE/FOWNER, so chowning a 0700 - // parent early would lock the host out before later files are copied. - set_mode(&home, 0o700)?; - let mut provider_dirs = std::collections::BTreeSet::new(); - provider_dirs.insert(home.clone()); - +fn write_provider_bootstrap_files( + files: &[ProviderBootstrapFile], + home: &Path, + source_root: &Path, +) -> Result, String> { + if !std::fs::symlink_metadata(home) + .is_ok_and(|metadata| metadata.is_dir() && !metadata.file_type().is_symlink()) + { + return Err("managed provider HOME volume is not mounted as a physical directory".into()); + } + let mut created = BootstrapCreatedPaths { + paths: std::collections::BTreeSet::new(), + }; for (index, file) in files.iter().enumerate() { - let source = PathBuf::from(format!("/run/freshell-bootstrap/provider-{index}")); - let relative = PathBuf::from(&file.provider_relative_path); - if file.provider_relative_path.is_empty() - || relative.is_absolute() + let relative = Path::new(&file.provider_relative_path); + if relative.as_os_str().is_empty() || relative .components() - .any(|component| !matches!(component, std::path::Component::Normal(_))) + .any(|part| !matches!(part, std::path::Component::Normal(_))) { return Err("provider bootstrap destination is unsafe".into()); } - let destination = home.join(&relative); - if !destination.starts_with(&home) { - return Err("provider bootstrap destination escaped HOME".into()); - } - if let Some(relative_parent) = relative.parent() { - let mut current = home.clone(); - for component in relative_parent.components() { - if let std::path::Component::Normal(component) = component { - current.push(component); - std::fs::create_dir_all(¤t).map_err(|error| error.to_string())?; - set_mode(¤t, 0o700)?; - provider_dirs.insert(current.clone()); + let mut parent = home.to_path_buf(); + for part in relative.parent().into_iter().flat_map(Path::components) { + parent.push(part.as_os_str()); + match std::fs::symlink_metadata(&parent) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + Ok(_) => { + return Err("provider bootstrap parent must be a physical directory".into()) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + created.reserve(&parent)?; + std::fs::create_dir(&parent).map_err(|error| error.to_string())?; + created.paths.insert(parent.clone()); + set_mode(&parent, 0o700)?; } + Err(error) => return Err(error.to_string()), } } - let bytes = std::fs::read(&source) - .map_err(|error| format!("read {}: {error}", source.display()))?; - let tmp = destination.with_extension(format!("freshell-tmp-{}", std::process::id())); + let destination = home.join(relative); + created.reserve(&destination)?; + if std::fs::symlink_metadata(&destination) + .is_ok_and(|metadata| metadata.file_type().is_symlink()) + { + return Err("provider bootstrap destination cannot be a symbolic link".into()); + } + let source = source_root.join(format!("provider-{index}")); + let bytes = std::fs::read(source).map_err(|error| error.to_string())?; + let temporary = destination.with_extension(format!("freshell-tmp-{}", std::process::id())); let mut options = std::fs::OpenOptions::new(); options.create_new(true).write(true); #[cfg(unix)] @@ -1227,29 +1293,27 @@ fn prepare_provider_bootstrap_files( use std::os::unix::fs::OpenOptionsExt; options.mode(0o600); } - use std::io::Write as _; - let mut out = options - .open(&tmp) - .map_err(|error| format!("create {}: {error}", tmp.display()))?; - out.write_all(&bytes).map_err(|error| error.to_string())?; - out.sync_all().map_err(|error| error.to_string())?; - set_mode(&tmp, 0o600)?; - std::fs::rename(&tmp, &destination).map_err(|error| error.to_string())?; - set_owner(&destination, run_as_uid, run_as_gid)?; - if let Some(parent) = destination.parent() { - let _ = std::fs::File::open(parent).and_then(|dir| dir.sync_all()); - } - } - - // Once all file writes are complete, transfer directory ownership from - // deepest to shallowest; HOME is last. No trusted-host filesystem access - // under provider HOME is required after this point. - let mut provider_dirs = provider_dirs.into_iter().collect::>(); - provider_dirs.sort_by_key(|path| std::cmp::Reverse(path.components().count())); - for dir in provider_dirs { - set_owner(&dir, run_as_uid, run_as_gid)?; - } - Ok(()) + use std::io::Write; + let result = (|| { + let mut output = options + .open(&temporary) + .map_err(|error| error.to_string())?; + output + .write_all(&bytes) + .map_err(|error| error.to_string())?; + output.sync_all().map_err(|error| error.to_string())?; + std::fs::rename(&temporary, &destination).map_err(|error| error.to_string()) + })(); + if result.is_err() { + let _ = std::fs::remove_file(&temporary); + } + result?; + created.paths.insert(destination); + } + // The exact mounted HOME is controller-created and is handed off last. + let mut paths = std::mem::take(&mut created.paths); + paths.insert(home.to_owned()); + Ok(paths.into_iter().collect()) } fn copy_provider_config_references( @@ -1268,16 +1332,9 @@ fn copy_provider_config_references( user_provider_root, provider_home, (run_as_uid, run_as_gid), - ProviderConfigOwnership::Immediate, ) } -#[derive(Clone, Copy, PartialEq, Eq)] -enum ProviderConfigOwnership { - Immediate, - DeferAmplifierTree, -} - fn copy_provider_config_references_with_ownership( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -1285,75 +1342,75 @@ fn copy_provider_config_references_with_ownership( user_provider_root: &Path, provider_home: &Path, run_as: (u32, u32), - ownership: ProviderConfigOwnership, ) -> Result<(), String> { + use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; use freshell_runtime_protocol::ProviderConfigRoot; - let (run_as_uid, run_as_gid) = run_as; context.validate(provider).map_err(|error| error.message)?; - let mut copied = Vec::new(); - let mut entries = 0usize; - for reference in provider_config_references(context) { - let source_root = match reference.root { - ProviderConfigRoot::Workspace => workspace.to_path_buf(), - ProviderConfigRoot::UserProvider => user_provider_root.to_path_buf(), - ProviderConfigRoot::Ephemeral => user_provider_root.join("ephemeral"), - }; - let source = source_root.join(&reference.relative_path); - let destination = provider_home.join(&reference.provider_relative_path); - if !destination.starts_with(provider_home) || !source.starts_with(&source_root) { - return Err("provider config escaped its approved root".into()); - } - copy_provider_config_entry( - &source_root, - &source, - &destination, - &mut copied, - &mut entries, - 0, - )?; - } - let projected = copied + let selections = provider_config_references(context) + .into_iter() + .map(|reference| { + let root = match reference.root { + ProviderConfigRoot::Workspace => workspace.to_path_buf(), + ProviderConfigRoot::UserProvider => user_provider_root.to_path_buf(), + ProviderConfigRoot::Ephemeral => user_provider_root.join("ephemeral"), + }; + CopySelection { + source: root.join(&reference.relative_path), + destination_relative: reference.provider_relative_path.clone().into(), + } + }) + .collect::>(); + let started = std::time::Instant::now(); + tracing::info!( + event = "provider.environment.copy.started", + stage = "session_host", + provider, + selections = selections.len() + ); + let excluded = provider_credential_sources(user_provider_root); + let report=copy_environment(&selections,provider_home,&excluded,CopyLimits::default()).map_err(|error| { + tracing::error!(event="provider.environment.copy.failed",stage="session_host",provider,relative_path=%error.relative_path.display(),operation=error.operation,limit_kind=error.limit_kind,observed=error.observed,permitted=error.permitted,corrective_message=%error.corrective_message,elapsed_ms=started.elapsed().as_millis() as u64); + error.to_string() + })?; + tracing::info!( + event = "provider.environment.copy.completed", + stage = "session_host", + provider, + entries = report.entries, + bytes = report.bytes, + elapsed_ms = started.elapsed().as_millis() as u64 + ); + let projected = report + .projected_paths .iter() - .filter_map(|path| path.strip_prefix(provider_home).ok()) - .filter(|path| !path.as_os_str().is_empty()) .map(|path| path.to_string_lossy().into_owned()) .collect::>(); - // A projected file can create several parent directories. Transfer all - // of them, including `.freshell` above an OpenCode TUI source, so the - // provider can create sibling state such as session-switch signals. - let mut ownership_paths = std::collections::BTreeSet::new(); - for path in copied { - for ancestor in path.ancestors() { - if ancestor == provider_home { - break; - } - if ancestor.starts_with(provider_home) { - ownership_paths.insert(ancestor.to_path_buf()); - } - } - } - let mut ownership_paths = ownership_paths.into_iter().collect::>(); + let mut ownership_paths = report + .projected_paths + .into_iter() + .map(|path| provider_home.join(path)) + .collect::>(); ownership_paths.sort_by_key(|path| std::cmp::Reverse(path.components().count())); for path in ownership_paths { - // Amplifier's first launch transfers the complete stub and projected - // config in one bounded walk. Chowning a selected 0700 directory here - // would make that later walk fail without DAC_OVERRIDE. - if should_transfer_projected_path(&path, provider_home, ownership) { - set_owner(&path, run_as_uid, run_as_gid)?; - } + set_owner(&path, run_as.0, run_as.1)?; } let manifest = write_projection_manifest(provider_home, &projected)?; - set_owner(&manifest, run_as_uid, run_as_gid)?; + set_owner(&manifest, run_as.0, run_as.1)?; Ok(()) } - -fn should_transfer_projected_path( - path: &Path, - provider_home: &Path, - ownership: ProviderConfigOwnership, -) -> bool { - ownership != ProviderConfigOwnership::DeferAmplifierTree - || !path.starts_with(provider_home.join(".amplifier")) +fn provider_credential_sources(root: &Path) -> Vec { + [ + ".credentials.json", + "auth.json", + "keys.env", + ".env", + "credentials", + "secrets", + "token.json", + ] + .into_iter() + .filter_map(|name| std::fs::canonicalize(root.join(name)).ok()) + .collect() } const PROVIDER_PROJECTION_MANIFEST: &str = ".freshell-config-projection.json"; @@ -1505,17 +1562,26 @@ fn provider_identity_args(uid: u32, gid: u32) -> Vec { fn allow_provider_read_stage(path: &Path, gid: u32) -> Result<(), String> { let metadata = std::fs::symlink_metadata(path).map_err(|error| error.to_string())?; + + set_owner(path, unsafe { libc::geteuid() }, gid)?; if metadata.file_type().is_symlink() { - return Err("staged provider config cannot be a symlink".into()); + return Ok(()); } - set_owner(path, unsafe { libc::geteuid() }, gid)?; if metadata.is_dir() { set_mode(path, 0o750)?; for entry in std::fs::read_dir(path).map_err(|error| error.to_string())? { allow_provider_read_stage(&entry.map_err(|error| error.to_string())?.path(), gid)?; } } else if metadata.is_file() { - set_mode(path, 0o640)?; + { + use std::os::unix::fs::PermissionsExt; + set_mode( + path, + freshell_platform::provider_environment::provider_read_stage_mode( + metadata.permissions().mode(), + ), + )?; + } } else { return Err("staged provider config must be a regular file or directory".into()); } @@ -1549,11 +1615,6 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() || path .components() .any(|part| !matches!(part, std::path::Component::Normal(_))) - || path.components().any(|part| { - part.as_os_str() - .to_str() - .is_some_and(is_provider_secret_component) - }) { return Err("provider config refresh destination is unsafe".into()); } @@ -1592,22 +1653,23 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() } } paths.sort_by_key(|path| path.components().count()); - let mut copied = Vec::new(); - let mut entries = 0; - for path in &paths { - let destination = provider_home.join(path); - let source = projection.join(path); - if source.exists() { - copy_provider_config_entry( - &projection, - &source, - &destination, - &mut copied, - &mut entries, - 0, - )?; - } - } + let selections = paths + .iter() + .filter(|path| std::fs::symlink_metadata(projection.join(path)).is_ok()) + .map( + |path| freshell_platform::provider_environment::CopySelection { + source: projection.join(path), + destination_relative: path.clone(), + }, + ) + .collect::>(); + freshell_platform::provider_environment::copy_environment( + &selections, + provider_home, + &[], + Default::default(), + ) + .map_err(|error| error.to_string())?; write_projection_manifest(provider_home, ¤t)?; Ok(()) } @@ -1626,15 +1688,12 @@ fn prepare_provider_context( run_as_uid, run_as_gid, &ProviderConfigPaths::default(), - ProviderConfigOwnership::Immediate, ) } - fn prepare_provider_context_for_fresh_launch( terminal: &TerminalLaunchSpec, paths: &ProviderConfigPaths, ) -> Result<(), String> { - let ownership = first_launch_config_ownership(&terminal.mode); prepare_provider_context_at( terminal.provider_launch_context.as_ref(), &terminal.mode, @@ -1642,18 +1701,8 @@ fn prepare_provider_context_for_fresh_launch( terminal.run_as_uid, terminal.run_as_gid, paths, - ownership, ) } - -fn first_launch_config_ownership(provider: &str) -> ProviderConfigOwnership { - if provider == "amplifier" { - ProviderConfigOwnership::DeferAmplifierTree - } else { - ProviderConfigOwnership::Immediate - } -} - fn prepare_provider_context_at( context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, provider: &str, @@ -1661,44 +1710,53 @@ fn prepare_provider_context_at( run_as_uid: u32, run_as_gid: u32, paths: &ProviderConfigPaths, - ownership: ProviderConfigOwnership, ) -> Result<(), String> { - let private_mount_parent = &paths.private_mount; - if !private_mount_parent.exists() { - if context.is_some_and(|context| { - provider_config_references(context).iter().any(|reference| { - reference.root == freshell_runtime_protocol::ProviderConfigRoot::UserProvider - }) - }) { - return Err("provider user config mount is unavailable".into()); - } - if let Some(context) = context { - return copy_provider_config_references_with_ownership( - context, - provider, - Path::new(workspace), - private_mount_parent, - &paths.provider_home, - (run_as_uid, run_as_gid), - ownership, - ); - } + let Some(context) = context else { return Ok(()); + }; + let user_root = paths.private_mount.join("user-provider"); + if provider_config_references(context).iter().any(|reference| { + reference.root == freshell_runtime_protocol::ProviderConfigRoot::UserProvider + }) && !user_root.is_dir() + { + return Err("provider user config mount is unavailable".into()); } - std::fs::create_dir_all(private_mount_parent).map_err(|error| error.to_string())?; - set_mode(private_mount_parent, 0o700)?; - if let Some(context) = context { - copy_provider_config_references_with_ownership( - context, - provider, - Path::new(workspace), - &private_mount_parent.join("user-provider"), - &paths.provider_home, - (run_as_uid, run_as_gid), - ownership, - )?; + if user_root.exists() { + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + let metadata = + std::fs::symlink_metadata(&user_root).map_err(|error| error.to_string())?; + if metadata.file_type().is_symlink() + || metadata.uid() != unsafe { libc::geteuid() } + || metadata.gid() != unsafe { libc::getegid() } + { + return Err("provider input bind does not map to trusted uid/group 0; stage with the same supported rootless daemon account and retry".into()); + } + } } - Ok(()) + // The trusted host's private bind parent permits provider group 0 traversal + // without exposing the separate private control/grant directories. + if paths.private_mount.exists() { + let metadata = + std::fs::symlink_metadata(&paths.private_mount).map_err(|error| error.to_string())?; + if !metadata.is_dir() || metadata.file_type().is_symlink() { + return Err("provider private bind parent must be a trusted physical directory".into()); + } + set_owner(&paths.private_mount, unsafe { libc::geteuid() }, unsafe { + libc::getegid() + })?; + set_mode(&paths.private_mount, 0o710)?; + } + refresh_provider_config_references( + context, + provider, + Path::new(workspace), + &user_root, + &paths.provider_home, + run_as_uid, + run_as_gid, + ) } fn prepare_provider_files( @@ -1807,121 +1865,40 @@ fn run_provider_path_command( } } +#[cfg(test)] fn copy_provider_config_entry( - source_root: &Path, + _source_root: &Path, source: &Path, destination: &Path, copied: &mut Vec, entries: &mut usize, - depth: usize, + _depth: usize, ) -> Result<(), String> { - if depth > 32 || *entries >= 4096 { - return Err("provider config copy exceeds bounds".into()); - } - *entries += 1; - if source.components().any(|component| { - component - .as_os_str() - .to_str() - .is_some_and(is_provider_secret_component) - }) { - return Err("provider config references a credential path".into()); - } - let Some(source) = resolve_provider_config_source(source_root, source)? else { - return Ok(()); - }; - if source.components().any(|component| { - component - .as_os_str() - .to_str() - .is_some_and(is_provider_secret_component) - }) { - return Err("provider config resolves to a credential path".into()); - } - let metadata = std::fs::metadata(&source) - .map_err(|error| format!("provider config source unavailable: {error}"))?; - if let Ok(existing) = std::fs::symlink_metadata(destination) { - if existing.file_type().is_symlink() { - return Err("provider config destination cannot be a symlink".into()); - } - } - if metadata.is_dir() { - std::fs::create_dir_all(destination).map_err(|error| error.to_string())?; - set_mode(destination, 0o700)?; - copied.push(destination.to_path_buf()); - for entry in std::fs::read_dir(&source).map_err(|error| error.to_string())? { - let entry = entry.map_err(|error| error.to_string())?; - let name = entry.file_name(); - if name.to_str().is_some_and(is_provider_secret_component) { - continue; - } - copy_provider_config_entry( - source_root, - &entry.path(), - &destination.join(name), - copied, - entries, - depth + 1, - )?; - } - } else if metadata.is_file() { - if metadata.len() > 16 * 1024 * 1024 { - return Err("provider config file exceeds bounds".into()); - } - if let Some(parent) = destination.parent() { - std::fs::create_dir_all(parent).map_err(|error| error.to_string())?; - copied.push(parent.to_path_buf()); - } - std::fs::copy(&source, destination).map_err(|error| error.to_string())?; - set_mode(destination, 0o600)?; - copied.push(destination.to_path_buf()); - } else { - return Err("provider config source is not a regular file or directory".into()); - } - Ok(()) -} - -/// Follow ordinary config symlinks only while every traversed component stays -/// inside its approved root. External links and vanished entries are omitted. -fn resolve_provider_config_source(root: &Path, source: &Path) -> Result, String> { - let canonical_root = std::fs::canonicalize(root) - .map_err(|error| format!("provider config root unavailable: {error}"))?; - let relative = source - .strip_prefix(root) - .map_err(|_| "provider config escaped its approved root")?; - let mut current = canonical_root.clone(); - for component in relative.components() { - if !matches!(component, std::path::Component::Normal(_)) { - return Err("provider config path is unsafe".into()); - } - current.push(component.as_os_str()); - current = match std::fs::canonicalize(¤t) { - Ok(path) => path, - Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), - Err(error) => return Err(format!("provider config source unavailable: {error}")), - }; - if !current.starts_with(&canonical_root) { - tracing::warn!( - event = "provider.config.skipped_symlink_escape", - "provider config symlink leaves approved root" - ); - return Ok(None); - } - } - Ok(Some(current)) -} - -fn is_provider_secret_component(value: &str) -> bool { - matches!( - value.to_ascii_lowercase().as_str(), - ".credentials.json" - | "auth.json" - | "keys.env" - | ".env" - | "credentials" - | "secrets" - | "token.json" + use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; + let root = destination + .parent() + .ok_or("provider destination has no parent")?; + let report = copy_environment( + &[CopySelection { + source: source.to_owned(), + destination_relative: destination + .file_name() + .ok_or("provider destination has no name")? + .into(), + }], + root, + &[], + CopyLimits::default(), ) + .map_err(|error| error.to_string())?; + *entries += report.entries; + copied.extend( + report + .projected_paths + .into_iter() + .map(|path| root.join(path)), + ); + Ok(()) } #[cfg(unix)] @@ -1930,12 +1907,15 @@ fn set_owner(path: &Path, uid: u32, gid: u32) -> Result<(), String> { let path_c = CString::new(path.as_os_str().as_bytes()) .map_err(|_| format!("path contains NUL: {}", path.display()))?; // SAFETY: path_c is NUL terminated and remains alive for the call. - let rc = unsafe { libc::chown(path_c.as_ptr(), uid, gid) }; + let rc = unsafe { libc::lchown(path_c.as_ptr(), uid, gid) }; if rc == 0 { + #[cfg(test)] + tests::created_path_ownership::OPERATIONS + .with(|operations| operations.borrow_mut().push(path.to_path_buf())); Ok(()) } else { Err(format!( - "chown {}: {}", + "lchown {}: {}", path.display(), std::io::Error::last_os_error() )) @@ -2567,8 +2547,29 @@ fn now_millis() -> i64 { mod tests { use super::*; + #[cfg(unix)] + #[test] + fn provider_config_two_stage_script_is_runnable() { + use std::{os::unix::fs::PermissionsExt, process::Command}; + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + std::fs::create_dir(&source).unwrap(); + std::fs::write(source.join("run"), "#!/bin/sh\nprintf resource-nonce\n").unwrap(); + std::fs::set_permissions(source.join("run"), std::fs::Permissions::from_mode(0o700)) + .unwrap(); + let stage = root.path().join("stage"); + let home = root.path().join("home"); + copy_provider_config_entry(&source, &source, &stage, &mut Vec::new(), &mut 0, 0).unwrap(); + copy_provider_config_entry(&stage, &stage, &home, &mut Vec::new(), &mut 0, 0).unwrap(); + let output = Command::new(home.join("run")) + .output() + .expect("two-stage executable spawn"); + assert!(output.status.success()); + assert_eq!(output.stdout, b"resource-nonce"); + } + #[test] - fn provider_secret_config_copy_preserves_settings_and_rejects_auth_files() { + fn provider_config_copy_preserves_package_resources_and_rejects_selected_credentials() { use freshell_runtime_protocol::{ ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, }; @@ -2616,7 +2617,10 @@ mod tests { b"{\"theme\":\"dark\"}" ); assert!(destination.join(".claude/plugins/ordinary.js").is_file()); - assert!(!destination.join(".claude/plugins/auth.json").exists()); + assert_eq!( + std::fs::read(destination.join(".claude/plugins/auth.json")).unwrap(), + b"test-secret-value" + ); assert!(!destination.join(".claude/.credentials.json").exists()); assert!(copy_provider_config_references( @@ -2645,7 +2649,7 @@ mod tests { #[cfg(unix)] #[test] - fn provider_config_copy_keeps_in_root_symlinks_and_skips_escaping_intermediate_link() { + fn provider_config_copy_keeps_selected_external_and_in_root_resources() { use freshell_runtime_protocol::{ ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, }; @@ -2694,9 +2698,10 @@ mod tests { unsafe { libc::getegid() }, ) .unwrap(); - assert!(!destination - .join(".config/opencode/project/escaped.json") - .exists()); + assert_eq!( + std::fs::read(destination.join(".config/opencode/project/escaped.json")).unwrap(), + b"outside-secret" + ); assert_eq!( std::fs::read(destination.join(".config/opencode/project/allowed.json")).unwrap(), b"inside" @@ -2704,7 +2709,7 @@ mod tests { } #[test] - fn exact_resume_refreshes_changed_config_and_removes_stale_projection() { + fn provider_config_exact_resume_refreshes_changes_and_removes_stale_projection() { use freshell_runtime_protocol::{ ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, }; @@ -2715,7 +2720,7 @@ mod tests { std::fs::create_dir_all(&destination).unwrap(); std::fs::write(source.join("settings.json"), b"first").unwrap(); std::fs::write(source.join("plugins/old.js"), b"old").unwrap(); - let context = ProviderLaunchContext { + let mut context = ProviderLaunchContext { preparation: ProviderPreparation::Claude { mcp_args: Vec::new(), }, @@ -2781,6 +2786,28 @@ mod tests { b"durable" ); std::fs::remove_file(source.join("settings.json")).unwrap(); + let error = refresh_provider_config_references( + &context, + "claude", + root.path(), + &source, + &destination, + uid, + gid, + ) + .unwrap_err(); + assert!(error.contains("settings.json") && error.contains("repair")); + assert_eq!( + std::fs::read(destination.join(".claude/settings.json")).unwrap(), + b"second" + ); + assert_eq!( + std::fs::read(destination.join(".claude/plugins/provider-created.js")).unwrap(), + b"durable" + ); + context + .config + .retain(|reference| reference.relative_path != "settings.json"); refresh_provider_config_references( &context, "claude", @@ -3496,61 +3523,201 @@ mod tests { format!("{}-{}", std::process::id(), now_millis()) } - #[test] - fn host_created_provider_tree_transfers_exact_paths_and_refuses_symlinks() { - use std::os::unix::fs::symlink; - - let dir = tempfile::tempdir().unwrap(); - let tree = dir.path().join("provider/.amplifier/projects/p/sessions/s"); - std::fs::create_dir_all(&tree).unwrap(); - std::fs::write(tree.join("metadata.json"), b"{}").unwrap(); - transfer_owned_tree( - &dir.path().join("provider/.amplifier"), - unsafe { libc::geteuid() }, - unsafe { libc::getegid() }, - 32, - ) - .unwrap(); - - let outside = dir.path().join("outside"); - std::fs::write(&outside, b"sentinel").unwrap(); - symlink(&outside, tree.join("escape")).unwrap(); - let error = transfer_owned_tree( - &dir.path().join("provider/.amplifier"), - unsafe { libc::geteuid() }, - unsafe { libc::getegid() }, - 32, - ) - .unwrap_err(); - assert!(error.contains("refuses symlink")); - assert_eq!(std::fs::read(&outside).unwrap(), b"sentinel"); + pub(crate) mod created_path_ownership { + use super::*; + use std::{ + cell::RefCell, + collections::BTreeSet, + os::unix::fs::{symlink, PermissionsExt}, + process::Command, + }; + thread_local! {pub(crate) static OPERATIONS: RefCell> = const {RefCell::new(Vec::new())};} + fn handoff(home: &Path, ledger: &[PathBuf]) -> Result<(), String> { + transfer_created_paths(home, ledger, unsafe { libc::geteuid() }, unsafe { + libc::getegid() + }) + } + #[test] + fn all_selected_entries_above_ten_thousand_remain_runnable() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + std::fs::create_dir(&source).unwrap(); + for index in 0..10_005 { + std::fs::write(source.join(format!("resource-{index}")), "nonce").unwrap(); + } + std::fs::write(source.join("run"), "#!/bin/sh\nprintf runnable").unwrap(); + std::fs::set_permissions(source.join("run"), std::fs::Permissions::from_mode(0o700)) + .unwrap(); + let home = root.path().join("provider"); + freshell_platform::provider_environment::copy_environment( + &[freshell_platform::provider_environment::CopySelection { + source, + destination_relative: "selected".into(), + }], + &home, + &[], + Default::default(), + ) + .unwrap(); + let ensured = freshell_sessions::amplifier_stub::ensure_session( + &home.join(".amplifier"), + "new-session", + root.path().to_str().unwrap(), + "terminal", + ) + .unwrap(); + let inputs = root.path().join("bootstrap-inputs"); + std::fs::create_dir(&inputs).unwrap(); + std::fs::write(inputs.join("provider-0"), "{}").unwrap(); + let mut ledger = ensured.created_paths; + ledger.extend( + write_provider_bootstrap_files( + &[ProviderBootstrapFile { + source_path: inputs.join("provider-0").to_string_lossy().into_owned(), + provider_relative_path: "bootstrap/config.json".into(), + }], + &home, + &inputs, + ) + .unwrap(), + ); + assert!(ledger.contains(&home.join("bootstrap/config.json"))); + assert!(ledger.contains(&home.join("bootstrap"))); + handoff(&home, &ledger).unwrap(); + assert_eq!( + std::fs::read_dir(home.join("selected")).unwrap().count(), + 10_006 + ); + let output = Command::new(home.join("selected/run")).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"runnable"); + } + #[test] + fn adopted_state_and_siblings_grant_no_recursive_authority() { + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("provider"); + let amplifier = home.join(".amplifier"); + std::fs::create_dir_all(&lifier).unwrap(); + std::fs::write(amplifier.join("sibling"), "sentinel").unwrap(); + let first = freshell_sessions::amplifier_stub::ensure_session( + &lifier, + "adopted", + root.path().to_str().unwrap(), + "terminal", + ) + .unwrap(); + let adopted = freshell_sessions::amplifier_stub::ensure_session( + &lifier, + "adopted", + root.path().to_str().unwrap(), + "terminal", + ) + .unwrap(); + assert!(!adopted.created); + OPERATIONS.with(|operations| operations.borrow_mut().clear()); + handoff(&home, &adopted.created_paths).unwrap(); + OPERATIONS.with(|operations| { + assert!( + operations.borrow().is_empty(), + "adoption cannot transfer sibling or stub ownership: {:?}", + *operations.borrow() + ) + }); + assert_eq!( + std::fs::read(amplifier.join("sibling")).unwrap(), + b"sentinel" + ); + assert!(first.session_dir.exists()); + } + #[test] + fn partial_bootstrap_failure_leaves_no_created_ownership_authority() { + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("provider"); + let inputs = root.path().join("inputs"); + std::fs::create_dir(&home).unwrap(); + std::fs::create_dir(&inputs).unwrap(); + std::fs::write(home.join("sibling"), "sentinel").unwrap(); + std::fs::write(inputs.join("provider-0"), "{}").unwrap(); + let files = [ + ProviderBootstrapFile { + source_path: "fixture".into(), + provider_relative_path: "bootstrap/first.json".into(), + }, + ProviderBootstrapFile { + source_path: "missing-fixture".into(), + provider_relative_path: "bootstrap/second.json".into(), + }, + ]; + assert!(write_provider_bootstrap_files(&files, &home, &inputs).is_err()); + assert!(!home.join("bootstrap").exists()); + assert_eq!(std::fs::read(home.join("sibling")).unwrap(), b"sentinel"); + } + + #[test] + fn physical_handoff_records_only_created_inodes_and_never_follows_links() { + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("provider"); + std::fs::create_dir(&home).unwrap(); + let outside = root.path().join("outside"); + std::fs::write(&outside, "sentinel").unwrap(); + use std::os::unix::fs::MetadataExt; + let before = std::fs::metadata(&outside).unwrap(); + let leaf = home.join("bootstrap.json"); + std::fs::write(&leaf, "{}").unwrap(); + let link = home.join("selected-link"); + symlink(&outside, &link).unwrap(); + let ledger = vec![leaf.clone(), link.clone(), home.clone()]; + OPERATIONS.with(|operations| operations.borrow_mut().clear()); + handoff(&home, &ledger).unwrap(); + OPERATIONS.with(|operations| { + let operations = operations.borrow(); + assert_eq!( + operations.iter().cloned().collect::>(), + ledger.into_iter().collect() + ); + assert_eq!(operations.last(), Some(&home)); + }); + assert_eq!(std::fs::read(&outside).unwrap(), b"sentinel"); + let after = std::fs::metadata(&outside).unwrap(); + assert_eq!( + (after.uid(), after.gid(), after.ctime(), after.ctime_nsec()), + ( + before.uid(), + before.gid(), + before.ctime(), + before.ctime_nsec() + ), + "physical lchown must leave the target inode untouched" + ); + } } #[cfg(unix)] #[test] - fn first_amplifier_launch_transfers_stub_and_projected_config_together() { + fn provider_config_first_amplifier_launch_transfers_exact_created_state_before_copy() { use freshell_runtime_protocol::{ ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, }; - use std::{cell::Cell, os::unix::fs::PermissionsExt, process::Command}; - + use std::{cell::Cell, process::Command}; let root = tempfile::tempdir().unwrap(); - let workspace = root.path().join("workspace"); - let user_provider_root = root.path().join("private/user-provider"); - let provider_home = root.path().join("provider"); - let amplifier_home = provider_home.join(".amplifier"); - let paths = ProviderConfigPaths { - private_mount: root.path().join("private"), - provider_home: provider_home.clone(), - }; - let selected_bundle = user_provider_root.join("bundles/ordinary"); - std::fs::create_dir_all(&workspace).unwrap(); - std::fs::create_dir_all(&selected_bundle).unwrap(); - std::fs::create_dir_all(&provider_home).unwrap(); - std::fs::write(selected_bundle.join("bundle.yaml"), b"name: ordinary\n").unwrap(); - std::fs::write(selected_bundle.join("auth.json"), b"fixture-secret").unwrap(); - - let context = ProviderLaunchContext { + let home = root.path().join("provider"); + let private = root.path().join("private"); + let source = private.join("user-provider"); + std::fs::create_dir_all(source.join("skills/fixture")).unwrap(); + std::fs::create_dir(&home).unwrap(); + std::fs::write( + source.join("skills/fixture/run"), + "#!/bin/sh\nprintf runnable", + ) + .unwrap(); + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions( + source.join("skills/fixture/run"), + std::fs::Permissions::from_mode(0o700), + ) + .unwrap(); + let mut terminal:TerminalLaunchSpec=serde_json::from_value(serde_json::json!({"terminalId":"first-amplifier","streamId":"stream","mode":"amplifier","program":"amplifier","args":[],"cwd":root.path(),"runAsUid":unsafe{libc::geteuid()},"runAsGid":unsafe{libc::getegid()},"cols":80,"rows":24,"projectKey":"project","workspacePath":root.path(),"resumeSessionId":"session"})).unwrap(); + terminal.provider_launch_context = Some(ProviderLaunchContext { preparation: ProviderPreparation::Amplifier { bundle: "default".into(), resume_args: Vec::new(), @@ -3558,93 +3725,48 @@ mod tests { mcp_capability: None, config: vec![ProviderConfigReference { root: ProviderConfigRoot::UserProvider, - relative_path: "bundles".into(), - provider_relative_path: ".amplifier/bundles".into(), + relative_path: "skills".into(), + provider_relative_path: ".amplifier/skills".into(), format: "directory".into(), }], - }; - let mut terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ - "terminalId":"amplifier-first-launch", "streamId":"amplifier-stream", - "mode":"amplifier", "program":"amplifier", "args":[], - "cwd":workspace, "runAsUid":unsafe { libc::geteuid() }, - "runAsGid":unsafe { libc::getegid() }, "cols":80, "rows":24, - "projectKey":"amplifier-project", "workspacePath":workspace, - "resumeSessionId":"session-first-launch" - })) - .unwrap(); - terminal.provider_launch_context = Some(context); - - // Docker's host has CHOWN but no DAC_OVERRIDE. Once the selected - // bundle is provider-owned and 0700, that host cannot walk it. The - // fixture uses one OS uid, so model the distinct host/provider DAC - // identities while the callbacks perform the real stub, ownership - // transfer, and config projection operations. - let provider_only_bundle = Cell::new(false); - let ownership_transferred = Cell::new(false); + }); + let handed_off = Cell::new(false); prepare_provider_state_before_bootstrap_with( &mut terminal, |terminal| { - freshell_sessions::amplifier_stub::ensure_session( - &lifier_home, - terminal.resume_session_id.as_deref().unwrap(), + let ensured = freshell_sessions::amplifier_stub::ensure_session( + &home.join(".amplifier"), + "session", &terminal.cwd, &terminal.terminal_id, ) - .map(|_| ()) - .map_err(|error| error.to_string()) + .map_err(|error| error.to_string())?; + let mut ledger = ensured.created_paths; + ledger.push(home.clone()); + Ok(ledger) }, - |terminal| { - if provider_only_bundle.get() { - return Err("host cannot traverse provider-owned 0700 bundle".into()); - } - transfer_owned_tree( - &lifier_home, - terminal.run_as_uid, - terminal.run_as_gid, - 10_000, - )?; - ownership_transferred.set(true); + |terminal, ledger| { + assert!(!home.join(".amplifier/skills").exists()); + transfer_created_paths(&home, ledger, terminal.run_as_uid, terminal.run_as_gid)?; + handed_off.set(true); Ok(()) }, |terminal| { - if ownership_transferred.get() { - return Err("host cannot project into a provider-owned Amplifier home".into()); - } - prepare_provider_context_for_fresh_launch(terminal, &paths)?; - let projected = amplifier_home.join("bundles"); - assert_eq!( - std::fs::metadata(&projected).unwrap().permissions().mode() & 0o777, - 0o700 - ); - provider_only_bundle.set(should_transfer_projected_path( - &projected, - &provider_home, - first_launch_config_ownership(&terminal.mode), - )); - Ok(()) + assert!(handed_off.get()); + prepare_provider_context_for_fresh_launch( + terminal, + &ProviderConfigPaths { + private_mount: private.clone(), + provider_home: home.clone(), + }, + ) }, ) - .expect("a first Amplifier launch must transfer state after an accessible projection"); - - let projected_bundle = amplifier_home.join("bundles/ordinary/bundle.yaml"); - assert!(ownership_transferred.get()); - assert!(!amplifier_home.join("bundles/ordinary/auth.json").exists()); - let output = Command::new("/usr/bin/setpriv") - .args([ - "--reuid", - &terminal.run_as_uid.to_string(), - "--regid", - &terminal.run_as_gid.to_string(), - "--keep-groups", - "--no-new-privs", - "--", - "cat", - ]) - .arg(&projected_bundle) + .unwrap(); + let output = Command::new(home.join(".amplifier/skills/fixture/run")) .output() .unwrap(); - assert!(output.status.success(), "provider child: {output:?}"); - assert_eq!(output.stdout, b"name: ordinary\n"); - assert!(amplifier_home.join("projects").is_dir()); + assert!(output.status.success()); + assert_eq!(output.stdout, b"runnable"); } } diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 3b73800b7..a01dc248f 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -33,9 +33,9 @@ pub struct PreparedTerminal { pub fn prepare_provider_state_before_bootstrap( terminal: &mut freshell_runtime_protocol::TerminalLaunchSpec, -) -> Result<(), String> { +) -> Result, String> { if terminal.mode != "amplifier" { - return Ok(()); + return Ok(Vec::new()); } let session_id = terminal .resume_session_id @@ -55,7 +55,7 @@ pub fn prepare_provider_state_before_bootstrap( if let Some(recorded_cwd) = ensured.working_dir_of_existing { terminal.cwd = recorded_cwd; } - Ok(()) + Ok(ensured.created_paths) } pub async fn prepare_terminal( diff --git a/crates/freshell-sessions/src/amplifier_stub.rs b/crates/freshell-sessions/src/amplifier_stub.rs index f00f9576e..7f59a64cf 100644 --- a/crates/freshell-sessions/src/amplifier_stub.rs +++ b/crates/freshell-sessions/src/amplifier_stub.rs @@ -85,6 +85,7 @@ pub fn resolve_amplifier_home() -> Option { /// slug, so spawning at the requested cwd would silently find nothing. #[derive(Debug, Clone)] pub struct EnsuredSession { + pub created_paths: Vec, pub session_dir: PathBuf, pub created: bool, pub found_under_divergent_slug: bool, @@ -115,11 +116,104 @@ pub struct EnsuredSession { /// resolves safely); plus empty `transcript.jsonl` and empty /// `events.jsonl` (the latter so the activity hub's create-time resolver /// attach finds a file — see the module design note). +/// Records only successful create operations; pre-existing directories never +/// confer ownership authority. Failed attempts remove only their own inodes. +fn create_directories_recorded(path: &Path, created: &mut Vec) -> std::io::Result<()> { + match std::fs::symlink_metadata(path) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => return Ok(()), + Ok(_) => { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidInput, + "session parent must be a physical directory", + )) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error), + } + if let Some(parent) = path.parent() { + create_directories_recorded(parent, created)?; + } + if created.len() >= freshell_platform::provider_environment::CopyLimits::default().max_entries { + return Err(std::io::Error::other( + "created provider paths exceed the shared entry bound", + )); + } + let mut builder = std::fs::DirBuilder::new(); + #[cfg(unix)] + { + use std::os::unix::fs::DirBuilderExt; + builder.mode(0o700); + } + match builder.create(path) { + Ok(()) => { + created.push(path.to_path_buf()); + Ok(()) + } + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + let metadata = std::fs::symlink_metadata(path)?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + Ok(()) + } else { + Err(error) + } + } + Err(error) => Err(error), + } +} +fn create_file_recorded( + path: &Path, + bytes: &[u8], + created: &mut Vec, +) -> std::io::Result<()> { + use std::io::Write; + if created.len() >= freshell_platform::provider_environment::CopyLimits::default().max_entries { + return Err(std::io::Error::other( + "created provider paths exceed the shared entry bound", + )); + } + let mut options = std::fs::OpenOptions::new(); + options.create_new(true).write(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600); + } + let mut file = options.open(path)?; + created.push(path.to_path_buf()); + file.write_all(bytes) +} +fn rollback_created_paths(created: &[PathBuf]) { + for path in created.iter().rev() { + if let Ok(metadata) = std::fs::symlink_metadata(path) { + if metadata.is_dir() { + let _ = std::fs::remove_dir(path); + } else { + let _ = std::fs::remove_file(path); + } + } + } +} + pub fn ensure_session( amplifier_home: &Path, session_id: &str, cwd: &str, terminal_id: &str, +) -> std::io::Result { + ensure_session_with_writer( + amplifier_home, + session_id, + cwd, + terminal_id, + create_file_recorded, + ) +} +fn ensure_session_with_writer( + amplifier_home: &Path, + session_id: &str, + cwd: &str, + terminal_id: &str, + mut write_file: impl FnMut(&Path, &[u8], &mut Vec) -> std::io::Result<()>, ) -> std::io::Result { // Path-safety gate (defense in depth): the id is joined into // filesystem paths (`projects//sessions/`) that this @@ -183,6 +277,7 @@ pub fn ensure_session( ); } return Ok(EnsuredSession { + created_paths: Vec::new(), session_dir: candidate, created: false, found_under_divergent_slug: divergent, @@ -196,7 +291,11 @@ pub fn ensure_session( .join(expected_slug) .join("sessions") .join(session_id); - std::fs::create_dir_all(&dir)?; + let mut created_paths = Vec::new(); + if let Err(error) = create_directories_recorded(&dir, &mut created_paths) { + rollback_created_paths(&created_paths); + return Err(error); + } let mut metadata = serde_json::json!({ "session_id": session_id, "created": chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Millis, true), @@ -220,30 +319,22 @@ pub fn ensure_session( if let Some(bundle) = crate::bundle_config::resolve_active_bundle(amplifier_home, &resolved) { metadata["bundle"] = serde_json::Value::String(bundle); } - // The three stub-file writes below can fail partway through (ENOSPC, - // permissions, ...) after create_dir_all already succeeded. On that - // path, best-effort roll back the directory rather than leaving a - // metadata-less dir behind: such a dir has no parseable metadata.json, - // so `stub_is_unused` conservatively KEEPS it forever (never GC-able), - // and a later `ensure_session` call for the same id would silently - // ADOPT it via the bare `candidate.is_dir()` check above, treating a - // half-written stub as a legitimate find. Council-scoped: ONE - // best-effort `remove_dir_all` — no atomic-write/rollback scaffolding. - // The rollback's own error is ignored; the original write error wins. let write_result: std::io::Result<()> = (|| { - std::fs::write( - dir.join("metadata.json"), - serde_json::to_string_pretty(&metadata)?, + write_file( + &dir.join("metadata.json"), + serde_json::to_string_pretty(&metadata)?.as_bytes(), + &mut created_paths, )?; - std::fs::write(dir.join("transcript.jsonl"), "")?; - std::fs::write(dir.join("events.jsonl"), "")?; + write_file(&dir.join("transcript.jsonl"), b"", &mut created_paths)?; + write_file(&dir.join("events.jsonl"), b"", &mut created_paths)?; Ok(()) })(); - if let Err(e) = write_result { - let _ = std::fs::remove_dir_all(&dir); - return Err(e); + if let Err(error) = write_result { + rollback_created_paths(&created_paths); + return Err(error); } Ok(EnsuredSession { + created_paths, session_dir: dir, created: true, found_under_divergent_slug: false, @@ -1021,6 +1112,118 @@ mod tests { assert!(meta.get("bundle").is_none()); let _ = std::fs::remove_dir_all(&home); } + mod created_path_ledger { + + use super::*; + struct Fixture(PathBuf); + impl Fixture { + fn path(&self) -> &Path { + &self.0 + } + } + impl Drop for Fixture { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.0); + } + } + fn fixture() -> Fixture { + Fixture(unique_temp_home("created-path-ledger")) + } + use std::collections::BTreeSet; + #[test] + fn created_path_ledger_records_exact_new_parents_and_leaves() { + let root = fixture(); + let home = root.path().join("provider/.amplifier"); + let ensured = + ensure_session(&home, "session", root.path().to_str().unwrap(), "terminal") + .unwrap(); + let mut expected = BTreeSet::new(); + for path in ensured + .session_dir + .ancestors() + .take_while(|path| *path != root.path()) + { + expected.insert(path.to_path_buf()); + } + for name in ["metadata.json", "transcript.jsonl", "events.jsonl"] { + expected.insert(ensured.session_dir.join(name)); + } + assert_eq!( + ensured + .created_paths + .iter() + .cloned() + .collect::>(), + expected + ); + assert!(ensured.created_paths.iter().all(|path| path.exists())); + } + #[test] + fn created_path_ledger_omits_preexisting_parents_and_adopted_siblings() { + let root = fixture(); + let home = root.path().join(".amplifier"); + std::fs::create_dir(&home).unwrap(); + std::fs::write(home.join("sibling"), "sentinel").unwrap(); + let first = ensure_session(&home, "session", root.path().to_str().unwrap(), "terminal") + .unwrap(); + assert!(!first.created_paths.contains(&home)); + assert_eq!(first.created_paths.len(), 7); + let adopted = + ensure_session(&home, "session", root.path().to_str().unwrap(), "terminal") + .unwrap(); + assert!(!adopted.created); + assert!(adopted.created_paths.is_empty()); + assert_eq!(std::fs::read(home.join("sibling")).unwrap(), b"sentinel"); + } + #[test] + fn partial_stub_write_rolls_back_only_this_attempts_created_paths() { + let root = fixture(); + let home = root.path().join(".amplifier"); + std::fs::create_dir(&home).unwrap(); + std::fs::write(home.join("sibling"), "sentinel").unwrap(); + let mut written = Vec::new(); + let result = ensure_session_with_writer( + &home, + "session", + root.path().to_str().unwrap(), + "terminal", + |path, bytes, created| { + if path.file_name().unwrap() == "events.jsonl" { + return Err(std::io::Error::other("synthetic fixture write failure")); + } + create_file_recorded(path, bytes, created)?; + written.push(path.to_owned()); + Ok(()) + }, + ); + assert!(result.is_err()); + assert_eq!(written.len(), 2); + assert!(written.iter().all(|path| !path.exists())); + assert!(!home.join("projects").exists()); + assert_eq!(std::fs::read(home.join("sibling")).unwrap(), b"sentinel"); + let retry = ensure_session(&home, "session", root.path().to_str().unwrap(), "terminal") + .unwrap(); + assert!(retry.created); + assert!(!retry.created_paths.contains(&home)); + assert!(!retry.created_paths.contains(&home.join("sibling"))); + } + #[test] + fn created_path_ledger_failed_creation_grants_no_surviving_authority() { + let root = fixture(); + let home = root.path().join(".amplifier"); + std::fs::create_dir(&home).unwrap(); + std::fs::write(home.join("projects"), "blocking-file").unwrap(); + assert!( + ensure_session(&home, "session", root.path().to_str().unwrap(), "terminal") + .is_err() + ); + assert_eq!( + std::fs::read(home.join("projects")).unwrap(), + b"blocking-file" + ); + assert_eq!(std::fs::read_dir(&home).unwrap().count(), 1); + } + } } #[cfg(test)] From 5033898b4826649597a6b5051ace6b47fe30574b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 00:08:31 -0700 Subject: [PATCH 008/493] fix: preserve provider HOME staging and bootstrap replacements --- .../src/provider_environment.rs | 120 ++++- crates/freshell-session-host/src/main.rs | 413 +++++++++++++++--- .../probe-provider-config-readonly-home.py | 138 ++++++ 3 files changed, 591 insertions(+), 80 deletions(-) create mode 100644 scripts/testing/probe-provider-config-readonly-home.py diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 42a1a3171..0fe997108 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -538,6 +538,14 @@ fn mode(path: &Path, permissions: u32, relative: &Path) -> Result<(), CopyError> Ok(()) } struct TemporaryProjection(PathBuf); +impl TemporaryProjection { + fn create(path: PathBuf) -> Result { + fs::create_dir(&path).map_err(|cause| io_error(Path::new(""), "create_stage", cause))?; + let temporary = Self(path); + mode(&temporary.0, 0o700, Path::new(""))?; + Ok(temporary) + } +} impl Drop for TemporaryProjection { fn drop(&mut self) { let _ = fs::remove_dir_all(&self.0); @@ -642,18 +650,35 @@ fn copy_environment_with_account_home( } } } - let parent = destination_root - .parent() - .ok_or_else(|| error(Path::new(""), "stage", "destination has no parent"))?; - fs::create_dir_all(parent) - .map_err(|cause| io_error(Path::new(""), "create_stage_parent", cause))?; - let temporary = TemporaryProjection(parent.join(format!( + // A mounted destination's parent may be read-only or on another filesystem. + // Keep its inode stable and stage inside the writable destination instead. + let existing_root = match fs::symlink_metadata(destination_root) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => true, + Ok(_) => { + return Err(error( + Path::new(""), + "publish", + "destination root must be a physical directory", + )) + } + Err(cause) if cause.kind() == std::io::ErrorKind::NotFound => false, + Err(cause) => return Err(io_error(Path::new(""), "inspect_destination", cause)), + }; + let parent = if existing_root { + destination_root + } else { + let parent = destination_root + .parent() + .ok_or_else(|| error(Path::new(""), "stage", "destination has no parent"))?; + fs::create_dir_all(parent) + .map_err(|cause| io_error(Path::new(""), "create_stage_parent", cause))?; + parent + }; + let temporary = TemporaryProjection::create(parent.join(format!( ".freshell-copy-{}-{}", std::process::id(), NEXT_PROJECTION.fetch_add(1, Ordering::Relaxed) - ))); - fs::create_dir(&temporary.0).map_err(|cause| io_error(Path::new(""), "create_stage", cause))?; - mode(&temporary.0, 0o700, Path::new(""))?; + )))?; let mut actual_bytes = 0u64; for (relative, node) in &plan.nodes { let destination = temporary.0.join(relative); @@ -727,7 +752,7 @@ fn copy_environment_with_account_home( } } } - if !destination_root.exists() { + if !existing_root { fs::rename(&temporary.0, destination_root) .map_err(|cause| io_error(Path::new(""), "publish", cause))?; } else { @@ -799,10 +824,7 @@ fn copy_environment_with_account_home( "existing destination type conflicts with selected resource", )) } - Ok(_) => { - fs::remove_file(&destination) - .map_err(|cause| io_error(relative, "replace_leaf", cause))?; - } + Ok(_) => {} Err(cause) if cause.kind() == std::io::ErrorKind::NotFound => {} Err(cause) => return Err(io_error(relative, "inspect_destination", cause)), } @@ -840,6 +862,76 @@ mod tests { } } + #[test] + fn failed_stage_creation_never_removes_a_preexisting_directory() { + let root = tempfile::tempdir().unwrap(); + let existing = root.path().join("existing-provider-sibling"); + fs::create_dir(&existing).unwrap(); + fs::write(existing.join("sentinel"), "provider-owned-nonce").unwrap(); + assert!(TemporaryProjection::create(existing.clone()).is_err()); + assert_eq!( + fs::read(existing.join("sentinel")).expect("failed create grants no cleanup authority"), + b"provider-owned-nonce" + ); + } + + #[cfg(unix)] + #[test] + fn existing_destination_keeps_root_and_siblings_when_staging_or_validation_fails() { + use std::os::unix::fs::MetadataExt; + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + let home = root.path().join("home"); + fs::create_dir_all(source.join("z-conflict")).unwrap(); + fs::create_dir_all(home.join("selected")).unwrap(); + fs::write(source.join("a-first"), "new-nonce").unwrap(); + fs::write(source.join("z-conflict/child"), "child-nonce").unwrap(); + fs::write(home.join("selected/a-first"), "prior-nonce").unwrap(); + fs::write(home.join("selected/z-conflict"), "existing-type").unwrap(); + fs::write(home.join("sibling"), "sibling-nonce").unwrap(); + let identity = |path: &Path| { + let metadata = fs::metadata(path).unwrap(); + ( + metadata.ino(), + metadata.uid(), + metadata.gid(), + metadata.mode(), + ) + }; + let before_home = identity(&home); + let before_sibling = identity(&home.join("sibling")); + let before_leaf = identity(&home.join("selected/a-first")); + let selected = [selection(&source, "selected")]; + assert!(copy_environment(&selected, &home, &[], CopyLimits::default()).is_err()); + assert_eq!( + fs::read(home.join("selected/a-first")).unwrap(), + b"prior-nonce" + ); + assert_eq!(identity(&home.join("selected/a-first")), before_leaf); + assert_eq!(identity(&home), before_home); + assert_eq!(identity(&home.join("sibling")), before_sibling); + assert_eq!( + fs::read_dir(&home).unwrap().count(), + 2, + "attempt stage must be removed" + ); + + fs::remove_file(home.join("selected/z-conflict")).unwrap(); + copy_environment(&selected, &home, &[], CopyLimits::default()).unwrap(); + assert_eq!( + fs::read(home.join("selected/a-first")).unwrap(), + b"new-nonce" + ); + assert_eq!( + fs::read(home.join("selected/z-conflict/child")).unwrap(), + b"child-nonce" + ); + assert_eq!(identity(&home), before_home); + assert_eq!(identity(&home.join("sibling")), before_sibling); + assert_eq!(fs::read(home.join("sibling")).unwrap(), b"sibling-nonce"); + assert_eq!(fs::read_dir(&home).unwrap().count(), 2); + } + #[cfg(unix)] #[test] fn copied_skill_executes_sibling_resources_after_two_stages() { diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index d79669582..d255dc186 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1215,15 +1215,20 @@ fn rollback_created_provider_paths(paths: &[PathBuf]) { } } } -struct BootstrapCreatedPaths { - paths: std::collections::BTreeSet, -} -impl BootstrapCreatedPaths { +struct BootstrapPublication { + // Every newly published inode needs ownership handoff, including replacements. + owned_paths: std::collections::BTreeSet, + // Only newly created pathnames grant removal authority on failure. + new_paths: Vec, + replacements: Vec<(PathBuf, PathBuf)>, +} +impl BootstrapPublication { fn reserve(&self, path: &Path) -> Result<(), String> { let observed = self - .paths + .owned_paths .len() - .checked_add(if self.paths.contains(path) { 1 } else { 2 }) + .checked_add(usize::from(!self.owned_paths.contains(path))) + .and_then(|entries| entries.checked_add(1)) // mounted HOME handoff .ok_or("created path accounting overflow")?; if observed > freshell_platform::provider_environment::CopyLimits::default().max_entries { return Err(format!("created provider paths exceed shared entry bound: observed {observed}, permitted 100000")); @@ -1231,9 +1236,14 @@ impl BootstrapCreatedPaths { Ok(()) } } -impl Drop for BootstrapCreatedPaths { +impl Drop for BootstrapPublication { fn drop(&mut self) { - rollback_created_provider_paths(&self.paths.iter().cloned().collect::>()); + rollback_created_provider_paths(&self.new_paths); + for (destination, backup) in self.replacements.iter().rev() { + if let Err(error) = std::fs::rename(backup, destination) { + tracing::error!(event = "provider.bootstrap.rollback.failed", stage = "session_host", operation = "restore_replacement", path = %destination.display(), %error); + } + } } } @@ -1242,15 +1252,25 @@ fn write_provider_bootstrap_files( home: &Path, source_root: &Path, ) -> Result, String> { + write_provider_bootstrap_files_with_publish(files, home, source_root, |source, destination| { + std::fs::rename(source, destination) + }) +} + +fn write_provider_bootstrap_files_with_publish( + files: &[ProviderBootstrapFile], + home: &Path, + source_root: &Path, + mut publish: impl FnMut(&Path, &Path) -> std::io::Result<()>, +) -> Result, String> { + use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; if !std::fs::symlink_metadata(home) .is_ok_and(|metadata| metadata.is_dir() && !metadata.file_type().is_symlink()) { return Err("managed provider HOME volume is not mounted as a physical directory".into()); } - let mut created = BootstrapCreatedPaths { - paths: std::collections::BTreeSet::new(), - }; - for (index, file) in files.iter().enumerate() { + let mut destinations = Vec::with_capacity(files.len()); + for file in files { let relative = Path::new(&file.provider_relative_path); if relative.as_os_str().is_empty() || relative @@ -1260,6 +1280,71 @@ fn write_provider_bootstrap_files( return Err("provider bootstrap destination is unsafe".into()); } let mut parent = home.to_path_buf(); + for part in relative.parent().into_iter().flat_map(Path::components) { + parent.push(part.as_os_str()); + match std::fs::symlink_metadata(&parent) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + Ok(_) => { + return Err("provider bootstrap parent must be a physical directory".into()) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.to_string()), + } + } + let destination = home.join(relative); + match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => {} + Ok(_) => return Err("provider bootstrap destination must be a regular file".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.to_string()), + } + if destinations.iter().any(|other: &PathBuf| { + destination.starts_with(other) || other.starts_with(&destination) + }) { + return Err("provider bootstrap destinations overlap".into()); + } + destinations.push(destination); + } + // Read and bound every input before changing any selected destination. Both + // staged files and replacement backups remain on the mounted HOME filesystem. + let stage = tempfile::Builder::new() + .prefix(".freshell-bootstrap-") + .tempdir_in(home) + .map_err(|error| error.to_string())?; + let selections = files + .iter() + .enumerate() + .map(|(index, _)| { + let source = source_root.join(format!("provider-{index}")); + if !std::fs::metadata(&source) + .map_err(|error| error.to_string())? + .is_file() + { + return Err("provider bootstrap input must be a regular file".to_owned()); + } + Ok(CopySelection { + source: std::fs::canonicalize(source).map_err(|error| error.to_string())?, + destination_relative: format!("provider-{index}").into(), + }) + }) + .collect::, String>>()?; + copy_environment( + &selections, + &stage.path().join("inputs"), + &[], + CopyLimits::default(), + ) + .map_err(|error| error.to_string())?; + let mut created = BootstrapPublication { + owned_paths: std::collections::BTreeSet::new(), + new_paths: Vec::new(), + replacements: Vec::new(), + }; + for (index, destination) in destinations.iter().enumerate() { + let relative = destination + .strip_prefix(home) + .map_err(|error| error.to_string())?; + let mut parent = home.to_path_buf(); for part in relative.parent().into_iter().flat_map(Path::components) { parent.push(part.as_os_str()); match std::fs::symlink_metadata(&parent) { @@ -1270,48 +1355,42 @@ fn write_provider_bootstrap_files( Err(error) if error.kind() == std::io::ErrorKind::NotFound => { created.reserve(&parent)?; std::fs::create_dir(&parent).map_err(|error| error.to_string())?; - created.paths.insert(parent.clone()); + created.owned_paths.insert(parent.clone()); + created.new_paths.push(parent.clone()); set_mode(&parent, 0o700)?; } Err(error) => return Err(error.to_string()), } } - let destination = home.join(relative); - created.reserve(&destination)?; - if std::fs::symlink_metadata(&destination) - .is_ok_and(|metadata| metadata.file_type().is_symlink()) - { - return Err("provider bootstrap destination cannot be a symbolic link".into()); - } - let source = source_root.join(format!("provider-{index}")); - let bytes = std::fs::read(source).map_err(|error| error.to_string())?; - let temporary = destination.with_extension(format!("freshell-tmp-{}", std::process::id())); - let mut options = std::fs::OpenOptions::new(); - options.create_new(true).write(true); - #[cfg(unix)] - { - use std::os::unix::fs::OpenOptionsExt; - options.mode(0o600); - } - use std::io::Write; - let result = (|| { - let mut output = options - .open(&temporary) - .map_err(|error| error.to_string())?; - output - .write_all(&bytes) - .map_err(|error| error.to_string())?; - output.sync_all().map_err(|error| error.to_string())?; - std::fs::rename(&temporary, &destination).map_err(|error| error.to_string()) - })(); - if result.is_err() { - let _ = std::fs::remove_file(&temporary); - } - result?; - created.paths.insert(destination); - } - // The exact mounted HOME is controller-created and is handed off last. - let mut paths = std::mem::take(&mut created.paths); + created.reserve(destination)?; + let replacement = match std::fs::symlink_metadata(destination) { + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => true, + Ok(_) => return Err("provider bootstrap destination must be a regular file".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => false, + Err(error) => return Err(error.to_string()), + }; + let input = stage.path().join(format!("inputs/provider-{index}")); + set_mode(&input, 0o600)?; + // Sync each complete input before publishing it, as the prior writer did. + std::fs::File::open(&input) + .and_then(|file| file.sync_all()) + .map_err(|error| error.to_string())?; + if replacement { + let backup = stage.path().join(format!("backup-{index}")); + std::fs::rename(destination, &backup).map_err(|error| error.to_string())?; + created.replacements.push((destination.clone(), backup)); + } + publish(&input, destination).map_err(|error| error.to_string())?; + created.owned_paths.insert(destination.clone()); + if !replacement { + created.new_paths.push(destination.clone()); + } + } + // Committing leaves only the exact new inode ownership ledger. Backups are + // attempt-owned and disappear with the stage; adopted parents are excluded. + created.new_paths.clear(); + created.replacements.clear(); + let mut paths = std::mem::take(&mut created.owned_paths); paths.insert(home.to_owned()); Ok(paths.into_iter().collect()) } @@ -1636,26 +1715,9 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() } paths.sort(); paths.dedup(); - let mut stale = previous.difference(¤t).collect::>(); - stale.sort_by_key(|path| std::cmp::Reverse(Path::new(path).components().count())); - for relative in stale { - let destination = provider_home.join(relative); - if let Ok(metadata) = std::fs::symlink_metadata(&destination) { - if metadata.is_dir() { - match std::fs::remove_dir(&destination) { - Ok(()) => {} - Err(error) if error.kind() == std::io::ErrorKind::DirectoryNotEmpty => {} - Err(error) => return Err(error.to_string()), - } - } else { - std::fs::remove_file(&destination).map_err(|error| error.to_string())?; - } - } - } paths.sort_by_key(|path| path.components().count()); let selections = paths .iter() - .filter(|path| std::fs::symlink_metadata(projection.join(path)).is_ok()) .map( |path| freshell_platform::provider_environment::CopySelection { source: projection.join(path), @@ -1670,6 +1732,22 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() Default::default(), ) .map_err(|error| error.to_string())?; + let mut stale = previous.difference(¤t).collect::>(); + stale.sort_by_key(|path| std::cmp::Reverse(Path::new(path).components().count())); + for relative in stale { + let destination = provider_home.join(relative); + if let Ok(metadata) = std::fs::symlink_metadata(&destination) { + if metadata.is_dir() { + match std::fs::remove_dir(&destination) { + Ok(()) => {} + Err(error) if error.kind() == std::io::ErrorKind::DirectoryNotEmpty => {} + Err(error) => return Err(error.to_string()), + } + } else { + std::fs::remove_file(&destination).map_err(|error| error.to_string())?; + } + } + } write_projection_manifest(provider_home, ¤t)?; Ok(()) } @@ -2708,6 +2786,61 @@ mod tests { ); } + #[cfg(unix)] + #[test] + fn provider_config_failed_staged_resource_preserves_previous_projection_and_retry() { + use std::os::unix::fs::{symlink, MetadataExt}; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let stage = root.path().join("stage"); + std::fs::create_dir_all(home.join("skills")).unwrap(); + std::fs::create_dir_all(stage.join("projection/skills")).unwrap(); + std::fs::write(home.join("skills/old"), "prior-nonce").unwrap(); + std::fs::write(home.join("skills/sibling"), "sibling-nonce").unwrap(); + std::fs::write( + home.join(PROVIDER_PROJECTION_MANIFEST), + r#"["skills","skills/old"]"#, + ) + .unwrap(); + std::fs::write( + stage.join("projection").join(PROVIDER_PROJECTION_MANIFEST), + r#"["skills","skills/new"]"#, + ) + .unwrap(); + std::fs::write(stage.join("manifest.json"), r#"["skills"]"#).unwrap(); + symlink("missing", stage.join("projection/skills/new")).unwrap(); + let before = std::fs::metadata(&home).unwrap().ino(); + assert!(apply_staged_provider_config(&stage, &home).is_err()); + assert_eq!( + std::fs::read(home.join("skills/old")) + .expect("prior projection must survive failed staging"), + b"prior-nonce" + ); + assert_eq!( + read_projection_manifest(&home).unwrap(), + ["skills".to_owned(), "skills/old".to_owned()] + .into_iter() + .collect() + ); + assert_eq!( + std::fs::read(home.join("skills/sibling")).unwrap(), + b"sibling-nonce" + ); + std::fs::remove_file(stage.join("projection/skills/new")).unwrap(); + std::fs::write(stage.join("projection/skills/new"), "new-nonce").unwrap(); + apply_staged_provider_config(&stage, &home).unwrap(); + assert_eq!(std::fs::metadata(&home).unwrap().ino(), before); + assert!(!home.join("skills/old").exists()); + assert_eq!( + std::fs::read(home.join("skills/new")).unwrap(), + b"new-nonce" + ); + assert_eq!( + std::fs::read(home.join("skills/sibling")).unwrap(), + b"sibling-nonce" + ); + } + #[test] fn provider_config_exact_resume_refreshes_changes_and_removes_stale_projection() { use freshell_runtime_protocol::{ @@ -3653,6 +3786,154 @@ mod tests { assert_eq!(std::fs::read(home.join("sibling")).unwrap(), b"sentinel"); } + #[test] + fn bootstrap_later_input_failure_preserves_existing_leaf_and_allows_retry() { + use std::os::unix::fs::MetadataExt; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("provider"); + let inputs = root.path().join("inputs"); + std::fs::create_dir_all(home.join("bootstrap")).unwrap(); + std::fs::create_dir(&inputs).unwrap(); + let first = home.join("bootstrap/first.json"); + let sibling = home.join("bootstrap/sibling"); + std::fs::write(&first, "prior-config").unwrap(); + std::fs::set_permissions(&first, std::fs::Permissions::from_mode(0o640)).unwrap(); + std::fs::write(&sibling, "provider-sibling").unwrap(); + let before = std::fs::metadata(&first).unwrap(); + let sibling_before = std::fs::metadata(&sibling).unwrap(); + std::fs::write(inputs.join("provider-0"), "new-config").unwrap(); + let files = [ + ProviderBootstrapFile { + source_path: "fixture".into(), + provider_relative_path: "bootstrap/first.json".into(), + }, + ProviderBootstrapFile { + source_path: "fixture".into(), + provider_relative_path: "bootstrap/second.json".into(), + }, + ]; + assert!(write_provider_bootstrap_files(&files, &home, &inputs).is_err()); + assert_eq!( + std::fs::read(&first).expect("prior config must survive failed bootstrap"), + b"prior-config" + ); + let after = std::fs::metadata(&first).unwrap(); + assert_eq!( + (after.ino(), after.uid(), after.gid(), after.mode()), + (before.ino(), before.uid(), before.gid(), before.mode()) + ); + assert_eq!(std::fs::read(&sibling).unwrap(), b"provider-sibling"); + assert_eq!( + std::fs::metadata(&sibling).unwrap().ino(), + sibling_before.ino() + ); + assert!(!home.join("bootstrap/second.json").exists()); + std::fs::write(inputs.join("provider-1"), "second-config").unwrap(); + let owned = write_provider_bootstrap_files(&files, &home, &inputs).unwrap(); + assert!( + owned.contains(&first), + "replacement inode needs exact ownership handoff" + ); + assert!(!owned.contains(&sibling)); + assert!(!owned.contains(&home.join("bootstrap"))); + assert_eq!(std::fs::read(&first).unwrap(), b"new-config"); + assert_eq!( + std::fs::read(home.join("bootstrap/second.json")).unwrap(), + b"second-config" + ); + assert_eq!(std::fs::read(&sibling).unwrap(), b"provider-sibling"); + } + + #[test] + fn bootstrap_late_publication_failure_restores_replacements_and_removes_only_new_paths() { + use std::os::unix::fs::MetadataExt; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("provider"); + let inputs = root.path().join("inputs"); + std::fs::create_dir_all(home.join("bootstrap")).unwrap(); + std::fs::create_dir(&inputs).unwrap(); + let first = home.join("bootstrap/first.json"); + let last = home.join("bootstrap/last.json"); + let sibling = home.join("bootstrap/sibling"); + for (path, bytes, mode) in [ + (&first, "prior-first", 0o640), + (&last, "prior-last", 0o700), + (&sibling, "provider-sibling", 0o600), + ] { + std::fs::write(path, bytes).unwrap(); + std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode)).unwrap(); + } + let metadata = |path: &Path| { + let value = std::fs::metadata(path).unwrap(); + (value.ino(), value.uid(), value.gid(), value.mode()) + }; + let before = [metadata(&first), metadata(&last), metadata(&sibling)]; + let files = [ + "bootstrap/first.json", + "new-parent/second.json", + "bootstrap/last.json", + ] + .map(|relative| ProviderBootstrapFile { + source_path: "fixture".into(), + provider_relative_path: relative.into(), + }); + for index in 0..files.len() { + std::fs::write( + inputs.join(format!("provider-{index}")), + format!("new-{index}"), + ) + .unwrap(); + } + let mut published = 0; + let failure = write_provider_bootstrap_files_with_publish( + &files, + &home, + &inputs, + |source, destination| { + published += 1; + if published == 3 { + assert_eq!( + std::fs::read(&first).unwrap(), + b"new-0", + "first replacement was actually published before failure" + ); + assert_eq!( + std::fs::read(home.join("new-parent/second.json")).unwrap(), + b"new-1" + ); + return Err(std::io::Error::other("synthetic late publication failure")); + } + std::fs::rename(source, destination) + }, + ); + assert!(failure + .unwrap_err() + .contains("synthetic late publication failure")); + assert_eq!(published, 3); + assert_eq!(std::fs::read(&first).unwrap(), b"prior-first"); + assert_eq!(std::fs::read(&last).unwrap(), b"prior-last"); + assert_eq!(std::fs::read(&sibling).unwrap(), b"provider-sibling"); + assert_eq!( + [metadata(&first), metadata(&last), metadata(&sibling)], + before + ); + assert!(!home.join("new-parent").exists()); + assert_eq!( + std::fs::read_dir(&home).unwrap().count(), + 1, + "attempt staging/backup paths must be cleaned" + ); + let owned = write_provider_bootstrap_files(&files, &home, &inputs).unwrap(); + assert!(owned.contains(&first)); + assert!(owned.contains(&last)); + assert!(owned.contains(&home.join("new-parent"))); + assert!(!owned.contains(&home.join("bootstrap"))); + assert!(!owned.contains(&sibling)); + assert_eq!(std::fs::read(&first).unwrap(), b"new-0"); + assert_eq!(std::fs::read(&last).unwrap(), b"new-2"); + assert_eq!(metadata(&sibling), before[2]); + } + #[test] fn physical_handoff_records_only_created_inodes_and_never_follows_links() { let root = tempfile::tempdir().unwrap(); diff --git a/scripts/testing/probe-provider-config-readonly-home.py b/scripts/testing/probe-provider-config-readonly-home.py new file mode 100644 index 000000000..6701000bb --- /dev/null +++ b/scripts/testing/probe-provider-config-readonly-home.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +"""Exercise the production refresh helper in an owned rootless runtime container.""" + +import argparse +import hashlib +import json +import os +from pathlib import Path +import subprocess +import tempfile +import uuid + + +def docker(*args, check=True): + return subprocess.run( + ["docker", *args], check=check, text=True, capture_output=True, timeout=60 + ) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--image", required=True) + parser.add_argument("--binary", required=True, type=Path) + args = parser.parse_args() + security = json.loads(docker("info", "--format", "{{json .SecurityOptions}}").stdout) + if "name=rootless" not in security: + raise RuntimeError("This probe requires the supported rootless Docker backend") + image = docker("image", "inspect", "--format", "{{.Id}}", args.image).stdout.strip() + binary = args.binary.resolve(strict=True) + with binary.open("rb") as reader: + binary_sha256 = hashlib.file_digest(reader, "sha256").hexdigest() + owned_name = "freshell-config-probe-" + uuid.uuid4().hex + label = "com.freshell.provider-config-probe=" + owned_name + with tempfile.TemporaryDirectory(prefix=owned_name) as temporary: + stage = Path(temporary) + stage.chmod(0o750) + skills = stage / "projection/skills" + skills.mkdir(parents=True) + (skills / "new.sh").write_text('#!/bin/sh\ncat "$(dirname "$0")/resource"\n') + (skills / "new.sh").chmod(0o750) + (skills / "resource").write_text("resource-nonce\n") + (stage / "manifest.json").write_text('["skills"]') + (stage / "projection/.freshell-config-projection.json").write_text( + '["skills","skills/new.sh","skills/resource"]' + ) + broken = stage / "broken/projection/skills" + broken.mkdir(parents=True) + (broken / "bad").symlink_to("missing-resource") + (stage / "broken/manifest.json").write_text('["skills"]') + (stage / "broken/projection/.freshell-config-projection.json").write_text( + '["skills","skills/bad"]' + ) + for path in stage.rglob("*"): + if path.is_symlink(): + continue + if path.is_dir(): + path.chmod(0o750) + elif not os.access(path, os.X_OK): + path.chmod(0o640) + docker("volume", "create", "--label", label, owned_name) + try: + docker( + "create", "--name", owned_name, "--label", label, + "--network", "none", "--read-only", "--cap-drop", "ALL", + "--cap-add", "CHOWN", "--cap-add", "SETUID", "--cap-add", "SETGID", + "--security-opt", "no-new-privileges:true", "--pids-limit", "64", + "--memory", "256m", "--cpus", "1", + "--tmpfs", "/tmp:rw,noexec,nosuid,nodev,size=16m,mode=1777", + "--mount", f"type=volume,src={owned_name},dst=/home/freshell/provider", + "--mount", f"type=bind,src={stage},dst=/probe-stage,readonly", + "--mount", f"type=bind,src={binary},dst=/runtime/freshell-session-host,readonly", + "--entrypoint", "/bin/sh", image, "-ceu", r''' +home=/home/freshell/provider +mkdir "$home/skills" +printf prior-nonce > "$home/skills/old.sh" +printf sibling-nonce > "$home/skills/provider-sibling" +printf '["skills","skills/old.sh"]' > "$home/.freshell-config-projection.json" +chmod 700 "$home" "$home/skills" +chmod 640 "$home/skills/provider-sibling" +chown -R 65534:0 "$home" +setpriv --reuid 65534 --regid 0 --clear-groups --no-new-privs -- /bin/sh -ceu ' +home=/home/freshell/provider +test "$(id -u)" = 65534 +test "$(id -g)" = 0 +test "$(sed -n "s/^CapEff:[[:space:]]*//p" /proc/self/status)" = 0000000000000000 +before_home=$(stat -c "%i:%u:%g:%a" "$home") +before_sibling=$(stat -c "%i:%u:%g:%a" "$home/skills/provider-sibling") +/runtime/freshell-session-host refresh-provider-config /probe-stage "$home" +test "$("$home/skills/new.sh")" = resource-nonce +test "$before_home" = "$(stat -c "%i:%u:%g:%a" "$home")" +test "$before_sibling" = "$(stat -c "%i:%u:%g:%a" "$home/skills/provider-sibling")" +test "$(cat "$home/skills/provider-sibling")" = sibling-nonce +test ! -e "$home/skills/old.sh" +before_manifest=$(cat "$home/.freshell-config-projection.json") +before_new=$(stat -c "%i:%u:%g:%a" "$home/skills/new.sh") +if /runtime/freshell-session-host refresh-provider-config /probe-stage/broken "$home"; then + echo "missing selected resource incorrectly succeeded" >&2 + exit 1 +fi +test "$before_manifest" = "$(cat "$home/.freshell-config-projection.json")" +test "$before_new" = "$(stat -c "%i:%u:%g:%a" "$home/skills/new.sh")" +test "$("$home/skills/new.sh")" = resource-nonce +test "$before_sibling" = "$(stat -c "%i:%u:%g:%a" "$home/skills/provider-sibling")" +/runtime/freshell-session-host refresh-provider-config /probe-stage "$home" +test "$("$home/skills/new.sh")" = resource-nonce +test "$before_home" = "$(stat -c "%i:%u:%g:%a" "$home")" +test "$before_sibling" = "$(stat -c "%i:%u:%g:%a" "$home/skills/provider-sibling")" +test -z "$(find "$home" -maxdepth 1 -name ".freshell-copy-*" -print)" +' +printf 'readonly-home-provider-helper PASS uid=65534 gid=0 caps=0 stable-home retained-sibling executable-resource stale-removed failed-refresh-preserved retry\n' +''' + ) + inspection = json.loads(docker("inspect", owned_name).stdout)[0] + assert inspection["HostConfig"]["ReadonlyRootfs"] + assert not inspection["HostConfig"]["Privileged"] + assert inspection["HostConfig"]["CapDrop"] == ["ALL"] + assert {cap.removeprefix("CAP_") for cap in inspection["HostConfig"]["CapAdd"]} == {"CHOWN", "SETUID", "SETGID"} + assert len(inspection["Mounts"]) == 3 + assert all("docker.sock" not in mount["Destination"] for mount in inspection["Mounts"]) + print(json.dumps({"probe": owned_name, "image": image, "binarySha256": binary_sha256, "rootless": True, + "readOnlyRootfs": True, "capAdd": inspection["HostConfig"]["CapAdd"], "mounts": inspection["Mounts"]}), flush=True) + outcome = docker("start", "--attach", owned_name, check=False) + print(outcome.stdout, end="", flush=True) + print(outcome.stderr, end="", flush=True) + state = json.loads(docker("inspect", owned_name).stdout)[0]["State"] + if state["ExitCode"] != 0: + raise RuntimeError(f"Provider helper container exited {state['ExitCode']}") + finally: + docker("rm", "--force", owned_name, check=False) + docker("volume", "rm", owned_name) + assert docker("container", "inspect", owned_name, check=False).returncode != 0 + assert docker("volume", "inspect", owned_name, check=False).returncode != 0 + print(json.dumps({"probe": owned_name, "ownedContainerRemoved": True, + "ownedVolumeRemoved": True}), flush=True) + + +if __name__ == "__main__": + main() From d39188f1ff054e2034b10a5b63b8c4cad7313f1c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 00:30:43 -0700 Subject: [PATCH 009/493] fix: preserve provider refresh aliases and preparation diagnostics --- .../src/managed_mcp_capability.rs | 138 +++++ crates/freshell-session-host/src/main.rs | 480 +++++++++++++++--- .../tests/provider_config_refresh.rs | 96 ++++ .../probe-provider-config-readonly-home.py | 33 +- 4 files changed, 688 insertions(+), 59 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 9a9c0b7fc..8ebcb909b 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1322,6 +1322,144 @@ pub(crate) async fn scoped_auth_middleware( mod tests { use super::*; + #[cfg(unix)] + #[test] + fn managed_codex_linked_skills_and_discovery_form_a_resumable_graph() { + use freshell_platform::provider_environment::{copy_environment, CopySelection}; + use std::{ + fs, + os::unix::fs::{symlink, MetadataExt, PermissionsExt}, + process::Command, + }; + let root = tempfile::tempdir().unwrap(); + let original = root.path().join("original"); + let external = root.path().join("external-skills"); + let discovered = original.join(".agents/skills"); + for directory in [&external, &discovered] { + fs::create_dir_all(directory.join("fixture/scripts")).unwrap(); + fs::write(directory.join("fixture/SKILL.md"), "fixture").unwrap(); + fs::write(directory.join("fixture/resource"), "resource-nonce\n").unwrap(); + let script = directory.join("fixture/scripts/run"); + fs::write( + &script, + "#!/bin/sh\ncat \"$(dirname \"$0\")/../resource\"\n", + ) + .unwrap(); + fs::set_permissions(script, fs::Permissions::from_mode(0o700)).unwrap(); + } + fs::create_dir_all(original.join(".codex")).unwrap(); + symlink(&external, original.join(".codex/skills")).unwrap(); + let context = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "codex", + root.path(), + &original, + None, + ) + .unwrap() + .unwrap(); + let controller = stage_provider_root( + &root.path().join("control/capabilities"), + "grant-codex-roundtrip", + "codex", + &original, + &context.config, + OpencodeEphemeralInput::default(), + None, + ) + .unwrap(); + let stage = root.path().join("host-stage"); + fs::create_dir(&stage).unwrap(); + let projection = stage.join("projection"); + let selections = context + .config + .iter() + .map(|reference| CopySelection { + source: controller.join(&reference.relative_path), + destination_relative: reference.provider_relative_path.clone().into(), + }) + .collect::>(); + let report = copy_environment(&selections, &projection, &[], Default::default()).unwrap(); + assert!(fs::symlink_metadata(projection.join(".codex/skills")) + .unwrap() + .file_type() + .is_symlink()); + let resources = [ + ".codex/skills/fixture/scripts/run", + ".codex/skills/freshell-discovered-agents/fixture/scripts/run", + ]; + for relative in resources { + let output = Command::new(projection.join(relative)).output().unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"resource-nonce\n"); + } + // The ordinary controller test always exercises discovery, transport, + // and resource execution. A focused integration invocation can also + // supply the separately built production host to exercise publication. + if let Some(binary) = std::env::var_os("FRESHELL_TEST_SESSION_HOST_BINARY") { + let binary = PathBuf::from(binary); + assert!(binary.is_absolute() && binary.is_file()); + fs::write(stage.join("provider.json"), r#""codex""#).unwrap(); + fs::write( + stage.join("manifest.json"), + serde_json::to_vec( + &context + .config + .iter() + .map(|reference| &reference.provider_relative_path) + .collect::>(), + ) + .unwrap(), + ) + .unwrap(); + fs::write( + projection.join(".freshell-config-projection.json"), + serde_json::to_vec( + &report + .projected_paths + .iter() + .map(|path| path.to_str().unwrap()) + .collect::>(), + ) + .unwrap(), + ) + .unwrap(); + let home = root.path().join("provider-home"); + fs::create_dir(&home).unwrap(); + fs::set_permissions(&home, fs::Permissions::from_mode(0o700)).unwrap(); + let run = || { + let output = Command::new(&binary) + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + assert!(output.stdout.is_empty()); + for line in std::str::from_utf8(&output.stderr).unwrap().lines() { + let event: serde_json::Value = serde_json::from_str(line).unwrap(); + assert_eq!(event["provider"], "codex"); + assert_eq!(event["stage"], "provider_publication"); + eprintln!("{line}"); + } + }; + run(); + let sibling = home.join(".codex/skills/provider-sibling"); + fs::write(&sibling, "sibling-nonce").unwrap(); + let inode = fs::metadata(&sibling).unwrap().ino(); + for _ in 0..3 { + run(); + for relative in resources { + let output = Command::new(home.join(relative)).output().unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"resource-nonce\n"); + } + assert_eq!(fs::metadata(&sibling).unwrap().ino(), inode); + assert_eq!(fs::read(&sibling).unwrap(), b"sibling-nonce"); + } + eprintln!("controller-host-codex-alias-roundtrip PASS initial-and-three-resumes executable-discovery preserved-sibling"); + } + } + #[cfg(unix)] #[test] fn managed_provider_resource_script_and_dependency_execute() { diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index d255dc186..61f53a5fe 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -959,6 +959,7 @@ async fn grant_execution( } if !exact_resume { prepare_provider_bootstrap_files( + launch.provider.as_str(), &launch.provider_bootstrap_files, launch.run_as_uid, launch.run_as_gid, @@ -1074,6 +1075,7 @@ fn prepare_provider_state_for_fresh_launch( |terminal| { let mut created = providers::prepare_provider_state_before_bootstrap(terminal)?; match write_provider_bootstrap_files( + &terminal.mode, &terminal.provider_bootstrap_files, Path::new("/home/freshell/provider"), Path::new("/run/freshell-bootstrap"), @@ -1088,6 +1090,7 @@ fn prepare_provider_state_for_fresh_launch( }, |terminal, created| { transfer_created_paths( + &terminal.mode, Path::new("/home/freshell/provider"), created, terminal.run_as_uid, @@ -1127,6 +1130,7 @@ fn prepare_provider_state_before_bootstrap_with( } fn transfer_created_paths( + provider: &str, home: &Path, created: &[PathBuf], uid: u32, @@ -1172,34 +1176,47 @@ fn transfer_created_paths( } } let started = std::time::Instant::now(); - tracing::info!( - event = "provider.environment.ownership.started", - stage = "session_host", - entries = paths.len() - ); + let entries = paths.len(); + emit_environment_event(serde_json::json!({ + "severity": "info", "event": "provider.environment.ownership.started", + "stage": "session_host", "provider": provider, "entries": entries + })); let mut ordered = std::mem::take(&mut paths).into_iter().collect::>(); ordered.sort_by_key(|path| std::cmp::Reverse(path.components().count())); for path in ordered { - set_owner(&path,uid,gid).inspect_err(|error| {tracing::error!(event="provider.environment.ownership.failed",stage="session_host",relative_path=%path.strip_prefix(home).unwrap_or(Path::new("")).display(),elapsed_ms=started.elapsed().as_millis() as u64,operation="lchown",target_uid=uid,target_gid=gid,%error);})?; + set_owner(&path, uid, gid).inspect_err(|error| { + emit_environment_event(serde_json::json!({ + "severity": "error", "event": "provider.environment.ownership.failed", + "stage": "session_host", "provider": provider, + "relative_path": path.strip_prefix(home).unwrap_or(Path::new("")), + "elapsed_ms": started.elapsed().as_millis() as u64, + "operation": "lchown", "target_uid": uid, "target_gid": gid, + "error": error + })); + })?; } - tracing::info!( - event = "provider.environment.ownership.completed", - stage = "session_host", - entries = created.len(), - elapsed_ms = started.elapsed().as_millis() as u64 - ); + emit_environment_event(serde_json::json!({ + "severity": "info", "event": "provider.environment.ownership.completed", + "stage": "session_host", "provider": provider, "entries": entries, + "elapsed_ms": started.elapsed().as_millis() as u64 + })); Ok(()) } fn prepare_provider_bootstrap_files( + provider: &str, files: &[ProviderBootstrapFile], run_as_uid: u32, run_as_gid: u32, ) -> Result<(), String> { let home = Path::new("/home/freshell/provider"); - let created = - write_provider_bootstrap_files(files, home, Path::new("/run/freshell-bootstrap"))?; - transfer_created_paths(home, &created, run_as_uid, run_as_gid) + let created = write_provider_bootstrap_files( + provider, + files, + home, + Path::new("/run/freshell-bootstrap"), + )?; + transfer_created_paths(provider, home, &created, run_as_uid, run_as_gid) } fn rollback_created_provider_paths(paths: &[PathBuf]) { @@ -1216,6 +1233,8 @@ fn rollback_created_provider_paths(paths: &[PathBuf]) { } } struct BootstrapPublication { + provider: String, + home: PathBuf, // Every newly published inode needs ownership handoff, including replacements. owned_paths: std::collections::BTreeSet, // Only newly created pathnames grant removal authority on failure. @@ -1241,34 +1260,49 @@ impl Drop for BootstrapPublication { rollback_created_provider_paths(&self.new_paths); for (destination, backup) in self.replacements.iter().rev() { if let Err(error) = std::fs::rename(backup, destination) { - tracing::error!(event = "provider.bootstrap.rollback.failed", stage = "session_host", operation = "restore_replacement", path = %destination.display(), %error); + emit_environment_event(serde_json::json!({ + "severity": "error", "event": "provider.bootstrap.rollback.failed", + "stage": "session_host", "provider": self.provider, + "operation": "restore_replacement", + "relative_path": destination.strip_prefix(&self.home).unwrap_or(Path::new("")), + "error_kind": format!("{:?}", error.kind()), "os_error": error.raw_os_error() + })); } } } } fn write_provider_bootstrap_files( + provider: &str, files: &[ProviderBootstrapFile], home: &Path, source_root: &Path, ) -> Result, String> { - write_provider_bootstrap_files_with_publish(files, home, source_root, |source, destination| { - std::fs::rename(source, destination) - }) + write_provider_bootstrap_files_with_publish( + provider, + files, + home, + source_root, + |source, destination| std::fs::rename(source, destination), + ) } fn write_provider_bootstrap_files_with_publish( + provider: &str, files: &[ProviderBootstrapFile], home: &Path, source_root: &Path, mut publish: impl FnMut(&Path, &Path) -> std::io::Result<()>, ) -> Result, String> { - use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; + use freshell_platform::provider_environment::CopySelection; if !std::fs::symlink_metadata(home) .is_ok_and(|metadata| metadata.is_dir() && !metadata.file_type().is_symlink()) { return Err("managed provider HOME volume is not mounted as a physical directory".into()); } + // This writer runs only during trusted initial preparation, before HOME + // ownership is handed to the provider. Refresh preserves the initialized root. + set_mode(home, 0o700)?; let mut destinations = Vec::with_capacity(files.len()); for file in files { let relative = Path::new(&file.provider_relative_path); @@ -1328,14 +1362,17 @@ fn write_provider_bootstrap_files_with_publish( }) }) .collect::, String>>()?; - copy_environment( + copy_environment_with_events( + "bootstrap_inputs", + provider, &selections, &stage.path().join("inputs"), &[], - CopyLimits::default(), ) .map_err(|error| error.to_string())?; let mut created = BootstrapPublication { + provider: provider.into(), + home: home.into(), owned_paths: std::collections::BTreeSet::new(), new_paths: Vec::new(), replacements: Vec::new(), @@ -1395,6 +1432,53 @@ fn write_provider_bootstrap_files_with_publish( Ok(paths.into_iter().collect()) } +// Preparation produces at most start/completion (or failure) records per copy, +// never per-file events. Keep protocol stdout exclusively for its existing uses. +fn emit_environment_event(event: serde_json::Value) { + use std::io::Write; + let line = freshell_runtime_observability::scrub(&event.to_string(), ""); + let _ = writeln!(std::io::stderr().lock(), "{line}"); +} + +fn copy_environment_with_events( + stage: &str, + provider: &str, + selections: &[freshell_platform::provider_environment::CopySelection], + destination: &Path, + excluded: &[PathBuf], +) -> Result< + freshell_platform::provider_environment::CopyReport, + freshell_platform::provider_environment::CopyError, +> { + let started = std::time::Instant::now(); + emit_environment_event(serde_json::json!({ + "severity": "info", "event": "provider.environment.copy.started", + "stage": stage, "provider": provider, "selections": selections.len() + })); + let report = freshell_platform::provider_environment::copy_environment( + selections, + destination, + excluded, + Default::default(), + ) + .inspect_err(|error| { + emit_environment_event(serde_json::json!({ + "severity": "error", "event": "provider.environment.copy.failed", + "stage": stage, "provider": provider, + "relative_path": error.relative_path, "operation": error.operation, + "limit_kind": error.limit_kind, "observed": error.observed, + "permitted": error.permitted, "corrective_message": error.corrective_message, + "elapsed_ms": started.elapsed().as_millis() as u64 + })); + })?; + emit_environment_event(serde_json::json!({ + "severity": "info", "event": "provider.environment.copy.completed", + "stage": stage, "provider": provider, "entries": report.entries, + "bytes": report.bytes, "elapsed_ms": started.elapsed().as_millis() as u64 + })); + Ok(report) +} + fn copy_provider_config_references( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -1422,7 +1506,7 @@ fn copy_provider_config_references_with_ownership( provider_home: &Path, run_as: (u32, u32), ) -> Result<(), String> { - use freshell_platform::provider_environment::{copy_environment, CopyLimits, CopySelection}; + use freshell_platform::provider_environment::CopySelection; use freshell_runtime_protocol::ProviderConfigRoot; context.validate(provider).map_err(|error| error.message)?; let selections = provider_config_references(context) @@ -1439,26 +1523,15 @@ fn copy_provider_config_references_with_ownership( } }) .collect::>(); - let started = std::time::Instant::now(); - tracing::info!( - event = "provider.environment.copy.started", - stage = "session_host", - provider, - selections = selections.len() - ); let excluded = provider_credential_sources(user_provider_root); - let report=copy_environment(&selections,provider_home,&excluded,CopyLimits::default()).map_err(|error| { - tracing::error!(event="provider.environment.copy.failed",stage="session_host",provider,relative_path=%error.relative_path.display(),operation=error.operation,limit_kind=error.limit_kind,observed=error.observed,permitted=error.permitted,corrective_message=%error.corrective_message,elapsed_ms=started.elapsed().as_millis() as u64); - error.to_string() - })?; - tracing::info!( - event = "provider.environment.copy.completed", - stage = "session_host", + let report = copy_environment_with_events( + "session_host", provider, - entries = report.entries, - bytes = report.bytes, - elapsed_ms = started.elapsed().as_millis() as u64 - ); + &selections, + provider_home, + &excluded, + ) + .map_err(|error| error.to_string())?; let projected = report .projected_paths .iter() @@ -1600,6 +1673,11 @@ fn refresh_provider_config_references( serde_json::to_vec(&destinations).map_err(|error| error.to_string())?, ) .map_err(|error| error.to_string())?; + std::fs::write( + stage.path().join("provider.json"), + serde_json::to_vec(provider).map_err(|error| error.to_string())?, + ) + .map_err(|error| error.to_string())?; allow_provider_read_stage(stage.path(), run_as_gid)?; if run_as_uid == unsafe { libc::geteuid() } && run_as_gid == unsafe { libc::getegid() } { return apply_staged_provider_config(stage.path(), provider_home); @@ -1697,13 +1775,21 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() { return Err("provider config refresh destination is unsafe".into()); } - let mut parent = provider_home.to_path_buf(); - for part in path.parent().into_iter().flat_map(Path::components) { - parent.push(part.as_os_str()); - if std::fs::symlink_metadata(&parent) - .is_ok_and(|metadata| metadata.file_type().is_symlink()) - { - return Err("provider config refresh parent cannot be a symlink".into()); + // Destination selections are logical graph routes, and may traverse a + // selected skills alias. The common copier resolves those routes from + // the staged graph, then validates every physical publication parent. + // Manifest entries instead name physical inodes used for stale removal. + if previous.contains(relative) || current.contains(relative) { + let mut parent = provider_home.to_path_buf(); + for part in path.parent().into_iter().flat_map(Path::components) { + parent.push(part.as_os_str()); + if std::fs::symlink_metadata(&parent) + .is_ok_and(|metadata| metadata.file_type().is_symlink()) + { + return Err( + "provider config refresh physical parent cannot be a symlink".into(), + ); + } } } if destinations @@ -1725,11 +1811,23 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() }, ) .collect::>(); - freshell_platform::provider_environment::copy_environment( + let provider = match std::fs::read(stage.join("provider.json")) { + Ok(bytes) => serde_json::from_slice::(&bytes).map_err(|error| error.to_string())?, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => "unknown".into(), + Err(error) => return Err(error.to_string()), + }; + if !matches!( + provider.as_str(), + "claude" | "codex" | "opencode" | "amplifier" | "unknown" + ) { + return Err("provider config refresh provider is unsupported".into()); + } + copy_environment_with_events( + "provider_publication", + &provider, &selections, provider_home, &[], - Default::default(), ) .map_err(|error| error.to_string())?; let mut stale = previous.difference(¤t).collect::>(); @@ -2954,6 +3052,204 @@ mod tests { assert!(!destination.join(".claude/settings.json").exists()); } + #[cfg(unix)] + #[test] + fn provider_config_codex_linked_skills_and_discovery_survive_repeated_resume() { + use freshell_platform::provider_environment::{copy_environment, CopySelection}; + use freshell_runtime_protocol::{ + ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, + }; + use std::os::unix::fs::{symlink, MetadataExt, PermissionsExt}; + let root = tempfile::tempdir().unwrap(); + let original = root.path().join("original"); + let external = root.path().join("external-skills"); + let discovered = original.join(".agents/skills"); + for directory in [&external, &discovered] { + std::fs::create_dir_all(directory.join("fixture/scripts")).unwrap(); + std::fs::write(directory.join("fixture/SKILL.md"), "fixture").unwrap(); + std::fs::write(directory.join("fixture/resource"), "resource-nonce\n").unwrap(); + let script = directory.join("fixture/scripts/run"); + std::fs::write( + &script, + "#!/bin/sh\ncat \"$(dirname \"$0\")/../resource\"\n", + ) + .unwrap(); + std::fs::set_permissions(script, std::fs::Permissions::from_mode(0o700)).unwrap(); + } + std::fs::create_dir_all(original.join(".codex")).unwrap(); + symlink(&external, original.join(".codex/skills")).unwrap(); + let controller = root.path().join("controller"); + copy_environment( + &[ + CopySelection { + source: original.join(".codex/skills"), + destination_relative: "skills".into(), + }, + CopySelection { + source: discovered, + destination_relative: "freshell-discovered-agents".into(), + }, + ], + &controller, + &[], + Default::default(), + ) + .unwrap(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Codex { + tui_args: vec![], + sidecar_args: vec![], + }, + mcp_capability: None, + config: [ + ("skills", ".codex/skills"), + ( + "freshell-discovered-agents", + ".codex/skills/freshell-discovered-agents", + ), + (".freshell/resources", ".freshell/resources"), + ] + .into_iter() + .map(|(source, destination)| ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: source.into(), + provider_relative_path: destination.into(), + format: "directory".into(), + }) + .collect(), + }; + let home = root.path().join("provider"); + std::fs::create_dir(&home).unwrap(); + let uid = unsafe { libc::geteuid() }; + let gid = unsafe { libc::getegid() }; + copy_provider_config_references( + &context, + "codex", + root.path(), + &controller, + &home, + uid, + gid, + ) + .unwrap(); + assert!(std::fs::symlink_metadata(home.join(".codex/skills")) + .unwrap() + .file_type() + .is_symlink()); + let sibling = home.join(".codex/skills/provider-sibling"); + std::fs::write(&sibling, "durable-nonce").unwrap(); + let inode = std::fs::metadata(&sibling).unwrap().ino(); + for _ in 0..3 { + refresh_provider_config_references( + &context, + "codex", + root.path(), + &controller, + &home, + uid, + gid, + ) + .unwrap(); + for relative in [ + ".codex/skills/fixture/scripts/run", + ".codex/skills/freshell-discovered-agents/fixture/scripts/run", + ] { + let output = std::process::Command::new(home.join(relative)) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"resource-nonce\n"); + } + assert_eq!(std::fs::metadata(&sibling).unwrap().ino(), inode); + assert_eq!(std::fs::read(&sibling).unwrap(), b"durable-nonce"); + } + } + + #[cfg(unix)] + #[test] + fn provider_config_unplanned_parent_alias_does_not_modify_outside_target() { + use std::os::unix::fs::{symlink, MetadataExt}; + let root = tempfile::tempdir().unwrap(); + let outside = root.path().join("outside"); + let home = root.path().join("home"); + let stage = root.path().join("stage"); + std::fs::create_dir(&outside).unwrap(); + std::fs::create_dir(&home).unwrap(); + std::fs::create_dir_all(stage.join("projection/.codex/skills")).unwrap(); + std::fs::write(outside.join("sentinel"), "unchanged").unwrap(); + let inode = std::fs::metadata(outside.join("sentinel")).unwrap().ino(); + symlink(&outside, home.join(".codex")).unwrap(); + std::fs::write(stage.join("manifest.json"), r#"[".codex/skills"]"#).unwrap(); + std::fs::write( + stage.join("projection").join(PROVIDER_PROJECTION_MANIFEST), + r#"[".codex",".codex/skills"]"#, + ) + .unwrap(); + assert!(apply_staged_provider_config(&stage, &home).is_err()); + assert_eq!( + std::fs::metadata(outside.join("sentinel")).unwrap().ino(), + inode + ); + assert_eq!( + std::fs::read(outside.join("sentinel")).unwrap(), + b"unchanged" + ); + assert!(!outside.join("skills").exists()); + } + + #[test] + fn provider_config_spawned_host_preparation_emits_safe_jsonl() { + // Re-exec an existing behavior test so the production preparation and + // ownership helpers run in a fresh process with no tracing subscriber. + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "tests::provider_config_first_amplifier_launch_transfers_exact_created_state_before_copy", "--nocapture"]) + .output().unwrap(); + assert!(output.status.success(), "{output:?}"); + let events = std::str::from_utf8(&output.stderr) + .unwrap() + .lines() + .map(|line| serde_json::from_str::(line).unwrap()) + .collect::>(); + for name in [ + "provider.environment.ownership.started", + "provider.environment.ownership.completed", + "provider.environment.copy.started", + "provider.environment.copy.completed", + ] { + let event = events + .iter() + .find(|event| event["event"] == name && event["stage"] == "session_host") + .expect("spawned preparation must emit each event"); + assert_eq!(event["provider"], "amplifier"); + } + // The existing selected-resource failure regression runs both a real + // successful copy and a missing selected input in the fresh process. + let failed = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "tests::provider_config_exact_resume_refreshes_changes_and_removes_stale_projection", "--nocapture"]) + .output().unwrap(); + assert!(failed.status.success(), "{failed:?}"); + let events = std::str::from_utf8(&failed.stderr) + .unwrap() + .lines() + .map(|line| serde_json::from_str::(line).unwrap()) + .collect::>(); + let event = events + .iter() + .find(|event| { + event["event"] == "provider.environment.copy.failed" + && event["stage"] == "session_host" + }) + .expect("spawned failed preparation must emit typed error"); + assert_eq!(event["provider"], "claude"); + assert_eq!(event["relative_path"], ".claude/settings.json"); + assert_eq!(event["operation"], "select"); + assert!(event["corrective_message"] + .as_str() + .unwrap() + .contains("repair")); + assert!(event["elapsed_ms"].as_u64().is_some()); + } + #[test] fn provider_identity_command_uses_clear_groups_only_when_switching_identity() { let uid = unsafe { libc::geteuid() }; @@ -3666,9 +3962,67 @@ mod tests { }; thread_local! {pub(crate) static OPERATIONS: RefCell> = const {RefCell::new(Vec::new())};} fn handoff(home: &Path, ledger: &[PathBuf]) -> Result<(), String> { - transfer_created_paths(home, ledger, unsafe { libc::geteuid() }, unsafe { - libc::getegid() - }) + transfer_created_paths( + "amplifier", + home, + ledger, + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + ) + } + #[test] + fn initial_home_becomes_private_before_handoff_and_refresh_keeps_its_inode() { + use std::os::unix::fs::MetadataExt; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("provider"); + std::fs::create_dir(&home).unwrap(); + std::fs::set_permissions(&home, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write(home.join("provider-sibling"), "durable").unwrap(); + let inode = std::fs::metadata(&home).unwrap().ino(); + let sibling = std::fs::metadata(home.join("provider-sibling")).unwrap(); + let inputs = root.path().join("inputs"); + std::fs::create_dir(&inputs).unwrap(); + let ledger = write_provider_bootstrap_files("codex", &[], &home, &inputs).unwrap(); + handoff(&home, &ledger).unwrap(); + let initialized = std::fs::metadata(&home).unwrap(); + assert_eq!(initialized.mode() & 0o777, 0o700); + assert_eq!(initialized.ino(), inode); + assert_eq!(initialized.uid(), unsafe { libc::geteuid() }); + assert_eq!(initialized.gid(), unsafe { libc::getegid() }); + let stage = root.path().join("stage"); + std::fs::create_dir_all(stage.join("projection")).unwrap(); + std::fs::write(stage.join("manifest.json"), "[]").unwrap(); + std::fs::write( + stage.join("projection").join(PROVIDER_PROJECTION_MANIFEST), + "[]", + ) + .unwrap(); + apply_staged_provider_config(&stage, &home).unwrap(); + let refreshed = std::fs::metadata(&home).unwrap(); + assert_eq!( + ( + refreshed.ino(), + refreshed.uid(), + refreshed.gid(), + refreshed.mode() + ), + ( + initialized.ino(), + initialized.uid(), + initialized.gid(), + initialized.mode() + ) + ); + assert_eq!( + std::fs::metadata(home.join("provider-sibling")) + .unwrap() + .ino(), + sibling.ino() + ); + assert_eq!( + std::fs::read(home.join("provider-sibling")).unwrap(), + b"durable" + ); } #[test] fn all_selected_entries_above_ten_thousand_remain_runnable() { @@ -3705,6 +4059,7 @@ mod tests { let mut ledger = ensured.created_paths; ledger.extend( write_provider_bootstrap_files( + "amplifier", &[ProviderBootstrapFile { source_path: inputs.join("provider-0").to_string_lossy().into_owned(), provider_relative_path: "bootstrap/config.json".into(), @@ -3781,7 +4136,7 @@ mod tests { provider_relative_path: "bootstrap/second.json".into(), }, ]; - assert!(write_provider_bootstrap_files(&files, &home, &inputs).is_err()); + assert!(write_provider_bootstrap_files("amplifier", &files, &home, &inputs).is_err()); assert!(!home.join("bootstrap").exists()); assert_eq!(std::fs::read(home.join("sibling")).unwrap(), b"sentinel"); } @@ -3812,7 +4167,7 @@ mod tests { provider_relative_path: "bootstrap/second.json".into(), }, ]; - assert!(write_provider_bootstrap_files(&files, &home, &inputs).is_err()); + assert!(write_provider_bootstrap_files("amplifier", &files, &home, &inputs).is_err()); assert_eq!( std::fs::read(&first).expect("prior config must survive failed bootstrap"), b"prior-config" @@ -3829,7 +4184,8 @@ mod tests { ); assert!(!home.join("bootstrap/second.json").exists()); std::fs::write(inputs.join("provider-1"), "second-config").unwrap(); - let owned = write_provider_bootstrap_files(&files, &home, &inputs).unwrap(); + let owned = + write_provider_bootstrap_files("amplifier", &files, &home, &inputs).unwrap(); assert!( owned.contains(&first), "replacement inode needs exact ownership handoff" @@ -3886,6 +4242,7 @@ mod tests { } let mut published = 0; let failure = write_provider_bootstrap_files_with_publish( + "amplifier", &files, &home, &inputs, @@ -3923,7 +4280,8 @@ mod tests { 1, "attempt staging/backup paths must be cleaned" ); - let owned = write_provider_bootstrap_files(&files, &home, &inputs).unwrap(); + let owned = + write_provider_bootstrap_files("amplifier", &files, &home, &inputs).unwrap(); assert!(owned.contains(&first)); assert!(owned.contains(&last)); assert!(owned.contains(&home.join("new-parent"))); @@ -4028,7 +4386,13 @@ mod tests { }, |terminal, ledger| { assert!(!home.join(".amplifier/skills").exists()); - transfer_created_paths(&home, ledger, terminal.run_as_uid, terminal.run_as_gid)?; + transfer_created_paths( + &terminal.mode, + &home, + ledger, + terminal.run_as_uid, + terminal.run_as_gid, + )?; handed_off.set(true); Ok(()) }, diff --git a/crates/freshell-session-host/tests/provider_config_refresh.rs b/crates/freshell-session-host/tests/provider_config_refresh.rs index 04d260430..9cd67ebff 100644 --- a/crates/freshell-session-host/tests/provider_config_refresh.rs +++ b/crates/freshell-session-host/tests/provider_config_refresh.rs @@ -55,3 +55,99 @@ fn provider_uid_refresh_helper_updates_projection_without_erasing_provider_state b"durable" ); } + +#[cfg(unix)] +#[test] +fn provider_config_spawned_helper_emits_safe_jsonl_for_success_and_typed_failure() { + use std::{fs, process::Command}; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let stage = root.path().join("stage"); + fs::create_dir(&home).unwrap(); + fs::create_dir_all(stage.join("projection/skills")).unwrap(); + fs::write(stage.join("provider.json"), r#""codex""#).unwrap(); + fs::write(stage.join("manifest.json"), r#"["skills"]"#).unwrap(); + fs::write( + stage.join("projection/.freshell-config-projection.json"), + r#"["skills","skills/config"]"#, + ) + .unwrap(); + let marker = "synthetic-config-secret-outgoing-message-marker"; + fs::write(stage.join("projection/skills/config"), marker).unwrap(); + let run = || { + Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home) + .output() + .unwrap() + }; + let success = run(); + assert!(success.status.success(), "{success:?}"); + assert!( + success.stdout.is_empty(), + "logging must preserve protocol stdout" + ); + assert_eq!( + fs::read(home.join("skills/config")).unwrap(), + marker.as_bytes() + ); + let parse = |bytes: &[u8]| -> Vec { + let text = std::str::from_utf8(bytes).unwrap(); + assert!(!text.contains(marker)); + text.lines() + .map(|line| serde_json::from_str(line).unwrap()) + .collect() + }; + let events = parse(&success.stderr); + assert_eq!(events.len(), 2, "copy logs must be per-operation"); + for event in &events { + eprintln!("{event}"); + } + let started = events + .iter() + .find(|event| event["event"] == "provider.environment.copy.started") + .expect("actual helper must emit copy start"); + assert_eq!(started["provider"], "codex"); + assert_eq!(started["stage"], "provider_publication"); + assert_eq!(started["selections"], 1); + let completed = events + .iter() + .find(|event| event["event"] == "provider.environment.copy.completed") + .expect("actual helper must emit completion"); + assert_eq!(completed["provider"], "codex"); + assert_eq!(completed["stage"], "provider_publication"); + assert!(completed["entries"].as_u64().unwrap() >= 2); + assert_eq!(completed["bytes"], marker.len()); + assert!(completed["elapsed_ms"].as_u64().is_some()); + // An oversized sparse fixture fails during planning without reading contents. + fs::File::create(stage.join("projection/skills/oversized")) + .unwrap() + .set_len(128 * 1024 * 1024 + 1) + .unwrap(); + let failed = run(); + assert!(!failed.status.success()); + assert!(failed.stdout.is_empty()); + let events = parse(&failed.stderr); + assert_eq!(events.len(), 3, "start, typed failure and final fatal only"); + for event in &events { + eprintln!("{event}"); + } + let failure = events + .iter() + .find(|event| event["event"] == "provider.environment.copy.failed") + .expect("actual helper must emit typed copy failure"); + assert_eq!(failure["provider"], "codex"); + assert_eq!(failure["stage"], "provider_publication"); + assert_eq!(failure["relative_path"], "skills/oversized"); + assert!(failure["operation"].as_str().is_some()); + assert_eq!(failure["limit_kind"], "file_bytes"); + assert_eq!(failure["observed"], 128 * 1024 * 1024 + 1_u64); + assert_eq!(failure["permitted"], 128 * 1024 * 1024_u64); + assert!(failure["corrective_message"].as_str().is_some()); + assert!(failure["elapsed_ms"].as_u64().is_some()); + assert_eq!( + fs::read(home.join("skills/config")).unwrap(), + marker.as_bytes() + ); +} diff --git a/scripts/testing/probe-provider-config-readonly-home.py b/scripts/testing/probe-provider-config-readonly-home.py index 6701000bb..01ec95657 100644 --- a/scripts/testing/probe-provider-config-readonly-home.py +++ b/scripts/testing/probe-provider-config-readonly-home.py @@ -40,6 +40,7 @@ def main(): (skills / "new.sh").chmod(0o750) (skills / "resource").write_text("resource-nonce\n") (stage / "manifest.json").write_text('["skills"]') + (stage / "provider.json").write_text('"codex"') (stage / "projection/.freshell-config-projection.json").write_text( '["skills","skills/new.sh","skills/resource"]' ) @@ -47,9 +48,27 @@ def main(): broken.mkdir(parents=True) (broken / "bad").symlink_to("missing-resource") (stage / "broken/manifest.json").write_text('["skills"]') + (stage / "broken/provider.json").write_text('"codex"') (stage / "broken/projection/.freshell-config-projection.json").write_text( '["skills","skills/bad"]' ) + aliases = stage / "aliases/projection" + for name in ["skill-root", "discovered-root"]: + resource = aliases / ".freshell/resources" / name + resource.mkdir(parents=True) + (resource / "resource").write_text(name + "-nonce\n") + (resource / "run").write_text('#!/bin/sh\ncat "$(dirname "$0")/resource"\n') + (resource / "run").chmod(0o750) + (aliases / ".codex").mkdir() + (aliases / ".codex/skills").symlink_to("../.freshell/resources/skill-root") + (aliases / ".freshell/resources/skill-root/freshell-discovered-agents").symlink_to("../discovered-root") + (stage / "aliases/provider.json").write_text('"codex"') + (stage / "aliases/manifest.json").write_text(json.dumps([ + ".codex/skills", ".codex/skills/freshell-discovered-agents", ".freshell/resources" + ])) + (aliases / ".freshell-config-projection.json").write_text(json.dumps([ + path.relative_to(aliases).as_posix() for path in aliases.rglob("*") + ])) for path in stage.rglob("*"): if path.is_symlink(): continue @@ -106,8 +125,20 @@ def main(): test "$before_home" = "$(stat -c "%i:%u:%g:%a" "$home")" test "$before_sibling" = "$(stat -c "%i:%u:%g:%a" "$home/skills/provider-sibling")" test -z "$(find "$home" -maxdepth 1 -name ".freshell-copy-*" -print)" +/runtime/freshell-session-host refresh-provider-config /probe-stage/aliases "$home" +printf alias-sibling-nonce > "$home/.codex/skills/provider-sibling" +before_alias_sibling=$(stat -c "%i:%u:%g:%a" "$home/.codex/skills/provider-sibling") +for attempt in 1 2 3; do + /runtime/freshell-session-host refresh-provider-config /probe-stage/aliases "$home" + test "$("$home/.codex/skills/run")" = skill-root-nonce + test "$("$home/.codex/skills/freshell-discovered-agents/run")" = discovered-root-nonce + test "$before_alias_sibling" = "$(stat -c "%i:%u:%g:%a" "$home/.codex/skills/provider-sibling")" + test "$(cat "$home/.codex/skills/provider-sibling")" = alias-sibling-nonce + test "$before_home" = "$(stat -c "%i:%u:%g:%a" "$home")" +done +test -z "$(find "$home" -maxdepth 1 -name ".freshell-copy-*" -print)" ' -printf 'readonly-home-provider-helper PASS uid=65534 gid=0 caps=0 stable-home retained-sibling executable-resource stale-removed failed-refresh-preserved retry\n' +printf 'readonly-home-provider-helper PASS uid=65534 gid=0 caps=0 stable-home retained-sibling executable-resource stale-removed failed-refresh-preserved retry nested-codex-alias-three-resumes\n' ''' ) inspection = json.loads(docker("inspect", owned_name).stdout)[0] From 7a3628b3f0d901fc499ed964dc230b7ee6bcac20 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 00:42:01 -0700 Subject: [PATCH 010/493] docs: record completed managed environment copier task --- .../2026-10-05-managed-provider-environment.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index e05c67a83..a591557a0 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -157,7 +157,7 @@ pub fn provider_read_stage_mode(source_mode: u32) -> u32 { 0o640 | (source_mode `CopyError` has safe `relative_path`, operation, limit kind, observed and permitted values, and a corrective message. It contains no bytes/config fragments. The caller supplies stage/provider fields when logging. Selection destination components must be normal bounded relative components, with no traversal. -- [ ] **Step 1: Add behavioral tests.** Create a disposable selection containing `SKILL.md`, `scripts/run` mode 0755 and then 0700, `references/data.json`, and `assets/template.txt`. Script body is `#!/bin/sh\nset -eu\ncd "$(dirname "$0")/.."\ncat references/data.json assets/template.txt\n`. Copy twice (controller root then provider HOME) and execute the final script with `Command::new`; assert exact nonce output and success. Create an ES module plugin importing `fixture-dependency` whose package reads its harmless `auth.json`; execute Node against the copied plugin and assert its exported nonce. Include actual relative `.pnpm` and `.bin` links, a transitive circular package link, and an external linked skill; assert runnable resource/dependency closure. Add missing/dangling/unsupported-node errors and exact/beyond tests for all four injected limits, checking observed error fields and no caller-published stage. Assert canonical credential source aliases are excluded but ordinary package `auth.json` survives. +- [x] **Step 1: Add behavioral tests.** Create a disposable selection containing `SKILL.md`, `scripts/run` mode 0755 and then 0700, `references/data.json`, and `assets/template.txt`. Script body is `#!/bin/sh\nset -eu\ncd "$(dirname "$0")/.."\ncat references/data.json assets/template.txt\n`. Copy twice (controller root then provider HOME) and execute the final script with `Command::new`; assert exact nonce output and success. Create an ES module plugin importing `fixture-dependency` whose package reads its harmless `auth.json`; execute Node against the copied plugin and assert its exported nonce. Include actual relative `.pnpm` and `.bin` links, a transitive circular package link, and an external linked skill; assert runnable resource/dependency closure. Add missing/dangling/unsupported-node errors and exact/beyond tests for all four injected limits, checking observed error fields and no caller-published stage. Assert canonical credential source aliases are excluded but ordinary package `auth.json` survives. ```rust #[cfg(unix)] @@ -190,15 +190,15 @@ fn copied_skill_executes_sibling_resources_after_two_stages() { } ``` -- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`. Expected missing implementation, then behavioral failure when the test is initially wired through current copying; record the old two-stage script execution PermissionDenied and missing dependency result, rather than treating initial compile setup as the required red evidence. Run matching controller/host tests against old code for the intended omissions before replacing it. +- [x] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`. Expected missing implementation, then behavioral failure when the test is initially wired through current copying; record the old two-stage script execution PermissionDenied and missing dependency result, rather than treating initial compile setup as the required red evidence. Run matching controller/host tests against old code for the intended omissions before replacing it. Add `created_path_ledger` tests calling the real `ensure_session` creation/adoption operations and asserting their returned exact paths against actual filesystem creation: new parents/leaves are recorded, pre-existing parents and adopted stubs/siblings are absent, partial-create rollback records no surviving authority. Add `created_path_ownership` tests invoking the production initial-preparation/ledger-transfer helper with >10,000 selected entries (including actual bootstrap-created leaves), an adopted sibling and a selected symlink to an unrelated target. Assert all selected entries remain present/runnable, only creator-recorded physical inodes are handed off, and sibling/target ownership and contents are unchanged. Observe actual creation and physical `lchown` operations/metadata, never source text; the same-UID unit fixture can capture invoked ownership operations, while Task 3's real provider-UID child smoke verifies the resulting transfer. Record behavioral failures against the old 10,000-cap/traversal path before replacing it. Run `mise exec rust@1.96 -- cargo test -p freshell-sessions --locked created_path_ledger` and `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked created_path_ownership`; require at least three executed tests in each named module, with no skipped cases or zero-match success. -- [ ] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Select complete skill/plugin/package resource units before mapping. External plugin entry files require their containing resource/package root, adjacent imports/assets and applicable package metadata; canonical target leaves alone are insufficient. Include existing installed node_modules lookup context and connecting ancestor skeleton without copying unrelated ancestor trees. Prefer one bounded complete installation node_modules unit for pnpm store/hoisted/.bin topology, preserving ancestor lookup order and package scope. Preplan coherent physical mappings for overlapping parent/child units, reuse one physical canonical target and preserve additional logical resource destinations as links; overlap itself is not an error. Reject incompatible destination/type claims before writing. Preserve symbolic links with rewritten relative targets and enqueue the required complete resource/package and installed lookup context under private `.freshell/resources` when outside selected roots, preserving relative ancestor/sibling hierarchy rather than flattening one canonical leaf per private ID. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. +- [x] **Step 3: Apply precise implementation patches.** Export `provider_environment`. Enumerate selected trees deterministically, count every planned entry and canonical regular-file byte once, reject unsupported nodes and actual read errors. Select complete skill/plugin/package resource units before mapping. External plugin entry files require their containing resource/package root, adjacent imports/assets and applicable package metadata; canonical target leaves alone are insufficient. Include existing installed node_modules lookup context and connecting ancestor skeleton without copying unrelated ancestor trees. Prefer one bounded complete installation node_modules unit for pnpm store/hoisted/.bin topology, preserving ancestor lookup order and package scope. Preplan coherent physical mappings for overlapping parent/child units, reuse one physical canonical target and preserve additional logical resource destinations as links; overlap itself is not an error. Reject incompatible destination/type claims before writing. Preserve symbolic links with rewritten relative targets and enqueue the required complete resource/package and installed lookup context under private `.freshell/resources` when outside selected roots, preserving relative ancestor/sibling hierarchy rather than flattening one canonical leaf per private ID. Reuse already-mapped directory targets so ordinary circular dependency links stay links rather than recursive copies. Fail dangling links; do not follow arbitrary HOME ancestors. Check excluded canonical source identities before copying aliases. Stream regular files in bounded chunks, enforce actual bytes while reading to catch growth, and create private files then apply only executable bits. One operation budget spans selections and closure; overflow/depth errors specify measured/path/bound and suggest reducing unrelated data or repairing the named selected resource. Replace both recursive copier bodies with this interface. Select complete known plugin/skill/resource trees; add OpenCode `node_modules` and actual package/lock forms including Bun locks, and Codex's discovered `.agents/skills` into the existing runtime-supported skill root. Record the single closure/context directory and required aliases as normal `.freshell/resources` selections in ProviderLaunchContext; transport the complete graph once and preserve its relative topology identically at the second stage, without independent package rediscovery or renesting. Do not put one config reference per dependency package or silently exceed the context's 128-selection limit. Keep history/cache/session exclusion explicit. Change `approved_config_path` and selected-path resolution to `Result`; absence is optional only before a configured selection is established. Replace the Amplifier 10,000-entry ownership cap with the exact created leaf/parent ledger in Selected C05 projection and exact initial ownership decisions; carry it from the creating operations, not a fresh recursive HOME walk. Preserve modes in `allow_provider_read_stage` and final copies. Callers log JSON start/completed/failed events with stage, provider, counts, bytes, elapsed time and safe error fields; no per-file success logs. Extend `EnsuredSession` with the exact paths its creating operations actually created, not a directory listing after creation. Preserve all existing identity/adoption fields and update its constructors/callers. Return the bounded ledger from `providers::prepare_provider_state_before_bootstrap` into host initial preparation and combine it with the bootstrap writer's created leaves/parents under the one shared entry budget. Adopted sessions contribute no recursive authority. The ownership helper consumes that ledger with physical `symlink_metadata`/`lchown`, deduplicates, transfers deepest-first and HOME last; no fresh Amplifier traversal. Include these creator/callsite/helper changes in Task 1 before Task 3 uses them. -- [ ] **Step 4: Green.** Repeat focused platform/controller/host commands; expected executable and Node resource/dependency assertions pass with correct boundary errors. -- [ ] **Step 5: Refactor.** Remove the two obsolete private copier policies and basename-based secret filtering; keep provider selection and ownership at callers and filesystem behavior in the common module. Avoid adding environment policy configuration. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`; the two `created_path_ledger`/`created_path_ownership` commands from Step 2. Expected all pass with positive counts, including at least three tests per new ledger/ownership module. These cover the shared policy, actual Amplifier creator/callsite and every existing caller. -- [ ] **Step 7: Commit.** `git add crates/freshell-platform/src/provider_environment.rs crates/freshell-platform/src/lib.rs crates/freshell-server/src/managed_provider_bootstrap.rs crates/freshell-server/src/managed_mcp_capability.rs crates/freshell-session-host/src/main.rs crates/freshell-session-host/src/providers/mod.rs crates/freshell-sessions/src/amplifier_stub.rs` then `git commit -m "fix: preserve bounded managed provider resources and executable modes"`. +- [x] **Step 4: Green.** Repeat focused platform/controller/host commands; expected executable and Node resource/dependency assertions pass with correct boundary errors. +- [x] **Step 5: Refactor.** Remove the two obsolete private copier policies and basename-based secret filtering; keep provider selection and ownership at callers and filesystem behavior in the common module. Avoid adding environment policy configuration. +- [x] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_bootstrap::tests`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp_capability::tests`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_config`; the two `created_path_ledger`/`created_path_ownership` commands from Step 2. Expected all pass with positive counts, including at least three tests per new ledger/ownership module. These cover the shared policy, actual Amplifier creator/callsite and every existing caller. +- [x] **Step 7: Commit.** `git add crates/freshell-platform/src/provider_environment.rs crates/freshell-platform/src/lib.rs crates/freshell-server/src/managed_provider_bootstrap.rs crates/freshell-server/src/managed_mcp_capability.rs crates/freshell-session-host/src/main.rs crates/freshell-session-host/src/providers/mod.rs crates/freshell-sessions/src/amplifier_stub.rs` then `git commit -m "fix: preserve bounded managed provider resources and executable modes"`. ### Task 2: Origin-correct configuration references through managed file grants From 37201d732c46b59c14309c04c48992bca16121ec Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 00:47:02 -0700 Subject: [PATCH 011/493] Share finite provider environment grant decoding and bound actual reads --- Cargo.lock | 1 + crates/freshell-platform/Cargo.toml | 1 + crates/freshell-platform/src/lib.rs | 1 + .../src/provider_environment_grants.rs | 545 ++++++++++++++++++ .../src/provider_secret_resolution.rs | 406 +------------ 5 files changed, 576 insertions(+), 378 deletions(-) create mode 100644 crates/freshell-platform/src/provider_environment_grants.rs diff --git a/Cargo.lock b/Cargo.lock index dcafa75d0..3bb2bce5a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1385,6 +1385,7 @@ dependencies = [ "percent-encoding", "serde_json", "tempfile", + "url", ] [[package]] diff --git a/crates/freshell-platform/Cargo.toml b/crates/freshell-platform/Cargo.toml index fea0e0db2..47d19e701 100644 --- a/crates/freshell-platform/Cargo.toml +++ b/crates/freshell-platform/Cargo.toml @@ -12,6 +12,7 @@ publish.workspace = true [dependencies] percent-encoding = "2" serde_json = { workspace = true } +url = "2" [dev-dependencies] tempfile = "3" diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index f020f4192..6c4b18c07 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -62,6 +62,7 @@ pub mod opencode_config; pub mod opencode_plugin; pub mod path; pub mod provider_environment; +pub mod provider_environment_grants; pub mod resume_gate; pub mod spawn; diff --git a/crates/freshell-platform/src/provider_environment_grants.rs b/crates/freshell-platform/src/provider_environment_grants.rs new file mode 100644 index 000000000..7bb6b846b --- /dev/null +++ b/crates/freshell-platform/src/provider_environment_grants.rs @@ -0,0 +1,545 @@ +//! Pure finite OneCLI environment decoding shared by controller and session host. +//! Resolved values are transient child-process data and must never be logged. +use std::{collections::BTreeMap, path::Path}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum EnvironmentProfile { + Claude, + Codex, + Opencode, + Amplifier, + AmplifierLunarouteGlm53, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum GrantOperation { + Read, + Decode, +} + +#[derive(Debug)] +pub struct GrantError { + pub operation: GrantOperation, + pub profile: EnvironmentProfile, + pub field: Option<&'static str>, + pub source_ordinal: Option, + reason: String, +} +impl std::fmt::Display for GrantError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "{:?} {:?} environment grant: {}", + self.operation, self.profile, self.reason + ) + } +} +impl std::error::Error for GrantError {} + +fn error(profile: EnvironmentProfile, operation: GrantOperation, reason: String) -> GrantError { + let source_ordinal = reason + .strip_prefix("OneCLI environment grant line ") + .and_then(|tail| tail.split_whitespace().next()) + .and_then(|number| number.parse().ok()); + let field = ALLOWED_NAMES + .iter() + .copied() + .find(|name| reason.contains(name)); + GrantError { + operation, + profile, + field, + source_ordinal, + reason, + } +} + +pub fn resolve_environment_profile( + profile: EnvironmentProfile, + raw: &str, +) -> Result, GrantError> { + if raw.len() > MAX_KEYS_BYTES as usize { + return Err(error( + profile, + GrantOperation::Decode, + "grant exceeds 128 KiB".into(), + )); + } + decode_profile(profile, raw).map_err(|reason| error(profile, GrantOperation::Decode, reason)) +} + +/// Charge actual bytes, including source growth after metadata inspection. +fn read_bounded( + reader: impl std::io::Read, + profile: EnvironmentProfile, +) -> Result { + use std::io::Read; + let mut bytes = Vec::new(); + reader + .take(MAX_KEYS_BYTES + 1) + .read_to_end(&mut bytes) + .map_err(|_| error(profile, GrantOperation::Read, "grant read failed".into()))?; + if bytes.len() > MAX_KEYS_BYTES as usize { + return Err(error( + profile, + GrantOperation::Read, + "grant exceeds 128 KiB".into(), + )); + } + String::from_utf8(bytes) + .map_err(|_| error(profile, GrantOperation::Decode, "grant is not UTF-8".into())) +} + +pub fn read_environment_grant( + path: &Path, + profile: EnvironmentProfile, +) -> Result, GrantError> { + let fail = |reason: &str| error(profile, GrantOperation::Read, reason.into()); + let canonical = std::fs::canonicalize(path).map_err(|_| fail("grant is unavailable"))?; + let metadata = std::fs::symlink_metadata(path).map_err(|_| fail("grant is unavailable"))?; + if canonical != path || !metadata.is_file() || metadata.file_type().is_symlink() { + return Err(fail("grant must be a canonical regular private file")); + } + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + if metadata.permissions().mode() & 0o077 != 0 { + return Err(fail("grant must be a canonical regular private file")); + } + } + let file = std::fs::File::open(path).map_err(|_| fail("grant is unavailable"))?; + let raw = read_bounded(file, profile)?; + resolve_environment_profile(profile, &raw) +} + +const MAX_KEYS_BYTES: u64 = 128 * 1024; +const MAX_VALUE_BYTES: usize = 16 * 1024; +const ALLOWED_NAMES: &[&str] = &[ + "ANTHROPIC_API_KEY", + "ANTHROPIC_BASE_URL", + "CLAUDE_CODE_OAUTH_TOKEN", + "CURL_CA_BUNDLE", + "GLM_RUNPOD_API_KEY", + "GLM_RUNPOD_BASE_URL", + "GEMINI_API_KEY", + "GOOGLE_API_KEY", + "GOOGLE_GENERATIVE_AI_API_KEY", + "HTTP_PROXY", + "HTTPS_PROXY", + "LUNAROUTE_API_KEY", + "LUNAROUTE_BASE_URL", + "NODE_EXTRA_CA_CERTS", + "NODE_USE_ENV_PROXY", + "NO_PROXY", + "ONECLI_GATEWAY", + "ONECLI_URL", + "OPENAI_API_KEY", + "OPENAI_BASE_URL", + "OPENCODE_API_KEY", + "OPENROUTER_API_KEY", + "REQUESTS_CA_BUNDLE", + "SSL_CERT_FILE", + "http_proxy", + "https_proxy", + "no_proxy", +]; +const CERTIFICATE_CHILD_NAMES: &[&str] = &[ + "CURL_CA_BUNDLE", + "NODE_EXTRA_CA_CERTS", + "REQUESTS_CA_BUNDLE", + "SSL_CERT_FILE", +]; + +fn decode_profile( + profile: EnvironmentProfile, + raw: &str, +) -> Result, String> { + let parsed = parse_keys_env(raw)?; + match profile { + EnvironmentProfile::Claude => resolve_provider_environment( + &parsed, + &[ + "ANTHROPIC_API_KEY", + "ANTHROPIC_BASE_URL", + "CLAUDE_CODE_OAUTH_TOKEN", + ], + false, + ), + EnvironmentProfile::Codex => { + resolve_provider_environment(&parsed, &["OPENAI_API_KEY", "OPENAI_BASE_URL"], false) + } + EnvironmentProfile::Opencode => resolve_provider_environment( + &parsed, + &[ + "ANTHROPIC_API_KEY", + "ANTHROPIC_BASE_URL", + "OPENAI_API_KEY", + "OPENAI_BASE_URL", + "GOOGLE_GENERATIVE_AI_API_KEY", + "GOOGLE_API_KEY", + "GEMINI_API_KEY", + "OPENROUTER_API_KEY", + "OPENCODE_API_KEY", + ], + true, + ), + EnvironmentProfile::Amplifier => resolve_provider_environment( + &parsed, + &[ + "ANTHROPIC_API_KEY", + "ANTHROPIC_BASE_URL", + "OPENAI_API_KEY", + "OPENAI_BASE_URL", + "LUNAROUTE_API_KEY", + "LUNAROUTE_BASE_URL", + "GLM_RUNPOD_API_KEY", + "GLM_RUNPOD_BASE_URL", + "GOOGLE_API_KEY", + "GEMINI_API_KEY", + "OPENROUTER_API_KEY", + ], + false, + ), + EnvironmentProfile::AmplifierLunarouteGlm53 => { + // The approved proxy may replace a provider placeholder key, so + // only non-emptiness is required here. The credentialed gateway + // URL itself must never retain its operator placeholder. + let api_key = required(&parsed, "LUNAROUTE_API_KEY")?; + let upstream = required(&parsed, "LUNAROUTE_BASE_URL")?; + validate_https_upstream(upstream)?; + let gateway = required(&parsed, "ONECLI_GATEWAY")?; + if gateway != "1" && !gateway.eq_ignore_ascii_case("enabled") { + return Err( + "Amplifier OneCLI profile requires ONECLI_GATEWAY to be enabled".into(), + ); + } + let upper_proxy = parsed.get("HTTPS_PROXY").map(String::as_str); + let lower_proxy = parsed.get("https_proxy").map(String::as_str); + if matches!((upper_proxy, lower_proxy), (Some(upper), Some(lower)) if upper != lower) { + return Err("Amplifier OneCLI HTTPS_PROXY and https_proxy values conflict".into()); + } + let proxy = upper_proxy.or(lower_proxy).ok_or_else(|| { + "Amplifier OneCLI profile requires a container-reachable HTTPS proxy".to_string() + })?; + reject_proxy_placeholder(proxy)?; + validate_container_proxy(proxy)?; + + // ONECLI_URL is a host-local control endpoint in the approved + // setup. Do not forward it into the container: 127.0.0.1 there is + // the soul, not the host gateway. NO_PROXY is also deliberately + // omitted so the approved upstream cannot bypass the gateway. + // Provider-vllm needs only its upstream/API-key pair plus the + // canonical proxy and certificate transport. + let mut child = BTreeMap::new(); + child.insert("VLLM_API_KEY".into(), api_key.to_string()); + child.insert("VLLM_BASE_URL".into(), upstream.to_string()); + child.insert("HTTPS_PROXY".into(), proxy.to_string()); + child.insert("https_proxy".into(), proxy.to_string()); + for name in CERTIFICATE_CHILD_NAMES { + if let Some(value) = parsed.get(*name) { + child.insert((*name).to_string(), value.clone()); + } + } + Ok(child) + } + } +} + +fn resolve_provider_environment( + parsed: &BTreeMap, + provider_names: &[&str], + allow_opencode_proxy_settings: bool, +) -> Result, String> { + let mut child = BTreeMap::new(); + for (name, value) in parsed { + if provider_names.contains(&name.as_str()) + || CERTIFICATE_CHILD_NAMES.contains(&name.as_str()) + { + child.insert(name.clone(), value.clone()); + } else if matches!(name.as_str(), "HTTPS_PROXY" | "https_proxy") { + reject_proxy_placeholder(value)?; + validate_container_proxy(value)?; + child.insert(name.clone(), value.clone()); + } else if matches!(name.as_str(), "HTTP_PROXY" | "http_proxy") { + if allow_opencode_proxy_settings { + reject_proxy_placeholder(value)?; + validate_container_proxy(value)?; + child.insert(name.clone(), value.clone()); + } + } else if name == "NODE_USE_ENV_PROXY" { + if allow_opencode_proxy_settings { + if value != "1" { + return Err("NODE_USE_ENV_PROXY must be set to 1".into()); + } + child.insert(name.clone(), value.clone()); + } + } else if matches!(name.as_str(), "NO_PROXY" | "no_proxy") { + if allow_opencode_proxy_settings { + validate_loopback_no_proxy(value)?; + child.insert(name.clone(), value.clone()); + } + } else if !matches!(name.as_str(), "ONECLI_URL" | "ONECLI_GATEWAY") { + return Err(format!("OneCLI environment profile does not allow {name}")); + } + } + if child.is_empty() + || !child + .keys() + .any(|name| provider_names.contains(&name.as_str())) + { + return Err("OneCLI environment profile has no provider values".into()); + } + for (upper_name, lower_name, display_name) in [ + ("HTTPS_PROXY", "https_proxy", "HTTPS_PROXY"), + ("HTTP_PROXY", "http_proxy", "HTTP_PROXY"), + ("NO_PROXY", "no_proxy", "NO_PROXY"), + ] { + if let (Some(upper), Some(lower)) = (child.get(upper_name), child.get(lower_name)) { + if upper != lower { + return Err(format!("OneCLI {display_name} values conflict")); + } + } + } + Ok(child) +} + +fn validate_loopback_no_proxy(value: &str) -> Result<(), String> { + let entries = value + .split(',') + .map(|entry| entry.trim().to_ascii_lowercase()) + .collect::>(); + if entries.is_empty() + || entries + .iter() + .any(|entry| !matches!(entry.as_str(), "localhost" | "127.0.0.1" | "::1" | "[::1]")) + { + return Err("OneCLI NO_PROXY may bypass only loopback hosts".into()); + } + Ok(()) +} + +fn validate_https_upstream(value: &str) -> Result<(), String> { + let url = url::Url::parse(value) + .map_err(|_| "LUNAROUTE_BASE_URL must be a valid credential-free HTTPS URL".to_string())?; + if url.scheme() != "https" + || url.host_str().is_none() + || !url.username().is_empty() + || url.password().is_some() + || url.query().is_some() + || url.fragment().is_some() + { + return Err("LUNAROUTE_BASE_URL must be a credential-free HTTPS URL".into()); + } + Ok(()) +} + +fn validate_container_proxy(value: &str) -> Result<(), String> { + let url = url::Url::parse(value) + .map_err(|_| "OneCLI proxy value must be a valid HTTP(S) URL".to_string())?; + if !matches!(url.scheme(), "http" | "https") || url.host_str().is_none() { + return Err("HTTPS_PROXY must be a valid HTTP(S) URL".into()); + } + let host = url.host_str().unwrap_or_default().to_ascii_lowercase(); + if host == "localhost" || host == "::1" || host.starts_with("127.") { + return Err( + "OneCLI proxy cannot use container loopback; use the approved proxy address reachable from the managed bridge" + .into(), + ); + } + if url.query().is_some() || url.fragment().is_some() { + return Err("OneCLI proxy cannot contain query or fragment data".into()); + } + Ok(()) +} + +fn parse_keys_env(raw: &str) -> Result, String> { + let mut values = BTreeMap::new(); + for (offset, original) in raw.lines().enumerate() { + let line_number = offset + 1; + let mut line = original.trim(); + if line.is_empty() || line.starts_with('#') { + continue; + } + if let Some(rest) = line.strip_prefix("export ") { + line = rest.trim_start(); + } + let (name, encoded) = line.split_once('=').ok_or_else(|| { + format!("OneCLI environment grant line {line_number} must be NAME=VALUE") + })?; + if !ALLOWED_NAMES.contains(&name) { + return Err(format!( + "OneCLI environment grant line {line_number} uses an unsupported name" + )); + } + if values.contains_key(name) { + return Err(format!( + "OneCLI environment grant line {line_number} duplicates {name}" + )); + } + let value = decode_value(encoded, &values, line_number)?; + if value.is_empty() || value.len() > MAX_VALUE_BYTES { + return Err(format!( + "OneCLI environment grant line {line_number} has an empty or oversized value" + )); + } + values.insert(name.to_string(), value); + } + Ok(values) +} + +fn decode_value( + encoded: &str, + prior: &BTreeMap, + line_number: usize, +) -> Result { + if !encoded.is_ascii() + || encoded.contains('`') + || encoded.contains("$(") + || encoded.contains('\0') + { + return Err(format!( + "OneCLI environment grant line {line_number} contains unsupported encoding or shell execution syntax" + )); + } + if encoded.starts_with('\'') { + if encoded.len() < 2 || !encoded.ends_with('\'') { + return Err(format!( + "OneCLI environment grant line {line_number} has malformed single quotes" + )); + } + return Ok(encoded[1..encoded.len() - 1].to_string()); + } + let value = if encoded.starts_with('"') { + if encoded.len() < 2 || !encoded.ends_with('"') { + return Err(format!( + "OneCLI environment grant line {line_number} has malformed double quotes" + )); + } + &encoded[1..encoded.len() - 1] + } else { + if encoded.chars().any(char::is_whitespace) { + return Err(format!( + "OneCLI environment grant line {line_number} must quote whitespace" + )); + } + encoded + }; + expand_references(value, prior, line_number) +} + +fn expand_references( + value: &str, + prior: &BTreeMap, + line_number: usize, +) -> Result { + let bytes = value.as_bytes(); + let mut out = String::new(); + let mut index = 0; + while index < bytes.len() { + if bytes[index] != b'$' { + out.push(bytes[index] as char); + index += 1; + continue; + } + let (name, next) = if bytes.get(index + 1) == Some(&b'{') { + let close = value[index + 2..].find('}').ok_or_else(|| { + format!("OneCLI environment grant line {line_number} has an unterminated reference") + })? + index + + 2; + (&value[index + 2..close], close + 1) + } else { + let start = index + 1; + let mut end = start; + while end < bytes.len() && (bytes[end].is_ascii_alphanumeric() || bytes[end] == b'_') { + end += 1; + } + (&value[start..end], end) + }; + if name.is_empty() || !ALLOWED_NAMES.contains(&name) { + return Err(format!( + "OneCLI environment grant line {line_number} has an unsupported reference" + )); + } + let resolved = prior.get(name).ok_or_else(|| { + format!("OneCLI environment grant line {line_number} references {name} before it is defined") + })?; + if out + .len() + .checked_add(resolved.len()) + .is_none_or(|size| size > MAX_VALUE_BYTES) + { + return Err(format!( + "OneCLI environment grant line {line_number} has an oversized value" + )); + } + out.push_str(resolved); + index = next; + } + Ok(out) +} + +fn reject_proxy_placeholder(value: &str) -> Result<(), String> { + let lower = value.to_ascii_lowercase(); + if lower.contains("") + || lower.contains("placeholder") + || lower.contains("changeme") + { + return Err("Amplifier OneCLI HTTPS proxy still contains an operator placeholder".into()); + } + Ok(()) +} + +fn required<'a>(values: &'a BTreeMap, name: &str) -> Result<&'a str, String> { + values + .get(name) + .map(String::as_str) + .ok_or_else(|| format!("Amplifier OneCLI profile requires {name} in keys.env")) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn oversized_grant_is_rejected_before_decode_even_when_comments_are_valid() { + let temp = tempfile::tempdir().unwrap(); + let file = temp.path().join("grant.env"); + let mut raw = "OPENAI_API_KEY=synthetic-only\n#".to_string(); + raw.push_str(&"x".repeat(128 * 1024)); + std::fs::write(&file, raw).unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + assert!(read_environment_grant(&file, EnvironmentProfile::Codex).is_err()); + } + #[test] + fn reader_charges_actual_growth_beyond_metadata_size() { + let reader = std::io::Cursor::new(vec![b'#'; 128 * 1024 + 1]); + let err = read_bounded(reader, EnvironmentProfile::Codex).unwrap_err(); + assert_eq!(err.operation, GrantOperation::Read); + } + #[test] + fn ordered_interpolation_and_provider_rules_remain_identical() { + let env=resolve_environment_profile(EnvironmentProfile::Codex, + "OPENAI_API_KEY='fixture'\nOPENAI_BASE_URL=https://example.invalid\nSSL_CERT_FILE=/runtime/ca\nREQUESTS_CA_BUNDLE=$SSL_CERT_FILE\n").unwrap(); + assert_eq!(env["REQUESTS_CA_BUNDLE"], "/runtime/ca"); + for raw in [ + "ANTHROPIC_API_KEY=fixture", + "OPENAI_API_KEY=$(synthetic)", + "OPENAI_API_KEY=one\nOPENAI_API_KEY=two", + "OPENAI_API_KEY=fixture\nHTTPS_PROXY=http://127.0.0.1:9", + ] { + assert!(resolve_environment_profile(EnvironmentProfile::Codex, raw).is_err()); + } + } + #[test] + fn unsupported_name_cannot_echo_private_input() { + let err = resolve_environment_profile( + EnvironmentProfile::Codex, + "sentinel-private-candidate=sentinel-private-value", + ) + .unwrap_err(); + assert!(!format!("{err:?} {err}").contains("sentinel-private")); + } +} diff --git a/crates/freshell-session-host/src/provider_secret_resolution.rs b/crates/freshell-session-host/src/provider_secret_resolution.rs index 5d356424b..8e1a0251e 100644 --- a/crates/freshell-session-host/src/provider_secret_resolution.rs +++ b/crates/freshell-session-host/src/provider_secret_resolution.rs @@ -4,43 +4,6 @@ use std::fs; use std::path::Path; const MAX_KEYS_BYTES: u64 = 128 * 1024; -const MAX_VALUE_BYTES: usize = 16 * 1024; -const ALLOWED_NAMES: &[&str] = &[ - "ANTHROPIC_API_KEY", - "ANTHROPIC_BASE_URL", - "CLAUDE_CODE_OAUTH_TOKEN", - "CURL_CA_BUNDLE", - "GLM_RUNPOD_API_KEY", - "GLM_RUNPOD_BASE_URL", - "GEMINI_API_KEY", - "GOOGLE_API_KEY", - "GOOGLE_GENERATIVE_AI_API_KEY", - "HTTP_PROXY", - "HTTPS_PROXY", - "LUNAROUTE_API_KEY", - "LUNAROUTE_BASE_URL", - "NODE_EXTRA_CA_CERTS", - "NODE_USE_ENV_PROXY", - "NO_PROXY", - "ONECLI_GATEWAY", - "ONECLI_URL", - "OPENAI_API_KEY", - "OPENAI_BASE_URL", - "OPENCODE_API_KEY", - "OPENROUTER_API_KEY", - "REQUESTS_CA_BUNDLE", - "SSL_CERT_FILE", - "http_proxy", - "https_proxy", - "no_proxy", -]; -const CERTIFICATE_CHILD_NAMES: &[&str] = &[ - "CURL_CA_BUNDLE", - "NODE_EXTRA_CA_CERTS", - "REQUESTS_CA_BUNDLE", - "SSL_CERT_FILE", -]; - #[derive(Default)] pub struct ResolvedProviderSecrets { pub environment: BTreeMap, @@ -66,7 +29,16 @@ pub fn resolve_child_secrets( { return Err("OneCLI grant must be a bounded regular file".into()); } - let raw = fs::read(&mount).map_err(|error| format!("read OneCLI grant: {error}"))?; + use std::io::Read; + let mut raw = Vec::new(); + fs::File::open(&mount) + .map_err(|_| "OneCLI grant is unavailable")? + .take(MAX_KEYS_BYTES + 1) + .read_to_end(&mut raw) + .map_err(|_| "read OneCLI grant failed")?; + if raw.len() > MAX_KEYS_BYTES as usize { + return Err("OneCLI grant exceeds 128 KiB".into()); + } if let Some(relative) = reference.profile.provider_file_relative_path() { if raw.is_empty() { return Err("OneCLI provider-file grant is empty".into()); @@ -85,351 +57,29 @@ pub fn resolve_child_secrets( Ok(resolved) } -fn resolve_profile( +fn environment_profile( profile: ProviderSecretProfile, - raw: &str, -) -> Result, String> { - let parsed = parse_keys_env(raw)?; +) -> Result { + use freshell_platform::provider_environment_grants::EnvironmentProfile; match profile { - ProviderSecretProfile::ClaudeOnecliEnvironment => { - resolve_provider_environment( - &parsed, - &["ANTHROPIC_API_KEY", "ANTHROPIC_BASE_URL", "CLAUDE_CODE_OAUTH_TOKEN"], - false, - ) - } - ProviderSecretProfile::CodexOnecliEnvironment => { - resolve_provider_environment(&parsed, &["OPENAI_API_KEY", "OPENAI_BASE_URL"], false) - } - ProviderSecretProfile::OpencodeOnecliEnvironment => { - resolve_provider_environment( - &parsed, - &[ - "ANTHROPIC_API_KEY", - "ANTHROPIC_BASE_URL", - "OPENAI_API_KEY", - "OPENAI_BASE_URL", - "GOOGLE_GENERATIVE_AI_API_KEY", - "GOOGLE_API_KEY", - "GEMINI_API_KEY", - "OPENROUTER_API_KEY", - "OPENCODE_API_KEY", - ], - true, - ) - } - ProviderSecretProfile::AmplifierOnecliEnvironment => { - resolve_provider_environment( - &parsed, - &[ - "ANTHROPIC_API_KEY", - "ANTHROPIC_BASE_URL", - "OPENAI_API_KEY", - "OPENAI_BASE_URL", - "LUNAROUTE_API_KEY", - "LUNAROUTE_BASE_URL", - "GLM_RUNPOD_API_KEY", - "GLM_RUNPOD_BASE_URL", - "GOOGLE_API_KEY", - "GEMINI_API_KEY", - "OPENROUTER_API_KEY", - ], - false, - ) - } - ProviderSecretProfile::ClaudeOnecliAuthFile - | ProviderSecretProfile::CodexOnecliAuthFile - | ProviderSecretProfile::OpencodeOnecliAuthFile - | ProviderSecretProfile::OpencodeOnecliCaFile - | ProviderSecretProfile::AmplifierOnecliKeysFile => Err("OneCLI auth-file grant cannot be parsed as an environment profile".into()), - ProviderSecretProfile::AmplifierOnecliLunarouteGlm53 => { - // The approved proxy may replace a provider placeholder key, so - // only non-emptiness is required here. The credentialed gateway - // URL itself must never retain its operator placeholder. - let api_key = required(&parsed, "LUNAROUTE_API_KEY")?; - let upstream = required(&parsed, "LUNAROUTE_BASE_URL")?; - validate_https_upstream(upstream)?; - let gateway = required(&parsed, "ONECLI_GATEWAY")?; - if gateway != "1" && !gateway.eq_ignore_ascii_case("enabled") { - return Err( - "Amplifier OneCLI profile requires ONECLI_GATEWAY to be enabled".into(), - ); - } - let upper_proxy = parsed.get("HTTPS_PROXY").map(String::as_str); - let lower_proxy = parsed.get("https_proxy").map(String::as_str); - if matches!((upper_proxy, lower_proxy), (Some(upper), Some(lower)) if upper != lower) { - return Err("Amplifier OneCLI HTTPS_PROXY and https_proxy values conflict".into()); - } - let proxy = upper_proxy.or(lower_proxy).ok_or_else(|| { - "Amplifier OneCLI profile requires a container-reachable HTTPS proxy".to_string() - })?; - reject_proxy_placeholder(proxy)?; - validate_container_proxy(proxy)?; - - // ONECLI_URL is a host-local control endpoint in the approved - // setup. Do not forward it into the container: 127.0.0.1 there is - // the soul, not the host gateway. NO_PROXY is also deliberately - // omitted so the approved upstream cannot bypass the gateway. - // Provider-vllm needs only its upstream/API-key pair plus the - // canonical proxy and certificate transport. - let mut child = BTreeMap::new(); - child.insert("VLLM_API_KEY".into(), api_key.to_string()); - child.insert("VLLM_BASE_URL".into(), upstream.to_string()); - child.insert("HTTPS_PROXY".into(), proxy.to_string()); - child.insert("https_proxy".into(), proxy.to_string()); - for name in CERTIFICATE_CHILD_NAMES { - if let Some(value) = parsed.get(*name) { - child.insert((*name).to_string(), value.clone()); - } - } - Ok(child) - } - ProviderSecretProfile::LegacyAmplifierOnecliAnthropicHaikuLow => Err( - "legacy Amplifier Anthropic/Haiku secret profile is no longer supported; recreate the managed runtime with the approved OneCLI/LunaRoute profile".into(), - ), + ProviderSecretProfile::ClaudeOnecliEnvironment => Ok(EnvironmentProfile::Claude), + ProviderSecretProfile::CodexOnecliEnvironment => Ok(EnvironmentProfile::Codex), + ProviderSecretProfile::OpencodeOnecliEnvironment => Ok(EnvironmentProfile::Opencode), + ProviderSecretProfile::AmplifierOnecliEnvironment => Ok(EnvironmentProfile::Amplifier), + ProviderSecretProfile::AmplifierOnecliLunarouteGlm53 => Ok(EnvironmentProfile::AmplifierLunarouteGlm53), + ProviderSecretProfile::LegacyAmplifierOnecliAnthropicHaikuLow => Err("legacy Amplifier Anthropic/Haiku secret profile is no longer supported; recreate the managed runtime with the approved OneCLI/LunaRoute profile".into()), + _ => Err("OneCLI auth-file grant cannot be parsed as an environment profile".into()), } } - -fn resolve_provider_environment( - parsed: &BTreeMap, - provider_names: &[&str], - allow_opencode_proxy_settings: bool, +fn resolve_profile( + profile: ProviderSecretProfile, + raw: &str, ) -> Result, String> { - let mut child = BTreeMap::new(); - for (name, value) in parsed { - if provider_names.contains(&name.as_str()) - || CERTIFICATE_CHILD_NAMES.contains(&name.as_str()) - { - child.insert(name.clone(), value.clone()); - } else if matches!(name.as_str(), "HTTPS_PROXY" | "https_proxy") { - reject_proxy_placeholder(value)?; - validate_container_proxy(value)?; - child.insert(name.clone(), value.clone()); - } else if matches!(name.as_str(), "HTTP_PROXY" | "http_proxy") { - if allow_opencode_proxy_settings { - reject_proxy_placeholder(value)?; - validate_container_proxy(value)?; - child.insert(name.clone(), value.clone()); - } - } else if name == "NODE_USE_ENV_PROXY" { - if allow_opencode_proxy_settings { - if value != "1" { - return Err("NODE_USE_ENV_PROXY must be set to 1".into()); - } - child.insert(name.clone(), value.clone()); - } - } else if matches!(name.as_str(), "NO_PROXY" | "no_proxy") { - if allow_opencode_proxy_settings { - validate_loopback_no_proxy(value)?; - child.insert(name.clone(), value.clone()); - } - } else if !matches!(name.as_str(), "ONECLI_URL" | "ONECLI_GATEWAY") { - return Err(format!("OneCLI environment profile does not allow {name}")); - } - } - if child.is_empty() - || !child - .keys() - .any(|name| provider_names.contains(&name.as_str())) - { - return Err("OneCLI environment profile has no provider values".into()); - } - for (upper_name, lower_name, display_name) in [ - ("HTTPS_PROXY", "https_proxy", "HTTPS_PROXY"), - ("HTTP_PROXY", "http_proxy", "HTTP_PROXY"), - ("NO_PROXY", "no_proxy", "NO_PROXY"), - ] { - if let (Some(upper), Some(lower)) = (child.get(upper_name), child.get(lower_name)) { - if upper != lower { - return Err(format!("OneCLI {display_name} values conflict")); - } - } - } - Ok(child) -} - -fn validate_loopback_no_proxy(value: &str) -> Result<(), String> { - let entries = value - .split(',') - .map(|entry| entry.trim().to_ascii_lowercase()) - .collect::>(); - if entries.is_empty() - || entries - .iter() - .any(|entry| !matches!(entry.as_str(), "localhost" | "127.0.0.1" | "::1" | "[::1]")) - { - return Err("OneCLI NO_PROXY may bypass only loopback hosts".into()); - } - Ok(()) -} - -fn validate_https_upstream(value: &str) -> Result<(), String> { - let url = url::Url::parse(value) - .map_err(|_| "LUNAROUTE_BASE_URL must be a valid credential-free HTTPS URL".to_string())?; - if url.scheme() != "https" - || url.host_str().is_none() - || !url.username().is_empty() - || url.password().is_some() - || url.query().is_some() - || url.fragment().is_some() - { - return Err("LUNAROUTE_BASE_URL must be a credential-free HTTPS URL".into()); - } - Ok(()) -} - -fn validate_container_proxy(value: &str) -> Result<(), String> { - let url = url::Url::parse(value) - .map_err(|_| "OneCLI proxy value must be a valid HTTP(S) URL".to_string())?; - if !matches!(url.scheme(), "http" | "https") || url.host_str().is_none() { - return Err("HTTPS_PROXY must be a valid HTTP(S) URL".into()); - } - let host = url.host_str().unwrap_or_default().to_ascii_lowercase(); - if host == "localhost" || host == "::1" || host.starts_with("127.") { - return Err( - "OneCLI proxy cannot use container loopback; use the approved proxy address reachable from the managed bridge" - .into(), - ); - } - if url.query().is_some() || url.fragment().is_some() { - return Err("OneCLI proxy cannot contain query or fragment data".into()); - } - Ok(()) -} - -fn parse_keys_env(raw: &str) -> Result, String> { - let mut values = BTreeMap::new(); - for (offset, original) in raw.lines().enumerate() { - let line_number = offset + 1; - let mut line = original.trim(); - if line.is_empty() || line.starts_with('#') { - continue; - } - if let Some(rest) = line.strip_prefix("export ") { - line = rest.trim_start(); - } - let (name, encoded) = line.split_once('=').ok_or_else(|| { - format!("OneCLI environment grant line {line_number} must be NAME=VALUE") - })?; - if !ALLOWED_NAMES.contains(&name) { - return Err(format!( - "OneCLI environment grant line {line_number} uses unsupported name {name:?}" - )); - } - if values.contains_key(name) { - return Err(format!( - "OneCLI environment grant line {line_number} duplicates {name}" - )); - } - let value = decode_value(encoded, &values, line_number)?; - if value.is_empty() || value.len() > MAX_VALUE_BYTES { - return Err(format!( - "OneCLI environment grant line {line_number} has an empty or oversized value" - )); - } - values.insert(name.to_string(), value); - } - Ok(values) -} - -fn decode_value( - encoded: &str, - prior: &BTreeMap, - line_number: usize, -) -> Result { - if !encoded.is_ascii() - || encoded.contains('`') - || encoded.contains("$(") - || encoded.contains('\0') - { - return Err(format!( - "OneCLI environment grant line {line_number} contains unsupported encoding or shell execution syntax" - )); - } - if encoded.starts_with('\'') { - if encoded.len() < 2 || !encoded.ends_with('\'') { - return Err(format!( - "OneCLI environment grant line {line_number} has malformed single quotes" - )); - } - return Ok(encoded[1..encoded.len() - 1].to_string()); - } - let value = if encoded.starts_with('"') { - if encoded.len() < 2 || !encoded.ends_with('"') { - return Err(format!( - "OneCLI environment grant line {line_number} has malformed double quotes" - )); - } - &encoded[1..encoded.len() - 1] - } else { - if encoded.chars().any(char::is_whitespace) { - return Err(format!( - "OneCLI environment grant line {line_number} must quote whitespace" - )); - } - encoded - }; - expand_references(value, prior, line_number) -} - -fn expand_references( - value: &str, - prior: &BTreeMap, - line_number: usize, -) -> Result { - let bytes = value.as_bytes(); - let mut out = String::new(); - let mut index = 0; - while index < bytes.len() { - if bytes[index] != b'$' { - out.push(bytes[index] as char); - index += 1; - continue; - } - let (name, next) = if bytes.get(index + 1) == Some(&b'{') { - let close = value[index + 2..].find('}').ok_or_else(|| { - format!("OneCLI environment grant line {line_number} has an unterminated reference") - })? + index - + 2; - (&value[index + 2..close], close + 1) - } else { - let start = index + 1; - let mut end = start; - while end < bytes.len() && (bytes[end].is_ascii_alphanumeric() || bytes[end] == b'_') { - end += 1; - } - (&value[start..end], end) - }; - if name.is_empty() || !ALLOWED_NAMES.contains(&name) { - return Err(format!( - "OneCLI environment grant line {line_number} has unsupported reference {name:?}" - )); - } - let resolved = prior.get(name).ok_or_else(|| { - format!("OneCLI environment grant line {line_number} references {name} before it is defined") - })?; - out.push_str(resolved); - index = next; - } - Ok(out) -} - -fn reject_proxy_placeholder(value: &str) -> Result<(), String> { - let lower = value.to_ascii_lowercase(); - if lower.contains("") - || lower.contains("placeholder") - || lower.contains("changeme") - { - return Err("Amplifier OneCLI HTTPS proxy still contains an operator placeholder".into()); - } - Ok(()) -} - -fn required<'a>(values: &'a BTreeMap, name: &str) -> Result<&'a str, String> { - values - .get(name) - .map(String::as_str) - .ok_or_else(|| format!("Amplifier OneCLI profile requires {name} in keys.env")) + freshell_platform::provider_environment_grants::resolve_environment_profile( + environment_profile(profile)?, + raw, + ) + .map_err(|error| error.to_string()) } #[cfg(test)] From 4117f0e72d83c1fa54f3b65175a5a2565989fa98 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 00:54:13 -0700 Subject: [PATCH 012/493] Bound finite configuration file grants independently of legacy OneCLI profiles --- crates/freshell-runtime-protocol/src/lib.rs | 136 ++++++++++++++++-- .../src/managed_provider_bootstrap.rs | 1 + crates/freshell-session-host/src/main.rs | 78 +++++++++- .../src/provider_secret_resolution.rs | 52 +++++++ .../src/providers/fresh_agent.rs | 2 + crates/freshell-supervisor/src/backend.rs | 1 + .../freshell-supervisor/src/backend/docker.rs | 16 ++- crates/freshell-supervisor/src/registry.rs | 1 + 8 files changed, 265 insertions(+), 22 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 445c5019e..49e1798b5 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -913,6 +913,10 @@ pub struct ProviderBootstrapFile { #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum ProviderSecretProfile { + ClaudeConfigurationFile, + CodexConfigurationFile, + OpencodeConfigurationFile, + AmplifierConfigurationFile, ClaudeOnecliEnvironment, ClaudeOnecliAuthFile, CodexOnecliEnvironment, @@ -936,18 +940,34 @@ pub enum ProviderSecretProfile { impl ProviderSecretProfile { pub fn provider(self) -> &'static str { match self { - Self::ClaudeOnecliEnvironment | Self::ClaudeOnecliAuthFile => "claude", - Self::CodexOnecliEnvironment | Self::CodexOnecliAuthFile => "codex", + Self::ClaudeOnecliEnvironment + | Self::ClaudeOnecliAuthFile + | Self::ClaudeConfigurationFile => "claude", + Self::CodexOnecliEnvironment + | Self::CodexOnecliAuthFile + | Self::CodexConfigurationFile => "codex", Self::OpencodeOnecliEnvironment | Self::OpencodeOnecliAuthFile - | Self::OpencodeOnecliCaFile => "opencode", + | Self::OpencodeOnecliCaFile + | Self::OpencodeConfigurationFile => "opencode", Self::AmplifierOnecliEnvironment + | Self::AmplifierConfigurationFile | Self::AmplifierOnecliKeysFile | Self::AmplifierOnecliLunarouteGlm53 | Self::LegacyAmplifierOnecliAnthropicHaikuLow => "amplifier", } } + pub fn is_configuration_file(self) -> bool { + matches!( + self, + Self::ClaudeConfigurationFile + | Self::CodexConfigurationFile + | Self::OpencodeConfigurationFile + | Self::AmplifierConfigurationFile + ) + } + pub fn auth_relative_path(self) -> Option<&'static str> { match self { Self::ClaudeOnecliAuthFile => Some(".claude/.credentials.json"), @@ -975,6 +995,8 @@ pub struct ProviderSecretReference { /// session host reads its read-only mount immediately before child spawn. pub source_path: String, pub profile: ProviderSecretProfile, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub provider_relative_path: Option, } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -1481,6 +1503,62 @@ mod provider_launch_context_tests { assert!(launch.validate().is_err()); } + #[test] + fn configuration_file_grants_allow_many_unique_destinations_in_both_launch_routes() { + let references: Vec<_> = (0..128) + .map(|index| ProviderSecretReference { + source_path: format!("/private/reference-{index}"), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{index}")), + }) + .collect(); + assert!(validate_provider_secrets("opencode", &references).is_ok()); + let mut terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"terminal", "streamId":"stream", "mode":"opencode", + "program":"opencode", "cwd":"/workspace", "runAsUid":65534, + "runAsGid":0, "cols":80, "rows":24, "projectKey":"project", + "workspacePath":"/workspace" + })) + .unwrap(); + terminal.provider_secret_references = references.clone(); + assert!(terminal.validate().is_ok()); + let mut fresh: FreshAgentLaunchSpec = serde_json::from_value(serde_json::json!({ + "sessionId":"presentation", "provider":"opencode", "sessionType":"freshopencode", + "runtimeVariant":"pinned", "providerStoreId":"store", "cwd":"/workspace", + "workspacePath":"/workspace", "runAsUid":65534, "runAsGid":0 + })) + .unwrap(); + fresh.provider_secret_references = references; + assert!(fresh.validate().is_ok()); + } + #[test] + fn configuration_file_grants_reject_cross_provider_duplicate_and_unsafe_destinations() { + let reference = ProviderSecretReference { + source_path: "/private/reference".into(), + profile: ProviderSecretProfile::ClaudeConfigurationFile, + provider_relative_path: Some(".freshell/references/one".into()), + }; + assert!(validate_provider_secrets("claude", &[reference.clone()]).is_ok()); + assert!(validate_provider_secrets("codex", &[reference.clone()]).is_err()); + assert!( + validate_provider_secrets("claude", &[reference.clone(), reference.clone()]).is_err() + ); + for path in [ + ".claude/auth.json", + ".freshell/references/../one", + ".freshell/references/one/two", + "/absolute", + ".freshell/references/", + ] { + let mut invalid = reference.clone(); + invalid.provider_relative_path = Some(path.into()); + assert!(validate_provider_secrets("claude", &[invalid]).is_err()); + } + let mut legacy = reference; + legacy.profile = ProviderSecretProfile::ClaudeOnecliAuthFile; + assert!(validate_provider_secrets("claude", &[legacy]).is_err()); + } + #[test] fn onecli_profiles_match_only_their_named_provider() { for (provider, profiles) in [ @@ -1516,6 +1594,7 @@ mod provider_launch_context_tests { ] { for profile in profiles { let references = [ProviderSecretReference { + provider_relative_path: None, source_path: "/private/onecli-grant".into(), profile, }]; @@ -1526,6 +1605,7 @@ mod provider_launch_context_tests { assert!(validate_provider_secrets( "amplifier", &[ProviderSecretReference { + provider_relative_path: None, source_path: "/private/legacy".into(), profile: ProviderSecretProfile::LegacyAmplifierOnecliAnthropicHaikuLow, }] @@ -1692,7 +1772,6 @@ impl FreshAgentLaunchSpec { .is_some_and(|native| native != &reference.session_id) }) || self.provider_bootstrap_files.len() > 16 - || self.provider_secret_references.len() > 4 { return Err(RuntimeError::new( RuntimeErrorCode::InvalidRequest, @@ -1878,10 +1957,7 @@ impl TerminalLaunchSpec { "managed terminal launch has an empty/root-uid required field", )); } - if self.env.len() > 512 - || self.args.len() > 512 - || self.provider_bootstrap_files.len() > 16 - || self.provider_secret_references.len() > 4 + if self.env.len() > 512 || self.args.len() > 512 || self.provider_bootstrap_files.len() > 16 { return Err(RuntimeError::new( RuntimeErrorCode::InvalidRequest, @@ -2139,26 +2215,56 @@ fn safe_selector(value: &str) -> bool { }) } -fn validate_provider_secrets( +pub fn validate_provider_secrets( provider: &str, references: &[ProviderSecretReference], ) -> Result<(), RuntimeError> { - let mut seen = std::collections::BTreeSet::new(); + let invalid = || { + RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "managed provider grant profile, destination or capacity is invalid", + ) + }; + if references.len() > 132 { + return Err(invalid()); + } + let mut legacy_profiles = std::collections::BTreeSet::new(); + let mut destinations = std::collections::BTreeSet::new(); for reference in references { if reference.profile.provider() != provider || !std::path::Path::new(&reference.source_path).is_absolute() || reference.source_path.len() > 1024 || reference.source_path.chars().any(char::is_control) - || !seen.insert(reference.profile as u8) || matches!( reference.profile, ProviderSecretProfile::LegacyAmplifierOnecliAnthropicHaikuLow ) { - return Err(RuntimeError::new( - RuntimeErrorCode::InvalidRequest, - "managed OneCLI profile is unsafe or belongs to another provider", - )); + return Err(invalid()); + } + if reference.profile.is_configuration_file() { + let destination = reference + .provider_relative_path + .as_deref() + .ok_or_else(invalid)?; + let id = destination + .strip_prefix(".freshell/references/") + .ok_or_else(invalid)?; + if id.is_empty() + || id.len() > 128 + || !id + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'-' | b'_')) + || !destinations.insert(destination) + || destinations.len() > 128 + { + return Err(invalid()); + } + } else if reference.provider_relative_path.is_some() + || !legacy_profiles.insert(reference.profile as u8) + || legacy_profiles.len() > 4 + { + return Err(invalid()); } } Ok(()) diff --git a/crates/freshell-server/src/managed_provider_bootstrap.rs b/crates/freshell-server/src/managed_provider_bootstrap.rs index 0a4a4637b..4d2819299 100644 --- a/crates/freshell-server/src/managed_provider_bootstrap.rs +++ b/crates/freshell-server/src/managed_provider_bootstrap.rs @@ -410,6 +410,7 @@ pub fn named_provider_onecli_references( .to_string_lossy() .into_owned(); references.push(ProviderSecretReference { + provider_relative_path: None, source_path, profile, }); diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 61f53a5fe..6d1f8f0a6 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -890,6 +890,7 @@ async fn grant_execution( prepare_provider_files( &terminal.mode, &resolved_secrets.provider_files, + &resolved_secrets.configuration_files, terminal.run_as_uid, terminal.run_as_gid, ) @@ -1008,6 +1009,7 @@ async fn grant_execution( prepare_provider_files( launch.provider.as_str(), &resolved_secrets.provider_files, + &resolved_secrets.configuration_files, launch.run_as_uid, launch.run_as_gid, ) @@ -1938,6 +1940,7 @@ fn prepare_provider_context_at( fn prepare_provider_files( provider: &str, provider_files: &[(&str, Vec)], + configuration_files: &[(String, PathBuf)], run_as_uid: u32, run_as_gid: u32, ) -> Result<(), String> { @@ -1945,7 +1948,7 @@ fn prepare_provider_files( std::fs::create_dir_all(auth_dir).map_err(|error| error.to_string())?; set_mode(auth_dir, 0o711)?; let home = Path::new("/home/freshell/provider"); - if !home.exists() && provider_files.is_empty() { + if !home.exists() && provider_files.is_empty() && configuration_files.is_empty() { return Ok(()); } let legacy_relative = match provider { @@ -2009,6 +2012,79 @@ fn prepare_provider_files( ], )?; } + let mut aggregate = 0u64; + for (index, (relative, source)) in configuration_files.iter().enumerate() { + let destination = home.join(relative); + let target = auth_dir.join(format!("configuration-{index}")); + let temporary = auth_dir.join(format!( + "configuration-{index}-tmp-{}", + uuid::Uuid::new_v4() + )); + let result = (|| { + use std::io::Read; + use std::os::unix::fs::OpenOptionsExt; + let input = std::fs::File::open(source) + .map_err(|_| "configuration-file grant is unavailable")?; + let mut output = std::fs::OpenOptions::new() + .create_new(true) + .write(true) + .mode(0o600) + .open(&temporary) + .map_err(|_| "configuration-file publication failed")?; + let bytes = std::io::copy(&mut input.take(16 * 1024 * 1024 + 1), &mut output) + .map_err(|_| "configuration-file publication read failed")?; + aggregate = aggregate + .checked_add(bytes) + .ok_or("configuration-file size overflow")?; + if bytes > 16 * 1024 * 1024 || aggregate > 32 * 1024 * 1024 { + return Err("configuration-file grant exceeds per-source or aggregate limit"); + } + output + .sync_all() + .map_err(|_| "configuration-file publication sync failed")?; + set_owner(&temporary, run_as_uid, run_as_gid) + .map_err(|_| "configuration-file ownership failed")?; + std::fs::rename(&temporary, &target) + .map_err(|_| "configuration-file publication failed")?; + let parent = destination + .parent() + .ok_or("configuration-file destination has no parent")?; + run_provider_path_command( + run_as_uid, + run_as_gid, + "/usr/bin/mkdir", + &[ + "-p", + "--", + parent + .to_str() + .ok_or("configuration-file parent is not UTF-8")?, + ], + ) + .map_err(|_| "configuration-file parent creation failed")?; + run_provider_path_command( + run_as_uid, + run_as_gid, + "/usr/bin/ln", + &[ + "-sfnT", + "--", + target + .to_str() + .ok_or("configuration-file target is not UTF-8")?, + destination + .to_str() + .ok_or("configuration-file destination is not UTF-8")?, + ], + ) + .map_err(|_| "configuration-file link publication failed")?; + Ok(()) + })(); + if result.is_err() { + let _ = std::fs::remove_file(&temporary); + } + result.map_err(str::to_string)?; + } Ok(()) } diff --git a/crates/freshell-session-host/src/provider_secret_resolution.rs b/crates/freshell-session-host/src/provider_secret_resolution.rs index 8e1a0251e..bc5ad8b4f 100644 --- a/crates/freshell-session-host/src/provider_secret_resolution.rs +++ b/crates/freshell-session-host/src/provider_secret_resolution.rs @@ -8,6 +8,7 @@ const MAX_KEYS_BYTES: u64 = 128 * 1024; pub struct ResolvedProviderSecrets { pub environment: BTreeMap, pub provider_files: Vec<(&'static str, Vec)>, + pub configuration_files: Vec<(String, std::path::PathBuf)>, } pub fn resolve_child_secrets( @@ -15,12 +16,41 @@ pub fn resolve_child_secrets( references: &[ProviderSecretReference], mount_root: &Path, ) -> Result { + freshell_runtime_protocol::validate_provider_secrets(provider, references) + .map_err(|error| error.message)?; let mut resolved = ResolvedProviderSecrets::default(); + let mut configuration_bytes = 0u64; for (index, reference) in references.iter().enumerate() { if reference.profile.provider() != provider { return Err("OneCLI profile belongs to another provider".into()); } let mount = mount_root.join(format!("provider-{index}")); + if reference.profile.is_configuration_file() { + let metadata = fs::symlink_metadata(&mount) + .map_err(|_| "configuration-file grant is unavailable")?; + if !metadata.is_file() + || metadata.file_type().is_symlink() + || metadata.len() > 16 * 1024 * 1024 + { + return Err( + "configuration-file grant must be a bounded regular file (16 MiB)".into(), + ); + } + configuration_bytes = configuration_bytes + .checked_add(metadata.len()) + .ok_or("configuration-file grant size overflow")?; + if configuration_bytes > 32 * 1024 * 1024 { + return Err("configuration-file grants exceed 32 MiB aggregate limit".into()); + } + resolved.configuration_files.push(( + reference + .provider_relative_path + .clone() + .ok_or("configuration-file grant destination missing")?, + mount, + )); + continue; + } let metadata = fs::symlink_metadata(&mount) .map_err(|error| format!("OneCLI grant is unavailable: {error}"))?; if !metadata.is_file() @@ -87,6 +117,22 @@ mod tests { use super::*; use freshell_runtime_protocol::ProviderSecretReference; + #[test] + fn configuration_file_grants_use_large_profile_without_expanding_payload_into_environment() { + let temp = tempfile::tempdir().unwrap(); + std::fs::write(temp.path().join("provider-0"), vec![b'x'; 128 * 1024 + 1]).unwrap(); + let reference = ProviderSecretReference { + source_path: "/synthetic/config-source".into(), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(".freshell/references/0".into()), + }; + let resolved = resolve_child_secrets("opencode", &[reference], temp.path()).unwrap(); + assert!(resolved.environment.is_empty()); + assert!(resolved.provider_files.is_empty()); + assert_eq!(resolved.configuration_files.len(), 1); + assert_eq!(resolved.configuration_files[0].0, ".freshell/references/0"); + } + #[test] fn provider_secret_profiles_accept_only_their_provider_and_keep_controls_out_of_child() { let cases = [ @@ -119,6 +165,7 @@ mod tests { ) .unwrap(); let references = [ProviderSecretReference { + provider_relative_path: None, source_path: root .path() .join("provider-0") @@ -165,6 +212,7 @@ mod tests { ) .unwrap(); let reference = ProviderSecretReference { + provider_relative_path: None, source_path: mount.to_string_lossy().into_owned(), profile: ProviderSecretProfile::OpencodeOnecliEnvironment, }; @@ -242,6 +290,7 @@ mod tests { ) .unwrap(); let reference = ProviderSecretReference { + provider_relative_path: None, source_path: mount.to_string_lossy().into_owned(), profile, }; @@ -296,6 +345,7 @@ mod tests { let mount = root.path().join("provider-0"); fs::write(&mount, "fake-OneCLI-auth-file").unwrap(); let references = [ProviderSecretReference { + provider_relative_path: None, source_path: mount.to_string_lossy().into_owned(), profile, }]; @@ -330,6 +380,7 @@ mod tests { ) .unwrap(); let reference = ProviderSecretReference { + provider_relative_path: None, source_path: mount.to_string_lossy().into_owned(), profile: ProviderSecretProfile::OpencodeOnecliEnvironment, }; @@ -409,6 +460,7 @@ mod tests { #[test] fn serialised_launch_contains_references_but_never_resolved_secret_bytes() { let reference = freshell_runtime_protocol::ProviderSecretReference { + provider_relative_path: None, source_path: "/private/keys.env".into(), profile: ProviderSecretProfile::AmplifierOnecliLunarouteGlm53, }; diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index 30be1f888..51ac09f0a 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -2089,6 +2089,7 @@ mod tests { }], }), provider_secret_references: vec![freshell_runtime_protocol::ProviderSecretReference { + provider_relative_path: None, source_path: "/run/freshell-secrets/onecli/env".into(), profile: match provider { FreshProvider::Claude => { @@ -2686,6 +2687,7 @@ mod tests { }], }), provider_secret_references: vec![freshell_runtime_protocol::ProviderSecretReference { + provider_relative_path: None, source_path: "/run/freshell-secrets/onecli".into(), profile: match provider { FreshProvider::Claude => freshell_runtime_protocol::ProviderSecretProfile::ClaudeOnecliEnvironment, diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index b80b3d937..40191b3d9 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -1677,6 +1677,7 @@ mod tests { provider_permission_mode: None, provider_bootstrap_files: Vec::new(), provider_secret_references: vec![ProviderSecretReference { + provider_relative_path: None, source_path: keys.to_string_lossy().into_owned(), profile: ProviderSecretProfile::AmplifierOnecliLunarouteGlm53, }], diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index be5c29f13..f56030668 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -74,6 +74,8 @@ fn workload_mounts( context: Option<&ProviderLaunchContext>, provider: &str, ) -> Result { + freshell_runtime_protocol::validate_provider_secrets(provider, secret_references) + .map_err(|error| error.message)?; let workspace = canonical_dir(Path::new(workspace_path), "workspace")?; if workspace != Path::new(workspace_path) { return Err("workspace path must already be canonical".into()); @@ -118,12 +120,13 @@ fn workload_mounts( #[cfg(unix)] { use std::os::unix::fs::PermissionsExt; - if std::fs::metadata(&source) - .map_err(|error| error.to_string())? - .permissions() - .mode() - & 0o077 - != 0 + if !secret.profile.is_configuration_file() + && std::fs::metadata(&source) + .map_err(|error| error.to_string())? + .permissions() + .mode() + & 0o077 + != 0 { return Err("provider OneCLI grant must be private".into()); } @@ -382,6 +385,7 @@ mod provider_secret_tests { })) .unwrap(); launch.provider_secret_references = vec![ProviderSecretReference { + provider_relative_path: None, source_path: canonical_grant.to_string_lossy().into_owned(), profile: ProviderSecretProfile::ClaudeOnecliEnvironment, }]; diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 33dfb52d3..a9588cbee 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -3044,6 +3044,7 @@ mod tests { provider_permission_mode: None, provider_bootstrap_files: Vec::new(), provider_secret_references: vec![freshell_runtime_protocol::ProviderSecretReference { + provider_relative_path: None, source_path: keys.to_string_lossy().into_owned(), profile: freshell_runtime_protocol::ProviderSecretProfile::AmplifierOnecliLunarouteGlm53, From de2845487aa4fb00b4cd9801d8440d0cc2a3858a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 01:00:25 -0700 Subject: [PATCH 013/493] Plan OpenCode file references against their original declaring origins --- .../freshell-platform/src/opencode_config.rs | 246 ++++++++++++ crates/freshell-server/src/main.rs | 2 + .../src/managed_provider_references.rs | 377 ++++++++++++++++++ 3 files changed, 625 insertions(+) create mode 100644 crates/freshell-server/src/managed_provider_references.rs diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index cf3f2dff6..93240bfeb 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -460,3 +460,249 @@ mod tests { assert_eq!(config["permission"]["bash"], "allow"); } } + +/// Input provenance for one token in the pinned document-level substitution. +/// Expanded paths are transient; never serialize or log this callback input. +pub struct FileToken<'a> { + pub path: &'a str, + pub original_span: std::ops::Range, + pub variables: Vec<&'a str>, + pub direct_expression: bool, + pub ordinal: usize, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum TransformError { + SourceLimit, + ExpansionLimit, + EventLimit, + VariableLimit, +} + +struct SubstitutionSegment<'a> { + expanded: std::ops::Range, + original: std::ops::Range, + variable: Option<&'a str>, +} + +const MAX_CONFIG_VIEW_BYTES: usize = 16 * 1024 * 1024; +const MAX_CONFIG_EVENTS: usize = 8192; + +fn bounded_append(out: &mut String, piece: &str) -> Result<(), TransformError> { + if out + .len() + .checked_add(piece.len()) + .is_none_or(|n| n > MAX_CONFIG_VIEW_BYTES) + { + return Err(TransformError::ExpansionLimit); + } + out.push_str(piece); + Ok(()) +} + +fn outer_file_end(raw: &str, start: usize) -> Option { + if !raw[start..].starts_with("{file:") { + return None; + } + let mut cursor = start + 6; + while cursor < raw.len() { + if raw[cursor..].starts_with("{env:") { + let close = raw[cursor + 5..].find('}')? + cursor + 5; + cursor = close + 1; + } else if raw.as_bytes()[cursor] == b'}' { + return Some(cursor + 1); + } else { + cursor += raw[cursor..].chars().next()?.len_utf8(); + } + } + None +} + +/// OpenCode 1.18.21 performs exactly one raw env pass, then snapshots file +/// matches from the resulting document. Keep raw escapes and full line-prefix +/// context; parsed JSON strings do not reproduce its substitution behavior. +/// Output may contain private child values and is process-only. +pub fn transform_file_tokens( + raw: &str, + env: &dyn Fn(&str) -> Option, + mut file: impl FnMut(FileToken<'_>) -> Result, +) -> Result> { + use TokenTransformError::{Bound, Selection}; + if raw.len() > MAX_CONFIG_VIEW_BYTES { + return Err(Bound(TransformError::SourceLimit)); + } + let mut expanded = String::new(); + let mut segments = Vec::new(); + let mut cursor = 0; + while let Some(relative) = raw[cursor..].find("{env:") { + let start = cursor + relative; + let Some(close_relative) = raw[start + 5..].find('}') else { + break; + }; + let close = start + 5 + close_relative; + let name = &raw[start + 5..close]; + if name.is_empty() { + cursor = close + 1; + continue; + } + if name.len() > 128 || name.chars().any(char::is_control) { + return Err(Bound(TransformError::VariableLimit)); + } + if segments.len() + 2 > MAX_CONFIG_EVENTS { + return Err(Bound(TransformError::EventLimit)); + } + let literal_start = segments + .last() + .map_or(0, |segment: &SubstitutionSegment<'_>| segment.original.end); + let begin = expanded.len(); + bounded_append(&mut expanded, &raw[literal_start..start]).map_err(Bound)?; + segments.push(SubstitutionSegment { + expanded: begin..expanded.len(), + original: literal_start..start, + variable: None, + }); + let begin = expanded.len(); + if let Some(value) = env(name) { + bounded_append(&mut expanded, &value).map_err(Bound)?; + } + segments.push(SubstitutionSegment { + expanded: begin..expanded.len(), + original: start..close + 1, + variable: Some(name), + }); + cursor = close + 1; + } + let literal_start = segments + .last() + .map_or(0, |segment: &SubstitutionSegment<'_>| segment.original.end); + let begin = expanded.len(); + bounded_append(&mut expanded, &raw[literal_start..]).map_err(Bound)?; + segments.push(SubstitutionSegment { + expanded: begin..expanded.len(), + original: literal_start..raw.len(), + variable: None, + }); + let mut out = String::new(); + let mut cursor = 0; + let mut scan = 0; + let mut ordinal = 0; + while let Some(relative) = expanded[scan..].find("{file:") { + let start = scan + relative; + let Some(close_relative) = expanded[start + 6..].find('}') else { + break; + }; + let end = start + 6 + close_relative + 1; + scan = end; + if end == start + 7 { + continue; + } + let line_start = expanded[..start].rfind('\n').map_or(0, |n| n + 1); + if expanded[line_start..start].trim_start().starts_with("//") { + continue; + } + if ordinal >= MAX_CONFIG_EVENTS { + return Err(Bound(TransformError::EventLimit)); + } + let mut original_start = usize::MAX; + let mut original_end = 0; + let mut variables = Vec::new(); + for segment in &segments { + if segment.expanded.start >= end || segment.expanded.end <= start { + continue; + } + let (a, b) = if let Some(variable) = segment.variable { + if !variables.contains(&variable) { + variables.push(variable); + } + (segment.original.start, segment.original.end) + } else { + ( + segment.original.start + start.saturating_sub(segment.expanded.start), + segment.original.start + + (end.min(segment.expanded.end) - segment.expanded.start), + ) + }; + original_start = original_start.min(a); + original_end = original_end.max(b); + } + let direct_expression = outer_file_end(raw, original_start) == Some(original_end); + let replacement = file(FileToken { + path: &expanded[start + 6..end - 1], + original_span: original_start..original_end, + variables, + direct_expression, + ordinal, + }) + .map_err(Selection)?; + bounded_append(&mut out, &expanded[cursor..start]).map_err(Bound)?; + bounded_append(&mut out, &replacement).map_err(Bound)?; + cursor = end; + ordinal += 1; + } + bounded_append(&mut out, &expanded[cursor..]).map_err(Bound)?; + Ok(out) +} + +#[derive(Debug)] +pub enum TokenTransformError { + Bound(TransformError), + Selection(E), +} + +#[cfg(test)] +mod substitution_tests { + use super::*; + #[test] + fn one_pass_preserves_introduced_env_tokens_and_snapshots_file_tokens() { + let raw = r#"{"x":"{env:INSERT}"}"#; + let out = transform_file_tokens( + raw, + &|name| (name == "INSERT").then(|| "prefix{file:rules}suffix{env:UNEXPANDED}".into()), + |token| { + assert!(!token.direct_expression); + assert_eq!(token.path, "rules"); + assert_eq!(token.variables, vec!["INSERT"]); + Ok::<_, ()>("{file:replacement}".into()) + }, + ) + .unwrap(); + assert_eq!( + out, + r#"{"x":"prefix{file:replacement}suffix{env:UNEXPANDED}"}"# + ); + } + #[test] + fn full_line_comment_prefix_is_preserved_and_empty_variable_matches_native() { + let raw = " // {file:missing}\n{\"x\":\"{file:{env:MISSING}}\"}"; + let out = transform_file_tokens::<()>(raw, &|_| None, |_| { + panic!("comment/empty file cannot select a source") + }) + .unwrap(); + assert_eq!(out, " // {file:missing}\n{\"x\":\"{file:}\"}"); + } + #[test] + fn nested_file_span_preserves_unrelated_env_tokens_in_controller_template() { + let raw = r#"{"x":"{file:{env:PATH}}","y":"{env:OTHER}"}"#; + let mut span = None; + transform_file_tokens( + raw, + &|name| { + Some( + if name == "PATH" { + "/approved" + } else { + "private-child" + } + .into(), + ) + }, + |token| { + assert!(token.direct_expression); + span = Some(token.original_span); + Ok::<_, ()>("runtime".into()) + }, + ) + .unwrap(); + assert_eq!(&raw[span.unwrap()], "{file:{env:PATH}}"); + } +} diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index 694da8c56..7bd5a96ae 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -44,6 +44,8 @@ mod managed_ports; #[cfg(feature = "managed-runtime-v1")] mod managed_provider_bootstrap; #[cfg(feature = "managed-runtime-v1")] +mod managed_provider_references; +#[cfg(feature = "managed-runtime-v1")] mod managed_runtime; #[cfg(feature = "managed-runtime-v1")] mod managed_runtime_api; diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs new file mode 100644 index 000000000..fa9c930a1 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -0,0 +1,377 @@ +//! Origin-aware managed provider configuration references. +//! Plans retain approved source metadata and unexpanded templates, never payload bytes. +use std::path::{Path, PathBuf}; + +pub struct FileSourceOrigin<'a> { + pub declaring_config: &'a Path, + pub cwd: &'a Path, + pub original_home: &'a Path, +} + +#[derive(Debug)] +pub struct ReferenceError { + pub operation: &'static str, + pub field: &'static str, + pub source_ordinal: usize, + pub declaring_source: PathBuf, + pub variables: Vec, + pub cause: &'static str, +} +impl std::fmt::Display for ReferenceError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "{} {} occurrence {} in {}: {}", + self.operation, + self.field, + self.source_ordinal, + self.declaring_source.display(), + self.cause + ) + } +} +impl std::error::Error for ReferenceError {} + +pub struct FileReferenceSelection { + pub canonical_source: PathBuf, + pub runtime_destination: String, + pub source_ordinal: usize, +} + +pub struct ResolvedFileReferencePlan { + pub template: String, + pub selections: Vec, + pub referenced_variables: Vec, +} + +pub fn resolve_file_references( + raw: &str, + origin: FileSourceOrigin<'_>, + env: &dyn Fn(&str) -> Option, +) -> Result { + use freshell_platform::opencode_config::{transform_file_tokens, TokenTransformError}; + let mut selections: Vec = Vec::new(); + let mut rewritten = Vec::new(); + let mut variables = std::collections::BTreeSet::new(); + let fail = |ordinal, variables: Vec, cause| ReferenceError { + operation: "resolve", + field: "OpenCode file", + source_ordinal: ordinal, + declaring_source: origin.declaring_config.into(), + variables, + cause, + }; + let transformed = transform_file_tokens(raw, env, |token| { + let names: Vec = token + .variables + .iter() + .map(|name| name.to_string()) + .collect(); + variables.extend(names.iter().cloned()); + if !token.direct_expression { + return Err(fail( + token.ordinal, + names, + "generated document recipe required", + )); + } + let path = if let Some(relative) = token.path.strip_prefix("~/") { + origin.original_home.join(relative) + } else { + let path = Path::new(token.path); + if path.is_absolute() { + path.into() + } else { + origin + .declaring_config + .parent() + .unwrap_or(origin.cwd) + .join(path) + } + }; + let canonical = std::fs::canonicalize(&path).map_err(|_| { + fail( + token.ordinal, + names.clone(), + "referenced file is missing, inaccessible or cyclic", + ) + })?; + let metadata = std::fs::metadata(&canonical).map_err(|_| { + fail( + token.ordinal, + names.clone(), + "referenced file metadata unavailable", + ) + })?; + if !metadata.is_file() || metadata.len() > 16 * 1024 * 1024 { + return Err(fail( + token.ordinal, + names, + "referenced target must be a regular file at most 16 MiB", + )); + } + std::fs::File::open(&canonical).map_err(|_| { + fail( + token.ordinal, + names.clone(), + "referenced file is unreadable", + ) + })?; + let destination = + if let Some(selection) = selections.iter().find(|s| s.canonical_source == canonical) { + selection.runtime_destination.clone() + } else { + if selections.len() >= 128 { + return Err(fail( + token.ordinal, + names, + "configuration reference count exceeds 128", + )); + } + let bytes = selections + .iter() + .try_fold(metadata.len(), |bytes, selection| { + std::fs::metadata(&selection.canonical_source) + .ok() + .and_then(|m| bytes.checked_add(m.len())) + }) + .ok_or_else(|| { + fail( + token.ordinal, + names.clone(), + "configuration reference accounting failed", + ) + })?; + if bytes > 32 * 1024 * 1024 { + return Err(fail( + token.ordinal, + names, + "configuration references exceed 32 MiB", + )); + } + let destination = format!( + "/home/freshell/provider/.freshell/references/{}", + selections.len() + ); + selections.push(FileReferenceSelection { + canonical_source: canonical, + runtime_destination: destination.clone(), + source_ordinal: token.ordinal, + }); + destination + }; + let replacement = format!("{{file:{destination}}}"); + rewritten.push((token.original_span, replacement.clone())); + Ok(replacement) + }) + .map_err(|error| match error { + TokenTransformError::Selection(error) => error, + TokenTransformError::Bound(_) => fail( + 0, + Vec::new(), + "configuration source, expansion or event profile limit exceeded", + ), + })?; + drop(transformed); // Expanded env text is transient and never retained in the plan. + let mut template = String::new(); + let mut cursor = 0; + for (span, replacement) in rewritten { + template.push_str(&raw[cursor..span.start]); + template.push_str(&replacement); + cursor = span.end; + } + template.push_str(&raw[cursor..]); + Ok(ResolvedFileReferencePlan { + template, + selections, + referenced_variables: variables.into_iter().collect(), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use sha2::{Digest, Sha256}; + + // Pinned resolver-equivalent consumer for OpenCode 1.18.21 variable.ts. + // Receipt is a digest of values consumed by its native JSONC/file substitution. + const CONSUMER: &str = r#" +const fs=require('fs'),path=require('path'),crypto=require('crypto'); +const source=process.argv[1],home=process.argv[2],env=JSON.parse(process.argv[3]); +let text=fs.readFileSync(source,'utf8').replace(/\{env:([^}]+)\}/g,(_,n)=>env[n]||''); +let out='',cursor=0; +for(const m of Array.from(text.matchAll(/\{file:[^}]+\}/g))) { + out+=text.slice(cursor,m.index); + if(text.slice(text.lastIndexOf('\n',m.index-1)+1,m.index).trimStart().startsWith('//')) out+=m[0]; + else { + const target=m[0].slice(6,-1),prefix='/home/freshell/provider/'; + if(!target.startsWith(prefix)) process.exit(71); + const bytes=fs.readFileSync(path.join(home,target.slice(prefix.length)),'utf8'); + out+=JSON.stringify(bytes.trim()).slice(1,-1); + } + cursor=m.index+m[0].length; +} +out+=text.slice(cursor); +const parsed=JSON.parse(out),receipts=Object.values(parsed).map(v=>crypto.createHash('sha256').update(v).digest('hex')); +process.stdout.write(JSON.stringify(receipts)); +"#; + + fn digest(value: &str) -> String { + format!("{:x}", Sha256::digest(value.as_bytes())) + } + fn consume(plan: &ResolvedFileReferencePlan, runtime: &Path) -> Vec { + std::fs::create_dir_all(runtime.join(".freshell/references")).unwrap(); + for selection in &plan.selections { + let relative = selection + .runtime_destination + .strip_prefix("/home/freshell/provider/") + .unwrap(); + std::fs::copy(&selection.canonical_source, runtime.join(relative)).unwrap(); + } + let source = runtime.join("source.json"); + std::fs::write(&source, &plan.template).unwrap(); + let result = std::process::Command::new("node") + .arg("-e") + .arg(CONSUMER) + .arg(&source) + .arg(runtime) + .arg("{}") + .output() + .unwrap(); + assert!( + result.status.success(), + "pinned consumer failed with {:?}", + result.status.code() + ); + serde_json::from_slice(&result.stdout).unwrap() + } + + #[test] + fn managed_provider_references_child_consumes_original_absolute_relative_home_spaces_and_symlink( + ) { + let temp = tempfile::tempdir().unwrap(); + let home = temp.path().join("original-home"); + let declaring = temp.path().join("project/nested"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&declaring).unwrap(); + let payload = " synthetic-nonce-quote\"slash\\\ntrim "; + for path in [ + home.join("home-ref"), + declaring.join("relative-ref"), + declaring.join("with spaces"), + temp.path().join("external-ref"), + ] { + std::fs::write(path, payload).unwrap(); + } + #[cfg(unix)] + std::os::unix::fs::symlink("relative-ref", declaring.join("linked-ref")).unwrap(); + let targets = vec![ + temp.path().join("external-ref").display().to_string(), + "relative-ref".into(), + "~/home-ref".into(), + "with spaces".into(), + "linked-ref".into(), + ]; + for target in targets { + let raw = serde_json::json!({"instructions":format!("{{file:{target}}}")}).to_string(); + let original = declaring.join("opencode.json"); + std::fs::write(&original, &raw).unwrap(); + let plan = resolve_file_references( + &raw, + FileSourceOrigin { + declaring_config: &original, + cwd: &declaring, + original_home: &home, + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest(payload.trim())] + ); + assert_eq!(std::fs::read_to_string(&original).unwrap(), raw); + assert!(!plan.template.contains("synthetic-nonce")); + } + } + #[test] + fn managed_provider_references_same_relative_spelling_keeps_each_declaring_origin() { + let temp = tempfile::tempdir().unwrap(); + for (index, nonce) in ["first-origin-nonce", "second-origin-nonce"] + .iter() + .enumerate() + { + let origin = temp.path().join(format!("origin-{index}")); + std::fs::create_dir_all(&origin).unwrap(); + std::fs::write(origin.join("rules"), nonce).unwrap(); + let config = origin.join("opencode.json"); + let raw = r#"{"instructions":"{file:rules}"}"#; + let plan = resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &config, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join(format!("runtime-{index}"))), + vec![digest(nonce)] + ); + } + } + #[test] + fn managed_provider_references_nested_path_env_is_consumed_without_forwarding_value() { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("private-path-ref"); + std::fs::write(&target, "nested-path-consumption-nonce").unwrap(); + let source = temp.path().join("opencode.json"); + let raw = r#"{"instructions":"{file:{env:NODE_EXTRA_CA_CERTS}}"}"#; + let plan = resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|name| (name == "NODE_EXTRA_CA_CERTS").then(|| target.display().to_string()), + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest("nested-path-consumption-nonce")] + ); + assert_eq!(plan.referenced_variables, vec!["NODE_EXTRA_CA_CERTS"]); + assert!(!plan.template.contains("private-path-ref")); + } + #[test] + fn managed_provider_references_missing_and_cyclic_targets_fail_without_echoing_candidate_values( + ) { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("opencode.json"); + #[cfg(unix)] + { + std::os::unix::fs::symlink("cycle-b", temp.path().join("cycle-a")).unwrap(); + std::os::unix::fs::symlink("cycle-a", temp.path().join("cycle-b")).unwrap(); + } + for target in ["private-missing-candidate", "cycle-a"] { + let result = resolve_file_references( + r#"{"x":"{file:{env:PATH_INPUT}}"}"#, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| Some(temp.path().join(target).display().to_string()), + ); + let err = result + .err() + .expect("missing or cyclic reference cannot launch"); + assert_eq!(err.source_ordinal, 0); + assert_eq!(err.variables, vec!["PATH_INPUT"]); + assert!(!format!("{err:?} {err}").contains(target)); + } + } +} From 6382ef718ba67a47ae5aff0012dcc14810cae47d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 01:10:02 -0700 Subject: [PATCH 014/493] Keep env introduced configuration references in bounded child only recipes --- .../freshell-platform/src/opencode_config.rs | 17 + .../src/managed_provider_references.rs | 292 +++++++++++++++--- 2 files changed, 267 insertions(+), 42 deletions(-) diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index 93240bfeb..8b5037e36 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -706,3 +706,20 @@ mod substitution_tests { assert_eq!(&raw[span.unwrap()], "{file:{env:PATH}}"); } } + +/// Resolve native path syntax without touching the original controller tree. +/// Hosts compare this candidate only to already approved source aliases. +pub fn file_reference_path( + path: &str, + declaring_config: &Path, + cwd: &Path, + home: &Path, +) -> PathBuf { + if let Some(relative) = path.strip_prefix("~/") { + home.join(relative) + } else if Path::new(path).is_absolute() { + path.into() + } else { + declaring_config.parent().unwrap_or(cwd).join(path) + } +} diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index fa9c930a1..af13de443 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -38,10 +38,25 @@ pub struct FileReferenceSelection { pub source_ordinal: usize, } +pub struct ApprovedFileOccurrence { + pub ordinal: usize, + pub approved_alias: PathBuf, + pub runtime_destination: String, +} +pub struct GeneratedFileRecipe { + pub generated_name: String, + pub unexpanded_source: String, + pub declaring_config: PathBuf, + pub cwd: PathBuf, + pub original_home: PathBuf, + pub occurrences: Vec, +} + pub struct ResolvedFileReferencePlan { pub template: String, pub selections: Vec, pub referenced_variables: Vec, + pub recipes: Vec, } pub fn resolve_file_references( @@ -49,9 +64,15 @@ pub fn resolve_file_references( origin: FileSourceOrigin<'_>, env: &dyn Fn(&str) -> Option, ) -> Result { - use freshell_platform::opencode_config::{transform_file_tokens, TokenTransformError}; + use freshell_platform::opencode_config::{ + file_reference_path, transform_file_tokens, TokenTransformError, + }; let mut selections: Vec = Vec::new(); let mut rewritten = Vec::new(); + let mut occurrences = Vec::new(); + let mut generated = false; + let mut planned_bytes = 0u64; + let mut aliases = std::collections::BTreeSet::new(); let mut variables = std::collections::BTreeSet::new(); let fail = |ordinal, variables: Vec, cause| ReferenceError { operation: "resolve", @@ -68,27 +89,20 @@ pub fn resolve_file_references( .map(|name| name.to_string()) .collect(); variables.extend(names.iter().cloned()); - if !token.direct_expression { + if variables.len() > 128 { return Err(fail( token.ordinal, names, - "generated document recipe required", + "referenced variable count exceeds 128", )); } - let path = if let Some(relative) = token.path.strip_prefix("~/") { - origin.original_home.join(relative) - } else { - let path = Path::new(token.path); - if path.is_absolute() { - path.into() - } else { - origin - .declaring_config - .parent() - .unwrap_or(origin.cwd) - .join(path) - } - }; + generated |= !token.direct_expression; + let path = file_reference_path( + token.path, + origin.declaring_config, + origin.cwd, + origin.original_home, + ); let canonical = std::fs::canonicalize(&path).map_err(|_| { fail( token.ordinal, @@ -128,21 +142,14 @@ pub fn resolve_file_references( "configuration reference count exceeds 128", )); } - let bytes = selections - .iter() - .try_fold(metadata.len(), |bytes, selection| { - std::fs::metadata(&selection.canonical_source) - .ok() - .and_then(|m| bytes.checked_add(m.len())) - }) - .ok_or_else(|| { - fail( - token.ordinal, - names.clone(), - "configuration reference accounting failed", - ) - })?; - if bytes > 32 * 1024 * 1024 { + planned_bytes = planned_bytes.checked_add(metadata.len()).ok_or_else(|| { + fail( + token.ordinal, + names.clone(), + "configuration reference size overflow", + ) + })?; + if planned_bytes > 32 * 1024 * 1024 { return Err(fail( token.ordinal, names, @@ -160,6 +167,18 @@ pub fn resolve_file_references( }); destination }; + if aliases.insert(path.clone()) && aliases.len() > 256 { + return Err(fail( + token.ordinal, + names, + "configuration alias count exceeds 256", + )); + } + occurrences.push(ApprovedFileOccurrence { + ordinal: token.ordinal, + approved_alias: path, + runtime_destination: destination.clone(), + }); let replacement = format!("{{file:{destination}}}"); rewritten.push((token.original_span, replacement.clone())); Ok(replacement) @@ -172,20 +191,87 @@ pub fn resolve_file_references( "configuration source, expansion or event profile limit exceeded", ), })?; - drop(transformed); // Expanded env text is transient and never retained in the plan. - let mut template = String::new(); - let mut cursor = 0; - for (span, replacement) in rewritten { - template.push_str(&raw[cursor..span.start]); - template.push_str(&replacement); - cursor = span.end; - } - template.push_str(&raw[cursor..]); + drop(transformed); + let (template, recipes) = if generated { + let generated_name = "FRESHELL_CONFIG_REFERENCE_0".to_string(); + ( + format!("{{env:{generated_name}}}"), + vec![GeneratedFileRecipe { + generated_name, + unexpanded_source: raw.into(), + declaring_config: origin.declaring_config.into(), + cwd: origin.cwd.into(), + original_home: origin.original_home.into(), + occurrences, + }], + ) + } else { + let mut template = String::new(); + let mut cursor = 0; + for (span, replacement) in rewritten { + template.push_str(&raw[cursor..span.start]); + template.push_str(&replacement); + cursor = span.end; + } + template.push_str(&raw[cursor..]); + (template, Vec::new()) + }; Ok(ResolvedFileReferencePlan { template, selections, referenced_variables: variables.into_iter().collect(), + recipes, + }) +} + +pub fn evaluate_generated_recipe( + recipe: &GeneratedFileRecipe, + env: &dyn Fn(&str) -> Option, +) -> Result { + use freshell_platform::opencode_config::{ + file_reference_path, transform_file_tokens, TokenTransformError, + }; + let fail = || ReferenceError { + operation: "prepare", + field: "OpenCode generated configuration", + source_ordinal: 0, + declaring_source: recipe.declaring_config.clone(), + variables: Vec::new(), + cause: "configuration reference selection changed; reprepare before dispatch", + }; + let mut observed = 0; + let text = transform_file_tokens(&recipe.unexpanded_source, env, |token| { + let path = file_reference_path( + token.path, + &recipe.declaring_config, + &recipe.cwd, + &recipe.original_home, + ); + let occurrence = recipe + .occurrences + .get(token.ordinal) + .filter(|approved| approved.ordinal == token.ordinal && approved.approved_alias == path) + .ok_or_else(fail)?; + observed += 1; + Ok(format!("{{file:{}}}", occurrence.runtime_destination)) }) + .map_err(|error| match error { + TokenTransformError::Selection(error) => error, + TokenTransformError::Bound(_) => ReferenceError { + cause: "configuration recipe profile limit exceeded", + ..fail() + }, + })?; + if observed != recipe.occurrences.len() { + return Err(fail()); + } + if text.len() > 64 * 1024 { + return Err(ReferenceError { + cause: "generated configuration exceeds 64 KiB child value limit", + ..fail() + }); + } + Ok(text) } #[cfg(test)] @@ -220,6 +306,13 @@ process.stdout.write(JSON.stringify(receipts)); format!("{:x}", Sha256::digest(value.as_bytes())) } fn consume(plan: &ResolvedFileReferencePlan, runtime: &Path) -> Vec { + consume_with_env(plan, runtime, &|_| None) + } + fn consume_with_env( + plan: &ResolvedFileReferencePlan, + runtime: &Path, + env: &dyn Fn(&str) -> Option, + ) -> Vec { std::fs::create_dir_all(runtime.join(".freshell/references")).unwrap(); for selection in &plan.selections { let relative = selection @@ -230,12 +323,19 @@ process.stdout.write(JSON.stringify(receipts)); } let source = runtime.join("source.json"); std::fs::write(&source, &plan.template).unwrap(); + let mut child = std::collections::BTreeMap::new(); + for recipe in &plan.recipes { + child.insert( + recipe.generated_name.clone(), + evaluate_generated_recipe(recipe, env).unwrap(), + ); + } let result = std::process::Command::new("node") .arg("-e") .arg(CONSUMER) .arg(&source) .arg(runtime) - .arg("{}") + .arg(serde_json::to_string(&child).unwrap()) .output() .unwrap(); assert!( @@ -374,4 +474,112 @@ process.stdout.write(JSON.stringify(receipts)); assert!(!format!("{err:?} {err}").contains(target)); } } + #[test] + fn managed_provider_references_env_introduced_file_tokens_and_private_prefix_use_child_recipe() + { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("reference"); + std::fs::write(&target, " payload-nonce ").unwrap(); + let source = temp.path().join("opencode.json"); + let raw = r#"{"instructions":"{env:OPENAI_API_KEY}"}"#; + let lookup = |name: &str| { + (name == "OPENAI_API_KEY").then(|| { + format!( + "sentinel-private-prefix{{file:{}}}sentinel-private-suffix", + target.display() + ) + }) + }; + let plan = resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &lookup, + ) + .unwrap(); + assert!(!plan.template.contains("sentinel-private")); + assert_eq!(plan.recipes.len(), 1); + assert!(!plan.recipes[0] + .unexpanded_source + .contains("sentinel-private")); + assert_eq!( + consume_with_env(&plan, &temp.path().join("runtime"), &lookup), + vec![digest( + "sentinel-private-prefixpayload-noncesentinel-private-suffix" + )] + ); + } + + #[test] + fn managed_provider_references_changed_grant_selection_blocks_before_child_and_keeps_prior_projection( + ) { + let temp = tempfile::tempdir().unwrap(); + let a = temp.path().join("a"); + let b = temp.path().join("b"); + std::fs::write(&a, "prior-usable-nonce").unwrap(); + std::fs::write(&b, "unapproved-new-nonce").unwrap(); + let source = temp.path().join("opencode.json"); + let initial = |_: &str| Some(format!("prefix{{file:{}}}", a.display())); + let plan = resolve_file_references( + r#"{"x":"{env:OPENAI_API_KEY}"}"#, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &initial, + ) + .unwrap(); + let previous = temp.path().join("runtime"); + assert_eq!( + consume_with_env(&plan, &previous, &initial), + vec![digest("prefixprior-usable-nonce")] + ); + let changed = |_: &str| Some(format!("sentinel-private-changed{{file:{}}}", b.display())); + let err = evaluate_generated_recipe(&plan.recipes[0], &changed) + .err() + .unwrap(); + assert!(err.cause.contains("selection changed")); + assert!(!format!("{err:?} {err}").contains("sentinel-private")); + assert_eq!( + std::fs::read_to_string(previous.join(".freshell/references/0")).unwrap(), + "prior-usable-nonce" + ); + let same_selection = |_: &str| Some(format!("different-prefix{{file:{}}}", a.display())); + assert!(evaluate_generated_recipe(&plan.recipes[0], &same_selection) + .unwrap() + .contains("different-prefix")); + } + #[test] + fn managed_provider_references_whole_document_native_wrapper_handles_outside_lexeme_expansion() + { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("reference"); + std::fs::write(&target, "document-consumption-nonce").unwrap(); + let source = temp.path().join("opencode.json"); + let lookup = |_: &str| { + Some( + serde_json::json!({"instructions":format!("{{file:{}}}",target.display())}) + .to_string(), + ) + }; + let plan = resolve_file_references( + "{env:OPENAI_API_KEY}", + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &lookup, + ) + .unwrap(); + assert_eq!( + consume_with_env(&plan, &temp.path().join("runtime"), &lookup), + vec![digest("document-consumption-nonce")] + ); + assert_eq!(plan.recipes[0].unexpanded_source, "{env:OPENAI_API_KEY}"); + } } From c8673245fe025eb7368b2b9767215cf1965ac812 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 01:40:33 -0700 Subject: [PATCH 015/493] Retain prior managed inputs until accepted preparation commit --- .../src/managed_mcp_capability.rs | 423 +++++++++++++++++- crates/freshell-supervisor/src/recovery.rs | 13 +- crates/freshell-supervisor/src/service.rs | 37 +- 3 files changed, 463 insertions(+), 10 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 8ebcb909b..6645edf5b 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -37,6 +37,21 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub workspace: Option<&'a Path>, } +#[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +enum InputPreparationPhase { + Pending, + Bound, + Committed, +} + +#[derive(Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct InputPreparation { + phase: InputPreparationPhase, + prior_grant_ids: Vec, +} + #[derive(Clone)] struct Grant { soul_id: SoulId, @@ -45,6 +60,7 @@ struct Grant { expires_at: u64, endpoint: String, path: PathBuf, + preparation: Option, } #[derive(Default)] @@ -224,8 +240,8 @@ fn apply_callback( endpoint: &str, home: &Path, ) -> Result, String> { - let callback: CapabilityCallback = - serde_json::from_slice(bytes).map_err(|error| error.to_string())?; + let callback: CapabilityCallback = serde_json::from_slice(bytes) + .map_err(|_| "managed provider callback metadata is invalid")?; if !freshell_api::check_auth(Some(&callback.control_secret), expected_control_secret) { return Err("managed runtime callback unauthorized".into()); } @@ -257,6 +273,23 @@ fn apply_callback( )?; Ok(b"ok".to_vec()) } + "commit" => { + let grant_id = callback + .grant_id + .ok_or("missing provider input preparation id")?; + let incarnation = callback + .incarnation_id + .ok_or("missing provider input preparation incarnation")?; + store.commit_preparation(&grant_id, &callback.soul_id, &incarnation, directory)?; + Ok(b"ok".to_vec()) + } + "abort" => { + let grant_id = callback + .grant_id + .ok_or("missing provider input preparation id")?; + store.abort_preparation(&grant_id, &callback.soul_id, directory)?; + Ok(b"ok".to_vec()) + } "revoke" => { let grant_id = callback.grant_id.ok_or("missing managed MCP grant id")?; store.revoke(&grant_id, directory)?; @@ -613,6 +646,51 @@ fn cleanup_unrecoverable_inputs( Ok(()) } +fn protected_preparation_ids(directory: &Path) -> Result, String> { + use std::io::Read; + let mut protected = HashSet::new(); + for entry in + std::fs::read_dir(directory).map_err(|_| "provider preparation inventory unavailable")? + { + let path = entry + .map_err(|_| "provider preparation entry unavailable")? + .path(); + let Some(id) = path.file_stem().and_then(|id| id.to_str()) else { + continue; + }; + if path.extension().and_then(|extension| extension.to_str()) != Some("json") + || grant_path(directory, id).is_err() + { + continue; + } + let mut bytes = Vec::new(); + let Ok(file) = std::fs::File::open(&path) else { + continue; + }; + file.take(128 * 1024 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "provider preparation metadata read failed")?; + if bytes.len() > 128 * 1024 { + return Err("provider preparation metadata profile limit exceeded".into()); + } + let Ok(stored) = serde_json::from_slice::(&bytes) else { + continue; + }; + if let Some(preparation) = stored.preparation { + if preparation.phase != InputPreparationPhase::Committed + || !preparation.prior_grant_ids.is_empty() + { + protected.insert(id.to_string()); + for prior in preparation.prior_grant_ids { + grant_path(directory, &prior)?; + protected.insert(prior); + } + } + } + } + Ok(protected) +} + fn cleanup_orphaned_provider_roots(directory: &Path) -> Result<(), String> { let staged_parent = directory .parent() @@ -621,6 +699,7 @@ fn cleanup_orphaned_provider_roots(directory: &Path) -> Result<(), String> { if !staged_parent.is_dir() { return Ok(()); } + let protected = protected_preparation_ids(directory)?; for entry in std::fs::read_dir(&staged_parent).map_err(|error| error.to_string())? { let entry = entry.map_err(|error| error.to_string())?; let Some(grant_id) = entry.file_name().to_str().map(str::to_owned) else { @@ -629,7 +708,7 @@ fn cleanup_orphaned_provider_roots(directory: &Path) -> Result<(), String> { let Ok(grant_file) = grant_path(directory, &grant_id) else { continue; }; - if !grant_file.exists() && entry.path().is_dir() { + if !grant_file.exists() && !protected.contains(&grant_id) && entry.path().is_dir() { std::fs::remove_dir_all(entry.path()).map_err(|error| error.to_string())?; } } @@ -936,6 +1015,8 @@ struct CapabilityFile<'a> { soul_id: &'a SoulId, incarnation_id: Option<&'a IncarnationId>, expires_at: u64, + #[serde(skip_serializing_if = "Option::is_none")] + preparation: Option<&'a InputPreparation>, } #[derive(Deserialize)] @@ -946,6 +1027,8 @@ struct StoredCapabilityFile { soul_id: SoulId, incarnation_id: Option, expires_at: u64, + #[serde(default)] + preparation: Option, } fn restored_grant_at( @@ -957,6 +1040,7 @@ fn restored_grant_at( ) -> Grant { let current = stored.endpoint == current_endpoint && !stored.scope.is_empty(); Grant { + preparation: stored.preparation, soul_id: stored.soul_id, incarnation_id: Some(incarnation_id), token: if current { stored.scope } else { String::new() }, @@ -988,6 +1072,7 @@ fn write_capability(path: &Path, grant: &Grant) -> Result<(), String> { soul_id: &grant.soul_id, incarnation_id: grant.incarnation_id.as_ref(), expires_at: grant.expires_at, + preparation: grant.preparation.as_ref(), }) .map_err(|error| error.to_string())?; let mut options = OpenOptions::new(); @@ -1046,6 +1131,10 @@ impl CapabilityStore { }) .transpose()?; let grant = Grant { + preparation: Some(InputPreparation { + phase: InputPreparationPhase::Pending, + prior_grant_ids: Vec::new(), + }), soul_id, incarnation_id: None, token: new_secret(), @@ -1077,6 +1166,22 @@ impl CapabilityStore { tui_source: Option<&ProviderConfigReference>, workspace: Option<&Path>, ) -> Result { + if let Some((grant_id, grant)) = self.grants.lock().unwrap().iter().find(|(_, grant)| { + grant.soul_id == soul_id + && grant.preparation.as_ref().is_some_and(|p| { + p.phase != InputPreparationPhase::Committed && !p.prior_grant_ids.is_empty() + }) + }) { + if grant.endpoint != endpoint { + return Err("pending provider input preparation requires reconciliation".into()); + } + return Ok(McpCapabilityReference { + grant_id: grant_id.clone(), + endpoint, + provider_relative_path: CAPABILITY_PATH.into(), + host_gateway_address: routed_host_ipv4(), + }); + } let retained = recoverable_input_path(directory, &soul_id)?; let prior_stage = self .grants @@ -1109,12 +1214,96 @@ impl CapabilityStore { }) .map(|(grant_id, _)| grant_id.clone()) .collect::>(); - for grant_id in prior { - self.revoke(&grant_id, directory)?; + let result = (|| { + let mut grants = self.grants.lock().unwrap(); + let pending = grants + .get_mut(&replacement.grant_id) + .ok_or("provider input preparation disappeared")?; + pending.preparation = Some(InputPreparation { + phase: InputPreparationPhase::Pending, + prior_grant_ids: prior, + }); + write_capability(&pending.path, pending) + })(); + if let Err(error) = result { + let _ = self.revoke(&replacement.grant_id, directory); + return Err(error); } Ok(replacement) } + fn commit_preparation( + &self, + grant_id: &str, + soul_id: &SoulId, + incarnation_id: &IncarnationId, + directory: &Path, + ) -> Result<(), String> { + let (prior, changed) = { + let mut grants = self.grants.lock().unwrap(); + let grant = grants + .get_mut(grant_id) + .ok_or("provider input preparation is unknown; reconcile delivery")?; + if &grant.soul_id != soul_id || grant.incarnation_id.as_ref() != Some(incarnation_id) { + return Err( + "provider input preparation commit has no matching accepted incarnation".into(), + ); + } + let Some(preparation) = grant.preparation.as_mut() else { + return Ok(()); + }; + if preparation.phase == InputPreparationPhase::Pending { + return Err("provider input preparation is not activated".into()); + } + let changed = preparation.phase != InputPreparationPhase::Committed; + preparation.phase = InputPreparationPhase::Committed; + let prior = preparation.prior_grant_ids.clone(); + write_capability(&grant.path, grant)?; + (prior, changed) + }; + for prior_id in prior { + self.revoke(&prior_id, directory)?; + } + let mut grants = self.grants.lock().unwrap(); + let grant = grants + .get_mut(grant_id) + .ok_or("committed provider preparation disappeared")?; + if let Some(preparation) = grant.preparation.as_mut() { + preparation.prior_grant_ids.clear(); + } + write_capability(&grant.path, grant)?; + if changed { + tracing::info!(event="provider.inputs.preparation.committed",grant_id, + soul_id=%soul_id,incarnation_id=%incarnation_id); + } + Ok(()) + } + + fn abort_preparation( + &self, + grant_id: &str, + soul_id: &SoulId, + directory: &Path, + ) -> Result<(), String> { + if let Some(grant) = self.grants.lock().unwrap().get(grant_id) { + if &grant.soul_id != soul_id { + return Err("provider input preparation belongs to another soul".into()); + } + if grant + .preparation + .as_ref() + .is_some_and(|p| p.phase == InputPreparationPhase::Committed) + { + return Err( + "provider input preparation is committed; reconcile before cleanup".into(), + ); + } + } + self.revoke(grant_id, directory)?; + tracing::info!(event="provider.inputs.preparation.aborted",grant_id,soul_id=%soul_id); + Ok(()) + } + fn activate( &self, grant_id: &str, @@ -1133,7 +1322,16 @@ impl CapabilityStore { return Ok(()); } } + let mut preparation = grants + .get(grant_id) + .and_then(|grant| grant.preparation.clone()); + if let Some(preparation) = preparation.as_mut() { + if preparation.phase != InputPreparationPhase::Committed { + preparation.phase = InputPreparationPhase::Bound; + } + } let grant = Grant { + preparation, soul_id, incarnation_id: Some(incarnation_id), token: new_secret(), @@ -1207,6 +1405,7 @@ pub(crate) async fn rehydrate_running_capabilities( .await .map_err(|error| error.to_string())?; let directory = capability_dir()?; + let protected = protected_preparation_ids(&directory)?; for entry in std::fs::read_dir(&directory).map_err(|error| error.to_string())? { let entry = entry.map_err(|error| error.to_string())?; let path = entry.path(); @@ -1241,6 +1440,27 @@ pub(crate) async fn rehydrate_running_capabilities( let Ok(stored) = serde_json::from_slice::(&file) else { continue; }; + if protected.contains(&grant_id) { + let current = stored.endpoint == endpoint()? && !stored.scope.is_empty(); + store().grants.lock().unwrap().insert( + grant_id.clone(), + Grant { + soul_id: stored.soul_id, + incarnation_id: stored.incarnation_id, + token: if current { stored.scope } else { String::new() }, + expires_at: if current { stored.expires_at } else { 0 }, + endpoint: stored.endpoint, + path, + preparation: stored.preparation, + }, + ); + tracing::info!( + event = "managed_mcp.preparation.retained", + grant_id, + "Retained uncertain provider input generation for supervisor reconciliation" + ); + continue; + } let Some(incarnation_id) = stored.incarnation_id.clone() else { if soul_wants_recovery(&views, &stored.soul_id) { retain_recoverable_inputs(&directory, &stored.soul_id, &grant_id)?; @@ -1767,6 +1987,7 @@ mod tests { let grant_file = grant_path(&capabilities, "grant-old").unwrap(); std::fs::write(&grant_file, "old grant file").unwrap(); let stored = StoredCapabilityFile { + preparation: None, endpoint: "http://host.docker.internal:3001".into(), scope: "old-scope".into(), soul_id: SoulId::new(), @@ -1849,6 +2070,7 @@ mod tests { )); let after_idle = restored_grant_at( StoredCapabilityFile { + preparation: None, expires_at: initial.expires_at - 1, ..renewed }, @@ -1888,6 +2110,185 @@ mod tests { assert!(!reopened.authenticates(&initial.scope)); } + #[test] + fn managed_mcp_lost_issue_response_reuses_one_pending_generation_and_keeps_prior() { + let control = tempfile::tempdir().unwrap(); + let directory = control.path().join("mcp-capabilities"); + std::fs::create_dir(&directory).unwrap(); + let home = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let first = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", home.path(), &[])), + Default::default(), + None, + ) + .unwrap(); + let callback = serde_json::to_vec( + &serde_json::json!({"action":"issue","controlSecret":"fixture-control", + "soulId":soul,"provider":"codex","workspace":home.path()}), + ) + .unwrap(); + let lost = apply_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + home.path(), + ) + .unwrap(); + let repeated = apply_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + home.path(), + ) + .unwrap(); + let pending: McpCapabilityReference = serde_json::from_slice(&lost).unwrap(); + let retry: McpCapabilityReference = serde_json::from_slice(&repeated).unwrap(); + assert_eq!(pending.grant_id, retry.grant_id); + assert!(staged_provider_root_path(&directory, &first.grant_id) + .unwrap() + .is_dir()); + assert_eq!( + std::fs::read_dir(control.path().join("provider-roots")) + .unwrap() + .count(), + 2 + ); + assert!(store + .abort_preparation(&pending.grant_id, &SoulId::new(), &directory) + .is_err()); + assert!(staged_provider_root_path(&directory, &pending.grant_id) + .unwrap() + .is_dir()); + } + + #[test] + fn managed_mcp_startup_orphan_cleanup_protects_uncertain_prior_and_pending_inputs() { + let control = tempfile::tempdir().unwrap(); + let directory = control.path().join("mcp-capabilities"); + std::fs::create_dir(&directory).unwrap(); + let home = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let pending = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", home.path(), &[])), + Default::default(), + None, + ) + .unwrap(); + let prior = staged_provider_root_path(&directory, "grant-prior-uncertain").unwrap(); + let orphan = staged_provider_root_path(&directory, "grant-confirmed-orphan").unwrap(); + std::fs::create_dir(&prior).unwrap(); + std::fs::create_dir(&orphan).unwrap(); + std::fs::write(prior.join("usable-resource"), "previous-projection-nonce").unwrap(); + { + let mut grants = store.grants.lock().unwrap(); + let grant = grants.get_mut(&pending.grant_id).unwrap(); + grant.preparation.as_mut().unwrap().prior_grant_ids = + vec!["grant-prior-uncertain".into()]; + write_capability(&grant.path, grant).unwrap(); + } + cleanup_orphaned_provider_roots(&directory).unwrap(); + assert_eq!( + std::fs::read_to_string(prior.join("usable-resource")).unwrap(), + "previous-projection-nonce" + ); + assert!(staged_provider_root_path(&directory, &pending.grant_id) + .unwrap() + .is_dir()); + assert!(!orphan.exists()); + } + + #[test] + fn managed_mcp_generation_commit_and_abort_preserve_prior_until_accepted() { + let control = tempfile::tempdir().unwrap(); + let dir = control.path().join("mcp-capabilities"); + std::fs::create_dir(&dir).unwrap(); + let home = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let first = store + .issue( + soul.clone(), + &dir, + "http://host.docker.internal:4000".into(), + Some(("codex", home.path(), &[])), + Default::default(), + None, + ) + .unwrap(); + let second = store + .issue_replacement( + soul.clone(), + &dir, + "http://host.docker.internal:4000".into(), + Some(("codex", home.path(), &[])), + None, + Some(home.path()), + ) + .unwrap(); + let prior = staged_provider_root_path(&dir, &first.grant_id).unwrap(); + let pending = staged_provider_root_path(&dir, &second.grant_id).unwrap(); + assert!(prior.exists(), "prepare cannot retire prior inputs"); + let incarnation = IncarnationId::new(); + assert!(store + .commit_preparation(&second.grant_id, &soul, &incarnation, &dir) + .is_err()); + store + .abort_preparation(&second.grant_id, &soul, &dir) + .unwrap(); + store + .abort_preparation(&second.grant_id, &soul, &dir) + .unwrap(); + assert!(prior.exists()); + assert!(!pending.exists()); + let replacement = store + .issue_replacement( + soul.clone(), + &dir, + "http://host.docker.internal:4000".into(), + Some(("codex", home.path(), &[])), + None, + Some(home.path()), + ) + .unwrap(); + store + .activate( + &replacement.grant_id, + soul.clone(), + incarnation.clone(), + &dir, + &replacement.endpoint, + ) + .unwrap(); + assert!( + prior.exists(), + "MCP activation is earlier than Host GrantAccepted" + ); + store + .commit_preparation(&replacement.grant_id, &soul, &incarnation, &dir) + .unwrap(); + store + .commit_preparation(&replacement.grant_id, &soul, &incarnation, &dir) + .unwrap(); + assert!(!prior.exists()); + assert!(store + .abort_preparation(&replacement.grant_id, &soul, &dir) + .is_err()); + } + #[test] fn opencode_private_stage_survives_reissue_without_durable_secret_bytes() { let control = tempfile::tempdir().unwrap(); @@ -1943,7 +2344,10 @@ mod tests { ) .unwrap(); let second_stage = staged_provider_root_path(&capabilities, &second.grant_id).unwrap(); - assert!(!first_stage.exists()); + assert!( + first_stage.exists(), + "replacement preparation must retain prior usable inputs" + ); assert_eq!( std::fs::read_to_string(second_stage.join("ephemeral/inline-config.json")).unwrap(), raw @@ -2426,8 +2830,11 @@ mod tests { .unwrap(); assert_ne!(replacement.grant_id, reference.grant_id); assert_eq!(replacement.endpoint, "http://host.docker.internal:4001"); - assert!(!path.exists()); - assert!(!store.authenticates(active["scope"].as_str().unwrap())); + assert!( + path.exists(), + "lost issue response must not retire prior grant" + ); + assert!(store.authenticates(active["scope"].as_str().unwrap())); } #[tokio::test] diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index 660e1d21f..f65b86977 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -852,12 +852,23 @@ impl Supervisor { let provider = start.context.provider.clone(); self.registry .mark_recovery_success( - soul_id, + soul_id.clone(), prepared.prepared.incarnation_id.clone(), attempt_id.clone(), ) .await .map_err(map_registry)?; + self.commit_provider_preparation( + &LaunchWorkload { + soul_id: soul_id.clone(), + fixture: prepared.fixture, + terminal: prepared.terminal.clone(), + fresh_agent: prepared.fresh_agent.clone(), + resume_spec: prepared.resume_spec.clone(), + }, + &prepared.prepared.incarnation_id, + ) + .await?; let elapsed_ms = recovery_started .elapsed() .as_millis() diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 94fd06d46..946548d1a 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -810,6 +810,13 @@ impl Supervisor { self.activation_failure(&prepared, state, "record_native_identity", error) })?; } + if workload.resume_spec.is_none() { + self.commit_provider_preparation(&workload, handle.incarnation_id()) + .await + .map_err(|error| { + self.activation_failure(&prepared, state, "commit_provider_inputs", error) + })?; + } append_event( &self.config.lifecycle_log, "supervisor.launch_running", @@ -1686,7 +1693,7 @@ impl Supervisor { "invalid managed MCP endpoint", )); } - if !matches!(action, "activate" | "revoke") { + if !matches!(action, "activate" | "revoke" | "commit" | "abort") { return Err(RuntimeError::new( RuntimeErrorCode::InvalidRequest, "invalid managed MCP action", @@ -1712,6 +1719,34 @@ impl Supervisor { Ok(()) } + pub(crate) async fn commit_provider_preparation( + &self, + workload: &LaunchWorkload, + incarnation: &IncarnationId, + ) -> Result<(), RuntimeError> { + let context = workload + .terminal + .as_ref() + .and_then(|terminal| terminal.provider_launch_context.as_ref()) + .or_else(|| { + workload + .fresh_agent + .as_ref() + .and_then(|agent| agent.provider_launch_context.as_ref()) + }); + if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { + self.notify_mcp_capability( + "commit", + &capability.endpoint, + &workload.soul_id, + incarnation, + &capability.grant_id, + ) + .await?; + } + Ok(()) + } + pub(crate) async fn issue_replacement_mcp_capability( &self, soul_id: &SoulId, From d8eacb9f29fa775be4745cf9a2967627a2986d33 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 02:36:09 -0700 Subject: [PATCH 016/493] Decode bounded selected OpenCode auth variables in native key order --- Cargo.lock | 1 + crates/freshell-platform/Cargo.toml | 1 + .../src/provider_environment_grants.rs | 151 +++++++++++++++++- 3 files changed, 151 insertions(+), 2 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 3bb2bce5a..368c3e6c5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1383,6 +1383,7 @@ name = "freshell-platform" version = "0.1.0" dependencies = [ "percent-encoding", + "serde", "serde_json", "tempfile", "url", diff --git a/crates/freshell-platform/Cargo.toml b/crates/freshell-platform/Cargo.toml index 47d19e701..8a4f32ce4 100644 --- a/crates/freshell-platform/Cargo.toml +++ b/crates/freshell-platform/Cargo.toml @@ -12,6 +12,7 @@ publish.workspace = true [dependencies] percent-encoding = "2" serde_json = { workspace = true } +serde = { workspace = true } url = "2" [dev-dependencies] diff --git a/crates/freshell-platform/src/provider_environment_grants.rs b/crates/freshell-platform/src/provider_environment_grants.rs index 7bb6b846b..f46c2de49 100644 --- a/crates/freshell-platform/src/provider_environment_grants.rs +++ b/crates/freshell-platform/src/provider_environment_grants.rs @@ -94,6 +94,19 @@ pub fn read_environment_grant( path: &Path, profile: EnvironmentProfile, ) -> Result, GrantError> { + let raw = read_private_grant(path, profile)?; + resolve_environment_profile(profile, &raw) +} + +pub fn read_opencode_auth_environment( + path: &Path, + needed: &[String], +) -> Result, GrantError> { + let raw = read_private_grant(path, EnvironmentProfile::Opencode)?; + resolve_opencode_auth_environment(&raw, needed) +} + +fn read_private_grant(path: &Path, profile: EnvironmentProfile) -> Result { let fail = |reason: &str| error(profile, GrantOperation::Read, reason.into()); let canonical = std::fs::canonicalize(path).map_err(|_| fail("grant is unavailable"))?; let metadata = std::fs::symlink_metadata(path).map_err(|_| fail("grant is unavailable"))?; @@ -108,8 +121,7 @@ pub fn read_environment_grant( } } let file = std::fs::File::open(path).map_err(|_| fail("grant is unavailable"))?; - let raw = read_bounded(file, profile)?; - resolve_environment_profile(profile, &raw) + read_bounded(file, profile) } const MAX_KEYS_BYTES: u64 = 128 * 1024; @@ -496,6 +508,102 @@ fn required<'a>(values: &'a BTreeMap, name: &str) -> Result<&'a .ok_or_else(|| format!("Amplifier OneCLI profile requires {name} in keys.env")) } +/// Read only the explicitly needed well-known auth environment. Returned +/// values are transient; callers must not serialize, cache or log the map. +pub fn resolve_opencode_auth_environment( + raw: &str, + needed: &[String], +) -> Result, GrantError> { + use serde::de::{Deserializer, MapAccess, Visitor}; + let fail = |reason: &str| { + error( + EnvironmentProfile::Opencode, + GrantOperation::Decode, + reason.into(), + ) + }; + if raw.len() > MAX_KEYS_BYTES as usize { + return Err(fail("auth grant exceeds 128 KiB")); + } + if needed.len() > 128 + || needed.iter().any(|name| { + name.is_empty() + || name.len() > 128 + || name.contains('=') + || name.chars().any(char::is_control) + }) + { + return Err(fail( + "auth lookup variable selection is outside the bounded profile", + )); + } + struct OrderedAuth; + impl<'de> Visitor<'de> for OrderedAuth { + type Value = Vec<(String, serde_json::Value)>; + fn expecting(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("a provider auth object") + } + fn visit_map>(self, mut map: M) -> Result { + let mut entries: Self::Value = Vec::new(); + while let Some((key, value)) = map.next_entry::()? { + // JSON.parse preserves a duplicate property's first insertion + // position while replacing its value with the final entry. + if let Some(index) = entries.iter().position(|(existing, _)| existing == &key) { + entries[index].1 = value; + } else { + entries.push((key, value)); + } + } + Ok(entries) + } + } + let mut decoder = serde_json::Deserializer::from_str(raw); + let mut entries = decoder + .deserialize_map(OrderedAuth) + .map_err(|_| fail("auth grant is not a valid JSON object"))?; + decoder + .end() + .map_err(|_| fail("auth grant has trailing data"))?; + let array_index = |name: &str| { + name.parse::() + .ok() + .filter(|index| *index != u32::MAX && index.to_string() == name) + }; + // Object.values visits integer-index keys first in ascending numeric order. + entries.sort_by(|(a, _), (b, _)| match (array_index(a), array_index(b)) { + (Some(a), Some(b)) => a.cmp(&b), + (Some(_), None) => std::cmp::Ordering::Less, + (None, Some(_)) => std::cmp::Ordering::Greater, + (None, None) => std::cmp::Ordering::Equal, + }); + let mut selected = BTreeMap::new(); + for (ordinal, (_, value)) in entries.into_iter().enumerate() { + if value.get("type").and_then(serde_json::Value::as_str) != Some("wellknown") { + continue; + } + let Some(key) = value.get("key").and_then(serde_json::Value::as_str) else { + continue; + }; + if !needed.iter().any(|name| name == key) { + continue; + } + let token = value + .get("token") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + let mut err = fail("selected well-known auth token must be a string"); + err.field = Some("auth.wellknown.token"); + err.source_ordinal = Some(ordinal); + err + })?; + if token.len() > 64 * 1024 { + return Err(fail("selected auth lookup value exceeds 64 KiB")); + } + selected.insert(key.to_string(), token.to_string()); + } + Ok(selected) +} + #[cfg(test)] mod tests { use super::*; @@ -542,4 +650,43 @@ mod tests { .unwrap_err(); assert!(!format!("{err:?} {err}").contains("sentinel-private")); } + #[test] + fn selected_native_auth_lookup_keeps_js_duplicate_key_order_and_excludes_unrequested_values() { + let raw = r#"{"first":{"type":"wellknown","key":"CA_PATH","token":"first-path"},"second":{"type":"wellknown","key":"CA_PATH","token":"second-path"},"first":{"type":"wellknown","key":"CA_PATH","token":"last-first-path"},"unrequested":{"type":"wellknown","key":"PRIVATE_UNUSED","token":"sentinel-private-unused"}}"#; + let env = resolve_opencode_auth_environment(raw, &["CA_PATH".into()]).unwrap(); + assert_eq!(env.get("CA_PATH").map(String::as_str), Some("second-path")); + assert_eq!(env.len(), 1); + let numeric=resolve_opencode_auth_environment(r#"{"2":{"type":"wellknown","key":"CA_PATH","token":"two"},"1":{"type":"wellknown","key":"CA_PATH","token":"one"}}"#,&["CA_PATH".into()]).unwrap(); + assert_eq!(numeric.get("CA_PATH").map(String::as_str), Some("two")); + } + + #[test] + fn auth_reader_is_private_bounded_and_does_not_echo_malformed_tokens() { + let temp = tempfile::tempdir().unwrap(); + let path = temp.path().join("auth.json"); + let raw = r#"{"source":{"type":"wellknown","key":"CA_PATH","token":"synthetic-path"}}"#; + std::fs::write(&path, raw).unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + let env = read_opencode_auth_environment(&path, &["CA_PATH".into()]).unwrap(); + assert_eq!( + env.get("CA_PATH").map(String::as_str), + Some("synthetic-path") + ); + assert_eq!(std::fs::read_to_string(&path).unwrap(), raw); + for malformed in [ + r#"{"source":{"type":"wellknown","key":"CA_PATH","token":["sentinel-private"]}}"#, + r#"{"source":{"type":"wellknown","key":"CA_PATH","token":"sentinel-private"}"#, + ] { + let err = + resolve_opencode_auth_environment(malformed, &["CA_PATH".into()]).unwrap_err(); + assert!(!format!("{err:?} {err}").contains("sentinel-private")); + } + std::fs::write(&path, vec![b' '; 128 * 1024 + 1]).unwrap(); + assert!(read_opencode_auth_environment(&path, &["CA_PATH".into()]).is_err()); + assert!(resolve_opencode_auth_environment(raw, &["x".repeat(129)]).is_err()); + } } From 5757973eb0c1bea127f05850da20786c64a272fb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 02:36:09 -0700 Subject: [PATCH 017/493] Resolve inline and symlinked relative config references with native origins --- .../freshell-platform/src/opencode_config.rs | 38 ++++++++++++-- .../src/managed_provider_references.rs | 50 +++++++++++++++++++ 2 files changed, 83 insertions(+), 5 deletions(-) diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index 8b5037e36..227d73f35 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -716,10 +716,38 @@ pub fn file_reference_path( home: &Path, ) -> PathBuf { if let Some(relative) = path.strip_prefix("~/") { - home.join(relative) - } else if Path::new(path).is_absolute() { - path.into() - } else { - declaring_config.parent().unwrap_or(cwd).join(path) + return lexical_path(&home.join(relative.trim_start_matches('/'))); } + if Path::new(path).is_absolute() { + return path.into(); + } + let declaring_parent = declaring_config + .parent() + .filter(|parent| !parent.as_os_str().is_empty()); + let parent = match declaring_parent { + Some(parent) if parent.is_absolute() => parent.to_owned(), + Some(parent) => cwd.join(parent), + None => cwd.to_owned(), + }; + // Node path.resolve normalizes '..' before filesystem symlinks are read. + lexical_path(&parent.join(path)) +} + +fn lexical_path(path: &Path) -> PathBuf { + use std::path::Component; + let mut result = PathBuf::new(); + for component in path.components() { + match component { + Component::CurDir => {} + Component::ParentDir => { + if matches!(result.components().next_back(), Some(Component::Normal(_))) { + result.pop(); + } else if !result.has_root() { + result.push(component.as_os_str()); + } + } + other => result.push(other.as_os_str()), + } + } + result } diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index af13de443..5941b121c 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -582,4 +582,54 @@ process.stdout.write(JSON.stringify(receipts)); ); assert_eq!(plan.recipes[0].unexpanded_source, "{env:OPENAI_API_KEY}"); } + #[test] + fn managed_provider_references_virtual_inline_uses_instance_cwd() { + let temp = tempfile::tempdir().unwrap(); + std::fs::write(temp.path().join("rules"), "inline-instance-nonce").unwrap(); + let plan = resolve_file_references( + r#"{"x":"{file:rules}"}"#, + FileSourceOrigin { + declaring_config: Path::new("OPENCODE_CONFIG_CONTENT"), + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest("inline-instance-nonce")] + ); + } + #[test] + fn managed_provider_references_relative_parent_is_lexical_before_following_directory_symlink() { + let temp = tempfile::tempdir().unwrap(); + let logical = temp.path().join("logical"); + let actual = temp.path().join("actual/config"); + std::fs::create_dir_all(&logical).unwrap(); + std::fs::create_dir_all(&actual).unwrap(); + std::fs::write(logical.join("rules"), "native-lexical-origin-nonce").unwrap(); + std::fs::write( + temp.path().join("actual/rules"), + "wrong-physical-origin-nonce", + ) + .unwrap(); + #[cfg(unix)] + std::os::unix::fs::symlink(&actual, logical.join("linked")).unwrap(); + let source = logical.join("linked/opencode.json"); + let plan = resolve_file_references( + r#"{"x":"{file:../rules}"}"#, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest("native-lexical-origin-nonce")] + ); + } } From eb9ad1454745a3fde7a0f6715435043ff6edde92 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 02:51:16 -0700 Subject: [PATCH 018/493] Read selected native auth only through an owned private helper exchange --- Cargo.lock | 1 + crates/freshell-platform/src/lib.rs | 1 + .../src/native_auth_exchange.rs | 210 ++++++++ crates/freshell-session-host/src/main.rs | 6 + crates/freshell-supervisor/Cargo.toml | 1 + crates/freshell-supervisor/src/backend.rs | 25 + .../src/backend/native_auth.rs | 490 ++++++++++++++++++ scripts/testing/runtime-test-broker.ts | 42 +- .../testing/runtime-test-broker.test.ts | 47 ++ 9 files changed, 817 insertions(+), 6 deletions(-) create mode 100644 crates/freshell-platform/src/native_auth_exchange.rs create mode 100644 crates/freshell-supervisor/src/backend/native_auth.rs diff --git a/Cargo.lock b/Cargo.lock index 368c3e6c5..8732fa6c2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1531,6 +1531,7 @@ dependencies = [ "async-trait", "chrono", "freshell-agent-runtime", + "freshell-platform", "freshell-runtime-observability", "freshell-runtime-protocol", "libc", diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 6c4b18c07..e11db93fe 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -58,6 +58,7 @@ pub mod git_meta; pub mod host_stats_readers; pub mod managed_child_secrets; pub mod mcp_inject; +pub mod native_auth_exchange; pub mod opencode_config; pub mod opencode_plugin; pub mod path; diff --git a/crates/freshell-platform/src/native_auth_exchange.rs b/crates/freshell-platform/src/native_auth_exchange.rs new file mode 100644 index 000000000..745e3c9a9 --- /dev/null +++ b/crates/freshell-platform/src/native_auth_exchange.rs @@ -0,0 +1,210 @@ +//! Single-use, memory-only selected OpenCode auth lookup over private IPC. +//! The caller supplies an exact owned provider HOME; no host HOME fallback exists. +use std::{collections::BTreeMap, path::Path}; + +pub const MAX_AUTH_FRAME: usize = 1024 * 1024; + +pub fn validate_keys(keys: &[String]) -> Result<(), &'static str> { + crate::provider_environment_grants::resolve_opencode_auth_environment("{}", keys) + .map(|_| ()) + .map_err(|_| "native auth selected keys are invalid") +} + +pub fn validate_selected( + keys: &[String], + values: &BTreeMap, +) -> Result<(), &'static str> { + validate_keys(keys)?; + let mut total = 0usize; + for (key, value) in values { + if !keys.contains(key) || value.len() > 64 * 1024 || value.contains('\0') { + return Err("native auth selected values are invalid"); + } + total = total + .checked_add(key.len()) + .and_then(|n| n.checked_add(value.len())) + .ok_or("native auth selected values exceed limit")?; + if total > 256 * 1024 { + return Err("native auth selected values exceed limit"); + } + } + Ok(()) +} + +struct BoundedFrame(Vec); +impl std::io::Write for BoundedFrame { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + if bytes.len() > MAX_AUTH_FRAME.saturating_sub(self.0.len()) { + return Err(std::io::Error::other("native auth frame exceeds limit")); + } + self.0.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } +} + +/// The only serialized auth reply is sent directly to this private socket. +#[cfg(unix)] +fn selected_native_auth( + home: &Path, + keys: &[String], +) -> Result, &'static str> { + use std::io::Read; + validate_keys(keys)?; + let home = std::fs::canonicalize(home).map_err(|_| "native auth HOME is unavailable")?; + let path = home.join(".local/share/opencode/auth.json"); + match std::fs::symlink_metadata(&path) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(BTreeMap::new()), + Err(_) => return Err("native auth file is unavailable"), + Ok(_) => {} + } + let canonical = std::fs::canonicalize(&path).map_err(|_| "native auth file is unavailable")?; + if !canonical.starts_with(&home) { + return Err("native auth file escaped owned HOME"); + } + let file = std::fs::File::open(canonical).map_err(|_| "native auth file is unavailable")?; + if !file + .metadata() + .map_err(|_| "native auth file metadata failed")? + .is_file() + { + return Err("native auth file is not regular"); + } + let mut bytes = Vec::new(); + file.take(128 * 1024 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "native auth file read failed")?; + if bytes.len() > 128 * 1024 { + return Err("native auth file exceeds limit"); + } + let raw = String::from_utf8(bytes).map_err(|_| "native auth file is not UTF-8")?; + let selected = + crate::provider_environment_grants::resolve_opencode_auth_environment(&raw, keys) + .map_err(|_| "native auth file cannot be decoded")?; + validate_selected(keys, &selected)?; + Ok(selected) +} + +#[cfg(unix)] +pub fn serve(home: &Path, socket: &Path) -> Result<(), &'static str> { + use std::{ + io::{Read, Write}, + os::unix::net::UnixStream, + }; + let mut stream = UnixStream::connect(socket).map_err(|_| "native auth exchange unavailable")?; + let timeout = Some(std::time::Duration::from_secs(15)); + stream + .set_read_timeout(timeout) + .map_err(|_| "native auth exchange unavailable")?; + stream + .set_write_timeout(timeout) + .map_err(|_| "native auth exchange unavailable")?; + let mut length = [0; 4]; + stream + .read_exact(&mut length) + .map_err(|_| "native auth request unavailable")?; + let length = u32::from_be_bytes(length) as usize; + if length > 32 * 1024 { + return Err("native auth request exceeds limit"); + } + let mut request = vec![0; length]; + stream + .read_exact(&mut request) + .map_err(|_| "native auth request unavailable")?; + let keys: Vec = + serde_json::from_slice(&request).map_err(|_| "native auth request invalid")?; + let selected = selected_native_auth(home, &keys)?; + let mut frame = BoundedFrame(Vec::new()); + serde_json::to_writer(&mut frame, &selected).map_err(|_| "native auth reply exceeds limit")?; + stream + .write_all(&(frame.0.len() as u32).to_be_bytes()) + .map_err(|_| "native auth reply unavailable")?; + stream + .write_all(&frame.0) + .map_err(|_| "native auth reply unavailable")?; + Ok(()) +} + +#[cfg(all(test, unix))] +mod tests { + use super::*; + use std::{ + io::{Read, Write}, + os::unix::net::UnixListener, + }; + + #[test] + fn selected_native_auth_values_travel_only_through_the_private_exchange() { + let temp = tempfile::tempdir().unwrap(); + let home = temp.path().join("owned-home"); + let path = home.join(".local/share/opencode/auth.json"); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + let raw = r#"{"a":{"type":"wellknown","key":"PATH_INPUT","token":"/synthetic/selected"},"b":{"type":"api","key":"UNSELECTED","token":"PRIVATE_UNSELECTED_SENTINEL"}}"#; + std::fs::write(&path, raw).unwrap(); + let socket = temp.path().join("reply.sock"); + let listener = UnixListener::bind(&socket).unwrap(); + let worker = std::thread::spawn(move || serve(&home, &socket)); + listener.set_nonblocking(true).unwrap(); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(1); + let (mut stream, _) = loop { + match listener.accept() { + Ok(pair) => break pair, + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + if std::time::Instant::now() >= deadline { + panic!( + "native auth consumer did not connect: {:?}", + worker.join().unwrap() + ); + } + std::thread::yield_now(); + } + Err(error) => panic!("{error}"), + } + }; + let request = serde_json::to_vec(&vec!["PATH_INPUT"]).unwrap(); + stream + .write_all(&(request.len() as u32).to_be_bytes()) + .unwrap(); + stream.write_all(&request).unwrap(); + let mut length = [0; 4]; + stream.read_exact(&mut length).unwrap(); + let mut reply = vec![0; u32::from_be_bytes(length) as usize]; + stream.read_exact(&mut reply).unwrap(); + let selected: BTreeMap = serde_json::from_slice(&reply).unwrap(); + assert_eq!( + selected.get("PATH_INPUT").map(String::as_str), + Some("/synthetic/selected") + ); + assert!(!String::from_utf8(reply) + .unwrap() + .contains("PRIVATE_UNSELECTED_SENTINEL")); + assert_eq!(worker.join().unwrap(), Ok(())); + assert_eq!(std::fs::read_to_string(path).unwrap(), raw); + } + + #[test] + fn native_auth_never_falls_back_to_an_outside_file_and_charges_actual_bytes() { + use std::os::unix::fs::symlink; + let temp = tempfile::tempdir().unwrap(); + let home = temp.path().join("owned"); + let path = home.join(".local/share/opencode/auth.json"); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + let outside = temp.path().join("outside-auth"); + std::fs::write(&outside, "PRIVATE_SENTINEL").unwrap(); + symlink(&outside, &path).unwrap(); + assert_eq!( + selected_native_auth(&home, &[]).unwrap_err(), + "native auth file escaped owned HOME" + ); + std::fs::remove_file(&path).unwrap(); + std::fs::write(&path, vec![b' '; 128 * 1024 + 1]).unwrap(); + assert_eq!( + selected_native_auth(&home, &[]).unwrap_err(), + "native auth file exceeds limit" + ); + std::fs::remove_file(&path).unwrap(); + assert!(selected_native_auth(&home, &[]).unwrap().is_empty()); + } +} diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 6d1f8f0a6..e856068b8 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -116,6 +116,12 @@ async fn run() -> Result<(), String> { match args.get(1).map(String::as_str) { Some("serve") => serve(&args[2..]).await, Some("worker") => worker(&args[2..]).await, + Some("native-auth-only") if args.len() == 2 => { + freshell_platform::native_auth_exchange::serve( + Path::new("/home/freshell/provider"), + Path::new("/run/freshell-auth-exchange/reply.sock"), + ).map_err(str::to_owned) + } Some("native-history-only") => { let snapshot = freshell_freshagent::native_history::read( &required_arg(&args[2..], "--provider")?, diff --git a/crates/freshell-supervisor/Cargo.toml b/crates/freshell-supervisor/Cargo.toml index 56dd2df06..99c1d70c2 100644 --- a/crates/freshell-supervisor/Cargo.toml +++ b/crates/freshell-supervisor/Cargo.toml @@ -13,6 +13,7 @@ provider-qualification = ["freshell-agent-runtime/provider-qualification"] [dependencies] freshell-agent-runtime = { path = "../freshell-agent-runtime" } freshell-runtime-protocol = { path = "../freshell-runtime-protocol" } +freshell-platform = { path = "../freshell-platform" } freshell-runtime-observability = { path = "../freshell-runtime-observability" } serde = { workspace = true } serde_json = { workspace = true } diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 40191b3d9..bb2ceedaa 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -1,4 +1,5 @@ pub mod docker; +mod native_auth; mod native_history; use crate::registry::OwnedRuntimeHandle; @@ -117,6 +118,16 @@ pub struct BackendInspection { #[async_trait] pub trait RuntimeBackend: Send + Sync { + async fn read_native_auth( + &self, + _handle: &OwnedRuntimeHandle, + _keys: &[String], + _reader_binary: &Path, + ) -> Result, BackendError> { + Err(BackendError::Unavailable( + "native auth reader unavailable".into(), + )) + } async fn read_native_history( &self, _handle: &OwnedRuntimeHandle, @@ -498,6 +509,20 @@ fn runtime_host_environment( #[async_trait] impl RuntimeBackend for DockerEngineBackend { + async fn read_native_auth( + &self, + handle: &OwnedRuntimeHandle, + keys: &[String], + reader_binary: &Path, + ) -> Result, BackendError> { + self.read_auth_helper( + handle, + keys, + reader_binary, + std::time::Duration::from_secs(20), + ) + .await + } async fn read_native_history( &self, handle: &OwnedRuntimeHandle, diff --git a/crates/freshell-supervisor/src/backend/native_auth.rs b/crates/freshell-supervisor/src/backend/native_auth.rs new file mode 100644 index 000000000..cad6c3e79 --- /dev/null +++ b/crates/freshell-supervisor/src/backend/native_auth.rs @@ -0,0 +1,490 @@ +//! Exact owned, quiescent soul auth lookup. Docker never carries reply values. +use super::*; +use std::time::Duration; +use tokio::net::UnixListener; + +fn fail(cause: &'static str) -> BackendError { + BackendError::Unavailable(cause.into()) +} + +struct ExchangeDirectory(PathBuf); +impl ExchangeDirectory { + fn create() -> Result { + use std::os::unix::ffi::OsStrExt; + // Deliberately short regardless of runtime_root length (Linux sun_path). + let path = std::env::temp_dir().join(format!("freshell-auth-{}", uuid::Uuid::new_v4())); + if path.join("reply.sock").as_os_str().as_bytes().len() > 107 { + return Err(fail("native auth exchange path exceeds limit")); + } + std::fs::create_dir(&path).map_err(|_| fail("native auth exchange creation failed"))?; + let owned = Self(path); + owned.mode(&owned.0, 0o710)?; + Ok(owned) + } + fn mode(&self, path: &Path, mode: u32) -> Result<(), BackendError> { + use std::os::unix::{ + ffi::OsStrExt, + fs::{MetadataExt, PermissionsExt}, + }; + let name = std::ffi::CString::new(path.as_os_str().as_bytes()) + .map_err(|_| fail("native auth exchange path is invalid"))?; + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + // Rootless namespace group 0 is the controller's primary group. + if unsafe { libc::lchown(name.as_ptr(), uid, gid) } != 0 { + return Err(fail("native auth mapped group is unavailable")); + } + std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode)) + .map_err(|_| fail("native auth exchange permission failed"))?; + let meta = std::fs::symlink_metadata(path) + .map_err(|_| fail("native auth exchange verification failed"))?; + if meta.uid() != uid || meta.gid() != gid || meta.mode() & 0o777 != mode { + return Err(fail("native auth exchange ownership changed")); + } + Ok(()) + } +} +impl Drop for ExchangeDirectory { + fn drop(&mut self) { + let _ = std::fs::remove_file(self.0.join("reply.sock")); + let _ = std::fs::remove_dir(&self.0); + } +} + +async fn exchange( + listener: &UnixListener, + keys: &[String], +) -> Result, BackendError> { + use freshell_platform::native_auth_exchange::{validate_selected, MAX_AUTH_FRAME}; + let (mut stream, _) = listener + .accept() + .await + .map_err(|_| fail("native auth exchange accept failed"))?; + let request = serde_json::to_vec(keys).map_err(|_| fail("native auth request invalid"))?; + if request.len() > 32 * 1024 { + return Err(fail("native auth request exceeds limit")); + } + stream + .write_all(&(request.len() as u32).to_be_bytes()) + .await + .map_err(|_| fail("native auth request failed"))?; + stream + .write_all(&request) + .await + .map_err(|_| fail("native auth request failed"))?; + let length = stream + .read_u32() + .await + .map_err(|_| fail("native auth reply unavailable"))? as usize; + if length > MAX_AUTH_FRAME { + return Err(fail("native auth reply exceeds limit")); + } + let mut reply = vec![0; length]; + stream + .read_exact(&mut reply) + .await + .map_err(|_| fail("native auth reply unavailable"))?; + let values = serde_json::from_slice(&reply).map_err(|_| fail("native auth reply invalid"))?; + validate_selected(keys, &values).map_err(fail)?; + Ok(values) +} + +impl DockerEngineBackend { + pub(super) async fn read_auth_helper( + &self, + handle: &OwnedRuntimeHandle, + keys: &[String], + binary: &Path, + budget: Duration, + ) -> Result, BackendError> { + freshell_platform::native_auth_exchange::validate_keys(keys).map_err(fail)?; + let provider = handle + .terminal() + .map(|spec| spec.mode.as_str()) + .or_else(|| handle.fresh_agent().map(|spec| spec.provider.as_str())); + let identity = handle + .terminal() + .map(|spec| (spec.run_as_uid, spec.run_as_gid)) + .or_else(|| { + handle + .fresh_agent() + .map(|spec| (spec.run_as_uid, spec.run_as_gid)) + }); + if provider != Some("opencode") || identity != Some((65534, 0)) { + return Err(fail("native auth source is not an owned OpenCode provider")); + } + let name = format!("freshell-auth-{}", uuid::Uuid::new_v4()); + let mut create_attempted = false; + let mut expected = None::; + let mut directory = None::; + let operation = async { + let inspection = self + .inspect(handle) + .await + .map_err(|_| fail("native auth owned source inspection failed"))?; + if !matches!( + inspection.state, + BackendRuntimeState::Created + | BackendRuntimeState::Exited + | BackendRuntimeState::Dead + | BackendRuntimeState::Missing + ) { + return Err(fail("native auth requires a quiescent owned soul")); + } + let volume = self + .request( + "GET", + &format!("{DOCKER_API}/volumes/{}", handle.provider_volume_name()), + None, + ) + .await + .map_err(|_| fail("native auth volume inspection failed"))?; + if volume.status != 200 { + return Err(fail("native auth expected owned volume is unavailable")); + } + let volume: Value = serde_json::from_slice(&volume.body) + .map_err(|_| fail("native auth volume inspection invalid"))?; + if volume["Name"].as_str() != Some(handle.provider_volume_name()) { + return Err(fail("native auth volume ownership changed")); + } + let binary = canonical_regular_file(binary) + .map_err(|_| fail("native auth reader binary unavailable"))?; + directory = Some(ExchangeDirectory::create()?); + let directory = directory.as_ref().unwrap(); + let socket = directory.0.join("reply.sock"); + let listener = UnixListener::bind(&socket) + .map_err(|_| fail("native auth exchange bind failed"))?; + directory.mode(&socket, 0o660)?; + let body = json!({ + "Image":handle.image_ref(),"User":"65534:0","Tty":false, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["native-auth-only"],"Env":["HOME=/home/freshell/provider"], + "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"SecurityOpt":["no-new-privileges"], + "Memory":256*1024*1024,"MemorySwap":256*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, + "LogConfig":{"Type":"none","Config":{}}, + "Mounts":[{"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, + {"Type":"volume","Source":handle.provider_volume_name(),"Target":"/home/freshell/provider","ReadOnly":true,"VolumeOptions":{"NoCopy":true}}, + {"Type":"bind","Source":directory.0,"Target":"/run/freshell-auth-exchange","ReadOnly":true}]} + }); + expected = Some(body.clone()); + create_attempted = true; + let created = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/create?name={name}"), + Some(&body), + 64 * 1024, + ) + .await + .map_err(|_| fail("native auth helper creation acknowledgement unavailable"))?; + if created.status == 409 { + create_attempted = false; + } + if created.status != 201 { + return Err(fail("native auth helper creation failed")); + } + let created: Value = serde_json::from_slice(&created.body) + .map_err(|_| fail("native auth helper creation invalid"))?; + let id = created["Id"] + .as_str() + .ok_or_else(|| fail("native auth helper identity unavailable"))?; + let supervision = async { + let started = self + .request("POST", &format!("{DOCKER_API}/containers/{id}/start"), None) + .await + .map_err(|_| fail("native auth helper start failed"))?; + if started.status != 204 { + return Err(fail("native auth helper start failed")); + } + let waited = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/wait?condition=not-running"), + None, + 64 * 1024, + ) + .await + .map_err(|_| fail("native auth helper wait failed"))?; + if waited.status != 200 { + return Err(fail("native auth helper wait failed")); + } + let waited: Value = serde_json::from_slice(&waited.body) + .map_err(|_| fail("native auth helper wait invalid"))?; + if waited["StatusCode"].as_i64() != Some(0) { + return Err(fail("native auth helper failed")); + } + Ok(()) + }; + let (values, ()) = tokio::try_join!(exchange(&listener, keys), supervision)?; + Ok(values) + }; + let result = tokio::time::timeout(budget, operation) + .await + .unwrap_or_else(|_| Err(fail("native auth lookup timed out"))); + if create_attempted { + let cleanup = async { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 1024 * 1024, + ) + .await + .map_err(|_| fail("native auth cleanup inspection failed"))?; + if inspected.status == 404 { + return Ok(()); + } + if inspected.status != 200 { + return Err(fail("native auth cleanup inspection failed")); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("native auth cleanup inspection invalid"))?; + let expected = expected + .as_ref() + .ok_or_else(|| fail("native auth cleanup authority unavailable"))?; + if ["Image", "User", "Tty", "Entrypoint", "Cmd", "Env"] + .iter() + .any(|key| actual["Config"][key] != expected[key]) + || actual["Config"]["Labels"] + .as_object() + .is_some_and(|labels| !labels.is_empty()) + || actual["HostConfig"]["Mounts"] != expected["HostConfig"]["Mounts"] + || actual["HostConfig"]["LogConfig"]["Type"] != "none" + { + return Err(fail("native auth cleanup ownership changed")); + } + let deleted = self + .request( + "DELETE", + &format!("{DOCKER_API}/containers/{name}?force=1"), + None, + ) + .await + .map_err(|_| fail("native auth cleanup failed"))?; + if deleted.status != 204 && deleted.status != 404 { + return Err(fail("native auth cleanup failed")); + } + Ok(()) + }; + tokio::time::timeout(Duration::from_secs(5), cleanup) + .await + .map_err(|_| fail("native auth cleanup timed out"))??; + } + tracing::debug!(soul_id=%handle.soul_id(),helper=%name,succeeded=result.is_ok(),"runtime.native_auth_lookup_settled"); + result + } +} + +#[cfg(test)] +mod tests { + use super::*; + use freshell_runtime_protocol::LaunchNonce; + use std::sync::{Arc, Mutex}; + + #[tokio::test] + async fn native_auth_uses_private_ipc_and_cleans_only_verified_owned_helper() { + for mode in [ + "success", + "failed_exit", + "timeout", + "lost_create_ack", + "cleanup_mismatch", + ] { + let temp = tempfile::tempdir().unwrap(); + let home = temp.path().join("owned-home"); + let auth = home.join(".local/share/opencode/auth.json"); + std::fs::create_dir_all(auth.parent().unwrap()).unwrap(); + std::fs::write(&auth, r#"{"selected":{"type":"wellknown","key":"FILE_PATH","token":"/synthetic/PRIVATE_AUTH_SENTINEL"}}"#).unwrap(); + let socket = temp.path().join("docker.sock"); + let listener = UnixListener::bind(&socket).unwrap(); + let daemon = DockerDaemonId::new(); + let binary = temp.path().join("reader"); + std::fs::write(&binary, "fixture executable").unwrap(); + let agent = serde_json::from_value(json!({ + "sessionId":"alias", "provider":"opencode", "sessionType":"freshopencode", "runtimeVariant":"opencode", "providerStoreId":"store", + "cwd":"/workspace", "workspacePath":"/workspace", "runAsUid":65534, "runAsGid":0 + })).unwrap(); + let handle = OwnedRuntimeHandle::from_registry( + InstallationId::new(), + SoulId::new(), + IncarnationId::new(), + LaunchNonce::new(), + daemon.clone(), + "provider-container-must-not-stop".into(), + "sha256:fixture".into(), + temp.path().into(), + binary.clone(), + "digest".into(), + RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + }, + None, + None, + Some(agent), + "owned-native-volume".into(), + ); + let seen = Arc::new(Mutex::new(Vec::<(String, Vec)>::new())); + let captured = seen.clone(); + let created = Arc::new(Mutex::new(None::)); + let server = tokio::spawn(async move { + loop { + let (mut stream, _) = listener.accept().await.unwrap(); + let (seen, created, daemon, home) = ( + captured.clone(), + created.clone(), + daemon.clone(), + home.clone(), + ); + tokio::spawn(async move { + let mut raw = Vec::new(); + let mut byte = [0]; + while !raw.ends_with(b"\r\n\r\n") { + stream.read_exact(&mut byte).await.unwrap(); + raw.push(byte[0]); + } + let head = String::from_utf8(raw).unwrap(); + let length = head + .lines() + .find_map(|line| line.strip_prefix("Content-Length: ")) + .unwrap() + .trim() + .parse::() + .unwrap(); + let mut body = vec![0; length]; + stream.read_exact(&mut body).await.unwrap(); + let request = head.lines().next().unwrap().to_string(); + seen.lock().unwrap().push((request.clone(), body.clone())); + let (status, body) = if request.contains("/info ") { + ( + 200, + json!({"ID":daemon,"SecurityOptions":["name=rootless"]}) + .to_string(), + ) + } else if request.contains("provider-container-must-not-stop/json") { + (404, String::new()) + } else if request.contains("/volumes/") { + (200, json!({"Name":"owned-native-volume"}).to_string()) + } else if request.contains("/create?") { + *created.lock().unwrap() = Some(serde_json::from_slice(&body).unwrap()); + ( + if mode == "lost_create_ack" { 503 } else { 201 }, + json!({"Id":"owned-auth-helper"}).to_string(), + ) + } else if request.contains("/start ") { + if mode != "timeout" { + let config = created.lock().unwrap().clone().unwrap(); + let exchange = config["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .find(|mount| mount["Target"] == "/run/freshell-auth-exchange") + .unwrap()["Source"] + .as_str() + .unwrap() + .to_owned(); + std::thread::spawn(move || { + let result = freshell_platform::native_auth_exchange::serve( + &home, + &Path::new(&exchange).join("reply.sock"), + ); + if mode == "success" { + assert_eq!(result, Ok(())); + } + }); + } + (204, String::new()) + } else if request.contains("/wait?") { + if mode == "timeout" { + tokio::time::sleep(Duration::from_secs(1)).await; + } + ( + 200, + json!({"StatusCode":if mode == "failed_exit" {1} else {0}}) + .to_string(), + ) + } else if request.contains("/json ") { + let config = created.lock().unwrap().clone().unwrap(); + (200, json!({"Config":if mode == "cleanup_mismatch" {json!({"Cmd":["other"]})} else {config.clone()},"HostConfig":config["HostConfig"],"Name":"owned-auth-helper"}).to_string()) + } else if request.starts_with("DELETE ") { + (204, String::new()) + } else { + panic!("unexpected Docker operation {request}"); + }; + let reply = format!("HTTP/1.1 {status} OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len()); + let _ = stream.write_all(reply.as_bytes()).await; + }); + } + }); + let result = DockerEngineBackend::new(&socket) + .read_auth_helper( + &handle, + &["FILE_PATH".into()], + &binary, + if mode == "timeout" { + Duration::from_millis(100) + } else { + Duration::from_secs(3) + }, + ) + .await; + assert_eq!( + result.is_ok(), + mode == "success", + "{mode}: {}", + result + .as_ref() + .err() + .map(ToString::to_string) + .unwrap_or_default() + ); + if mode == "success" { + assert_eq!( + result.unwrap()["FILE_PATH"], + "/synthetic/PRIVATE_AUTH_SENTINEL" + ); + } + let records = seen.lock().unwrap(); + let create = records + .iter() + .find(|(request, _)| request.contains("/create?")) + .expect("helper must actually run"); + let body: Value = serde_json::from_slice(&create.1).unwrap(); + assert!(!String::from_utf8_lossy(&create.1).contains("PRIVATE_AUTH_SENTINEL")); + assert_eq!(body["Cmd"], json!(["native-auth-only"])); + assert_eq!(body["HostConfig"]["LogConfig"]["Type"], "none"); + assert_eq!(body["User"], "65534:0"); + assert_eq!(body["HostConfig"]["Mounts"].as_array().unwrap().len(), 3); + assert!(records.iter().all(|(request, _)| !request.contains("/logs") + && !(request.starts_with("DELETE") && request.contains(handle.container_id())) + && !request.contains("/stop"))); + let deletes: Vec<_> = records + .iter() + .filter(|(request, _)| request.starts_with("DELETE ")) + .collect(); + assert_eq!( + deletes.len(), + if mode == "cleanup_mismatch" { 0 } else { 1 }, + "{mode}" + ); + if let Some(delete) = deletes.first() { + assert!(delete.0.contains("/containers/freshell-auth-")); + } + let exchange = PathBuf::from( + body["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .find(|mount| mount["Target"] == "/run/freshell-auth-exchange") + .unwrap()["Source"] + .as_str() + .unwrap(), + ); + assert!(!exchange.exists(), "private exchange cleanup failed"); + drop(records); + server.abort(); + let _ = server.await; + } + } +} diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 09cd50544..a88a82b9d 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -58,7 +58,7 @@ type DockerResponse = { export class RestrictedDockerBroker { private readonly knownContainerIds = new Set() private readonly receiptsById = new Map() - private readonly historyHelpers = new Map() + private readonly historyHelpers = new Map() private readonly events: BrokerEvent[] = [] private server?: http.Server private stopFailuresRemaining = 0 @@ -144,7 +144,7 @@ export class RestrictedDockerBroker { return } - const helperTarget = url.match(/^\/v1\.47\/containers\/([^/?]+)(?:\/(start|wait|logs))?(?:\?.*)?$/) + const helperTarget = url.match(/^\/v1\.47\/containers\/([^/?]+)(?:\/(start|wait|logs|json))?(?:\?.*)?$/) const helperEntry = helperTarget && [...this.historyHelpers].find(([name, helper]) => ( helperTarget[1] === name || helperTarget[1] === helper.containerId )) @@ -153,7 +153,8 @@ export class RestrictedDockerBroker { const exactId = helperTarget![1] === helper.containerId const allowed = (method === 'POST' && exactId && url === `${DOCKER_API_PREFIX}/containers/${helper.containerId}/start`) || (method === 'POST' && exactId && url === `${DOCKER_API_PREFIX}/containers/${helper.containerId}/wait?condition=not-running`) - || (method === 'GET' && exactId && [0, 1].some((stderr) => url === `${DOCKER_API_PREFIX}/containers/${helper.containerId}/logs?stdout=1&stderr=${stderr}`)) + || (method === 'GET' && helper.kind === 'history' && exactId && [0, 1].some((stderr) => url === `${DOCKER_API_PREFIX}/containers/${helper.containerId}/logs?stdout=1&stderr=${stderr}`)) + || (method === 'GET' && helper.kind === 'auth' && url === `${DOCKER_API_PREFIX}/containers/${helperTarget![1]}/json`) || (method === 'DELETE' && url === `${DOCKER_API_PREFIX}/containers/${helperTarget![1]}?force=1`) if (allowed) { const forwarded = await this.forward(request, body) @@ -166,14 +167,15 @@ export class RestrictedDockerBroker { if (method === 'POST' && url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`)) { const name = new URL(url, 'http://docker').searchParams.get('name') ?? '' - if (name.startsWith('freshell-history-')) { - const owner = this.validateHistoryHelper(name, body) + if (name.startsWith('freshell-history-') || name.startsWith('freshell-auth-')) { + const kind = name.startsWith('freshell-auth-') ? 'auth' : 'history' + const owner = kind === 'auth' ? this.validateAuthHelper(name, body) : this.validateHistoryHelper(name, body) if (!owner || this.historyHelpers.has(name)) { this.block(response, method, url, 403, 'history helper does not match an owned read-only source', false) return } // Reserve the exact validated name before forwarding: Docker may create it but lose its acknowledgement. - const helper: { owner: BrokerReceipt; containerId?: string } = { owner } + const helper: { owner: BrokerReceipt; containerId?: string; kind: 'history' | 'auth' } = { owner, kind } this.historyHelpers.set(name, helper) try { const forwarded = await this.forward(request, body) @@ -424,6 +426,34 @@ export class RestrictedDockerBroker { && receipt.imageRef === parsed.Image && receipt.hostBinaryPath === binary.Source) } + private validateAuthHelper(name: string, body: Buffer): BrokerReceipt | undefined { + const uuid = '[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}' + if (!new RegExp(`^freshell-auth-${uuid}$`).test(name)) return + let parsed: Record + try { parsed = JSON.parse(body.toString('utf8')) } catch { return } + const host = parsed.HostConfig ?? {} + const same = (actual: unknown, expected: unknown) => JSON.stringify(actual) === JSON.stringify(expected) + if (Object.keys(parsed.Labels ?? {}).length !== 0 || parsed.User !== '65534:0' || parsed.Tty !== false + || !same(parsed.Entrypoint, ['/runtime/freshell-session-host']) || !same(parsed.Cmd, ['native-auth-only']) + || !same(parsed.Env, ['HOME=/home/freshell/provider'])) return + if (host.NetworkMode !== 'none' || host.ReadonlyRootfs !== true || host.Privileged === true + || (host.PidMode ?? '') !== '' || (host.Binds?.length ?? 0) !== 0 || (host.CapAdd?.length ?? 0) !== 0 + || !same(host.CapDrop, ['ALL']) || !same(host.SecurityOpt, ['no-new-privileges']) + || host.Memory !== 256 * 1024 * 1024 || host.MemorySwap !== 256 * 1024 * 1024 + || host.NanoCpus !== 500_000_000 || host.PidsLimit !== 32 || Object.keys(host.Tmpfs ?? {}).length !== 0 + || !same(host.LogConfig, { Type: 'none', Config: {} })) return + const mounts = host.Mounts + if (!Array.isArray(mounts) || mounts.length !== 3) return + const binary = mounts.find((mount) => mount.Type === 'bind' && mount.Target === '/runtime/freshell-session-host' && mount.ReadOnly === true) + const volume = mounts.find((mount) => mount.Type === 'volume' && mount.Target === '/home/freshell/provider' && mount.ReadOnly === true && same(mount.VolumeOptions, { NoCopy: true })) + const exchange = mounts.find((mount) => mount.Type === 'bind' && mount.Target === '/run/freshell-auth-exchange' && mount.ReadOnly === true) + if (!binary || !volume || !exchange || typeof exchange.Source !== 'string' + || !new RegExp(`^freshell-auth-${uuid}$`).test(path.basename(exchange.Source)) || path.dirname(exchange.Source) !== os.tmpdir() + || !this.policy.allowedHostBinaryPaths.has(binary.Source) || !this.policy.allowedImageRefs.has(parsed.Image)) return + return this.receipts().find((receipt) => receipt.providerVolumeName === volume.Source + && receipt.imageRef === parsed.Image && receipt.hostBinaryPath === binary.Source) + } + private isAllowedWorkspacePath(candidate: string): boolean { const roots = this.policy.allowedWorkspaceRoots ?? new Set() for (const root of roots) { diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 35897a4fc..e2f2886c4 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -183,6 +183,53 @@ function historyHelperBody() { } } +const authName = 'freshell-auth-11111111-1111-4111-8111-111111111111' +function authHelperBody(): any { + const body: any = historyHelperBody() + body.Tty = false + body.Cmd = ['native-auth-only'] + delete body.HostConfig.Tmpfs + body.HostConfig.LogConfig = { Type: 'none', Config: {} } + body.HostConfig.Mounts[1].VolumeOptions = { NoCopy: true } + body.HostConfig.Mounts.push({ Type: 'bind', Source: `/tmp/${authName}`, Target: '/run/freshell-auth-exchange', ReadOnly: true }) + return body +} + +describe('receipt-owned auth exchange HTTP routes', () => { + it('allows only the fixed private helper, lifecycle and ownership inspection with Docker logs disabled', async () => { + await withHttpBroker(async ({ request, forwarded }) => { + await request('POST', '/v1.47/containers/create?name=owned-runtime', createBody([...ordinaryBinds, actorBind])) + const mutations = [ + (body: any) => { body.Cmd.push('PRIVATE_SENTINEL') }, + (body: any) => { body.HostConfig.LogConfig.Type = 'json-file' }, + (body: any) => { body.HostConfig.Mounts[1].ReadOnly = false }, + (body: any) => { body.HostConfig.Mounts[2].Source = '/tmp/unapproved' }, + (body: any) => { body.HostConfig.Mounts[2].ReadOnly = false }, + (body: any) => { body.Env.push('SECRET=PRIVATE_SENTINEL') }, + ] + for (const mutate of mutations) { + const body = authHelperBody() + mutate(body) + const before = forwarded.length + expect((await request('POST', `/v1.47/containers/create?name=${authName}`, body)).status).toBe(403) + expect(forwarded).toHaveLength(before) + } + expect((await request('POST', `/v1.47/containers/create?name=${authName}`, authHelperBody())).status).toBe(201) + expect((await request('POST', `/v1.47/containers/${helperId}/start`)).status).toBe(204) + expect((await request('POST', `/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200) + expect((await request('GET', `/v1.47/containers/${authName}/json`)).status).toBe(200) + const before = forwarded.length + expect((await request('GET', `/v1.47/containers/${helperId}/logs?stdout=1&stderr=1`)).status).toBe(403) + expect((await request('POST', `/v1.47/containers/${helperId}/stop`)).status).toBe(403) + expect(forwarded).toHaveLength(before) + expect((await request('DELETE', `/v1.47/containers/${authName}?force=1`)).status).toBe(204) + expect(forwarded.filter((operation) => operation.method === 'DELETE')).toEqual([ + { method: 'DELETE', url: `/v1.47/containers/${authName}?force=1`, body: undefined }, + ]) + }) + }) +}) + type ForwardedRequest = { method: string; url: string; body: any } type BrokerHttpFixture = { request(method: string, url: string, body?: unknown): Promise<{ status: number; body: string }> From 18dba51f44c158bc78261306c741ef25b39a7c90 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 03:00:23 -0700 Subject: [PATCH 019/493] Scope nested native auth lookup to the current recovery attempt --- crates/freshell-runtime-client/src/lib.rs | 20 ++ crates/freshell-runtime-protocol/src/lib.rs | 109 +++++++++- .../src/managed_mcp_capability.rs | 9 +- .../src/native_auth_lookup_tests.rs | 200 ++++++++++++++++++ crates/freshell-supervisor/src/recovery.rs | 2 +- crates/freshell-supervisor/src/registry.rs | 14 ++ crates/freshell-supervisor/src/service.rs | 168 +++++++++++++-- 7 files changed, 502 insertions(+), 20 deletions(-) create mode 100644 crates/freshell-supervisor/src/native_auth_lookup_tests.rs diff --git a/crates/freshell-runtime-client/src/lib.rs b/crates/freshell-runtime-client/src/lib.rs index 085c80ca8..3d80fbeb9 100644 --- a/crates/freshell-runtime-client/src/lib.rs +++ b/crates/freshell-runtime-client/src/lib.rs @@ -51,6 +51,26 @@ pub struct RuntimeClient { } impl RuntimeClient { + pub async fn native_auth_lookup( + &self, + soul_id: SoulId, + attempt_authority: String, + keys: Vec, + ) -> Result { + let request = freshell_runtime_protocol::NativeAuthLookupRequest { + expected_control_epoch: Some(self.current_epoch().await?), + soul_id, + attempt_authority, + keys, + }; + match self + .request(RequestId::new(), AdminCommand::NativeAuthLookup(request)) + .await? + { + AdminResult::NativeAuthLookup(values) => Ok(values), + _ => Err(ClientError::UnexpectedResult), + } + } pub async fn from_secret_file( socket_path: impl Into, secret_file: impl Into, diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 49e1798b5..6add6069c 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2687,6 +2687,7 @@ pub struct QualificationWriterClaimEvidence { #[serde(tag = "method", content = "params", rename_all = "snake_case")] pub enum AdminCommand { Health, + NativeAuthLookup(NativeAuthLookupRequest), Launch(Box), Stop(StopRequest), Inventory, @@ -2721,6 +2722,28 @@ pub enum AdminCommand { Recover(RecoverRequest), } +/// Transient nested callback request; the authority is scoped by the supervisor +/// to the recovery attempt whose lifecycle lock is already held. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct NativeAuthLookupRequest { + pub expected_control_epoch: Option, + pub soul_id: SoulId, + pub attempt_authority: String, + pub keys: Vec, +} + +#[derive(Clone, Serialize, Deserialize)] +#[serde(transparent)] +pub struct NativeAuthValues(pub std::collections::BTreeMap); +impl fmt::Debug for NativeAuthValues { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("NativeAuthValues") + .field("selected_count", &self.0.len()) + .finish() + } +} + #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct RuntimeView { @@ -2835,6 +2858,7 @@ pub struct LaunchResult { #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(tag = "kind", content = "data", rename_all = "snake_case")] pub enum AdminResult { + NativeAuthLookup(NativeAuthValues), Health { control_epoch: u64, installation_id: InstallationId, @@ -3103,6 +3127,57 @@ where write_frame_with_limit(writer, value, MAX_CONTROL_FRAME_BYTES).await } +pub fn bounded_json_bytes(value: &T, limit: usize) -> Result, FrameError> { + struct BoundedWriter { + bytes: Vec, + limit: usize, + overflow: bool, + } + impl std::io::Write for BoundedWriter { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + let Some(required) = self + .bytes + .len() + .checked_add(bytes.len()) + .filter(|n| *n <= self.limit) + else { + self.overflow = true; + return Err(std::io::Error::other("control frame exceeds limit")); + }; + if required > self.bytes.capacity() { + let capacity = self + .bytes + .capacity() + .saturating_mul(2) + .max(required) + .min(self.limit); + self.bytes + .try_reserve_exact(capacity - self.bytes.len()) + .map_err(std::io::Error::other)?; + } + self.bytes.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + let limit = limit.min(u32::MAX as usize); + let mut writer = BoundedWriter { + bytes: Vec::with_capacity(limit.min(4096)), + limit, + overflow: false, + }; + if let Err(error) = serde_json::to_writer(&mut writer, value) { + return Err(if writer.overflow { + FrameError::TooLarge + } else { + FrameError::Json(error) + }); + } + Ok(writer.bytes) +} + pub async fn write_frame_with_limit( writer: &mut W, value: &T, @@ -3112,10 +3187,7 @@ where W: AsyncWrite + Unpin, T: Serialize, { - let bytes = serde_json::to_vec(value)?; - if bytes.len() > limit { - return Err(FrameError::TooLarge); - } + let bytes = bounded_json_bytes(value, limit)?; writer .write_all(&(bytes.len() as u32).to_be_bytes()) .await?; @@ -3152,6 +3224,35 @@ where mod tests { use super::*; + #[test] + fn provider_preparation_serialization_stops_before_materializing_oversized_metadata() { + use std::sync::atomic::{AtomicUsize, Ordering}; + struct Many<'a>(&'a AtomicUsize); + impl Serialize for Many<'_> { + fn serialize(&self, serializer: S) -> Result { + use serde::ser::SerializeSeq; + let mut seq = serializer.serialize_seq(Some(100_000))?; + for _ in 0..100_000 { + self.0.fetch_add(1, Ordering::SeqCst); + seq.serialize_element("metadata")?; + } + seq.end() + } + } + let count = AtomicUsize::new(0); + assert!(matches!( + bounded_json_bytes(&Many(&count), 1024), + Err(FrameError::TooLarge) + )); + assert!( + count.load(Ordering::SeqCst) < 200, + "oversized metadata was fully materialized before its bound" + ); + let exact = serde_json::json!({"prepared":"metadata"}); + let bytes = serde_json::to_vec(&exact).unwrap(); + assert_eq!(bounded_json_bytes(&exact, bytes.len()).unwrap(), bytes); + } + #[test] fn durable_terminal_argument_recipe_preserves_provider_options_and_legacy_modes() { for (provider, args) in [ diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 6645edf5b..b435d79d0 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -157,10 +157,13 @@ pub(crate) async fn spawn_callback_server() -> Result<(), String> { let mut bytes = Vec::new(); let result = async { (&mut socket) - .take(8192) + .take((freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + 1) as u64) .read_to_end(&mut bytes) .await .map_err(|error| error.to_string())?; + if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err("managed provider callback exceeds metadata limit".into()); + } let expected = &SETTINGS .get() .ok_or("managed MCP controller is unconfigured")? @@ -230,6 +233,7 @@ struct CapabilityCallback { provider: Option, tui_source: Option, workspace: Option, + auth_lookup_authority: Option, } fn apply_callback( @@ -245,6 +249,9 @@ fn apply_callback( if !freshell_api::check_auth(Some(&callback.control_secret), expected_control_secret) { return Err("managed runtime callback unauthorized".into()); } + if callback.auth_lookup_authority.is_some() && callback.action != "issue" { + return Err("native auth lookup authority belongs only to provider preparation".into()); + } match callback.action.as_str() { "issue" => { let provider = callback.provider.ok_or("missing managed MCP provider")?; diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs new file mode 100644 index 000000000..6e4db60e5 --- /dev/null +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -0,0 +1,200 @@ +use crate::{backend::*, registry::*, service::*}; +use async_trait::async_trait; +use freshell_runtime_protocol::*; +use std::{ + collections::BTreeMap, + path::Path, + sync::{ + atomic::{AtomicUsize, Ordering}, + Arc, + }, +}; + +struct AuthBackend(AtomicUsize); +#[async_trait] +impl RuntimeBackend for AuthBackend { + async fn read_native_auth( + &self, + handle: &OwnedRuntimeHandle, + keys: &[String], + _: &Path, + ) -> Result, BackendError> { + assert_eq!( + handle.provider_volume_name(), + stable_provider_volume_name(handle.installation_id(), handle.soul_id()) + ); + assert_eq!(keys, &["PATH_INPUT"]); + self.0.fetch_add(1, Ordering::SeqCst); + Ok(BTreeMap::from([( + "PATH_INPUT".into(), + "/synthetic/selected".into(), + )])) + } + async fn create_stopped(&self, _: &CreateRuntimeSpec) -> Result { + unreachable!() + } + async fn start_host(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + async fn inspect(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + unreachable!() + } + async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + async fn verify_empty(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn list_known( + &self, + _: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + unreachable!() + } + async fn read_limits(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } +} + +#[tokio::test] +async fn provider_native_auth_nested_callback_uses_current_attempt_without_relocking_soul() { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("host"); + std::fs::write(&binary, "synthetic helper").unwrap(); + let registry = Registry::open(temp.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let limits = RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + }; + let launch = registry + .prepare_launch(LaunchPreparation { + soul_id: soul.clone(), + provider: "opencode".into(), + provider_store_id: "store".into(), + native_session_id: None, + creation_seed_ref: "explicit-test-seed".into(), + request_id: RequestId::new(), + payload_digest: "digest".into(), + requested_limits: limits, + profile: RuntimeProfile::Custom, + project_key: "workspace".into(), + fixture: None, + terminal: None, + fresh_agent: None, + view_intent: None, + admission: crate::admission::AdmissionPolicy::default(), + }) + .await + .unwrap(); + registry + .commit_created( + launch.incarnation_id, + BackendCreatedRecord { + daemon_id: DockerDaemonId::new(), + container_id: "owned-container".into(), + image_ref: "sha256:synthetic-image".into(), + runtime_dir: temp.path().into(), + host_binary_path: binary.clone(), + immutable_config_digest: "digest".into(), + }, + ) + .await + .unwrap(); + let start = registry + .begin_recovery( + soul.clone(), + RecoveryTrigger::ManualRetry, + RecoveryPath::NativeResume, + ) + .await + .unwrap(); + let backend = Arc::new(AuthBackend(AtomicUsize::new(0))); + let secret = "synthetic-control-secret"; + let supervisor = Supervisor::new( + registry, + backend.clone(), + SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: binary, + image_ref: "sha256:synthetic-image".into(), + test_run_id: "auth-nested".into(), + control_secret: secret.into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let lock = supervisor.lifecycle_lock(&soul).await; + let _held = lock.lock().await; + let scope = supervisor.auth_lookup_scope(&start); + let key = scope.key.clone(); + let request = |soul: SoulId, key: String| { + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::NativeAuthLookup(NativeAuthLookupRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul, + attempt_authority: key, + keys: vec!["PATH_INPUT".into()], + }), + ); + envelope.auth = Some(secret.into()); + envelope + }; + let reply = tokio::time::timeout( + std::time::Duration::from_millis(250), + supervisor.dispatch(request(soul.clone(), key.clone())), + ) + .await + .expect("nested callback reacquired held soul lock"); + match reply.result.unwrap() { + AdminResult::NativeAuthLookup(values) => { + assert_eq!(values.0["PATH_INPUT"], "/synthetic/selected") + } + _ => panic!("unexpected auth reply"), + } + assert_eq!(backend.0.load(Ordering::SeqCst), 1); + assert!(supervisor + .dispatch(request(SoulId::new(), key.clone())) + .await + .result + .is_err()); + assert!(supervisor + .dispatch(request(soul.clone(), "foreign-authority".into())) + .await + .result + .is_err()); + drop(scope); + assert!(supervisor + .dispatch(request(soul.clone(), key)) + .await + .result + .is_err()); + let scope = supervisor.auth_lookup_scope(&start); + supervisor + .registry + .begin_stop_expected(soul.clone(), None) + .await + .unwrap(); + assert!(supervisor + .dispatch(request(soul, scope.key.clone())) + .await + .result + .is_err()); + assert_eq!( + backend.0.load(Ordering::SeqCst), + 1, + "expired/foreign/stopped attempt must never inspect auth" + ); +} diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index f65b86977..fba9e153f 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -632,7 +632,7 @@ impl Supervisor { } }); let refreshed_capability = self - .issue_replacement_mcp_capability(&soul_id, &terminal.mode, tui_source, &terminal.workspace_path) + .issue_replacement_mcp_capability(&start, &terminal.mode, tui_source, &terminal.workspace_path) .await .and_then(|reference| { crate::backend::docker::staged_provider_root_for_grant( diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index a9588cbee..1c85be94f 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -1460,6 +1460,20 @@ impl Registry { .await } + pub(crate) async fn validate_auth_lookup_attempt( + &self, + start: RecoveryStart, + ) -> Result { + self.run_blocking(move |conn| { + let current:bool = conn.query_row( + "SELECT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1 AND desired_state='running' AND recovery_state='recovering' AND intent_revision=?2 AND recovery_attempt_id=?3)", + params![start.context.soul_id.as_str(),start.context.intent_revision,start.attempt_id.as_str()], + |row|row.get(0), + )?; + Ok(current) + }).await + } + pub async fn prepare_replacement( &self, start: RecoveryStart, diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 946548d1a..2e37a2a2f 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -60,6 +60,29 @@ struct McpCapabilityRequest<'a> { provider: Option<&'a str>, tui_source: Option<&'a freshell_runtime_protocol::ProviderConfigReference>, workspace: Option<&'a str>, + auth_lookup_authority: Option<&'a str>, +} + +#[derive(Clone)] +struct AuthLookupAuthority { + start: crate::registry::RecoveryStart, + expires: Instant, +} +struct AuthLookupScope { + authorities: Arc>>, + key: String, +} + +#[cfg(test)] +#[path = "native_auth_lookup_tests.rs"] +mod native_auth_lookup_tests; +impl Drop for AuthLookupScope { + fn drop(&mut self) { + self.authorities + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .remove(&self.key); + } } #[derive(Clone)] @@ -71,6 +94,7 @@ pub struct Supervisor { /// remain independently recoverable. Stop intentionally does not take this /// lock: its durable intent revision must be able to preempt recovery. pub(crate) recovery_locks: Arc>>>>, + auth_lookup_authorities: Arc>>, } impl Supervisor { @@ -117,6 +141,7 @@ impl Supervisor { backend, config: Arc::new(config), recovery_locks: Arc::new(Mutex::new(HashMap::new())), + auth_lookup_authorities: Arc::new(StdMutex::new(HashMap::new())), }) } @@ -162,6 +187,14 @@ impl Supervisor { )); } match envelope.body { + AdminCommand::NativeAuthLookup(request) => { + self.registry + .assert_epoch(request.expected_control_epoch) + .map_err(map_registry)?; + self.lookup_native_auth(request) + .await + .map(AdminResult::NativeAuthLookup) + } AdminCommand::Health => Ok(AdminResult::Health { control_epoch: self.registry.control_epoch(), installation_id: self.registry.installation_id().clone(), @@ -523,6 +556,98 @@ impl Supervisor { } } + fn auth_lookup_scope(&self, start: &crate::registry::RecoveryStart) -> AuthLookupScope { + let key = uuid::Uuid::new_v4().to_string(); + let mut authorities = self + .auth_lookup_authorities + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + authorities.retain(|_, authority| authority.expires > Instant::now()); + authorities.insert( + key.clone(), + AuthLookupAuthority { + start: start.clone(), + expires: Instant::now() + Duration::from_secs(30), + }, + ); + AuthLookupScope { + authorities: self.auth_lookup_authorities.clone(), + key, + } + } + + async fn lookup_native_auth( + &self, + request: freshell_runtime_protocol::NativeAuthLookupRequest, + ) -> Result { + let denied = || { + RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "native auth lookup requires the current provider preparation attempt", + ) + }; + freshell_platform::native_auth_exchange::validate_keys(&request.keys) + .map_err(|_| denied())?; + if request.attempt_authority.len() != 36 { + return Err(denied()); + } + let authority = { + let authorities = self + .auth_lookup_authorities + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + authorities + .get(&request.attempt_authority) + .filter(|authority| authority.expires > Instant::now()) + .cloned() + .ok_or_else(denied)? + }; + if authority.start.context.soul_id != request.soul_id + || authority.start.context.provider != "opencode" + || !self + .registry + .validate_auth_lookup_attempt(authority.start.clone()) + .await + .map_err(map_registry)? + { + return Err(denied()); + } + // The recovery owner already holds the soul lock. This operation uses + // its short-lived authority and drops the map/registry locks before IPC. + let values = tokio::time::timeout( + Duration::from_secs(25), + self.backend.read_native_auth( + &authority.start.context.prior_handle, + &request.keys, + &self.config.host_binary_path, + ), + ) + .await + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "native auth lookup timed out", + ) + })? + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "native auth owned lookup failed", + ) + })?; + freshell_platform::native_auth_exchange::validate_selected(&request.keys, &values) + .map_err(|_| denied())?; + if !self + .registry + .validate_auth_lookup_attempt(authority.start) + .await + .map_err(map_registry)? + { + return Err(denied()); + } + Ok(freshell_runtime_protocol::NativeAuthValues(values)) + } + async fn launch( &self, request_id: RequestId, @@ -1708,6 +1833,7 @@ impl Supervisor { provider: None, tui_source: None, workspace: None, + auth_lookup_authority: None, }) .await?; if response != b"ok" { @@ -1749,20 +1875,22 @@ impl Supervisor { pub(crate) async fn issue_replacement_mcp_capability( &self, - soul_id: &SoulId, + start: &crate::registry::RecoveryStart, provider: &str, tui_source: Option<&freshell_runtime_protocol::ProviderConfigReference>, workspace: &str, ) -> Result { + let scope = self.auth_lookup_scope(start); let response = self .mcp_capability_callback(McpCapabilityRequest { action: "issue", - soul_id, + soul_id: &start.context.soul_id, incarnation_id: None, grant_id: None, provider: Some(provider), tui_source, workspace: Some(workspace), + auth_lookup_authority: Some(&scope.key), }) .await?; serde_json::from_slice(&response).map_err(|_| { @@ -1783,16 +1911,20 @@ impl Supervisor { "invalid managed runtime control secret", )); } - let body = serde_json::to_vec(&serde_json::json!({ - "action": request.action, - "controlSecret": self.config.control_secret, - "soulId": request.soul_id, - "incarnationId": request.incarnation_id, - "grantId": request.grant_id, - "provider": request.provider, - "tuiSource": request.tui_source, - "workspace": request.workspace, - })) + let body = freshell_runtime_protocol::bounded_json_bytes( + &serde_json::json!({ + "action": request.action, + "controlSecret": self.config.control_secret, + "soulId": request.soul_id, + "incarnationId": request.incarnation_id, + "grantId": request.grant_id, + "provider": request.provider, + "tuiSource": request.tui_source, + "workspace": request.workspace, + "authLookupAuthority": request.auth_lookup_authority, + }), + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) .map_err(|error| RuntimeError::new(RuntimeErrorCode::InvalidRequest, error.to_string()))?; let callback_socket = self.config.control_socket_path.with_file_name("mcp.sock"); let exchange = async { @@ -1800,10 +1932,18 @@ impl Supervisor { socket.write_all(&body).await?; socket.shutdown().await?; let mut response = Vec::new(); - socket.read_to_end(&mut response).await?; + (&mut socket) + .take((freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + 1) as u64) + .read_to_end(&mut response) + .await?; + if response.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err(std::io::Error::other( + "provider preparation reply exceeds limit", + )); + } Ok::<_, std::io::Error>(response) }; - let response = tokio::time::timeout(Duration::from_secs(5), exchange) + let response = tokio::time::timeout(Duration::from_secs(35), exchange) .await .map_err(|_| { RuntimeError::new( From c81841e9d84ac1ff3421a67e7e7f6c3871efc12a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 03:18:16 -0700 Subject: [PATCH 020/493] Retain native auth until a matching typed grant replaces it --- crates/freshell-session-host/src/main.rs | 134 ++++++++++++++++------- 1 file changed, 92 insertions(+), 42 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index e856068b8..b2f834801 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1950,34 +1950,33 @@ fn prepare_provider_files( run_as_uid: u32, run_as_gid: u32, ) -> Result<(), String> { - let auth_dir = Path::new("/tmp/freshell-provider-auth"); - std::fs::create_dir_all(auth_dir).map_err(|error| error.to_string())?; - set_mode(auth_dir, 0o711)?; - let home = Path::new("/home/freshell/provider"); - if !home.exists() && provider_files.is_empty() && configuration_files.is_empty() { + prepare_provider_files_at( + Path::new("/home/freshell/provider"), + Path::new("/tmp/freshell-provider-auth"), + provider, + provider_files, + configuration_files, + run_as_uid, + run_as_gid, + ) +} + +fn prepare_provider_files_at( + home: &Path, + auth_dir: &Path, + _provider: &str, + provider_files: &[(&str, Vec)], + configuration_files: &[(String, PathBuf)], + run_as_uid: u32, + run_as_gid: u32, +) -> Result<(), String> { + if provider_files.is_empty() && configuration_files.is_empty() { return Ok(()); } - let legacy_relative = match provider { - "claude" => Some(".claude/.credentials.json"), - "codex" => Some(".codex/auth.json"), - "opencode" => Some(".local/share/opencode/auth.json"), - _ => None, - }; - if let Some(relative) = legacy_relative { - let destination = home.join(relative); - // A pre-OneCLI soul can contain a raw bootstrap copy. The provider - // uid owns the durable HOME and removes that copy before any child runs. - run_provider_path_command( - run_as_uid, - run_as_gid, - "/usr/bin/rm", - &[ - "-f", - "--", - destination.to_str().ok_or("auth path is not UTF-8")?, - ], - )?; - } + std::fs::create_dir_all(auth_dir).map_err(|error| error.to_string())?; + set_mode(auth_dir, 0o711)?; + // Retain native auth until a matching typed grant has been written and + // owned. The final destination replacement occurs only for that grant. for (index, (relative, contents)) in provider_files.iter().enumerate() { let destination = home.join(relative); let target = auth_dir.join(format!("provider-{index}")); @@ -2101,16 +2100,8 @@ fn run_provider_path_command( args: &[&str], ) -> Result<(), String> { let status = std::process::Command::new("/usr/bin/setpriv") - .args([ - "--reuid", - &uid.to_string(), - "--regid", - &gid.to_string(), - "--clear-groups", - "--no-new-privs", - "--", - program, - ]) + .args(provider_identity_args(uid, gid)) + .arg(program) .args(args) .status() .map_err(|error| format!("provider-owned auth path operation failed to start: {error}"))?; @@ -2803,6 +2794,56 @@ fn now_millis() -> i64 { #[cfg(test)] mod tests { + #[test] + fn provider_native_auth_survives_without_matching_typed_replacement() { + use super::*; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let auth = home.join(".local/share/opencode/auth.json"); + std::fs::create_dir_all(auth.parent().unwrap()).unwrap(); + let native = r#"{"native":{"type":"api","key":"durable-nonce"}}"#; + std::fs::write(&auth, native).unwrap(); + let uid = unsafe { libc::geteuid() }; + let gid = unsafe { libc::getegid() }; + prepare_provider_files_at( + &home, + &root.path().join("empty-auth"), + "opencode", + &[], + &[], + uid, + gid, + ) + .unwrap(); + assert_eq!(std::fs::read_to_string(&auth).unwrap(), native); + prepare_provider_files_at( + &home, + &root.path().join("ca-only"), + "opencode", + &[(".freshell/opencode/ca.pem", b"synthetic-ca".to_vec())], + &[], + uid, + gid, + ) + .unwrap(); + assert_eq!(std::fs::read_to_string(&auth).unwrap(), native); + let replacement = b"{\"typed\":true}"; + prepare_provider_files_at( + &home, + &root.path().join("typed-auth"), + "opencode", + &[(".local/share/opencode/auth.json", replacement.to_vec())], + &[], + uid, + gid, + ) + .unwrap(); + assert_eq!(std::fs::read(&auth).unwrap(), replacement); + assert!(std::fs::symlink_metadata(&auth) + .unwrap() + .file_type() + .is_symlink()); + } use super::*; #[cfg(unix)] @@ -3802,7 +3843,13 @@ mod tests { }; use std::os::unix::fs::PermissionsExt; - let workspace = tempfile::tempdir().unwrap(); + // runtime-suite /tmp is deliberately non-executable. Its existing + // cargo target volume is the owned executable-fixture filesystem. + let workspace = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(directory) => tempfile::tempdir_in(directory), + None => tempfile::tempdir(), + } + .unwrap(); std::fs::set_permissions(workspace.path(), std::fs::Permissions::from_mode(0o755)).unwrap(); let provider_home = workspace.path().join("provider-home"); let source_config = workspace.path().join("opencode.json"); @@ -3849,7 +3896,7 @@ mod tests { let launcher = workspace.path().join("exact-resume-provider"); std::fs::write( &launcher, - "#!/bin/sh\ncat \"$TEST_CONFIG_PATH\" > \"$TEST_OUTPUT_PATH\"\nexec sleep 60\n", + "#!/bin/sh\nset -e\ncat \"$TEST_CONFIG_PATH\" > \"$TEST_OUTPUT_PATH.tmp\"\nmv \"$TEST_OUTPUT_PATH.tmp\" \"$TEST_OUTPUT_PATH\"\nexec sleep 60\n", ) .unwrap(); std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)).unwrap(); @@ -3971,14 +4018,17 @@ mod tests { } tokio::time::sleep(std::time::Duration::from_millis(20)).await; } - assert_eq!( - observed.as_deref(), - Some(br#"{"model":"second"}"#.as_slice()) - ); let mut launched_pty = state.pty.lock().await.take(); + let mut diagnostics = None; if let Some(mut pty) = launched_pty.take() { + diagnostics = Some(pty.read_output(state.incarnation_id.clone(), 0, 64 * 1024, None)); pty.stop().await; } + assert_eq!( + observed.as_deref(), + Some(br#"{"model":"second"}"#.as_slice()), + "synthetic provider fixture output: {diagnostics:?}", + ); } #[test] From 85363419ee8dfbacf5b9f0845cf1ed35485bf39c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 04:33:27 -0700 Subject: [PATCH 021/493] fix: validate prepared provider input mounts and immutable metadata --- crates/freshell-runtime-protocol/src/lib.rs | 11 + .../src/provider_inputs.rs | 353 ++++++++++++++++++ .../src/managed_provider_bootstrap.rs | 1 + crates/freshell-session-host/src/main.rs | 6 + .../src/providers/fresh_agent.rs | 2 + .../src/providers/mod.rs | 1 + crates/freshell-supervisor/src/backend.rs | 145 ++++++- .../freshell-supervisor/src/backend/docker.rs | 294 +++++++++++++++ .../src/backend/prepared_inputs.rs | 230 ++++++++++++ crates/freshell-supervisor/src/recovery.rs | 2 + crates/freshell-supervisor/src/registry.rs | 2 + 11 files changed, 1046 insertions(+), 1 deletion(-) create mode 100644 crates/freshell-runtime-protocol/src/provider_inputs.rs create mode 100644 crates/freshell-supervisor/src/backend/prepared_inputs.rs diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 6add6069c..4186b6819 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -18,6 +18,8 @@ type HmacSha256 = Hmac; pub mod fresh_agent_ops; pub use fresh_agent_ops::*; +pub mod provider_inputs; +pub use provider_inputs::*; #[cfg(test)] #[path = "fresh_agent_ops_tests.rs"] @@ -1002,6 +1004,8 @@ pub struct ProviderSecretReference { #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct ProviderLaunchContext { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub prepared_inputs: Option, pub preparation: ProviderPreparation, #[serde(default, skip_serializing_if = "Option::is_none")] pub mcp_capability: Option, @@ -1074,6 +1078,9 @@ pub struct ProviderConfigReference { impl ProviderLaunchContext { pub fn validate(&self, provider: &str) -> Result<(), RuntimeError> { + if let Some(inputs) = &self.prepared_inputs { + inputs.validate(self.mcp_capability.as_ref())?; + } let invalid = || { RuntimeError::new( RuntimeErrorCode::InvalidRequest, @@ -1321,6 +1328,7 @@ mod provider_launch_context_tests { host_gateway_address: None, }; let mut context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Opencode { project_config: Vec::new(), tui_config: Some(reference), @@ -1359,6 +1367,7 @@ mod provider_launch_context_tests { #[test] fn typed_provider_context_round_trips_without_secret_material() { let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Claude { mcp_args: vec!["--mcp-config".into(), ".claude/freshell-mcp.json".into()], }, @@ -1436,6 +1445,7 @@ mod provider_launch_context_tests { assert!(serde_json::from_value::(nested).is_err()); let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Claude { mcp_args: vec![] }, mcp_capability: None, config: vec![ProviderConfigReference { @@ -1475,6 +1485,7 @@ mod provider_launch_context_tests { })) .unwrap(); launch.provider_launch_context = Some(ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Claude { mcp_args: vec!["--api-key".into(), "fixture-secret-bytes".into()], }, diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs new file mode 100644 index 000000000..597a029a1 --- /dev/null +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -0,0 +1,353 @@ +//! Prepared provider input metadata. Payloads and unexpanded source documents +//! remain on private stage filesystems; the control plane carries descriptors. +use crate::{McpCapabilityReference, RuntimeError, RuntimeErrorCode, SoulId}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct PreparedProviderInputs { + pub generation_id: String, + pub manifest: ProviderInputManifestReference, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub config_binds: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub resource_mounts: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub indexed_files: Option, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ProviderInputManifestReference { + pub relative_path: String, + pub sha256: String, + pub source_count: u16, + pub alias_count: u16, + pub recipe_count: u16, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ProviderConfigBind { + pub source_relative_path: String, + pub logical_destination: String, + pub effective_destination: String, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ProviderResourceMount { + pub stage_relative_path: String, + pub provider_relative_path: String, + pub logical_destination: String, + pub effective_destination: String, + pub kind: ProviderResourceKind, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum ProviderResourceKind { + File, + Directory, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct IndexedProviderFiles { + pub volume_name: String, + #[serde(default)] + pub aliases: Vec, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct IndexedProviderFileAlias { + pub source_id: u16, + pub logical_destination: String, + pub effective_destination: String, +} + +pub fn indexed_provider_volume_name(soul: &SoulId, generation: &str) -> String { + let digest = Sha256::digest(format!("{}\0{generation}", soul.as_str()).as_bytes()); + format!("freshell-inputs-{}", &format!("{digest:x}")[..24]) +} + +impl PreparedProviderInputs { + pub fn validate( + &self, + capability: Option<&McpCapabilityReference>, + ) -> Result<(), RuntimeError> { + use std::collections::BTreeMap; + let invalid = |field: &str| { + RuntimeError::new(RuntimeErrorCode::InvalidRequest,format!("prepared provider inputs {field}: invalid, conflicting or over-bound metadata; reprepare before launch")) + }; + crate::bounded_json_bytes(self, crate::MAX_CONTROL_FRAME_BYTES) + .map_err(|_| invalid("control byte limit"))?; + if capability.map(|value| value.grant_id.as_str()) != Some(self.generation_id.as_str()) + || !self.generation_id.starts_with("grant-") + || !identifier(&self.generation_id) + { + return Err(invalid("generation")); + } + if self.manifest.relative_path != ".freshell/input-manifest.json" + || self.manifest.sha256.len() != 64 + || !self.manifest.sha256.bytes().all(|b| b.is_ascii_hexdigit()) + || self.manifest.source_count > 128 + || self.manifest.alias_count > 256 + || self.manifest.recipe_count > 128 + { + return Err(invalid("manifest")); + } + if self + .config_binds + .len() + .checked_add(self.resource_mounts.len()) + .is_none_or(|n| n > 100_000) + { + return Err(invalid("resource entries")); + } + let mut destinations: BTreeMap = BTreeMap::new(); + let mut logical = BTreeMap::::new(); + let mut add = |original: &str, + effective: &str, + identity: String, + kind: ProviderResourceKind| + -> Result<(), RuntimeError> { + if !mount_destination(original) || !mount_destination(effective) { + return Err(invalid("mount destination")); + } + if logical + .insert(original.to_owned(), identity.clone()) + .is_some_and(|prior| prior != identity) + { + return Err(invalid("logical destination")); + } + if destinations + .insert(effective.to_owned(), (identity.clone(), kind)) + .is_some_and(|prior| prior != (identity, kind)) + { + return Err(invalid("effective destination")); + } + Ok(()) + }; + for bind in &self.config_binds { + if !unit_path(&bind.source_relative_path, ".freshell/templates", false) { + return Err(invalid("configuration source")); + } + add( + &bind.logical_destination, + &bind.effective_destination, + format!("configuration:{}", bind.source_relative_path), + ProviderResourceKind::File, + )?; + } + for mount in &self.resource_mounts { + let file = mount.kind == ProviderResourceKind::File; + if !unit_path(&mount.stage_relative_path, ".freshell/resource-units", file) + || !unit_path( + &mount.provider_relative_path, + ".freshell/provider-roots", + file, + ) + || mount + .stage_relative_path + .strip_prefix(".freshell/resource-units/") + != mount + .provider_relative_path + .strip_prefix(".freshell/provider-roots/") + { + return Err(invalid("resource source")); + } + add( + &mount.logical_destination, + &mount.effective_destination, + format!("resource:{}", mount.provider_relative_path), + mount.kind, + )?; + } + if let Some(files) = &self.indexed_files { + if !files + .volume_name + .strip_prefix("freshell-inputs-") + .is_some_and(|suffix| { + suffix.len() == 24 && suffix.bytes().all(|b| b.is_ascii_hexdigit()) + }) + || files.aliases.len() > 256 + || files.aliases.len() != self.manifest.alias_count as usize + { + return Err(invalid("indexed file publication")); + } + for alias in &files.aliases { + if alias.source_id >= self.manifest.source_count { + return Err(invalid("indexed source ID")); + } + add( + &alias.logical_destination, + &alias.effective_destination, + format!("indexed:{}", alias.source_id), + ProviderResourceKind::File, + )?; + } + } else if self.manifest.source_count != 0 || self.manifest.alias_count != 0 { + return Err(invalid("indexed file publication")); + } + for (path, (_, kind)) in &destinations { + let mut cursor = path.as_str(); + while let Some((parent, _)) = cursor.rsplit_once('/') { + if parent.is_empty() { + break; + } + if destinations.get(parent).is_some_and(|(_, parent_kind)| { + *parent_kind == ProviderResourceKind::File + || *kind == ProviderResourceKind::Directory + }) { + return Err(invalid("ancestor destination")); + } + cursor = parent; + } + } + Ok(()) + } +} + +fn identifier(value: &str) -> bool { + !value.is_empty() + && value.len() <= 128 + && value + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'-' | b'_')) +} +fn unit_path(value: &str, prefix: &str, file: bool) -> bool { + let Some(tail) = value + .strip_prefix(prefix) + .and_then(|tail| tail.strip_prefix('/')) + else { + return false; + }; + let (id, leaf) = tail + .split_once('/') + .map_or((tail, None), |(id, leaf)| (id, Some(leaf))); + identifier(id) && leaf == if file { Some("source") } else { None } +} +fn mount_destination(value: &str) -> bool { + if !value.starts_with('/') + || value.len() > 64 * 1024 + || value.chars().any(char::is_control) + || value[1..] + .split('/') + .any(|part| part.is_empty() || matches!(part, "." | "..")) + { + return false; + } + for reserved in [ + "/runtime", + "/run/freshell", + "/run/freshell-private", + "/run/freshell-secrets", + "/run/freshell-bootstrap", + "/home/freshell/provider/.freshell/provider-roots", + "/home/freshell/provider/.freshell/mcp-capability.json", + ] { + if value == reserved + || value.starts_with(&format!("{reserved}/")) + || reserved.starts_with(&format!("{value}/")) + { + return false; + } + } + value != "/home/freshell/provider" +} + +#[cfg(test)] +mod tests { + use super::*; + fn prepared() -> PreparedProviderInputs { + PreparedProviderInputs { + generation_id: "grant-synthetic".into(), + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 2, + alias_count: 2, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: Some(IndexedProviderFiles { + volume_name: indexed_provider_volume_name(&SoulId::new(), "grant-synthetic"), + aliases: vec![ + IndexedProviderFileAlias { + source_id: 0, + logical_destination: "/fixture/rules".into(), + effective_destination: "/fixture/rules".into(), + }, + IndexedProviderFileAlias { + source_id: 1, + logical_destination: "/fixture/other".into(), + effective_destination: "/fixture/other".into(), + }, + ], + }), + } + } + fn capability() -> McpCapabilityReference { + McpCapabilityReference { + grant_id: "grant-synthetic".into(), + endpoint: "http://host.docker.internal:3344".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + } + } + #[test] + fn provider_prepared_mounts_reject_different_sources_over_one_effective_inode() { + let mut value = prepared(); + value.indexed_files.as_mut().unwrap().aliases[1].effective_destination = + "/fixture/rules".into(); + assert!( + value.validate(Some(&capability())).is_err(), + "different approved sources must never win by mount order" + ); + } + #[test] + fn provider_prepared_mounts_preserve_home_and_control_parents() { + for destination in [ + "/home/freshell/provider", + "/run/freshell/secret", + "/runtime/freshell-session-host", + ] { + let mut value = prepared(); + value.resource_mounts.push(ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/unit-0".into(), + provider_relative_path: ".freshell/provider-roots/unit-0".into(), + logical_destination: destination.into(), + effective_destination: destination.into(), + kind: ProviderResourceKind::Directory, + }); + assert!( + value.validate(Some(&capability())).is_err(), + "reserved destination accepted" + ); + } + } + #[test] + fn provider_prepared_manifest_belongs_to_the_same_generation() { + let mut value = prepared(); + value.generation_id = "grant-other".into(); + assert!( + value.validate(Some(&capability())).is_err(), + "stale generation must fail before mount construction" + ); + } + #[test] + fn provider_prepared_mounts_allow_one_source_through_multiple_original_aliases() { + let mut value = prepared(); + value.indexed_files.as_mut().unwrap().aliases[1].source_id = 0; + value.indexed_files.as_mut().unwrap().aliases[1].effective_destination = + "/fixture/rules".into(); + value.validate(Some(&capability())).unwrap(); + let bytes = crate::bounded_json_bytes(&value, crate::MAX_CONTROL_FRAME_BYTES).unwrap(); + let restored: PreparedProviderInputs = serde_json::from_slice(&bytes).unwrap(); + assert_eq!(restored, value); + } +} diff --git a/crates/freshell-server/src/managed_provider_bootstrap.rs b/crates/freshell-server/src/managed_provider_bootstrap.rs index 4d2819299..54c8bb944 100644 --- a/crates/freshell-server/src/managed_provider_bootstrap.rs +++ b/crates/freshell-server/src/managed_provider_bootstrap.rs @@ -292,6 +292,7 @@ fn provider_launch_context_from_home_at_with_xdg( ); } Ok(Some(ProviderLaunchContext { + prepared_inputs: None, preparation, mcp_capability, config, diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index b2f834801..90a757045 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -2882,6 +2882,7 @@ mod tests { std::fs::write(source.join("plugins/ordinary.js"), b"export default {};").unwrap(); std::fs::write(source.join("plugins/auth.json"), b"test-secret-value").unwrap(); let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Claude { mcp_args: Vec::new(), }, @@ -2965,6 +2966,7 @@ mod tests { symlink(&outside, workspace.join(".opencode")).unwrap(); symlink(workspace.join("inside"), workspace.join("linked-inside")).unwrap(); let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Opencode { project_config: vec![ ProviderConfigReference { @@ -3075,6 +3077,7 @@ mod tests { std::fs::write(source.join("settings.json"), b"first").unwrap(); std::fs::write(source.join("plugins/old.js"), b"old").unwrap(); let mut context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Claude { mcp_args: Vec::new(), }, @@ -3219,6 +3222,7 @@ mod tests { ) .unwrap(); let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Codex { tui_args: vec![], sidecar_args: vec![], @@ -3857,6 +3861,7 @@ mod tests { std::fs::create_dir(&provider_home).unwrap(); std::fs::write(&source_config, br#"{"model":"first"}"#).unwrap(); let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Opencode { project_config: vec![ProviderConfigReference { root: ProviderConfigRoot::Workspace, @@ -4489,6 +4494,7 @@ mod tests { .unwrap(); let mut terminal:TerminalLaunchSpec=serde_json::from_value(serde_json::json!({"terminalId":"first-amplifier","streamId":"stream","mode":"amplifier","program":"amplifier","args":[],"cwd":root.path(),"runAsUid":unsafe{libc::geteuid()},"runAsGid":unsafe{libc::getegid()},"cols":80,"rows":24,"projectKey":"project","workspacePath":root.path(),"resumeSessionId":"session"})).unwrap(); terminal.provider_launch_context = Some(ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Amplifier { bundle: "default".into(), resume_args: Vec::new(), diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index 51ac09f0a..19a173a9a 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -2079,6 +2079,7 @@ mod tests { session_id: format!("native-{}", provider.as_str()), }), provider_launch_context: Some(ProviderLaunchContext { + prepared_inputs: None, preparation, mcp_capability: None, config: vec![freshell_runtime_protocol::ProviderConfigReference { @@ -2677,6 +2678,7 @@ mod tests { provider: provider.as_str().into(), session_id: "exact-native".into(), }), provider_launch_context: Some(ProviderLaunchContext { + prepared_inputs: None, preparation, mcp_capability: None, config: vec![freshell_runtime_protocol::ProviderConfigReference { diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index a01dc248f..db00e29c8 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -582,6 +582,7 @@ mod provider_secret_context_tests { })) .unwrap(); terminal.provider_launch_context = Some(ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Claude { mcp_args: vec!["--mcp-config".into(), ".claude/freshell-mcp.json".into()], }, diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index bb2ceedaa..c92063209 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -1,6 +1,7 @@ pub mod docker; mod native_auth; mod native_history; +mod prepared_inputs; use crate::registry::OwnedRuntimeHandle; use async_trait::async_trait; @@ -263,6 +264,45 @@ impl DockerEngineBackend { )); } verify_inspect_config(handle, &value)?; + let mounts = match (handle.terminal(), handle.fresh_agent()) { + (Some(terminal), None) => Some(docker::terminal_mounts(terminal)), + (None, Some(agent)) => Some(docker::fresh_agent_mounts(agent)), + _ => None, + } + .transpose() + .map_err(BackendError::OwnershipMismatch)?; + if let Some(mounts) = mounts { + let mut volumes = BTreeMap::new(); + let mut names = std::collections::BTreeSet::new(); + if !mounts.resource_mounts.is_empty() { + names.insert(handle.provider_volume_name().to_owned()); + } + if let Some(files) = &mounts.indexed_files { + names.insert(files.volume_name.clone()); + } + for name in names { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{name}"), + None, + 1024 * 1024, + ) + .await?; + if inspected.status != 200 { + return Err(BackendError::OwnershipMismatch( + "prepared owned volume is unavailable".into(), + )); + } + let inspected = serde_json::from_slice(&inspected.body).map_err(|_| { + BackendError::OwnershipMismatch( + "prepared owned volume inspection invalid".into(), + ) + })?; + volumes.insert(name, inspected); + } + prepared_inputs::verify(&value, &mounts, handle.provider_volume_name(), &volumes)?; + } Ok(Some(value)) } @@ -662,7 +702,7 @@ impl RuntimeBackend for DockerEngineBackend { } else { Vec::new() }; - let body = docker_create_body( + let mut body = docker_create_body( spec, binds, host_env, @@ -671,6 +711,36 @@ impl RuntimeBackend for DockerEngineBackend { requested_limits, memory_swap, ); + if let Some(mounts) = &terminal_mounts { + let context = spec + .terminal + .as_ref() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + spec.fresh_agent + .as_ref() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }); + if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { + if inputs.indexed_files.as_ref().is_some_and(|files| { + files.volume_name + != freshell_runtime_protocol::indexed_provider_volume_name( + &spec.soul_id, + &inputs.generation_id, + ) + }) { + return Err(BackendError::InvalidConfig( + "indexed provider volume belongs to another soul/preparation".into(), + )); + } + } + body["HostConfig"]["Mounts"] = serde_json::to_value( + prepared_inputs::mount_descriptors(mounts, &spec.provider_volume_name)?, + ) + .map_err(|_| { + BackendError::InvalidConfig("prepared mount metadata cannot be encoded".into()) + })?; + } // The name is diagnostic only and is deliberately unique per create attempt. // A crash after Docker create but before the container-id commit can leave a // stopped, unregistered object. Retrying the PREPARED transaction must be able @@ -1856,4 +1926,77 @@ mod tests { running.provider_store_id = "store-two".into(); assert_ne!(digest(running), original); } + + #[test] + fn provider_prepared_metadata_changes_both_route_immutable_digests() { + use freshell_runtime_protocol::{ + PreparedProviderInputs, ProviderInputManifestReference, ProviderLaunchContext, + ProviderPreparation, + }; + let inputs = PreparedProviderInputs { + generation_id: "grant-digest".into(), + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }; + let context = ProviderLaunchContext { + prepared_inputs: Some(inputs), + preparation: ProviderPreparation::Codex { + tui_args: vec![], + sidecar_args: vec![], + }, + mcp_capability: None, + config: vec![], + }; + let terminal:TerminalLaunchSpec=serde_json::from_value(json!({ + "terminalId":"terminal-digest","streamId":"stream-digest","mode":"codex","program":"codex", + "cwd":"/workspace","workspacePath":"/workspace","runAsUid":65534,"runAsGid":0, + "cols":80,"rows":24,"projectKey":"project-digest","providerLaunchContext":context, + })).unwrap(); + let fresh:FreshAgentLaunchSpec=serde_json::from_value(json!({ + "sessionId":"session-digest","provider":"codex","sessionType":"freshcodex","runtimeVariant":"codex-app-server", + "providerStoreId":"store-digest","cwd":"/workspace","workspacePath":"/workspace","runAsUid":65534,"runAsGid":0, + "providerLaunchContext":context, + })).unwrap(); + for fresh_route in [false, true] { + let mut expected = ExpectedConfig { + image_ref: "sha256:image".into(), + installation_id: "installation-digest".into(), + soul_id: "soul-digest".into(), + incarnation_id: "incarnation-digest".into(), + host_binary_path: "/runtime/host".into(), + runtime_dir: "/runtime/incarnation".into(), + limits: RuntimeLimits { + cpu_milli: 500, + memory_bytes: 128 * 1024 * 1024, + swap_bytes: 0, + pids_max: 64, + }, + terminal: (!fresh_route).then_some(terminal.clone()), + fresh_agent: fresh_route.then(|| (&fresh).into()), + provider_volume_name: "owned-provider".into(), + }; + let before = digest_expected(&expected).unwrap(); + let context = if let Some(terminal) = expected.terminal.as_mut() { + terminal.provider_launch_context.as_mut().unwrap() + } else { + expected + .fresh_agent + .as_mut() + .unwrap() + .provider_launch_context + .as_mut() + .unwrap() + }; + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = "b".repeat(64); + assert_ne!(digest_expected(&expected).unwrap(),before,"prepared metadata must participate in the immutable proof for route fresh={fresh_route}"); + } + } } diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index f56030668..42ea5fb04 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -39,6 +39,16 @@ pub struct TerminalMounts { pub provider_secret_files: Vec, pub provider_user_root: Option, pub mcp_capability_file: Option, + pub input_manifest: Option, + pub config_binds: Vec, + pub resource_mounts: Vec, + pub indexed_files: Option, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ResolvedProviderConfigBind { + pub source: PathBuf, + pub reference: freshell_runtime_protocol::ProviderConfigBind, } pub fn terminal_mounts(spec: &TerminalLaunchSpec) -> Result { @@ -74,6 +84,9 @@ fn workload_mounts( context: Option<&ProviderLaunchContext>, provider: &str, ) -> Result { + if let Some(context) = context { + context.validate(provider).map_err(|error| error.message)?; + } freshell_runtime_protocol::validate_provider_secrets(provider, secret_references) .map_err(|error| error.message)?; let workspace = canonical_dir(Path::new(workspace_path), "workspace")?; @@ -190,6 +203,11 @@ fn workload_mounts( Ok(canonical) }) .transpose()?; + let prepared = prepared_input_mounts( + &workspace, + provider_user_root.as_deref(), + context.and_then(|context| context.prepared_inputs.as_ref()), + )?; Ok(TerminalMounts { workspace, git_common_dir, @@ -197,9 +215,284 @@ fn workload_mounts( provider_secret_files, provider_user_root, mcp_capability_file, + input_manifest: prepared.input_manifest, + config_binds: prepared.config_binds, + resource_mounts: prepared.resource_mounts, + indexed_files: prepared.indexed_files, }) } +#[derive(Default)] +struct PreparedInputMounts { + input_manifest: Option, + config_binds: Vec, + resource_mounts: Vec, + indexed_files: Option, +} + +fn prepared_input_mounts( + workspace: &Path, + root: Option<&Path>, + inputs: Option<&freshell_runtime_protocol::PreparedProviderInputs>, +) -> Result { + let mut prepared = PreparedInputMounts::default(); + if let Some(inputs) = inputs { + let root = root.ok_or("prepared provider stage is unavailable")?; + let manifest = canonical_stage_file(root, &inputs.manifest.relative_path)?; + verify_input_manifest_digest(&manifest, &inputs.manifest.sha256)?; + for reference in &inputs.config_binds { + let source = canonical_stage_file(root, &reference.source_relative_path)?; + verify_template_limit(&source)?; + verify_workspace_destination( + &workspace, + &reference.logical_destination, + &reference.effective_destination, + false, + )?; + prepared.config_binds.push(ResolvedProviderConfigBind { + source, + reference: reference.clone(), + }); + } + for reference in &inputs.resource_mounts { + let source = root.join(&reference.stage_relative_path); + let canonical = std::fs::canonicalize(&source) + .map_err(|_| "selected prepared resource is unavailable")?; + let metadata = std::fs::symlink_metadata(&source) + .map_err(|_| "selected prepared resource metadata failed")?; + let directory = + reference.kind == freshell_runtime_protocol::ProviderResourceKind::Directory; + if canonical != source + || metadata.file_type().is_symlink() + || if directory { + !metadata.is_dir() + } else { + !metadata.is_file() + } + { + return Err("prepared resource is not a physical staged unit".into()); + } + verify_workspace_destination( + &workspace, + &reference.logical_destination, + &reference.effective_destination, + directory, + )?; + prepared.resource_mounts.push(reference.clone()); + } + if let Some(files) = &inputs.indexed_files { + for alias in &files.aliases { + verify_workspace_destination( + &workspace, + &alias.logical_destination, + &alias.effective_destination, + false, + )?; + } + prepared.indexed_files = Some(files.clone()); + } + prepared.input_manifest = Some(manifest); + } + Ok(prepared) +} + +fn canonical_stage_file(root: &Path, relative: &str) -> Result { + let path = root.join(relative); + let canonical = + std::fs::canonicalize(&path).map_err(|_| "prepared input file is unavailable")?; + let metadata = + std::fs::symlink_metadata(&path).map_err(|_| "prepared input file metadata failed")?; + if canonical != path || !metadata.is_file() || metadata.file_type().is_symlink() { + return Err("prepared input file must be physical and stage-owned".into()); + } + Ok(canonical) +} +fn verify_template_limit(path: &Path) -> Result<(), String> { + use std::io::Read; + let mut reader = std::fs::File::open(path) + .map_err(|_| "prepared config template is unavailable")? + .take(16 * 1024 * 1024 + 1); + let mut bytes = 0u64; + let mut chunk = [0; 64 * 1024]; + loop { + let count = reader + .read(&mut chunk) + .map_err(|_| "prepared config template read failed")?; + if count == 0 { + return Ok(()); + } + bytes += count as u64; + if bytes > 16 * 1024 * 1024 { + return Err("prepared config template exceeds source-view limit".into()); + } + } +} +fn verify_input_manifest_digest(path: &Path, expected: &str) -> Result<(), String> { + use sha2::{Digest, Sha256}; + use std::io::Read; + let mut reader = std::fs::File::open(path) + .map_err(|_| "prepared input manifest is unavailable")? + .take(64 * 1024 * 1024 + 1); + let mut hash = Sha256::new(); + let mut bytes = 0u64; + let mut chunk = [0; 64 * 1024]; + loop { + let count = reader + .read(&mut chunk) + .map_err(|_| "prepared input manifest read failed")?; + if count == 0 { + break; + } + bytes += count as u64; + if bytes > 64 * 1024 * 1024 { + return Err("prepared input manifest exceeds stored metadata limit".into()); + } + hash.update(&chunk[..count]); + } + if format!("{:x}", hash.finalize()) != expected.to_ascii_lowercase() { + return Err("prepared input manifest changed; reprepare before mount construction".into()); + } + Ok(()) +} +fn verify_workspace_destination( + workspace: &Path, + logical: &str, + effective: &str, + directory: bool, +) -> Result<(), String> { + let logical = Path::new(logical); + if !logical.starts_with(workspace) { + return Ok(()); + } + // These mountpoints and parents must already exist. This operation never + // creates, chmods or edits the original workspace or its links. + let canonical = std::fs::canonicalize(logical) + .map_err(|_| "selected workspace input mountpoint is unavailable")?; + let metadata = std::fs::metadata(logical) + .map_err(|_| "selected workspace input mountpoint metadata failed")?; + if canonical != Path::new(effective) + || if directory { + !metadata.is_dir() + } else { + !metadata.is_file() + } + { + return Err("selected workspace input topology changed; reprepare before launch".into()); + } + Ok(()) +} + +#[cfg(test)] +mod prepared_stage_tests { + use super::*; + use freshell_runtime_protocol::{ + PreparedProviderInputs, ProviderConfigBind, ProviderInputManifestReference, + }; + use sha2::{Digest, Sha256}; + + fn fixture() -> (tempfile::TempDir, PathBuf, PathBuf, PreparedProviderInputs) { + let root = tempfile::tempdir().unwrap(); + let stage = root.path().join("stage"); + let workspace = root.path().join("workspace"); + std::fs::create_dir_all(stage.join(".freshell/templates")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let manifest = b"{\"sources\":[]}"; + std::fs::write(stage.join(".freshell/input-manifest.json"), manifest).unwrap(); + std::fs::write( + stage.join(".freshell/templates/source-0"), + b"{\"model\":\"transported\"}", + ) + .unwrap(); + let target = workspace.join("opencode.json"); + std::fs::write(&target, b"{\"model\":\"original\"}").unwrap(); + let inputs = PreparedProviderInputs { + generation_id: "grant-prepared-stage".into(), + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(manifest)), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![ProviderConfigBind { + source_relative_path: ".freshell/templates/source-0".into(), + logical_destination: target.to_string_lossy().into_owned(), + effective_destination: target.to_string_lossy().into_owned(), + }], + resource_mounts: vec![], + indexed_files: None, + }; + (root, stage, workspace, inputs) + } + + #[test] + fn provider_prepared_stage_bounds_actual_config_template() { + let (_owned, stage, workspace, inputs) = fixture(); + std::fs::OpenOptions::new() + .write(true) + .open(stage.join(".freshell/templates/source-0")) + .unwrap() + .set_len(16 * 1024 * 1024 + 1) + .unwrap(); + assert!( + prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).is_err(), + "oversized selected template must be rejected before mount construction" + ); + } + + #[test] + fn provider_prepared_stage_consumes_template_and_preserves_workspace_topology() { + let (_owned, stage, workspace, mut inputs) = fixture(); + let original = workspace.join("opencode.json"); + let logical = workspace.join("linked.json"); + std::os::unix::fs::symlink("opencode.json", &logical).unwrap(); + inputs.config_binds[0].logical_destination = logical.to_string_lossy().into_owned(); + let mounts = prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).unwrap(); + assert_eq!( + std::fs::read(&mounts.config_binds[0].source).unwrap(), + b"{\"model\":\"transported\"}" + ); + assert_eq!( + std::fs::read(&original).unwrap(), + b"{\"model\":\"original\"}" + ); + assert_eq!( + std::fs::read_link(&logical).unwrap(), + Path::new("opencode.json") + ); + inputs.config_binds[0].logical_destination = workspace + .join("missing/selected.json") + .to_string_lossy() + .into_owned(); + assert!(prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).is_err()); + assert!(!workspace.join("missing").exists()); + } + + #[test] + fn provider_prepared_stage_rejects_changed_manifest_and_escaped_template() { + let (_owned, stage, workspace, inputs) = fixture(); + std::fs::write( + stage.join(".freshell/input-manifest.json"), + b"{\"sources\":[1]}", + ) + .unwrap(); + assert!(prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).is_err()); + std::fs::write( + stage.join(".freshell/input-manifest.json"), + b"{\"sources\":[]}", + ) + .unwrap(); + let template = stage.join(".freshell/templates/source-0"); + std::fs::remove_file(&template).unwrap(); + std::os::unix::fs::symlink(workspace.join("opencode.json"), &template).unwrap(); + assert!(prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).is_err()); + assert_eq!( + std::fs::read(workspace.join("opencode.json")).unwrap(), + b"{\"model\":\"original\"}" + ); + } +} + fn capability_directory() -> Result { let directory = CAPABILITY_DIR .get() @@ -337,6 +630,7 @@ mod provider_secret_tests { fn nested_opencode_user_config_requires_the_approved_root_mount() { use freshell_runtime_protocol::{ProviderConfigReference, ProviderPreparation}; let context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Opencode { project_config: vec![ProviderConfigReference { root: ProviderConfigRoot::UserProvider, diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs new file mode 100644 index 000000000..308bc7912 --- /dev/null +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -0,0 +1,230 @@ +use super::*; + +pub const INDEXED_VOLUME_OPTIONS: &str = + "size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec"; + +pub(super) fn mount_descriptors( + mounts: &docker::TerminalMounts, + provider_volume: &str, +) -> Result, BackendError> { + let mut selected = BTreeMap::::new(); + let mut add = |destination: &str, value: Value| -> Result<(), BackendError> { + if selected + .insert(destination.to_owned(), value.clone()) + .is_some_and(|prior| prior != value) + { + return Err(BackendError::InvalidConfig( + "prepared mounts have conflicting effective destinations".into(), + )); + } + Ok(()) + }; + for bind in &mounts.config_binds { + add( + &bind.reference.effective_destination, + json!({"Type":"bind","Source":bind.source,"Target":bind.reference.effective_destination,"ReadOnly":true}), + )?; + } + for root in &mounts.resource_mounts { + add( + &root.effective_destination, + json!({"Type":"volume","Source":provider_volume,"Target":root.effective_destination, + "ReadOnly":root.kind==freshell_runtime_protocol::ProviderResourceKind::File, + "VolumeOptions":{"NoCopy":true,"Subpath":root.provider_relative_path}}), + )?; + } + if let Some(files) = &mounts.indexed_files { + for alias in &files.aliases { + add( + &alias.effective_destination, + json!({"Type":"volume","Source":files.volume_name,"Target":alias.effective_destination,"ReadOnly":true, + "VolumeOptions":{"NoCopy":true,"Subpath":format!("files/source-{}",alias.source_id)}}), + )?; + } + } + Ok(selected.into_values().collect()) +} + +pub(super) fn verify( + actual: &Value, + mounts: &docker::TerminalMounts, + provider_volume: &str, + volumes: &BTreeMap, +) -> Result<(), BackendError> { + let expected = mount_descriptors(mounts, provider_volume)?; + if expected.is_empty() { + return Ok(()); + } + let mismatch = || { + BackendError::OwnershipMismatch( + "prepared provider mount/subpath/owned-volume topology changed".into(), + ) + }; + let declared = actual["HostConfig"]["Mounts"] + .as_array() + .ok_or_else(mismatch)?; + let reported = actual["Mounts"].as_array().ok_or_else(mismatch)?; + if declared.len() != expected.len() { + return Err(mismatch()); + } + for mount in expected { + let target = mount["Target"].as_str().ok_or_else(mismatch)?; + let declarations = declared + .iter() + .filter(|value| value["Target"] == target) + .collect::>(); + let reports = reported + .iter() + .filter(|value| value["Destination"] == target) + .collect::>(); + if declarations.len() != 1 || reports.len() != 1 { + return Err(mismatch()); + } + let declaration = declarations[0]; + let report = reports[0]; + if ["Type", "Source", "Target", "ReadOnly"] + .iter() + .any(|key| declaration[key] != mount[key]) + || report["Type"] != mount["Type"] + || report["RW"].as_bool() != mount["ReadOnly"].as_bool().map(|readonly| !readonly) + { + return Err(mismatch()); + } + if mount["Type"] == "bind" { + if report["Source"] != mount["Source"] { + return Err(mismatch()); + } + } else { + if declaration["VolumeOptions"]["Subpath"] != mount["VolumeOptions"]["Subpath"] + || declaration["VolumeOptions"]["NoCopy"] != true + || !declaration["VolumeOptions"]["DriverConfig"].is_null() + || report["Name"] != mount["Source"] + { + return Err(mismatch()); + } + let name = mount["Source"].as_str().ok_or_else(mismatch)?; + let volume = volumes.get(name).ok_or_else(mismatch)?; + if volume["Name"] != name + || volume["Driver"] != "local" + || !volume["Mountpoint"].is_string() + || report["Source"] != volume["Mountpoint"] + { + return Err(mismatch()); + } + if mounts + .indexed_files + .as_ref() + .is_some_and(|files| files.volume_name == name) + && volume["Options"] + != json!({"type":"tmpfs","device":"tmpfs","o":INDEXED_VOLUME_OPTIONS}) + { + return Err(mismatch()); + } + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use freshell_runtime_protocol::{ + IndexedProviderFileAlias, IndexedProviderFiles, ProviderConfigBind, ProviderResourceKind, + ProviderResourceMount, + }; + fn fixture() -> (docker::TerminalMounts, Value, BTreeMap) { + let mounts = docker::TerminalMounts { + workspace: "/fixture/workspace".into(), + git_common_dir: None, + provider_bootstrap_files: vec![], + provider_secret_files: vec![], + provider_user_root: None, + mcp_capability_file: None, + input_manifest: None, + config_binds: vec![docker::ResolvedProviderConfigBind { + source: "/private-stage/template".into(), + reference: ProviderConfigBind { + source_relative_path: ".freshell/templates/source-0".into(), + logical_destination: "/fixture/opencode.json".into(), + effective_destination: "/fixture/opencode.json".into(), + }, + }], + resource_mounts: vec![ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/plugin-0".into(), + provider_relative_path: ".freshell/provider-roots/plugin-0".into(), + logical_destination: "/outside/plugin".into(), + effective_destination: "/outside/plugin".into(), + kind: ProviderResourceKind::Directory, + }], + indexed_files: Some(IndexedProviderFiles { + volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), + aliases: vec![IndexedProviderFileAlias { + source_id: 0, + logical_destination: "/outside/credential".into(), + effective_destination: "/outside/credential".into(), + }], + }), + }; + let descriptors = mount_descriptors(&mounts, "owned-provider").unwrap(); + let actual = json!({"HostConfig":{"Mounts":descriptors},"Mounts":[ + {"Type":"bind","Source":"/private-stage/template","Destination":"/fixture/opencode.json","RW":false}, + {"Type":"volume","Name":"owned-provider","Source":"/docker/volumes/owned-provider/_data","Destination":"/outside/plugin","RW":true}, + {"Type":"volume","Name":"freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa","Source":"/docker/volumes/freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa/_data","Destination":"/outside/credential","RW":false}, + ]}); + let volumes = BTreeMap::from([ + ( + "owned-provider".into(), + json!({"Name":"owned-provider","Driver":"local","Mountpoint":"/docker/volumes/owned-provider/_data","Options":{}}), + ), + ( + "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), + json!({"Name":"freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa","Driver":"local","Mountpoint":"/docker/volumes/freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa/_data","Options":{"type":"tmpfs","device":"tmpfs","o":INDEXED_VOLUME_OPTIONS}}), + ), + ]); + (mounts, actual, volumes) + } + #[test] + fn provider_prepared_adoption_checks_subpaths_nocopy_and_actual_volume_options() { + let (mounts, actual, volumes) = fixture(); + verify(&actual, &mounts, "owned-provider", &volumes).unwrap(); + let alias = actual["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .position(|mount| mount["Target"] == "/outside/credential") + .unwrap(); + for mode in [ + "subpath", + "nocopy", + "mount-type", + "writable", + "source", + "volume-options", + ] { + let mut changed = actual.clone(); + let mut changed_volumes = volumes.clone(); + match mode { + "subpath" => { + changed["HostConfig"]["Mounts"][alias]["VolumeOptions"]["Subpath"] = + json!("files/source-1") + } + "nocopy" => { + changed["HostConfig"]["Mounts"][alias]["VolumeOptions"]["NoCopy"] = json!(false) + } + "mount-type" => changed["Mounts"][2]["Type"] = json!("bind"), + "writable" => changed["Mounts"][2]["RW"] = json!(true), + "source" => changed["Mounts"][2]["Source"] = json!("/other/volume/_data"), + "volume-options" => { + changed_volumes + .get_mut("freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa") + .unwrap()["Options"]["o"] = json!("size=512m") + } + _ => unreachable!(), + } + assert!( + verify(&changed, &mounts, "owned-provider", &changed_volumes).is_err(), + "changed {mode} must prevent adoption" + ); + } + } +} diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index fba9e153f..25f08682b 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -2410,6 +2410,7 @@ mod tests { let mut replacement = replacement; replacement.provider_launch_context = Some(freshell_runtime_protocol::ProviderLaunchContext { + prepared_inputs: None, preparation: freshell_runtime_protocol::ProviderPreparation::Opencode { project_config: Vec::new(), tui_config: None, @@ -2451,6 +2452,7 @@ mod tests { std::fs::create_dir_all(&user_root).unwrap(); std::fs::write(user_root.join("old.jsonc"), "{}").unwrap(); let mut context = ProviderLaunchContext { + prepared_inputs: None, preparation: ProviderPreparation::Opencode { project_config: vec![], tui_config: None, diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 1c85be94f..c1c6ad562 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -3064,6 +3064,7 @@ mod tests { freshell_runtime_protocol::ProviderSecretProfile::AmplifierOnecliLunarouteGlm53, }], provider_launch_context: Some(freshell_runtime_protocol::ProviderLaunchContext { + prepared_inputs: None, preparation: freshell_runtime_protocol::ProviderPreparation::Amplifier { bundle: "default".into(), resume_args: Vec::new(), @@ -3172,6 +3173,7 @@ mod tests { provider_bootstrap_files: Vec::new(), provider_secret_references: Vec::new(), provider_launch_context: Some(freshell_runtime_protocol::ProviderLaunchContext { + prepared_inputs: None, preparation: freshell_runtime_protocol::ProviderPreparation::Opencode { project_config: Vec::new(), tui_config: None, From 43c8a9a3575a998c7a0ccd0a7205b26784805b06 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 04:50:24 -0700 Subject: [PATCH 022/493] fix: preserve ordered OpenCode source origins and shared reference identities --- crates/freshell-platform/src/lib.rs | 1 + .../freshell-platform/src/opencode_recipe.rs | 58 +++ .../src/managed_provider_references.rs | 120 +++-- .../managed_provider_references/sources.rs | 443 ++++++++++++++++++ 4 files changed, 557 insertions(+), 65 deletions(-) create mode 100644 crates/freshell-platform/src/opencode_recipe.rs create mode 100644 crates/freshell-server/src/managed_provider_references/sources.rs diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index e11db93fe..0419571eb 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -61,6 +61,7 @@ pub mod mcp_inject; pub mod native_auth_exchange; pub mod opencode_config; pub mod opencode_plugin; +pub mod opencode_recipe; pub mod path; pub mod provider_environment; pub mod provider_environment_grants; diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs new file mode 100644 index 000000000..921fe035a --- /dev/null +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -0,0 +1,58 @@ +//! Private, unexpanded OpenCode recipes shared by controller and session host. +//! Expanded text is returned only to the caller's transient child environment. +use serde::{Deserialize, Serialize}; +use std::path::PathBuf; + +#[derive(Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ApprovedFileOccurrence { + pub ordinal: usize, + pub approved_alias: PathBuf, + pub runtime_destination: String, +} + +#[derive(Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct GeneratedFileRecipe { + pub generated_name: String, + pub unexpanded_source: String, + pub declaring_config: PathBuf, + pub cwd: PathBuf, + pub original_home: PathBuf, + pub occurrences: Vec, +} + +pub fn evaluate( + recipe: &GeneratedFileRecipe, + env: &dyn Fn(&str) -> Option, +) -> Result { + use crate::opencode_config::{file_reference_path, transform_file_tokens, TokenTransformError}; + let changed = "configuration reference selection changed; reprepare before dispatch"; + let mut observed = 0; + let text = transform_file_tokens(&recipe.unexpanded_source, env, |token| { + let path = file_reference_path( + token.path, + &recipe.declaring_config, + &recipe.cwd, + &recipe.original_home, + ); + let occurrence = recipe + .occurrences + .get(token.ordinal) + .filter(|approved| approved.ordinal == token.ordinal && approved.approved_alias == path) + .ok_or(changed)?; + observed += 1; + Ok(format!("{{file:{}}}", occurrence.runtime_destination)) + }) + .map_err(|error| match error { + TokenTransformError::Selection(cause) => cause, + TokenTransformError::Bound(_) => "configuration recipe profile limit exceeded", + })?; + if observed != recipe.occurrences.len() { + return Err(changed); + } + if text.len() > 64 * 1024 { + return Err("generated configuration exceeds 64 KiB child value limit"); + } + Ok(text) +} diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 5941b121c..e58924fd1 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -2,6 +2,8 @@ //! Plans retain approved source metadata and unexpanded templates, never payload bytes. use std::path::{Path, PathBuf}; +pub(crate) mod sources; + pub struct FileSourceOrigin<'a> { pub declaring_config: &'a Path, pub cwd: &'a Path, @@ -38,31 +40,35 @@ pub struct FileReferenceSelection { pub source_ordinal: usize, } -pub struct ApprovedFileOccurrence { - pub ordinal: usize, - pub approved_alias: PathBuf, - pub runtime_destination: String, -} -pub struct GeneratedFileRecipe { - pub generated_name: String, - pub unexpanded_source: String, - pub declaring_config: PathBuf, - pub cwd: PathBuf, - pub original_home: PathBuf, - pub occurrences: Vec, -} +pub use freshell_platform::opencode_recipe::{ApprovedFileOccurrence, GeneratedFileRecipe}; pub struct ResolvedFileReferencePlan { pub template: String, pub selections: Vec, pub referenced_variables: Vec, pub recipes: Vec, + pub occurrences: Vec, } pub fn resolve_file_references( raw: &str, origin: FileSourceOrigin<'_>, env: &dyn Fn(&str) -> Option, +) -> Result { + let mut next = 0usize; + resolve_file_references_allocated(raw, origin, env, 0, &mut |_| { + let destination = format!("/home/freshell/provider/.freshell/references/{next}"); + next += 1; + Ok(destination) + }) +} + +pub(super) fn resolve_file_references_allocated( + raw: &str, + origin: FileSourceOrigin<'_>, + env: &dyn Fn(&str) -> Option, + recipe_ordinal: usize, + allocate: &mut dyn FnMut(&Path) -> Result, ) -> Result { use freshell_platform::opencode_config::{ file_reference_path, transform_file_tokens, TokenTransformError, @@ -156,10 +162,8 @@ pub fn resolve_file_references( "configuration references exceed 32 MiB", )); } - let destination = format!( - "/home/freshell/provider/.freshell/references/{}", - selections.len() - ); + let destination = allocate(&canonical) + .map_err(|cause| fail(token.ordinal, names.clone(), cause))?; selections.push(FileReferenceSelection { canonical_source: canonical, runtime_destination: destination.clone(), @@ -191,9 +195,22 @@ pub fn resolve_file_references( "configuration source, expansion or event profile limit exceeded", ), })?; - drop(transformed); let (template, recipes) = if generated { - let generated_name = "FRESHELL_CONFIG_REFERENCE_0".to_string(); + let generated_name = format!("FRESHELL_CONFIG_REFERENCE_{recipe_ordinal}"); + if transformed.len() > 64 * 1024 { + return Err(fail( + 0, + Vec::new(), + "generated configuration exceeds 64 KiB child value limit", + )); + } + if env(&generated_name).is_some() || raw.contains(&format!("{{env:{generated_name}}}")) { + return Err(fail( + 0, + vec![generated_name], + "generated configuration name collides with an original child input", + )); + } ( format!("{{env:{generated_name}}}"), vec![GeneratedFileRecipe { @@ -202,7 +219,7 @@ pub fn resolve_file_references( declaring_config: origin.declaring_config.into(), cwd: origin.cwd.into(), original_home: origin.original_home.into(), - occurrences, + occurrences: occurrences.clone(), }], ) } else { @@ -216,11 +233,13 @@ pub fn resolve_file_references( template.push_str(&raw[cursor..]); (template, Vec::new()) }; + drop(transformed); Ok(ResolvedFileReferencePlan { template, selections, referenced_variables: variables.into_iter().collect(), recipes, + occurrences, }) } @@ -228,50 +247,14 @@ pub fn evaluate_generated_recipe( recipe: &GeneratedFileRecipe, env: &dyn Fn(&str) -> Option, ) -> Result { - use freshell_platform::opencode_config::{ - file_reference_path, transform_file_tokens, TokenTransformError, - }; - let fail = || ReferenceError { + freshell_platform::opencode_recipe::evaluate(recipe, env).map_err(|cause| ReferenceError { operation: "prepare", field: "OpenCode generated configuration", source_ordinal: 0, declaring_source: recipe.declaring_config.clone(), variables: Vec::new(), - cause: "configuration reference selection changed; reprepare before dispatch", - }; - let mut observed = 0; - let text = transform_file_tokens(&recipe.unexpanded_source, env, |token| { - let path = file_reference_path( - token.path, - &recipe.declaring_config, - &recipe.cwd, - &recipe.original_home, - ); - let occurrence = recipe - .occurrences - .get(token.ordinal) - .filter(|approved| approved.ordinal == token.ordinal && approved.approved_alias == path) - .ok_or_else(fail)?; - observed += 1; - Ok(format!("{{file:{}}}", occurrence.runtime_destination)) + cause, }) - .map_err(|error| match error { - TokenTransformError::Selection(error) => error, - TokenTransformError::Bound(_) => ReferenceError { - cause: "configuration recipe profile limit exceeded", - ..fail() - }, - })?; - if observed != recipe.occurrences.len() { - return Err(fail()); - } - if text.len() > 64 * 1024 { - return Err(ReferenceError { - cause: "generated configuration exceeds 64 KiB child value limit", - ..fail() - }); - } - Ok(text) } #[cfg(test)] @@ -522,7 +505,9 @@ process.stdout.write(JSON.stringify(receipts)); std::fs::write(&a, "prior-usable-nonce").unwrap(); std::fs::write(&b, "unapproved-new-nonce").unwrap(); let source = temp.path().join("opencode.json"); - let initial = |_: &str| Some(format!("prefix{{file:{}}}", a.display())); + let initial = |name: &str| { + (name == "OPENAI_API_KEY").then(|| format!("prefix{{file:{}}}", a.display())) + }; let plan = resolve_file_references( r#"{"x":"{env:OPENAI_API_KEY}"}"#, FileSourceOrigin { @@ -538,7 +523,10 @@ process.stdout.write(JSON.stringify(receipts)); consume_with_env(&plan, &previous, &initial), vec![digest("prefixprior-usable-nonce")] ); - let changed = |_: &str| Some(format!("sentinel-private-changed{{file:{}}}", b.display())); + let changed = |name: &str| { + (name == "OPENAI_API_KEY") + .then(|| format!("sentinel-private-changed{{file:{}}}", b.display())) + }; let err = evaluate_generated_recipe(&plan.recipes[0], &changed) .err() .unwrap(); @@ -548,7 +536,9 @@ process.stdout.write(JSON.stringify(receipts)); std::fs::read_to_string(previous.join(".freshell/references/0")).unwrap(), "prior-usable-nonce" ); - let same_selection = |_: &str| Some(format!("different-prefix{{file:{}}}", a.display())); + let same_selection = |name: &str| { + (name == "OPENAI_API_KEY").then(|| format!("different-prefix{{file:{}}}", a.display())) + }; assert!(evaluate_generated_recipe(&plan.recipes[0], &same_selection) .unwrap() .contains("different-prefix")); @@ -560,11 +550,11 @@ process.stdout.write(JSON.stringify(receipts)); let target = temp.path().join("reference"); std::fs::write(&target, "document-consumption-nonce").unwrap(); let source = temp.path().join("opencode.json"); - let lookup = |_: &str| { - Some( + let lookup = |name: &str| { + (name == "OPENAI_API_KEY").then(|| { serde_json::json!({"instructions":format!("{{file:{}}}",target.display())}) - .to_string(), - ) + .to_string() + }) }; let plan = resolve_file_references( "{env:OPENAI_API_KEY}", diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs new file mode 100644 index 000000000..eaf561149 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -0,0 +1,443 @@ +//! Ordered native source occurrences for managed OpenCode preparation. +use std::path::{Path, PathBuf}; + +#[derive(Clone, Copy, PartialEq, Eq)] +pub enum SourceLane { + Main, + Tui, +} + +pub struct SourceOccurrence { + pub logical_path: PathBuf, + pub canonical_source: PathBuf, + pub lane: SourceLane, +} + +pub struct OpencodeDiscovery<'a> { + pub home: &'a Path, + pub cwd: &'a Path, + pub worktree: &'a Path, + pub config_root: &'a Path, + pub env: &'a dyn Fn(&str) -> Option, +} + +pub fn discover_opencode( + request: OpencodeDiscovery<'_>, +) -> Result, &'static str> { + let mut sources = Vec::new(); + let disabled = (request.env)("OPENCODE_DISABLE_PROJECT_CONFIG") + .is_some_and(|value| matches!(value.as_str(), "1" | "true")); + for name in ["config.json", "opencode.json", "opencode.jsonc"] { + add_source( + &mut sources, + request.config_root.join(name), + SourceLane::Main, + )?; + } + if let Some(path) = (request.env)("OPENCODE_CONFIG").filter(|value| !value.is_empty()) { + add_source( + &mut sources, + absolute_from_cwd(&path, request.cwd), + SourceLane::Main, + )?; + } + if !disabled { + for directory in ancestors(request.cwd, Some(request.worktree)) + .into_iter() + .rev() + { + for name in ["opencode.json", "opencode.jsonc"] { + add_source(&mut sources, directory.join(name), SourceLane::Main)?; + } + } + } + for directory in config_directories(&request, Some(request.worktree), disabled)? { + if directory.file_name() == Some(std::ffi::OsStr::new(".opencode")) + || (request.env)("OPENCODE_CONFIG_DIR") + .is_some_and(|path| absolute_from_cwd(&path, request.cwd) == directory) + { + for name in ["opencode.json", "opencode.jsonc"] { + add_source(&mut sources, directory.join(name), SourceLane::Main)?; + } + } + } + for name in ["tui.json", "tui.jsonc"] { + add_source( + &mut sources, + request.config_root.join(name), + SourceLane::Tui, + )?; + } + if let Some(path) = (request.env)("OPENCODE_TUI_CONFIG").filter(|value| !value.is_empty()) { + add_source( + &mut sources, + absolute_from_cwd(&path, request.cwd), + SourceLane::Tui, + )?; + } + if !disabled { + for directory in ancestors(request.cwd, None).into_iter().rev() { + for name in ["tui.json", "tui.jsonc"] { + add_source(&mut sources, directory.join(name), SourceLane::Tui)?; + } + } + } + for directory in config_directories(&request, None, disabled)? { + if directory.file_name() == Some(std::ffi::OsStr::new(".opencode")) + || (request.env)("OPENCODE_CONFIG_DIR") + .is_some_and(|path| absolute_from_cwd(&path, request.cwd) == directory) + { + for name in ["tui.json", "tui.jsonc"] { + add_source(&mut sources, directory.join(name), SourceLane::Tui)?; + } + } + } + Ok(sources) +} + +fn absolute_from_cwd(path: &str, cwd: &Path) -> PathBuf { + let path = Path::new(path); + if path.is_absolute() { + path.into() + } else { + cwd.join(path) + } +} + +fn ancestors(start: &Path, stop: Option<&Path>) -> Vec { + let mut result = Vec::new(); + for directory in start.ancestors() { + result.push(directory.to_owned()); + if Some(directory) == stop { + break; + } + } + result +} + +pub fn config_directories( + request: &OpencodeDiscovery<'_>, + stop: Option<&Path>, + disabled: bool, +) -> Result, &'static str> { + let mut result = vec![request.config_root.to_owned()]; + let mut candidates = Vec::new(); + if !disabled { + candidates.extend( + ancestors(request.cwd, stop) + .into_iter() + .map(|directory| directory.join(".opencode")), + ); + } + candidates.push(request.home.join(".opencode")); + for directory in candidates { + match std::fs::symlink_metadata(&directory) { + Ok(_) => { + if !std::fs::metadata(&directory) + .map_err(|_| "selected OpenCode config directory is unavailable")? + .is_dir() + { + return Err("selected OpenCode config directory is not a directory"); + } + if !result.contains(&directory) { + result.push(directory); + } + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("selected OpenCode config directory metadata failed"), + } + } + if let Some(path) = (request.env)("OPENCODE_CONFIG_DIR").filter(|value| !value.is_empty()) { + let directory = absolute_from_cwd(&path, request.cwd); + if !result.contains(&directory) { + result.push(directory); + } + } + Ok(result) +} + +fn add_source( + sources: &mut Vec, + logical_path: PathBuf, + lane: SourceLane, +) -> Result<(), &'static str> { + match std::fs::symlink_metadata(&logical_path) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), + Err(_) => return Err("selected configuration source metadata failed"), + Ok(_) => {} + } + if sources.len() >= 8192 { + return Err("configuration source occurrence budget exceeded"); + } + let canonical_source = std::fs::canonicalize(&logical_path) + .map_err(|_| "selected configuration source is unavailable, dangling or cyclic")?; + let metadata = std::fs::metadata(&canonical_source) + .map_err(|_| "selected configuration source metadata failed")?; + if !metadata.is_file() || metadata.len() > 16 * 1024 * 1024 { + return Err("selected configuration source is not a bounded regular file"); + } + sources.push(SourceOccurrence { + logical_path, + canonical_source, + lane, + }); + Ok(()) +} + +pub struct PlannedSourceOccurrence { + pub source: SourceOccurrence, + pub references: super::ResolvedFileReferencePlan, +} + +pub fn plan_opencode_sources( + sources: Vec, + cwd: &Path, + home: &Path, + lookup: &dyn Fn(SourceLane, &Path, &str) -> Option, +) -> Result, super::ReferenceError> { + use std::io::Read; + let mut result = Vec::new(); + let mut selected = std::collections::BTreeMap::::new(); + let mut selected_bytes = 0u64; + let mut recipe_count = 0usize; + let mut aliases = std::collections::BTreeSet::new(); + let mut variables = std::collections::BTreeSet::new(); + let mut event_count = 0usize; + let mut retained_view_bytes = 0usize; + for (ordinal, source) in sources.into_iter().enumerate() { + let fail = |cause| super::ReferenceError { + operation: "read", + field: "OpenCode source", + source_ordinal: ordinal, + declaring_source: source.logical_path.clone(), + variables: vec![], + cause, + }; + let mut raw = String::new(); + std::fs::File::open(&source.canonical_source) + .map_err(|_| fail("selected source read failed"))? + .take(16 * 1024 * 1024 + 1) + .read_to_string(&mut raw) + .map_err(|_| fail("selected source read failed or is not UTF-8"))?; + if raw.len() > 16 * 1024 * 1024 { + return Err(fail("selected source exceeds source-view limit")); + } + let mut allocate = |canonical: &Path| { + if let Some(destination) = selected.get(canonical) { + return Ok(destination.clone()); + } + if selected.len() >= 128 { + return Err("configuration reference count exceeds 128 across sources"); + } + selected_bytes += std::fs::metadata(canonical) + .map_err(|_| "selected source metadata failed")? + .len(); + if selected_bytes > 32 * 1024 * 1024 { + return Err("configuration references exceed 32 MiB across sources"); + } + let destination = format!( + "/home/freshell/provider/.freshell/references/{}", + selected.len() + ); + selected.insert(canonical.to_owned(), destination.clone()); + Ok(destination) + }; + let references = super::resolve_file_references_allocated( + &raw, + super::FileSourceOrigin { + declaring_config: &source.logical_path, + cwd, + original_home: home, + }, + &|name| lookup(source.lane, &source.logical_path, name), + recipe_count, + &mut allocate, + )?; + recipe_count += references.recipes.len(); + retained_view_bytes = retained_view_bytes + .checked_add(references.template.len()) + .and_then(|bytes| { + references.recipes.iter().try_fold(bytes, |bytes, recipe| { + bytes.checked_add(recipe.unexpanded_source.len()) + }) + }) + .ok_or_else(|| fail("source-view byte count overflow"))?; + if retained_view_bytes > 64 * 1024 * 1024 { + return Err(fail("simultaneous retained source views exceed 64 MiB")); + } + aliases.extend( + references + .occurrences + .iter() + .map(|occurrence| occurrence.approved_alias.clone()), + ); + variables.extend(references.referenced_variables.iter().cloned()); + event_count += references.occurrences.len(); + if recipe_count > 128 || aliases.len() > 256 || variables.len() > 128 || event_count > 8192 + { + return Err(fail( + "combined source recipe, alias, variable or event limit exceeded", + )); + } + result.push(PlannedSourceOccurrence { source, references }); + } + Ok(result) +} + +#[cfg(test)] +mod tests { + use super::*; + use sha2::{Digest, Sha256}; + fn digest(path: &Path) -> String { + format!("{:x}", Sha256::digest(std::fs::read(path).unwrap())) + } + #[test] + fn managed_provider_references_discovery_keeps_native_order_origins_and_tui_ancestors() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let config = home.join(".config/opencode"); + let worktree = owned.path().join("project"); + let cwd = worktree.join("sub"); + let explicit = owned.path().join("selected/custom.json"); + let directory = owned.path().join("selected/config-dir"); + let files = [ + config.join("config.json"), + config.join("opencode.json"), + config.join("opencode.jsonc"), + explicit.clone(), + worktree.join("opencode.json"), + worktree.join("opencode.jsonc"), + cwd.join("opencode.json"), + cwd.join(".opencode/opencode.json"), + worktree.join(".opencode/opencode.json"), + home.join(".opencode/opencode.json"), + directory.join("opencode.json"), + config.join("tui.json"), + owned.path().join("tui.json"), + worktree.join("tui.json"), + cwd.join("tui.json"), + ]; + for (ordinal, path) in files.iter().enumerate() { + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(path, format!("{{\"model\":\"nonce-{ordinal}\"}}")).unwrap(); + } + let env = |key: &str| match key { + "OPENCODE_CONFIG" => Some(explicit.to_string_lossy().into_owned()), + "OPENCODE_CONFIG_DIR" => Some(directory.to_string_lossy().into_owned()), + _ => None, + }; + let sources = discover_opencode(OpencodeDiscovery { + home: &home, + cwd: &cwd, + worktree: &worktree, + config_root: &config, + env: &env, + }) + .unwrap(); + let expected = files.iter().map(|path| digest(path)).collect::>(); + let consumed = sources + .iter() + .map(|source| digest(&source.canonical_source)) + .collect::>(); + assert_eq!( + consumed, expected, + "all native-selected configuration bytes must be available in their loader order" + ); + for (source, logical) in sources.iter().zip(files) { + assert_eq!(source.logical_path, logical); + } + assert!(sources[..11] + .iter() + .all(|source| source.lane == SourceLane::Main)); + assert!(sources[11..] + .iter() + .all(|source| source.lane == SourceLane::Tui)); + assert_eq!(digest(&explicit), expected[3]); + } + + #[test] + fn managed_provider_references_source_plan_assigns_distinct_global_payload_destinations() { + let owned = tempfile::tempdir().unwrap(); + let cwd = owned.path().join("workspace"); + let home = owned.path().join("home"); + let mut sources = Vec::new(); + for name in ["first", "second"] { + let dir = cwd.join(name); + std::fs::create_dir_all(&dir).unwrap(); + let config = dir.join("opencode.json"); + std::fs::write(&config, "{\"instructions\":[\"{file:rules}\"]}").unwrap(); + std::fs::write(dir.join("rules"), name).unwrap(); + sources.push(SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + }); + } + let plan = plan_opencode_sources(sources, &cwd, &home, &|_, _, _| None).unwrap(); + let destinations = plan + .iter() + .map(|source| source.references.selections[0].runtime_destination.clone()) + .collect::>(); + assert_eq!( + destinations.len(), + 2, + "distinct source-origin payloads cannot overwrite the same runtime destination" + ); + for source in plan { + assert_eq!( + std::fs::read_to_string(&source.source.canonical_source).unwrap(), + "{\"instructions\":[\"{file:rules}\"]}" + ); + assert!(source + .references + .template + .contains(&source.references.selections[0].runtime_destination)); + } + } + + #[test] + fn managed_provider_references_source_plan_shares_one_identity_and_rejects_generated_name_collision( + ) { + let owned = tempfile::tempdir().unwrap(); + let config = owned.path().join("opencode.json"); + let payload = owned.path().join("rules"); + std::fs::write(&payload, "approved-payload").unwrap(); + std::fs::write(&config, "{\"instructions\":[\"{env:INSERT}\"]}").unwrap(); + let sources = || { + [SourceLane::Main, SourceLane::Tui] + .into_iter() + .map(|lane| SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane, + }) + .collect() + }; + let plan = plan_opencode_sources(sources(), owned.path(), owned.path(), &|_, _, name| { + (name == "INSERT").then(|| "prefix{file:rules}suffix".into()) + }) + .unwrap(); + assert_eq!( + plan[0].references.selections[0].runtime_destination, + plan[1].references.selections[0].runtime_destination + ); + assert_ne!( + plan[0].references.recipes[0].generated_name, + plan[1].references.recipes[0].generated_name + ); + let collision = plan_opencode_sources( + sources(), + owned.path(), + owned.path(), + &|_, _, name| match name { + "INSERT" => Some("prefix{file:rules}suffix".into()), + "FRESHELL_CONFIG_REFERENCE_0" => Some("user-owned-value".into()), + _ => None, + }, + ); + assert!( + collision.is_err(), + "generated child names cannot shadow user or source-specific auth values" + ); + } +} From 5b8e1126a0a2715255e532642350859713e47472 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 04:55:49 -0700 Subject: [PATCH 023/493] fix: keep prepared OpenCode inline values in child environment --- .../src/providers/mod.rs | 42 +++++++++++++++---- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index db00e29c8..13c94b79e 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -106,7 +106,7 @@ pub async fn prepare_terminal( "/run/freshell-private/user-provider/ephemeral/inline-config.json", ) .map_err(|_| "managed OpenCode inline config is unavailable")?; - terminal.env.insert("OPENCODE_CONFIG_CONTENT".into(), raw); + child_env.insert("OPENCODE_CONFIG_CONTENT".into(), raw); } } } @@ -151,11 +151,11 @@ pub async fn prepare_terminal( .map_err(|error| error.to_string())?; let inherited = terminal .env - .get("OPENCODE_CONFIG_CONTENT") - .or_else(|| child_env.get("OPENCODE_CONFIG_CONTENT")); + .remove("OPENCODE_CONFIG_CONTENT") + .or_else(|| child_env.get("OPENCODE_CONFIG_CONTENT").cloned()); if inherited.is_some() || permission.is_some() || has_mcp { let config = freshell_platform::opencode_config::merge_owned_entries( - inherited.map(String::as_str), + inherited.as_deref(), &plan, &freshell_platform::opencode_config::OwnedConfigEdits { disable_snapshots: false, @@ -165,9 +165,6 @@ pub async fn prepare_terminal( }, ) .map_err(|error| error.to_string())?; - terminal - .env - .insert("OPENCODE_CONFIG_CONTENT".into(), config.clone()); child_env.insert("OPENCODE_CONFIG_CONTENT".into(), config); } } @@ -551,6 +548,37 @@ mod provider_secret_context_tests { McpCapabilityReference, ProviderLaunchContext, ProviderPreparation, }; + #[tokio::test] + async fn provider_secret_inline_configuration_stays_only_in_actual_child_environment() { + let owned = tempfile::tempdir().unwrap(); + let terminal=serde_json::from_value(serde_json::json!({ + "terminalId":"terminal-inline", "streamId":"stream-inline", "mode":"opencode", "program":"opencode", + "cwd":owned.path(),"workspacePath":owned.path(),"runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"project-inline", + "env":{"FRESHELL_OPENCODE_REBIND":"0"} + })).unwrap(); + let secrets = std::collections::BTreeMap::from([( + "OPENCODE_CONFIG_CONTENT".into(), + "{\"model\":\"sentinel-private-inline\"}".into(), + )]); + let prepared = prepare_terminal(terminal, &secrets).await.unwrap(); + let metadata = serde_json::to_string(&prepared.terminal).unwrap(); + assert!( + !metadata.contains("sentinel-private-inline"), + "private normalized config cannot enter terminal launch serialization" + ); + assert!(!prepared + .terminal + .env + .contains_key("OPENCODE_CONFIG_CONTENT")); + let receipt=std::process::Command::new("node").args(["-e","const c=JSON.parse(process.env.OPENCODE_CONFIG_CONTENT);process.stdout.write(require('crypto').createHash('sha256').update(c.model).digest('hex'));"]).envs(&prepared.child_env).output().unwrap(); + assert!(receipt.status.success()); + use sha2::{Digest, Sha256}; + assert_eq!( + String::from_utf8(receipt.stdout).unwrap(), + format!("{:x}", Sha256::digest(b"sentinel-private-inline")) + ); + } + #[test] fn scoped_mcp_grant_becomes_provider_process_environment() { let grant = tempfile::NamedTempFile::new().unwrap(); From 3799a69e75b8a59fb3be378788a13a423c435f45 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 06:39:36 -0700 Subject: [PATCH 024/493] Stream indexed provider inputs through an owned private publisher --- crates/freshell-platform/src/lib.rs | 1 + .../src/provider_input_exchange.rs | 372 +++++++ crates/freshell-runtime-protocol/src/lib.rs | 14 + .../src/provider_inputs.rs | 69 ++ crates/freshell-session-host/src/main.rs | 42 + crates/freshell-supervisor/Cargo.toml | 2 +- crates/freshell-supervisor/src/backend.rs | 48 + .../freshell-supervisor/src/backend/docker.rs | 5 + .../src/backend/input_publisher.rs | 915 ++++++++++++++++++ .../src/backend/native_auth.rs | 34 +- crates/freshell-supervisor/src/service.rs | 14 + 11 files changed, 1502 insertions(+), 14 deletions(-) create mode 100644 crates/freshell-platform/src/provider_input_exchange.rs create mode 100644 crates/freshell-supervisor/src/backend/input_publisher.rs diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 0419571eb..0e2d10ee8 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -65,6 +65,7 @@ pub mod opencode_recipe; pub mod path; pub mod provider_environment; pub mod provider_environment_grants; +pub mod provider_input_exchange; pub mod resume_gate; pub mod spawn; diff --git a/crates/freshell-platform/src/provider_input_exchange.rs b/crates/freshell-platform/src/provider_input_exchange.rs new file mode 100644 index 000000000..946faa969 --- /dev/null +++ b/crates/freshell-platform/src/provider_input_exchange.rs @@ -0,0 +1,372 @@ +//! One attempt's bounded indexed file publication over private Unix IPC. +//! Frames and acknowledgements contain no payload in diagnostic output. +use serde::{Deserialize, Serialize}; + +pub const MAX_CHUNK_BYTES: usize = 64 * 1024; +pub const MAX_FRAME_BYTES: usize = 1024 * 1024; +pub const MAX_FILE_BYTES: u64 = 16 * 1024 * 1024; +pub const MAX_TOTAL_BYTES: u64 = 32 * 1024 * 1024; + +#[derive(Serialize, Deserialize)] +#[serde(tag = "operation", rename_all = "snake_case", deny_unknown_fields)] +pub enum Request { + Begin { + source_ids: Vec, + }, + Open { + source_id: u16, + }, + Chunk { + #[serde(deserialize_with = "bounded_chunk")] + bytes: Vec, + }, + Finish, + Seal, + Release, +} +#[derive(Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum Acknowledgement { + Begun, + Opened, + Written, + Finished, + Populated, + Released, +} + +#[cfg(unix)] +pub fn serve(root: &std::path::Path, socket: &std::path::Path) -> Result<(), &'static str> { + use std::{ + collections::BTreeSet, + io::Write, + os::unix::{ + fs::{DirBuilderExt, OpenOptionsExt}, + net::UnixStream, + }, + }; + if std::fs::canonicalize(root).map_err(|_| "indexed input root is unavailable")? != root + || !std::fs::symlink_metadata(root) + .map_err(|_| "indexed input root metadata failed")? + .is_dir() + { + return Err("indexed input root must be physical"); + } + let mut stream = + UnixStream::connect(socket).map_err(|_| "input publication exchange is unavailable")?; + let operation_timeout = Some(std::time::Duration::from_secs(20)); + stream + .set_read_timeout(operation_timeout) + .map_err(|_| "input publication timeout setup failed")?; + stream + .set_write_timeout(operation_timeout) + .map_err(|_| "input publication timeout setup failed")?; + let Request::Begin { source_ids } = read_request(&mut stream)? else { + return Err("input publication must begin with selected source ids"); + }; + let selected = source_ids.iter().copied().collect::>(); + if selected.len() != source_ids.len() + || selected.len() > 128 + || selected.iter().any(|id| *id >= 128) + { + return Err("input publication source ids exceed profile"); + } + let files = root.join("files"); + std::fs::DirBuilder::new() + .mode(0o700) + .create(&files) + .map_err(|_| "input publication directory is not fresh")?; + write_acknowledgement(&mut stream, &Acknowledgement::Begun)?; + let mut completed = BTreeSet::new(); + let mut active: Option<(u16, std::fs::File, u64)> = None; + let mut total = 0u64; + loop { + let request = read_request(&mut stream)?; + let acknowledgement = match request { + Request::Open { source_id } => { + if active.is_some() + || !selected.contains(&source_id) + || completed.contains(&source_id) + { + return Err("input publication source order is invalid"); + } + let file = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(files.join(format!("source-{source_id}"))) + .map_err(|_| "input publication private file creation failed")?; + active = Some((source_id, file, 0)); + Acknowledgement::Opened + } + Request::Chunk { bytes } => { + let (_, file, written) = active + .as_mut() + .ok_or("input publication chunk has no selected source")?; + if bytes.is_empty() || bytes.len() > MAX_CHUNK_BYTES { + return Err("input publication chunk exceeds profile"); + } + let next = written + .checked_add(bytes.len() as u64) + .ok_or("input publication size overflow")?; + let next_total = total + .checked_add(bytes.len() as u64) + .ok_or("input publication size overflow")?; + if next > MAX_FILE_BYTES || next_total > MAX_TOTAL_BYTES { + return Err("input publication payload exceeds profile"); + } + file.write_all(&bytes) + .map_err(|_| "input publication private file write failed")?; + *written = next; + total = next_total; + Acknowledgement::Written + } + Request::Finish => { + let (id, file, _) = active + .take() + .ok_or("input publication completion has no source")?; + file.sync_all() + .map_err(|_| "input publication private file flush failed")?; + completed.insert(id); + Acknowledgement::Finished + } + Request::Seal => { + if active.is_some() || completed != selected { + return Err("input publication cannot seal incomplete sources"); + } + write_acknowledgement(&mut stream, &Acknowledgement::Populated)?; + // This connection keeps the exact helper and its active tmpfs + // mount alive while the workload mounts and accepts the grant. + stream + .set_read_timeout(Some(std::time::Duration::from_secs(120))) + .map_err(|_| "input publication holding timeout setup failed")?; + if !matches!(read_request(&mut stream)?, Request::Release) { + return Err("sealed input publication accepts only owned release"); + } + write_acknowledgement(&mut stream, &Acknowledgement::Released)?; + return Ok(()); + } + Request::Begin { .. } | Request::Release => { + return Err("input publication phase is invalid") + } + }; + write_acknowledgement(&mut stream, &acknowledgement)?; + } +} + +fn bounded_chunk<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result, D::Error> { + struct ChunkVisitor; + impl<'de> serde::de::Visitor<'de> for ChunkVisitor { + type Value = Vec; + fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result { + formatter.write_str("a bounded private byte chunk") + } + fn visit_seq>( + self, + mut sequence: A, + ) -> Result { + let mut bytes = Vec::new(); + while let Some(byte) = sequence.next_element::()? { + if bytes.len() == MAX_CHUNK_BYTES { + return Err(serde::de::Error::custom("private chunk exceeds limit")); + } + if bytes.len() == bytes.capacity() { + bytes.reserve((MAX_CHUNK_BYTES - bytes.len()).min(bytes.capacity().max(64))); + } + bytes.push(byte); + } + Ok(bytes) + } + } + deserializer.deserialize_seq(ChunkVisitor) +} + +pub fn read_request(reader: &mut impl std::io::Read) -> Result { + let mut length = [0; 4]; + reader + .read_exact(&mut length) + .map_err(|_| "input publication frame is unavailable")?; + let length = u32::from_be_bytes(length) as usize; + if length >= MAX_FRAME_BYTES { + return Err("input publication frame exceeds limit"); + } + let mut bytes = vec![0; length]; + reader + .read_exact(&mut bytes) + .map_err(|_| "input publication frame is incomplete")?; + serde_json::from_slice(&bytes).map_err(|_| "input publication frame is invalid or over-bound") +} + +struct FrameBuffer(Vec); +impl std::io::Write for FrameBuffer { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + if bytes.len() >= MAX_FRAME_BYTES.saturating_sub(self.0.len()) { + return Err(std::io::Error::other( + "input publication frame exceeds limit", + )); + } + self.0.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } +} + +pub fn write_request( + writer: &mut impl std::io::Write, + request: &Request, +) -> Result<(), &'static str> { + if matches!(request,Request::Chunk{bytes} if bytes.len()>MAX_CHUNK_BYTES) { + return Err("input publication chunk exceeds profile"); + } + write_frame(writer, request) +} +pub fn write_acknowledgement( + writer: &mut impl std::io::Write, + acknowledgement: &Acknowledgement, +) -> Result<(), &'static str> { + write_frame(writer, acknowledgement) +} +fn write_frame( + writer: &mut impl std::io::Write, + value: &impl Serialize, +) -> Result<(), &'static str> { + let mut frame = FrameBuffer(Vec::new()); + serde_json::to_writer(&mut frame, value) + .map_err(|_| "input publication frame exceeds limit")?; + writer + .write_all(&(frame.0.len() as u32).to_be_bytes()) + .map_err(|_| "input publication frame write failed")?; + writer + .write_all(&frame.0) + .map_err(|_| "input publication frame write failed")?; + Ok(()) +} + +#[cfg(all(test, unix))] +mod tests { + use super::*; + use std::{ + io::{Read, Write}, + os::unix::net::UnixListener, + }; + + fn send(stream: &mut std::os::unix::net::UnixStream, request: &Request) -> Acknowledgement { + let frame = serde_json::to_vec(request).unwrap(); + assert!(frame.len() < MAX_FRAME_BYTES); + stream + .write_all(&(frame.len() as u32).to_be_bytes()) + .unwrap(); + stream.write_all(&frame).unwrap(); + let mut length = [0; 4]; + stream.read_exact(&mut length).unwrap(); + let mut response = vec![0; u32::from_be_bytes(length) as usize]; + stream.read_exact(&mut response).unwrap(); + serde_json::from_slice(&response).unwrap() + } + + #[test] + fn provider_input_exchange_publishes_private_files_and_holds_active_generation_until_release() { + let owned = tempfile::tempdir().unwrap(); + let root = owned.path().join("indexed"); + std::fs::create_dir(&root).unwrap(); + let socket = owned.path().join("exchange.sock"); + let listener = UnixListener::bind(&socket).unwrap(); + listener.set_nonblocking(true).unwrap(); + let published_root = root.clone(); + let worker = std::thread::spawn(move || serve(&published_root, &socket)); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(1); + let (mut stream, _) = loop { + match listener.accept() { + Ok(pair) => break pair, + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + assert!( + std::time::Instant::now() < deadline, + "publisher did not connect to private exchange" + ); + std::thread::yield_now(); + } + Err(error) => panic!("owned exchange accept failed: {error}"), + } + }; + assert!( + send( + &mut stream, + &Request::Begin { + source_ids: vec![0, 1] + } + ) == Acknowledgement::Begun + ); + for (source_id, nonce) in [ + (0, b"first-private-nonce".as_slice()), + (1, b"second-private-nonce".as_slice()), + ] { + assert!(send(&mut stream, &Request::Open { source_id }) == Acknowledgement::Opened); + assert!( + send( + &mut stream, + &Request::Chunk { + bytes: nonce.to_vec() + } + ) == Acknowledgement::Written + ); + assert!(send(&mut stream, &Request::Finish) == Acknowledgement::Finished); + } + assert!(send(&mut stream, &Request::Seal) == Acknowledgement::Populated); + assert!( + !worker.is_finished(), + "publisher must retain its active mount until workload acceptance" + ); + use std::os::unix::fs::PermissionsExt; + for (source_id, nonce) in [ + (0, b"first-private-nonce".as_slice()), + (1, b"second-private-nonce".as_slice()), + ] { + let file = root.join(format!("files/source-{source_id}")); + assert_eq!(std::fs::read(&file).unwrap(), nonce); + assert_eq!( + std::fs::metadata(&file).unwrap().permissions().mode() & 0o777, + 0o600 + ); + } + assert!(send(&mut stream, &Request::Release) == Acknowledgement::Released); + assert!(worker.join().unwrap().is_ok()); + } + + #[test] + fn provider_input_exchange_rejects_actual_oversized_chunks_and_frames_before_publication() { + let too_large = serde_json::to_vec(&Request::Chunk { + bytes: vec![255; MAX_CHUNK_BYTES + 1], + }) + .unwrap(); + let mut wire = (too_large.len() as u32).to_be_bytes().to_vec(); + wire.extend(too_large); + let error = read_request(&mut std::io::Cursor::new(wire)).err().unwrap(); + assert!(!error.contains("255")); + let mut frame_header = std::io::Cursor::new((MAX_FRAME_BYTES as u32).to_be_bytes()); + assert!(read_request(&mut frame_header).is_err()); + let mut encoded = Vec::new(); + assert!(write_request( + &mut encoded, + &Request::Chunk { + bytes: vec![0; MAX_CHUNK_BYTES + 1] + } + ) + .is_err()); + assert!(encoded.is_empty()); + let mut accepted = Vec::new(); + write_request( + &mut accepted, + &Request::Chunk { + bytes: vec![255; MAX_CHUNK_BYTES], + }, + ) + .unwrap(); + let Request::Chunk { bytes } = read_request(&mut std::io::Cursor::new(accepted)).unwrap() + else { + panic!("expected private chunk"); + }; + assert_eq!(bytes.len(), MAX_CHUNK_BYTES); + } +} diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 4186b6819..09fef8182 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -1852,6 +1852,13 @@ impl FreshAgentLaunchSpec { context.validate(self.provider.as_str())?; } validate_provider_secrets(self.provider.as_str(), &self.provider_secret_references)?; + if let Some(inputs) = self + .provider_launch_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + { + inputs.validate_references(&self.provider_secret_references)?; + } if self.provider != FreshProvider::Kilroy && !self.provider_bootstrap_files.is_empty() { return Err(RuntimeError::new( RuntimeErrorCode::InvalidRequest, @@ -2049,6 +2056,13 @@ impl TerminalLaunchSpec { } } validate_provider_secrets(&self.mode, &self.provider_secret_references)?; + if let Some(inputs) = self + .provider_launch_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + { + inputs.validate_references(&self.provider_secret_references)?; + } if let Some(context) = &self.provider_launch_context { context.validate(&self.mode)?; } diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 597a029a1..7ef98fe8d 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -74,6 +74,29 @@ pub fn indexed_provider_volume_name(soul: &SoulId, generation: &str) -> String { } impl PreparedProviderInputs { + pub fn validate_references( + &self, + references: &[crate::ProviderSecretReference], + ) -> Result<(), RuntimeError> { + let invalid = || { + RuntimeError::new(crate::RuntimeErrorCode::InvalidRequest,"prepared provider file indexes do not match approved configuration grants; reprepare before launch") + }; + let configurations = references + .iter() + .filter(|reference| reference.profile.is_configuration_file()) + .collect::>(); + if configurations.len() != self.manifest.source_count as usize { + return Err(invalid()); + } + for (source_id, reference) in configurations.into_iter().enumerate() { + if reference.provider_relative_path.as_deref() + != Some(format!(".freshell/references/{source_id}").as_str()) + { + return Err(invalid()); + } + } + Ok(()) + } pub fn validate( &self, capability: Option<&McpCapabilityReference>, @@ -350,4 +373,50 @@ mod tests { let restored: PreparedProviderInputs = serde_json::from_slice(&bytes).unwrap(); assert_eq!(restored, value); } + + #[test] + fn provider_prepared_file_indexes_must_name_real_configuration_grants_in_both_routes() { + let context = crate::ProviderLaunchContext { + prepared_inputs: Some(prepared()), + preparation: crate::ProviderPreparation::Codex { + tui_args: vec![], + sidecar_args: vec![], + }, + mcp_capability: Some(capability()), + config: vec![], + }; + let mut terminal:crate::TerminalLaunchSpec=serde_json::from_value(serde_json::json!({ + "terminalId":"terminal-index","streamId":"stream-index","mode":"codex","program":"codex", + "cwd":"/fixture/workspace","workspacePath":"/fixture/workspace","runAsUid":65534,"runAsGid":0, + "cols":80,"rows":24,"projectKey":"project-index","providerLaunchContext":context + })).unwrap(); + let mut fresh:crate::FreshAgentLaunchSpec=serde_json::from_value(serde_json::json!({ + "sessionId":"session-index","provider":"codex","sessionType":"freshcodex","runtimeVariant":"codex-app-server", + "providerStoreId":"store-index","cwd":"/fixture/workspace","workspacePath":"/fixture/workspace","runAsUid":65534,"runAsGid":0, + "providerLaunchContext":context + })).unwrap(); + assert!( + terminal.validate().is_err(), + "file subpaths without approved input grants must fail before Docker" + ); + assert!( + fresh.validate().is_err(), + "fresh route requires the same source identity proof" + ); + let references = (0..2) + .map(|source_id| crate::ProviderSecretReference { + source_path: format!("/private/source-{source_id}"), + profile: crate::ProviderSecretProfile::CodexConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{source_id}")), + }) + .collect::>(); + terminal.provider_secret_references = references.clone(); + fresh.provider_secret_references = references; + terminal.validate().unwrap(); + fresh.validate().unwrap(); + terminal.provider_secret_references.swap(0, 1); + fresh.provider_secret_references.swap(0, 1); + assert!(terminal.validate().is_err()); + assert!(fresh.validate().is_err()); + } } diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 90a757045..6551370c9 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -117,11 +117,25 @@ async fn run() -> Result<(), String> { Some("serve") => serve(&args[2..]).await, Some("worker") => worker(&args[2..]).await, Some("native-auth-only") if args.len() == 2 => { + verify_private_helper_namespace( + Path::new("/home/freshell/provider"), + Path::new("/run/freshell-auth-exchange/reply.sock"), + )?; freshell_platform::native_auth_exchange::serve( Path::new("/home/freshell/provider"), Path::new("/run/freshell-auth-exchange/reply.sock"), ).map_err(str::to_owned) } + Some("publish-provider-inputs") if args.len() == 2 => { + verify_private_helper_namespace( + Path::new("/run/freshell-input-files"), + Path::new("/run/freshell-input-exchange/reply.sock"), + )?; + freshell_platform::provider_input_exchange::serve( + Path::new("/run/freshell-input-files"), + Path::new("/run/freshell-input-exchange/reply.sock"), + ).map_err(str::to_owned) + } Some("native-history-only") => { let snapshot = freshell_freshagent::native_history::read( &required_arg(&args[2..], "--provider")?, @@ -144,6 +158,34 @@ async fn run() -> Result<(), String> { } } +fn verify_private_helper_namespace(root: &Path, socket: &Path) -> Result<(), String> { + use std::os::unix::fs::{FileTypeExt, MetadataExt}; + let invalid = || "private provider helper namespace mapping is unavailable".to_string(); + if unsafe { libc::geteuid() } != 65534 || unsafe { libc::getegid() } != 0 { + return Err(invalid()); + } + let root_meta = std::fs::symlink_metadata(root).map_err(|_| invalid())?; + let directory = + std::fs::symlink_metadata(socket.parent().ok_or_else(invalid)?).map_err(|_| invalid())?; + let endpoint = std::fs::symlink_metadata(socket).map_err(|_| invalid())?; + if !root_meta.is_dir() + || root_meta.uid() != 65534 + || root_meta.gid() != 0 + || root_meta.mode() & 0o777 != 0o700 + || !directory.is_dir() + || directory.uid() != 0 + || directory.gid() != 0 + || directory.mode() & 0o777 != 0o710 + || !endpoint.file_type().is_socket() + || endpoint.uid() != 0 + || endpoint.gid() != 0 + || endpoint.mode() & 0o777 != 0o660 + { + return Err(invalid()); + } + Ok(()) +} + async fn serve(args: &[String]) -> Result<(), String> { let socket_path = required_arg(args, "--control-socket")?; let state_dir = PathBuf::from(required_arg(args, "--state-dir")?); diff --git a/crates/freshell-supervisor/Cargo.toml b/crates/freshell-supervisor/Cargo.toml index 99c1d70c2..70f627b71 100644 --- a/crates/freshell-supervisor/Cargo.toml +++ b/crates/freshell-supervisor/Cargo.toml @@ -18,7 +18,7 @@ freshell-runtime-observability = { path = "../freshell-runtime-observability" } serde = { workspace = true } serde_json = { workspace = true } rusqlite = { package = "rusqlite", version = "0.31", features = ["bundled"] } -tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "io-util"] } +tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "io-util", "fs"] } uuid = { version = "1", features = ["v4"] } sha2 = "0.10" async-trait = "0.1" diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index c92063209..68bccb68b 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -1,4 +1,5 @@ pub mod docker; +mod input_publisher; mod native_auth; mod native_history; mod prepared_inputs; @@ -119,6 +120,12 @@ pub struct BackendInspection { #[async_trait] pub trait RuntimeBackend: Send + Sync { + async fn commit_prepared_inputs( + &self, + _handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + Ok(()) + } async fn read_native_auth( &self, _handle: &OwnedRuntimeHandle, @@ -168,12 +175,17 @@ pub trait RuntimeBackend: Send + Sync { #[derive(Clone)] pub struct DockerEngineBackend { socket_path: PathBuf, + input_publishers: + std::sync::Arc>>, + input_publication_claims: std::sync::Arc>>, } impl DockerEngineBackend { pub fn new(socket_path: impl Into) -> Self { Self { socket_path: socket_path.into(), + input_publishers: Default::default(), + input_publication_claims: Default::default(), } } @@ -549,6 +561,28 @@ fn runtime_host_environment( #[async_trait] impl RuntimeBackend for DockerEngineBackend { + async fn commit_prepared_inputs( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + let context = handle + .terminal() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + handle + .fresh_agent() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }); + if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { + self.release_indexed_publication( + handle.soul_id(), + handle.incarnation_id(), + &inputs.generation_id, + ) + .await?; + } + Ok(()) + } async fn read_native_auth( &self, handle: &OwnedRuntimeHandle, @@ -740,6 +774,14 @@ impl RuntimeBackend for DockerEngineBackend { .map_err(|_| { BackendError::InvalidConfig("prepared mount metadata cannot be encoded".into()) })?; + if mounts.indexed_files.is_some() && !rootless { + return Err(BackendError::Unavailable( + "indexed provider input publication requires the configured rootless backend" + .into(), + )); + } + self.begin_indexed_publication(spec, mounts, &binary) + .await?; } // The name is diagnostic only and is deliberately unique per create attempt. // A crash after Docker create but before the container-id commit can leave a @@ -759,6 +801,12 @@ impl RuntimeBackend for DockerEngineBackend { ) .await?; if response.status != 201 { + // Transport failures and server errors can hide a successful create. + // Only a definitive request rejection proves that no workload has + // acquired this generation's mounts. + if matches!(response.status, 400 | 401 | 403 | 404 | 409 | 422) { + self.abort_indexed_publication(spec).await?; + } return Err(response.as_error()); } let value: Value = serde_json::from_slice(&response.body) diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 42ea5fb04..988b52f98 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -89,6 +89,11 @@ fn workload_mounts( } freshell_runtime_protocol::validate_provider_secrets(provider, secret_references) .map_err(|error| error.message)?; + if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { + inputs + .validate_references(secret_references) + .map_err(|error| error.message)?; + } let workspace = canonical_dir(Path::new(workspace_path), "workspace")?; if workspace != Path::new(workspace_path) { return Err("workspace path must already be canonical".into()); diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs new file mode 100644 index 000000000..34ce35988 --- /dev/null +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -0,0 +1,915 @@ +//! Fixed provider-UID indexed input publisher and its owned active mount lease. +use super::*; +use freshell_platform::provider_input_exchange::{ + Acknowledgement, Request, MAX_CHUNK_BYTES, MAX_FILE_BYTES, MAX_FRAME_BYTES, MAX_TOTAL_BYTES, +}; +use std::time::Duration; +use tokio::net::UnixListener; + +fn fail(cause: &'static str) -> BackendError { + BackendError::Unavailable(cause.into()) +} + +pub(super) struct HeldPublisher { + soul: String, + incarnation: String, + name: String, + expected: Value, + context: freshell_runtime_protocol::ProviderLaunchContext, + references: Vec, + stream: UnixStream, + _directory: native_auth::ExchangeDirectory, +} +struct PublicationClaim { + claims: std::sync::Arc>>, + soul: String, +} +impl Drop for PublicationClaim { + fn drop(&mut self) { + self.claims + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .remove(&self.soul); + } +} + +async fn acknowledgement( + stream: &mut UnixStream, + expected: Acknowledgement, +) -> Result<(), BackendError> { + let length = stream + .read_u32() + .await + .map_err(|_| fail("input publisher acknowledgement unavailable"))? + as usize; + if length >= MAX_FRAME_BYTES { + return Err(fail("input publisher acknowledgement exceeds limit")); + } + let mut bytes = vec![0; length]; + stream + .read_exact(&mut bytes) + .await + .map_err(|_| fail("input publisher acknowledgement incomplete"))?; + let actual: Acknowledgement = serde_json::from_slice(&bytes) + .map_err(|_| fail("input publisher acknowledgement invalid"))?; + if actual != expected { + return Err(fail("input publisher acknowledgement phase changed")); + } + Ok(()) +} +async fn request( + stream: &mut UnixStream, + value: &Request, + expected: Acknowledgement, +) -> Result<(), BackendError> { + let mut bytes = Vec::new(); + freshell_platform::provider_input_exchange::write_request(&mut bytes, value).map_err(fail)?; + stream + .write_all(&bytes) + .await + .map_err(|_| fail("input publisher private request failed"))?; + acknowledgement(stream, expected).await +} + +fn labels(spec: &CreateRuntimeSpec, generation: &str) -> Value { + json!({"com.freshell.managed-inputs":"true","com.freshell.installation-id":spec.installation_id, + "com.freshell.soul-id":spec.soul_id,"com.freshell.incarnation-id":spec.incarnation_id, + "com.freshell.input-generation":generation}) +} + +fn environment(value: &Value) -> Option> { + let mut result = BTreeMap::new(); + for entry in value.as_array()? { + let (key, value) = entry.as_str()?.split_once('=')?; + if result.insert(key, value).is_some() { + return None; + } + } + Some(result) +} + +pub(super) fn owned_helper(actual: &Value, expected: &Value) -> bool { + let mounts_match = actual["HostConfig"]["Mounts"] + .as_array() + .zip(expected["HostConfig"]["Mounts"].as_array()) + .is_some_and(|(actual, expected)| { + actual.len() == expected.len() + && actual.iter().zip(expected).all(|(actual, expected)| { + ["Type", "Source", "Target", "ReadOnly"] + .iter() + .all(|key| actual[key] == expected[key]) + && actual["VolumeOptions"]["NoCopy"] == expected["VolumeOptions"]["NoCopy"] + && actual["VolumeOptions"]["Subpath"].as_str().unwrap_or("") + == expected["VolumeOptions"]["Subpath"].as_str().unwrap_or("") + }) + }); + ["Image", "User", "Tty", "Entrypoint", "Cmd", "Labels"] + .iter() + .all(|key| actual["Config"][key] == expected[key]) + && environment(&actual["Config"]["Env"]) + .is_some_and(|actual| Some(actual) == environment(&expected["Env"])) + && mounts_match + && actual["HostConfig"]["LogConfig"]["Type"] == "none" +} + +impl DockerEngineBackend { + pub(super) async fn private_helper_environment( + &self, + image: &str, + explicit: &[&str], + ) -> Result, BackendError> { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/images/{image}/json"), + None, + MAX_FRAME_BYTES as u64, + ) + .await + .map_err(|_| fail("private helper image inspection unavailable"))?; + if inspected.status != 200 { + return Err(fail("private helper image inspection failed")); + } + let image: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("private helper image metadata invalid"))?; + let mut values = BTreeMap::::new(); + if let Some(entries) = image["Config"]["Env"].as_array() { + if entries.len() > 128 { + return Err(fail("private helper image environment exceeds limit")); + } + for entry in entries { + let (key, _) = entry + .as_str() + .and_then(|entry| entry.split_once('=')) + .ok_or_else(|| fail("private helper image environment invalid"))?; + if key.is_empty() + || key.len() > 128 + || !key + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_') + { + return Err(fail("private helper image environment name invalid")); + } + // Explicit empty overrides prevent Docker's image defaults + // from introducing provider credentials into this helper. + values.insert(key.to_owned(), String::new()); + } + } + for entry in explicit { + let (key, value) = entry + .split_once('=') + .ok_or_else(|| fail("private helper environment invalid"))?; + values.insert(key.to_owned(), value.to_owned()); + } + Ok(values + .into_iter() + .map(|(key, value)| format!("{key}={value}")) + .collect()) + } + pub(super) async fn abort_indexed_publication( + &self, + spec: &CreateRuntimeSpec, + ) -> Result<(), BackendError> { + let context = spec + .terminal + .as_ref() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + spec.fresh_agent + .as_ref() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }); + let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) else { + return Ok(()); + }; + let Some(files) = &inputs.indexed_files else { + return Ok(()); + }; + { + let held = self + .input_publishers + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))?; + let Some(publisher) = held.get(&files.volume_name) else { + return Ok(()); + }; + if publisher.soul != spec.soul_id.as_str() + || publisher.incarnation != spec.incarnation_id.as_str() + || publisher.context != *context.unwrap() + { + return Err(fail("input publisher abort belongs to another preparation")); + } + } + self.release_indexed_publication( + &spec.soul_id, + &spec.incarnation_id, + &inputs.generation_id, + ) + .await?; + self.cleanup_input_volume( + &files.volume_name, + &labels(spec, &inputs.generation_id), + &json!({"type":"tmpfs","device":"tmpfs","o":prepared_inputs::INDEXED_VOLUME_OPTIONS}), + ) + .await + } + pub(super) async fn begin_indexed_publication( + &self, + spec: &CreateRuntimeSpec, + mounts: &docker::TerminalMounts, + binary: &Path, + ) -> Result<(), BackendError> { + let Some(files) = &mounts.indexed_files else { + return Ok(()); + }; + let context = spec + .terminal + .as_ref() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + spec.fresh_agent + .as_ref() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }) + .ok_or_else(|| fail("input publisher context unavailable"))?; + let inputs = context + .prepared_inputs + .as_ref() + .ok_or_else(|| fail("input publisher preparation unavailable"))?; + if files.volume_name + != freshell_runtime_protocol::indexed_provider_volume_name( + &spec.soul_id, + &inputs.generation_id, + ) + { + return Err(fail("input publisher volume does not belong to generation")); + } + let references = spec + .terminal + .as_ref() + .map(|spec| &spec.provider_secret_references) + .or_else(|| { + spec.fresh_agent + .as_ref() + .map(|spec| &spec.provider_secret_references) + }) + .ok_or_else(|| fail("input publisher references unavailable"))?; + inputs + .validate_references(references) + .map_err(|_| fail("input publisher source identity mismatch"))?; + let sources = references + .iter() + .filter(|reference| reference.profile.is_configuration_file()) + .map(|reference| PathBuf::from(&reference.source_path)) + .collect::>(); + let soul = spec.soul_id.to_string(); + { + let held = self + .input_publishers + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))?; + if let Some(publisher) = held.get(&files.volume_name) { + if publisher.soul == soul + && publisher.incarnation == spec.incarnation_id.as_str() + && publisher.context == *context + && publisher.references == *references + && publisher.expected["Image"] == spec.image_ref + && publisher.expected["HostConfig"]["Mounts"][0]["Source"] + == binary.to_string_lossy().as_ref() + { + let mut unexpected = [0]; + if matches!(publisher.stream.try_read(&mut unexpected), Err(error) if error.kind()==std::io::ErrorKind::WouldBlock) + { + return Ok(()); + } + return Err(fail("pending input publisher lease is no longer available")); + } + return Err(fail("pending input publisher preparation changed")); + } + if held.values().any(|publisher| publisher.soul == soul) { + return Err(fail( + "pending input publisher requires exact preparation reconciliation", + )); + } + } + { + let mut claims = self + .input_publication_claims + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))?; + if !claims.insert(soul.clone()) { + return Err(fail("input publication already pending for soul")); + } + } + let _claim = PublicationClaim { + claims: self.input_publication_claims.clone(), + soul: soul.clone(), + }; + let directory = native_auth::ExchangeDirectory::create_input()?; + let socket = directory.0.join("reply.sock"); + let listener = UnixListener::bind(&socket) + .map_err(|_| fail("input publisher private exchange bind failed"))?; + directory.mode(&socket, 0o660)?; + let name = format!("{}-publisher", files.volume_name); + let ownership = labels(spec, &inputs.generation_id); + let mut body = json!({ + "Image":spec.image_ref,"User":"65534:0","Tty":false,"Labels":ownership, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["publish-provider-inputs"],"Env":[], + "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"SecurityOpt":["no-new-privileges"], + "Memory":128*1024*1024,"MemorySwap":128*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, + "LogConfig":{"Type":"none","Config":{}}, + "Mounts":[{"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, + {"Type":"volume","Source":files.volume_name,"Target":"/run/freshell-input-files","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}, + {"Type":"bind","Source":directory.0,"Target":"/run/freshell-input-exchange","ReadOnly":true}]} + }); + let options = + json!({"type":"tmpfs","device":"tmpfs","o":prepared_inputs::INDEXED_VOLUME_OPTIONS}); + let mut volume_attempted = false; + let mut helper_attempted = false; + let operation = async { + body["Env"] = serde_json::to_value( + self.private_helper_environment(&spec.image_ref, &[]) + .await?, + ) + .map_err(|_| fail("private helper environment encoding failed"))?; + let prior = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", files.volume_name), + None, + 1024 * 1024, + ) + .await + .map_err(|_| fail("input publisher expected-volume query failed"))?; + if prior.status != 404 { + return Err(fail( + "input publisher volume already exists; reconcile before preparation", + )); + } + volume_attempted = true; + let volume=self.request_bounded("POST",&format!("{DOCKER_API}/volumes/create"),Some(&json!({"Name":files.volume_name,"Driver":"local","Labels":ownership,"DriverOpts":options})),64*1024).await.map_err(|_|fail("input publisher volume creation acknowledgement unavailable"))?; + if volume.status != 201 { + return Err(fail("input publisher volume creation failed")); + } + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", files.volume_name), + None, + 1024 * 1024, + ) + .await + .map_err(|_| fail("input publisher volume inspection failed"))?; + let inspected: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("input publisher volume inspection invalid"))?; + if inspected["Name"] != files.volume_name + || inspected["Driver"] != "local" + || inspected["Options"] != options + || inspected["Labels"] != ownership + { + return Err(fail( + "input publisher exact tmpfs volume ownership mismatch", + )); + } + helper_attempted = true; + let created = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/create?name={name}"), + Some(&body), + 64 * 1024, + ) + .await + .map_err(|_| fail("input publisher creation acknowledgement unavailable"))?; + if created.status == 409 { + helper_attempted = false; + } + if created.status != 201 { + return Err(fail("input publisher creation failed")); + } + let created: Value = serde_json::from_slice(&created.body) + .map_err(|_| fail("input publisher creation invalid"))?; + let id = created["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) + .ok_or_else(|| fail("input publisher identity invalid"))?; + let start = async { + let started = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/start"), + None, + 64 * 1024, + ) + .await + .map_err(|_| fail("input publisher start acknowledgement unavailable"))?; + if started.status != 204 { + return Err(fail("input publisher start failed")); + } + Ok::<(), BackendError>(()) + }; + let accept = async { + listener + .accept() + .await + .map(|(stream, _)| stream) + .map_err(|_| fail("input publisher private exchange accept failed")) + }; + let (mut stream, ()) = tokio::try_join!(accept, start)?; + request( + &mut stream, + &Request::Begin { + source_ids: (0..sources.len() as u16).collect(), + }, + Acknowledgement::Begun, + ) + .await?; + let mut total = 0u64; + for (source_id, path) in sources.iter().enumerate() { + if std::fs::canonicalize(path) + .map_err(|_| fail("approved input source unavailable"))? + != *path + { + return Err(fail("approved input source selection changed")); + } + let file = tokio::fs::File::open(path) + .await + .map_err(|_| fail("approved input source read failed"))?; + if !file + .metadata() + .await + .map_err(|_| fail("approved input source metadata failed"))? + .is_file() + { + return Err(fail("approved input source is not regular")); + } + let mut file = + file.take(MAX_FILE_BYTES.min(MAX_TOTAL_BYTES.saturating_sub(total)) + 1); + request( + &mut stream, + &Request::Open { + source_id: source_id as u16, + }, + Acknowledgement::Opened, + ) + .await?; + let mut count = 0u64; + let mut chunk = vec![0; MAX_CHUNK_BYTES]; + loop { + let read = file + .read(&mut chunk) + .await + .map_err(|_| fail("approved input source read failed"))?; + if read == 0 { + break; + } + count += read as u64; + total += read as u64; + if count > MAX_FILE_BYTES || total > MAX_TOTAL_BYTES { + return Err(fail("approved input source actual bytes exceed profile")); + } + request( + &mut stream, + &Request::Chunk { + bytes: chunk[..read].to_vec(), + }, + Acknowledgement::Written, + ) + .await?; + } + request(&mut stream, &Request::Finish, Acknowledgement::Finished).await?; + } + request(&mut stream, &Request::Seal, Acknowledgement::Populated).await?; + Ok::(stream) + }; + let result = tokio::time::timeout(Duration::from_secs(20), operation) + .await + .unwrap_or_else(|_| Err(fail("input publication timed out"))); + match result { + Ok(stream) => { + self.input_publishers + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))? + .insert( + files.volume_name.clone(), + HeldPublisher { + soul, + incarnation: spec.incarnation_id.to_string(), + name, + expected: body, + context: context.clone(), + references: references.clone(), + stream, + _directory: directory, + }, + ); + tracing::info!(soul_id=%spec.soul_id,generation_id=%inputs.generation_id,source_count=sources.len(),"runtime.input_publication_ready"); + Ok(()) + } + Err(error) => { + if helper_attempted { + self.cleanup_input_helper(&name, &body).await?; + } + if volume_attempted { + self.cleanup_input_volume(&files.volume_name, &ownership, &options) + .await?; + } + Err(error) + } + } + } + + async fn cleanup_input_helper(&self, name: &str, expected: &Value) -> Result<(), BackendError> { + let cleanup = async { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 1024 * 1024, + ) + .await + .map_err(|_| fail("input publisher cleanup inspection failed"))?; + if inspected.status == 404 { + return Ok(()); + } + if inspected.status != 200 { + return Err(fail("input publisher cleanup inspection failed")); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("input publisher cleanup inspection invalid"))?; + if !owned_helper(&actual, expected) { + return Err(fail("input publisher cleanup ownership changed")); + } + let deleted = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/containers/{name}?force=1"), + None, + 64 * 1024, + ) + .await + .map_err(|_| fail("input publisher cleanup failed"))?; + if !matches!(deleted.status, 204 | 404) { + return Err(fail("input publisher cleanup failed")); + } + Ok(()) + }; + tokio::time::timeout(Duration::from_secs(5), cleanup) + .await + .map_err(|_| fail("input publisher cleanup timed out"))? + } + + async fn cleanup_input_volume( + &self, + name: &str, + ownership: &Value, + options: &Value, + ) -> Result<(), BackendError> { + let cleanup = async { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{name}"), + None, + 1024 * 1024, + ) + .await + .map_err(|_| fail("input volume cleanup inspection failed"))?; + if inspected.status == 404 { + return Ok(()); + } + if inspected.status != 200 { + return Err(fail("input volume cleanup inspection failed")); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("input volume cleanup inspection invalid"))?; + if actual["Name"] != name + || actual["Driver"] != "local" + || actual["Labels"] != *ownership + || actual["Options"] != *options + { + return Err(fail("input volume cleanup ownership changed")); + } + let deleted = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/volumes/{name}"), + None, + 64 * 1024, + ) + .await + .map_err(|_| fail("input volume cleanup failed"))?; + if !matches!(deleted.status, 204 | 404) { + return Err(fail( + "input volume cleanup failed; preserve generation for reconciliation", + )); + } + Ok(()) + }; + tokio::time::timeout(Duration::from_secs(5), cleanup) + .await + .map_err(|_| fail("input volume cleanup timed out"))? + } + + pub(super) async fn release_indexed_publication( + &self, + soul: &SoulId, + incarnation: &IncarnationId, + generation: &str, + ) -> Result<(), BackendError> { + let volume = freshell_runtime_protocol::indexed_provider_volume_name(soul, generation); + let held = self + .input_publishers + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))? + .remove(&volume); + let Some(mut held) = held else { + return Ok(()); + }; + if held.soul != soul.as_str() || held.incarnation != incarnation.as_str() { + self.input_publishers + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))? + .insert(volume, held); + return Err(fail("input publisher release belongs to another attempt")); + } + let released = tokio::time::timeout( + Duration::from_secs(5), + request( + &mut held.stream, + &Request::Release, + Acknowledgement::Released, + ), + ) + .await; + let cleanup = self.cleanup_input_helper(&held.name, &held.expected).await; + if cleanup.is_err() { + self.input_publishers + .lock() + .map_err(|_| fail("input publisher ownership unavailable"))? + .insert(volume, held); + } + cleanup?; + if !matches!(released, Ok(Ok(()))) { + tracing::warn!(soul_id=%soul,generation_id=generation,"runtime.input_publication_release_without_ack"); + } + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::{Arc, Mutex}; + use tokio::net::UnixListener; + + #[tokio::test] + async fn provider_input_publisher_streams_only_private_ipc_and_pins_tmpfs_before_workload_creation( + ) { + publication_fixture(false).await; + } + + #[tokio::test] + async fn provider_input_publisher_abort_removes_only_the_pending_owned_generation() { + publication_fixture(true).await; + } + + async fn publication_fixture(abort: bool) { + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("approved-source"); + std::fs::write(&source, b"PRIVATE_PUBLISHER_SENTINEL").unwrap(); + let binary = owned.path().join("session-host"); + std::fs::write(&binary, b"host").unwrap(); + let volume_root = owned.path().join("indexed-volume"); + std::fs::create_dir(&volume_root).unwrap(); + let soul = SoulId::parse("soul-input-publisher").unwrap(); + let generation = "grant-input-publisher"; + let volume_name = + freshell_runtime_protocol::indexed_provider_volume_name(&soul, generation); + let terminal:TerminalLaunchSpec=serde_json::from_value(json!({ + "terminalId":"terminal-input","streamId":"stream-input","mode":"opencode","program":"opencode", + "cwd":"/fixture/workspace","workspacePath":"/fixture/workspace","runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"project-input", + "providerSecretReferences":[{"sourcePath":source,"profile":"opencode_configuration_file","providerRelativePath":".freshell/references/0"}], + "providerLaunchContext":{ + "preparation":{"opencode":{"project_config":[],"tui_config":null}}, + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:3344","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":1,"aliasCount":1,"recipeCount":0}, + "indexedFiles":{"volumeName":volume_name,"aliases":[{"sourceId":0,"logicalDestination":"/outside/selected","effectiveDestination":"/outside/selected"}]}} + } + })).unwrap(); + terminal.validate().unwrap(); + let spec = CreateRuntimeSpec { + installation_id: InstallationId::parse("installation-input").unwrap(), + soul_id: soul, + incarnation_id: IncarnationId::parse("incarnation-input").unwrap(), + image_ref: format!("sha256:{}", "a".repeat(64)), + host_binary_path: binary.clone(), + runtime_dir: owned.path().join("runtime"), + limits: RuntimeLimits { + cpu_milli: 500, + memory_bytes: 128 * 1024 * 1024, + swap_bytes: 0, + pids_max: 64, + }, + test_run_id: "input-publication-fixture".into(), + terminal: Some(terminal), + fresh_agent: None, + provider_volume_name: "freshell-provider-owned-input".into(), + }; + let mounts = docker::TerminalMounts { + workspace: "/fixture/workspace".into(), + git_common_dir: None, + provider_bootstrap_files: vec![], + provider_secret_files: vec![source.clone()], + provider_user_root: None, + mcp_capability_file: None, + input_manifest: None, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: spec + .terminal + .as_ref() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .indexed_files + .clone(), + }; + let socket = owned.path().join("docker.sock"); + let listener = UnixListener::bind(&socket).unwrap(); + let captured = Arc::new(Mutex::new(Vec::<(String, Vec)>::new())); + let seen = captured.clone(); + let created = Arc::new(Mutex::new(None::)); + let volume = Arc::new(Mutex::new(None::)); + let helper = Arc::new(Mutex::new( + None::>>, + )); + let active_helper = helper.clone(); + let private_root = volume_root.clone(); + let created_for_server = created.clone(); + let server = tokio::spawn(async move { + loop { + let (mut stream, _) = listener.accept().await.unwrap(); + let (seen, created, volume, helper, private_root) = ( + seen.clone(), + created_for_server.clone(), + volume.clone(), + active_helper.clone(), + private_root.clone(), + ); + tokio::spawn(async move { + let mut header = Vec::new(); + let mut byte = [0]; + while !header.ends_with(b"\r\n\r\n") { + stream.read_exact(&mut byte).await.unwrap(); + header.push(byte[0]); + } + let header = String::from_utf8(header).unwrap(); + let length = header + .lines() + .find_map(|line| line.strip_prefix("Content-Length: ")) + .unwrap() + .trim() + .parse::() + .unwrap(); + let mut body = vec![0; length]; + stream.read_exact(&mut body).await.unwrap(); + let request = header.lines().next().unwrap().to_string(); + seen.lock().unwrap().push((request.clone(), body.clone())); + let (status, reply) = if request.contains("/images/") { + (200,json!({"Config":{"Env":["PATH=/image/path","INHERITED_TOKEN=PRIVATE_IMAGE_SENTINEL"]}}).to_string()) + } else if request.contains("/volumes/create ") { + let mut value: Value = serde_json::from_slice(&body).unwrap(); + value["Mountpoint"] = json!("/owned/daemon/volume"); + value["Options"] = value["DriverOpts"].clone(); + *volume.lock().unwrap() = Some(value.clone()); + (201, value.to_string()) + } else if request.starts_with("DELETE ") && request.contains("/volumes/") { + *volume.lock().unwrap() = None; + (204, String::new()) + } else if request.contains("/volumes/") { + match volume.lock().unwrap().as_ref() { + Some(value) => (200, value.to_string()), + None => (404, String::new()), + } + } else if request.contains("/containers/create?") { + *created.lock().unwrap() = Some(serde_json::from_slice(&body).unwrap()); + (201, json!({"Id":"b".repeat(64)}).to_string()) + } else if request.contains("/start ") { + let config = created.lock().unwrap().clone().unwrap(); + let directory = config["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .find(|mount| mount["Target"] == "/run/freshell-input-exchange") + .unwrap()["Source"] + .as_str() + .unwrap() + .to_owned(); + *helper.lock().unwrap() = Some(std::thread::spawn(move || { + freshell_platform::provider_input_exchange::serve( + &private_root, + &Path::new(&directory).join("reply.sock"), + ) + })); + (204, String::new()) + } else if request.contains("/json ") { + let mut config = created.lock().unwrap().clone().unwrap(); + // Docker inspection includes normalized default fields. + config["HostConfig"]["Mounts"][0]["Consistency"] = json!("default"); + (200,json!({"Config":config,"HostConfig":config["HostConfig"],"State":{"Status":"running"}}).to_string()) + } else if request.starts_with("DELETE ") { + (204, String::new()) + } else { + panic!("unexpected owned Docker request {request}"); + }; + let response=format!("HTTP/1.1 {status} OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{reply}",reply.len()); + let _ = stream.write_all(response.as_bytes()).await; + }); + } + }); + let backend = DockerEngineBackend::new(&socket); + backend + .begin_indexed_publication(&spec, &mounts, &binary) + .await + .unwrap(); + assert_eq!( + std::fs::read(volume_root.join("files/source-0")).unwrap(), + b"PRIVATE_PUBLISHER_SENTINEL" + ); + assert!( + !helper.lock().unwrap().as_ref().unwrap().is_finished(), + "publisher must pin completed tmpfs before workload creation" + ); + // A lost workload-create acknowledgement retries the same frozen + // preparation. It must reuse the pinned bytes, even after the source + // disappears, rather than publish a second generation. + std::fs::remove_file(&source).unwrap(); + let before_retry = captured.lock().unwrap().len(); + backend + .begin_indexed_publication(&spec, &mounts, &binary) + .await + .unwrap(); + assert_eq!(captured.lock().unwrap().len(), before_retry); + let mut changed = spec.clone(); + changed + .terminal + .as_mut() + .unwrap() + .provider_launch_context + .as_mut() + .unwrap() + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .sha256 = "b".repeat(64); + assert!(backend + .begin_indexed_publication(&changed, &mounts, &binary) + .await + .is_err()); + assert_eq!(captured.lock().unwrap().len(), before_retry); + for (_, body) in captured.lock().unwrap().iter() { + assert!(!String::from_utf8_lossy(body).contains("PRIVATE_PUBLISHER_SENTINEL")); + assert!(!String::from_utf8_lossy(body).contains("PRIVATE_IMAGE_SENTINEL")); + } + assert!(created.lock().unwrap().as_ref().unwrap()["Env"] + .as_array() + .unwrap() + .contains(&json!("INHERITED_TOKEN="))); + if abort { + backend.abort_indexed_publication(&spec).await.unwrap(); + } else { + backend + .release_indexed_publication(&spec.soul_id, &spec.incarnation_id, generation) + .await + .unwrap(); + } + assert_eq!( + captured + .lock() + .unwrap() + .iter() + .filter( + |(request, _)| request.starts_with("DELETE ") && request.contains("/volumes/") + ) + .count(), + usize::from(abort) + ); + assert!(helper + .lock() + .unwrap() + .take() + .unwrap() + .join() + .unwrap() + .is_ok()); + drop(backend); + server.abort(); + } +} diff --git a/crates/freshell-supervisor/src/backend/native_auth.rs b/crates/freshell-supervisor/src/backend/native_auth.rs index cad6c3e79..396c770d3 100644 --- a/crates/freshell-supervisor/src/backend/native_auth.rs +++ b/crates/freshell-supervisor/src/backend/native_auth.rs @@ -7,12 +7,18 @@ fn fail(cause: &'static str) -> BackendError { BackendError::Unavailable(cause.into()) } -struct ExchangeDirectory(PathBuf); +pub(super) struct ExchangeDirectory(pub(super) PathBuf); impl ExchangeDirectory { fn create() -> Result { + Self::create_owned("freshell-auth") + } + pub(super) fn create_input() -> Result { + Self::create_owned("freshell-inputs") + } + fn create_owned(prefix: &str) -> Result { use std::os::unix::ffi::OsStrExt; // Deliberately short regardless of runtime_root length (Linux sun_path). - let path = std::env::temp_dir().join(format!("freshell-auth-{}", uuid::Uuid::new_v4())); + let path = std::env::temp_dir().join(format!("{prefix}-{}", uuid::Uuid::new_v4())); if path.join("reply.sock").as_os_str().as_bytes().len() > 107 { return Err(fail("native auth exchange path exceeds limit")); } @@ -21,7 +27,7 @@ impl ExchangeDirectory { owned.mode(&owned.0, 0o710)?; Ok(owned) } - fn mode(&self, path: &Path, mode: u32) -> Result<(), BackendError> { + pub(super) fn mode(&self, path: &Path, mode: u32) -> Result<(), BackendError> { use std::os::unix::{ ffi::OsStrExt, fs::{MetadataExt, PermissionsExt}, @@ -154,9 +160,12 @@ impl DockerEngineBackend { let listener = UnixListener::bind(&socket) .map_err(|_| fail("native auth exchange bind failed"))?; directory.mode(&socket, 0o660)?; + let helper_env = self + .private_helper_environment(handle.image_ref(), &["HOME=/home/freshell/provider"]) + .await?; let body = json!({ "Image":handle.image_ref(),"User":"65534:0","Tty":false, - "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["native-auth-only"],"Env":["HOME=/home/freshell/provider"], + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["native-auth-only"],"Env":helper_env, "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"SecurityOpt":["no-new-privileges"], "Memory":256*1024*1024,"MemorySwap":256*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, "LogConfig":{"Type":"none","Config":{}}, @@ -241,15 +250,7 @@ impl DockerEngineBackend { let expected = expected .as_ref() .ok_or_else(|| fail("native auth cleanup authority unavailable"))?; - if ["Image", "User", "Tty", "Entrypoint", "Cmd", "Env"] - .iter() - .any(|key| actual["Config"][key] != expected[key]) - || actual["Config"]["Labels"] - .as_object() - .is_some_and(|labels| !labels.is_empty()) - || actual["HostConfig"]["Mounts"] != expected["HostConfig"]["Mounts"] - || actual["HostConfig"]["LogConfig"]["Type"] != "none" - { + if !input_publisher::owned_helper(&actual, expected) { return Err(fail("native auth cleanup ownership changed")); } let deleted = self @@ -364,6 +365,8 @@ mod tests { ) } else if request.contains("provider-container-must-not-stop/json") { (404, String::new()) + } else if request.contains("/images/") { + (200,json!({"Config":{"Env":["PATH=/image/path","INHERITED_TOKEN=PRIVATE_IMAGE_SENTINEL"]}}).to_string()) } else if request.contains("/volumes/") { (200, json!({"Name":"owned-native-volume"}).to_string()) } else if request.contains("/create?") { @@ -455,6 +458,11 @@ mod tests { assert_eq!(body["Cmd"], json!(["native-auth-only"])); assert_eq!(body["HostConfig"]["LogConfig"]["Type"], "none"); assert_eq!(body["User"], "65534:0"); + assert!(body["Env"] + .as_array() + .unwrap() + .contains(&json!("INHERITED_TOKEN="))); + assert!(!String::from_utf8_lossy(&create.1).contains("PRIVATE_IMAGE_SENTINEL")); assert_eq!(body["HostConfig"]["Mounts"].as_array().unwrap().len(), 3); assert!(records.iter().all(|(request, _)| !request.contains("/logs") && !(request.starts_with("DELETE") && request.contains(handle.container_id())) diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 2e37a2a2f..e5d36ebd3 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -1870,6 +1870,20 @@ impl Supervisor { ) .await?; } + if context + .and_then(|context| context.prepared_inputs.as_ref()) + .is_some() + { + let handle = self + .registry + .owned_handle(incarnation.clone()) + .await + .map_err(map_registry)?; + self.backend + .commit_prepared_inputs(&handle) + .await + .map_err(map_backend)?; + } Ok(()) } From e803ebb2cbbc307e5de5a932d4f50019ec8c48ca Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 06:41:46 -0700 Subject: [PATCH 025/493] Allow selected configuration paths within the preparation frame budget --- crates/freshell-runtime-protocol/src/lib.rs | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 09fef8182..4f0baa16d 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -1541,6 +1541,13 @@ mod provider_launch_context_tests { .unwrap(); fresh.provider_secret_references = references; assert!(fresh.validate().is_ok()); + // A valid filesystem path can exceed the legacy credential reference + // limit. Configuration references use the preparation frame budget. + let deep_source = format!("/private/{}/selected", "directory/".repeat(205)); + terminal.provider_secret_references[0].source_path = deep_source.clone(); + fresh.provider_secret_references[0].source_path = deep_source; + assert!(terminal.validate().is_ok()); + assert!(fresh.validate().is_ok()); } #[test] fn configuration_file_grants_reject_cross_provider_duplicate_and_unsafe_destinations() { @@ -2258,7 +2265,12 @@ pub fn validate_provider_secrets( for reference in references { if reference.profile.provider() != provider || !std::path::Path::new(&reference.source_path).is_absolute() - || reference.source_path.len() > 1024 + || reference.source_path.len() + > if reference.profile.is_configuration_file() { + MAX_CONTROL_FRAME_BYTES + } else { + 1024 + } || reference.source_path.chars().any(char::is_control) || matches!( reference.profile, From 1262197a8c5837169bff4d1d3f1806c435769a1c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 06:54:23 -0700 Subject: [PATCH 026/493] Select Codex typed file inputs by declaring layer and role origin --- Cargo.lock | 1 + crates/freshell-server/Cargo.toml | 2 + .../src/managed_mcp_capability.rs | 16 +- .../src/managed_provider_references.rs | 1 + .../src/managed_provider_references/codex.rs | 380 ++++++++++++++++++ 5 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 crates/freshell-server/src/managed_provider_references/codex.rs diff --git a/Cargo.lock b/Cargo.lock index 8732fa6c2..ffd4847fc 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1473,6 +1473,7 @@ dependencies = [ "tempfile", "tokio", "tokio-tungstenite 0.24.0", + "toml 0.9.12+spec-1.1.0", "tower", "tracing", "tracing-subscriber", diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index 5dfc48388..0bc10cb7c 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -22,6 +22,7 @@ managed-runtime-v1 = [ "dep:freshell-agent-runtime", "dep:freshell-runtime-client", "dep:freshell-runtime-protocol", + "dep:toml", ] managed-provider-qualification = [ "managed-runtime-v1", @@ -73,6 +74,7 @@ tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "proces axum = "0.8" async-trait = "0.1" serde_json = { workspace = true } +toml = { version = "=0.9.12", optional = true } # Boot-scoped identifiers: `srv-` / `boot-`. uuid = { version = "1", features = ["v4"] } # The browser-pane HTTP reverse proxy (`/api/proxy/http/{port}/*`) forwards to a diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index b435d79d0..decd9157b 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1687,14 +1687,26 @@ mod tests { } } + #[cfg(unix)] + fn executable_fixture_directory() -> tempfile::TempDir { + match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(directory) => tempfile::tempdir_in(directory), + None => tempfile::tempdir(), + } + .unwrap() + } + #[cfg(unix)] #[test] fn managed_provider_resource_script_and_dependency_execute() { use std::{os::unix::fs::PermissionsExt, process::Command}; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture_directory(); let source = root.path().join("source"); std::fs::create_dir_all(source.join("scripts")).unwrap(); std::fs::create_dir_all(source.join("node_modules/fixture-dependency")).unwrap(); + // Keep this transported package's module kind explicit when its + // disposable fixture lives below the repository's ESM package. + std::fs::write(source.join("package.json"), r#"{"type":"commonjs"}"#).unwrap(); std::fs::write(source.join("data"), "resource-nonce\n").unwrap(); std::fs::write( source.join("scripts/run"), @@ -1747,7 +1759,7 @@ mod tests { #[test] fn managed_provider_two_stage_script_retains_execute_permission() { use std::{os::unix::fs::PermissionsExt, process::Command}; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture_directory(); let source = root.path().join("source"); std::fs::create_dir(&source).unwrap(); std::fs::write(source.join("run"), "#!/bin/sh\nprintf resource-nonce\n").unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index e58924fd1..d3c7bb6d7 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -2,6 +2,7 @@ //! Plans retain approved source metadata and unexpanded templates, never payload bytes. use std::path::{Path, PathBuf}; +pub(crate) mod codex; pub(crate) mod sources; pub struct FileSourceOrigin<'a> { diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs new file mode 100644 index 000000000..462f1a814 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -0,0 +1,380 @@ +//! Finite Codex typed inputs, resolved per declaring logical TOML layer. +use std::path::{Path, PathBuf}; + +#[derive(Clone, Copy, PartialEq, Eq)] +pub enum InputKind { + Configuration, + Instructions, + CompactPrompt, + ModelCatalog, + SkillSelector, +} +pub struct SelectedInput { + pub logical_path: PathBuf, + pub canonical_source: PathBuf, + pub kind: InputKind, +} + +#[derive(Clone, Copy, PartialEq, Eq)] +pub enum LayerKind { + User, + Project, + Role, +} +pub struct SourceLayer { + pub logical_path: PathBuf, + pub enabled: bool, + pub kind: LayerKind, +} + +pub fn select_inputs( + layers: &[SourceLayer], + home: &Path, + profile: Option<&str>, +) -> Result, &'static str> { + let mut selection = Selection { + home, + profile, + inputs: Vec::new(), + active: std::collections::BTreeSet::new(), + metadata_bytes: 0, + }; + for layer in layers { + selection.configuration(&layer.logical_path, layer.enabled, layer.kind, 0)?; + } + Ok(selection.inputs) +} + +struct Selection<'a> { + home: &'a Path, + profile: Option<&'a str>, + inputs: Vec, + active: std::collections::BTreeSet<(PathBuf, PathBuf)>, + metadata_bytes: usize, +} + +impl Selection<'_> { + fn add(&mut self, logical_path: PathBuf, kind: InputKind) -> Result { + if self.inputs.len() >= 8192 { + return Err("Codex selected input event budget exceeded"); + } + let canonical_source = std::fs::canonicalize(&logical_path) + .map_err(|_| "Codex selected input is missing, inaccessible or cyclic")?; + let metadata = std::fs::metadata(&canonical_source) + .map_err(|_| "Codex selected input metadata unavailable")?; + if !metadata.is_file() { + return Err("Codex selected input must be a regular file"); + } + self.metadata_bytes = self + .metadata_bytes + .checked_add(logical_path.as_os_str().len()) + .and_then(|bytes| bytes.checked_add(canonical_source.as_os_str().len())) + .ok_or("Codex selected input metadata size overflow")?; + if self.metadata_bytes > 8 * 1024 * 1024 { + return Err("Codex selected input metadata budget exceeded"); + } + self.inputs.push(SelectedInput { + logical_path, + canonical_source: canonical_source.clone(), + kind, + }); + Ok(canonical_source) + } + + fn configuration( + &mut self, + path: &Path, + enabled: bool, + kind: LayerKind, + depth: usize, + ) -> Result<(), &'static str> { + use std::io::Read; + if depth > 64 { + return Err("Codex role configuration depth exceeded"); + } + let logical = normalize(path); + let canonical = self.add(logical.clone(), InputKind::Configuration)?; + if !enabled { + return Ok(()); + } + let identity = (canonical.clone(), logical.clone()); + if !self.active.insert(identity.clone()) { + return Err("Codex role configuration cycle detected"); + } + let result = (|| { + let mut raw = String::new(); + std::fs::File::open(&canonical) + .map_err(|_| "Codex selected configuration unreadable")? + .take(16 * 1024 * 1024 + 1) + .read_to_string(&mut raw) + .map_err(|_| "Codex selected configuration is unreadable or not UTF-8")?; + if raw.len() > 16 * 1024 * 1024 { + return Err("Codex selected configuration source view exceeded"); + } + let value: toml::Value = + toml::from_str(&raw).map_err(|_| "Codex selected configuration TOML is invalid")?; + let parent = logical + .parent() + .ok_or("Codex declaring configuration parent unavailable")?; + self.file_fields(&value, parent)?; + if let Some(profile) = self.profile.filter(|_| kind == LayerKind::User) { + if let Some(value) = value + .get("profiles") + .and_then(|profiles| profiles.get(profile)) + { + self.file_fields(value, parent)?; + } + } + let mut role_paths = Vec::new(); + let mut role_path_bytes = 0usize; + if let Some(roles) = value.get("agents").and_then(toml::Value::as_table) { + if roles.len() > 8192 - self.inputs.len() { + return Err("Codex selected role event budget exceeded"); + } + for role in roles.values() { + if let Some(path) = string_field(role, "config_file")? { + let path = typed_path(path, parent, self.home); + role_path_bytes = role_path_bytes + .checked_add(path.as_os_str().len()) + .ok_or("Codex selected role metadata overflow")?; + if role_path_bytes > 8 * 1024 * 1024 - self.metadata_bytes { + return Err("Codex selected role metadata budget exceeded"); + } + role_paths.push(path); + } + } + } + if let Some(rules) = value + .get("skills") + .and_then(|skills| skills.get("config")) + .filter(|_| kind == LayerKind::User) + { + let rules = rules + .as_array() + .ok_or("Codex skills config has an unsupported type")?; + for rule in rules { + if let Some(path) = string_field(rule, "path")? { + self.add( + typed_path(path, parent, self.home), + InputKind::SkillSelector, + )?; + } + } + } + // Retain only finite edge metadata across recursion. Native TOML + // documents and raw source views leave scope before loading roles. + drop(value); + drop(raw); + for role in role_paths { + self.configuration(&role, true, LayerKind::Role, depth + 1)?; + } + Ok(()) + })(); + self.active.remove(&identity); + result + } + + fn file_fields(&mut self, value: &toml::Value, parent: &Path) -> Result<(), &'static str> { + for (field, kind) in [ + ("model_instructions_file", InputKind::Instructions), + ("experimental_compact_prompt_file", InputKind::CompactPrompt), + ("model_catalog_json", InputKind::ModelCatalog), + ] { + if let Some(path) = string_field(value, field)? { + self.add(typed_path(path, parent, self.home), kind)?; + } + } + Ok(()) + } +} + +fn string_field<'a>(value: &'a toml::Value, field: &str) -> Result, &'static str> { + value + .get(field) + .map(|value| { + value + .as_str() + .ok_or("Codex selected typed input has an unsupported type") + }) + .transpose() +} + +fn typed_path(value: &str, parent: &Path, home: &Path) -> PathBuf { + let selected = if value == "~" { + home.to_owned() + } else if let Some(suffix) = value.strip_prefix("~/") { + home.join(suffix.trim_start_matches('/')) + } else { + PathBuf::from(value) + }; + normalize(&if selected.is_absolute() { + selected + } else { + parent.join(selected) + }) +} + +fn normalize(path: &Path) -> PathBuf { + use std::path::Component; + let mut result = PathBuf::new(); + for component in path.components() { + match component { + Component::CurDir => {} + Component::ParentDir => { + result.pop(); + } + other => result.push(other), + } + } + result +} + +#[cfg(test)] +mod tests { + use super::*; + use sha2::{Digest, Sha256}; + fn digest(path: &Path) -> String { + format!("{:x}", Sha256::digest(std::fs::read(path).unwrap())) + } + fn layer(path: PathBuf, enabled: bool) -> SourceLayer { + SourceLayer { + logical_path: path, + enabled, + kind: LayerKind::User, + } + } + #[test] + fn managed_provider_codex_typed_inputs_keep_each_logical_role_and_profile_origin() { + let fixture = tempfile::tempdir().unwrap(); + let home = fixture.path().join("home"); + let logical = fixture.path().join("project/.codex"); + let actual = fixture.path().join("storage"); + for path in [&home, &logical, &actual] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::create_dir(logical.join("roles")).unwrap(); + std::fs::create_dir(logical.join("skills")).unwrap(); + std::fs::write(logical.join("instructions.md"), "logical-layer").unwrap(); + std::fs::write(actual.join("instructions.md"), "wrong-storage-origin").unwrap(); + std::fs::write(logical.join("roles/instructions.md"), "logical-role").unwrap(); + std::fs::write(logical.join("profile.md"), "selected-profile").unwrap(); + std::fs::write(logical.join("skills/SKILL.md"), "selected-skill").unwrap(); + std::fs::write( + actual.join("layer.toml"), + r#" +model_instructions_file = "./instructions.md" +notify = ["./missing-command", "./missing-argument"] +[mcp_servers.fixture] +command = "./missing-command" +args = ["./missing-argument"] +cwd = "./missing-cwd" +[agents.worker] +config_file = "./roles/worker.toml" +[profiles.chosen] +experimental_compact_prompt_file = "./profile.md" +[profiles.unselected] +model_instructions_file = "./missing-unselected.md" +[[skills.config]] +path = "./skills/SKILL.md" +enabled = false +"#, + ) + .unwrap(); + std::fs::write( + actual.join("role.toml"), + "model_instructions_file='./instructions.md'", + ) + .unwrap(); + std::os::unix::fs::symlink(actual.join("layer.toml"), logical.join("config.toml")).unwrap(); + std::os::unix::fs::symlink(actual.join("role.toml"), logical.join("roles/worker.toml")) + .unwrap(); + let selected = select_inputs( + &[layer(logical.join("config.toml"), true)], + &home, + Some("chosen"), + ) + .unwrap(); + let instructions = selected + .iter() + .filter(|input| input.kind == InputKind::Instructions) + .map(|input| digest(&input.canonical_source)) + .collect::>(); + assert_eq!( + instructions, + vec![ + digest(&logical.join("instructions.md")), + digest(&logical.join("roles/instructions.md")) + ] + ); + assert!(selected + .iter() + .any(|input| input.kind == InputKind::CompactPrompt + && digest(&input.canonical_source) == digest(&logical.join("profile.md")))); + assert!(selected + .iter() + .any(|input| input.kind == InputKind::SkillSelector + && input.logical_path == logical.join("skills/SKILL.md"))); + assert_eq!( + std::fs::read_to_string(actual.join("role.toml")).unwrap(), + "model_instructions_file='./instructions.md'" + ); + } + + #[test] + fn managed_provider_codex_typed_inputs_respect_disabled_layers_and_classify_cycles_and_parse_errors( + ) { + let fixture = tempfile::tempdir().unwrap(); + let path = fixture.path().join("config.toml"); + std::fs::write(&path, "[agents.loop]\nconfig_file='./config.toml'\n").unwrap(); + assert_eq!( + select_inputs(&[layer(path.clone(), true)], fixture.path(), None).err(), + Some("Codex role configuration cycle detected") + ); + std::fs::write(&path, "model_instructions_file='./missing'\n").unwrap(); + let disabled = select_inputs(&[layer(path.clone(), false)], fixture.path(), None).unwrap(); + assert_eq!(disabled.len(), 1); + assert!(disabled[0].kind == InputKind::Configuration); + std::fs::write( + &path, + "model_instructions_file='PRIVATE_INVALID_TOML_SENTINEL\n", + ) + .unwrap(); + let error = select_inputs(&[layer(path, true)], fixture.path(), None) + .err() + .unwrap(); + assert_eq!(error, "Codex selected configuration TOML is invalid"); + assert!(!error.contains("PRIVATE_INVALID_TOML_SENTINEL")); + } + + #[test] + fn managed_provider_codex_project_layer_cannot_promote_user_profiles_or_skill_rules() { + let fixture = tempfile::tempdir().unwrap(); + let path = fixture.path().join("config.toml"); + std::fs::write(fixture.path().join("project.md"), "project-input").unwrap(); + std::fs::write( + &path, + r#"model_instructions_file='./project.md' +[profiles.chosen] +model_instructions_file='./missing-profile-input' +[[skills.config]] +path='./missing-user-only-selector' +"#, + ) + .unwrap(); + let selected = select_inputs( + &[SourceLayer { + logical_path: path, + enabled: true, + kind: LayerKind::Project, + }], + fixture.path(), + Some("chosen"), + ) + .unwrap(); + assert_eq!(selected.len(), 2); + assert_eq!( + digest(&selected[1].canonical_source), + digest(&fixture.path().join("project.md")) + ); + } +} From 04d162155d6bf1b975e1631f628e3dda448693cb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 07:07:02 -0700 Subject: [PATCH 027/493] Consume published provider input generations through private subpath mounts --- crates/freshell-session-host/src/main.rs | 166 ++++++++++++++---- crates/freshell-supervisor/src/backend.rs | 8 + .../freshell-supervisor/src/backend/docker.rs | 13 +- .../src/backend/input_publisher.rs | 1 + .../src/backend/prepared_inputs.rs | 48 +++++ 5 files changed, 201 insertions(+), 35 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 6551370c9..dd1600bf5 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -941,6 +941,11 @@ async fn grant_execution( &resolved_secrets.configuration_files, terminal.run_as_uid, terminal.run_as_gid, + terminal + .provider_launch_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + .is_some_and(|inputs| inputs.indexed_files.is_some()), ) .map_err(|error| { RuntimeError::new( @@ -1060,6 +1065,11 @@ async fn grant_execution( &resolved_secrets.configuration_files, launch.run_as_uid, launch.run_as_gid, + launch + .provider_launch_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + .is_some_and(|inputs| inputs.indexed_files.is_some()), ) .map_err(|error| { RuntimeError::new( @@ -1991,6 +2001,7 @@ fn prepare_provider_files( configuration_files: &[(String, PathBuf)], run_as_uid: u32, run_as_gid: u32, + indexed_configuration: bool, ) -> Result<(), String> { prepare_provider_files_at( Path::new("/home/freshell/provider"), @@ -2000,6 +2011,7 @@ fn prepare_provider_files( configuration_files, run_as_uid, run_as_gid, + indexed_configuration, ) } @@ -2011,12 +2023,15 @@ fn prepare_provider_files_at( configuration_files: &[(String, PathBuf)], run_as_uid: u32, run_as_gid: u32, + indexed_configuration: bool, ) -> Result<(), String> { if provider_files.is_empty() && configuration_files.is_empty() { return Ok(()); } - std::fs::create_dir_all(auth_dir).map_err(|error| error.to_string())?; - set_mode(auth_dir, 0o711)?; + if !provider_files.is_empty() || !indexed_configuration { + std::fs::create_dir_all(auth_dir).map_err(|error| error.to_string())?; + set_mode(auth_dir, 0o711)?; + } // Retain native auth until a matching typed grant has been written and // owned. The final destination replacement occurs only for that grant. for (index, (relative, contents)) in provider_files.iter().enumerate() { @@ -2062,6 +2077,29 @@ fn prepare_provider_files_at( let mut aggregate = 0u64; for (index, (relative, source)) in configuration_files.iter().enumerate() { let destination = home.join(relative); + if indexed_configuration { + use std::io::Read; + let metadata = std::fs::symlink_metadata(source) + .map_err(|_| "indexed configuration-file mount unavailable")?; + if !metadata.is_file() { + return Err("indexed configuration-file mount is not regular".into()); + } + let bytes = std::io::copy( + &mut std::fs::File::open(source) + .map_err(|_| "indexed configuration-file mount unreadable")? + .take(16 * 1024 * 1024 + 1), + &mut std::io::sink(), + ) + .map_err(|_| "indexed configuration-file read failed")?; + aggregate = aggregate + .checked_add(bytes) + .ok_or("indexed configuration-file size overflow")?; + if bytes > 16 * 1024 * 1024 || aggregate > 32 * 1024 * 1024 { + return Err("indexed configuration-file actual bytes exceed profile".into()); + } + publish_configuration_link(&destination, source, run_as_uid, run_as_gid)?; + continue; + } let target = auth_dir.join(format!("configuration-{index}")); let temporary = auth_dir.join(format!( "configuration-{index}-tmp-{}", @@ -2093,38 +2131,7 @@ fn prepare_provider_files_at( .map_err(|_| "configuration-file ownership failed")?; std::fs::rename(&temporary, &target) .map_err(|_| "configuration-file publication failed")?; - let parent = destination - .parent() - .ok_or("configuration-file destination has no parent")?; - run_provider_path_command( - run_as_uid, - run_as_gid, - "/usr/bin/mkdir", - &[ - "-p", - "--", - parent - .to_str() - .ok_or("configuration-file parent is not UTF-8")?, - ], - ) - .map_err(|_| "configuration-file parent creation failed")?; - run_provider_path_command( - run_as_uid, - run_as_gid, - "/usr/bin/ln", - &[ - "-sfnT", - "--", - target - .to_str() - .ok_or("configuration-file target is not UTF-8")?, - destination - .to_str() - .ok_or("configuration-file destination is not UTF-8")?, - ], - ) - .map_err(|_| "configuration-file link publication failed")?; + publish_configuration_link(&destination, &target, run_as_uid, run_as_gid)?; Ok(()) })(); if result.is_err() { @@ -2135,6 +2142,46 @@ fn prepare_provider_files_at( Ok(()) } +fn publish_configuration_link( + destination: &Path, + target: &Path, + uid: u32, + gid: u32, +) -> Result<(), &'static str> { + let parent = destination + .parent() + .ok_or("configuration-file destination has no parent")?; + run_provider_path_command( + uid, + gid, + "/usr/bin/mkdir", + &[ + "-p", + "--", + parent + .to_str() + .ok_or("configuration-file parent is not UTF-8")?, + ], + ) + .map_err(|_| "configuration-file parent creation failed")?; + run_provider_path_command( + uid, + gid, + "/usr/bin/ln", + &[ + "-sfnT", + "--", + target + .to_str() + .ok_or("configuration-file target is not UTF-8")?, + destination + .to_str() + .ok_or("configuration-file destination is not UTF-8")?, + ], + ) + .map_err(|_| "configuration-file link publication failed") +} + fn run_provider_path_command( uid: u32, gid: u32, @@ -2855,6 +2902,7 @@ mod tests { &[], uid, gid, + false, ) .unwrap(); assert_eq!(std::fs::read_to_string(&auth).unwrap(), native); @@ -2866,6 +2914,7 @@ mod tests { &[], uid, gid, + false, ) .unwrap(); assert_eq!(std::fs::read_to_string(&auth).unwrap(), native); @@ -2878,6 +2927,7 @@ mod tests { &[], uid, gid, + false, ) .unwrap(); assert_eq!(std::fs::read(&auth).unwrap(), replacement); @@ -2888,6 +2938,54 @@ mod tests { } use super::*; + #[cfg(unix)] + #[test] + fn provider_secret_indexed_configuration_consumer_keeps_the_mounted_inode_without_a_second_copy( + ) { + use sha2::{Digest, Sha256}; + use std::os::unix::fs::MetadataExt; + let fixture = tempfile::tempdir().unwrap(); + let home = fixture.path().join("home"); + let private = fixture.path().join("legacy-private-copy"); + let mounted = fixture.path().join("published-index-0"); + std::fs::create_dir(&home).unwrap(); + std::fs::write(&mounted, b"PRIVATE_INDEXED_GENERATION_SENTINEL").unwrap(); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + prepare_provider_files_at( + &home, + &private, + "opencode", + &[], + &[(".freshell/references/0".into(), mounted.clone())], + uid, + gid, + true, + ) + .unwrap(); + let selected = home.join(".freshell/references/0"); + let receipt=std::process::Command::new("node").env_clear() + .env("PATH",std::env::var_os("PATH").unwrap()) + .args(["-e","process.stdout.write(require('crypto').createHash('sha256').update(require('fs').readFileSync(process.argv[1])).digest('hex'))"]) + .arg(&selected).output().unwrap(); + assert!(receipt.status.success()); + assert_eq!( + String::from_utf8(receipt.stdout).unwrap(), + format!( + "{:x}", + Sha256::digest(b"PRIVATE_INDEXED_GENERATION_SENTINEL") + ) + ); + assert_eq!( + std::fs::metadata(&selected).unwrap().ino(), + std::fs::metadata(&mounted).unwrap().ino() + ); + assert_eq!(std::fs::read_link(&selected).unwrap(), mounted); + assert!( + !private.exists(), + "indexed publication must not make another private payload copy" + ); + } + #[cfg(unix)] #[test] fn provider_config_two_stage_script_is_runnable() { diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 68bccb68b..bfe92221e 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -697,6 +697,9 @@ impl RuntimeBackend for DockerEngineBackend { )); } for (index, source) in mounts.provider_secret_files.iter().enumerate() { + if mounts.indexed_secret_sources.contains_key(&index) { + continue; + } binds.push(format!( "{}:/run/freshell-secrets/provider-{index}:ro", source.display() @@ -1389,6 +1392,11 @@ fn verify_workload_mounts( } } for (index, source) in expected.provider_secret_files.iter().enumerate() { + // The prepared mount verifier establishes the owned tmpfs/subpath + // identity for these inputs; they are no longer controller binds. + if expected.indexed_secret_sources.contains_key(&index) { + continue; + } let source_text = source.to_string_lossy(); let destination = format!("/run/freshell-secrets/provider-{index}"); let found = mounts.iter().any(|mount| { diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 988b52f98..b9e4d5909 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -37,6 +37,7 @@ pub struct TerminalMounts { pub git_common_dir: Option, pub provider_bootstrap_files: Vec, pub provider_secret_files: Vec, + pub indexed_secret_sources: std::collections::BTreeMap, pub provider_user_root: Option, pub mcp_capability_file: Option, pub input_manifest: Option, @@ -129,7 +130,12 @@ fn workload_mounts( provider_bootstrap_files.push(source); } let mut provider_secret_files = Vec::with_capacity(secret_references.len()); - for secret in secret_references { + let mut indexed_secret_sources = std::collections::BTreeMap::new(); + let indexed = context + .and_then(|context| context.prepared_inputs.as_ref()) + .is_some_and(|inputs| inputs.indexed_files.is_some()); + let mut source_id = 0u16; + for (index, secret) in secret_references.iter().enumerate() { let source = std::fs::canonicalize(&secret.source_path) .map_err(|error| format!("provider secret reference: {error}"))?; if !source.is_file() || source != Path::new(&secret.source_path) { @@ -151,6 +157,10 @@ fn workload_mounts( } reject_management_path(&source)?; provider_secret_files.push(source); + if indexed && secret.profile.is_configuration_file() { + indexed_secret_sources.insert(index, source_id); + source_id += 1; + } } let provider_user_root = if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { @@ -218,6 +228,7 @@ fn workload_mounts( git_common_dir, provider_bootstrap_files, provider_secret_files, + indexed_secret_sources, provider_user_root, mcp_capability_file, input_manifest: prepared.input_manifest, diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index 34ce35988..f6093dcc9 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -722,6 +722,7 @@ mod tests { git_common_dir: None, provider_bootstrap_files: vec![], provider_secret_files: vec![source.clone()], + indexed_secret_sources: BTreeMap::from([(0, 0)]), provider_user_root: None, mcp_capability_file: None, input_manifest: None, diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs index 308bc7912..344402892 100644 --- a/crates/freshell-supervisor/src/backend/prepared_inputs.rs +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -34,6 +34,19 @@ pub(super) fn mount_descriptors( )?; } if let Some(files) = &mounts.indexed_files { + for (index, source_id) in &mounts.indexed_secret_sources { + if *index >= mounts.provider_secret_files.len() || *source_id >= 128 { + return Err(BackendError::InvalidConfig( + "prepared private input index is invalid".into(), + )); + } + let destination = format!("/run/freshell-secrets/provider-{index}"); + add( + &destination, + json!({"Type":"volume","Source":files.volume_name,"Target":destination,"ReadOnly":true, + "VolumeOptions":{"NoCopy":true,"Subpath":format!("files/source-{source_id}")}}), + )?; + } for alias in &files.aliases { add( &alias.effective_destination, @@ -41,6 +54,10 @@ pub(super) fn mount_descriptors( "VolumeOptions":{"NoCopy":true,"Subpath":format!("files/source-{}",alias.source_id)}}), )?; } + } else if !mounts.indexed_secret_sources.is_empty() { + return Err(BackendError::InvalidConfig( + "prepared private input volume is missing".into(), + )); } Ok(selected.into_values().collect()) } @@ -138,6 +155,7 @@ mod tests { git_common_dir: None, provider_bootstrap_files: vec![], provider_secret_files: vec![], + indexed_secret_sources: BTreeMap::new(), provider_user_root: None, mcp_capability_file: None, input_manifest: None, @@ -183,6 +201,36 @@ mod tests { ]); (mounts, actual, volumes) } + + #[test] + fn provider_prepared_indexed_secret_read_uses_the_published_generation() { + let (mut mounts, mut actual, volumes) = fixture(); + mounts.provider_secret_files = vec!["/approved/controller-source".into()]; + mounts.indexed_secret_sources.insert(0, 0); + let descriptors = mount_descriptors(&mounts, "owned-provider").unwrap(); + let private=descriptors.iter().find(|mount|mount["Target"]=="/run/freshell-secrets/provider-0") + .expect("host input must bind the already-published generation, not the changing controller source"); + assert_eq!(private["Type"], "volume"); + assert_eq!( + private["Source"], + "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa" + ); + assert_eq!(private["VolumeOptions"]["Subpath"], "files/source-0"); + assert_eq!(private["ReadOnly"], true); + actual["HostConfig"]["Mounts"] = json!(descriptors); + actual["Mounts"].as_array_mut().unwrap().push(json!({"Type":"volume","Name":"freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa","Source":"/docker/volumes/freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa/_data","Destination":"/run/freshell-secrets/provider-0","RW":false})); + verify(&actual, &mounts, "owned-provider", &volumes).unwrap(); + actual["Mounts"].as_array_mut().unwrap().push(json!({"Type":"bind","Source":"/fixture/workspace","Destination":"/fixture/workspace","RW":true})); + super::super::verify_workload_mounts(actual["Mounts"].as_array().unwrap(), &mounts) + .unwrap(); + actual["HostConfig"]["Mounts"] + .as_array_mut() + .unwrap() + .iter_mut() + .find(|mount| mount["Target"] == "/run/freshell-secrets/provider-0") + .unwrap()["VolumeOptions"]["Subpath"] = json!("files/source-1"); + assert!(verify(&actual, &mounts, "owned-provider", &volumes).is_err()); + } #[test] fn provider_prepared_adoption_checks_subpaths_nocopy_and_actual_volume_options() { let (mounts, actual, volumes) = fixture(); From 9c96d0895529b95132f7fe3c10670acf09f2cfc4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 08:03:53 -0700 Subject: [PATCH 028/493] Constrain prepared input mounts and publisher operations in runtime test broker --- scripts/testing/runtime-test-broker.ts | 275 +++++++++++++++++- .../testing/runtime-test-broker.test.ts | 164 ++++++++++- 2 files changed, 433 insertions(+), 6 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index a88a82b9d..c326e564d 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -6,6 +6,76 @@ import path from 'node:path' const DOCKER_API_PREFIX = '/v1.47' const MAX_REQUEST_BYTES = 2 * 1024 * 1024 +const INDEXED_TMPFS_OPTIONS = 'size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec' + +function indexedLabels(input: PreparedInputAuthorization): Record { + return { + 'com.freshell.managed-inputs': 'true', + 'com.freshell.installation-id': input.installationId, + 'com.freshell.soul-id': input.soulId, + 'com.freshell.incarnation-id': input.incarnationId, + 'com.freshell.input-generation': input.generationId, + } +} + +function equalRecord(actual: unknown, expected: Record): boolean { + if (!actual || typeof actual !== 'object' || Array.isArray(actual)) return false + const record = actual as Record + return Object.keys(record).length === Object.keys(expected).length + && Object.entries(expected).every(([key, value]) => record[key] === value) +} + +function privateHelperEnvironment(value: unknown, home: boolean): boolean { + if (!Array.isArray(value) || value.length > 128) return false + const names = new Set() + for (const entry of value) { + if (typeof entry !== 'string') return false + const separator = entry.indexOf('=') + const name = entry.slice(0, separator) + const contents = entry.slice(separator + 1) + const expected = name === 'HOME' && home ? '/home/freshell/provider' : '' + if (separator < 1 || !/^[a-zA-Z0-9_]{1,128}$/.test(name) + || names.has(name) || contents !== expected) return false + names.add(name) + } + return !home || names.has('HOME') +} + +function preparedMountsMatch(actual: unknown[], input: PreparedInputAuthorization): boolean { + const expected: Record[] = input.configurationBinds.map((bind) => ({ + Type: 'bind', Source: path.join(input.stageRoot, bind.sourceRelativePath), Target: bind.effectiveDestination, ReadOnly: true, + })) + expected.push(...input.resourceMounts.map((mount) => ({ + Type: 'volume', Source: input.providerVolumeName, Target: mount.effectiveDestination, ReadOnly: mount.kind === 'file', + VolumeOptions: { NoCopy: true, Subpath: mount.providerRelativePath }, + }))) + if (input.indexedFiles) { + expected.push(...input.indexedFiles.aliases.map((alias) => ({ + Type: 'volume', Source: input.indexedFiles!.volumeName, Target: alias.effectiveDestination, ReadOnly: true, + VolumeOptions: { NoCopy: true, Subpath: `files/source-${alias.sourceId}` }, + }))) + expected.push(...input.indexedSecretSources.map((source) => ({ + Type: 'volume', Source: input.indexedFiles!.volumeName, Target: `/run/freshell-secrets/provider-${source.grantIndex}`, ReadOnly: true, + VolumeOptions: { NoCopy: true, Subpath: `files/source-${source.sourceId}` }, + }))) + } + if (actual.length !== expected.length) return false + const remaining = new Map(expected.map((mount) => [mount.Target, mount])) + for (const value of actual) { + if (!value || typeof value !== 'object' || Array.isArray(value)) return false + const mount = value as Record + const wanted = remaining.get(mount.Target) + if (!wanted || Object.keys(mount).some((key) => !['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key)) + || ['Type','Source','Target','ReadOnly'].some((key) => mount[key] !== wanted[key])) return false + if (wanted.Type === 'volume') { + const options = mount.VolumeOptions + const wantedOptions = wanted.VolumeOptions as Record + if (!options || Object.keys(options).length !== 2 || options.NoCopy !== true || options.Subpath !== wantedOptions.Subpath) return false + } else if (mount.VolumeOptions !== undefined) return false + remaining.delete(mount.Target) + } + return remaining.size === 0 +} export type BrokerReceipt = { containerId: string @@ -49,6 +119,21 @@ export type RestrictedDockerBrokerPolicy = { logPath: string } +export type PreparedInputAuthorization = { + installationId: string + soulId: string + incarnationId: string + imageRef: string + hostBinaryPath: string + providerVolumeName: string + stageRoot: string + generationId: string + configurationBinds: { sourceRelativePath: string; effectiveDestination: string }[] + resourceMounts: { providerRelativePath: string; effectiveDestination: string; kind: 'file' | 'directory' }[] + indexedFiles?: { volumeName: string; aliases: { sourceId: number; effectiveDestination: string }[] } + indexedSecretSources: { grantIndex: number; sourceId: number }[] +} + type DockerResponse = { statusCode: number headers: http.IncomingHttpHeaders @@ -56,6 +141,13 @@ type DockerResponse = { } export class RestrictedDockerBroker { + private readonly preparedInputs = new Map() + private readonly inputVolumes = new Set() + private readonly inputPublishers = new Map + containerId?: string + }>() private readonly knownContainerIds = new Set() private readonly receiptsById = new Map() private readonly historyHelpers = new Map() @@ -65,6 +157,58 @@ export class RestrictedDockerBroker { constructor(private readonly policy: RestrictedDockerBrokerPolicy) {} + authorizePreparedInputs(input: PreparedInputAuthorization): void { + const safeId = (value: string) => /^[a-zA-Z0-9_-]{1,128}$/.test(value) + const relative = path.relative(path.dirname(this.policy.runtimeRootPrefix), input.stageRoot) + if (!safeId(input.installationId) || !safeId(input.soulId) || !safeId(input.incarnationId) + || !/^grant-[a-zA-Z0-9-]{1,122}$/.test(input.generationId) + || !new RegExp(`^scenarios/[^/]+/control/provider-roots/${input.generationId}$`).test(relative) + || !this.policy.allowedHostBinaryPaths.has(input.hostBinaryPath) + || !this.policy.allowedImageRefs.has(input.imageRef) + || !/^freshell-provider-[a-f0-9]{24}$/.test(input.providerVolumeName)) { + throw new Error('prepared input owner/stage is outside this fixture run') + } + if (Buffer.byteLength(JSON.stringify(input)) >= 1024 * 1024) throw new Error('prepared input metadata exceeds control profile') + const unit = (value: string, prefix: string) => value.startsWith(prefix) + && /^[a-zA-Z0-9_-]{1,128}$/.test(value.slice(prefix.length)) + const destination = (value: string) => path.isAbsolute(value) && path.normalize(value) === value + && !/[\x00-\x1f\x7f]/.test(value) + && (this.isAllowedWorkspacePath(value) + || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) + const targets = new Set() + const claim = (value: string) => { + if (!destination(value) || targets.has(value)) throw new Error('prepared input target is outside selected fixture roots or duplicated') + targets.add(value) + } + for (const bind of input.configurationBinds) { + if (!unit(bind.sourceRelativePath, '.freshell/templates/')) throw new Error('prepared template source is invalid') + claim(bind.effectiveDestination) + } + for (const mount of input.resourceMounts) { + const relative=mount.kind==='file' && mount.providerRelativePath.endsWith('/source') ? mount.providerRelativePath.slice(0,-'/source'.length) : mount.providerRelativePath; + if (!unit(relative, '.freshell/provider-roots/') || (mount.kind==='file' && !mount.providerRelativePath.endsWith('/source')) || !['file', 'directory'].includes(mount.kind)) throw new Error('prepared resource source is invalid') + claim(mount.effectiveDestination) + } + if (input.indexedFiles) { + const expected = `freshell-inputs-${createHash('sha256').update(`${input.soulId}\0${input.generationId}`).digest('hex').slice(0,24)}` + if (input.indexedFiles.volumeName !== expected || input.indexedFiles.aliases.length > 256 || input.indexedSecretSources.length > 128) throw new Error('prepared indexed volume identity/profile is invalid') + for (const alias of input.indexedFiles.aliases) { + if (!Number.isInteger(alias.sourceId) || alias.sourceId < 0 || alias.sourceId >= 128) throw new Error('prepared indexed source id is invalid') + claim(alias.effectiveDestination) + } + } else if (input.indexedSecretSources.length) throw new Error('prepared private source has no indexed volume') + const indexes = new Set() + for (const index of input.indexedSecretSources) { + if (!Number.isInteger(index.grantIndex) || index.grantIndex < 0 || index.grantIndex >= 132 + || !Number.isInteger(index.sourceId) || index.sourceId < 0 || index.sourceId >= 128 || indexes.has(index.grantIndex)) throw new Error('prepared private input index is invalid') + indexes.add(index.grantIndex) + } + const key = `${input.installationId}\0${input.soulId}\0${input.incarnationId}\0${input.generationId}` + const previous = this.preparedInputs.get(key) + if (previous && JSON.stringify(previous) !== JSON.stringify(input)) throw new Error('prepared input generation changed; reconcile before replacement') + this.preparedInputs.set(key, structuredClone(input)) + } + get proxySocketPath(): string { return this.policy.proxySocketPath } @@ -74,7 +218,7 @@ export class RestrictedDockerBroker { } receiptIds(): Set { - return new Set([...this.knownContainerIds, ...[...this.historyHelpers.values()].flatMap((helper) => helper.containerId ? [helper.containerId] : [])]) + return new Set([...this.knownContainerIds, ...[...this.historyHelpers.values(),...this.inputPublishers.values()].flatMap((helper) => helper.containerId ? [helper.containerId] : [])]) } eventsSnapshot(): BrokerEvent[] { @@ -137,14 +281,79 @@ export class RestrictedDockerBroker { await this.forwardAndReply(request, response, body, { destructive: false }) return } + if (method==='GET' && [...this.policy.allowedImageRefs].some((image)=>url===`${DOCKER_API_PREFIX}/images/${image}/json`)) { + await this.forwardAndReply(request,response,body,{destructive:false}) + return + } const volume = url.match(/^\/v1\.47\/volumes\/(freshell-provider-[a-f0-9]{24})$/)?.[1] if (method === 'GET' && volume && this.receipts().some((receipt) => receipt.providerVolumeName === volume)) { await this.forwardAndReply(request, response, body, { destructive: false }) return } + const indexedVolume=url.match(/^\/v1\.47\/volumes\/(freshell-inputs-[a-f0-9]{24})$/)?.[1] + const input=indexedVolume && [...this.preparedInputs.values()].find((candidate)=>candidate.indexedFiles?.volumeName===indexedVolume) + if (method==='GET' && input) { + await this.forwardAndReply(request,response,body,{destructive:false}) + return + } + if (method==='DELETE' && input && this.inputVolumes.has(indexedVolume!)) { + const inspected=await this.forward({method:'GET',url:`${DOCKER_API_PREFIX}/volumes/${indexedVolume}`,headers:{}} as IncomingMessage,Buffer.alloc(0)) + let metadata:Record={} + try {metadata=JSON.parse(inspected.body.toString())} catch {} + if (inspected.statusCode!==404 && (metadata.Name!==indexedVolume || metadata.Driver!=='local' + || !equalRecord(metadata.Labels,indexedLabels(input)) || !equalRecord(metadata.Options,{type:'tmpfs',device:'tmpfs',o:INDEXED_TMPFS_OPTIONS}))) { + this.block(response,method,url,403,'indexed volume ownership changed',true) + return + } + await this.forwardAndReply(request,response,body,{destructive:true}) + return + } + if (method==='POST' && url===`${DOCKER_API_PREFIX}/volumes/create`) { + let parsed:Record={} + try {parsed=JSON.parse(body.toString())} catch {} + const input=[...this.preparedInputs.values()].find((candidate)=>candidate.indexedFiles?.volumeName===parsed.Name) + if (!input || Object.keys(parsed).length!==4 || parsed.Driver!=='local' + || !equalRecord(parsed.DriverOpts,{type:'tmpfs',device:'tmpfs',o:INDEXED_TMPFS_OPTIONS}) || !equalRecord(parsed.Labels,indexedLabels(input))) { + this.block(response,method,url,403,'indexed volume must match an owned bounded preparation',true) + return + } + this.inputVolumes.add(parsed.Name) + await this.forwardAndReply(request,response,body,{destructive:false}) + return + } const helperTarget = url.match(/^\/v1\.47\/containers\/([^/?]+)(?:\/(start|wait|logs|json))?(?:\?.*)?$/) + const publisher=helperTarget && [...this.inputPublishers].find(([name,publisher])=>helperTarget[1]===name || helperTarget[1]===publisher.containerId) + if (publisher) { + const [name,owned]=publisher + const exactId=helperTarget![1]===owned.containerId + if (method==='DELETE' && !exactId) { + const inspected=await this.forward({method:'GET',url:`${DOCKER_API_PREFIX}/containers/${name}/json`,headers:{}} as IncomingMessage,Buffer.alloc(0)) + let actual:Record={} + try {actual=JSON.parse(inspected.body.toString())} catch {} + const environment=(entries:unknown)=>Array.isArray(entries)?JSON.stringify([...entries].sort()):undefined + const ownedConfig=inspected.statusCode===404 || (inspected.statusCode===200 && ['Image','User','Tty'].every((key)=>actual.Config?.[key]===owned.expected[key]) + && ['Entrypoint','Cmd'].every((key)=>JSON.stringify(actual.Config?.[key])===JSON.stringify(owned.expected[key])) + && equalRecord(actual.Config?.Labels,owned.expected.Labels) + && environment(actual.Config?.Env)===environment(owned.expected.Env) + && Array.isArray(actual.HostConfig?.Mounts) && actual.HostConfig.Mounts.length===3 + && owned.expected.HostConfig.Mounts.every((expected:Record)=>actual.HostConfig.Mounts.some((mount:Record)=> + ['Type','Source','Target','ReadOnly'].every((key)=>mount[key]===expected[key]) + && (expected.Type!=='volume' || (mount.VolumeOptions?.NoCopy===true && (mount.VolumeOptions.Subpath ?? '')==='' && !mount.VolumeOptions.DriverConfig)))) + && actual.HostConfig?.LogConfig?.Type==='none') + if (!ownedConfig) { + this.block(response,method,url,403,'indexed publisher named cleanup ownership changed',true) + return + } + } + if ((method==='GET' && url===`${DOCKER_API_PREFIX}/containers/${helperTarget![1]}/json`) + || (method==='POST' && exactId && url===`${DOCKER_API_PREFIX}/containers/${owned.containerId}/start`) + || (method==='DELETE' && url===`${DOCKER_API_PREFIX}/containers/${helperTarget![1]}?force=1`)) { + await this.forwardAndReply(request,response,body,{destructive:method!=='GET',containerId:owned.containerId,helperName:name}) + return + } + } const helperEntry = helperTarget && [...this.historyHelpers].find(([name, helper]) => ( helperTarget[1] === name || helperTarget[1] === helper.containerId )) @@ -167,6 +376,29 @@ export class RestrictedDockerBroker { if (method === 'POST' && url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`)) { const name = new URL(url, 'http://docker').searchParams.get('name') ?? '' + if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name)) { + const input=this.validateInputPublisher(name,body) + if (!input || this.inputPublishers.has(name)) { + this.block(response,method,url,403,'indexed publisher must match an owned fixed preparation',false) + return + } + const publisher:{input:PreparedInputAuthorization;expected:Record;containerId?:string}={input,expected:JSON.parse(body.toString())} + this.inputPublishers.set(name,publisher) + try { + const forwarded=await this.forward(request,body) + if (forwarded.statusCode===201) { + const parsed=JSON.parse(forwarded.body.toString()) + if (typeof parsed.Id!=='string' || !/^[a-f0-9]{64}$/.test(parsed.Id) || this.receiptIds().has(parsed.Id)) throw new Error('invalid indexed publisher identity') + publisher.containerId=parsed.Id + } + this.record({method,url,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name,containerId:publisher.containerId,providerVolumeName:input.indexedFiles!.volumeName}) + replyDocker(response,forwarded) + } catch { + response.statusCode=503 + response.end(JSON.stringify({message:'indexed publisher acknowledgement unavailable; exact ownership inspection required'})) + } + return + } if (name.startsWith('freshell-history-') || name.startsWith('freshell-auth-')) { const kind = name.startsWith('freshell-auth-') ? 'auth' : 'history' const owner = kind === 'auth' ? this.validateAuthHelper(name, body) : this.validateHistoryHelper(name, body) @@ -283,6 +515,9 @@ export class RestrictedDockerBroker { if (!incarnationId || !soulId || !installationId) return { ok: false, reason: 'missing installation/incarnation/soul labels' } const host = parsed.HostConfig ?? {} + if (host.Mounts !== undefined && !Array.isArray(host.Mounts)) { + return { ok: false, reason: 'typed mounts require an exact owned provider preparation authorization' } + } const terminalWorkload = host.NetworkMode === 'bridge' const freshAgentWorkload = Array.isArray(parsed.Env) && parsed.Env.some((value: unknown) => ( typeof value === 'string' && value.startsWith('FRESHELL_HOSTED_FRESH_AGENT=') @@ -393,9 +628,41 @@ export class RestrictedDockerBroker { if (binds.some((bind) => bind.includes('docker.sock') || bind.includes('/var/lib/freshell-supervisor') || bind.includes('/run/freshell-supervisor'))) { return { ok: false, reason: 'management-state mount is forbidden' } } + const prepared=[...this.preparedInputs.values()].filter((input) => input.installationId === installationId + && input.soulId === soulId && input.incarnationId === incarnationId && input.imageRef === imageRef + && input.hostBinaryPath === hostBinaryPath && input.providerVolumeName === providerVolumeName); + if ((host.Mounts?.length || prepared.length) && !prepared.some((input)=>preparedMountsMatch(host.Mounts ?? [],input))) return { ok: false, reason: 'typed mounts differ from this attempt owned prepared input set' } return { ok: true, incarnationId, soulId, imageRef, runtimeDir, hostBinaryPath, ...(providerVolumeName ? { providerVolumeName } : {}), ...(workspacePath ? { workspacePath } : {}) } } + private validateInputPublisher(name:string,body:Buffer):PreparedInputAuthorization | undefined { + let parsed:Record + try {parsed=JSON.parse(body.toString())} catch {return} + const input=[...this.preparedInputs.values()].find((input)=>`${input.indexedFiles?.volumeName}-publisher`===name) + if (!input || !this.inputVolumes.has(input.indexedFiles!.volumeName) || parsed.Image!==input.imageRef + || parsed.User!=='65534:0' || parsed.Tty!==false || !equalRecord(parsed.Labels,indexedLabels(input)) + || JSON.stringify(parsed.Entrypoint)!==JSON.stringify(['/runtime/freshell-session-host']) + || JSON.stringify(parsed.Cmd)!==JSON.stringify(['publish-provider-inputs']) || !privateHelperEnvironment(parsed.Env,false)) return + const host=parsed.HostConfig ?? {} + if (Object.keys(parsed).some((key)=>!['Image','User','Tty','Labels','Entrypoint','Cmd','Env','HostConfig'].includes(key)) + || Object.keys(host).some((key)=>!['NetworkMode','ReadonlyRootfs','CapDrop','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','LogConfig','Mounts'].includes(key))) return + if (host.NetworkMode!=='none' || host.ReadonlyRootfs!==true || host.Privileged===true || (host.PidMode ?? '')!=='' + || (host.Binds?.length ?? 0)!==0 || (host.CapAdd?.length ?? 0)!==0 || Object.keys(host.Tmpfs ?? {}).length + || JSON.stringify(host.CapDrop)!==JSON.stringify(['ALL']) || JSON.stringify(host.SecurityOpt)!==JSON.stringify(['no-new-privileges']) + || host.Memory!==128*1024*1024 || host.MemorySwap!==128*1024*1024 || host.NanoCpus!==500_000_000 || host.PidsLimit!==32 + || !equalRecord(host.LogConfig,{Type:'none',Config:host.LogConfig?.Config}) || Object.keys(host.LogConfig?.Config ?? {}).length) return + const mounts=host.Mounts + if (!Array.isArray(mounts) || mounts.length!==3) return + const binary=mounts.find((mount)=>mount.Type==='bind' && mount.Source===input.hostBinaryPath && mount.Target==='/runtime/freshell-session-host' && mount.ReadOnly===true) + const volume=mounts.find((mount)=>mount.Type==='volume' && mount.Source===input.indexedFiles!.volumeName && mount.Target==='/run/freshell-input-files' && mount.ReadOnly===false && equalRecord(mount.VolumeOptions,{NoCopy:true})) + const exchange=mounts.find((mount)=>mount.Type==='bind' && mount.Target==='/run/freshell-input-exchange' && mount.ReadOnly===true + && typeof mount.Source==='string' && path.dirname(mount.Source)===os.tmpdir() + && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + if (!binary || !volume || !exchange || [binary,volume,exchange].some((mount)=>Object.keys(mount).some((key)=>!['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key))) + || binary.VolumeOptions!==undefined || exchange.VolumeOptions!==undefined) return + return input + } + private validateHistoryHelper(name: string, body: Buffer): BrokerReceipt | undefined { if (!/^freshell-history-[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/.test(name)) return let parsed: Record @@ -435,7 +702,7 @@ export class RestrictedDockerBroker { const same = (actual: unknown, expected: unknown) => JSON.stringify(actual) === JSON.stringify(expected) if (Object.keys(parsed.Labels ?? {}).length !== 0 || parsed.User !== '65534:0' || parsed.Tty !== false || !same(parsed.Entrypoint, ['/runtime/freshell-session-host']) || !same(parsed.Cmd, ['native-auth-only']) - || !same(parsed.Env, ['HOME=/home/freshell/provider'])) return + || !privateHelperEnvironment(parsed.Env,true)) return if (host.NetworkMode !== 'none' || host.ReadonlyRootfs !== true || host.Privileged === true || (host.PidMode ?? '') !== '' || (host.Binds?.length ?? 0) !== 0 || (host.CapAdd?.length ?? 0) !== 0 || !same(host.CapDrop, ['ALL']) || !same(host.SecurityOpt, ['no-new-privileges']) @@ -462,9 +729,9 @@ export class RestrictedDockerBroker { return false } - private async forwardAndReply(request: IncomingMessage, response: ServerResponse, body: Buffer, event: { destructive: boolean; containerId?: string }): Promise { + private async forwardAndReply(request: IncomingMessage, response: ServerResponse, body: Buffer, event: { destructive: boolean; containerId?: string; helperName?:string }): Promise { const forwarded = await this.forward(request, body) - this.record({ method: request.method ?? 'GET', url: request.url ?? '/', decision: 'forward', destructive: event.destructive, unsafeAttempt: false, containerId: event.containerId }) + this.record({ method: request.method ?? 'GET', url: request.url ?? '/', decision: 'forward', destructive: event.destructive, unsafeAttempt: false, containerId: event.containerId,helperName:event.helperName }) replyDocker(response, forwarded) } diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index e2f2886c4..025fb3b60 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -5,7 +5,7 @@ import http from 'node:http' import path from 'node:path' import { describe, expect, it } from 'vitest' -import { RestrictedDockerBroker } from '../../../../scripts/testing/runtime-test-broker.js' +import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' const runtimeDir = '/tmp/owned/r/scenario/x/incarnation-one' const actorKey = createHash('sha256').update('installation-one\0soul-one').digest('hex') @@ -70,6 +70,75 @@ const ordinaryBinds = [ ] describe('managed runtime broker actor storage', () => { + it('accepts only blanked image defaults and fixed HOME for the owned auth helper',async()=>{ + await withHttpBroker(async ({request,forwarded})=>{ + expect((await request('POST','/v1.47/containers/create?name=owned',createBody([...ordinaryBinds,actorBind]))).status).toBe(201); + expect((await request('GET','/v1.47/images/sha256:fixture-image/json')).status).toBe(200); + const body=authHelperBody(); + body.Env=['PATH=','DEBIAN_FRONTEND=','HOME=/home/freshell/provider']; + expect((await request('POST',`/v1.47/containers/create?name=${authName}`,body)).status).toBe(201); + const before=forwarded.length; + for (const Env of [['PATH=PRIVATE_VALUE','HOME=/home/freshell/provider'],['HOME=/outside'],['PATH=','PATH=','HOME=/home/freshell/provider'],['PATH=']]) { + const changed={...body,Env}; + expect((await request('POST',`/v1.47/containers/create?name=${authName.replace('111111111111','222222222222')}`,changed)).status).toBe(403); + expect(forwarded).toHaveLength(before); + } + expect((await request('GET','/v1.47/images/sha256:foreign/json')).status).toBe(403); + expect(forwarded).toHaveLength(before); + }); + }); + it('forwards only the exact attempt-owned typed mounts and refuses altered sources and controls', async () => { + await withHttpBroker(async ({request,forwarded,broker:instance})=>{ + const generationId='grant-prepared'; + const volumeName=`freshell-inputs-${createHash('sha256').update(`soul-one\0${generationId}`).digest('hex').slice(0,24)}`; + const authorization:PreparedInputAuthorization={ + installationId:'installation-one',soulId:'soul-one',incarnationId:'incarnation-one',generationId, + imageRef:'sha256:fixture-image',hostBinaryPath:'/tmp/owned/freshell-session-host',providerVolumeName:providerVolume, + stageRoot:`/tmp/owned/scenarios/scenario/control/provider-roots/${generationId}`, + configurationBinds:[{sourceRelativePath:'.freshell/templates/source-0',effectiveDestination:'/workspace/opencode.json'}], + resourceMounts:[{providerRelativePath:'.freshell/provider-roots/plugin-0',effectiveDestination:'/workspace/plugin',kind:'directory'}, + {providerRelativePath:'.freshell/provider-roots/instructions-0/source',effectiveDestination:'/workspace/instructions.md',kind:'file'}], + indexedFiles:{volumeName,aliases:[{sourceId:0,effectiveDestination:'/workspace/selected-input'}]}, + indexedSecretSources:[{grantIndex:1,sourceId:0}], + }; + instance.authorizePreparedInputs(authorization); + const body=JSON.parse(createBody([...ordinaryBinds,actorBind]).toString()); + body.HostConfig.Mounts=[ + {Type:'bind',Source:`${authorization.stageRoot}/.freshell/templates/source-0`,Target:'/workspace/opencode.json',ReadOnly:true}, + {Type:'volume',Source:providerVolume,Target:'/workspace/plugin',ReadOnly:false,VolumeOptions:{NoCopy:true,Subpath:'.freshell/provider-roots/plugin-0'}}, + {Type:'volume',Source:volumeName,Target:'/workspace/selected-input',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'files/source-0'}}, + {Type:'volume',Source:volumeName,Target:'/run/freshell-secrets/provider-1',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'files/source-0'}}, + {Type:'volume',Source:providerVolume,Target:'/workspace/instructions.md',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'.freshell/provider-roots/instructions-0/source'}}, + ]; + expect((await request('POST','/v1.47/containers/create?name=owned-prepared',body)).status).toBe(201); + expect(forwarded).toHaveLength(1); + for (const mutation of ['source','target','writable','subpath','nocopy','missing','empty','duplicate','owner','image','volume']) { + const changed=structuredClone(body); + if (mutation==='source') changed.HostConfig.Mounts[0].Source='/var/run/docker.sock'; + if (mutation==='target') changed.HostConfig.Mounts[0].Target='/run/freshell'; + if (mutation==='writable') changed.HostConfig.Mounts[2].ReadOnly=false; + if (mutation==='subpath') changed.HostConfig.Mounts[2].VolumeOptions.Subpath='files/source-1'; + if (mutation==='nocopy') changed.HostConfig.Mounts[1].VolumeOptions.NoCopy=false; + if (mutation==='missing') changed.HostConfig.Mounts.pop(); + if (mutation==='empty') changed.HostConfig.Mounts=[]; + if (mutation==='duplicate') changed.HostConfig.Mounts.push(changed.HostConfig.Mounts[0]); + if (mutation==='owner') changed.Labels['com.freshell.incarnation-id']='incarnation-other'; + if (mutation==='image') changed.Image='sha256:other-approved-image'; + if (mutation==='volume') changed.HostConfig.Binds=changed.HostConfig.Binds.map((bind:string)=>bind.replace(providerVolume,`freshell-provider-${'c'.repeat(24)}`)); + expect((await request('POST',`/v1.47/containers/create?name=changed-${mutation}`,changed)).status,mutation).toBe(403); + expect(forwarded).toHaveLength(1); + } + }); + }); + it('blocks unapproved typed mounts before any Docker request is forwarded', async () => { + await withHttpBroker(async ({request,forwarded})=>{ + const body=JSON.parse(createBody([...ordinaryBinds,actorBind]).toString()); + body.HostConfig.Mounts=[{Type:'bind',Source:'/var/run/docker.sock',Target:'/outside/source',ReadOnly:true}]; + const response=await request('POST','/v1.47/containers/create?name=typed-fixture',body); + expect(response.status).toBe(403); + expect(forwarded).toEqual([]); + }); + }); it('allows only the durable host-owned directory for the labeled soul', () => { const validate = (body: Buffer): { ok: boolean } => ( broker() as unknown as { validateCreate(body: Buffer): { ok: boolean } } @@ -237,8 +306,86 @@ type BrokerHttpFixture = { broker: RestrictedDockerBroker loseHelperAck(): void returnHelperId(id: string): void + replaceHelperInspection():void +} + +function publisherAuthorization():PreparedInputAuthorization { + const generationId='grant-publisher'; + return {installationId:'installation-one',soulId:'soul-one',incarnationId:'incarnation-one',generationId, + imageRef:'sha256:fixture-image',hostBinaryPath:'/tmp/owned/freshell-session-host',providerVolumeName:providerVolume, + stageRoot:`/tmp/owned/scenarios/scenario/control/provider-roots/${generationId}`,configurationBinds:[],resourceMounts:[], + indexedFiles:{volumeName:`freshell-inputs-${createHash('sha256').update(`soul-one\0${generationId}`).digest('hex').slice(0,24)}`,aliases:[{sourceId:0,effectiveDestination:'/workspace/input'}]}, + indexedSecretSources:[{grantIndex:0,sourceId:0}]}; +} + +const indexedOptions={type:'tmpfs',device:'tmpfs',o:'size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec'}; +function publisherLabels(input:PreparedInputAuthorization) { + return {'com.freshell.managed-inputs':'true','com.freshell.installation-id':input.installationId, + 'com.freshell.soul-id':input.soulId,'com.freshell.incarnation-id':input.incarnationId,'com.freshell.input-generation':input.generationId}; +} +function publisherBody(input:PreparedInputAuthorization):any { + return {Image:input.imageRef,User:'65534:0',Tty:false,Labels:publisherLabels(input),Entrypoint:['/runtime/freshell-session-host'],Cmd:['publish-provider-inputs'],Env:['PATH='], + HostConfig:{NetworkMode:'none',ReadonlyRootfs:true,CapDrop:['ALL'],SecurityOpt:['no-new-privileges'],Memory:128*1024*1024,MemorySwap:128*1024*1024,NanoCpus:500_000_000,PidsLimit:32,LogConfig:{Type:'none',Config:{}}, + Mounts:[{Type:'bind',Source:input.hostBinaryPath,Target:'/runtime/freshell-session-host',ReadOnly:true}, + {Type:'volume',Source:input.indexedFiles!.volumeName,Target:'/run/freshell-input-files',ReadOnly:false,VolumeOptions:{NoCopy:true}}, + {Type:'bind',Source:path.join(os.tmpdir(),'freshell-inputs-11111111-1111-4111-8111-111111111111'),Target:'/run/freshell-input-exchange',ReadOnly:true}]}}; } +describe('fixed indexed publisher broker',()=>{ + it('preserves exact name ownership on lost publisher acknowledgement and refuses replacement-helper cleanup',async()=>{ + await withHttpBroker(async({request,forwarded,broker:instance,loseHelperAck,replaceHelperInspection})=>{ + const input=publisherAuthorization();instance.authorizePreparedInputs(input); + const volume={Name:input.indexedFiles!.volumeName,Driver:'local',DriverOpts:indexedOptions,Labels:publisherLabels(input)}; + expect((await request('POST','/v1.47/volumes/create',volume)).status).toBe(201); + const name=`${volume.Name}-publisher`; + loseHelperAck(); + expect((await request('POST',`/v1.47/containers/create?name=${name}`,publisherBody(input))).status).toBe(503); + expect((await request('GET',`/v1.47/containers/${name}/json`)).status).toBe(200); + replaceHelperInspection(); + expect((await request('DELETE',`/v1.47/containers/${name}?force=1`)).status).toBe(403); + expect(forwarded.filter((request)=>request.method==='DELETE')).toEqual([]); + }); + }); + it('limits tmpfs and publisher operations to the exact preparation and refuses extra host access',async()=>{ + await withHttpBroker(async({request,forwarded,broker:instance})=>{ + const input=publisherAuthorization(); + instance.authorizePreparedInputs(input); + const volume={Name:input.indexedFiles!.volumeName,Driver:'local',DriverOpts:indexedOptions,Labels:publisherLabels(input)}; + const invalid=structuredClone(volume);invalid.DriverOpts.o='size=128m'; + expect((await request('POST','/v1.47/volumes/create',invalid)).status).toBe(403); + expect(forwarded).toHaveLength(0); + expect((await request('POST','/v1.47/volumes/create',volume)).status).toBe(201); + const name=`${volume.Name}-publisher`; + const body=publisherBody(input); + const before=forwarded.length; + for (const mutation of ['devices','socket','volume','uid','network','credentials','argv','capabilities','labels']) { + const changed=structuredClone(body); + if (mutation==='devices') changed.HostConfig.Devices=[{PathOnHost:'/dev/foreign',PathInContainer:'/dev/foreign',CgroupPermissions:'rwm'}]; + if (mutation==='socket') changed.HostConfig.Mounts[2].Source='/var/run/docker.sock'; + if (mutation==='volume') changed.HostConfig.Mounts[1].Source=providerVolume; + if (mutation==='uid') changed.User='0:0'; + if (mutation==='network') changed.HostConfig.NetworkMode='host'; + if (mutation==='credentials') changed.Env=['API_KEY=PRIVATE_VALUE']; + if (mutation==='argv') changed.Cmd.push('arbitrary-command'); + if (mutation==='capabilities') changed.HostConfig.CapAdd=['SYS_ADMIN']; + if (mutation==='labels') changed.Labels['com.freshell.incarnation-id']='foreign'; + expect((await request('POST',`/v1.47/containers/create?name=${name}`,changed)).status,mutation).toBe(403); + expect(forwarded).toHaveLength(before); + } + expect((await request('POST',`/v1.47/containers/create?name=${name}`,body)).status).toBe(201); + expect((await request('POST',`/v1.47/containers/${helperId}/start`)).status).toBe(204); + const afterStart=forwarded.length; + for (const url of [`/v1.47/containers/${helperId}/logs?stdout=1`, `/v1.47/containers/${helperId}/exec`, `/v1.47/containers/${ownedId}/start`, '/v1.47/volumes/freshell-inputs-ffffffffffffffffffffffff']) { + expect((await request('GET',url)).status).toBe(403); + expect(forwarded).toHaveLength(afterStart); + } + expect((await request('GET',`/v1.47/containers/${name}/json`)).status).toBe(200); + expect((await request('DELETE',`/v1.47/containers/${helperId}?force=1`)).status).toBe(204); + expect((await request('DELETE',`/v1.47/volumes/${volume.Name}`)).status).toBe(204); + }); + }); +}); + async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise) { const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-broker-history-')) const socket = path.join(dir, 'docker.sock') @@ -246,12 +393,25 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise const forwarded: ForwardedRequest[] = [] let loseAck = false let helperAckId = helperId + let indexedVolumeMetadata:unknown + let foreignHelper=false const upstream = http.createServer(async (req, res) => { const chunks: Buffer[] = [] for await (const chunk of req) chunks.push(Buffer.from(chunk)) const raw = Buffer.concat(chunks).toString() const body = raw ? JSON.parse(raw) : undefined forwarded.push({ method: req.method!, url: req.url!, body }) + if (req.url==='/v1.47/volumes/create') { + indexedVolumeMetadata={Name:body.Name,Driver:body.Driver,Options:body.DriverOpts,Labels:body.Labels}; + res.statusCode=201;res.end(JSON.stringify(indexedVolumeMetadata));return; + } + if (req.method==='GET' && req.url?.startsWith('/v1.47/volumes/freshell-inputs-') && indexedVolumeMetadata) { + res.statusCode=200;res.end(JSON.stringify(indexedVolumeMetadata));return; + } + if (req.method==='GET' && req.url?.includes('-publisher/json')) { + const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]==='publish-provider-inputs')!.body; + res.statusCode=200;res.end(JSON.stringify({Config:foreignHelper?{...config,User:'0:0'}:config,HostConfig:config.HostConfig}));return; + } if (req.url?.startsWith('/v1.47/containers/create?')) { if (body.Entrypoint?.[0] === '/runtime/freshell-session-host' && loseAck) { req.socket.destroy() @@ -293,7 +453,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise try { await new Promise((resolve) => upstream.listen(socket, resolve)) await instance.start() - await run({ request, forwarded, broker: instance, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id } }) + await run({ request, forwarded, broker: instance, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true} }) } finally { await instance.close() await new Promise((resolve) => upstream.close(() => resolve())) From 6f8d0309f416d940f46d101601fbdb3c7365cc95 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 08:21:32 -0700 Subject: [PATCH 029/493] test: prove rootless indexed provider input mount lifetime --- scripts/sandbox-test.sh | 5 + .../probe-provider-config-readonly-home.py | 216 ++++++++++++++++++ 2 files changed, 221 insertions(+) diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 30c10581e..acf4bda82 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -32,6 +32,11 @@ set -euo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" IMAGE_TAG="freshell-sandbox:latest" +if [ "${1:-}" = "--provider-inputs" ]; then + shift + exec python3 "${REPO_ROOT}/scripts/testing/probe-provider-config-readonly-home.py" --indexed-inputs "$@" +fi + if [ "${1:-}" = "--native-tools" ]; then NATIVE_MANIFEST="${2:-}" if [ -z "${NATIVE_MANIFEST}" ]; then diff --git a/scripts/testing/probe-provider-config-readonly-home.py b/scripts/testing/probe-provider-config-readonly-home.py index 01ec95657..bcffb308d 100644 --- a/scripts/testing/probe-provider-config-readonly-home.py +++ b/scripts/testing/probe-provider-config-readonly-home.py @@ -9,6 +9,9 @@ import subprocess import tempfile import uuid +import socket +import struct +import time def docker(*args, check=True): @@ -17,10 +20,218 @@ def docker(*args, check=True): ) +def indexed_inputs_probe(image, binary, binary_sha256, receipt_directory): + """Closed synthetic publisher/file-subpath fixture; permissions run in Docker.""" + expected_image = "sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" + if image != expected_image: + raise RuntimeError("indexed input fixture requires the pinned already-present runtime image") + run_id = "freshell-indexed-probe-" + uuid.uuid4().hex + label_key = "com.freshell.indexed-input-probe" + labels = {label_key: run_id} + volume = run_id + "-volume" + containers = [] + payload = b"synthetic-indexed-source\n" + expected_digest = hashlib.sha256(payload).hexdigest() + receipt = {"image": image, "binarySha256": binary_sha256, "rootless": True, "runId": run_id} + + def remove_owned(name): + result = docker("container", "inspect", name, check=False) + if result.returncode != 0: + return + value = json.loads(result.stdout)[0] + if value["Config"].get("Labels", {}).get(label_key) != run_id: + raise RuntimeError("indexed fixture cleanup ownership changed") + docker("rm", "--force", value["Id"]) + + with tempfile.TemporaryDirectory(prefix=run_id) as temporary: + exchange = Path(temporary) + # This fresh directory contains only IPC/receipts. Setup and every + # chmod/chown/denial fixture executes inside the owned namespace. + stream = None + listener = None + volume_attempted = False + try: + setup = run_id + "-setup" + containers.append(setup) + docker("create", "--name", setup, "--label", f"{label_key}={run_id}", + "--network", "none", "--read-only", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--pids-limit", "32", "--memory", "64m", + "--mount", f"type=bind,src={exchange},dst=/fixture-exchange", + "--entrypoint", "/bin/sh", image, "-ceu", "chmod 710 /fixture-exchange") + setup_result = docker("start", "--attach", setup, check=False) + if setup_result.returncode != 0: + raise RuntimeError("indexed fixture namespace setup failed") + remove_owned(setup) + listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + listener.settimeout(20) + socket_path = exchange / "reply.sock" + listener.bind(str(socket_path)) + listener.listen(1) + mode_setup = run_id + "-socket-mode" + containers.append(mode_setup) + docker("create", "--name", mode_setup, "--label", f"{label_key}={run_id}", + "--network", "none", "--read-only", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--pids-limit", "32", "--memory", "64m", + "--mount", f"type=bind,src={exchange},dst=/fixture-exchange", + "--entrypoint", "/bin/sh", image, "-ceu", "chmod 660 /fixture-exchange/reply.sock") + if docker("start", "--attach", mode_setup, check=False).returncode != 0: + raise RuntimeError("indexed fixture socket setup failed") + remove_owned(mode_setup) + volume_attempted = True + docker("volume", "create", "--label", f"{label_key}={run_id}", "--driver", "local", + "--opt", "type=tmpfs", "--opt", "device=tmpfs", "--opt", + "o=size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec", volume) + publisher = run_id + "-publisher" + containers.append(publisher) + docker("create", "--name", publisher, "--label", f"{label_key}={run_id}", + "--user", "65534:0", "--network", "none", "--read-only", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--log-driver", "none", + "--pids-limit", "32", "--memory", "128m", "--cpus", ".5", + "--mount", f"type=bind,src={binary},dst=/runtime/freshell-session-host,readonly", + "--mount", f"type=volume,src={volume},dst=/run/freshell-input-files,volume-nocopy", + "--mount", f"type=bind,src={exchange},dst=/run/freshell-input-exchange,readonly", + "--entrypoint", "/runtime/freshell-session-host", image, "publish-provider-inputs") + docker("start", publisher) + stream, _ = listener.accept() + stream.settimeout(20) + + def receive_exact(length): + result = bytearray() + while len(result) < length: + chunk = stream.recv(length - len(result)) + if not chunk: + raise RuntimeError("indexed publisher private acknowledgement incomplete") + result.extend(chunk) + return bytes(result) + + def send(request, expected): + frame = json.dumps(request, separators=(",", ":")).encode() + if len(frame) >= 1024 * 1024: + raise RuntimeError("indexed fixture private frame exceeds profile") + stream.sendall(struct.pack(">I", len(frame)) + frame) + length = struct.unpack(">I", receive_exact(4))[0] + if length >= 1024 * 1024 or json.loads(receive_exact(length)) != expected: + raise RuntimeError("indexed publisher acknowledgement phase changed") + + send({"operation": "begin", "source_ids": [0]}, "begun") + send({"operation": "open", "source_id": 0}, "opened") + send({"operation": "chunk", "bytes": list(payload)}, "written") + send({"operation": "finish"}, "finished") + send({"operation": "seal"}, "populated") + consumer = run_id + "-consumer" + containers.append(consumer) + probe = r''' +const fs=require('fs'), crypto=require('crypto'); +const file='/fixture/selected'; +const stat=fs.lstatSync(file); +let writable=true;try{fs.writeFileSync(file,'forbidden-write')}catch{writable=false} +const status=fs.readFileSync('/proc/self/status','utf8'); +console.log(JSON.stringify({regular:stat.isFile(),symlink:stat.isSymbolicLink(),uid:process.getuid(),gid:process.getgid(),fileUid:stat.uid,fileGid:stat.gid,mode:stat.mode&511,writable,sha256:crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex'),capEff:status.match(/^CapEff:\s*(\w+)/m)?.[1],uidMap:fs.readFileSync('/proc/self/uid_map','utf8').trim(),gidMap:fs.readFileSync('/proc/self/gid_map','utf8').trim()})); +setInterval(()=>{},1000); +''' + docker("create", "--name", consumer, "--label", f"{label_key}={run_id}", + "--user", "65534:0", "--network", "none", "--read-only", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--pids-limit", "32", "--memory", "128m", + "--mount", f"type=volume,src={volume},dst=/fixture/selected,volume-subpath=files/source-0,volume-nocopy,readonly", + "--entrypoint", "/usr/local/bin/node", image, "-e", probe) + inspected = json.loads(docker("inspect", consumer).stdout)[0] + mount = inspected["HostConfig"]["Mounts"][0] + assert mount["Type"] == "volume" and mount["Source"] == volume and mount["ReadOnly"] + assert mount["VolumeOptions"]["NoCopy"] and mount["VolumeOptions"]["Subpath"] == "files/source-0" + docker("start", consumer) + active = json.loads(docker("inspect", consumer).stdout)[0] + flattened = active["Mounts"] + volume_metadata = json.loads(docker("volume", "inspect", volume).stdout)[0] + assert len(flattened) == 1 + assert flattened[0]["Type"] == "volume" and flattened[0]["Name"] == volume + assert flattened[0]["Destination"] == "/fixture/selected" and not flattened[0]["RW"] + assert flattened[0]["Source"] == volume_metadata["Mountpoint"] + deadline = time.monotonic() + 20 + observed = None + while time.monotonic() < deadline: + output = docker("logs", consumer).stdout.strip() + if output: + observed = json.loads(output.splitlines()[0]) + break + time.sleep(.1) + if observed is None: + raise RuntimeError("indexed consumer produced no bounded receipt") + assert observed["regular"] and not observed["symlink"] and not observed["writable"] + assert observed["uid"] == 65534 and observed["gid"] == 0 + assert observed["fileUid"] == 65534 and observed["fileGid"] == 0 and observed["mode"] == 0o600 + assert observed["capEff"] == "0000000000000000" and observed["sha256"] == expected_digest + assert observed["uidMap"] and observed["gidMap"] + assert json.loads(docker("inspect", publisher).stdout)[0]["State"]["Running"] + send({"operation": "release"}, "released") + remove_owned(publisher) + assert json.loads(docker("inspect", consumer).stdout)[0]["State"]["Running"] + post_release = docker("exec", "--user", "65534:0", consumer, "/usr/local/bin/node", "-e", + "process.stdout.write(require('crypto').createHash('sha256').update(require('fs').readFileSync('/fixture/selected')).digest('hex'))").stdout + assert post_release == expected_digest + receipt.update({"consumer": observed, "subpath": mount["VolumeOptions"]["Subpath"], "flattenedMountAgreement": True, "publisherConsumerOverlap": True}) + remove_owned(consumer) + stream.close() + stream = None + # After the last active mount goes away, the local tmpfs is empty. + # Repopulate the same exact owned volume before any cold consumer. + cold_publisher = run_id + "-cold-publisher" + containers.append(cold_publisher) + docker("create", "--name", cold_publisher, "--label", f"{label_key}={run_id}", + "--user", "65534:0", "--network", "none", "--read-only", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--log-driver", "none", + "--pids-limit", "32", "--memory", "128m", "--cpus", ".5", + "--mount", f"type=bind,src={binary},dst=/runtime/freshell-session-host,readonly", + "--mount", f"type=volume,src={volume},dst=/run/freshell-input-files,volume-nocopy", + "--mount", f"type=bind,src={exchange},dst=/run/freshell-input-exchange,readonly", + "--entrypoint", "/runtime/freshell-session-host", image, "publish-provider-inputs") + docker("start", cold_publisher) + stream, _ = listener.accept() + stream.settimeout(20) + cold_payload = b"synthetic-cold-repopulation\n" + send({"operation": "begin", "source_ids": [0]}, "begun") + send({"operation": "open", "source_id": 0}, "opened") + send({"operation": "chunk", "bytes": list(cold_payload)}, "written") + send({"operation": "finish"}, "finished") + send({"operation": "seal"}, "populated") + cold_consumer = run_id + "-cold-consumer" + containers.append(cold_consumer) + docker("create", "--name", cold_consumer, "--label", f"{label_key}={run_id}", + "--user", "65534:0", "--network", "none", "--read-only", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--pids-limit", "32", "--memory", "128m", + "--mount", f"type=volume,src={volume},dst=/fixture/selected,volume-subpath=files/source-0,volume-nocopy,readonly", + "--entrypoint", "/usr/local/bin/node", image, "-e", + "process.stdout.write(require('crypto').createHash('sha256').update(require('fs').readFileSync('/fixture/selected')).digest('hex'))") + cold = docker("start", "--attach", cold_consumer, check=False) + assert cold.returncode == 0 and cold.stdout == hashlib.sha256(cold_payload).hexdigest() + send({"operation": "release"}, "released") + receipt["postReleaseRead"] = True + receipt["coldLastUnmountRepopulated"] = True + finally: + if stream is not None: + stream.close() + if listener is not None: + listener.close() + for container in reversed(containers): + remove_owned(container) + volume_state = docker("volume", "inspect", volume, check=False) + if volume_attempted and volume_state.returncode == 0: + metadata = json.loads(volume_state.stdout)[0] + if metadata.get("Labels", {}).get(label_key) != run_id: + raise RuntimeError("indexed fixture volume cleanup ownership changed") + docker("volume", "rm", volume) + receipt["ownedCleanup"] = all(docker("container", "inspect", name, check=False).returncode != 0 for name in containers) + receipt["ownedCleanup"] &= docker("volume", "inspect", volume, check=False).returncode != 0 + output = receipt_directory / "task002-indexed-provider-inputs.json" + output.write_text(json.dumps(receipt, sort_keys=True) + "\n") + print(json.dumps(receipt, sort_keys=True), flush=True) + + def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--image", required=True) parser.add_argument("--binary", required=True, type=Path) + parser.add_argument("--indexed-inputs", action="store_true") + parser.add_argument("--receipt-dir", type=Path) args = parser.parse_args() security = json.loads(docker("info", "--format", "{{json .SecurityOptions}}").stdout) if "name=rootless" not in security: @@ -29,6 +240,11 @@ def main(): binary = args.binary.resolve(strict=True) with binary.open("rb") as reader: binary_sha256 = hashlib.file_digest(reader, "sha256").hexdigest() + if args.indexed_inputs: + if not args.receipt_dir or not args.receipt_dir.is_dir(): + raise RuntimeError("indexed fixture requires an existing receipt directory") + indexed_inputs_probe(image, binary, binary_sha256, args.receipt_dir) + return owned_name = "freshell-config-probe-" + uuid.uuid4().hex label = "com.freshell.provider-config-probe=" + owned_name with tempfile.TemporaryDirectory(prefix=owned_name) as temporary: From d628047a441dee290beb9d88f780fb2a5ea4aabf Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 08:41:04 -0700 Subject: [PATCH 030/493] fix: retain private source views before capability publication --- .../freshell-platform/src/opencode_recipe.rs | 35 +++ .../src/managed_mcp_capability.rs | 254 +++++++++++++++++- .../managed_mcp_capability/source_stage.rs | 89 ++++++ .../src/managed_provider_references.rs | 44 ++- .../managed_provider_references/sources.rs | 8 +- crates/freshell-server/src/managed_runtime.rs | 1 + scripts/sandbox-test.sh | 5 + 7 files changed, 423 insertions(+), 13 deletions(-) create mode 100644 crates/freshell-server/src/managed_mcp_capability/source_stage.rs diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index 921fe035a..b002eb195 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -22,6 +22,41 @@ pub struct GeneratedFileRecipe { pub occurrences: Vec, } +/// Private filesystem metadata linking one native loader occurrence to its +/// original source view. Documents and expanded values are never in this DTO. +#[derive(Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct SourceManifest { + pub version: u8, + pub cwd: PathBuf, + pub original_home: PathBuf, + pub sources: Vec, + pub files: Vec, +} +#[derive(Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct SourceView { + pub logical_path: PathBuf, + pub lane: SourceLane, + pub unexpanded_relative_path: String, + pub template_relative_path: String, + pub generated_name: Option, + pub occurrences: Vec, + pub referenced_variables: Vec, +} +#[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum SourceLane { + Main, + Tui, +} +#[derive(Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct SelectedSourceFile { + pub source_id: u16, + pub canonical_source: PathBuf, +} + pub fn evaluate( recipe: &GeneratedFileRecipe, env: &dyn Fn(&str) -> Option, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index decd9157b..00a5960dd 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -27,6 +27,7 @@ use tokio::io::{AsyncReadExt, AsyncWriteExt}; const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; +mod source_stage; #[derive(Clone, Copy, Default)] pub(crate) struct OpencodeEphemeralInput<'a> { @@ -35,6 +36,8 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub tui_source: Option<&'a ProviderConfigReference>, pub cwd: Option<&'a Path>, pub workspace: Option<&'a Path>, + pub source_plan: + Option<&'a [crate::managed_provider_references::sources::PlannedSourceOccurrence]>, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -917,6 +920,14 @@ fn stage_provider_root( bytes = report.bytes, elapsed_ms = started.elapsed().as_millis() as u64 ); + if let Some(plan) = opencode_input.source_plan { + source_stage::stage( + &temporary, + home, + opencode_input.cwd.ok_or("source plan requires cwd")?, + plan, + )?; + } allow_controller_stage_read(&temporary)?; std::fs::rename(&temporary, &staged).map_err(|error| error.to_string())?; Ok(staged.clone()) @@ -1558,12 +1569,20 @@ mod tests { os::unix::fs::{symlink, MetadataExt, PermissionsExt}, process::Command, }; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture_directory(); let original = root.path().join("original"); let external = root.path().join("external-skills"); let discovered = original.join(".agents/skills"); for directory in [&external, &discovered] { fs::create_dir_all(directory.join("fixture/scripts")).unwrap(); + // Give this synthetic package its own native resolution boundary; + // an executable tempdir under /workspace must not inherit the + // repository's unrelated installed dependency tree. + fs::write( + directory.join("package.json"), + r#"{"name":"synthetic-codex-skills","private":true,"type":"commonjs"}"#, + ) + .unwrap(); fs::write(directory.join("fixture/SKILL.md"), "fixture").unwrap(); fs::write(directory.join("fixture/resource"), "resource-nonce\n").unwrap(); let script = directory.join("fixture/scripts/run"); @@ -2338,6 +2357,7 @@ mod tests { tui_source: Some(&tui_source), cwd: Some(home.path()), workspace: Some(home.path()), + source_plan: None, }, None, ) @@ -2380,6 +2400,237 @@ mod tests { .contains("nested-secret-byte")); } + #[test] + fn managed_provider_source_stage_retains_unexpanded_direct_origins() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let config = workspace.join("opencode.json"); + let direct = r#"{"instructions":["{file:rules}"],"model":"{env:MODEL}"}"#; + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(&config, direct).unwrap(); + std::fs::write(workspace.join("rules"), "synthetic-private-selected-value").unwrap(); + use crate::managed_provider_references::sources::*; + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let staged = stage_provider_root( + &owned.path().join("capabilities"), + "grant-source-stage", + "opencode", + &home, + &[], + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + ..Default::default() + }, + None, + ) + .unwrap(); + let manifest: serde_json::Value = serde_json::from_slice( + &std::fs::read(staged.join(".freshell/input-manifest.json")) + .expect("preparation must publish its source manifest before the capability"), + ) + .unwrap(); + let source = &manifest["sources"][0]; + assert_eq!(source["logicalPath"], config.to_string_lossy().as_ref()); + assert_eq!(source["referencedVariables"], serde_json::json!(["MODEL"]), "host refresh must retain every finite source input, including variables that can introduce a later file token"); + let raw = std::fs::read_to_string( + staged.join(source["unexpandedRelativePath"].as_str().unwrap()), + ) + .unwrap(); + assert_eq!( + raw, direct, + "direct recipes also need the original source for host selection revalidation" + ); + let template = + std::fs::read_to_string(staged.join(source["templateRelativePath"].as_str().unwrap())) + .unwrap(); + assert!(template.contains("{env:MODEL}")); + assert!(template.contains("{file:/home/freshell/provider/.freshell/references/0}")); + assert!( + !std::fs::read_to_string(staged.join(".freshell/input-manifest.json")) + .unwrap() + .contains("synthetic-private-selected-value") + ); + assert_eq!(std::fs::read_to_string(&config).unwrap(), direct); + std::fs::remove_file(config).unwrap(); + assert_eq!( + std::fs::read_to_string( + staged.join(source["unexpandedRelativePath"].as_str().unwrap()) + ) + .unwrap(), + direct + ); + } + + #[test] + fn managed_provider_source_stage_generated_view_replays_without_original_tree() { + use crate::managed_provider_references::sources::*; + use freshell_platform::opencode_recipe::{GeneratedFileRecipe, SourceManifest}; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let config = workspace.join("tui.json"); + let raw = r#"{"message":"{env:INTRO}"}"#; + std::fs::write(&config, raw).unwrap(); + std::fs::write(workspace.join("rules"), "synthetic-file-value").unwrap(); + let lookup = + |name: &str| (name == "INTRO").then(|| "synthetic-child-value{file:rules}".into()); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Tui, + }], + &workspace, + &home, + &|_, _, name| lookup(name), + ) + .unwrap(); + let store = CapabilityStore::default(); + let directory = owned.path().join("capabilities"); + std::fs::create_dir(&directory).unwrap(); + let grant = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + ..Default::default() + }, + None, + ) + .unwrap(); + let staged = staged_provider_root_path(&directory, &grant.grant_id).unwrap(); + let metadata = std::fs::read(staged.join(".freshell/input-manifest.json")).unwrap(); + let manifest: SourceManifest = serde_json::from_slice(&metadata).unwrap(); + let view = &manifest.sources[0]; + assert!(view.lane == freshell_platform::opencode_recipe::SourceLane::Tui); + let raw_view = + std::fs::read_to_string(staged.join(&view.unexpanded_relative_path)).unwrap(); + assert_eq!(raw_view, raw); + let template = std::fs::read_to_string(staged.join(&view.template_relative_path)).unwrap(); + assert!(!template.contains("synthetic-child-value")); + assert!(!String::from_utf8(metadata) + .unwrap() + .contains("synthetic-child-value")); + assert!(!serde_json::to_string(&grant) + .unwrap() + .contains("synthetic-child-value")); + std::fs::remove_dir_all(&workspace).unwrap(); + let recipe = GeneratedFileRecipe { + generated_name: view.generated_name.clone().unwrap(), + unexpanded_source: raw_view, + declaring_config: view.logical_path.clone(), + cwd: manifest.cwd.clone(), + original_home: manifest.original_home.clone(), + occurrences: view.occurrences.clone(), + }; + assert_eq!( + freshell_platform::opencode_recipe::evaluate(&recipe, &lookup).unwrap(), + r#"{"message":"synthetic-child-value{file:/home/freshell/provider/.freshell/references/0}"}"# + ); + assert!( + freshell_platform::opencode_recipe::evaluate(&recipe, &|name| (name == "INTRO") + .then(|| "synthetic-child-value{file:other}".into())) + .is_err() + ); + } + + #[test] + fn managed_provider_source_stage_failure_preserves_prior_capability_and_views() { + use crate::managed_provider_references::sources::*; + let owned = tempfile::tempdir().unwrap(); + let directory = owned.path().join("capabilities"); + let workspace = owned.path().join("workspace"); + let home = owned.path().join("home"); + for path in [&directory, &workspace, &home] { + std::fs::create_dir(path).unwrap(); + } + let config = workspace.join("opencode.json"); + std::fs::write(&config, r#"{"instructions":["{file:rules}"]}"#).unwrap(); + std::fs::write(workspace.join("rules"), "synthetic-selected-file").unwrap(); + let mut plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let soul = SoulId::new(); + let store = CapabilityStore::default(); + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + ..Default::default() + }, + None, + ) + .unwrap(); + let old_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + let old_manifest = std::fs::read(old_stage.join(".freshell/input-manifest.json")).unwrap(); + plan[0].references.selections[0].runtime_destination = + "/home/freshell/provider/.freshell/references/99".into(); + assert!(store + .issue( + soul, + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + ..Default::default() + }, + None + ) + .is_err()); + assert_eq!( + std::fs::read(old_stage.join(".freshell/input-manifest.json")).unwrap(), + old_manifest + ); + assert!(grant_path(&directory, &prior.grant_id).unwrap().is_file()); + assert_eq!(store.grants.lock().unwrap().len(), 1); + assert_eq!(std::fs::read_dir(&directory).unwrap().count(), 1); + assert_eq!( + std::fs::read_dir(owned.path().join("provider-roots")) + .unwrap() + .count(), + 1, + "failed preparation must remove only its fresh temporary stage" + ); + } + #[test] fn opencode_private_inputs_survive_restarting_after_container_exits() { let control = tempfile::tempdir().unwrap(); @@ -2410,6 +2661,7 @@ mod tests { tui_source: Some(&tui_source), cwd: Some(home.path()), workspace: Some(home.path()), + source_plan: None, }, None, ) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs new file mode 100644 index 000000000..5ba267adb --- /dev/null +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -0,0 +1,89 @@ +//! Stage source views in the grant's private filesystem before publication. +use super::{private_directory, write_private_bytes}; +use crate::managed_provider_references::sources::{PlannedSourceOccurrence, SourceLane}; +use freshell_platform::opencode_recipe::{SelectedSourceFile, SourceManifest, SourceView}; +use std::{collections::BTreeMap, path::Path}; + +pub(super) fn stage( + root: &Path, + home: &Path, + cwd: &Path, + plan: &[PlannedSourceOccurrence], +) -> Result<(), String> { + let mut manifest = SourceManifest { + version: 1, + cwd: cwd.into(), + original_home: home.into(), + sources: Vec::new(), + files: Vec::new(), + }; + private_directory(&root.join(".freshell/source-views"))?; + private_directory(&root.join(".freshell/templates"))?; + let mut files = BTreeMap::new(); + let mut view_bytes = 0usize; + for (ordinal, source) in plan.iter().enumerate() { + view_bytes = view_bytes + .checked_add(source.references.unexpanded_source().len()) + .and_then(|value| value.checked_add(source.references.template.len())) + .ok_or("source view size overflow")?; + if source.references.unexpanded_source().len() > 16 * 1024 * 1024 + || source.references.template.len() > 16 * 1024 * 1024 + || view_bytes > 64 * 1024 * 1024 + || ordinal >= 8192 + { + return Err("source view profile exceeded before stage publication".into()); + } + let raw = format!(".freshell/source-views/source-{ordinal}"); + let template = format!(".freshell/templates/source-{ordinal}"); + write_private_bytes( + &root.join(&raw), + source.references.unexpanded_source().as_bytes(), + )?; + write_private_bytes(&root.join(&template), source.references.template.as_bytes())?; + for selection in &source.references.selections { + let index = selection + .runtime_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .and_then(|value| value.parse::().ok()) + .filter(|value| *value < 128) + .ok_or("source plan contains an invalid indexed destination")?; + if files + .insert(index, selection.canonical_source.clone()) + .is_some_and(|prior| prior != selection.canonical_source) + { + return Err("source plan changes one indexed source identity".into()); + } + } + manifest.sources.push(SourceView { + logical_path: source.source.logical_path.clone(), + lane: match source.source.lane { + SourceLane::Main => freshell_platform::opencode_recipe::SourceLane::Main, + SourceLane::Tui => freshell_platform::opencode_recipe::SourceLane::Tui, + }, + unexpanded_relative_path: raw, + template_relative_path: template, + generated_name: source + .references + .recipes + .first() + .map(|recipe| recipe.generated_name.clone()), + occurrences: source.references.occurrences.clone(), + referenced_variables: source.references.referenced_variables.clone(), + }); + } + for (expected, (source_id, canonical_source)) in files.into_iter().enumerate() { + if source_id as usize != expected { + return Err("source indexes are not contiguous".into()); + } + manifest.files.push(SelectedSourceFile { + source_id, + canonical_source, + }); + } + let metadata = freshell_runtime_protocol::bounded_json_bytes( + &manifest, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "source manifest metadata profile exceeded")?; + write_private_bytes(&root.join(".freshell/input-manifest.json"), &metadata) +} diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index d3c7bb6d7..248ac07e0 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -45,10 +45,25 @@ pub use freshell_platform::opencode_recipe::{ApprovedFileOccurrence, GeneratedFi pub struct ResolvedFileReferencePlan { pub template: String, + // Generated recipes already own the raw view. Keep one raw view for + // direct references too; a rewritten template cannot revalidate selection. + unexpanded_direct_source: Option, pub selections: Vec, pub referenced_variables: Vec, pub recipes: Vec, pub occurrences: Vec, + pub substitution_events: usize, +} +impl ResolvedFileReferencePlan { + pub fn unexpanded_source(&self) -> &str { + self.unexpanded_direct_source.as_deref().unwrap_or_else(|| { + &self + .recipes + .first() + .expect("generated plan owns its source") + .unexpanded_source + }) + } } pub fn resolve_file_references( @@ -80,7 +95,9 @@ pub(super) fn resolve_file_references_allocated( let mut generated = false; let mut planned_bytes = 0u64; let mut aliases = std::collections::BTreeSet::new(); - let mut variables = std::collections::BTreeSet::new(); + let variables = std::cell::RefCell::new(std::collections::BTreeSet::new()); + let substitution_events = std::cell::Cell::new(0usize); + let input_error = std::cell::Cell::new(None); let fail = |ordinal, variables: Vec, cause| ReferenceError { operation: "resolve", field: "OpenCode file", @@ -89,18 +106,28 @@ pub(super) fn resolve_file_references_allocated( variables, cause, }; - let transformed = transform_file_tokens(raw, env, |token| { + let lookup = |name: &str| { + let mut variables = variables.borrow_mut(); + if variables.len() >= 128 && !variables.contains(name) { + input_error.set(Some("referenced variable count exceeds 128")); + return None; + } + variables.insert(name.to_owned()); + substitution_events.set(substitution_events.get() + 1); + env(name) + }; + let transformed = transform_file_tokens(raw, &lookup, |token| { let names: Vec = token .variables .iter() .map(|name| name.to_string()) .collect(); - variables.extend(names.iter().cloned()); - if variables.len() > 128 { + substitution_events.set(substitution_events.get() + 1); + if substitution_events.get() > 8192 { return Err(fail( token.ordinal, names, - "referenced variable count exceeds 128", + "source substitution event count exceeds 8192", )); } generated |= !token.direct_expression; @@ -196,6 +223,9 @@ pub(super) fn resolve_file_references_allocated( "configuration source, expansion or event profile limit exceeded", ), })?; + if let Some(cause) = input_error.get() { + return Err(fail(0, Vec::new(), cause)); + } let (template, recipes) = if generated { let generated_name = format!("FRESHELL_CONFIG_REFERENCE_{recipe_ordinal}"); if transformed.len() > 64 * 1024 { @@ -237,10 +267,12 @@ pub(super) fn resolve_file_references_allocated( drop(transformed); Ok(ResolvedFileReferencePlan { template, + unexpanded_direct_source: (!generated).then(|| raw.to_owned()), selections, - referenced_variables: variables.into_iter().collect(), + referenced_variables: variables.into_inner().into_iter().collect(), recipes, occurrences, + substitution_events: substitution_events.get(), }) } diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index eaf561149..8af0e4653 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -256,11 +256,7 @@ pub fn plan_opencode_sources( recipe_count += references.recipes.len(); retained_view_bytes = retained_view_bytes .checked_add(references.template.len()) - .and_then(|bytes| { - references.recipes.iter().try_fold(bytes, |bytes, recipe| { - bytes.checked_add(recipe.unexpanded_source.len()) - }) - }) + .and_then(|bytes| bytes.checked_add(references.unexpanded_source().len())) .ok_or_else(|| fail("source-view byte count overflow"))?; if retained_view_bytes > 64 * 1024 * 1024 { return Err(fail("simultaneous retained source views exceed 64 MiB")); @@ -272,7 +268,7 @@ pub fn plan_opencode_sources( .map(|occurrence| occurrence.approved_alias.clone()), ); variables.extend(references.referenced_variables.iter().cloned()); - event_count += references.occurrences.len(); + event_count += references.substitution_events; if recipe_count > 128 || aliases.len() > 256 || variables.len() > 128 || event_count > 8192 { return Err(fail( diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 7cd0687bf..8f7a02f45 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -206,6 +206,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { tui_source: tui_source.as_ref(), cwd: Some(&cwd), workspace: Some(&workspace), + source_plan: None, } } else { Default::default() diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index acf4bda82..7516f732e 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -140,6 +140,11 @@ if [ "${RUNTIME_SUITE}" -eq 1 ]; then --read-only --tmpfs /tmp:rw,nosuid,nodev,size=512m --tmpfs /home/sandbox/.cache:rw,nosuid,nodev,size=256m + # Executable fixtures need a private package-lookup boundary. Reuse the + # same sandbox-owned target volume outside /workspace, where native + # ancestor lookup would otherwise select the repo's real node_modules. + -v freshell-sandbox-cargo-target:/freshell-test-artifacts + -e FRESHELL_TEST_EXECUTABLE_TMP_DIR=/freshell-test-artifacts --security-opt no-new-privileges -e FRESHELL_SANDBOX_MODE=runtime-suite ) From f89773d00d2ba66cf580037d52cb4fae15719e3d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 09:08:25 -0700 Subject: [PATCH 031/493] fix: bound final managed terminal and Claude sidecar exec inputs --- Cargo.lock | 1 + crates/freshell-freshagent/src/claude.rs | 60 ++++ crates/freshell-platform/Cargo.toml | 3 + crates/freshell-platform/src/lib.rs | 1 + crates/freshell-platform/src/provider_exec.rs | 277 ++++++++++++++++++ crates/freshell-session-host/src/pty.rs | 69 ++++- 6 files changed, 402 insertions(+), 9 deletions(-) create mode 100644 crates/freshell-platform/src/provider_exec.rs diff --git a/Cargo.lock b/Cargo.lock index ffd4847fc..d6bd18654 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1382,6 +1382,7 @@ dependencies = [ name = "freshell-platform" version = "0.1.0" dependencies = [ + "libc", "percent-encoding", "serde", "serde_json", diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index 107ad64f7..45501ae96 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -9535,6 +9535,8 @@ async fn spawn_sidecar() -> Result<(Child, ChildStdin, ChildStdout, String), Str let run_as = std::env::var("FRESHELL_PROVIDER_RUN_AS_UID") .ok() .zip(std::env::var("FRESHELL_PROVIDER_RUN_AS_GID").ok()); + #[cfg(target_os = "linux")] + let run_as_wrapped = run_as.is_some(); let mut cmd = if let Some((uid, gid)) = run_as { let mut command = tokio::process::Command::new("/usr/bin/setpriv"); command.args([ @@ -9570,6 +9572,17 @@ async fn spawn_sidecar() -> Result<(Child, ChildStdin, ChildStdout, String), Str .stderr(Stdio::piped()); cmd.kill_on_drop(true); + #[cfg(target_os = "linux")] + if freshell_platform::managed_child_secrets::snapshot().is_some() { + let wrapper_targets = [node.as_str()]; + let additional = if run_as_wrapped { + &wrapper_targets[..] + } else { + &[] + }; + freshell_platform::provider_exec::check_command(cmd.as_std(), true, &additional) + .map_err(|error| error.to_string())?; + } let mut child = cmd.spawn().map_err(|e| { format!( "claude sidecar spawn failed ({node} {}): {e}", @@ -9585,6 +9598,53 @@ async fn spawn_sidecar() -> Result<(Child, ChildStdin, ChildStdout, String), Str Ok((child, stdin, stdout, ownership_id)) } +#[cfg(all(test, target_os = "linux"))] +mod provider_exec_tests { + use super::*; + + #[tokio::test] + async fn provider_exec_profile_checks_actual_managed_claude_sidecar_spawn() { + const CASE: &str = "FRESHELL_EXEC_PROFILE_FIXTURE_CHILD"; + if let Ok(case) = std::env::var(CASE) { + let owned = tempfile::tempdir().unwrap(); + let script = owned.path().join("synthetic-sidecar.mjs"); + std::fs::write(&script, "setInterval(()=>{},1000)\n").unwrap(); + std::env::set_var("FRESHELL_CLAUDE_SIDECAR", &script); + let count = if case == "large" { 17 } else { 1 }; + freshell_platform::managed_child_secrets::install( + (0..count) + .map(|i| (format!("FRESHELL_EXEC_FIXTURE_{i}"), "x".repeat(64 * 1024))) + .collect(), + ) + .unwrap(); + let mut result = spawn_sidecar().await; + if let Ok((child, _, _, _)) = &mut result { + let pid = child.id().unwrap(); + child.kill().await.unwrap(); + child.wait().await.unwrap(); + eprintln!("synthetic owned sidecar pid {pid} stopped in fixture namespace"); + } + if case == "large" { + assert!(matches!(result,Err(ref cause) if cause.contains("provider exec profile")),"managed fresh sidecar must reject oversized actual inherited+selected environment before spawn"); + } else { + assert!(result.is_ok(),"valid finite selected environment must still launch the actual sidecar command"); + } + return; + } + for case in ["large", "small"] { + let output=std::process::Command::new(std::env::current_exe().unwrap()).env_clear() + .env("PATH","/usr/local/bin:/usr/bin:/bin").env(CASE,case) + .args(["--exact","claude::provider_exec_tests::provider_exec_profile_checks_actual_managed_claude_sidecar_spawn","--nocapture"]) + .output().unwrap(); + assert!( + output.status.success(), + "actual sidecar fixture case {case} failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + } + } +} + /// Read the sidecar's stdout until the `created` (→ the nanoid placeholder) or /// `create.failed` control line, bounded by `budget`. EOF before either is a failure. async fn read_created( diff --git a/crates/freshell-platform/Cargo.toml b/crates/freshell-platform/Cargo.toml index 8a4f32ce4..d1862a611 100644 --- a/crates/freshell-platform/Cargo.toml +++ b/crates/freshell-platform/Cargo.toml @@ -15,5 +15,8 @@ serde_json = { workspace = true } serde = { workspace = true } url = "2" +[target.'cfg(target_os = "linux")'.dependencies] +libc = "=0.2.186" + [dev-dependencies] tempfile = "3" diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 0e2d10ee8..d5c0b7011 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -65,6 +65,7 @@ pub mod opencode_recipe; pub mod path; pub mod provider_environment; pub mod provider_environment_grants; +pub mod provider_exec; pub mod provider_input_exchange; pub mod resume_gate; pub mod spawn; diff --git a/crates/freshell-platform/src/provider_exec.rs b/crates/freshell-platform/src/provider_exec.rs new file mode 100644 index 000000000..7b58b83c0 --- /dev/null +++ b/crates/freshell-platform/src/provider_exec.rs @@ -0,0 +1,277 @@ +//! Final managed-provider exec limits. Diagnostics carry classifications only. +use std::{collections::BTreeMap, io::Read}; + +pub const MAX_EXEC_BYTES: usize = 1024 * 1024; +pub const MIN_STACK_BYTES: u64 = 8 * 1024 * 1024; + +#[derive(Clone, Copy)] +pub struct Profile { + pub page_bytes: usize, + pub pointer_bytes: usize, + pub soft_stack_bytes: u64, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ExecError { + NativeProfile, + Stack, + String, + Total, + Executable, + Interpreter, +} +impl std::fmt::Display for ExecError { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + formatter, + "provider exec profile {:?}: repair the launch inputs before dispatch", + self + ) + } +} +impl std::error::Error for ExecError {} + +#[cfg(target_os = "linux")] +pub fn process_profile() -> Result { + let mut stack = std::mem::MaybeUninit::::uninit(); + // getrlimit initializes the complete result on success; it changes no + // process limits. sysconf queries the actual target's page size. + if unsafe { libc::getrlimit(libc::RLIMIT_STACK, stack.as_mut_ptr()) } != 0 { + return Err(ExecError::NativeProfile); + } + let page = unsafe { libc::sysconf(libc::_SC_PAGESIZE) }; + if page <= 0 { + return Err(ExecError::NativeProfile); + } + let stack = unsafe { stack.assume_init() }; + Ok(Profile { + page_bytes: page as usize, + pointer_bytes: std::mem::size_of::(), + soft_stack_bytes: stack.rlim_cur, + }) +} + +/// Check the exact standard Command overlay. The caller states whether its +/// command inherits the process environment; Rust exposes overlays separately. +#[cfg(target_os = "linux")] +pub fn check_command( + command: &std::process::Command, + inherit_environment: bool, + additional_programs: &[&str], +) -> Result<(), ExecError> { + let mut environment = BTreeMap::::new(); + if inherit_environment { + for (name, value) in std::env::vars_os() { + environment.insert( + name.into_string().map_err(|_| ExecError::String)?, + value.into_string().map_err(|_| ExecError::String)?, + ); + } + } + for (name, value) in command.get_envs() { + let name = name.to_str().ok_or(ExecError::String)?; + match value { + Some(value) => { + environment.insert(name.into(), value.to_str().ok_or(ExecError::String)?.into()); + } + None => { + environment.remove(name); + } + } + } + let program = command + .get_program() + .to_str() + .ok_or(ExecError::Executable)?; + let args = command + .get_args() + .map(|arg| arg.to_str().map(str::to_owned).ok_or(ExecError::String)) + .collect::, _>>()?; + check_final_exec( + program, + &args, + &environment, + additional_programs, + process_profile()?, + ) +} + +struct Accounting { + profile: Profile, + bytes: usize, + strings: usize, +} +impl Accounting { + fn string(&mut self, bytes: usize) -> Result<(), ExecError> { + let bytes = bytes.checked_add(1).ok_or(ExecError::String)?; + if bytes + > self + .profile + .page_bytes + .checked_mul(32) + .ok_or(ExecError::NativeProfile)? + { + return Err(ExecError::String); + } + self.bytes = self.bytes.checked_add(bytes).ok_or(ExecError::Total)?; + self.strings = self.strings.checked_add(1).ok_or(ExecError::Total)?; + self.check() + } + fn check(&self) -> Result<(), ExecError> { + let pointers = self + .strings + .checked_add(2) + .and_then(|count| count.checked_mul(self.profile.pointer_bytes)) + .ok_or(ExecError::Total)?; + if self + .bytes + .checked_add(pointers) + .is_none_or(|bytes| bytes > MAX_EXEC_BYTES) + { + return Err(ExecError::Total); + } + Ok(()) + } +} + +/// `additional_programs` names fixed wrapper/provider executables whose own +/// exec occurs after the first program (for example setpriv -> provider). +/// This inspects only executable headers, never interprets command/argv text. +pub fn check_final_exec( + program: &str, + args: &[String], + environment: &BTreeMap, + additional_programs: &[&str], + profile: Profile, +) -> Result<(), ExecError> { + if profile.soft_stack_bytes < MIN_STACK_BYTES { + return Err(ExecError::Stack); + } + if profile.page_bytes == 0 || profile.pointer_bytes == 0 { + return Err(ExecError::NativeProfile); + } + let mut accounting = Accounting { + profile, + bytes: 0, + strings: 0, + }; + if program.contains('\0') { + return Err(ExecError::String); + } + accounting.string(program.len())?; + for argument in args { + if argument.contains('\0') { + return Err(ExecError::String); + } + accounting.string(argument.len())?; + } + for (name, value) in environment { + if name.is_empty() || name.contains(['=', '\0']) || value.contains('\0') { + return Err(ExecError::String); + } + accounting.string( + name.len() + .checked_add(1) + .and_then(|bytes| bytes.checked_add(value.len())) + .ok_or(ExecError::String)?, + )?; + } + for executable in std::iter::once(program).chain(additional_programs.iter().copied()) { + let resolved = crate::path::resolve_program_via_path( + executable, + environment.get("PATH").map(String::as_str), + ) + .map_err(|_| ExecError::Executable)?; + account_executable(&resolved, &mut accounting, 0)?; + } + accounting.check() +} + +fn account_executable( + program: &str, + accounting: &mut Accounting, + depth: usize, +) -> Result<(), ExecError> { + if depth > 4 { + return Err(ExecError::Interpreter); + } + // execfn and every kernel-added script/interpreter string consume space + // in addition to the supplied argv and its pointers. + accounting.string(program.len())?; + let mut header = [0u8; 256]; + let mut file = std::fs::File::open(program).map_err(|_| ExecError::Executable)?; + let count = file.read(&mut header).map_err(|_| ExecError::Executable)?; + if count < 2 || &header[..2] != b"#!" { + return Ok(()); + } + let line = &header[2..count]; + let end = line + .iter() + .position(|byte| matches!(byte, b'\n' | 0)) + .unwrap_or(line.len()); + let line = std::str::from_utf8(&line[..end]) + .map_err(|_| ExecError::Interpreter)? + .trim_matches([' ', '\t']); + let split = line.find([' ', '\t']).unwrap_or(line.len()); + let interpreter = &line[..split]; + if interpreter.is_empty() { + return Err(ExecError::Interpreter); + } + let argument = line[split..].trim_matches([' ', '\t']); + if !argument.is_empty() { + accounting.string(argument.len())?; + } + account_executable(interpreter, accounting, depth + 1) +} + +#[cfg(all(test, target_os = "linux"))] +mod tests { + use super::*; + fn profile() -> Profile { + Profile { + page_bytes: 4096, + pointer_bytes: 8, + soft_stack_bytes: MIN_STACK_BYTES, + } + } + #[test] + fn provider_exec_profile_counts_nul_pointers_and_complete_environment_strings() { + let arguments = vec!["x".repeat(131_060); 8]; + assert_eq!( + check_final_exec("/bin/true", &arguments, &BTreeMap::new(), &[], profile()), + Err(ExecError::Total) + ); + let environment = BTreeMap::from([("INPUT".into(), "x".repeat(32 * 4096 - 6))]); + assert_eq!( + check_final_exec("/bin/true", &[], &environment, &[], profile()), + Err(ExecError::String) + ); + assert!(check_final_exec( + "/bin/true", + &[], + &BTreeMap::from([("INPUT".into(), "x".repeat(64 * 1024))]), + &[], + profile() + ) + .is_ok()); + let mut low = profile(); + low.soft_stack_bytes -= 1; + assert_eq!( + check_final_exec("/bin/true", &[], &BTreeMap::new(), &[], low), + Err(ExecError::Stack) + ); + } + #[test] + fn provider_exec_profile_accounts_actual_shebang_additions() { + let owned = tempfile::tempdir().unwrap(); + let script = owned.path().join("fixture"); + std::fs::write(&script, b"#!/bin/sh -eu\nexit 0\n").unwrap(); + let program = script.to_str().unwrap(); + assert!(check_final_exec(program, &[], &BTreeMap::new(), &[], profile()).is_ok()); + std::fs::write(&script, format!("#!{}\n", script.display())).unwrap(); + assert_eq!( + check_final_exec(program, &[], &BTreeMap::new(), &[], profile()), + Err(ExecError::Interpreter) + ); + } +} diff --git a/crates/freshell-session-host/src/pty.rs b/crates/freshell-session-host/src/pty.rs index 6129c97d2..de4047da7 100644 --- a/crates/freshell-session-host/src/pty.rs +++ b/crates/freshell-session-host/src/pty.rs @@ -110,15 +110,29 @@ impl HostedPty { provider_env.insert("GIT_CONFIG_COUNT".into(), "1".into()); provider_env.insert("GIT_CONFIG_KEY_0".into(), "safe.directory".into()); provider_env.insert("GIT_CONFIG_VALUE_0".into(), launch.workspace_path.clone()); - let mut pty = match PtyTerminal::spawn_with_sink( - &spec, - &provider_env, - &launch.terminal_id, - &launch.stream_id, - None, - Some(sink), - Some(on_exit), - ) { + let spawn = (|| { + #[cfg(target_os = "linux")] + freshell_platform::provider_exec::check_final_exec( + &spec.program, + &spec.args, + &provider_env, + &[&launch.program], + freshell_platform::provider_exec::process_profile() + .map_err(|error| error.to_string())?, + ) + .map_err(|error| error.to_string())?; + PtyTerminal::spawn_with_sink( + &spec, + &provider_env, + &launch.terminal_id, + &launch.stream_id, + None, + Some(sink), + Some(on_exit), + ) + .map_err(|error| error.to_string()) + })(); + let mut pty = match spawn { Ok(pty) => pty, Err(error) => { if let Some(prepared) = prepared_codex { @@ -706,3 +720,40 @@ mod codex_identity_tests { assert!(validate_codex_candidate(None, Some("thread-a"), "thread-b", false).is_err()); } } + +#[cfg(all(test, target_os = "linux"))] +mod provider_exec_tests { + use super::*; + + #[tokio::test] + async fn provider_exec_profile_rejects_combined_final_environment_before_pty_dispatch() { + let owned = tempfile::tempdir().unwrap(); + let launch: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"exec-profile","streamId":"exec-profile-stream","mode":"codex", + "program":"/usr/bin/true","cwd":owned.path(),"workspacePath":owned.path(), + "runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"exec-profile", + "env":{"PATH":"/usr/bin:/bin"} + })) + .unwrap(); + let environment = (0..17) + .map(|i| (format!("FRESHELL_EXEC_FIXTURE_{i}"), "x".repeat(64 * 1024))) + .collect(); + let mut result = HostedPty::spawn( + owned.path(), + IncarnationId::new(), + &launch, + environment, + None, + ) + .await; + // This regression owns and stops only the unexpectedly dispatched + // synthetic PTY, inside the required lifecycle sandbox. + if let Ok(pty) = &mut result { + pty.stop().await; + } + assert!( + matches!(result,Err(ref cause) if cause.contains("provider exec profile")), + "oversized final argv+env must fail with a classified pre-dispatch profile error" + ); + } +} From 269b31ffdebabf72938d6a7a923c3222faeffa76 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 09:44:24 -0700 Subject: [PATCH 032/493] fix: check managed Codex and SDK grandchild exec accounting --- Cargo.lock | 1 + .../freshell-claude-sidecar/exec-profile.mjs | 92 +++++++++++++++++++ crates/freshell-claude-sidecar/index.mjs | 7 +- .../freshell-claude-sidecar/model-catalog.mjs | 7 +- crates/freshell-codex/Cargo.toml | 3 +- crates/freshell-codex/src/launch_lifecycle.rs | 44 +++++++++ crates/freshell-codex/src/launch_plan.rs | 4 + .../freshell-codex/tests/launch_lifecycle.rs | 1 + crates/freshell-freshagent/src/codex.rs | 69 ++++++++++++++ .../freshell-freshagent/src/terminal_tabs.rs | 1 + .../src/providers/codex.rs | 1 + .../src/providers/fresh_agent.rs | 1 + crates/freshell-ws/src/terminal.rs | 1 + test/unit/claude-sidecar/exec-profile.test.ts | 54 +++++++++++ .../fixtures/exec-profile-query-module.mjs | 35 +++++++ 15 files changed, 318 insertions(+), 3 deletions(-) create mode 100644 crates/freshell-claude-sidecar/exec-profile.mjs create mode 100644 test/unit/claude-sidecar/exec-profile.test.ts create mode 100644 test/unit/claude-sidecar/fixtures/exec-profile-query-module.mjs diff --git a/Cargo.lock b/Cargo.lock index d6bd18654..a2501e9f3 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1306,6 +1306,7 @@ dependencies = [ name = "freshell-codex" version = "0.1.0" dependencies = [ + "freshell-platform", "futures-util", "libc", "serde", diff --git a/crates/freshell-claude-sidecar/exec-profile.mjs b/crates/freshell-claude-sidecar/exec-profile.mjs new file mode 100644 index 000000000..d6aa400bb --- /dev/null +++ b/crates/freshell-claude-sidecar/exec-profile.mjs @@ -0,0 +1,92 @@ +// Final exec accounting for the pinned SDK's supported spawn callback. +// Expanded values remain in process memory; errors contain classifications. +import { spawn } from 'node:child_process' +import { openSync, readSync, closeSync, statSync, accessSync, constants } from 'node:fs' +import { endianness } from 'node:os' +import path from 'node:path' + +const fail = kind => { throw new Error(`provider exec profile ${kind}: repair inputs before dispatch`) } +function readBounded(file, limit) { + let fd + try { + fd = openSync(file, 'r') + const data = Buffer.alloc(limit) + return data.subarray(0, readSync(fd, data, 0, limit, 0)) + } catch { return fail('executable-or-native-profile') } + finally { if (fd !== undefined) closeSync(fd) } +} +function nativeProfile() { + const pointerBytes = ['ia32', 'arm'].includes(process.arch) ? 4 : 8 + const data = readBounded('/proc/self/auxv', 4096) + const little = endianness() === 'LE' + const read = offset => pointerBytes === 4 + ? (little ? data.readUInt32LE(offset) : data.readUInt32BE(offset)) + : Number(little ? data.readBigUInt64LE(offset) : data.readBigUInt64BE(offset)) + let pageBytes + for (let offset = 0; offset + 2 * pointerBytes <= data.length; offset += 2 * pointerBytes) { + if (read(offset) === 6) { pageBytes = read(offset + pointerBytes); break } + } + const limits = readBounded('/proc/self/limits', 16384).toString() + const soft = limits.split('\n').find(line => line.startsWith('Max stack size'))?.slice('Max stack size'.length).trim().split(/\s+/)[0] + const stackBytes = soft === 'unlimited' ? Infinity : Number(soft) + if (!Number.isSafeInteger(pageBytes) || pageBytes <= 0 || !soft || Number.isNaN(stackBytes)) fail('native-profile') + if (stackBytes < 8 * 1024 * 1024) fail('stack') + return { pointerBytes, pageBytes } +} +function resolveProgram(command, cwd, environment) { + if (command.includes('/')) return path.resolve(cwd || process.cwd(), command) + for (const directory of (environment.PATH ?? '/usr/bin:/bin').split(':')) { + const candidate = path.resolve(cwd || process.cwd(), directory, command) + try { if (!statSync(candidate).isFile()) continue; accessSync(candidate, constants.X_OK); return candidate } catch {} + } + return fail('executable') +} + +export function checkSdkExec(options) { + const profile = nativeProfile() + let bytes = 0 + let strings = 0 + const account = text => { + if (typeof text !== 'string' || text.includes('\0')) fail('string') + const size = Buffer.byteLength(text) + 1 + if (size > 32 * profile.pageBytes) fail('string') + bytes += size + strings += 1 + if (bytes + (strings + 2) * profile.pointerBytes > 1024 * 1024) fail('total') + } + account(options.command) + for (const arg of options.args) account(arg) + const environment = options.env ?? process.env + for (const [name, value] of Object.entries(environment)) { + if (value === undefined) continue + if (!name || name.includes('=') || name.includes('\0')) fail('string') + account(`${name}=${value}`) + } + const executable = (program, depth = 0) => { + if (depth > 4) fail('interpreter') + account(program) + const header = readBounded(program, 256) + if (header.subarray(0, 2).toString() !== '#!') return + const line = header.subarray(2).toString().split(/[\n\0]/)[0].replace(/^[ \t]+|[ \t]+$/g, '') + const split = line.search(/[ \t]/) + const interpreter = split < 0 ? line : line.slice(0, split) + if (!interpreter) fail('interpreter') + const argument = split < 0 ? '' : line.slice(split).replace(/^[ \t]+|[ \t]+$/g, '') + if (argument) account(argument) + executable(interpreter, depth + 1) + } + executable(resolveProgram(options.command, options.cwd, environment)) +} + +export function managedSdkSpawn(stderr, environment = process.env) { + if (process.platform !== 'linux' || environment.FRESHELL_HOSTED_FRESH_AGENT !== 'claude') return undefined + return options => { + checkSdkExec(options) + const child = spawn(options.command, options.args, { + cwd: options.cwd, env: options.env, signal: options.signal, + stdio: ['pipe', 'pipe', 'pipe'], windowsHide: true, + }) + child.stderr.on('data', stderr) + return child + } +} diff --git a/crates/freshell-claude-sidecar/index.mjs b/crates/freshell-claude-sidecar/index.mjs index 26017022f..7299e768b 100644 --- a/crates/freshell-claude-sidecar/index.mjs +++ b/crates/freshell-claude-sidecar/index.mjs @@ -78,6 +78,7 @@ import { randomBytes } from 'node:crypto' import { configureSession, userMessageContent, resultErrorMessage } from './session-settings.mjs' import { createTurnCompleteGate } from './turn-complete-gate.mjs' import { nextMonotonic } from './monotonic-clock.mjs' +import { managedSdkSpawn } from './exec-profile.mjs' import { canUseTool as routeCanUseTool, cancelPending, @@ -395,6 +396,9 @@ function handleCreate(req) { // events (macrotasks) cannot interleave inside that finally. sessions.set(sessionId, state) + const sdkStderr = process.env.FRESHELL_HOSTED_FRESH_AGENT === 'claude' + ? data => process.stderr.write(`${JSON.stringify({ severity: 'warn', component: 'claude-sidecar', event: 'provider.stderr', bytes: Buffer.byteLength(String(data)) })}\n`) + : data => logerr(`sdk stderr: ${String(data).trimEnd()}`) const sdkQuery = query({ prompt: iterable, options: { @@ -420,7 +424,8 @@ function handleCreate(req) { abortController: abort, env: createClaudeSdkCleanEnv(process.env), settingSources: ['user', 'project', 'local'], - stderr: (data) => logerr(`sdk stderr: ${String(data).trimEnd()}`), + spawnClaudeCodeProcess: managedSdkSpawn(sdkStderr), + stderr: sdkStderr, canUseTool: async (toolName, input, options) => { const s = sessions.get(sessionId) if (!s) return { behavior: 'allow', updatedInput: input } diff --git a/crates/freshell-claude-sidecar/model-catalog.mjs b/crates/freshell-claude-sidecar/model-catalog.mjs index 5e171e0ca..cb1bc5308 100644 --- a/crates/freshell-claude-sidecar/model-catalog.mjs +++ b/crates/freshell-claude-sidecar/model-catalog.mjs @@ -1,5 +1,6 @@ import { realpathSync } from 'node:fs' import { fileURLToPath, pathToFileURL } from 'node:url' +import { managedSdkSpawn } from './exec-profile.mjs' const log = (message, details = {}) => process.stderr.write(`${JSON.stringify({ severity: 'warn', component: 'claude-model-catalog', message, ...details })}\n`) @@ -7,6 +8,9 @@ const log = (message, details = {}) => process.stderr.write(`${JSON.stringify({ export async function probeModelCatalog(query, { env = process.env, timeoutMs = 10_000 } = {}) { const { CLAUDECODE: _claudeCode, ANTHROPIC_API_KEY: _apiKey, ...cleanEnv } = env const abortController = new AbortController() + const stderr = env.FRESHELL_HOSTED_FRESH_AGENT === 'claude' + ? data => log('Claude capability probe stderr', { bytes: Buffer.byteLength(String(data)) }) + : data => log('Claude capability probe stderr', { data: String(data).trimEnd() }) const session = query({ prompt: (async function* () {})(), options: { @@ -14,7 +18,8 @@ export async function probeModelCatalog(query, { env = process.env, timeoutMs = env: cleanEnv, pathToClaudeCodeExecutable: env.CLAUDE_CMD || undefined, settingSources: ['user', 'project', 'local'], - stderr: (data) => log('Claude capability probe stderr', { data: String(data).trimEnd() }), + spawnClaudeCodeProcess: managedSdkSpawn(stderr, env), + stderr, }, }) let timer diff --git a/crates/freshell-codex/Cargo.toml b/crates/freshell-codex/Cargo.toml index d35758714..76dae4163 100644 --- a/crates/freshell-codex/Cargo.toml +++ b/crates/freshell-codex/Cargo.toml @@ -31,7 +31,7 @@ publish.workspace = true # feature (verified to compile; wired live in the next step). Default-off keeps the shared # workspace build lean. default = [] -real-transport = ["dep:tokio-tungstenite", "dep:futures-util", "dep:libc", "tokio/net", "tokio/io-util", "tokio/process"] +real-transport = ["dep:tokio-tungstenite", "dep:futures-util", "dep:libc", "dep:freshell-platform", "tokio/net", "tokio/io-util", "tokio/process"] [dependencies] # Derive for the durable sidecar record rows (`sidecar_store`) — the versioned @@ -61,6 +61,7 @@ futures-util = { version = "0.3", default-features = false, features = ["sink", # FRESHELL_CODEX_SIDECAR_ID tag (runtime.ts:494) so no orphan survives (the oracle # `ownership.cleanup` invariant). Only needed by the real transport. libc = { version = "0.2", optional = true } +freshell-platform = { path = "../freshell-platform", optional = true } [dev-dependencies] # Tempdirs for the sidecar-store unit tests — no global state, nothing outside diff --git a/crates/freshell-codex/src/launch_lifecycle.rs b/crates/freshell-codex/src/launch_lifecycle.rs index 2a7c6f0a1..720236e7c 100644 --- a/crates/freshell-codex/src/launch_lifecycle.rs +++ b/crates/freshell-codex/src/launch_lifecycle.rs @@ -1212,6 +1212,16 @@ impl CodexLaunchRuntime for SpawnedCodexAppServerRuntime { cmd.kill_on_drop(true); } + #[cfg(target_os = "linux")] + if self.sidecar_context.provider_exec_profile { + let additional = if program == "/usr/bin/setpriv" { + &["codex"][..] + } else { + &[] + }; + freshell_platform::provider_exec::check_command(cmd.as_std(), true, additional) + .map_err(|error| error.to_string())?; + } let mut child = cmd .spawn() .map_err(|error| format!("codex app-server spawn failed ({command}): {error}"))?; @@ -1483,6 +1493,40 @@ async fn persist_record_or_terminate_exact_child( #[cfg(test)] mod tests { + #[cfg(all(feature = "real-transport", target_os = "linux"))] + #[tokio::test] + async fn provider_exec_profile_rejects_actual_managed_codex_app_server_spawn() { + use super::*; + for count in [17, 1] { + let context = crate::launch_plan::CodexSidecarLaunchContext { + provider_exec_profile: true, + config_args: vec![], + env: (0..count) + .map(|i| (format!("FRESHELL_EXEC_FIXTURE_{i}"), "x".repeat(64 * 1024))) + .collect(), + }; + let runtime = SpawnedCodexAppServerRuntime { + codex_command: Some(format!( + "node {}/../../test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs", + env!("CARGO_MANIFEST_DIR") + )), + ..SpawnedCodexAppServerRuntime::with_context(context) + }; + let result = runtime.ensure_ready(None).await; + runtime + .shutdown() + .await + .expect("cleanup only this exact synthetic app-server in lifecycle sandbox"); + if count == 17 { + assert!(matches!(result,Err(ref cause) if cause.contains("provider exec profile")),"managed terminal app-server must reject its complete oversized environment before dispatch"); + } else { + assert!( + result.is_ok(), + "valid selected inputs must still execute the actual app-server handshake" + ); + } + } + } use super::*; #[tokio::test] diff --git a/crates/freshell-codex/src/launch_plan.rs b/crates/freshell-codex/src/launch_plan.rs index 613958866..2af1c6bb2 100644 --- a/crates/freshell-codex/src/launch_plan.rs +++ b/crates/freshell-codex/src/launch_plan.rs @@ -324,6 +324,8 @@ pub fn codex_remote_args(proxy_ws_url: &str) -> Result<[String; 4], CodexRemoteA /// child process environment and are intentionally redacted from [`fmt::Debug`]. #[derive(Clone, PartialEq, Eq, Default)] pub struct CodexSidecarLaunchContext { + /// The managed soul host requests final argv/env validation before spawn. + pub provider_exec_profile: bool, pub config_args: Vec, pub env: BTreeMap, } @@ -331,6 +333,7 @@ pub struct CodexSidecarLaunchContext { impl fmt::Debug for CodexSidecarLaunchContext { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.debug_struct("CodexSidecarLaunchContext") + .field("provider_exec_profile", &self.provider_exec_profile) .field("config_args", &self.config_args) .field("env_keys", &self.env.keys().collect::>()) .finish() @@ -748,6 +751,7 @@ mod tests { #[test] fn sidecar_spawn_spec_places_context_before_app_server_and_redacts_values() { let context = CodexSidecarLaunchContext { + provider_exec_profile: false, config_args: vec![ "-c".to_string(), "mcp_servers.freshell.command=\"node\"".to_string(), diff --git a/crates/freshell-codex/tests/launch_lifecycle.rs b/crates/freshell-codex/tests/launch_lifecycle.rs index ba7abaef9..cb9ae9473 100644 --- a/crates/freshell-codex/tests/launch_lifecycle.rs +++ b/crates/freshell-codex/tests/launch_lifecycle.rs @@ -78,6 +78,7 @@ fn context_config_args() -> Vec { fn sidecar_context(arg_log: &std::path::Path) -> CodexSidecarLaunchContext { CodexSidecarLaunchContext { + provider_exec_profile: false, config_args: context_config_args(), env: BTreeMap::from([ ( diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index 06fe77a32..73e641cc3 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -7164,6 +7164,16 @@ impl FreshCodexState { .stderr(Stdio::piped()); cmd.kill_on_drop(true); + #[cfg(target_os = "linux")] + if self.sidecar_launch_context.provider_exec_profile { + let additional = if codex_program == "/usr/bin/setpriv" { + &["codex"][..] + } else { + &[] + }; + freshell_platform::provider_exec::check_command(cmd.as_std(), true, additional) + .map_err(|error| error.to_string())?; + } let mut child = cmd .spawn() .map_err(|e| format!("codex app-server spawn failed ({codex_cmd}): {e}"))?; @@ -28765,3 +28775,62 @@ pub(crate) mod tests { ); } } + +#[cfg(all(test, target_os = "linux"))] +mod provider_exec_tests { + use super::*; + #[tokio::test] + async fn provider_exec_profile_rejects_actual_managed_fresh_codex_spawn() { + const CASE: &str = "FRESHELL_CODEX_EXEC_PROFILE_CHILD"; + if let Ok(case) = std::env::var(CASE) { + let count = if case == "large" { 17 } else { 1 }; + let (tx, _rx) = tokio::sync::broadcast::channel(16); + let state = FreshCodexState::new( + Arc::new("synthetic-token".into()), + Arc::new(tx), + serde_json::json!({"freshAgent":{"enabled":false}}), + ) + .with_sidecar_launch_context( + freshell_codex::launch_plan::CodexSidecarLaunchContext { + provider_exec_profile: true, + config_args: vec![], + env: (0..count) + .map(|i| (format!("FRESHELL_EXEC_FIXTURE_{i}"), "x".repeat(64 * 1024))) + .collect(), + }, + ); + std::env::set_var( + "CODEX_CMD", + format!( + "node {}/../../test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs", + env!("CARGO_MANIFEST_DIR") + ), + ); + let mut result = state.spawn_sidecar_notifying(None, None).await; + if let Ok((_, _, _, child)) = &mut result { + child.kill().await.unwrap(); + child.wait().await.unwrap(); + } + if case == "large" { + assert!( + matches!(result,Err(ref cause) if cause.contains("provider exec profile")), + "managed fresh Codex must reject its actual full argv+env before spawn" + ); + } else { + assert!(result.is_ok(),"valid selected fresh Codex inputs must execute the actual app-server handshake"); + } + return; + } + for case in ["large", "small"] { + let output=std::process::Command::new(std::env::current_exe().unwrap()).env_clear() + .env("PATH","/usr/local/bin:/usr/bin:/bin").env(CASE,case) + .args(["--exact","codex::provider_exec_tests::provider_exec_profile_rejects_actual_managed_fresh_codex_spawn","--nocapture"]) + .output().unwrap(); + assert!( + output.status.success(), + "actual fresh Codex exec fixture failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + } + } +} diff --git a/crates/freshell-freshagent/src/terminal_tabs.rs b/crates/freshell-freshagent/src/terminal_tabs.rs index 92b4f638c..6d426131e 100644 --- a/crates/freshell-freshagent/src/terminal_tabs.rs +++ b/crates/freshell-freshagent/src/terminal_tabs.rs @@ -456,6 +456,7 @@ fn build_codex_managed_launch_setup( tui_mcp_injection: tui, terminal_env, sidecar_context: freshell_codex::launch_plan::CodexSidecarLaunchContext { + provider_exec_profile: false, config_args: sidecar.args, env: sidecar_env, }, diff --git a/crates/freshell-session-host/src/providers/codex.rs b/crates/freshell-session-host/src/providers/codex.rs index c4ef05b7e..2acaac4ff 100644 --- a/crates/freshell-session-host/src/providers/codex.rs +++ b/crates/freshell-session-host/src/providers/codex.rs @@ -35,6 +35,7 @@ pub async fn prepare( // It has no web-owned MCP context to inherit, so use the explicit // empty launch context rather than silently reintroducing web state. sidecar_context: CodexSidecarLaunchContext { + provider_exec_profile: true, config_args: terminal .provider_launch_context .as_ref() diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index 19a173a9a..bd07cc4c7 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -173,6 +173,7 @@ impl HostedTransport { json!({"freshAgent":{"enabled":true}}), ) .with_sidecar_launch_context(CodexSidecarLaunchContext { + provider_exec_profile: true, config_args: sidecar_args, env: Default::default(), }), diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index 1da77fc97..961c295c7 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -2857,6 +2857,7 @@ fn build_codex_managed_launch_setup( tui_mcp_injection: tui, terminal_env, sidecar_context: freshell_codex::launch_plan::CodexSidecarLaunchContext { + provider_exec_profile: false, config_args: sidecar.args, env: sidecar_env, }, diff --git a/test/unit/claude-sidecar/exec-profile.test.ts b/test/unit/claude-sidecar/exec-profile.test.ts new file mode 100644 index 000000000..90f40f343 --- /dev/null +++ b/test/unit/claude-sidecar/exec-profile.test.ts @@ -0,0 +1,54 @@ +// @vitest-environment node +import { expect, it } from 'vitest' +import { spawn } from 'node:child_process' +import { once } from 'node:events' +import { createHash } from 'node:crypto' +import { createInterface } from 'node:readline' +import { fileURLToPath } from 'node:url' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import path from 'node:path' + +async function consume(model: string) { + const cwd = await mkdtemp(path.join(tmpdir(), 'sdk-exec-profile-')) + const child = spawn(process.execPath, [fileURLToPath(new URL('../../../crates/freshell-claude-sidecar/index.mjs', import.meta.url))], { + env: { + PATH: '/usr/local/bin:/usr/bin:/bin', HOME: cwd, + FRESHELL_HOSTED_FRESH_AGENT: 'claude', + FRESHELL_CLAUDE_SDK_QUERY_MODULE: fileURLToPath(new URL('./fixtures/exec-profile-query-module.mjs', import.meta.url)), + }, + stdio: ['pipe', 'pipe', 'pipe'], + }) + const frames: any[] = [] + let stderr = '' + child.stderr!.on('data', chunk => { stderr += chunk.toString() }) + const lines = createInterface({ input: child.stdout! }) + lines.on('line', line => frames.push(JSON.parse(line))) + child.stdin!.write(`${JSON.stringify({ type: 'create', requestId: 'exec-profile', model, cwd })}\n`) + try { + const deadline = Date.now() + 5_000 + while (!frames.some(frame => ['create.failed', 'sdk.assistant', 'sdk.error'].includes(frame.type)) && Date.now() < deadline) { + await new Promise(resolve => setTimeout(resolve, 10)) + } + return { frames, stderr } + } finally { + lines.close() + if (child.exitCode === null) { child.kill(); await once(child, 'exit') } + await rm(cwd, { recursive: true }) + } +} + +it('rejects the actual SDK grandchild final environment after SDK additions', async () => { + const { frames } = await consume('oversized') + expect(frames.find(frame => frame.type === 'create.failed')?.message).toContain('provider exec profile') + expect(frames.some(frame => frame.type === 'sdk.assistant')).toBe(false) +}) + +it('lets a valid SDK child consume its finite environment through the real spawn callback', async () => { + const { frames, stderr } = await consume('valid') + const digest = createHash('sha256').update('x'.repeat(64 * 1024)).digest('hex') + expect(frames.find(frame => frame.type === 'sdk.assistant')?.content).toEqual([{ type: 'text', text: digest }]) + expect(frames.some(frame => frame.type === 'create.failed')).toBe(false) + expect(stderr).not.toContain('synthetic-private-sdk-stderr') + expect(stderr.split('\n').filter(Boolean).map(line => JSON.parse(line))).toContainEqual({ severity: 'warn', component: 'claude-sidecar', event: 'provider.stderr', bytes: 28 }) +}) diff --git a/test/unit/claude-sidecar/fixtures/exec-profile-query-module.mjs b/test/unit/claude-sidecar/fixtures/exec-profile-query-module.mjs new file mode 100644 index 000000000..c729a05ac --- /dev/null +++ b/test/unit/claude-sidecar/fixtures/exec-profile-query-module.mjs @@ -0,0 +1,35 @@ +import { spawn } from 'node:child_process' + +// Public pinned SDK SpawnOptions contract, exercising a real private child. +// The synthetic SDK-added environment is created after the Rust sidecar spawn. +export function query({ options }) { + options.stderr('synthetic-private-sdk-stderr') + const count = options.model === 'oversized' ? 17 : 1 + const env = Object.fromEntries(Array.from({ length: count }, (_, i) => [`FRESHELL_EXEC_FIXTURE_${i}`, 'x'.repeat(64 * 1024)])) + const command = { + command: process.execPath, + args: ['-e', "process.stdout.write(require('crypto').createHash('sha256').update(process.env.FRESHELL_EXEC_FIXTURE_0).digest('hex'))"], + cwd: options.cwd, + env, + signal: new AbortController().signal, + } + const child = options.spawnClaudeCodeProcess + ? options.spawnClaudeCodeProcess(command) + : spawn(command.command, command.args, { cwd: command.cwd, env, signal: command.signal, stdio: ['pipe', 'pipe', 'pipe'] }) + const complete = new Promise((resolve, reject) => { + let digest = '' + child.stdout.on('data', chunk => { digest += chunk.toString() }) + child.on('error', reject) + child.on('close', code => code === 0 ? resolve(digest) : reject(new Error('synthetic grandchild failed'))) + }) + return { + async *[Symbol.asyncIterator]() { + const digest = await complete + yield { type: 'system', subtype: 'init', session_id: 'synthetic-exec-session', model: 'synthetic', tools: [] } + yield { type: 'assistant', message: { role: 'assistant', content: [{ type: 'text', text: digest }] } } + yield { type: 'result', subtype: 'success', session_id: 'synthetic-exec-session', result: '' } + }, + close() { if (child.exitCode === null) child.kill() }, + interrupt() {}, + } +} From 854a08479fdb754ad7a47256ff3d921ffc47e4bc Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 10:17:49 -0700 Subject: [PATCH 033/493] fix: bind initial provider auth absence to reserved launch ownership --- crates/freshell-runtime-client/src/lib.rs | 23 + crates/freshell-runtime-protocol/src/lib.rs | 37 ++ .../src/provider_inputs.rs | 10 + crates/freshell-supervisor/src/backend.rs | 39 ++ .../freshell-supervisor/src/backend/docker.rs | 7 +- .../src/backend/input_publisher.rs | 1 + .../src/backend/native_auth.rs | 409 +++++++++++++++++- .../src/native_auth_lookup_tests.rs | 85 ++++ crates/freshell-supervisor/src/registry.rs | 127 +++++- .../src/registry/input_reservation.rs | 241 +++++++++++ crates/freshell-supervisor/src/service.rs | 124 ++++++ 11 files changed, 1095 insertions(+), 8 deletions(-) create mode 100644 crates/freshell-supervisor/src/registry/input_reservation.rs diff --git a/crates/freshell-runtime-client/src/lib.rs b/crates/freshell-runtime-client/src/lib.rs index 3d80fbeb9..46120e1b3 100644 --- a/crates/freshell-runtime-client/src/lib.rs +++ b/crates/freshell-runtime-client/src/lib.rs @@ -51,6 +51,29 @@ pub struct RuntimeClient { } impl RuntimeClient { + pub async fn reserve_provider_inputs( + &self, + soul_id: SoulId, + provider: String, + creation_seed_ref: String, + ) -> Result { + let request = freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(self.current_epoch().await?), + soul_id, + provider, + creation_seed_ref, + }; + match self + .request( + RequestId::new(), + AdminCommand::ReserveProviderInputs(request), + ) + .await? + { + AdminResult::ProviderInputReservation(receipt) => Ok(receipt), + _ => Err(ClientError::UnexpectedResult), + } + } pub async fn native_auth_lookup( &self, soul_id: SoulId, diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 4f0baa16d..c8b1aba3d 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2724,6 +2724,7 @@ pub struct QualificationWriterClaimEvidence { #[serde(tag = "method", content = "params", rename_all = "snake_case")] pub enum AdminCommand { Health, + ReserveProviderInputs(ReserveProviderInputsRequest), NativeAuthLookup(NativeAuthLookupRequest), Launch(Box), Stop(StopRequest), @@ -2759,6 +2760,41 @@ pub enum AdminCommand { Recover(RecoverRequest), } +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ReserveProviderInputsRequest { + pub expected_control_epoch: Option, + pub soul_id: SoulId, + pub provider: String, + pub creation_seed_ref: String, +} + +/// Exact registry-backed creation identity. The observation is recorded only +/// after the selected daemon confirms the reserved volume is absent. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ProviderInputReservation { + pub reservation_id: String, + pub installation_id: InstallationId, + pub soul_id: SoulId, + pub provider: String, + pub creation_seed_ref: String, + pub intent_revision: u64, + pub provider_volume_name: String, + pub image_ref: String, + pub host_binary_path: String, + pub host_binary_sha256: String, + pub observation: InitialAuthVolumeObservation, +} +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)] +pub enum InitialAuthVolumeObservation { + Absent { + #[serde(rename = "daemonId")] + daemon_id: DockerDaemonId, + }, +} + /// Transient nested callback request; the authority is scoped by the supervisor /// to the recovery attempt whose lifecycle lock is already held. #[derive(Debug, Clone, Serialize, Deserialize)] @@ -2895,6 +2931,7 @@ pub struct LaunchResult { #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(tag = "kind", content = "data", rename_all = "snake_case")] pub enum AdminResult { + ProviderInputReservation(ProviderInputReservation), NativeAuthLookup(NativeAuthValues), Health { control_epoch: u64, diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 7ef98fe8d..a04208f2d 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -8,6 +8,8 @@ use sha2::{Digest, Sha256}; #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct PreparedProviderInputs { pub generation_id: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub initial_auth_reservation: Option, pub manifest: ProviderInputManifestReference, #[serde(default, skip_serializing_if = "Vec::is_empty")] pub config_binds: Vec, @@ -107,6 +109,13 @@ impl PreparedProviderInputs { }; crate::bounded_json_bytes(self, crate::MAX_CONTROL_FRAME_BYTES) .map_err(|_| invalid("control byte limit"))?; + if self + .initial_auth_reservation + .as_ref() + .is_some_and(|id| uuid::Uuid::parse_str(id).is_err()) + { + return Err(invalid("initial auth reservation")); + } if capability.map(|value| value.grant_id.as_str()) != Some(self.generation_id.as_str()) || !self.generation_id.starts_with("grant-") || !identifier(&self.generation_id) @@ -288,6 +297,7 @@ mod tests { fn prepared() -> PreparedProviderInputs { PreparedProviderInputs { generation_id: "grant-synthetic".into(), + initial_auth_reservation: None, manifest: ProviderInputManifestReference { relative_path: ".freshell/input-manifest.json".into(), sha256: "a".repeat(64), diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index bfe92221e..ce7e2237a 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -40,6 +40,7 @@ pub enum BackendError { #[derive(Debug, Clone)] pub struct CreateRuntimeSpec { + pub initial_input_reservation: Option, pub installation_id: InstallationId, pub soul_id: SoulId, pub incarnation_id: IncarnationId, @@ -120,6 +121,14 @@ pub struct BackendInspection { #[async_trait] pub trait RuntimeBackend: Send + Sync { + async fn observe_initial_auth_volume( + &self, + _reservation: &crate::registry::OwnedInputReservation, + ) -> Result { + Err(BackendError::Unavailable( + "initial auth source observation unavailable".into(), + )) + } async fn commit_prepared_inputs( &self, _handle: &OwnedRuntimeHandle, @@ -561,6 +570,12 @@ fn runtime_host_environment( #[async_trait] impl RuntimeBackend for DockerEngineBackend { + async fn observe_initial_auth_volume( + &self, + reservation: &crate::registry::OwnedInputReservation, + ) -> Result { + self.observe_initial_volume(reservation).await + } async fn commit_prepared_inputs( &self, handle: &OwnedRuntimeHandle, @@ -626,6 +641,26 @@ impl RuntimeBackend for DockerEngineBackend { )); } let binary = canonical_regular_file(&spec.host_binary_path)?; + let initial_reference = spec + .terminal + .as_ref() + .and_then(|workload| workload.provider_launch_context.as_ref()) + .or_else(|| { + spec.fresh_agent + .as_ref() + .and_then(|workload| workload.provider_launch_context.as_ref()) + }) + .and_then(|context| context.prepared_inputs.as_ref()) + .and_then(|inputs| inputs.initial_auth_reservation.as_deref()); + if initial_reference.is_some_and(|id| { + spec.initial_input_reservation + .as_ref() + .is_none_or(|bound| bound.reservation.id != id) + }) { + return Err(BackendError::InvalidConfig( + "initial provider source lost its current prepared reservation".into(), + )); + } let runtime_dir = canonical_dir(&spec.runtime_dir)?; let actor_state_dir = if spec.fresh_agent.is_some() { let runtime_root = runtime_dir.parent().ok_or_else(|| { @@ -661,6 +696,8 @@ impl RuntimeBackend for DockerEngineBackend { let expected = ExpectedConfig::new(spec, &binary, &runtime_dir); let immutable_config_digest = digest_expected(&expected)?; let (daemon_id, rootless) = self.daemon_info().await?; + self.ensure_reserved_initial_volume(spec, &daemon_id, rootless) + .await?; let memory_swap = spec .limits .memory_bytes @@ -1847,6 +1884,7 @@ mod tests { }) .collect(); let spec = CreateRuntimeSpec { + initial_input_reservation: None, installation_id: InstallationId::parse("installation-test").unwrap(), soul_id: SoulId::parse("soul-test").unwrap(), incarnation_id: IncarnationId::parse("incarnation-test").unwrap(), @@ -1991,6 +2029,7 @@ mod tests { }; let inputs = PreparedProviderInputs { generation_id: "grant-digest".into(), + initial_auth_reservation: None, manifest: ProviderInputManifestReference { relative_path: ".freshell/input-manifest.json".into(), sha256: "a".repeat(64), diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index b9e4d5909..c06f8ceaa 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -260,7 +260,7 @@ fn prepared_input_mounts( let source = canonical_stage_file(root, &reference.source_relative_path)?; verify_template_limit(&source)?; verify_workspace_destination( - &workspace, + workspace, &reference.logical_destination, &reference.effective_destination, false, @@ -289,7 +289,7 @@ fn prepared_input_mounts( return Err("prepared resource is not a physical staged unit".into()); } verify_workspace_destination( - &workspace, + workspace, &reference.logical_destination, &reference.effective_destination, directory, @@ -299,7 +299,7 @@ fn prepared_input_mounts( if let Some(files) = &inputs.indexed_files { for alias in &files.aliases { verify_workspace_destination( - &workspace, + workspace, &alias.logical_destination, &alias.effective_destination, false, @@ -423,6 +423,7 @@ mod prepared_stage_tests { std::fs::write(&target, b"{\"model\":\"original\"}").unwrap(); let inputs = PreparedProviderInputs { generation_id: "grant-prepared-stage".into(), + initial_auth_reservation: None, manifest: ProviderInputManifestReference { relative_path: ".freshell/input-manifest.json".into(), sha256: format!("{:x}", Sha256::digest(manifest)), diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index f6093dcc9..46964f760 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -700,6 +700,7 @@ mod tests { })).unwrap(); terminal.validate().unwrap(); let spec = CreateRuntimeSpec { + initial_input_reservation: None, installation_id: InstallationId::parse("installation-input").unwrap(), soul_id: soul, incarnation_id: IncarnationId::parse("incarnation-input").unwrap(), diff --git a/crates/freshell-supervisor/src/backend/native_auth.rs b/crates/freshell-supervisor/src/backend/native_auth.rs index 396c770d3..345df1807 100644 --- a/crates/freshell-supervisor/src/backend/native_auth.rs +++ b/crates/freshell-supervisor/src/backend/native_auth.rs @@ -95,6 +95,152 @@ async fn exchange( } impl DockerEngineBackend { + pub(super) async fn ensure_reserved_initial_volume( + &self, + spec: &CreateRuntimeSpec, + daemon: &DockerDaemonId, + rootless: bool, + ) -> Result<(), BackendError> { + let Some(bound) = &spec.initial_input_reservation else { + return Ok(()); + }; + let reservation = &bound.reservation; + let freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: observed, + } = &bound.observation; + if !rootless + || observed != daemon + || reservation.installation_id != spec.installation_id + || reservation.soul_id != spec.soul_id + || bound.incarnation_id != spec.incarnation_id + || reservation.image_ref != spec.image_ref + || reservation.binary != spec.host_binary_path + || reservation.provider_volume_name != spec.provider_volume_name + || spec.provider_volume_name + != crate::registry::stable_provider_volume_name( + &spec.installation_id, + &spec.soul_id, + ) + { + return Err(fail( + "initial auth reservation does not match the prepared runtime", + )); + } + bound + .verify_artifact() + .map_err(|_| fail("initial auth reserved runtime artifact changed or unreadable"))?; + let labels = json!({ + "com.freshell.installation-id":spec.installation_id.as_str(), + "com.freshell.soul-id":spec.soul_id.as_str(), + "freshell.input.reservation":reservation.id, + "freshell.input.incarnation":spec.incarnation_id.as_str(), + "freshell.input.intent":"1", + "freshell.input.daemon":daemon.as_str(), + "freshell.input.image":reservation.image_ref, + "freshell.input.binary-sha256":reservation.binary_sha256 + }); + let path = format!("{DOCKER_API}/volumes/{}", spec.provider_volume_name); + let response = self + .request_bounded("GET", &path, None, 64 * 1024) + .await + .map_err(|_| fail("reserved initial volume observation failed"))?; + if response.status == 404 { + if self + .daemon_id() + .await + .map_err(|_| fail("initial auth daemon revalidation failed"))? + != *daemon + { + return Err(fail( + "initial auth source daemon changed before volume creation", + )); + } + let created=self.request_bounded("POST",&format!("{DOCKER_API}/volumes/create"),Some(&json!({ + "Name":spec.provider_volume_name,"Driver":"local","Labels":labels + })),64*1024).await.map_err(|_|fail("reserved initial volume create acknowledgement uncertain; retry the same bound launch"))?; + if created.status != 201 { + return Err(fail( + "reserved initial volume creation failed; retry the same bound launch", + )); + } + } else if response.status != 200 { + return Err(fail( + "reserved initial volume observation did not prove absence or ownership", + )); + } + // Docker volume-create is idempotent by name; its acknowledgement alone + // cannot establish that a pre-existing object belongs to this receipt. + let readback = self + .request_bounded("GET", &path, None, 64 * 1024) + .await + .map_err(|_| fail("reserved initial volume ownership readback failed"))?; + let value: Value = serde_json::from_slice(&readback.body) + .map_err(|_| fail("reserved initial volume ownership response invalid"))?; + if readback.status != 200 + || value["Name"] != spec.provider_volume_name + || value["Driver"] != "local" + || value["Labels"] != labels + || !(value["Options"].is_null() || value["Options"] == json!({})) + { + return Err(fail( + "reserved initial volume belongs to a different preparation", + )); + } + if self + .daemon_id() + .await + .map_err(|_| fail("initial auth daemon final observation failed"))? + != *daemon + { + return Err(fail( + "initial auth source daemon changed after volume creation", + )); + } + Ok(()) + } + pub(super) async fn observe_initial_volume( + &self, + reservation: &crate::registry::OwnedInputReservation, + ) -> Result { + let (daemon_id, rootless) = self + .daemon_info() + .await + .map_err(|_| fail("initial auth daemon observation failed"))?; + if !rootless { + return Err(fail( + "initial auth source requires the qualified rootless daemon", + )); + } + if reservation.provider_volume_name + != crate::registry::stable_provider_volume_name( + &reservation.installation_id, + &reservation.soul_id, + ) + { + return Err(fail("initial auth reservation volume identity changed")); + } + let response = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", reservation.provider_volume_name), + None, + 64 * 1024, + ) + .await + .map_err(|_| fail("initial auth reserved volume observation failed"))?; + if response.status != 404 { + return Err(fail("initial auth absence requires an actual reserved-volume 404; use an owned existing source")); + } + if self + .daemon_id() + .await + .map_err(|_| fail("initial auth daemon revalidation failed"))? + != daemon_id + { + return Err(fail("initial auth source daemon changed")); + } + Ok(freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { daemon_id }) + } pub(super) async fn read_auth_helper( &self, handle: &OwnedRuntimeHandle, @@ -281,6 +427,259 @@ mod tests { use freshell_runtime_protocol::LaunchNonce; use std::sync::{Arc, Mutex}; + #[tokio::test] + async fn provider_initial_absence_requires_rootless_same_daemon_exact_volume_404() { + for mode in [ + "absent", + "existing", + "denied", + "server_error", + "nonrootless", + "daemon_changed", + ] { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, b"synthetic artifact").unwrap(); + let registry = + crate::registry::Registry::open(temp.path().join("registry"), None).unwrap(); + let reserved = registry + .reserve_initial_inputs( + freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: SoulId::new(), + provider: "opencode".into(), + creation_seed_ref: "seed".into(), + }, + format!("sha256:{}", "a".repeat(64)), + binary, + ) + .await + .unwrap(); + let daemon = DockerDaemonId::new(); + let changed = DockerDaemonId::new(); + let listener = UnixListener::bind(temp.path().join("docker.sock")).unwrap(); + let requests = Arc::new(Mutex::new(Vec::::new())); + let seen = requests.clone(); + let server = tokio::spawn(async move { + let mut observations = 0; + loop { + let (mut stream, _) = listener.accept().await.unwrap(); + let mut raw = Vec::new(); + let mut byte = [0]; + while !raw.ends_with(b"\r\n\r\n") { + stream.read_exact(&mut byte).await.unwrap(); + raw.push(byte[0]); + } + let head = String::from_utf8(raw).unwrap(); + let request = head.lines().next().unwrap().to_owned(); + seen.lock().unwrap().push(request.clone()); + let (status, body) = if request.contains("/info ") { + observations += 1; + ( + 200, + json!({"ID":if mode=="daemon_changed" && observations>1 {&changed} else {&daemon}, + "SecurityOptions":if mode=="nonrootless" {vec![]} else {vec!["name=rootless"]}}), + ) + } else if request.starts_with("GET") && request.contains("/volumes/") { + ( + match mode { + "existing" => 200, + "denied" => 403, + "server_error" => 500, + _ => 404, + }, + json!({}), + ) + } else { + (500, json!({})) + }; + let body = body.to_string(); + stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",body.len()).as_bytes()).await.unwrap(); + } + }); + let result = DockerEngineBackend::new(temp.path().join("docker.sock")) + .observe_initial_volume(&reserved) + .await; + assert_eq!(result.is_ok(), mode == "absent", "{mode}"); + let seen = requests.lock().unwrap().clone(); + assert!( + seen.iter().all(|r| r.starts_with("GET ")), + "absence observation must not create a helper, volume or workload" + ); + assert!(seen + .iter() + .filter(|r| r.contains("/volumes/")) + .all(|r| r.contains(&format!("/volumes/{} ", reserved.provider_volume_name)))); + drop(seen); + server.abort(); + let _ = server.await; + } + } + + #[tokio::test] + async fn provider_initial_volume_is_labelled_once_and_reused_only_by_bound_launch() { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, b"synthetic executable artifact").unwrap(); + let runtime = temp.path().join("runtime"); + std::fs::create_dir(&runtime).unwrap(); + let registry = crate::registry::Registry::open(temp.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let image = format!("sha256:{}", "a".repeat(64)); + let reserved = registry + .reserve_initial_inputs( + freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "opencode".into(), + creation_seed_ref: "seed".into(), + }, + image.clone(), + binary.clone(), + ) + .await + .unwrap(); + let daemon = DockerDaemonId::new(); + let binding = crate::registry::BoundInputReservation { + reservation: reserved, + incarnation_id: IncarnationId::new(), + observation: freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: daemon.clone(), + }, + }; + let volume = Arc::new(Mutex::new(None::)); + let lose_create_ack = Arc::new(std::sync::atomic::AtomicBool::new(true)); + let requests = Arc::new(Mutex::new(Vec::::new())); + let listener = UnixListener::bind(temp.path().join("docker.sock")).unwrap(); + let captured = volume.clone(); + let seen = requests.clone(); + let lose_ack = lose_create_ack.clone(); + let server = tokio::spawn(async move { + loop { + let (mut stream, _) = listener.accept().await.unwrap(); + let mut raw = Vec::new(); + let mut byte = [0]; + while !raw.ends_with(b"\r\n\r\n") { + stream.read_exact(&mut byte).await.unwrap(); + raw.push(byte[0]); + } + let head = String::from_utf8(raw).unwrap(); + let length = head + .lines() + .find_map(|line| line.strip_prefix("Content-Length: ")) + .unwrap() + .trim() + .parse::() + .unwrap(); + let mut body = vec![0; length]; + stream.read_exact(&mut body).await.unwrap(); + let request = head.lines().next().unwrap().to_string(); + seen.lock().unwrap().push(request.clone()); + let (status, reply) = if request.contains("/info ") { + ( + 200, + json!({"ID":daemon,"SecurityOptions":["name=rootless"]}), + ) + } else if request.starts_with("POST") && request.contains("/volumes/create ") { + let mut created: Value = serde_json::from_slice(&body).unwrap(); + created["Driver"] = json!("local"); + created["Options"] = json!({}); + *captured.lock().unwrap() = Some(created.clone()); + ( + if lose_ack.swap(false, std::sync::atomic::Ordering::SeqCst) { + 503 + } else { + 201 + }, + created, + ) + } else if request.starts_with("GET") && request.contains("/volumes/") { + match captured.lock().unwrap().clone() { + Some(value) => (200, value), + None => (404, json!({"message":"synthetic missing volume"})), + } + } else if request.starts_with("POST") && request.contains("/containers/create?") { + (201, json!({"Id":"b".repeat(64)})) + } else { + (500, json!({"message":"unexpected synthetic operation"})) + }; + let reply = reply.to_string(); + stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{reply}",reply.len()).as_bytes()).await.unwrap(); + } + }); + let backend = DockerEngineBackend::new(temp.path().join("docker.sock")); + let spec = CreateRuntimeSpec { + initial_input_reservation: Some(binding.clone()), + installation_id: registry.installation_id().clone(), + soul_id: soul, + incarnation_id: binding.incarnation_id.clone(), + image_ref: image, + host_binary_path: binary, + runtime_dir: runtime, + limits: RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + }, + test_run_id: "initial-volume".into(), + terminal: None, + fresh_agent: None, + provider_volume_name: binding.reservation.provider_volume_name.clone(), + }; + assert!(backend.create_stopped(&spec).await.is_err(),"uncertain volume-create acknowledgement keeps the exact reservation available for retry"); + assert!(!requests + .lock() + .unwrap() + .iter() + .any(|r| r.contains("/containers/create?"))); + backend.create_stopped(&spec).await.unwrap(); + assert_eq!( + requests + .lock() + .unwrap() + .iter() + .filter(|r| r.starts_with("POST") && r.contains("/volumes/create ")) + .count(), + 1, + "physical provider volume must be explicitly owned before stopped workload creation" + ); + backend.create_stopped(&spec).await.unwrap(); + assert_eq!(requests.lock().unwrap().iter().filter(|r|r.starts_with("POST") && r.contains("/volumes/create ")).count(),1, + "lost workload-create acknowledgement must reuse the same exact labelled reservation volume"); + volume.lock().unwrap().as_mut().unwrap()["Labels"]["freshell.input.reservation"] = + json!("foreign-reservation"); + assert!( + backend.create_stopped(&spec).await.is_err(), + "existing differently owned volume must not authorize initial absence" + ); + assert_eq!( + requests + .lock() + .unwrap() + .iter() + .filter(|r| r.contains("/containers/create?")) + .count(), + 2 + ); + std::fs::write(&spec.host_binary_path, b"changed synthetic artifact").unwrap(); + assert!( + backend.create_stopped(&spec).await.is_err(), + "changed helper artifact cannot consume the old reservation" + ); + assert_eq!( + requests + .lock() + .unwrap() + .iter() + .filter(|r| r.contains("/containers/create?")) + .count(), + 2 + ); + server.abort(); + let _ = server.await; + } + #[tokio::test] async fn native_auth_uses_private_ipc_and_cleans_only_verified_owned_helper() { for mode in [ @@ -448,7 +847,7 @@ mod tests { "/synthetic/PRIVATE_AUTH_SENTINEL" ); } - let records = seen.lock().unwrap(); + let records = seen.lock().unwrap().clone(); let create = records .iter() .find(|(request, _)| request.contains("/create?")) @@ -464,9 +863,11 @@ mod tests { .contains(&json!("INHERITED_TOKEN="))); assert!(!String::from_utf8_lossy(&create.1).contains("PRIVATE_IMAGE_SENTINEL")); assert_eq!(body["HostConfig"]["Mounts"].as_array().unwrap().len(), 3); - assert!(records.iter().all(|(request, _)| !request.contains("/logs") - && !(request.starts_with("DELETE") && request.contains(handle.container_id())) - && !request.contains("/stop"))); + assert!(records + .iter() + .all(|(request, _)| !(request.contains("/logs") + || request.contains("/stop") + || request.starts_with("DELETE") && request.contains(handle.container_id())))); let deletes: Vec<_> = records .iter() .filter(|(request, _)| request.starts_with("DELETE ")) diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index 6e4db60e5..8659884e1 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -11,8 +11,93 @@ use std::{ }; struct AuthBackend(AtomicUsize); + +#[tokio::test] +async fn provider_initial_auth_requires_creation_reservation_before_absence_receipt() { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic helper artifact").unwrap(); + let registry = Registry::open(temp.path().join("registry"), None).unwrap(); + let backend = Arc::new(AuthBackend(AtomicUsize::new(0))); + let secret = "synthetic-control-secret"; + let supervisor = Supervisor::new( + registry, + backend.clone(), + SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: binary, + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "initial-input-reservation".into(), + control_secret: secret.into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let soul = SoulId::new(); + let command:AdminCommand=serde_json::from_value(serde_json::json!({ + "method":"reserve_provider_inputs","params":{ + "expectedControlEpoch":supervisor.registry.control_epoch(),"soulId":soul, + "provider":"opencode","creationSeedRef":"synthetic-creation-seed" + } + })).expect("initial source selection needs a real reservation operation, not missing native identity as auth absence"); + let mut envelope = Envelope::new(RequestId::new(), ControlRole::Web, command); + envelope.auth = Some(secret.into()); + let reply = supervisor.dispatch(envelope).await.result.unwrap(); + let receipt = serde_json::to_value(reply).unwrap(); + assert_eq!(receipt["kind"], "provider_input_reservation"); + assert_eq!(receipt["data"]["soulId"], soul.as_str()); + assert_eq!(receipt["data"]["intentRevision"], 1); + assert_eq!(receipt["data"]["observation"]["kind"], "absent"); + assert_eq!( + backend.0.load(Ordering::SeqCst), + 1, + "daemon observation happens only after the registry reserves this creation" + ); + assert!( + supervisor.registry.inventory().await.unwrap().is_empty(), + "input reservation must not invent a workload/container or native session identity" + ); + let mut lookup = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::NativeAuthLookup(NativeAuthLookupRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul, + attempt_authority: receipt["data"]["reservationId"].as_str().unwrap().into(), + keys: vec!["PATH_INPUT".into()], + }), + ); + lookup.auth = Some(secret.into()); + match supervisor.dispatch(lookup).await.result.expect( + "recorded initial absence must be available only through its current reserved source", + ) { + AdminResult::NativeAuthLookup(values) => assert!(values.0.is_empty()), + _ => panic!("unexpected initial source reply"), + } + assert_eq!( + backend.0.load(Ordering::SeqCst), + 2, + "absence is reobserved rather than inferred from native identity" + ); +} #[async_trait] impl RuntimeBackend for AuthBackend { + async fn observe_initial_auth_volume( + &self, + reservation: &OwnedInputReservation, + ) -> Result { + assert_eq!( + reservation.provider_volume_name, + stable_provider_volume_name(&reservation.installation_id, &reservation.soul_id) + ); + self.0.fetch_add(1, Ordering::SeqCst); + static DAEMON: std::sync::OnceLock = std::sync::OnceLock::new(); + Ok(InitialAuthVolumeObservation::Absent { + daemon_id: DAEMON.get_or_init(DockerDaemonId::new).clone(), + }) + } async fn read_native_auth( &self, handle: &OwnedRuntimeHandle, diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index c1c6ad562..a40d46cd3 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -17,7 +17,9 @@ use std::{ time::{SystemTime, UNIX_EPOCH}, }; -pub const SCHEMA_VERSION: u32 = 7; +pub const SCHEMA_VERSION: u32 = 8; +mod input_reservation; +pub use input_reservation::{BoundInputReservation, OwnedInputReservation}; const ACTIVE_STATES: &str = "'prepared','created','starting','running','stopping'"; #[derive(Debug, thiserror::Error)] @@ -438,6 +440,7 @@ impl Registry { } let incarnation_id = IncarnationId::parse(incarnation_raw) .map_err(|_| RegistryError::Integrity("invalid incarnation id".into()))?; + input_reservation::check_bound_reference(&tx, &input, &incarnation_id)?; let row = load_prepared_row(&tx, &incarnation_id)?; tx.commit()?; return Ok(PreparedLaunch { existing_request: true, ..row }); @@ -520,6 +523,7 @@ impl Registry { } let incarnation_id = IncarnationId::parse(incarnation_raw) .map_err(|_| RegistryError::Integrity("invalid incarnation id".into()))?; + input_reservation::check_bound_reference(&tx, &input, &incarnation_id)?; tx.execute( "INSERT INTO commands (soul_id,request_id,payload_digest,protected_payload_ref,state,provider_ack_id,incarnation_id,event_cursor,created_at,updated_at) VALUES (?1,?2,?3,NULL,'running',NULL,?4,0,?5,?5)", params![input.soul_id.as_str(), input.request_id.as_str(), input.payload_digest, incarnation_id.as_str(), now], @@ -542,6 +546,7 @@ impl Registry { } let incarnation_id = IncarnationId::new(); + input_reservation::bind_in_transaction(&tx, &input, &incarnation_id)?; let launch_nonce = LaunchNonce::new(); let limits_json = serde_json::to_string(&input.requested_limits)?; let terminal_json = input.terminal.as_ref().map(serde_json::to_string).transpose()?; @@ -2117,6 +2122,7 @@ fn create_schema( CREATE TRIGGER view_intents_inventory_update AFTER UPDATE ON view_intents BEGIN UPDATE installation SET inventory_revision=inventory_revision+1 WHERE singleton=1; END; CREATE TRIGGER view_intents_inventory_delete AFTER DELETE ON view_intents BEGIN UPDATE installation SET inventory_revision=inventory_revision+1 WHERE singleton=1; END; "#))?; + input_reservation::create_table(&tx)?; tx.execute("INSERT INTO installation (singleton,installation_id,schema_version,control_epoch,inventory_revision,initial_scan_state) VALUES (1,?1,?2,0,0,'pending')", params![installation_id.as_str(), SCHEMA_VERSION])?; tx.execute( "INSERT INTO rollout_state (singleton,mode,updated_at) VALUES (1,'legacy',?1)", @@ -2379,6 +2385,13 @@ fn migrate_schema(conn: &mut Connection) -> Result<(), RegistryError> { tx.commit()?; schema = 7; } + if schema == 7 { + let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?; + input_reservation::create_table(&tx)?; + tx.execute_batch("UPDATE installation SET schema_version=8 WHERE singleton=1;")?; + tx.commit()?; + schema = 8; + } if schema != SCHEMA_VERSION { return Err(RegistryError::Integrity(format!( "unsupported older schema {schema}" @@ -2864,6 +2877,118 @@ mod tests { } } + #[tokio::test] + async fn provider_initial_input_reservation_binds_to_first_prepared_incarnation() { + let dir = tempfile::tempdir().unwrap(); + let binary = dir.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic artifact").unwrap(); + let registry = Registry::open(dir.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let reserved = registry + .reserve_initial_inputs( + freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "opencode".into(), + creation_seed_ref: "seed".into(), + }, + format!("sha256:{}", "a".repeat(64)), + binary, + ) + .await + .unwrap(); + let observation = freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new(), + }; + let receipt = registry + .record_initial_observation(reserved.clone(), observation.clone()) + .await + .unwrap(); + assert_eq!( + registry + .record_initial_observation(reserved.clone(), observation) + .await + .unwrap(), + receipt + ); + assert!( + registry + .record_initial_observation( + reserved, + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new() + } + ) + .await + .is_err(), + "a repeated reserve cannot overwrite the original daemon observation" + ); + let mut launch = prep(soul.clone(), RequestId::new(), "reserved-inputs"); + launch.provider = "opencode".into(); + launch.fresh_agent = Some(serde_json::from_value(serde_json::json!({ + "sessionId":"synthetic-session", "provider":"opencode", "sessionType":"freshopencode", + "runtimeVariant":"opencode", "providerStoreId":"store", "cwd":"/synthetic/workspace", + "workspacePath":"/synthetic/workspace", "runAsUid":65534, "runAsGid":0, + "providerLaunchContext": { + "preparation":{"opencode":{"project_config":[],"tui_config":null}}, + "mcpCapability":{"grantId":"grant-reservation", "endpoint":"http://host.docker.internal:3001/api/mcp", + "providerRelativePath":".config/opencode/freshell-mcp.json"}, + "preparedInputs":{"generationId":"grant-reservation", "initialAuthReservation":receipt.reservation_id, + "manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64), + "sourceCount":0,"aliasCount":0,"recipeCount":0}} + } + })).expect("the complete prepared context must carry its initial auth reservation")); + let first = registry.prepare_launch(launch.clone()).await.unwrap(); + let replay = registry.prepare_launch(launch.clone()).await.unwrap(); + assert_eq!(first.incarnation_id, replay.incarnation_id); + assert!( + registry + .bound_input_reservation(first.incarnation_id.clone()) + .await + .unwrap() + .is_some(), + "creation authority remains available after the soul row is created" + ); + assert!(registry + .bound_input_reservation(IncarnationId::new()) + .await + .unwrap() + .is_none()); + assert!( + registry + .initial_input_reservation(soul.clone(), receipt.reservation_id.clone()) + .await + .unwrap() + .is_none(), + "bound authority must no longer be reusable for a pre-soul absence lookup" + ); + launch + .fresh_agent + .as_mut() + .unwrap() + .provider_launch_context + .as_mut() + .unwrap() + .prepared_inputs + .as_mut() + .unwrap() + .initial_auth_reservation = None; + assert!(matches!(registry.prepare_launch(launch).await,Err(RegistryError::RequestConflict)), + "a pending retry cannot remove or swap creation authority even with the same semantic digest"); + let conn = open_connection(®istry.inner.db_path).unwrap(); + let consumed: i64 = conn + .query_row( + "SELECT consumed FROM provider_input_preparations WHERE soul_id=?1", + params![soul.as_str()], + |row| row.get(0), + ) + .unwrap(); + assert_eq!( + consumed, 1, + "preparing the workload must atomically bind, not leave reusable pre-soul authority" + ); + } + async fn materialize_test_runtime(registry: &Registry, soul_id: SoulId) -> PreparedLaunch { let prepared = registry .prepare_launch(prep(soul_id, RequestId::new(), "materialized")) diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs new file mode 100644 index 000000000..18f04b15c --- /dev/null +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -0,0 +1,241 @@ +//! A creation reservation precedes any initial provider-auth absence claim. +use super::*; +use freshell_runtime_protocol::{ + InitialAuthVolumeObservation, ProviderInputReservation, ReserveProviderInputsRequest, +}; +use rusqlite::Transaction; + +#[derive(Debug, Clone)] +pub struct OwnedInputReservation { + pub(crate) id: String, + pub(crate) installation_id: InstallationId, + pub(crate) soul_id: SoulId, + pub(crate) creation_seed_ref: String, + pub(crate) provider_volume_name: String, + pub(crate) image_ref: String, + pub(crate) binary: PathBuf, + pub(crate) binary_sha256: String, +} +#[derive(Debug, Clone)] +pub struct BoundInputReservation { + pub(crate) reservation: OwnedInputReservation, + pub(crate) incarnation_id: IncarnationId, + pub(crate) observation: InitialAuthVolumeObservation, +} +impl BoundInputReservation { + pub(crate) fn verify_artifact(&self) -> Result<(), RegistryError> { + if binary_digest(&self.reservation.binary)? != self.reservation.binary_sha256 { + return Err(RegistryError::InvalidState( + "initial auth reserved runtime artifact changed".into(), + )); + } + Ok(()) + } +} +pub(super) fn create_table(conn: &Connection) -> Result<(), RegistryError> { + conn.execute_batch( + "CREATE TABLE IF NOT EXISTS provider_input_preparations ( + soul_id TEXT PRIMARY KEY, reservation_id TEXT NOT NULL UNIQUE, + creation_seed_ref TEXT NOT NULL, image_ref TEXT NOT NULL, + host_binary_path TEXT NOT NULL, host_binary_sha256 TEXT NOT NULL, + intent_revision INTEGER NOT NULL CHECK(intent_revision=1), + observation TEXT, consumed INTEGER NOT NULL DEFAULT 0 CHECK(consumed IN (0,1)), + bound_incarnation_id TEXT UNIQUE + );", + )?; + Ok(()) +} +fn binary_digest(binary: &Path) -> Result { + use std::io::Read; + use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; + let metadata = std::fs::symlink_metadata(binary)?; + if !metadata.is_file() + || metadata.file_type().is_symlink() + || std::fs::canonicalize(binary)? != binary + { + return Err(RegistryError::InvalidState( + "input reservation binary must be canonical and regular".into(), + )); + } + let file = std::fs::OpenOptions::new() + .read(true) + .custom_flags(libc::O_NOFOLLOW) + .open(binary)?; + let opened = file.metadata()?; + if opened.dev() != metadata.dev() || opened.ino() != metadata.ino() { + return Err(RegistryError::InvalidState( + "input binary changed during reservation".into(), + )); + } + let mut reader = file.take( + metadata + .len() + .checked_add(1) + .ok_or_else(|| RegistryError::InvalidState("input binary size overflow".into()))?, + ); + let mut hash = Sha256::new(); + let mut buffer = [0; 64 * 1024]; + let mut observed = 0u64; + loop { + let count = reader.read(&mut buffer)?; + if count == 0 { + break; + } + observed += count as u64; + if observed > metadata.len() { + return Err(RegistryError::InvalidState( + "input binary changed during reservation".into(), + )); + } + hash.update(&buffer[..count]); + } + let after = reader.get_ref().metadata()?; + let path_after = std::fs::symlink_metadata(binary)?; + if observed != metadata.len() + || path_after.dev() != metadata.dev() + || path_after.ino() != metadata.ino() + || after.len() != metadata.len() + || after.mtime() != metadata.mtime() + || after.mtime_nsec() != metadata.mtime_nsec() + || after.ctime() != metadata.ctime() + || after.ctime_nsec() != metadata.ctime_nsec() + { + return Err(RegistryError::InvalidState( + "input binary changed during reservation".into(), + )); + } + Ok(format!("{:x}", hash.finalize())) +} +impl Registry { + pub(crate) async fn bound_input_reservation( + &self, + incarnation_id: IncarnationId, + ) -> Result, RegistryError> { + let installation = self.installation_id().clone(); + self.run_blocking(move |conn| { + let row: Option<(String,String,String,String,String,String,String)> = conn.query_row( + "SELECT p.soul_id,p.reservation_id,p.creation_seed_ref,p.image_ref,p.host_binary_path,p.host_binary_sha256,p.observation FROM provider_input_preparations p JOIN incarnations i ON i.incarnation_id=p.bound_incarnation_id JOIN souls s ON s.soul_id=p.soul_id WHERE p.bound_incarnation_id=?1 AND p.consumed=1 AND i.launch_state='prepared' AND i.soul_id=p.soul_id AND s.desired_state='running' AND s.intent_revision=p.intent_revision AND s.provider='opencode' AND s.creation_seed_ref=p.creation_seed_ref", + params![incarnation_id.as_str()], |row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?)) + ).optional()?; + let Some((soul,id,seed,image,path,sha,observation))=row else {return Ok(None)}; + let binary=PathBuf::from(path); + if binary_digest(&binary)?!=sha {return Err(RegistryError::InvalidState("initial auth source artifact changed before launch".into()));} + let soul=SoulId::parse(soul).map_err(|_|RegistryError::Integrity("invalid reserved soul".into()))?; + Ok(Some(BoundInputReservation { + reservation:OwnedInputReservation{id,provider_volume_name:stable_provider_volume_name(&installation,&soul),installation_id:installation,soul_id:soul,creation_seed_ref:seed,image_ref:image,binary,binary_sha256:sha}, + incarnation_id,observation:serde_json::from_str(&observation)? + })) + }).await + } + pub(crate) async fn initial_input_reservation( + &self, + soul: SoulId, + id: String, + ) -> Result, RegistryError> { + let installation = self.installation_id().clone(); + self.run_blocking(move|conn|{ + let row:Option<(String,String,String,String,String)>=conn.query_row("SELECT creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,observation FROM provider_input_preparations WHERE soul_id=?1 AND reservation_id=?2 AND consumed=0 AND observation IS NOT NULL AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![soul.as_str(),id],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?))).optional()?; + let Some((seed,image,path,digest,observation))=row else{return Ok(None)}; + let binary=PathBuf::from(path); + if binary_digest(&binary)?!=digest{return Err(RegistryError::InvalidState("initial auth source artifact changed; reprepare before dispatch".into()));} + let volume=stable_provider_volume_name(&installation,&soul); + Ok(Some((OwnedInputReservation{id,installation_id:installation,soul_id:soul,creation_seed_ref:seed,provider_volume_name:volume,image_ref:image,binary,binary_sha256:digest},serde_json::from_str(&observation)?))) + }).await + } + pub(crate) async fn reserve_initial_inputs( + &self, + request: ReserveProviderInputsRequest, + image_ref: String, + binary: PathBuf, + ) -> Result { + if request.provider != "opencode" + || request.creation_seed_ref.is_empty() + || request.creation_seed_ref.len() > 4096 + || request.creation_seed_ref.chars().any(char::is_control) + { + return Err(RegistryError::InvalidState( + "initial input reservation requires a bounded OpenCode creation identity".into(), + )); + } + let installation = self.installation_id().clone(); + self.run_blocking(move|mut conn|{ + let sha = binary_digest(&binary)?; + let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; + let exists:bool=tx.query_row("SELECT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![request.soul_id.as_str()],|row|row.get(0))?; + if exists {return Err(RegistryError::InvalidState("initial source requires a creation reservation; existing souls use their owned recovery source".into()));} + let stored:Option<(String,String,String,String,String,i64)>=tx.query_row("SELECT reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,consumed FROM provider_input_preparations WHERE soul_id=?1",params![request.soul_id.as_str()],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?))).optional()?; + let id=if let Some((id,seed,image,path,digest,consumed))=stored { + if seed!=request.creation_seed_ref || image!=image_ref || Path::new(&path)!=binary || digest!=sha || consumed!=0 {return Err(RegistryError::RequestConflict);} + id + } else { + let id=uuid::Uuid::new_v4().to_string(); + tx.execute("INSERT INTO provider_input_preparations (soul_id,reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,intent_revision) VALUES (?1,?2,?3,?4,?5,?6,1)",params![request.soul_id.as_str(),id,request.creation_seed_ref,image_ref,binary.to_string_lossy(),sha])?; + id + }; + let volume=stable_provider_volume_name(&installation,&request.soul_id); + tx.commit()?; + Ok(OwnedInputReservation{id,installation_id:installation,soul_id:request.soul_id,creation_seed_ref:request.creation_seed_ref,provider_volume_name:volume,image_ref,binary,binary_sha256:sha}) + }).await + } + pub(crate) async fn record_initial_observation( + &self, + reservation: OwnedInputReservation, + observation: InitialAuthVolumeObservation, + ) -> Result { + let stored = serde_json::to_string(&observation)?; + self.run_blocking(move|mut conn|{ + let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; + let updated=tx.execute("UPDATE provider_input_preparations SET observation=?1 WHERE soul_id=?2 AND reservation_id=?3 AND consumed=0 AND host_binary_sha256=?4 AND (observation IS NULL OR observation=?1) AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?2)",params![stored,reservation.soul_id.as_str(),reservation.id,reservation.binary_sha256])?; + if updated!=1 {return Err(RegistryError::RequestConflict);} + tx.commit()?; + Ok(ProviderInputReservation{reservation_id:reservation.id,installation_id:reservation.installation_id,soul_id:reservation.soul_id,provider:"opencode".into(),creation_seed_ref:reservation.creation_seed_ref,intent_revision:1,provider_volume_name:reservation.provider_volume_name,image_ref:reservation.image_ref,host_binary_path:reservation.binary.to_string_lossy().into_owned(),host_binary_sha256:reservation.binary_sha256,observation}) + }).await + } +} + +pub(super) fn reservation_reference(input: &LaunchPreparation) -> Option<&str> { + input + .terminal + .as_ref() + .and_then(|workload| workload.provider_launch_context.as_ref()) + .or_else(|| { + input + .fresh_agent + .as_ref() + .and_then(|workload| workload.provider_launch_context.as_ref()) + }) + .and_then(|context| context.prepared_inputs.as_ref()) + .and_then(|inputs| inputs.initial_auth_reservation.as_deref()) +} + +pub(super) fn bind_in_transaction( + tx: &Transaction<'_>, + input: &LaunchPreparation, + incarnation: &IncarnationId, +) -> Result<(), RegistryError> { + let Some(id) = reservation_reference(input) else { + return Ok(()); + }; + if input.provider != "opencode" || input.native_session_id.is_some() { + return Err(RegistryError::RequestConflict); + } + let changed=tx.execute( + "UPDATE provider_input_preparations SET consumed=1,bound_incarnation_id=?1 WHERE soul_id=?2 AND reservation_id=?3 AND creation_seed_ref=?4 AND observation IS NOT NULL AND consumed=0 AND bound_incarnation_id IS NULL AND EXISTS(SELECT 1 FROM souls WHERE soul_id=?2 AND desired_state='running' AND intent_revision=1 AND provider='opencode' AND creation_seed_ref=?4)", + params![incarnation.as_str(),input.soul_id.as_str(),id,input.creation_seed_ref])?; + if changed != 1 { + return Err(RegistryError::RequestConflict); + } + Ok(()) +} + +pub(super) fn check_bound_reference( + conn: &Connection, + input: &LaunchPreparation, + incarnation: &IncarnationId, +) -> Result<(), RegistryError> { + let stored:Option=conn.query_row("SELECT reservation_id FROM provider_input_preparations WHERE bound_incarnation_id=?1 AND soul_id=?2 AND consumed=1",params![incarnation.as_str(),input.soul_id.as_str()],|row|row.get(0)).optional()?; + if stored.as_deref() != reservation_reference(input) { + return Err(RegistryError::RequestConflict); + } + Ok(()) +} diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index e5d36ebd3..c023fa305 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -187,6 +187,44 @@ impl Supervisor { )); } match envelope.body { + AdminCommand::ReserveProviderInputs(request) => { + self.registry + .assert_epoch(request.expected_control_epoch) + .map_err(map_registry)?; + let lock = self.lifecycle_lock(&request.soul_id).await; + let _held = lock.lock().await; + let reservation = self + .registry + .reserve_initial_inputs( + request, + self.config.image_ref.clone(), + self.config.host_binary_path.clone(), + ) + .await + .map_err(map_registry)?; + let observation = tokio::time::timeout( + Duration::from_secs(10), + self.backend.observe_initial_auth_volume(&reservation), + ) + .await + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial auth source observation timed out", + ) + })? + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial auth source observation failed; retry preparation", + ) + })?; + self.registry + .record_initial_observation(reservation, observation) + .await + .map(AdminResult::ProviderInputReservation) + .map_err(map_registry) + } AdminCommand::NativeAuthLookup(request) => { self.registry .assert_epoch(request.expected_control_epoch) @@ -591,6 +629,51 @@ impl Supervisor { if request.attempt_authority.len() != 36 { return Err(denied()); } + if let Some((reservation, expected)) = self + .registry + .initial_input_reservation(request.soul_id.clone(), request.attempt_authority.clone()) + .await + .map_err(map_registry)? + { + if reservation.image_ref != self.config.image_ref + || reservation.binary != self.config.host_binary_path + { + return Err(denied()); + } + let observed = tokio::time::timeout( + Duration::from_secs(10), + self.backend.observe_initial_auth_volume(&reservation), + ) + .await + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial auth revalidation timed out", + ) + })? + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial auth source changed; reprepare before dispatch", + ) + })?; + if observed != expected + || self + .registry + .initial_input_reservation( + request.soul_id.clone(), + request.attempt_authority.clone(), + ) + .await + .map_err(map_registry)? + .is_none() + { + return Err(denied()); + } + return Ok(freshell_runtime_protocol::NativeAuthValues( + std::collections::BTreeMap::new(), + )); + } let authority = { let authorities = self .auth_lookup_authorities @@ -653,6 +736,8 @@ impl Supervisor { request_id: RequestId, request: freshell_runtime_protocol::LaunchRequest, ) -> Result { + let lock = self.lifecycle_lock(&request.soul_id).await; + let _guard = lock.lock().await; let limits = request.limits.validate()?; request.validate_workload()?; if !release_qualified_workload( @@ -763,6 +848,19 @@ impl Supervisor { let created = self .backend .create_stopped(&CreateRuntimeSpec { + initial_input_reservation: self + .registry + .bound_input_reservation(prepared.incarnation_id.clone()) + .await + .map_err(map_registry) + .map_err(|error| { + self.activation_failure( + &prepared, + state, + "revalidate_initial_provider_source", + error, + ) + })?, installation_id: self.registry.installation_id().clone(), soul_id: prepared.soul_id.clone(), incarnation_id: prepared.incarnation_id.clone(), @@ -2881,6 +2979,31 @@ mod release_scope_tests { RuntimeErrorCode, SoulId, }; + #[test] + fn provider_fresh_launch_digest_rejects_changed_prepared_workload() { + let mut request: freshell_runtime_protocol::LaunchRequest = serde_json::from_value( + serde_json::json!({ + "soulId": SoulId::new(), "provider":"claude", "providerStoreId":"synthetic-store", + "creationSeedRef":"synthetic-seed", "limits": {"cpuMilli":1000,"memoryBytes":1073741824,"swapBytes":0,"pidsMax":128}, + "freshAgent": { + "sessionId":"synthetic-session", "provider":"claude", "sessionType":"freshclaude", + "runtimeVariant":"claude-sdk-sidecar", "providerStoreId":"synthetic-store", + "cwd":"/synthetic/workspace", "workspacePath":"/synthetic/workspace", + "runAsUid":65534, "runAsGid":0, "model":"synthetic-model" + } + }) + ).unwrap(); + let original = super::semantic_launch_digest(&request).unwrap(); + request.fresh_agent.as_mut().unwrap().model = Some("changed-model".into()); + assert_ne!( + original, + super::semantic_launch_digest(&request).unwrap(), + "changed fresh workload must not coalesce with the previous immutable launch" + ); + request.fresh_agent.as_mut().unwrap().model = Some("synthetic-model".into()); + assert_eq!(original, super::semantic_launch_digest(&request).unwrap()); + } + #[test] fn semantic_operation_identity_classifies_record_transition_and_replay() { assert_eq!( @@ -3042,6 +3165,7 @@ fn semantic_launch_digest( "nativeSessionId": request.native_session_id, "fixture": request.fixture, "terminal": request.terminal, + "freshAgent": request.fresh_agent, }); let bytes = serde_json::to_vec(&value) .map_err(|e| RuntimeError::new(RuntimeErrorCode::InvalidRequest, e.to_string()))?; From 456942ff0fbe587c4bc3e9acf4ab44ca24655f8c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 10:27:46 -0700 Subject: [PATCH 034/493] fix: share retained source views across provider auth planning --- .../src/managed_provider_references.rs | 21 +- .../managed_provider_references/sources.rs | 274 ++++++++++++++++-- 2 files changed, 266 insertions(+), 29 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 248ac07e0..9245f0c3e 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -47,7 +47,7 @@ pub struct ResolvedFileReferencePlan { pub template: String, // Generated recipes already own the raw view. Keep one raw view for // direct references too; a rewritten template cannot revalidate selection. - unexpanded_direct_source: Option, + unexpanded_direct_source: Option>, pub selections: Vec, pub referenced_variables: Vec, pub recipes: Vec, @@ -85,6 +85,17 @@ pub(super) fn resolve_file_references_allocated( env: &dyn Fn(&str) -> Option, recipe_ordinal: usize, allocate: &mut dyn FnMut(&Path) -> Result, +) -> Result { + resolve_file_references_from_view(raw, None, origin, env, recipe_ordinal, allocate) +} + +pub(super) fn resolve_file_references_from_view( + raw: &str, + source_view: Option>, + origin: FileSourceOrigin<'_>, + env: &dyn Fn(&str) -> Option, + recipe_ordinal: usize, + allocate: &mut dyn FnMut(&Path) -> Result, ) -> Result { use freshell_platform::opencode_config::{ file_reference_path, transform_file_tokens, TokenTransformError, @@ -226,9 +237,11 @@ pub(super) fn resolve_file_references_allocated( if let Some(cause) = input_error.get() { return Err(fail(0, Vec::new(), cause)); } + let transformed_length = transformed.len(); + drop(transformed); let (template, recipes) = if generated { let generated_name = format!("FRESHELL_CONFIG_REFERENCE_{recipe_ordinal}"); - if transformed.len() > 64 * 1024 { + if transformed_length > 64 * 1024 { return Err(fail( 0, Vec::new(), @@ -264,10 +277,10 @@ pub(super) fn resolve_file_references_allocated( template.push_str(&raw[cursor..]); (template, Vec::new()) }; - drop(transformed); Ok(ResolvedFileReferencePlan { template, - unexpanded_direct_source: (!generated).then(|| raw.to_owned()), + unexpanded_direct_source: (!generated) + .then(|| source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned()))), selections, referenced_variables: variables.into_inner().into_iter().collect(), recipes, diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 8af0e4653..b3bd2eb70 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -189,13 +189,122 @@ pub struct PlannedSourceOccurrence { pub references: super::ResolvedFileReferencePlan, } +pub struct LoadedSourceOccurrence { + pub source: SourceOccurrence, + pub view: std::sync::Arc, +} + +/// Retain one bounded canonical read while keeping every native logical origin. +/// Authentication lookup can select its finite keys from these same views. +pub fn read_opencode_sources( + sources: Vec, +) -> Result, super::ReferenceError> { + use std::io::Read; + let mut loaded = Vec::new(); + let mut canonical = std::collections::BTreeMap::>::new(); + let mut retained = 0usize; + for (ordinal, source) in sources.into_iter().enumerate() { + let fail = |cause| super::ReferenceError { + operation: "read", + field: "OpenCode source", + source_ordinal: ordinal, + declaring_source: source.logical_path.clone(), + variables: vec![], + cause, + }; + if ordinal >= 8192 { + return Err(fail("source occurrence budget exceeded")); + } + let view = if let Some(view) = canonical.get(&source.canonical_source) { + view.clone() + } else { + let remaining = (64 * 1024 * 1024usize) + .checked_sub(retained) + .ok_or_else(|| fail("source-view byte count overflow"))?; + let limit = remaining.min(16 * 1024 * 1024); + let mut raw = String::new(); + std::fs::File::open(&source.canonical_source) + .map_err(|_| fail("selected source read failed"))? + .take(limit as u64 + 1) + .read_to_string(&mut raw) + .map_err(|_| fail("selected source read failed or is not UTF-8"))?; + if raw.len() > limit { + return Err(fail("selected source exceeds source-view limit")); + } + let view = std::sync::Arc::new(raw); + canonical.insert(source.canonical_source.clone(), view.clone()); + view + }; + retained = retained + .checked_add(view.len()) + .ok_or_else(|| fail("source-view byte count overflow"))?; + if retained > 64 * 1024 * 1024 { + return Err(fail("simultaneous retained source views exceed 64 MiB")); + } + loaded.push(LoadedSourceOccurrence { source, view }); + } + Ok(loaded) +} + +pub fn source_environment_names( + sources: &[LoadedSourceOccurrence], +) -> Result, super::ReferenceError> { + let mut names = std::collections::BTreeSet::new(); + let mut events = 0usize; + for (ordinal, source) in sources.iter().enumerate() { + let fail = |cause| super::ReferenceError { + operation: "select", + field: "OpenCode source", + source_ordinal: ordinal, + declaring_source: source.source.logical_path.clone(), + variables: vec![], + cause, + }; + let mut cursor = 0; + while let Some(relative) = source.view[cursor..].find("{env:") { + let start = cursor + relative + 5; + let Some(close) = source.view[start..] + .find('}') + .map(|relative| start + relative) + else { + break; + }; + cursor = close + 1; + let name = &source.view[start..close]; + if name.is_empty() { + continue; + } + if name.len() > 128 || name.chars().any(char::is_control) { + return Err(fail("referenced variable name profile exceeded")); + } + events = events + .checked_add(1) + .ok_or_else(|| fail("substitution event count overflow"))?; + if events > 8192 || names.len() >= 128 && !names.contains(name) { + return Err(fail("combined source variable or event limit exceeded")); + } + names.insert(name.to_owned()); + } + } + Ok(names.into_iter().collect()) +} + pub fn plan_opencode_sources( sources: Vec, cwd: &Path, home: &Path, lookup: &dyn Fn(SourceLane, &Path, &str) -> Option, ) -> Result, super::ReferenceError> { - use std::io::Read; + plan_loaded_opencode_sources(read_opencode_sources(sources)?, cwd, home, lookup) +} + +pub fn plan_loaded_opencode_sources( + sources: Vec, + cwd: &Path, + home: &Path, + lookup: &dyn Fn(SourceLane, &Path, &str) -> Option, +) -> Result, super::ReferenceError> { + let original_names = source_environment_names(&sources)?; let mut result = Vec::new(); let mut selected = std::collections::BTreeMap::::new(); let mut selected_bytes = 0u64; @@ -204,7 +313,8 @@ pub fn plan_opencode_sources( let mut variables = std::collections::BTreeSet::new(); let mut event_count = 0usize; let mut retained_view_bytes = 0usize; - for (ordinal, source) in sources.into_iter().enumerate() { + for (ordinal, loaded) in sources.into_iter().enumerate() { + let source = loaded.source; let fail = |cause| super::ReferenceError { operation: "read", field: "OpenCode source", @@ -213,15 +323,6 @@ pub fn plan_opencode_sources( variables: vec![], cause, }; - let mut raw = String::new(); - std::fs::File::open(&source.canonical_source) - .map_err(|_| fail("selected source read failed"))? - .take(16 * 1024 * 1024 + 1) - .read_to_string(&mut raw) - .map_err(|_| fail("selected source read failed or is not UTF-8"))?; - if raw.len() > 16 * 1024 * 1024 { - return Err(fail("selected source exceeds source-view limit")); - } let mut allocate = |canonical: &Path| { if let Some(destination) = selected.get(canonical) { return Ok(destination.clone()); @@ -229,9 +330,13 @@ pub fn plan_opencode_sources( if selected.len() >= 128 { return Err("configuration reference count exceeds 128 across sources"); } - selected_bytes += std::fs::metadata(canonical) - .map_err(|_| "selected source metadata failed")? - .len(); + selected_bytes = selected_bytes + .checked_add( + std::fs::metadata(canonical) + .map_err(|_| "selected source metadata failed")? + .len(), + ) + .ok_or("selected reference size overflow")?; if selected_bytes > 32 * 1024 * 1024 { return Err("configuration references exceed 32 MiB across sources"); } @@ -242,8 +347,9 @@ pub fn plan_opencode_sources( selected.insert(canonical.to_owned(), destination.clone()); Ok(destination) }; - let references = super::resolve_file_references_allocated( - &raw, + let references = super::resolve_file_references_from_view( + &loaded.view, + Some(loaded.view.clone()), super::FileSourceOrigin { declaring_config: &source.logical_path, cwd, @@ -253,7 +359,18 @@ pub fn plan_opencode_sources( recipe_count, &mut allocate, )?; - recipe_count += references.recipes.len(); + if references + .recipes + .iter() + .any(|recipe| original_names.contains(&recipe.generated_name)) + { + return Err(fail( + "generated configuration name collides with an original source input", + )); + } + recipe_count = recipe_count + .checked_add(references.recipes.len()) + .ok_or_else(|| fail("source recipe count overflow"))?; retained_view_bytes = retained_view_bytes .checked_add(references.template.len()) .and_then(|bytes| bytes.checked_add(references.unexpanded_source().len())) @@ -261,14 +378,21 @@ pub fn plan_opencode_sources( if retained_view_bytes > 64 * 1024 * 1024 { return Err(fail("simultaneous retained source views exceed 64 MiB")); } - aliases.extend( - references - .occurrences - .iter() - .map(|occurrence| occurrence.approved_alias.clone()), - ); - variables.extend(references.referenced_variables.iter().cloned()); - event_count += references.substitution_events; + for occurrence in &references.occurrences { + if aliases.len() >= 256 && !aliases.contains(&occurrence.approved_alias) { + return Err(fail("configuration alias count exceeds 256 across sources")); + } + aliases.insert(occurrence.approved_alias.clone()); + } + for name in &references.referenced_variables { + if variables.len() >= 128 && !variables.contains(name) { + return Err(fail("referenced variable count exceeds 128 across sources")); + } + variables.insert(name.clone()); + } + event_count = event_count + .checked_add(references.substitution_events) + .ok_or_else(|| fail("source substitution event count overflow"))?; if recipe_count > 128 || aliases.len() > 256 || variables.len() > 128 || event_count > 8192 { return Err(fail( @@ -288,6 +412,106 @@ mod tests { format!("{:x}", Sha256::digest(std::fs::read(path).unwrap())) } #[test] + fn managed_provider_references_shared_source_is_read_once_before_auth_lookup() { + let owned = tempfile::tempdir().unwrap(); + let left = owned.path().join("left"); + let right = owned.path().join("right"); + std::fs::create_dir(&left).unwrap(); + std::fs::create_dir(&right).unwrap(); + std::fs::write(left.join("rules"), b"LEFT_PRIVATE_PAYLOAD").unwrap(); + std::fs::write(right.join("rules"), b"RIGHT_PRIVATE_PAYLOAD").unwrap(); + let shared = owned.path().join("shared.json"); + let raw = r#"{"model":"{env:MODEL}","instructions":["{file:./rules}"]}"#; + std::fs::write(&shared, raw).unwrap(); + let sources = vec![ + SourceOccurrence { + logical_path: left.join("opencode.json"), + canonical_source: shared.clone(), + lane: SourceLane::Main, + }, + SourceOccurrence { + logical_path: right.join("opencode.json"), + canonical_source: shared.clone(), + lane: SourceLane::Main, + }, + ]; + let lookups = std::cell::Cell::new(0); + let plans = plan_opencode_sources(sources, owned.path(), owned.path(), &|_, _, name| { + if name == "MODEL" { + if lookups.get() == 0 { + std::fs::write(&shared, r#"{"model":"CHANGED_SOURCE"}"#).unwrap(); + } + lookups.set(lookups.get() + 1); + Some("synthetic-model".into()) + } else { + None + } + }) + .unwrap(); + assert_eq!(plans.len(), 2); + assert_eq!(plans[0].references.unexpanded_source(), raw); + assert_eq!(plans[1].references.unexpanded_source(),raw, + "auth selection and planning must share the same bounded source read for every canonical source occurrence"); + assert_eq!( + plans[0].references.selections[0].canonical_source, + left.join("rules") + ); + assert_eq!( + plans[1].references.selections[0].canonical_source, + right.join("rules"), + "one canonical read must retain both logical declaring parents" + ); + assert_eq!(lookups.get(), 2); + } + + #[test] + fn managed_provider_references_retained_views_select_keys_and_reject_cross_source_name_collision( + ) { + let owned = tempfile::tempdir().unwrap(); + std::fs::write(owned.path().join("rules"), b"PRIVATE_RULES").unwrap(); + let first = owned.path().join("opencode.json"); + let second = owned.path().join("tui.json"); + std::fs::write( + &first, + r#"{"instructions":["{env:INSERT}"],"model":"{env:MODEL}"}"#, + ) + .unwrap(); + std::fs::write(&second, r#"{"model":"{env:FRESHELL_CONFIG_REFERENCE_0}"}"#).unwrap(); + let sources = vec![ + SourceOccurrence { + logical_path: first.clone(), + canonical_source: first.clone(), + lane: SourceLane::Main, + }, + SourceOccurrence { + logical_path: second.clone(), + canonical_source: second, + lane: SourceLane::Tui, + }, + ]; + let loaded = read_opencode_sources(sources).unwrap(); + assert_eq!( + source_environment_names(&loaded).unwrap(), + vec!["FRESHELL_CONFIG_REFERENCE_0", "INSERT", "MODEL"] + ); + std::fs::remove_file(first).unwrap(); + let error = plan_loaded_opencode_sources( + loaded, + owned.path(), + owned.path(), + &|_, _, name| (name == "INSERT").then(|| "{file:rules}".into()), + ) + .err() + .expect( + "an original selected source variable cannot become an unrelated generated document", + ); + assert_eq!( + error.cause, + "generated configuration name collides with an original source input" + ); + assert!(!error.to_string().contains("PRIVATE_RULES")); + } + #[test] fn managed_provider_references_discovery_keeps_native_order_origins_and_tui_ancestors() { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); From dfa22f7dad88d5850dfd97487c5fb7184d8184ff Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 10:38:11 -0700 Subject: [PATCH 035/493] fix: retain implicitly discovered Codex role inputs --- .../src/managed_provider_references/codex.rs | 158 ++++++++++++++++++ 1 file changed, 158 insertions(+) diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 462f1a814..ea4b05cb7 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -161,6 +161,23 @@ impl Selection<'_> { } } } + if kind != LayerKind::Role { + let declared = role_paths.iter().cloned().collect(); + let remaining_events = 8192usize + .checked_sub(self.inputs.len()) + .and_then(|value| value.checked_sub(role_paths.len())) + .ok_or("Codex selected role event budget exceeded")?; + let remaining_metadata = (8 * 1024 * 1024usize) + .checked_sub(self.metadata_bytes) + .and_then(|value| value.checked_sub(role_path_bytes)) + .ok_or("Codex selected role metadata budget exceeded")?; + role_paths.extend(discover_role_files( + &parent.join("agents"), + &declared, + remaining_events, + remaining_metadata, + )?); + } // Retain only finite edge metadata across recursion. Native TOML // documents and raw source views leave scope before loading roles. drop(value); @@ -188,6 +205,70 @@ impl Selection<'_> { } } +fn discover_role_files( + root: &Path, + declared: &std::collections::BTreeSet, + max_files: usize, + max_path_bytes: usize, +) -> Result, &'static str> { + let limits = freshell_platform::provider_environment::CopyLimits::default(); + let mut directories = vec![(root.to_owned(), 0usize)]; + let mut files = Vec::new(); + let mut entries = 0usize; + let mut metadata_bytes = 0usize; + let mut selected_bytes = 0usize; + while let Some((directory, depth)) = directories.pop() { + if depth > limits.max_depth { + return Err("Codex agent discovery depth exceeded"); + } + let children = match std::fs::read_dir(&directory) { + Ok(children) => children, + Err(error) if error.kind() == std::io::ErrorKind::NotFound && depth == 0 => continue, + Err(_) => return Err("Codex selected agent directory is unreadable or changed"), + }; + for child in children { + entries = entries + .checked_add(1) + .ok_or("Codex agent discovery entry count overflow")?; + if entries > limits.max_entries { + return Err("Codex agent discovery entry limit exceeded"); + } + let child = child.map_err(|_| "Codex selected agent directory entry unreadable")?; + let path = child.path(); + metadata_bytes = metadata_bytes + .checked_add(path.as_os_str().len()) + .ok_or("Codex agent discovery metadata overflow")?; + if metadata_bytes > 64 * 1024 * 1024 { + return Err("Codex agent discovery metadata budget exceeded"); + } + let kind = child + .file_type() + .map_err(|_| "Codex selected agent entry metadata unavailable")?; + if kind.is_dir() { + directories.push((path, depth + 1)); + } else if kind.is_file() + && path + .extension() + .is_some_and(|extension| extension == "toml") + && !declared.contains(&path) + { + if files.len() >= max_files { + return Err("Codex selected role event budget exceeded"); + } + selected_bytes = selected_bytes + .checked_add(path.as_os_str().len()) + .ok_or("Codex selected role metadata overflow")?; + if selected_bytes > max_path_bytes { + return Err("Codex selected role metadata budget exceeded"); + } + files.push(path); + } + } + } + files.sort(); + Ok(files) +} + fn string_field<'a>(value: &'a toml::Value, field: &str) -> Result, &'static str> { value .get(field) @@ -243,6 +324,83 @@ mod tests { kind: LayerKind::User, } } + #[test] + fn managed_provider_codex_discovered_roles_keep_nested_instruction_origins() { + let fixture = tempfile::tempdir().unwrap(); + let root = fixture.path().join(".codex"); + let nested = root.join("agents/nested"); + std::fs::create_dir_all(&nested).unwrap(); + std::fs::write(root.join("config.toml"), "model='synthetic-model'").unwrap(); + std::fs::write(root.join("agents/z.toml"),"name='z'\ndescription='synthetic role'\ndeveloper_instructions='synthetic developer'\nmodel_instructions_file='./z.md'\n").unwrap(); + std::fs::write(nested.join("a.toml"),"name='a'\ndescription='synthetic role'\ndeveloper_instructions='synthetic developer'\nmodel_instructions_file='./a.md'\n").unwrap(); + std::fs::write(root.join("agents/z.md"), b"Z_INSTRUCTIONS").unwrap(); + std::fs::write(nested.join("a.md"), b"A_NESTED_INSTRUCTIONS").unwrap(); + std::fs::write(nested.join("not-a-role.json"), b"not a TOML role").unwrap(); + let selected = select_inputs( + &[layer(root.join("config.toml"), true)], + fixture.path(), + None, + ) + .unwrap(); + let instructions = selected + .iter() + .filter(|input| input.kind == InputKind::Instructions) + .map(|input| digest(&input.canonical_source)) + .collect::>(); + assert_eq!(instructions,vec![digest(&nested.join("a.md")),digest(&root.join("agents/z.md"))], + "native recursive role discovery must retain each role's declaring parent in sorted file order"); + assert_eq!( + select_inputs( + &[layer(root.join("config.toml"), false)], + fixture.path(), + None + ) + .unwrap() + .len(), + 1, + "disabled project configuration must not activate its discovered role inputs" + ); + } + + #[test] + fn managed_provider_codex_discovered_roles_exclude_declared_files_and_bound_metadata() { + let fixture = tempfile::tempdir().unwrap(); + let agents = fixture.path().join("agents"); + std::fs::create_dir(&agents).unwrap(); + let role = agents.join("declared.toml"); + std::fs::write(&role, "model_instructions_file='./instructions.md'").unwrap(); + std::fs::write( + agents.join("instructions.md"), + b"SYNTHETIC_ROLE_INSTRUCTIONS", + ) + .unwrap(); + let config = fixture.path().join("config.toml"); + std::fs::write( + &config, + "[agents.worker]\nconfig_file='./agents/declared.toml'\n", + ) + .unwrap(); + let selected = select_inputs(&[layer(config, true)], fixture.path(), None).unwrap(); + assert_eq!( + selected + .iter() + .filter(|input| input.kind == InputKind::Instructions) + .count(), + 1, + "explicit role must not be selected again through discovery" + ); + assert!(discover_role_files(&agents, &std::collections::BTreeSet::new(), 0, 1024).is_err()); + assert!(discover_role_files(&agents, &std::collections::BTreeSet::new(), 1, 0).is_err()); + assert!(discover_role_files( + &fixture.path().join("absent"), + &std::collections::BTreeSet::new(), + 0, + 0 + ) + .unwrap() + .is_empty()); + } + #[test] fn managed_provider_codex_typed_inputs_keep_each_logical_role_and_profile_origin() { let fixture = tempfile::tempdir().unwrap(); From c87682a45cdfdbfd5a5c2f90e1872093794b837a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 11:38:31 -0700 Subject: [PATCH 036/493] fix: transport selected local Amplifier bundle resources --- Cargo.lock | 2 + crates/freshell-server/Cargo.toml | 4 + .../src/managed_mcp_capability.rs | 286 ++++++++-- .../amplifier_resources.rs | 515 ++++++++++++++++++ scripts/sandbox-test.sh | 5 + .../managed-provider-source-sandbox.py | 94 ++++ 6 files changed, 874 insertions(+), 32 deletions(-) create mode 100644 crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs create mode 100644 scripts/testing/managed-provider-source-sandbox.py diff --git a/Cargo.lock b/Cargo.lock index a2501e9f3..dfcd8812b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1467,6 +1467,8 @@ dependencies = [ "regex", "reqwest", "rusqlite", + "saphyr", + "saphyr-parser", "serde", "serde_json", "sha1", diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index 0bc10cb7c..23b6477eb 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -23,6 +23,8 @@ managed-runtime-v1 = [ "dep:freshell-runtime-client", "dep:freshell-runtime-protocol", "dep:toml", + "dep:saphyr", + "dep:saphyr-parser", ] managed-provider-qualification = [ "managed-runtime-v1", @@ -75,6 +77,8 @@ axum = "0.8" async-trait = "0.1" serde_json = { workspace = true } toml = { version = "=0.9.12", optional = true } +saphyr = { version = "=0.0.11", default-features = false, optional = true } +saphyr-parser = { version = "=0.0.11", optional = true } # Boot-scoped identifiers: `srv-` / `boot-`. uuid = { version = "1", features = ["v4"] } # The browser-pane HTTP reverse proxy (`/api/proxy/http/{port}/*`) forwards to a diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 00a5960dd..63027c7c4 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -27,6 +27,7 @@ use tokio::io::{AsyncReadExt, AsyncWriteExt}; const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; +mod amplifier_resources; mod source_stage; #[derive(Clone, Copy, Default)] @@ -774,6 +775,50 @@ fn stage_provider_root( source: resource_root, destination_relative: ".freshell/resources".into(), }]; + let mut excluded = provider_credential_sources(&source); + for path in [ + ".claude/.credentials.json", + ".codex/auth.json", + ".local/share/opencode/auth.json", + ".config/opencode/auth.json", + ".amplifier/keys.env", + ".amplifier/.env", + // Native conversations/history/logs remain in their own stores; + // an alias cannot turn them into selected package resources. + ".claude/projects", + ".claude/history.jsonl", + ".claude/debug", + ".codex/sessions", + ".codex/archived_sessions", + ".codex/history.jsonl", + ".codex/log", + ".amplifier/projects", + ".local/share/opencode/storage", + ".local/share/opencode/opencode.db", + ".local/share/opencode/opencode.db-wal", + ".local/share/opencode/opencode.db-shm", + ] { + if let Ok(canonical) = std::fs::canonicalize(home.join(path)) { + excluded.push(canonical); + } + } + excluded.extend( + crate::managed_provider_bootstrap::named_provider_onecli_references(provider)? + .into_iter() + .map(|reference| PathBuf::from(reference.source_path)), + ); + let amplifier_resources = if provider == "amplifier" { + Some(amplifier_resources::prepare( + home, + opencode_input.cwd.unwrap_or(home), + &excluded, + )?) + } else { + None + }; + if let Some(resources) = &lifier_resources { + selections.extend(resources.selections.iter().cloned()); + } for reference in references { if reference.root != ProviderConfigRoot::UserProvider { return Err("managed provider selector is unsafe".into()); @@ -876,38 +921,6 @@ fn stage_provider_root( provider, selections = selections.len() ); - let mut excluded = provider_credential_sources(&source); - for path in [ - ".claude/.credentials.json", - ".codex/auth.json", - ".local/share/opencode/auth.json", - ".config/opencode/auth.json", - ".amplifier/keys.env", - ".amplifier/.env", - // Native conversations/history/logs remain in their own stores; - // an alias cannot turn them into selected package resources. - ".claude/projects", - ".claude/history.jsonl", - ".claude/debug", - ".codex/sessions", - ".codex/archived_sessions", - ".codex/history.jsonl", - ".codex/log", - ".amplifier/projects", - ".local/share/opencode/storage", - ".local/share/opencode/opencode.db", - ".local/share/opencode/opencode.db-wal", - ".local/share/opencode/opencode.db-shm", - ] { - if let Ok(canonical) = std::fs::canonicalize(home.join(path)) { - excluded.push(canonical); - } - } - excluded.extend( - crate::managed_provider_bootstrap::named_provider_onecli_references(provider)? - .into_iter() - .map(|reference| PathBuf::from(reference.source_path)), - ); let report=copy_environment(&selections,&temporary,&excluded,CopyLimits::default()).map_err(|error| { tracing::error!(event="provider.environment.copy.failed",stage="controller",provider,operation=error.operation,relative_path=%error.relative_path.display(),limit_kind=error.limit_kind,observed=error.observed,permitted=error.permitted,elapsed_ms=started.elapsed().as_millis() as u64,corrective_message=%error.corrective_message); error.to_string() @@ -920,6 +933,17 @@ fn stage_provider_root( bytes = report.bytes, elapsed_ms = started.elapsed().as_millis() as u64 ); + if let Some(resources) = amplifier_resources { + if let Some(settings) = resources.rewritten_settings { + replace_staged_private_bytes( + &temporary.join("settings.yaml"), + settings.as_bytes(), + )?; + } + for (relative, template) in resources.rewritten_resources { + replace_staged_private_bytes(&temporary.join(relative), template.as_bytes())?; + } + } if let Some(plan) = opencode_input.source_plan { source_stage::stage( &temporary, @@ -938,6 +962,24 @@ fn stage_provider_root( result } +fn replace_staged_private_bytes(path: &Path, bytes: &[u8]) -> Result<(), String> { + let metadata = std::fs::symlink_metadata(path) + .map_err(|_| "selected source was not copied into its owned stage")?; + if !(metadata.is_file() || metadata.file_type().is_symlink()) { + return Err("selected source stage destination is not a file".into()); + } + let parent = path + .parent() + .ok_or("selected source stage parent unavailable")?; + let temporary = parent.join(format!(".source-template-{}", uuid::Uuid::new_v4())); + write_private_bytes(&temporary, bytes)?; + if let Err(error) = std::fs::rename(&temporary, path) { + let _ = std::fs::remove_file(&temporary); + return Err(error.to_string()); + } + Ok(()) +} + fn stage_tui_source( reference: &ProviderConfigReference, workspace: &Path, @@ -1560,6 +1602,186 @@ pub(crate) async fn scoped_auth_middleware( mod tests { use super::*; + #[cfg(all(unix, feature = "managed-provider-qualification"))] + #[test] + fn managed_provider_amplifier_native_local_bundle_survives_transport() { + use freshell_platform::provider_environment::{ + copy_environment, CopyLimits, CopySelection, + }; + use sha2::{Digest, Sha256}; + for case in [ + "collection-alias", + "merge-key", + "relative-context", + "absolute-context", + "parent-context", + "unicode-source", + "scalar-alias", + "home-document", + "directory-source", + ] { + let root = executable_fixture_directory(); + let home = root.path().join("source-home"); + let bundle = if case == "home-document" { + home.clone() + } else { + root.path().join(if case == "unicode-source" { + "资源 with spaces" + } else { + "outside-bundle" + }) + }; + std::fs::create_dir_all(home.join(".amplifier")).unwrap(); + std::fs::create_dir_all(&bundle).unwrap(); + let payload = b"synthetic-amplifier-context-nonce"; + let context = if case == "parent-context" { + root.path().join("context.txt") + } else { + bundle.join("context.txt") + }; + std::fs::write(&context, payload).unwrap(); + let context_reference = if case == "absolute-context" { + context.to_string_lossy().into_owned() + } else if case == "parent-context" { + "../context.txt".into() + } else { + "./context.txt".into() + }; + std::fs::write( + bundle.join("bundle.yaml"), + format!( + "bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n probe: {}\n", + serde_json::to_string(&context_reference).unwrap() + ), + ) + .unwrap(); + let selected = if case == "directory-source" { + bundle.clone() + } else { + bundle.join("bundle.yaml") + }; + let quoted = serde_json::to_string(&format!("file://{}", selected.display())).unwrap(); + let settings = if case == "scalar-alias" { + format!("selected: &bundle_source {quoted}\nbundle:\n active: *bundle_source\n") + } else if case == "collection-alias" { + format!("defaults: &bundle_map\n active: {quoted}\nbundle: *bundle_map\nadjacent: *bundle_map\n") + } else if case == "merge-key" { + format!("defaults: &bundle_map\n active: {quoted}\nbundle:\n <<: *bundle_map\nadjacent: *bundle_map\n") + } else { + format!("# unchanged α comment\nbundle:\n active: {quoted}\n") + }; + std::fs::write(home.join(".amplifier/settings.yaml"), settings).unwrap(); + std::fs::write(bundle.join("private-unselected"), b"UNSELECTED_HOME_NONCE").unwrap(); + let references = provider_source_references("amplifier", &home).unwrap(); + let capabilities = root.path().join("capabilities"); + std::fs::create_dir(&capabilities).unwrap(); + let staged = stage_provider_root( + &capabilities, + "grant-native-amplifier", + "amplifier", + &home, + &references, + OpencodeEphemeralInput::default(), + None, + ) + .unwrap(); + let selections = references + .iter() + .map(|reference| CopySelection { + source: staged.join(&reference.relative_path), + destination_relative: reference.provider_relative_path.clone().into(), + }) + .collect::>(); + let runtime_home = Path::new("/home/freshell/provider"); + copy_environment(&selections, runtime_home, &[], CopyLimits::default()).unwrap(); + if case == "home-document" { + assert!(!runtime_home.join(".freshell/resources/amplifier-local-0/private-unselected").exists(), + "selecting a standalone document in HOME must not transport unrelated home files"); + } + std::fs::remove_dir_all(&bundle).unwrap(); + if case == "parent-context" { + std::fs::remove_file(context).unwrap(); + } + let consumer = root.path().join("native-amplifier-consumer.py"); + std::fs::write(&consumer,r#"import asyncio, hashlib, json, os, sys +from pathlib import Path +phase = 'native_settings_shape' +async def consume(): + global phase + from amplifier_app_cli.lib.settings import AppSettings, SettingsPaths + home = Path(os.environ['HOME']) + settings = AppSettings(SettingsPaths(global_settings=home/'.amplifier/settings.yaml', project_settings=home/'absent-project.yaml', local_settings=home/'absent-local.yaml', session_settings=None)) + phase = 'native_settings_load' + selected = settings.get_active_bundle() + if not isinstance(selected, str) or not selected.startswith('file://'): + raise TypeError('fixture requires one local bundle source') + from amplifier_foundation import BundleRegistry, load_bundle + phase = 'native_bundle_load' + bundle = await load_bundle(selected, registry=BundleRegistry(strict=True)) + phase = 'native_context_resolution' + bundle.resolve_pending_context() + resource = Path(bundle.context['probe']) + phase = 'native_context_read' + data = resource.read_bytes() + adjacent = settings.get_merged_settings().get('adjacent', {}).get('active') + print(json.dumps({'ok':True,'digest':hashlib.sha256(data).hexdigest(), 'adjacentDigest':hashlib.sha256(adjacent.encode()).hexdigest() if isinstance(adjacent,str) else None})) +try: + asyncio.run(consume()) +except Exception as error: + print(json.dumps({'ok':False,'phase':phase,'errorClass':type(error).__name__})) + sys.exit(73) +"#).unwrap(); + let output = std::process::Command::new("/opt/amplifier-src/.venv/bin/python") + .arg("-B") + .arg(&consumer) + .env_clear() + .env("HOME", runtime_home) + .env("AMPLIFIER_HOME", runtime_home.join(".amplifier")) + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .env("LANG", "C.UTF-8") + .current_dir(root.path()) + .output() + .expect("closed pinned native fixture must provide its image interpreter"); + let line = std::str::from_utf8(&output.stdout) + .unwrap() + .lines() + .last() + .unwrap_or(""); + let receipt: serde_json::Value = serde_json::from_str(line) + .expect("native fixture emits only bounded classified receipt"); + assert!( + output.status.success(), + "transported native consumer failed: phase={} errorClass={}", + receipt["phase"], + receipt["errorClass"] + ); + assert_eq!(receipt["digest"], format!("{:x}", Sha256::digest(payload))); + if matches!(case, "collection-alias" | "merge-key") { + assert_eq!(receipt["adjacentDigest"],format!("{:x}",Sha256::digest(format!("file://{}",selected.display()).as_bytes())), + "rewriting a recognized bundle source must retain the neighboring native alias value"); + } + println!( + "provider_source_consumer {}", + serde_json::json!({"provider":"amplifier","case":case,"digest":receipt["digest"]}) + ); + // Each case consumes one physical synthetic HOME; remove only its + // fixture projection before the next independent native consumer. + for relative in [ + ".amplifier", + ".freshell", + "absent-project.yaml", + "absent-local.yaml", + ] { + let path = runtime_home.join(relative); + if path.is_dir() { + std::fs::remove_dir_all(path).unwrap(); + } else if path.exists() { + std::fs::remove_file(path).unwrap(); + } + } + } + } + #[cfg(unix)] #[test] fn managed_codex_linked_skills_and_discovery_form_a_resumable_graph() { diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs new file mode 100644 index 000000000..d70a8f634 --- /dev/null +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -0,0 +1,515 @@ +//! Native local Amplifier resource selection. Commands and unrelated YAML +//! retain their original bytes; only selected source scalar nodes are rewritten. +use freshell_platform::provider_environment::CopySelection; +use saphyr::{LoadableYamlNode, MarkedYamlOwned}; +use std::{ + collections::BTreeMap, + io::Read, + path::{Path, PathBuf}, +}; + +pub(super) struct ResourcePreparation { + pub selections: Vec, + pub rewritten_settings: Option, + pub rewritten_resources: Vec<(PathBuf, String)>, +} + +pub(super) fn prepare( + home: &Path, + cwd: &Path, + excluded: &[PathBuf], +) -> Result { + let path = home.join(".amplifier/settings.yaml"); + let mut result = ResourcePreparation { + selections: Vec::new(), + rewritten_settings: None, + rewritten_resources: Vec::new(), + }; + let file = match std::fs::File::open(&path) { + Ok(file) => file, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(result), + Err(_) => return Err("Amplifier selected settings source unreadable".into()), + }; + let mut raw = String::new(); + file.take(16 * 1024 * 1024 + 1) + .read_to_string(&mut raw) + .map_err(|_| "Amplifier selected settings source unreadable or not UTF-8")?; + if raw.len() > 16 * 1024 * 1024 { + return Err("Amplifier selected settings source exceeds 16 MiB".into()); + } + bound_yaml_expansion(&raw)?; + let documents = MarkedYamlOwned::load_from_str(&raw) + .map_err(|_| "Amplifier selected settings YAML is invalid")?; + if documents.len() > 1 { + return Err("Amplifier selected settings must contain one YAML document".into()); + } + let Some(document) = documents.first() else { + return Ok(result); + }; + let Some(bundle) = native_mapping_value(document, "bundle", 0)? else { + return Ok(result); + }; + let Some(active) = native_mapping_value(bundle, "active", 0)? else { + return Ok(result); + }; + let Some(source) = active.data.as_str() else { + return Err("Amplifier bundle.active source has an unsupported type".into()); + }; + let Some(local) = source.strip_prefix("file://") else { + return Ok(result); + }; + if local.is_empty() || local.chars().any(char::is_control) { + return Err("Amplifier local source has an invalid path".into()); + } + let selected = PathBuf::from(local); + let selected = if selected.is_absolute() { + selected + } else { + cwd.join(selected) + }; + let canonical = std::fs::canonicalize(&selected) + .map_err(|_| "Amplifier selected local bundle is missing, inaccessible or cyclic")?; + if excluded.iter().any(|path| canonical.starts_with(path)) { + return Err("Amplifier selected bundle aliases credentials or native state; use the approved private grant source".into()); + } + let metadata = std::fs::metadata(&canonical) + .map_err(|_| "Amplifier selected local bundle metadata unavailable")?; + let (root, leaf) = if metadata.is_file() { + ( + canonical + .parent() + .ok_or("Amplifier selected local bundle parent unavailable")? + .to_owned(), + canonical + .file_name() + .ok_or("Amplifier selected local bundle filename unavailable")? + .to_owned(), + ) + } else if metadata.is_dir() { + (canonical.clone(), std::ffi::OsString::new()) + } else { + return Err("Amplifier selected local bundle has an unsupported filesystem type".into()); + }; + let relative = Path::new(".freshell/resources/amplifier-local-0"); + let runtime = Path::new("/home/freshell/provider") + .join(relative) + .join(leaf); + let runtime = runtime + .to_str() + .ok_or("Amplifier projected bundle path is not UTF-8")?; + let replacement = serde_json::to_string(&format!("file://{runtime}")) + .map_err(|_| "Amplifier projected source could not be encoded")?; + let template = rewrite_bundle_active(&raw, document, bundle, active, &replacement)?; + if metadata.is_dir() { + result.selections.push(CopySelection { + source: root, + destination_relative: relative.into(), + }); + } else { + // A standalone bundle document does not select its entire parent + // directory. Preserve only its finite resources and native package + // closure; selecting a document in HOME must not copy HOME wholesale. + if !canonical + .extension() + .is_some_and(|extension| extension == "yaml" || extension == "yml" || extension == "md") + { + return Err("Amplifier local bundle source has an unsupported native format".into()); + } + result.selections.push(CopySelection { + source: canonical.clone(), + destination_relative: relative.join( + canonical + .file_name() + .ok_or("Amplifier bundle filename unavailable")?, + ), + }); + let mut bundle_raw = String::new(); + std::fs::File::open(&canonical) + .map_err(|_| "Amplifier selected bundle unreadable")? + .take(16 * 1024 * 1024 + 1) + .read_to_string(&mut bundle_raw) + .map_err(|_| "Amplifier selected bundle is unreadable or not UTF-8")?; + if bundle_raw.len() > 16 * 1024 * 1024 { + return Err("Amplifier selected bundle source exceeds 16 MiB".into()); + } + if canonical + .extension() + .is_some_and(|extension| extension == "yaml" || extension == "yml") + { + bound_yaml_expansion(&bundle_raw)?; + let bundle_documents = MarkedYamlOwned::load_from_str(&bundle_raw) + .map_err(|_| "Amplifier selected bundle YAML is invalid")?; + if bundle_documents.len() != 1 { + return Err("Amplifier selected bundle requires one YAML document".into()); + } + let mut bundle_replacements = Vec::new(); + if let Some(context) = bundle_documents[0] + .data + .as_mapping_get("context") + .and_then(|node| node.data.as_mapping()) + { + for (name, node) in context { + if name.data.as_str() == Some("include") { + continue; + } + let Some(value) = node.data.as_str() else { + continue; + }; + let path = Path::new(value); + let selected = if path.is_absolute() { + path.to_owned() + } else { + root.join(path) + }; + let independent = path.is_absolute() + || path + .components() + .any(|component| matches!(component, std::path::Component::ParentDir)); + let destination = if independent { + PathBuf::from(format!( + ".freshell/resources/amplifier-context-{}", + result.selections.len() + )) + } else { + relative.join(path) + }; + if independent { + let runtime = Path::new("/home/freshell/provider").join(&destination); + let replacement = serde_json::to_string( + runtime + .to_str() + .ok_or("Amplifier context destination is not UTF-8")?, + ) + .map_err(|_| "Amplifier context destination could not be encoded")?; + let start = source_offset(&bundle_raw, node.span.start)?; + let end = source_offset(&bundle_raw, node.span.end)?; + bundle_replacements.push((start, end, replacement)); + } + if result.selections.len() >= 8192 { + return Err("Amplifier source selection event budget exceeded".into()); + } + let canonical_context = std::fs::canonicalize(&selected).map_err(|_| { + "Amplifier selected context is missing, inaccessible or cyclic" + })?; + if excluded + .iter() + .any(|path| canonical_context.starts_with(path)) + { + return Err("Amplifier selected context aliases credentials or native state; use the approved private grant source".into()); + } + result.selections.push(CopySelection { + source: selected, + destination_relative: destination, + }); + } + } + { + let mut template = bundle_raw; + bundle_replacements.sort_by_key(|(start, _, _)| *start); + for (start, end, replacement) in bundle_replacements.into_iter().rev() { + if start > end + || end > template.len() + || template + .len() + .checked_sub(end - start) + .and_then(|size| size.checked_add(replacement.len())) + .is_none_or(|size| size > 16 * 1024 * 1024) + { + return Err( + "Amplifier bundle source rewrite exceeds its source-view profile" + .into(), + ); + } + template.replace_range(start..end, &replacement); + } + result.rewritten_resources.push(( + relative.join( + canonical + .file_name() + .ok_or("Amplifier bundle filename unavailable")?, + ), + template, + )); + } + } + } + result.rewritten_settings = Some(template); + Ok(result) +} + +fn native_mapping_value<'a>( + node: &'a MarkedYamlOwned, + key: &str, + depth: usize, +) -> Result, String> { + if depth > 64 { + return Err("Amplifier YAML merge depth exceeded".into()); + } + if let Some(value) = node.data.as_mapping_get(key) { + return Ok(Some(value)); + } + let Some(merged) = node.data.as_mapping_get("<<") else { + return Ok(None); + }; + if merged.data.as_mapping().is_some() { + return native_mapping_value(merged, key, depth + 1); + } + if let Some(sequence) = merged.data.as_sequence() { + for map in sequence { + if map.data.as_mapping().is_none() { + return Err("Amplifier YAML merge requires mappings".into()); + } + if let Some(value) = native_mapping_value(map, key, depth + 1)? { + return Ok(Some(value)); + } + } + return Ok(None); + } + Err("Amplifier YAML merge requires a mapping or sequence".into()) +} + +fn rewrite_bundle_active( + raw: &str, + document: &MarkedYamlOwned, + bundle: &MarkedYamlOwned, + active: &MarkedYamlOwned, + replacement: &str, +) -> Result { + if document.data.as_mapping_get("bundle").is_none() { + return Err( + "Amplifier inherited bundle source requires an independent YAML occurrence projection" + .into(), + ); + } + let bundle_start = source_offset(raw, bundle.span.start)?; + let bundle_end = source_offset(raw, bundle.span.end)?; + let start = source_offset(raw, active.span.start)?; + let end = source_offset(raw, active.span.end)?; + let token = raw + .get(bundle_start..bundle_end) + .ok_or("Amplifier bundle source span is invalid")?; + let (start, end, replacement) = if token.starts_with('*') { + ( + bundle_start, + bundle_end, + format!("{{<<: {token}, active: {replacement}}}"), + ) + } else if bundle.data.as_mapping_get("active").is_some() + && start >= bundle_start + && end <= bundle_end + { + (start, end, replacement.to_owned()) + } else if token.starts_with('{') && token.ends_with('}') { + ( + bundle_end - 1, + bundle_end - 1, + format!(", active: {replacement}"), + ) + } else { + let newline = if bundle_end > 0 && raw.as_bytes()[bundle_end - 1] != b'\n' { + "\n" + } else { + "" + }; + ( + bundle_end, + bundle_end, + format!( + "{newline}{}active: {replacement}\n", + " ".repeat(bundle.span.start.col()) + ), + ) + }; + if start > end { + return Err("Amplifier selected source span is invalid".into()); + } + let size = raw + .len() + .checked_sub(end - start) + .and_then(|size| size.checked_add(replacement.len())) + .ok_or("Amplifier source template size overflow")?; + if size > 16 * 1024 * 1024 { + return Err("Amplifier source template exceeds 16 MiB".into()); + } + let mut template = String::with_capacity(size); + template.push_str(&raw[..start]); + template.push_str(&replacement); + template.push_str(&raw[end..]); + Ok(template) +} + +fn source_offset(raw: &str, marker: saphyr::Marker) -> Result { + let mut start = 0usize; + for _ in 1..marker.line() { + start = start + .checked_add( + raw[start..] + .find('\n') + .ok_or("Amplifier source span line is unavailable")? + + 1, + ) + .ok_or("Amplifier source span offset overflow")?; + } + let line = &raw[start..]; + let offset = line + .char_indices() + .nth(marker.col()) + .map(|(index, _)| index) + .or_else(|| (line.chars().count() == marker.col()).then_some(line.len())) + .ok_or("Amplifier source span column is unavailable")?; + start + .checked_add(offset) + .ok_or_else(|| "Amplifier source span offset overflow".into()) +} + +/// The loader clones anchored subtrees. Charge those clones before loading, +/// without retaining source scalar values in an auxiliary tree. +fn bound_yaml_expansion(raw: &str) -> Result<(), String> { + use saphyr_parser::{Event, Parser}; + let mut anchors = BTreeMap::::new(); + let mut stack = Vec::<(usize, usize)>::new(); + let mut document_bytes = 0usize; + let mut anchor_bytes = 0usize; + let mut documents = 0usize; + let limit = 64 * 1024 * 1024usize; + for event in Parser::new_from_iter(raw.chars()) { + let (event, _) = event.map_err(|_| "Amplifier selected settings YAML is invalid")?; + let (weight, anchor, collection) = match event { + Event::DocumentStart(_) => { + documents += 1; + if documents > 1 { + return Err("Amplifier selected settings must contain one YAML document".into()); + } + continue; + } + Event::Scalar(value, _, anchor, _) => ( + 128usize + .checked_add(value.len()) + .ok_or("Amplifier YAML node size overflow")?, + anchor, + false, + ), + Event::SequenceStart(anchor, _) | Event::MappingStart(anchor, _) => (128, anchor, true), + Event::Alias(id) => ( + *anchors + .get(&id) + .ok_or("Amplifier YAML alias is cyclic or unavailable")?, + 0, + false, + ), + Event::SequenceEnd | Event::MappingEnd => { + let (anchor, weight) = stack.pop().ok_or("Amplifier YAML structure is invalid")?; + if anchor != 0 { + anchor_bytes = anchor_bytes + .checked_add(weight) + .ok_or("Amplifier YAML anchor size overflow")?; + anchors.insert(anchor, weight); + } + if let Some((_, parent)) = stack.last_mut() { + *parent = parent + .checked_add(weight) + .ok_or("Amplifier YAML node size overflow")?; + } + if raw + .len() + .checked_add(document_bytes) + .and_then(|bytes| bytes.checked_add(anchor_bytes)) + .is_none_or(|bytes| bytes > limit) + { + return Err("Amplifier YAML source-view expansion exceeds 64 MiB".into()); + } + continue; + } + _ => continue, + }; + document_bytes = document_bytes + .checked_add(weight) + .ok_or("Amplifier YAML source-view size overflow")?; + if collection { + stack.push((anchor, weight)); + } else { + if anchor != 0 { + anchor_bytes = anchor_bytes + .checked_add(weight) + .ok_or("Amplifier YAML anchor size overflow")?; + anchors.insert(anchor, weight); + } + if let Some((_, parent)) = stack.last_mut() { + *parent = parent + .checked_add(weight) + .ok_or("Amplifier YAML node size overflow")?; + } + } + if raw + .len() + .checked_add(document_bytes) + .and_then(|bytes| bytes.checked_add(anchor_bytes)) + .is_none_or(|bytes| bytes > limit) + { + return Err("Amplifier YAML source-view expansion exceeds 64 MiB".into()); + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn managed_provider_amplifier_resource_errors_preserve_private_input() { + let fixture = tempfile::tempdir().unwrap(); + let settings = fixture.path().join(".amplifier/settings.yaml"); + std::fs::create_dir(settings.parent().unwrap()).unwrap(); + let raw = "private: PRIVATE_YAML_SENTINEL\nbundle: [invalid\n"; + std::fs::write(&settings, raw).unwrap(); + let error = prepare(fixture.path(), fixture.path(), &[]).err().unwrap(); + assert!(!error.contains("PRIVATE_YAML_SENTINEL")); + assert_eq!(std::fs::read_to_string(&settings).unwrap(), raw); + } + #[test] + fn managed_provider_amplifier_alias_expansion_is_bounded_before_loader_clone() { + let fixture = tempfile::tempdir().unwrap(); + let settings = fixture.path().join(".amplifier/settings.yaml"); + std::fs::create_dir(settings.parent().unwrap()).unwrap(); + let mut raw = String::from("v0: &a0 [synthetic, synthetic]\n"); + for depth in 1..20 { + raw.push_str(&format!( + "v{depth}: &a{depth} [*a{}, *a{}]\n", + depth - 1, + depth - 1 + )); + } + std::fs::write(&settings, raw).unwrap(); + assert_eq!( + prepare(fixture.path(), fixture.path(), &[]).err().unwrap(), + "Amplifier YAML source-view expansion exceeds 64 MiB" + ); + assert!(bound_yaml_expansion("root: &loop [*loop]\n").is_err()); + } + #[test] + fn managed_provider_amplifier_resource_selection_rejects_credential_alias_before_copy() { + let fixture = tempfile::tempdir().unwrap(); + let settings = fixture.path().join(".amplifier/settings.yaml"); + std::fs::create_dir(settings.parent().unwrap()).unwrap(); + let credential = fixture.path().join("credential.yaml"); + std::fs::write(&credential, "PRIVATE_CREDENTIAL_SENTINEL").unwrap(); + std::fs::write( + &settings, + format!("bundle:\n active: 'file://{}'\n", credential.display()), + ) + .unwrap(); + let error = prepare( + fixture.path(), + fixture.path(), + std::slice::from_ref(&credential), + ) + .err() + .unwrap(); + assert!(error.contains("aliases credentials")); + assert!(!error.contains("PRIVATE_CREDENTIAL_SENTINEL")); + assert_eq!( + std::fs::read_to_string(credential).unwrap(), + "PRIVATE_CREDENTIAL_SENTINEL" + ); + } +} diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 7516f732e..445b03275 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -37,6 +37,11 @@ if [ "${1:-}" = "--provider-inputs" ]; then exec python3 "${REPO_ROOT}/scripts/testing/probe-provider-config-readonly-home.py" --indexed-inputs "$@" fi +if [ "${1:-}" = "--provider-sources" ]; then + shift + exec python3 "${REPO_ROOT}/scripts/testing/managed-provider-source-sandbox.py" "$@" +fi + if [ "${1:-}" = "--native-tools" ]; then NATIVE_MANIFEST="${2:-}" if [ -z "${NATIVE_MANIFEST}" ]; then diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py new file mode 100644 index 000000000..3eb678509 --- /dev/null +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +"""Closed pinned-provider source consumer in an owned rootless container.""" +import argparse +import hashlib +import json +from pathlib import Path +import re +import subprocess +import uuid + +IMAGE = "sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" +TEST = "managed_mcp_capability::tests::managed_provider_amplifier_native_local_bundle_survives_transport" +LABEL = "com.freshell.provider-source-probe" + + +def docker(*args, check=True): + return subprocess.run(["docker", *args], text=True, capture_output=True, + timeout=180, check=check) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--test-binary", type=Path, required=True) + parser.add_argument("--receipt-dir", type=Path, required=True) + args = parser.parse_args() + binary = args.test_binary.resolve(strict=True) + if not binary.is_file() or not re.fullmatch(r"freshell_server-[0-9a-f]{16}", binary.name): + raise RuntimeError("source fixture requires its compiled server test artifact") + expected_directory = Path(__file__).resolve().parents[2] / "target/debug/deps" + if binary.parent != expected_directory.resolve(): + raise RuntimeError("source fixture test artifact must belong to this worktree") + listing = subprocess.run([str(binary), "--exact", TEST, "--list"], + env={"PATH": "/usr/bin:/bin"}, text=True, + capture_output=True, timeout=20, check=True) + if f"{TEST}: test" not in listing.stdout.splitlines(): + raise RuntimeError("source fixture compiled artifact contains no exact selected test") + if not args.receipt_dir.is_dir(): + raise RuntimeError("source fixture receipt directory must already exist") + if "name=rootless" not in json.loads(docker("info", "--format", "{{json .SecurityOptions}}").stdout): + raise RuntimeError("source fixture requires the qualified rootless daemon") + if docker("image", "inspect", "--format", "{{.Id}}", IMAGE).stdout.strip() != IMAGE: + raise RuntimeError("source fixture requires its exact already-present provider image") + with binary.open("rb") as stream: + digest = hashlib.file_digest(stream, "sha256").hexdigest() + name = "freshell-source-probe-" + uuid.uuid4().hex + created = False + receipt = {"image": IMAGE, "testArtifactSha256": digest, "test": TEST, + "runId": name, "rootless": True} + try: + docker("create", "--name", name, "--label", f"{LABEL}={name}", + "--network", "none", "--read-only", "--user", "65534:0", "--cap-drop", "ALL", + "--security-opt", "no-new-privileges:true", "--pids-limit", "128", "--memory", "2g", + "--mount", f"type=bind,src={binary},dst=/runtime/source-consumer-tests,readonly", + "--tmpfs", "/fixture:rw,exec,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", + "--tmpfs", "/home/freshell/provider:rw,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", + "--tmpfs", "/tmp:rw,nosuid,nodev,noexec,size=256m,uid=65534,gid=0,mode=0700", + "--env", "HOME=/home/freshell/provider", + "--env", "AMPLIFIER_HOME=/home/freshell/provider/.amplifier", + "--env", "FRESHELL_TEST_EXECUTABLE_TMP_DIR=/fixture", + "--entrypoint", "/runtime/source-consumer-tests", IMAGE, + "--exact", TEST, "--nocapture", "--test-threads", "1") + created = True + result = docker("start", "--attach", name, check=False) + receipt["exitCode"] = result.returncode + cases = [] + for line in result.stdout.splitlines(): + marker = "provider_source_consumer " + if marker in line: + value = json.loads(line.split(marker, 1)[1]) + cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) + receipt["consumedCases"] = cases + if result.returncode == 0 and (len(cases) != 9 or "test result: ok. 1 passed;" not in result.stdout): + receipt["exitCode"] = 2 + receipt["failureClass"] = "native_consumer_count_mismatch" + # Only the Rust fixture's classified assertion/digest receipt is emitted. + # The native provider's raw streams stay inside its test process. + print(result.stdout, end="", flush=True) + print(result.stderr, end="", flush=True) + finally: + observed = docker("container", "inspect", name, check=False) + if observed.returncode == 0: + value = json.loads(observed.stdout)[0] + if value["Config"].get("Labels", {}).get(LABEL) != name: + raise RuntimeError("source fixture cleanup ownership changed") + docker("rm", "--force", value["Id"]) + receipt["ownedCleanup"] = docker("container", "inspect", name, check=False).returncode != 0 + (args.receipt_dir / "task002-native-source-consumers.json").write_text(json.dumps(receipt, sort_keys=True) + "\n") + if not created or not receipt["ownedCleanup"]: + raise RuntimeError("source fixture failed its owned cleanup proof") + raise SystemExit(receipt["exitCode"]) + + +if __name__ == "__main__": + main() From 9c0b86691f539df767aeedeb305b56c893dac062 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 11:45:16 -0700 Subject: [PATCH 037/493] fix: retain Amplifier Markdown bundle context resources --- .../src/managed_mcp_capability.rs | 55 ++++++++++++++----- .../amplifier_resources.rs | 31 +++++++++-- .../managed-provider-source-sandbox.py | 2 +- 3 files changed, 66 insertions(+), 22 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 63027c7c4..cac661818 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1610,6 +1610,8 @@ mod tests { }; use sha2::{Digest, Sha256}; for case in [ + "markdown-bundle", + "markdown-absolute-context", "collection-alias", "merge-key", "relative-context", @@ -1640,25 +1642,33 @@ mod tests { bundle.join("context.txt") }; std::fs::write(&context, payload).unwrap(); - let context_reference = if case == "absolute-context" { - context.to_string_lossy().into_owned() - } else if case == "parent-context" { - "../context.txt".into() + let context_reference = + if matches!(case, "absolute-context" | "markdown-absolute-context") { + context.to_string_lossy().into_owned() + } else if case == "parent-context" { + "../context.txt".into() + } else { + "./context.txt".into() + }; + let bundle_name = if case.starts_with("markdown-") { + "bundle.md" } else { - "./context.txt".into() + "bundle.yaml" }; - std::fs::write( - bundle.join("bundle.yaml"), - format!( - "bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n probe: {}\n", - serde_json::to_string(&context_reference).unwrap() - ), - ) - .unwrap(); + let header = format!( + "bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n probe: {}\n", + serde_json::to_string(&context_reference).unwrap() + ); + let document = if case.starts_with("markdown-") { + format!("---\n{header}---\n# Native retained instructions\nplain body source\n") + } else { + header + }; + std::fs::write(bundle.join(bundle_name), document).unwrap(); let selected = if case == "directory-source" { bundle.clone() } else { - bundle.join("bundle.yaml") + bundle.join(bundle_name) }; let quoted = serde_json::to_string(&format!("file://{}", selected.display())).unwrap(); let settings = if case == "scalar-alias" { @@ -1723,8 +1733,13 @@ async def consume(): resource = Path(bundle.context['probe']) phase = 'native_context_read' data = resource.read_bytes() + body_digest = None + if selected.endswith('.md'): + from amplifier_foundation.io import parse_frontmatter + _, body = parse_frontmatter(Path(selected[len('file://'):]).read_text()) + body_digest = hashlib.sha256(body.encode()).hexdigest() adjacent = settings.get_merged_settings().get('adjacent', {}).get('active') - print(json.dumps({'ok':True,'digest':hashlib.sha256(data).hexdigest(), 'adjacentDigest':hashlib.sha256(adjacent.encode()).hexdigest() if isinstance(adjacent,str) else None})) + print(json.dumps({'ok':True,'digest':hashlib.sha256(data).hexdigest(), 'bodyDigest':body_digest, 'adjacentDigest':hashlib.sha256(adjacent.encode()).hexdigest() if isinstance(adjacent,str) else None})) try: asyncio.run(consume()) except Exception as error: @@ -1756,6 +1771,16 @@ except Exception as error: receipt["errorClass"] ); assert_eq!(receipt["digest"], format!("{:x}", Sha256::digest(payload))); + if case.starts_with("markdown-") { + assert_eq!( + receipt["bodyDigest"], + format!( + "{:x}", + Sha256::digest(b"# Native retained instructions\nplain body source\n") + ), + "native frontmatter parsing must retain the transported Markdown body" + ); + } if matches!(case, "collection-alias" | "merge-key") { assert_eq!(receipt["adjacentDigest"],format!("{:x}",Sha256::digest(format!("file://{}",selected.display()).as_bytes())), "rewriting a recognized bundle source must retain the neighboring native alias value"); diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index d70a8f634..9a6a03072 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -132,12 +132,17 @@ pub(super) fn prepare( if bundle_raw.len() > 16 * 1024 * 1024 { return Err("Amplifier selected bundle source exceeds 16 MiB".into()); } - if canonical + let yaml = if canonical .extension() - .is_some_and(|extension| extension == "yaml" || extension == "yml") + .is_some_and(|extension| extension == "md") { - bound_yaml_expansion(&bundle_raw)?; - let bundle_documents = MarkedYamlOwned::load_from_str(&bundle_raw) + markdown_frontmatter(&bundle_raw) + } else { + Some((bundle_raw.as_str(), 0)) + }; + if let Some((yaml, header_offset)) = yaml { + bound_yaml_expansion(yaml)?; + let bundle_documents = MarkedYamlOwned::load_from_str(yaml) .map_err(|_| "Amplifier selected bundle YAML is invalid")?; if bundle_documents.len() != 1 { return Err("Amplifier selected bundle requires one YAML document".into()); @@ -181,8 +186,12 @@ pub(super) fn prepare( .ok_or("Amplifier context destination is not UTF-8")?, ) .map_err(|_| "Amplifier context destination could not be encoded")?; - let start = source_offset(&bundle_raw, node.span.start)?; - let end = source_offset(&bundle_raw, node.span.end)?; + let start = header_offset + .checked_add(source_offset(yaml, node.span.start)?) + .ok_or("Amplifier Markdown source offset overflow")?; + let end = header_offset + .checked_add(source_offset(yaml, node.span.end)?) + .ok_or("Amplifier Markdown source offset overflow")?; bundle_replacements.push((start, end, replacement)); } if result.selections.len() >= 8192 { @@ -237,6 +246,16 @@ pub(super) fn prepare( Ok(result) } +fn markdown_frontmatter(raw: &str) -> Option<(&str, usize)> { + static HEADER: std::sync::OnceLock = std::sync::OnceLock::new(); + let expression = HEADER.get_or_init(|| { + regex::Regex::new(r"(?s)\A---\s*\n(.*?)\n---\s*\n?") + .expect("fixed native Markdown frontmatter expression") + }); + let header = expression.captures(raw)?.get(1)?; + Some((header.as_str(), header.start())) +} + fn native_mapping_value<'a>( node: &'a MarkedYamlOwned, key: &str, diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 3eb678509..ba39370d3 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -69,7 +69,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != 9 or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != 11 or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. From 7516515131485b7969b4466f84b804f364734b07 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 11:51:22 -0700 Subject: [PATCH 038/493] fix: select Amplifier context include resources --- .../src/managed_mcp_capability.rs | 16 ++- .../amplifier_resources.rs | 125 ++++++++++-------- .../managed-provider-source-sandbox.py | 2 +- 3 files changed, 84 insertions(+), 59 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index cac661818..16093743e 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1610,6 +1610,8 @@ mod tests { }; use sha2::{Digest, Sha256}; for case in [ + "context-include", + "context-include-leading-slash", "markdown-bundle", "markdown-absolute-context", "collection-alias", @@ -1655,10 +1657,14 @@ mod tests { } else { "bundle.yaml" }; - let header = format!( - "bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n probe: {}\n", - serde_json::to_string(&context_reference).unwrap() - ); + let header = if case.starts_with("context-include") { + format!("bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n include: [{}context.txt]\n",if case=="context-include-leading-slash" {"/"}else{""}) + } else { + format!( + "bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n probe: {}\n", + serde_json::to_string(&context_reference).unwrap() + ) + }; let document = if case.starts_with("markdown-") { format!("---\n{header}---\n# Native retained instructions\nplain body source\n") } else { @@ -1730,7 +1736,7 @@ async def consume(): bundle = await load_bundle(selected, registry=BundleRegistry(strict=True)) phase = 'native_context_resolution' bundle.resolve_pending_context() - resource = Path(bundle.context['probe']) + resource = Path(bundle.context.get('probe') or next(iter(bundle.context.values()))) phase = 'native_context_read' data = resource.read_bytes() body_digest = None diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 9a6a03072..5444a4c78 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -154,62 +154,81 @@ pub(super) fn prepare( .and_then(|node| node.data.as_mapping()) { for (name, node) in context { - if name.data.as_str() == Some("include") { - continue; - } - let Some(value) = node.data.as_str() else { - continue; - }; - let path = Path::new(value); - let selected = if path.is_absolute() { - path.to_owned() - } else { - root.join(path) - }; - let independent = path.is_absolute() - || path - .components() - .any(|component| matches!(component, std::path::Component::ParentDir)); - let destination = if independent { - PathBuf::from(format!( - ".freshell/resources/amplifier-context-{}", - result.selections.len() - )) + let include = name.data.as_str() == Some("include"); + let entries = if include { + node.data + .as_sequence() + .ok_or("Amplifier context.include must be a sequence")? + .as_slice() } else { - relative.join(path) + std::slice::from_ref(node) }; - if independent { - let runtime = Path::new("/home/freshell/provider").join(&destination); - let replacement = serde_json::to_string( - runtime - .to_str() - .ok_or("Amplifier context destination is not UTF-8")?, - ) - .map_err(|_| "Amplifier context destination could not be encoded")?; - let start = header_offset - .checked_add(source_offset(yaml, node.span.start)?) - .ok_or("Amplifier Markdown source offset overflow")?; - let end = header_offset - .checked_add(source_offset(yaml, node.span.end)?) - .ok_or("Amplifier Markdown source offset overflow")?; - bundle_replacements.push((start, end, replacement)); - } - if result.selections.len() >= 8192 { - return Err("Amplifier source selection event budget exceeded".into()); - } - let canonical_context = std::fs::canonicalize(&selected).map_err(|_| { - "Amplifier selected context is missing, inaccessible or cyclic" - })?; - if excluded - .iter() - .any(|path| canonical_context.starts_with(path)) - { - return Err("Amplifier selected context aliases credentials or native state; use the approved private grant source".into()); + for node in entries { + let Some(value) = node.data.as_str() else { + continue; + }; + let value = if include { + value.trim_start_matches('/') + } else { + value + }; + if include && value.contains(':') { + return Err("Amplifier namespace context requires its retained source-base projection".into()); + } + let path = Path::new(value); + let selected = if path.is_absolute() { + path.to_owned() + } else { + root.join(path) + }; + let independent = path.is_absolute() + || path.components().any(|component| { + matches!(component, std::path::Component::ParentDir) + }); + if include && independent { + return Err("Amplifier context.include outside its source base requires an independent namespace projection".into()); + } + let destination = if independent { + PathBuf::from(format!( + ".freshell/resources/amplifier-context-{}", + result.selections.len() + )) + } else { + relative.join(path) + }; + if independent { + let runtime = Path::new("/home/freshell/provider").join(&destination); + let replacement = serde_json::to_string( + runtime + .to_str() + .ok_or("Amplifier context destination is not UTF-8")?, + ) + .map_err(|_| "Amplifier context destination could not be encoded")?; + let start = header_offset + .checked_add(source_offset(yaml, node.span.start)?) + .ok_or("Amplifier Markdown source offset overflow")?; + let end = header_offset + .checked_add(source_offset(yaml, node.span.end)?) + .ok_or("Amplifier Markdown source offset overflow")?; + bundle_replacements.push((start, end, replacement)); + } + if result.selections.len() >= 8192 { + return Err("Amplifier source selection event budget exceeded".into()); + } + let canonical_context = std::fs::canonicalize(&selected).map_err(|_| { + "Amplifier selected context is missing, inaccessible or cyclic" + })?; + if excluded + .iter() + .any(|path| canonical_context.starts_with(path)) + { + return Err("Amplifier selected context aliases credentials or native state; use the approved private grant source".into()); + } + result.selections.push(CopySelection { + source: selected, + destination_relative: destination, + }); } - result.selections.push(CopySelection { - source: selected, - destination_relative: destination, - }); } } { diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index ba39370d3..27490e21e 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -69,7 +69,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != 11 or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != 13 or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. From 0078652dac46e547cd082d8bf9a1337e17c898d2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 12:05:33 -0700 Subject: [PATCH 039/493] fix: preserve local Amplifier bundle include origins --- .../src/managed_mcp_capability.rs | 38 +++- .../amplifier_resources.rs | 167 ++++++++++++++++-- crates/freshell-server/src/managed_runtime.rs | 6 +- .../managed-provider-source-sandbox.py | 2 +- 4 files changed, 197 insertions(+), 16 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 16093743e..e74ef36d9 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1610,6 +1610,9 @@ mod tests { }; use sha2::{Digest, Sha256}; for case in [ + "bundle-includes", + "bundle-object-includes", + "bundle-relative-includes", "context-include", "context-include-leading-slash", "markdown-bundle", @@ -1657,7 +1660,29 @@ mod tests { } else { "bundle.yaml" }; - let header = if case.starts_with("context-include") { + let included = root.path().join("included-bundle"); + let header = if matches!( + case, + "bundle-includes" | "bundle-object-includes" | "bundle-relative-includes" + ) { + std::fs::create_dir(&included).unwrap(); + std::fs::write(included.join("context.txt"), payload).unwrap(); + std::fs::write(included.join("bundle.yaml"),"bundle:\n name: included-probe\n version: 1.0.0\ncontext:\n probe: ./context.txt\n").unwrap(); + let source = if case == "bundle-relative-includes" { + "./included-bundle/bundle.yaml".to_owned() + } else { + format!("file://{}", included.join("bundle.yaml").display()) + }; + format!( + "bundle:\n name: source-probe\n version: 1.0.0\nincludes:\n - {}{}\n", + if case == "bundle-object-includes" { + "bundle: " + } else { + "" + }, + serde_json::to_string(&source).unwrap() + ) + } else if case.starts_with("context-include") { format!("bundle:\n name: source-probe\n version: 1.0.0\ncontext:\n include: [{}context.txt]\n",if case=="context-include-leading-slash" {"/"}else{""}) } else { format!( @@ -1697,7 +1722,10 @@ mod tests { "amplifier", &home, &references, - OpencodeEphemeralInput::default(), + OpencodeEphemeralInput { + cwd: Some(root.path()), + ..Default::default() + }, None, ) .unwrap(); @@ -1715,6 +1743,12 @@ mod tests { "selecting a standalone document in HOME must not transport unrelated home files"); } std::fs::remove_dir_all(&bundle).unwrap(); + if matches!( + case, + "bundle-includes" | "bundle-object-includes" | "bundle-relative-includes" + ) { + std::fs::remove_dir_all(&included).unwrap(); + } if case == "parent-context" { std::fs::remove_file(context).unwrap(); } diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 5444a4c78..3c71b2177 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -55,8 +55,51 @@ pub(super) fn prepare( let Some(source) = active.data.as_str() else { return Err("Amplifier bundle.active source has an unsupported type".into()); }; - let Some(local) = source.strip_prefix("file://") else { + if !source.starts_with("file://") { return Ok(result); + } + let mut state = BundleSelection { + view_bytes: raw.len(), + ..Default::default() + }; + if let Some(runtime) = select_local_bundle(source, cwd, excluded, &mut result, &mut state)? { + let replacement = serde_json::to_string(&runtime) + .map_err(|_| "Amplifier projected source could not be encoded")?; + result.rewritten_settings = Some(rewrite_bundle_active( + &raw, + document, + bundle, + active, + &replacement, + )?); + } + Ok(result) +} + +#[derive(Default)] +struct BundleSelection { + next_id: usize, + active: std::collections::BTreeSet, + view_bytes: usize, + metadata_bytes: usize, + completed: BTreeMap, +} +fn select_local_bundle( + source: &str, + cwd: &Path, + excluded: &[PathBuf], + result: &mut ResourcePreparation, + state: &mut BundleSelection, +) -> Result, String> { + let local = if let Some(local) = source.strip_prefix("file://") { + local + } else if source.starts_with("./") + || source.starts_with("../") + || Path::new(source).is_absolute() + { + source + } else { + return Ok(None); }; if local.is_empty() || local.chars().any(char::is_control) { return Err("Amplifier local source has an invalid path".into()); @@ -72,6 +115,9 @@ pub(super) fn prepare( if excluded.iter().any(|path| canonical.starts_with(path)) { return Err("Amplifier selected bundle aliases credentials or native state; use the approved private grant source".into()); } + if let Some(runtime) = state.completed.get(&canonical) { + return Ok(Some(runtime.clone())); + } let metadata = std::fs::metadata(&canonical) .map_err(|_| "Amplifier selected local bundle metadata unavailable")?; let (root, leaf) = if metadata.is_file() { @@ -90,20 +136,34 @@ pub(super) fn prepare( } else { return Err("Amplifier selected local bundle has an unsupported filesystem type".into()); }; - let relative = Path::new(".freshell/resources/amplifier-local-0"); + if state.active.contains(&canonical) { + return Err("Amplifier local include cycle detected".into()); + } + if state.active.len() >= 64 || state.next_id >= 8192 { + return Err("Amplifier local include depth or event budget exceeded".into()); + } + state.metadata_bytes = state + .metadata_bytes + .checked_add(canonical.as_os_str().len()) + .ok_or("Amplifier selected source metadata overflow")?; + if state.metadata_bytes > 8 * 1024 * 1024 { + return Err("Amplifier selected source metadata budget exceeded".into()); + } + state.active.insert(canonical.clone()); + let id = state.next_id; + state.next_id += 1; + let relative = PathBuf::from(format!(".freshell/resources/amplifier-local-{id}")); let runtime = Path::new("/home/freshell/provider") - .join(relative) + .join(&relative) .join(leaf); let runtime = runtime .to_str() .ok_or("Amplifier projected bundle path is not UTF-8")?; - let replacement = serde_json::to_string(&format!("file://{runtime}")) - .map_err(|_| "Amplifier projected source could not be encoded")?; - let template = rewrite_bundle_active(&raw, document, bundle, active, &replacement)?; + let runtime_uri = format!("file://{runtime}"); if metadata.is_dir() { result.selections.push(CopySelection { source: root, - destination_relative: relative.into(), + destination_relative: relative.clone(), }); } else { // A standalone bundle document does not select its entire parent @@ -124,14 +184,22 @@ pub(super) fn prepare( ), }); let mut bundle_raw = String::new(); + let remaining = (64 * 1024 * 1024usize) + .checked_sub(state.view_bytes) + .ok_or("Amplifier retained source view size overflow")?; + let limit = remaining.min(16 * 1024 * 1024); std::fs::File::open(&canonical) .map_err(|_| "Amplifier selected bundle unreadable")? - .take(16 * 1024 * 1024 + 1) + .take(limit as u64 + 1) .read_to_string(&mut bundle_raw) .map_err(|_| "Amplifier selected bundle is unreadable or not UTF-8")?; - if bundle_raw.len() > 16 * 1024 * 1024 { - return Err("Amplifier selected bundle source exceeds 16 MiB".into()); + if bundle_raw.len() > limit { + return Err("Amplifier selected bundle exceeds the shared source-view profile".into()); } + state.view_bytes = state + .view_bytes + .checked_add(bundle_raw.len()) + .ok_or("Amplifier retained source view size overflow")?; let yaml = if canonical .extension() .is_some_and(|extension| extension == "md") @@ -148,6 +216,38 @@ pub(super) fn prepare( return Err("Amplifier selected bundle requires one YAML document".into()); } let mut bundle_replacements = Vec::new(); + if let Some(includes) = bundle_documents[0].data.as_mapping_get("includes") { + for item in includes + .data + .as_sequence() + .ok_or("Amplifier includes must be a sequence")? + { + let node = if item.data.as_str().is_some() { + item + } else { + item.data + .as_mapping_get("bundle") + .ok_or("Amplifier include requires its bundle source")? + }; + let source = node + .data + .as_str() + .ok_or("Amplifier include bundle source must be a string")?; + if let Some(projected) = + select_local_bundle(source, cwd, excluded, result, state)? + { + let replacement = serde_json::to_string(&projected) + .map_err(|_| "Amplifier include source could not be encoded")?; + let start = header_offset + .checked_add(source_offset(yaml, node.span.start)?) + .ok_or("Amplifier include source offset overflow")?; + let end = header_offset + .checked_add(source_offset(yaml, node.span.end)?) + .ok_or("Amplifier include source offset overflow")?; + bundle_replacements.push((start, end, replacement)); + } + } + } if let Some(context) = bundle_documents[0] .data .as_mapping_get("context") @@ -261,8 +361,9 @@ pub(super) fn prepare( } } } - result.rewritten_settings = Some(template); - Ok(result) + state.active.remove(&canonical); + state.completed.insert(canonical, runtime_uri.clone()); + Ok(Some(runtime_uri)) } fn markdown_frontmatter(raw: &str) -> Option<(&str, usize)> { @@ -493,6 +594,48 @@ fn bound_yaml_expansion(raw: &str) -> Result<(), String> { #[cfg(test)] mod tests { use super::*; + + #[test] + fn managed_provider_amplifier_local_graph_rejects_cycles_and_reuses_completed_diamonds() { + let fixture = tempfile::tempdir().unwrap(); + let settings = fixture.path().join(".amplifier/settings.yaml"); + std::fs::create_dir(settings.parent().unwrap()).unwrap(); + let a = fixture.path().join("a.yaml"); + let b = fixture.path().join("b.yaml"); + let c = fixture.path().join("c.yaml"); + let d = fixture.path().join("d.yaml"); + std::fs::write( + &settings, + format!("bundle:\n active: 'file://{}'\n", a.display()), + ) + .unwrap(); + std::fs::write(&a, format!("includes: ['file://{}']\n", b.display())).unwrap(); + std::fs::write(&b, format!("includes: ['file://{}']\n", a.display())).unwrap(); + assert_eq!( + prepare(fixture.path(), fixture.path(), &[]).err().unwrap(), + "Amplifier local include cycle detected" + ); + std::fs::write( + &a, + format!( + "includes: ['file://{}','file://{}']\n", + b.display(), + c.display() + ), + ) + .unwrap(); + std::fs::write(&b, format!("includes: ['file://{}']\n", d.display())).unwrap(); + std::fs::write(&c, format!("includes: ['file://{}']\n", d.display())).unwrap(); + std::fs::write(&d, "bundle: {name: shared}\n").unwrap(); + let prepared = prepare(fixture.path(), fixture.path(), &[]).unwrap(); + assert_eq!(prepared.selections.iter().filter(|selection|selection.source==d).count(),1, + "a completed local include shared by two branches retains one approved input generation"); + std::fs::remove_file(d).unwrap(); + assert!( + prepare(fixture.path(), fixture.path(), &[]).is_err(), + "missing declared include must not be silently skipped" + ); + } #[test] fn managed_provider_amplifier_resource_errors_preserve_private_input() { let fixture = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 8f7a02f45..280adc20e 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -209,7 +209,11 @@ impl ManagedTerminalController for ServerManagedRuntimeController { source_plan: None, } } else { - Default::default() + crate::managed_mcp_capability::OpencodeEphemeralInput { + cwd: Some(&cwd), + workspace: Some(&workspace), + ..Default::default() + } }; let mcp_capability = matches!( request.mode.as_str(), diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 27490e21e..405248cf7 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -69,7 +69,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != 13 or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != 16 or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. From 637137f96d41a52cc1ad7f47b7caf30d597a351b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 12:11:11 -0700 Subject: [PATCH 040/493] fix: follow includes from native Amplifier directory entries --- .../src/managed_mcp_capability.rs | 46 +++++++++---- .../amplifier_resources.rs | 66 +++++++++++++------ .../managed-provider-source-sandbox.py | 2 +- 3 files changed, 79 insertions(+), 35 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e74ef36d9..bbcb592e0 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1610,6 +1610,8 @@ mod tests { }; use sha2::{Digest, Sha256}; for case in [ + "directory-markdown-precedence", + "directory-bundle-includes", "bundle-includes", "bundle-object-includes", "bundle-relative-includes", @@ -1655,15 +1657,19 @@ mod tests { } else { "./context.txt".into() }; - let bundle_name = if case.starts_with("markdown-") { - "bundle.md" - } else { - "bundle.yaml" - }; + let bundle_name = + if case.starts_with("markdown-") || case == "directory-markdown-precedence" { + "bundle.md" + } else { + "bundle.yaml" + }; let included = root.path().join("included-bundle"); let header = if matches!( case, - "bundle-includes" | "bundle-object-includes" | "bundle-relative-includes" + "directory-bundle-includes" + | "bundle-includes" + | "bundle-object-includes" + | "bundle-relative-includes" ) { std::fs::create_dir(&included).unwrap(); std::fs::write(included.join("context.txt"), payload).unwrap(); @@ -1690,13 +1696,24 @@ mod tests { serde_json::to_string(&context_reference).unwrap() ) }; - let document = if case.starts_with("markdown-") { - format!("---\n{header}---\n# Native retained instructions\nplain body source\n") - } else { - header - }; + let document = + if case.starts_with("markdown-") || case == "directory-markdown-precedence" { + format!("---\n{header}---\n# Native retained instructions\nplain body source\n") + } else { + header + }; std::fs::write(bundle.join(bundle_name), document).unwrap(); - let selected = if case == "directory-source" { + if case == "directory-markdown-precedence" { + std::fs::write( + bundle.join("bundle.yaml"), + "bundle: {name: wrong-native-entry}\ncontext: {probe: ./must-not-select}\n", + ) + .unwrap(); + } + let selected = if matches!( + case, + "directory-source" | "directory-bundle-includes" | "directory-markdown-precedence" + ) { bundle.clone() } else { bundle.join(bundle_name) @@ -1745,7 +1762,10 @@ mod tests { std::fs::remove_dir_all(&bundle).unwrap(); if matches!( case, - "bundle-includes" | "bundle-object-includes" | "bundle-relative-includes" + "directory-bundle-includes" + | "bundle-includes" + | "bundle-object-includes" + | "bundle-relative-includes" ) { std::fs::remove_dir_all(&included).unwrap(); } diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 3c71b2177..55f0393bd 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -160,35 +160,64 @@ fn select_local_bundle( .to_str() .ok_or("Amplifier projected bundle path is not UTF-8")?; let runtime_uri = format!("file://{runtime}"); - if metadata.is_dir() { + let document_path = if metadata.is_dir() { result.selections.push(CopySelection { - source: root, + source: root.clone(), destination_relative: relative.clone(), }); + let mut entry = None; + for name in ["bundle.md", "bundle.yaml"] { + let candidate = root.join(name); + match std::fs::symlink_metadata(&candidate) { + Ok(_) => { + entry = Some(candidate); + break; + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => { + return Err("Amplifier directory bundle entry metadata unavailable".into()) + } + } + } + entry.ok_or("Amplifier selected directory has no native bundle entry")? } else { + canonical.clone() + }; + let document = std::fs::canonicalize(&document_path) + .map_err(|_| "Amplifier directory bundle entry unavailable, dangling or cyclic")?; + if excluded.iter().any(|path| document.starts_with(path)) { + return Err("Amplifier selected bundle entry aliases credentials or native state; use the approved private grant source".into()); + } + let document_relative = relative.join( + document_path + .file_name() + .ok_or("Amplifier bundle filename unavailable")?, + ); + let root = document + .parent() + .ok_or("Amplifier bundle parent unavailable")?; + { // A standalone bundle document does not select its entire parent // directory. Preserve only its finite resources and native package // closure; selecting a document in HOME must not copy HOME wholesale. - if !canonical + if !document .extension() .is_some_and(|extension| extension == "yaml" || extension == "yml" || extension == "md") { return Err("Amplifier local bundle source has an unsupported native format".into()); } - result.selections.push(CopySelection { - source: canonical.clone(), - destination_relative: relative.join( - canonical - .file_name() - .ok_or("Amplifier bundle filename unavailable")?, - ), - }); + if metadata.is_file() { + result.selections.push(CopySelection { + source: document.clone(), + destination_relative: document_relative.clone(), + }); + } let mut bundle_raw = String::new(); let remaining = (64 * 1024 * 1024usize) .checked_sub(state.view_bytes) .ok_or("Amplifier retained source view size overflow")?; let limit = remaining.min(16 * 1024 * 1024); - std::fs::File::open(&canonical) + std::fs::File::open(&document) .map_err(|_| "Amplifier selected bundle unreadable")? .take(limit as u64 + 1) .read_to_string(&mut bundle_raw) @@ -200,7 +229,7 @@ fn select_local_bundle( .view_bytes .checked_add(bundle_raw.len()) .ok_or("Amplifier retained source view size overflow")?; - let yaml = if canonical + let yaml = if document .extension() .is_some_and(|extension| extension == "md") { @@ -350,14 +379,9 @@ fn select_local_bundle( } template.replace_range(start..end, &replacement); } - result.rewritten_resources.push(( - relative.join( - canonical - .file_name() - .ok_or("Amplifier bundle filename unavailable")?, - ), - template, - )); + result + .rewritten_resources + .push((document_relative, template)); } } } diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 405248cf7..a437a5ca5 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -69,7 +69,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != 16 or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != 18 or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. From e382843c071204f577305836b7929a065874989e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 12:24:16 -0700 Subject: [PATCH 041/493] feat: deliver frozen provider path inputs through private execution IPC --- crates/freshell-runtime-client/src/lib.rs | 23 +++ crates/freshell-runtime-protocol/src/lib.rs | 4 + .../src/provider_inputs.rs | 151 ++++++++++++++++++ crates/freshell-session-host/src/main.rs | 6 + .../src/native_auth_lookup_tests.rs | 85 ++++++++++ crates/freshell-supervisor/src/service.rs | 72 +++++++++ 6 files changed, 341 insertions(+) diff --git a/crates/freshell-runtime-client/src/lib.rs b/crates/freshell-runtime-client/src/lib.rs index 46120e1b3..daee99f3e 100644 --- a/crates/freshell-runtime-client/src/lib.rs +++ b/crates/freshell-runtime-client/src/lib.rs @@ -51,6 +51,29 @@ pub struct RuntimeClient { } impl RuntimeClient { + pub async fn bind_provider_path_inputs( + &self, + soul_id: SoulId, + generation_id: String, + inputs: freshell_runtime_protocol::ProviderPathInputs, + ) -> Result<(), ClientError> { + let request = freshell_runtime_protocol::BindProviderPathInputsRequest { + expected_control_epoch: Some(self.current_epoch().await?), + soul_id, + generation_id, + inputs, + }; + match self + .request( + RequestId::new(), + AdminCommand::BindProviderPathInputs(request), + ) + .await? + { + AdminResult::ProviderPathInputsBound => Ok(()), + _ => Err(ClientError::UnexpectedResult), + } + } pub async fn reserve_provider_inputs( &self, soul_id: SoulId, diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index c8b1aba3d..1028fde52 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2726,6 +2726,7 @@ pub enum AdminCommand { Health, ReserveProviderInputs(ReserveProviderInputsRequest), NativeAuthLookup(NativeAuthLookupRequest), + BindProviderPathInputs(BindProviderPathInputsRequest), Launch(Box), Stop(StopRequest), Inventory, @@ -2933,6 +2934,7 @@ pub struct LaunchResult { pub enum AdminResult { ProviderInputReservation(ProviderInputReservation), NativeAuthLookup(NativeAuthValues), + ProviderPathInputsBound, Health { control_epoch: u64, installation_id: InstallationId, @@ -2997,6 +2999,8 @@ pub enum HostCommand { grant_id: GrantId, #[serde(default, skip_serializing_if = "Option::is_none")] fixture: Option, + #[serde(default, skip_serializing_if = "ProviderPathInputs::is_empty")] + provider_path_inputs: ProviderPathInputs, #[serde(default, skip_serializing_if = "Option::is_none")] terminal: Option>, #[serde(default, skip_serializing_if = "Option::is_none")] diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index a04208f2d..e5547b799 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -4,6 +4,109 @@ use crate::{McpCapabilityReference, RuntimeError, RuntimeErrorCode, SoulId}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; +/// Transient frozen path inputs travel only through authenticated process IPC. +/// This type must never be embedded in a durable launch specification. +#[derive(Clone, Default, PartialEq, Eq, Serialize)] +#[serde(transparent)] +pub struct ProviderPathInputs(std::collections::BTreeMap); +impl std::fmt::Debug for ProviderPathInputs { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("ProviderPathInputs") + .field("selected_count", &self.0.len()) + .finish() + } +} +impl ProviderPathInputs { + pub fn new(values: std::collections::BTreeMap) -> Result { + let mut result = Self::default(); + for (name, value) in values { + result.insert(name, value)?; + } + Ok(result) + } + pub fn insert(&mut self, name: String, value: String) -> Result<(), RuntimeError> { + let invalid = || { + RuntimeError::new(RuntimeErrorCode::InvalidRequest,"selected provider path inputs have invalid names, controls, collisions or exceeded profile") + }; + if name.is_empty() + || name.len() > 128 + || name.contains('=') + || name.chars().any(char::is_control) + || value.len() > 64 * 1024 + || value.contains('\0') + || self.0.contains_key(&name) + || self.0.len() >= 128 + { + return Err(invalid()); + } + let used = self + .0 + .iter() + .fold( + name.len().checked_add(value.len()), + |size, (name, value)| { + size.and_then(|size| size.checked_add(name.len())) + .and_then(|size| size.checked_add(value.len())) + }, + ) + .ok_or_else(invalid)?; + if used > 256 * 1024 { + return Err(invalid()); + } + self.0.insert(name, value); + Ok(()) + } + pub fn get(&self, name: &str) -> Option<&str> { + self.0.get(name).map(String::as_str) + } + pub fn iter(&self) -> impl Iterator { + self.0 + .iter() + .map(|(name, value)| (name.as_str(), value.as_str())) + } + pub fn is_empty(&self) -> bool { + self.0.is_empty() + } +} +impl<'de> Deserialize<'de> for ProviderPathInputs { + fn deserialize>(deserializer: D) -> Result { + struct Visitor; + impl<'de> serde::de::Visitor<'de> for Visitor { + type Value = ProviderPathInputs; + fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { + f.write_str("bounded transient provider path inputs") + } + fn visit_map>( + self, + mut map: A, + ) -> Result { + let mut inputs = ProviderPathInputs::default(); + while let Some(name) = map.next_key::()? { + if inputs.0.len() >= 128 || name.len() > 128 { + return Err(serde::de::Error::custom( + "selected input name profile exceeded", + )); + } + let value = map.next_value::()?; + inputs + .insert(name, value) + .map_err(|_| serde::de::Error::custom("selected input profile exceeded"))?; + } + Ok(inputs) + } + } + deserializer.deserialize_map(Visitor) + } +} +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct BindProviderPathInputsRequest { + pub expected_control_epoch: Option, + pub soul_id: SoulId, + pub generation_id: String, + pub inputs: ProviderPathInputs, +} + #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct PreparedProviderInputs { @@ -294,6 +397,54 @@ fn mount_destination(value: &str) -> bool { #[cfg(test)] mod tests { use super::*; + #[test] + fn provider_frozen_path_inputs_use_private_binding_rpc() { + let request = serde_json::json!({"method":"bind_provider_path_inputs","params":{ + "expectedControlEpoch":1,"soulId":crate::SoulId::new(), + "generationId":"grant-synthetic-input-generation", + "inputs":{"PUBLIC_RULE_ROOT":"/synthetic/original/rules"} + }}); + assert!(serde_json::from_value::(request).is_ok(), + "frozen path inputs need a private process-only delivery separate from durable LaunchRequest"); + } + + #[test] + fn provider_frozen_path_inputs_enforce_shared_profile_and_redact_diagnostics() { + let mut inputs = ProviderPathInputs::default(); + for name in ["A", "B", "C", "D"] { + inputs + .insert(name.into(), "x".repeat(64 * 1024 - 1)) + .unwrap(); + } + assert!(inputs.insert("E".into(), "x".into()).is_err()); + assert!(ProviderPathInputs::new(std::collections::BTreeMap::from([( + "A".into(), + "x".repeat(64 * 1024 + 1) + )])) + .is_err()); + let mut inputs = ProviderPathInputs::default(); + for id in 0..128 { + inputs + .insert(format!("PATH_{id}"), "/synthetic".into()) + .unwrap(); + } + assert!(inputs + .insert("PATH_128".into(), "/synthetic".into()) + .is_err()); + for raw in [ + r#"{"PATH":"/first","PATH":"/second"}"#, + r#"{"PATH":"bad\u0000value"}"#, + ] { + assert!(serde_json::from_str::(raw).is_err()); + } + let inputs: ProviderPathInputs = + serde_json::from_str(r#"{"PATH":"PRIVATE_PATH_INPUT_SENTINEL"}"#).unwrap(); + let wire = serde_json::to_vec(&inputs).unwrap(); + let received: ProviderPathInputs = serde_json::from_slice(&wire).unwrap(); + assert_eq!(received.get("PATH"), Some("PRIVATE_PATH_INPUT_SENTINEL")); + assert!(!format!("{received:?}").contains("PRIVATE_PATH_INPUT_SENTINEL")); + } + fn prepared() -> PreparedProviderInputs { PreparedProviderInputs { generation_id: "grant-synthetic".into(), diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index dd1600bf5..16de2cac1 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -337,6 +337,7 @@ async fn dispatch( execution_generation, grant_id, fixture, + provider_path_inputs, terminal, fresh_agent, resume_spec, @@ -356,6 +357,7 @@ async fn dispatch( execution_generation, grant_id, fixture, + provider_path_inputs, terminal: terminal.map(|terminal| *terminal), fresh_agent: fresh_agent.map(|agent| *agent), resume_spec: resume_spec.map(|resume_spec| *resume_spec), @@ -785,6 +787,7 @@ struct GrantExecutionRequest { execution_generation: u64, grant_id: GrantId, fixture: Option, + provider_path_inputs: freshell_runtime_protocol::ProviderPathInputs, terminal: Option, fresh_agent: Option, resume_spec: Option, @@ -800,6 +803,7 @@ async fn grant_execution( execution_generation, grant_id, fixture, + provider_path_inputs: _provider_path_inputs, terminal, fresh_agent, resume_spec, @@ -3952,6 +3956,7 @@ mod tests { execution_generation: 1, grant_id: GrantId::new(), fixture: Some(FixtureKind::Heartbeat), + provider_path_inputs: Default::default(), terminal: None, fresh_agent: None, resume_spec: None, @@ -4137,6 +4142,7 @@ mod tests { execution_generation: 1, grant_id: GrantId::new(), fixture: None, + provider_path_inputs: Default::default(), terminal: Some(Box::new(terminal)), fresh_agent: None, resume_spec: Some(Box::new(resume_spec)), diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index 8659884e1..d10cfa043 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -82,6 +82,91 @@ async fn provider_initial_auth_requires_creation_reservation_before_absence_rece "absence is reobserved rather than inferred from native identity" ); } +#[tokio::test] +async fn provider_frozen_path_binding_is_idempotent_private_and_generation_scoped() { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic helper artifact").unwrap(); + let registry_root = temp.path().join("registry"); + let registry = Registry::open(®istry_root, None).unwrap(); + let secret = "synthetic-control-secret"; + let supervisor = Supervisor::new( + registry, + Arc::new(AuthBackend(AtomicUsize::new(0))), + SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: binary, + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "private-path-binding".into(), + control_secret: secret.into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let soul = SoulId::new(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let sentinel = "PRIVATE_FROZEN_INPUT_SENTINEL"; + for (value, success) in [(sentinel, true), (sentinel, true), ("changed-input", false)] { + let request = BindProviderPathInputsRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul.clone(), + generation_id: generation.clone(), + inputs: ProviderPathInputs::new(BTreeMap::from([("RULE_ROOT".into(), value.into())])) + .unwrap(), + }; + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::BindProviderPathInputs(request), + ); + envelope.auth = Some(secret.into()); + assert_eq!(supervisor.dispatch(envelope).await.result.is_ok(), success); + } + assert_eq!( + supervisor + .provider_path_inputs + .lock() + .unwrap() + .get(&(soul.clone(), generation.clone())) + .unwrap() + .0 + .get("RULE_ROOT"), + Some(sentinel) + ); + assert!(supervisor.registry.inventory().await.unwrap().is_empty()); + for file in std::fs::read_dir(®istry_root).unwrap() { + let file = file.unwrap().path(); + if file.is_file() { + assert!( + !std::fs::read(file) + .unwrap() + .windows(sentinel.len()) + .any(|bytes| bytes == sentinel.as_bytes()), + "private binding must not write an input snapshot to registry or its WAL" + ); + } + } + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::BindProviderPathInputs(BindProviderPathInputsRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul.clone(), + generation_id: generation.clone(), + inputs: ProviderPathInputs::default(), + }), + ); + envelope.auth = Some(secret.into()); + supervisor.dispatch(envelope).await.result.unwrap(); + assert!(!supervisor + .provider_path_inputs + .lock() + .unwrap() + .contains_key(&(soul, generation))); +} + #[async_trait] impl RuntimeBackend for AuthBackend { async fn observe_initial_auth_volume( diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index c023fa305..e4d931fea 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -95,6 +95,11 @@ pub struct Supervisor { /// lock: its durable intent revision must be able to preempt recovery. pub(crate) recovery_locks: Arc>>>>, auth_lookup_authorities: Arc>>, + provider_path_inputs: Arc< + StdMutex< + HashMap<(SoulId, String), (freshell_runtime_protocol::ProviderPathInputs, Instant)>, + >, + >, } impl Supervisor { @@ -142,6 +147,7 @@ impl Supervisor { config: Arc::new(config), recovery_locks: Arc::new(Mutex::new(HashMap::new())), auth_lookup_authorities: Arc::new(StdMutex::new(HashMap::new())), + provider_path_inputs: Arc::new(StdMutex::new(HashMap::new())), }) } @@ -225,6 +231,38 @@ impl Supervisor { .map(AdminResult::ProviderInputReservation) .map_err(map_registry) } + AdminCommand::BindProviderPathInputs(request) => { + self.registry + .assert_epoch(request.expected_control_epoch) + .map_err(map_registry)?; + let invalid = || { + RuntimeError::new(RuntimeErrorCode::InvalidRequest,"provider path input binding has invalid or changed generation; reprepare before dispatch") + }; + let id = request + .generation_id + .strip_prefix("grant-") + .ok_or_else(invalid)?; + uuid::Uuid::parse_str(id).map_err(|_| invalid())?; + let mut bindings = self + .provider_path_inputs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + bindings.retain(|_, (_, expires)| *expires > Instant::now()); + let key = (request.soul_id, request.generation_id); + if request.inputs.is_empty() { + bindings.remove(&key); + } else if let Some((prior, _)) = bindings.get(&key) { + if prior != &request.inputs { + return Err(invalid()); + } + } else { + bindings.insert( + key, + (request.inputs, Instant::now() + Duration::from_secs(600)), + ); + } + Ok(AdminResult::ProviderPathInputsBound) + } AdminCommand::NativeAuthLookup(request) => { self.registry .assert_epoch(request.expected_control_epoch) @@ -1940,6 +1978,12 @@ impl Supervisor { "capability_pending: server rejected scoped grant", )); } + if matches!(action, "abort" | "revoke") { + self.provider_path_inputs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .remove(&(soul_id.clone(), grant_id.into())); + } Ok(()) } @@ -1982,6 +2026,12 @@ impl Supervisor { .await .map_err(map_backend)?; } + if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { + self.provider_path_inputs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .remove(&(workload.soul_id.clone(), inputs.generation_id.clone())); + } Ok(()) } @@ -2310,6 +2360,27 @@ impl Supervisor { grant: &ExecutionGrantRecord, ) -> Result { let incarnation_id = handle.incarnation_id().clone(); + let preparation = workload + .terminal + .as_ref() + .and_then(|launch| launch.provider_launch_context.as_ref()) + .or_else(|| { + workload + .fresh_agent + .as_ref() + .and_then(|launch| launch.provider_launch_context.as_ref()) + }) + .and_then(|context| context.prepared_inputs.as_ref()); + let provider_path_inputs = preparation + .and_then(|inputs| { + self.provider_path_inputs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .get(&(workload.soul_id.clone(), inputs.generation_id.clone())) + .filter(|(_, expires)| *expires > Instant::now()) + .map(|(values, _)| values.clone()) + }) + .unwrap_or_default(); let command = HostCommand::GrantExecution { incarnation_id: incarnation_id.clone(), soul_id: workload.soul_id.clone(), @@ -2318,6 +2389,7 @@ impl Supervisor { execution_generation: grant.execution_generation, grant_id: grant.grant_id.clone(), fixture: workload.fixture, + provider_path_inputs, terminal: workload.terminal.clone().map(Box::new), fresh_agent: workload.fresh_agent.clone().map(Box::new), resume_spec: workload.resume_spec.clone().map(Box::new), From 6bc57677670d639e55691c7bbc95dc5b40ae088f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 13:42:08 -0700 Subject: [PATCH 042/493] Connect shared OpenCode source preparation to both launch routes --- .../src/native_auth_exchange.rs | 10 +- .../freshell-platform/src/opencode_recipe.rs | 6 + crates/freshell-runtime-protocol/src/lib.rs | 1 + .../freshell-server/src/fresh_agent_proxy.rs | 64 +++- .../src/managed_mcp_capability.rs | 209 ++++++++++- .../managed_mcp_capability/source_stage.rs | 148 +++++++- .../src/managed_provider_bootstrap.rs | 2 +- .../src/managed_provider_references.rs | 181 +++++++++- .../environment.rs | 338 ++++++++++++++++++ .../managed_provider_references/sources.rs | 18 +- crates/freshell-server/src/managed_runtime.rs | 71 +++- crates/freshell-session-host/src/main.rs | 124 ++++++- .../src/provider_input_preparation.rs | 268 ++++++++++++++ 13 files changed, 1371 insertions(+), 69 deletions(-) create mode 100644 crates/freshell-server/src/managed_provider_references/environment.rs create mode 100644 crates/freshell-session-host/src/provider_input_preparation.rs diff --git a/crates/freshell-platform/src/native_auth_exchange.rs b/crates/freshell-platform/src/native_auth_exchange.rs index 745e3c9a9..24012e3a4 100644 --- a/crates/freshell-platform/src/native_auth_exchange.rs +++ b/crates/freshell-platform/src/native_auth_exchange.rs @@ -47,7 +47,7 @@ impl std::io::Write for BoundedFrame { /// The only serialized auth reply is sent directly to this private socket. #[cfg(unix)] -fn selected_native_auth( +pub fn read_selected_native_auth( home: &Path, keys: &[String], ) -> Result, &'static str> { @@ -115,7 +115,7 @@ pub fn serve(home: &Path, socket: &Path) -> Result<(), &'static str> { .map_err(|_| "native auth request unavailable")?; let keys: Vec = serde_json::from_slice(&request).map_err(|_| "native auth request invalid")?; - let selected = selected_native_auth(home, &keys)?; + let selected = read_selected_native_auth(home, &keys)?; let mut frame = BoundedFrame(Vec::new()); serde_json::to_writer(&mut frame, &selected).map_err(|_| "native auth reply exceeds limit")?; stream @@ -195,16 +195,16 @@ mod tests { std::fs::write(&outside, "PRIVATE_SENTINEL").unwrap(); symlink(&outside, &path).unwrap(); assert_eq!( - selected_native_auth(&home, &[]).unwrap_err(), + read_selected_native_auth(&home, &[]).unwrap_err(), "native auth file escaped owned HOME" ); std::fs::remove_file(&path).unwrap(); std::fs::write(&path, vec![b' '; 128 * 1024 + 1]).unwrap(); assert_eq!( - selected_native_auth(&home, &[]).unwrap_err(), + read_selected_native_auth(&home, &[]).unwrap_err(), "native auth file exceeds limit" ); std::fs::remove_file(&path).unwrap(); - assert!(selected_native_auth(&home, &[]).unwrap().is_empty()); + assert!(read_selected_native_auth(&home, &[]).unwrap().is_empty()); } } diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index b002eb195..9ac068d8c 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -32,6 +32,8 @@ pub struct SourceManifest { pub original_home: PathBuf, pub sources: Vec, pub files: Vec, + #[serde(default)] + pub frozen_inputs: Vec, } #[derive(Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] @@ -43,6 +45,10 @@ pub struct SourceView { pub generated_name: Option, pub occurrences: Vec, pub referenced_variables: Vec, + #[serde(default)] + pub path_variables: Vec, + #[serde(default)] + pub original_form: bool, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 1028fde52..b2ec8c1c2 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -702,6 +702,7 @@ pub enum StopOutcome { #[serde(rename_all = "SCREAMING_SNAKE_CASE")] pub enum RuntimeErrorCode { InvalidRequest, + ProviderInputsChanged, ProtocolVersionMismatch, UnauthorizedRole, FrameTooLarge, diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 4e23748b5..9f5b07ec2 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -448,31 +448,57 @@ impl HostedFreshAgentProxy { let session_type = session_type_wire(message.session_type); let request_id = stable_request_id(&message.request_id).map_err(|_| ())?; let provider_bootstrap_files = provider_bootstrap_files(&provider).map_err(|_| ())?; - let provider_secret_references = + let mut provider_secret_references = crate::managed_provider_bootstrap::named_provider_onecli_references(provider.as_str()) .map_err(|_| ())?; - let mcp_capability = crate::managed_mcp_capability::issue_mcp_capability( - &soul, - provider.as_str(), - crate::managed_mcp_capability::OpencodeEphemeralInput { - cwd: Some(&cwd), - workspace: Some(&workspace), - ..Default::default() - }, - ) - .map_err(|error| { - tracing::warn!(%error, "fresh_agent.create.capability_failed"); - })?; - let mut provider_launch_context = - crate::managed_provider_bootstrap::provider_launch_context_for_managed_at( - provider.as_str(), - &workspace, + let (mcp_capability, mut provider_launch_context) = if provider == FreshProvider::Opencode { + let home = std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .ok_or(())?; + let original = |name: &str| std::env::var(name).ok(); + let prepared = crate::managed_provider_references::prepare_managed_opencode( + &self.client, + &soul, + &message.request_id, + &home, &cwd, - Some(mcp_capability.clone()), + &workspace, + &original, + provider_secret_references, + None, ) + .await .map_err(|error| { - tracing::warn!(event="fresh_agent.create.provider_environment_failed", %error); + tracing::warn!(event="fresh_agent.create.provider_inputs_failed",%error); })?; + provider_secret_references = prepared.references; + let capability = prepared.context.mcp_capability.clone().ok_or(())?; + (capability, Some(prepared.context)) + } else { + let mcp_capability = crate::managed_mcp_capability::issue_mcp_capability( + &soul, + provider.as_str(), + crate::managed_mcp_capability::OpencodeEphemeralInput { + cwd: Some(&cwd), + workspace: Some(&workspace), + ..Default::default() + }, + ) + .map_err(|error| { + tracing::warn!(%error, "fresh_agent.create.capability_failed"); + })?; + let provider_launch_context = + crate::managed_provider_bootstrap::provider_launch_context_for_managed_at( + provider.as_str(), + &workspace, + &cwd, + Some(mcp_capability.clone()), + ) + .map_err(|error| { + tracing::warn!(event="fresh_agent.create.provider_environment_failed", %error); + })?; + (mcp_capability, provider_launch_context) + }; // Fresh agents use the direct provider transports, which currently do // not inject Freshell's terminal MCP server. Retain ordinary provider // MCP/config files, plugins, and the staged user root; only remove diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index bbcb592e0..787113da9 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -39,6 +39,11 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub workspace: Option<&'a Path>, pub source_plan: Option<&'a [crate::managed_provider_references::sources::PlannedSourceOccurrence]>, + pub initial_auth_reservation: Option<&'a str>, + pub secret_references: Option<&'a [freshell_runtime_protocol::ProviderSecretReference]>, + pub config_root: Option<&'a Path>, + pub launch_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, + pub frozen_inputs: Option<&'a [String]>, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -749,7 +754,15 @@ fn stage_provider_root( .filter(|value| !value.is_empty()) .map(PathBuf::from) .unwrap_or_else(|| home.join(".config")); - (config_home.join("opencode"), config_home) + let root = opencode_input + .config_root + .map(Path::to_path_buf) + .unwrap_or_else(|| config_home.join("opencode")); + let boundary = root + .parent() + .ok_or("managed config root parent unavailable")? + .to_owned(); + (root, boundary) } else { (home.join(provider_dir), home.to_path_buf()) }; @@ -802,10 +815,18 @@ fn stage_provider_root( excluded.push(canonical); } } + let fallback_secrets; + let secret_references = if let Some(references) = opencode_input.secret_references { + references + } else { + fallback_secrets = + crate::managed_provider_bootstrap::named_provider_onecli_references(provider)?; + &fallback_secrets + }; excluded.extend( - crate::managed_provider_bootstrap::named_provider_onecli_references(provider)? - .into_iter() - .map(|reference| PathBuf::from(reference.source_path)), + secret_references + .iter() + .map(|reference| PathBuf::from(&reference.source_path)), ); let amplifier_resources = if provider == "amplifier" { Some(amplifier_resources::prepare( @@ -950,6 +971,7 @@ fn stage_provider_root( home, opencode_input.cwd.ok_or("source plan requires cwd")?, plan, + opencode_input.frozen_inputs.unwrap_or(&[]), )?; } allow_controller_stage_read(&temporary)?; @@ -1190,6 +1212,59 @@ impl CapabilityStore { ) }) .transpose()?; + if let (Some(root), Some(plan)) = (&staged_provider_root, opencode_input.source_plan) { + let reference = McpCapabilityReference { + grant_id: grant_id.clone(), + endpoint: endpoint.clone(), + provider_relative_path: CAPABILITY_PATH.into(), + host_gateway_address: routed_host_ipv4(), + }; + let home = source + .ok_or("prepared source requires original provider HOME")? + .1; + let prepared = source_stage::describe( + root, + home, + plan, + &soul_id, + &reference, + opencode_input.initial_auth_reservation, + opencode_input.config_root, + ) + .and_then(|inputs| { + if let Some(context) = opencode_input.launch_context { + let mut context = context.clone(); + context.mcp_capability = Some(reference.clone()); + context.config.retain(|selected| { + let destination = format!( + "/home/freshell/provider/{}", + selected.provider_relative_path + ); + !inputs.config_binds.iter().any(|bind| { + bind.logical_destination == destination + || bind.effective_destination == destination + }) + }); + context.prepared_inputs = Some(inputs); + context + .validate("opencode") + .map_err(|error| error.message)?; + let bytes = freshell_runtime_protocol::bounded_json_bytes( + &context, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "complete prepared context exceeds metadata profile")?; + let path = root.join(".freshell/prepared-context.json"); + write_private_bytes(&path, &bytes)?; + allow_controller_stage_read(&path)?; + } + Ok(()) + }); + if let Err(error) = prepared { + let _ = std::fs::remove_dir_all(root); + return Err(error); + } + } let grant = Grant { preparation: Some(InputPreparation { phase: InputPreparationPhase::Pending, @@ -1454,6 +1529,51 @@ pub(crate) fn issue_mcp_capability( ) } +pub(crate) fn issue_prepared_opencode_context( + soul: &SoulId, + home: &Path, + context: &freshell_runtime_protocol::ProviderLaunchContext, + input: OpencodeEphemeralInput<'_>, +) -> Result { + let input = OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }; + let directory = capability_dir()?; + let capability = store().issue( + soul.clone(), + &directory, + endpoint()?, + Some(("opencode", home, &context.config)), + input, + None, + )?; + let result = (|| { + use std::io::Read; + let stage = staged_provider_root_path(&directory, &capability.grant_id)?; + let mut bytes = Vec::new(); + std::fs::File::open(stage.join(".freshell/prepared-context.json")) + .map_err(|_| "complete prepared provider context unavailable")? + .take((freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + 1) as u64) + .read_to_end(&mut bytes) + .map_err(|_| "complete prepared provider context read failed")?; + if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err("complete prepared provider context exceeds metadata profile".into()); + } + let context: freshell_runtime_protocol::ProviderLaunchContext = + serde_json::from_slice(&bytes) + .map_err(|_| "complete prepared provider context metadata invalid")?; + if context.mcp_capability.as_ref() != Some(&capability) { + return Err("complete prepared context changed generation before publication".into()); + } + Ok(context) + })(); + if result.is_err() { + let _ = store().revoke(&capability.grant_id, &directory); + } + result +} + /// A web-server restart restores only grants whose exact incarnation remains /// live in the supervisor inventory. Recoverable souls retain their private /// OpenCode inputs before stale scopes and incarnation stages are revoked. @@ -2665,6 +2785,7 @@ except Exception as error: cwd: Some(home.path()), workspace: Some(home.path()), source_plan: None, + ..Default::default() }, None, ) @@ -2707,6 +2828,85 @@ except Exception as error: .contains("nested-secret-byte")); } + #[test] + fn managed_provider_source_preparation_publishes_complete_descriptor_before_capability() { + use crate::managed_provider_references::sources::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let config_root = home.join(".config/opencode"); + std::fs::create_dir_all(&config_root).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let config = config_root.join("opencode.json"); + let raw = r#"{"instructions":["{file:rules}"]}"#; + std::fs::write(&config, raw).unwrap(); + std::fs::write(config_root.join("rules"), "PRIVATE_REFERENCED_NONCE").unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let context = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "opencode", &workspace, &home, None, + ) + .unwrap() + .unwrap(); + let store = CapabilityStore::default(); + let capabilities = owned.path().join("capabilities"); + std::fs::create_dir(&capabilities).unwrap(); + let soul = SoulId::new(); + let capability = store + .issue( + soul.clone(), + &capabilities, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &context.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let staged = staged_provider_root_path(&capabilities, &capability.grant_id).unwrap(); + let metadata = std::fs::read(staged.join(".freshell/prepared-inputs.json")).expect( + "complete prepared input descriptor must exist when its capability is published", + ); + let inputs: freshell_runtime_protocol::PreparedProviderInputs = + serde_json::from_slice(&metadata).unwrap(); + inputs.validate(Some(&capability)).unwrap(); + let ready: freshell_runtime_protocol::ProviderLaunchContext = serde_json::from_slice( + &std::fs::read(staged.join(".freshell/prepared-context.json")).unwrap(), + ) + .unwrap(); + assert_eq!(ready.prepared_inputs.as_ref(), Some(&inputs)); + assert_eq!(ready.mcp_capability.as_ref(), Some(&capability)); + assert!(!ready.config.iter().any(|selected|selected.provider_relative_path==".config/opencode/opencode.json"),"ordinary HOME copying must not overwrite the exact read-only prepared config mount"); + assert_eq!(inputs.manifest.source_count, 1); + assert_eq!(inputs.config_binds.len(), 1); + assert_eq!( + inputs.config_binds[0].logical_destination, + "/home/freshell/provider/.config/opencode/opencode.json" + ); + assert_eq!( + inputs.indexed_files.as_ref().unwrap().volume_name, + freshell_runtime_protocol::indexed_provider_volume_name(&soul, &capability.grant_id) + ); + assert!(!metadata + .windows(b"PRIVATE_REFERENCED_NONCE".len()) + .any(|bytes| bytes == b"PRIVATE_REFERENCED_NONCE")); + assert_eq!(std::fs::read_to_string(config).unwrap(), raw); + } + #[test] fn managed_provider_source_stage_retains_unexpanded_direct_origins() { let owned = tempfile::tempdir().unwrap(); @@ -2969,6 +3169,7 @@ except Exception as error: cwd: Some(home.path()), workspace: Some(home.path()), source_plan: None, + ..Default::default() }, None, ) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 5ba267adb..cbaa4733c 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -9,6 +9,7 @@ pub(super) fn stage( home: &Path, cwd: &Path, plan: &[PlannedSourceOccurrence], + frozen_inputs: &[String], ) -> Result<(), String> { let mut manifest = SourceManifest { version: 1, @@ -16,6 +17,7 @@ pub(super) fn stage( original_home: home.into(), sources: Vec::new(), files: Vec::new(), + frozen_inputs: frozen_inputs.to_vec(), }; private_directory(&root.join(".freshell/source-views"))?; private_directory(&root.join(".freshell/templates"))?; @@ -69,6 +71,8 @@ pub(super) fn stage( .map(|recipe| recipe.generated_name.clone()), occurrences: source.references.occurrences.clone(), referenced_variables: source.references.referenced_variables.clone(), + path_variables: source.references.path_variables.clone(), + original_form: false, }); } for (expected, (source_id, canonical_source)) in files.into_iter().enumerate() { @@ -80,10 +84,144 @@ pub(super) fn stage( canonical_source, }); } - let metadata = freshell_runtime_protocol::bounded_json_bytes( - &manifest, - freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, - ) - .map_err(|_| "source manifest metadata profile exceeded")?; + let metadata = freshell_runtime_protocol::bounded_json_bytes(&manifest, 64 * 1024 * 1024) + .map_err(|_| "source manifest metadata profile exceeded")?; write_private_bytes(&root.join(".freshell/input-manifest.json"), &metadata) } + +pub(super) fn describe( + root: &Path, + home: &Path, + plan: &[PlannedSourceOccurrence], + soul: &freshell_runtime_protocol::SoulId, + capability: &freshell_runtime_protocol::McpCapabilityReference, + initial_auth_reservation: Option<&str>, + config_root: Option<&Path>, +) -> Result { + use freshell_runtime_protocol::*; + use sha2::{Digest, Sha256}; + use std::io::Read; + let runtime_home = Path::new("/home/freshell/provider"); + let default_global = home.join(".config/opencode"); + let global = config_root.unwrap_or(&default_global); + let mut sources = BTreeMap::new(); + let mut binds = Vec::new(); + let mut recipes = 0usize; + let mut canonical_occurrences = BTreeMap::<&Path, usize>::new(); + for occurrence in plan { + *canonical_occurrences + .entry(&occurrence.source.canonical_source) + .or_default() += 1; + } + for (ordinal, occurrence) in plan.iter().enumerate() { + if canonical_occurrences[occurrence.source.canonical_source.as_path()] > 1 { + return Err( + "shared source preparation requires its original-form aliases before publication" + .into(), + ); + } + let logical = &occurrence.source.logical_path; + let destination = if let Ok(relative) = logical.strip_prefix(global) { + runtime_home.join(".config/opencode").join(relative) + } else { + logical + .strip_prefix(home) + .map(|relative| runtime_home.join(relative)) + .unwrap_or_else(|_| logical.clone()) + }; + let effective = if let Ok(relative) = logical.strip_prefix(&global) { + let physical = std::fs::canonicalize(root.join(relative)) + .map_err(|_| "selected global config is unavailable in its private stage")?; + let relative = physical + .strip_prefix(root) + .map_err(|_| "selected global config escaped its owned stage")?; + if relative.starts_with(".freshell/resources") { + runtime_home.join(relative) + } else { + runtime_home.join(".config/opencode").join(relative) + } + } else { + occurrence + .source + .canonical_source + .strip_prefix(home) + .map(|relative| runtime_home.join(relative)) + .unwrap_or_else(|_| occurrence.source.canonical_source.clone()) + }; + binds.push(ProviderConfigBind { + source_relative_path: format!(".freshell/templates/source-{ordinal}"), + logical_destination: destination + .to_str() + .ok_or("config destination is not UTF-8")? + .into(), + effective_destination: effective + .to_str() + .ok_or("config destination is not UTF-8")? + .into(), + }); + recipes = recipes + .checked_add(occurrence.references.recipes.len()) + .ok_or("source recipe count overflow")?; + for selected in &occurrence.references.selections { + let id = selected + .runtime_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .and_then(|id| id.parse::().ok()) + .filter(|id| *id < 128) + .ok_or("invalid selected source index")?; + if sources + .insert(id, &selected.canonical_source) + .is_some_and(|prior| prior != &selected.canonical_source) + { + return Err("selected source identity changed under one index".into()); + } + } + } + let mut file = std::fs::File::open(root.join(".freshell/input-manifest.json")) + .map_err(|_| "prepared source manifest unavailable")? + .take(64 * 1024 * 1024 + 1); + let mut hash = Sha256::new(); + let mut used = 0usize; + let mut buffer = [0; 64 * 1024]; + loop { + let size = file + .read(&mut buffer) + .map_err(|_| "prepared source manifest read failed")?; + if size == 0 { + break; + } + used = used + .checked_add(size) + .ok_or("source manifest size overflow")?; + if used > 64 * 1024 * 1024 { + return Err("source manifest metadata exceeds 64 MiB".into()); + } + hash.update(&buffer[..size]); + } + let inputs = PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: initial_auth_reservation.map(str::to_string), + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", hash.finalize()), + source_count: u16::try_from(sources.len()) + .map_err(|_| "selected source count overflow")?, + alias_count: 0, + recipe_count: u16::try_from(recipes).map_err(|_| "recipe count overflow")?, + }, + config_binds: binds, + resource_mounts: Vec::new(), + indexed_files: (!sources.is_empty()).then(|| IndexedProviderFiles { + volume_name: indexed_provider_volume_name(soul, &capability.grant_id), + aliases: Vec::new(), + }), + }; + inputs + .validate(Some(capability)) + .map_err(|error| error.message)?; + let metadata = bounded_json_bytes(&inputs, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| "prepared descriptor exceeds control profile")?; + write_private_bytes(&root.join(".freshell/prepared-inputs.json"), &metadata)?; + super::allow_controller_stage_read(&root.join(".freshell/prepared-inputs.json"))?; + Ok(inputs) +} diff --git a/crates/freshell-server/src/managed_provider_bootstrap.rs b/crates/freshell-server/src/managed_provider_bootstrap.rs index 54c8bb944..9797f3e7b 100644 --- a/crates/freshell-server/src/managed_provider_bootstrap.rs +++ b/crates/freshell-server/src/managed_provider_bootstrap.rs @@ -70,7 +70,7 @@ fn provider_launch_context_from_home_at( ) } -fn provider_launch_context_from_home_at_with_xdg( +pub(crate) fn provider_launch_context_from_home_at_with_xdg( provider: &str, workspace: &Path, project_dir: &Path, diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 9245f0c3e..ee4754a72 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -3,6 +3,7 @@ use std::path::{Path, PathBuf}; pub(crate) mod codex; +pub(crate) mod environment; pub(crate) mod sources; pub struct FileSourceOrigin<'a> { @@ -50,6 +51,7 @@ pub struct ResolvedFileReferencePlan { unexpanded_direct_source: Option>, pub selections: Vec, pub referenced_variables: Vec, + pub path_variables: Vec, pub recipes: Vec, pub occurrences: Vec, pub substitution_events: usize, @@ -96,6 +98,25 @@ pub(super) fn resolve_file_references_from_view( env: &dyn Fn(&str) -> Option, recipe_ordinal: usize, allocate: &mut dyn FnMut(&Path) -> Result, +) -> Result { + resolve_file_references_from_view_relocated( + raw, + source_view, + origin, + env, + recipe_ordinal, + allocate, + &std::collections::BTreeMap::new(), + ) +} +pub(super) fn resolve_file_references_from_view_relocated( + raw: &str, + source_view: Option>, + origin: FileSourceOrigin<'_>, + env: &dyn Fn(&str) -> Option, + recipe_ordinal: usize, + allocate: &mut dyn FnMut(&Path) -> Result, + relocated_files: &std::collections::BTreeMap, ) -> Result { use freshell_platform::opencode_config::{ file_reference_path, transform_file_tokens, TokenTransformError, @@ -103,6 +124,7 @@ pub(super) fn resolve_file_references_from_view( let mut selections: Vec = Vec::new(); let mut rewritten = Vec::new(); let mut occurrences = Vec::new(); + let mut path_variables = std::collections::BTreeSet::new(); let mut generated = false; let mut planned_bytes = 0u64; let mut aliases = std::collections::BTreeSet::new(); @@ -133,6 +155,7 @@ pub(super) fn resolve_file_references_from_view( .iter() .map(|name| name.to_string()) .collect(); + path_variables.extend(names.iter().cloned()); substitution_events.set(substitution_events.get() + 1); if substitution_events.get() > 8192 { return Err(fail( @@ -148,7 +171,8 @@ pub(super) fn resolve_file_references_from_view( origin.cwd, origin.original_home, ); - let canonical = std::fs::canonicalize(&path).map_err(|_| { + let physical_source = relocated_files.get(&path).unwrap_or(&path); + let canonical = std::fs::canonicalize(physical_source).map_err(|_| { fail( token.ordinal, names.clone(), @@ -283,6 +307,7 @@ pub(super) fn resolve_file_references_from_view( .then(|| source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned()))), selections, referenced_variables: variables.into_inner().into_iter().collect(), + path_variables: path_variables.into_iter().collect(), recipes, occurrences, substitution_events: substitution_events.get(), @@ -669,3 +694,157 @@ process.stdout.write(JSON.stringify(receipts)); ); } } + +pub(crate) struct PreparedManagedOpencode { + pub context: freshell_runtime_protocol::ProviderLaunchContext, + pub references: Vec, +} + +/// Terminal, fresh and scoped replacement callers share this one preparation. +/// The original lookup and decoded grants never enter the returned metadata. +pub(crate) async fn prepare_managed_opencode( + client: &freshell_runtime_client::RuntimeClient, + soul: &freshell_runtime_protocol::SoulId, + creation_seed: &str, + home: &Path, + cwd: &Path, + workspace: &Path, + original: &(dyn Fn(&str) -> Option + Sync), + mut references: Vec, + auth_authority: Option<&str>, +) -> Result { + use sources::*; + if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) + || original("OPENCODE_TUI_CONFIG").is_some_and(|value| !value.is_empty()) + { + return Err( + "selected OpenCode inline/TUI source requires its owned pre-recipe view".into(), + ); + } + let xdg = original("XDG_CONFIG_HOME") + .filter(|value| !value.is_empty()) + .map(PathBuf::from) + .unwrap_or_else(|| home.join(".config")); + let xdg = if xdg.is_absolute() { + xdg + } else { + cwd.join(xdg) + }; + let config_root = xdg.join("opencode"); + let base = crate::managed_provider_bootstrap::provider_launch_context_from_home_at_with_xdg( + "opencode", + workspace, + cwd, + home, + Some(&xdg), + None, + )? + .ok_or("OpenCode provider preparation unavailable")?; + let selected = discover_opencode(OpencodeDiscovery { + home, + cwd, + worktree: workspace, + config_root: &config_root, + env: original, + }) + .map_err(str::to_string)?; + let loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; + if loaded + .iter() + .any(|source| source.source.lane == SourceLane::Tui) + { + return Err("selected OpenCode TUI source requires its owned pre-recipe view".into()); + } + let names = source_environment_names(&loaded).map_err(|error| error.to_string())?; + let mut environment = environment::SourceEnvironment::read(names, &references, original)?; + let mut reservation = None; + if environment.needs_native_auth() { + let authority = if let Some(authority) = auth_authority { + authority.to_owned() + } else { + let receipt = client + .reserve_provider_inputs(soul.clone(), "opencode".into(), creation_seed.into()) + .await + .map_err(|error| error.to_string())?; + reservation = Some(receipt.reservation_id.clone()); + receipt.reservation_id + }; + let values = client + .native_auth_lookup(soul.clone(), authority, environment.names().to_vec()) + .await + .map_err(|error| error.to_string())?; + environment.set_native_auth(values.0)?; + } + let plan = environment + .plan_sources(loaded, cwd, home) + .map_err(|error| error.to_string())?; + let frozen = environment.frozen_inputs(&plan)?; + let frozen_names = frozen + .iter() + .map(|(name, _)| name.to_owned()) + .collect::>(); + let mut files = std::collections::BTreeMap::new(); + for source in &plan { + for selection in &source.references.selections { + let id = selection + .runtime_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .and_then(|value| value.parse::().ok()) + .ok_or("invalid prepared source index")?; + if files + .insert(id, selection.canonical_source.clone()) + .is_some_and(|prior| prior != selection.canonical_source) + { + return Err("prepared source changes one selected index".into()); + } + } + } + for (expected, (id, path)) in files.into_iter().enumerate() { + if id as usize != expected { + return Err("prepared source indexes are not contiguous".into()); + } + references.push(freshell_runtime_protocol::ProviderSecretReference { + source_path: path.to_str().ok_or("selected source is not UTF-8")?.into(), + profile: freshell_runtime_protocol::ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{id}")), + }); + } + freshell_runtime_protocol::validate_provider_secrets("opencode", &references) + .map_err(|error| error.message)?; + let context = crate::managed_mcp_capability::issue_prepared_opencode_context( + soul, + home, + &base, + crate::managed_mcp_capability::OpencodeEphemeralInput { + cwd: Some(cwd), + workspace: Some(workspace), + config_root: Some(&config_root), + source_plan: Some(&plan), + initial_auth_reservation: reservation.as_deref(), + secret_references: Some(&references), + frozen_inputs: Some(&frozen_names), + ..Default::default() + }, + )?; + let capability = context + .mcp_capability + .as_ref() + .ok_or("prepared source capability unavailable")?; + if let Err(error) = client + .bind_provider_path_inputs(soul.clone(), capability.grant_id.clone(), frozen) + .await + { + let _ = crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id); + return Err(error.to_string()); + } + context + .prepared_inputs + .as_ref() + .ok_or("prepared source descriptors unavailable")? + .validate_references(&references) + .map_err(|error| error.message)?; + Ok(PreparedManagedOpencode { + context, + references, + }) +} diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs new file mode 100644 index 000000000..005cd3030 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -0,0 +1,338 @@ +//! One bounded controller read of selected child grants and original inputs. +use super::sources::{PlannedSourceOccurrence, SourceLane}; +use freshell_runtime_protocol::{ProviderPathInputs, ProviderSecretReference}; +use std::collections::BTreeMap; + +pub(super) struct SourceEnvironment { + ambient: BTreeMap, + onecli: BTreeMap, + auth: Option>, + names: Vec, + relocated_files: BTreeMap, +} +impl SourceEnvironment { + pub fn read( + names: Vec, + references: &[ProviderSecretReference], + original: &dyn Fn(&str) -> Option, + ) -> Result { + use freshell_platform::provider_environment_grants::{ + read_environment_grant, read_opencode_auth_environment, EnvironmentProfile, + }; + use freshell_runtime_protocol::ProviderSecretProfile; + freshell_runtime_protocol::validate_provider_secrets("opencode", references) + .map_err(|error| error.message)?; + freshell_platform::native_auth_exchange::validate_keys(&names).map_err(str::to_string)?; + let mut ambient = BTreeMap::new(); + let mut ambient_bytes = 0usize; + for name in &names { + if let Some(value) = original(name) { + ambient_bytes = ambient_bytes + .checked_add(value.len()) + .ok_or("source input size overflow")?; + if value.len() > 16 * 1024 * 1024 || ambient_bytes > 64 * 1024 * 1024 { + return Err("source input view profile exceeded".into()); + } + ambient.insert(name.clone(), value); + } + } + let mut onecli = BTreeMap::new(); + let mut auth = None; + let mut relocated_files = BTreeMap::new(); + for reference in references { + match reference.profile { + ProviderSecretProfile::OpencodeOnecliEnvironment => { + let values = read_environment_grant( + std::path::Path::new(&reference.source_path), + EnvironmentProfile::Opencode, + ) + .map_err(|error| error.to_string())?; + onecli.extend(values.into_iter().filter(|(name, _)| names.contains(name))); + } + ProviderSecretProfile::OpencodeOnecliAuthFile => { + auth = Some( + read_opencode_auth_environment( + std::path::Path::new(&reference.source_path), + &names, + ) + .map_err(|error| error.to_string())?, + ); + } + ProviderSecretProfile::OpencodeOnecliCaFile => { + let destination = reference + .profile + .provider_file_relative_path() + .ok_or("CA grant destination unavailable")?; + relocated_files.insert( + std::path::Path::new("/home/freshell/provider").join(destination), + std::path::PathBuf::from(&reference.source_path), + ); + } + _ => {} + } + } + Ok(Self { + ambient, + onecli, + auth, + names, + relocated_files, + }) + } + + pub fn lookup(&self, lane: SourceLane, name: &str) -> Option { + if lane == SourceLane::Main { + if let Some(value) = self.auth.as_ref().and_then(|auth| auth.get(name)) { + return Some(value.clone()); + } + } + self.onecli + .get(name) + .or_else(|| self.ambient.get(name)) + .cloned() + } + pub fn needs_native_auth(&self) -> bool { + self.auth.is_none() + } + pub fn names(&self) -> &[String] { + &self.names + } + pub fn set_native_auth(&mut self, values: BTreeMap) -> Result<(), String> { + freshell_platform::native_auth_exchange::validate_selected(&self.names, &values) + .map_err(str::to_string)?; + if self.auth.is_some() { + return Err("selected typed auth must not be replaced by native auth fallback".into()); + } + self.auth = Some(values); + Ok(()) + } + pub fn plan_sources( + &self, + sources: Vec, + cwd: &std::path::Path, + home: &std::path::Path, + ) -> Result, super::ReferenceError> { + if self.needs_native_auth() + && sources + .iter() + .any(|source| source.source.lane == SourceLane::Main) + { + return Err(super::ReferenceError { + operation: "prepare", + field: "OpenCode native auth source", + source_ordinal: 0, + declaring_source: cwd.into(), + variables: vec![], + cause: "native auth selection requires the current provider preparation attempt", + }); + } + super::sources::plan_loaded_opencode_sources_relocated( + sources, + cwd, + home, + &|lane, _, name| self.lookup(lane, name), + &self.relocated_files, + ) + } + pub fn frozen_inputs( + &self, + plan: &[PlannedSourceOccurrence], + ) -> Result { + let mut selected = std::collections::BTreeSet::new(); + for source in plan { + let names = if source.references.recipes.is_empty() { + &source.references.path_variables + } else { + &source.references.referenced_variables + }; + for name in names { + if self.onecli.contains_key(name) + || (source.source.lane == SourceLane::Main + && self + .auth + .as_ref() + .is_some_and(|auth| auth.contains_key(name))) + { + continue; + } + if self.ambient.contains_key(name) { + selected.insert(name.clone()); + } + } + } + let mut result = ProviderPathInputs::default(); + for name in selected { + if matches!( + name.as_str(), + "AUTH_TOKEN" | "FRESHELL_TOKEN" | "FRESHELL_URL" + ) || name.starts_with("FRESHELL_CONFIG_REFERENCE_") + { + return Err("selected original source input collides with controller authentication or generated names".into()); + } + result + .insert(name.clone(), self.ambient[&name].clone()) + .map_err(|error| error.message)?; + } + Ok(result) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use freshell_runtime_protocol::ProviderSecretProfile; + use std::os::unix::fs::PermissionsExt; + #[test] + fn managed_provider_source_environment_freezes_only_selected_original_inputs() { + use super::super::sources::{LoadedSourceOccurrence, SourceOccurrence}; + let owned = tempfile::tempdir().unwrap(); + let rules = owned.path().join("rules"); + std::fs::write(&rules, "synthetic-rule-nonce").unwrap(); + let raw = r#"{"instructions":["{env:INSERT}"],"model":"{env:UNRELATED}"}"#; + let config = owned.path().join("opencode.json"); + std::fs::write(&config, raw).unwrap(); + let input = format!("PRIVATE_PREFIX{{file:{}}}PRIVATE_SUFFIX", rules.display()); + let mut environment = + SourceEnvironment::read(vec!["INSERT".into(), "UNRELATED".into()], &[], &|name| { + (name == "INSERT").then(|| input.clone()) + }) + .unwrap(); + let loaded = || { + vec![LoadedSourceOccurrence { + source: SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + }, + view: std::sync::Arc::new(raw.into()), + }] + }; + assert!( + environment + .plan_sources(loaded(), owned.path(), owned.path()) + .is_err(), + "absent metadata must not stand in for a selected native auth source" + ); + environment.set_native_auth(BTreeMap::new()).unwrap(); + let plan = environment + .plan_sources(loaded(), owned.path(), owned.path()) + .unwrap(); + let frozen = environment.frozen_inputs(&plan).unwrap(); + assert_eq!(frozen.get("INSERT"), Some(input.as_str())); + assert!(frozen.get("UNRELATED").is_none()); + assert!(!plan[0].references.template.contains("PRIVATE_PREFIX")); + assert!(!plan[0] + .references + .unexpanded_source() + .contains("PRIVATE_PREFIX")); + } + #[test] + fn managed_provider_source_environment_maps_known_runtime_ca_to_its_typed_source() { + use super::super::sources::{LoadedSourceOccurrence, SourceOccurrence}; + let owned = tempfile::tempdir().unwrap(); + let grant = owned.path().join("environment.env"); + let auth = owned.path().join("auth.json"); + let ca = owned.path().join("ca.pem"); + let config = owned.path().join("opencode.json"); + std::fs::write(&grant,"OPENAI_API_KEY=synthetic-key\nNODE_EXTRA_CA_CERTS=/home/freshell/provider/.config/onecli/gateway-ca.pem\n").unwrap(); + std::fs::write(&auth, "{}").unwrap(); + std::fs::write(&ca, "synthetic-ca-nonce").unwrap(); + for path in [&grant, &auth, &ca] { + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + let refs = [ + (grant, ProviderSecretProfile::OpencodeOnecliEnvironment), + (auth, ProviderSecretProfile::OpencodeOnecliAuthFile), + (ca.clone(), ProviderSecretProfile::OpencodeOnecliCaFile), + ] + .map(|(path, profile)| ProviderSecretReference { + source_path: path.to_str().unwrap().into(), + profile, + provider_relative_path: None, + }); + let environment = + SourceEnvironment::read(vec!["NODE_EXTRA_CA_CERTS".into()], &refs, &|_| None).unwrap(); + let raw = r#"{"instructions":["{file:{env:NODE_EXTRA_CA_CERTS}}","{file:/home/freshell/provider/.config/onecli/gateway-ca.pem}"]}"#; + std::fs::write(&config, raw).unwrap(); + let plan = environment + .plan_sources( + vec![LoadedSourceOccurrence { + source: SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + }, + view: std::sync::Arc::new(raw.into()), + }], + owned.path(), + owned.path(), + ) + .unwrap(); + assert_eq!(plan[0].references.selections.len(), 1); + assert_eq!(plan[0].references.selections[0].canonical_source, ca); + assert!(environment.frozen_inputs(&plan).unwrap().is_empty(),"typed child grant inputs must be refreshed from the host's actual read, not frozen as ambient paths"); + } + #[test] + fn managed_provider_source_environment_reads_typed_grants_and_keeps_auth_source_specific() { + let owned = tempfile::tempdir().unwrap(); + let grant = owned.path().join("environment.env"); + let auth = owned.path().join("auth.json"); + std::fs::write( + &grant, + "OPENAI_API_KEY=synthetic-key\nNODE_EXTRA_CA_CERTS=/synthetic/onecli-ca\n", + ) + .unwrap(); + std::fs::write(&auth,r#"{"first":{"type":"wellknown","key":"PATH_INPUT","token":"/synthetic/first"},"last":{"type":"wellknown","key":"PATH_INPUT","token":"/synthetic/typed-auth"}}"#).unwrap(); + for path in [&grant, &auth] { + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + let references = [ + ProviderSecretReference { + source_path: grant.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeOnecliEnvironment, + provider_relative_path: None, + }, + ProviderSecretReference { + source_path: auth.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeOnecliAuthFile, + provider_relative_path: None, + }, + ]; + let environment = SourceEnvironment::read( + vec!["NODE_EXTRA_CA_CERTS".into(), "PATH_INPUT".into()], + &references, + &|_| Some("/synthetic/ambient".into()), + ) + .unwrap(); + assert_eq!( + environment + .lookup(SourceLane::Main, "NODE_EXTRA_CA_CERTS") + .as_deref(), + Some("/synthetic/onecli-ca"), + "approved OneCLI child overlay must win over original ambient input" + ); + assert_eq!( + environment + .lookup(SourceLane::Main, "PATH_INPUT") + .as_deref(), + Some("/synthetic/typed-auth") + ); + assert_eq!( + environment.lookup(SourceLane::Tui, "PATH_INPUT").as_deref(), + Some("/synthetic/ambient"), + "TUI must not receive main loader's authEnv" + ); + assert!( + !environment.needs_native_auth(), + "selected typed auth grant has precedence over native fallback" + ); + std::fs::write(&grant, "NODE_EXTRA_CA_CERTS=/synthetic/changed\n").unwrap(); + assert_eq!( + environment + .lookup(SourceLane::Main, "NODE_EXTRA_CA_CERTS") + .as_deref(), + Some("/synthetic/onecli-ca"), + "one bounded read must remain the preparation's actual grant generation" + ); + } +} diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index b3bd2eb70..7c881627c 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -303,6 +303,21 @@ pub fn plan_loaded_opencode_sources( cwd: &Path, home: &Path, lookup: &dyn Fn(SourceLane, &Path, &str) -> Option, +) -> Result, super::ReferenceError> { + plan_loaded_opencode_sources_relocated( + sources, + cwd, + home, + lookup, + &std::collections::BTreeMap::new(), + ) +} +pub fn plan_loaded_opencode_sources_relocated( + sources: Vec, + cwd: &Path, + home: &Path, + lookup: &dyn Fn(SourceLane, &Path, &str) -> Option, + relocated_files: &std::collections::BTreeMap, ) -> Result, super::ReferenceError> { let original_names = source_environment_names(&sources)?; let mut result = Vec::new(); @@ -347,7 +362,7 @@ pub fn plan_loaded_opencode_sources( selected.insert(canonical.to_owned(), destination.clone()); Ok(destination) }; - let references = super::resolve_file_references_from_view( + let references = super::resolve_file_references_from_view_relocated( &loaded.view, Some(loaded.view.clone()), super::FileSourceOrigin { @@ -358,6 +373,7 @@ pub fn plan_loaded_opencode_sources( &|name| lookup(source.lane, &source.logical_path, name), recipe_count, &mut allocate, + relocated_files, )?; if references .recipes diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 280adc20e..25c870a42 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -175,7 +175,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { // server's resolved child environment wholesale (which can carry // AUTH_TOKEN, provider API keys, cloud credentials, proxy creds, ...). let env = managed_provider_env(&request.mode, &request.env); - let provider_secret_references = { + let mut provider_secret_references = { crate::managed_provider_bootstrap::named_provider_onecli_references(&request.mode)? }; let args = managed_provider_args(&request.mode, request.spec.args)?; @@ -207,6 +207,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { cwd: Some(&cwd), workspace: Some(&workspace), source_plan: None, + ..Default::default() } } else { crate::managed_mcp_capability::OpencodeEphemeralInput { @@ -215,25 +216,59 @@ impl ManagedTerminalController for ServerManagedRuntimeController { ..Default::default() } }; - let mcp_capability = matches!( - request.mode.as_str(), - "claude" | "codex" | "opencode" | "amplifier" - ) - .then(|| { - crate::managed_mcp_capability::issue_mcp_capability( + let (mcp_capability, mut provider_launch_context) = if request.mode == "opencode" { + let home = request + .env + .get("HOME") + .map(std::path::PathBuf::from) + .or_else(|| std::env::var_os("HOME").map(std::path::PathBuf::from)) + .ok_or("managed provider HOME unavailable")?; + let original = |name: &str| { + request + .env + .get(name) + .cloned() + .or_else(|| std::env::var(name).ok()) + }; + let prepared = crate::managed_provider_references::prepare_managed_opencode( + &self.client, &soul_id, - &request.mode, - opencode_input, - ) - }) - .transpose()?; - let mut provider_launch_context = - crate::managed_provider_bootstrap::provider_launch_context_for_managed_at( - &request.mode, - &workspace, + create_key, + &home, &cwd, - mcp_capability.clone(), - )?; + &workspace, + &original, + provider_secret_references, + None, + ) + .await?; + provider_secret_references = prepared.references; + ( + prepared.context.mcp_capability.clone(), + Some(prepared.context), + ) + } else { + let mcp_capability = matches!( + request.mode.as_str(), + "claude" | "codex" | "opencode" | "amplifier" + ) + .then(|| { + crate::managed_mcp_capability::issue_mcp_capability( + &soul_id, + &request.mode, + opencode_input, + ) + }) + .transpose()?; + let provider_launch_context = + crate::managed_provider_bootstrap::provider_launch_context_for_managed_at( + &request.mode, + &workspace, + &cwd, + mcp_capability.clone(), + )?; + (mcp_capability, provider_launch_context) + }; if let Some(context) = provider_launch_context.as_mut() { if let freshell_runtime_protocol::ProviderPreparation::Opencode { inline_config, diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 16de2cac1..7a13cedbc 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1,6 +1,7 @@ mod command_journal; mod control; mod output_journal; +mod provider_input_preparation; mod provider_secret_resolution; mod providers; mod pty; @@ -803,7 +804,7 @@ async fn grant_execution( execution_generation, grant_id, fixture, - provider_path_inputs: _provider_path_inputs, + provider_path_inputs, terminal, fresh_agent, resume_spec, @@ -844,6 +845,77 @@ async fn grant_execution( )); } + // Resolve every mounted grant once before consuming the execution grant. + // A changed source selection remains a pre-dispatch preparation failure. + let source = terminal + .as_ref() + .map(|launch| { + ( + launch.mode.as_str(), + launch.provider_secret_references.as_slice(), + launch.provider_launch_context.as_ref(), + ) + }) + .or_else(|| { + fresh_agent.as_ref().map(|launch| { + ( + launch.provider.as_str(), + launch.provider_secret_references.as_slice(), + launch.provider_launch_context.as_ref(), + ) + }) + }); + let mut prepared_secrets = if let Some((provider, references, context)) = source { + let mut secrets = provider_secret_resolution::resolve_child_secrets( + provider, + references, + Path::new("/run/freshell-secrets"), + ) + .map_err(|error| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + format!("resolve managed provider credentials: {error}"), + ) + })?; + if provider == "opencode" { + let generated = provider_input_preparation::prepare( + context, + &mut secrets, + &provider_path_inputs, + &state + .provider_config_paths + .private_mount + .join("user-provider"), + &state.provider_config_paths.provider_home, + ) + .map_err(|error| { + RuntimeError::new( + if error.starts_with("configuration reference selection changed") { + RuntimeErrorCode::ProviderInputsChanged + } else { + RuntimeErrorCode::HostUnreachable + }, + error, + ) + })?; + secrets.environment.extend(generated); + } else if !provider_path_inputs.is_empty() { + return Err(RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "private source input binding belongs to another provider", + )); + } + Some(secrets) + } else { + if !provider_path_inputs.is_empty() { + return Err(RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "private source input binding has no provider workload", + )); + } + None + }; + persisted.max_control_epoch = control_epoch; persisted.max_execution_generation = execution_generation; persisted.grant_id = Some(grant_id.clone()); @@ -928,15 +1000,10 @@ async fn grant_execution( ) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; } - let resolved_secrets = provider_secret_resolution::resolve_child_secrets( - &terminal.mode, - &terminal.provider_secret_references, - Path::new("/run/freshell-secrets"), - ) - .map_err(|error| { + let resolved_secrets = prepared_secrets.take().ok_or_else(|| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, - format!("resolve managed provider credentials: {error}"), + "prepared terminal credentials unavailable", ) })?; prepare_provider_files( @@ -1052,15 +1119,10 @@ async fn grant_execution( ) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; } - let mut resolved_secrets = provider_secret_resolution::resolve_child_secrets( - launch.provider.as_str(), - &launch.provider_secret_references, - Path::new("/run/freshell-secrets"), - ) - .map_err(|error| { + let mut resolved_secrets = prepared_secrets.take().ok_or_else(|| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, - format!("resolve hosted OneCLI grant: {error}"), + "prepared fresh-agent credentials unavailable", ) })?; prepare_provider_files( @@ -3900,6 +3962,38 @@ mod tests { assert_eq!(history.history, vec!["create", "resume"]); } + #[tokio::test] + async fn provider_secret_missing_prepared_source_does_not_consume_execution_grant() { + let state = test_host_state(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let terminal:TerminalLaunchSpec=serde_json::from_value(serde_json::json!({ + "terminalId":"synthetic-preparation","streamId":"synthetic-stream","mode":"opencode","program":"/usr/bin/true","args":[],"env":{}, + "cwd":"/tmp","workspacePath":"/tmp","runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"synthetic", + "providerLaunchContext":{"preparation":{"opencode":{"project_config":[],"inline_config":false}}, + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":0,"aliasCount":0,"recipeCount":0}}} + })).unwrap(); + let result = grant_execution( + &state, + GrantExecutionRequest { + soul_id: SoulId::new(), + control_epoch: 1, + execution_generation: 1, + grant_id: GrantId::new(), + fixture: None, + provider_path_inputs: Default::default(), + terminal: Some(terminal), + fresh_agent: None, + resume_spec: None, + }, + ) + .await; + assert!(result.is_err()); + let persisted = state.persisted.lock().await; + assert!(persisted.grant_id.is_none(),"source preflight failure must leave the grant unconsumed and safe to reprepare before dispatch"); + assert_eq!(persisted.worker_launch_count, 0); + } + #[tokio::test] async fn dispatch_rejects_wrong_role_before_any_host_mutation() { let state = test_host_state(); diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs new file mode 100644 index 000000000..414713d8f --- /dev/null +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -0,0 +1,268 @@ +//! Private prepared-source validation before an execution grant is consumed. +use crate::provider_secret_resolution::ResolvedProviderSecrets; +use freshell_runtime_protocol::{ProviderLaunchContext, ProviderPathInputs}; +use std::{collections::BTreeMap, path::Path}; + +fn read_private(root: &Path, relative: &str, limit: usize) -> Result, String> { + use std::io::Read; + if std::path::Path::new(relative) + .components() + .any(|component| !matches!(component, std::path::Component::Normal(_))) + { + return Err("prepared source has an invalid private relative path".into()); + } + let path = root.join(relative); + let metadata = + std::fs::symlink_metadata(&path).map_err(|_| "prepared private source unavailable")?; + if !metadata.is_file() + || metadata.file_type().is_symlink() + || std::fs::canonicalize(&path).map_err(|_| "prepared private source unavailable")? != path + { + return Err("prepared private source must be a physical stage file".into()); + } + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "prepared private source unreadable")? + .take(limit as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "prepared private source read failed")?; + if bytes.len() > limit { + return Err("prepared source-view or filesystem metadata profile exceeded".into()); + } + Ok(bytes) +} +pub fn prepare( + context: Option<&ProviderLaunchContext>, + secrets: &mut ResolvedProviderSecrets, + frozen: &ProviderPathInputs, + stage: &Path, + home: &Path, +) -> Result, String> { + use freshell_platform::opencode_recipe::{GeneratedFileRecipe, SourceLane, SourceManifest}; + use sha2::{Digest, Sha256}; + let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) else { + return if frozen.is_empty() { + Ok(BTreeMap::new()) + } else { + Err("private source inputs have no prepared generation".into()) + }; + }; + context + .ok_or("prepared provider context unavailable")? + .validate("opencode") + .map_err(|error| error.message)?; + let metadata = read_private(stage, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + if format!("{:x}", Sha256::digest(&metadata)) != inputs.manifest.sha256 { + return Err("prepared source manifest changed before dispatch".into()); + } + let manifest: SourceManifest = serde_json::from_slice(&metadata) + .map_err(|_| "prepared source manifest metadata invalid")?; + if manifest.version != 1 + || manifest.sources.len() > 8192 + || manifest.files.len() > 128 + || manifest.frozen_inputs.len() > 128 + { + return Err("prepared source metadata profile exceeded".into()); + } + if frozen.iter().any(|(name, _)| { + !manifest + .frozen_inputs + .iter() + .any(|approved| approved == name) + }) || manifest + .frozen_inputs + .iter() + .any(|name| frozen.get(name).is_none()) + { + return Err("configuration reference selection changed; frozen input binding is unavailable or changed; reprepare before dispatch".into()); + } + let names = manifest + .sources + .iter() + .flat_map(|source| source.referenced_variables.iter().cloned()) + .collect::>() + .into_iter() + .collect::>(); + freshell_platform::native_auth_exchange::validate_keys(&names).map_err(str::to_string)?; + let auth = if let Some((_, raw)) = secrets + .provider_files + .iter() + .find(|(relative, _)| *relative == ".local/share/opencode/auth.json") + { + let raw = + std::str::from_utf8(raw).map_err(|_| "selected typed auth source is not UTF-8")?; + freshell_platform::provider_environment_grants::resolve_opencode_auth_environment( + raw, &names, + ) + .map_err(|error| error.to_string())? + } else { + freshell_platform::native_auth_exchange::read_selected_native_auth(home, &names) + .map_err(str::to_string)? + }; + let mut generated = BTreeMap::new(); + let mut views = 0usize; + let mut budget = frozen.clone(); + for source in manifest.sources { + let remaining = (64 * 1024 * 1024usize) + .checked_sub(views) + .ok_or("source view profile overflow")?; + let raw = read_private( + stage, + &source.unexpanded_relative_path, + remaining.min(16 * 1024 * 1024), + )?; + views = views + .checked_add(raw.len()) + .ok_or("source view profile overflow")?; + let raw = String::from_utf8(raw).map_err(|_| "prepared source is not UTF-8")?; + let recipe = GeneratedFileRecipe { + generated_name: source.generated_name.clone().unwrap_or_default(), + unexpanded_source: raw, + declaring_config: source.logical_path, + cwd: manifest.cwd.clone(), + original_home: manifest.original_home.clone(), + occurrences: source.occurrences, + }; + let lookup = |name: &str| { + if source.lane == SourceLane::Main { + auth.get(name) + .map(String::as_str) + .or_else(|| secrets.environment.get(name).map(String::as_str)) + .or_else(|| frozen.get(name)) + .map(str::to_string) + } else { + secrets + .environment + .get(name) + .map(String::as_str) + .or_else(|| frozen.get(name)) + .map(str::to_string) + } + }; + let value = freshell_platform::opencode_recipe::evaluate(&recipe, &lookup) + .map_err(str::to_string)?; + if let Some(name) = source.generated_name { + if secrets.environment.contains_key(&name) + || auth.contains_key(&name) + || frozen.get(&name).is_some() + || generated.contains_key(&name) + { + return Err( + "generated source input collides with a child credential or source input" + .into(), + ); + } + budget + .insert(name.clone(), value.clone()) + .map_err(|error| error.message)?; + generated.insert(name, value); + } + if source.original_form { + for name in source.path_variables { + if let Some(value) = frozen.get(&name) { + generated.entry(name).or_insert_with(|| value.into()); + } + } + } + } + Ok(generated) +} + +#[cfg(test)] +mod tests { + use super::*; + use freshell_platform::opencode_recipe::*; + use freshell_runtime_protocol::*; + use sha2::{Digest, Sha256}; + #[test] + fn provider_secret_prepared_recipe_consumes_private_frozen_inputs_without_original_tree() { + let owned = tempfile::tempdir().unwrap(); + let stage = owned.path().join("stage"); + let home = owned.path().join("home"); + std::fs::create_dir_all(stage.join(".freshell/source-views")).unwrap(); + std::fs::create_dir(&home).unwrap(); + let raw = r#"{"instructions":["{env:INSERT}"]}"#; + std::fs::write(stage.join(".freshell/source-views/source-0"), raw).unwrap(); + let manifest = SourceManifest { + version: 1, + cwd: "/original-unavailable/project".into(), + original_home: "/original-unavailable/home".into(), + sources: vec![SourceView { + logical_path: "/original-unavailable/project/opencode.json".into(), + lane: SourceLane::Main, + unexpanded_relative_path: ".freshell/source-views/source-0".into(), + template_relative_path: ".freshell/templates/source-0".into(), + generated_name: Some("FRESHELL_CONFIG_REFERENCE_0".into()), + occurrences: vec![ApprovedFileOccurrence { + ordinal: 0, + approved_alias: "/original-unavailable/payload/rules".into(), + runtime_destination: "/home/freshell/provider/.freshell/references/0".into(), + }], + referenced_variables: vec!["INSERT".into()], + path_variables: vec!["INSERT".into()], + original_form: false, + }], + frozen_inputs: vec!["INSERT".into()], + files: vec![SelectedSourceFile { + source_id: 0, + canonical_source: "/original-unavailable/payload/rules".into(), + }], + }; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 1, + alias_count: 0, + recipe_count: 1, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: Some(IndexedProviderFiles { + volume_name: indexed_provider_volume_name(&SoulId::new(), &capability.grant_id), + aliases: vec![], + }), + }), + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + mcp_capability: Some(capability), + config: vec![], + }; + let frozen = ProviderPathInputs::new(BTreeMap::from([( + "INSERT".into(), + "PRIVATE_PREFIX{file:/original-unavailable/payload/rules}PRIVATE_SUFFIX".into(), + )])) + .unwrap(); + let mut secrets = ResolvedProviderSecrets::default(); + let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap(); + let value=generated.get("FRESHELL_CONFIG_REFERENCE_0").expect("host must generate the selected native child source from private path inputs before dispatch"); + let expected = r#"{"instructions":["PRIVATE_PREFIX{file:/home/freshell/provider/.freshell/references/0}PRIVATE_SUFFIX"]}"#; + assert_eq!(value, expected); + assert!(!generated.contains_key("INSERT"),"independent generated recipes do not forward an original ambient variable into the native child"); + let changed = ProviderPathInputs::new(BTreeMap::from([( + "INSERT".into(), + "{file:/different-unapproved/rules}".into(), + )])) + .unwrap(); + assert!(prepare(Some(&context),&mut secrets,&changed,&stage,&home).is_err(),"changed lexical selection must fail without canonicalizing or reading an unavailable original tree"); + assert_eq!( + std::fs::read_to_string(stage.join(".freshell/source-views/source-0")).unwrap(), + raw + ); + } +} From 3fb46bd46202e00d5e130022e1e143044be65759 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 14:03:51 -0700 Subject: [PATCH 043/493] Precreate owned provider mounts and preserve prepared source authority --- crates/freshell-platform/Cargo.toml | 2 +- crates/freshell-platform/src/lib.rs | 1 + .../freshell-platform/src/opencode_config.rs | 19 + .../freshell-platform/src/opencode_recipe.rs | 2 + .../src/provider_mountpoints.rs | 324 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 10 + .../src/managed_provider_references.rs | 23 +- crates/freshell-session-host/src/main.rs | 222 +++++++++++- .../src/provider_input_preparation.rs | 14 + .../src/provider_secret_resolution.rs | 1 + .../src/providers/mod.rs | 55 ++- crates/freshell-supervisor/src/backend.rs | 7 + .../src/backend/input_publisher.rs | 1 + .../src/backend/native_auth.rs | 1 + .../src/backend/provider_mountpoints.rs | 251 ++++++++++++++ crates/freshell-supervisor/src/registry.rs | 23 ++ crates/freshell-supervisor/src/service.rs | 5 + scripts/sandbox-test.sh | 5 + .../managed-provider-mountpoints-sandbox.py | 108 ++++++ scripts/testing/runtime-test-broker.ts | 44 ++- .../testing/runtime-test-broker.test.ts | 28 ++ 21 files changed, 1104 insertions(+), 42 deletions(-) create mode 100644 crates/freshell-platform/src/provider_mountpoints.rs create mode 100644 crates/freshell-supervisor/src/backend/provider_mountpoints.rs create mode 100644 scripts/testing/managed-provider-mountpoints-sandbox.py diff --git a/crates/freshell-platform/Cargo.toml b/crates/freshell-platform/Cargo.toml index d1862a611..3b459ee5f 100644 --- a/crates/freshell-platform/Cargo.toml +++ b/crates/freshell-platform/Cargo.toml @@ -15,7 +15,7 @@ serde_json = { workspace = true } serde = { workspace = true } url = "2" -[target.'cfg(target_os = "linux")'.dependencies] +[target.'cfg(unix)'.dependencies] libc = "=0.2.186" [dev-dependencies] diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index d5c0b7011..e33f2cbe3 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -67,6 +67,7 @@ pub mod provider_environment; pub mod provider_environment_grants; pub mod provider_exec; pub mod provider_input_exchange; +pub mod provider_mountpoints; pub mod resume_gate; pub mod spawn; diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index 227d73f35..1268acd74 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -166,6 +166,25 @@ fn validate_mcp_field(value: &Value, path: &Path) -> Result<(), ConfigError> { Ok(()) } +/// Inspect the retained, accepted raw view without reading its source again. +pub fn source_owns_freshell_mcp( + raw: &str, + path: &Path, + project_dir: &Path, +) -> Result { + let value = + parse_jsonc_object(raw).ok_or_else(|| ConfigError::Malformed { path: path.into() })?; + validate_mcp_field(&value, path)?; + let entry = value + .get("mcp") + .and_then(|mcp| mcp.get("freshell")) + .filter(|entry| !entry.is_null()); + Ok(entry.is_some_and(|entry| { + path != project_dir.join(".opencode/opencode.json") + || !is_direct_owned_mcp_entry(project_dir, entry) + })) +} + /// Merge only Freshell-owned keys into the inline lane. The returned document /// is process-local and must never be serialized into a durable launch plan. pub fn merge_owned_entries( diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index 9ac068d8c..15d85bcbd 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -34,6 +34,8 @@ pub struct SourceManifest { pub files: Vec, #[serde(default)] pub frozen_inputs: Vec, + #[serde(default)] + pub user_owns_freshell_mcp: bool, } #[derive(Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] diff --git a/crates/freshell-platform/src/provider_mountpoints.rs b/crates/freshell-platform/src/provider_mountpoints.rs new file mode 100644 index 000000000..44cea904d --- /dev/null +++ b/crates/freshell-platform/src/provider_mountpoints.rs @@ -0,0 +1,324 @@ +//! Descriptor-owned placeholders in a verified soul volume, before mounting. +use serde::{Deserialize, Serialize}; +use std::path::Path; + +#[derive(Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct Mountpoint { + pub relative_path: String, + pub directory: bool, +} + +#[cfg(unix)] +pub fn prepare(home: &Path, selected: &[Mountpoint]) -> Result<(), &'static str> { + prepare_for_identity(home, selected, 65534, 0) +} +#[cfg(unix)] +fn prepare_for_identity( + home: &Path, + selected: &[Mountpoint], + uid: u32, + gid: u32, +) -> Result<(), &'static str> { + use std::{ + collections::BTreeMap, + os::unix::fs::{DirBuilderExt, MetadataExt, OpenOptionsExt}, + }; + let mut paths = BTreeMap::new(); + let mut bytes = 0usize; + for entry in selected { + let relative = Path::new(&entry.relative_path); + if relative.as_os_str().is_empty() + || relative.components().count() > 64 + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("provider mountpoint must be a bounded normal relative path"); + } + for parent in relative + .parent() + .into_iter() + .flat_map(Path::ancestors) + .filter(|path| !path.as_os_str().is_empty()) + { + if paths.insert(parent.to_owned(), true) == Some(false) { + return Err("provider file mountpoint masks a parent directory"); + } + } + if paths + .insert(relative.to_owned(), entry.directory) + .is_some_and(|prior| prior != entry.directory) + { + return Err("provider mountpoint kinds conflict"); + } + if paths.len() > 100_000 { + return Err("provider mountpoint entry profile exceeded"); + } + bytes = bytes + .checked_add(entry.relative_path.len()) + .ok_or("provider mountpoint metadata overflow")?; + if bytes > 64 * 1024 * 1024 { + return Err("provider mountpoint metadata profile exceeded"); + } + } + let metadata = + std::fs::symlink_metadata(home).map_err(|_| "provider HOME mountpoint unavailable")?; + if !metadata.is_dir() + || metadata.file_type().is_symlink() + || std::fs::canonicalize(home).map_err(|_| "provider HOME mountpoint unavailable")? != home + { + return Err("provider HOME mountpoint must be physical"); + } + let initial = metadata.uid() == 0 && metadata.gid() == 0 && uid != 0; + if initial { + if std::fs::read_dir(home) + .map_err(|_| "initial provider HOME unreadable")? + .next() + .is_some() + { + return Err("initial provider HOME must be empty before ownership transfer"); + } + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(home, std::fs::Permissions::from_mode(0o700)) + .map_err(|_| "initial provider HOME mode setup failed")?; + } else if metadata.uid() != uid || metadata.gid() != gid { + return Err("provider HOME ownership changed"); + } + // Inspection precedes every mutation, so a conflicting retained leaf never + // causes a partially populated preparation or ownership adoption. + for (relative, directory) in &paths { + match std::fs::symlink_metadata(home.join(relative)) { + Ok(metadata) + if !metadata.file_type().is_symlink() + && metadata.uid() == uid + && metadata.gid() == gid + && if *directory { + metadata.is_dir() + } else { + metadata.is_file() + } => {} + Ok(_) => return Err("retained provider mountpoint kind or ownership changed"), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("retained provider mountpoint inspection failed"), + } + } + let mut created = Vec::new(); + let result = (|| { + for (relative, directory) in paths { + let path = home.join(relative); + if std::fs::symlink_metadata(&path).is_ok() { + continue; + } + if directory { + std::fs::DirBuilder::new() + .mode(0o700) + .create(&path) + .map_err(|_| "provider directory mountpoint creation failed")?; + } else { + std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&path) + .map_err(|_| "provider file mountpoint creation failed")?; + } + created.push(path); + } + for path in created + .iter() + .rev() + .chain(initial.then_some(home.to_owned()).as_ref()) + { + use std::os::unix::ffi::OsStrExt; + let name = std::ffi::CString::new(path.as_os_str().as_bytes()) + .map_err(|_| "provider mountpoint path invalid")?; + if unsafe { libc::lchown(name.as_ptr(), uid, gid) } != 0 { + return Err("provider mountpoint ownership transfer failed"); + } + } + Ok(()) + })(); + if result.is_err() { + for path in created.iter().rev() { + if std::fs::symlink_metadata(path).is_ok_and(|metadata| metadata.is_dir()) { + let _ = std::fs::remove_dir(path); + } else { + let _ = std::fs::remove_file(path); + } + } + } + result +} + +#[cfg(unix)] +pub fn serve(home: &Path, socket: &Path) -> Result<(), &'static str> { + use std::{ + io::{Read, Write}, + os::unix::{fs::MetadataExt, net::UnixStream}, + }; + let root = + std::fs::symlink_metadata(home).map_err(|_| "provider mountpoint HOME unavailable")?; + // The retained provider identity performs its own writes. The initial + // empty volume is the only root-owned inode this helper may transfer. + if root.uid() == 65534 && root.gid() == 0 { + if unsafe { libc::setgroups(0, std::ptr::null()) } != 0 + || unsafe { libc::setgid(0) } != 0 + || unsafe { libc::setuid(65534) } != 0 + { + return Err("provider mountpoint identity transition failed"); + } + } + let mut stream = + UnixStream::connect(socket).map_err(|_| "provider mountpoint exchange unavailable")?; + let timeout = Some(std::time::Duration::from_secs(20)); + stream + .set_read_timeout(timeout) + .map_err(|_| "provider mountpoint timeout setup failed")?; + stream + .set_write_timeout(timeout) + .map_err(|_| "provider mountpoint timeout setup failed")?; + let mut length = [0; 4]; + stream + .read_exact(&mut length) + .map_err(|_| "provider mountpoint request unavailable")?; + let length = u32::from_be_bytes(length) as usize; + if length > 1024 * 1024 { + return Err("provider mountpoint control metadata profile exceeded"); + } + let mut bytes = vec![0; length]; + stream + .read_exact(&mut bytes) + .map_err(|_| "provider mountpoint request incomplete")?; + let selected: Vec = + serde_json::from_slice(&bytes).map_err(|_| "provider mountpoint metadata invalid")?; + prepare(home, &selected)?; + stream + .write_all(b"ok") + .map_err(|_| "provider mountpoint acknowledgement failed") +} + +#[cfg(all(test, unix))] +mod tests { + use super::*; + use std::os::unix::fs::MetadataExt; + + #[test] + fn provider_mountpoints_create_physical_files_and_preserve_existing_inodes() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir(&home).unwrap(); + let selected = vec![ + Mountpoint { + relative_path: ".config/opencode/opencode.json".into(), + directory: false, + }, + Mountpoint { + relative_path: ".freshell/provider-roots/plugin-0".into(), + directory: true, + }, + ]; + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + prepare_for_identity(&home, &selected, uid, gid).unwrap(); + let file = home.join(&selected[0].relative_path); + let root = home.join(&selected[1].relative_path); + let metadata = std::fs::symlink_metadata(&file) + .expect("the file mountpoint must exist before Docker create"); + assert!(metadata.is_file()); + assert!(!metadata.file_type().is_symlink()); + assert_eq!((metadata.uid(), metadata.gid()), (uid, gid)); + assert!(std::fs::symlink_metadata(&root).unwrap().is_dir()); + std::fs::write(&file, "retained-owned-file").unwrap(); + let inode = metadata.ino(); + prepare_for_identity(&home, &selected, uid, gid).unwrap(); + assert_eq!(std::fs::metadata(&file).unwrap().ino(), inode); + assert_eq!( + std::fs::read_to_string(&file).unwrap(), + "retained-owned-file" + ); + } + + #[test] + fn provider_mountpoints_refuse_symlink_parents_before_creating_other_paths() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let outside = owned.path().join("outside"); + std::fs::create_dir(&home).unwrap(); + std::fs::create_dir(&outside).unwrap(); + std::fs::write(outside.join("kept"), "outside-original").unwrap(); + std::os::unix::fs::symlink(&outside, home.join("link")).unwrap(); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + let selected = [ + Mountpoint { + relative_path: "aaa/new-file".into(), + directory: false, + }, + Mountpoint { + relative_path: "link/kept".into(), + directory: false, + }, + ]; + assert!(prepare_for_identity(&home, &selected, uid, gid).is_err()); + assert!(!home.join("aaa").exists()); + assert_eq!( + std::fs::read_to_string(outside.join("kept")).unwrap(), + "outside-original" + ); + assert!(std::fs::symlink_metadata(home.join("link")) + .unwrap() + .file_type() + .is_symlink()); + for relative in ["../escape", "/outside", "", "a/../../escape"] { + assert!(prepare_for_identity( + &home, + &[Mountpoint { + relative_path: relative.into(), + directory: false + }], + uid, + gid + ) + .is_err()); + } + } + + #[test] + fn provider_mountpoints_refuse_file_directory_collisions_without_overwrite() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir(&home).unwrap(); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + std::fs::write(home.join("retained"), "kept").unwrap(); + assert!(prepare_for_identity( + &home, + &[Mountpoint { + relative_path: "retained/child".into(), + directory: false + }], + uid, + gid + ) + .is_err()); + assert_eq!( + std::fs::read_to_string(home.join("retained")).unwrap(), + "kept" + ); + assert!(prepare_for_identity( + &home, + &[ + Mountpoint { + relative_path: "file".into(), + directory: false + }, + Mountpoint { + relative_path: "file/child".into(), + directory: false + } + ], + uid, + gid + ) + .is_err()); + assert!(!home.join("file").exists()); + } +} diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index cbaa4733c..24e785297 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -18,12 +18,22 @@ pub(super) fn stage( sources: Vec::new(), files: Vec::new(), frozen_inputs: frozen_inputs.to_vec(), + user_owns_freshell_mcp: false, }; private_directory(&root.join(".freshell/source-views"))?; private_directory(&root.join(".freshell/templates"))?; let mut files = BTreeMap::new(); let mut view_bytes = 0usize; for (ordinal, source) in plan.iter().enumerate() { + if source.source.lane == SourceLane::Main { + manifest.user_owns_freshell_mcp |= + freshell_platform::opencode_config::source_owns_freshell_mcp( + source.references.unexpanded_source(), + &source.source.logical_path, + cwd, + ) + .map_err(|error| error.to_string())?; + } view_bytes = view_bytes .checked_add(source.references.unexpanded_source().len()) .and_then(|value| value.checked_add(source.references.template.len())) diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index ee4754a72..5d630b87f 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -757,17 +757,26 @@ pub(crate) async fn prepare_managed_opencode( } let names = source_environment_names(&loaded).map_err(|error| error.to_string())?; let mut environment = environment::SourceEnvironment::read(names, &references, original)?; - let mut reservation = None; + // Even a typed auth grant needs explicit ownership of a newly absent soul + // volume before its descriptor-owned mountpoints can be constructed. + let reservation = if auth_authority.is_none() { + Some( + client + .reserve_provider_inputs(soul.clone(), "opencode".into(), creation_seed.into()) + .await + .map_err(|error| error.to_string())? + .reservation_id, + ) + } else { + None + }; if environment.needs_native_auth() { let authority = if let Some(authority) = auth_authority { authority.to_owned() } else { - let receipt = client - .reserve_provider_inputs(soul.clone(), "opencode".into(), creation_seed.into()) - .await - .map_err(|error| error.to_string())?; - reservation = Some(receipt.reservation_id.clone()); - receipt.reservation_id + reservation + .clone() + .ok_or("initial provider input reservation unavailable")? }; let values = client .native_auth_lookup(soul.clone(), authority, environment.names().to_vec()) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 7a13cedbc..125dffe2c 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -137,6 +137,15 @@ async fn run() -> Result<(), String> { Path::new("/run/freshell-input-exchange/reply.sock"), ).map_err(str::to_owned) } + Some("prepare-provider-mountpoints") if args.len() == 2 => { + if unsafe { libc::geteuid() } != 0 || unsafe { libc::getegid() } != 0 { + return Err("provider mountpoint helper requires the fixed host identity".into()); + } + freshell_platform::provider_mountpoints::serve( + Path::new("/home/freshell/provider"), + Path::new("/run/freshell-input-exchange/reply.sock"), + ).map_err(str::to_owned) + } Some("native-history-only") => { let snapshot = freshell_freshagent::native_history::read( &required_arg(&args[2..], "--provider")?, @@ -1024,14 +1033,18 @@ async fn grant_execution( format!("prepare OneCLI provider files: {error}"), ) })?; - let prepared = providers::prepare_terminal(terminal, &resolved_secrets.environment) - .await - .map_err(|error| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - format!("prepare managed provider: {error}"), - ) - })?; + let prepared = providers::prepare_terminal( + terminal, + &resolved_secrets.environment, + resolved_secrets.opencode_plan.as_ref(), + ) + .await + .map_err(|error| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + format!("prepare managed provider: {error}"), + ) + })?; let terminal = prepared.terminal; let hosted = HostedPty::spawn( &state.state_dir, @@ -1146,6 +1159,7 @@ async fn grant_execution( providers::prepare_fresh_agent_child_environment( &launch, &mut resolved_secrets.environment, + resolved_secrets.opencode_plan.as_ref(), ) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; freshell_platform::managed_child_secrets::install(resolved_secrets.environment) @@ -1649,7 +1663,17 @@ fn copy_provider_config_references_with_ownership( } }) .collect::>(); - let excluded = provider_credential_sources(user_provider_root); + let protected = private_bound_destinations(context); + let mut excluded = provider_credential_sources(user_provider_root); + for selection in &selections { + for relative in &protected { + if let Ok(suffix) = Path::new(relative).strip_prefix(&selection.destination_relative) { + if let Ok(source) = std::fs::canonicalize(selection.source.join(suffix)) { + excluded.push(source); + } + } + } + } let report = copy_environment_with_events( "session_host", provider, @@ -1691,6 +1715,35 @@ fn provider_credential_sources(root: &Path) -> Vec { .collect() } +fn private_bound_destinations( + context: &freshell_runtime_protocol::ProviderLaunchContext, +) -> std::collections::BTreeSet { + let home = Path::new("/home/freshell/provider"); + context + .prepared_inputs + .iter() + .flat_map(|inputs| { + inputs + .config_binds + .iter() + .map(|bind| bind.effective_destination.as_str()) + .chain(inputs.indexed_files.iter().flat_map(|files| { + files + .aliases + .iter() + .map(|alias| alias.effective_destination.as_str()) + })) + }) + .filter_map(|destination| { + Path::new(destination) + .strip_prefix(home) + .ok() + .and_then(Path::to_str) + .map(str::to_owned) + }) + .collect() +} + const PROVIDER_PROJECTION_MANIFEST: &str = ".freshell-config-projection.json"; fn write_projection_manifest( @@ -1804,6 +1857,12 @@ fn refresh_provider_config_references( serde_json::to_vec(provider).map_err(|error| error.to_string())?, ) .map_err(|error| error.to_string())?; + std::fs::write( + stage.path().join("private-mountpoints.json"), + serde_json::to_vec(&private_bound_destinations(context)) + .map_err(|_| "private mountpoint metadata invalid")?, + ) + .map_err(|_| "private mountpoint metadata write failed")?; allow_provider_read_stage(stage.path(), run_as_gid)?; if run_as_uid == unsafe { libc::geteuid() } && run_as_gid == unsafe { libc::getegid() } { return apply_staged_provider_config(stage.path(), provider_home); @@ -1886,6 +1945,22 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() let projection = stage.join("projection"); let previous = read_projection_manifest(provider_home)?; let current = read_projection_manifest(&projection)?; + let protected: std::collections::BTreeSet = + match std::fs::read(stage.join("private-mountpoints.json")) { + Ok(bytes) => { + serde_json::from_slice(&bytes).map_err(|_| "private mountpoint metadata invalid")? + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Default::default(), + Err(_) => return Err("private mountpoint metadata unavailable".into()), + }; + if protected.iter().any(|relative| { + Path::new(relative).as_os_str().is_empty() + || Path::new(relative) + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + }) { + return Err("private mountpoint metadata has an invalid relative path".into()); + } let mut paths = Vec::with_capacity(destinations.len()); for relative in destinations .iter() @@ -1959,6 +2034,9 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() let mut stale = previous.difference(¤t).collect::>(); stale.sort_by_key(|path| std::cmp::Reverse(Path::new(path).components().count())); for relative in stale { + if protected.contains(relative) { + continue; + } let destination = provider_home.join(relative); if let Ok(metadata) = std::fs::symlink_metadata(&destination) { if metadata.is_dir() { @@ -2949,6 +3027,13 @@ fn now_millis() -> i64 { #[cfg(test)] mod tests { + fn executable_fixture() -> tempfile::TempDir { + match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(directory) => tempfile::tempdir_in(directory), + None => tempfile::tempdir(), + } + .expect("owned executable fixture directory") + } #[test] fn provider_native_auth_survives_without_matching_typed_replacement() { use super::*; @@ -3056,7 +3141,7 @@ mod tests { #[test] fn provider_config_two_stage_script_is_runnable() { use std::{os::unix::fs::PermissionsExt, process::Command}; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture(); let source = root.path().join("source"); std::fs::create_dir(&source).unwrap(); std::fs::write(source.join("run"), "#!/bin/sh\nprintf resource-nonce\n").unwrap(); @@ -3073,6 +3158,113 @@ mod tests { assert_eq!(output.stdout, b"resource-nonce"); } + #[test] + fn provider_config_directory_copy_and_refresh_leave_private_bound_leaf_untouched() { + use freshell_runtime_protocol::*; + use std::os::unix::fs::MetadataExt; + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("stage"); + let home = owned.path().join("home"); + std::fs::create_dir_all(source.join("nested")).unwrap(); + std::fs::create_dir_all(home.join(".config/opencode/nested")).unwrap(); + std::fs::write( + source.join("nested/opencode.json"), + "ordinary-copy-must-exclude-this", + ) + .unwrap(); + std::fs::write(source.join("nested/resource"), "ordinary-resource").unwrap(); + let leaf = home.join(".config/opencode/nested/opencode.json"); + std::fs::write(&leaf, "private-bound-leaf").unwrap(); + let inode = std::fs::metadata(&leaf).unwrap().ino(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![ProviderConfigBind { + source_relative_path: ".freshell/templates/source-0".into(), + logical_destination: + "/home/freshell/provider/.config/opencode/nested/opencode.json".into(), + effective_destination: + "/home/freshell/provider/.config/opencode/nested/opencode.json".into(), + }], + resource_mounts: vec![], + indexed_files: None, + }), + mcp_capability: Some(capability), + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + config: vec![ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "nested".into(), + provider_relative_path: ".config/opencode/nested".into(), + format: "directory".into(), + }], + }; + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + copy_provider_config_references( + &context, + "opencode", + owned.path(), + &source, + &home, + uid, + gid, + ) + .unwrap(); + assert_eq!( + std::fs::read_to_string(&leaf).unwrap(), + "private-bound-leaf", + "ordinary directory copy must exclude private bound files" + ); + assert_eq!(std::fs::metadata(&leaf).unwrap().ino(), inode); + assert_eq!( + std::fs::read_to_string(home.join(".config/opencode/nested/resource")).unwrap(), + "ordinary-resource" + ); + let protected = ".config/opencode/nested/opencode.json"; + let mut previous = read_projection_manifest(&home).unwrap(); + previous.insert(protected.into()); + write_projection_manifest(&home, &previous).unwrap(); + std::fs::write(source.join("nested/resource"), "new-resource").unwrap(); + refresh_provider_config_references( + &context, + "opencode", + owned.path(), + &source, + &home, + uid, + gid, + ) + .unwrap(); + assert_eq!(std::fs::metadata(&leaf).unwrap().ino(), inode); + assert_eq!( + std::fs::read_to_string(&leaf).unwrap(), + "private-bound-leaf" + ); + assert!(!read_projection_manifest(&home).unwrap().contains(protected)); + assert_eq!( + std::fs::read_to_string(home.join(".config/opencode/nested/resource")).unwrap(), + "new-resource" + ); + } + #[test] fn provider_config_copy_preserves_package_resources_and_rejects_selected_credentials() { use freshell_runtime_protocol::{ @@ -3392,7 +3584,7 @@ mod tests { ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, }; use std::os::unix::fs::{symlink, MetadataExt, PermissionsExt}; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture(); let original = root.path().join("original"); let external = root.path().join("external-skills"); let discovered = original.join(".agents/skills"); @@ -4088,11 +4280,7 @@ mod tests { // runtime-suite /tmp is deliberately non-executable. Its existing // cargo target volume is the owned executable-fixture filesystem. - let workspace = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { - Some(directory) => tempfile::tempdir_in(directory), - None => tempfile::tempdir(), - } - .unwrap(); + let workspace = executable_fixture(); std::fs::set_permissions(workspace.path(), std::fs::Permissions::from_mode(0o755)).unwrap(); let provider_home = workspace.path().join("provider-home"); let source_config = workspace.path().join("opencode.json"); @@ -4715,7 +4903,7 @@ mod tests { ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, }; use std::{cell::Cell, process::Command}; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture(); let home = root.path().join("provider"); let private = root.path().join("private"); let source = private.join("user-provider"); diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 414713d8f..892b8a74d 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -99,6 +99,10 @@ pub fn prepare( freshell_platform::native_auth_exchange::read_selected_native_auth(home, &names) .map_err(str::to_string)? }; + let plan = freshell_platform::opencode_config::OpencodeConfigPlan { + sources: vec![], + user_owns_freshell_mcp: manifest.user_owns_freshell_mcp, + }; let mut generated = BTreeMap::new(); let mut views = 0usize; let mut budget = frozen.clone(); @@ -165,6 +169,7 @@ pub fn prepare( } } } + secrets.opencode_plan = Some(plan); Ok(generated) } @@ -203,6 +208,7 @@ mod tests { original_form: false, }], frozen_inputs: vec!["INSERT".into()], + user_owns_freshell_mcp: true, files: vec![SelectedSourceFile { source_id: 0, canonical_source: "/original-unavailable/payload/rules".into(), @@ -250,6 +256,14 @@ mod tests { .unwrap(); let mut secrets = ResolvedProviderSecrets::default(); let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap(); + assert!(secrets.opencode_plan.is_some(), "the accepted source plan must reach provider preparation without rescanning mounted templates"); + assert!( + secrets + .opencode_plan + .as_ref() + .unwrap() + .user_owns_freshell_mcp + ); let value=generated.get("FRESHELL_CONFIG_REFERENCE_0").expect("host must generate the selected native child source from private path inputs before dispatch"); let expected = r#"{"instructions":["PRIVATE_PREFIX{file:/home/freshell/provider/.freshell/references/0}PRIVATE_SUFFIX"]}"#; assert_eq!(value, expected); diff --git a/crates/freshell-session-host/src/provider_secret_resolution.rs b/crates/freshell-session-host/src/provider_secret_resolution.rs index bc5ad8b4f..0db90b7c3 100644 --- a/crates/freshell-session-host/src/provider_secret_resolution.rs +++ b/crates/freshell-session-host/src/provider_secret_resolution.rs @@ -6,6 +6,7 @@ use std::path::Path; const MAX_KEYS_BYTES: u64 = 128 * 1024; #[derive(Default)] pub struct ResolvedProviderSecrets { + pub opencode_plan: Option, pub environment: BTreeMap, pub provider_files: Vec<(&'static str, Vec)>, pub configuration_files: Vec<(String, std::path::PathBuf)>, diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 13c94b79e..2a3379933 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -61,6 +61,7 @@ pub fn prepare_provider_state_before_bootstrap( pub async fn prepare_terminal( mut terminal: freshell_runtime_protocol::TerminalLaunchSpec, child_secret_env: &std::collections::BTreeMap, + prepared_opencode_plan: Option<&freshell_platform::opencode_config::OpencodeConfigPlan>, ) -> Result { let mut child_env = child_secret_env.clone(); if let Some(context) = terminal.provider_launch_context.as_ref() { @@ -144,11 +145,11 @@ pub async fn prepare_terminal( .provider_launch_context .as_ref() .is_some_and(|context| context.mcp_capability.is_some()); - let plan = freshell_platform::opencode_config::prepare_opencode_config( - std::path::Path::new(&terminal.cwd), - std::path::Path::new("/home/freshell/provider/.config/opencode"), - ) - .map_err(|error| error.to_string())?; + let plan = opencode_plan( + &terminal.cwd, + terminal.provider_launch_context.as_ref(), + prepared_opencode_plan, + )?; let inherited = terminal .env .remove("OPENCODE_CONFIG_CONTENT") @@ -266,6 +267,7 @@ fn prepare_provider_features_for_identity(uid: u32, gid: u32, args: &[&str]) -> pub fn prepare_fresh_agent_child_environment( launch: &freshell_runtime_protocol::FreshAgentLaunchSpec, child_env: &mut std::collections::BTreeMap, + prepared_opencode_plan: Option<&freshell_platform::opencode_config::OpencodeConfigPlan>, ) -> Result<(), String> { use freshell_runtime_protocol::{FreshProvider, ProviderPreparation}; if launch.provider != FreshProvider::Opencode { @@ -296,16 +298,24 @@ pub fn prepare_fresh_agent_child_environment( } } } - let plan = freshell_platform::opencode_config::prepare_opencode_config( - std::path::Path::new(&launch.cwd), - std::path::Path::new("/home/freshell/provider/.config/opencode"), - ) - .map_err(|error| error.to_string())?; - if let Some(inherited) = child_env.get("OPENCODE_CONFIG_CONTENT") { + let plan = opencode_plan( + &launch.cwd, + launch.provider_launch_context.as_ref(), + prepared_opencode_plan, + )?; + let has_mcp = launch + .provider_launch_context + .as_ref() + .is_some_and(|context| context.mcp_capability.is_some()); + if child_env.contains_key("OPENCODE_CONFIG_CONTENT") || has_mcp { let normalized = freshell_platform::opencode_config::merge_owned_entries( - Some(inherited), + child_env.get("OPENCODE_CONFIG_CONTENT").map(String::as_str), &plan, - &freshell_platform::opencode_config::OwnedConfigEdits::default(), + &freshell_platform::opencode_config::OwnedConfigEdits { + freshell_command: has_mcp + .then(|| vec!["node".into(), "/opt/freshell-mcp/server.js".into()]), + ..Default::default() + }, ) .map_err(|error| error.to_string())?; child_env.insert("OPENCODE_CONFIG_CONTENT".into(), normalized); @@ -330,6 +340,23 @@ pub fn prepare_fresh_agent_child_environment( Ok(()) } +fn opencode_plan( + cwd: &str, + context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, + prepared: Option<&freshell_platform::opencode_config::OpencodeConfigPlan>, +) -> Result { + if context.is_some_and(|context| context.prepared_inputs.is_some()) { + return prepared + .cloned() + .ok_or_else(|| "accepted OpenCode source ownership plan is unavailable".into()); + } + freshell_platform::opencode_config::prepare_opencode_config( + std::path::Path::new(cwd), + std::path::Path::new("/home/freshell/provider/.config/opencode"), + ) + .map_err(|error| error.to_string()) +} + pub(crate) fn run_provider_features_worker(args: &[String]) -> Result<(), String> { let provider_home = std::path::Path::new("/home/freshell/provider"); match args { @@ -560,7 +587,7 @@ mod provider_secret_context_tests { "OPENCODE_CONFIG_CONTENT".into(), "{\"model\":\"sentinel-private-inline\"}".into(), )]); - let prepared = prepare_terminal(terminal, &secrets).await.unwrap(); + let prepared = prepare_terminal(terminal, &secrets, None).await.unwrap(); let metadata = serde_json::to_string(&prepared.terminal).unwrap(); assert!( !metadata.contains("sentinel-private-inline"), diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index ce7e2237a..a58e204a1 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -3,6 +3,7 @@ mod input_publisher; mod native_auth; mod native_history; mod prepared_inputs; +mod provider_mountpoints; use crate::registry::OwnedRuntimeHandle; use async_trait::async_trait; @@ -41,6 +42,7 @@ pub enum BackendError { #[derive(Debug, Clone)] pub struct CreateRuntimeSpec { pub initial_input_reservation: Option, + pub prior_provider_owner: Option, pub installation_id: InstallationId, pub soul_id: SoulId, pub incarnation_id: IncarnationId, @@ -698,6 +700,10 @@ impl RuntimeBackend for DockerEngineBackend { let (daemon_id, rootless) = self.daemon_info().await?; self.ensure_reserved_initial_volume(spec, &daemon_id, rootless) .await?; + if let Some(mounts) = &terminal_mounts { + self.prepare_provider_mountpoints(spec, mounts, &binary, &daemon_id) + .await?; + } let memory_swap = spec .limits .memory_bytes @@ -1884,6 +1890,7 @@ mod tests { }) .collect(); let spec = CreateRuntimeSpec { + prior_provider_owner: None, initial_input_reservation: None, installation_id: InstallationId::parse("installation-test").unwrap(), soul_id: SoulId::parse("soul-test").unwrap(), diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index 46964f760..7b7f2eb7f 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -700,6 +700,7 @@ mod tests { })).unwrap(); terminal.validate().unwrap(); let spec = CreateRuntimeSpec { + prior_provider_owner: None, initial_input_reservation: None, installation_id: InstallationId::parse("installation-input").unwrap(), soul_id: soul, diff --git a/crates/freshell-supervisor/src/backend/native_auth.rs b/crates/freshell-supervisor/src/backend/native_auth.rs index 345df1807..01a91076b 100644 --- a/crates/freshell-supervisor/src/backend/native_auth.rs +++ b/crates/freshell-supervisor/src/backend/native_auth.rs @@ -609,6 +609,7 @@ mod tests { }); let backend = DockerEngineBackend::new(temp.path().join("docker.sock")); let spec = CreateRuntimeSpec { + prior_provider_owner: None, initial_input_reservation: Some(binding.clone()), installation_id: registry.installation_id().clone(), soul_id: soul, diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs new file mode 100644 index 000000000..247808cfd --- /dev/null +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -0,0 +1,251 @@ +//! Fixed helper for descriptor-owned placeholders in the exact soul volume. +use super::*; +use freshell_platform::provider_mountpoints::Mountpoint; +use std::time::Duration; +use tokio::net::UnixListener; + +fn fail(cause: &'static str) -> BackendError { + BackendError::Unavailable(cause.into()) +} + +fn selections(mounts: &docker::TerminalMounts) -> Result, BackendError> { + let home = Path::new("/home/freshell/provider"); + let mut selected = BTreeMap::::new(); + let mut add = |path: &Path, directory: bool| -> Result<(), BackendError> { + if let Ok(relative) = path.strip_prefix(home) { + if relative.as_os_str().is_empty() + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err(fail("provider mountpoint destination is invalid")); + } + let name = relative + .to_str() + .ok_or_else(|| fail("provider mountpoint is not UTF-8"))?; + if selected + .insert(name.into(), directory) + .is_some_and(|prior| prior != directory) + { + return Err(fail("provider mountpoint kinds conflict")); + } + } + Ok(()) + }; + for bind in &mounts.config_binds { + add(Path::new(&bind.reference.effective_destination), false)?; + } + for root in &mounts.resource_mounts { + let directory = root.kind == freshell_runtime_protocol::ProviderResourceKind::Directory; + add(&home.join(&root.provider_relative_path), directory)?; + add(Path::new(&root.effective_destination), directory)?; + } + if let Some(files) = &mounts.indexed_files { + for alias in &files.aliases { + add(Path::new(&alias.effective_destination), false)?; + } + } + Ok(selected + .into_iter() + .map(|(relative_path, directory)| Mountpoint { + relative_path, + directory, + }) + .collect()) +} + +impl DockerEngineBackend { + pub(super) async fn prepare_provider_mountpoints( + &self, + spec: &CreateRuntimeSpec, + mounts: &docker::TerminalMounts, + binary: &Path, + daemon: &DockerDaemonId, + ) -> Result<(), BackendError> { + let selected = selections(mounts)?; + if selected.is_empty() { + return Ok(()); + } + let request = freshell_runtime_protocol::bounded_json_bytes(&selected, 1024 * 1024) + .map_err(|_| fail("provider mountpoint metadata profile exceeded"))?; + if spec.initial_input_reservation.is_none() { + let prior = spec.prior_provider_owner.as_ref().ok_or_else(|| fail("provider mountpoints require a bound initial reservation or retained registry owner"))?; + if prior.soul_id() != &spec.soul_id + || prior.installation_id() != &spec.installation_id + || prior.daemon_id() != daemon + || prior.provider_volume_name() != spec.provider_volume_name + || prior.image_ref() != spec.image_ref + { + return Err(fail("provider mountpoint retained ownership changed")); + } + if !matches!( + self.inspect(prior).await?.state, + BackendRuntimeState::Created + | BackendRuntimeState::Exited + | BackendRuntimeState::Dead + | BackendRuntimeState::Missing + ) { + return Err(fail( + "provider mountpoints require a quiescent retained owner", + )); + } + } + let volume = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", spec.provider_volume_name), + None, + 64 * 1024, + ) + .await?; + let value: Value = serde_json::from_slice(&volume.body) + .map_err(|_| fail("provider mountpoint volume metadata invalid"))?; + if volume.status != 200 + || value["Name"] != spec.provider_volume_name + || value["Driver"] != "local" + || !(value["Options"].is_null() || value["Options"] == json!({})) + { + return Err(fail("provider mountpoint owned local volume unavailable")); + } + if self.daemon_id().await? != *daemon { + return Err(fail("provider mountpoint daemon changed")); + } + let directory = native_auth::ExchangeDirectory::create_input()?; + let socket = directory.0.join("reply.sock"); + let listener = UnixListener::bind(&socket) + .map_err(|_| fail("provider mountpoint exchange bind failed"))?; + directory.mode(&socket, 0o660)?; + let name = format!("freshell-mountpoints-{}", uuid::Uuid::new_v4()); + let body = json!({ + "Image":spec.image_ref,"User":"0:0","Tty":false, + "Labels":{"com.freshell.installation-id":spec.installation_id,"com.freshell.soul-id":spec.soul_id,"com.freshell.incarnation-id":spec.incarnation_id}, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["prepare-provider-mountpoints"], + "Env":self.private_helper_environment(&spec.image_ref,&[]).await?, + "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"CapAdd":["CHOWN","SETGID","SETUID"], + "SecurityOpt":["no-new-privileges"],"Memory":128*1024*1024,"MemorySwap":128*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, + "LogConfig":{"Type":"none","Config":{}}, + "Mounts":[{"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, + {"Type":"volume","Source":spec.provider_volume_name,"Target":"/home/freshell/provider","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}, + {"Type":"bind","Source":directory.0,"Target":"/run/freshell-input-exchange","ReadOnly":true}]} + }); + let mut attempted = false; + let operation = async { + attempted = true; + let created = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/create?name={name}"), + Some(&body), + 64 * 1024, + ) + .await?; + if created.status == 409 { + attempted = false; + } + if created.status != 201 { + return Err(fail("provider mountpoint helper creation failed")); + } + let created: Value = serde_json::from_slice(&created.body) + .map_err(|_| fail("provider mountpoint helper identity unavailable"))?; + let id = created["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) + .ok_or_else(|| fail("provider mountpoint helper identity invalid"))?; + let started = self + .request("POST", &format!("{DOCKER_API}/containers/{id}/start"), None) + .await?; + if started.status != 204 { + return Err(fail("provider mountpoint helper start failed")); + } + let exchange = async { + let (mut stream, _) = listener + .accept() + .await + .map_err(|_| fail("provider mountpoint exchange failed"))?; + stream + .write_all(&(request.len() as u32).to_be_bytes()) + .await + .map_err(|_| fail("provider mountpoint request failed"))?; + stream + .write_all(&request) + .await + .map_err(|_| fail("provider mountpoint request incomplete"))?; + let mut acknowledgement = [0; 2]; + stream + .read_exact(&mut acknowledgement) + .await + .map_err(|_| fail("provider mountpoint acknowledgement unavailable"))?; + if &acknowledgement != b"ok" { + return Err(fail("provider mountpoint acknowledgement invalid")); + } + Ok(()) + }; + let wait = async { + let response = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/wait?condition=not-running"), + None, + 64 * 1024, + ) + .await?; + let value: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("provider mountpoint exit metadata invalid"))?; + if response.status != 200 || value["StatusCode"] != 0 { + return Err(fail("provider mountpoint helper failed")); + } + Ok(()) + }; + tokio::try_join!(exchange, wait)?; + if self.daemon_id().await? != *daemon { + return Err(fail( + "provider mountpoint daemon changed before main create", + )); + } + Ok(()) + }; + let result = tokio::time::timeout(Duration::from_secs(30), operation) + .await + .unwrap_or_else(|_| Err(fail("provider mountpoint preparation timed out"))); + if attempted { + let cleanup = async { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 1024 * 1024, + ) + .await?; + if inspected.status == 404 { + return Ok(()); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("provider mountpoint cleanup metadata invalid"))?; + let id = actual["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) + .ok_or_else(|| fail("provider mountpoint cleanup identity invalid"))?; + if inspected.status != 200 || !input_publisher::owned_helper(&actual, &body) { + return Err(fail("provider mountpoint cleanup ownership changed")); + } + let removed = self + .request( + "DELETE", + &format!("{DOCKER_API}/containers/{id}?force=1"), + None, + ) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail("provider mountpoint helper cleanup failed")); + } + Ok(()) + }; + tokio::time::timeout(Duration::from_secs(5), cleanup) + .await + .map_err(|_| fail("provider mountpoint cleanup timed out"))??; + } + tracing::debug!(soul_id=%spec.soul_id,entries=selected.len(),succeeded=result.is_ok(),"runtime.provider_mountpoints_settled"); + result + } +} diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index a40d46cd3..2b3625fe2 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -683,6 +683,29 @@ impl Registry { .await } + pub(crate) async fn prior_provider_owner( + &self, + incarnation: IncarnationId, + ) -> Result, RegistryError> { + let installation = self.installation_id().clone(); + self.run_blocking(move |conn| { + let prior: Option = conn.query_row( + "SELECT prior_incarnation_id FROM incarnations WHERE incarnation_id=?1", + params![incarnation.as_str()], + |row| row.get(0), + )?; + prior + .map(|raw| { + let prior = IncarnationId::parse(raw).map_err(|_| { + RegistryError::Integrity("invalid prior provider owner".into()) + })?; + load_owned_handle(&conn, installation, &prior) + }) + .transpose() + }) + .await + } + pub async fn active_handle_for_soul( &self, soul_id: SoulId, diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index e4d931fea..21dc48e9c 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -886,6 +886,11 @@ impl Supervisor { let created = self .backend .create_stopped(&CreateRuntimeSpec { + prior_provider_owner: self + .registry + .prior_provider_owner(prepared.incarnation_id.clone()) + .await + .map_err(map_registry)?, initial_input_reservation: self .registry .bound_input_reservation(prepared.incarnation_id.clone()) diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 445b03275..9969a0e64 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -32,6 +32,11 @@ set -euo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" IMAGE_TAG="freshell-sandbox:latest" +if [ "${1:-}" = "--provider-mountpoints" ]; then + shift + exec python3 "${REPO_ROOT}/scripts/testing/managed-provider-mountpoints-sandbox.py" "$@" +fi + if [ "${1:-}" = "--provider-inputs" ]; then shift exec python3 "${REPO_ROOT}/scripts/testing/probe-provider-config-readonly-home.py" --indexed-inputs "$@" diff --git a/scripts/testing/managed-provider-mountpoints-sandbox.py b/scripts/testing/managed-provider-mountpoints-sandbox.py new file mode 100644 index 000000000..d88e27e87 --- /dev/null +++ b/scripts/testing/managed-provider-mountpoints-sandbox.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python3 +"""Closed owned-volume qualification of the production mountpoint helper.""" +import argparse +import hashlib +import json +from pathlib import Path +import socket +import struct +import subprocess +import tempfile +import uuid + +IMAGE = "sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" +LABEL = "com.freshell.mountpoint-probe" + +def docker(*args, check=True): + return subprocess.run(["docker", *args], capture_output=True, text=True, timeout=60, check=check) + +def main(): + parser=argparse.ArgumentParser(description=__doc__) + parser.add_argument("--binary", type=Path, required=True) + parser.add_argument("--receipt-dir", type=Path, required=True) + args=parser.parse_args() + binary=args.binary.resolve(strict=True) + expected=Path(__file__).resolve().parents[2]/"target/debug/freshell-session-host" + if binary != expected or not binary.is_file() or not args.receipt_dir.is_dir(): + raise RuntimeError("mountpoint fixture requires this worktree's compiled host and existing receipt directory") + if "name=rootless" not in json.loads(docker("info","--format","{{json .SecurityOptions}}").stdout): + raise RuntimeError("mountpoint fixture requires the qualified rootless daemon") + if docker("image","inspect","--format","{{.Id}}",IMAGE).stdout.strip()!=IMAGE: + raise RuntimeError("mountpoint fixture requires the exact cached provider image") + run="freshell-mountpoint-probe-"+uuid.uuid4().hex + volume=run+"-volume" + owned=[] + receipt={"image":IMAGE,"runId":run,"rootless":True,"binarySha256":hashlib.file_digest(binary.open("rb"),"sha256").hexdigest()} + def remove(name): + result=docker("container","inspect",name,check=False) + if result.returncode: + return + actual=json.loads(result.stdout)[0] + if actual["Config"].get("Labels",{}).get(LABEL)!=run: + raise RuntimeError("mountpoint cleanup ownership changed") + docker("rm","--force",actual["Id"]) + def create(name,*args): + owned.append(name) + docker("create","--name",name,"--label",f"{LABEL}={run}","--network","none","--read-only", + "--cap-drop","ALL","--security-opt","no-new-privileges:true","--pids-limit","32","--memory","128m",*args) + with tempfile.TemporaryDirectory(prefix=run) as temporary: + exchange=Path(temporary) + listener=socket.socket(socket.AF_UNIX,socket.SOCK_STREAM) + listener.settimeout(20) + listener.bind(str(exchange/"reply.sock"));listener.listen(1) + try: + setup=run+"-socket-mode" + create(setup,"--mount",f"type=bind,src={exchange},dst=/exchange","--entrypoint","/bin/sh",IMAGE, + "-ceu","chmod 710 /exchange; chmod 660 /exchange/reply.sock") + if docker("start","--attach",setup).returncode: + raise RuntimeError("mountpoint socket setup failed") + remove(setup) + docker("volume","create","--label",f"{LABEL}={run}","--driver","local",volume) + selected=[{"relativePath":".config/opencode/opencode.json","directory":False}, + {"relativePath":".freshell/provider-roots/plugin-0","directory":True}] + for attempt in range(2): + helper=run+f"-helper-{attempt}" + create(helper,"--user","0:0","--cap-add","CHOWN","--cap-add","SETGID","--cap-add","SETUID","--log-driver","none", + "--mount",f"type=bind,src={binary},dst=/runtime/freshell-session-host,readonly", + "--mount",f"type=volume,src={volume},dst=/home/freshell/provider,volume-nocopy", + "--mount",f"type=bind,src={exchange},dst=/run/freshell-input-exchange,readonly", + "--entrypoint","/runtime/freshell-session-host",IMAGE,"prepare-provider-mountpoints") + docker("start",helper) + stream,_=listener.accept();stream.settimeout(20) + with stream: + frame=json.dumps(selected,separators=(",",":")).encode() + stream.sendall(struct.pack(">I",len(frame))+frame) + if stream.recv(2)!=b"ok": + raise RuntimeError("mountpoint helper failed its bounded acknowledgement") + if docker("wait",helper).stdout.strip()!="0": + raise RuntimeError("mountpoint helper failed") + remove(helper) + consumer=run+f"-consumer-{attempt}" + probe="""const fs=require('fs');const h='/home/freshell/provider',f=h+'/.config/opencode/opencode.json',r=h+'/.freshell/provider-roots/plugin-0';const s=fs.lstatSync(f),d=fs.lstatSync(r),hs=fs.lstatSync(h);if(!s.isFile()||s.isSymbolicLink()||!d.isDirectory()||s.uid!==65534||d.uid!==65534||hs.uid!==65534)throw Error('placeholder ownership');const text=fs.readFileSync(f,'utf8');if(process.argv[1]==='0'){if(text)throw Error('initial file');fs.writeFileSync(f,'retained-owned-nonce')}else if(text!=='retained-owned-nonce')throw Error('retained overwrite');console.log(JSON.stringify({inode:s.ino,rootInode:d.ino,homeInode:hs.ino,uid:process.getuid(),gid:process.getgid(),capEff:fs.readFileSync('/proc/self/status','utf8').match(/^CapEff:\\s*(\\w+)/m)[1]}));""" + create(consumer,"--user","65534:0","--mount",f"type=volume,src={volume},dst=/home/freshell/provider,volume-nocopy", + "--entrypoint","/usr/local/bin/node",IMAGE,"-e",probe,str(attempt)) + actual=json.loads(docker("start","--attach",consumer).stdout) + if actual["uid"]!=65534 or actual["gid"]!=0 or actual["capEff"]!="0000000000000000": + raise RuntimeError("mountpoint consumer identity changed") + if attempt==0: + receipt["initial"]=actual + elif actual!=receipt["initial"]: + raise RuntimeError("retained mountpoint or HOME inode changed") + remove(consumer) + receipt["retainedInodesAndBytesPreserved"]=True + finally: + listener.close() + for name in reversed(owned): + remove(name) + result=docker("volume","inspect",volume,check=False) + if result.returncode==0: + actual=json.loads(result.stdout)[0] + if actual.get("Labels",{}).get(LABEL)!=run: + raise RuntimeError("mountpoint volume cleanup ownership changed") + docker("volume","rm",volume) + receipt["ownedCleanup"]=all(docker("container","inspect",name,check=False).returncode!=0 for name in owned) and docker("volume","inspect",volume,check=False).returncode!=0 + (args.receipt_dir/"task002-native-provider-mountpoints.json").write_text(json.dumps(receipt,sort_keys=True)+"\n") + print(json.dumps(receipt,sort_keys=True),flush=True) + +if __name__=="__main__": + main() diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index c326e564d..3ca09e3c0 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -349,6 +349,8 @@ export class RestrictedDockerBroker { } if ((method==='GET' && url===`${DOCKER_API_PREFIX}/containers/${helperTarget![1]}/json`) || (method==='POST' && exactId && url===`${DOCKER_API_PREFIX}/containers/${owned.containerId}/start`) + || (method==='POST' && exactId && owned.expected.Cmd?.[0]==='prepare-provider-mountpoints' + && url===`${DOCKER_API_PREFIX}/containers/${owned.containerId}/wait?condition=not-running`) || (method==='DELETE' && url===`${DOCKER_API_PREFIX}/containers/${helperTarget![1]}?force=1`)) { await this.forwardAndReply(request,response,body,{destructive:method!=='GET',containerId:owned.containerId,helperName:name}) return @@ -376,8 +378,9 @@ export class RestrictedDockerBroker { if (method === 'POST' && url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`)) { const name = new URL(url, 'http://docker').searchParams.get('name') ?? '' - if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name)) { - const input=this.validateInputPublisher(name,body) + const mountpointHelper=/^freshell-mountpoints-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) + if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name) || mountpointHelper) { + const input=mountpointHelper ? this.validateMountpointHelper(body) : this.validateInputPublisher(name,body) if (!input || this.inputPublishers.has(name)) { this.block(response,method,url,403,'indexed publisher must match an owned fixed preparation',false) return @@ -391,7 +394,7 @@ export class RestrictedDockerBroker { if (typeof parsed.Id!=='string' || !/^[a-f0-9]{64}$/.test(parsed.Id) || this.receiptIds().has(parsed.Id)) throw new Error('invalid indexed publisher identity') publisher.containerId=parsed.Id } - this.record({method,url,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name,containerId:publisher.containerId,providerVolumeName:input.indexedFiles!.volumeName}) + this.record({method,url,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name,containerId:publisher.containerId,providerVolumeName:mountpointHelper ? input.providerVolumeName : input.indexedFiles!.volumeName}) replyDocker(response,forwarded) } catch { response.statusCode=503 @@ -663,6 +666,41 @@ export class RestrictedDockerBroker { return input } + private validateMountpointHelper(body:Buffer):PreparedInputAuthorization | undefined { + let parsed:Record + try {parsed=JSON.parse(body.toString())} catch {return} + const input=[...this.preparedInputs.values()].find((input)=>parsed.Image===input.imageRef + && equalRecord(parsed.Labels,{'com.freshell.installation-id':input.installationId, + 'com.freshell.soul-id':input.soulId,'com.freshell.incarnation-id':input.incarnationId})) + if (!input || parsed.User!=='0:0' || parsed.Tty!==false + || JSON.stringify(parsed.Entrypoint)!==JSON.stringify(['/runtime/freshell-session-host']) + || JSON.stringify(parsed.Cmd)!==JSON.stringify(['prepare-provider-mountpoints']) + || !privateHelperEnvironment(parsed.Env,false)) return + const host=parsed.HostConfig ?? {} + if (Object.keys(parsed).some((key)=>!['Image','User','Tty','Labels','Entrypoint','Cmd','Env','HostConfig'].includes(key)) + || Object.keys(host).some((key)=>!['NetworkMode','ReadonlyRootfs','CapDrop','CapAdd','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','LogConfig','Mounts'].includes(key))) return + if (host.NetworkMode!=='none' || host.ReadonlyRootfs!==true + || JSON.stringify(host.CapDrop)!==JSON.stringify(['ALL']) + || JSON.stringify(host.CapAdd)!==JSON.stringify(['CHOWN','SETGID','SETUID']) + || JSON.stringify(host.SecurityOpt)!==JSON.stringify(['no-new-privileges']) + || host.Memory!==128*1024*1024 || host.MemorySwap!==128*1024*1024 + || host.NanoCpus!==500_000_000 || host.PidsLimit!==32 + || !equalRecord(host.LogConfig,{Type:'none',Config:host.LogConfig?.Config}) + || Object.keys(host.LogConfig?.Config ?? {}).length) return + const mounts=host.Mounts + if (!Array.isArray(mounts) || mounts.length!==3) return + const binary=mounts.find((mount)=>mount.Type==='bind' && mount.Source===input.hostBinaryPath + && mount.Target==='/runtime/freshell-session-host' && mount.ReadOnly===true) + const volume=mounts.find((mount)=>mount.Type==='volume' && mount.Source===input.providerVolumeName + && mount.Target==='/home/freshell/provider' && mount.ReadOnly===false && equalRecord(mount.VolumeOptions,{NoCopy:true})) + const exchange=mounts.find((mount)=>mount.Type==='bind' && mount.Target==='/run/freshell-input-exchange' && mount.ReadOnly===true + && typeof mount.Source==='string' && path.dirname(mount.Source)===os.tmpdir() + && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + if (!binary || !volume || !exchange || [binary,volume,exchange].some((mount)=>Object.keys(mount).some((key)=>!['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key))) + || binary.VolumeOptions!==undefined || exchange.VolumeOptions!==undefined) return + return input + } + private validateHistoryHelper(name: string, body: Buffer): BrokerReceipt | undefined { if (!/^freshell-history-[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/.test(name)) return let parsed: Record diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 025fb3b60..1754a3fde 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -332,6 +332,34 @@ function publisherBody(input:PreparedInputAuthorization):any { } describe('fixed indexed publisher broker',()=>{ + it('allows the fixed owned mountpoint helper before workload creation and refuses extra authority',async()=>{ + await withHttpBroker(async ({request,forwarded,broker:instance})=>{ + const input=publisherAuthorization();instance.authorizePreparedInputs(input); + const name='freshell-mountpoints-11111111-1111-4111-8111-111111111111'; + const body=publisherBody(input); + body.User='0:0';body.Cmd=['prepare-provider-mountpoints']; + body.Labels={'com.freshell.installation-id':input.installationId,'com.freshell.soul-id':input.soulId,'com.freshell.incarnation-id':input.incarnationId}; + body.HostConfig.CapAdd=['CHOWN','SETGID','SETUID']; + body.HostConfig.Mounts[1]={Type:'volume',Source:input.providerVolumeName,Target:'/home/freshell/provider',ReadOnly:false,VolumeOptions:{NoCopy:true}}; + expect((await request('POST',`/v1.47/containers/create?name=${name}`,body)).status).toBe(201); + const before=forwarded.length; + for (const mutate of [ + (value:any)=>value.HostConfig.CapAdd.push('DAC_OVERRIDE'), + (value:any)=>value.Cmd.push('/outside'), + (value:any)=>value.HostConfig.Mounts[1].Source='freshell-provider-ffffffffffffffffffffffff', + (value:any)=>value.HostConfig.Mounts[1].VolumeOptions.NoCopy=false, + (value:any)=>value.HostConfig.Mounts.push({Type:'bind',Source:'/operator-home',Target:'/outside',ReadOnly:false}), + (value:any)=>value.Labels['com.freshell.soul-id']='foreign', + ]) { + const changed=structuredClone(body);mutate(changed); + expect((await request('POST',`/v1.47/containers/create?name=${name.replace('111111111111','222222222222')}`,changed)).status).toBe(403); + expect(forwarded).toHaveLength(before); + } + expect((await request('POST',`/v1.47/containers/${helperId}/start`)).status).toBe(204); + expect((await request('POST',`/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200); + expect((await request('DELETE',`/v1.47/containers/${helperId}?force=1`)).status).toBe(204); + }); + }); it('preserves exact name ownership on lost publisher acknowledgement and refuses replacement-helper cleanup',async()=>{ await withHttpBroker(async({request,forwarded,broker:instance,loseHelperAck,replaceHelperInspection})=>{ const input=publisherAuthorization();instance.authorizePreparedInputs(input); From 2c4fe7a7c3a7d3531e3d1ece5979f9455efa8b62 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 15:13:37 -0700 Subject: [PATCH 044/493] Prove prepared OpenCode inputs through terminal and fresh native startup --- crates/freshell-freshagent/src/claude.rs | 2 +- .../src/provider_mountpoints.rs | 12 +- crates/freshell-server/src/main.rs | 2 + .../src/managed_provider_launch_tests.rs | 476 ++++++++++++++++++ crates/freshell-session-host/src/main.rs | 162 ++++-- .../src/backend/input_publisher.rs | 66 ++- .../src/backend/prepared_inputs.rs | 27 +- crates/freshell-supervisor/src/service.rs | 9 +- scripts/sandbox-test.sh | 5 + .../testing/managed-provider-launch-broker.ts | 70 +++ .../managed-provider-launch-sandbox.py | 222 ++++++++ scripts/testing/runtime-test-broker.ts | 59 ++- .../testing/runtime-test-broker.test.ts | 47 +- 13 files changed, 1111 insertions(+), 48 deletions(-) create mode 100644 crates/freshell-server/src/managed_provider_launch_tests.rs create mode 100644 scripts/testing/managed-provider-launch-broker.ts create mode 100644 scripts/testing/managed-provider-launch-sandbox.py diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index 45501ae96..249bfc6de 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -9580,7 +9580,7 @@ async fn spawn_sidecar() -> Result<(Child, ChildStdin, ChildStdout, String), Str } else { &[] }; - freshell_platform::provider_exec::check_command(cmd.as_std(), true, &additional) + freshell_platform::provider_exec::check_command(cmd.as_std(), true, additional) .map_err(|error| error.to_string())?; } let mut child = cmd.spawn().map_err(|e| { diff --git a/crates/freshell-platform/src/provider_mountpoints.rs b/crates/freshell-platform/src/provider_mountpoints.rs index 44cea904d..023807653 100644 --- a/crates/freshell-platform/src/provider_mountpoints.rs +++ b/crates/freshell-platform/src/provider_mountpoints.rs @@ -161,13 +161,13 @@ pub fn serve(home: &Path, socket: &Path) -> Result<(), &'static str> { std::fs::symlink_metadata(home).map_err(|_| "provider mountpoint HOME unavailable")?; // The retained provider identity performs its own writes. The initial // empty volume is the only root-owned inode this helper may transfer. - if root.uid() == 65534 && root.gid() == 0 { - if unsafe { libc::setgroups(0, std::ptr::null()) } != 0 + if root.uid() == 65534 + && root.gid() == 0 + && (unsafe { libc::setgroups(0, std::ptr::null()) } != 0 || unsafe { libc::setgid(0) } != 0 - || unsafe { libc::setuid(65534) } != 0 - { - return Err("provider mountpoint identity transition failed"); - } + || unsafe { libc::setuid(65534) } != 0) + { + return Err("provider mountpoint identity transition failed"); } let mut stream = UnixStream::connect(socket).map_err(|_| "provider mountpoint exchange unavailable")?; diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index 7bd5a96ae..e3521a2c8 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -43,6 +43,8 @@ mod managed_mcp_capability; mod managed_ports; #[cfg(feature = "managed-runtime-v1")] mod managed_provider_bootstrap; +#[cfg(all(test, unix, feature = "managed-provider-qualification"))] +mod managed_provider_launch_tests; #[cfg(feature = "managed-runtime-v1")] mod managed_provider_references; #[cfg(feature = "managed-runtime-v1")] diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs new file mode 100644 index 000000000..533e3b35b --- /dev/null +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -0,0 +1,476 @@ +//! Closed synthetic native startup qualification. The external sandbox owns +//! the broker, pinned image, shared scratch and every process/container cleanup. +use async_trait::async_trait; +use freshell_runtime_protocol::*; +use freshell_supervisor::{ + backend::{ + BackendCreated, BackendError, BackendInspection, CreateRuntimeSpec, DockerEngineBackend, + RuntimeBackend, + }, + registry::{OwnedInputReservation, OwnedRuntimeHandle}, +}; +use serde_json::{json, Value}; +use std::{ + collections::BTreeMap, + path::{Path, PathBuf}, + time::Duration, +}; + +/// Delegates every lifecycle operation to the actual Docker backend. The only +/// added operation delivers metadata to the closed fixture's trusted broker. +struct BrokerAuthorizedBackend { + real: DockerEngineBackend, + control: PathBuf, + handoff: PathBuf, +} +impl BrokerAuthorizedBackend { + async fn authorize(&self, spec: &CreateRuntimeSpec) -> Result<(), BackendError> { + let invalid = + || BackendError::InvalidConfig("closed native fixture metadata handoff invalid".into()); + let (context, references) = spec + .terminal + .as_ref() + .map(|workload| { + ( + &workload.provider_launch_context, + &workload.provider_secret_references, + ) + }) + .or_else(|| { + spec.fresh_agent.as_ref().map(|workload| { + ( + &workload.provider_launch_context, + &workload.provider_secret_references, + ) + }) + }) + .ok_or_else(invalid)?; + let context = context.as_ref().ok_or_else(invalid)?; + let inputs = context.prepared_inputs.as_ref().ok_or_else(invalid)?; + inputs + .validate_references(references) + .map_err(|_| invalid())?; + let indexed = references + .iter() + .enumerate() + .filter(|(_, reference)| reference.profile.is_configuration_file()) + .map(|(index, reference)| { + let source = reference + .provider_relative_path + .as_deref() + .and_then(|path| path.strip_prefix(".freshell/references/")) + .and_then(|id| id.parse::().ok()) + .ok_or_else(invalid)?; + Ok(json!({"grantIndex":index,"sourceId":source})) + }) + .collect::, BackendError>>()?; + let value = json!({ + "installationId":spec.installation_id,"soulId":spec.soul_id,"incarnationId":spec.incarnation_id, + "imageRef":spec.image_ref,"hostBinaryPath":spec.host_binary_path,"providerVolumeName":spec.provider_volume_name, + "stageRoot":self.control.join("provider-roots").join(&inputs.generation_id),"generationId":inputs.generation_id, + "initialAuthReservation":inputs.initial_auth_reservation, + "configurationBinds":inputs.config_binds.iter().map(|bind|json!({"sourceRelativePath":bind.source_relative_path,"effectiveDestination":bind.effective_destination})).collect::>(), + "resourceMounts":inputs.resource_mounts.iter().map(|mount|json!({"providerRelativePath":mount.provider_relative_path,"effectiveDestination":mount.effective_destination,"kind":mount.kind})).collect::>(), + "indexedFiles":inputs.indexed_files.as_ref().map(|files|json!({"volumeName":files.volume_name,"aliases":files.aliases.iter().map(|alias|json!({"sourceId":alias.source_id,"effectiveDestination":alias.effective_destination})).collect::>()})), + "indexedSecretSources":indexed, + }); + let metadata = + bounded_json_bytes(&value, MAX_CONTROL_FRAME_BYTES).map_err(|_| invalid())?; + let path = self.handoff.join(format!("{}.json", spec.incarnation_id)); + let temporary = path.with_extension("pending"); + std::fs::write(&temporary, metadata).map_err(|_| invalid())?; + std::fs::rename(&temporary, &path).map_err(|_| invalid())?; + let receipt = path.with_extension("accepted"); + tokio::time::timeout(Duration::from_secs(10), async { + while !receipt.is_file() { + tokio::time::sleep(Duration::from_millis(20)).await; + } + }) + .await + .map_err(|_| { + BackendError::Unavailable("closed native fixture broker authorization timed out".into()) + })?; + Ok(()) + } +} +#[async_trait] +impl RuntimeBackend for BrokerAuthorizedBackend { + async fn observe_initial_auth_volume( + &self, + reservation: &OwnedInputReservation, + ) -> Result { + self.real.observe_initial_auth_volume(reservation).await + } + async fn commit_prepared_inputs( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + self.real.commit_prepared_inputs(handle).await + } + async fn read_native_auth( + &self, + handle: &OwnedRuntimeHandle, + keys: &[String], + binary: &Path, + ) -> Result, BackendError> { + self.real.read_native_auth(handle, keys, binary).await + } + async fn read_native_history( + &self, + handle: &OwnedRuntimeHandle, + provider: &str, + id: &str, + binary: &Path, + ) -> Result { + self.real + .read_native_history(handle, provider, id, binary) + .await + } + async fn create_stopped( + &self, + spec: &CreateRuntimeSpec, + ) -> Result { + self.authorize(spec).await?; + self.real.create_stopped(spec).await + } + async fn start_host(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { + self.real.start_host(handle).await + } + async fn inspect( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result { + self.real.inspect(handle).await + } + async fn request_stop( + &self, + handle: &OwnedRuntimeHandle, + grace: u64, + ) -> Result<(), BackendError> { + self.real.request_stop(handle, grace).await + } + async fn force_stop(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { + self.real.force_stop(handle).await + } + async fn verify_empty(&self, handle: &OwnedRuntimeHandle) -> Result { + self.real.verify_empty(handle).await + } + async fn list_known( + &self, + handles: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + self.real.list_known(handles).await + } + async fn read_limits( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result { + self.real.read_limits(handle).await + } + async fn enable_long_lived(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { + self.real.enable_long_lived(handle).await + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_prepared_inputs() { + use freshell_supervisor::{ + admission::AdmissionPolicy, + registry::Registry, + service::{serve_control, Supervisor, SupervisorConfig}, + }; + use freshell_terminal::registry::{ManagedTerminalController, ManagedTerminalLaunch}; + use freshell_ws::hosted_fresh_agent::{HostedFreshAgentCommand, HostedFreshAgentGateway}; + use sha2::{Digest, Sha256}; + use std::sync::Arc; + let fixture = PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_FIXTURE") + .expect("run the closed --provider-launches sandbox"), + ); + assert_eq!(fixture.parent(), Some(Path::new("/tmp"))); + assert!(fixture + .file_name() + .and_then(|name| name.to_str()) + .is_some_and(|name| name.starts_with("fln-") && name.len() == 20)); + let control = fixture.join("scenarios/native/control"); + let runtime = fixture.join("scenarios/n/r"); + for directory in [&control, &runtime, &fixture.join("handoff")] { + std::fs::create_dir_all(directory).unwrap(); + } + let binary = PathBuf::from(std::env::var_os("FRESHELL_PROVIDER_LAUNCH_HOST_BINARY").unwrap()); + let image = std::env::var("FRESHELL_PROVIDER_LAUNCH_IMAGE").unwrap(); + let installation = + InstallationId::parse(std::env::var("FRESHELL_PROVIDER_LAUNCH_INSTALLATION").unwrap()) + .unwrap(); + let registry = Registry::open(fixture.join("registry"), Some(installation)).unwrap(); + let backend = Arc::new(BrokerAuthorizedBackend { + real: DockerEngineBackend::new(fixture.join("broker.sock")), + control: control.clone(), + handoff: fixture.join("handoff"), + }); + let socket = control.join("control.sock"); + let secret_file = control.join("secret"); + std::fs::write(&secret_file, "closed-synthetic-control-secret").unwrap(); + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&secret_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + freshell_supervisor::backend::docker::configure_capability_dir(&socket).unwrap(); + let supervisor = Supervisor::new( + registry.clone(), + backend, + SupervisorConfig { + runtime_root: runtime, + control_socket_path: socket.clone(), + host_binary_path: binary, + image_ref: image, + test_run_id: std::env::var("FRESHELL_PROVIDER_LAUNCH_RUN_ID").unwrap(), + control_secret: "closed-synthetic-control-secret".into(), + lifecycle_log: fixture.join("lifecycle.jsonl"), + admission: AdmissionPolicy::default(), + }, + ) + .unwrap(); + let control_task = tokio::spawn({ + let socket = socket.clone(); + async move { + serve_control(supervisor, &socket).await.unwrap(); + } + }); + tokio::time::timeout(Duration::from_secs(5), async { + while !socket.exists() { + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .unwrap(); + let web = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap(); + let port = web.local_addr().unwrap().port(); + // Native well-known authentication fetches configuration from the outer + // auth-store key before reading local configuration. Keep that dependency + // inside the same owned synthetic fixture as the launch consumers. + let well_known_url = format!("http://host.docker.internal:{port}"); + std::fs::write( + fixture.join("grants/auth.json"), + serde_json::to_vec(&json!({well_known_url: { + "type": "wellknown", "key": "OPENAI_API_KEY", + "token": "AUTH_PREFIX{file:./rules.txt}AUTH_SUFFIX" + }})) + .unwrap(), + ) + .unwrap(); + let web_task = tokio::spawn(async move { + loop { + let Ok((mut stream, _)) = web.accept().await else { + break; + }; + tokio::spawn(async move { + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + let mut bytes = [0; 4096]; + let _ = stream.read(&mut bytes).await; + let _=stream.write_all(b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: 13\r\nConnection: close\r\n\r\n{\"config\":{}}").await; + }); + } + }); + crate::managed_mcp_capability::configure("0.0.0.0", port, &socket, &secret_file).unwrap(); + crate::managed_mcp_capability::spawn_callback_server() + .await + .unwrap(); + std::env::set_var("FRESHELL_MANAGED_RUNTIME_V1", "1"); + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SOCKET", &socket); + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SECRET_FILE", &secret_file); + std::env::set_var("FRESHELL_MANAGED_FRESH_AGENT_V1", "1"); + let controller = crate::managed_runtime::ServerManagedRuntimeController::from_env() + .await + .unwrap() + .unwrap(); + let cwd = fixture.join("workspace").to_string_lossy().into_owned(); + let original_config = fixture.join("source-home/.config/opencode/opencode.json"); + let original_bytes = std::fs::read(&original_config).unwrap(); + let request: ManagedTerminalLaunch = ManagedTerminalLaunch { + spec: freshell_platform::spawn::SpawnSpec { + program: "opencode".into(), + args: vec![], + env_overrides: BTreeMap::new(), + cwd: Some(cwd.clone()), + cols: 120, + rows: 30, + }, + env: BTreeMap::new(), + terminal_id: "closed-native-terminal".into(), + stream_id: "closed-native-stream".into(), + mode: "opencode".into(), + resume_session_id: None, + provider_model: None, + provider_reasoning_effort: None, + provider_sandbox: None, + provider_permission_mode: None, + view_tab_id: None, + view_pane_id: None, + create_request_id: Some("closed-native-terminal-create".into()), + }; + let terminal = controller + .launch(request) + .await + .expect("actual terminal controller/supervisor/host launch"); + let receipt = fixture.join("workspace/native-consumed.json"); + async fn read_receipt(path: &Path) -> Result { + tokio::time::timeout(Duration::from_secs(45), async { + loop { + if let Ok(bytes) = std::fs::read(path) { + if let Ok(value) = serde_json::from_slice(&bytes) { + return value; + } + } + tokio::time::sleep(Duration::from_millis(50)).await; + } + }) + .await + .map_err(|_| "actual pinned provider startup did not consume selected config and closure") + } + let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXnative-ruleAUTH_SUFFIX")), + "description":format!("{:x}",Sha256::digest(b"native-second-rule")), + "references":(["third","fourth","fifth","sixth"].map(|ordinal|format!("{:x}",Sha256::digest(format!("native-{ordinal}-rule").as_bytes())))), + "resource":format!("{:x}",Sha256::digest(b"plugin-relative-resource")), + "dependency":format!("{:x}",Sha256::digest(b"plugin-actual-local-dependency")),"sdk":format!("{:x}",Sha256::digest(b"function"))}); + let client = controller.runtime_client(); + let terminal_receipt = match read_receipt(&receipt).await { + Ok(value) => value, + Err(error) => { + let output = client + .read_output( + SoulId::parse(terminal.soul_id.clone()).unwrap(), + 0, + 64 * 1024, + ) + .await + .unwrap(); + let text = output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(); + let classes = [ + "ConfigInvalidError", + "ConfigJsonError", + "JSON parsing", + "SyntaxError", + "EACCES", + "ENOENT", + "Failed to import", + "plugin", + "opencode.json", + "bun install", + "ModuleNotFound", + ] + .into_iter() + .filter(|class| text.contains(class)) + .collect::>(); + println!( + "provider_launch_diagnostic {}", + json!({"route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes,"containsAuthSentinel":text.contains("AUTH_PREFIX"),"containsGeneratedName":text.contains("FRESHELL_CONFIG_REFERENCE_0")}) + ); + panic!("{error}"); + } + }; + assert_eq!(terminal_receipt, expected); + assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + println!( + "provider_launch_consumer {}", + json!({"provider":"opencode","route":"terminal","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) + ); + client + .stop(SoulId::parse(terminal.soul_id).unwrap()) + .await + .unwrap(); + std::fs::remove_file(&receipt).unwrap(); + let (broadcast, _) = tokio::sync::broadcast::channel(256); + let mut events = broadcast.subscribe(); + let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( + Some(client.clone()), + Arc::new(broadcast), + ) + .unwrap() + .unwrap(); + let message:freshell_protocol::FreshAgentCreate=serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":"freshopencode","provider":"opencode","cwd":cwd,"model":"closed-fixture/unavailable"})).unwrap(); + Arc::clone(&proxy) + .dispatch(HostedFreshAgentCommand::Create(message)) + .await; + let created = tokio::time::timeout(Duration::from_secs(10), async { + loop { + let raw = events.recv().await.unwrap(); + let frame: Value = serde_json::from_str(&raw).unwrap(); + if frame["type"] == "freshAgent.create.failed" { + panic!("actual fresh launch failed: {}", frame["message"]); + } + if frame["type"] == "freshAgent.created" { + break frame; + } + } + }) + .await + .expect("actual fresh create response"); + assert!(created["sessionId"].as_str().is_some()); + // OpenCode preserves a zero-turn placeholder until its first send. The + // synthetic native plugin rejects chat.message before the model/title loop; + // exercise the real gateway's first-start path without a model request. + let send = serde_json::from_value(json!({ + "requestId":"closed-native-fresh-startup", "provider":"opencode", + "sessionType":"freshopencode", "sessionId":created["sessionId"], + "text":"closed synthetic startup qualification", "cwd":cwd + })) + .unwrap(); + Arc::clone(&proxy) + .dispatch(HostedFreshAgentCommand::Send(send)) + .await; + assert_eq!(read_receipt(&receipt).await.unwrap(), expected); + assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + let stopped = fixture.join("workspace/native-consumed.json.stopped"); + tokio::time::timeout(Duration::from_secs(10), async { + loop { + if std::fs::read(&stopped) + .is_ok_and(|value| value == b"FRESHELL_SYNTHETIC_STARTUP_STOP") + { + break; + } + tokio::time::sleep(Duration::from_millis(25)).await; + } + }) + .await + .expect("native plugin must stop the fresh request before the model loop"); + println!( + "provider_launch_consumer {}", + json!({"provider":"opencode","route":"fresh","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) + ); + // Scan only run-owned durable/staged surfaces; referenced payloads belong + // in private indexed files, not ordinary resource/configuration trees. + fn assert_private_sentinels_absent(root: &Path) { + for entry in std::fs::read_dir(root).unwrap() { + let entry = entry.unwrap(); + let metadata = entry.file_type().unwrap(); + if metadata.is_dir() { + assert_private_sentinels_absent(&entry.path()); + } else if metadata.is_file() { + let bytes = std::fs::read(entry.path()).unwrap(); + for sentinel in [ + b"AUTH_PREFIX".as_slice(), + b"native-rule".as_slice(), + b"synthetic-onecli-shadowed".as_slice(), + ] { + assert!( + !bytes.windows(sentinel.len()).any(|value| value == sentinel), + "private input leaked to an owned durable/staged surface" + ); + } + } + } + } + assert_private_sentinels_absent(&fixture.join("registry")); + assert_private_sentinels_absent(&control.join("provider-roots")); + assert_private_sentinels_absent(&fixture.join("handoff")); + for view in client.inventory().await.unwrap() { + if view.desired_state == DesiredState::Running { + client.stop(view.soul_id).await.unwrap(); + } + } + web_task.abort(); + control_task.abort(); +} diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 125dffe2c..bac71816e 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1101,6 +1101,10 @@ async fn grant_execution( &launch.provider_bootstrap_files, launch.run_as_uid, launch.run_as_gid, + launch + .provider_launch_context + .as_ref() + .is_some_and(|context| context.prepared_inputs.is_some()), ) .map_err(|error| { RuntimeError::new( @@ -1214,12 +1218,26 @@ fn prepare_provider_state_for_fresh_launch( terminal, |terminal| { let mut created = providers::prepare_provider_state_before_bootstrap(terminal)?; - match write_provider_bootstrap_files( - &terminal.mode, - &terminal.provider_bootstrap_files, - Path::new("/home/freshell/provider"), - Path::new("/run/freshell-bootstrap"), - ) { + let bootstrap = if terminal.provider_bootstrap_files.is_empty() + && terminal + .provider_launch_context + .as_ref() + .is_some_and(|context| context.prepared_inputs.is_some()) + { + verify_initialized_provider_home( + Path::new("/home/freshell/provider"), + terminal.run_as_uid, + terminal.run_as_gid, + ) + } else { + write_provider_bootstrap_files( + &terminal.mode, + &terminal.provider_bootstrap_files, + Path::new("/home/freshell/provider"), + Path::new("/run/freshell-bootstrap"), + ) + }; + match bootstrap { Ok(bootstrap) => created.extend(bootstrap), Err(error) => { rollback_created_provider_paths(&created); @@ -1348,14 +1366,14 @@ fn prepare_provider_bootstrap_files( files: &[ProviderBootstrapFile], run_as_uid: u32, run_as_gid: u32, + initialized: bool, ) -> Result<(), String> { let home = Path::new("/home/freshell/provider"); - let created = write_provider_bootstrap_files( - provider, - files, - home, - Path::new("/run/freshell-bootstrap"), - )?; + let created = if initialized && files.is_empty() { + verify_initialized_provider_home(home, run_as_uid, run_as_gid) + } else { + write_provider_bootstrap_files(provider, files, home, Path::new("/run/freshell-bootstrap")) + }?; transfer_created_paths(provider, home, &created, run_as_uid, run_as_gid) } @@ -1427,6 +1445,26 @@ fn write_provider_bootstrap_files( ) } +fn verify_initialized_provider_home( + home: &Path, + uid: u32, + gid: u32, +) -> Result, String> { + use std::os::unix::fs::MetadataExt; + let metadata = + std::fs::symlink_metadata(home).map_err(|_| "initialized provider HOME unavailable")?; + if !metadata.is_dir() + || metadata.file_type().is_symlink() + || metadata.uid() != uid + || metadata.gid() != gid + || metadata.mode() & 0o777 != 0o700 + || std::fs::canonicalize(home).map_err(|_| "initialized provider HOME unavailable")? != home + { + return Err("initialized provider HOME physical ownership changed".into()); + } + Ok(Vec::new()) +} + fn write_provider_bootstrap_files_with_publish( provider: &str, files: &[ProviderBootstrapFile], @@ -2140,7 +2178,7 @@ fn prepare_provider_context_at( } fn prepare_provider_files( - provider: &str, + _provider: &str, provider_files: &[(&str, Vec)], configuration_files: &[(String, PathBuf)], run_as_uid: u32, @@ -2150,7 +2188,6 @@ fn prepare_provider_files( prepare_provider_files_at( Path::new("/home/freshell/provider"), Path::new("/tmp/freshell-provider-auth"), - provider, provider_files, configuration_files, run_as_uid, @@ -2162,7 +2199,6 @@ fn prepare_provider_files( fn prepare_provider_files_at( home: &Path, auth_dir: &Path, - _provider: &str, provider_files: &[(&str, Vec)], configuration_files: &[(String, PathBuf)], run_as_uid: u32, @@ -2222,19 +2258,12 @@ fn prepare_provider_files_at( for (index, (relative, source)) in configuration_files.iter().enumerate() { let destination = home.join(relative); if indexed_configuration { - use std::io::Read; let metadata = std::fs::symlink_metadata(source) .map_err(|_| "indexed configuration-file mount unavailable")?; if !metadata.is_file() { return Err("indexed configuration-file mount is not regular".into()); } - let bytes = std::io::copy( - &mut std::fs::File::open(source) - .map_err(|_| "indexed configuration-file mount unreadable")? - .take(16 * 1024 * 1024 + 1), - &mut std::io::sink(), - ) - .map_err(|_| "indexed configuration-file read failed")?; + let bytes = count_configuration_bytes_as_provider(source, run_as_uid, run_as_gid)?; aggregate = aggregate .checked_add(bytes) .ok_or("indexed configuration-file size overflow")?; @@ -2286,6 +2315,64 @@ fn prepare_provider_files_at( Ok(()) } +fn count_configuration_bytes_as_provider( + source: &Path, + uid: u32, + gid: u32, +) -> Result { + let limit = 16 * 1024 * 1024 + 1; + if uid == unsafe { libc::geteuid() } && gid == unsafe { libc::getegid() } { + use std::io::Read; + return std::io::copy( + &mut std::fs::File::open(source) + .map_err(|_| "indexed configuration-file mount unreadable")? + .take(limit), + &mut std::io::sink(), + ) + .map_err(|_| "indexed configuration-file read failed"); + } + // The indexed publisher owns 0600 files as the provider identity. Keep + // actual-byte validation under that identity; pipe bytes only to a sink. + let mut command = std::process::Command::new("/usr/bin/setpriv"); + command + .args(provider_identity_args(uid, gid)) + .args([ + "/usr/bin/head", + "-c", + "16777217", + "--", + source + .to_str() + .ok_or("indexed configuration-file path is not UTF-8")?, + ]) + .env_clear() + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()); + #[cfg(target_os = "linux")] + freshell_platform::provider_exec::check_command(&command, false, &["/usr/bin/head"]) + .map_err(|_| "indexed configuration-file provider exec profile invalid")?; + let mut child = command + .spawn() + .map_err(|_| "indexed configuration-file provider read failed to start")?; + let bytes = std::io::copy( + &mut child + .stdout + .take() + .ok_or("indexed configuration-file private stream unavailable")?, + &mut std::io::sink(), + ) + .map_err(|_| "indexed configuration-file provider read failed")?; + if !child + .wait() + .map_err(|_| "indexed configuration-file provider read failed")? + .success() + { + return Err("indexed configuration-file provider read failed"); + } + Ok(bytes) +} + fn publish_configuration_link( destination: &Path, target: &Path, @@ -3035,6 +3122,29 @@ mod tests { .expect("owned executable fixture directory") } #[test] + fn provider_config_precreated_home_needs_no_empty_bootstrap_handoff() { + use super::*; + use std::os::unix::fs::{MetadataExt, PermissionsExt}; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + std::fs::create_dir(&home).unwrap(); + std::fs::set_permissions(&home, std::fs::Permissions::from_mode(0o700)).unwrap(); + std::fs::write(home.join("retained"), "retained-provider-state").unwrap(); + let before = std::fs::metadata(&home).unwrap(); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + let ledger = verify_initialized_provider_home(&home, uid, gid).unwrap(); + assert!(ledger.is_empty(),"an already initialized HOME must not be put in the trusted initial ownership ledger again"); + let after = std::fs::metadata(&home).unwrap(); + assert_eq!( + (after.ino(), after.uid(), after.gid(), after.mode()), + (before.ino(), before.uid(), before.gid(), before.mode()) + ); + assert_eq!( + std::fs::read_to_string(home.join("retained")).unwrap(), + "retained-provider-state" + ); + } + #[test] fn provider_native_auth_survives_without_matching_typed_replacement() { use super::*; let root = tempfile::tempdir().unwrap(); @@ -3048,7 +3158,6 @@ mod tests { prepare_provider_files_at( &home, &root.path().join("empty-auth"), - "opencode", &[], &[], uid, @@ -3060,7 +3169,6 @@ mod tests { prepare_provider_files_at( &home, &root.path().join("ca-only"), - "opencode", &[(".freshell/opencode/ca.pem", b"synthetic-ca".to_vec())], &[], uid, @@ -3073,7 +3181,6 @@ mod tests { prepare_provider_files_at( &home, &root.path().join("typed-auth"), - "opencode", &[(".local/share/opencode/auth.json", replacement.to_vec())], &[], uid, @@ -3105,7 +3212,6 @@ mod tests { prepare_provider_files_at( &home, &private, - "opencode", &[], &[(".freshell/references/0".into(), mounted.clone())], uid, @@ -4591,7 +4697,7 @@ mod tests { } #[test] fn all_selected_entries_above_ten_thousand_remain_runnable() { - let root = tempfile::tempdir().unwrap(); + let root = super::executable_fixture(); let source = root.path().join("source"); std::fs::create_dir(&source).unwrap(); for index in 0..10_005 { diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index 7b7f2eb7f..a58826716 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -95,23 +95,52 @@ pub(super) fn owned_helper(actual: &Value, expected: &Value) -> bool { .is_some_and(|(actual, expected)| { actual.len() == expected.len() && actual.iter().zip(expected).all(|(actual, expected)| { - ["Type", "Source", "Target", "ReadOnly"] + ["Type", "Source", "Target"] .iter() .all(|key| actual[key] == expected[key]) + && mount_readonly(&actual["ReadOnly"]) + == mount_readonly(&expected["ReadOnly"]) && actual["VolumeOptions"]["NoCopy"] == expected["VolumeOptions"]["NoCopy"] && actual["VolumeOptions"]["Subpath"].as_str().unwrap_or("") == expected["VolumeOptions"]["Subpath"].as_str().unwrap_or("") }) }); - ["Image", "User", "Tty", "Entrypoint", "Cmd", "Labels"] + ["Image", "User", "Tty", "Entrypoint", "Cmd"] .iter() .all(|key| actual["Config"][key] == expected[key]) + && owned_labels(&actual["Config"]["Labels"], &expected["Labels"]) && environment(&actual["Config"]["Env"]) .is_some_and(|actual| Some(actual) == environment(&expected["Env"])) && mounts_match && actual["HostConfig"]["LogConfig"]["Type"] == "none" } +pub(super) fn mount_readonly(value: &Value) -> Option { + if value.is_null() { + Some(false) + } else { + value.as_bool() + } +} +fn owned_labels(actual: &Value, expected: &Value) -> bool { + let expected = expected.as_object(); + let Some(actual) = actual.as_object() else { + return actual.is_null() && expected.is_none_or(|labels| labels.is_empty()); + }; + expected.is_none_or(|labels| { + labels + .iter() + .all(|(key, value)| actual.get(key) == Some(value)) + }) && actual + .iter() + .filter(|(key, _)| { + key.starts_with("com.freshell.") + || key.starts_with("freshell.") + || key.as_str() == "project" + }) + .all(|(key, value)| expected.and_then(|labels| labels.get(key)) == Some(value)) +} + impl DockerEngineBackend { pub(super) async fn private_helper_environment( &self, @@ -661,6 +690,39 @@ impl DockerEngineBackend { #[cfg(test)] mod tests { use super::*; + + #[test] + fn provider_input_helper_accepts_docker_inherited_labels_and_omitted_false_without_losing_owner_fences( + ) { + let expected = json!({"Image":"sha256:fixture","User":"0:0","Tty":false,"Entrypoint":["/runtime/host"],"Cmd":["prepare-provider-mountpoints"], + "Env":["PATH="],"Labels":{"com.freshell.soul-id":"soul-owned"}, + "HostConfig":{"LogConfig":{"Type":"none"},"Mounts":[{"Type":"volume","Source":"owned-provider","Target":"/home/freshell/provider","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}]}}); + let mut actual = json!({"Config":expected,"HostConfig":expected["HostConfig"]}); + actual["Config"]["Labels"]["org.opencontainers.image.version"] = json!("24.04"); + actual["HostConfig"]["Mounts"][0] + .as_object_mut() + .unwrap() + .remove("ReadOnly"); + assert!(owned_helper(&actual,&expected),"native Docker omitted defaults and inherited image labels preserve the exact helper owner"); + for field in ["owner", "source", "nocopy", "readonly", "extra-owner"] { + let mut changed = actual.clone(); + match field { + "owner" => changed["Config"]["Labels"]["com.freshell.soul-id"] = json!("foreign"), + "source" => { + changed["HostConfig"]["Mounts"][0]["Source"] = json!("foreign-provider") + } + "nocopy" => { + changed["HostConfig"]["Mounts"][0]["VolumeOptions"]["NoCopy"] = json!(false) + } + "readonly" => changed["HostConfig"]["Mounts"][0]["ReadOnly"] = json!(true), + _ => { + changed["Config"]["Labels"]["com.freshell.incarnation-id"] = + json!("foreign-attempt") + } + } + assert!(!owned_helper(&changed, &expected), "{field}"); + } + } use std::sync::{Arc, Mutex}; use tokio::net::UnixListener; diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs index 344402892..7818fbe8a 100644 --- a/crates/freshell-supervisor/src/backend/prepared_inputs.rs +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -99,9 +99,11 @@ pub(super) fn verify( } let declaration = declarations[0]; let report = reports[0]; - if ["Type", "Source", "Target", "ReadOnly"] + if ["Type", "Source", "Target"] .iter() .any(|key| declaration[key] != mount[key]) + || input_publisher::mount_readonly(&declaration["ReadOnly"]) + != input_publisher::mount_readonly(&mount["ReadOnly"]) || report["Type"] != mount["Type"] || report["RW"].as_bool() != mount["ReadOnly"].as_bool().map(|readonly| !readonly) { @@ -202,6 +204,29 @@ mod tests { (mounts, actual, volumes) } + #[test] + fn provider_prepared_adoption_accepts_docker_omitted_writable_default() { + let (mounts, mut actual, volumes) = fixture(); + actual["HostConfig"]["Mounts"] + .as_array_mut() + .unwrap() + .iter_mut() + .find(|mount| mount["ReadOnly"] == false) + .unwrap() + .as_object_mut() + .unwrap() + .remove("ReadOnly"); + verify(&actual, &mounts, "owned-provider", &volumes) + .expect("Docker's absent ReadOnly field means the exact selected writable directory"); + actual["HostConfig"]["Mounts"] + .as_array_mut() + .unwrap() + .iter_mut() + .find(|mount| mount["Target"] == "/outside/plugin") + .unwrap()["ReadOnly"] = json!(true); + assert!(verify(&actual, &mounts, "owned-provider", &volumes).is_err()); + } + #[test] fn provider_prepared_indexed_secret_read_uses_the_published_generation() { let (mut mounts, mut actual, volumes) = fixture(); diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 21dc48e9c..69d887ebd 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -85,6 +85,9 @@ impl Drop for AuthLookupScope { } } +type ProviderPathInputBindings = + HashMap<(SoulId, String), (freshell_runtime_protocol::ProviderPathInputs, Instant)>; + #[derive(Clone)] pub struct Supervisor { pub(crate) registry: Registry, @@ -95,11 +98,7 @@ pub struct Supervisor { /// lock: its durable intent revision must be able to preempt recovery. pub(crate) recovery_locks: Arc>>>>, auth_lookup_authorities: Arc>>, - provider_path_inputs: Arc< - StdMutex< - HashMap<(SoulId, String), (freshell_runtime_protocol::ProviderPathInputs, Instant)>, - >, - >, + provider_path_inputs: Arc>, } impl Supervisor { diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 9969a0e64..a709a98a0 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -37,6 +37,11 @@ if [ "${1:-}" = "--provider-mountpoints" ]; then exec python3 "${REPO_ROOT}/scripts/testing/managed-provider-mountpoints-sandbox.py" "$@" fi +if [ "${1:-}" = "--provider-launches" ]; then + shift + exec python3 "${REPO_ROOT}/scripts/testing/managed-provider-launch-sandbox.py" "$@" +fi + if [ "${1:-}" = "--provider-inputs" ]; then shift exec python3 "${REPO_ROOT}/scripts/testing/probe-provider-config-readonly-home.py" --indexed-inputs "$@" diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts new file mode 100644 index 000000000..c4a39aec3 --- /dev/null +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -0,0 +1,70 @@ +//! Trusted closed-fixture entrypoint. Only this container receives Docker IPC; +//! the native startup fixture receives this broker's restricted proxy socket. +import fs from 'node:fs' +import path from 'node:path' +import { RestrictedDockerBroker, type PreparedInputAuthorization } from './runtime-test-broker.js' + +const root=process.env.FRESHELL_PROVIDER_LAUNCH_FIXTURE ?? '' +if (path.dirname(root)!=='/tmp' || !/^fln-[a-f0-9]{16}$/.test(path.basename(root))) { + throw new Error('closed provider startup requires its exact owned fixture root') +} +const configuration=JSON.parse(fs.readFileSync(path.join(root,'broker-configuration.json'),'utf8')) +const image='sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c' +if (configuration.imageRef!==image || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==2) { + throw new Error('closed provider startup image or finite source set changed') +} +const gateways=new Set() +const broker=new RestrictedDockerBroker({ + realSocketPath:'/run/fixture-docker.sock',proxySocketPath:path.join(root,'broker.sock'), + runtimeRootPrefix:path.join(root,'scenarios'),allowedHostBinaryPaths:new Set([configuration.hostBinaryPath]), + allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, + allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), + allowedProviderUserRoots:new Set(['/home/freshell/provider']), + allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json')]), + allowedHostGatewayAddresses:gateways, + testRunId:configuration.runId,logPath:path.join(root,'broker.jsonl'), +}) +for (const authorization of configuration.initialVolumes) broker.authorizeInitialProviderVolume(authorization) +await broker.start() +const accepted=new Set() +const timer=setInterval(()=>{ + try { + const gatewayFile=path.join(root,'controller-gateway.json') + if (fs.existsSync(gatewayFile) && gateways.size===0) { + const gateway=JSON.parse(fs.readFileSync(gatewayFile,'utf8')) + if (typeof gateway.address!=='string' || !/^(?:\d{1,3}\.){3}\d{1,3}$/.test(gateway.address)) throw new Error('closed controller gateway is invalid') + gateways.add(gateway.address) + } + const directory=path.join(root,'handoff') + const entries=fs.readdirSync(directory).filter((name)=>name.endsWith('.json')) + if (entries.length>2) throw new Error('closed provider fixture exceeds its finite two launch attempts') + for (const name of entries) { + if (accepted.has(name)) continue + if (!/^[a-zA-Z0-9_-]{1,128}\.json$/.test(name)) throw new Error('closed provider fixture handoff name invalid') + const file=path.join(directory,name) + const stat=fs.lstatSync(file) + if (!stat.isFile() || stat.isSymbolicLink() || stat.size>1024*1024) throw new Error('closed provider fixture handoff profile exceeded') + const input=JSON.parse(fs.readFileSync(file,'utf8')) as PreparedInputAuthorization + if (!configuration.initialVolumes.some((source:any)=>source.installationId===input.installationId + && source.soulId===input.soulId && source.providerVolumeName===input.providerVolumeName)) { + throw new Error('closed provider fixture handoff owner changed') + } + broker.authorizePreparedInputs(input) + accepted.add(name) + fs.writeFileSync(file.slice(0,-'.json'.length)+'.accepted','ok',{mode:0o600}) + } + if (fs.existsSync(path.join(root,'broker-complete'))) { + clearInterval(timer) + void broker.close().then(()=>{ + fs.writeFileSync(path.join(root,'broker-receipt.json'),JSON.stringify({ + events:broker.eventsSnapshot(),receipts:broker.receipts(),unsafeAttempts:broker.unsafeAttempts(),acceptedPreparations:accepted.size, + })+'\n',{mode:0o600}) + }) + } + } catch { + clearInterval(timer) + fs.writeFileSync(path.join(root,'broker-failure'),'closed fixture authorization failed',{mode:0o600}) + void broker.close() + process.exitCode=1 + } +},20) diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py new file mode 100644 index 000000000..baebea5ad --- /dev/null +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Closed actual OpenCode controller/supervisor/host startup qualification.""" +import argparse +import hashlib +import json +from pathlib import Path +import re +import subprocess +import tempfile +import time +import uuid + +IMAGE="sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" +TEST="managed_provider_launch_tests::managed_provider_opencode_terminal_and_fresh_native_startup_consumes_prepared_inputs" +LABEL="com.freshell.provider-launch-probe" + +def docker(*args,check=True,timeout=180): + return subprocess.run(["docker",*args],text=True,capture_output=True,timeout=timeout,check=check) + +def digest(path): + with path.open("rb") as stream: + return hashlib.file_digest(stream,"sha256").hexdigest() + +def main(): + parser=argparse.ArgumentParser(description=__doc__) + parser.add_argument("--test-binary",type=Path,required=True) + parser.add_argument("--host-binary",type=Path,required=True) + parser.add_argument("--broker-binary",type=Path,required=True) + parser.add_argument("--receipt-dir",type=Path,required=True) + args=parser.parse_args() + repo=Path(__file__).resolve().parents[2] + test=args.test_binary.resolve(strict=True);host=args.host_binary.resolve(strict=True);broker=args.broker_binary.resolve(strict=True) + if test.parent!=repo/"target/debug/deps" or not re.fullmatch(r"freshell_server-[a-f0-9]{16}",test.name) or host!=repo/"target/debug/freshell-session-host" or broker!=repo/"target/managed-provider-launch-broker.mjs" or not args.receipt_dir.is_dir(): + raise RuntimeError("closed startup requires this worktree's exact compiled artifacts and existing receipts") + listing=subprocess.run([str(test),"--exact",TEST,"--list"],env={"PATH":"/usr/bin:/bin"},text=True,capture_output=True,timeout=20,check=True) + if f"{TEST}: test" not in listing.stdout.splitlines(): + raise RuntimeError("closed startup compiled test is missing") + info=json.loads(docker("info","--format","{{json .}}").stdout) + if "name=rootless" not in info["SecurityOptions"] or docker("image","inspect","--format","{{.Id}}",IMAGE).stdout.strip()!=IMAGE: + raise RuntimeError("closed startup requires the qualified rootless daemon and exact cached image") + endpoint=json.loads(docker("context","inspect").stdout)[0]["Endpoints"]["docker"]["Host"] + if not endpoint.startswith("unix://"): + raise RuntimeError("closed startup requires the existing local rootless daemon IPC") + run="freshell-launch-"+uuid.uuid4().hex;installation=str(uuid.uuid4());owned=[] + receipt={"runId":run,"image":IMAGE,"test":TEST,"rootless":True,"testArtifactSha256":digest(test),"hostArtifactSha256":digest(host),"brokerArtifactSha256":digest(broker)} + volumes=set() + def remove(name): + result=docker("container","inspect",name,check=False) + if result.returncode: + return + value=json.loads(result.stdout)[0] + labels=value["Config"].get("Labels",{}) + known_helpers=[] + if labels.get("com.freshell.installation-id")==installation: + for candidate in (root/"handoff").glob("*.json"): + metadata=json.loads(candidate.read_text()) + if labels.get("com.freshell.soul-id")==metadata["soulId"] and labels.get("com.freshell.incarnation-id")==metadata["incarnationId"]: + known_helpers.append(metadata) + fixed_helper=value["Config"].get("Image")==IMAGE and value["Config"].get("Cmd") in [["publish-provider-inputs"],["prepare-provider-mountpoints"]] and bool(known_helpers) + if fixed_helper: + receipt.setdefault("helperInspections",[]).append({ + "id":value["Id"],"image":value["Config"]["Image"],"user":value["Config"]["User"], + "entrypoint":value["Config"]["Entrypoint"],"command":value["Config"]["Cmd"], + "labels":labels,"mounts":value["HostConfig"].get("Mounts"),"logConfig":value["HostConfig"].get("LogConfig"), + "tty":value["Config"].get("Tty"),"exitCode":value["State"].get("ExitCode"), + "environment": [{"name":entry.split("=",1)[0],"empty":entry.endswith("=")} for entry in value["Config"].get("Env",[])], + }) + if labels.get(LABEL)!=run and labels.get("com.freshell.runtime-test-run-id")!=run and not fixed_helper: + raise RuntimeError("closed startup container cleanup ownership changed") + docker("rm","--force",value["Id"]) + def create(name,*arguments): + owned.append(name) + docker("create","--name",name,"--label",f"{LABEL}={run}","--read-only","--security-opt","no-new-privileges:true", + "--pids-limit","256","--memory","2g",*arguments) + with tempfile.TemporaryDirectory(prefix=run+"-") as temporary, tempfile.TemporaryDirectory(prefix="fl-"+uuid.uuid4().hex[:8]+"-") as ipc: + # Keep an exact UUID-only basename for both trusted and fixture checks. + root=Path(temporary).with_name("fln-"+uuid.uuid4().hex[:16]) + Path(temporary).rename(root) + try: + for name in ["workspace","source-home/.config/opencode","grants","handoff","tmp","scenarios"]: + (root/name).mkdir(parents=True,exist_ok=True) + initial=[] + for soul in ["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32],"soul-fresh-opencode-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]]: + volume="freshell-provider-"+hashlib.sha256((installation+"\0"+soul).encode()).hexdigest()[:24] + volumes.add(volume) + initial.append({"installationId":installation,"soulId":soul,"providerVolumeName":volume,"imageRef":IMAGE,"hostBinaryPath":str(host),"binarySha256":digest(host),"daemonId":info["ID"]}) + (root/"broker-configuration.json").write_text(json.dumps({"imageRef":IMAGE,"hostBinaryPath":str(host),"runId":run,"initialVolumes":initial})) + setup=run+"-dependencies" + # Install the actual exact published SDK tree into synthetic config + # storage. No read-only install bypass, fake SDK or model call. + create(setup,"--network","bridge","--user","0:0","--cap-drop","ALL", + "--mount",f"type=bind,src={root},dst={root}","--tmpfs","/tmp:rw,nosuid,nodev,size=128m", + "--env",f"HOME={root}/source-home","--entrypoint","/usr/local/bin/npm",IMAGE, + "install","--prefix",str(root/"source-home/.config/opencode"),"@opencode-ai/plugin@1.18.21") + outcome=docker("start","--attach",setup,check=False) + if outcome.returncode: + raise RuntimeError("closed startup actual SDK dependency preparation failed") + remove(setup) + writable=run+"-workspace-mode" + create(writable,"--network","none","--user","0:0","--cap-drop","ALL", + "--mount",f"type=bind,src={root},dst={root}","--entrypoint","/usr/bin/chmod",IMAGE, + "0770",str(root/"workspace")) + docker("start","--attach",writable) + remove(writable) + config=root/"source-home/.config/opencode" + plugin=config/"plugins/probe-server";plugin.mkdir(parents=True) + (plugin/"resource.txt").write_text("plugin-relative-resource") + (plugin/"dependency.mjs").write_text("export const value='plugin-actual-local-dependency';\n") + (plugin/"index.mjs").write_text("""import fs from 'node:fs';import crypto from 'node:crypto';import {tool} from '@opencode-ai/plugin';import {value} from './dependency.mjs'; +const hash=(value)=>crypto.createHash('sha256').update(value).digest('hex'); +export default {id:'closed-native-fixture',async server(input,options){return {async config(config){ + const receipt={prompt:hash(config.agent.probe.prompt),description:hash(config.agent.probe.description),references:['third','fourth','fifth','sixth'].map(name=>hash(config.agent[name].prompt)),resource:hash(fs.readFileSync(new URL('./resource.txt',import.meta.url))),dependency:hash(value),sdk:hash(typeof tool)}; + fs.writeFileSync(options.receipt,JSON.stringify(receipt)); +},async 'chat.message'(){ + fs.writeFileSync(options.receipt+'.stopped','FRESHELL_SYNTHETIC_STARTUP_STOP'); + throw new Error('FRESHELL_SYNTHETIC_STARTUP_STOP'); +}}}}; +""") + syntax=subprocess.run(["node","--check",str(plugin/"index.mjs")],text=True,capture_output=True,timeout=20) + if syntax.returncode: + raise RuntimeError("closed startup synthetic plugin syntax invalid") + (config/"rules.txt").write_text("native-rule\n");(config/"second.txt").write_text("native-second-rule\n") + agents={"probe":{"prompt":"{env:OPENAI_API_KEY}","description":"{file:{env:RULE_DIR}/second.txt}"}} + for ordinal in ["third","fourth","fifth","sixth"]: + (config/f"{ordinal}.txt").write_text(f"native-{ordinal}-rule\n") + agents[ordinal]={"prompt":"{file:./"+ordinal+".txt}"} + (config/"opencode.json").write_text(json.dumps({"agent":agents,"plugin":[["./plugins/probe-server/index.mjs",{"receipt":str(root/"workspace/native-consumed.json")}]]})) + (root/"grants/environment.env").write_text("OPENAI_API_KEY=synthetic-onecli-shadowed\n") + (root/"grants/auth.json").write_text(json.dumps({"fixture":{"type":"wellknown","key":"OPENAI_API_KEY","token":"AUTH_PREFIX{file:./rules.txt}AUTH_SUFFIX"}})) + # These newly created synthetic grants alone receive private mode. + (root/"grants/environment.env").chmod(0o600);(root/"grants/auth.json").chmod(0o600) + broker_name=run+"-broker" + create(broker_name,"--network","none","--user","0:0","--cap-drop","ALL", + "--mount",f"type=bind,src={root},dst={root}", + "--mount",f"type=bind,src={ipc},dst={ipc}", + "--mount",f"type=bind,src={endpoint[7:]},dst=/run/fixture-docker.sock,readonly", + "--mount",f"type=bind,src={broker},dst=/runtime/startup-broker.mjs,readonly", + "--env",f"FRESHELL_PROVIDER_LAUNCH_FIXTURE={root}","--env",f"TMPDIR={ipc}", + "--entrypoint","/usr/local/bin/node",IMAGE,"/runtime/startup-broker.mjs") + docker("start",broker_name) + deadline=time.monotonic()+20 + while not (root/"broker.sock").exists() and time.monotonic()): boolea && Object.entries(expected).every(([key, value]) => record[key] === value) } +function ownedHelperLabels(actual:Record | undefined,expected:Record):boolean { + return !!actual && Object.entries(expected).every(([key,value])=>actual[key]===value) + && Object.entries(actual).filter(([key])=>key.startsWith('com.freshell.') || key.startsWith('freshell.') || key==='project') + .every(([key,value])=>expected[key]===value) +} + function privateHelperEnvironment(value: unknown, home: boolean): boolean { if (!Array.isArray(value) || value.length > 128) return false const names = new Set() @@ -115,6 +121,7 @@ export type RestrictedDockerBrokerPolicy = { allowedWorkspaceRoots?: Set allowedProviderUserRoots?: Set allowedBootstrapFiles?: Set + allowedHostGatewayAddresses?: Set testRunId: string logPath: string } @@ -134,6 +141,16 @@ export type PreparedInputAuthorization = { indexedSecretSources: { grantIndex: number; sourceId: number }[] } +export type InitialProviderVolumeAuthorization = { + installationId:string + soulId:string + providerVolumeName:string + imageRef:string + hostBinaryPath:string + binarySha256:string + daemonId:string +} + type DockerResponse = { statusCode: number headers: http.IncomingHttpHeaders @@ -141,6 +158,7 @@ type DockerResponse = { } export class RestrictedDockerBroker { + private readonly initialProviderVolumes = new Map() private readonly preparedInputs = new Map() private readonly inputVolumes = new Set() private readonly inputPublishers = new Map /^[a-zA-Z0-9_-]{1,128}$/.test(value) const relative = path.relative(path.dirname(this.policy.runtimeRootPrefix), input.stageRoot) @@ -287,7 +318,8 @@ export class RestrictedDockerBroker { } const volume = url.match(/^\/v1\.47\/volumes\/(freshell-provider-[a-f0-9]{24})$/)?.[1] - if (method === 'GET' && volume && this.receipts().some((receipt) => receipt.providerVolumeName === volume)) { + if (method === 'GET' && volume && (this.initialProviderVolumes.has(volume) + || this.receipts().some((receipt) => receipt.providerVolumeName === volume))) { await this.forwardAndReply(request, response, body, { destructive: false }) return } @@ -312,6 +344,25 @@ export class RestrictedDockerBroker { if (method==='POST' && url===`${DOCKER_API_PREFIX}/volumes/create`) { let parsed:Record={} try {parsed=JSON.parse(body.toString())} catch {} + const initial=this.initialProviderVolumes.get(parsed.Name) + if (initial) { + const labels=parsed.Labels ?? {} + const expected={ + 'com.freshell.installation-id':initial.installationId,'com.freshell.soul-id':initial.soulId, + 'freshell.input.reservation':labels['freshell.input.reservation'], + 'freshell.input.incarnation':labels['freshell.input.incarnation'],'freshell.input.intent':'1', + 'freshell.input.daemon':initial.daemonId,'freshell.input.image':initial.imageRef, + 'freshell.input.binary-sha256':initial.binarySha256, + } + if (Object.keys(parsed).length!==3 || parsed.Driver!=='local' || !equalRecord(labels,expected) + || !/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(labels['freshell.input.reservation']) + || !/^[a-zA-Z0-9_-]{1,128}$/.test(labels['freshell.input.incarnation'])) { + this.block(response,method,url,403,'initial volume must match the exact reserved soul source',true) + return + } + await this.forwardAndReply(request,response,body,{destructive:false}) + return + } const input=[...this.preparedInputs.values()].find((candidate)=>candidate.indexedFiles?.volumeName===parsed.Name) if (!input || Object.keys(parsed).length!==4 || parsed.Driver!=='local' || !equalRecord(parsed.DriverOpts,{type:'tmpfs',device:'tmpfs',o:INDEXED_TMPFS_OPTIONS}) || !equalRecord(parsed.Labels,indexedLabels(input))) { @@ -335,11 +386,12 @@ export class RestrictedDockerBroker { const environment=(entries:unknown)=>Array.isArray(entries)?JSON.stringify([...entries].sort()):undefined const ownedConfig=inspected.statusCode===404 || (inspected.statusCode===200 && ['Image','User','Tty'].every((key)=>actual.Config?.[key]===owned.expected[key]) && ['Entrypoint','Cmd'].every((key)=>JSON.stringify(actual.Config?.[key])===JSON.stringify(owned.expected[key])) - && equalRecord(actual.Config?.Labels,owned.expected.Labels) + && ownedHelperLabels(actual.Config?.Labels,owned.expected.Labels) && environment(actual.Config?.Env)===environment(owned.expected.Env) && Array.isArray(actual.HostConfig?.Mounts) && actual.HostConfig.Mounts.length===3 && owned.expected.HostConfig.Mounts.every((expected:Record)=>actual.HostConfig.Mounts.some((mount:Record)=> - ['Type','Source','Target','ReadOnly'].every((key)=>mount[key]===expected[key]) + ['Type','Source','Target'].every((key)=>mount[key]===expected[key]) + && (mount.ReadOnly ?? false)===(expected.ReadOnly ?? false) && (expected.Type!=='volume' || (mount.VolumeOptions?.NoCopy===true && (mount.VolumeOptions.Subpath ?? '')==='' && !mount.VolumeOptions.DriverConfig)))) && actual.HostConfig?.LogConfig?.Type==='none') if (!ownedConfig) { @@ -530,6 +582,7 @@ export class RestrictedDockerBroker { const localAddresses = new Set(Object.values(os.networkInterfaces()).flatMap((interfaces) => ( interfaces?.filter((network) => network.family === 'IPv4' && !network.internal).map((network) => network.address) ?? [] ))) + for (const address of this.policy.allowedHostGatewayAddresses ?? []) localAddresses.add(address) if (extraHosts.some((entry: unknown) => { if (entry === 'host.docker.internal:host-gateway') return false if (typeof entry !== 'string' || !entry.startsWith('host.docker.internal:')) return true diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 1754a3fde..e02f22095 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -209,6 +209,16 @@ describe('managed provider tmpfs', () => { }) describe('managed provider host gateway', () => { + it('allows only the exact owned controller gateway supplied by the closed fixture',async()=>{ + await withHttpBroker(async({request,forwarded})=>{ + const accepted=createBody([...ordinaryBinds,actorBind],undefined,['host.docker.internal:172.17.0.241']); + expect((await request('POST','/v1.47/containers/create?name=owned-controller',accepted)).status).toBe(201); + const before=forwarded.length; + const refused=createBody([...ordinaryBinds,actorBind],undefined,['host.docker.internal:172.17.0.242']); + expect((await request('POST','/v1.47/containers/create?name=foreign-controller',refused)).status).toBe(403); + expect(forwarded).toHaveLength(before); + },new Set(['172.17.0.241'])); + }); it('accepts the rootless daemon mapping to a local host address', () => { const validate = (body: Buffer): { ok: boolean } => ( broker() as unknown as { validateCreate(body: Buffer): { ok: boolean } } @@ -332,6 +342,35 @@ function publisherBody(input:PreparedInputAuthorization):any { } describe('fixed indexed publisher broker',()=>{ + it('observes and creates only the explicitly reserved initial soul volume before any workload receipt',async()=>{ + await withHttpBroker(async ({request,forwarded,broker:instance})=>{ + const installationId='installation-one',soulId='soul-initial'; + const providerVolumeName=`freshell-provider-${createHash('sha256').update(`${installationId}\0${soulId}`).digest('hex').slice(0,24)}`; + const authorization={installationId,soulId,providerVolumeName,imageRef:'sha256:fixture-image',hostBinaryPath:'/tmp/owned/freshell-session-host',binarySha256:'a'.repeat(64),daemonId:'fixture-daemon'}; + instance.authorizeInitialProviderVolume(authorization); + expect((await request('GET',`/v1.47/volumes/${providerVolumeName}`)).status).toBe(200); + const body={Name:providerVolumeName,Driver:'local',Labels:{ + 'com.freshell.installation-id':installationId,'com.freshell.soul-id':soulId, + 'freshell.input.reservation':'11111111-1111-4111-8111-111111111111', + 'freshell.input.incarnation':'incarnation-initial','freshell.input.intent':'1', + 'freshell.input.daemon':authorization.daemonId,'freshell.input.image':authorization.imageRef, + 'freshell.input.binary-sha256':authorization.binarySha256, + }}; + expect((await request('POST','/v1.47/volumes/create',body)).status).toBe(201); + const before=forwarded.length; + for (const mutate of [ + (value:any)=>value.Name='freshell-provider-'+ 'f'.repeat(24), + (value:any)=>value.DriverOpts={type:'tmpfs'}, + (value:any)=>value.Labels['freshell.input.daemon']='changed', + (value:any)=>value.Labels['freshell.input.binary-sha256']='b'.repeat(64), + (value:any)=>value.Labels['freshell.input.reservation']='arbitrary', + ]) { + const changed=structuredClone(body);mutate(changed); + expect((await request('POST','/v1.47/volumes/create',changed)).status).toBe(403); + expect(forwarded).toHaveLength(before); + } + }); + }); it('allows the fixed owned mountpoint helper before workload creation and refuses extra authority',async()=>{ await withHttpBroker(async ({request,forwarded,broker:instance})=>{ const input=publisherAuthorization();instance.authorizePreparedInputs(input); @@ -414,7 +453,7 @@ describe('fixed indexed publisher broker',()=>{ }); }); -async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise) { +async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise,gateways?:Set) { const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-broker-history-')) const socket = path.join(dir, 'docker.sock') const proxy = path.join(dir, 'broker.sock') @@ -438,7 +477,10 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise } if (req.method==='GET' && req.url?.includes('-publisher/json')) { const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]==='publish-provider-inputs')!.body; - res.statusCode=200;res.end(JSON.stringify({Config:foreignHelper?{...config,User:'0:0'}:config,HostConfig:config.HostConfig}));return; + const inspected=structuredClone(config); + inspected.Labels['org.opencontainers.image.version']='24.04'; + for (const mount of inspected.HostConfig.Mounts) if (mount.ReadOnly===false) delete mount.ReadOnly; + res.statusCode=200;res.end(JSON.stringify({Config:foreignHelper?{...inspected,User:'0:0'}:inspected,HostConfig:inspected.HostConfig}));return; } if (req.url?.startsWith('/v1.47/containers/create?')) { if (body.Entrypoint?.[0] === '/runtime/freshell-session-host' && loseAck) { @@ -459,6 +501,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise : req.url?.includes('/logs?') ? '{"threadId":"native-owned"}' : JSON.stringify({ Name: providerVolume })) }) const instance = new RestrictedDockerBroker({ + allowedHostGatewayAddresses:gateways, realSocketPath: socket, proxySocketPath: proxy, runtimeRootPrefix: '/tmp/owned/r', From 61e8d87f27ed2343b798e8b9cf6de4482d37c3ee Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 15:47:45 -0700 Subject: [PATCH 045/493] Carry complete provider contexts through replacement and retry callbacks --- .../src/provider_inputs.rs | 67 +++ .../freshell-server/src/fresh_agent_proxy.rs | 20 +- .../src/managed_mcp_capability.rs | 530 +++++++++++++++++- .../src/managed_provider_references.rs | 65 ++- .../environment.rs | 2 +- crates/freshell-server/src/managed_runtime.rs | 20 +- .../src/native_auth_lookup_tests.rs | 142 ++++- crates/freshell-supervisor/src/recovery.rs | 180 ++++-- crates/freshell-supervisor/src/service.rs | 39 +- 9 files changed, 962 insertions(+), 103 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index e5547b799..9ecd3095b 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -4,6 +4,31 @@ use crate::{McpCapabilityReference, RuntimeError, RuntimeErrorCode, SoulId}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; +/// One immutable preparation result for every launch and replacement route. +/// Source documents and selected payloads stay in their private filesystem. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct PreparedProviderContext { + pub provider_launch_context: crate::ProviderLaunchContext, + pub provider_secret_references: Vec, +} +impl PreparedProviderContext { + pub fn validate(&self, provider: &str) -> Result<(), RuntimeError> { + crate::bounded_json_bytes(self, crate::MAX_CONTROL_FRAME_BYTES).map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "prepared provider context exceeds control metadata profile", + ) + })?; + self.provider_launch_context.validate(provider)?; + crate::validate_provider_secrets(provider, &self.provider_secret_references)?; + if let Some(inputs) = &self.provider_launch_context.prepared_inputs { + inputs.validate_references(&self.provider_secret_references)?; + } + Ok(()) + } +} + /// Transient frozen path inputs travel only through authenticated process IPC. /// This type must never be embedded in a durable launch specification. #[derive(Clone, Default, PartialEq, Eq, Serialize)] @@ -483,6 +508,48 @@ mod tests { host_gateway_address: None, } } + #[test] + fn provider_prepared_complete_context_rejects_mixed_generation_and_reference_vectors() { + let value = PreparedProviderContext { + provider_launch_context: crate::ProviderLaunchContext { + prepared_inputs: Some(prepared()), + preparation: crate::ProviderPreparation::Codex { + tui_args: vec![], + sidecar_args: vec![], + }, + mcp_capability: Some(capability()), + config: vec![], + }, + provider_secret_references: (0..2) + .map(|index| crate::ProviderSecretReference { + source_path: format!("/private/source-{index}"), + profile: crate::ProviderSecretProfile::CodexConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{index}")), + }) + .collect(), + }; + let wire = crate::bounded_json_bytes(&value, crate::MAX_CONTROL_FRAME_BYTES).unwrap(); + let mut received: PreparedProviderContext = serde_json::from_slice(&wire).unwrap(); + received.validate("codex").unwrap(); + assert!(received.validate("opencode").is_err()); + received.provider_secret_references.swap(0, 1); + assert!( + received.validate("codex").is_err(), + "stale/new indexed reference vectors cannot accompany a prepared generation" + ); + received = value; + received + .provider_launch_context + .mcp_capability + .as_mut() + .unwrap() + .grant_id = "grant-different".into(); + assert!( + received.validate("codex").is_err(), + "one complete reply must retain its exact generation authority" + ); + } + #[test] fn provider_prepared_mounts_reject_different_sources_over_one_effective_inode() { let mut value = prepared(); diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 9f5b07ec2..71af97ca1 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -457,15 +457,17 @@ impl HostedFreshAgentProxy { .ok_or(())?; let original = |name: &str| std::env::var(name).ok(); let prepared = crate::managed_provider_references::prepare_managed_opencode( - &self.client, - &soul, - &message.request_id, - &home, - &cwd, - &workspace, - &original, - provider_secret_references, - None, + crate::managed_provider_references::OpencodePreparation { + client: &self.client, + soul: &soul, + creation_seed: &message.request_id, + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &original, + references: provider_secret_references, + auth_authority: None, + }, ) .await .map_err(|error| { diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 787113da9..460642edc 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -85,6 +85,7 @@ struct CapabilitySettings { endpoint: String, callback_socket: PathBuf, control_secret: String, + control_socket: PathBuf, } fn store() -> &'static CapabilityStore { @@ -140,6 +141,7 @@ pub(crate) fn configure( endpoint: format!("http://host.docker.internal:{port}"), callback_socket: control_dir.join("mcp.sock"), control_secret, + control_socket: control_socket.to_owned(), }) .map_err(|_| "managed MCP controller already configured".into()) } @@ -180,14 +182,20 @@ pub(crate) async fn spawn_callback_server() -> Result<(), String> { let home = std::env::var_os("HOME") .map(PathBuf::from) .ok_or("managed provider HOME is unavailable")?; - apply_callback( + let client = freshell_runtime_client::RuntimeClient::new( + SETTINGS.get().unwrap().control_socket.clone(), + expected.clone(), + ); + apply_prepared_callback( &bytes, expected, store(), &capability_dir()?, &endpoint()?, &home, + &client, ) + .await } .await; match result { @@ -243,6 +251,142 @@ struct CapabilityCallback { tui_source: Option, workspace: Option, auth_lookup_authority: Option, + cwd: Option, + provider_secret_references: Option>, + prior_context: Option, +} + +async fn apply_prepared_callback( + bytes: &[u8], + expected: &str, + store: &CapabilityStore, + directory: &Path, + endpoint: &str, + home: &Path, + client: &freshell_runtime_client::RuntimeClient, +) -> Result, String> { + let callback: CapabilityCallback = serde_json::from_slice(bytes) + .map_err(|_| "managed provider callback metadata is invalid")?; + if !freshell_api::check_auth(Some(&callback.control_secret), expected) { + return Err("managed runtime callback unauthorized".into()); + } + if callback.action != "issue" { + return apply_callback(bytes, expected, store, directory, endpoint, home); + } + let provider = callback + .provider + .as_deref() + .ok_or("missing managed MCP provider")?; + if let Some(reply) = store.pending_prepared_context( + &callback.soul_id, + directory, + endpoint, + provider, + callback.prior_context.as_ref(), + )? { + return freshell_runtime_protocol::bounded_json_bytes( + &reply, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "replacement provider context exceeds control metadata profile".into()); + } + if provider != "opencode" { + let references = provider_source_references(provider, home)?; + let mut context = callback + .prior_context + .clone() + .ok_or("complete replacement requires prior provider context")?; + if context.prepared_inputs.is_some() { + return Err( + "selected provider prepared sources require complete replacement planning".into(), + ); + } + context.config = references.clone(); + let secret_references = callback.provider_secret_references.unwrap_or_default(); + let capability = store.issue_replacement_with_inputs( + callback.soul_id, + directory, + endpoint.into(), + Some((provider, home, &references)), + OpencodeEphemeralInput { + launch_context: Some(&context), + secret_references: Some(&secret_references), + tui_source: callback.tui_source.as_ref(), + workspace: callback.workspace.as_deref(), + ..Default::default() + }, + )?; + let context = read_prepared_context(directory, &capability)?; + let reply = freshell_runtime_protocol::PreparedProviderContext { + provider_launch_context: context, + provider_secret_references: secret_references, + }; + reply + .validate(provider) + .map_err(|_| "replacement provider context metadata invalid")?; + return freshell_runtime_protocol::bounded_json_bytes( + &reply, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "replacement provider context exceeds control metadata profile".into()); + } + let workspace = callback + .workspace + .as_deref() + .ok_or("missing provider workspace")?; + let cwd = callback.cwd.as_deref().unwrap_or(workspace); + let references = callback + .provider_secret_references + .unwrap_or_default() + .into_iter() + .filter(|reference| !reference.profile.is_configuration_file()) + .collect(); + let original = |name: &str| std::env::var(name).ok(); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client, + soul: &callback.soul_id, + creation_seed: "replacement", + home, + cwd, + workspace, + original: &original, + references, + auth_authority: Some( + callback + .auth_lookup_authority + .as_deref() + .ok_or("replacement provider preparation requires attempt authority")?, + ), + }, + |soul, home, context, input| { + let input = OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }; + let capability = store.issue_replacement_with_inputs( + soul.clone(), + directory, + endpoint.into(), + Some(("opencode", home, &context.config)), + input, + )?; + read_prepared_context(directory, &capability) + }, + ) + .await?; + let reply = freshell_runtime_protocol::PreparedProviderContext { + provider_launch_context: prepared.context, + provider_secret_references: prepared.references, + }; + reply + .validate(provider) + .map_err(|_| "replacement provider context metadata invalid")?; + freshell_runtime_protocol::bounded_json_bytes( + &reply, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "replacement provider context exceeds control metadata profile".into()) } fn apply_callback( @@ -1170,6 +1314,52 @@ fn write_capability(path: &Path, grant: &Grant) -> Result<(), String> { Ok(()) } +fn publish_prepared_context_metadata( + root: &Path, + provider: &str, + context: &freshell_runtime_protocol::ProviderLaunchContext, + references: Option<&[freshell_runtime_protocol::ProviderSecretReference]>, +) -> Result<(), String> { + context + .validate(provider) + .map_err(|_| "complete provider context metadata invalid")?; + let parent = root.join(".freshell"); + if !parent.exists() { + private_directory(&parent)?; + } + if !std::fs::symlink_metadata(&parent) + .is_ok_and(|metadata| metadata.is_dir() && !metadata.file_type().is_symlink()) + { + return Err("complete provider metadata parent must be a physical directory".into()); + } + let bytes = freshell_runtime_protocol::bounded_json_bytes( + context, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "complete provider context exceeds metadata profile")?; + let path = parent.join("prepared-context.json"); + write_private_bytes(&path, &bytes)?; + allow_controller_stage_read(&path)?; + if let Some(references) = references { + let reply = freshell_runtime_protocol::PreparedProviderContext { + provider_launch_context: context.clone(), + provider_secret_references: references.to_vec(), + }; + reply + .validate(provider) + .map_err(|_| "complete provider result metadata invalid")?; + let bytes = freshell_runtime_protocol::bounded_json_bytes( + &reply, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "complete provider result exceeds metadata profile")?; + let path = parent.join("prepared-result.json"); + write_private_bytes(&path, &bytes)?; + allow_controller_stage_read(&path)?; + } + Ok(()) +} + impl CapabilityStore { fn renew_active_at(&self, now: u64, active: &HashSet<(SoulId, IncarnationId)>) { for grant in self.grants.lock().unwrap().values_mut() { @@ -1246,17 +1436,12 @@ impl CapabilityStore { }) }); context.prepared_inputs = Some(inputs); - context - .validate("opencode") - .map_err(|error| error.message)?; - let bytes = freshell_runtime_protocol::bounded_json_bytes( + publish_prepared_context_metadata( + root, + "opencode", &context, - freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, - ) - .map_err(|_| "complete prepared context exceeds metadata profile")?; - let path = root.join(".freshell/prepared-context.json"); - write_private_bytes(&path, &bytes)?; - allow_controller_stage_read(&path)?; + opencode_input.secret_references, + )?; } Ok(()) }); @@ -1265,6 +1450,29 @@ impl CapabilityStore { return Err(error); } } + if opencode_input.source_plan.is_none() { + if let (Some(root), Some(context), Some((provider, _, _))) = + (&staged_provider_root, opencode_input.launch_context, source) + { + let mut context = context.clone(); + context.mcp_capability = Some(McpCapabilityReference { + grant_id: grant_id.clone(), + endpoint: endpoint.clone(), + provider_relative_path: CAPABILITY_PATH.into(), + host_gateway_address: routed_host_ipv4(), + }); + let result = publish_prepared_context_metadata( + root, + provider, + &context, + opencode_input.secret_references, + ); + if let Err(error) = result { + let _ = std::fs::remove_dir_all(root); + return Err(error); + } + } + } let grant = Grant { preparation: Some(InputPreparation { phase: InputPreparationPhase::Pending, @@ -1292,6 +1500,78 @@ impl CapabilityStore { }) } + fn pending_prepared_context( + &self, + soul: &SoulId, + directory: &Path, + endpoint: &str, + provider: &str, + prior: Option<&freshell_runtime_protocol::ProviderLaunchContext>, + ) -> Result, String> { + let candidate = { + let grants = self.grants.lock().unwrap(); + grants + .iter() + .find(|(_, grant)| { + grant.soul_id == *soul + && grant.preparation.as_ref().is_some_and(|prepared| { + prepared.phase != InputPreparationPhase::Committed + && !prepared.prior_grant_ids.is_empty() + }) + }) + .map(|(id, grant)| { + ( + id.clone(), + grant.endpoint.clone(), + grant.preparation.as_ref().unwrap().prior_grant_ids.clone(), + ) + }) + }; + let Some((id, pending_endpoint, prior_ids)) = candidate else { + return Ok(None); + }; + if pending_endpoint != endpoint + || prior + .and_then(|context| context.mcp_capability.as_ref()) + .is_some_and(|reference| !prior_ids.contains(&reference.grant_id)) + { + return Err( + "pending complete provider preparation requires exact ownership reconciliation" + .into(), + ); + } + let stage = staged_provider_root_path(directory, &id)?; + let path = stage.join(".freshell/prepared-result.json"); + if !path.exists() { + return Ok(None); + } + use std::io::Read; + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "pending prepared result unavailable")? + .take((freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + 1) as u64) + .read_to_end(&mut bytes) + .map_err(|_| "pending prepared result read failed")?; + if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err("pending prepared result exceeds metadata profile".into()); + } + let result: freshell_runtime_protocol::PreparedProviderContext = + serde_json::from_slice(&bytes) + .map_err(|_| "pending prepared result metadata invalid")?; + result + .validate(provider) + .map_err(|_| "pending prepared result metadata invalid")?; + if result + .provider_launch_context + .mcp_capability + .as_ref() + .is_none_or(|capability| capability.grant_id != id || capability.endpoint != endpoint) + { + return Err("pending prepared result changed generation".into()); + } + Ok(Some(result)) + } + fn issue_replacement( &self, soul_id: SoulId, @@ -1300,6 +1580,27 @@ impl CapabilityStore { source: Option<(&str, &Path, &[ProviderConfigReference])>, tui_source: Option<&ProviderConfigReference>, workspace: Option<&Path>, + ) -> Result { + self.issue_replacement_with_inputs( + soul_id, + directory, + endpoint, + source, + OpencodeEphemeralInput { + tui_source, + workspace, + ..Default::default() + }, + ) + } + + fn issue_replacement_with_inputs( + &self, + soul_id: SoulId, + directory: &Path, + endpoint: String, + source: Option<(&str, &Path, &[ProviderConfigReference])>, + input: OpencodeEphemeralInput<'_>, ) -> Result { if let Some((grant_id, grant)) = self.grants.lock().unwrap().iter().find(|(_, grant)| { grant.soul_id == soul_id @@ -1332,11 +1633,7 @@ impl CapabilityStore { directory, endpoint, source, - OpencodeEphemeralInput { - tui_source, - workspace, - ..Default::default() - }, + input, prior_stage.as_deref(), )?; let prior = self @@ -1548,6 +1845,17 @@ pub(crate) fn issue_prepared_opencode_context( input, None, )?; + let result = read_prepared_context(&directory, &capability); + if result.is_err() { + let _ = store().revoke(&capability.grant_id, &directory); + } + result +} + +fn read_prepared_context( + directory: &Path, + capability: &McpCapabilityReference, +) -> Result { let result = (|| { use std::io::Read; let stage = staged_provider_root_path(&directory, &capability.grant_id)?; @@ -1563,14 +1871,11 @@ pub(crate) fn issue_prepared_opencode_context( let context: freshell_runtime_protocol::ProviderLaunchContext = serde_json::from_slice(&bytes) .map_err(|_| "complete prepared provider context metadata invalid")?; - if context.mcp_capability.as_ref() != Some(&capability) { + if context.mcp_capability.as_ref() != Some(capability) { return Err("complete prepared context changed generation before publication".into()); } Ok(context) })(); - if result.is_err() { - let _ = store().revoke(&capability.grant_id, &directory); - } result } @@ -2828,6 +3133,191 @@ except Exception as error: .contains("nested-secret-byte")); } + #[tokio::test] + async fn managed_provider_replacement_complete_codex_metadata_replays_before_rescanning_roots() + { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".codex/skills/owned")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + std::fs::write(home.join(".codex/config.toml"), "model = 'synthetic'\n").unwrap(); + std::fs::write( + home.join(".codex/skills/owned/runner.sh"), + "synthetic retained resource", + ) + .unwrap(); + let mut context = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "codex", &workspace, &home, None, + ) + .unwrap() + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", &home, &context.config)), + Default::default(), + None, + ) + .unwrap(); + context.mcp_capability = Some(prior); + let callback = serde_json::to_vec( + &serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul, + "provider":"codex","workspace":workspace,"cwd":workspace,"priorContext":context, + "providerSecretReferences":[],"authLookupAuthority":"fixture-owned-attempt"}), + ) + .unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let first = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(); + std::fs::remove_file(home.join(".codex/config.toml")).unwrap(); + std::fs::write(home.join(".codex/new.config.toml"), "model = 'changed'\n").unwrap(); + let repeated = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(); + assert_eq!(serde_json::from_slice::(&repeated).unwrap(),serde_json::from_slice::(&first).unwrap(), + "a complete pending context must retain its exact selected root metadata in every provider route"); + let ready: freshell_runtime_protocol::PreparedProviderContext = + serde_json::from_slice(&repeated).unwrap(); + let stage = staged_provider_root_path( + &directory, + &ready + .provider_launch_context + .mcp_capability + .unwrap() + .grant_id, + ) + .unwrap(); + assert_eq!( + std::fs::read_to_string(stage.join("skills/owned/runner.sh")).unwrap(), + "synthetic retained resource" + ); + assert_eq!( + std::fs::read_dir(owned.path().join("control/provider-roots")) + .unwrap() + .count(), + 2 + ); + } + + #[tokio::test] + async fn managed_provider_replacement_callback_returns_complete_source_context() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let config = home.join(".config/opencode"); + std::fs::create_dir_all(&config).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + std::fs::write( + config.join("opencode.json"), + r#"{"agent":{"probe":{"prompt":"{file:rules}"}}}"#, + ) + .unwrap(); + std::fs::write(config.join("rules"), "PRIVATE_REPLACEMENT_SOURCE_NONCE").unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let prior_references = provider_source_references("opencode", &home).unwrap(); + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &prior_references)), + Default::default(), + None, + ) + .unwrap(); + let callback = serde_json::to_vec(&serde_json::json!({ + "action":"issue", "controlSecret":"fixture-control", "soulId":soul, + "provider":"opencode", "workspace":workspace, + "authLookupAuthority":"fixture-owned-attempt", "providerSecretReferences":[] + })) + .unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let reply = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(); + let reply: serde_json::Value = serde_json::from_slice(&reply).unwrap(); + assert!(reply["providerLaunchContext"]["preparedInputs"].is_object(), + "replacement must deliver the complete prepared source context before registry/Docker publication"); + assert_eq!( + reply["providerSecretReferences"].as_array().unwrap().len(), + 1 + ); + assert!( + staged_provider_root_path(&directory, &prior.grant_id) + .unwrap() + .is_dir(), + "an unaccepted replacement must preserve the prior projection" + ); + assert!(!String::from_utf8(serde_json::to_vec(&reply).unwrap()) + .unwrap() + .contains("PRIVATE_REPLACEMENT_SOURCE_NONCE")); + // A lost metadata response retries the same pending generation. It + // must not rescan changed originals and mix new references with it. + std::fs::remove_file(config.join("opencode.json")).unwrap(); + let repeated = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .expect("a pending complete preparation must replay its accepted metadata"); + assert_eq!( + serde_json::from_slice::(&repeated).unwrap(), + reply + ); + assert_eq!( + std::fs::read_dir(owned.path().join("control/provider-roots")) + .unwrap() + .count(), + 2 + ); + } + #[test] fn managed_provider_source_preparation_publishes_complete_descriptor_before_capability() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 5d630b87f..7ca81f91e 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -702,17 +702,48 @@ pub(crate) struct PreparedManagedOpencode { /// Terminal, fresh and scoped replacement callers share this one preparation. /// The original lookup and decoded grants never enter the returned metadata. +pub(crate) struct OpencodePreparation<'a> { + pub client: &'a freshell_runtime_client::RuntimeClient, + pub soul: &'a freshell_runtime_protocol::SoulId, + pub creation_seed: &'a str, + pub home: &'a Path, + pub cwd: &'a Path, + pub workspace: &'a Path, + pub original: &'a (dyn Fn(&str) -> Option + Sync), + pub references: Vec, + pub auth_authority: Option<&'a str>, +} + pub(crate) async fn prepare_managed_opencode( - client: &freshell_runtime_client::RuntimeClient, - soul: &freshell_runtime_protocol::SoulId, - creation_seed: &str, - home: &Path, - cwd: &Path, - workspace: &Path, - original: &(dyn Fn(&str) -> Option + Sync), - mut references: Vec, - auth_authority: Option<&str>, + request: OpencodePreparation<'_>, +) -> Result { + prepare_managed_opencode_with_publisher( + request, + crate::managed_mcp_capability::issue_prepared_opencode_context, + ) + .await +} + +pub(crate) async fn prepare_managed_opencode_with_publisher( + request: OpencodePreparation<'_>, + publish: impl FnOnce( + &freshell_runtime_protocol::SoulId, + &Path, + &freshell_runtime_protocol::ProviderLaunchContext, + crate::managed_mcp_capability::OpencodeEphemeralInput<'_>, + ) -> Result, ) -> Result { + let OpencodePreparation { + client, + soul, + creation_seed, + home, + cwd, + workspace, + original, + mut references, + auth_authority, + } = request; use sources::*; if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) || original("OPENCODE_TUI_CONFIG").is_some_and(|value| !value.is_empty()) @@ -820,7 +851,7 @@ pub(crate) async fn prepare_managed_opencode( } freshell_runtime_protocol::validate_provider_secrets("opencode", &references) .map_err(|error| error.message)?; - let context = crate::managed_mcp_capability::issue_prepared_opencode_context( + let context = publish( soul, home, &base, @@ -839,12 +870,14 @@ pub(crate) async fn prepare_managed_opencode( .mcp_capability .as_ref() .ok_or("prepared source capability unavailable")?; - if let Err(error) = client - .bind_provider_path_inputs(soul.clone(), capability.grant_id.clone(), frozen) - .await - { - let _ = crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id); - return Err(error.to_string()); + if !frozen.is_empty() { + if let Err(error) = client + .bind_provider_path_inputs(soul.clone(), capability.grant_id.clone(), frozen) + .await + { + let _ = crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id); + return Err(error.to_string()); + } } context .prepared_inputs diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index 005cd3030..e0561639f 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -92,7 +92,7 @@ impl SourceEnvironment { .cloned() } pub fn needs_native_auth(&self) -> bool { - self.auth.is_none() + self.auth.is_none() && !self.names.is_empty() } pub fn names(&self) -> &[String] { &self.names diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 25c870a42..c4b4677fb 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -231,15 +231,17 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .or_else(|| std::env::var(name).ok()) }; let prepared = crate::managed_provider_references::prepare_managed_opencode( - &self.client, - &soul_id, - create_key, - &home, - &cwd, - &workspace, - &original, - provider_secret_references, - None, + crate::managed_provider_references::OpencodePreparation { + client: &self.client, + soul: &soul_id, + creation_seed: create_key, + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &original, + references: provider_secret_references, + auth_authority: None, + }, ) .await?; provider_secret_references = prepared.references; diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index d10cfa043..4733c18e1 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -335,6 +335,146 @@ async fn provider_native_auth_nested_callback_uses_current_attempt_without_reloc _ => panic!("unexpected auth reply"), } assert_eq!(backend.0.load(Ordering::SeqCst), 1); + // Exercise the real callback and private control sockets while the same + // soul's lifecycle lock is held. The nested lookup must use its scoped + // attempt and the response must retain one complete generation/vector. + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + let control_task = tokio::spawn({ + let supervisor = supervisor.clone(); + let path = supervisor.config.control_socket_path.clone(); + async move { crate::service::serve_control(supervisor, &path).await } + }); + tokio::time::timeout(std::time::Duration::from_secs(5), async { + while !supervisor.config.control_socket_path.exists() { + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + }) + .await + .unwrap(); + let callback_listener = tokio::net::UnixListener::bind( + supervisor + .config + .control_socket_path + .with_file_name("mcp.sock"), + ) + .unwrap(); + let control_socket = supervisor.config.control_socket_path.clone(); + let epoch = supervisor.registry.control_epoch(); + let callback_soul = soul.clone(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let reference = McpCapabilityReference { + grant_id: generation.clone(), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + config: vec![], + mcp_capability: Some(reference.clone()), + prepared_inputs: None, + }; + let mut reply_context = context.clone(); + reply_context.prepared_inputs = Some(PreparedProviderInputs { + generation_id: generation.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 6, + alias_count: 6, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: Some(IndexedProviderFiles { + volume_name: indexed_provider_volume_name(&soul, &generation), + aliases: (0..6) + .map(|index| IndexedProviderFileAlias { + source_id: index, + logical_destination: format!("/fixture/source-{index}"), + effective_destination: format!("/fixture/source-{index}"), + }) + .collect(), + }), + }); + let references = (0..6) + .map(|index| ProviderSecretReference { + source_path: format!("/private/source-{index}"), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{index}")), + }) + .collect::>(); + let reply = PreparedProviderContext { + provider_launch_context: reply_context, + provider_secret_references: references.clone(), + }; + let callback_task = tokio::spawn(async move { + let (mut stream, _) = callback_listener.accept().await.unwrap(); + let mut bytes = Vec::new(); + (&mut stream) + .take((MAX_CONTROL_FRAME_BYTES + 1) as u64) + .read_to_end(&mut bytes) + .await + .unwrap(); + let request: serde_json::Value = serde_json::from_slice(&bytes).unwrap(); + assert_eq!(request["soulId"], callback_soul.as_str()); + assert_eq!(request["cwd"], "/fixture/instance"); + assert!(request["priorContext"].is_object()); + let mut control = tokio::net::UnixStream::connect(control_socket) + .await + .unwrap(); + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::NativeAuthLookup(NativeAuthLookupRequest { + expected_control_epoch: Some(epoch), + soul_id: callback_soul, + attempt_authority: request["authLookupAuthority"].as_str().unwrap().into(), + keys: vec!["PATH_INPUT".into()], + }), + ); + envelope.auth = Some(secret.into()); + write_frame(&mut control, &envelope).await.unwrap(); + let response: AdminReply = read_frame(&mut control).await.unwrap(); + let AdminResult::NativeAuthLookup(values) = response.result.unwrap() else { + panic!("unexpected nested private RPC result") + }; + assert_eq!(values.0["PATH_INPUT"], "/synthetic/selected"); + let bytes = bounded_json_bytes(&reply, MAX_CONTROL_FRAME_BYTES).unwrap(); + stream.write_all(&bytes).await.unwrap(); + }); + let returned = tokio::time::timeout( + std::time::Duration::from_secs(5), + supervisor.issue_replacement_mcp_capability( + &start, + "opencode", + &context, + &[], + "/fixture/instance", + "/fixture/workspace", + ), + ) + .await + .expect("complete callback lookup reacquired the held soul lock") + .unwrap(); + assert_eq!(returned.provider_secret_references, references); + assert_eq!( + returned + .provider_launch_context + .prepared_inputs + .unwrap() + .generation_id, + generation + ); + callback_task.await.unwrap(); + control_task.abort(); + assert_eq!(backend.0.load(Ordering::SeqCst), 2); assert!(supervisor .dispatch(request(SoulId::new(), key.clone())) .await @@ -364,7 +504,7 @@ async fn provider_native_auth_nested_callback_uses_current_attempt_without_reloc .is_err()); assert_eq!( backend.0.load(Ordering::SeqCst), - 1, + 2, "expired/foreign/stopped attempt must never inspect auth" ); } diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index 25f08682b..1559a8f16 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -572,6 +572,7 @@ impl Supervisor { start.context.terminal.clone() }; let mut replacement_terminal = replacement_terminal; + let mut replacement_fresh = fresh_without_raw_bootstrap(start.context.fresh_agent.clone()); if replacement_terminal.as_ref().is_some_and(|terminal| { matches!( terminal.mode.as_str(), @@ -610,43 +611,45 @@ impl Supervisor { incident_id: None, }); } - if let Some(terminal) = replacement_terminal.as_mut() { - if terminal - .provider_launch_context - .as_ref() - .and_then(|context| context.mcp_capability.as_ref()) - .is_some() - { - let tui_source = terminal - .provider_launch_context - .as_ref() - .and_then(|context| { - if let freshell_runtime_protocol::ProviderPreparation::Opencode { - tui_source, - .. - } = &context.preparation - { - tui_source.as_ref() - } else { - None - } - }); - let refreshed_capability = self - .issue_replacement_mcp_capability(&start, &terminal.mode, tui_source, &terminal.workspace_path) - .await - .and_then(|reference| { - crate::backend::docker::staged_provider_root_for_grant( - &reference.grant_id, + let replacement_context = replacement_terminal + .as_ref() + .and_then(|terminal| { + terminal.provider_launch_context.as_ref().map(|context| { + ( + terminal.mode.clone(), + context.clone(), + terminal.provider_secret_references.clone(), + terminal.cwd.clone(), + terminal.workspace_path.clone(), + ) + }) + }) + .or_else(|| { + replacement_fresh.as_ref().and_then(|agent| { + agent.provider_launch_context.as_ref().map(|context| { + ( + agent.provider.as_str().to_owned(), + context.clone(), + agent.provider_secret_references.clone(), + agent.cwd.clone(), + agent.workspace_path.clone(), ) - .map(|staged_root| (reference, staged_root)) - .map_err(|error| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - format!("capability_pending: staged provider root unavailable: {error}"), - ) - }) - }); - let (reference, staged_root) = match refreshed_capability { + }) + }) + }); + if let Some((provider, context, references, cwd, workspace)) = replacement_context { + if context.mcp_capability.is_some() { + let refreshed_capability = self + .issue_replacement_mcp_capability( + &start, + &provider, + &context, + &references, + &cwd, + &workspace, + ) + .await; + let prepared_context = match refreshed_capability { Ok(refreshed) => refreshed, Err(error) => { let reason = RecoveryBlockReason::CapabilityPending; @@ -673,8 +676,11 @@ impl Supervisor { }); } }; - replace_terminal_mcp_reference(terminal, reference); - refresh_terminal_provider_config(terminal, &staged_root); + install_prepared_provider_context( + replacement_terminal.as_mut(), + replacement_fresh.as_mut(), + prepared_context, + )?; } } let prepared = match self @@ -682,7 +688,7 @@ impl Supervisor { .prepare_replacement( start.clone(), replacement_terminal, - fresh_without_raw_bootstrap(start.context.fresh_agent.clone()), + replacement_fresh, resume_spec.clone(), path, ) @@ -2097,6 +2103,32 @@ fn replace_terminal_mcp_reference( } } +fn install_prepared_provider_context( + terminal: Option<&mut TerminalLaunchSpec>, + fresh: Option<&mut freshell_runtime_protocol::FreshAgentLaunchSpec>, + prepared: freshell_runtime_protocol::PreparedProviderContext, +) -> Result<(), RuntimeError> { + match (terminal, fresh) { + (Some(terminal), None) => { + prepared.validate(&terminal.mode)?; + terminal.provider_launch_context = Some(prepared.provider_launch_context); + terminal.provider_secret_references = prepared.provider_secret_references; + } + (None, Some(agent)) => { + prepared.validate(agent.provider.as_str())?; + agent.provider_launch_context = Some(prepared.provider_launch_context); + agent.provider_secret_references = prepared.provider_secret_references; + } + _ => { + return Err(RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "complete provider replacement requires exactly one workload", + )) + } + } + Ok(()) +} + fn refresh_provider_context_from_user_root( context: &mut freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -2436,11 +2468,81 @@ mod tests { ); let reference = replacement .provider_launch_context + .as_ref() .unwrap() .mcp_capability + .as_ref() .unwrap(); assert_eq!(reference.grant_id, "grant-new"); assert_eq!(reference.endpoint, "http://host.docker.internal:4001"); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let references = (0..6) + .map(|index| freshell_runtime_protocol::ProviderSecretReference { + source_path: format!("/private/current-source-{index}"), + profile: + freshell_runtime_protocol::ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{index}")), + }) + .collect::>(); + let mut context = replacement + .provider_launch_context + .as_ref() + .unwrap() + .clone(); + context.mcp_capability.as_mut().unwrap().grant_id = generation.clone(); + context.prepared_inputs = Some(freshell_runtime_protocol::PreparedProviderInputs { + generation_id: generation.clone(), + initial_auth_reservation: None, + manifest: freshell_runtime_protocol::ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 6, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: Some(freshell_runtime_protocol::IndexedProviderFiles { + volume_name: freshell_runtime_protocol::indexed_provider_volume_name( + &SoulId::new(), + &generation, + ), + aliases: vec![], + }), + }); + let complete = freshell_runtime_protocol::PreparedProviderContext { + provider_launch_context: context, + provider_secret_references: references.clone(), + }; + let mut fresh:freshell_runtime_protocol::FreshAgentLaunchSpec=serde_json::from_value(serde_json::json!({ + "sessionId":"fresh-replacement","provider":"opencode","sessionType":"freshopencode","runtimeVariant":"opencode-serve", + "providerStoreId":"store","cwd":"/workspace","workspacePath":"/workspace","runAsUid":65534,"runAsGid":0 + })).unwrap(); + install_prepared_provider_context(Some(&mut replacement), None, complete.clone()).unwrap(); + install_prepared_provider_context(None, Some(&mut fresh), complete).unwrap(); + assert_eq!(replacement.provider_secret_references, references); + assert_eq!(fresh.provider_secret_references, references); + assert_eq!( + replacement + .provider_launch_context + .unwrap() + .prepared_inputs + .unwrap() + .generation_id, + generation + ); + assert_eq!( + fresh + .provider_launch_context + .unwrap() + .prepared_inputs + .unwrap() + .generation_id, + generation + ); + assert_eq!(replacement.program, terminal.program); + assert_eq!(replacement.args, terminal.args); + assert!(replacement.provider_bootstrap_files.is_empty()); } #[test] diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 69d887ebd..c1e8d034d 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -61,6 +61,9 @@ struct McpCapabilityRequest<'a> { tui_source: Option<&'a freshell_runtime_protocol::ProviderConfigReference>, workspace: Option<&'a str>, auth_lookup_authority: Option<&'a str>, + cwd: Option<&'a str>, + prior_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, + provider_secret_references: Option<&'a [freshell_runtime_protocol::ProviderSecretReference]>, } #[derive(Clone)] @@ -1974,6 +1977,9 @@ impl Supervisor { tui_source: None, workspace: None, auth_lookup_authority: None, + cwd: None, + prior_context: None, + provider_secret_references: None, }) .await?; if response != b"ok" { @@ -2043,9 +2049,17 @@ impl Supervisor { &self, start: &crate::registry::RecoveryStart, provider: &str, - tui_source: Option<&freshell_runtime_protocol::ProviderConfigReference>, + context: &freshell_runtime_protocol::ProviderLaunchContext, + references: &[freshell_runtime_protocol::ProviderSecretReference], + cwd: &str, workspace: &str, - ) -> Result { + ) -> Result { + let tui_source = match &context.preparation { + freshell_runtime_protocol::ProviderPreparation::Opencode { tui_source, .. } => { + tui_source.as_ref() + } + _ => None, + }; let scope = self.auth_lookup_scope(start); let response = self .mcp_capability_callback(McpCapabilityRequest { @@ -2057,14 +2071,20 @@ impl Supervisor { tui_source, workspace: Some(workspace), auth_lookup_authority: Some(&scope.key), + cwd: Some(cwd), + prior_context: Some(context), + provider_secret_references: Some(references), }) .await?; - serde_json::from_slice(&response).map_err(|_| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - "capability_pending: server returned an invalid scoped grant reference", - ) - }) + let prepared: freshell_runtime_protocol::PreparedProviderContext = + serde_json::from_slice(&response).map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "capability_pending: server returned an invalid scoped grant reference", + ) + })?; + prepared.validate(provider)?; + Ok(prepared) } async fn mcp_capability_callback( @@ -2088,6 +2108,9 @@ impl Supervisor { "tuiSource": request.tui_source, "workspace": request.workspace, "authLookupAuthority": request.auth_lookup_authority, + "cwd":request.cwd, + "priorContext":request.prior_context, + "providerSecretReferences":request.provider_secret_references, }), freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, ) From 531136a49bf67afcb4ba1f107d549782368aeec8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 16:13:07 -0700 Subject: [PATCH 046/493] Preserve accepted source origins and reject generated auth collisions --- .../src/native_auth_exchange.rs | 12 + .../freshell-platform/src/opencode_config.rs | 6 + .../src/provider_environment_grants.rs | 21 ++ .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 235 ++++++++++++++++++ .../src/managed_provider_references.rs | 25 +- .../environment.rs | 47 +++- crates/freshell-server/src/managed_runtime.rs | 1 + 8 files changed, 337 insertions(+), 11 deletions(-) diff --git a/crates/freshell-platform/src/native_auth_exchange.rs b/crates/freshell-platform/src/native_auth_exchange.rs index 24012e3a4..f1a1625d9 100644 --- a/crates/freshell-platform/src/native_auth_exchange.rs +++ b/crates/freshell-platform/src/native_auth_exchange.rs @@ -135,6 +135,18 @@ mod tests { os::unix::net::UnixListener, }; + #[test] + fn native_auth_reserved_generated_key_cannot_escape_the_finite_lookup_filter() { + let owned = tempfile::tempdir().unwrap(); + let path = owned.path().join(".local/share/opencode/auth.json"); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + let raw = r#"{"shadow":{"type":"wellknown","key":"FRESHELL_CONFIG_REFERENCE_0","token":"PRIVATE_NATIVE_SHADOW_SENTINEL"},"selected":{"type":"wellknown","key":"PATH_INPUT","token":"/synthetic/selected"}}"#; + std::fs::write(&path, raw).unwrap(); + let error = read_selected_native_auth(owned.path(), &["PATH_INPUT".into()]).unwrap_err(); + assert!(!error.contains("PRIVATE_NATIVE_SHADOW_SENTINEL")); + assert_eq!(std::fs::read_to_string(path).unwrap(), raw); + } + #[test] fn selected_native_auth_values_travel_only_through_the_private_exchange() { let temp = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index 1268acd74..84bf63768 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -6,6 +6,12 @@ use serde_json::Value; use std::path::{Path, PathBuf}; +pub const GENERATED_CONFIG_REFERENCE_PREFIX: &str = "FRESHELL_CONFIG_REFERENCE_"; +pub const MAX_GENERATED_CONFIG_REFERENCES: usize = 128; +pub fn generated_config_reference_name(ordinal: usize) -> String { + format!("{GENERATED_CONFIG_REFERENCE_PREFIX}{ordinal}") +} + #[derive(Clone, Debug, PartialEq, Eq)] pub struct ConfigSource { pub path: PathBuf, diff --git a/crates/freshell-platform/src/provider_environment_grants.rs b/crates/freshell-platform/src/provider_environment_grants.rs index f46c2de49..42f49577b 100644 --- a/crates/freshell-platform/src/provider_environment_grants.rs +++ b/crates/freshell-platform/src/provider_environment_grants.rs @@ -584,6 +584,13 @@ pub fn resolve_opencode_auth_environment( let Some(key) = value.get("key").and_then(serde_json::Value::as_str) else { continue; }; + if key.starts_with(crate::opencode_config::GENERATED_CONFIG_REFERENCE_PREFIX) { + let mut error = + fail("well-known auth key occupies the reserved generated configuration namespace"); + error.field = Some("auth.wellknown.key"); + error.source_ordinal = Some(ordinal); + return Err(error); + } if !needed.iter().any(|name| name == key) { continue; } @@ -650,6 +657,20 @@ mod tests { .unwrap_err(); assert!(!format!("{err:?} {err}").contains("sentinel-private")); } + #[test] + fn auth_generated_namespace_collision_is_checked_before_selected_value_filtering() { + let raw = r#"{"shadow":{"type":"wellknown","key":"FRESHELL_CONFIG_REFERENCE_0","token":"PRIVATE_GENERATED_SHADOW_SENTINEL"},"selected":{"type":"wellknown","key":"RULE_ROOT","token":"/synthetic/rules"}}"#; + let error=resolve_opencode_auth_environment(raw,&["RULE_ROOT".into()]) + .expect_err("native authEnv must not shadow child-only generated config variables even when that auth key was not selected"); + assert!(!format!("{error:?} {error}").contains("PRIVATE_GENERATED_SHADOW_SENTINEL")); + // JS duplicate-property semantics apply before namespace inspection; + // the obsolete entry contributes neither an overlay nor a collision. + let replaced = r#"{"shadow":{"type":"wellknown","key":"FRESHELL_CONFIG_REFERENCE_0","token":"PRIVATE_GENERATED_SHADOW_SENTINEL"},"shadow":{"type":"api","key":"synthetic-api"},"selected":{"type":"wellknown","key":"RULE_ROOT","token":"/synthetic/rules"}}"#; + let values = resolve_opencode_auth_environment(replaced, &["RULE_ROOT".into()]).unwrap(); + assert_eq!(values.len(), 1); + assert_eq!(values["RULE_ROOT"], "/synthetic/rules"); + } + #[test] fn selected_native_auth_lookup_keeps_js_duplicate_key_order_and_excludes_unrequested_values() { let raw = r#"{"first":{"type":"wellknown","key":"CA_PATH","token":"first-path"},"second":{"type":"wellknown","key":"CA_PATH","token":"second-path"},"first":{"type":"wellknown","key":"CA_PATH","token":"last-first-path"},"unrequested":{"type":"wellknown","key":"PRIVATE_UNUSED","token":"sentinel-private-unused"}}"#; diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 71af97ca1..42878031b 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -467,6 +467,7 @@ impl HostedFreshAgentProxy { original: &original, references: provider_secret_references, auth_authority: None, + selected_sources: None, }, ) .await diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 460642edc..2965bc38f 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -330,6 +330,16 @@ async fn apply_prepared_callback( ) .map_err(|_| "replacement provider context exceeds control metadata profile".into()); } + let accepted_manifest = read_prior_source_manifest( + store, + directory, + &callback.soul_id, + callback.prior_context.as_ref(), + )?; + let home = accepted_manifest + .as_ref() + .map(|manifest| manifest.original_home.as_path()) + .unwrap_or(home); let workspace = callback .workspace .as_deref() @@ -342,6 +352,17 @@ async fn apply_prepared_callback( .filter(|reference| !reference.profile.is_configuration_file()) .collect(); let original = |name: &str| std::env::var(name).ok(); + let selected_sources=accepted_manifest.as_ref().map(|manifest|{ + manifest.sources.iter().enumerate().map(|(ordinal,source)|{ + let canonical=std::fs::canonicalize(&source.logical_path) + .map_err(|_|format!("prior selected configuration source {ordinal} is unavailable; repair it before reissue"))?; + Ok(crate::managed_provider_references::sources::SourceOccurrence { + logical_path:source.logical_path.clone(),canonical_source:canonical, + lane:match source.lane {freshell_platform::opencode_recipe::SourceLane::Main=>crate::managed_provider_references::sources::SourceLane::Main, + freshell_platform::opencode_recipe::SourceLane::Tui=>crate::managed_provider_references::sources::SourceLane::Tui}, + }) + }).collect::,String>>() + }).transpose()?; let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( crate::managed_provider_references::OpencodePreparation { client, @@ -358,6 +379,7 @@ async fn apply_prepared_callback( .as_deref() .ok_or("replacement provider preparation requires attempt authority")?, ), + selected_sources, }, |soul, home, context, input| { let input = OpencodeEphemeralInput { @@ -389,6 +411,67 @@ async fn apply_prepared_callback( .map_err(|_| "replacement provider context exceeds control metadata profile".into()) } +fn read_prior_source_manifest( + store: &CapabilityStore, + directory: &Path, + soul: &SoulId, + context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, +) -> Result, String> { + let Some(context) = context else { + return Ok(None); + }; + context + .validate("opencode") + .map_err(|_| "prior provider source context metadata invalid")?; + let Some(inputs) = context.prepared_inputs.as_ref() else { + return Ok(None); + }; + let capability = context + .mcp_capability + .as_ref() + .ok_or("prior source context lacks generation authority")?; + let owned = store + .grants + .lock() + .unwrap() + .get(&capability.grant_id) + .is_some_and(|grant| grant.soul_id == *soul); + if !owned { + return Err("prior provider source context belongs to another owner".into()); + } + let root = staged_provider_root_path(directory, &capability.grant_id)?; + let path = root.join(&inputs.manifest.relative_path); + if !std::fs::symlink_metadata(&path) + .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) + { + return Err("prior source manifest must be a physical private file".into()); + } + use std::io::Read; + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "prior source manifest unavailable")? + .take(64 * 1024 * 1024 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "prior source manifest read failed")?; + if bytes.len() > 64 * 1024 * 1024 + || format!("{:x}", sha2::Sha256::digest(&bytes)) != inputs.manifest.sha256 + { + return Err("prior source manifest profile or identity changed".into()); + } + let manifest: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice(&bytes).map_err(|_| "prior source manifest metadata invalid")?; + if manifest.version != 1 + || manifest.sources.len() > 8192 + || manifest.files.len() > 128 + || manifest.frozen_inputs.len() > 128 + || !manifest.original_home.is_absolute() + || !manifest.cwd.is_absolute() + { + return Err("prior source manifest origin profile invalid".into()); + } + Ok(Some(manifest)) +} + fn apply_callback( bytes: &[u8], expected_control_secret: &str, @@ -3133,6 +3216,158 @@ except Exception as error: .contains("nested-secret-byte")); } + #[tokio::test] + async fn managed_provider_replacement_keeps_accepted_original_home_origin() { + use crate::managed_provider_references::sources::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("accepted-home"); + let later_home = owned.path().join("later-controller-home"); + let workspace = owned.path().join("workspace"); + let config = home.join(".config/opencode/opencode.json"); + std::fs::create_dir_all(config.parent().unwrap()).unwrap(); + std::fs::create_dir_all(later_home.join(".config/opencode")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let raw = r#"{"agent":{"probe":{"prompt":"{file:rules}"}}}"#; + std::fs::write(&config, raw).unwrap(); + let rules = config.parent().unwrap().join("rules"); + std::fs::write(&rules, "ACCEPTED_HOME_RESOURCE_NONCE").unwrap(); + let explicit = home.join("selected-explicit/custom.jsonc"); + std::fs::create_dir_all(explicit.parent().unwrap()).unwrap(); + let explicit_raw = r#"{"agent":{"extra":{"prompt":"{file:rules}"}}}"#; + std::fs::write(&explicit, explicit_raw).unwrap(); + let explicit_rules = explicit.parent().unwrap().join("rules"); + std::fs::write(&explicit_rules, "EXPLICIT_ORIGIN_RESOURCE_NONCE").unwrap(); + let plan = plan_opencode_sources( + vec![ + SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + }, + SourceOccurrence { + logical_path: explicit.clone(), + canonical_source: explicit.clone(), + lane: SourceLane::Main, + }, + ], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let context = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "opencode", &workspace, &home, None, + ) + .unwrap() + .unwrap(); + let references = [rules.clone(), explicit_rules.clone()] + .into_iter() + .enumerate() + .map( + |(index, path)| freshell_runtime_protocol::ProviderSecretReference { + source_path: path.to_str().unwrap().into(), + profile: + freshell_runtime_protocol::ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{index}")), + }, + ) + .collect::>(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &context.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + launch_context: Some(&context), + secret_references: Some(&references), + ..Default::default() + }, + None, + ) + .unwrap(); + let accepted = read_prepared_context(&directory, &prior).unwrap(); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + let manifest_path = prior_stage.join(".freshell/input-manifest.json"); + let manifest_bytes = std::fs::read(&manifest_path).unwrap(); + let manifest_before: serde_json::Value = serde_json::from_slice(&manifest_bytes).unwrap(); + let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul, + "provider":"opencode","workspace":workspace,"cwd":workspace,"priorContext":accepted,"providerSecretReferences":references, + "authLookupAuthority":"fixture-owned-attempt"})).unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let mut foreign: serde_json::Value = serde_json::from_slice(&callback).unwrap(); + foreign["soulId"] = serde_json::json!(SoulId::new()); + assert!( + apply_prepared_callback( + &serde_json::to_vec(&foreign).unwrap(), + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &later_home, + &client + ) + .await + .is_err(), + "another soul cannot select this private origin manifest" + ); + let mut corrupted = manifest_before; + corrupted["originalHome"] = serde_json::json!(later_home); + std::fs::write(&manifest_path, serde_json::to_vec(&corrupted).unwrap()).unwrap(); + assert!( + apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &later_home, + &client + ) + .await + .is_err(), + "origin metadata must retain its approved manifest identity" + ); + std::fs::write(&manifest_path, &manifest_bytes).unwrap(); + let reply = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &later_home, + &client, + ) + .await + .unwrap(); + let ready: freshell_runtime_protocol::PreparedProviderContext = + serde_json::from_slice(&reply).unwrap(); + assert_eq!(ready.provider_secret_references.len(),2,"reissue must keep accepted original HOME and ordered explicit sources instead of rediscovering the controller environment"); + assert_eq!( + ready.provider_secret_references[0].source_path, + rules.to_str().unwrap() + ); + assert_eq!( + ready.provider_secret_references[1].source_path, + explicit_rules.to_str().unwrap() + ); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + assert_eq!(std::fs::read_to_string(&explicit).unwrap(), explicit_raw); + assert!(staged_provider_root_path(&directory, &prior.grant_id) + .unwrap() + .is_dir()); + } + #[tokio::test] async fn managed_provider_replacement_complete_codex_metadata_replays_before_rescanning_roots() { diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 7ca81f91e..7f6c22df9 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -264,7 +264,8 @@ pub(super) fn resolve_file_references_from_view_relocated( let transformed_length = transformed.len(); drop(transformed); let (template, recipes) = if generated { - let generated_name = format!("FRESHELL_CONFIG_REFERENCE_{recipe_ordinal}"); + let generated_name = + freshell_platform::opencode_config::generated_config_reference_name(recipe_ordinal); if transformed_length > 64 * 1024 { return Err(fail( 0, @@ -712,6 +713,7 @@ pub(crate) struct OpencodePreparation<'a> { pub original: &'a (dyn Fn(&str) -> Option + Sync), pub references: Vec, pub auth_authority: Option<&'a str>, + pub selected_sources: Option>, } pub(crate) async fn prepare_managed_opencode( @@ -743,6 +745,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( original, mut references, auth_authority, + selected_sources, } = request; use sources::*; if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) @@ -771,14 +774,18 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( None, )? .ok_or("OpenCode provider preparation unavailable")?; - let selected = discover_opencode(OpencodeDiscovery { - home, - cwd, - worktree: workspace, - config_root: &config_root, - env: original, - }) - .map_err(str::to_string)?; + let selected = if let Some(sources) = selected_sources { + sources + } else { + discover_opencode(OpencodeDiscovery { + home, + cwd, + worktree: workspace, + config_root: &config_root, + env: original, + }) + .map_err(str::to_string)? + }; let loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; if loaded .iter() diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index e0561639f..824ecc95f 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -23,6 +23,14 @@ impl SourceEnvironment { freshell_runtime_protocol::validate_provider_secrets("opencode", references) .map_err(|error| error.message)?; freshell_platform::native_auth_exchange::validate_keys(&names).map_err(str::to_string)?; + // Probe only the finite internal names. Retain neither the unrelated + // original/request values nor an ambient environment snapshot. + for ordinal in 0..freshell_platform::opencode_config::MAX_GENERATED_CONFIG_REFERENCES { + let name = freshell_platform::opencode_config::generated_config_reference_name(ordinal); + if original(&name).is_some() { + return Err("original/request input occupies the reserved generated configuration namespace".into()); + } + } let mut ambient = BTreeMap::new(); let mut ambient_bytes = 0usize; for name in &names { @@ -165,7 +173,8 @@ impl SourceEnvironment { if matches!( name.as_str(), "AUTH_TOKEN" | "FRESHELL_TOKEN" | "FRESHELL_URL" - ) || name.starts_with("FRESHELL_CONFIG_REFERENCE_") + ) || name + .starts_with(freshell_platform::opencode_config::GENERATED_CONFIG_REFERENCE_PREFIX) { return Err("selected original source input collides with controller authentication or generated names".into()); } @@ -180,6 +189,37 @@ impl SourceEnvironment { #[cfg(test)] mod tests { use super::*; + #[test] + fn managed_provider_source_environment_rejects_unselected_typed_auth_generated_key() { + let owned = tempfile::tempdir().unwrap(); + let path = owned.path().join("auth.json"); + let raw = r#"{"shadow":{"type":"wellknown","key":"FRESHELL_CONFIG_REFERENCE_0","token":"PRIVATE_TYPED_SHADOW_SENTINEL"},"selected":{"type":"wellknown","key":"RULE_ROOT","token":"/synthetic/selected"}}"#; + std::fs::write(&path, raw).unwrap(); + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).unwrap(); + let references = [ProviderSecretReference { + source_path: path.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeOnecliAuthFile, + provider_relative_path: None, + }]; + let error = SourceEnvironment::read(vec!["RULE_ROOT".into()], &references, &|_| None) + .err() + .expect( + "typed auth must check native generated-name overlays before source publication", + ); + assert!(!error.contains("PRIVATE_TYPED_SHADOW_SENTINEL")); + assert_eq!(std::fs::read_to_string(path).unwrap(), raw); + } + + #[test] + fn managed_provider_source_environment_rejects_unselected_request_generated_namespace() { + let original = |name: &str| match name { + "FRESHELL_CONFIG_REFERENCE_0" => Some("PRIVATE_REQUEST_SHADOW_SENTINEL".into()), + _ => None, + }; + let error=SourceEnvironment::read(vec!["RULE_ROOT".into()],&[],&original).err() + .expect("an unselected original/request value must not occupy the generated configuration namespace"); + assert!(!error.contains("PRIVATE_REQUEST_SHADOW_SENTINEL")); + } use freshell_runtime_protocol::ProviderSecretProfile; use std::os::unix::fs::PermissionsExt; #[test] @@ -301,7 +341,10 @@ mod tests { let environment = SourceEnvironment::read( vec!["NODE_EXTRA_CA_CERTS".into(), "PATH_INPUT".into()], &references, - &|_| Some("/synthetic/ambient".into()), + &|name| { + matches!(name, "PATH_INPUT" | "NODE_EXTRA_CA_CERTS") + .then(|| "/synthetic/ambient".into()) + }, ) .unwrap(); assert_eq!( diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index c4b4677fb..4a1011966 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -241,6 +241,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { original: &original, references: provider_secret_references, auth_authority: None, + selected_sources: None, }, ) .await?; From 6d0e308406b6a3b99b00ca0cdb5ed119c393d14c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 16:23:17 -0700 Subject: [PATCH 047/493] fix: validate direct provider sources under document limits --- .../freshell-platform/src/opencode_recipe.rs | 17 +++++- .../src/provider_input_preparation.rs | 60 +++++++++++++++++-- 2 files changed, 70 insertions(+), 7 deletions(-) diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index 15d85bcbd..2ce1f79c7 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -65,9 +65,23 @@ pub struct SelectedSourceFile { pub canonical_source: PathBuf, } +/// Evaluate a generated child environment value under its 64 KiB value profile. pub fn evaluate( recipe: &GeneratedFileRecipe, env: &dyn Fn(&str) -> Option, +) -> Result { + let text = evaluate_document(recipe, env)?; + if text.len() > 64 * 1024 { + return Err("generated configuration exceeds 64 KiB child value limit"); + } + Ok(text) +} + +/// Validate a direct source using the transformer’s 16 MiB document profile. +/// Direct filesystem sources never become child environment values. +pub fn evaluate_document( + recipe: &GeneratedFileRecipe, + env: &dyn Fn(&str) -> Option, ) -> Result { use crate::opencode_config::{file_reference_path, transform_file_tokens, TokenTransformError}; let changed = "configuration reference selection changed; reprepare before dispatch"; @@ -94,8 +108,5 @@ pub fn evaluate( if observed != recipe.occurrences.len() { return Err(changed); } - if text.len() > 64 * 1024 { - return Err("generated configuration exceeds 64 KiB child value limit"); - } Ok(text) } diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 892b8a74d..bb1073410 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -143,8 +143,12 @@ pub fn prepare( .map(str::to_string) } }; - let value = freshell_platform::opencode_recipe::evaluate(&recipe, &lookup) - .map_err(str::to_string)?; + let value = if source.generated_name.is_some() { + freshell_platform::opencode_recipe::evaluate(&recipe, &lookup) + } else { + freshell_platform::opencode_recipe::evaluate_document(&recipe, &lookup) + } + .map_err(str::to_string)?; if let Some(name) = source.generated_name { if secrets.environment.contains_key(&name) || auth.contains_key(&name) @@ -188,7 +192,7 @@ mod tests { std::fs::create_dir(&home).unwrap(); let raw = r#"{"instructions":["{env:INSERT}"]}"#; std::fs::write(stage.join(".freshell/source-views/source-0"), raw).unwrap(); - let manifest = SourceManifest { + let mut manifest = SourceManifest { version: 1, cwd: "/original-unavailable/project".into(), original_home: "/original-unavailable/home".into(), @@ -222,7 +226,7 @@ mod tests { provider_relative_path: ".freshell/mcp-capability.json".into(), host_gateway_address: None, }; - let context = ProviderLaunchContext { + let mut context = ProviderLaunchContext { prepared_inputs: Some(PreparedProviderInputs { generation_id: capability.grant_id.clone(), initial_auth_reservation: None, @@ -278,5 +282,53 @@ mod tests { std::fs::read_to_string(stage.join(".freshell/source-views/source-0")).unwrap(), raw ); + // Direct sources are filesystem documents, not generated child env values. + // A complete accepted file reference remains valid in a source above 64 KiB. + let large_raw = format!("{}{}", " ".repeat(128 * 1024), raw); + std::fs::write(stage.join(".freshell/source-views/source-0"), &large_raw).unwrap(); + manifest.sources[0].generated_name = None; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .expect("direct source must use the 16 MiB document profile"); + assert!(generated.is_empty()); + assert_eq!( + std::fs::read(stage.join(".freshell/source-views/source-0")).unwrap(), + large_raw.as_bytes() + ); + let document_limit = 16 * 1024 * 1024; + let occurrences = std::mem::take(&mut manifest.sources[0].occurrences); + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let at_limit = format!("{}{{}}", " ".repeat(document_limit - 2)); + std::fs::write(stage.join(".freshell/source-views/source-0"), &at_limit).unwrap(); + assert!(prepare(Some(&context), &mut secrets, &frozen, &stage, &home).is_ok()); + std::fs::write( + stage.join(".freshell/source-views/source-0"), + format!(" {at_limit}"), + ) + .unwrap(); + assert!( + prepare(Some(&context), &mut secrets, &frozen, &stage, &home).is_err(), + "direct source growth above the 16 MiB profile must fail before dispatch" + ); + std::fs::write(stage.join(".freshell/source-views/source-0"), &large_raw).unwrap(); + + manifest.sources[0].occurrences = occurrences; + manifest.sources[0].generated_name = Some("FRESHELL_CONFIG_REFERENCE_0".into()); + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + assert!( + prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .unwrap_err() + .contains("64 KiB"), + "generated child values retain the declared stricter limit" + ); } } From 09e81890e7e8a85ebfeaa26269cf7621898337b0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 16:26:34 -0700 Subject: [PATCH 048/493] fix: share selected and generated provider input name limits --- .../environment.rs | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index 824ecc95f..65a9cfff6 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -168,6 +168,20 @@ impl SourceEnvironment { } } } + let generated_count = plan + .iter() + .map(|source| source.references.recipes.len()) + .try_fold(0usize, usize::checked_add) + .ok_or("selected/generated source name count overflow")?; + if selected + .len() + .checked_add(generated_count) + .is_none_or(|count| count > 128) + { + return Err( + "selected and generated source inputs exceed shared 128-name profile".into(), + ); + } let mut result = ProviderPathInputs::default(); for name in selected { if matches!( @@ -266,6 +280,64 @@ mod tests { .unexpanded_source() .contains("PRIVATE_PREFIX")); } + #[test] + fn managed_provider_source_environment_combines_selected_and_generated_name_budget() { + use super::super::sources::{LoadedSourceOccurrence, SourceOccurrence}; + let owned = tempfile::tempdir().unwrap(); + let rules = owned.path().join("rules"); + std::fs::write(&rules, "synthetic-budget-rule-nonce").unwrap(); + let names = (0..127) + .map(|index| format!("ROOT_{index}")) + .chain(std::iter::once("INSERT".into())) + .collect::>(); + let input = format!("{{file:{}}}", rules.display()); + let original = |name: &str| { + if name == "INSERT" { + Some(input.clone()) + } else if name.starts_with("ROOT_") { + Some(rules.to_str().unwrap().into()) + } else { + None + } + }; + let mut environment = SourceEnvironment::read(names, &[], &original).unwrap(); + environment.set_native_auth(BTreeMap::new()).unwrap(); + for selected_names in [127, 128] { + let mut fields = (0..selected_names - 1) + .map(|index| format!("{{file:{{env:ROOT_{index}}}}}")) + .collect::>(); + fields.push("{env:INSERT}".into()); + let raw = serde_json::json!({"instructions": fields}).to_string(); + let config = owned.path().join("opencode.json"); + let plan = environment + .plan_sources( + vec![LoadedSourceOccurrence { + source: SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + }, + view: std::sync::Arc::new(raw), + }], + owned.path(), + owned.path(), + ) + .unwrap(); + assert_eq!(plan[0].references.recipes.len(), 1); + let frozen = environment.frozen_inputs(&plan); + if selected_names == 127 { + assert_eq!( + frozen.unwrap().iter().count(), + 127, + "the shared profile must allow exactly 128 selected plus generated names" + ); + } else { + assert!(frozen.is_err(), + "128 selected names plus a generated name must fail before capability publication"); + } + } + } + #[test] fn managed_provider_source_environment_maps_known_runtime_ca_to_its_typed_source() { use super::super::sources::{LoadedSourceOccurrence, SourceOccurrence}; From c440a11f1d1c2606b815eadf554706d2b1e0ea03 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 16:35:15 -0700 Subject: [PATCH 049/493] fix: retain scoped provider path selections through replacement --- .../src/managed_mcp_capability.rs | 176 +++++++++++++++++- .../src/native_auth_lookup_tests.rs | 100 +++++++++- crates/freshell-supervisor/src/service.rs | 86 +++++++-- 3 files changed, 341 insertions(+), 21 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 2965bc38f..716799c1a 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -254,6 +254,8 @@ struct CapabilityCallback { cwd: Option, provider_secret_references: Option>, prior_context: Option, + #[serde(default)] + provider_path_inputs: Option, } async fn apply_prepared_callback( @@ -351,7 +353,26 @@ async fn apply_prepared_callback( .into_iter() .filter(|reference| !reference.profile.is_configuration_file()) .collect(); - let original = |name: &str| std::env::var(name).ok(); + let path_inputs = callback.provider_path_inputs.unwrap_or_default(); + let frozen_names = accepted_manifest + .as_ref() + .map(|manifest| manifest.frozen_inputs.as_slice()) + .unwrap_or(&[]); + if path_inputs + .iter() + .any(|(name, _)| !frozen_names.iter().any(|approved| approved == name)) + || frozen_names + .iter() + .any(|name| path_inputs.get(name).is_none()) + { + return Err("accepted request-only source inputs are unavailable or differ from their generation; reprepare before dispatch".into()); + } + let original = |name: &str| { + path_inputs + .get(name) + .map(str::to_owned) + .or_else(|| std::env::var(name).ok()) + }; let selected_sources=accepted_manifest.as_ref().map(|manifest|{ manifest.sources.iter().enumerate().map(|(ordinal,source)|{ let canonical=std::fs::canonicalize(&source.logical_path) @@ -3461,6 +3482,159 @@ except Exception as error: ); } + #[tokio::test] + async fn managed_provider_replacement_preserves_request_only_path_inputs() { + use crate::managed_provider_references::sources::*; + use freshell_runtime_protocol::*; + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let root = home.join(".config/opencode"); + std::fs::create_dir_all(&root).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let config = root.join("opencode.json"); + let raw = r#"{"agent":{"probe":{"prompt":"{file:{env:REQUEST_RULE_ROOT}/rules}"}}}"#; + std::fs::write(&config, raw).unwrap(); + let rules = root.join("rules"); + let nonce = "PRIVATE_REQUEST_REISSUE_NONCE"; + std::fs::write(&rules, nonce).unwrap(); + let auth = owned.path().join("auth.json"); + std::fs::write(&auth, "{}").unwrap(); + std::fs::set_permissions(&auth, std::fs::Permissions::from_mode(0o600)).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + }], + &workspace, + &home, + &|_, _, name| (name == "REQUEST_RULE_ROOT").then(|| root.to_str().unwrap().into()), + ) + .unwrap(); + let references = vec![ + ProviderSecretReference { + source_path: auth.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeOnecliAuthFile, + provider_relative_path: None, + }, + ProviderSecretReference { + source_path: rules.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(".freshell/references/0".into()), + }, + ]; + let base = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "opencode", &workspace, &home, None, + ) + .unwrap() + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let frozen_names = vec!["REQUEST_RULE_ROOT".into()]; + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &base.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + launch_context: Some(&base), + secret_references: Some(&references), + frozen_inputs: Some(&frozen_names), + ..Default::default() + }, + None, + ) + .unwrap(); + let accepted = read_prepared_context(&directory, &prior).unwrap(); + let inputs = ProviderPathInputs::new(std::collections::BTreeMap::from([( + "REQUEST_RULE_ROOT".into(), + root.to_str().unwrap().into(), + )])) + .unwrap(); + let callback = serde_json::to_vec(&serde_json::json!({"action":"issue", "controlSecret":"fixture-control", "soulId":soul, + "provider":"opencode", "workspace":workspace, "cwd":workspace, "priorContext":accepted, + "providerSecretReferences":references, "authLookupAuthority":"fixture-owned-attempt", "providerPathInputs":inputs})).unwrap(); + let control = owned.path().join("control.sock"); + let listener = tokio::net::UnixListener::bind(&control).unwrap(); + let expected_root = root.clone(); + let bound = tokio::spawn(async move { + tokio::time::timeout(std::time::Duration::from_secs(5), async move { + for operation in 0..2 { + let (mut socket, _) = listener.accept().await.unwrap(); + let request: Envelope = read_frame(&mut socket).await.unwrap(); + assert_eq!(request.auth.as_deref(), Some("fixture-control")); + let result = match (operation, request.body) { + (0, AdminCommand::Health) => AdminResult::Health { + control_epoch: 1, + installation_id: InstallationId::parse("installation-fixture").unwrap(), + }, + (1, AdminCommand::BindProviderPathInputs(request)) => { + assert_eq!( + request.inputs.get("REQUEST_RULE_ROOT"), + expected_root.to_str() + ); + AdminResult::ProviderPathInputsBound + } + _ => panic!("unexpected private replacement operation"), + }; + write_frame( + &mut socket, + &AdminReply { + request_id: request.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + }) + .await + .unwrap(); + }); + let client = freshell_runtime_client::RuntimeClient::new(control, "fixture-control"); + let reply = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await; + if reply.is_err() { + bound.abort(); + } + let reply = reply.expect("replacement must resolve the accepted request-only selection without current controller env"); + bound.await.unwrap(); + let reply: PreparedProviderContext = serde_json::from_slice(&reply).unwrap(); + let selected = reply + .provider_secret_references + .iter() + .find(|reference| reference.profile.is_configuration_file()) + .unwrap(); + assert_eq!( + std::fs::read(&selected.source_path).unwrap(), + nonce.as_bytes() + ); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + let durable = bounded_json_bytes(&reply, MAX_CONTROL_FRAME_BYTES).unwrap(); + assert!(!durable + .windows(nonce.len()) + .any(|bytes| bytes == nonce.as_bytes())); + assert!(staged_provider_root_path(&directory, &prior.grant_id) + .unwrap() + .is_dir()); + } + #[tokio::test] async fn managed_provider_replacement_callback_returns_complete_source_context() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index 4733c18e1..cca91cdd2 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -136,6 +136,28 @@ async fn provider_frozen_path_binding_is_idempotent_private_and_generation_scope Some(sentinel) ); assert!(supervisor.registry.inventory().await.unwrap().is_empty()); + supervisor.commit_provider_path_inputs(&soul, &generation); + assert!( + supervisor + .provider_path_inputs + .lock() + .unwrap() + .contains_key(&(soul.clone(), generation.clone())), + "an accepted generation must retain process-only request path selections for replacement" + ); + assert_eq!( + supervisor + .bound_provider_path_inputs(&soul, Some(&generation)) + .get("RULE_ROOT"), + Some(sentinel) + ); + assert!( + supervisor + .bound_provider_path_inputs(&SoulId::new(), Some(&generation)) + .is_empty(), + "generation names never authorize another soul's selection" + ); + for file in std::fs::read_dir(®istry_root).unwrap() { let file = file.unwrap().path(); if file.is_file() { @@ -148,13 +170,87 @@ async fn provider_frozen_path_binding_is_idempotent_private_and_generation_scope ); } } + let pending = format!("grant-{}", uuid::Uuid::new_v4()); + let excess = format!("grant-{}", uuid::Uuid::new_v4()); + for (candidate, success) in [(&pending, true), (&excess, false), (&pending, true)] { + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::BindProviderPathInputs(BindProviderPathInputsRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul.clone(), + generation_id: candidate.clone(), + inputs: ProviderPathInputs::new(BTreeMap::from([( + "RULE_ROOT".into(), + sentinel.into(), + )])) + .unwrap(), + }), + ); + envelope.auth = Some(secret.into()); + assert_eq!( + supervisor.dispatch(envelope).await.result.is_ok(), + success, + "a soul may retain only its active and one pending input generation" + ); + } + supervisor + .provider_path_inputs + .lock() + .unwrap() + .get_mut(&(soul.clone(), pending.clone())) + .unwrap() + .1 = Some(tokio::time::Instant::now() - std::time::Duration::from_secs(1)); + assert!( + supervisor + .bound_provider_path_inputs(&soul, Some(&pending)) + .is_empty(), + "an expired unaccepted preparation cannot provide path inputs" + ); + assert_eq!( + supervisor + .bound_provider_path_inputs(&soul, Some(&generation)) + .get("RULE_ROOT"), + Some(sentinel), + "the pending timeout must not remove the accepted selection" + ); + let mut renewal = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::BindProviderPathInputs(BindProviderPathInputsRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul.clone(), + generation_id: pending.clone(), + inputs: ProviderPathInputs::new(BTreeMap::from([( + "RULE_ROOT".into(), + sentinel.into(), + )])) + .unwrap(), + }), + ); + renewal.auth = Some(secret.into()); + supervisor.dispatch(renewal).await.result.unwrap(); + supervisor.commit_provider_path_inputs(&soul, &pending); + assert!( + !supervisor + .provider_path_inputs + .lock() + .unwrap() + .contains_key(&(soul.clone(), generation.clone())), + "accepting replacement must retire the previous process-only selection" + ); + assert!(supervisor + .provider_path_inputs + .lock() + .unwrap() + .contains_key(&(soul.clone(), pending.clone()))); let mut envelope = Envelope::new( RequestId::new(), ControlRole::Web, AdminCommand::BindProviderPathInputs(BindProviderPathInputsRequest { expected_control_epoch: Some(supervisor.registry.control_epoch()), soul_id: soul.clone(), - generation_id: generation.clone(), + generation_id: pending.clone(), inputs: ProviderPathInputs::default(), }), ); @@ -164,7 +260,7 @@ async fn provider_frozen_path_binding_is_idempotent_private_and_generation_scope .provider_path_inputs .lock() .unwrap() - .contains_key(&(soul, generation))); + .contains_key(&(soul, pending))); } #[async_trait] diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index c1e8d034d..7a8d04d9d 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -64,6 +64,7 @@ struct McpCapabilityRequest<'a> { cwd: Option<&'a str>, prior_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, provider_secret_references: Option<&'a [freshell_runtime_protocol::ProviderSecretReference]>, + provider_path_inputs: Option<&'a freshell_runtime_protocol::ProviderPathInputs>, } #[derive(Clone)] @@ -88,8 +89,13 @@ impl Drop for AuthLookupScope { } } -type ProviderPathInputBindings = - HashMap<(SoulId, String), (freshell_runtime_protocol::ProviderPathInputs, Instant)>; +type ProviderPathInputBindings = HashMap< + (SoulId, String), + ( + freshell_runtime_protocol::ProviderPathInputs, + Option, + ), +>; #[derive(Clone)] pub struct Supervisor { @@ -249,7 +255,9 @@ impl Supervisor { .provider_path_inputs .lock() .unwrap_or_else(std::sync::PoisonError::into_inner); - bindings.retain(|_, (_, expires)| *expires > Instant::now()); + bindings.retain(|_, (_, expires)| { + expires.is_none_or(|expires| expires > Instant::now()) + }); let key = (request.soul_id, request.generation_id); if request.inputs.is_empty() { bindings.remove(&key); @@ -258,9 +266,18 @@ impl Supervisor { return Err(invalid()); } } else { + if bindings + .iter() + .any(|((soul, _), (_, expires))| *soul == key.0 && expires.is_some()) + { + return Err(invalid()); + } bindings.insert( key, - (request.inputs, Instant::now() + Duration::from_secs(600)), + ( + request.inputs, + Some(Instant::now() + Duration::from_secs(600)), + ), ); } Ok(AdminResult::ProviderPathInputsBound) @@ -1980,6 +1997,7 @@ impl Supervisor { cwd: None, prior_context: None, provider_secret_references: None, + provider_path_inputs: None, }) .await?; if response != b"ok" { @@ -2037,14 +2055,43 @@ impl Supervisor { .map_err(map_backend)?; } if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { - self.provider_path_inputs - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - .remove(&(workload.soul_id.clone(), inputs.generation_id.clone())); + self.commit_provider_path_inputs(&workload.soul_id, &inputs.generation_id); } Ok(()) } + fn bound_provider_path_inputs( + &self, + soul_id: &SoulId, + generation_id: Option<&str>, + ) -> freshell_runtime_protocol::ProviderPathInputs { + generation_id + .and_then(|generation| { + self.provider_path_inputs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .get(&(soul_id.clone(), generation.into())) + .filter(|(_, expires)| expires.is_none_or(|expires| expires > Instant::now())) + .map(|(inputs, _)| inputs.clone()) + }) + .unwrap_or_default() + } + + fn commit_provider_path_inputs(&self, soul_id: &SoulId, generation_id: &str) { + let mut bindings = self + .provider_path_inputs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let current = (soul_id.clone(), generation_id.to_string()); + // The controller has retired the prior generation only after acceptance. + // Keep the current selection process-local for replacement, with no + // preparation timeout once it belongs to the accepted running context. + bindings.retain(|key, _| key.0 != *soul_id || *key == current); + if let Some((_, expires)) = bindings.get_mut(¤t) { + *expires = None; + } + } + pub(crate) async fn issue_replacement_mcp_capability( &self, start: &crate::registry::RecoveryStart, @@ -2061,6 +2108,13 @@ impl Supervisor { _ => None, }; let scope = self.auth_lookup_scope(start); + let path_inputs = self.bound_provider_path_inputs( + &start.context.soul_id, + context + .prepared_inputs + .as_ref() + .map(|inputs| inputs.generation_id.as_str()), + ); let response = self .mcp_capability_callback(McpCapabilityRequest { action: "issue", @@ -2074,6 +2128,7 @@ impl Supervisor { cwd: Some(cwd), prior_context: Some(context), provider_secret_references: Some(references), + provider_path_inputs: Some(&path_inputs), }) .await?; let prepared: freshell_runtime_protocol::PreparedProviderContext = @@ -2111,6 +2166,7 @@ impl Supervisor { "cwd":request.cwd, "priorContext":request.prior_context, "providerSecretReferences":request.provider_secret_references, + "providerPathInputs":request.provider_path_inputs, }), freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, ) @@ -2398,16 +2454,10 @@ impl Supervisor { .and_then(|launch| launch.provider_launch_context.as_ref()) }) .and_then(|context| context.prepared_inputs.as_ref()); - let provider_path_inputs = preparation - .and_then(|inputs| { - self.provider_path_inputs - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - .get(&(workload.soul_id.clone(), inputs.generation_id.clone())) - .filter(|(_, expires)| *expires > Instant::now()) - .map(|(values, _)| values.clone()) - }) - .unwrap_or_default(); + let provider_path_inputs = self.bound_provider_path_inputs( + &workload.soul_id, + preparation.map(|inputs| inputs.generation_id.as_str()), + ); let command = HostCommand::GrantExecution { incarnation_id: incarnation_id.clone(), soul_id: workload.soul_id.clone(), From ba08d171fb35ec91513c12f6c594094309048b1f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 16:50:57 -0700 Subject: [PATCH 050/493] fix: deliver approved native provider source selections to children --- .../freshell-platform/src/opencode_recipe.rs | 23 +++ .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 136 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 13 +- .../src/managed_provider_references.rs | 41 ++++++ .../managed_provider_references/sources.rs | 12 ++ crates/freshell-server/src/managed_runtime.rs | 1 + .../src/provider_input_preparation.rs | 41 +++++- 8 files changed, 258 insertions(+), 10 deletions(-) diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index 2ce1f79c7..c7a1f2433 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -36,6 +36,29 @@ pub struct SourceManifest { pub frozen_inputs: Vec, #[serde(default)] pub user_owns_freshell_mcp: bool, + #[serde(default)] + pub native_source_flags: Vec, +} + +#[derive(Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct NativeSourceFlag { + pub kind: NativeSourceFlagKind, + pub runtime_destination: String, +} +#[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum NativeSourceFlagKind { + Main, + Tui, +} +impl NativeSourceFlagKind { + pub fn variable(self) -> &'static str { + match self { + Self::Main => "OPENCODE_CONFIG", + Self::Tui => "OPENCODE_TUI_CONFIG", + } + } } #[derive(Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 42878031b..2a0aa8077 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -468,6 +468,7 @@ impl HostedFreshAgentProxy { references: provider_secret_references, auth_authority: None, selected_sources: None, + native_source_flags: None, }, ) .await diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 716799c1a..ecaf5502c 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -44,6 +44,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub config_root: Option<&'a Path>, pub launch_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, pub frozen_inputs: Option<&'a [String]>, + pub native_source_flags: Option<&'a [freshell_platform::opencode_recipe::NativeSourceFlag]>, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -401,6 +402,9 @@ async fn apply_prepared_callback( .ok_or("replacement provider preparation requires attempt authority")?, ), selected_sources, + native_source_flags: accepted_manifest + .as_ref() + .map(|manifest| manifest.native_source_flags.clone()), }, |soul, home, context, input| { let input = OpencodeEphemeralInput { @@ -1220,6 +1224,7 @@ fn stage_provider_root( opencode_input.cwd.ok_or("source plan requires cwd")?, plan, opencode_input.frozen_inputs.unwrap_or(&[]), + opencode_input.native_source_flags.unwrap_or(&[]), )?; } allow_controller_stage_read(&temporary)?; @@ -3482,6 +3487,137 @@ except Exception as error: ); } + #[tokio::test] + async fn managed_provider_explicit_native_source_selection_survives_preparation() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let config = workspace.join("selected.json"); + let raw = r#"{"agent":{"probe":{"prompt":"{file:rules}"}}}"#; + std::fs::write(&config, raw).unwrap(); + let rules = workspace.join("rules"); + std::fs::write(&rules, "EXPLICIT_SOURCE_SELECTION_NONCE").unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "fixture-existing-soul", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| (name == "OPENCODE_CONFIG").then(|| "selected.json".into()), + references: vec![], + auth_authority: Some("fixture-owned-attempt"), + selected_sources: None, + native_source_flags: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .unwrap(); + let capability = prepared.context.mcp_capability.as_ref().unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert_eq!( + manifest.native_source_flags.len(), + 1, + "native explicit selection must be preserved beside its approved exact config bind" + ); + assert_eq!( + manifest.native_source_flags[0].kind.variable(), + "OPENCODE_CONFIG" + ); + assert_eq!( + manifest.native_source_flags[0].runtime_destination, + config.to_str().unwrap() + ); + assert_eq!(prepared.references[0].source_path, rules.to_str().unwrap()); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + let callback = serde_json::to_vec(&serde_json::json!({"action":"issue", "controlSecret":"fixture-control", "soulId":soul, + "provider":"opencode", "workspace":workspace, "cwd":workspace, "priorContext":prepared.context, + "providerSecretReferences":prepared.references, "authLookupAuthority":"fixture-owned-attempt"})).unwrap(); + let reissued = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(); + let reissued: PreparedProviderContext = serde_json::from_slice(&reissued).unwrap(); + let replacement_stage = staged_provider_root_path( + &directory, + &reissued + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap() + .grant_id, + ) + .unwrap(); + let replacement: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(replacement_stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert_eq!(replacement.native_source_flags.len(), 1); + assert_eq!( + replacement.native_source_flags[0].runtime_destination, + config.to_str().unwrap(), + "request-only native controls must survive current controller env absence on reissue" + ); + let missing = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &SoulId::new(), + creation_seed: "fixture-existing-soul", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| (name == "OPENCODE_CONFIG").then(|| "missing.json".into()), + references: vec![], + auth_authority: Some("fixture-owned-attempt"), + selected_sources: None, + native_source_flags: None, + }, + |_, _, _, _| panic!("missing selected native source must fail before publication"), + ) + .await + .err() + .expect("a missing explicitly selected source cannot be omitted"); + assert!(missing.contains("OPENCODE_CONFIG")); + } + #[tokio::test] async fn managed_provider_replacement_preserves_request_only_path_inputs() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 24e785297..9c9b1c053 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -10,6 +10,7 @@ pub(super) fn stage( cwd: &Path, plan: &[PlannedSourceOccurrence], frozen_inputs: &[String], + native_source_flags: &[freshell_platform::opencode_recipe::NativeSourceFlag], ) -> Result<(), String> { let mut manifest = SourceManifest { version: 1, @@ -19,6 +20,7 @@ pub(super) fn stage( files: Vec::new(), frozen_inputs: frozen_inputs.to_vec(), user_owns_freshell_mcp: false, + native_source_flags: native_source_flags.to_vec(), }; private_directory(&root.join(".freshell/source-views"))?; private_directory(&root.join(".freshell/templates"))?; @@ -131,14 +133,9 @@ pub(super) fn describe( ); } let logical = &occurrence.source.logical_path; - let destination = if let Ok(relative) = logical.strip_prefix(global) { - runtime_home.join(".config/opencode").join(relative) - } else { - logical - .strip_prefix(home) - .map(|relative| runtime_home.join(relative)) - .unwrap_or_else(|_| logical.clone()) - }; + let destination = crate::managed_provider_references::sources::runtime_config_destination( + home, global, logical, + ); let effective = if let Ok(relative) = logical.strip_prefix(&global) { let physical = std::fs::canonicalize(root.join(relative)) .map_err(|_| "selected global config is unavailable in its private stage")?; diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 7f6c22df9..34aeca410 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -714,6 +714,7 @@ pub(crate) struct OpencodePreparation<'a> { pub references: Vec, pub auth_authority: Option<&'a str>, pub selected_sources: Option>, + pub native_source_flags: Option>, } pub(crate) async fn prepare_managed_opencode( @@ -746,6 +747,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( mut references, auth_authority, selected_sources, + native_source_flags, } = request; use sources::*; if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) @@ -787,6 +789,44 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( .map_err(str::to_string)? }; let loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; + let native_source_flags = if let Some(flags) = native_source_flags { + flags + } else { + let mut flags = Vec::new(); + for (name, kind) in [ + ( + "OPENCODE_CONFIG", + freshell_platform::opencode_recipe::NativeSourceFlagKind::Main, + ), + ( + "OPENCODE_TUI_CONFIG", + freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui, + ), + ] { + if let Some(path) = original(name).filter(|path| !path.is_empty()) { + let path = if Path::new(&path).is_absolute() { + PathBuf::from(path) + } else { + cwd.join(path) + }; + if !loaded + .iter() + .any(|source| source.source.logical_path == path) + { + return Err(format!("selected {name} source is missing or unavailable; repair it before dispatch")); + } + let destination = sources::runtime_config_destination(home, &config_root, &path); + flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { + kind, + runtime_destination: destination + .to_str() + .ok_or("selected native source destination is not UTF-8")? + .into(), + }); + } + } + flags + }; if loaded .iter() .any(|source| source.source.lane == SourceLane::Tui) @@ -870,6 +910,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( initial_auth_reservation: reservation.as_deref(), secret_references: Some(&references), frozen_inputs: Some(&frozen_names), + native_source_flags: Some(&native_source_flags), ..Default::default() }, )?; diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 7c881627c..77ab0c9f5 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -95,6 +95,18 @@ pub fn discover_opencode( Ok(sources) } +pub fn runtime_config_destination(home: &Path, global: &Path, logical: &Path) -> PathBuf { + let runtime_home = Path::new("/home/freshell/provider"); + if let Ok(relative) = logical.strip_prefix(global) { + runtime_home.join(".config/opencode").join(relative) + } else { + logical + .strip_prefix(home) + .map(|relative| runtime_home.join(relative)) + .unwrap_or_else(|_| logical.into()) + } +} + fn absolute_from_cwd(path: &str, cwd: &Path) -> PathBuf { let path = Path::new(path); if path.is_absolute() { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 4a1011966..0e86141bd 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -242,6 +242,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { references: provider_secret_references, auth_authority: None, selected_sources: None, + native_source_flags: None, }, ) .await?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index bb1073410..30a18336b 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -104,6 +104,22 @@ pub fn prepare( user_owns_freshell_mcp: manifest.user_owns_freshell_mcp, }; let mut generated = BTreeMap::new(); + if manifest.native_source_flags.len() > 2 { + return Err("prepared native source selector profile exceeded".into()); + } + for flag in &manifest.native_source_flags { + let name = flag.kind.variable(); + if !inputs + .config_binds + .iter() + .any(|bind| bind.logical_destination == flag.runtime_destination) + || generated.contains_key(name) + || secrets.environment.contains_key(name) + { + return Err("prepared native source selector has no approved config bind or collides with child input".into()); + } + generated.insert(name.into(), flag.runtime_destination.clone()); + } let mut views = 0usize; let mut budget = frozen.clone(); for source in manifest.sources { @@ -213,6 +229,10 @@ mod tests { }], frozen_inputs: vec!["INSERT".into()], user_owns_freshell_mcp: true, + native_source_flags: vec![NativeSourceFlag { + kind: NativeSourceFlagKind::Main, + runtime_destination: "/original-unavailable/project/opencode.json".into(), + }], files: vec![SelectedSourceFile { source_id: 0, canonical_source: "/original-unavailable/payload/rules".into(), @@ -237,7 +257,11 @@ mod tests { alias_count: 0, recipe_count: 1, }, - config_binds: vec![], + config_binds: vec![ProviderConfigBind { + source_relative_path: ".freshell/templates/source-0".into(), + logical_destination: "/original-unavailable/project/opencode.json".into(), + effective_destination: "/original-unavailable/project/opencode.json".into(), + }], resource_mounts: vec![], indexed_files: Some(IndexedProviderFiles { volume_name: indexed_provider_volume_name(&SoulId::new(), &capability.grant_id), @@ -271,6 +295,9 @@ mod tests { let value=generated.get("FRESHELL_CONFIG_REFERENCE_0").expect("host must generate the selected native child source from private path inputs before dispatch"); let expected = r#"{"instructions":["PRIVATE_PREFIX{file:/home/freshell/provider/.freshell/references/0}PRIVATE_SUFFIX"]}"#; assert_eq!(value, expected); + assert_eq!(generated.get("OPENCODE_CONFIG").map(String::as_str), + Some("/original-unavailable/project/opencode.json"), + "an explicitly selected native source must be delivered to the actual child environment"); assert!(!generated.contains_key("INSERT"),"independent generated recipes do not forward an original ambient variable into the native child"); let changed = ProviderPathInputs::new(BTreeMap::from([( "INSERT".into(), @@ -293,7 +320,8 @@ mod tests { format!("{:x}", Sha256::digest(&bytes)); let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) .expect("direct source must use the 16 MiB document profile"); - assert!(generated.is_empty()); + assert_eq!(generated.len(), 1); + assert!(generated.contains_key("OPENCODE_CONFIG")); assert_eq!( std::fs::read(stage.join(".freshell/source-views/source-0")).unwrap(), large_raw.as_bytes() @@ -330,5 +358,14 @@ mod tests { .contains("64 KiB"), "generated child values retain the declared stricter limit" ); + manifest.native_source_flags[0].runtime_destination = + "/unapproved/native/config.json".into(); + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap_err(); + assert!(error.contains("approved config bind")); + assert!(!error.contains("/unapproved/native/config.json")); } } From 14243b2053be8505d3d1f7a0ffd7bdaaa54d9cf4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 17:19:05 -0700 Subject: [PATCH 051/493] fix: limit frozen source inputs to approved path variables --- .../environment.rs | 23 +++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index 65a9cfff6..bad92c90e 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -148,12 +148,10 @@ impl SourceEnvironment { ) -> Result { let mut selected = std::collections::BTreeSet::new(); for source in plan { - let names = if source.references.recipes.is_empty() { - &source.references.path_variables - } else { - &source.references.referenced_variables - }; - for name in names { + // A whole-document recipe does not grant unrelated ambient keys + // new forwarding authority. Auth and OneCLI remain host-resolved; + // freeze only original inputs that select accepted file paths. + for name in &source.references.path_variables { if self.onecli.contains_key(name) || (source.source.lane == SourceLane::Main && self @@ -248,7 +246,11 @@ mod tests { let input = format!("PRIVATE_PREFIX{{file:{}}}PRIVATE_SUFFIX", rules.display()); let mut environment = SourceEnvironment::read(vec!["INSERT".into(), "UNRELATED".into()], &[], &|name| { - (name == "INSERT").then(|| input.clone()) + match name { + "INSERT" => Some(input.clone()), + "UNRELATED" => Some("PRIVATE_UNRELATED_AMBIENT_SECRET".into()), + _ => None, + } }) .unwrap(); let loaded = || { @@ -274,6 +276,13 @@ mod tests { let frozen = environment.frozen_inputs(&plan).unwrap(); assert_eq!(frozen.get("INSERT"), Some(input.as_str())); assert!(frozen.get("UNRELATED").is_none()); + let handoff = serde_json::to_vec(&frozen).unwrap(); + assert!( + !handoff + .windows(b"PRIVATE_UNRELATED_AMBIENT_SECRET".len()) + .any(|bytes| bytes == b"PRIVATE_UNRELATED_AMBIENT_SECRET"), + "private IPC serialization must not gain unrelated ambient credentials" + ); assert!(!plan[0].references.template.contains("PRIVATE_PREFIX")); assert!(!plan[0] .references From fe8baf7fc59cb8f92770356ea7bf88e096db80ff Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 17:35:58 -0700 Subject: [PATCH 052/493] test: prove native OpenCode consumes selected config on both launch routes --- .../src/managed_provider_launch_tests.rs | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 533e3b35b..fbdf442f1 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -285,6 +285,12 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr let cwd = fixture.join("workspace").to_string_lossy().into_owned(); let original_config = fixture.join("source-home/.config/opencode/opencode.json"); let original_bytes = std::fs::read(&original_config).unwrap(); + let explicit_config = fixture.join("workspace/selected-config.json"); + let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; + std::fs::write(&explicit_config, explicit_bytes).unwrap(); + let explicit_rule = fixture.join("workspace/selected-rule"); + std::fs::write(&explicit_rule, "native-explicit-selected-rule").unwrap(); + std::fs::set_permissions(&explicit_rule, std::fs::Permissions::from_mode(0o600)).unwrap(); let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { program: "opencode".into(), @@ -294,7 +300,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr cols: 120, rows: 30, }, - env: BTreeMap::new(), + env: BTreeMap::from([("OPENCODE_CONFIG".into(), "selected-config.json".into())]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), mode: "opencode".into(), @@ -327,7 +333,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr .map_err(|_| "actual pinned provider startup did not consume selected config and closure") } let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXnative-ruleAUTH_SUFFIX")), - "description":format!("{:x}",Sha256::digest(b"native-second-rule")), + "description":format!("{:x}",Sha256::digest(b"native-explicit-selected-rule")), "references":(["third","fourth","fifth","sixth"].map(|ordinal|format!("{:x}",Sha256::digest(format!("native-{ordinal}-rule").as_bytes())))), "resource":format!("{:x}",Sha256::digest(b"plugin-relative-resource")), "dependency":format!("{:x}",Sha256::digest(b"plugin-actual-local-dependency")),"sdk":format!("{:x}",Sha256::digest(b"function"))}); @@ -373,6 +379,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr }; assert_eq!(terminal_receipt, expected); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); println!( "provider_launch_consumer {}", json!({"provider":"opencode","route":"terminal","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) @@ -382,6 +389,9 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr .await .unwrap(); std::fs::remove_file(&receipt).unwrap(); + // Fresh launches use current server controls; the terminal exercised a + // request-only relative control. Both must activate the explicit layer. + std::env::set_var("OPENCODE_CONFIG", &explicit_config); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( @@ -423,6 +433,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr .await; assert_eq!(read_receipt(&receipt).await.unwrap(), expected); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); let stopped = fixture.join("workspace/native-consumed.json.stopped"); tokio::time::timeout(Duration::from_secs(10), async { loop { @@ -453,6 +464,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr for sentinel in [ b"AUTH_PREFIX".as_slice(), b"native-rule".as_slice(), + b"native-explicit-selected-rule".as_slice(), b"synthetic-onecli-shadowed".as_slice(), ] { assert!( From 1bfbe5f17c6f2c062b0e04a3da3211880daa391a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 17:48:23 -0700 Subject: [PATCH 053/493] fix: prepare selected OpenCode TUI sources before reference recipes --- .../freshell-platform/src/opencode_plugin.rs | 99 ++++++++++++++----- .../src/managed_mcp_capability.rs | 85 ++++++++++++++++ .../src/managed_provider_launch_tests.rs | 32 +++++- .../src/managed_provider_references.rs | 54 ++++++++-- crates/freshell-server/src/managed_runtime.rs | 74 +------------- .../src/providers/mod.rs | 67 +++++++++---- .../managed-provider-launch-sandbox.py | 4 +- 7 files changed, 291 insertions(+), 124 deletions(-) diff --git a/crates/freshell-platform/src/opencode_plugin.rs b/crates/freshell-platform/src/opencode_plugin.rs index 2ac439ba3..e789a74e3 100644 --- a/crates/freshell-platform/src/opencode_plugin.rs +++ b/crates/freshell-platform/src/opencode_plugin.rs @@ -122,31 +122,7 @@ pub fn ensure_rebind_plugin_with_user_config( let tui_json = tui_config_path(home); let content = if let Some(selected) = selected { let raw = std::fs::read_to_string(selected)?; - let mut config = crate::opencode_config::parse_jsonc_object(&raw).ok_or_else(|| { - std::io::Error::new( - std::io::ErrorKind::InvalidData, - "OpenCode TUI config contains malformed JSONC or is not an object", - ) - })?; - let plugins = config - .as_object_mut() - .expect("validated object") - .entry("plugin") - .or_insert_with(|| serde_json::json!([])); - let Some(plugins) = plugins.as_array_mut() else { - return Err(std::io::Error::new( - std::io::ErrorKind::InvalidData, - "OpenCode TUI plugin list is not an array", - )); - }; - let own = plugin_file_spec(&plugin); - if !plugins - .iter() - .any(|entry| entry.as_str() == Some(own.as_str())) - { - plugins.push(serde_json::Value::String(own)); - } - config.to_string() + selected_tui_content(&raw, &plugin)? } else { tui_config_content(&plugin) }; @@ -154,6 +130,79 @@ pub fn ensure_rebind_plugin_with_user_config( Ok(tui_json) } +/// Normalize the selected source before reference spans are captured. This +/// function never resolves environment or file tokens and never changes the +/// declaring origin. +pub fn selected_tui_content(raw: &str, plugin: &Path) -> std::io::Result { + selected_tui_content_bounded(raw, plugin, 16 * 1024 * 1024) +} + +pub fn selected_tui_content_bounded( + raw: &str, + plugin: &Path, + remaining: usize, +) -> std::io::Result { + let mut config = crate::opencode_config::parse_jsonc_object(raw).ok_or_else(|| { + std::io::Error::new( + std::io::ErrorKind::InvalidData, + "OpenCode TUI config contains malformed JSONC or is not an object", + ) + })?; + let plugins = config + .as_object_mut() + .expect("validated object") + .entry("plugin") + .or_insert_with(|| serde_json::json!([])); + let Some(plugins) = plugins.as_array_mut() else { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "OpenCode TUI plugin list is not an array", + )); + }; + let own = plugin_file_spec(plugin); + if !plugins + .iter() + .any(|entry| entry.as_str() == Some(own.as_str())) + { + plugins.push(serde_json::Value::String(own)); + } + struct BoundedView { + bytes: Vec, + limit: usize, + } + impl Write for BoundedView { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + if self + .bytes + .len() + .checked_add(bytes.len()) + .is_none_or(|size| size > self.limit) + { + return Err(std::io::Error::other( + "OpenCode effective TUI source-view profile exceeded", + )); + } + self.bytes.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + let mut view = BoundedView { + bytes: Vec::new(), + limit: remaining.min(16 * 1024 * 1024), + }; + serde_json::to_writer(&mut view, &config).map_err(std::io::Error::other)?; + String::from_utf8(view.bytes).map_err(std::io::Error::other) +} + +/// Install only the implementation when the controller already prepared the +/// selected TUI source. Re-parsing that source here would change recipe spans. +pub fn ensure_rebind_plugin_source(home: &Path) -> std::io::Result<()> { + write_atomic_if_changed(&rebind_plugin_path(home), REBIND_PLUGIN_SOURCE) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index ecaf5502c..b5391fc4c 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -3487,6 +3487,91 @@ except Exception as error: ); } + #[tokio::test] + async fn managed_provider_selected_tui_keeps_origin_and_applies_owned_edits_before_references() + { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let config = workspace.join("selected-tui.jsonc"); + let raw = "{ // original comment\n\"theme\":\"{file:./theme.txt}\",\"plugin\":[],}"; + std::fs::write(&config, raw).unwrap(); + let theme = workspace.join("theme.txt"); + std::fs::write(&theme, "dark").unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "fixture-existing-soul", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| { + (name == "OPENCODE_TUI_CONFIG").then(|| "selected-tui.jsonc".into()) + }, + references: vec![], + auth_authority: Some("fixture-owned-attempt"), + selected_sources: None, + native_source_flags: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .expect("selected TUI must remain supported through the shared managed producer"); + let stage = staged_provider_root_path( + &directory, + &prepared.context.mcp_capability.as_ref().unwrap().grant_id, + ) + .unwrap(); + let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert_eq!(manifest.native_source_flags.len(), 1); + assert_eq!( + manifest.native_source_flags[0].kind.variable(), + "OPENCODE_TUI_CONFIG" + ); + let effective = + std::fs::read_to_string(stage.join(&manifest.sources[0].unexpanded_relative_path)) + .unwrap(); + let effective: serde_json::Value = serde_json::from_str(&effective).unwrap(); + assert_eq!(effective["theme"], "{file:./theme.txt}"); + assert_eq!( + effective["plugin"], + serde_json::json!([ + "file:///home/freshell/provider/.freshell/opencode/freshell-rebind-plugin.ts" + ]), + "selected rebind normalization must precede the source view used by reference recipes" + ); + assert_eq!(prepared.references[0].source_path, theme.to_str().unwrap()); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + } + #[tokio::test] async fn managed_provider_explicit_native_source_selection_survives_preparation() { use freshell_runtime_protocol::*; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index fbdf442f1..534bfdf57 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -291,6 +291,18 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr let explicit_rule = fixture.join("workspace/selected-rule"); std::fs::write(&explicit_rule, "native-explicit-selected-rule").unwrap(); std::fs::set_permissions(&explicit_rule, std::fs::Permissions::from_mode(0o600)).unwrap(); + let tui_plugin = fixture.join("source-home/.config/opencode/plugins/probe-tui"); + std::fs::create_dir_all(&tui_plugin).unwrap(); + std::fs::write(tui_plugin.join("index.mjs"), r#"import fs from 'node:fs';import crypto from 'node:crypto'; +export default {id:'closed-native-tui-fixture',async tui(api,options){ + fs.writeFileSync(options.receipt,JSON.stringify({theme:crypto.createHash('sha256').update(api.tuiConfig.theme).digest('hex')})); +}}; +"#).unwrap(); + let selected_tui = fixture.join("workspace/selected-tui.jsonc"); + let tui_receipt = fixture.join("workspace/native-tui-consumed.json"); + let selected_tui_bytes = format!("{{ // selected origin\n\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&tui_receipt).unwrap()); + std::fs::write(&selected_tui, &selected_tui_bytes).unwrap(); + std::fs::write(fixture.join("workspace/selected-theme"), " dark\n").unwrap(); let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { program: "opencode".into(), @@ -300,7 +312,10 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr cols: 120, rows: 30, }, - env: BTreeMap::from([("OPENCODE_CONFIG".into(), "selected-config.json".into())]), + env: BTreeMap::from([ + ("OPENCODE_CONFIG".into(), "selected-config.json".into()), + ("OPENCODE_TUI_CONFIG".into(), "selected-tui.jsonc".into()), + ]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), mode: "opencode".into(), @@ -378,6 +393,20 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr } }; assert_eq!(terminal_receipt, expected); + assert_eq!( + read_receipt(&tui_receipt) + .await + .expect("actual TUI plugin must consume the selected effective TUI configuration"), + json!({"theme":format!("{:x}",Sha256::digest(b"dark"))}) + ); + assert_eq!( + std::fs::read_to_string(&selected_tui).unwrap(), + selected_tui_bytes + ); + println!( + "provider_launch_consumer {}", + json!({"provider":"opencode","route":"terminal-tui","digest":format!("{:x}",Sha256::digest(b"dark"))}) + ); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); println!( @@ -392,6 +421,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. std::env::set_var("OPENCODE_CONFIG", &explicit_config); + std::env::set_var("OPENCODE_TUI_CONFIG", &selected_tui); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 34aeca410..ebf70fa01 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -750,9 +750,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( native_source_flags, } = request; use sources::*; - if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) - || original("OPENCODE_TUI_CONFIG").is_some_and(|value| !value.is_empty()) - { + if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) { return Err( "selected OpenCode inline/TUI source requires its owned pre-recipe view".into(), ); @@ -788,7 +786,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( }) .map_err(str::to_string)? }; - let loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; + let mut loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; let native_source_flags = if let Some(flags) = native_source_flags { flags } else { @@ -827,11 +825,45 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( } flags }; - if loaded + let mut retained_views = loaded .iter() - .any(|source| source.source.lane == SourceLane::Tui) - { - return Err("selected OpenCode TUI source requires its owned pre-recipe view".into()); + .try_fold(0usize, |bytes, source| bytes.checked_add(source.view.len())) + .ok_or("source-view byte count overflow")?; + for source in &mut loaded { + if source.source.lane == SourceLane::Tui + && native_source_flags.iter().any(|flag| { + flag.kind == freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui + && Path::new(&flag.runtime_destination) + == sources::runtime_config_destination( + home, + &config_root, + &source.source.logical_path, + ) + }) + { + let remaining = (64 * 1024 * 1024usize) + .checked_sub(retained_views) + .ok_or("simultaneous source views exceed 64 MiB")?; + let content = freshell_platform::opencode_plugin::selected_tui_content_bounded( + &source.view, + &freshell_platform::opencode_plugin::rebind_plugin_path(Path::new( + "/home/freshell/provider", + )), remaining, + ) + .map_err(|_| { + format!( + "selected OpenCode TUI source {} has invalid JSONC/plugin list or exceeds the source-view profile", + source.source.logical_path.display() + ) + })?; + if content.len() > 16 * 1024 * 1024 { + return Err("selected OpenCode effective TUI source exceeds 16 MiB".into()); + } + retained_views = retained_views + .checked_add(content.len()) + .ok_or("source-view byte count overflow")?; + source.view = std::sync::Arc::new(content); + } } let names = source_environment_names(&loaded).map_err(|error| error.to_string())?; let mut environment = environment::SourceEnvironment::read(names, &references, original)?; @@ -848,7 +880,11 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( } else { None }; - if environment.needs_native_auth() { + if environment.needs_native_auth() + && loaded + .iter() + .any(|source| source.source.lane == SourceLane::Main) + { let authority = if let Some(authority) = auth_authority { authority.to_owned() } else { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 0e86141bd..ba4cf8e50 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -180,43 +180,12 @@ impl ManagedTerminalController for ServerManagedRuntimeController { }; let args = managed_provider_args(&request.mode, request.spec.args)?; let request_id = stable_request_id(create_key)?; - let tui_source = if request.mode == "opencode" { - request - .env - .get("OPENCODE_TUI_CONFIG") - .map(|raw| { - let home = std::env::var_os("HOME") - .map(std::path::PathBuf::from) - .ok_or("managed provider HOME is unavailable")?; - crate::managed_mcp_capability::approved_tui_source( - raw, &cwd, &workspace, &home, - ) - }) - .transpose()? - } else { - None - }; - let opencode_input = if request.mode == "opencode" { - crate::managed_mcp_capability::OpencodeEphemeralInput { - inline_config: request - .env - .get("OPENCODE_CONFIG_CONTENT") - .map(String::as_str), - tui_config_path: request.env.get("OPENCODE_TUI_CONFIG").map(String::as_str), - tui_source: tui_source.as_ref(), - cwd: Some(&cwd), - workspace: Some(&workspace), - source_plan: None, - ..Default::default() - } - } else { - crate::managed_mcp_capability::OpencodeEphemeralInput { - cwd: Some(&cwd), - workspace: Some(&workspace), - ..Default::default() - } + let opencode_input = crate::managed_mcp_capability::OpencodeEphemeralInput { + cwd: Some(&cwd), + workspace: Some(&workspace), + ..Default::default() }; - let (mcp_capability, mut provider_launch_context) = if request.mode == "opencode" { + let (mcp_capability, provider_launch_context) = if request.mode == "opencode" { let home = request .env .get("HOME") @@ -273,39 +242,6 @@ impl ManagedTerminalController for ServerManagedRuntimeController { )?; (mcp_capability, provider_launch_context) }; - if let Some(context) = provider_launch_context.as_mut() { - if let freshell_runtime_protocol::ProviderPreparation::Opencode { - inline_config, - tui_config, - tui_source: context_tui_source, - .. - } = &mut context.preparation - { - *inline_config = opencode_input.inline_config.is_some(); - if opencode_input.tui_config_path.is_some() { - let jsonc = opencode_input - .tui_config_path - .is_some_and(|path| path.to_ascii_lowercase().ends_with(".jsonc")); - *tui_config = Some(freshell_runtime_protocol::ProviderConfigReference { - root: freshell_runtime_protocol::ProviderConfigRoot::Ephemeral, - relative_path: if jsonc { - "tui-config.jsonc" - } else { - "tui-config.json" - } - .into(), - provider_relative_path: if jsonc { - ".freshell/opencode/user-tui.jsonc" - } else { - ".freshell/opencode/user-tui.json" - } - .into(), - format: if jsonc { "jsonc" } else { "json" }.into(), - }); - *context_tui_source = tui_source.clone(); - } - } - } if mcp_capability.is_some() && provider_launch_context.is_none() { if let Some(capability) = &mcp_capability { let _ = diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 2a3379933..71f94dcf9 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -119,24 +119,26 @@ pub async fn prepare_terminal( Some("0" | "false") ); if !rebind_disabled { + let prepared_tui = terminal + .provider_launch_context + .as_ref() + .is_some_and(|context| context.prepared_inputs.is_some()) + && child_env.contains_key("OPENCODE_TUI_CONFIG"); let source = terminal .env .get("OPENCODE_TUI_CONFIG") .or_else(|| child_env.get("OPENCODE_TUI_CONFIG")) .cloned(); - let mut args = vec!["opencode-rebind"]; - if let Some(source) = source.as_deref() { - args.push(source); - } - prepare_provider_features_for_identity( + if let Some(selector) = prepare_opencode_rebind( terminal.run_as_uid, terminal.run_as_gid, - &args, - )?; - terminal.env.insert( - "OPENCODE_TUI_CONFIG".into(), - "/home/freshell/provider/.freshell/opencode/tui.json".into(), - ); + prepared_tui, + source.as_deref(), + )? { + terminal + .env + .insert("OPENCODE_TUI_CONFIG".into(), selector.into()); + } } let permission = terminal .env @@ -326,20 +328,43 @@ pub fn prepare_fresh_agent_child_environment( .map(String::as_str), Some("0" | "false") ) { + let prepared_tui = launch + .provider_launch_context + .as_ref() + .is_some_and(|context| context.prepared_inputs.is_some()) + && child_env.contains_key("OPENCODE_TUI_CONFIG"); let source = child_env.get("OPENCODE_TUI_CONFIG").cloned(); - let mut args = vec!["opencode-rebind"]; - if let Some(source) = source.as_deref() { - args.push(source); + if let Some(selector) = prepare_opencode_rebind( + launch.run_as_uid, + launch.run_as_gid, + prepared_tui, + source.as_deref(), + )? { + child_env.insert("OPENCODE_TUI_CONFIG".into(), selector.into()); } - prepare_provider_features_for_identity(launch.run_as_uid, launch.run_as_gid, &args)?; - child_env.insert( - "OPENCODE_TUI_CONFIG".into(), - "/home/freshell/provider/.freshell/opencode/tui.json".into(), - ); } Ok(()) } +fn prepare_opencode_rebind( + uid: u32, + gid: u32, + prepared_source: bool, + selected: Option<&str>, +) -> Result, String> { + if prepared_source { + prepare_provider_features_for_identity(uid, gid, &["opencode-rebind-source"])?; + Ok(None) + } else { + let mut args = vec!["opencode-rebind"]; + if let Some(selected) = selected { + args.push(selected); + } + prepare_provider_features_for_identity(uid, gid, &args)?; + Ok(Some("/home/freshell/provider/.freshell/opencode/tui.json")) + } +} + fn opencode_plan( cwd: &str, context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, @@ -378,6 +403,10 @@ pub(crate) fn run_provider_features_worker(args: &[String]) -> Result<(), String .and_then(|mut file| std::io::Write::write_all(&mut file, &bytes)) .map_err(|error| error.to_string()) } + [operation] if operation == "opencode-rebind-source" => { + freshell_platform::opencode_plugin::ensure_rebind_plugin_source(provider_home) + .map_err(|_| "managed OpenCode rebind plugin installation failed".into()) + } [operation] | [operation, _] if operation == "opencode-rebind" => { let selected = args.get(1).map(std::path::Path::new); if let Some(selected) = selected { diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index baebea5ad..3c0dfec14 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -167,7 +167,9 @@ def create(name,*arguments): receipt["exitCode"]=exit_code receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) - if exit_code==0 and (len(receipt["consumedRoutes"])!=2 or "test result: ok. 1 passed;" not in outcome.stdout): + expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} + observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} + if exit_code==0 and (len(receipt["consumedRoutes"])!=3 or observed_routes!=expected_routes or "test result: ok. 1 passed;" not in outcome.stdout): receipt["exitCode"]=2;receipt["failureClass"]="native_startup_count_mismatch" (root/"broker-complete").write_text("ok") deadline=time.monotonic()+10 From 7e0ae0296e05f58b45789e88db2104febe54409d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 17:52:45 -0700 Subject: [PATCH 054/493] fix: validate managed Claude arguments against the shipped hook recipe --- Cargo.lock | 1 + crates/freshell-platform/src/cli_launch.rs | 15 +++++++- crates/freshell-runtime-protocol/Cargo.toml | 1 + crates/freshell-runtime-protocol/src/lib.rs | 38 ++++++++++++++++----- 4 files changed, 46 insertions(+), 9 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index dfcd8812b..787a804c6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1426,6 +1426,7 @@ dependencies = [ name = "freshell-runtime-protocol" version = "0.1.0" dependencies = [ + "freshell-platform", "hmac", "serde", "serde_json", diff --git a/crates/freshell-platform/src/cli_launch.rs b/crates/freshell-platform/src/cli_launch.rs index 0a307962d..fb5115548 100644 --- a/crates/freshell-platform/src/cli_launch.rs +++ b/crates/freshell-platform/src/cli_launch.rs @@ -241,6 +241,20 @@ pub const CLAUDE_SESSION_START_COMMAND_WINDOWS: &str = "powershell.exe -NoLogo - /// Built via `serde_json` (the workspace enables `preserve_order`, so key /// order is insertion order: `SessionStart` before `Stop`). pub fn claude_settings_json(target: ProviderTarget) -> String { + claude_settings_value(target).to_string() +} + +/// Validate against the same shipped recipe used by launch construction. +/// JSON key order is immaterial; additional settings or commands are rejected. +pub fn is_owned_claude_settings(raw: &str) -> bool { + serde_json::from_str::(raw).is_ok_and(|value| { + [ProviderTarget::Unix, ProviderTarget::Windows] + .into_iter() + .any(|target| value == claude_settings_value(target)) + }) +} + +fn claude_settings_value(target: ProviderTarget) -> serde_json::Value { let (session_start, bell) = match target { ProviderTarget::Windows => ( CLAUDE_SESSION_START_COMMAND_WINDOWS, @@ -254,7 +268,6 @@ pub fn claude_settings_json(target: ProviderTarget) -> String { "Stop": [{"hooks": [{"type": "command", "command": bell}]}], } }) - .to_string() } /// Truthy env lookup (JS `env.X` truthiness: unset and `''` are both falsy). diff --git a/crates/freshell-runtime-protocol/Cargo.toml b/crates/freshell-runtime-protocol/Cargo.toml index 0d6952761..de787d767 100644 --- a/crates/freshell-runtime-protocol/Cargo.toml +++ b/crates/freshell-runtime-protocol/Cargo.toml @@ -6,6 +6,7 @@ rust-version.workspace = true publish.workspace = true [dependencies] +freshell-platform = { path = "../freshell-platform" } serde = { workspace = true } serde_json = { workspace = true } tokio = { version = "1", features = ["io-util", "macros", "rt"] } diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index b2ec8c1c2..d653797c1 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2124,14 +2124,7 @@ pub fn validate_managed_terminal_args(mode: &str, args: &[String]) -> Result<(), } let value = args.get(index + 1).ok_or_else(invalid)?; let allowed = match (mode, flag) { - ("claude", "--settings") => { - let digest = format!("{:x}", sha2::Sha256::digest(value.as_bytes())); - matches!( - digest.as_str(), - "aa8900bab7bd9bcbb64634d60593a9217c1639b539cea23318122b3428d123c6" - | "d2dcb96c036160b2469bcd732fdc8d767dd481f9c0c864ce47d58ed057652c4d" - ) - } + ("claude", "--settings") => freshell_platform::cli_launch::is_owned_claude_settings(value), ("claude", "--mcp-config") => matches!( value.as_str(), ".claude/freshell-mcp.json" | "/home/freshell/provider/.claude/freshell-mcp.json" @@ -3403,6 +3396,35 @@ mod tests { ); } + #[test] + fn managed_claude_shipped_hook_recipe_is_accepted_and_custom_hooks_are_rejected() { + use freshell_platform::cli_launch::{claude_settings_json, ProviderTarget}; + for target in [ProviderTarget::Unix, ProviderTarget::Windows] { + let settings = claude_settings_json(target); + assert!(validate_managed_terminal_args( + "claude", + &["--settings".into(), settings.clone()] + ) + .is_ok()); + let mut changed: serde_json::Value = serde_json::from_str(&settings).unwrap(); + changed["hooks"]["Stop"][0]["hooks"][0]["command"] = + serde_json::json!("echo PRIVATE_UNAPPROVED_HOOK"); + let error = validate_managed_terminal_args( + "claude", + &["--settings".into(), changed.to_string()], + ) + .unwrap_err(); + assert!(!error.message.contains("PRIVATE_UNAPPROVED_HOOK")); + changed = serde_json::from_str(&settings).unwrap(); + changed["env"] = serde_json::json!({"API_KEY":"PRIVATE_UNAPPROVED_SETTING"}); + assert!(validate_managed_terminal_args( + "claude", + &["--settings".into(), changed.to_string()] + ) + .is_err()); + } + } + #[test] fn ids_reject_control_characters_and_are_distinct() { assert!(SoulId::parse("bad\nvalue").is_err()); From 60d09aa8ca34bbebdce928f4180aa8bc346a5db4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 17:57:13 -0700 Subject: [PATCH 055/493] fix: retain selected OpenCode rebind policy through preparation and reissue --- .../freshell-platform/src/opencode_recipe.rs | 5 + .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 168 ++++++++++++------ .../managed_mcp_capability/source_stage.rs | 2 + .../src/managed_provider_references.rs | 12 +- crates/freshell-server/src/managed_runtime.rs | 1 + .../src/provider_input_preparation.rs | 1 + 7 files changed, 133 insertions(+), 57 deletions(-) diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index c7a1f2433..c698e8271 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -38,6 +38,11 @@ pub struct SourceManifest { pub user_owns_freshell_mcp: bool, #[serde(default)] pub native_source_flags: Vec, + #[serde(default = "default_rebind_enabled")] + pub rebind_enabled: bool, +} +fn default_rebind_enabled() -> bool { + true } #[derive(Clone, Serialize, Deserialize)] diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 2a0aa8077..01dd40392 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -469,6 +469,7 @@ impl HostedFreshAgentProxy { auth_authority: None, selected_sources: None, native_source_flags: None, + rebind_enabled: None, }, ) .await diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index b5391fc4c..e342d2346 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -45,6 +45,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub launch_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, pub frozen_inputs: Option<&'a [String]>, pub native_source_flags: Option<&'a [freshell_platform::opencode_recipe::NativeSourceFlag]>, + pub rebind_enabled: Option, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -405,6 +406,9 @@ async fn apply_prepared_callback( native_source_flags: accepted_manifest .as_ref() .map(|manifest| manifest.native_source_flags.clone()), + rebind_enabled: accepted_manifest + .as_ref() + .map(|manifest| manifest.rebind_enabled), }, |soul, home, context, input| { let input = OpencodeEphemeralInput { @@ -1225,6 +1229,7 @@ fn stage_provider_root( plan, opencode_input.frozen_inputs.unwrap_or(&[]), opencode_input.native_source_flags.unwrap_or(&[]), + opencode_input.rebind_enabled.unwrap_or(true), )?; } allow_controller_stage_read(&temporary)?; @@ -3504,72 +3509,121 @@ except Exception as error: let directory = owned.path().join("control/mcp-capabilities"); std::fs::create_dir_all(&directory).unwrap(); let store = CapabilityStore::default(); - let soul = SoulId::new(); let client = freshell_runtime_client::RuntimeClient::new( owned.path().join("unused.sock"), "fixture-control", ); - let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( - crate::managed_provider_references::OpencodePreparation { - client: &client, - soul: &soul, - creation_seed: "fixture-existing-soul", - home: &home, - cwd: &workspace, - workspace: &workspace, - original: &|name| { - (name == "OPENCODE_TUI_CONFIG").then(|| "selected-tui.jsonc".into()) - }, - references: vec![], - auth_authority: Some("fixture-owned-attempt"), - selected_sources: None, - native_source_flags: None, - }, - |soul, home, context, input| { - let capability = store.issue( - soul.clone(), - &directory, - "http://host.docker.internal:4000".into(), - Some(("opencode", home, &context.config)), - OpencodeEphemeralInput { - launch_context: Some(context), - ..input + for disabled in [false, true] { + let soul = SoulId::new(); + let prepared = + crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "fixture-existing-soul", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| match name { + "OPENCODE_TUI_CONFIG" => Some("selected-tui.jsonc".into()), + "FRESHELL_OPENCODE_REBIND" if disabled => Some("0".into()), + _ => None, + }, + references: vec![], + auth_authority: Some("fixture-owned-attempt"), + selected_sources: None, + native_source_flags: None, + rebind_enabled: None, }, - None, - )?; - read_prepared_context(&directory, &capability) - }, - ) - .await - .expect("selected TUI must remain supported through the shared managed producer"); - let stage = staged_provider_root_path( - &directory, - &prepared.context.mcp_capability.as_ref().unwrap().grant_id, - ) - .unwrap(); - let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice( - &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), - ) - .unwrap(); - assert_eq!(manifest.native_source_flags.len(), 1); - assert_eq!( - manifest.native_source_flags[0].kind.variable(), - "OPENCODE_TUI_CONFIG" - ); - let effective = - std::fs::read_to_string(stage.join(&manifest.sources[0].unexpanded_relative_path)) + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .expect("selected TUI must remain supported through the shared managed producer"); + let stage = staged_provider_root_path( + &directory, + &prepared.context.mcp_capability.as_ref().unwrap().grant_id, + ) + .unwrap(); + let manifest: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) .unwrap(); - let effective: serde_json::Value = serde_json::from_str(&effective).unwrap(); - assert_eq!(effective["theme"], "{file:./theme.txt}"); - assert_eq!( + assert_eq!(manifest.rebind_enabled, !disabled); + assert_eq!(manifest.native_source_flags.len(), 1); + assert_eq!( + manifest.native_source_flags[0].kind.variable(), + "OPENCODE_TUI_CONFIG" + ); + let effective = + std::fs::read_to_string(stage.join(&manifest.sources[0].unexpanded_relative_path)) + .unwrap(); + let effective = + freshell_platform::opencode_config::parse_jsonc_object(&effective).unwrap(); + assert_eq!(effective["theme"], "{file:./theme.txt}"); + assert_eq!( effective["plugin"], - serde_json::json!([ + if disabled { serde_json::json!([]) } else { serde_json::json!([ "file:///home/freshell/provider/.freshell/opencode/freshell-rebind-plugin.ts" - ]), + ]) }, "selected rebind normalization must precede the source view used by reference recipes" ); - assert_eq!(prepared.references[0].source_path, theme.to_str().unwrap()); - assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + assert_eq!(prepared.references[0].source_path, theme.to_str().unwrap()); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + let callback = serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul, + "provider":"opencode","workspace":workspace,"cwd":workspace,"priorContext":prepared.context, + "providerSecretReferences":prepared.references,"authLookupAuthority":"fixture-owned-attempt"})).unwrap(); + let reissued = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(); + let reissued: PreparedProviderContext = serde_json::from_slice(&reissued).unwrap(); + let stage = staged_provider_root_path( + &directory, + &reissued + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap() + .grant_id, + ) + .unwrap(); + let retained: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert_eq!( + retained.rebind_enabled, !disabled, + "reissue must preserve request-only owned edits with current control absent" + ); + let retained = freshell_platform::opencode_config::parse_jsonc_object( + &std::fs::read_to_string(stage.join(&retained.sources[0].unexpanded_relative_path)) + .unwrap(), + ) + .unwrap(); + assert_eq!(retained["plugin"], effective["plugin"]); + } } #[tokio::test] @@ -3606,6 +3660,7 @@ except Exception as error: auth_authority: Some("fixture-owned-attempt"), selected_sources: None, native_source_flags: None, + rebind_enabled: None, }, |soul, home, context, input| { let capability = store.issue( @@ -3694,6 +3749,7 @@ except Exception as error: auth_authority: Some("fixture-owned-attempt"), selected_sources: None, native_source_flags: None, + rebind_enabled: None, }, |_, _, _, _| panic!("missing selected native source must fail before publication"), ) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 9c9b1c053..710aa5b12 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -11,6 +11,7 @@ pub(super) fn stage( plan: &[PlannedSourceOccurrence], frozen_inputs: &[String], native_source_flags: &[freshell_platform::opencode_recipe::NativeSourceFlag], + rebind_enabled: bool, ) -> Result<(), String> { let mut manifest = SourceManifest { version: 1, @@ -21,6 +22,7 @@ pub(super) fn stage( frozen_inputs: frozen_inputs.to_vec(), user_owns_freshell_mcp: false, native_source_flags: native_source_flags.to_vec(), + rebind_enabled, }; private_directory(&root.join(".freshell/source-views"))?; private_directory(&root.join(".freshell/templates"))?; diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index ebf70fa01..44ed958ce 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -715,6 +715,7 @@ pub(crate) struct OpencodePreparation<'a> { pub auth_authority: Option<&'a str>, pub selected_sources: Option>, pub native_source_flags: Option>, + pub rebind_enabled: Option, } pub(crate) async fn prepare_managed_opencode( @@ -748,7 +749,14 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( auth_authority, selected_sources, native_source_flags, + rebind_enabled, } = request; + let rebind_enabled = rebind_enabled.unwrap_or_else(|| { + !matches!( + original("FRESHELL_OPENCODE_REBIND").as_deref(), + Some("0" | "false") + ) + }); use sources::*; if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) { return Err( @@ -830,7 +838,8 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( .try_fold(0usize, |bytes, source| bytes.checked_add(source.view.len())) .ok_or("source-view byte count overflow")?; for source in &mut loaded { - if source.source.lane == SourceLane::Tui + if rebind_enabled + && source.source.lane == SourceLane::Tui && native_source_flags.iter().any(|flag| { flag.kind == freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui && Path::new(&flag.runtime_destination) @@ -947,6 +956,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( secret_references: Some(&references), frozen_inputs: Some(&frozen_names), native_source_flags: Some(&native_source_flags), + rebind_enabled: Some(rebind_enabled), ..Default::default() }, )?; diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index ba4cf8e50..38d8d0f7d 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -212,6 +212,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { auth_authority: None, selected_sources: None, native_source_flags: None, + rebind_enabled: None, }, ) .await?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 30a18336b..fd57f7c8a 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -229,6 +229,7 @@ mod tests { }], frozen_inputs: vec!["INSERT".into()], user_owns_freshell_mcp: true, + rebind_enabled: true, native_source_flags: vec![NativeSourceFlag { kind: NativeSourceFlagKind::Main, runtime_destination: "/original-unavailable/project/opencode.json".into(), From 9474e879f1362ce84b4e8192c9e7648ee5947207 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 18:16:38 -0700 Subject: [PATCH 056/493] fix: prepare inline OpenCode configuration before native file resolution --- crates/freshell-opencode/src/serve.rs | 1 + .../freshell-platform/src/opencode_config.rs | 54 +++++- .../freshell-platform/src/opencode_plugin.rs | 30 +-- .../freshell-platform/src/opencode_recipe.rs | 5 + crates/freshell-runtime-protocol/src/lib.rs | 2 +- .../freshell-server/src/fresh_agent_proxy.rs | 5 + .../src/managed_mcp_capability.rs | 180 +++++++++++++++++- .../managed_mcp_capability/source_stage.rs | 119 ++++++++---- .../src/managed_provider_launch_tests.rs | 29 ++- .../src/managed_provider_references.rs | 67 ++++++- .../environment.rs | 5 +- .../managed_provider_references/sources.rs | 1 + crates/freshell-server/src/managed_runtime.rs | 2 + .../src/provider_input_preparation.rs | 63 +++++- .../src/providers/mod.rs | 5 +- .../managed-provider-launch-sandbox.py | 2 +- 16 files changed, 479 insertions(+), 91 deletions(-) diff --git a/crates/freshell-opencode/src/serve.rs b/crates/freshell-opencode/src/serve.rs index fe910db49..1b67993d2 100644 --- a/crates/freshell-opencode/src/serve.rs +++ b/crates/freshell-opencode/src/serve.rs @@ -64,6 +64,7 @@ pub fn merged_opencode_config_content(inherited: Option<&str>) -> Result, pub user_owns_freshell_mcp: bool, + pub inline_prepared: bool, } -#[derive(Clone, Debug, Default, PartialEq, Eq)] +#[derive(Clone, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)] pub struct OwnedConfigEdits { pub disable_snapshots: bool, pub bash_permission: Option, @@ -136,6 +137,7 @@ pub fn prepare_opencode_config( Ok(OpencodeConfigPlan { sources, user_owns_freshell_mcp, + inline_prepared: false, }) } @@ -197,6 +199,15 @@ pub fn merge_owned_entries( inherited: Option<&str>, plan: &OpencodeConfigPlan, edits: &OwnedConfigEdits, +) -> Result { + merge_owned_entries_bounded(inherited, plan, edits, 16 * 1024 * 1024) +} + +pub fn merge_owned_entries_bounded( + inherited: Option<&str>, + plan: &OpencodeConfigPlan, + edits: &OwnedConfigEdits, + limit: usize, ) -> Result { let mut config = match inherited.filter(|raw| !raw.is_empty()) { Some(raw) => parse_jsonc_object(raw).ok_or_else(|| ConfigError::Malformed { @@ -239,7 +250,44 @@ pub fn merge_owned_entries( ); } } - Ok(config.to_string()) + serialize_source_view(&config, limit).map_err(|source| ConfigError::Io { + path: "OPENCODE_CONFIG_CONTENT".into(), + source, + }) +} + +/// Serialize an effective source without allocating beyond its accepted view. +pub fn serialize_source_view(config: &Value, limit: usize) -> std::io::Result { + use std::io::Write; + struct BoundedView { + bytes: Vec, + limit: usize, + } + impl Write for BoundedView { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + if self + .bytes + .len() + .checked_add(bytes.len()) + .is_none_or(|size| size > self.limit) + { + return Err(std::io::Error::other( + "OpenCode effective source-view profile exceeded", + )); + } + self.bytes.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + let mut view = BoundedView { + bytes: Vec::new(), + limit: limit.min(16 * 1024 * 1024), + }; + serde_json::to_writer(&mut view, config).map_err(std::io::Error::other)?; + String::from_utf8(view.bytes).map_err(std::io::Error::other) } /// Normalize the JSONC accepted by OpenCode's `jsonc-parser` (comments and @@ -380,6 +428,7 @@ mod tests { &OpencodeConfigPlan { sources: vec![], user_owns_freshell_mcp: false, + inline_prepared: false, }, &OwnedConfigEdits::default(), ) @@ -463,6 +512,7 @@ mod tests { let plan = OpencodeConfigPlan { sources: vec![], user_owns_freshell_mcp: false, + inline_prepared: false, }; let raw = "{ // inline user settings\n\"provider\":{\"demo\":{}},\"mcp\":{\"freshell\":{\"type\":\"local\",\"command\":[\"user-command\"]}},\"permission\":{\"edit\":\"ask\"},}"; let merged = merge_owned_entries( diff --git a/crates/freshell-platform/src/opencode_plugin.rs b/crates/freshell-platform/src/opencode_plugin.rs index e789a74e3..e6cf9ba14 100644 --- a/crates/freshell-platform/src/opencode_plugin.rs +++ b/crates/freshell-platform/src/opencode_plugin.rs @@ -166,35 +166,7 @@ pub fn selected_tui_content_bounded( { plugins.push(serde_json::Value::String(own)); } - struct BoundedView { - bytes: Vec, - limit: usize, - } - impl Write for BoundedView { - fn write(&mut self, bytes: &[u8]) -> std::io::Result { - if self - .bytes - .len() - .checked_add(bytes.len()) - .is_none_or(|size| size > self.limit) - { - return Err(std::io::Error::other( - "OpenCode effective TUI source-view profile exceeded", - )); - } - self.bytes.extend_from_slice(bytes); - Ok(bytes.len()) - } - fn flush(&mut self) -> std::io::Result<()> { - Ok(()) - } - } - let mut view = BoundedView { - bytes: Vec::new(), - limit: remaining.min(16 * 1024 * 1024), - }; - serde_json::to_writer(&mut view, &config).map_err(std::io::Error::other)?; - String::from_utf8(view.bytes).map_err(std::io::Error::other) + crate::opencode_config::serialize_source_view(&config, remaining) } /// Install only the implementation when the controller already prepared the diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index c698e8271..f42a3d1cc 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -40,6 +40,8 @@ pub struct SourceManifest { pub native_source_flags: Vec, #[serde(default = "default_rebind_enabled")] pub rebind_enabled: bool, + #[serde(default)] + pub inline_edits: crate::opencode_config::OwnedConfigEdits, } fn default_rebind_enabled() -> bool { true @@ -72,6 +74,8 @@ pub struct SourceView { pub lane: SourceLane, pub unexpanded_relative_path: String, pub template_relative_path: String, + #[serde(default)] + pub template_sha256: Option, pub generated_name: Option, pub occurrences: Vec, pub referenced_variables: Vec, @@ -85,6 +89,7 @@ pub struct SourceView { pub enum SourceLane { Main, Tui, + Inline, } #[derive(Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index d653797c1..235bb7d59 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -6,7 +6,7 @@ use hmac::{Hmac, Mac}; use serde::{de::DeserializeOwned, Deserialize, Serialize}; -use sha2::{Digest, Sha256}; +use sha2::Sha256; use std::fmt; use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}; diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 01dd40392..a1e7852bb 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -470,6 +470,11 @@ impl HostedFreshAgentProxy { selected_sources: None, native_source_flags: None, rebind_enabled: None, + inherited_inline: None, + inline_edits: Some(freshell_platform::opencode_config::OwnedConfigEdits { + disable_snapshots: true, + ..Default::default() + }), }, ) .await diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e342d2346..021941d95 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -46,6 +46,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub frozen_inputs: Option<&'a [String]>, pub native_source_flags: Option<&'a [freshell_platform::opencode_recipe::NativeSourceFlag]>, pub rebind_enabled: Option, + pub inline_edits: Option<&'a freshell_platform::opencode_config::OwnedConfigEdits>, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -376,16 +377,47 @@ async fn apply_prepared_callback( .or_else(|| std::env::var(name).ok()) }; let selected_sources=accepted_manifest.as_ref().map(|manifest|{ - manifest.sources.iter().enumerate().map(|(ordinal,source)|{ + manifest.sources.iter().enumerate().filter(|(_, source)| source.lane != freshell_platform::opencode_recipe::SourceLane::Inline).map(|(ordinal,source)|{ let canonical=std::fs::canonicalize(&source.logical_path) .map_err(|_|format!("prior selected configuration source {ordinal} is unavailable; repair it before reissue"))?; Ok(crate::managed_provider_references::sources::SourceOccurrence { logical_path:source.logical_path.clone(),canonical_source:canonical, lane:match source.lane {freshell_platform::opencode_recipe::SourceLane::Main=>crate::managed_provider_references::sources::SourceLane::Main, - freshell_platform::opencode_recipe::SourceLane::Tui=>crate::managed_provider_references::sources::SourceLane::Tui}, + freshell_platform::opencode_recipe::SourceLane::Tui=>crate::managed_provider_references::sources::SourceLane::Tui, + freshell_platform::opencode_recipe::SourceLane::Inline=>crate::managed_provider_references::sources::SourceLane::Inline}, }) }).collect::,String>>() }).transpose()?; + let inherited_inline = accepted_manifest + .as_ref() + .map(|manifest| { + let mut sources = manifest.sources.iter().filter(|source| { + source.lane == freshell_platform::opencode_recipe::SourceLane::Inline + }); + let Some(source) = sources.next() else { + return Ok(None); + }; + if sources.next().is_some() + || source.logical_path != Path::new("OPENCODE_CONFIG_CONTENT") + { + return Err("accepted inline source identity is invalid".to_string()); + } + let context = callback + .prior_context + .as_ref() + .ok_or("accepted inline source has no prior context")?; + let root = staged_provider_root_path( + directory, + &context + .mcp_capability + .as_ref() + .ok_or("accepted inline source has no generation")? + .grant_id, + )?; + source_stage::read_source_view(&root, &source.unexpanded_relative_path).map(Some) + }) + .transpose()? + .flatten(); let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( crate::managed_provider_references::OpencodePreparation { client, @@ -409,6 +441,10 @@ async fn apply_prepared_callback( rebind_enabled: accepted_manifest .as_ref() .map(|manifest| manifest.rebind_enabled), + inherited_inline, + inline_edits: accepted_manifest + .as_ref() + .map(|manifest| manifest.inline_edits.clone()), }, |soul, home, context, input| { let input = OpencodeEphemeralInput { @@ -1227,9 +1263,7 @@ fn stage_provider_root( home, opencode_input.cwd.ok_or("source plan requires cwd")?, plan, - opencode_input.frozen_inputs.unwrap_or(&[]), - opencode_input.native_source_flags.unwrap_or(&[]), - opencode_input.rebind_enabled.unwrap_or(true), + opencode_input, )?; } allow_controller_stage_read(&temporary)?; @@ -3534,6 +3568,8 @@ except Exception as error: selected_sources: None, native_source_flags: None, rebind_enabled: None, + inherited_inline: None, + inline_edits: None, }, |soul, home, context, input| { let capability = store.issue( @@ -3626,6 +3662,136 @@ except Exception as error: } } + #[tokio::test] + async fn managed_provider_inline_source_uses_instance_origin_after_owned_mcp_edits() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let config = workspace.join("selected-tui.jsonc"); + let raw = r#"{"agent":{"probe":{"prompt":"{file:./theme.txt}"}}}"#; + std::fs::write(&config, raw).unwrap(); + let theme = workspace.join("theme.txt"); + std::fs::write(&theme, "dark").unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + for fresh in [false, true] { + let soul = SoulId::new(); + let prepared = + crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "fixture-existing-soul", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| (name == "OPENCODE_CONFIG_CONTENT").then(|| raw.into()), + references: vec![], + auth_authority: Some("fixture-owned-attempt"), + selected_sources: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: Some(freshell_platform::opencode_config::OwnedConfigEdits { + disable_snapshots: fresh, + ..Default::default() + }), + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .expect("inline config must remain supported through the shared managed producer"); + let stage = staged_provider_root_path( + &directory, + &prepared.context.mcp_capability.as_ref().unwrap().grant_id, + ) + .unwrap(); + let manifest: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert!(manifest.native_source_flags.is_empty()); + let effective = + std::fs::read_to_string(stage.join(&manifest.sources[0].unexpanded_relative_path)) + .unwrap(); + let effective: serde_json::Value = serde_json::from_str(&effective).unwrap(); + assert_eq!(effective["agent"]["probe"]["prompt"], "{file:./theme.txt}"); + assert_eq!( + effective["mcp"]["freshell"]["command"], + serde_json::json!(["node", "/opt/freshell-mcp/server.js"]) + ); + assert_eq!(manifest.inline_edits.disable_snapshots, fresh); + assert_eq!( + effective["snapshot"], + if fresh { + serde_json::json!(false) + } else { + serde_json::Value::Null + } + ); + assert_eq!(prepared.references[0].source_path, theme.to_str().unwrap()); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + let callback = serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul, + "provider":"opencode","workspace":workspace,"cwd":workspace,"priorContext":prepared.context, + "providerSecretReferences":prepared.references,"authLookupAuthority":"fixture-owned-attempt"})).unwrap(); + let reissued = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(); + let reissued: PreparedProviderContext = serde_json::from_slice(&reissued).unwrap(); + let stage = staged_provider_root_path( + &directory, + &reissued + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap() + .grant_id, + ) + .unwrap(); + let retained: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + let retained = freshell_platform::opencode_config::parse_jsonc_object( + &std::fs::read_to_string(stage.join(&retained.sources[0].unexpanded_relative_path)) + .unwrap(), + ) + .unwrap(); + assert_eq!(retained, effective, "request-only inline source and owned edits must survive reissue without current inline env"); + } + } + #[tokio::test] async fn managed_provider_explicit_native_source_selection_survives_preparation() { use freshell_runtime_protocol::*; @@ -3661,6 +3827,8 @@ except Exception as error: selected_sources: None, native_source_flags: None, rebind_enabled: None, + inherited_inline: None, + inline_edits: None, }, |soul, home, context, input| { let capability = store.issue( @@ -3750,6 +3918,8 @@ except Exception as error: selected_sources: None, native_source_flags: None, rebind_enabled: None, + inherited_inline: None, + inline_edits: None, }, |_, _, _, _| panic!("missing selected native source must fail before publication"), ) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 710aa5b12..beab21523 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -4,14 +4,39 @@ use crate::managed_provider_references::sources::{PlannedSourceOccurrence, Sourc use freshell_platform::opencode_recipe::{SelectedSourceFile, SourceManifest, SourceView}; use std::{collections::BTreeMap, path::Path}; +pub(super) fn read_source_view(root: &Path, relative: &str) -> Result { + use std::io::Read; + if Path::new(relative) + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("accepted source view has an invalid private path".into()); + } + let path = root.join(relative); + if !std::fs::symlink_metadata(&path) + .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) + || std::fs::canonicalize(&path).map_err(|_| "accepted source view is unavailable")? != path + { + return Err("accepted source view must be a physical stage file".into()); + } + let mut raw = String::new(); + std::fs::File::open(path) + .map_err(|_| "accepted source view is unavailable")? + .take(16 * 1024 * 1024 + 1) + .read_to_string(&mut raw) + .map_err(|_| "accepted source view read failed")?; + if raw.len() > 16 * 1024 * 1024 { + return Err("accepted source view exceeds 16 MiB".into()); + } + Ok(raw) +} + pub(super) fn stage( root: &Path, home: &Path, cwd: &Path, plan: &[PlannedSourceOccurrence], - frozen_inputs: &[String], - native_source_flags: &[freshell_platform::opencode_recipe::NativeSourceFlag], - rebind_enabled: bool, + input: super::OpencodeEphemeralInput<'_>, ) -> Result<(), String> { let mut manifest = SourceManifest { version: 1, @@ -19,10 +44,11 @@ pub(super) fn stage( original_home: home.into(), sources: Vec::new(), files: Vec::new(), - frozen_inputs: frozen_inputs.to_vec(), + frozen_inputs: input.frozen_inputs.unwrap_or(&[]).to_vec(), user_owns_freshell_mcp: false, - native_source_flags: native_source_flags.to_vec(), - rebind_enabled, + native_source_flags: input.native_source_flags.unwrap_or(&[]).to_vec(), + rebind_enabled: input.rebind_enabled.unwrap_or(true), + inline_edits: input.inline_edits.cloned().unwrap_or_default(), }; private_directory(&root.join(".freshell/source-views"))?; private_directory(&root.join(".freshell/templates"))?; @@ -75,9 +101,17 @@ pub(super) fn stage( lane: match source.source.lane { SourceLane::Main => freshell_platform::opencode_recipe::SourceLane::Main, SourceLane::Tui => freshell_platform::opencode_recipe::SourceLane::Tui, + SourceLane::Inline => freshell_platform::opencode_recipe::SourceLane::Inline, }, unexpanded_relative_path: raw, template_relative_path: template, + template_sha256: Some({ + use sha2::Digest; + format!( + "{:x}", + sha2::Sha256::digest(source.references.template.as_bytes()) + ) + }), generated_name: source .references .recipes @@ -128,46 +162,51 @@ pub(super) fn describe( .or_default() += 1; } for (ordinal, occurrence) in plan.iter().enumerate() { - if canonical_occurrences[occurrence.source.canonical_source.as_path()] > 1 { + if occurrence.source.lane != SourceLane::Inline + && canonical_occurrences[occurrence.source.canonical_source.as_path()] > 1 + { return Err( "shared source preparation requires its original-form aliases before publication" .into(), ); } - let logical = &occurrence.source.logical_path; - let destination = crate::managed_provider_references::sources::runtime_config_destination( - home, global, logical, - ); - let effective = if let Ok(relative) = logical.strip_prefix(&global) { - let physical = std::fs::canonicalize(root.join(relative)) - .map_err(|_| "selected global config is unavailable in its private stage")?; - let relative = physical - .strip_prefix(root) - .map_err(|_| "selected global config escaped its owned stage")?; - if relative.starts_with(".freshell/resources") { - runtime_home.join(relative) + if occurrence.source.lane != SourceLane::Inline { + let logical = &occurrence.source.logical_path; + let destination = + crate::managed_provider_references::sources::runtime_config_destination( + home, global, logical, + ); + let effective = if let Ok(relative) = logical.strip_prefix(&global) { + let physical = std::fs::canonicalize(root.join(relative)) + .map_err(|_| "selected global config is unavailable in its private stage")?; + let relative = physical + .strip_prefix(root) + .map_err(|_| "selected global config escaped its owned stage")?; + if relative.starts_with(".freshell/resources") { + runtime_home.join(relative) + } else { + runtime_home.join(".config/opencode").join(relative) + } } else { - runtime_home.join(".config/opencode").join(relative) - } - } else { - occurrence - .source - .canonical_source - .strip_prefix(home) - .map(|relative| runtime_home.join(relative)) - .unwrap_or_else(|_| occurrence.source.canonical_source.clone()) - }; - binds.push(ProviderConfigBind { - source_relative_path: format!(".freshell/templates/source-{ordinal}"), - logical_destination: destination - .to_str() - .ok_or("config destination is not UTF-8")? - .into(), - effective_destination: effective - .to_str() - .ok_or("config destination is not UTF-8")? - .into(), - }); + occurrence + .source + .canonical_source + .strip_prefix(home) + .map(|relative| runtime_home.join(relative)) + .unwrap_or_else(|_| occurrence.source.canonical_source.clone()) + }; + binds.push(ProviderConfigBind { + source_relative_path: format!(".freshell/templates/source-{ordinal}"), + logical_destination: destination + .to_str() + .ok_or("config destination is not UTF-8")? + .into(), + effective_destination: effective + .to_str() + .ok_or("config destination is not UTF-8")? + .into(), + }); + } recipes = recipes .checked_add(occurrence.references.recipes.len()) .ok_or("source recipe count overflow")?; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 534bfdf57..7c22fb550 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -248,12 +248,13 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr // auth-store key before reading local configuration. Keep that dependency // inside the same owned synthetic fixture as the launch consumers. let well_known_url = format!("http://host.docker.internal:{port}"); + let inline_auth_url = format!("{well_known_url}/inline-auth"); std::fs::write( fixture.join("grants/auth.json"), serde_json::to_vec(&json!({well_known_url: { "type": "wellknown", "key": "OPENAI_API_KEY", "token": "AUTH_PREFIX{file:./rules.txt}AUTH_SUFFIX" - }})) + }, inline_auth_url: {"type":"wellknown","key":"INLINE_VALUE","token":"AUTH_INLINE_SHADOW_NO_FILE"}})) .unwrap(), ) .unwrap(); @@ -303,6 +304,15 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ let selected_tui_bytes = format!("{{ // selected origin\n\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&tui_receipt).unwrap()); std::fs::write(&selected_tui, &selected_tui_bytes).unwrap(); std::fs::write(fixture.join("workspace/selected-theme"), " dark\n").unwrap(); + let inline_source = r#"{ // inline instance origin +"agent":{"probe":{"description":"{env:INLINE_VALUE}"}}}"#; + let inline_value = "INLINE_PREFIX{file:./inline-rule}INLINE_SUFFIX"; + std::fs::write( + fixture.join("workspace/inline-rule"), + "native-inline-selected-rule\n", + ) + .unwrap(); + let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { program: "opencode".into(), @@ -315,6 +325,8 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ env: BTreeMap::from([ ("OPENCODE_CONFIG".into(), "selected-config.json".into()), ("OPENCODE_TUI_CONFIG".into(), "selected-tui.jsonc".into()), + ("OPENCODE_CONFIG_CONTENT".into(), inline_source.into()), + ("INLINE_VALUE".into(), inline_value.into()), ]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), @@ -348,10 +360,10 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .map_err(|_| "actual pinned provider startup did not consume selected config and closure") } let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXnative-ruleAUTH_SUFFIX")), - "description":format!("{:x}",Sha256::digest(b"native-explicit-selected-rule")), + "description":format!("{:x}",Sha256::digest(b"INLINE_PREFIXnative-inline-selected-ruleINLINE_SUFFIX")), "references":(["third","fourth","fifth","sixth"].map(|ordinal|format!("{:x}",Sha256::digest(format!("native-{ordinal}-rule").as_bytes())))), "resource":format!("{:x}",Sha256::digest(b"plugin-relative-resource")), - "dependency":format!("{:x}",Sha256::digest(b"plugin-actual-local-dependency")),"sdk":format!("{:x}",Sha256::digest(b"function"))}); + "dependency":format!("{:x}",Sha256::digest(b"plugin-actual-local-dependency")),"sdk":format!("{:x}",Sha256::digest(b"function")),"snapshotDisabled":false}); let client = controller.runtime_client(); let terminal_receipt = match read_receipt(&receipt).await { Ok(value) => value, @@ -422,6 +434,10 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ // request-only relative control. Both must activate the explicit layer. std::env::set_var("OPENCODE_CONFIG", &explicit_config); std::env::set_var("OPENCODE_TUI_CONFIG", &selected_tui); + std::env::set_var("OPENCODE_CONFIG_CONTENT", inline_source); + std::env::set_var("INLINE_VALUE", inline_value); + let mut fresh_expected = expected.clone(); + fresh_expected["snapshotDisabled"] = json!(true); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( @@ -461,7 +477,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ Arc::clone(&proxy) .dispatch(HostedFreshAgentCommand::Send(send)) .await; - assert_eq!(read_receipt(&receipt).await.unwrap(), expected); + assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); let stopped = fixture.join("workspace/native-consumed.json.stopped"); @@ -479,7 +495,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .expect("native plugin must stop the fresh request before the model loop"); println!( "provider_launch_consumer {}", - json!({"provider":"opencode","route":"fresh","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) + json!({"provider":"opencode","route":"fresh","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&fresh_expected).unwrap()))}) ); // Scan only run-owned durable/staged surfaces; referenced payloads belong // in private indexed files, not ordinary resource/configuration trees. @@ -495,6 +511,9 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ b"AUTH_PREFIX".as_slice(), b"native-rule".as_slice(), b"native-explicit-selected-rule".as_slice(), + b"native-inline-selected-rule".as_slice(), + b"INLINE_PREFIX".as_slice(), + b"AUTH_INLINE_SHADOW_NO_FILE".as_slice(), b"synthetic-onecli-shadowed".as_slice(), ] { assert!( diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 44ed958ce..01c04b401 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -716,6 +716,8 @@ pub(crate) struct OpencodePreparation<'a> { pub selected_sources: Option>, pub native_source_flags: Option>, pub rebind_enabled: Option, + pub inherited_inline: Option, + pub inline_edits: Option, } pub(crate) async fn prepare_managed_opencode( @@ -750,6 +752,8 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( selected_sources, native_source_flags, rebind_enabled, + inherited_inline, + inline_edits, } = request; let rebind_enabled = rebind_enabled.unwrap_or_else(|| { !matches!( @@ -758,10 +762,14 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( ) }); use sources::*; - if original("OPENCODE_CONFIG_CONTENT").is_some_and(|value| !value.is_empty()) { - return Err( - "selected OpenCode inline/TUI source requires its owned pre-recipe view".into(), - ); + let inherited_inline = inherited_inline + .or_else(|| original("OPENCODE_CONFIG_CONTENT")) + .filter(|raw| !raw.is_empty()); + let mut inline_edits = inline_edits.unwrap_or_default(); + inline_edits.freshell_command = Some(vec!["node".into(), "/opt/freshell-mcp/server.js".into()]); + if inline_edits.bash_permission.is_none() { + inline_edits.bash_permission = original("FRESHELL_MANAGED_OPENCODE_BASH_PERMISSION") + .filter(|value| matches!(value.as_str(), "allow" | "ask" | "deny")); } let xdg = original("XDG_CONFIG_HOME") .filter(|value| !value.is_empty()) @@ -795,6 +803,56 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( .map_err(str::to_string)? }; let mut loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; + if let Some(raw) = inherited_inline { + if raw.len() > 16 * 1024 * 1024 { + return Err("OpenCode inherited inline source exceeds 16 MiB".into()); + } + let retained = loaded + .iter() + .try_fold(raw.len(), |bytes, source| { + bytes.checked_add(source.view.len()) + }) + .ok_or("source-view size overflow")?; + let remaining = (64 * 1024 * 1024usize) + .checked_sub(retained) + .ok_or("simultaneous source views exceed 64 MiB")?; + let user_owns_freshell_mcp = loaded + .iter() + .filter(|source| source.source.lane == SourceLane::Main) + .try_fold(false, |owned, source| { + freshell_platform::opencode_config::source_owns_freshell_mcp( + &source.view, + &source.source.logical_path, + cwd, + ) + .map(|user_owns| owned || user_owns) + }) + .map_err(|error| error.to_string())?; + let view = freshell_platform::opencode_config::merge_owned_entries_bounded( + Some(&raw), + &freshell_platform::opencode_config::OpencodeConfigPlan { + sources: vec![], + user_owns_freshell_mcp, + inline_prepared: false, + }, + &inline_edits, + remaining.min(64 * 1024), + ) + .map_err(|error| error.to_string())?; + let inline = LoadedSourceOccurrence { + source: SourceOccurrence { + logical_path: "OPENCODE_CONFIG_CONTENT".into(), + canonical_source: "OPENCODE_CONFIG_CONTENT".into(), + lane: SourceLane::Inline, + }, + view: std::sync::Arc::new(view), + }; + let position = loaded + .iter() + .position(|source| source.source.lane == SourceLane::Tui) + .unwrap_or(loaded.len()); + loaded.insert(position, inline); + } let native_source_flags = if let Some(flags) = native_source_flags { flags } else { @@ -957,6 +1015,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( frozen_inputs: Some(&frozen_names), native_source_flags: Some(&native_source_flags), rebind_enabled: Some(rebind_enabled), + inline_edits: Some(&inline_edits), ..Default::default() }, )?; diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index bad92c90e..7aafc3cfd 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -168,7 +168,10 @@ impl SourceEnvironment { } let generated_count = plan .iter() - .map(|source| source.references.recipes.len()) + .map(|source| { + source.references.recipes.len() + + usize::from(source.source.lane == SourceLane::Inline) + }) .try_fold(0usize, usize::checked_add) .ok_or("selected/generated source name count overflow")?; if selected diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 77ab0c9f5..3dd655a2b 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -5,6 +5,7 @@ use std::path::{Path, PathBuf}; pub enum SourceLane { Main, Tui, + Inline, } pub struct SourceOccurrence { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 38d8d0f7d..2a781af7e 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -213,6 +213,8 @@ impl ManagedTerminalController for ServerManagedRuntimeController { selected_sources: None, native_source_flags: None, rebind_enabled: None, + inherited_inline: None, + inline_edits: None, }, ) .await?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index fd57f7c8a..31e07f123 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -102,8 +102,13 @@ pub fn prepare( let plan = freshell_platform::opencode_config::OpencodeConfigPlan { sources: vec![], user_owns_freshell_mcp: manifest.user_owns_freshell_mcp, + inline_prepared: manifest + .sources + .iter() + .any(|source| source.lane == SourceLane::Inline), }; let mut generated = BTreeMap::new(); + let mut budget = frozen.clone(); if manifest.native_source_flags.len() > 2 { return Err("prepared native source selector profile exceeded".into()); } @@ -118,10 +123,12 @@ pub fn prepare( { return Err("prepared native source selector has no approved config bind or collides with child input".into()); } + budget + .insert(name.into(), flag.runtime_destination.clone()) + .map_err(|error| error.message)?; generated.insert(name.into(), flag.runtime_destination.clone()); } let mut views = 0usize; - let mut budget = frozen.clone(); for source in manifest.sources { let remaining = (64 * 1024 * 1024usize) .checked_sub(views) @@ -165,6 +172,28 @@ pub fn prepare( freshell_platform::opencode_recipe::evaluate_document(&recipe, &lookup) } .map_err(str::to_string)?; + if source.lane == SourceLane::Inline { + let template = String::from_utf8(read_private( + stage, + &source.template_relative_path, + 64 * 1024, + )?) + .map_err(|_| "prepared inline source is not UTF-8")?; + if source.template_sha256.as_deref() + != Some(format!("{:x}", Sha256::digest(template.as_bytes())).as_str()) + { + return Err("prepared inline template changed before dispatch; reprepare the source generation".into()); + } + if generated.contains_key("OPENCODE_CONFIG_CONTENT") + || secrets.environment.contains_key("OPENCODE_CONFIG_CONTENT") + { + return Err("prepared inline source collides with another child source".into()); + } + budget + .insert("OPENCODE_CONFIG_CONTENT".into(), template.clone()) + .map_err(|error| error.message)?; + generated.insert("OPENCODE_CONFIG_CONTENT".into(), template); + } if let Some(name) = source.generated_name { if secrets.environment.contains_key(&name) || auth.contains_key(&name) @@ -217,6 +246,7 @@ mod tests { lane: SourceLane::Main, unexpanded_relative_path: ".freshell/source-views/source-0".into(), template_relative_path: ".freshell/templates/source-0".into(), + template_sha256: None, generated_name: Some("FRESHELL_CONFIG_REFERENCE_0".into()), occurrences: vec![ApprovedFileOccurrence { ordinal: 0, @@ -229,6 +259,7 @@ mod tests { }], frozen_inputs: vec!["INSERT".into()], user_owns_freshell_mcp: true, + inline_edits: Default::default(), rebind_enabled: true, native_source_flags: vec![NativeSourceFlag { kind: NativeSourceFlagKind::Main, @@ -300,6 +331,36 @@ mod tests { Some("/original-unavailable/project/opencode.json"), "an explicitly selected native source must be delivered to the actual child environment"); assert!(!generated.contains_key("INSERT"),"independent generated recipes do not forward an original ambient variable into the native child"); + std::fs::create_dir_all(stage.join(".freshell/templates")).unwrap(); + let inline_template = "{env:FRESHELL_CONFIG_REFERENCE_0}"; + std::fs::write(stage.join(".freshell/templates/source-0"), inline_template).unwrap(); + manifest.sources[0].lane = SourceLane::Inline; + manifest.sources[0].template_sha256 = + Some(format!("{:x}", Sha256::digest(inline_template.as_bytes()))); + let native_flags = std::mem::take(&mut manifest.native_source_flags); + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let inline = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap(); + assert_eq!( + inline.get("OPENCODE_CONFIG_CONTENT").map(String::as_str), + Some(inline_template) + ); + assert!(secrets.opencode_plan.as_ref().unwrap().inline_prepared); + std::fs::write( + stage.join(".freshell/templates/source-0"), + r#"{"instructions":["{file:/unapproved/private-file}"]}"#, + ) + .unwrap(); + assert!(prepare(Some(&context), &mut secrets, &frozen, &stage, &home).is_err(), "changed inline template must fail before an unapproved file can reach the native child"); + manifest.sources[0].lane = SourceLane::Main; + manifest.sources[0].template_sha256 = None; + manifest.native_source_flags = native_flags; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); let changed = ProviderPathInputs::new(BTreeMap::from([( "INSERT".into(), "{file:/different-unapproved/rules}".into(), diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 71f94dcf9..3893008ef 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -156,7 +156,7 @@ pub async fn prepare_terminal( .env .remove("OPENCODE_CONFIG_CONTENT") .or_else(|| child_env.get("OPENCODE_CONFIG_CONTENT").cloned()); - if inherited.is_some() || permission.is_some() || has_mcp { + if !plan.inline_prepared && (inherited.is_some() || permission.is_some() || has_mcp) { let config = freshell_platform::opencode_config::merge_owned_entries( inherited.as_deref(), &plan, @@ -309,11 +309,12 @@ pub fn prepare_fresh_agent_child_environment( .provider_launch_context .as_ref() .is_some_and(|context| context.mcp_capability.is_some()); - if child_env.contains_key("OPENCODE_CONFIG_CONTENT") || has_mcp { + if !plan.inline_prepared && (child_env.contains_key("OPENCODE_CONFIG_CONTENT") || has_mcp) { let normalized = freshell_platform::opencode_config::merge_owned_entries( child_env.get("OPENCODE_CONFIG_CONTENT").map(String::as_str), &plan, &freshell_platform::opencode_config::OwnedConfigEdits { + disable_snapshots: true, freshell_command: has_mcp .then(|| vec!["node".into(), "/opt/freshell-mcp/server.js".into()]), ..Default::default() diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 3c0dfec14..e570819e8 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -109,7 +109,7 @@ def create(name,*arguments): (plugin/"index.mjs").write_text("""import fs from 'node:fs';import crypto from 'node:crypto';import {tool} from '@opencode-ai/plugin';import {value} from './dependency.mjs'; const hash=(value)=>crypto.createHash('sha256').update(value).digest('hex'); export default {id:'closed-native-fixture',async server(input,options){return {async config(config){ - const receipt={prompt:hash(config.agent.probe.prompt),description:hash(config.agent.probe.description),references:['third','fourth','fifth','sixth'].map(name=>hash(config.agent[name].prompt)),resource:hash(fs.readFileSync(new URL('./resource.txt',import.meta.url))),dependency:hash(value),sdk:hash(typeof tool)}; + const receipt={prompt:hash(config.agent.probe.prompt),description:hash(config.agent.probe.description),references:['third','fourth','fifth','sixth'].map(name=>hash(config.agent[name].prompt)),resource:hash(fs.readFileSync(new URL('./resource.txt',import.meta.url))),dependency:hash(value),sdk:hash(typeof tool),snapshotDisabled:config.snapshot===false}; fs.writeFileSync(options.receipt,JSON.stringify(receipt)); },async 'chat.message'(){ fs.writeFileSync(options.receipt+'.stopped','FRESHELL_SYNTHETIC_STARTUP_STOP'); From 08335a6de4b864d335f865f0f07b0a59cf797452 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 18:55:10 -0700 Subject: [PATCH 057/493] fix: preserve shared OpenCode source origins through private aliases --- .../freshell-platform/src/opencode_config.rs | 2 +- .../freshell-platform/src/opencode_recipe.rs | 4 + .../src/managed_mcp_capability.rs | 104 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 83 ++++++++++---- .../src/managed_provider_launch_tests.rs | 89 +++++++++++++-- .../src/managed_provider_references.rs | 33 ++++-- .../managed_provider_references/sources.rs | 51 ++++++++- .../src/provider_input_preparation.rs | 76 +++++++++++-- 8 files changed, 388 insertions(+), 54 deletions(-) diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index b294c582a..d848f88fa 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -808,7 +808,7 @@ pub fn file_reference_path( lexical_path(&parent.join(path)) } -fn lexical_path(path: &Path) -> PathBuf { +pub fn lexical_path(path: &Path) -> PathBuf { use std::path::Component; let mut result = PathBuf::new(); for component in path.components() { diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index f42a3d1cc..a2054e85c 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -52,6 +52,8 @@ fn default_rebind_enabled() -> bool { pub struct NativeSourceFlag { pub kind: NativeSourceFlagKind, pub runtime_destination: String, + #[serde(default)] + pub source_ordinal: Option, } #[derive(Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] @@ -71,6 +73,8 @@ impl NativeSourceFlagKind { #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct SourceView { pub logical_path: PathBuf, + #[serde(default)] + pub native_destination: Option, pub lane: SourceLane, pub unexpanded_relative_path: String, pub template_relative_path: String, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 021941d95..39821bf55 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -3792,6 +3792,110 @@ except Exception as error: } } + #[tokio::test] + async fn managed_provider_shared_workspace_sources_keep_original_form_and_private_aliases() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let cwd = workspace.join("nested"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + let raw = r#"{"agent":{"probe":{"prompt":"{file:./rules}"}}}"#; + let shared = workspace.join("shared.json"); + std::fs::write(&shared, raw).unwrap(); + std::os::unix::fs::symlink("shared.json", workspace.join("opencode.json")).unwrap(); + std::os::unix::fs::symlink("../shared.json", cwd.join("opencode.json")).unwrap(); + std::fs::write(workspace.join("rules"), "PRIVATE_ROOT_ORIGIN").unwrap(); + std::fs::write(cwd.join("rules"), "PRIVATE_NESTED_ORIGIN").unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "fixture-existing-soul", + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &|_| None, + references: vec![], + auth_authority: Some("fixture-owned-attempt"), + selected_sources: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .expect("representable shared source origins cannot be rejected or flattened"); + let inputs = prepared.context.prepared_inputs.as_ref().unwrap(); + assert!( + inputs.config_binds.is_empty(), + "two origin-dependent rewrites cannot bind over one shared source inode" + ); + let aliases = &inputs.indexed_files.as_ref().unwrap().aliases; + assert_eq!(aliases.len(), 2); + assert_eq!( + aliases + .iter() + .map(|alias| alias.logical_destination.as_str()) + .collect::>(), + [ + workspace.join("rules").to_str().unwrap(), + cwd.join("rules").to_str().unwrap() + ] + ); + assert_ne!(aliases[0].source_id, aliases[1].source_id); + let stage = staged_provider_root_path( + &directory, + &prepared.context.mcp_capability.as_ref().unwrap().grant_id, + ) + .unwrap(); + let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert_eq!(manifest.sources.len(), 2); + for source in manifest.sources { + assert!(source.original_form); + assert_eq!( + std::fs::read_to_string(stage.join(source.unexpanded_relative_path)).unwrap(), + raw + ); + } + assert_eq!(std::fs::read_to_string(shared).unwrap(), raw); + assert!(std::fs::symlink_metadata(workspace.join("opencode.json")) + .unwrap() + .file_type() + .is_symlink()); + assert!(std::fs::symlink_metadata(cwd.join("opencode.json")) + .unwrap() + .file_type() + .is_symlink()); + } + #[tokio::test] async fn managed_provider_explicit_native_source_selection_survives_preparation() { use freshell_runtime_protocol::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index beab21523..b181fb0d6 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -98,6 +98,16 @@ pub(super) fn stage( } manifest.sources.push(SourceView { logical_path: source.source.logical_path.clone(), + native_destination: Some( + crate::managed_provider_references::sources::runtime_config_destination( + home, + input.config_root.unwrap_or(&home.join(".config/opencode")), + &source.source.logical_path, + ) + .to_str() + .ok_or("native source destination is not UTF-8")? + .into(), + ), lane: match source.source.lane { SourceLane::Main => freshell_platform::opencode_recipe::SourceLane::Main, SourceLane::Tui => freshell_platform::opencode_recipe::SourceLane::Tui, @@ -120,7 +130,7 @@ pub(super) fn stage( occurrences: source.references.occurrences.clone(), referenced_variables: source.references.referenced_variables.clone(), path_variables: source.references.path_variables.clone(), - original_form: false, + original_form: source.original_form, }); } for (expected, (source_id, canonical_source)) in files.into_iter().enumerate() { @@ -155,27 +165,16 @@ pub(super) fn describe( let mut sources = BTreeMap::new(); let mut binds = Vec::new(); let mut recipes = 0usize; - let mut canonical_occurrences = BTreeMap::<&Path, usize>::new(); - for occurrence in plan { - *canonical_occurrences - .entry(&occurrence.source.canonical_source) - .or_default() += 1; - } + let mut aliases = Vec::new(); + let mut alias_sources = BTreeMap::new(); for (ordinal, occurrence) in plan.iter().enumerate() { - if occurrence.source.lane != SourceLane::Inline - && canonical_occurrences[occurrence.source.canonical_source.as_path()] > 1 - { - return Err( - "shared source preparation requires its original-form aliases before publication" - .into(), - ); - } - if occurrence.source.lane != SourceLane::Inline { + if occurrence.source.lane != SourceLane::Inline && !occurrence.original_form { let logical = &occurrence.source.logical_path; - let destination = - crate::managed_provider_references::sources::runtime_config_destination( + let destination = freshell_platform::opencode_config::lexical_path( + &crate::managed_provider_references::sources::runtime_config_destination( home, global, logical, - ); + ), + ); let effective = if let Ok(relative) = logical.strip_prefix(&global) { let physical = std::fs::canonicalize(root.join(relative)) .map_err(|_| "selected global config is unavailable in its private stage")?; @@ -224,6 +223,48 @@ pub(super) fn describe( return Err("selected source identity changed under one index".into()); } } + if occurrence.original_form { + for approved in &occurrence.references.occurrences { + let source_id = approved + .runtime_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .and_then(|value| value.parse::().ok()) + .ok_or("original-form alias has no approved source index")?; + let selected = sources + .get(&source_id) + .ok_or("original-form alias index is unavailable")?; + let logical = + crate::managed_provider_references::sources::runtime_config_destination( + home, + global, + &approved.approved_alias, + ); + let effective = + crate::managed_provider_references::sources::runtime_config_destination( + home, global, selected, + ); + if let Some(prior) = alias_sources.insert(logical.clone(), source_id) { + if prior != source_id { + return Err("shared source has conflicting private alias selections; reprepare before dispatch".into()); + } + continue; + } + if aliases.len() >= 256 { + return Err("configuration alias count exceeds 256".into()); + } + aliases.push(IndexedProviderFileAlias { + source_id, + logical_destination: logical + .to_str() + .ok_or("alias destination is not UTF-8")? + .into(), + effective_destination: effective + .to_str() + .ok_or("alias destination is not UTF-8")? + .into(), + }); + } + } } let mut file = std::fs::File::open(root.join(".freshell/input-manifest.json")) .map_err(|_| "prepared source manifest unavailable")? @@ -254,14 +295,14 @@ pub(super) fn describe( sha256: format!("{:x}", hash.finalize()), source_count: u16::try_from(sources.len()) .map_err(|_| "selected source count overflow")?, - alias_count: 0, + alias_count: u16::try_from(aliases.len()).map_err(|_| "source alias count overflow")?, recipe_count: u16::try_from(recipes).map_err(|_| "recipe count overflow")?, }, config_binds: binds, resource_mounts: Vec::new(), indexed_files: (!sources.is_empty()).then(|| IndexedProviderFiles { volume_name: indexed_provider_volume_name(soul, &capability.grant_id), - aliases: Vec::new(), + aliases, }), }; inputs diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 7c22fb550..2f5faaba2 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -283,7 +283,49 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr .await .unwrap() .unwrap(); - let cwd = fixture.join("workspace").to_string_lossy().into_owned(); + let nested = fixture.join("workspace/nested"); + std::fs::create_dir_all(&nested).unwrap(); + assert!(std::process::Command::new("git") + .args(["init", "--quiet"]) + .arg(fixture.join("workspace")) + .status() + .unwrap() + .success()); + let shared_source = fixture.join("workspace/shared.json"); + let shared_receipt = fixture.join("workspace/shared-origin-consumed.json"); + let shared_bytes = serde_json::to_vec(&json!({"agent":{"probe":{"prompt":"{env:OPENAI_API_KEY}"}}, "plugin":[["./shared-probe.mjs", {"receipt":shared_receipt}]]})).unwrap(); + for (directory, label) in [ + (fixture.join("workspace"), "root"), + (nested.clone(), "nested"), + ] { + let plugin = format!( + r#"import fs from 'node:fs';import crypto from 'node:crypto'; +export default {{id:'closed-shared-{label}',async server(input,options){{return {{async config(config){{ + let receipt=fs.existsSync(options.receipt)?JSON.parse(fs.readFileSync(options.receipt,'utf8')):{{}}; + receipt.{label}=crypto.createHash('sha256').update(fs.readFileSync(new URL('./rules.txt',import.meta.url))).digest('hex'); + fs.writeFileSync(options.receipt,JSON.stringify(receipt)); +}}}}}}}}; +"# + ); + std::fs::write(directory.join("shared-probe.mjs"), plugin).unwrap(); + } + std::fs::write(&shared_source, &shared_bytes).unwrap(); + std::os::unix::fs::symlink("shared.json", fixture.join("workspace/opencode.json")).unwrap(); + std::os::unix::fs::symlink("../shared.json", nested.join("opencode.json")).unwrap(); + for (path, bytes) in [ + ( + fixture.join("workspace/rules.txt"), + b"native-shared-root-rule".as_slice(), + ), + ( + nested.join("rules.txt"), + b"native-shared-nested-rule".as_slice(), + ), + ] { + std::fs::write(&path, bytes).unwrap(); + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + let cwd = nested.to_string_lossy().into_owned(); let original_config = fixture.join("source-home/.config/opencode/opencode.json"); let original_bytes = std::fs::read(&original_config).unwrap(); let explicit_config = fixture.join("workspace/selected-config.json"); @@ -307,11 +349,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ let inline_source = r#"{ // inline instance origin "agent":{"probe":{"description":"{env:INLINE_VALUE}"}}}"#; let inline_value = "INLINE_PREFIX{file:./inline-rule}INLINE_SUFFIX"; - std::fs::write( - fixture.join("workspace/inline-rule"), - "native-inline-selected-rule\n", - ) - .unwrap(); + std::fs::write(nested.join("inline-rule"), "native-inline-selected-rule\n").unwrap(); let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { @@ -323,8 +361,8 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ rows: 30, }, env: BTreeMap::from([ - ("OPENCODE_CONFIG".into(), "selected-config.json".into()), - ("OPENCODE_TUI_CONFIG".into(), "selected-tui.jsonc".into()), + ("OPENCODE_CONFIG".into(), "../selected-config.json".into()), + ("OPENCODE_TUI_CONFIG".into(), "../selected-tui.jsonc".into()), ("OPENCODE_CONFIG_CONTENT".into(), inline_source.into()), ("INLINE_VALUE".into(), inline_value.into()), ]), @@ -359,7 +397,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .await .map_err(|_| "actual pinned provider startup did not consume selected config and closure") } - let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXnative-ruleAUTH_SUFFIX")), + let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXnative-shared-nested-ruleAUTH_SUFFIX")), "description":format!("{:x}",Sha256::digest(b"INLINE_PREFIXnative-inline-selected-ruleINLINE_SUFFIX")), "references":(["third","fourth","fifth","sixth"].map(|ordinal|format!("{:x}",Sha256::digest(format!("native-{ordinal}-rule").as_bytes())))), "resource":format!("{:x}",Sha256::digest(b"plugin-relative-resource")), @@ -405,6 +443,10 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ } }; assert_eq!(terminal_receipt, expected); + assert_eq!( + read_receipt(&shared_receipt).await.unwrap(), + json!({"root":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"nested":format!("{:x}",Sha256::digest(b"native-shared-nested-rule"))}) + ); assert_eq!( read_receipt(&tui_receipt) .await @@ -420,6 +462,17 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ json!({"provider":"opencode","route":"terminal-tui","digest":format!("{:x}",Sha256::digest(b"dark"))}) ); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + assert_eq!(std::fs::read(&shared_source).unwrap(), shared_bytes); + assert!( + std::fs::symlink_metadata(fixture.join("workspace/opencode.json")) + .unwrap() + .file_type() + .is_symlink() + ); + assert!(std::fs::symlink_metadata(nested.join("opencode.json")) + .unwrap() + .file_type() + .is_symlink()); assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); println!( "provider_launch_consumer {}", @@ -430,6 +483,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .await .unwrap(); std::fs::remove_file(&receipt).unwrap(); + std::fs::remove_file(&shared_receipt).unwrap(); // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. std::env::set_var("OPENCODE_CONFIG", &explicit_config); @@ -478,7 +532,22 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .dispatch(HostedFreshAgentCommand::Send(send)) .await; assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); + assert_eq!( + read_receipt(&shared_receipt).await.unwrap(), + json!({"root":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"nested":format!("{:x}",Sha256::digest(b"native-shared-nested-rule"))}) + ); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + assert_eq!(std::fs::read(&shared_source).unwrap(), shared_bytes); + assert!( + std::fs::symlink_metadata(fixture.join("workspace/opencode.json")) + .unwrap() + .file_type() + .is_symlink() + ); + assert!(std::fs::symlink_metadata(nested.join("opencode.json")) + .unwrap() + .file_type() + .is_symlink()); assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); let stopped = fixture.join("workspace/native-consumed.json.stopped"); tokio::time::timeout(Duration::from_secs(10), async { @@ -512,6 +581,8 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ b"native-rule".as_slice(), b"native-explicit-selected-rule".as_slice(), b"native-inline-selected-rule".as_slice(), + b"native-shared-root-rule".as_slice(), + b"native-shared-nested-rule".as_slice(), b"INLINE_PREFIX".as_slice(), b"AUTH_INLINE_SHADOW_NO_FILE".as_slice(), b"synthetic-onecli-shadowed".as_slice(), diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 01c04b401..88759526c 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -106,9 +106,16 @@ pub(super) fn resolve_file_references_from_view( env, recipe_ordinal, allocate, - &std::collections::BTreeMap::new(), + ReferenceProjection { + relocated_files: &std::collections::BTreeMap::new(), + original_form: false, + }, ) } +pub(super) struct ReferenceProjection<'a> { + pub relocated_files: &'a std::collections::BTreeMap, + pub original_form: bool, +} pub(super) fn resolve_file_references_from_view_relocated( raw: &str, source_view: Option>, @@ -116,7 +123,7 @@ pub(super) fn resolve_file_references_from_view_relocated( env: &dyn Fn(&str) -> Option, recipe_ordinal: usize, allocate: &mut dyn FnMut(&Path) -> Result, - relocated_files: &std::collections::BTreeMap, + projection: ReferenceProjection<'_>, ) -> Result { use freshell_platform::opencode_config::{ file_reference_path, transform_file_tokens, TokenTransformError, @@ -171,7 +178,7 @@ pub(super) fn resolve_file_references_from_view_relocated( origin.cwd, origin.original_home, ); - let physical_source = relocated_files.get(&path).unwrap_or(&path); + let physical_source = projection.relocated_files.get(&path).unwrap_or(&path); let canonical = std::fs::canonicalize(physical_source).map_err(|_| { fail( token.ordinal, @@ -263,7 +270,9 @@ pub(super) fn resolve_file_references_from_view_relocated( } let transformed_length = transformed.len(); drop(transformed); - let (template, recipes) = if generated { + let (template, recipes) = if projection.original_form { + (raw.to_owned(), Vec::new()) + } else if generated { let generated_name = freshell_platform::opencode_config::generated_config_reference_name(recipe_ordinal); if transformed_length > 64 * 1024 { @@ -304,7 +313,7 @@ pub(super) fn resolve_file_references_from_view_relocated( }; Ok(ResolvedFileReferencePlan { template, - unexpanded_direct_source: (!generated) + unexpanded_direct_source: (!generated || projection.original_form) .then(|| source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned()))), selections, referenced_variables: variables.into_inner().into_iter().collect(), @@ -873,15 +882,19 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( } else { cwd.join(path) }; - if !loaded + let source_ordinal = loaded .iter() - .any(|source| source.source.logical_path == path) - { - return Err(format!("selected {name} source is missing or unavailable; repair it before dispatch")); - } + .position(|source| source.source.logical_path == path && match kind { + freshell_platform::opencode_recipe::NativeSourceFlagKind::Main => source.source.lane == SourceLane::Main, + freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => source.source.lane == SourceLane::Tui, + }).ok_or_else(|| format!("selected {name} source is missing or unavailable; repair it before dispatch"))?; let destination = sources::runtime_config_destination(home, &config_root, &path); flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { kind, + source_ordinal: Some( + u16::try_from(source_ordinal) + .map_err(|_| "native source ordinal overflow")?, + ), runtime_destination: destination .to_str() .ok_or("selected native source destination is not UTF-8")? diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 3dd655a2b..5facc02bf 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -200,6 +200,7 @@ fn add_source( pub struct PlannedSourceOccurrence { pub source: SourceOccurrence, pub references: super::ResolvedFileReferencePlan, + pub original_form: bool, } pub struct LoadedSourceOccurrence { @@ -333,6 +334,12 @@ pub fn plan_loaded_opencode_sources_relocated( relocated_files: &std::collections::BTreeMap, ) -> Result, super::ReferenceError> { let original_names = source_environment_names(&sources)?; + let mut canonical_occurrences = std::collections::BTreeMap::new(); + for source in &sources { + *canonical_occurrences + .entry(source.source.canonical_source.clone()) + .or_insert(0usize) += 1; + } let mut result = Vec::new(); let mut selected = std::collections::BTreeMap::::new(); let mut selected_bytes = 0u64; @@ -343,6 +350,8 @@ pub fn plan_loaded_opencode_sources_relocated( let mut retained_view_bytes = 0usize; for (ordinal, loaded) in sources.into_iter().enumerate() { let source = loaded.source; + let original_form = source.lane != SourceLane::Inline + && canonical_occurrences[&source.canonical_source] > 1; let fail = |cause| super::ReferenceError { operation: "read", field: "OpenCode source", @@ -351,6 +360,19 @@ pub fn plan_loaded_opencode_sources_relocated( variables: vec![], cause, }; + for index in 0..freshell_platform::opencode_config::MAX_GENERATED_CONFIG_REFERENCES { + if lookup( + source.lane, + &source.logical_path, + &freshell_platform::opencode_config::generated_config_reference_name(index), + ) + .is_some() + { + return Err(fail( + "original or source-specific input occupies the reserved generated namespace", + )); + } + } let mut allocate = |canonical: &Path| { if let Some(destination) = selected.get(canonical) { return Ok(destination.clone()); @@ -386,7 +408,10 @@ pub fn plan_loaded_opencode_sources_relocated( &|name| lookup(source.lane, &source.logical_path, name), recipe_count, &mut allocate, - relocated_files, + super::ReferenceProjection { + relocated_files, + original_form, + }, )?; if references .recipes @@ -428,7 +453,11 @@ pub fn plan_loaded_opencode_sources_relocated( "combined source recipe, alias, variable or event limit exceeded", )); } - result.push(PlannedSourceOccurrence { source, references }); + result.push(PlannedSourceOccurrence { + source, + references, + original_form, + }); } Ok(result) } @@ -670,10 +699,20 @@ mod tests { plan[0].references.selections[0].runtime_destination, plan[1].references.selections[0].runtime_destination ); - assert_ne!( - plan[0].references.recipes[0].generated_name, - plan[1].references.recipes[0].generated_name - ); + for source in &plan { + assert!( + source.original_form, + "shared main/TUI source must retain native lane evaluation" + ); + assert!( + source.references.recipes.is_empty(), + "one shared source cannot receive two lane-specific document wrappers" + ); + assert_eq!( + source.references.unexpanded_source(), + "{\"instructions\":[\"{env:INSERT}\"]}" + ); + } let collision = plan_opencode_sources( sources(), owned.path(), diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 31e07f123..58fd093c0 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -114,13 +114,32 @@ pub fn prepare( } for flag in &manifest.native_source_flags { let name = flag.kind.variable(); - if !inputs - .config_binds - .iter() - .any(|bind| bind.logical_destination == flag.runtime_destination) - || generated.contains_key(name) - || secrets.environment.contains_key(name) - { + let has_bind = inputs.config_binds.iter().any(|bind| { + std::path::Path::new(&bind.logical_destination) + == freshell_platform::opencode_config::lexical_path(std::path::Path::new( + &flag.runtime_destination, + )) + }); + let approved = if let Some(ordinal) = flag.source_ordinal { + manifest + .sources + .get(usize::from(ordinal)) + .is_some_and(|source| { + source.native_destination.as_deref() == Some(flag.runtime_destination.as_str()) + && match flag.kind { + freshell_platform::opencode_recipe::NativeSourceFlagKind::Main => { + source.lane == SourceLane::Main + } + freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => { + source.lane == SourceLane::Tui + } + } + && (source.original_form || has_bind) + }) + } else { + has_bind + }; + if !approved || generated.contains_key(name) || secrets.environment.contains_key(name) { return Err("prepared native source selector has no approved config bind or collides with child input".into()); } budget @@ -243,6 +262,7 @@ mod tests { original_home: "/original-unavailable/home".into(), sources: vec![SourceView { logical_path: "/original-unavailable/project/opencode.json".into(), + native_destination: Some("/original-unavailable/project/opencode.json".into()), lane: SourceLane::Main, unexpanded_relative_path: ".freshell/source-views/source-0".into(), template_relative_path: ".freshell/templates/source-0".into(), @@ -263,6 +283,7 @@ mod tests { rebind_enabled: true, native_source_flags: vec![NativeSourceFlag { kind: NativeSourceFlagKind::Main, + source_ordinal: Some(0), runtime_destination: "/original-unavailable/project/opencode.json".into(), }], files: vec![SelectedSourceFile { @@ -331,6 +352,47 @@ mod tests { Some("/original-unavailable/project/opencode.json"), "an explicitly selected native source must be delivered to the actual child environment"); assert!(!generated.contains_key("INSERT"),"independent generated recipes do not forward an original ambient variable into the native child"); + // Original-form shared configurations have approved native source + // identity but deliberately no replacement config bind. + manifest.sources[0].original_form = true; + manifest.sources[0].generated_name = None; + let binds = std::mem::take(&mut context.prepared_inputs.as_mut().unwrap().config_binds); + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let original = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .expect("native selection of an approved original-form source cannot require a replacement bind"); + assert_eq!( + original.get("OPENCODE_CONFIG").map(String::as_str), + Some("/original-unavailable/project/opencode.json") + ); + let approved_flag = manifest.native_source_flags[0].clone(); + for invalid in [0, 1, 2] { + manifest.native_source_flags[0] = approved_flag.clone(); + match invalid { + 0 => { + manifest.native_source_flags[0].runtime_destination = + "/unapproved/source.json".into() + } + 1 => manifest.native_source_flags[0].source_ordinal = Some(42), + _ => manifest.native_source_flags[0].kind = NativeSourceFlagKind::Tui, + } + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap_err(); + assert!(!error.contains("/unapproved/source.json")); + } + manifest.native_source_flags[0] = approved_flag; + manifest.sources[0].original_form = false; + manifest.sources[0].generated_name = Some("FRESHELL_CONFIG_REFERENCE_0".into()); + context.prepared_inputs.as_mut().unwrap().config_binds = binds; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); std::fs::create_dir_all(stage.join(".freshell/templates")).unwrap(); let inline_template = "{env:FRESHELL_CONFIG_REFERENCE_0}"; std::fs::write(stage.join(".freshell/templates/source-0"), inline_template).unwrap(); From 350c436f771781d428c094a53676276738ac3f4a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 19:13:36 -0700 Subject: [PATCH 058/493] fix: project selected OpenCode roots into writable soul subpaths --- .../src/managed_mcp_capability.rs | 93 +++++++++++ .../managed_mcp_capability/source_stage.rs | 73 ++++++++- .../src/managed_provider_launch_tests.rs | 48 ++++++ crates/freshell-session-host/src/main.rs | 148 +++++++++++++++++- 4 files changed, 357 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 39821bf55..39966fa8b 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1120,6 +1120,23 @@ fn stage_provider_root( .iter() .map(|reference| PathBuf::from(&reference.source_path)), ); + if let Some(plan) = opencode_input.source_plan { + let units = source_stage::resource_units(home, &source, plan)?; + selections.extend(units.iter().map(|(source, mount)| CopySelection { + source: source.clone(), + destination_relative: mount.stage_relative_path.clone().into(), + })); + let descriptors = units + .into_iter() + .map(|(_, mount)| mount) + .collect::>(); + let metadata = freshell_runtime_protocol::bounded_json_bytes( + &descriptors, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "prepared resource descriptor exceeds control profile")?; + write_private_bytes(&temporary.join(".freshell/resource-mounts.json"), &metadata)?; + } let amplifier_resources = if provider == "amplifier" { Some(amplifier_resources::prepare( home, @@ -3792,6 +3809,82 @@ except Exception as error: } } + #[test] + fn managed_provider_selected_config_root_stages_complete_writable_dependencies() { + use crate::managed_provider_references::sources::*; + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let selected = workspace.join(".opencode"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(selected.join("plugins/probe/node_modules/dependency")).unwrap(); + let raw = r#"{"plugin":["./plugins/probe/index.js"]}"#; + std::fs::write(selected.join("opencode.json"), raw).unwrap(); + std::fs::write( + selected.join("plugins/probe/index.js"), + "export default {};", + ) + .unwrap(); + std::fs::write( + selected.join("plugins/probe/node_modules/dependency/index.js"), + "dependency-nonce", + ) + .unwrap(); + let sources = discover_opencode(OpencodeDiscovery { + home: &home, + cwd: &workspace, + worktree: &workspace, + config_root: &home.join(".config/opencode"), + env: &|_| None, + }) + .unwrap(); + let plan = plan_opencode_sources(sources, &workspace, &home, &|_, _, _| None).unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let capability = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + secret_references: Some(&[]), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let root = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == selected.to_str().unwrap()) + .expect("selected config directories require complete writable soul-backed roots"); + assert_eq!(root.kind, ProviderResourceKind::Directory); + assert_eq!( + std::fs::read_to_string( + stage + .join(&root.stage_relative_path) + .join("plugins/probe/node_modules/dependency/index.js") + ) + .unwrap(), + "dependency-nonce" + ); + assert_eq!( + std::fs::read_to_string(selected.join("opencode.json")).unwrap(), + raw + ); + } + #[tokio::test] async fn managed_provider_shared_workspace_sources_keep_original_form_and_private_aliases() { use freshell_runtime_protocol::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index b181fb0d6..1010f3478 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -4,6 +4,66 @@ use crate::managed_provider_references::sources::{PlannedSourceOccurrence, Sourc use freshell_platform::opencode_recipe::{SelectedSourceFile, SourceManifest, SourceView}; use std::{collections::BTreeMap, path::Path}; +/// Selected discovery roots are complete writable units, independent of the +/// private rewritten configuration leaves mounted inside them. +pub(super) fn resource_units( + home: &Path, + global: &Path, + plan: &[PlannedSourceOccurrence], +) -> Result< + Vec<( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )>, + String, +> { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + let mut roots = BTreeMap::new(); + for source in plan { + if source.source.lane == SourceLane::Inline { + continue; + } + let Some(parent) = source.source.logical_path.parent() else { + continue; + }; + if parent == global || parent.file_name() != Some(std::ffi::OsStr::new(".opencode")) { + continue; + } + let canonical = std::fs::canonicalize(parent) + .map_err(|_| "selected OpenCode resource root is unavailable, dangling or cyclic")?; + if !canonical.is_dir() { + return Err("selected OpenCode resource root must be a directory".into()); + } + roots.insert(parent.to_path_buf(), canonical); + } + let mut units = Vec::new(); + for (ordinal, (logical, canonical)) in roots.into_iter().enumerate() { + let destination = crate::managed_provider_references::sources::runtime_config_destination( + home, global, &logical, + ); + let effective = crate::managed_provider_references::sources::runtime_config_destination( + home, global, &canonical, + ); + units.push(( + canonical, + ProviderResourceMount { + stage_relative_path: format!(".freshell/resource-units/root-{ordinal}"), + provider_relative_path: format!(".freshell/provider-roots/root-{ordinal}"), + logical_destination: destination + .to_str() + .ok_or("resource destination is not UTF-8")? + .into(), + effective_destination: effective + .to_str() + .ok_or("resource destination is not UTF-8")? + .into(), + kind: ProviderResourceKind::Directory, + }, + )); + } + Ok(units) +} + pub(super) fn read_source_view(root: &Path, relative: &str) -> Result { use std::io::Read; if Path::new(relative) @@ -299,7 +359,18 @@ pub(super) fn describe( recipe_count: u16::try_from(recipes).map_err(|_| "recipe count overflow")?, }, config_binds: binds, - resource_mounts: Vec::new(), + resource_mounts: { + let mut bytes = Vec::new(); + std::fs::File::open(root.join(".freshell/resource-mounts.json")) + .map_err(|_| "prepared resource manifest unavailable")? + .take(MAX_CONTROL_FRAME_BYTES as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "prepared resource manifest read failed")?; + if bytes.len() > MAX_CONTROL_FRAME_BYTES { + return Err("prepared resource descriptor exceeds control profile".into()); + } + serde_json::from_slice(&bytes).map_err(|_| "prepared resource descriptors invalid")? + }, indexed_files: (!sources.is_empty()).then(|| IndexedProviderFiles { volume_name: indexed_provider_volume_name(soul, &capability.grant_id), aliases, diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 2f5faaba2..44dad50fb 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -350,6 +350,37 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ "agent":{"probe":{"description":"{env:INLINE_VALUE}"}}}"#; let inline_value = "INLINE_PREFIX{file:./inline-rule}INLINE_SUFFIX"; std::fs::write(nested.join("inline-rule"), "native-inline-selected-rule\n").unwrap(); + let resource_root = fixture.join("workspace/.opencode"); + let root_receipt = fixture.join("workspace/native-root-consumed.json"); + std::fs::create_dir_all(resource_root.join("plugins/selected/node_modules/fixture-dependency")) + .unwrap(); + std::fs::write( + resource_root.join("plugins/selected/node_modules/fixture-dependency/package.json"), + r#"{"name":"fixture-dependency","type":"module","exports":"./index.mjs"}"#, + ) + .unwrap(); + std::fs::write( + resource_root.join("plugins/selected/node_modules/fixture-dependency/index.mjs"), + "export default 'selected-root-dependency';", + ) + .unwrap(); + std::fs::write( + resource_root.join("plugins/selected/resource"), + "selected-root-resource", + ) + .unwrap(); + std::fs::write(resource_root.join("plugins/selected/index.mjs"), r#"import fs from 'node:fs';import crypto from 'node:crypto';import dependency from 'fixture-dependency'; +export default {id:'closed-writable-selected-root',async server(input,options){return {async config(config){ + const root=new URL('../../',import.meta.url); const path=new URL('provider-owned-write',root); + fs.writeFileSync(path,'selected-root-write'); + fs.writeFileSync(options.receipt,JSON.stringify({dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); +}}}}; +"#).unwrap(); + let root_config = serde_json::to_vec( + &json!({"plugin":[["./plugins/selected/index.mjs", {"receipt":root_receipt}]]}), + ) + .unwrap(); + std::fs::write(resource_root.join("opencode.json"), &root_config).unwrap(); let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { @@ -443,6 +474,16 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ } }; assert_eq!(terminal_receipt, expected); + let expected_root = json!({"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); + assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); + assert!( + !resource_root.join("provider-owned-write").exists(), + "native writes must land in the selected soul subpath, preserving originals" + ); + assert_eq!( + std::fs::read(resource_root.join("opencode.json")).unwrap(), + root_config + ); assert_eq!( read_receipt(&shared_receipt).await.unwrap(), json!({"root":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"nested":format!("{:x}",Sha256::digest(b"native-shared-nested-rule"))}) @@ -484,6 +525,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .unwrap(); std::fs::remove_file(&receipt).unwrap(); std::fs::remove_file(&shared_receipt).unwrap(); + std::fs::remove_file(&root_receipt).unwrap(); // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. std::env::set_var("OPENCODE_CONFIG", &explicit_config); @@ -532,6 +574,12 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ .dispatch(HostedFreshAgentCommand::Send(send)) .await; assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); + assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); + assert!(!resource_root.join("provider-owned-write").exists()); + assert_eq!( + std::fs::read(resource_root.join("opencode.json")).unwrap(), + root_config + ); assert_eq!( read_receipt(&shared_receipt).await.unwrap(), json!({"root":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"nested":format!("{:x}",Sha256::digest(b"native-shared-nested-rule"))}) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index bac71816e..ce8234b43 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1687,7 +1687,7 @@ fn copy_provider_config_references_with_ownership( use freshell_platform::provider_environment::CopySelection; use freshell_runtime_protocol::ProviderConfigRoot; context.validate(provider).map_err(|error| error.message)?; - let selections = provider_config_references(context) + let mut selections = provider_config_references(context) .into_iter() .map(|reference| { let root = match reference.root { @@ -1701,6 +1701,16 @@ fn copy_provider_config_references_with_ownership( } }) .collect::>(); + selections.extend( + context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .map(|unit| CopySelection { + source: user_provider_root.join(&unit.stage_relative_path), + destination_relative: unit.provider_relative_path.clone().into(), + }), + ); let protected = private_bound_destinations(context); let mut excluded = provider_credential_sources(user_provider_root); for selection in &selections { @@ -1757,7 +1767,7 @@ fn private_bound_destinations( context: &freshell_runtime_protocol::ProviderLaunchContext, ) -> std::collections::BTreeSet { let home = Path::new("/home/freshell/provider"); - context + let mut protected = context .prepared_inputs .iter() .flat_map(|inputs| { @@ -1779,7 +1789,34 @@ fn private_bound_destinations( .and_then(Path::to_str) .map(str::to_owned) }) - .collect() + .collect::>(); + if let Some(inputs) = &context.prepared_inputs { + for destination in inputs + .config_binds + .iter() + .map(|bind| &bind.effective_destination) + .chain(inputs.indexed_files.iter().flat_map(|files| { + files + .aliases + .iter() + .map(|alias| &alias.effective_destination) + })) + { + for unit in &inputs.resource_mounts { + if let Ok(relative) = + Path::new(destination).strip_prefix(&unit.effective_destination) + { + protected.insert( + Path::new(&unit.provider_relative_path) + .join(relative) + .to_string_lossy() + .into_owned(), + ); + } + } + } + } + protected } const PROVIDER_PROJECTION_MANIFEST: &str = ".freshell-config-projection.json"; @@ -1881,10 +1918,17 @@ fn refresh_provider_config_references( unsafe { libc::geteuid() }, unsafe { libc::getegid() }, )?; - let destinations = provider_config_references(context) + let mut destinations = provider_config_references(context) .iter() .map(|reference| reference.provider_relative_path.clone()) .collect::>(); + destinations.extend( + context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .map(|unit| unit.provider_relative_path.clone()), + ); std::fs::write( stage.path().join("manifest.json"), serde_json::to_vec(&destinations).map_err(|error| error.to_string())?, @@ -3264,6 +3308,102 @@ mod tests { assert_eq!(output.stdout, b"resource-nonce"); } + #[test] + fn provider_config_selected_resource_root_populates_and_refreshes_stable_soul_directory() { + use freshell_runtime_protocol::*; + use std::os::unix::fs::MetadataExt; + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("stage"); + let home = owned.path().join("home"); + let unit = source.join(".freshell/resource-units/root-0"); + let destination = home.join(".freshell/provider-roots/root-0"); + std::fs::create_dir_all(unit.join("node_modules/dependency")).unwrap(); + std::fs::create_dir_all(&destination).unwrap(); + std::fs::write( + unit.join("node_modules/dependency/index.js"), + "dependency-initial", + ) + .unwrap(); + std::fs::write(destination.join("native-owned-sibling"), "preserved").unwrap(); + let inode = std::fs::metadata(&destination).unwrap().ino(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".freshell/provider-roots/root-0".into(), + logical_destination: "/synthetic/outside/.opencode".into(), + effective_destination: "/synthetic/outside/.opencode".into(), + kind: ProviderResourceKind::Directory, + }], + indexed_files: None, + }), + mcp_capability: Some(capability), + config: vec![], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + }; + let uid = unsafe { libc::geteuid() }; + let gid = unsafe { libc::getegid() }; + refresh_provider_config_references( + &context, + "opencode", + owned.path(), + &source, + &home, + uid, + gid, + ) + .unwrap(); + assert_eq!( + std::fs::read_to_string(destination.join("node_modules/dependency/index.js")).unwrap(), + "dependency-initial" + ); + std::fs::write( + unit.join("node_modules/dependency/index.js"), + "dependency-refreshed", + ) + .unwrap(); + refresh_provider_config_references( + &context, + "opencode", + owned.path(), + &source, + &home, + uid, + gid, + ) + .unwrap(); + assert_eq!( + std::fs::read_to_string(destination.join("node_modules/dependency/index.js")).unwrap(), + "dependency-refreshed" + ); + assert_eq!(std::fs::metadata(&destination).unwrap().ino(), inode); + assert_eq!( + std::fs::read_to_string(destination.join("native-owned-sibling")).unwrap(), + "preserved" + ); + } + #[test] fn provider_config_directory_copy_and_refresh_leave_private_bound_leaf_untouched() { use freshell_runtime_protocol::*; From 70425196ca573ec55dd6ca6461fecddccf2fc710 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 19:26:55 -0700 Subject: [PATCH 059/493] fix: retain and activate selected OpenCode resource directories --- .../freshell-platform/src/opencode_recipe.rs | 4 + .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 158 +++++++++++++++++- .../managed_mcp_capability/source_stage.rs | 19 ++- .../src/managed_provider_launch_tests.rs | 60 +++++++ .../src/managed_provider_references.rs | 40 +++++ .../managed_provider_references/sources.rs | 44 +++++ crates/freshell-server/src/managed_runtime.rs | 1 + .../src/provider_input_preparation.rs | 16 +- .../src/backend/provider_mountpoints.rs | 59 +++++++ 10 files changed, 395 insertions(+), 7 deletions(-) diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index a2054e85c..0269ac999 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -38,6 +38,8 @@ pub struct SourceManifest { pub user_owns_freshell_mcp: bool, #[serde(default)] pub native_source_flags: Vec, + #[serde(default)] + pub resource_roots: Vec, #[serde(default = "default_rebind_enabled")] pub rebind_enabled: bool, #[serde(default)] @@ -60,12 +62,14 @@ pub struct NativeSourceFlag { pub enum NativeSourceFlagKind { Main, Tui, + Directory, } impl NativeSourceFlagKind { pub fn variable(self) -> &'static str { match self { Self::Main => "OPENCODE_CONFIG", Self::Tui => "OPENCODE_TUI_CONFIG", + Self::Directory => "OPENCODE_CONFIG_DIR", } } } diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index a1e7852bb..0f512dc19 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -468,6 +468,7 @@ impl HostedFreshAgentProxy { references: provider_secret_references, auth_authority: None, selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 39966fa8b..385574ecc 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -39,6 +39,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub workspace: Option<&'a Path>, pub source_plan: Option<&'a [crate::managed_provider_references::sources::PlannedSourceOccurrence]>, + pub resource_roots: Option<&'a [PathBuf]>, pub initial_auth_reservation: Option<&'a str>, pub secret_references: Option<&'a [freshell_runtime_protocol::ProviderSecretReference]>, pub config_root: Option<&'a Path>, @@ -435,6 +436,9 @@ async fn apply_prepared_callback( .ok_or("replacement provider preparation requires attempt authority")?, ), selected_sources, + selected_resource_roots: accepted_manifest + .as_ref() + .map(|manifest| manifest.resource_roots.clone()), native_source_flags: accepted_manifest .as_ref() .map(|manifest| manifest.native_source_flags.clone()), @@ -1121,7 +1125,8 @@ fn stage_provider_root( .map(|reference| PathBuf::from(&reference.source_path)), ); if let Some(plan) = opencode_input.source_plan { - let units = source_stage::resource_units(home, &source, plan)?; + let units = + source_stage::resource_units(home, &source, plan, opencode_input.resource_roots)?; selections.extend(units.iter().map(|(source, mount)| CopySelection { source: source.clone(), destination_relative: mount.stage_relative_path.clone().into(), @@ -3583,6 +3588,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, @@ -3714,6 +3720,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, @@ -3885,6 +3892,152 @@ except Exception as error: ); } + #[tokio::test] + async fn managed_provider_selected_directory_without_config_retains_discovery_resources() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let selected = owned.path().join("selected-config-directory"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(selected.join("skills/probe")).unwrap(); + std::fs::write( + selected.join("skills/probe/SKILL.md"), + "skill-resource-nonce", + ) + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "existing-fixture", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| { + (name == "OPENCODE_CONFIG_DIR").then(|| selected.to_str().unwrap().into()) + }, + references: vec![], + auth_authority: Some("existing-owned-attempt"), + selected_sources: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .unwrap(); + let inputs = prepared.context.prepared_inputs.as_ref().unwrap(); + let unit = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == selected.to_str().unwrap()) + .expect( + "explicit config directory resources remain selected even with no JSON config leaf", + ); + let stage = staged_provider_root_path(&directory, &inputs.generation_id).unwrap(); + assert_eq!( + std::fs::read_to_string( + stage + .join(&unit.stage_relative_path) + .join("skills/probe/SKILL.md") + ) + .unwrap(), + "skill-resource-nonce" + ); + let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert!( + manifest + .native_source_flags + .iter() + .any(|flag| flag.kind.variable() == "OPENCODE_CONFIG_DIR" + && flag.runtime_destination == selected.to_str().unwrap()), + "the native directory selection must activate the exact selected root" + ); + std::fs::write( + selected.join("skills/probe/SKILL.md"), + "skill-resource-refreshed", + ) + .unwrap(); + let callback = serde_json::to_vec(&serde_json::json!({ + "controlSecret":"fixture-control", "action":"issue", "provider":"opencode", "soulId":soul, + "priorContext":prepared.context, "providerSecretReferences":[], "workspace":workspace, + "cwd":workspace, "authLookupAuthority":"existing-owned-attempt", + })).unwrap(); + let reissued: PreparedProviderContext = serde_json::from_slice( + &apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(), + ) + .unwrap(); + let next = reissued + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap(); + let next_unit = next + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == selected.to_str().unwrap()) + .unwrap(); + let next_stage = staged_provider_root_path(&directory, &next.generation_id).unwrap(); + assert_eq!( + std::fs::read_to_string( + next_stage + .join(&next_unit.stage_relative_path) + .join("skills/probe/SKILL.md") + ) + .unwrap(), + "skill-resource-refreshed" + ); + let next_manifest: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(next_stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert!(next_manifest.native_source_flags.iter().any(|flag| flag.kind.variable() == "OPENCODE_CONFIG_DIR" && flag.runtime_destination == selected.to_str().unwrap()), "reissue must retain accepted directory selection when current controller env has no selector"); + assert!( + stage.exists(), + "pending replacement cannot retire the previous usable roots" + ); + } + #[tokio::test] async fn managed_provider_shared_workspace_sources_keep_original_form_and_private_aliases() { use freshell_runtime_protocol::*; @@ -3921,6 +4074,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, @@ -4022,6 +4176,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, @@ -4113,6 +4268,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 1010f3478..498235353 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -10,6 +10,7 @@ pub(super) fn resource_units( home: &Path, global: &Path, plan: &[PlannedSourceOccurrence], + accepted_roots: Option<&[std::path::PathBuf]>, ) -> Result< Vec<( std::path::PathBuf, @@ -19,6 +20,14 @@ pub(super) fn resource_units( > { use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; let mut roots = BTreeMap::new(); + for root in accepted_roots.unwrap_or_default() { + let canonical = std::fs::canonicalize(root) + .map_err(|_| "selected OpenCode resource root is unavailable, dangling or cyclic")?; + if !canonical.is_dir() { + return Err("selected OpenCode resource root must be a directory".into()); + } + roots.insert(root.clone(), canonical); + } for source in plan { if source.source.lane == SourceLane::Inline { continue; @@ -38,10 +47,11 @@ pub(super) fn resource_units( } let mut units = Vec::new(); for (ordinal, (logical, canonical)) in roots.into_iter().enumerate() { - let destination = crate::managed_provider_references::sources::runtime_config_destination( - home, global, &logical, - ); - let effective = crate::managed_provider_references::sources::runtime_config_destination( + let destination = + crate::managed_provider_references::sources::runtime_directory_destination( + home, global, &logical, + ); + let effective = crate::managed_provider_references::sources::runtime_directory_destination( home, global, &canonical, ); units.push(( @@ -107,6 +117,7 @@ pub(super) fn stage( frozen_inputs: input.frozen_inputs.unwrap_or(&[]).to_vec(), user_owns_freshell_mcp: false, native_source_flags: input.native_source_flags.unwrap_or(&[]).to_vec(), + resource_roots: input.resource_roots.unwrap_or(&[]).to_vec(), rebind_enabled: input.rebind_enabled.unwrap_or(true), inline_edits: input.inline_edits.cloned().unwrap_or_default(), }; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 44dad50fb..13ae61fe1 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -381,6 +381,40 @@ export default {id:'closed-writable-selected-root',async server(input,options){r ) .unwrap(); std::fs::write(resource_root.join("opencode.json"), &root_config).unwrap(); + let explicit_directory = fixture.join("source-home/selected-directory"); + let directory_receipt = fixture.join("workspace/native-directory-consumed.json"); + std::fs::create_dir_all(explicit_directory.join("plugins")).unwrap(); + std::fs::create_dir_all(explicit_directory.join("skills/probe")).unwrap(); + std::fs::copy( + fixture.join("source-home/.config/opencode/package.json"), + explicit_directory.join("package.json"), + ) + .unwrap(); + std::os::unix::fs::symlink( + fixture.join("source-home/.config/opencode/node_modules"), + explicit_directory.join("node_modules"), + ) + .unwrap(); + std::fs::write( + explicit_directory.join("skills/probe/SKILL.md"), + "selected-directory-skill", + ) + .unwrap(); + let directory_plugin = format!( + r#"import fs from 'node:fs';import crypto from 'node:crypto'; +export default {{id:'closed-directory-discovery',async server(){{return {{async config(){{ + const root=new URL('../',import.meta.url);const write=new URL('provider-owned-auto-write',root); + fs.writeFileSync(write,'selected-directory-write'); + fs.writeFileSync({},JSON.stringify({{skill:crypto.createHash('sha256').update(fs.readFileSync(new URL('skills/probe/SKILL.md',root))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(write)).digest('hex')}})); +}}}}}}}}; +"#, + serde_json::to_string(&directory_receipt).unwrap() + ); + std::fs::write( + explicit_directory.join("plugins/auto-probe.js"), + directory_plugin, + ) + .unwrap(); let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { @@ -392,6 +426,10 @@ export default {id:'closed-writable-selected-root',async server(input,options){r rows: 30, }, env: BTreeMap::from([ + ( + "OPENCODE_CONFIG_DIR".into(), + "../../source-home/selected-directory".into(), + ), ("OPENCODE_CONFIG".into(), "../selected-config.json".into()), ("OPENCODE_TUI_CONFIG".into(), "../selected-tui.jsonc".into()), ("OPENCODE_CONFIG_CONTENT".into(), inline_source.into()), @@ -461,6 +499,11 @@ export default {id:'closed-writable-selected-root',async server(input,options){r "plugin", "opencode.json", "bun install", + "install", + "Resolving", + "package", + "lockfile", + "Permission denied", "ModuleNotFound", ] .into_iter() @@ -476,6 +519,14 @@ export default {id:'closed-writable-selected-root',async server(input,options){r assert_eq!(terminal_receipt, expected); let expected_root = json!({"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); + let expected_directory = json!({"skill":format!("{:x}",Sha256::digest(b"selected-directory-skill")),"write":format!("{:x}",Sha256::digest(b"selected-directory-write"))}); + assert_eq!( + read_receipt(&directory_receipt).await.unwrap(), + expected_directory + ); + assert!(!explicit_directory + .join("provider-owned-auto-write") + .exists()); assert!( !resource_root.join("provider-owned-write").exists(), "native writes must land in the selected soul subpath, preserving originals" @@ -526,9 +577,11 @@ export default {id:'closed-writable-selected-root',async server(input,options){r std::fs::remove_file(&receipt).unwrap(); std::fs::remove_file(&shared_receipt).unwrap(); std::fs::remove_file(&root_receipt).unwrap(); + std::fs::remove_file(&directory_receipt).unwrap(); // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. std::env::set_var("OPENCODE_CONFIG", &explicit_config); + std::env::set_var("OPENCODE_CONFIG_DIR", &explicit_directory); std::env::set_var("OPENCODE_TUI_CONFIG", &selected_tui); std::env::set_var("OPENCODE_CONFIG_CONTENT", inline_source); std::env::set_var("INLINE_VALUE", inline_value); @@ -575,6 +628,13 @@ export default {id:'closed-writable-selected-root',async server(input,options){r .await; assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); + assert_eq!( + read_receipt(&directory_receipt).await.unwrap(), + expected_directory + ); + assert!(!explicit_directory + .join("provider-owned-auto-write") + .exists()); assert!(!resource_root.join("provider-owned-write").exists()); assert_eq!( std::fs::read(resource_root.join("opencode.json")).unwrap(), diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 88759526c..4a61f30fc 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -723,6 +723,7 @@ pub(crate) struct OpencodePreparation<'a> { pub references: Vec, pub auth_authority: Option<&'a str>, pub selected_sources: Option>, + pub selected_resource_roots: Option>, pub native_source_flags: Option>, pub rebind_enabled: Option, pub inherited_inline: Option, @@ -759,6 +760,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( mut references, auth_authority, selected_sources, + selected_resource_roots, native_source_flags, rebind_enabled, inherited_inline, @@ -790,6 +792,18 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( cwd.join(xdg) }; let config_root = xdg.join("opencode"); + let resource_roots = if let Some(roots) = selected_resource_roots { + roots + } else { + sources::select_opencode_resource_roots(&OpencodeDiscovery { + home, + cwd, + worktree: workspace, + config_root: &config_root, + env: original, + }) + .map_err(str::to_string)? + }; let base = crate::managed_provider_bootstrap::provider_launch_context_from_home_at_with_xdg( "opencode", workspace, @@ -887,6 +901,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( .position(|source| source.source.logical_path == path && match kind { freshell_platform::opencode_recipe::NativeSourceFlagKind::Main => source.source.lane == SourceLane::Main, freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => source.source.lane == SourceLane::Tui, + freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory => false, }).ok_or_else(|| format!("selected {name} source is missing or unavailable; repair it before dispatch"))?; let destination = sources::runtime_config_destination(home, &config_root, &path); flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { @@ -902,6 +917,30 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( }); } } + if let Some(path) = original("OPENCODE_CONFIG_DIR").filter(|path| !path.is_empty()) { + let path = if Path::new(&path).is_absolute() { + PathBuf::from(path) + } else { + cwd.join(path) + }; + if !resource_roots.contains(&path) { + return Err( + "selected OPENCODE_CONFIG_DIR root is unavailable before dispatch".into(), + ); + } + flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { + kind: freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory, + source_ordinal: None, + runtime_destination: sources::runtime_directory_destination( + home, + &config_root, + &path, + ) + .to_str() + .ok_or("selected directory is not UTF-8")? + .into(), + }); + } flags }; let mut retained_views = loaded @@ -1023,6 +1062,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( workspace: Some(workspace), config_root: Some(&config_root), source_plan: Some(&plan), + resource_roots: Some(&resource_roots), initial_auth_reservation: reservation.as_deref(), secret_references: Some(&references), frozen_inputs: Some(&frozen_names), diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 5facc02bf..5bb268c70 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -108,6 +108,14 @@ pub fn runtime_config_destination(home: &Path, global: &Path, logical: &Path) -> } } +pub fn runtime_directory_destination(home: &Path, global: &Path, logical: &Path) -> PathBuf { + runtime_config_destination( + home, + global, + &freshell_platform::opencode_config::lexical_path(logical), + ) +} + fn absolute_from_cwd(path: &str, cwd: &Path) -> PathBuf { let path = Path::new(path); if path.is_absolute() { @@ -169,6 +177,42 @@ pub fn config_directories( Ok(result) } +/// Native discovery includes directories with plugins or skills even when no +/// configuration document exists. Keep these selections independent of files. +pub fn select_opencode_resource_roots( + request: &OpencodeDiscovery<'_>, +) -> Result, &'static str> { + let disabled = (request.env)("OPENCODE_DISABLE_PROJECT_CONFIG") + .is_some_and(|value| matches!(value.as_str(), "1" | "true")); + let mut roots = config_directories(request, None, disabled)?; + for directory in std::iter::once(request.home.to_path_buf()) + .chain(ancestors(request.cwd, Some(request.worktree))) + { + for name in [".claude/skills", ".agents/skills"] { + let path = directory.join(name); + match std::fs::symlink_metadata(&path) { + Ok(_) => { + if !roots.contains(&path) { + roots.push(path); + } + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("selected OpenCode skill root metadata failed"), + } + } + } + roots.retain(|root| root != request.config_root); + for root in &roots { + let canonical = std::fs::canonicalize(root).map_err(|_| { + "selected OpenCode resource directory is missing, inaccessible or cyclic" + })?; + if !canonical.is_dir() || canonical == request.home || canonical == Path::new("/") { + return Err("selected OpenCode resource requires a bounded directory scope"); + } + } + Ok(roots) +} + fn add_source( sources: &mut Vec, logical_path: PathBuf, diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 2a781af7e..4c3e0eecd 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -211,6 +211,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { references: provider_secret_references, auth_authority: None, selected_sources: None, + selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, inherited_inline: None, diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 58fd093c0..997e25253 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -109,7 +109,7 @@ pub fn prepare( }; let mut generated = BTreeMap::new(); let mut budget = frozen.clone(); - if manifest.native_source_flags.len() > 2 { + if manifest.native_source_flags.len() > 3 { return Err("prepared native source selector profile exceeded".into()); } for flag in &manifest.native_source_flags { @@ -120,7 +120,15 @@ pub fn prepare( &flag.runtime_destination, )) }); - let approved = if let Some(ordinal) = flag.source_ordinal { + let approved = if flag.kind + == freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory + { + flag.source_ordinal.is_none() + && inputs.resource_mounts.iter().any(|mount| { + mount.logical_destination == flag.runtime_destination + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::Directory + }) + } else if let Some(ordinal) = flag.source_ordinal { manifest .sources .get(usize::from(ordinal)) @@ -133,6 +141,9 @@ pub fn prepare( freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => { source.lane == SourceLane::Tui } + freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory => { + false + } } && (source.original_form || has_bind) }) @@ -257,6 +268,7 @@ mod tests { let raw = r#"{"instructions":["{env:INSERT}"]}"#; std::fs::write(stage.join(".freshell/source-views/source-0"), raw).unwrap(); let mut manifest = SourceManifest { + resource_roots: vec![], version: 1, cwd: "/original-unavailable/project".into(), original_home: "/original-unavailable/home".into(), diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index 247808cfd..3e55f1acd 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -39,6 +39,24 @@ fn selections(mounts: &docker::TerminalMounts) -> Result, Backen let directory = root.kind == freshell_runtime_protocol::ProviderResourceKind::Directory; add(&home.join(&root.provider_relative_path), directory)?; add(Path::new(&root.effective_destination), directory)?; + for leaf in mounts + .config_binds + .iter() + .map(|bind| &bind.reference.effective_destination) + .chain(mounts.indexed_files.iter().flat_map(|files| { + files + .aliases + .iter() + .map(|alias| &alias.effective_destination) + })) + { + if let Ok(relative) = Path::new(leaf).strip_prefix(&root.effective_destination) { + add( + &home.join(&root.provider_relative_path).join(relative), + false, + )?; + } + } } if let Some(files) = &mounts.indexed_files { for alias in &files.aliases { @@ -249,3 +267,44 @@ impl DockerEngineBackend { result } } + +#[cfg(test)] +mod tests { + use super::*; + use freshell_runtime_protocol::{ + ProviderConfigBind, ProviderResourceKind, ProviderResourceMount, + }; + + #[test] + fn provider_mountpoints_precreate_private_leaf_inside_external_selected_root() { + let mounts = docker::TerminalMounts { + workspace: "/synthetic/workspace".into(), + git_common_dir: None, + provider_bootstrap_files: vec![], + provider_secret_files: vec![], + indexed_secret_sources: BTreeMap::new(), + provider_user_root: None, + mcp_capability_file: None, + input_manifest: None, + indexed_files: None, + config_binds: vec![docker::ResolvedProviderConfigBind { + source: "/private/template".into(), + reference: ProviderConfigBind { + source_relative_path: ".freshell/templates/source-0".into(), + logical_destination: "/synthetic/workspace/.opencode/opencode.json".into(), + effective_destination: "/synthetic/workspace/.opencode/opencode.json".into(), + }, + }], + resource_mounts: vec![ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".freshell/provider-roots/root-0".into(), + logical_destination: "/synthetic/workspace/.opencode".into(), + effective_destination: "/synthetic/workspace/.opencode".into(), + kind: ProviderResourceKind::Directory, + }], + }; + let selected = selections(&mounts).unwrap(); + assert!(selected.iter().any(|point| point.relative_path == ".freshell/provider-roots/root-0/opencode.json" && !point.directory), + "exact private leaves must exist in the owned soul subpath before Docker mounts the external root"); + } +} From 4c878a719df2e3a26c091bf63c373c61f8c6957a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 20:02:27 -0700 Subject: [PATCH 060/493] fix: preserve provider root aliases and cross-root dependency links --- .../src/provider_environment.rs | 79 ++++++- .../src/managed_mcp_capability.rs | 61 +++++ .../managed_mcp_capability/source_stage.rs | 17 +- .../src/managed_provider_launch_tests.rs | 63 ++++- crates/freshell-session-host/src/main.rs | 223 ++++++++++++++++-- 5 files changed, 401 insertions(+), 42 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 0fe997108..b5d088454 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -578,6 +578,59 @@ fn copy_environment_with_account_home( limits: CopyLimits, account_home: Option, ) -> Result { + copy_environment_projection( + selections, + destination_root, + excluded_canonical_sources, + limits, + account_home, + &[], + ) +} + +/// Directory subpaths mounted at native discovery aliases retain in-unit +/// relative links, while cross-unit links address the final physical HOME. +pub fn copy_environment_for_mounts( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + limits: CopyLimits, + mounted_directories: &[PathBuf], +) -> Result { + let account_home = std::env::var_os("HOME").and_then(|home| fs::canonicalize(home).ok()); + copy_environment_projection( + selections, + destination_root, + excluded_canonical_sources, + limits, + account_home, + mounted_directories, + ) +} + +fn copy_environment_projection( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + limits: CopyLimits, + account_home: Option, + mounted_directories: &[PathBuf], +) -> Result { + bound( + Path::new(""), + "entries", + mounted_directories.len() as u64, + limits.max_entries as u64, + )?; + for root in mounted_directories { + if !valid_relative(root) || !destination_root.is_absolute() { + return Err(error( + root, + "select_mount_root", + "mounted resource requires a normal relative root and absolute destination HOME", + )); + } + } let mut plan = Plan { nodes: BTreeMap::new(), mappings: BTreeMap::new(), @@ -737,12 +790,17 @@ fn copy_environment_with_account_home( mode(&destination, private_file_mode(*source_mode), relative)?; } Node::Link(target) => { + let mounted = mounted_directories + .iter() + .find(|root| relative.starts_with(root)); + let target = if mounted.is_some_and(|root| !target.starts_with(root)) { + destination_root.join(target) + } else { + relative_link(relative.parent().unwrap_or(Path::new("")), target) + }; #[cfg(unix)] - std::os::unix::fs::symlink( - relative_link(relative.parent().unwrap_or(Path::new("")), target), - &destination, - ) - .map_err(|cause| io_error(relative, "create_link", cause))?; + std::os::unix::fs::symlink(target, &destination) + .map_err(|cause| io_error(relative, "create_link", cause))?; #[cfg(not(unix))] return Err(error( relative, @@ -862,6 +920,13 @@ mod tests { } } + fn executable_fixture() -> tempfile::TempDir { + match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::tempdir_in(root).unwrap(), + None => tempfile::tempdir().unwrap(), + } + } + #[test] fn failed_stage_creation_never_removes_a_preexisting_directory() { let root = tempfile::tempdir().unwrap(); @@ -935,7 +1000,7 @@ mod tests { #[cfg(unix)] #[test] fn copied_skill_executes_sibling_resources_after_two_stages() { - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture(); let source = root.path().join("source/skill"); fs::create_dir_all(source.join("scripts")).unwrap(); fs::create_dir_all(source.join("references")).unwrap(); @@ -991,7 +1056,7 @@ mod tests { #[cfg(unix)] #[test] fn copied_pnpm_plugin_external_resources_and_circular_dependencies_execute() { - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture(); let source = root.path().join("source"); let package = source.join("node_modules/.pnpm/fixture-dependency@1/node_modules/fixture-dependency"); diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 385574ecc..2cff63993 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4038,6 +4038,67 @@ except Exception as error: ); } + #[test] + fn managed_provider_home_and_workspace_root_aliases_share_one_physical_unit() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let shared = owned.path().join("shared-root"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(shared.join("skills/probe")).unwrap(); + std::fs::write(shared.join("skills/probe/SKILL.md"), "shared-root-resource").unwrap(); + std::os::unix::fs::symlink(&shared, home.join(".opencode")).unwrap(); + std::os::unix::fs::symlink(&shared, workspace.join(".opencode")).unwrap(); + let roots = vec![home.join(".opencode"), workspace.join(".opencode")]; + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let capability = store.issue(SoulId::new(), &directory, "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), OpencodeEphemeralInput { + cwd: Some(&workspace), workspace: Some(&workspace), source_plan: Some(&[]), + resource_roots: Some(&roots), secret_references: Some(&[]), ..Default::default() + }, None).expect("shared selected roots must keep both native discovery origins without competing physical copies"); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + assert_eq!(inputs.resource_mounts.len(), 2); + assert_eq!( + inputs.resource_mounts[0].stage_relative_path, + inputs.resource_mounts[1].stage_relative_path + ); + assert_eq!( + inputs.resource_mounts[0].provider_relative_path, + inputs.resource_mounts[1].provider_relative_path + ); + assert!(std::fs::symlink_metadata( + stage.join(&inputs.resource_mounts[0].stage_relative_path) + ) + .unwrap() + .is_dir()); + assert_eq!( + inputs.resource_mounts[0].effective_destination, + "/home/freshell/provider/.opencode" + ); + assert_eq!( + inputs.resource_mounts[1].effective_destination, + shared.to_str().unwrap() + ); + assert_eq!( + std::fs::read_to_string( + stage + .join(&inputs.resource_mounts[0].stage_relative_path) + .join("skills/probe/SKILL.md") + ) + .unwrap(), + "shared-root-resource" + ); + assert_eq!(std::fs::read_link(home.join(".opencode")).unwrap(), shared); + } + #[tokio::test] async fn managed_provider_shared_workspace_sources_keep_original_form_and_private_aliases() { use freshell_runtime_protocol::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 498235353..a92d1e304 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -46,14 +46,23 @@ pub(super) fn resource_units( roots.insert(parent.to_path_buf(), canonical); } let mut units = Vec::new(); - for (ordinal, (logical, canonical)) in roots.into_iter().enumerate() { + let mut identities = BTreeMap::new(); + for (logical, canonical) in roots { + let next = identities.len(); + let ordinal = *identities.entry(canonical.clone()).or_insert(next); let destination = crate::managed_provider_references::sources::runtime_directory_destination( home, global, &logical, ); - let effective = crate::managed_provider_references::sources::runtime_directory_destination( - home, global, &canonical, - ); + let effective = if destination.starts_with("/home/freshell/provider") { + // HOME is a new physical graph, so its native discovery root is + // mounted directly; the original user symlink remains untouched. + destination.clone() + } else { + crate::managed_provider_references::sources::runtime_directory_destination( + home, global, &canonical, + ) + }; units.push(( canonical, ProviderResourceMount { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 13ae61fe1..2d394b84c 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -352,6 +352,7 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ std::fs::write(nested.join("inline-rule"), "native-inline-selected-rule\n").unwrap(); let resource_root = fixture.join("workspace/.opencode"); let root_receipt = fixture.join("workspace/native-root-consumed.json"); + let home_root_receipt = fixture.join("workspace/native-home-root-consumed.json"); std::fs::create_dir_all(resource_root.join("plugins/selected/node_modules/fixture-dependency")) .unwrap(); std::fs::write( @@ -370,26 +371,58 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ ) .unwrap(); std::fs::write(resource_root.join("plugins/selected/index.mjs"), r#"import fs from 'node:fs';import crypto from 'node:crypto';import dependency from 'fixture-dependency'; -export default {id:'closed-writable-selected-root',async server(input,options){return {async config(config){ +const home=import.meta.url.startsWith('file:///home/freshell/provider/.opencode/'); +export default {id:'closed-writable-selected-root-'+(home?'home':'workspace'),async server(input,options){return {async config(config){ const root=new URL('../../',import.meta.url); const path=new URL('provider-owned-write',root); fs.writeFileSync(path,'selected-root-write'); - fs.writeFileSync(options.receipt,JSON.stringify({dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); + fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); }}}}; "#).unwrap(); let root_config = serde_json::to_vec( - &json!({"plugin":[["./plugins/selected/index.mjs", {"receipt":root_receipt}]]}), + &json!({"plugin":[["./plugins/selected/index.mjs", {"receipt":root_receipt, "homeReceipt":home_root_receipt}]]}), ) .unwrap(); std::fs::write(resource_root.join("opencode.json"), &root_config).unwrap(); + for name in ["package.json", "package-lock.json"] { + std::fs::copy( + fixture.join("source-home/.config/opencode").join(name), + resource_root.join(name), + ) + .unwrap(); + } + std::os::unix::fs::symlink( + fixture.join("source-home/.config/opencode/node_modules"), + resource_root.join("node_modules"), + ) + .unwrap(); + std::os::unix::fs::symlink(&resource_root, fixture.join("source-home/.opencode")).unwrap(); let explicit_directory = fixture.join("source-home/selected-directory"); let directory_receipt = fixture.join("workspace/native-directory-consumed.json"); std::fs::create_dir_all(explicit_directory.join("plugins")).unwrap(); std::fs::create_dir_all(explicit_directory.join("skills/probe")).unwrap(); + let transported_dependency = + fixture.join("source-home/.config/opencode/node_modules/transport-only"); + std::fs::create_dir_all(&transported_dependency).unwrap(); + std::fs::write( + transported_dependency.join("package.json"), + r#"{"name":"transport-only","type":"module","exports":"./index.mjs"}"#, + ) + .unwrap(); + std::fs::write( + transported_dependency.join("index.mjs"), + "export default 'transport-only-cross-root-dependency';", + ) + .unwrap(); std::fs::copy( fixture.join("source-home/.config/opencode/package.json"), explicit_directory.join("package.json"), ) .unwrap(); + std::fs::copy( + fixture.join("source-home/.config/opencode/package-lock.json"), + explicit_directory.join("package-lock.json"), + ) + .unwrap(); std::os::unix::fs::symlink( fixture.join("source-home/.config/opencode/node_modules"), explicit_directory.join("node_modules"), @@ -402,12 +435,20 @@ export default {id:'closed-writable-selected-root',async server(input,options){r .unwrap(); let directory_plugin = format!( r#"import fs from 'node:fs';import crypto from 'node:crypto'; -export default {{id:'closed-directory-discovery',async server(){{return {{async config(){{ +export default {{id:'closed-directory-discovery',async server(){{ + let dependency; + try {{dependency=(await import('transport-only')).default;}} catch(error) {{ + const link=new URL('../node_modules',import.meta.url); + fs.writeFileSync({},JSON.stringify({{failureClass:'dependency_load',linkIsSymbolic:fs.lstatSync(link).isSymbolicLink(),linkAbsolute:fs.lstatSync(link).isSymbolicLink()&&fs.readlinkSync(link).startsWith('/'),modulePresent:fs.existsSync(new URL('../node_modules/transport-only/index.mjs',import.meta.url))}})); + throw error; + }} + return {{async config(){{ const root=new URL('../',import.meta.url);const write=new URL('provider-owned-auto-write',root); fs.writeFileSync(write,'selected-directory-write'); - fs.writeFileSync({},JSON.stringify({{skill:crypto.createHash('sha256').update(fs.readFileSync(new URL('skills/probe/SKILL.md',root))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(write)).digest('hex')}})); + fs.writeFileSync({},JSON.stringify({{linkIsSymbolic:fs.lstatSync(new URL('node_modules',root)).isSymbolicLink(),linkAbsolute:fs.lstatSync(new URL('node_modules',root)).isSymbolicLink()&&fs.readlinkSync(new URL('node_modules',root)).startsWith('/'),dependency:crypto.createHash('sha256').update(dependency).digest('hex'),skill:crypto.createHash('sha256').update(fs.readFileSync(new URL('skills/probe/SKILL.md',root))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(write)).digest('hex')}})); }}}}}}}}; "#, + serde_json::to_string(&directory_receipt).unwrap(), serde_json::to_string(&directory_receipt).unwrap() ); std::fs::write( @@ -519,7 +560,12 @@ export default {{id:'closed-directory-discovery',async server(){{return {{async assert_eq!(terminal_receipt, expected); let expected_root = json!({"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); - let expected_directory = json!({"skill":format!("{:x}",Sha256::digest(b"selected-directory-skill")),"write":format!("{:x}",Sha256::digest(b"selected-directory-write"))}); + assert_eq!( + read_receipt(&home_root_receipt).await.unwrap(), + expected_root, + "actual native HOME and workspace discovery must both consume the shared transported root" + ); + let expected_directory = json!({"linkIsSymbolic":true,"linkAbsolute":true,"dependency":format!("{:x}",Sha256::digest(b"transport-only-cross-root-dependency")),"skill":format!("{:x}",Sha256::digest(b"selected-directory-skill")),"write":format!("{:x}",Sha256::digest(b"selected-directory-write"))}); assert_eq!( read_receipt(&directory_receipt).await.unwrap(), expected_directory @@ -577,6 +623,7 @@ export default {{id:'closed-directory-discovery',async server(){{return {{async std::fs::remove_file(&receipt).unwrap(); std::fs::remove_file(&shared_receipt).unwrap(); std::fs::remove_file(&root_receipt).unwrap(); + std::fs::remove_file(&home_root_receipt).unwrap(); std::fs::remove_file(&directory_receipt).unwrap(); // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. @@ -628,6 +675,10 @@ export default {{id:'closed-directory-discovery',async server(){{return {{async .await; assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); + assert_eq!( + read_receipt(&home_root_receipt).await.unwrap(), + expected_root + ); assert_eq!( read_receipt(&directory_receipt).await.unwrap(), expected_directory diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index ce8234b43..fbec49265 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1627,17 +1627,32 @@ fn copy_environment_with_events( ) -> Result< freshell_platform::provider_environment::CopyReport, freshell_platform::provider_environment::CopyError, +> { + copy_environment_with_events_for_mounts(stage, provider, selections, destination, excluded, &[]) +} + +fn copy_environment_with_events_for_mounts( + stage: &str, + provider: &str, + selections: &[freshell_platform::provider_environment::CopySelection], + destination: &Path, + excluded: &[PathBuf], + mounted_directories: &[PathBuf], +) -> Result< + freshell_platform::provider_environment::CopyReport, + freshell_platform::provider_environment::CopyError, > { let started = std::time::Instant::now(); emit_environment_event(serde_json::json!({ "severity": "info", "event": "provider.environment.copy.started", "stage": stage, "provider": provider, "selections": selections.len() })); - let report = freshell_platform::provider_environment::copy_environment( + let report = freshell_platform::provider_environment::copy_environment_for_mounts( selections, destination, excluded, Default::default(), + mounted_directories, ) .inspect_err(|error| { emit_environment_event(serde_json::json!({ @@ -1657,31 +1672,13 @@ fn copy_environment_with_events( Ok(report) } -fn copy_provider_config_references( - context: &freshell_runtime_protocol::ProviderLaunchContext, - provider: &str, - workspace: &Path, - user_provider_root: &Path, - provider_home: &Path, - run_as_uid: u32, - run_as_gid: u32, -) -> Result<(), String> { - copy_provider_config_references_with_ownership( - context, - provider, - workspace, - user_provider_root, - provider_home, - (run_as_uid, run_as_gid), - ) -} - fn copy_provider_config_references_with_ownership( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, workspace: &Path, user_provider_root: &Path, provider_home: &Path, + final_home: &Path, run_as: (u32, u32), ) -> Result<(), String> { use freshell_platform::provider_environment::CopySelection; @@ -1722,12 +1719,18 @@ fn copy_provider_config_references_with_ownership( } } } - let report = copy_environment_with_events( + let mounted = if provider_home == final_home { + resource_mount_roots(context) + } else { + Vec::new() + }; + let report = copy_environment_with_events_for_mounts( "session_host", provider, &selections, provider_home, &excluded, + &mounted, ) .map_err(|error| error.to_string())?; let projected = report @@ -1748,6 +1751,19 @@ fn copy_provider_config_references_with_ownership( set_owner(&manifest, run_as.0, run_as.1)?; Ok(()) } + +fn resource_mount_roots( + context: &freshell_runtime_protocol::ProviderLaunchContext, +) -> Vec { + context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .filter(|root| root.kind == freshell_runtime_protocol::ProviderResourceKind::Directory) + .map(|root| PathBuf::from(&root.provider_relative_path)) + .collect() +} + fn provider_credential_sources(root: &Path) -> Vec { [ ".credentials.json", @@ -1909,14 +1925,14 @@ fn refresh_provider_config_references( let stage = tempfile::tempdir().map_err(|error| error.to_string())?; let projection = stage.path().join("projection"); std::fs::create_dir(&projection).map_err(|error| error.to_string())?; - copy_provider_config_references( + copy_provider_config_references_with_ownership( context, provider, workspace, user_provider_root, &projection, - unsafe { libc::geteuid() }, - unsafe { libc::getegid() }, + provider_home, + (unsafe { libc::geteuid() }, unsafe { libc::getegid() }), )?; let mut destinations = provider_config_references(context) .iter() @@ -1945,6 +1961,13 @@ fn refresh_provider_config_references( .map_err(|_| "private mountpoint metadata invalid")?, ) .map_err(|_| "private mountpoint metadata write failed")?; + let mounted = freshell_runtime_protocol::bounded_json_bytes( + &resource_mount_roots(context), + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "resource mountpoint metadata profile exceeded")?; + std::fs::write(stage.path().join("resource-mountpoints.json"), mounted) + .map_err(|_| "resource mountpoint metadata write failed")?; allow_provider_read_stage(stage.path(), run_as_gid)?; if run_as_uid == unsafe { libc::geteuid() } && run_as_gid == unsafe { libc::getegid() } { return apply_staged_provider_config(stage.path(), provider_home); @@ -2105,12 +2128,39 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() ) { return Err("provider config refresh provider is unsupported".into()); } - copy_environment_with_events( + let mounted: Vec = { + use std::io::Read; + match std::fs::File::open(stage.join("resource-mountpoints.json")) { + Ok(file) => { + let mut bytes = Vec::new(); + file.take(freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "resource mountpoint metadata read failed")?; + if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err("resource mountpoint metadata profile exceeded".into()); + } + serde_json::from_slice(&bytes) + .map_err(|_| "resource mountpoint metadata invalid")? + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Vec::new(), + Err(_) => return Err("resource mountpoint metadata unavailable".into()), + } + }; + if mounted.iter().any(|path| { + !path.starts_with(".freshell/provider-roots") + || path + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + }) { + return Err("resource mountpoint metadata has an invalid owned root".into()); + } + copy_environment_with_events_for_mounts( "provider_publication", &provider, &selections, provider_home, &[], + &mounted, ) .map_err(|error| error.to_string())?; let mut stale = previous.difference(¤t).collect::>(); @@ -3158,6 +3208,26 @@ fn now_millis() -> i64 { #[cfg(test)] mod tests { + fn copy_provider_config_references( + context: &freshell_runtime_protocol::ProviderLaunchContext, + provider: &str, + workspace: &std::path::Path, + user_provider_root: &std::path::Path, + provider_home: &std::path::Path, + run_as_uid: u32, + run_as_gid: u32, + ) -> Result<(), String> { + super::copy_provider_config_references_with_ownership( + context, + provider, + workspace, + user_provider_root, + provider_home, + provider_home, + (run_as_uid, run_as_gid), + ) + } + fn executable_fixture() -> tempfile::TempDir { match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { Some(directory) => tempfile::tempdir_in(directory), @@ -3308,6 +3378,109 @@ mod tests { assert_eq!(output.stdout, b"resource-nonce"); } + #[test] + fn provider_config_external_mount_preserves_cross_root_dependency_resolution() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let stage = owned.path().join("stage"); + let home = owned.path().join("home"); + let native = owned.path().join("native-directory"); + std::fs::create_dir_all(stage.join("node_modules/transport-only")).unwrap(); + std::fs::create_dir_all(stage.join(".freshell/resource-units/root-0")).unwrap(); + std::fs::create_dir_all(&native).unwrap(); + std::fs::write( + stage.join("node_modules/transport-only/nonce"), + "cross-root-dependency-nonce", + ) + .unwrap(); + std::os::unix::fs::symlink( + "../../../node_modules", + stage.join(".freshell/resource-units/root-0/node_modules"), + ) + .unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + config: vec![ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "node_modules".into(), + provider_relative_path: ".config/opencode/node_modules".into(), + format: "directory".into(), + }], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + indexed_files: None, + resource_mounts: vec![ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".freshell/provider-roots/root-0".into(), + logical_destination: native.to_str().unwrap().into(), + effective_destination: native.to_str().unwrap().into(), + kind: ProviderResourceKind::Directory, + }], + }), + mcp_capability: Some(capability), + }; + copy_provider_config_references( + &context, + "opencode", + owned.path(), + &stage, + &home, + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + ) + .unwrap(); + let link = + std::fs::read_link(home.join(".freshell/provider-roots/root-0/node_modules")).unwrap(); + // A volume subpath is mounted at `native`, so this link is evaluated + // from that discovery parent rather than its physical soul parent. + let selected = native.join(link).join("transport-only/nonce"); + assert_eq!( + std::fs::read_to_string(selected).unwrap(), + "cross-root-dependency-nonce" + ); + std::fs::write( + stage.join("node_modules/transport-only/nonce"), + "cross-root-refreshed-nonce", + ) + .unwrap(); + refresh_provider_config_references( + &context, + "opencode", + owned.path(), + &stage, + &home, + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + ) + .unwrap(); + let link = + std::fs::read_link(home.join(".freshell/provider-roots/root-0/node_modules")).unwrap(); + assert_eq!( + std::fs::read_to_string(native.join(link).join("transport-only/nonce")).unwrap(), + "cross-root-refreshed-nonce" + ); + } + #[test] fn provider_config_selected_resource_root_populates_and_refreshes_stable_soul_directory() { use freshell_runtime_protocol::*; From bf1e3f44b4c49ad2c519fb40be0973d7ab67657a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 20:12:28 -0700 Subject: [PATCH 061/493] fix: project private files through each provider root alias --- .../src/managed_mcp_capability.rs | 92 +++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 41 ++++++--- .../src/managed_provider_launch_tests.rs | 17 +++- 3 files changed, 133 insertions(+), 17 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 2cff63993..ffb9d1e73 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4099,6 +4099,98 @@ except Exception as error: assert_eq!(std::fs::read_link(home.join(".opencode")).unwrap(), shared); } + #[tokio::test] + async fn managed_provider_shared_home_root_private_references_mount_at_each_native_origin() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let shared = owned.path().join("shared-root"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&shared).unwrap(); + let raw = r#"{"agent":{"probe":{"prompt":"{file:./rules}"}}}"#; + std::fs::write(shared.join("opencode.json"), raw).unwrap(); + std::fs::write(shared.join("rules"), "PRIVATE_SHARED_HOME_ROOT_NONCE").unwrap(); + std::os::unix::fs::symlink(&shared, home.join(".opencode")).unwrap(); + std::os::unix::fs::symlink(&shared, workspace.join(".opencode")).unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "existing-fixture", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|_| None, + references: vec![], + auth_authority: Some("existing-owned-attempt"), + selected_sources: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .unwrap(); + let inputs = prepared.context.prepared_inputs.as_ref().unwrap(); + let files = inputs.indexed_files.as_ref().unwrap(); + let home_alias = files + .aliases + .iter() + .find(|alias| alias.logical_destination == "/home/freshell/provider/.opencode/rules") + .unwrap(); + assert_eq!(home_alias.effective_destination, "/home/freshell/provider/.opencode/rules", "private child mounts do not propagate between two volume aliases; HOME needs its own exact file mount"); + let workspace_alias = files + .aliases + .iter() + .find(|alias| { + alias.logical_destination == workspace.join(".opencode/rules").to_str().unwrap() + }) + .unwrap(); + assert_eq!( + workspace_alias.effective_destination, + shared.join("rules").to_str().unwrap() + ); + assert_eq!(home_alias.source_id, workspace_alias.source_id); + let stage = staged_provider_root_path(&directory, &inputs.generation_id).unwrap(); + assert!( + !stage + .join(&inputs.resource_mounts[0].stage_relative_path) + .join("rules") + .exists(), + "sensitive referenced bytes cannot enter ordinary resource units" + ); + assert_eq!( + std::fs::read_to_string(shared.join("opencode.json")).unwrap(), + raw + ); + } + #[tokio::test] async fn managed_provider_shared_workspace_sources_keep_original_form_and_private_aliases() { use freshell_runtime_protocol::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index a92d1e304..b8cedcb36 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -242,6 +242,18 @@ pub(super) fn describe( let runtime_home = Path::new("/home/freshell/provider"); let default_global = home.join(".config/opencode"); let global = config_root.unwrap_or(&default_global); + let resource_mounts: Vec = { + let mut bytes = Vec::new(); + std::fs::File::open(root.join(".freshell/resource-mounts.json")) + .map_err(|_| "prepared resource manifest unavailable")? + .take(MAX_CONTROL_FRAME_BYTES as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "prepared resource manifest read failed")?; + if bytes.len() > MAX_CONTROL_FRAME_BYTES { + return Err("prepared resource descriptor exceeds control profile".into()); + } + serde_json::from_slice(&bytes).map_err(|_| "prepared resource descriptors invalid")? + }; let mut sources = BTreeMap::new(); let mut binds = Vec::new(); let mut recipes = 0usize; @@ -319,10 +331,22 @@ pub(super) fn describe( global, &approved.approved_alias, ); - let effective = + let effective = if let Some(unit) = resource_mounts.iter().find(|unit| { + unit.kind == ProviderResourceKind::Directory + && logical.starts_with(&unit.logical_destination) + }) { + Path::new(&unit.effective_destination).join( + logical + .strip_prefix(&unit.logical_destination) + .map_err(|_| { + "selected private alias lost its approved resource origin" + })?, + ) + } else { crate::managed_provider_references::sources::runtime_config_destination( home, global, selected, - ); + ) + }; if let Some(prior) = alias_sources.insert(logical.clone(), source_id) { if prior != source_id { return Err("shared source has conflicting private alias selections; reprepare before dispatch".into()); @@ -379,18 +403,7 @@ pub(super) fn describe( recipe_count: u16::try_from(recipes).map_err(|_| "recipe count overflow")?, }, config_binds: binds, - resource_mounts: { - let mut bytes = Vec::new(); - std::fs::File::open(root.join(".freshell/resource-mounts.json")) - .map_err(|_| "prepared resource manifest unavailable")? - .take(MAX_CONTROL_FRAME_BYTES as u64 + 1) - .read_to_end(&mut bytes) - .map_err(|_| "prepared resource manifest read failed")?; - if bytes.len() > MAX_CONTROL_FRAME_BYTES { - return Err("prepared resource descriptor exceeds control profile".into()); - } - serde_json::from_slice(&bytes).map_err(|_| "prepared resource descriptors invalid")? - }, + resource_mounts, indexed_files: (!sources.is_empty()).then(|| IndexedProviderFiles { volume_name: indexed_provider_volume_name(soul, &capability.grant_id), aliases, diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 2d394b84c..e1c49a832 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -375,14 +375,24 @@ const home=import.meta.url.startsWith('file:///home/freshell/provider/.opencode/ export default {id:'closed-writable-selected-root-'+(home?'home':'workspace'),async server(input,options){return {async config(config){ const root=new URL('../../',import.meta.url); const path=new URL('provider-owned-write',root); fs.writeFileSync(path,'selected-root-write'); - fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); + fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({privateRule:crypto.createHash('sha256').update(config.agent.rootScope.prompt).digest('hex'),dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); }}}}; "#).unwrap(); let root_config = serde_json::to_vec( - &json!({"plugin":[["./plugins/selected/index.mjs", {"receipt":root_receipt, "homeReceipt":home_root_receipt}]]}), + &json!({"agent":{"rootScope":{"prompt":"{file:./private-root-rule}"}}, "plugin":[["./plugins/selected/index.mjs", {"receipt":root_receipt, "homeReceipt":home_root_receipt}]]}), ) .unwrap(); std::fs::write(resource_root.join("opencode.json"), &root_config).unwrap(); + std::fs::write( + resource_root.join("private-root-rule"), + "shared-home-private-rule\n", + ) + .unwrap(); + std::fs::set_permissions( + resource_root.join("private-root-rule"), + std::fs::Permissions::from_mode(0o600), + ) + .unwrap(); for name in ["package.json", "package-lock.json"] { std::fs::copy( fixture.join("source-home/.config/opencode").join(name), @@ -558,7 +568,7 @@ export default {{id:'closed-directory-discovery',async server(){{ } }; assert_eq!(terminal_receipt, expected); - let expected_root = json!({"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); + let expected_root = json!({"privateRule":format!("{:x}",Sha256::digest(b"shared-home-private-rule")),"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); assert_eq!( read_receipt(&home_root_receipt).await.unwrap(), @@ -742,6 +752,7 @@ export default {{id:'closed-directory-discovery',async server(){{ b"native-inline-selected-rule".as_slice(), b"native-shared-root-rule".as_slice(), b"native-shared-nested-rule".as_slice(), + b"shared-home-private-rule".as_slice(), b"INLINE_PREFIX".as_slice(), b"AUTH_INLINE_SHADOW_NO_FILE".as_slice(), b"synthetic-onecli-shadowed".as_slice(), From cb7ba7683f8eac2616b3f8d3f827c8468ca93c33 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 20:45:17 -0700 Subject: [PATCH 062/493] fix: retain native provider HOME and workspace discovery roles --- .../freshell-platform/src/opencode_recipe.rs | 22 ++- .../src/managed_mcp_capability.rs | 174 +++++++++++++++++- .../managed_mcp_capability/source_stage.rs | 71 +++++-- .../src/managed_provider_launch_tests.rs | 25 ++- .../src/managed_provider_references.rs | 16 +- .../environment.rs | 3 + .../managed_provider_references/sources.rs | 132 +++++++++---- .../src/provider_input_preparation.rs | 1 + .../testing/managed-provider-launch-broker.ts | 2 +- 9 files changed, 374 insertions(+), 72 deletions(-) diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index 0269ac999..06955dfd4 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -24,6 +24,24 @@ pub struct GeneratedFileRecipe { /// Private filesystem metadata linking one native loader occurrence to its /// original source view. Documents and expanded values are never in this DTO. +/// Native discovery role, independent of the selected file's physical inode. +/// A project below the original HOME remains at its workspace path. +#[derive(Clone, Copy, Default, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum DiscoveryRole { + #[default] + Workspace, + Global, + Home, +} + +#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct DiscoveryRoot { + pub logical_path: PathBuf, + pub role: DiscoveryRole, +} + #[derive(Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct SourceManifest { @@ -39,7 +57,7 @@ pub struct SourceManifest { #[serde(default)] pub native_source_flags: Vec, #[serde(default)] - pub resource_roots: Vec, + pub resource_roots: Vec, #[serde(default = "default_rebind_enabled")] pub rebind_enabled: bool, #[serde(default)] @@ -78,6 +96,8 @@ impl NativeSourceFlagKind { pub struct SourceView { pub logical_path: PathBuf, #[serde(default)] + pub discovery_role: DiscoveryRole, + #[serde(default)] pub native_destination: Option, pub lane: SourceLane, pub unexpanded_relative_path: String, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index ffb9d1e73..afc46efa6 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -39,7 +39,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub workspace: Option<&'a Path>, pub source_plan: Option<&'a [crate::managed_provider_references::sources::PlannedSourceOccurrence]>, - pub resource_roots: Option<&'a [PathBuf]>, + pub resource_roots: Option<&'a [freshell_platform::opencode_recipe::DiscoveryRoot]>, pub initial_auth_reservation: Option<&'a str>, pub secret_references: Option<&'a [freshell_runtime_protocol::ProviderSecretReference]>, pub config_root: Option<&'a Path>, @@ -382,7 +382,7 @@ async fn apply_prepared_callback( let canonical=std::fs::canonicalize(&source.logical_path) .map_err(|_|format!("prior selected configuration source {ordinal} is unavailable; repair it before reissue"))?; Ok(crate::managed_provider_references::sources::SourceOccurrence { - logical_path:source.logical_path.clone(),canonical_source:canonical, + logical_path:source.logical_path.clone(),canonical_source:canonical,role:source.discovery_role, lane:match source.lane {freshell_platform::opencode_recipe::SourceLane::Main=>crate::managed_provider_references::sources::SourceLane::Main, freshell_platform::opencode_recipe::SourceLane::Tui=>crate::managed_provider_references::sources::SourceLane::Tui, freshell_platform::opencode_recipe::SourceLane::Inline=>crate::managed_provider_references::sources::SourceLane::Inline}, @@ -1125,8 +1125,16 @@ fn stage_provider_root( .map(|reference| PathBuf::from(&reference.source_path)), ); if let Some(plan) = opencode_input.source_plan { - let units = - source_stage::resource_units(home, &source, plan, opencode_input.resource_roots)?; + let units = source_stage::resource_units( + home, + &source, + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + plan, + opencode_input.resource_roots, + )?; selections.extend(units.iter().map(|(source, mount)| CopySelection { source: source.clone(), destination_relative: mount.stage_relative_path.clone().into(), @@ -1590,6 +1598,10 @@ impl CapabilityStore { &reference, opencode_input.initial_auth_reservation, opencode_input.config_root, + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), ) .and_then(|inputs| { if let Some(context) = opencode_input.launch_context { @@ -3330,11 +3342,13 @@ except Exception as error: logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, SourceOccurrence { logical_path: explicit.clone(), canonical_source: explicit.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, ], &workspace, @@ -4038,6 +4052,142 @@ except Exception as error: ); } + #[test] + fn managed_provider_ancestor_root_has_its_own_native_mount_location() { + use crate::managed_provider_references::sources::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path(); + let workspace = home.join("workspace"); + let root = workspace.join(".opencode"); + std::fs::create_dir_all(&root).unwrap(); + std::os::unix::fs::symlink(&root, home.join(".opencode")).unwrap(); + let roots = vec![ + DiscoveryRoot { + logical_path: root.clone(), + role: DiscoveryRole::Workspace, + }, + DiscoveryRoot { + logical_path: home.join(".opencode"), + role: DiscoveryRole::Workspace, + }, + DiscoveryRoot { + logical_path: home.join(".opencode"), + role: DiscoveryRole::Home, + }, + ]; + let units = source_stage::resource_units( + home, + &home.join(".config/opencode"), + &workspace, + &[], + Some(&roots), + ) + .unwrap(); + let ancestor = units + .iter() + .find(|(_, mount)| Path::new(&mount.logical_destination) == home.join(".opencode")) + .unwrap(); + assert_eq!(Path::new(&ancestor.1.effective_destination), home.join(".opencode"), + "ancestor discovery is outside the mounted workspace: its original logical symlink is absent in the container, so mount that exact logical root"); + let effective = units + .iter() + .map(|(_, mount)| &mount.effective_destination) + .collect::>(); + assert_eq!(effective.len(),3,"native HOME, workspace and outside ancestor discovery each need a mount without duplicate effective targets"); + assert!(units + .iter() + .all(|(_, mount)| mount.provider_relative_path == units[0].1.provider_relative_path)); + } + + #[test] + fn managed_provider_home_workspace_discovery_keeps_project_origins() { + use crate::managed_provider_references::sources::*; + use freshell_runtime_protocol::*; + for nested_workspace in [false, true] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = if nested_workspace { + home.join("project") + } else { + home.clone() + }; + let global = home.join(".config/opencode"); + std::fs::create_dir_all(&global).unwrap(); + std::fs::create_dir_all(workspace.join(".opencode/skills/probe")).unwrap(); + std::fs::write( + workspace.join("opencode.json"), + r#"{"agent":{"probe":{"description":"project-nonce"}}}"#, + ) + .unwrap(); + std::fs::write( + workspace.join(".opencode/opencode.json"), + r#"{"agent":{"probe":{"prompt":"root-nonce"}}}"#, + ) + .unwrap(); + std::fs::write( + workspace.join(".opencode/skills/probe/SKILL.md"), + "skill-nonce", + ) + .unwrap(); + let discovery = OpencodeDiscovery { + home: &home, + cwd: &workspace, + worktree: &workspace, + config_root: &global, + env: &|_| None, + }; + let roots = select_opencode_resource_roots(&discovery).unwrap(); + let sources = discover_opencode(discovery).unwrap(); + let plan = plan_opencode_sources(sources, &workspace, &home, &|_, _, _| None).unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let capability = CapabilityStore::default() + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + resource_roots: Some(&roots), + secret_references: Some(&[]), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + assert!(inputs.config_binds.iter().any(|bind| Path::new(&bind.logical_destination) == workspace.join("opencode.json") && Path::new(&bind.effective_destination) == workspace.join("opencode.json")), + "native project discovery beneath original HOME must consume the original workspace location"); + assert!(inputs.resource_mounts.iter().any(|root| Path::new(&root.logical_destination) == workspace.join(".opencode")), + "native workspace discovery must retain its original root independently of physical provider HOME"); + if !nested_workspace { + assert!(inputs.resource_mounts.iter().any(|root| root.logical_destination == "/home/freshell/provider/.opencode"), + "the same original HOME/workspace root also supplies provider-HOME discovery"); + } + let manifest: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert!(manifest + .sources + .iter() + .any(|source| source.native_destination.as_deref() + == workspace.join("opencode.json").to_str())); + assert_eq!( + std::fs::read_to_string(workspace.join("opencode.json")).unwrap(), + r#"{"agent":{"probe":{"description":"project-nonce"}}}"# + ); + } + } + #[test] fn managed_provider_home_and_workspace_root_aliases_share_one_physical_unit() { use freshell_runtime_protocol::*; @@ -4051,7 +4201,16 @@ except Exception as error: std::fs::write(shared.join("skills/probe/SKILL.md"), "shared-root-resource").unwrap(); std::os::unix::fs::symlink(&shared, home.join(".opencode")).unwrap(); std::os::unix::fs::symlink(&shared, workspace.join(".opencode")).unwrap(); - let roots = vec![home.join(".opencode"), workspace.join(".opencode")]; + let roots = vec![ + freshell_platform::opencode_recipe::DiscoveryRoot { + logical_path: home.join(".opencode"), + role: freshell_platform::opencode_recipe::DiscoveryRole::Home, + }, + freshell_platform::opencode_recipe::DiscoveryRoot { + logical_path: workspace.join(".opencode"), + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, + }, + ]; let directory = owned.path().join("control/mcp-capabilities"); std::fs::create_dir_all(&directory).unwrap(); let store = CapabilityStore::default(); @@ -4460,6 +4619,7 @@ except Exception as error: logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }], &workspace, &home, @@ -4698,6 +4858,7 @@ except Exception as error: logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }], &workspace, &home, @@ -4776,6 +4937,7 @@ except Exception as error: logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }], &workspace, &home, @@ -4854,6 +5016,7 @@ except Exception as error: logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Tui, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }], &workspace, &home, @@ -4932,6 +5095,7 @@ except Exception as error: logical_path: config.clone(), canonical_source: config, lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }], &workspace, &home, diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index b8cedcb36..bf2f9a4b7 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -9,8 +9,9 @@ use std::{collections::BTreeMap, path::Path}; pub(super) fn resource_units( home: &Path, global: &Path, + workspace: &Path, plan: &[PlannedSourceOccurrence], - accepted_roots: Option<&[std::path::PathBuf]>, + accepted_roots: Option<&[freshell_platform::opencode_recipe::DiscoveryRoot]>, ) -> Result< Vec<( std::path::PathBuf, @@ -21,12 +22,12 @@ pub(super) fn resource_units( use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; let mut roots = BTreeMap::new(); for root in accepted_roots.unwrap_or_default() { - let canonical = std::fs::canonicalize(root) + let canonical = std::fs::canonicalize(&root.logical_path) .map_err(|_| "selected OpenCode resource root is unavailable, dangling or cyclic")?; if !canonical.is_dir() { return Err("selected OpenCode resource root must be a directory".into()); } - roots.insert(root.clone(), canonical); + roots.insert((root.logical_path.clone(), root.role), canonical); } for source in plan { if source.source.lane == SourceLane::Inline { @@ -43,24 +44,27 @@ pub(super) fn resource_units( if !canonical.is_dir() { return Err("selected OpenCode resource root must be a directory".into()); } - roots.insert(parent.to_path_buf(), canonical); + roots.insert((parent.to_path_buf(), source.source.role), canonical); } let mut units = Vec::new(); let mut identities = BTreeMap::new(); - for (logical, canonical) in roots { + for ((logical, role), canonical) in roots { let next = identities.len(); let ordinal = *identities.entry(canonical.clone()).or_insert(next); let destination = crate::managed_provider_references::sources::runtime_directory_destination( - home, global, &logical, + home, global, &logical, role, ); - let effective = if destination.starts_with("/home/freshell/provider") { - // HOME is a new physical graph, so its native discovery root is - // mounted directly; the original user symlink remains untouched. + let effective = if destination.starts_with("/home/freshell/provider") + || !logical.starts_with(workspace) + { + // HOME is a new physical graph. Ancestor links outside the mounted + // workspace are also absent there, so mount their logical roots + // directly without modifying the original user topology. destination.clone() } else { crate::managed_provider_references::sources::runtime_directory_destination( - home, global, &canonical, + home, global, &canonical, role, ) }; units.push(( @@ -178,11 +182,13 @@ pub(super) fn stage( } manifest.sources.push(SourceView { logical_path: source.source.logical_path.clone(), + discovery_role: source.source.role, native_destination: Some( crate::managed_provider_references::sources::runtime_config_destination( home, input.config_root.unwrap_or(&home.join(".config/opencode")), &source.source.logical_path, + source.source.role, ) .to_str() .ok_or("native source destination is not UTF-8")? @@ -235,6 +241,7 @@ pub(super) fn describe( capability: &freshell_runtime_protocol::McpCapabilityReference, initial_auth_reservation: Option<&str>, config_root: Option<&Path>, + workspace: &Path, ) -> Result { use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; @@ -264,10 +271,18 @@ pub(super) fn describe( let logical = &occurrence.source.logical_path; let destination = freshell_platform::opencode_config::lexical_path( &crate::managed_provider_references::sources::runtime_config_destination( - home, global, logical, + home, + global, + logical, + occurrence.source.role, ), ); - let effective = if let Ok(relative) = logical.strip_prefix(&global) { + let effective = if occurrence.source.role + == freshell_platform::opencode_recipe::DiscoveryRole::Global + { + let relative = logical + .strip_prefix(global) + .map_err(|_| "global source is outside its discovery root")?; let physical = std::fs::canonicalize(root.join(relative)) .map_err(|_| "selected global config is unavailable in its private stage")?; let relative = physical @@ -278,13 +293,27 @@ pub(super) fn describe( } else { runtime_home.join(".config/opencode").join(relative) } + } else if let Some(unit) = resource_mounts + .iter() + .find(|unit| destination.starts_with(&unit.logical_destination)) + { + Path::new(&unit.effective_destination).join( + destination + .strip_prefix(&unit.logical_destination) + .map_err(|_| "config origin is outside its selected resource root")?, + ) + } else if occurrence.source.role + == freshell_platform::opencode_recipe::DiscoveryRole::Workspace + && !logical.starts_with(workspace) + { + destination.clone() } else { - occurrence - .source - .canonical_source - .strip_prefix(home) - .map(|relative| runtime_home.join(relative)) - .unwrap_or_else(|_| occurrence.source.canonical_source.clone()) + crate::managed_provider_references::sources::runtime_config_destination( + home, + global, + &occurrence.source.canonical_source, + occurrence.source.role, + ) }; binds.push(ProviderConfigBind { source_relative_path: format!(".freshell/templates/source-{ordinal}"), @@ -330,6 +359,7 @@ pub(super) fn describe( home, global, &approved.approved_alias, + occurrence.source.role, ); let effective = if let Some(unit) = resource_mounts.iter().find(|unit| { unit.kind == ProviderResourceKind::Directory @@ -344,7 +374,10 @@ pub(super) fn describe( ) } else { crate::managed_provider_references::sources::runtime_config_destination( - home, global, selected, + home, + global, + selected, + occurrence.source.role, ) }; if let Some(prior) = alias_sources.insert(logical.clone(), source_id) { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index e1c49a832..f3aff3e36 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -375,7 +375,7 @@ const home=import.meta.url.startsWith('file:///home/freshell/provider/.opencode/ export default {id:'closed-writable-selected-root-'+(home?'home':'workspace'),async server(input,options){return {async config(config){ const root=new URL('../../',import.meta.url); const path=new URL('provider-owned-write',root); fs.writeFileSync(path,'selected-root-write'); - fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({privateRule:crypto.createHash('sha256').update(config.agent.rootScope.prompt).digest('hex'),dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); + fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({physicalHome:process.env.HOME,privateRule:crypto.createHash('sha256').update(config.agent.rootScope.prompt).digest('hex'),dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); }}}}; "#).unwrap(); let root_config = serde_json::to_vec( @@ -406,7 +406,7 @@ export default {id:'closed-writable-selected-root-'+(home?'home':'workspace'),as ) .unwrap(); std::os::unix::fs::symlink(&resource_root, fixture.join("source-home/.opencode")).unwrap(); - let explicit_directory = fixture.join("source-home/selected-directory"); + let explicit_directory = fixture.join("workspace/selected-directory"); let directory_receipt = fixture.join("workspace/native-directory-consumed.json"); std::fs::create_dir_all(explicit_directory.join("plugins")).unwrap(); std::fs::create_dir_all(explicit_directory.join("skills/probe")).unwrap(); @@ -467,6 +467,12 @@ export default {{id:'closed-directory-discovery',async server(){{ ) .unwrap(); + // Qualify original HOME equal to the Git workspace, with a nested launch + // cwd. Native project discovery must stay in the workspace while provider + // HOME remains its independent physical volume. + let terminal_home = fixture.join("workspace"); + std::env::set_var("XDG_CONFIG_HOME", fixture.join("source-home/.config")); + std::env::set_var("HOME", &terminal_home); let request: ManagedTerminalLaunch = ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { program: "opencode".into(), @@ -477,10 +483,7 @@ export default {{id:'closed-directory-discovery',async server(){{ rows: 30, }, env: BTreeMap::from([ - ( - "OPENCODE_CONFIG_DIR".into(), - "../../source-home/selected-directory".into(), - ), + ("OPENCODE_CONFIG_DIR".into(), "../selected-directory".into()), ("OPENCODE_CONFIG".into(), "../selected-config.json".into()), ("OPENCODE_TUI_CONFIG".into(), "../selected-tui.jsonc".into()), ("OPENCODE_CONFIG_CONTENT".into(), inline_source.into()), @@ -568,7 +571,7 @@ export default {{id:'closed-directory-discovery',async server(){{ } }; assert_eq!(terminal_receipt, expected); - let expected_root = json!({"privateRule":format!("{:x}",Sha256::digest(b"shared-home-private-rule")),"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); + let expected_root = json!({"physicalHome":"/home/freshell/provider","privateRule":format!("{:x}",Sha256::digest(b"shared-home-private-rule")),"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); assert_eq!( read_receipt(&home_root_receipt).await.unwrap(), @@ -624,7 +627,7 @@ export default {{id:'closed-directory-discovery',async server(){{ assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); println!( "provider_launch_consumer {}", - json!({"provider":"opencode","route":"terminal","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) + json!({"provider":"opencode","route":"terminal","workspaceScope":"default-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) ); client .stop(SoulId::parse(terminal.soul_id).unwrap()) @@ -637,6 +640,10 @@ export default {{id:'closed-directory-discovery',async server(){{ std::fs::remove_file(&directory_receipt).unwrap(); // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. + // The fresh route has a workspace beneath original HOME. Both role + // topologies consume the same independently retained synthetic fixtures. + std::os::unix::fs::symlink(&resource_root, fixture.join(".opencode")).unwrap(); + std::env::set_var("HOME", &fixture); std::env::set_var("OPENCODE_CONFIG", &explicit_config); std::env::set_var("OPENCODE_CONFIG_DIR", &explicit_directory); std::env::set_var("OPENCODE_TUI_CONFIG", &selected_tui); @@ -733,7 +740,7 @@ export default {{id:'closed-directory-discovery',async server(){{ .expect("native plugin must stop the fresh request before the model loop"); println!( "provider_launch_consumer {}", - json!({"provider":"opencode","route":"fresh","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&fresh_expected).unwrap()))}) + json!({"provider":"opencode","route":"fresh","workspaceScope":"workspace-under-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&fresh_expected).unwrap()))}) ); // Scan only run-owned durable/staged surfaces; referenced payloads belong // in private indexed files, not ordinary resource/configuration trees. diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 4a61f30fc..eaa619ee1 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -723,7 +723,7 @@ pub(crate) struct OpencodePreparation<'a> { pub references: Vec, pub auth_authority: Option<&'a str>, pub selected_sources: Option>, - pub selected_resource_roots: Option>, + pub selected_resource_roots: Option>, pub native_source_flags: Option>, pub rebind_enabled: Option, pub inherited_inline: Option, @@ -867,6 +867,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( logical_path: "OPENCODE_CONFIG_CONTENT".into(), canonical_source: "OPENCODE_CONFIG_CONTENT".into(), lane: SourceLane::Inline, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, view: std::sync::Arc::new(view), }; @@ -898,12 +899,17 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( }; let source_ordinal = loaded .iter() - .position(|source| source.source.logical_path == path && match kind { + .position(|source| source.source.logical_path == path && source.source.role == DiscoveryRole::Workspace && match kind { freshell_platform::opencode_recipe::NativeSourceFlagKind::Main => source.source.lane == SourceLane::Main, freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => source.source.lane == SourceLane::Tui, freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory => false, }).ok_or_else(|| format!("selected {name} source is missing or unavailable; repair it before dispatch"))?; - let destination = sources::runtime_config_destination(home, &config_root, &path); + let destination = sources::runtime_config_destination( + home, + &config_root, + &path, + loaded[source_ordinal].source.role, + ); flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { kind, source_ordinal: Some( @@ -923,7 +929,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( } else { cwd.join(path) }; - if !resource_roots.contains(&path) { + if !resource_roots.iter().any(|root| root.logical_path == path) { return Err( "selected OPENCODE_CONFIG_DIR root is unavailable before dispatch".into(), ); @@ -935,6 +941,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( home, &config_root, &path, + DiscoveryRole::Workspace, ) .to_str() .ok_or("selected directory is not UTF-8")? @@ -957,6 +964,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( home, &config_root, &source.source.logical_path, + source.source.role, ) }) { diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index 7aafc3cfd..edf62daa5 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -262,6 +262,7 @@ mod tests { logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, view: std::sync::Arc::new(raw.into()), }] @@ -328,6 +329,7 @@ mod tests { logical_path: config.clone(), canonical_source: config, lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, view: std::sync::Arc::new(raw), }], @@ -385,6 +387,7 @@ mod tests { logical_path: config.clone(), canonical_source: config, lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, view: std::sync::Arc::new(raw.into()), }], diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 5bb268c70..aabb0ef03 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1,4 +1,5 @@ //! Ordered native source occurrences for managed OpenCode preparation. +pub use freshell_platform::opencode_recipe::{DiscoveryRole, DiscoveryRoot}; use std::path::{Path, PathBuf}; #[derive(Clone, Copy, PartialEq, Eq)] @@ -12,6 +13,7 @@ pub struct SourceOccurrence { pub logical_path: PathBuf, pub canonical_source: PathBuf, pub lane: SourceLane, + pub role: DiscoveryRole, } pub struct OpencodeDiscovery<'a> { @@ -33,6 +35,7 @@ pub fn discover_opencode( &mut sources, request.config_root.join(name), SourceLane::Main, + DiscoveryRole::Global, )?; } if let Some(path) = (request.env)("OPENCODE_CONFIG").filter(|value| !value.is_empty()) { @@ -40,6 +43,7 @@ pub fn discover_opencode( &mut sources, absolute_from_cwd(&path, request.cwd), SourceLane::Main, + DiscoveryRole::Workspace, )?; } if !disabled { @@ -48,17 +52,27 @@ pub fn discover_opencode( .rev() { for name in ["opencode.json", "opencode.jsonc"] { - add_source(&mut sources, directory.join(name), SourceLane::Main)?; + add_source( + &mut sources, + directory.join(name), + SourceLane::Main, + DiscoveryRole::Workspace, + )?; } } } for directory in config_directories(&request, Some(request.worktree), disabled)? { - if directory.file_name() == Some(std::ffi::OsStr::new(".opencode")) + if directory.logical_path.file_name() == Some(std::ffi::OsStr::new(".opencode")) || (request.env)("OPENCODE_CONFIG_DIR") - .is_some_and(|path| absolute_from_cwd(&path, request.cwd) == directory) + .is_some_and(|path| absolute_from_cwd(&path, request.cwd) == directory.logical_path) { for name in ["opencode.json", "opencode.jsonc"] { - add_source(&mut sources, directory.join(name), SourceLane::Main)?; + add_source( + &mut sources, + directory.logical_path.join(name), + SourceLane::Main, + directory.role, + )?; } } } @@ -67,6 +81,7 @@ pub fn discover_opencode( &mut sources, request.config_root.join(name), SourceLane::Tui, + DiscoveryRole::Global, )?; } if let Some(path) = (request.env)("OPENCODE_TUI_CONFIG").filter(|value| !value.is_empty()) { @@ -74,45 +89,70 @@ pub fn discover_opencode( &mut sources, absolute_from_cwd(&path, request.cwd), SourceLane::Tui, + DiscoveryRole::Workspace, )?; } if !disabled { for directory in ancestors(request.cwd, None).into_iter().rev() { for name in ["tui.json", "tui.jsonc"] { - add_source(&mut sources, directory.join(name), SourceLane::Tui)?; + add_source( + &mut sources, + directory.join(name), + SourceLane::Tui, + DiscoveryRole::Workspace, + )?; } } } for directory in config_directories(&request, None, disabled)? { - if directory.file_name() == Some(std::ffi::OsStr::new(".opencode")) + if directory.logical_path.file_name() == Some(std::ffi::OsStr::new(".opencode")) || (request.env)("OPENCODE_CONFIG_DIR") - .is_some_and(|path| absolute_from_cwd(&path, request.cwd) == directory) + .is_some_and(|path| absolute_from_cwd(&path, request.cwd) == directory.logical_path) { for name in ["tui.json", "tui.jsonc"] { - add_source(&mut sources, directory.join(name), SourceLane::Tui)?; + add_source( + &mut sources, + directory.logical_path.join(name), + SourceLane::Tui, + directory.role, + )?; } } } Ok(sources) } -pub fn runtime_config_destination(home: &Path, global: &Path, logical: &Path) -> PathBuf { +pub fn runtime_config_destination( + home: &Path, + global: &Path, + logical: &Path, + role: DiscoveryRole, +) -> PathBuf { let runtime_home = Path::new("/home/freshell/provider"); - if let Ok(relative) = logical.strip_prefix(global) { - runtime_home.join(".config/opencode").join(relative) - } else { - logical + match role { + DiscoveryRole::Global => logical + .strip_prefix(global) + .map(|relative| runtime_home.join(".config/opencode").join(relative)) + .unwrap_or_else(|_| logical.into()), + DiscoveryRole::Home => logical .strip_prefix(home) .map(|relative| runtime_home.join(relative)) - .unwrap_or_else(|_| logical.into()) + .unwrap_or_else(|_| logical.into()), + DiscoveryRole::Workspace => logical.into(), } } -pub fn runtime_directory_destination(home: &Path, global: &Path, logical: &Path) -> PathBuf { +pub fn runtime_directory_destination( + home: &Path, + global: &Path, + logical: &Path, + role: DiscoveryRole, +) -> PathBuf { runtime_config_destination( home, global, &freshell_platform::opencode_config::lexical_path(logical), + role, ) } @@ -140,21 +180,28 @@ pub fn config_directories( request: &OpencodeDiscovery<'_>, stop: Option<&Path>, disabled: bool, -) -> Result, &'static str> { - let mut result = vec![request.config_root.to_owned()]; +) -> Result, &'static str> { + let mut result = vec![DiscoveryRoot { + logical_path: request.config_root.to_owned(), + role: DiscoveryRole::Global, + }]; let mut candidates = Vec::new(); if !disabled { - candidates.extend( - ancestors(request.cwd, stop) - .into_iter() - .map(|directory| directory.join(".opencode")), - ); + candidates.extend(ancestors(request.cwd, stop).into_iter().map(|directory| { + DiscoveryRoot { + logical_path: directory.join(".opencode"), + role: DiscoveryRole::Workspace, + } + })); } - candidates.push(request.home.join(".opencode")); + candidates.push(DiscoveryRoot { + logical_path: request.home.join(".opencode"), + role: DiscoveryRole::Home, + }); for directory in candidates { - match std::fs::symlink_metadata(&directory) { + match std::fs::symlink_metadata(&directory.logical_path) { Ok(_) => { - if !std::fs::metadata(&directory) + if !std::fs::metadata(&directory.logical_path) .map_err(|_| "selected OpenCode config directory is unavailable")? .is_dir() { @@ -169,7 +216,10 @@ pub fn config_directories( } } if let Some(path) = (request.env)("OPENCODE_CONFIG_DIR").filter(|value| !value.is_empty()) { - let directory = absolute_from_cwd(&path, request.cwd); + let directory = DiscoveryRoot { + logical_path: absolute_from_cwd(&path, request.cwd), + role: DiscoveryRole::Workspace, + }; if !result.contains(&directory) { result.push(directory); } @@ -181,19 +231,27 @@ pub fn config_directories( /// configuration document exists. Keep these selections independent of files. pub fn select_opencode_resource_roots( request: &OpencodeDiscovery<'_>, -) -> Result, &'static str> { +) -> Result, &'static str> { let disabled = (request.env)("OPENCODE_DISABLE_PROJECT_CONFIG") .is_some_and(|value| matches!(value.as_str(), "1" | "true")); let mut roots = config_directories(request, None, disabled)?; - for directory in std::iter::once(request.home.to_path_buf()) - .chain(ancestors(request.cwd, Some(request.worktree))) + for (directory, role) in std::iter::once((request.home.to_path_buf(), DiscoveryRole::Home)) + .chain( + ancestors(request.cwd, Some(request.worktree)) + .into_iter() + .map(|path| (path, DiscoveryRole::Workspace)), + ) { for name in [".claude/skills", ".agents/skills"] { let path = directory.join(name); match std::fs::symlink_metadata(&path) { Ok(_) => { - if !roots.contains(&path) { - roots.push(path); + let root = DiscoveryRoot { + logical_path: path, + role, + }; + if !roots.contains(&root) { + roots.push(root); } } Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} @@ -201,9 +259,9 @@ pub fn select_opencode_resource_roots( } } } - roots.retain(|root| root != request.config_root); + roots.retain(|root| root.role != DiscoveryRole::Global); for root in &roots { - let canonical = std::fs::canonicalize(root).map_err(|_| { + let canonical = std::fs::canonicalize(&root.logical_path).map_err(|_| { "selected OpenCode resource directory is missing, inaccessible or cyclic" })?; if !canonical.is_dir() || canonical == request.home || canonical == Path::new("/") { @@ -217,6 +275,7 @@ fn add_source( sources: &mut Vec, logical_path: PathBuf, lane: SourceLane, + role: DiscoveryRole, ) -> Result<(), &'static str> { match std::fs::symlink_metadata(&logical_path) { Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), @@ -237,6 +296,7 @@ fn add_source( logical_path, canonical_source, lane, + role, }); Ok(()) } @@ -530,11 +590,13 @@ mod tests { logical_path: left.join("opencode.json"), canonical_source: shared.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, SourceOccurrence { logical_path: right.join("opencode.json"), canonical_source: shared.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, ]; let lookups = std::cell::Cell::new(0); @@ -584,11 +646,13 @@ mod tests { logical_path: first.clone(), canonical_source: first.clone(), lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, SourceOccurrence { logical_path: second.clone(), canonical_source: second, lane: SourceLane::Tui, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }, ]; let loaded = read_opencode_sources(sources).unwrap(); @@ -693,6 +757,7 @@ mod tests { logical_path: config.clone(), canonical_source: config, lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, }); } let plan = plan_opencode_sources(sources, &cwd, &home, &|_, _, _| None).unwrap(); @@ -732,6 +797,7 @@ mod tests { logical_path: config.clone(), canonical_source: config.clone(), lane, + role: DiscoveryRole::Workspace, }) .collect() }; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 997e25253..a6bf37bdf 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -273,6 +273,7 @@ mod tests { cwd: "/original-unavailable/project".into(), original_home: "/original-unavailable/home".into(), sources: vec![SourceView { + discovery_role: DiscoveryRole::Workspace, logical_path: "/original-unavailable/project/opencode.json".into(), native_destination: Some("/original-unavailable/project/opencode.json".into()), lane: SourceLane::Main, diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index c4a39aec3..8b21fb0a2 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -19,7 +19,7 @@ const broker=new RestrictedDockerBroker({ runtimeRootPrefix:path.join(root,'scenarios'),allowedHostBinaryPaths:new Set([configuration.hostBinaryPath]), allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), - allowedProviderUserRoots:new Set(['/home/freshell/provider']), + allowedProviderUserRoots:new Set(['/home/freshell/provider',path.join(root,'.opencode')]), allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json')]), allowedHostGatewayAddresses:gateways, testRunId:configuration.runId,logPath:path.join(root,'broker.jsonl'), From 968a613fad7d0d47af0ee8c749fe4cc123f08e84 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 21:06:52 -0700 Subject: [PATCH 063/493] fix: selectively project shared provider file references --- .../src/managed_mcp_capability.rs | 122 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 84 ++++++++---- .../src/managed_provider_launch_tests.rs | 31 +++-- .../src/managed_provider_references.rs | 33 ++++- .../managed_provider_references/sources.rs | 12 +- 5 files changed, 244 insertions(+), 38 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index afc46efa6..86cc69861 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4052,6 +4052,128 @@ except Exception as error: ); } + #[test] + fn managed_provider_shared_source_rewrites_only_home_and_absolute_file_tokens() { + use crate::managed_provider_references::sources::*; + use freshell_runtime_protocol::*; + for native_home_in_auth in [false, true] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let left = workspace.join("left"); + let right = workspace.join("right"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + for directory in [&left, &right] { + std::fs::create_dir_all(directory).unwrap(); + std::fs::write( + directory.join("relative"), + directory.file_name().unwrap().as_encoded_bytes(), + ) + .unwrap(); + } + let absolute = home.join("absolute-rule"); + let home_rule = home.join("home-rule"); + std::fs::write(&absolute, "absolute-private-rule").unwrap(); + std::fs::write(&home_rule, "home-private-rule").unwrap(); + let shared = workspace.join("shared.json"); + let raw = serde_json::to_string(&serde_json::json!({"instructions":["{file:./relative}",format!("{{file:{}}}",absolute.display()),"{file:~/home-rule}"],"agent":{"probe":{"prompt":"{env:AUTH}"}}})).unwrap(); + std::fs::write(&shared, &raw).unwrap(); + let sources = [&left, &right] + .into_iter() + .map(|directory| { + let logical = directory.join("opencode.json"); + std::os::unix::fs::symlink("../shared.json", &logical).unwrap(); + SourceOccurrence { + logical_path: logical, + canonical_source: shared.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + } + }) + .collect(); + let plan = plan_opencode_sources(sources, &workspace, &home, &|_, source, name| { + (name == "AUTH").then(|| { + let file = if native_home_in_auth { + if source.parent() == Some(left.as_path()) { + "~/home-rule".to_string() + } else { + absolute.display().to_string() + } + } else { + "./relative".to_string() + }; + format!("AUTH_PREFIX{{file:{file}}}_END") + }) + }) + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let capability = CapabilityStore::default() + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + secret_references: Some(&[]), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + assert_eq!( + inputs.config_binds.len(), + 1, + "one physical shared source must receive one uniform selective template" + ); + assert_eq!( + Path::new(&inputs.config_binds[0].effective_destination), + shared + ); + let template = + std::fs::read_to_string(stage.join(&inputs.config_binds[0].source_relative_path)) + .unwrap(); + assert!( + template.contains("{file:./relative}"), + "logical declaring parents must retain their distinct relative meanings" + ); + assert!( + template.contains("{env:AUTH}"), + "native loader-specific auth must remain native, without flattening source values" + ); + assert!( + !template.contains("{file:~/") && !template.contains(absolute.to_str().unwrap()), + "shared HOME and absolute file tokens must point directly to approved indexed payloads" + ); + assert_eq!( + inputs.indexed_files.as_ref().unwrap().aliases.len(), + if native_home_in_auth { 4 } else { 2 }, + "relative and interpolated native file expressions retain their exact aliases" + ); + if native_home_in_auth { + for logical in [ + PathBuf::from("/home/freshell/provider/home-rule"), + absolute.clone(), + ] { + assert!(inputs.indexed_files.as_ref().unwrap().aliases.iter().any(|alias| Path::new(&alias.logical_destination) == logical && Path::new(&alias.effective_destination) == logical), + "native interpolated HOME/absolute aliases must mount at the actual runtime path, not the controller source path"); + } + } + assert_eq!(std::fs::read_to_string(&shared).unwrap(), raw); + assert!(plan + .iter() + .all(|source| source.original_form && source.references.recipes.is_empty())); + } + } + #[test] fn managed_provider_ancestor_root_has_its_own_native_mount_location() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index bf2f9a4b7..35a143691 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -263,11 +263,15 @@ pub(super) fn describe( }; let mut sources = BTreeMap::new(); let mut binds = Vec::new(); + let mut bind_sources = BTreeMap::::new(); let mut recipes = 0usize; let mut aliases = Vec::new(); let mut alias_sources = BTreeMap::new(); for (ordinal, occurrence) in plan.iter().enumerate() { - if occurrence.source.lane != SourceLane::Inline && !occurrence.original_form { + if occurrence.source.lane != SourceLane::Inline + && (!occurrence.original_form + || occurrence.references.template != occurrence.references.unexpanded_source()) + { let logical = &occurrence.source.logical_path; let destination = freshell_platform::opencode_config::lexical_path( &crate::managed_provider_references::sources::runtime_config_destination( @@ -315,17 +319,25 @@ pub(super) fn describe( occurrence.source.role, ) }; - binds.push(ProviderConfigBind { - source_relative_path: format!(".freshell/templates/source-{ordinal}"), - logical_destination: destination - .to_str() - .ok_or("config destination is not UTF-8")? - .into(), - effective_destination: effective - .to_str() - .ok_or("config destination is not UTF-8")? - .into(), - }); + let relative = format!(".freshell/templates/source-{ordinal}"); + let prior = bind_sources.get(&effective); + if prior.is_some_and(|template| *template != occurrence.references.template) { + return Err("shared effective config target has incompatible templates; retain prior inputs and reprepare".into()); + } + if prior.is_none() { + bind_sources.insert(effective.clone(), &occurrence.references.template); + binds.push(ProviderConfigBind { + source_relative_path: relative, + logical_destination: destination + .to_str() + .ok_or("config destination is not UTF-8")? + .into(), + effective_destination: effective + .to_str() + .ok_or("config destination is not UTF-8")? + .into(), + }); + } } recipes = recipes .checked_add(occurrence.references.recipes.len()) @@ -346,6 +358,13 @@ pub(super) fn describe( } if occurrence.original_form { for approved in &occurrence.references.occurrences { + let Some(kind) = occurrence + .references + .native_alias_kinds + .get(&approved.ordinal) + else { + continue; + }; let source_id = approved .runtime_destination .strip_prefix("/home/freshell/provider/.freshell/references/") @@ -354,13 +373,27 @@ pub(super) fn describe( let selected = sources .get(&source_id) .ok_or("original-form alias index is unavailable")?; - let logical = - crate::managed_provider_references::sources::runtime_config_destination( - home, - global, - &approved.approved_alias, - occurrence.source.role, - ); + let logical = match kind { + crate::managed_provider_references::NativeAliasKind::Absolute => { + approved.approved_alias.clone() + } + crate::managed_provider_references::NativeAliasKind::Home => { + crate::managed_provider_references::sources::runtime_config_destination( + home, + global, + &approved.approved_alias, + crate::managed_provider_references::sources::DiscoveryRole::Home, + ) + } + crate::managed_provider_references::NativeAliasKind::Relative => { + crate::managed_provider_references::sources::runtime_config_destination( + home, + global, + &approved.approved_alias, + occurrence.source.role, + ) + } + }; let effective = if let Some(unit) = resource_mounts.iter().find(|unit| { unit.kind == ProviderResourceKind::Directory && logical.starts_with(&unit.logical_destination) @@ -373,12 +406,13 @@ pub(super) fn describe( })?, ) } else { - crate::managed_provider_references::sources::runtime_config_destination( - home, - global, - selected, - occurrence.source.role, - ) + match kind { + crate::managed_provider_references::NativeAliasKind::Absolute if logical.starts_with(workspace) => selected.as_path().to_path_buf(), + crate::managed_provider_references::NativeAliasKind::Relative + if occurrence.source.role == crate::managed_provider_references::sources::DiscoveryRole::Workspace + && occurrence.source.logical_path.starts_with(workspace) => selected.as_path().to_path_buf(), + _ => logical.clone(), + } }; if let Some(prior) = alias_sources.insert(logical.clone(), source_id) { if prior != source_id { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index f3aff3e36..92e83ae4a 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -251,10 +251,10 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr let inline_auth_url = format!("{well_known_url}/inline-auth"); std::fs::write( fixture.join("grants/auth.json"), - serde_json::to_vec(&json!({well_known_url: { + serde_json::to_vec(&json!({well_known_url.clone(): { "type": "wellknown", "key": "OPENAI_API_KEY", - "token": "AUTH_PREFIX{file:./rules.txt}AUTH_SUFFIX" - }, inline_auth_url: {"type":"wellknown","key":"INLINE_VALUE","token":"AUTH_INLINE_SHADOW_NO_FILE"}})) + "token": format!("AUTH_PREFIX{{file:{}}}AUTH_SUFFIX", fixture.join("workspace/home rule with spaces.txt").display()) + }, inline_auth_url.clone(): {"type":"wellknown","key":"INLINE_VALUE","token":"AUTH_INLINE_SHADOW_NO_FILE"}})) .unwrap(), ) .unwrap(); @@ -293,7 +293,16 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr .success()); let shared_source = fixture.join("workspace/shared.json"); let shared_receipt = fixture.join("workspace/shared-origin-consumed.json"); - let shared_bytes = serde_json::to_vec(&json!({"agent":{"probe":{"prompt":"{env:OPENAI_API_KEY}"}}, "plugin":[["./shared-probe.mjs", {"receipt":shared_receipt}]]})).unwrap(); + let shared_home_rule = fixture.join("workspace/home rule with spaces.txt"); + std::fs::write(&shared_home_rule, "shared-home-sensitive-file\n").unwrap(); + std::fs::set_permissions(&shared_home_rule, std::fs::Permissions::from_mode(0o600)).unwrap(); + let shared_bytes = serde_json::to_vec(&json!({"agent":{ + "probe":{"prompt":"{env:OPENAI_API_KEY}"}, + "sharedRelative":{"prompt":"{file:./rules.txt}"}, + "sharedAbsolute":{"prompt":format!("{{file:{}}}", shared_home_rule.display())}, + "sharedHome":{"prompt":"{file:~/home rule with spaces.txt}"} + }, "plugin":[["./shared-probe.mjs", {"receipt":shared_receipt}]]})) + .unwrap(); for (directory, label) in [ (fixture.join("workspace"), "root"), (nested.clone(), "nested"), @@ -302,7 +311,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr r#"import fs from 'node:fs';import crypto from 'node:crypto'; export default {{id:'closed-shared-{label}',async server(input,options){{return {{async config(config){{ let receipt=fs.existsSync(options.receipt)?JSON.parse(fs.readFileSync(options.receipt,'utf8')):{{}}; - receipt.{label}=crypto.createHash('sha256').update(fs.readFileSync(new URL('./rules.txt',import.meta.url))).digest('hex'); + receipt.{label}={{relative:crypto.createHash('sha256').update(fs.readFileSync(new URL('./rules.txt',import.meta.url))).digest('hex'),absolute:crypto.createHash('sha256').update(config.agent.sharedAbsolute.prompt).digest('hex'),home:crypto.createHash('sha256').update(config.agent.sharedHome.prompt).digest('hex')}}; fs.writeFileSync(options.receipt,JSON.stringify(receipt)); }}}}}}}}; "# @@ -520,7 +529,7 @@ export default {{id:'closed-directory-discovery',async server(){{ .await .map_err(|_| "actual pinned provider startup did not consume selected config and closure") } - let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXnative-shared-nested-ruleAUTH_SUFFIX")), + let expected = json!({"prompt":format!("{:x}",Sha256::digest(b"AUTH_PREFIXshared-home-sensitive-fileAUTH_SUFFIX")), "description":format!("{:x}",Sha256::digest(b"INLINE_PREFIXnative-inline-selected-ruleINLINE_SUFFIX")), "references":(["third","fourth","fifth","sixth"].map(|ordinal|format!("{:x}",Sha256::digest(format!("native-{ordinal}-rule").as_bytes())))), "resource":format!("{:x}",Sha256::digest(b"plugin-relative-resource")), @@ -596,7 +605,7 @@ export default {{id:'closed-directory-discovery',async server(){{ ); assert_eq!( read_receipt(&shared_receipt).await.unwrap(), - json!({"root":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"nested":format!("{:x}",Sha256::digest(b"native-shared-nested-rule"))}) + json!({"root":{"relative":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"absolute":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file")),"home":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file"))},"nested":{"relative":format!("{:x}",Sha256::digest(b"native-shared-nested-rule")),"absolute":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file")),"home":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file"))}}) ); assert_eq!( read_receipt(&tui_receipt) @@ -643,6 +652,11 @@ export default {{id:'closed-directory-discovery',async server(){{ // The fresh route has a workspace beneath original HOME. Both role // topologies consume the same independently retained synthetic fixtures. std::os::unix::fs::symlink(&resource_root, fixture.join(".opencode")).unwrap(); + std::os::unix::fs::symlink(&shared_home_rule, fixture.join("home rule with spaces.txt")) + .unwrap(); + std::fs::write(fixture.join("grants/auth.json"), serde_json::to_vec(&json!({well_known_url: { + "type":"wellknown", "key":"OPENAI_API_KEY", "token":"AUTH_PREFIX{file:~/home rule with spaces.txt}AUTH_SUFFIX" + }, inline_auth_url: {"type":"wellknown", "key":"INLINE_VALUE", "token":"AUTH_INLINE_SHADOW_NO_FILE"}})).unwrap()).unwrap(); std::env::set_var("HOME", &fixture); std::env::set_var("OPENCODE_CONFIG", &explicit_config); std::env::set_var("OPENCODE_CONFIG_DIR", &explicit_directory); @@ -710,7 +724,7 @@ export default {{id:'closed-directory-discovery',async server(){{ ); assert_eq!( read_receipt(&shared_receipt).await.unwrap(), - json!({"root":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"nested":format!("{:x}",Sha256::digest(b"native-shared-nested-rule"))}) + json!({"root":{"relative":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"absolute":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file")),"home":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file"))},"nested":{"relative":format!("{:x}",Sha256::digest(b"native-shared-nested-rule")),"absolute":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file")),"home":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file"))}}) ); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); assert_eq!(std::fs::read(&shared_source).unwrap(), shared_bytes); @@ -760,6 +774,7 @@ export default {{id:'closed-directory-discovery',async server(){{ b"native-shared-root-rule".as_slice(), b"native-shared-nested-rule".as_slice(), b"shared-home-private-rule".as_slice(), + b"shared-home-sensitive-file".as_slice(), b"INLINE_PREFIX".as_slice(), b"AUTH_INLINE_SHADOW_NO_FILE".as_slice(), b"synthetic-onecli-shadowed".as_slice(), diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index eaa619ee1..b48c9b452 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -44,6 +44,13 @@ pub struct FileReferenceSelection { pub use freshell_platform::opencode_recipe::{ApprovedFileOccurrence, GeneratedFileRecipe}; +#[derive(Clone, Copy)] +pub(crate) enum NativeAliasKind { + Relative, + Absolute, + Home, +} + pub struct ResolvedFileReferencePlan { pub template: String, // Generated recipes already own the raw view. Keep one raw view for @@ -55,6 +62,7 @@ pub struct ResolvedFileReferencePlan { pub recipes: Vec, pub occurrences: Vec, pub substitution_events: usize, + pub native_alias_kinds: std::collections::BTreeMap, } impl ResolvedFileReferencePlan { pub fn unexpanded_source(&self) -> &str { @@ -130,6 +138,7 @@ pub(super) fn resolve_file_references_from_view_relocated( }; let mut selections: Vec = Vec::new(); let mut rewritten = Vec::new(); + let mut native_alias_kinds = std::collections::BTreeMap::new(); let mut occurrences = Vec::new(); let mut path_variables = std::collections::BTreeSet::new(); let mut generated = false; @@ -254,7 +263,24 @@ pub(super) fn resolve_file_references_from_view_relocated( runtime_destination: destination.clone(), }); let replacement = format!("{{file:{destination}}}"); - rewritten.push((token.original_span, replacement.clone())); + let kind = if token.path.starts_with("~/") { + NativeAliasKind::Home + } else if Path::new(token.path).is_absolute() { + NativeAliasKind::Absolute + } else { + NativeAliasKind::Relative + }; + // Only literal source-independent paths can receive an identical edit + // at a shared inode. Relative paths and loader-specific interpolation + // retain native evaluation and their approved occurrence aliases. + let selective = token.direct_expression + && token.variables.is_empty() + && !matches!(kind, NativeAliasKind::Relative); + if !projection.original_form || selective { + rewritten.push((token.original_span, replacement.clone())); + } else { + native_alias_kinds.insert(token.ordinal, kind); + } Ok(replacement) }) .map_err(|error| match error { @@ -270,9 +296,7 @@ pub(super) fn resolve_file_references_from_view_relocated( } let transformed_length = transformed.len(); drop(transformed); - let (template, recipes) = if projection.original_form { - (raw.to_owned(), Vec::new()) - } else if generated { + let (template, recipes) = if generated && !projection.original_form { let generated_name = freshell_platform::opencode_config::generated_config_reference_name(recipe_ordinal); if transformed_length > 64 * 1024 { @@ -321,6 +345,7 @@ pub(super) fn resolve_file_references_from_view_relocated( recipes, occurrences, substitution_events: substitution_events.get(), + native_alias_kinds, }) } diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index aabb0ef03..495392bea 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -444,7 +444,7 @@ pub fn plan_loaded_opencode_sources_relocated( .entry(source.source.canonical_source.clone()) .or_insert(0usize) += 1; } - let mut result = Vec::new(); + let mut result: Vec = Vec::new(); let mut selected = std::collections::BTreeMap::::new(); let mut selected_bytes = 0u64; let mut recipe_count = 0usize; @@ -517,6 +517,16 @@ pub fn plan_loaded_opencode_sources_relocated( original_form, }, )?; + if original_form { + if let Some(prior) = result + .iter() + .find(|prior| prior.source.canonical_source == source.canonical_source) + { + if prior.references.template != references.template { + return Err(fail("shared source requires incompatible native lane edits; retain its inputs and reprepare before dispatch")); + } + } + } if references .recipes .iter() From 69965d9b08bc0b0b27b8c80ff1573a05a6b45d82 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 21:40:30 -0700 Subject: [PATCH 064/493] fix: retain accepted provider roots and block unknown discovery roles --- .../freshell-platform/src/opencode_recipe.rs | 21 ++ .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 180 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 13 ++ .../src/managed_provider_references.rs | 33 +++- .../managed_provider_references/sources.rs | 2 +- crates/freshell-server/src/managed_runtime.rs | 1 + .../src/provider_input_preparation.rs | 22 +++ 8 files changed, 263 insertions(+), 10 deletions(-) diff --git a/crates/freshell-platform/src/opencode_recipe.rs b/crates/freshell-platform/src/opencode_recipe.rs index 06955dfd4..127100c44 100644 --- a/crates/freshell-platform/src/opencode_recipe.rs +++ b/crates/freshell-platform/src/opencode_recipe.rs @@ -30,6 +30,7 @@ pub struct GeneratedFileRecipe { #[serde(rename_all = "snake_case")] pub enum DiscoveryRole { #[default] + Unknown, Workspace, Global, Home, @@ -48,6 +49,8 @@ pub struct SourceManifest { pub version: u8, pub cwd: PathBuf, pub original_home: PathBuf, + #[serde(default)] + pub global_config_root: Option, pub sources: Vec, pub files: Vec, #[serde(default)] @@ -63,6 +66,24 @@ pub struct SourceManifest { #[serde(default)] pub inline_edits: crate::opencode_config::OwnedConfigEdits, } +impl SourceManifest { + pub fn require_known_discovery_roles(&self) -> Result<(), &'static str> { + if self + .sources + .iter() + .any(|source| source.discovery_role == DiscoveryRole::Unknown) + || self + .resource_roots + .iter() + .any(|root| root.role == DiscoveryRole::Unknown) + { + Err("native discovery role is unrecorded") + } else { + Ok(()) + } + } +} + fn default_rebind_enabled() -> bool { true } diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 0f512dc19..b32b25018 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -468,6 +468,7 @@ impl HostedFreshAgentProxy { references: provider_secret_references, auth_authority: None, selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 86cc69861..bdfc877a5 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -436,6 +436,7 @@ async fn apply_prepared_callback( .ok_or("replacement provider preparation requires attempt authority")?, ), selected_sources, + accepted_global_root: accepted_manifest.as_ref().map(|manifest| manifest.global_config_root.clone().ok_or_else(|| "accepted global configuration root is unrecorded; retain prior inputs and reprepare before dispatch".to_string())).transpose()?, selected_resource_roots: accepted_manifest .as_ref() .map(|manifest| manifest.resource_roots.clone()), @@ -538,6 +539,10 @@ fn read_prior_source_manifest( { return Err("prior source manifest origin profile invalid".into()); } + manifest.require_known_discovery_roles().map_err(|_| { + "prior native discovery role is unrecorded; retain inputs and reprepare before dispatch" + .to_string() + })?; Ok(Some(manifest)) } @@ -3602,6 +3607,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, @@ -3734,6 +3740,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, @@ -3943,6 +3950,7 @@ except Exception as error: references: vec![], auth_authority: Some("existing-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, @@ -4052,6 +4060,174 @@ except Exception as error: ); } + #[tokio::test] + async fn managed_provider_replacement_retains_accepted_global_root_without_current_xdg() { + use freshell_runtime_protocol::*; + for document in [false, true] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let xdg = owned.path().join("accepted-xdg"); + let global = xdg.join("opencode"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(global.join("plugins/probe")).unwrap(); + std::fs::write( + global.join("plugins/probe/resource"), + "accepted-global-resource", + ) + .unwrap(); + if document { + std::fs::write( + global.join("opencode.json"), + r#"{"plugin":["./plugins/probe/index.mjs"]}"#, + ) + .unwrap(); + } + std::fs::write( + global.join("plugins/probe/index.mjs"), + "export default async()=>({});", + ) + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = + crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "existing-fixture", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| { + (name == "XDG_CONFIG_HOME").then(|| xdg.to_str().unwrap().into()) + }, + references: vec![], + auth_authority: Some("existing-owned-attempt"), + selected_sources: None, + accepted_global_root: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .unwrap(); + let prior = prepared.context.mcp_capability.as_ref().unwrap(); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + assert_eq!( + std::fs::read_to_string(prior_stage.join("plugins/probe/resource")).unwrap(), + "accepted-global-resource" + ); + if document { + use sha2::Digest; + let manifest_path = prior_stage.join(".freshell/input-manifest.json"); + let current = std::fs::read(&manifest_path).unwrap(); + let mut legacy: serde_json::Value = serde_json::from_slice(¤t).unwrap(); + for source in legacy["sources"].as_array_mut().unwrap() { + source.as_object_mut().unwrap().remove("discoveryRole"); + } + let bytes = serde_json::to_vec(&legacy).unwrap(); + std::fs::write(&manifest_path, &bytes).unwrap(); + let mut legacy_context = prepared.context.clone(); + legacy_context + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .sha256 = format!("{:x}", sha2::Sha256::digest(&bytes)); + let legacy_callback = serde_json::to_vec(&serde_json::json!({"controlSecret":"fixture-control", "action":"issue", "provider":"opencode", "soulId":soul, + "priorContext":legacy_context, "providerSecretReferences":[], "workspace":workspace, "cwd":workspace, "authLookupAuthority":"existing-owned-attempt"})).unwrap(); + let error = apply_prepared_callback( + &legacy_callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .err() + .unwrap_or_default(); + assert!(error.contains("discovery role"), "missing role metadata is unknown evidence and must block before a replacement generation is prepared"); + assert!(prior_stage.is_dir()); + std::fs::write(&manifest_path, current).unwrap(); + } + std::fs::write( + global.join("plugins/probe/resource"), + "refreshed-accepted-global-resource", + ) + .unwrap(); + let callback = serde_json::to_vec(&serde_json::json!({"controlSecret":"fixture-control", "action":"issue", "provider":"opencode", "soulId":soul, + "priorContext":prepared.context, "providerSecretReferences":[], "workspace":workspace, "cwd":workspace, "authLookupAuthority":"existing-owned-attempt"})).unwrap(); + let ready: PreparedProviderContext = serde_json::from_slice( + &apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &home, + &client, + ) + .await + .unwrap(), + ) + .unwrap(); + let next_stage = staged_provider_root_path( + &directory, + &ready + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap() + .grant_id, + ) + .unwrap(); + assert_eq!(std::fs::read_to_string(next_stage.join("plugins/probe/resource")).expect("reissue must refresh the accepted global resource graph independently of the current XDG env"), "refreshed-accepted-global-resource"); + assert!( + prior_stage.is_dir(), + "replacement preparation cannot retire prior usable inputs" + ); + if document { + let bind = &ready + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap() + .config_binds[0]; + assert_eq!( + bind.logical_destination, + "/home/freshell/provider/.config/opencode/opencode.json" + ); + } + } + } + #[test] fn managed_provider_shared_source_rewrites_only_home_and_absolute_file_tokens() { use crate::managed_provider_references::sources::*; @@ -4415,6 +4591,7 @@ except Exception as error: references: vec![], auth_authority: Some("existing-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, @@ -4508,6 +4685,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, @@ -4610,6 +4788,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, @@ -4702,6 +4881,7 @@ except Exception as error: references: vec![], auth_authority: Some("fixture-owned-attempt"), selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 35a143691..09e4d814c 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -22,6 +22,9 @@ pub(super) fn resource_units( use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; let mut roots = BTreeMap::new(); for root in accepted_roots.unwrap_or_default() { + if root.role == crate::managed_provider_references::sources::DiscoveryRole::Unknown { + return Err("selected resource discovery role is unrecorded".into()); + } let canonical = std::fs::canonicalize(&root.logical_path) .map_err(|_| "selected OpenCode resource root is unavailable, dangling or cyclic")?; if !canonical.is_dir() { @@ -125,6 +128,12 @@ pub(super) fn stage( version: 1, cwd: cwd.into(), original_home: home.into(), + global_config_root: Some( + input + .config_root + .unwrap_or(&home.join(".config/opencode")) + .to_path_buf(), + ), sources: Vec::new(), files: Vec::new(), frozen_inputs: input.frozen_inputs.unwrap_or(&[]).to_vec(), @@ -139,6 +148,10 @@ pub(super) fn stage( let mut files = BTreeMap::new(); let mut view_bytes = 0usize; for (ordinal, source) in plan.iter().enumerate() { + if source.source.role == crate::managed_provider_references::sources::DiscoveryRole::Unknown + { + return Err("selected configuration discovery role is unrecorded".into()); + } if source.source.lane == SourceLane::Main { manifest.user_owns_freshell_mcp |= freshell_platform::opencode_config::source_owns_freshell_mcp( diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index b48c9b452..4f5fb983d 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -748,6 +748,7 @@ pub(crate) struct OpencodePreparation<'a> { pub references: Vec, pub auth_authority: Option<&'a str>, pub selected_sources: Option>, + pub accepted_global_root: Option, pub selected_resource_roots: Option>, pub native_source_flags: Option>, pub rebind_enabled: Option, @@ -785,6 +786,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( mut references, auth_authority, selected_sources, + accepted_global_root, selected_resource_roots, native_source_flags, rebind_enabled, @@ -807,16 +809,29 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( inline_edits.bash_permission = original("FRESHELL_MANAGED_OPENCODE_BASH_PERMISSION") .filter(|value| matches!(value.as_str(), "allow" | "ask" | "deny")); } - let xdg = original("XDG_CONFIG_HOME") - .filter(|value| !value.is_empty()) - .map(PathBuf::from) - .unwrap_or_else(|| home.join(".config")); - let xdg = if xdg.is_absolute() { - xdg + let config_root = if let Some(root) = accepted_global_root { + if !root.is_absolute() || root.file_name() != Some(std::ffi::OsStr::new("opencode")) { + return Err( + "accepted global configuration root is invalid; retain prior inputs and reprepare" + .into(), + ); + } + root } else { - cwd.join(xdg) + let xdg = original("XDG_CONFIG_HOME") + .filter(|value| !value.is_empty()) + .map(PathBuf::from) + .unwrap_or_else(|| home.join(".config")); + let xdg = if xdg.is_absolute() { + xdg + } else { + cwd.join(xdg) + }; + xdg.join("opencode") }; - let config_root = xdg.join("opencode"); + let xdg = config_root + .parent() + .ok_or("global configuration root parent unavailable")?; let resource_roots = if let Some(roots) = selected_resource_roots { roots } else { @@ -834,7 +849,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( workspace, cwd, home, - Some(&xdg), + Some(xdg), None, )? .ok_or("OpenCode provider preparation unavailable")?; diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 495392bea..6323d33d0 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -138,7 +138,7 @@ pub fn runtime_config_destination( .strip_prefix(home) .map(|relative| runtime_home.join(relative)) .unwrap_or_else(|_| logical.into()), - DiscoveryRole::Workspace => logical.into(), + DiscoveryRole::Workspace | DiscoveryRole::Unknown => logical.into(), } } diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 4c3e0eecd..bfecc59d9 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -211,6 +211,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { references: provider_secret_references, auth_authority: None, selected_sources: None, + accepted_global_root: None, selected_resource_roots: None, native_source_flags: None, rebind_enabled: None, diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index a6bf37bdf..ccc16fbc4 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -64,6 +64,7 @@ pub fn prepare( { return Err("prepared source metadata profile exceeded".into()); } + manifest.require_known_discovery_roles().map_err(|_| "prepared native discovery role is unrecorded; retain inputs and reprepare before dispatch".to_string())?; if frozen.iter().any(|(name, _)| { !manifest .frozen_inputs @@ -272,6 +273,7 @@ mod tests { version: 1, cwd: "/original-unavailable/project".into(), original_home: "/original-unavailable/home".into(), + global_config_root: None, sources: vec![SourceView { discovery_role: DiscoveryRole::Workspace, logical_path: "/original-unavailable/project/opencode.json".into(), @@ -365,6 +367,26 @@ mod tests { Some("/original-unavailable/project/opencode.json"), "an explicitly selected native source must be delivered to the actual child environment"); assert!(!generated.contains_key("INSERT"),"independent generated recipes do not forward an original ambient variable into the native child"); + let current_bytes = serde_json::to_vec(&manifest).unwrap(); + let mut legacy: serde_json::Value = serde_json::from_slice(¤t_bytes).unwrap(); + legacy["sources"][0] + .as_object_mut() + .unwrap() + .remove("discoveryRole"); + let bytes = serde_json::to_vec(&legacy).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .err() + .unwrap_or_default(); + assert!( + error.contains("discovery role"), + "a role-less prepared source must block before the execution grant is consumed" + ); + std::fs::write(stage.join(".freshell/input-manifest.json"), ¤t_bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(¤t_bytes)); // Original-form shared configurations have approved native source // identity but deliberately no replacement config bind. manifest.sources[0].original_form = true; From af9f30d64afe8961e345f7848b770220b7b663ba Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 21:58:43 -0700 Subject: [PATCH 065/493] fix: account shared provider source views once --- .../managed_mcp_capability/source_stage.rs | 23 +++-- .../src/managed_provider_references.rs | 31 ++++--- .../managed_provider_references/sources.rs | 88 ++++++++++++++++--- 3 files changed, 103 insertions(+), 39 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 09e4d814c..a6732e178 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -146,7 +146,7 @@ pub(super) fn stage( private_directory(&root.join(".freshell/source-views"))?; private_directory(&root.join(".freshell/templates"))?; let mut files = BTreeMap::new(); - let mut view_bytes = 0usize; + let mut view_budget = crate::managed_provider_references::sources::SourceViewBudget::default(); for (ordinal, source) in plan.iter().enumerate() { if source.source.role == crate::managed_provider_references::sources::DiscoveryRole::Unknown { @@ -161,15 +161,13 @@ pub(super) fn stage( ) .map_err(|error| error.to_string())?; } - view_bytes = view_bytes - .checked_add(source.references.unexpanded_source().len()) - .and_then(|value| value.checked_add(source.references.template.len())) - .ok_or("source view size overflow")?; - if source.references.unexpanded_source().len() > 16 * 1024 * 1024 - || source.references.template.len() > 16 * 1024 * 1024 - || view_bytes > 64 * 1024 * 1024 - || ordinal >= 8192 - { + view_budget + .charge(source.references.unexpanded_source()) + .map_err(str::to_string)?; + view_budget + .charge(&source.references.template) + .map_err(str::to_string)?; + if ordinal >= 8192 { return Err("source view profile exceeded before stage publication".into()); } let raw = format!(".freshell/source-views/source-{ordinal}"); @@ -283,7 +281,8 @@ pub(super) fn describe( for (ordinal, occurrence) in plan.iter().enumerate() { if occurrence.source.lane != SourceLane::Inline && (!occurrence.original_form - || occurrence.references.template != occurrence.references.unexpanded_source()) + || occurrence.references.template.as_str() + != occurrence.references.unexpanded_source()) { let logical = &occurrence.source.logical_path; let destination = freshell_platform::opencode_config::lexical_path( @@ -334,7 +333,7 @@ pub(super) fn describe( }; let relative = format!(".freshell/templates/source-{ordinal}"); let prior = bind_sources.get(&effective); - if prior.is_some_and(|template| *template != occurrence.references.template) { + if prior.is_some_and(|template| *template != occurrence.references.template.as_str()) { return Err("shared effective config target has incompatible templates; retain prior inputs and reprepare".into()); } if prior.is_none() { diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 4f5fb983d..cabe68a6b 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -52,7 +52,7 @@ pub(crate) enum NativeAliasKind { } pub struct ResolvedFileReferencePlan { - pub template: String, + pub template: std::sync::Arc, // Generated recipes already own the raw view. Keep one raw view for // direct references too; a rewritten template cannot revalidate selection. unexpanded_direct_source: Option>, @@ -335,10 +335,19 @@ pub(super) fn resolve_file_references_from_view_relocated( template.push_str(&raw[cursor..]); (template, Vec::new()) }; + let unexpanded_direct_source = (!generated || projection.original_form) + .then(|| source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned()))); + let template = if template == raw { + unexpanded_direct_source + .as_ref() + .cloned() + .unwrap_or_else(|| std::sync::Arc::new(template)) + } else { + std::sync::Arc::new(template) + }; Ok(ResolvedFileReferencePlan { template, - unexpanded_direct_source: (!generated || projection.original_form) - .then(|| source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned()))), + unexpanded_direct_source, selections, referenced_variables: variables.into_inner().into_iter().collect(), path_variables: path_variables.into_iter().collect(), @@ -411,7 +420,7 @@ process.stdout.write(JSON.stringify(receipts)); std::fs::copy(&selection.canonical_source, runtime.join(relative)).unwrap(); } let source = runtime.join("source.json"); - std::fs::write(&source, &plan.template).unwrap(); + std::fs::write(&source, plan.template.as_bytes()).unwrap(); let mut child = std::collections::BTreeMap::new(); for recipe in &plan.recipes { child.insert( @@ -870,11 +879,9 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( if raw.len() > 16 * 1024 * 1024 { return Err("OpenCode inherited inline source exceeds 16 MiB".into()); } - let retained = loaded - .iter() - .try_fold(raw.len(), |bytes, source| { - bytes.checked_add(source.view.len()) - }) + let retained = sources::retained_source_view_bytes(&loaded) + .map_err(str::to_string)? + .checked_add(raw.len()) .ok_or("source-view size overflow")?; let remaining = (64 * 1024 * 1024usize) .checked_sub(retained) @@ -990,10 +997,8 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( } flags }; - let mut retained_views = loaded - .iter() - .try_fold(0usize, |bytes, source| bytes.checked_add(source.view.len())) - .ok_or("source-view byte count overflow")?; + let mut retained_views = + sources::retained_source_view_bytes(&loaded).map_err(str::to_string)?; for source in &mut loaded { if rebind_enabled && source.source.lane == SourceLane::Tui diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 6323d33d0..8e7f65195 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -307,6 +307,43 @@ pub struct PlannedSourceOccurrence { pub original_form: bool, } +#[derive(Default)] +pub(crate) struct SourceViewBudget { + views: std::collections::BTreeSet<(usize, usize)>, + pub bytes: usize, +} +impl SourceViewBudget { + pub fn charge(&mut self, view: &str) -> Result<(), &'static str> { + if view.len() > 16 * 1024 * 1024 { + return Err("source view exceeds 16 MiB"); + } + let identity = (view.as_ptr() as usize, view.len()); + if self.views.contains(&identity) { + return Ok(()); + } + let bytes = self + .bytes + .checked_add(view.len()) + .ok_or("source-view byte count overflow")?; + if bytes > 64 * 1024 * 1024 { + return Err("simultaneous retained source views exceed 64 MiB"); + } + self.views.insert(identity); + self.bytes = bytes; + Ok(()) + } +} + +pub fn retained_source_view_bytes( + sources: &[LoadedSourceOccurrence], +) -> Result { + let mut budget = SourceViewBudget::default(); + for source in sources { + budget.charge(&source.view)?; + } + Ok(budget.bytes) +} + pub struct LoadedSourceOccurrence { pub source: SourceOccurrence, pub view: std::sync::Arc, @@ -349,16 +386,13 @@ pub fn read_opencode_sources( if raw.len() > limit { return Err(fail("selected source exceeds source-view limit")); } + retained = retained + .checked_add(raw.len()) + .ok_or_else(|| fail("source-view byte count overflow"))?; let view = std::sync::Arc::new(raw); canonical.insert(source.canonical_source.clone(), view.clone()); view }; - retained = retained - .checked_add(view.len()) - .ok_or_else(|| fail("source-view byte count overflow"))?; - if retained > 64 * 1024 * 1024 { - return Err(fail("simultaneous retained source views exceed 64 MiB")); - } loaded.push(LoadedSourceOccurrence { source, view }); } Ok(loaded) @@ -451,7 +485,7 @@ pub fn plan_loaded_opencode_sources_relocated( let mut aliases = std::collections::BTreeSet::new(); let mut variables = std::collections::BTreeSet::new(); let mut event_count = 0usize; - let mut retained_view_bytes = 0usize; + let mut view_budget = SourceViewBudget::default(); for (ordinal, loaded) in sources.into_iter().enumerate() { let source = loaded.source; let original_form = source.lane != SourceLane::Inline @@ -539,13 +573,10 @@ pub fn plan_loaded_opencode_sources_relocated( recipe_count = recipe_count .checked_add(references.recipes.len()) .ok_or_else(|| fail("source recipe count overflow"))?; - retained_view_bytes = retained_view_bytes - .checked_add(references.template.len()) - .and_then(|bytes| bytes.checked_add(references.unexpanded_source().len())) - .ok_or_else(|| fail("source-view byte count overflow"))?; - if retained_view_bytes > 64 * 1024 * 1024 { - return Err(fail("simultaneous retained source views exceed 64 MiB")); - } + view_budget.charge(&references.template).map_err(fail)?; + view_budget + .charge(references.unexpanded_source()) + .map_err(fail)?; for occurrence in &references.occurrences { if aliases.len() >= 256 && !aliases.contains(&occurrence.approved_alias) { return Err(fail("configuration alias count exceeds 256 across sources")); @@ -583,6 +614,35 @@ mod tests { fn digest(path: &Path) -> String { format!("{:x}", Sha256::digest(std::fs::read(path).unwrap())) } + #[test] + fn managed_provider_references_shared_large_source_uses_unique_view_budget() { + let owned = tempfile::tempdir().unwrap(); + let shared = owned.path().join("shared.json"); + let raw = format!( + "{{\"agent\":{{\"large\":{{\"description\":\"{}\"}}}}}}", + "x".repeat(9 * 1024 * 1024) + ); + std::fs::write(&shared, &raw).unwrap(); + let sources = (0..8) + .map(|ordinal| SourceOccurrence { + logical_path: owned.path().join(format!("origin-{ordinal}/opencode.json")), + canonical_source: shared.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }) + .collect(); + let plan = plan_opencode_sources(sources, owned.path(), owned.path(), &|_, _, _| None) + .expect( + "eight native occurrences of one 9 MiB source fit the shared 64 MiB view profile", + ); + assert_eq!(plan.len(), 8); + for source in plan { + assert_eq!(source.references.template.as_str(), raw); + assert_eq!(source.references.unexpanded_source(), raw); + assert!(source.original_form && source.references.selections.is_empty()); + } + } + #[test] fn managed_provider_references_shared_source_is_read_once_before_auth_lookup() { let owned = tempfile::tempdir().unwrap(); From f2ebd9e8218841c3114dcf2837018e04a2fef5df Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 22:43:37 -0700 Subject: [PATCH 066/493] fix: preserve shared OpenCode main and TUI source semantics --- .../freshell-platform/src/opencode_plugin.rs | 222 ++++++++++++++++-- .../src/managed_mcp_capability.rs | 201 ++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 21 ++ .../src/managed_provider_launch_tests.rs | 196 +++++++++++++--- .../src/managed_provider_references.rs | 33 ++- .../environment.rs | 3 + .../managed_provider_references/sources.rs | 9 +- .../testing/managed-provider-launch-broker.ts | 2 +- .../managed-provider-launch-sandbox.py | 5 +- 9 files changed, 626 insertions(+), 66 deletions(-) diff --git a/crates/freshell-platform/src/opencode_plugin.rs b/crates/freshell-platform/src/opencode_plugin.rs index e6cf9ba14..c8115e63f 100644 --- a/crates/freshell-platform/src/opencode_plugin.rs +++ b/crates/freshell-platform/src/opencode_plugin.rs @@ -20,6 +20,12 @@ pub fn rebind_plugin_path(home: &Path) -> PathBuf { .join("freshell-rebind-plugin.ts") } +/// Native package entrypoints let a shared Main/TUI source select this +/// directory: the server loader skips it and the TUI loader selects ./tui. +pub fn rebind_plugin_package_path(home: &Path) -> PathBuf { + home.join(".freshell/opencode") +} + /// The freshell-owned TUI config injected per-pane via OPENCODE_TUI_CONFIG. /// Lives under ~/.freshell/opencode/ — deliberately NOT an ancestor of any /// pane cwd (opencode's project tui.json discovery walks up to `/`). @@ -130,9 +136,8 @@ pub fn ensure_rebind_plugin_with_user_config( Ok(tui_json) } -/// Normalize the selected source before reference spans are captured. This -/// function never resolves environment or file tokens and never changes the -/// declaring origin. +/// Append the owned plugin before reference spans are captured while retaining +/// the source's native text and declaring origin. Tokens remain unresolved. pub fn selected_tui_content(raw: &str, plugin: &Path) -> std::io::Result { selected_tui_content_bounded(raw, plugin, 16 * 1024 * 1024) } @@ -142,37 +147,153 @@ pub fn selected_tui_content_bounded( plugin: &Path, remaining: usize, ) -> std::io::Result { - let mut config = crate::opencode_config::parse_jsonc_object(raw).ok_or_else(|| { + if raw.len() > remaining.min(16 * 1024 * 1024) { + return Err(std::io::Error::other( + "OpenCode TUI source-view profile exceeded", + )); + } + let config = crate::opencode_config::parse_jsonc_object(raw).ok_or_else(|| { std::io::Error::new( std::io::ErrorKind::InvalidData, "OpenCode TUI config contains malformed JSONC or is not an object", ) })?; - let plugins = config - .as_object_mut() - .expect("validated object") - .entry("plugin") - .or_insert_with(|| serde_json::json!([])); - let Some(plugins) = plugins.as_array_mut() else { + let plugins = config.get("plugin"); + if plugins.is_some_and(|plugins| !plugins.is_array()) { return Err(std::io::Error::new( std::io::ErrorKind::InvalidData, "OpenCode TUI plugin list is not an array", )); - }; + } let own = plugin_file_spec(plugin); - if !plugins - .iter() - .any(|entry| entry.as_str() == Some(own.as_str())) + if plugins + .and_then(serde_json::Value::as_array) + .is_some_and(|plugins| { + plugins + .iter() + .any(|entry| entry.as_str() == Some(own.as_str())) + }) { - plugins.push(serde_json::Value::String(own)); + return Ok(raw.into()); + } + // Keep the native document's comments, escapes and replacement spans. + // Only append one literal plugin entry to the effective top-level array. + let (offset, separator) = plugin_insertion(raw, plugins.is_some())?; + let own = serde_json::to_string(&own).map_err(std::io::Error::other)?; + let addition = if plugins.is_some() { + format!("{separator}{own}") + } else { + format!("{separator}\"plugin\":[{own}]") + }; + if raw + .len() + .checked_add(addition.len()) + .is_none_or(|length| length > remaining.min(16 * 1024 * 1024)) + { + return Err(std::io::Error::other( + "OpenCode TUI source-view profile exceeded", + )); + } + let mut content = String::with_capacity(raw.len() + addition.len()); + content.push_str(&raw[..offset]); + content.push_str(&addition); + content.push_str(&raw[offset..]); + Ok(content) +} + +/// Scan validated JSONC without allocating tokens or decoding values. The +/// last top-level plugin array is effective when a source repeats that key. +fn plugin_insertion(raw: &str, has_plugins: bool) -> std::io::Result<(usize, &'static str)> { + let bytes = raw.as_bytes(); + let mut cursor = 0; + let mut depth = 0usize; + let mut previous = b' '; + let mut plugin_key = false; + let mut plugin_array = false; + let mut insertion = None; + while cursor < bytes.len() { + let start = cursor; + let token = bytes[cursor]; + cursor += 1; + if token.is_ascii_whitespace() { + continue; + } + if token == b'/' && bytes.get(cursor) == Some(&b'/') { + while cursor < bytes.len() && !matches!(bytes[cursor], b'\n' | b'\r') { + cursor += 1; + } + continue; + } + if token == b'/' && bytes.get(cursor) == Some(&b'*') { + cursor += 1; + while cursor + 1 < bytes.len() && &bytes[cursor..cursor + 2] != b"*/" { + cursor += 1; + } + cursor += 2; + continue; + } + if token == b'"' { + while cursor < bytes.len() { + let byte = bytes[cursor]; + cursor += 1; + if byte == b'\\' { + cursor += 1; + } else if byte == b'"' { + break; + } + } + if depth == 1 && matches!(previous, b'{' | b',') { + plugin_key = serde_json::from_str::(&raw[start..cursor]) + .is_ok_and(|key| key == "plugin"); + } + } else { + match token { + b'[' | b'{' => { + if depth == 1 && previous == b':' { + plugin_array = plugin_key && token == b'['; + } + depth += 1; + } + b']' | b'}' => { + if depth == 2 && plugin_array && token == b']' { + insertion = Some(( + start, + if matches!(previous, b'[' | b',') { + "" + } else { + "," + }, + )); + plugin_array = false; + } + if depth == 1 && !has_plugins { + insertion = Some(( + start, + if matches!(previous, b'{' | b',') { + "" + } else { + "," + }, + )); + } + depth -= 1; + } + _ => {} + } + } + previous = token; } - crate::opencode_config::serialize_source_view(&config, remaining) + insertion.ok_or_else(|| std::io::Error::other("OpenCode TUI plugin array is unavailable")) } /// Install only the implementation when the controller already prepared the /// selected TUI source. Re-parsing that source here would change recipe spans. pub fn ensure_rebind_plugin_source(home: &Path) -> std::io::Result<()> { - write_atomic_if_changed(&rebind_plugin_path(home), REBIND_PLUGIN_SOURCE) + write_atomic_if_changed(&rebind_plugin_path(home), REBIND_PLUGIN_SOURCE)?; + write_atomic_if_changed( + &rebind_plugin_package_path(home).join("package.json"), + r#"{"exports":{"./tui":"./freshell-rebind-plugin.ts"}}"#, + ) } #[cfg(test)] @@ -279,14 +400,77 @@ mod tests { std::fs::write(&selected, raw).unwrap(); let generated = ensure_rebind_plugin_with_user_config(home.path(), Some(&selected)).unwrap(); - let effective: serde_json::Value = - serde_json::from_slice(&std::fs::read(generated).unwrap()).unwrap(); + let effective = crate::opencode_config::parse_jsonc_object( + &std::fs::read_to_string(generated).unwrap(), + ) + .unwrap(); assert_eq!(effective["theme"], "dark"); assert_eq!(effective["plugin"].as_array().unwrap().len(), 2); assert_eq!(effective["plugin"][0], "file:///user.ts"); assert_eq!(std::fs::read_to_string(selected).unwrap(), raw); } + #[test] + fn shared_tui_edit_preserves_native_file_and_auth_lexemes() { + let raw = "{\n // native comment {file:./missing}\n\"agent\":{\"probe\":{\"prompt\":\"{env:AUTH}\",\"description\":\"\\u007bfile:./literal\\u007d\"}},\"plugin\":[[\"./probe.mjs\",{\"key\":\"{file:./rule}\"}], /* tail */], }"; + let content = selected_tui_content(raw, Path::new("/owned/rebind.ts")).unwrap(); + assert!( + content.contains("// native comment {file:./missing}"), + "normalizing a shared source changes the native comment-prefix file exclusion" + ); + assert!(content.contains("\\u007bfile:./literal\\u007d"), + "native replacement happens before JSONC decoding, so raw escape spellings must survive"); + assert!(content.contains("\"prompt\":\"{env:AUTH}\"")); + let mut selected = Vec::new(); + crate::opencode_config::transform_file_tokens( + &content, + &|name| (name == "AUTH").then(|| "{file:./auth-rule}".into()), + |token| { + selected.push(token.path.to_string()); + Ok::<_, ()>(format!("{{file:{}}}", token.path)) + }, + ) + .unwrap(); + assert_eq!( + selected, + ["./auth-rule", "./rule"], + "native replacement must not select comment-prefixed or escaped file-looking strings" + ); + let config = crate::opencode_config::parse_jsonc_object(&content).unwrap(); + assert_eq!(config["plugin"].as_array().unwrap().len(), 2); + assert_eq!(config["plugin"][0][0], "./probe.mjs"); + } + + #[test] + fn tui_plugin_insertion_handles_native_arrays_comments_and_effective_duplicate_keys() { + for raw in [ + "{}", + "{\"theme\":\"dark\", /* tail */}", + "{\"plugin\":[]}", + "{\"plugin\":[\"file:///user.ts\", /* tail */]}", + "{\"plugin\":[[\"file:///user.ts\",{\"nested\":[1,2],\"text\":\"]}\\\"\"}]]}", + "{\"plugin\":[\"discarded\"],\"pl\\u0075gin\":[],\"nested\":{\"plugin\":[\"unrelated\"]}}", + ] { + let before = crate::opencode_config::parse_jsonc_object(raw).unwrap(); + let plugin = Path::new("/owned/rebind"); + let content = selected_tui_content(raw, plugin).unwrap(); + let after = crate::opencode_config::parse_jsonc_object(&content).unwrap(); + let before_plugins = before.get("plugin").and_then(serde_json::Value::as_array); + let after_plugins = after["plugin"].as_array().unwrap(); + assert_eq!(after_plugins.len(), before_plugins.map_or(0, Vec::len) + 1); + if let Some(before_plugins) = before_plugins { + assert_eq!(&after_plugins[..before_plugins.len()], before_plugins); + } + assert_eq!(after_plugins.last().unwrap(), "file:///owned/rebind"); + for (key, value) in before.as_object().unwrap() { + if key != "plugin" { assert_eq!(&after[key], value); } + } + assert_eq!(selected_tui_content(&content, plugin).unwrap(), content); + assert!(selected_tui_content_bounded(raw, plugin, content.len() - 1).is_err()); + assert_eq!(selected_tui_content_bounded(raw, plugin, content.len()).unwrap(), content); + } + } + #[test] fn malformed_selected_tui_config_is_refused_without_editing_source() { let home = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index bdfc877a5..a667ef714 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1607,6 +1607,7 @@ impl CapabilityStore { .workspace .or(opencode_input.cwd) .unwrap_or(home), + opencode_input.secret_references.unwrap_or(&[]), ) .and_then(|inputs| { if let Some(context) = opencode_input.launch_context { @@ -4228,6 +4229,206 @@ except Exception as error: } } + #[test] + fn managed_provider_shared_runtime_ca_reuses_the_selected_typed_file() { + use crate::managed_provider_references::sources::*; + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let ca = home.join("synthetic-ca.pem"); + std::fs::write(&ca, "PRIVATE_SYNTHETIC_CA_PAYLOAD").unwrap(); + let runtime_ca = PathBuf::from("/home/freshell/provider/.config/onecli/gateway-ca.pem"); + let shared = workspace.join("shared.json"); + let raw = r#"{"agent":{"probe":{"prompt":"{file:{env:NODE_EXTRA_CA_CERTS}}"}}}"#; + std::fs::write(&shared, raw).unwrap(); + let sources = [SourceLane::Main, SourceLane::Tui] + .into_iter() + .map(|lane| LoadedSourceOccurrence { + requires_config_bind: false, + source: SourceOccurrence { + logical_path: workspace.join(if lane == SourceLane::Main { + "opencode.json" + } else { + "tui.json" + }), + canonical_source: shared.clone(), + lane, + role: DiscoveryRole::Workspace, + }, + view: std::sync::Arc::new(raw.into()), + }) + .collect(); + let plan = plan_loaded_opencode_sources_relocated( + sources, + &workspace, + &home, + &|_, _, name| { + (name == "NODE_EXTRA_CA_CERTS").then(|| runtime_ca.to_str().unwrap().into()) + }, + &std::collections::BTreeMap::from([(runtime_ca.clone(), ca.clone())]), + ) + .unwrap(); + let references = [ + ProviderSecretReference { + source_path: ca.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeOnecliCaFile, + provider_relative_path: None, + }, + ProviderSecretReference { + source_path: ca.to_str().unwrap().into(), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(".freshell/references/0".into()), + }, + ]; + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let capability = CapabilityStore::default() + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + secret_references: Some(&references), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + assert!(inputs.indexed_files.as_ref().unwrap().aliases.is_empty(), + "the host's typed CA projection already owns this exact file; a readonly indexed alias must not cover its writable installation path"); + assert_eq!(inputs.manifest.source_count, 1); + assert_eq!(std::fs::read_to_string(&shared).unwrap(), raw); + assert_eq!( + std::fs::read_to_string(&ca).unwrap(), + "PRIVATE_SYNTHETIC_CA_PAYLOAD" + ); + let different_ca = home.join("changed-ca.pem"); + std::fs::write(&different_ca, "PRIVATE_CHANGED_CA_PAYLOAD").unwrap(); + let mut changed = references.clone(); + changed[0].source_path = different_ca.to_str().unwrap().into(); + let error = CapabilityStore::default() + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + secret_references: Some(&changed), + ..Default::default() + }, + None, + ) + .unwrap_err(); + assert!(error.contains("runtime CA alias conflicts")); + assert!(!error.contains("PRIVATE_CHANGED_CA_PAYLOAD")); + assert!( + stage.join(".freshell/prepared-inputs.json").is_file(), + "a changed typed source cannot retire the prior usable projection" + ); + } + + #[tokio::test] + async fn managed_provider_selected_tui_can_share_a_main_source_without_flattening_it() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let shared = workspace.join("shared.jsonc"); + let raw = "{\n // comment {file:./unselected-missing}\n\"agent\":{\"probe\":{\"prompt\":\"{file:./rules}\",\"description\":\"\\u007bfile:./literal\\u007d\"}},\"plugin\":[],}"; + std::fs::write(&shared, raw).unwrap(); + std::fs::write(workspace.join("rules"), "PRIVATE_SHARED_LANE_RULE").unwrap(); + std::os::unix::fs::symlink("shared.jsonc", workspace.join("opencode.jsonc")).unwrap(); + std::os::unix::fs::symlink("shared.jsonc", workspace.join("selected-tui.jsonc")).unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( + crate::managed_provider_references::OpencodePreparation { + client: &client, + soul: &soul, + creation_seed: "existing-fixture", + home: &home, + cwd: &workspace, + workspace: &workspace, + original: &|name| { + (name == "OPENCODE_TUI_CONFIG").then(|| "selected-tui.jsonc".into()) + }, + references: vec![], + auth_authority: Some("existing-owned-attempt"), + selected_sources: None, + accepted_global_root: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: None, + }, + |soul, home, context, input| { + let capability = store.issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(context), + ..input + }, + None, + )?; + read_prepared_context(&directory, &capability) + }, + ) + .await + .expect("one shared Main/TUI source retains a uniform original-form projection"); + let inputs = prepared.context.prepared_inputs.as_ref().unwrap(); + assert_eq!(inputs.config_binds.len(), 1); + assert_eq!( + Path::new(&inputs.config_binds[0].effective_destination), + shared + ); + let stage = staged_provider_root_path(&directory, &inputs.generation_id).unwrap(); + let template = + std::fs::read_to_string(stage.join(&inputs.config_binds[0].source_relative_path)) + .unwrap(); + assert!(template.contains("// comment {file:./unselected-missing}")); + assert!(template.contains("\\u007bfile:./literal\\u007d")); + assert!(template.contains("{file:./rules}")); + let config = freshell_platform::opencode_config::parse_jsonc_object(&template).unwrap(); + assert_eq!( + config["plugin"], + serde_json::json!(["file:///home/freshell/provider/.freshell/opencode"]) + ); + assert_eq!(inputs.indexed_files.as_ref().unwrap().aliases.len(), 1); + assert_eq!(std::fs::read_to_string(shared).unwrap(), raw); + assert!( + std::fs::symlink_metadata(workspace.join("selected-tui.jsonc")) + .unwrap() + .is_symlink() + ); + } + #[test] fn managed_provider_shared_source_rewrites_only_home_and_absolute_file_tokens() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index a6732e178..e8df5216f 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -253,6 +253,7 @@ pub(super) fn describe( initial_auth_reservation: Option<&str>, config_root: Option<&Path>, workspace: &Path, + secret_references: &[freshell_runtime_protocol::ProviderSecretReference], ) -> Result { use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; @@ -281,6 +282,7 @@ pub(super) fn describe( for (ordinal, occurrence) in plan.iter().enumerate() { if occurrence.source.lane != SourceLane::Inline && (!occurrence.original_form + || occurrence.requires_config_bind || occurrence.references.template.as_str() != occurrence.references.unexpanded_source()) { @@ -406,6 +408,25 @@ pub(super) fn describe( ) } }; + // The host installs the selected typed file at this path. + // Covering it with a readonly alias would prevent its atomic + // refresh, even when both projections selected the same bytes. + if let Some(reference) = secret_references.iter().find(|reference| { + reference.profile == ProviderSecretProfile::OpencodeOnecliCaFile + && reference + .profile + .provider_file_relative_path() + .is_some_and(|relative| runtime_home.join(relative) == logical) + }) { + let typed_source = + std::fs::canonicalize(&reference.source_path).map_err(|_| { + "selected typed CA source is unavailable before publication" + })?; + if typed_source != **selected { + return Err("selected runtime CA alias conflicts with its typed source; retain prior inputs and reprepare".into()); + } + continue; + } let effective = if let Some(unit) = resource_mounts.iter().find(|unit| { unit.kind == ProviderResourceKind::Directory && logical.starts_with(&unit.logical_destination) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 92e83ae4a..60c9f2ca7 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -16,12 +16,48 @@ use std::{ time::Duration, }; +fn assert_private_sentinels_absent(root: &Path, certificate_sentinel: &[u8]) { + for entry in std::fs::read_dir(root).unwrap() { + let entry = entry.unwrap(); + let metadata = entry.file_type().unwrap(); + if metadata.is_dir() { + assert_private_sentinels_absent(&entry.path(), certificate_sentinel); + } else if metadata.is_file() { + let bytes = std::fs::read(entry.path()).unwrap(); + for (ordinal, sentinel) in [ + certificate_sentinel, + b"AUTH_PREFIX".as_slice(), + b"native-rule".as_slice(), + b"native-explicit-selected-rule".as_slice(), + b"native-inline-selected-rule".as_slice(), + b"native-shared-root-rule".as_slice(), + b"native-shared-nested-rule".as_slice(), + b"shared-home-private-rule".as_slice(), + b"shared-home-sensitive-file".as_slice(), + b"INLINE_PREFIX".as_slice(), + b"AUTH_INLINE_SHADOW_NO_FILE".as_slice(), + b"synthetic-onecli-shadowed".as_slice(), + ] + .into_iter() + .enumerate() + { + assert!( + !bytes.windows(sentinel.len()).any(|value| value == sentinel), + "private input sentinel {ordinal} leaked to owned surface {}", + entry.path().display() + ); + } + } + } +} + /// Delegates every lifecycle operation to the actual Docker backend. The only /// added operation delivers metadata to the closed fixture's trusted broker. struct BrokerAuthorizedBackend { real: DockerEngineBackend, control: PathBuf, handoff: PathBuf, + certificate_sentinel: Vec, } impl BrokerAuthorizedBackend { async fn authorize(&self, spec: &CreateRuntimeSpec) -> Result<(), BackendError> { @@ -50,6 +86,13 @@ impl BrokerAuthorizedBackend { inputs .validate_references(references) .map_err(|_| invalid())?; + assert_private_sentinels_absent( + &self + .control + .join("provider-roots") + .join(&inputs.generation_id), + &self.certificate_sentinel, + ); let indexed = references .iter() .enumerate() @@ -203,10 +246,19 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr InstallationId::parse(std::env::var("FRESHELL_PROVIDER_LAUNCH_INSTALLATION").unwrap()) .unwrap(); let registry = Registry::open(fixture.join("registry"), Some(installation)).unwrap(); + let certificate = std::fs::read_to_string(fixture.join("grants/ca.pem")).unwrap(); + let certificate_sentinel = certificate + .lines() + .find(|line| !line.starts_with("-----")) + .unwrap() + .as_bytes() + .to_vec(); + assert!(certificate_sentinel.len() >= 32); let backend = Arc::new(BrokerAuthorizedBackend { real: DockerEngineBackend::new(fixture.join("broker.sock")), control: control.clone(), handoff: fixture.join("handoff"), + certificate_sentinel: certificate_sentinel.clone(), }); let socket = control.join("control.sock"); let secret_file = control.join("secret"); @@ -296,13 +348,19 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr let shared_home_rule = fixture.join("workspace/home rule with spaces.txt"); std::fs::write(&shared_home_rule, "shared-home-sensitive-file\n").unwrap(); std::fs::set_permissions(&shared_home_rule, std::fs::Permissions::from_mode(0o600)).unwrap(); - let shared_bytes = serde_json::to_vec(&json!({"agent":{ + let shared_lane_receipt = fixture.join("workspace/shared-lane-consumed"); + let shared_json = serde_json::to_string(&json!({"agent":{ "probe":{"prompt":"{env:OPENAI_API_KEY}"}, "sharedRelative":{"prompt":"{file:./rules.txt}"}, "sharedAbsolute":{"prompt":format!("{{file:{}}}", shared_home_rule.display())}, "sharedHome":{"prompt":"{file:~/home rule with spaces.txt}"} - }, "plugin":[["./shared-probe.mjs", {"receipt":shared_receipt}]]})) + }, "plugin":[["./shared-probe", {"receipt":shared_receipt,"laneReceipt":shared_lane_receipt,"auth":"{env:OPENAI_API_KEY}","ca":"{file:{env:NODE_EXTRA_CA_CERTS}}"}]]})) .unwrap(); + let shared_bytes = format!( + "{{\n // native comment {{file:./unselected-missing}}\n{}", + &shared_json[1..] + ) + .into_bytes(); for (directory, label) in [ (fixture.join("workspace"), "root"), (nested.clone(), "nested"), @@ -311,12 +369,29 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr r#"import fs from 'node:fs';import crypto from 'node:crypto'; export default {{id:'closed-shared-{label}',async server(input,options){{return {{async config(config){{ let receipt=fs.existsSync(options.receipt)?JSON.parse(fs.readFileSync(options.receipt,'utf8')):{{}}; - receipt.{label}={{relative:crypto.createHash('sha256').update(fs.readFileSync(new URL('./rules.txt',import.meta.url))).digest('hex'),absolute:crypto.createHash('sha256').update(config.agent.sharedAbsolute.prompt).digest('hex'),home:crypto.createHash('sha256').update(config.agent.sharedHome.prompt).digest('hex')}}; + receipt.{label}={{relative:crypto.createHash('sha256').update(fs.readFileSync(new URL('../rules.txt',import.meta.url))).digest('hex'),absolute:crypto.createHash('sha256').update(config.agent.sharedAbsolute.prompt).digest('hex'),home:crypto.createHash('sha256').update(config.agent.sharedHome.prompt).digest('hex')}}; fs.writeFileSync(options.receipt,JSON.stringify(receipt)); + fs.writeFileSync(options.laneReceipt+'.{label}.main',JSON.stringify({{auth:crypto.createHash('sha256').update(options.auth).digest('hex'),ca:crypto.createHash('sha256').update(options.ca).digest('hex')}})); }}}}}}}}; "# ); - std::fs::write(directory.join("shared-probe.mjs"), plugin).unwrap(); + let tui_plugin = format!( + r#"import fs from 'node:fs';import crypto from 'node:crypto'; +export default {{id:'closed-shared-{label}',async tui(api,options){{ + fs.writeFileSync(options.laneReceipt+'.{label}.tui',JSON.stringify({{auth:crypto.createHash('sha256').update(options.auth).digest('hex'),ca:crypto.createHash('sha256').update(options.ca).digest('hex'),theme:crypto.createHash('sha256').update(api.tuiConfig.theme).digest('hex')}})); + setTimeout(()=>{{const dir='/home/freshell/provider/.freshell/session-signals/opencode';const alive=fs.existsSync(dir)&&fs.readdirSync(dir).some(name=>name.startsWith('closed-native-terminal__')&&JSON.parse(fs.readFileSync(dir+'/'+name,'utf8')).hello===true);fs.writeFileSync(options.laneReceipt+'.{label}.rebind',JSON.stringify({{alive}}));}},1500); +}}}}; +"# + ); + let package = directory.join("shared-probe"); + std::fs::create_dir_all(&package).unwrap(); + std::fs::write(package.join("server.mjs"), plugin).unwrap(); + std::fs::write(package.join("tui.mjs"), tui_plugin).unwrap(); + std::fs::write( + package.join("package.json"), + r#"{"exports":{"./server":"./server.mjs","./tui":"./tui.mjs"}}"#, + ) + .unwrap(); } std::fs::write(&shared_source, &shared_bytes).unwrap(); std::os::unix::fs::symlink("shared.json", fixture.join("workspace/opencode.json")).unwrap(); @@ -351,9 +426,11 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ }}; "#).unwrap(); let selected_tui = fixture.join("workspace/selected-tui.jsonc"); + std::os::unix::fs::symlink("shared.json", &selected_tui).unwrap(); + let project_tui = fixture.join("workspace/tui.jsonc"); let tui_receipt = fixture.join("workspace/native-tui-consumed.json"); let selected_tui_bytes = format!("{{ // selected origin\n\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&tui_receipt).unwrap()); - std::fs::write(&selected_tui, &selected_tui_bytes).unwrap(); + std::fs::write(&project_tui, &selected_tui_bytes).unwrap(); std::fs::write(fixture.join("workspace/selected-theme"), " dark\n").unwrap(); let inline_source = r#"{ // inline instance origin "agent":{"probe":{"description":"{env:INLINE_VALUE}"}}}"#; @@ -492,6 +569,10 @@ export default {{id:'closed-directory-discovery',async server(){{ rows: 30, }, env: BTreeMap::from([ + ( + "FRESHELL_TERMINAL_ID".into(), + "closed-native-terminal".into(), + ), ("OPENCODE_CONFIG_DIR".into(), "../selected-directory".into()), ("OPENCODE_CONFIG".into(), "../selected-config.json".into()), ("OPENCODE_TUI_CONFIG".into(), "../selected-tui.jsonc".into()), @@ -603,8 +684,45 @@ export default {{id:'closed-directory-discovery',async server(){{ std::fs::read(resource_root.join("opencode.json")).unwrap(), root_config ); + let shared_consumption = read_receipt(&shared_receipt).await; + if shared_consumption.is_err() { + let output = client + .read_output( + SoulId::parse(terminal.soul_id.clone()).unwrap(), + 0, + 1024 * 1024, + ) + .await + .unwrap(); + let text = output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(); + let classes = [ + "shared-probe", + "SyntaxError", + "TypeError", + "ConfigInvalidError", + "ConfigJsonError", + "ENOENT", + "EACCES", + "failed to load plugin", + "Plugin export", + "OpenSSL", + "certificate", + "Failed to import", + ] + .into_iter() + .filter(|class| text.contains(class)) + .collect::>(); + println!( + "provider_launch_diagnostic {}", + json!({"route":"shared-main","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes}) + ); + } assert_eq!( - read_receipt(&shared_receipt).await.unwrap(), + shared_consumption.unwrap(), json!({"root":{"relative":format!("{:x}",Sha256::digest(b"native-shared-root-rule")),"absolute":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file")),"home":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file"))},"nested":{"relative":format!("{:x}",Sha256::digest(b"native-shared-nested-rule")),"absolute":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file")),"home":format!("{:x}",Sha256::digest(b"shared-home-sensitive-file"))}}) ); assert_eq!( @@ -613,8 +731,31 @@ export default {{id:'closed-directory-discovery',async server(){{ .expect("actual TUI plugin must consume the selected effective TUI configuration"), json!({"theme":format!("{:x}",Sha256::digest(b"dark"))}) ); + let ca_digest = format!( + "{:x}", + Sha256::digest( + std::fs::read_to_string(fixture.join("grants/ca.pem")) + .unwrap() + .trim() + .as_bytes() + ) + ); assert_eq!( - std::fs::read_to_string(&selected_tui).unwrap(), + read_receipt(&shared_lane_receipt.with_extension("root.main")) + .await + .unwrap(), + json!({"auth":format!("{:x}",Sha256::digest(b"AUTH_PREFIXshared-home-sensitive-fileAUTH_SUFFIX")),"ca":ca_digest}) + ); + assert_eq!( + read_receipt(&shared_lane_receipt.with_extension("root.tui")) + .await + .unwrap(), + json!({"auth":format!("{:x}",Sha256::digest(b"synthetic-onecli-shadowed")),"ca":ca_digest,"theme":format!("{:x}",Sha256::digest(b"dark"))}) + ); + assert_eq!(read_receipt(&shared_lane_receipt.with_extension("root.rebind")).await.unwrap(), json!({"alive":true}), + "the actual native TUI loader must execute the owned package entrypoint, not merely retain its config spec"); + assert_eq!( + std::fs::read_to_string(&project_tui).unwrap(), selected_tui_bytes ); println!( @@ -647,6 +788,7 @@ export default {{id:'closed-directory-discovery',async server(){{ std::fs::remove_file(&root_receipt).unwrap(); std::fs::remove_file(&home_root_receipt).unwrap(); std::fs::remove_file(&directory_receipt).unwrap(); + std::fs::remove_file(shared_lane_receipt.with_extension("root.main")).unwrap(); // Fresh launches use current server controls; the terminal exercised a // request-only relative control. Both must activate the explicit layer. // The fresh route has a workspace beneath original HOME. Both role @@ -705,6 +847,12 @@ export default {{id:'closed-directory-discovery',async server(){{ .dispatch(HostedFreshAgentCommand::Send(send)) .await; assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); + assert_eq!( + read_receipt(&shared_lane_receipt.with_extension("root.main")) + .await + .unwrap(), + json!({"auth":format!("{:x}",Sha256::digest(b"AUTH_PREFIXshared-home-sensitive-fileAUTH_SUFFIX")),"ca":ca_digest}) + ); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); assert_eq!( read_receipt(&home_root_receipt).await.unwrap(), @@ -758,38 +906,8 @@ export default {{id:'closed-directory-discovery',async server(){{ ); // Scan only run-owned durable/staged surfaces; referenced payloads belong // in private indexed files, not ordinary resource/configuration trees. - fn assert_private_sentinels_absent(root: &Path) { - for entry in std::fs::read_dir(root).unwrap() { - let entry = entry.unwrap(); - let metadata = entry.file_type().unwrap(); - if metadata.is_dir() { - assert_private_sentinels_absent(&entry.path()); - } else if metadata.is_file() { - let bytes = std::fs::read(entry.path()).unwrap(); - for sentinel in [ - b"AUTH_PREFIX".as_slice(), - b"native-rule".as_slice(), - b"native-explicit-selected-rule".as_slice(), - b"native-inline-selected-rule".as_slice(), - b"native-shared-root-rule".as_slice(), - b"native-shared-nested-rule".as_slice(), - b"shared-home-private-rule".as_slice(), - b"shared-home-sensitive-file".as_slice(), - b"INLINE_PREFIX".as_slice(), - b"AUTH_INLINE_SHADOW_NO_FILE".as_slice(), - b"synthetic-onecli-shadowed".as_slice(), - ] { - assert!( - !bytes.windows(sentinel.len()).any(|value| value == sentinel), - "private input leaked to an owned durable/staged surface" - ); - } - } - } - } - assert_private_sentinels_absent(&fixture.join("registry")); - assert_private_sentinels_absent(&control.join("provider-roots")); - assert_private_sentinels_absent(&fixture.join("handoff")); + assert_private_sentinels_absent(&fixture.join("registry"), &certificate_sentinel); + assert_private_sentinels_absent(&fixture.join("handoff"), &certificate_sentinel); for view in client.inventory().await.unwrap() { if view.desired_state == DesiredState::Running { client.stop(view.soul_id).await.unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index cabe68a6b..09c7f73ed 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -910,6 +910,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( ) .map_err(|error| error.to_string())?; let inline = LoadedSourceOccurrence { + requires_config_bind: true, source: SourceOccurrence { logical_path: "OPENCODE_CONFIG_CONTENT".into(), canonical_source: "OPENCODE_CONFIG_CONTENT".into(), @@ -999,7 +1000,9 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( }; let mut retained_views = sources::retained_source_view_bytes(&loaded).map_err(str::to_string)?; - for source in &mut loaded { + let mut edited = std::collections::BTreeSet::new(); + for ordinal in 0..loaded.len() { + let source = &loaded[ordinal]; if rebind_enabled && source.source.lane == SourceLane::Tui && native_source_flags.iter().any(|flag| { @@ -1013,14 +1016,28 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( ) }) { + let canonical = source.source.canonical_source.clone(); + if !edited.insert(canonical.clone()) { + continue; + } + let shared = loaded + .iter() + .filter(|source| source.source.canonical_source == canonical) + .count() + > 1; let remaining = (64 * 1024 * 1024usize) .checked_sub(retained_views) .ok_or("simultaneous source views exceed 64 MiB")?; + let runtime_home = Path::new("/home/freshell/provider"); + let plugin = if shared { + freshell_platform::opencode_plugin::rebind_plugin_package_path(runtime_home) + } else { + freshell_platform::opencode_plugin::rebind_plugin_path(runtime_home) + }; let content = freshell_platform::opencode_plugin::selected_tui_content_bounded( &source.view, - &freshell_platform::opencode_plugin::rebind_plugin_path(Path::new( - "/home/freshell/provider", - )), remaining, + &plugin, + remaining, ) .map_err(|_| { format!( @@ -1034,7 +1051,13 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( retained_views = retained_views .checked_add(content.len()) .ok_or("source-view byte count overflow")?; - source.view = std::sync::Arc::new(content); + let view = std::sync::Arc::new(content); + for (index, source) in loaded.iter_mut().enumerate() { + if index == ordinal || shared && source.source.canonical_source == canonical { + source.view = view.clone(); + source.requires_config_bind = true; + } + } } } let names = source_environment_names(&loaded).map_err(|error| error.to_string())?; diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index edf62daa5..7de17f00e 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -258,6 +258,7 @@ mod tests { .unwrap(); let loaded = || { vec![LoadedSourceOccurrence { + requires_config_bind: false, source: SourceOccurrence { logical_path: config.clone(), canonical_source: config.clone(), @@ -325,6 +326,7 @@ mod tests { let plan = environment .plan_sources( vec![LoadedSourceOccurrence { + requires_config_bind: false, source: SourceOccurrence { logical_path: config.clone(), canonical_source: config, @@ -383,6 +385,7 @@ mod tests { let plan = environment .plan_sources( vec![LoadedSourceOccurrence { + requires_config_bind: false, source: SourceOccurrence { logical_path: config.clone(), canonical_source: config, diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 8e7f65195..48cf575df 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -305,6 +305,7 @@ pub struct PlannedSourceOccurrence { pub source: SourceOccurrence, pub references: super::ResolvedFileReferencePlan, pub original_form: bool, + pub requires_config_bind: bool, } #[derive(Default)] @@ -347,6 +348,7 @@ pub fn retained_source_view_bytes( pub struct LoadedSourceOccurrence { pub source: SourceOccurrence, pub view: std::sync::Arc, + pub requires_config_bind: bool, } /// Retain one bounded canonical read while keeping every native logical origin. @@ -393,7 +395,11 @@ pub fn read_opencode_sources( canonical.insert(source.canonical_source.clone(), view.clone()); view }; - loaded.push(LoadedSourceOccurrence { source, view }); + loaded.push(LoadedSourceOccurrence { + source, + view, + requires_config_bind: false, + }); } Ok(loaded) } @@ -602,6 +608,7 @@ pub fn plan_loaded_opencode_sources_relocated( source, references, original_form, + requires_config_bind: loaded.requires_config_bind, }); } Ok(result) diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 8b21fb0a2..04bca884e 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -20,7 +20,7 @@ const broker=new RestrictedDockerBroker({ allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), allowedProviderUserRoots:new Set(['/home/freshell/provider',path.join(root,'.opencode')]), - allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json')]), + allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json'),path.join(root,'grants/ca.pem')]), allowedHostGatewayAddresses:gateways, testRunId:configuration.runId,logPath:path.join(root,'broker.jsonl'), }) diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index e570819e8..554c2e1aa 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -125,10 +125,12 @@ def create(name,*arguments): (config/f"{ordinal}.txt").write_text(f"native-{ordinal}-rule\n") agents[ordinal]={"prompt":"{file:./"+ordinal+".txt}"} (config/"opencode.json").write_text(json.dumps({"agent":agents,"plugin":[["./plugins/probe-server/index.mjs",{"receipt":str(root/"workspace/native-consumed.json")}]]})) - (root/"grants/environment.env").write_text("OPENAI_API_KEY=synthetic-onecli-shadowed\n") + (root/"grants/environment.env").write_text("OPENAI_API_KEY=synthetic-onecli-shadowed\nNODE_EXTRA_CA_CERTS=/home/freshell/provider/.config/onecli/gateway-ca.pem\n") + (root/"grants/ca.pem").write_bytes((repo/"test/e2e-browser/fixtures/tls/ca.cert.pem").read_bytes()) (root/"grants/auth.json").write_text(json.dumps({"fixture":{"type":"wellknown","key":"OPENAI_API_KEY","token":"AUTH_PREFIX{file:./rules.txt}AUTH_SUFFIX"}})) # These newly created synthetic grants alone receive private mode. (root/"grants/environment.env").chmod(0o600);(root/"grants/auth.json").chmod(0o600) + (root/"grants/ca.pem").chmod(0o600) broker_name=run+"-broker" create(broker_name,"--network","none","--user","0:0","--cap-drop","ALL", "--mount",f"type=bind,src={root},dst={root}", @@ -154,6 +156,7 @@ def create(name,*arguments): "--env",f"FRESHELL_PROVIDER_LAUNCH_INSTALLATION={installation}","--env",f"FRESHELL_PROVIDER_LAUNCH_RUN_ID={run}", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_ENV_FILE={root}/grants/environment.env", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE={root}/grants/auth.json", + "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE={root}/grants/ca.pem", "--entrypoint","/runtime/startup-tests",IMAGE,"--exact",TEST,"--nocapture","--test-threads","1") docker("start",controller) controller_metadata=json.loads(docker("container","inspect",controller).stdout)[0] From 7a68e5972043889d01fe68c572af13e7eb4fbb2c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 23:29:29 -0700 Subject: [PATCH 067/493] fix: stage native-selected Codex inputs in prepared generations --- crates/freshell-platform/src/codex_inputs.rs | 56 ++ crates/freshell-platform/src/lib.rs | 1 + crates/freshell-runtime-protocol/src/lib.rs | 24 + .../freshell-server/src/fresh_agent_proxy.rs | 51 +- .../src/fresh_agent_proxy_tests.rs | 30 + .../src/managed_mcp_capability.rs | 533 +++++++++++++++++- .../src/managed_mcp_capability/codex_stage.rs | 176 ++++++ .../src/managed_provider_bootstrap.rs | 2 +- .../src/managed_provider_references/codex.rs | 335 ++++++++++- crates/freshell-server/src/managed_runtime.rs | 24 + crates/freshell-session-host/src/main.rs | 3 +- .../src/provider_input_preparation.rs | 100 +++- .../src/providers/mod.rs | 45 +- .../managed-provider-source-sandbox.py | 16 +- 14 files changed, 1338 insertions(+), 58 deletions(-) create mode 100644 crates/freshell-platform/src/codex_inputs.rs create mode 100644 crates/freshell-server/src/managed_mcp_capability/codex_stage.rs diff --git a/crates/freshell-platform/src/codex_inputs.rs b/crates/freshell-platform/src/codex_inputs.rs new file mode 100644 index 000000000..e5bea88ed --- /dev/null +++ b/crates/freshell-platform/src/codex_inputs.rs @@ -0,0 +1,56 @@ +//! Accepted native Codex discovery context. Source contents stay in private +//! stage files; only the finite native source controls travel in metadata. +use serde::{Deserialize, Serialize}; +use std::path::{Path, PathBuf}; + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct CodexInputManifest { + pub version: u8, + pub original_home: PathBuf, + pub cwd: PathBuf, + pub codex_home: PathBuf, + pub named_profile: Option, +} +impl CodexInputManifest { + pub fn validate(&self) -> Result<(), &'static str> { + if self.version != 1 + || [&self.original_home, &self.cwd, &self.codex_home] + .iter() + .any(|path| { + !path.is_absolute() + || path.as_os_str().len() > 64 * 1024 + || path.to_string_lossy().chars().any(char::is_control) + }) + || self + .named_profile + .as_ref() + .is_some_and(|name| !valid_profile(name)) + { + return Err("accepted Codex discovery context is invalid"); + } + Ok(()) + } + pub fn runtime_path(&self, logical: &Path) -> PathBuf { + logical + .strip_prefix(&self.codex_home) + .map(|relative| self.runtime_home().join(relative)) + .unwrap_or_else(|_| runtime_path(&self.original_home, logical)) + } + pub fn runtime_home(&self) -> PathBuf { + PathBuf::from("/home/freshell/provider/.codex") + } +} +pub fn valid_profile(name: &str) -> bool { + !name.is_empty() + && name.len() <= 128 + && name + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'-')) +} +pub fn runtime_path(original_home: &Path, logical: &Path) -> PathBuf { + logical + .strip_prefix(original_home) + .map(|relative| Path::new("/home/freshell/provider").join(relative)) + .unwrap_or_else(|_| logical.to_path_buf()) +} diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index e33f2cbe3..4ee8a457d 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -53,6 +53,7 @@ pub mod cli_launch; pub mod clock; +pub mod codex_inputs; pub mod detect; pub mod git_meta; pub mod host_stats_readers; diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 235bb7d59..b0cc7313f 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -1419,6 +1419,29 @@ mod provider_launch_context_tests { ); } + #[test] + fn managed_codex_named_profile_control_reaches_durable_launch_arguments() { + for flag in ["--profile", "-p"] { + let original = vec![ + flag.into(), + "selected_profile-1".into(), + "--model".into(), + "fixture-model".into(), + ]; + assert_eq!( + durable_managed_terminal_args("codex", original.clone(), None) + .expect("native named profile control must remain usable"), + original + ); + assert!(durable_managed_terminal_args( + "codex", + vec![flag.into(), "../config".into()], + None + ) + .is_err()); + } + } + #[test] fn managed_codex_accepts_only_the_packaged_mcp_recipe_after_preparation() { let args = vec![ @@ -2146,6 +2169,7 @@ pub fn validate_managed_terminal_args(mode: &str, args: &[String]) -> Result<(), .and_then(|part| part.strip_suffix('"')) .is_some_and(safe_selector) } + ("codex", "--profile" | "-p") => freshell_platform::codex_inputs::valid_profile(value), ("codex", "--model" | "--sandbox") => safe_selector(value), ("codex", "--remote") => value .strip_prefix("ws://127.0.0.1:") diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index b32b25018..ef80056a4 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -40,6 +40,19 @@ const FIXTURE_MODES_ENV: &str = "FRESHELL_MANAGED_FRESH_AGENT_FIXTURE_MODES"; const PROVIDER_UID: u32 = 65_534; const PROVIDER_GID: u32 = 0; +pub(crate) fn context_for_fresh_provider( + mut provider_launch_context: Option, +) -> Option { + if let Some(freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Claude { mcp_args }, + .. + }) = provider_launch_context.as_mut() + { + mcp_args.clear(); + } + provider_launch_context +} + pub(crate) struct HostedFreshAgentProxy { client: RuntimeClient, broadcast: Arc>, @@ -486,6 +499,24 @@ impl HostedFreshAgentProxy { provider_secret_references = prepared.references; let capability = prepared.context.mcp_capability.clone().ok_or(())?; (capability, Some(prepared.context)) + } else if provider == FreshProvider::Codex { + let home = std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .ok_or(())?; + let original = |name: &str| std::env::var(name).ok(); + let context = crate::managed_provider_references::codex::prepare_managed( + &soul, + &home, + &cwd, + &workspace, + &original, + &[], + &provider_secret_references, + ) + .map_err(|error| { + tracing::warn!(event="fresh_agent.create.provider_inputs_failed",%error); + })?; + (context.mcp_capability.clone().ok_or(())?, Some(context)) } else { let mcp_capability = crate::managed_mcp_capability::issue_mcp_capability( &soul, @@ -511,25 +542,7 @@ impl HostedFreshAgentProxy { })?; (mcp_capability, provider_launch_context) }; - // Fresh agents use the direct provider transports, which currently do - // not inject Freshell's terminal MCP server. Retain ordinary provider - // MCP/config files, plugins, and the staged user root; only remove - // terminal-specific Freshell injection to match the direct route. - if let Some(context) = provider_launch_context.as_mut() { - match &mut context.preparation { - freshell_runtime_protocol::ProviderPreparation::Claude { mcp_args } => { - mcp_args.clear() - } - freshell_runtime_protocol::ProviderPreparation::Codex { - tui_args, - sidecar_args, - } => { - tui_args.clear(); - sidecar_args.clear(); - } - _ => {} - } - } + provider_launch_context = context_for_fresh_provider(provider_launch_context); let launch = FreshAgentLaunchSpec { session_id: public_session_id.clone(), provider: provider.clone(), diff --git a/crates/freshell-server/src/fresh_agent_proxy_tests.rs b/crates/freshell-server/src/fresh_agent_proxy_tests.rs index 4eb2eb7ef..0f856db36 100644 --- a/crates/freshell-server/src/fresh_agent_proxy_tests.rs +++ b/crates/freshell-server/src/fresh_agent_proxy_tests.rs @@ -6,6 +6,36 @@ use freshell_runtime_protocol::{ }; use tokio::net::UnixListener; +#[test] +fn managed_provider_fresh_codex_retains_terminal_and_native_sidecar_arguments() { + let args = crate::managed_provider_bootstrap::managed_codex_mcp_args().unwrap(); + let context = freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Codex { + tui_args: args.clone(), + sidecar_args: args.clone(), + }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let context = context_for_fresh_provider(Some(context)).unwrap(); + let freshell_runtime_protocol::ProviderPreparation::Codex { + tui_args, + sidecar_args, + } = context.preparation + else { + panic!("Codex preparation missing") + }; + assert_eq!( + tui_args, args, + "terminal arguments must remain in accepted fresh context" + ); + assert_eq!( + sidecar_args, args, + "the actual native app-server receives the retained configuration pairs" + ); +} + #[test] fn hosted_created_frame_uses_the_public_request_identity() { let (broadcast, mut receiver) = broadcast::channel(8); diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index a667ef714..fc4d72493 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -28,10 +28,12 @@ const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; mod amplifier_resources; +mod codex_stage; mod source_stage; #[derive(Clone, Copy, Default)] pub(crate) struct OpencodeEphemeralInput<'a> { + pub codex_manifest: Option<&'a freshell_platform::codex_inputs::CodexInputManifest>, pub inline_config: Option<&'a str>, pub tui_config_path: Option<&'a str>, pub tui_source: Option<&'a ProviderConfigReference>, @@ -296,6 +298,62 @@ async fn apply_prepared_callback( ) .map_err(|_| "replacement provider context exceeds control metadata profile".into()); } + if provider == "codex" { + let mut context = callback + .prior_context + .clone() + .ok_or("complete replacement requires prior provider context")?; + let manifest = if context.prepared_inputs.is_some() { + let bytes = + read_prior_manifest_bytes("codex", store, directory, &callback.soul_id, &context)?; + let manifest: freshell_platform::codex_inputs::CodexInputManifest = + serde_json::from_slice(&bytes) + .map_err(|_| "prior Codex source manifest metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + manifest + } else { + let cwd = callback + .cwd + .as_deref() + .or(callback.workspace.as_deref()) + .ok_or("missing provider working directory")?; + freshell_platform::codex_inputs::CodexInputManifest { + version: 1, + original_home: home.into(), + cwd: cwd.into(), + codex_home: home.join(".codex"), + named_profile: None, + } + }; + context.config.clear(); + let references = callback.provider_secret_references.unwrap_or_default(); + let capability = store.issue_replacement_with_inputs( + callback.soul_id, + directory, + endpoint.into(), + Some((provider, &manifest.original_home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(&context), + secret_references: Some(&references), + cwd: Some(&manifest.cwd), + workspace: callback.workspace.as_deref(), + codex_manifest: Some(&manifest), + ..Default::default() + }, + )?; + let reply = freshell_runtime_protocol::PreparedProviderContext { + provider_launch_context: read_prepared_context(directory, &capability)?, + provider_secret_references: references, + }; + reply + .validate(provider) + .map_err(|_| "replacement Codex context metadata invalid")?; + return freshell_runtime_protocol::bounded_json_bytes( + &reply, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "replacement Codex context exceeds control metadata profile".into()); + } if provider != "opencode" { let references = provider_source_references(provider, home)?; let mut context = callback @@ -481,6 +539,56 @@ async fn apply_prepared_callback( .map_err(|_| "replacement provider context exceeds control metadata profile".into()) } +fn read_prior_manifest_bytes( + provider: &str, + store: &CapabilityStore, + directory: &Path, + soul: &SoulId, + context: &freshell_runtime_protocol::ProviderLaunchContext, +) -> Result, String> { + context + .validate(provider) + .map_err(|_| "prior provider source context metadata invalid")?; + let inputs = context + .prepared_inputs + .as_ref() + .ok_or("prior prepared provider inputs unavailable")?; + let capability = context + .mcp_capability + .as_ref() + .ok_or("prior source context lacks generation authority")?; + if !store + .grants + .lock() + .unwrap() + .get(&capability.grant_id) + .is_some_and(|grant| grant.soul_id == *soul) + { + return Err("prior provider source context belongs to another owner".into()); + } + let root = staged_provider_root_path(directory, &capability.grant_id)?; + let path = root.join(&inputs.manifest.relative_path); + if !std::fs::symlink_metadata(&path) + .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) + || std::fs::canonicalize(&path).map_err(|_| "prior source manifest unavailable")? != path + { + return Err("prior source manifest must be a physical private file".into()); + } + use std::io::Read; + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "prior source manifest unavailable")? + .take(64 * 1024 * 1024 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "prior source manifest read failed")?; + if bytes.len() > 64 * 1024 * 1024 + || format!("{:x}", sha2::Sha256::digest(&bytes)) != inputs.manifest.sha256 + { + return Err("prior source manifest profile or identity changed".into()); + } + Ok(bytes) +} + fn read_prior_source_manifest( store: &CapabilityStore, directory: &Path, @@ -1155,6 +1263,18 @@ fn stage_provider_root( .map_err(|_| "prepared resource descriptor exceeds control profile")?; write_private_bytes(&temporary.join(".freshell/resource-mounts.json"), &metadata)?; } + let codex_resources = if provider == "codex" && opencode_input.launch_context.is_some() { + Some(codex_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input.codex_manifest, + )?) + } else { + None + }; + if let Some(resources) = &codex_resources { + selections.extend(resources.selections.iter().cloned()); + } let amplifier_resources = if provider == "amplifier" { Some(amplifier_resources::prepare( home, @@ -1168,6 +1288,9 @@ fn stage_provider_root( selections.extend(resources.selections.iter().cloned()); } for reference in references { + if codex_resources.is_some() { + continue; + } if reference.root != ProviderConfigRoot::UserProvider { return Err("managed provider selector is unsafe".into()); } @@ -1301,6 +1424,23 @@ fn stage_provider_root( opencode_input, )?; } + if let Some(resources) = &codex_resources { + let capability = McpCapabilityReference { + grant_id: grant_id.into(), + endpoint: "http://host.docker.internal:1".into(), + provider_relative_path: CAPABILITY_PATH.into(), + host_gateway_address: None, + }; + let inputs = codex_stage::publish(&temporary, resources, &capability)?; + write_private_bytes( + &temporary.join(".freshell/prepared-inputs.json"), + &freshell_runtime_protocol::bounded_json_bytes( + &inputs, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "Codex prepared context exceeds control profile")?, + )?; + } allow_controller_stage_read(&temporary)?; std::fs::rename(&temporary, &staged).map_err(|error| error.to_string())?; Ok(staged.clone()) @@ -1649,6 +1789,15 @@ impl CapabilityStore { provider_relative_path: CAPABILITY_PATH.into(), host_gateway_address: routed_host_ipv4(), }); + if provider == "codex" { + let bytes = + source_stage::read_source_view(root, ".freshell/prepared-inputs.json")?; + context.prepared_inputs = Some( + serde_json::from_str(&bytes) + .map_err(|_| "Codex prepared inputs unavailable")?, + ); + context.config.clear(); + } let result = publish_prepared_context_metadata( root, provider, @@ -2019,6 +2168,15 @@ pub(crate) fn issue_prepared_opencode_context( home: &Path, context: &freshell_runtime_protocol::ProviderLaunchContext, input: OpencodeEphemeralInput<'_>, +) -> Result { + issue_prepared_provider_context("opencode", soul, home, context, input) +} +pub(crate) fn issue_prepared_provider_context( + provider: &str, + soul: &SoulId, + home: &Path, + context: &freshell_runtime_protocol::ProviderLaunchContext, + input: OpencodeEphemeralInput<'_>, ) -> Result { let input = OpencodeEphemeralInput { launch_context: Some(context), @@ -2029,7 +2187,7 @@ pub(crate) fn issue_prepared_opencode_context( soul.clone(), &directory, endpoint()?, - Some(("opencode", home, &context.config)), + Some((provider, home, &context.config)), input, None, )?; @@ -3475,6 +3633,355 @@ except Exception as error: .is_dir()); } + #[cfg(feature = "managed-provider-qualification")] + #[tokio::test] + async fn managed_provider_codex_native_typed_inputs_survive_transport() { + use freshell_platform::provider_environment::{ + copy_environment, CopyLimits, CopySelection, + }; + use freshell_runtime_protocol::{ProviderLaunchContext, ProviderPreparation}; + use sha2::{Digest, Sha256}; + use std::process::Stdio; + use tokio::io::{AsyncBufReadExt, AsyncWriteExt}; + let owned = executable_fixture_directory(); + let original = owned.path().join("original-home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(original.join(".codex/agents/nested")).unwrap(); + std::fs::create_dir_all(workspace.join(".codex/skills/probe/scripts")).unwrap(); + std::fs::create_dir(workspace.join(".git")).unwrap(); + let config = format!("model_instructions_file = 'user.md'\n[[skills.config]]\npath = 'absent-skill'\nenabled = false\n[projects.{}]\ntrust_level = 'trusted'\n[agents.reader]\nconfig_file = 'agents/nested/reader.toml'\n",serde_json::to_string(workspace.to_str().unwrap()).unwrap()); + std::fs::write(original.join(".codex/config.toml"), &config).unwrap(); + std::fs::write(original.join(".codex/user.md"), "user-instructions-nonce").unwrap(); + std::fs::write( + original.join(".codex/agents/nested/reader.toml"), + "model_instructions_file = 'role.md'\n", + ) + .unwrap(); + std::fs::write( + original.join(".codex/agents/nested/role.md"), + "role-instructions-nonce", + ) + .unwrap(); + std::fs::write( + workspace.join(".codex/config.toml"), + "model_instructions_file = 'project.md'\n", + ) + .unwrap(); + std::fs::write( + workspace.join(".codex/project.md"), + "project-instructions-nonce", + ) + .unwrap(); + std::fs::write(workspace.join(".codex/skills/probe/SKILL.md"),"---\nname: transport-probe\ndescription: Synthetic executable resource fixture\n---\nRun scripts/run to read the adjacent complete resource.\n").unwrap(); + std::fs::write( + workspace.join(".codex/skills/probe/resource.txt"), + "skill-resource-nonce", + ) + .unwrap(); + let script = workspace.join(".codex/skills/probe/scripts/run"); + std::fs::write( + &script, + "#!/bin/sh\ncat \"$(dirname \"$0\")/../resource.txt\"\n", + ) + .unwrap(); + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o700)).unwrap(); + let mcp = crate::managed_provider_bootstrap::managed_codex_mcp_args().unwrap(); + let context = ProviderLaunchContext { + config: vec![], + mcp_capability: None, + prepared_inputs: None, + preparation: ProviderPreparation::Codex { + tui_args: mcp.clone(), + sidecar_args: mcp, + }, + }; + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let grant = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", &original, &[])), + OpencodeEphemeralInput { + launch_context: Some(&context), + cwd: Some(&workspace), + workspace: Some(&workspace), + ..Default::default() + }, + None, + ) + .unwrap(); + let context = crate::fresh_agent_proxy::context_for_fresh_provider(Some( + read_prepared_context(&directory, &grant).unwrap(), + )) + .unwrap(); + let inputs = context.prepared_inputs.as_ref().unwrap(); + let stage = staged_provider_root_path(&directory, &grant.grant_id).unwrap(); + let physical = Path::new("/home/freshell/provider"); + assert!(physical.is_dir()); + let selections = inputs + .resource_mounts + .iter() + .map(|unit| CopySelection { + source: stage.join(&unit.stage_relative_path), + destination_relative: unit.provider_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&selections, physical, &[], CopyLimits::default()).unwrap(); + // The original fixture remains preserved byte-for-byte. Native reads + // only descriptor projections at each accepted logical path. + let archived = owned.path().join("archived-home"); + std::fs::rename(&original, &archived).unwrap(); + let archived_project = owned.path().join("archived-project-config"); + std::fs::rename(workspace.join(".codex"), &archived_project).unwrap(); + for bind in &inputs.config_binds { + let destination = Path::new(&bind.effective_destination); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + std::os::unix::fs::symlink(stage.join(&bind.source_relative_path), destination) + .unwrap(); + } + for unit in &inputs.resource_mounts { + let destination = Path::new(&unit.effective_destination); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + std::os::unix::fs::symlink(physical.join(&unit.provider_relative_path), destination) + .unwrap(); + } + let version = std::process::Command::new("/usr/local/bin/codex") + .arg("--version") + .output() + .unwrap(); + assert!(version.status.success()); + assert_eq!( + String::from_utf8(version.stdout).unwrap().trim(), + "codex-cli 0.147.0" + ); + let mut child = tokio::process::Command::new("/usr/local/bin/codex") + .arg("app-server") + .args(match &context.preparation { + ProviderPreparation::Codex { sidecar_args, .. } => sidecar_args.clone(), + _ => panic!("Codex context missing"), + }) + .current_dir(&workspace) + .env("HOME", physical) + .env("CODEX_HOME", physical.join(".codex")) + .env_remove("OPENAI_API_KEY") + .env_remove("FRESHELL_CLAUDE_SDK_QUERY_MODULE") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .kill_on_drop(true) + .spawn() + .unwrap(); + let mut stdin = child.stdin.take().unwrap(); + let mut stdout = tokio::io::BufReader::new(child.stdout.take().unwrap()).lines(); + async fn request( + stdin: &mut tokio::process::ChildStdin, + stdout: &mut tokio::io::Lines>, + id: u64, + method: &str, + params: serde_json::Value, + ) -> serde_json::Value { + stdin + .write_all( + format!( + "{}\n", + serde_json::json!({"id":id,"method":method,"params":params}) + ) + .as_bytes(), + ) + .await + .unwrap(); + tokio::time::timeout(std::time::Duration::from_secs(25), async { + loop { + let line = stdout + .next_line() + .await + .unwrap() + .expect("native Codex app-server ended before response"); + let response: serde_json::Value = serde_json::from_str(&line).unwrap(); + if response.get("id").and_then(serde_json::Value::as_u64) == Some(id) { + assert!( + response.get("error").is_none(), + "native Codex request {method} failed" + ); + return response["result"].clone(); + } + } + }) + .await + .expect("native Codex source consumer response deadline") + } + request(&mut stdin,&mut stdout,1,"initialize",serde_json::json!({"clientInfo":{"name":"freshell-fixture","version":"1"},"capabilities":{"experimentalApi":true}})).await; + stdin + .write_all(b"{\"method\":\"initialized\"}\n") + .await + .unwrap(); + let resolved = request( + &mut stdin, + &mut stdout, + 2, + "config/read", + serde_json::json!({"includeLayers":true,"cwd":workspace}), + ) + .await; + assert_eq!( + resolved["config"]["model_instructions_file"].as_str(), + Some(workspace.join(".codex/project.md").to_str().unwrap()) + ); + assert_eq!( + resolved["config"]["mcp_servers"]["freshell"]["command"], "node", + "actual native app-server must consume the retained fresh provider argv" + ); + assert_eq!( + resolved["config"]["mcp_servers"]["freshell"]["args"], + serde_json::json!(["/opt/freshell-mcp/server.js"]) + ); + let skills = request( + &mut stdin, + &mut stdout, + 3, + "skills/list", + serde_json::json!({"cwds":[workspace],"forceReload":true}), + ) + .await; + let found = skills["data"] + .as_array() + .unwrap() + .iter() + .flat_map(|data| data["skills"].as_array().unwrap()) + .any(|skill| skill["name"] == "transport-probe"); + assert!( + found, + "native Codex must discover the transported project skill" + ); + child.kill().await.unwrap(); + child.wait().await.unwrap(); + let resource = + std::process::Command::new(workspace.join(".codex/skills/probe/scripts/run")) + .output() + .unwrap(); + assert!(resource.status.success()); + assert_eq!(resource.stdout, b"skill-resource-nonce"); + assert_eq!( + std::fs::read_to_string(archived.join(".codex/config.toml")).unwrap(), + config + ); + for (case, bytes) in [ + ( + "native-config-read", + b"project-instructions-nonce".as_slice(), + ), + ("native-skills-list", b"transport-probe".as_slice()), + ("executable-resource", b"skill-resource-nonce".as_slice()), + ( + "native-retained-mcp-args", + b"node:/opt/freshell-mcp/server.js".as_slice(), + ), + ] { + println!( + "provider_source_consumer {}", + serde_json::json!({"provider":"codex","case":case,"digest":format!("{:x}",Sha256::digest(bytes))}) + ); + } + } + + #[test] + fn managed_provider_codex_selected_inputs_reach_private_generation() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".codex/agents/nested")).unwrap(); + std::fs::create_dir_all(workspace.join(".codex/skills/fixture/scripts")).unwrap(); + std::fs::create_dir(workspace.join(".git")).unwrap(); + let raw = format!("model_instructions_file = 'instructions.md'\n[projects.{}]\ntrust_level = 'trusted'\n[agents.reader]\nconfig_file = 'agents/nested/reader.toml'\n", serde_json::to_string(workspace.to_str().unwrap()).unwrap()); + std::fs::write(home.join(".codex/config.toml"), &raw).unwrap(); + std::fs::write(home.join(".codex/instructions.md"), "home-input").unwrap(); + std::fs::write( + home.join(".codex/agents/nested/reader.toml"), + "model_instructions_file = 'instructions.md'\n", + ) + .unwrap(); + std::fs::write( + home.join(".codex/agents/nested/instructions.md"), + "role-input", + ) + .unwrap(); + std::fs::write( + workspace.join(".codex/config.toml"), + "model_instructions_file = 'instructions.md'\n", + ) + .unwrap(); + std::fs::write(workspace.join(".codex/instructions.md"), "project-input").unwrap(); + std::fs::write( + workspace.join(".codex/skills/fixture/SKILL.md"), + "skill-resource", + ) + .unwrap(); + let context = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "codex", &workspace, &home, None, + ) + .unwrap() + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let capability = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", &home, &context.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let accepted = read_prepared_context(&directory, &capability).unwrap(); + let inputs = accepted + .prepared_inputs + .expect("Codex selected layers and typed resources must be accepted as one generation"); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + for (logical, expected) in [ + (home.join(".codex/instructions.md"), "home-input"), + ( + home.join(".codex/agents/nested/instructions.md"), + "role-input", + ), + (workspace.join(".codex/instructions.md"), "project-input"), + ] { + let mount = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == logical.to_str().unwrap()) + .expect("ordinary selected resource must retain its original absolute alias"); + assert_eq!( + std::fs::read_to_string(stage.join(&mount.stage_relative_path)).unwrap(), + expected + ); + } + assert!(inputs + .resource_mounts + .iter() + .any(|mount| mount.logical_destination + == workspace.join(".codex/skills").to_str().unwrap())); + assert!(inputs + .config_binds + .iter() + .any(|bind| bind.logical_destination + == workspace.join(".codex/config.toml").to_str().unwrap())); + assert_eq!( + std::fs::read_to_string(home.join(".codex/config.toml")).unwrap(), + raw + ); + } + #[tokio::test] async fn managed_provider_replacement_complete_codex_metadata_replays_before_rescanning_roots() { @@ -3504,11 +4011,17 @@ except Exception as error: &directory, "http://host.docker.internal:4000".into(), Some(("codex", &home, &context.config)), - Default::default(), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + ..Default::default() + }, None, ) .unwrap(); - context.mcp_capability = Some(prior); + context = read_prepared_context(&directory, &prior).unwrap(); + assert!(context.prepared_inputs.is_some()); let callback = serde_json::to_vec( &serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul, "provider":"codex","workspace":workspace,"cwd":workspace,"priorContext":context, @@ -3547,17 +4060,29 @@ except Exception as error: "a complete pending context must retain its exact selected root metadata in every provider route"); let ready: freshell_runtime_protocol::PreparedProviderContext = serde_json::from_slice(&repeated).unwrap(); + let resource = ready + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap() + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == "/home/freshell/provider/.codex/skills") + .unwrap() + .stage_relative_path + .clone(); let stage = staged_provider_root_path( &directory, &ready .provider_launch_context .mcp_capability + .as_ref() .unwrap() .grant_id, ) .unwrap(); assert_eq!( - std::fs::read_to_string(stage.join("skills/owned/runner.sh")).unwrap(), + std::fs::read_to_string(stage.join(resource).join("owned/runner.sh")).unwrap(), "synthetic retained resource" ); assert_eq!( diff --git a/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs b/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs new file mode 100644 index 000000000..1861d4625 --- /dev/null +++ b/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs @@ -0,0 +1,176 @@ +//! Native-selected ordinary Codex inputs use the shared resource copier and +//! existing exact mounts. No configuration payload enters callback metadata. +use super::{private_directory, write_private_bytes}; +use crate::managed_provider_references::codex::{discover, Discovery, InputKind}; +use freshell_platform::{codex_inputs::CodexInputManifest, provider_environment::CopySelection}; +use freshell_runtime_protocol::*; +use std::{ + collections::BTreeMap, + path::{Path, PathBuf}, +}; + +pub(super) struct Preparation { + pub discovery: Discovery, + pub selections: Vec, + pub binds: Vec, + pub mounts: Vec, +} +pub(super) fn prepare( + home: &Path, + cwd: &Path, + accepted: Option<&CodexInputManifest>, +) -> Result { + let manifest = accepted.cloned().unwrap_or_else(|| CodexInputManifest { + version: 1, + original_home: home.into(), + cwd: cwd.into(), + codex_home: home.join(".codex"), + named_profile: None, + }); + let discovery = discover(manifest).map_err(str::to_string)?; + let mut result = Preparation { + discovery, + selections: Vec::new(), + binds: Vec::new(), + mounts: Vec::new(), + }; + let mut files = BTreeMap::::new(); + for input in &result.discovery.inputs { + if input.kind != InputKind::Configuration { + continue; + } + let next = files.len(); + let source = files + .entry(input.canonical_source.clone()) + .or_insert_with(|| { + let relative = format!(".freshell/templates/codex-{next}"); + result.selections.push(CopySelection { + source: input.canonical_source.clone(), + destination_relative: relative.clone().into(), + }); + relative + }); + for destination in destinations(&result.discovery.manifest, &input.logical_path) { + let bind = ProviderConfigBind { + source_relative_path: source.clone(), + logical_destination: destination + .to_str() + .ok_or("Codex config destination is not UTF-8")? + .into(), + effective_destination: effective( + &destination, + &input.canonical_source, + &result.discovery.manifest, + ) + .to_str() + .ok_or("Codex config destination is not UTF-8")? + .into(), + }; + if !result.binds.contains(&bind) { + result.binds.push(bind); + } + } + } + let mut ordinary = result.discovery.resource_roots.clone(); + for input in &result.discovery.inputs { + if input.kind == InputKind::Configuration { + continue; + } + if !ordinary + .iter() + .any(|root| input.logical_path.starts_with(root)) + { + ordinary.push(input.logical_path.clone()); + } + } + ordinary.sort(); + ordinary.dedup(); + let mut units = BTreeMap::::new(); + for logical in ordinary { + let source = std::fs::canonicalize(&logical) + .map_err(|_| "Codex selected resource is missing, inaccessible or cyclic")?; + let metadata = std::fs::metadata(&source) + .map_err(|_| "Codex selected resource metadata unavailable")?; + let kind = if metadata.is_dir() { + ProviderResourceKind::Directory + } else if metadata.is_file() { + ProviderResourceKind::File + } else { + return Err("Codex selected resource has an unsupported type".into()); + }; + let next = units.len(); + let (name, _) = units.entry(source.clone()).or_insert_with(|| { + let name = if kind == ProviderResourceKind::File { + format!("root-{next}/source") + } else { + format!("root-{next}") + }; + result.selections.push(CopySelection { + source: source.clone(), + destination_relative: format!(".freshell/resource-units/{name}").into(), + }); + (name, kind) + }); + for destination in destinations(&result.discovery.manifest, &logical) { + result.mounts.push(ProviderResourceMount { + stage_relative_path: format!(".freshell/resource-units/{name}"), + provider_relative_path: format!(".freshell/provider-roots/{name}"), + logical_destination: destination + .to_str() + .ok_or("Codex resource destination is not UTF-8")? + .into(), + effective_destination: effective(&destination, &source, &result.discovery.manifest) + .to_str() + .ok_or("Codex resource destination is not UTF-8")? + .into(), + kind, + }); + } + } + Ok(result) +} +fn destinations(manifest: &CodexInputManifest, logical: &Path) -> Vec { + let runtime = manifest.runtime_path(logical); + if runtime == logical { + vec![runtime] + } else { + vec![runtime, logical.into()] + } +} +fn effective(destination: &Path, canonical: &Path, manifest: &CodexInputManifest) -> PathBuf { + if destination.starts_with(&manifest.cwd) && !destination.starts_with("/home/freshell/provider") + { + canonical.into() + } else { + destination.into() + } +} +pub(super) fn publish( + root: &Path, + preparation: &Preparation, + capability: &McpCapabilityReference, +) -> Result { + use sha2::{Digest, Sha256}; + private_directory(&root.join(".freshell"))?; + let bytes = bounded_json_bytes(&preparation.discovery.manifest, 64 * 1024 * 1024) + .map_err(|_| "Codex source metadata budget exceeded")?; + write_private_bytes(&root.join(".freshell/input-manifest.json"), &bytes)?; + let inputs = PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: preparation.binds.clone(), + resource_mounts: preparation.mounts.clone(), + indexed_files: None, + }; + inputs + .validate(Some(capability)) + .map_err(|error| error.message)?; + Ok(inputs) +} diff --git a/crates/freshell-server/src/managed_provider_bootstrap.rs b/crates/freshell-server/src/managed_provider_bootstrap.rs index 9797f3e7b..8faa62f74 100644 --- a/crates/freshell-server/src/managed_provider_bootstrap.rs +++ b/crates/freshell-server/src/managed_provider_bootstrap.rs @@ -323,7 +323,7 @@ impl freshell_platform::mcp_inject::McpRuntime for ManagedImageMcpRuntime { } } -fn managed_codex_mcp_args() -> Option> { +pub(crate) fn managed_codex_mcp_args() -> Option> { let renderings = freshell_platform::mcp_inject::build_managed_codex_mcp_renderings( &ManagedImageMcpRuntime, &freshell_platform::RealEnv, diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index ea4b05cb7..40185216a 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -7,7 +7,6 @@ pub enum InputKind { Instructions, CompactPrompt, ModelCatalog, - SkillSelector, } pub struct SelectedInput { pub logical_path: PathBuf, @@ -153,12 +152,9 @@ impl Selection<'_> { .as_array() .ok_or("Codex skills config has an unsupported type")?; for rule in rules { - if let Some(path) = string_field(rule, "path")? { - self.add( - typed_path(path, parent, self.home), - InputKind::SkillSelector, - )?; - } + // Native paths are enable/disable selectors. They remain + // in the transported TOML and confer no resource authority. + string_field(rule, "path")?; } } if kind != LayerKind::Role { @@ -310,6 +306,305 @@ fn normalize(path: &Path) -> PathBuf { result } +pub(crate) fn prepare_managed( + soul: &freshell_runtime_protocol::SoulId, + home: &Path, + cwd: &Path, + workspace: &Path, + original: &dyn Fn(&str) -> Option, + args: &[String], + references: &[freshell_runtime_protocol::ProviderSecretReference], +) -> Result { + let codex_home = + if let Some(selected) = original("CODEX_HOME").filter(|value| !value.is_empty()) { + let selected = PathBuf::from(selected); + let path = std::fs::canonicalize(if selected.is_absolute() { + selected + } else { + cwd.join(selected) + }) + .map_err(|_| { + "selected CODEX_HOME is unavailable; repair the selected directory and retry" + })?; + if !path.is_dir() { + return Err("selected CODEX_HOME must be a directory".into()); + } + path + } else { + home.join(".codex") + }; + let named_profile = args + .windows(2) + .filter(|pair| matches!(pair[0].as_str(), "--profile" | "-p")) + .map(|pair| pair[1].clone()) + .next_back(); + let manifest = freshell_platform::codex_inputs::CodexInputManifest { + version: 1, + original_home: home.into(), + cwd: cwd.into(), + codex_home, + named_profile, + }; + let mcp = crate::managed_provider_bootstrap::managed_codex_mcp_args() + .ok_or("managed Codex MCP preparation unavailable")?; + let context = freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Codex { + tui_args: mcp.clone(), + sidecar_args: mcp, + }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + crate::managed_mcp_capability::issue_prepared_provider_context( + "codex", + soul, + home, + &context, + crate::managed_mcp_capability::OpencodeEphemeralInput { + codex_manifest: Some(&manifest), + cwd: Some(cwd), + workspace: Some(workspace), + secret_references: Some(references), + ..Default::default() + }, + ) +} + +pub struct Discovery { + pub manifest: freshell_platform::codex_inputs::CodexInputManifest, + pub inputs: Vec, + pub resource_roots: Vec, +} + +pub fn discover( + manifest: freshell_platform::codex_inputs::CodexInputManifest, +) -> Result { + manifest.validate()?; + let mut layers = Vec::new(); + let mut preceding = toml::Value::Table(Default::default()); + let base = manifest.codex_home.join("config.toml"); + if exists(&base)? { + let value = read_config(&base)?; + if let Some(name) = &manifest.named_profile { + if value.get("profile").and_then(toml::Value::as_str) == Some(name.as_str()) + || value + .get("profiles") + .and_then(|profiles| profiles.get(name)) + .is_some() + { + return Err("Codex named profile conflicts with legacy profile settings"); + } + } + merge(&mut preceding, value); + layers.push(SourceLayer { + logical_path: base, + enabled: true, + kind: LayerKind::User, + }); + } + if let Some(name) = &manifest.named_profile { + let path = manifest.codex_home.join(format!("{name}.config.toml")); + // A selected named configuration is required, unlike an absent base. + merge(&mut preceding, read_config(&path)?); + layers.push(SourceLayer { + logical_path: path, + enabled: true, + kind: LayerKind::User, + }); + } + let markers: Vec = if let Some(value) = preceding.get("project_root_markers") { + value + .as_array() + .ok_or("Codex project markers must be an array")? + .iter() + .map(|value| { + value + .as_str() + .map(str::to_owned) + .ok_or("Codex project marker must be a string") + }) + .collect::>()? + } else { + vec![".git".into()] + }; + let root = manifest + .cwd + .ancestors() + .find(|directory| markers.iter().any(|marker| directory.join(marker).exists())) + .unwrap_or(&manifest.cwd); + let mut directories = manifest + .cwd + .ancestors() + .take_while(|directory| directory.starts_with(root)) + .map(Path::to_path_buf) + .collect::>(); + directories.reverse(); + let mut resources = Vec::new(); + for path in [ + manifest.codex_home.join("skills"), + manifest.original_home.join(".agents/skills"), + ] { + if exists(&path)? { + resources.push(path); + } + } + for directory in &directories { + let folder = directory.join(".codex"); + if exists(&folder)? + && normalize(&folder) != normalize(&manifest.codex_home) + && std::fs::canonicalize(&folder).ok() + != std::fs::canonicalize(&manifest.codex_home).ok() + { + if !std::fs::metadata(&folder) + .map_err(|_| "Codex project root is unreadable")? + .is_dir() + { + return Err("Codex project config root must be a directory"); + } + let skills = folder.join("skills"); + if exists(&skills)? { + resources.push(skills); + } + let path = folder.join("config.toml"); + if exists(&path)? { + layers.push(SourceLayer { + logical_path: path, + enabled: trusted(&preceding, directory, root), + kind: LayerKind::Project, + }); + } + } + let skills = directory.join(".agents/skills"); + if exists(&skills)? { + resources.push(skills); + } + } + // User configuration alone selects installed plugins. Project tables do + // not change the selection. Copy complete installed package/data units. + if let Some(plugins) = preceding.get("plugins").and_then(toml::Value::as_table) { + for (identity, value) in plugins { + if value.get("enabled").and_then(toml::Value::as_bool) != Some(true) { + continue; + } + let (plugin, market) = identity + .split_once('@') + .ok_or("Codex installed plugin identity is invalid")?; + if [plugin, market].iter().any(|part| { + part.is_empty() + || Path::new(part) + .components() + .any(|component| !matches!(component, std::path::Component::Normal(_))) + }) { + return Err("Codex installed plugin identity is invalid"); + } + let cache = manifest + .codex_home + .join("plugins/cache") + .join(market) + .join(plugin); + if !exists(&cache)? { + return Err( + "Codex selected installed plugin is unavailable; repair installation and retry", + ); + } + resources.push(cache); + let data = manifest + .codex_home + .join("plugins/data") + .join(format!("{plugin}-{market}")); + if exists(&data)? { + resources.push(data); + } + } + } + let legacy = if manifest.named_profile.is_none() { + preceding.get("profile").and_then(toml::Value::as_str) + } else { + None + }; + let inputs = select_inputs(&layers, &manifest.original_home, legacy)?; + resources.sort(); + resources.dedup(); + Ok(Discovery { + manifest, + inputs, + resource_roots: resources, + }) +} +fn exists(path: &Path) -> Result { + match std::fs::symlink_metadata(path) { + Ok(_) => Ok(true), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(_) => Err("Codex selected resource is inaccessible; repair permissions and retry"), + } +} +fn read_config(path: &Path) -> Result { + use std::io::Read; + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "Codex selected configuration is unavailable")? + .take(16 * 1024 * 1024 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "Codex selected configuration is unreadable")?; + if bytes.len() > 16 * 1024 * 1024 { + return Err("Codex selected configuration source view exceeded"); + } + let raw = + std::str::from_utf8(&bytes).map_err(|_| "Codex selected configuration is not UTF-8")?; + toml::from_str(raw).map_err(|_| "Codex selected configuration TOML is invalid") +} +fn merge(target: &mut toml::Value, value: toml::Value) { + if let (Some(target), Some(value)) = (target.as_table_mut(), value.as_table()) { + for (key, value) in value { + if let Some(existing) = target.get_mut(key) { + merge(existing, value.clone()); + } else { + target.insert(key.clone(), value.clone()); + } + } + } else { + *target = value; + } +} +fn trusted(value: &toml::Value, directory: &Path, root: &Path) -> bool { + let Some(projects) = value.get("projects") else { + return false; + }; + let mut paths = vec![directory.to_path_buf()]; + if let Ok(canonical) = std::fs::canonicalize(directory) { + paths.push(canonical); + } + paths.push(root.to_path_buf()); + if let Ok(canonical) = std::fs::canonicalize(root) { + paths.push(canonical); + } + // Linked worktrees share the root checkout's trust selection. + if let Ok(gitfile) = std::fs::read_to_string(root.join(".git")) { + if let Some(gitdir) = gitfile.trim().strip_prefix("gitdir: ") { + let path = normalize(&root.join(gitdir)); + if let Some(checkout) = path + .ancestors() + .find(|path| path.file_name() == Some(std::ffi::OsStr::new(".git"))) + .and_then(Path::parent) + { + paths.push(checkout.to_path_buf()); + } + } + } + for path in paths { + if let Some(trust) = path + .to_str() + .and_then(|key| projects.get(key)) + .and_then(|project| project.get("trust_level")) + .and_then(toml::Value::as_str) + { + return trust == "trusted"; + } + } + false +} + #[cfg(test)] mod tests { use super::*; @@ -324,6 +619,22 @@ mod tests { kind: LayerKind::User, } } + #[test] + fn managed_provider_codex_skill_path_selectors_do_not_create_resource_authority() { + let owned = tempfile::tempdir().unwrap(); + let config = owned.path().join("config.toml"); + std::fs::write( + &config, + "[[skills.config]]\npath = 'absent-skill'\nenabled = false\n", + ) + .unwrap(); + let inputs = select_inputs(&[layer(config,true)],owned.path(),None).expect("native disabled path selectors do not require an existing file or grant a new skill discovery root"); + assert_eq!(inputs.len(), 1); + assert!(inputs + .iter() + .all(|input| input.kind == InputKind::Configuration)); + } + #[test] fn managed_provider_codex_discovered_roles_keep_nested_instruction_origins() { let fixture = tempfile::tempdir().unwrap(); @@ -468,10 +779,12 @@ enabled = false .iter() .any(|input| input.kind == InputKind::CompactPrompt && digest(&input.canonical_source) == digest(&logical.join("profile.md")))); - assert!(selected - .iter() - .any(|input| input.kind == InputKind::SkillSelector - && input.logical_path == logical.join("skills/SKILL.md"))); + assert!( + !selected + .iter() + .any(|input| input.logical_path == logical.join("skills/SKILL.md")), + "a skill path selector cannot grant ordinary file/resource authority" + ); assert_eq!( std::fs::read_to_string(actual.join("role.toml")).unwrap(), "model_instructions_file='./instructions.md'" diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index bfecc59d9..548a237c7 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -225,6 +225,30 @@ impl ManagedTerminalController for ServerManagedRuntimeController { prepared.context.mcp_capability.clone(), Some(prepared.context), ) + } else if request.mode == "codex" { + let home = request + .env + .get("HOME") + .map(std::path::PathBuf::from) + .or_else(|| std::env::var_os("HOME").map(std::path::PathBuf::from)) + .ok_or("managed provider HOME unavailable")?; + let original = |name: &str| { + request + .env + .get(name) + .cloned() + .or_else(|| std::env::var(name).ok()) + }; + let context = crate::managed_provider_references::codex::prepare_managed( + &soul_id, + &home, + &cwd, + &workspace, + &original, + &args, + &provider_secret_references, + )?; + (context.mcp_capability.clone(), Some(context)) } else { let mcp_capability = matches!( request.mode.as_str(), diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index fbec49265..8501d81e1 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -886,7 +886,7 @@ async fn grant_execution( format!("resolve managed provider credentials: {error}"), ) })?; - if provider == "opencode" { + if matches!(provider, "opencode" | "codex") { let generated = provider_input_preparation::prepare( context, &mut secrets, @@ -1165,6 +1165,7 @@ async fn grant_execution( &mut resolved_secrets.environment, resolved_secrets.opencode_plan.as_ref(), ) + .await .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; freshell_platform::managed_child_secrets::install(resolved_secrets.environment) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index ccc16fbc4..60ea46ae0 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -49,12 +49,38 @@ pub fn prepare( }; context .ok_or("prepared provider context unavailable")? - .validate("opencode") + .validate(context.unwrap().preparation.provider()) .map_err(|error| error.message)?; let metadata = read_private(stage, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; if format!("{:x}", Sha256::digest(&metadata)) != inputs.manifest.sha256 { return Err("prepared source manifest changed before dispatch".into()); } + if context.unwrap().preparation.provider() == "codex" { + if !frozen.is_empty() + || inputs.manifest.source_count != 0 + || inputs.manifest.alias_count != 0 + || inputs.manifest.recipe_count != 0 + || inputs.indexed_files.is_some() + { + return Err( + "Codex ordinary source context carries foreign private reference inputs".into(), + ); + } + let manifest: freshell_platform::codex_inputs::CodexInputManifest = + serde_json::from_slice(&metadata) + .map_err(|_| "prepared Codex discovery metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + let mut generated = BTreeMap::new(); + generated.insert( + "CODEX_HOME".into(), + manifest + .runtime_home() + .to_str() + .ok_or("native Codex home is not UTF-8")? + .into(), + ); + return Ok(generated); + } let manifest: SourceManifest = serde_json::from_slice(&metadata) .map_err(|_| "prepared source manifest metadata invalid")?; if manifest.version != 1 @@ -259,6 +285,78 @@ mod tests { use freshell_platform::opencode_recipe::*; use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + #[test] + fn provider_secret_codex_prepared_manifest_retains_native_home_without_original_tree() { + let owned = tempfile::tempdir().unwrap(); + std::fs::create_dir(owned.path().join(".freshell")).unwrap(); + let manifest = freshell_platform::codex_inputs::CodexInputManifest { + version: 1, + original_home: "/absent-original-home".into(), + cwd: "/absent-original-workspace".into(), + codex_home: "/absent-original-home/custom-codex".into(), + named_profile: Some("selected".into()), + }; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Codex { + tui_args: vec![], + sidecar_args: vec![], + }, + config: vec![], + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }), + mcp_capability: Some(capability), + }; + let mut secrets = ResolvedProviderSecrets::default(); + let generated = prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + owned.path(), + owned.path(), + ) + .expect( + "accepted native Codex inputs must validate without reading original controller paths", + ); + assert_eq!( + generated.get("CODEX_HOME").unwrap(), + "/home/freshell/provider/.codex" + ); + std::fs::write( + owned.path().join(".freshell/input-manifest.json"), + b"changed", + ) + .unwrap(); + assert!(prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + owned.path(), + owned.path() + ) + .unwrap_err() + .contains("changed")); + } + #[test] fn provider_secret_prepared_recipe_consumes_private_frozen_inputs_without_original_tree() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 3893008ef..332a1c700 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -64,19 +64,8 @@ pub async fn prepare_terminal( prepared_opencode_plan: Option<&freshell_platform::opencode_config::OpencodeConfigPlan>, ) -> Result { let mut child_env = child_secret_env.clone(); - if let Some(context) = terminal.provider_launch_context.as_ref() { - if let Some(capability) = context.mcp_capability.as_ref() { - if capability.endpoint.is_empty() { - return Err("managed MCP endpoint is unavailable".into()); - } - let grant = read_mcp_grant(std::path::Path::new("/run/freshell/mcp-capability.json"))?; - if grant.get("FRESHELL_URL") != Some(&capability.endpoint) { - return Err("managed MCP grant endpoint differs from launch context".into()); - } - probe_managed_endpoint(&capability.endpoint).await?; - child_env.extend(grant); - } - } + prepare_scoped_mcp_environment(terminal.provider_launch_context.as_ref(), &mut child_env) + .await?; if terminal.mode == "claude" && terminal .provider_launch_context @@ -192,6 +181,21 @@ pub async fn prepare_terminal( }) } +async fn prepare_scoped_mcp_environment( + context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, + child_env: &mut std::collections::BTreeMap, +) -> Result<(), String> { + if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { + let grant = read_mcp_grant(std::path::Path::new("/run/freshell/mcp-capability.json"))?; + if grant.get("FRESHELL_URL") != Some(&capability.endpoint) { + return Err("managed MCP grant endpoint differs from launch context".into()); + } + probe_managed_endpoint(&capability.endpoint).await?; + child_env.extend(grant); + } + Ok(()) +} + fn read_mcp_grant( path: &std::path::Path, ) -> Result, String> { @@ -266,12 +270,13 @@ fn prepare_provider_features_for_identity(uid: u32, gid: u32, args: &[&str]) -> } } -pub fn prepare_fresh_agent_child_environment( +pub async fn prepare_fresh_agent_child_environment( launch: &freshell_runtime_protocol::FreshAgentLaunchSpec, child_env: &mut std::collections::BTreeMap, prepared_opencode_plan: Option<&freshell_platform::opencode_config::OpencodeConfigPlan>, ) -> Result<(), String> { use freshell_runtime_protocol::{FreshProvider, ProviderPreparation}; + prepare_scoped_mcp_environment(launch.provider_launch_context.as_ref(), child_env).await?; if launch.provider != FreshProvider::Opencode { return Ok(()); } @@ -636,6 +641,18 @@ mod provider_secret_context_tests { ); } + #[tokio::test] + async fn provider_secret_fresh_codex_requires_its_selected_scoped_mcp_grant() { + let launch: freshell_runtime_protocol::FreshAgentLaunchSpec=serde_json::from_value(serde_json::json!({"sessionId":"synthetic-fresh","provider":"codex","sessionType":"freshcodex","runtimeVariant":"codex-app-server","providerStoreId":"synthetic-store","cwd":"/synthetic-workspace","workspacePath":"/synthetic-workspace","runAsUid":65534,"runAsGid":0,"providerLaunchContext":{"preparation":{"codex":{"tui_args":[],"sidecar_args":[]}},"mcpCapability":{"grantId":"grant-synthetic","endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}}})).unwrap(); + let mut env = Default::default(); + let error = prepare_fresh_agent_child_environment(&launch, &mut env, None) + .await + .expect_err( + "fresh native Codex must require the selected scoped grant before it starts", + ); + assert!(error.contains("grant is unavailable")); + } + #[test] fn scoped_mcp_grant_becomes_provider_process_environment() { let grant = tempfile::NamedTempFile::new().unwrap(); diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index a437a5ca5..3c697534f 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -20,19 +20,21 @@ def docker(*args, check=True): def main(): parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--provider",choices=["amplifier","codex"],default="amplifier") parser.add_argument("--test-binary", type=Path, required=True) parser.add_argument("--receipt-dir", type=Path, required=True) args = parser.parse_args() + selected_test = TEST if args.provider == "amplifier" else "managed_mcp_capability::tests::managed_provider_codex_native_typed_inputs_survive_transport" binary = args.test_binary.resolve(strict=True) if not binary.is_file() or not re.fullmatch(r"freshell_server-[0-9a-f]{16}", binary.name): raise RuntimeError("source fixture requires its compiled server test artifact") expected_directory = Path(__file__).resolve().parents[2] / "target/debug/deps" if binary.parent != expected_directory.resolve(): raise RuntimeError("source fixture test artifact must belong to this worktree") - listing = subprocess.run([str(binary), "--exact", TEST, "--list"], + listing = subprocess.run([str(binary), "--exact", selected_test, "--list"], env={"PATH": "/usr/bin:/bin"}, text=True, capture_output=True, timeout=20, check=True) - if f"{TEST}: test" not in listing.stdout.splitlines(): + if f"{selected_test}: test" not in listing.stdout.splitlines(): raise RuntimeError("source fixture compiled artifact contains no exact selected test") if not args.receipt_dir.is_dir(): raise RuntimeError("source fixture receipt directory must already exist") @@ -44,7 +46,7 @@ def main(): digest = hashlib.file_digest(stream, "sha256").hexdigest() name = "freshell-source-probe-" + uuid.uuid4().hex created = False - receipt = {"image": IMAGE, "testArtifactSha256": digest, "test": TEST, + receipt = {"image": IMAGE, "testArtifactSha256": digest, "test": selected_test, "runId": name, "rootless": True} try: docker("create", "--name", name, "--label", f"{LABEL}={name}", @@ -52,13 +54,13 @@ def main(): "--security-opt", "no-new-privileges:true", "--pids-limit", "128", "--memory", "2g", "--mount", f"type=bind,src={binary},dst=/runtime/source-consumer-tests,readonly", "--tmpfs", "/fixture:rw,exec,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", - "--tmpfs", "/home/freshell/provider:rw,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", + "--tmpfs", f"/home/freshell/provider:rw,{'exec' if args.provider == 'codex' else 'noexec'},nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", "--tmpfs", "/tmp:rw,nosuid,nodev,noexec,size=256m,uid=65534,gid=0,mode=0700", "--env", "HOME=/home/freshell/provider", "--env", "AMPLIFIER_HOME=/home/freshell/provider/.amplifier", "--env", "FRESHELL_TEST_EXECUTABLE_TMP_DIR=/fixture", "--entrypoint", "/runtime/source-consumer-tests", IMAGE, - "--exact", TEST, "--nocapture", "--test-threads", "1") + "--exact", selected_test, "--nocapture", "--test-threads", "1") created = True result = docker("start", "--attach", name, check=False) receipt["exitCode"] = result.returncode @@ -69,7 +71,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != 18 or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != (18 if args.provider == "amplifier" else 4) or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. @@ -84,7 +86,7 @@ def main(): raise RuntimeError("source fixture cleanup ownership changed") docker("rm", "--force", value["Id"]) receipt["ownedCleanup"] = docker("container", "inspect", name, check=False).returncode != 0 - (args.receipt_dir / "task002-native-source-consumers.json").write_text(json.dumps(receipt, sort_keys=True) + "\n") + (args.receipt_dir / ("task002-native-source-consumers.json" if args.provider == "amplifier" else "task002-native-codex-source-consumers.json")).write_text(json.dumps(receipt, sort_keys=True) + "\n") if not created or not receipt["ownedCleanup"]: raise RuntimeError("source fixture failed its owned cleanup proof") raise SystemExit(receipt["exitCode"]) From 8f2df22f66ff9b14c965050ae010b79c5a301b55 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 6 Oct 2026 23:48:42 -0700 Subject: [PATCH 068/493] fix: bind initial provider resources to their provider identity --- .../src/native_auth_lookup_tests.rs | 63 +++++++ crates/freshell-supervisor/src/registry.rs | 170 +++++++++++++++++- .../src/registry/input_reservation.rs | 50 +++--- crates/freshell-supervisor/src/service.rs | 3 +- 4 files changed, 261 insertions(+), 25 deletions(-) diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index cca91cdd2..14c4c6517 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -12,6 +12,69 @@ use std::{ struct AuthBackend(AtomicUsize); +#[tokio::test] +async fn provider_ordinary_initial_reservation_cannot_authorize_opencode_auth_lookup() { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic helper artifact").unwrap(); + let registry = Registry::open(temp.path().join("registry"), None).unwrap(); + let backend = Arc::new(AuthBackend(AtomicUsize::new(0))); + let secret = "synthetic-control-secret"; + let supervisor = Supervisor::new( + registry, + backend.clone(), + SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: binary, + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "ordinary-input-reservation".into(), + control_secret: secret.into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let soul = SoulId::new(); + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::ReserveProviderInputs(ReserveProviderInputsRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul.clone(), + provider: "codex".into(), + creation_seed_ref: "synthetic-creation-seed".into(), + }), + ); + envelope.auth = Some(secret.into()); + let AdminResult::ProviderInputReservation(receipt) = + supervisor.dispatch(envelope).await.result.unwrap() + else { + panic!("unexpected reservation reply") + }; + assert_eq!(receipt.provider, "codex"); + let mut lookup = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::NativeAuthLookup(NativeAuthLookupRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul, + attempt_authority: receipt.reservation_id, + keys: vec!["PATH_INPUT".into()], + }), + ); + lookup.auth = Some(secret.into()); + assert_eq!( + supervisor.dispatch(lookup).await.result.unwrap_err().code, + RuntimeErrorCode::InvalidRequest + ); + assert_eq!( + backend.0.load(Ordering::SeqCst), + 1, + "ordinary creation authority must not perform or claim OpenCode auth lookup" + ); +} + #[tokio::test] async fn provider_initial_auth_requires_creation_reservation_before_absence_receipt() { let temp = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 2b3625fe2..2b6134c2c 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -17,7 +17,7 @@ use std::{ time::{SystemTime, UNIX_EPOCH}, }; -pub const SCHEMA_VERSION: u32 = 8; +pub const SCHEMA_VERSION: u32 = 9; mod input_reservation; pub use input_reservation::{BoundInputReservation, OwnedInputReservation}; const ACTIVE_STATES: &str = "'prepared','created','starting','running','stopping'"; @@ -2415,6 +2415,16 @@ fn migrate_schema(conn: &mut Connection) -> Result<(), RegistryError> { tx.commit()?; schema = 8; } + if schema == 8 { + let add_provider = !column_exists(conn, "provider_input_preparations", "provider")?; + let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?; + if add_provider { + tx.execute_batch("ALTER TABLE provider_input_preparations ADD COLUMN provider TEXT NOT NULL DEFAULT 'opencode';")?; + } + tx.execute_batch("UPDATE installation SET schema_version=9 WHERE singleton=1;")?; + tx.commit()?; + schema = 9; + } if schema != SCHEMA_VERSION { return Err(RegistryError::Integrity(format!( "unsupported older schema {schema}" @@ -2900,6 +2910,164 @@ mod tests { } } + #[tokio::test] + async fn provider_initial_reservation_migration_retains_legacy_opencode_authority() { + let dir = tempfile::tempdir().unwrap(); + let binary = dir.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic artifact").unwrap(); + let root = dir.path().join("registry"); + let registry = Registry::open(&root, None).unwrap(); + let request = freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: None, + soul_id: SoulId::new(), + provider: "opencode".into(), + creation_seed_ref: "seed".into(), + }; + let image = format!("sha256:{}", "a".repeat(64)); + let reservation = registry + .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) + .await + .unwrap(); + let observation = freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new(), + }; + let expected = registry + .record_initial_observation(reservation, observation.clone()) + .await + .unwrap(); + let conn = open_connection(®istry.inner.db_path).unwrap(); + conn.execute_batch("ALTER TABLE provider_input_preparations DROP COLUMN provider; UPDATE installation SET schema_version=8;").unwrap(); + drop(conn); + drop(registry); + let restored = Registry::open(&root, None).unwrap(); + let reservation = restored + .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) + .await + .unwrap(); + assert_eq!( + restored + .record_initial_observation(reservation, observation) + .await + .unwrap(), + expected + ); + let mut changed = request; + changed.provider = "codex".into(); + assert!( + matches!( + restored + .reserve_initial_inputs(changed, image, binary) + .await, + Err(RegistryError::RequestConflict) + ), + "legacy OpenCode reservations retain exact provider identity on reopen" + ); + } + + #[tokio::test] + async fn provider_ordinary_initial_reservation_preserves_provider_identity() { + for provider in ["codex", "claude", "amplifier"] { + let dir = tempfile::tempdir().unwrap(); + let binary = dir.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic artifact").unwrap(); + let registry = Registry::open(dir.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let request = freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: provider.into(), + creation_seed_ref: "seed".into(), + }; + let image = format!("sha256:{}", "a".repeat(64)); + let reserved = registry + .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) + .await + .expect("ordinary selected inputs require their own provider's creation authority"); + let retry = registry + .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) + .await + .unwrap(); + assert_eq!(reserved.id, retry.id); + let mut foreign = request.clone(); + foreign.provider = "opencode".into(); + assert!(matches!( + registry + .reserve_initial_inputs(foreign, image, binary) + .await, + Err(RegistryError::RequestConflict) + )); + let receipt = registry + .record_initial_observation( + reserved, + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new(), + }, + ) + .await + .unwrap(); + assert_eq!(receipt.provider, provider); + let mut launch = prep(soul.clone(), RequestId::new(), "ordinary-reserved-inputs"); + launch.provider = provider.into(); + // Reservation binding protects actual workload preparation, regardless + // of which finite provider's selected ordinary resources it owns. + launch.terminal = Some(serde_json::from_value(serde_json::json!({ + "terminalId":"synthetic-terminal", "streamId":"synthetic-stream", "projectKey":"test-project", "mode":provider, + "program":provider,"args":[],"env":{}, "cwd":"/synthetic/workspace", + "workspacePath":"/synthetic/workspace","cols":80,"rows":24,"runAsUid":65534,"runAsGid":0, + "providerLaunchContext": { + "preparation":match provider { + "codex" => serde_json::json!({"codex":{"tui_args":[],"sidecar_args":[]}}), + "claude" => serde_json::json!({"claude":{"mcp_args":[]}}), + "amplifier" => serde_json::json!({"amplifier":{"bundle":"default","resume_args":[]}}), + _ => unreachable!(), + }, + "mcpCapability":{"grantId":"grant-ordinary", "endpoint":"http://host.docker.internal:3001/api/mcp", "providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":"grant-ordinary", "initialAuthReservation":receipt.reservation_id, + "manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":0,"aliasCount":0,"recipeCount":0}} + } + })).unwrap()); + let mut wrong = launch.clone(); + wrong.provider = "opencode".into(); + let workload = wrong.terminal.as_mut().unwrap(); + workload.mode = "opencode".into(); + workload + .provider_launch_context + .as_mut() + .unwrap() + .preparation = freshell_runtime_protocol::ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }; + let wrong_result = registry.prepare_launch(wrong).await; + assert!( + matches!(wrong_result, Err(RegistryError::RequestConflict)), + "foreign provider bind outcome: {wrong_result:?}" + ); + let prepared = registry.prepare_launch(launch.clone()).await.unwrap(); + assert_eq!( + registry + .prepare_launch(launch) + .await + .unwrap() + .incarnation_id, + prepared.incarnation_id + ); + let bound = registry + .bound_input_reservation(prepared.incarnation_id) + .await + .unwrap() + .unwrap(); + assert_eq!(bound.reservation.soul_id, soul); + assert!(registry + .initial_input_reservation(soul, receipt.reservation_id) + .await + .unwrap() + .is_none()); + } + } + #[tokio::test] async fn provider_initial_input_reservation_binds_to_first_prepared_incarnation() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index 18f04b15c..5487b3f7f 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -1,4 +1,4 @@ -//! A creation reservation precedes any initial provider-auth absence claim. +//! Provider-bound creation authority for ordinary inputs and OpenCode auth lookup. use super::*; use freshell_runtime_protocol::{ InitialAuthVolumeObservation, ProviderInputReservation, ReserveProviderInputsRequest, @@ -8,6 +8,7 @@ use rusqlite::Transaction; #[derive(Debug, Clone)] pub struct OwnedInputReservation { pub(crate) id: String, + pub(crate) provider: String, pub(crate) installation_id: InstallationId, pub(crate) soul_id: SoulId, pub(crate) creation_seed_ref: String, @@ -36,7 +37,7 @@ pub(super) fn create_table(conn: &Connection) -> Result<(), RegistryError> { conn.execute_batch( "CREATE TABLE IF NOT EXISTS provider_input_preparations ( soul_id TEXT PRIMARY KEY, reservation_id TEXT NOT NULL UNIQUE, - creation_seed_ref TEXT NOT NULL, image_ref TEXT NOT NULL, + creation_seed_ref TEXT NOT NULL, provider TEXT NOT NULL DEFAULT 'opencode', image_ref TEXT NOT NULL, host_binary_path TEXT NOT NULL, host_binary_sha256 TEXT NOT NULL, intent_revision INTEGER NOT NULL CHECK(intent_revision=1), observation TEXT, consumed INTEGER NOT NULL DEFAULT 0 CHECK(consumed IN (0,1)), @@ -113,16 +114,16 @@ impl Registry { ) -> Result, RegistryError> { let installation = self.installation_id().clone(); self.run_blocking(move |conn| { - let row: Option<(String,String,String,String,String,String,String)> = conn.query_row( - "SELECT p.soul_id,p.reservation_id,p.creation_seed_ref,p.image_ref,p.host_binary_path,p.host_binary_sha256,p.observation FROM provider_input_preparations p JOIN incarnations i ON i.incarnation_id=p.bound_incarnation_id JOIN souls s ON s.soul_id=p.soul_id WHERE p.bound_incarnation_id=?1 AND p.consumed=1 AND i.launch_state='prepared' AND i.soul_id=p.soul_id AND s.desired_state='running' AND s.intent_revision=p.intent_revision AND s.provider='opencode' AND s.creation_seed_ref=p.creation_seed_ref", - params![incarnation_id.as_str()], |row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?)) + let row: Option<(String,String,String,String,String,String,String,String)> = conn.query_row( + "SELECT p.soul_id,p.reservation_id,p.creation_seed_ref,p.image_ref,p.host_binary_path,p.host_binary_sha256,p.observation,p.provider FROM provider_input_preparations p JOIN incarnations i ON i.incarnation_id=p.bound_incarnation_id JOIN souls s ON s.soul_id=p.soul_id WHERE p.bound_incarnation_id=?1 AND p.consumed=1 AND i.launch_state='prepared' AND i.soul_id=p.soul_id AND s.desired_state='running' AND s.intent_revision=p.intent_revision AND s.provider=p.provider AND s.creation_seed_ref=p.creation_seed_ref", + params![incarnation_id.as_str()], |row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?,row.get(7)?)) ).optional()?; - let Some((soul,id,seed,image,path,sha,observation))=row else {return Ok(None)}; + let Some((soul,id,seed,image,path,sha,observation,provider))=row else {return Ok(None)}; let binary=PathBuf::from(path); if binary_digest(&binary)?!=sha {return Err(RegistryError::InvalidState("initial auth source artifact changed before launch".into()));} let soul=SoulId::parse(soul).map_err(|_|RegistryError::Integrity("invalid reserved soul".into()))?; Ok(Some(BoundInputReservation { - reservation:OwnedInputReservation{id,provider_volume_name:stable_provider_volume_name(&installation,&soul),installation_id:installation,soul_id:soul,creation_seed_ref:seed,image_ref:image,binary,binary_sha256:sha}, + reservation:OwnedInputReservation{id,provider,provider_volume_name:stable_provider_volume_name(&installation,&soul),installation_id:installation,soul_id:soul,creation_seed_ref:seed,image_ref:image,binary,binary_sha256:sha}, incarnation_id,observation:serde_json::from_str(&observation)? })) }).await @@ -134,12 +135,12 @@ impl Registry { ) -> Result, RegistryError> { let installation = self.installation_id().clone(); self.run_blocking(move|conn|{ - let row:Option<(String,String,String,String,String)>=conn.query_row("SELECT creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,observation FROM provider_input_preparations WHERE soul_id=?1 AND reservation_id=?2 AND consumed=0 AND observation IS NOT NULL AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![soul.as_str(),id],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?))).optional()?; - let Some((seed,image,path,digest,observation))=row else{return Ok(None)}; + let row:Option<(String,String,String,String,String,String)>=conn.query_row("SELECT creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,observation,provider FROM provider_input_preparations WHERE soul_id=?1 AND reservation_id=?2 AND consumed=0 AND observation IS NOT NULL AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![soul.as_str(),id],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?))).optional()?; + let Some((seed,image,path,digest,observation,provider))=row else{return Ok(None)}; let binary=PathBuf::from(path); if binary_digest(&binary)?!=digest{return Err(RegistryError::InvalidState("initial auth source artifact changed; reprepare before dispatch".into()));} let volume=stable_provider_volume_name(&installation,&soul); - Ok(Some((OwnedInputReservation{id,installation_id:installation,soul_id:soul,creation_seed_ref:seed,provider_volume_name:volume,image_ref:image,binary,binary_sha256:digest},serde_json::from_str(&observation)?))) + Ok(Some((OwnedInputReservation{id,provider,installation_id:installation,soul_id:soul,creation_seed_ref:seed,provider_volume_name:volume,image_ref:image,binary,binary_sha256:digest},serde_json::from_str(&observation)?))) }).await } pub(crate) async fn reserve_initial_inputs( @@ -148,13 +149,16 @@ impl Registry { image_ref: String, binary: PathBuf, ) -> Result { - if request.provider != "opencode" - || request.creation_seed_ref.is_empty() + if !matches!( + request.provider.as_str(), + "opencode" | "codex" | "claude" | "amplifier" + ) || request.creation_seed_ref.is_empty() || request.creation_seed_ref.len() > 4096 || request.creation_seed_ref.chars().any(char::is_control) { return Err(RegistryError::InvalidState( - "initial input reservation requires a bounded OpenCode creation identity".into(), + "initial input reservation requires a bounded supported-provider creation identity" + .into(), )); } let installation = self.installation_id().clone(); @@ -163,18 +167,18 @@ impl Registry { let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; let exists:bool=tx.query_row("SELECT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![request.soul_id.as_str()],|row|row.get(0))?; if exists {return Err(RegistryError::InvalidState("initial source requires a creation reservation; existing souls use their owned recovery source".into()));} - let stored:Option<(String,String,String,String,String,i64)>=tx.query_row("SELECT reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,consumed FROM provider_input_preparations WHERE soul_id=?1",params![request.soul_id.as_str()],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?))).optional()?; - let id=if let Some((id,seed,image,path,digest,consumed))=stored { - if seed!=request.creation_seed_ref || image!=image_ref || Path::new(&path)!=binary || digest!=sha || consumed!=0 {return Err(RegistryError::RequestConflict);} + let stored:Option<(String,String,String,String,String,i64,String)>=tx.query_row("SELECT reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,consumed,provider FROM provider_input_preparations WHERE soul_id=?1",params![request.soul_id.as_str()],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?))).optional()?; + let id=if let Some((id,seed,image,path,digest,consumed,provider))=stored { + if provider!=request.provider || seed!=request.creation_seed_ref || image!=image_ref || Path::new(&path)!=binary || digest!=sha || consumed!=0 {return Err(RegistryError::RequestConflict);} id } else { let id=uuid::Uuid::new_v4().to_string(); - tx.execute("INSERT INTO provider_input_preparations (soul_id,reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,intent_revision) VALUES (?1,?2,?3,?4,?5,?6,1)",params![request.soul_id.as_str(),id,request.creation_seed_ref,image_ref,binary.to_string_lossy(),sha])?; + tx.execute("INSERT INTO provider_input_preparations (soul_id,reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,intent_revision,provider) VALUES (?1,?2,?3,?4,?5,?6,1,?7)",params![request.soul_id.as_str(),id,request.creation_seed_ref,image_ref,binary.to_string_lossy(),sha,request.provider])?; id }; let volume=stable_provider_volume_name(&installation,&request.soul_id); tx.commit()?; - Ok(OwnedInputReservation{id,installation_id:installation,soul_id:request.soul_id,creation_seed_ref:request.creation_seed_ref,provider_volume_name:volume,image_ref,binary,binary_sha256:sha}) + Ok(OwnedInputReservation{id,provider:request.provider,installation_id:installation,soul_id:request.soul_id,creation_seed_ref:request.creation_seed_ref,provider_volume_name:volume,image_ref,binary,binary_sha256:sha}) }).await } pub(crate) async fn record_initial_observation( @@ -185,10 +189,10 @@ impl Registry { let stored = serde_json::to_string(&observation)?; self.run_blocking(move|mut conn|{ let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; - let updated=tx.execute("UPDATE provider_input_preparations SET observation=?1 WHERE soul_id=?2 AND reservation_id=?3 AND consumed=0 AND host_binary_sha256=?4 AND (observation IS NULL OR observation=?1) AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?2)",params![stored,reservation.soul_id.as_str(),reservation.id,reservation.binary_sha256])?; + let updated=tx.execute("UPDATE provider_input_preparations SET observation=?1 WHERE soul_id=?2 AND reservation_id=?3 AND consumed=0 AND host_binary_sha256=?4 AND provider=?5 AND (observation IS NULL OR observation=?1) AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?2)",params![stored,reservation.soul_id.as_str(),reservation.id,reservation.binary_sha256,reservation.provider])?; if updated!=1 {return Err(RegistryError::RequestConflict);} tx.commit()?; - Ok(ProviderInputReservation{reservation_id:reservation.id,installation_id:reservation.installation_id,soul_id:reservation.soul_id,provider:"opencode".into(),creation_seed_ref:reservation.creation_seed_ref,intent_revision:1,provider_volume_name:reservation.provider_volume_name,image_ref:reservation.image_ref,host_binary_path:reservation.binary.to_string_lossy().into_owned(),host_binary_sha256:reservation.binary_sha256,observation}) + Ok(ProviderInputReservation{reservation_id:reservation.id,installation_id:reservation.installation_id,soul_id:reservation.soul_id,provider:reservation.provider,creation_seed_ref:reservation.creation_seed_ref,intent_revision:1,provider_volume_name:reservation.provider_volume_name,image_ref:reservation.image_ref,host_binary_path:reservation.binary.to_string_lossy().into_owned(),host_binary_sha256:reservation.binary_sha256,observation}) }).await } } @@ -216,12 +220,12 @@ pub(super) fn bind_in_transaction( let Some(id) = reservation_reference(input) else { return Ok(()); }; - if input.provider != "opencode" || input.native_session_id.is_some() { + if input.native_session_id.is_some() { return Err(RegistryError::RequestConflict); } let changed=tx.execute( - "UPDATE provider_input_preparations SET consumed=1,bound_incarnation_id=?1 WHERE soul_id=?2 AND reservation_id=?3 AND creation_seed_ref=?4 AND observation IS NOT NULL AND consumed=0 AND bound_incarnation_id IS NULL AND EXISTS(SELECT 1 FROM souls WHERE soul_id=?2 AND desired_state='running' AND intent_revision=1 AND provider='opencode' AND creation_seed_ref=?4)", - params![incarnation.as_str(),input.soul_id.as_str(),id,input.creation_seed_ref])?; + "UPDATE provider_input_preparations SET consumed=1,bound_incarnation_id=?1 WHERE soul_id=?2 AND reservation_id=?3 AND creation_seed_ref=?4 AND observation IS NOT NULL AND consumed=0 AND bound_incarnation_id IS NULL AND EXISTS(SELECT 1 FROM souls WHERE soul_id=?2 AND desired_state='running' AND intent_revision=1 AND provider=?5 AND creation_seed_ref=?4) AND provider=?5", + params![incarnation.as_str(),input.soul_id.as_str(),id,input.creation_seed_ref,input.provider])?; if changed != 1 { return Err(RegistryError::RequestConflict); } diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 7a8d04d9d..c14fe7fd4 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -692,7 +692,8 @@ impl Supervisor { .await .map_err(map_registry)? { - if reservation.image_ref != self.config.image_ref + if reservation.provider != "opencode" + || reservation.image_ref != self.config.image_ref || reservation.binary != self.config.host_binary_path { return Err(denied()); From 813c2d2298619512ac05abba7749e1c090c481b8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 00:56:05 -0700 Subject: [PATCH 069/493] fix: preserve native file aliases during provider input publication --- .../src/provider_environment.rs | 286 +++++++++++++++++- crates/freshell-session-host/src/main.rs | 1 - 2 files changed, 276 insertions(+), 11 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index b5d088454..0aebe9819 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -588,14 +588,14 @@ fn copy_environment_with_account_home( ) } -/// Directory subpaths mounted at native discovery aliases retain in-unit -/// relative links, while cross-unit links address the final physical HOME. +/// Mounted directory aliases retain in-unit relative links. Mounted regular +/// files retain their existing inode, so native aliases see the published bytes. pub fn copy_environment_for_mounts( selections: &[CopySelection], destination_root: &Path, excluded_canonical_sources: &[PathBuf], limits: CopyLimits, - mounted_directories: &[PathBuf], + mounted_roots: &[PathBuf], ) -> Result { let account_home = std::env::var_os("HOME").and_then(|home| fs::canonicalize(home).ok()); copy_environment_projection( @@ -604,7 +604,7 @@ pub fn copy_environment_for_mounts( excluded_canonical_sources, limits, account_home, - mounted_directories, + mounted_roots, ) } @@ -614,15 +614,15 @@ fn copy_environment_projection( excluded_canonical_sources: &[PathBuf], limits: CopyLimits, account_home: Option, - mounted_directories: &[PathBuf], + mounted_roots: &[PathBuf], ) -> Result { bound( Path::new(""), "entries", - mounted_directories.len() as u64, + mounted_roots.len() as u64, limits.max_entries as u64, )?; - for root in mounted_directories { + for root in mounted_roots { if !valid_relative(root) || !destination_root.is_absolute() { return Err(error( root, @@ -717,6 +717,37 @@ fn copy_environment_projection( Err(cause) if cause.kind() == std::io::ErrorKind::NotFound => false, Err(cause) => return Err(io_error(Path::new(""), "inspect_destination", cause)), }; + let mut prior_mounted_bytes = 0u64; + for root in mounted_roots.iter().collect::>() { + if !matches!(plan.nodes.get(root), Some(Node::File(..))) { + continue; + } + let metadata = fs::symlink_metadata(destination_root.join(root)) + .map_err(|cause| io_error(root, "inspect_mounted_file", cause))?; + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err(error( + root, + "inspect_mounted_file", + "mounted resource must retain an owned physical regular inode", + )); + } + bound(root, "file_bytes", metadata.len(), limits.max_file_bytes)?; + prior_mounted_bytes = prior_mounted_bytes + .checked_add(metadata.len()) + .ok_or_else(|| { + error( + root, + "inspect_mounted_file", + "prior mounted byte accounting overflow", + ) + })?; + bound( + root, + "total_bytes", + prior_mounted_bytes, + limits.max_total_bytes, + )?; + } let parent = if existing_root { destination_root } else { @@ -790,9 +821,7 @@ fn copy_environment_projection( mode(&destination, private_file_mode(*source_mode), relative)?; } Node::Link(target) => { - let mounted = mounted_directories - .iter() - .find(|root| relative.starts_with(root)); + let mounted = mounted_roots.iter().find(|root| relative.starts_with(root)); let target = if mounted.is_some_and(|root| !target.starts_with(root)) { destination_root.join(target) } else { @@ -892,6 +921,21 @@ fn copy_environment_projection( .map_err(|cause| io_error(relative, "publish_directory", cause))?; } mode(&destination, 0o700, relative)?; + } else if mounted_roots.contains(relative) && matches!(node, Node::File(..)) { + let Node::File(_, source_mode, _) = node else { + unreachable!() + }; + publish_mounted_file( + &temporary.0.join(relative), + &destination, + &temporary.0.join(format!( + ".freshell-mounted-backup-{}", + NEXT_PROJECTION.fetch_add(1, Ordering::Relaxed) + )), + relative, + *source_mode, + limits, + )?; } else { fs::rename(temporary.0.join(relative), destination) .map_err(|cause| io_error(relative, "publish_leaf", cause))?; @@ -906,8 +950,230 @@ fn copy_environment_projection( }) } +fn publish_mounted_file( + source: &Path, + destination: &Path, + backup: &Path, + relative: &Path, + source_mode: u32, + limits: CopyLimits, +) -> Result<(), CopyError> { + use std::io::{Seek, SeekFrom}; + #[cfg(unix)] + use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; + let metadata = fs::symlink_metadata(destination) + .map_err(|cause| io_error(relative, "inspect_mounted_file", cause))?; + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err(error( + relative, + "inspect_mounted_file", + "mounted resource must retain an owned physical regular inode", + )); + } + bound( + relative, + "file_bytes", + metadata.len(), + limits.max_file_bytes, + )?; + let mut options = OpenOptions::new(); + options.read(true).write(true); + #[cfg(unix)] + options.custom_flags(libc::O_NOFOLLOW); + let mut target = options + .open(destination) + .map_err(|cause| io_error(relative, "open_mounted_file", cause))?; + #[cfg(unix)] + { + let opened = target + .metadata() + .map_err(|cause| io_error(relative, "inspect_mounted_file", cause))?; + if opened.dev() != metadata.dev() || opened.ino() != metadata.ino() { + return Err(error( + relative, + "inspect_mounted_file", + "mounted resource inode changed before publication", + )); + } + } + // Preserve the previous selected bytes before modifying an inode that is + // already bound into the worker. Both copies are streamed on its own HOME. + let mut saved = OpenOptions::new() + .read(true) + .write(true) + .create_new(true) + .open(backup) + .map_err(|cause| io_error(relative, "backup_mounted_file", cause))?; + mode(backup, 0o600, relative)?; + let observed = std::io::copy( + &mut (&mut target).take(metadata.len().checked_add(1).ok_or_else(|| { + error( + relative, + "backup_mounted_file", + "mounted file size overflow", + ) + })?), + &mut saved, + ) + .map_err(|cause| io_error(relative, "backup_mounted_file", cause))?; + if observed != metadata.len() { + return Err(error( + relative, + "backup_mounted_file", + "mounted resource changed before publication; retry with a quiescent owner", + )); + } + let result = (|| -> std::io::Result<()> { + let incoming = File::open(source)?; + let expected = incoming.metadata()?.len(); + if expected > limits.max_file_bytes { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "staged mounted file exceeds copy profile", + )); + } + let maximum = expected.checked_add(1).ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidData, "staged file size overflow") + })?; + target.seek(SeekFrom::Start(0))?; + let copied = std::io::copy(&mut incoming.take(maximum), &mut target)?; + if copied != expected { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "staged mounted file changed", + )); + } + target.set_len(copied)?; + target.sync_all()?; + mode(destination, private_file_mode(source_mode), relative).map_err(|_| { + std::io::Error::new( + std::io::ErrorKind::PermissionDenied, + "mounted file mode publication failed", + ) + })?; + Ok(()) + })(); + if let Err(cause) = result { + let restore = (|| -> std::io::Result<()> { + saved.seek(SeekFrom::Start(0))?; + target.seek(SeekFrom::Start(0))?; + let restored = std::io::copy(&mut saved, &mut target)?; + target.set_len(restored)?; + target.sync_all()?; + fs::set_permissions(destination, metadata.permissions())?; + Ok(()) + })(); + if let Err(restore) = restore { + return Err(io_error(relative, "restore_mounted_file", restore)); + } + return Err(io_error(relative, "publish_mounted_file", cause)); + } + Ok(()) +} + #[cfg(test)] mod tests { + #[test] + fn provider_environment_mounted_file_alias_reads_complete_publication_on_same_inode() { + use std::io::{Read, Seek}; + let fixture = tempfile::tempdir().unwrap(); + let source = fixture.path().join("selected-instructions"); + fs::write(&source, b"complete selected instructions").unwrap(); + let home = fixture.path().join("provider-home"); + let leaf = PathBuf::from(".freshell/provider-roots/root-0/source"); + fs::create_dir_all(home.join(leaf.parent().unwrap())).unwrap(); + fs::write(home.join(&leaf), b"").unwrap(); + // Docker binds the already owned regular inode at a native alias + // before the session host publishes its complete selected inputs. + let mut alias = fs::File::open(home.join(&leaf)).unwrap(); + let selection = [CopySelection { + source: source.clone(), + destination_relative: leaf.clone(), + }]; + for content in [ + b"complete selected instructions".as_slice(), + b"replacement selected instructions".as_slice(), + ] { + fs::write(&source, content).unwrap(); + copy_environment_for_mounts( + &selection, + &home, + &[], + CopyLimits::default(), + std::slice::from_ref(&leaf), + ) + .unwrap(); + alias.rewind().unwrap(); + let mut observed = Vec::new(); + alias.read_to_end(&mut observed).unwrap(); + assert_eq!(observed,content,"an existing native file alias must see complete initial and refreshed resource bytes"); + } + } + + #[cfg(unix)] + #[test] + fn provider_environment_mounted_file_invalid_target_and_profile_failure_preserve_prior_bytes() { + let fixture = tempfile::tempdir().unwrap(); + let source = fixture.path().join("source"); + fs::write(&source, b"x").unwrap(); + let home = fixture.path().join("provider-home"); + let leaf = PathBuf::from(".freshell/provider-roots/root-0/source"); + fs::create_dir_all(home.join(leaf.parent().unwrap())).unwrap(); + fs::write(home.join(&leaf), b"prior selected bytes").unwrap(); + let selection = [CopySelection { + source: source.clone(), + destination_relative: leaf.clone(), + }]; + let limits = CopyLimits { + max_total_bytes: 2, + ..Default::default() + }; + assert!(copy_environment_for_mounts( + &selection, + &home, + &[], + limits, + std::slice::from_ref(&leaf) + ) + .is_err()); + assert_eq!(fs::read(home.join(&leaf)).unwrap(), b"prior selected bytes"); + let outside = fixture.path().join("unselected"); + fs::write(&outside, b"unselected original bytes").unwrap(); + fs::remove_file(home.join(&leaf)).unwrap(); + std::os::unix::fs::symlink(&outside, home.join(&leaf)).unwrap(); + assert!(copy_environment_for_mounts( + &selection, + &home, + &[], + Default::default(), + std::slice::from_ref(&leaf) + ) + .is_err()); + assert_eq!(fs::read(&outside).unwrap(), b"unselected original bytes"); + assert!(fs::symlink_metadata(home.join(&leaf)) + .unwrap() + .file_type() + .is_symlink()); + } + + #[test] + fn provider_environment_shared_mounted_file_aliases_use_one_prior_byte_budget() { + let fixture = tempfile::tempdir().unwrap(); + let source = fixture.path().join("source"); + fs::write(&source, b"y").unwrap(); + let home = fixture.path().join("provider-home"); + let leaf = PathBuf::from(".freshell/provider-roots/root-0/source"); + fs::create_dir_all(home.join(leaf.parent().unwrap())).unwrap(); + fs::write(home.join(&leaf), b"x").unwrap(); + let selections = [CopySelection { + source, + destination_relative: leaf.clone(), + }]; + copy_environment_for_mounts(&selections,&home,&[],CopyLimits {max_total_bytes:1,..Default::default()},&[leaf.clone(),leaf.clone(),leaf.clone()]) + .expect("three native aliases to one physical resource must retain the full shared byte capacity"); + assert_eq!(fs::read(home.join(leaf)).unwrap(), b"y"); + } + use super::*; #[cfg(unix)] use std::os::unix::fs::{symlink, PermissionsExt}; diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 8501d81e1..ec21acb40 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1760,7 +1760,6 @@ fn resource_mount_roots( .prepared_inputs .iter() .flat_map(|inputs| &inputs.resource_mounts) - .filter(|root| root.kind == freshell_runtime_protocol::ProviderResourceKind::Directory) .map(|root| PathBuf::from(&root.provider_relative_path)) .collect() } From 70f4ee578e8c2924ae85b625faf66f09971cb628 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 00:56:14 -0700 Subject: [PATCH 070/493] fix: prepare owned Codex inputs for terminal and fresh launches --- .../freshell-server/src/fresh_agent_proxy.rs | 19 +- .../src/managed_mcp_capability.rs | 7 +- .../src/managed_mcp_capability/codex_stage.rs | 3 +- .../src/managed_provider_launch_tests.rs | 320 ++++++++++++++++++ .../src/managed_provider_references/codex.rs | 46 ++- crates/freshell-server/src/managed_runtime.rs | 21 +- .../testing/managed-provider-launch-broker.ts | 6 +- .../managed-provider-launch-sandbox.py | 40 ++- 8 files changed, 417 insertions(+), 45 deletions(-) diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index ef80056a4..8ecae40d6 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -505,14 +505,19 @@ impl HostedFreshAgentProxy { .ok_or(())?; let original = |name: &str| std::env::var(name).ok(); let context = crate::managed_provider_references::codex::prepare_managed( - &soul, - &home, - &cwd, - &workspace, - &original, - &[], - &provider_secret_references, + crate::managed_provider_references::codex::CodexPreparation { + client: &self.client, + soul: &soul, + creation_seed: &message.request_id, + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &original, + args: &[], + references: &provider_secret_references, + }, ) + .await .map_err(|error| { tracing::warn!(event="fresh_agent.create.provider_inputs_failed",%error); })?; diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index fc4d72493..2a63b003a 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1431,7 +1431,12 @@ fn stage_provider_root( provider_relative_path: CAPABILITY_PATH.into(), host_gateway_address: None, }; - let inputs = codex_stage::publish(&temporary, resources, &capability)?; + let inputs = codex_stage::publish( + &temporary, + resources, + &capability, + opencode_input.initial_auth_reservation, + )?; write_private_bytes( &temporary.join(".freshell/prepared-inputs.json"), &freshell_runtime_protocol::bounded_json_bytes( diff --git a/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs b/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs index 1861d4625..c17642012 100644 --- a/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs @@ -149,6 +149,7 @@ pub(super) fn publish( root: &Path, preparation: &Preparation, capability: &McpCapabilityReference, + initial_reservation: Option<&str>, ) -> Result { use sha2::{Digest, Sha256}; private_directory(&root.join(".freshell"))?; @@ -157,7 +158,7 @@ pub(super) fn publish( write_private_bytes(&root.join(".freshell/input-manifest.json"), &bytes)?; let inputs = PreparedProviderInputs { generation_id: capability.grant_id.clone(), - initial_auth_reservation: None, + initial_auth_reservation: initial_reservation.map(str::to_string), manifest: ProviderInputManifestReference { relative_path: ".freshell/input-manifest.json".into(), sha256: format!("{:x}", Sha256::digest(&bytes)), diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 60c9f2ca7..6b4630eb1 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -215,6 +215,326 @@ impl RuntimeBackend for BrokerAuthorizedBackend { } } +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn managed_provider_codex_terminal_and_fresh_native_startup_consumes_prepared_inputs() { + use freshell_supervisor::{ + admission::AdmissionPolicy, + registry::Registry, + service::{serve_control, Supervisor, SupervisorConfig}, + }; + use freshell_terminal::registry::{ManagedTerminalController, ManagedTerminalLaunch}; + use freshell_ws::hosted_fresh_agent::{HostedFreshAgentCommand, HostedFreshAgentGateway}; + use sha2::{Digest, Sha256}; + use std::{os::unix::fs::PermissionsExt, sync::Arc}; + let fixture = PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_FIXTURE") + .expect("run the closed --provider-launches --provider codex sandbox"), + ); + assert_eq!(fixture.parent(), Some(Path::new("/tmp"))); + assert!(fixture + .file_name() + .unwrap() + .to_str() + .unwrap() + .starts_with("fln-")); + let home = fixture.join("source-home"); + let workspace = fixture.join("workspace"); + let control = fixture.join("scenarios/native/control"); + let runtime = fixture.join("scenarios/n/r"); + let skill = home.join(".codex/skills/probe"); + for path in [ + &control, + &runtime, + &skill.join("scripts/node_modules/probe-dependency"), + &workspace.join(".codex/agents"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + assert!(std::process::Command::new("git") + .args(["init", "--quiet"]) + .arg(&workspace) + .status() + .unwrap() + .success()); + std::fs::write(skill.join("SKILL.md"), "---\nname: transport-probe\ndescription: complete launch fixture\n---\nExecute the adjacent synthetic resource.\n").unwrap(); + std::fs::write(skill.join("scripts/resource"), b"closed-codex-resource").unwrap(); + std::fs::write( + skill.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='closed-codex-dependency'\n", + ) + .unwrap(); + std::fs::write( + skill.join("scripts/run"), + "#!/bin/sh\ncat \"$(dirname \"$0\")/resource\"\n", + ) + .unwrap(); + std::fs::set_permissions( + skill.join("scripts/run"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + let receipt = workspace.join("codex-consumed.json"); + let server = r#"const fs=require('node:fs'),crypto=require('node:crypto'),http=require('node:http'),path=require('node:path'); +const digest=v=>crypto.createHash('sha256').update(v).digest('hex'); +const readline=require('node:readline').createInterface({input:process.stdin}); +readline.on('line',async line=>{let message;try{message=JSON.parse(line)}catch{return}if(message.id===undefined)return; +let result={};if(message.method==='initialize'){ + const endpoint=process.env.FRESHELL_URL,token=process.env.FRESHELL_TOKEN; + const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); + const nativeArgv=fs.readFileSync('/proc/'+process.ppid+'/cmdline','utf8').split('\0'); + const retainedMcpRecipe=nativeArgv.includes('mcp_servers.freshell.command="node"')&&nativeArgv.includes('mcp_servers.freshell.args=["/opt/freshell-mcp/server.js"]')&&nativeArgv.some(arg=>arg.startsWith('mcp_servers.freshell.env_vars=')); + const record={physicalHome:process.env.HOME,initialized:true,authenticated,retainedMcpRecipe,scopedAlias:process.env.FRESHELL==='1',endpointPresent:!!endpoint,tokenPresent:!!token, + user:digest(fs.readFileSync(path.join(process.env.CODEX_HOME,'user.md'))),project:digest(fs.readFileSync(path.join(process.cwd(),'.codex/project.md'))), + role:digest(fs.readFileSync(path.join(process.cwd(),'.codex/agents/role.md'))),skill:digest(fs.readFileSync(path.join(__dirname,'../SKILL.md'))), + resource:digest(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:digest(require('probe-dependency'))}; + fs.writeFileSync(process.argv[2],JSON.stringify(record));result={protocolVersion:'2024-11-05',capabilities:{tools:{}},serverInfo:{name:'closed-managed-source-consumer',version:'1'}}; +}else if(message.method==='tools/list')result={tools:[]};else if(message.method==='resources/list')result={resources:[]};else if(message.method==='prompts/list')result={prompts:[]}; +process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n');}); +"#; + std::fs::write(skill.join("scripts/server.cjs"), server).unwrap(); + let config = format!("model='synthetic-model'\nmodel_provider='fixture'\nmodel_instructions_file='./user.md'\n[model_providers.fixture]\nname='Closed fixture'\nbase_url='http://127.0.0.1:9'\nwire_api='responses'\nrequires_openai_auth=false\n[projects.{}]\ntrust_level='trusted'\n[mcp_servers.probe]\ncommand='node'\nenv_vars=['FRESHELL_URL','FRESHELL_TOKEN','FRESHELL','CODEX_HOME']\nargs=[{},{}]\n", serde_json::to_string(workspace.to_str().unwrap()).unwrap(), serde_json::to_string(skill.join("scripts/server.cjs").to_str().unwrap()).unwrap(), serde_json::to_string(receipt.to_str().unwrap()).unwrap()); + std::fs::write(home.join(".codex/config.toml"), &config).unwrap(); + std::fs::write(home.join(".codex/user.md"), b"closed-user-input").unwrap(); + let project_config = "model_instructions_file='./project.md'\n[agents.worker]\nconfig_file='./agents/worker.toml'\n"; + std::fs::write(workspace.join(".codex/config.toml"), project_config).unwrap(); + std::fs::write(workspace.join(".codex/project.md"), b"closed-project-input").unwrap(); + std::fs::write(workspace.join(".codex/agents/worker.toml"), "name='worker'\ndescription='synthetic role'\ndeveloper_instructions='synthetic developer'\nmodel_instructions_file='./role.md'\n").unwrap(); + std::fs::write( + workspace.join(".codex/agents/role.md"), + b"closed-role-input", + ) + .unwrap(); + let registry = Registry::open( + fixture.join("registry"), + Some( + InstallationId::parse(std::env::var("FRESHELL_PROVIDER_LAUNCH_INSTALLATION").unwrap()) + .unwrap(), + ), + ) + .unwrap(); + std::env::set_var("FRESHELL_MANAGED_PROVIDER_QUALIFICATION", "codex"); + let socket = control.join("control.sock"); + let secret_file = control.join("secret"); + std::fs::write(&secret_file, "closed-synthetic-control-secret").unwrap(); + std::fs::set_permissions(&secret_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + freshell_supervisor::backend::docker::configure_capability_dir(&socket).unwrap(); + let supervisor = Supervisor::new( + registry.clone(), + Arc::new(BrokerAuthorizedBackend { + real: DockerEngineBackend::new(fixture.join("broker.sock")), + control: control.clone(), + handoff: fixture.join("handoff"), + certificate_sentinel: b"unselected-private-sentinel".to_vec(), + }), + SupervisorConfig { + runtime_root: runtime, + control_socket_path: socket.clone(), + host_binary_path: PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_HOST_BINARY").unwrap(), + ), + image_ref: std::env::var("FRESHELL_PROVIDER_LAUNCH_IMAGE").unwrap(), + test_run_id: std::env::var("FRESHELL_PROVIDER_LAUNCH_RUN_ID").unwrap(), + control_secret: "closed-synthetic-control-secret".into(), + lifecycle_log: fixture.join("lifecycle.jsonl"), + admission: AdmissionPolicy::default(), + }, + ) + .unwrap(); + let control_task = tokio::spawn({ + let socket = socket.clone(); + async move { + serve_control(supervisor, &socket).await.unwrap(); + } + }); + tokio::time::timeout(Duration::from_secs(5), async { + while !socket.exists() { + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .unwrap(); + let web = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap(); + let port = web.local_addr().unwrap().port(); + let web_token = Arc::new("closed-synthetic-web-token".to_string()); + let app = axum::Router::new() + .route("/api/health", axum::routing::get(|| async { "ok" })) + .route( + "/api/mcp", + axum::routing::post(|headers: axum::http::HeaderMap| async move { + if headers + .get("x-auth-token") + .is_some_and(|token| token == "closed-synthetic-web-token") + { + axum::http::StatusCode::OK + } else { + axum::http::StatusCode::UNAUTHORIZED + } + }), + ) + .layer(axum::middleware::from_fn_with_state( + web_token, + crate::managed_mcp_capability::scoped_auth_middleware, + )); + let web_task = tokio::spawn(async move { axum::serve(web, app).await.unwrap() }); + crate::managed_mcp_capability::configure("0.0.0.0", port, &socket, &secret_file).unwrap(); + crate::managed_mcp_capability::spawn_callback_server() + .await + .unwrap(); + for (name, value) in [ + ("FRESHELL_MANAGED_RUNTIME_V1", "1"), + ("FRESHELL_MANAGED_FRESH_AGENT_V1", "1"), + ] { + std::env::set_var(name, value); + } + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SOCKET", &socket); + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SECRET_FILE", &secret_file); + std::env::set_var("HOME", &home); + let controller = crate::managed_runtime::ServerManagedRuntimeController::from_env() + .await + .unwrap() + .unwrap(); + let cwd = workspace.to_str().unwrap().to_owned(); + let terminal = controller + .launch(ManagedTerminalLaunch { + spec: freshell_platform::spawn::SpawnSpec { + program: "codex".into(), + args: vec![], + env_overrides: BTreeMap::new(), + cwd: Some(cwd.clone()), + cols: 120, + rows: 30, + }, + env: BTreeMap::new(), + terminal_id: "closed-native-terminal".into(), + stream_id: "closed-native-stream".into(), + mode: "codex".into(), + resume_session_id: None, + provider_model: None, + provider_reasoning_effort: None, + provider_sandbox: None, + provider_permission_mode: None, + view_tab_id: None, + view_pane_id: None, + create_request_id: Some("closed-native-terminal-create".into()), + }) + .await + .expect("actual Codex controller/supervisor/session-host terminal startup"); + let client = controller.runtime_client(); + async fn consumed(path: &Path) -> Result { + tokio::time::timeout(Duration::from_secs(45), async { + loop { + if let Ok(bytes) = std::fs::read(path) { + if let Ok(value) = serde_json::from_slice(&bytes) { + break value; + } + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await + .map_err(|_| "native Codex must initialize its configured MCP consumer") + } + let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); + let expected = json!({"physicalHome":"/home/freshell/provider","initialized":true,"authenticated":true,"retainedMcpRecipe":true,"scopedAlias":true,"endpointPresent":true,"tokenPresent":true,"user":hash(b"closed-user-input"),"project":hash(b"closed-project-input"),"role":hash(b"closed-role-input"),"skill":hash(std::fs::read(skill.join("SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-codex-resource"),"dependency":hash(b"closed-codex-dependency")}); + let consumed_terminal = consumed(&receipt).await; + if consumed_terminal.is_err() { + let output = client + .read_output( + SoulId::parse(terminal.soul_id.clone()).unwrap(), + 0, + 64 * 1024, + ) + .await + .unwrap(); + let text = output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(); + let classes = [ + "ERR_INVALID_ARG_TYPE", + "ENOENT", + "EACCES", + "onboarding", + "Sign in", + "trust", + "API key", + "MCP", + "mcp", + "initialize", + "failed", + "Permission denied", + "Terminal", + "TERM", + "user.md", + "project.md", + "role.md", + "server.cjs", + "node", + "config.toml", + ] + .into_iter() + .filter(|class| text.contains(class)) + .collect::>(); + println!( + "provider_launch_diagnostic {}", + json!({"provider":"codex","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes}) + ); + } + assert_eq!(consumed_terminal.unwrap(), expected); + println!( + "provider_launch_consumer {}", + json!({"provider":"codex","route":"terminal","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + ); + client + .stop(SoulId::parse(terminal.soul_id).unwrap()) + .await + .unwrap(); + std::fs::remove_file(&receipt).unwrap(); + let (broadcast, _) = tokio::sync::broadcast::channel(256); + let mut events = broadcast.subscribe(); + let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( + Some(client.clone()), + Arc::new(broadcast), + ) + .unwrap() + .unwrap(); + Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":"freshcodex","provider":"codex","cwd":cwd})).unwrap())).await; + tokio::time::timeout(Duration::from_secs(45), async { + loop { + let frame: Value = serde_json::from_str(&events.recv().await.unwrap()).unwrap(); + if frame["type"] == "freshAgent.create.failed" { + panic!("actual fresh Codex failed: {}", frame["message"]); + } + if frame["type"] == "freshAgent.created" { + break; + } + } + }) + .await + .unwrap(); + assert_eq!(consumed(&receipt).await.unwrap(), expected); + println!( + "provider_launch_consumer {}", + json!({"provider":"codex","route":"fresh","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + ); + assert_eq!( + std::fs::read_to_string(home.join(".codex/config.toml")).unwrap(), + config + ); + assert_eq!( + std::fs::read_to_string(workspace.join(".codex/config.toml")).unwrap(), + project_config + ); + for view in client.inventory().await.unwrap() { + if view.desired_state == DesiredState::Running { + client.stop(view.soul_id).await.unwrap(); + } + } + web_task.abort(); + control_task.abort(); +} + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_prepared_inputs() { use freshell_supervisor::{ diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 40185216a..234aa56b8 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -306,15 +306,32 @@ fn normalize(path: &Path) -> PathBuf { result } -pub(crate) fn prepare_managed( - soul: &freshell_runtime_protocol::SoulId, - home: &Path, - cwd: &Path, - workspace: &Path, - original: &dyn Fn(&str) -> Option, - args: &[String], - references: &[freshell_runtime_protocol::ProviderSecretReference], +pub(crate) struct CodexPreparation<'a> { + pub client: &'a freshell_runtime_client::RuntimeClient, + pub soul: &'a freshell_runtime_protocol::SoulId, + pub creation_seed: &'a str, + pub home: &'a Path, + pub cwd: &'a Path, + pub workspace: &'a Path, + pub original: &'a (dyn Fn(&str) -> Option + Sync), + pub args: &'a [String], + pub references: &'a [freshell_runtime_protocol::ProviderSecretReference], +} + +pub(crate) async fn prepare_managed( + request: CodexPreparation<'_>, ) -> Result { + let CodexPreparation { + client, + soul, + creation_seed, + home, + cwd, + workspace, + original, + args, + references, + } = request; let codex_home = if let Some(selected) = original("CODEX_HOME").filter(|value| !value.is_empty()) { let selected = PathBuf::from(selected); @@ -356,6 +373,18 @@ pub(crate) fn prepare_managed( mcp_capability: None, prepared_inputs: None, }; + // Ordinary Codex resource roots require exact first-volume authority. + // This reservation makes no OpenCode native-auth or recovery-loss claim. + let reservation = client + .reserve_provider_inputs(soul.clone(), "codex".into(), creation_seed.into()) + .await + .map_err(|error| error.to_string())?; + if reservation.provider != "codex" + || &reservation.soul_id != soul + || reservation.creation_seed_ref != creation_seed + { + return Err("Codex selected resource reservation does not match this creation".into()); + } crate::managed_mcp_capability::issue_prepared_provider_context( "codex", soul, @@ -366,6 +395,7 @@ pub(crate) fn prepare_managed( cwd: Some(cwd), workspace: Some(workspace), secret_references: Some(references), + initial_auth_reservation: Some(&reservation.reservation_id), ..Default::default() }, ) diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 548a237c7..6598f9ae6 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -240,14 +240,19 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .or_else(|| std::env::var(name).ok()) }; let context = crate::managed_provider_references::codex::prepare_managed( - &soul_id, - &home, - &cwd, - &workspace, - &original, - &args, - &provider_secret_references, - )?; + crate::managed_provider_references::codex::CodexPreparation { + client: &self.client, + soul: &soul_id, + creation_seed: create_key, + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &original, + args: &args, + references: &provider_secret_references, + }, + ) + .await?; (context.mcp_capability.clone(), Some(context)) } else { let mcp_capability = matches!( diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 04bca884e..9ba50aa89 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -10,7 +10,7 @@ if (path.dirname(root)!=='/tmp' || !/^fln-[a-f0-9]{16}$/.test(path.basename(root } const configuration=JSON.parse(fs.readFileSync(path.join(root,'broker-configuration.json'),'utf8')) const image='sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c' -if (configuration.imageRef!==image || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==2) { +if (configuration.imageRef!==image || !['opencode','codex'].includes(configuration.provider) || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==2) { throw new Error('closed provider startup image or finite source set changed') } const gateways=new Set() @@ -19,7 +19,9 @@ const broker=new RestrictedDockerBroker({ runtimeRootPrefix:path.join(root,'scenarios'),allowedHostBinaryPaths:new Set([configuration.hostBinaryPath]), allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), - allowedProviderUserRoots:new Set(['/home/freshell/provider',path.join(root,'.opencode')]), + allowedProviderUserRoots:new Set(configuration.provider==='codex' + ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] + : ['/home/freshell/provider',path.join(root,'.opencode')]), allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json'),path.join(root,'grants/ca.pem')]), allowedHostGatewayAddresses:gateways, testRunId:configuration.runId,logPath:path.join(root,'broker.jsonl'), diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 554c2e1aa..f31d22d95 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -12,6 +12,7 @@ IMAGE="sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" TEST="managed_provider_launch_tests::managed_provider_opencode_terminal_and_fresh_native_startup_consumes_prepared_inputs" +CODEX_TEST="managed_provider_launch_tests::managed_provider_codex_terminal_and_fresh_native_startup_consumes_prepared_inputs" LABEL="com.freshell.provider-launch-probe" def docker(*args,check=True,timeout=180): @@ -27,13 +28,15 @@ def main(): parser.add_argument("--host-binary",type=Path,required=True) parser.add_argument("--broker-binary",type=Path,required=True) parser.add_argument("--receipt-dir",type=Path,required=True) + parser.add_argument("--provider",choices=["opencode","codex"],default="opencode") args=parser.parse_args() + selected_test=CODEX_TEST if args.provider=="codex" else TEST repo=Path(__file__).resolve().parents[2] test=args.test_binary.resolve(strict=True);host=args.host_binary.resolve(strict=True);broker=args.broker_binary.resolve(strict=True) if test.parent!=repo/"target/debug/deps" or not re.fullmatch(r"freshell_server-[a-f0-9]{16}",test.name) or host!=repo/"target/debug/freshell-session-host" or broker!=repo/"target/managed-provider-launch-broker.mjs" or not args.receipt_dir.is_dir(): raise RuntimeError("closed startup requires this worktree's exact compiled artifacts and existing receipts") - listing=subprocess.run([str(test),"--exact",TEST,"--list"],env={"PATH":"/usr/bin:/bin"},text=True,capture_output=True,timeout=20,check=True) - if f"{TEST}: test" not in listing.stdout.splitlines(): + listing=subprocess.run([str(test),"--exact",selected_test,"--list"],env={"PATH":"/usr/bin:/bin"},text=True,capture_output=True,timeout=20,check=True) + if f"{selected_test}: test" not in listing.stdout.splitlines(): raise RuntimeError("closed startup compiled test is missing") info=json.loads(docker("info","--format","{{json .}}").stdout) if "name=rootless" not in info["SecurityOptions"] or docker("image","inspect","--format","{{.Id}}",IMAGE).stdout.strip()!=IMAGE: @@ -42,7 +45,7 @@ def main(): if not endpoint.startswith("unix://"): raise RuntimeError("closed startup requires the existing local rootless daemon IPC") run="freshell-launch-"+uuid.uuid4().hex;installation=str(uuid.uuid4());owned=[] - receipt={"runId":run,"image":IMAGE,"test":TEST,"rootless":True,"testArtifactSha256":digest(test),"hostArtifactSha256":digest(host),"brokerArtifactSha256":digest(broker)} + receipt={"runId":run,"image":IMAGE,"test":selected_test,"provider":args.provider,"rootless":True,"testArtifactSha256":digest(test),"hostArtifactSha256":digest(host),"brokerArtifactSha256":digest(broker)} volumes=set() def remove(name): result=docker("container","inspect",name,check=False) @@ -80,22 +83,23 @@ def create(name,*arguments): for name in ["workspace","source-home/.config/opencode","grants","handoff","tmp","scenarios"]: (root/name).mkdir(parents=True,exist_ok=True) initial=[] - for soul in ["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32],"soul-fresh-opencode-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]]: + for soul in ["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32],"soul-fresh-"+args.provider+"-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]]: volume="freshell-provider-"+hashlib.sha256((installation+"\0"+soul).encode()).hexdigest()[:24] volumes.add(volume) initial.append({"installationId":installation,"soulId":soul,"providerVolumeName":volume,"imageRef":IMAGE,"hostBinaryPath":str(host),"binarySha256":digest(host),"daemonId":info["ID"]}) - (root/"broker-configuration.json").write_text(json.dumps({"imageRef":IMAGE,"hostBinaryPath":str(host),"runId":run,"initialVolumes":initial})) + (root/"broker-configuration.json").write_text(json.dumps({"imageRef":IMAGE,"hostBinaryPath":str(host),"runId":run,"provider":args.provider,"initialVolumes":initial})) setup=run+"-dependencies" # Install the actual exact published SDK tree into synthetic config # storage. No read-only install bypass, fake SDK or model call. - create(setup,"--network","bridge","--user","0:0","--cap-drop","ALL", - "--mount",f"type=bind,src={root},dst={root}","--tmpfs","/tmp:rw,nosuid,nodev,size=128m", - "--env",f"HOME={root}/source-home","--entrypoint","/usr/local/bin/npm",IMAGE, - "install","--prefix",str(root/"source-home/.config/opencode"),"@opencode-ai/plugin@1.18.21") - outcome=docker("start","--attach",setup,check=False) - if outcome.returncode: - raise RuntimeError("closed startup actual SDK dependency preparation failed") - remove(setup) + if args.provider=="opencode": + create(setup,"--network","bridge","--user","0:0","--cap-drop","ALL", + "--mount",f"type=bind,src={root},dst={root}","--tmpfs","/tmp:rw,nosuid,nodev,size=128m", + "--env",f"HOME={root}/source-home","--entrypoint","/usr/local/bin/npm",IMAGE, + "install","--prefix",str(root/"source-home/.config/opencode"),"@opencode-ai/plugin@1.18.21") + outcome=docker("start","--attach",setup,check=False) + if outcome.returncode: + raise RuntimeError("closed startup actual SDK dependency preparation failed") + remove(setup) writable=run+"-workspace-mode" create(writable,"--network","none","--user","0:0","--cap-drop","ALL", "--mount",f"type=bind,src={root},dst={root}","--entrypoint","/usr/bin/chmod",IMAGE, @@ -157,7 +161,7 @@ def create(name,*arguments): "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_ENV_FILE={root}/grants/environment.env", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE={root}/grants/auth.json", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE={root}/grants/ca.pem", - "--entrypoint","/runtime/startup-tests",IMAGE,"--exact",TEST,"--nocapture","--test-threads","1") + "--entrypoint","/runtime/startup-tests",IMAGE,"--exact",selected_test,"--nocapture","--test-threads","1") docker("start",controller) controller_metadata=json.loads(docker("container","inspect",controller).stdout)[0] gateway=controller_metadata["NetworkSettings"]["Networks"]["bridge"]["IPAddress"] @@ -170,9 +174,9 @@ def create(name,*arguments): receipt["exitCode"]=exit_code receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) - expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} + expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else {("codex","terminal"),("codex","fresh")} observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} - if exit_code==0 and (len(receipt["consumedRoutes"])!=3 or observed_routes!=expected_routes or "test result: ok. 1 passed;" not in outcome.stdout): + if exit_code==0 and (len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes or "test result: ok. 1 passed;" not in outcome.stdout): receipt["exitCode"]=2;receipt["failureClass"]="native_startup_count_mismatch" (root/"broker-complete").write_text("ok") deadline=time.monotonic()+10 @@ -208,7 +212,7 @@ def create(name,*arguments): "freshell.input.reservation":prepared["initialAuthReservation"],"freshell.input.incarnation":prepared["incarnationId"], "freshell.input.intent":"1","freshell.input.daemon":source["daemonId"],"freshell.input.image":IMAGE, "freshell.input.binary-sha256":source["binarySha256"]} - elif prepared.get("indexedFiles",{}).get("volumeName")==volume: + elif (prepared.get("indexedFiles") or {}).get("volumeName")==volume: labels={"com.freshell.managed-inputs":"true","com.freshell.installation-id":installation, "com.freshell.soul-id":prepared["soulId"],"com.freshell.incarnation-id":prepared["incarnationId"], "com.freshell.input-generation":prepared["generationId"]} @@ -216,7 +220,7 @@ def create(name,*arguments): raise RuntimeError("closed startup volume cleanup ownership changed") docker("volume","rm",volume) receipt["ownedCleanup"]=all(docker("container","inspect",name,check=False).returncode!=0 for name in owned) and all(docker("volume","inspect",volume,check=False).returncode!=0 for volume in volumes) - (args.receipt_dir/"task002-native-opencode-launch.json").write_text(json.dumps(receipt,sort_keys=True)+"\n") + (args.receipt_dir/f"task002-native-{args.provider}-launch.json").write_text(json.dumps(receipt,sort_keys=True)+"\n") # Rename back so TemporaryDirectory removes only its owned scratch. root.rename(temporary) if not receipt.get("ownedCleanup"): From 3455f53d02ba6c97468169966c6f98df999ec05a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 00:56:15 -0700 Subject: [PATCH 071/493] fix: retain provider creation identity across reservation retries --- crates/freshell-supervisor/src/registry.rs | 15 +++++++++++++-- .../src/registry/input_reservation.rs | 8 ++++++-- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 2b6134c2c..aa4a14b1d 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -3040,7 +3040,7 @@ mod tests { tui_source: None, inline_config: false, }; - let wrong_result = registry.prepare_launch(wrong).await; + let wrong_result = registry.prepare_launch(wrong.clone()).await; assert!( matches!(wrong_result, Err(RegistryError::RequestConflict)), "foreign provider bind outcome: {wrong_result:?}" @@ -3048,12 +3048,23 @@ mod tests { let prepared = registry.prepare_launch(launch.clone()).await.unwrap(); assert_eq!( registry - .prepare_launch(launch) + .prepare_launch(launch.clone()) .await .unwrap() .incarnation_id, prepared.incarnation_id ); + assert!(matches!(registry.prepare_launch(wrong).await, Err(RegistryError::RequestConflict)), + "a bound reservation retry cannot change provider even when its caller reuses the payload digest"); + let mut changed_seed = launch; + changed_seed.creation_seed_ref = "changed-creation-seed".into(); + assert!( + matches!( + registry.prepare_launch(changed_seed).await, + Err(RegistryError::RequestConflict) + ), + "a bound reservation retry cannot change its creation seed" + ); let bound = registry .bound_input_reservation(prepared.incarnation_id) .await diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index 5487b3f7f..bf14b4abf 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -237,8 +237,12 @@ pub(super) fn check_bound_reference( input: &LaunchPreparation, incarnation: &IncarnationId, ) -> Result<(), RegistryError> { - let stored:Option=conn.query_row("SELECT reservation_id FROM provider_input_preparations WHERE bound_incarnation_id=?1 AND soul_id=?2 AND consumed=1",params![incarnation.as_str(),input.soul_id.as_str()],|row|row.get(0)).optional()?; - if stored.as_deref() != reservation_reference(input) { + let stored:Option<(String,String,String)>=conn.query_row("SELECT reservation_id,provider,creation_seed_ref FROM provider_input_preparations WHERE bound_incarnation_id=?1 AND soul_id=?2 AND consumed=1",params![incarnation.as_str(),input.soul_id.as_str()],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?))).optional()?; + if stored.as_ref().map(|(id, _, _)| id.as_str()) != reservation_reference(input) + || stored.as_ref().is_some_and(|(_, provider, seed)| { + provider != &input.provider || seed != &input.creation_seed_ref + }) + { return Err(RegistryError::RequestConflict); } Ok(()) From aa10f0b0f60c63ed465318355b60fc40f9a3bead Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 01:06:36 -0700 Subject: [PATCH 072/493] fix: block Codex replacement when original input origins are unknown --- .../src/managed_mcp_capability.rs | 103 ++++++++++++++++-- 1 file changed, 91 insertions(+), 12 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 2a63b003a..e487271b2 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -312,18 +312,10 @@ async fn apply_prepared_callback( manifest.validate().map_err(str::to_string)?; manifest } else { - let cwd = callback - .cwd - .as_deref() - .or(callback.workspace.as_deref()) - .ok_or("missing provider working directory")?; - freshell_platform::codex_inputs::CodexInputManifest { - version: 1, - original_home: home.into(), - cwd: cwd.into(), - codex_home: home.join(".codex"), - named_profile: None, - } + tracing::warn!(event="provider.inputs.preparation.blocked",stage="controller_callback",operation="select_original_sources",provider, + soul_id=%callback.soul_id,reason="origin_unknown", + "Original source origins are unrecorded; retained inputs were preserved"); + return Err("original Codex source origins are unrecorded; retain the existing inputs and reprepare from their original source before dispatch".into()); }; context.config.clear(); let references = callback.provider_secret_references.unwrap_or_default(); @@ -3987,6 +3979,93 @@ except Exception as error: ); } + #[tokio::test] + async fn managed_provider_legacy_codex_origin_unknown_blocks_and_retains_inputs() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("current-home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".codex")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + std::fs::write( + home.join(".codex/config.toml"), + "model_instructions_file='./current.md'\n", + ) + .unwrap(); + std::fs::write( + home.join(".codex/current.md"), + b"must not substitute current HOME", + ) + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let endpoint = "http://host.docker.internal:4000"; + let prior = store + .issue( + soul.clone(), + &directory, + endpoint.into(), + None, + Default::default(), + None, + ) + .unwrap(); + let legacy_root = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + std::fs::create_dir_all(&legacy_root).unwrap(); + std::fs::write( + legacy_root.join("retained-resource"), + b"retained legacy resource", + ) + .unwrap(); + let private_before = + std::fs::read(grant_path(&directory, &prior.grant_id).unwrap()).unwrap(); + let context = freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Codex { + tui_args: vec![], + sidecar_args: vec![], + }, + config: vec![], + mcp_capability: Some(prior.clone()), + prepared_inputs: None, + }; + let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"codex","workspace":workspace,"cwd":workspace,"priorContext":context,"providerSecretReferences":[],"authLookupAuthority":"fixture-owned-attempt"})).unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let error = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + endpoint, + &home, + &client, + ) + .await + .err() + .expect( + "legacy metadata cannot grant current HOME authority for an unrecorded original source", + ); + assert!(error.contains("original Codex source origins are unrecorded")); + assert!( + std::fs::read(grant_path(&directory, &prior.grant_id).unwrap()).unwrap() + == private_before, + "unknown origins must retain the original private capability unchanged" + ); + assert_eq!( + std::fs::read(legacy_root.join("retained-resource")).unwrap(), + b"retained legacy resource" + ); + assert_eq!( + std::fs::read_dir(directory.parent().unwrap().join("provider-roots")) + .unwrap() + .count(), + 1 + ); + } + #[tokio::test] async fn managed_provider_replacement_complete_codex_metadata_replays_before_rescanning_roots() { From 7b3e5d221f2846d7b2005786deab350b7525a598 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 01:20:16 -0700 Subject: [PATCH 073/493] fix: retain global Codex instructions and native rules in prepared inputs --- .../src/managed_provider_launch_tests.rs | 12 ++++++++++-- .../src/managed_provider_references/codex.rs | 2 ++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 6b4630eb1..dfd8a3a55 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -283,7 +283,8 @@ let result={};if(message.method==='initialize'){ const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); const nativeArgv=fs.readFileSync('/proc/'+process.ppid+'/cmdline','utf8').split('\0'); const retainedMcpRecipe=nativeArgv.includes('mcp_servers.freshell.command="node"')&&nativeArgv.includes('mcp_servers.freshell.args=["/opt/freshell-mcp/server.js"]')&&nativeArgv.some(arg=>arg.startsWith('mcp_servers.freshell.env_vars=')); - const record={physicalHome:process.env.HOME,initialized:true,authenticated,retainedMcpRecipe,scopedAlias:process.env.FRESHELL==='1',endpointPresent:!!endpoint,tokenPresent:!!token, + const policy=JSON.parse(require('node:child_process').execFileSync('codex',['execpolicy','check','--rules',path.join(process.env.CODEX_HOME,'rules/probe.rules'),'closed-fixture-never-run'],{encoding:'utf8'})); + const record={globalInstructions:digest(fs.readFileSync(path.join(process.env.CODEX_HOME,'AGENTS.md'))),nativeRuleForbidden:policy.decision==='forbidden',physicalHome:process.env.HOME,initialized:true,authenticated,retainedMcpRecipe,scopedAlias:process.env.FRESHELL==='1',endpointPresent:!!endpoint,tokenPresent:!!token, user:digest(fs.readFileSync(path.join(process.env.CODEX_HOME,'user.md'))),project:digest(fs.readFileSync(path.join(process.cwd(),'.codex/project.md'))), role:digest(fs.readFileSync(path.join(process.cwd(),'.codex/agents/role.md'))),skill:digest(fs.readFileSync(path.join(__dirname,'../SKILL.md'))), resource:digest(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:digest(require('probe-dependency'))}; @@ -292,6 +293,13 @@ let result={};if(message.method==='initialize'){ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n');}); "#; std::fs::write(skill.join("scripts/server.cjs"), server).unwrap(); + std::fs::create_dir_all(home.join(".codex/rules")).unwrap(); + std::fs::write(home.join(".codex/AGENTS.md"), b"closed-global-instructions").unwrap(); + std::fs::write( + home.join(".codex/rules/probe.rules"), + "prefix_rule(pattern=['closed-fixture-never-run'], decision='forbidden')\n", + ) + .unwrap(); let config = format!("model='synthetic-model'\nmodel_provider='fixture'\nmodel_instructions_file='./user.md'\n[model_providers.fixture]\nname='Closed fixture'\nbase_url='http://127.0.0.1:9'\nwire_api='responses'\nrequires_openai_auth=false\n[projects.{}]\ntrust_level='trusted'\n[mcp_servers.probe]\ncommand='node'\nenv_vars=['FRESHELL_URL','FRESHELL_TOKEN','FRESHELL','CODEX_HOME']\nargs=[{},{}]\n", serde_json::to_string(workspace.to_str().unwrap()).unwrap(), serde_json::to_string(skill.join("scripts/server.cjs").to_str().unwrap()).unwrap(), serde_json::to_string(receipt.to_str().unwrap()).unwrap()); std::fs::write(home.join(".codex/config.toml"), &config).unwrap(); std::fs::write(home.join(".codex/user.md"), b"closed-user-input").unwrap(); @@ -435,7 +443,7 @@ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n'); .map_err(|_| "native Codex must initialize its configured MCP consumer") } let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"physicalHome":"/home/freshell/provider","initialized":true,"authenticated":true,"retainedMcpRecipe":true,"scopedAlias":true,"endpointPresent":true,"tokenPresent":true,"user":hash(b"closed-user-input"),"project":hash(b"closed-project-input"),"role":hash(b"closed-role-input"),"skill":hash(std::fs::read(skill.join("SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-codex-resource"),"dependency":hash(b"closed-codex-dependency")}); + let expected = json!({"globalInstructions":hash(b"closed-global-instructions"),"nativeRuleForbidden":true,"physicalHome":"/home/freshell/provider","initialized":true,"authenticated":true,"retainedMcpRecipe":true,"scopedAlias":true,"endpointPresent":true,"tokenPresent":true,"user":hash(b"closed-user-input"),"project":hash(b"closed-project-input"),"role":hash(b"closed-role-input"),"skill":hash(std::fs::read(skill.join("SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-codex-resource"),"dependency":hash(b"closed-codex-dependency")}); let consumed_terminal = consumed(&receipt).await; if consumed_terminal.is_err() { let output = client diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 234aa56b8..1ac48d70e 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -472,6 +472,8 @@ pub fn discover( directories.reverse(); let mut resources = Vec::new(); for path in [ + manifest.codex_home.join("AGENTS.md"), + manifest.codex_home.join("rules"), manifest.codex_home.join("skills"), manifest.original_home.join(".agents/skills"), ] { From 7754771447fa54efa217db12857968d61434f98e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 01:41:56 -0700 Subject: [PATCH 074/493] fix: retain hosted Claude MCP selection through the default SDK --- crates/freshell-claude-sidecar/hosted-mcp.mjs | 6 + crates/freshell-claude-sidecar/index.mjs | 5 +- crates/freshell-freshagent/src/claude.rs | 10 ++ .../freshell-server/src/fresh_agent_proxy.rs | 9 +- .../src/fresh_agent_proxy_tests.rs | 22 +++ .../src/providers/fresh_agent.rs | 127 +++++++++++++++++- .../fixtures/settings-query-module.mjs | 2 + .../claude-sidecar/session-settings.test.ts | 15 +++ 8 files changed, 184 insertions(+), 12 deletions(-) create mode 100644 crates/freshell-claude-sidecar/hosted-mcp.mjs diff --git a/crates/freshell-claude-sidecar/hosted-mcp.mjs b/crates/freshell-claude-sidecar/hosted-mcp.mjs new file mode 100644 index 000000000..79ad74ed0 --- /dev/null +++ b/crates/freshell-claude-sidecar/hosted-mcp.mjs @@ -0,0 +1,6 @@ +// The managed launch context selects one fixed recipe. Scoped authentication +// stays in the inherited child environment and never enters SDK argv/IPC. +export function hostedMcpServers(selected, env = process.env) { + if (selected !== true || env.FRESHELL_HOSTED_FRESH_AGENT !== 'claude') return undefined + return { freshell: { command: 'node', args: ['/opt/freshell-mcp/server.js'] } } +} diff --git a/crates/freshell-claude-sidecar/index.mjs b/crates/freshell-claude-sidecar/index.mjs index 7299e768b..fe9665a29 100644 --- a/crates/freshell-claude-sidecar/index.mjs +++ b/crates/freshell-claude-sidecar/index.mjs @@ -79,6 +79,7 @@ import { configureSession, userMessageContent, resultErrorMessage } from './sess import { createTurnCompleteGate } from './turn-complete-gate.mjs' import { nextMonotonic } from './monotonic-clock.mjs' import { managedSdkSpawn } from './exec-profile.mjs' +import { hostedMcpServers } from './hosted-mcp.mjs' import { canUseTool as routeCanUseTool, cancelPending, @@ -419,6 +420,7 @@ function handleCreate(req) { plugins: Array.isArray(req.plugins) ? req.plugins.map(path => ({ type: 'local', path })) : undefined, + mcpServers: hostedMcpServers(req.hostedMcp), pathToClaudeCodeExecutable: process.env.CLAUDE_CMD || undefined, includePartialMessages: true, abortController: abort, @@ -448,8 +450,7 @@ function handleCreate(req) { }).catch((error) => logerr(`session commands unavailable: ${error?.message || error}`)) } - // Placeholder returns IMMEDIATELY (the SDK query is lazy) — exactly as - // SdkBridge.createSession returns the nanoid before system/init arrives. + // Return the placeholder before the native system/init identity arrives. emit({ type: 'created', requestId, sessionId }) consumeStream(sessionId, sdkQuery).catch((err) => logerr(`consume error: ${err?.message}`)) } catch (err) { diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index 249bfc6de..d60ac8a85 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -100,6 +100,9 @@ const MID_ROLLBACK_PARK_TICK: Duration = Duration::from_millis(10); /// Shared, cheaply-cloneable freshclaude WS state (mergeable into the server app + WsState). #[derive(Clone)] pub struct FreshClaudeState { + /// Host-only selection of the fixed Freshell stdio MCP recipe. Its endpoint + /// and grant remain in the child environment, never in create IPC. + hosted_mcp: bool, /// The shared WS broadcast bus (pre-serialized frames), fanned out by every /// `freshell-ws` connection so the oracle's capture socket records /// `freshAgent.created` / `freshAgent.send.accepted` / `freshAgent.event`. @@ -763,6 +766,7 @@ impl FreshClaudeState { /// Build the state around the shared broadcast bus. pub fn new(broadcast_tx: Arc>) -> Self { Self { + hosted_mcp: false, broadcast_tx, sessions: Arc::new(TokioMutex::new(HashMap::new())), cli_index: Arc::new(TokioMutex::new(HashMap::new())), @@ -796,6 +800,11 @@ impl FreshClaudeState { } } + pub fn with_hosted_mcp(mut self, selected: bool) -> Self { + self.hosted_mcp = selected; + self + } + /// b8ke focused episode-2 round-3 F3: crate-visible session-key resolver /// for in-crate cross-module tests. #[cfg(test)] @@ -2294,6 +2303,7 @@ impl FreshClaudeState { "permissionMode": msg.permission_mode, "effort": msg.effort, "plugins": msg.plugins, + "hostedMcp": self.hosted_mcp, "modelSelection": msg.model_selection, "resumeSessionId": resume_sid, }); diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 8ecae40d6..76eb48602 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -41,15 +41,8 @@ const PROVIDER_UID: u32 = 65_534; const PROVIDER_GID: u32 = 0; pub(crate) fn context_for_fresh_provider( - mut provider_launch_context: Option, + provider_launch_context: Option, ) -> Option { - if let Some(freshell_runtime_protocol::ProviderLaunchContext { - preparation: freshell_runtime_protocol::ProviderPreparation::Claude { mcp_args }, - .. - }) = provider_launch_context.as_mut() - { - mcp_args.clear(); - } provider_launch_context } diff --git a/crates/freshell-server/src/fresh_agent_proxy_tests.rs b/crates/freshell-server/src/fresh_agent_proxy_tests.rs index 0f856db36..83bd9e6e0 100644 --- a/crates/freshell-server/src/fresh_agent_proxy_tests.rs +++ b/crates/freshell-server/src/fresh_agent_proxy_tests.rs @@ -6,6 +6,28 @@ use freshell_runtime_protocol::{ }; use tokio::net::UnixListener; +#[test] +fn managed_provider_fresh_claude_retains_validated_mcp_recipe() { + let args = vec!["--mcp-config".into(), ".claude/freshell-mcp.json".into()]; + let context = freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Claude { + mcp_args: args.clone(), + }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let context = context_for_fresh_provider(Some(context)).unwrap(); + let freshell_runtime_protocol::ProviderPreparation::Claude { mcp_args } = context.preparation + else { + panic!("Claude preparation missing") + }; + assert_eq!( + mcp_args, args, + "fresh Claude must retain the selected MCP recipe" + ); +} + #[test] fn managed_provider_fresh_codex_retains_terminal_and_native_sidecar_arguments() { let args = crate::managed_provider_bootstrap::managed_codex_mcp_args().unwrap(); diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index bd07cc4c7..e897b7296 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -155,7 +155,15 @@ impl HostedTransport { let broadcast_tx = Arc::new(broadcast_tx); let state = match provider { FreshProvider::Claude | FreshProvider::Kilroy => { - ProviderState::Claude(FreshClaudeState::new(Arc::clone(&broadcast_tx))) + let hosted_mcp = context.is_some_and(|context| { + matches!( + &context.preparation, + ProviderPreparation::Claude { mcp_args } if !mcp_args.is_empty() + ) + }); + ProviderState::Claude( + FreshClaudeState::new(Arc::clone(&broadcast_tx)).with_hosted_mcp(hosted_mcp), + ) } FreshProvider::Codex => { let sidecar_args = context @@ -1115,7 +1123,7 @@ fn spawn_broadcast_bridge( .get("type") .and_then(Value::as_str) .unwrap_or_default(); - if event_type == "sdk.session.init" { + if matches!(event_type, "sdk.session.init" | "freshAgent.session.init") { if let Some(session_id) = event .get("cliSessionId") .or_else(|| event.get("sessionId")) @@ -1311,6 +1319,43 @@ mod tests { static OPENCODE_SNAPSHOT_ENV_LOCK: Mutex<()> = Mutex::const_new(()); + #[tokio::test] + async fn managed_provider_claude_native_identity_uses_actual_adapter_event_name() { + let (bus, rx) = broadcast::channel(8); + let (created, _) = watch::channel(None); + let (native, mut identity) = watch::channel(None); + let (events, mut event_rx) = mpsc::channel(8); + let (send_outcomes, _) = broadcast::channel(8); + let (kill_outcomes, _) = broadcast::channel(8); + spawn_broadcast_bridge( + FreshProvider::Claude, + rx, + BroadcastBridgeOutputs { + created, + native, + events, + send_outcomes, + kill_outcomes, + suppressed_kills: Arc::new(std::sync::Mutex::new(HashSet::new())), + }, + ); + bus.send(json!({"type":"freshAgent.event","sessionId":"owned-placeholder", + "event":{"type":"freshAgent.session.init","cliSessionId":"aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"}}).to_string()).unwrap(); + let received = tokio::time::timeout(Duration::from_millis(250), identity.changed()).await; + assert!( + received.is_ok(), + "the actual Claude adapter identity must reach the host" + ); + assert_eq!( + identity.borrow().as_deref(), + Some("aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa") + ); + assert!( + matches!(event_rx.recv().await, Some(AgentEvent::Started { native_session_id }) + if native_session_id == "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa") + ); + } + struct SnapshotProviderEnv(Vec<(&'static str, Option)>); impl Drop for SnapshotProviderEnv { fn drop(&mut self) { @@ -1323,6 +1368,84 @@ mod tests { } } + #[tokio::test] + async fn managed_provider_fresh_claude_sends_only_fixed_hosted_mcp_selection() { + let _lock = OPENCODE_SNAPSHOT_ENV_LOCK.lock().await; + let dir = tempfile::tempdir().unwrap(); + let script = dir.path().join("sidecar.mjs"); + let receipt = dir.path().join("create.json"); + std::fs::write(&script, r#" +import {createInterface} from 'node:readline'; +import {writeFileSync} from 'node:fs'; +const lines=createInterface({input:process.stdin}); +lines.on('line', line=>{ + const req=JSON.parse(line); + if(req.type==='create'){ + writeFileSync(process.env.FRESHELL_TEST_MCP_RECEIPT,JSON.stringify(req)); + process.stdout.write(JSON.stringify({type:'created',requestId:req.requestId,sessionId:'owned-mcp-fixture'})+'\n'); + process.stdout.write(JSON.stringify({type:'sdk.session.init',sessionId:'owned-mcp-fixture',cliSessionId:'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',model:'fixture',cwd:req.cwd,tools:[],slashCommands:[]})+'\n'); + } + if(req.type==='shutdown')process.exit(0); +}); +"#).unwrap(); + let keys = [ + "FRESHELL_CLAUDE_SIDECAR", + "FRESHELL_CLAUDE_NODE", + "FRESHELL_TEST_MCP_RECEIPT", + ]; + let _env = SnapshotProviderEnv( + keys.iter() + .map(|key| (*key, std::env::var_os(key))) + .collect(), + ); + std::env::set_var("FRESHELL_CLAUDE_SIDECAR", &script); + std::env::set_var("FRESHELL_CLAUDE_NODE", "node"); + std::env::set_var("FRESHELL_TEST_MCP_RECEIPT", &receipt); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { + mcp_args: vec!["--mcp-config".into(), ".claude/freshell-mcp.json".into()], + }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + for selected in [true, false] { + let transport = HostedTransport::new_with_context( + FreshProvider::Claude, + selected.then_some(&context), + ) + .await; + let profile = FreshAgentProfile { + provider: FreshProvider::Claude, + runtime_variant: "freshclaude".into(), + cwd: dir.path().to_string_lossy().into(), + model: None, + effort: None, + permission_mode: None, + sandbox: None, + provider_store_id: "owned-store".into(), + native_session_id: None, + plugins: None, + model_selection: None, + session_ref: None, + provider_launch_context: selected.then_some(context.clone()), + provider_secret_references: vec![], + }; + transport.start(&profile).await.unwrap(); + let observed: Value = + serde_json::from_slice(&std::fs::read(&receipt).unwrap()).unwrap(); + transport.stop().await.unwrap(); + assert_eq!( + observed["hostedMcp"], selected, + "only accepted hosted selection enables the SDK recipe" + ); + assert!( + observed.get("mcpServers").is_none(), + "arbitrary MCP maps are never sent over private create IPC" + ); + } + } + #[tokio::test] async fn actual_opencode_first_send_materializes_once_and_reuses_owned_native_http_session() { let _lock = OPENCODE_SNAPSHOT_ENV_LOCK.lock().await; diff --git a/test/unit/claude-sidecar/fixtures/settings-query-module.mjs b/test/unit/claude-sidecar/fixtures/settings-query-module.mjs index 2125de3f3..d694426a8 100644 --- a/test/unit/claude-sidecar/fixtures/settings-query-module.mjs +++ b/test/unit/claude-sidecar/fixtures/settings-query-module.mjs @@ -1,6 +1,8 @@ // Controlled SDK implementation used through the production sidecar import seam. let generations = 0 export function query({ prompt, options }) { + const mcp = options.mcpServers?.freshell + process.stdout.write(`${JSON.stringify({ type: 'probe.hosted-mcp', selected: !!mcp, fixedRecipe: mcp?.command === 'node' && JSON.stringify(mcp?.args) === JSON.stringify(['/opt/freshell-mcp/server.js']) && mcp?.env === undefined })}\n`) process.stdout.write(`${JSON.stringify({ type: 'probe.auth-env', apiKeyVisible: options.env.ANTHROPIC_API_KEY === 'onecli-test-key', oauthVisible: options.env.CLAUDE_CODE_OAUTH_TOKEN === 'onecli-test-oauth' })}\n`) const generation = ++generations const durable = options.resume ?? 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa' diff --git a/test/unit/claude-sidecar/session-settings.test.ts b/test/unit/claude-sidecar/session-settings.test.ts index 8c5fe3179..1f23a6673 100644 --- a/test/unit/claude-sidecar/session-settings.test.ts +++ b/test/unit/claude-sidecar/session-settings.test.ts @@ -105,6 +105,21 @@ function sidecar(extraEnv: Record = {}) { } describe('Claude sidecar OneCLI child environment', () => { + it('passes the fixed selected hosted MCP recipe through the supported SDK option', async () => { + const bridge = sidecar({ FRESHELL_HOSTED_FRESH_AGENT: 'claude' }) + bridge.send({ type: 'create', requestId: 'hosted-mcp', cwd: '/tmp', hostedMcp: true }) + expect(await bridge.waitFor('probe.hosted-mcp')).toMatchObject({ selected: true, fixedRecipe: true }) + }) + + it('keeps unselected and ordinary Claude SDK sessions free of the hosted recipe', async () => { + const hosted = sidecar({ FRESHELL_HOSTED_FRESH_AGENT: 'claude' }) + hosted.send({ type: 'create', requestId: 'unselected-mcp', cwd: '/tmp', hostedMcp: false }) + expect(await hosted.waitFor('probe.hosted-mcp')).toMatchObject({ selected: false }) + const ordinary = sidecar({ FRESHELL_HOSTED_FRESH_AGENT: '' }) + ordinary.send({ type: 'create', requestId: 'ordinary-mcp', cwd: '/tmp', hostedMcp: true }) + expect(await ordinary.waitFor('probe.hosted-mcp')).toMatchObject({ selected: false }) + }) + it('passes a managed OneCLI API key to the SDK child without inheriting ambient keys', async () => { const unmanaged = sidecar({ ANTHROPIC_API_KEY: 'onecli-test-key' }) unmanaged.send({ type: 'create', requestId: 'unmanaged', cwd: '/tmp' }) From d63f02c7182109f091731ae76e4f9a0dc59c4b71 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 01:52:43 -0700 Subject: [PATCH 075/493] test: qualify genuine default Claude SDK MCP startup without a model turn --- crates/freshell-server/src/main.rs | 2 + .../src/managed_provider_sdk_probe.mjs | 107 ++++++++++++++++++ .../src/managed_provider_sdk_tests.rs | 60 ++++++++++ .../managed-provider-source-sandbox.py | 13 ++- 4 files changed, 178 insertions(+), 4 deletions(-) create mode 100644 crates/freshell-server/src/managed_provider_sdk_probe.mjs create mode 100644 crates/freshell-server/src/managed_provider_sdk_tests.rs diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index e3521a2c8..fba8dd876 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -47,6 +47,8 @@ mod managed_provider_bootstrap; mod managed_provider_launch_tests; #[cfg(feature = "managed-runtime-v1")] mod managed_provider_references; +#[cfg(all(test, unix, feature = "managed-provider-qualification"))] +mod managed_provider_sdk_tests; #[cfg(feature = "managed-runtime-v1")] mod managed_runtime; #[cfg(feature = "managed-runtime-v1")] diff --git a/crates/freshell-server/src/managed_provider_sdk_probe.mjs b/crates/freshell-server/src/managed_provider_sdk_probe.mjs new file mode 100644 index 000000000..af7d343f6 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_sdk_probe.mjs @@ -0,0 +1,107 @@ +import { readFileSync, existsSync } from 'node:fs' +import { spawnSync } from 'node:child_process' +import { query } from '/opt/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk/sdk.mjs' +import { hostedMcpServers } from './hosted-mcp.mjs' +import { managedSdkSpawn } from './exec-profile.mjs' + +const abort = new AbortController() +const pause = ms => new Promise(resolve => setTimeout(resolve, ms)) +let ended = false, settleNext, prompts = 0, stage = 'query', sdkQuery, nativeChild, nativeExit +const prompt = { + [Symbol.asyncIterator]() { return this }, + next() { + if (ended) return Promise.resolve({ done: true }) + return new Promise(resolve => { settleNext = resolve }) + }, + end() { ended = true; settleNext?.({ done: true }); settleNext = undefined }, + return() { this.end(); return Promise.resolve({ done: true }) }, +} +const token = 'synthetic-sdk-scoped-child-value' +const env = { ...process.env, FRESHELL: '1', FRESHELL_URL: 'http://127.0.0.1:9', FRESHELL_TOKEN: token } +const receipt = { + sdkVersion: JSON.parse(readFileSync('/opt/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk/package.json')).version, + nativeVersion: '', defaultChild: false, connected: false, fixedRecipe: false, + scopedEnvironment: false, zeroPrompts: false, nativeExit: false, mcpExit: false, +} +const mcpPids = new Set() +function observeMcp(parent) { + const file = `/proc/${parent}/task/${parent}/children` + if (!existsSync(file)) return + for (const pid of readFileSync(file, 'utf8').trim().split(/\s+/).filter(Boolean)) { + try { + const argv = readFileSync(`/proc/${pid}/cmdline`, 'utf8').split('\0') + if (argv.includes('/opt/freshell-mcp/server.js')) { + mcpPids.add(pid) + const effective = readFileSync(`/proc/${pid}/environ`, 'utf8').split('\0') + receipt.scopedEnvironment = effective.includes(`FRESHELL_TOKEN=${token}`) + && effective.includes('FRESHELL=1') && effective.includes(`FRESHELL_URL=${env.FRESHELL_URL}`) + } + observeMcp(pid) + } catch { /* The observed native-owned child may exit during inspection. */ } + } +} +const deadline = setTimeout(() => { prompt.end(); abort.abort(); sdkQuery?.close() }, 90_000) +try { + const launch = managedSdkSpawn(() => {}) + sdkQuery = query({ prompt, options: { + cwd: process.cwd(), env, abortController: abort, + settingSources: ['user', 'project', 'local'], mcpServers: hostedMcpServers(true, env), + spawnClaudeCodeProcess: options => { + receipt.defaultChild = options.command.includes('@anthropic-ai/claude-agent-sdk-linux-x64/') && options.command.endsWith('/claude') + const version = spawnSync(options.command, ['--version'], { env: options.env, encoding: 'utf8', timeout: 10_000 }) + receipt.nativeVersion = /^2\.1\.237(?:\s|$)/.test(version.stdout ?? '') ? '2.1.237' : 'unexpected' + const index = options.args.indexOf('--mcp-config') + const config = index < 0 ? {} : JSON.parse(options.args[index + 1]) + receipt.fixedRecipe = JSON.stringify(config) === JSON.stringify({ mcpServers: hostedMcpServers(true, env) }) + && !options.args.some(arg => arg.includes(token)) + nativeChild = launch(options) + const write = nativeChild.stdin.write + let pending = '' + nativeChild.stdin.write = function(chunk, ...args) { + pending += chunk.toString() + let boundary + while ((boundary = pending.indexOf('\n')) >= 0) { + const frame = pending.slice(0, boundary) + pending = pending.slice(boundary + 1) + try { if (JSON.parse(frame).type === 'user') prompts++ } catch {} + } + if (pending.length > 1024 * 1024) throw new Error('fixture-native-input-profile') + return write.call(this, chunk, ...args) + } + nativeExit = new Promise(resolve => nativeChild.once('exit', resolve)) + return nativeChild + }, + } }) + stage = 'initialization' + await sdkQuery.initializationResult() + stage = 'native-mcp-startup' + for (let attempt = 0; attempt < 160; attempt++) { + const statuses = await sdkQuery.mcpServerStatus() + receipt.connected = statuses.some(status => status.name === 'freshell' && status.status === 'connected' && status.tools?.length > 0) + observeMcp(nativeChild.pid) + if (receipt.connected && receipt.scopedEnvironment) break + await pause(250) + } +} catch { + receipt.failureClass = stage +} finally { + prompt.end() + abort.abort() + sdkQuery?.close() + if (sdkQuery) await sdkQuery.return() + if (nativeExit) { + let exitTimer + receipt.nativeExit = await Promise.race([ + nativeExit.then(() => true), + new Promise(resolve => { exitTimer = setTimeout(() => resolve(false), 10_000) }), + ]) + clearTimeout(exitTimer) + } + receipt.zeroPrompts = prompts === 0 + receipt.mcpExit = mcpPids.size > 0 && [...mcpPids].every(pid => { + try { return /^State:\s+Z/m.test(readFileSync(`/proc/${pid}/status`, 'utf8')) } catch { return true } + }) + clearTimeout(deadline) +} +process.stdout.write(JSON.stringify(receipt)) +if (receipt.failureClass || Object.values(receipt).includes(false)) process.exitCode = 1 diff --git a/crates/freshell-server/src/managed_provider_sdk_tests.rs b/crates/freshell-server/src/managed_provider_sdk_tests.rs new file mode 100644 index 000000000..9e3133af2 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_sdk_tests.rs @@ -0,0 +1,60 @@ +//! Genuine default SDK startup qualification in the closed provider sandbox. +use sha2::{Digest, Sha256}; + +#[test] +fn managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt() { + let root = std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") + .expect("native SDK qualification requires its owned executable sandbox"); + let fixture = tempfile::Builder::new() + .prefix("claude-sdk-") + .tempdir_in(root) + .unwrap(); + for (name, source) in [ + ( + "hosted-mcp.mjs", + include_str!("../../freshell-claude-sidecar/hosted-mcp.mjs"), + ), + ( + "exec-profile.mjs", + include_str!("../../freshell-claude-sidecar/exec-profile.mjs"), + ), + ("probe.mjs", include_str!("managed_provider_sdk_probe.mjs")), + ] { + std::fs::write(fixture.path().join(name), source).unwrap(); + } + let output = std::process::Command::new("node") + .arg(fixture.path().join("probe.mjs")) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .env("HOME", "/home/freshell/provider") + .env("FRESHELL_HOSTED_FRESH_AGENT", "claude") + .current_dir(fixture.path()) + .output() + .unwrap(); + assert!( + output.status.success(), + "native SDK failed its classified model-free startup: {}", + String::from_utf8_lossy(&output.stdout) + ); + let receipt: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(receipt["sdkVersion"], "0.3.237"); + assert_eq!(receipt["nativeVersion"], "2.1.237"); + for field in [ + "defaultChild", + "connected", + "fixedRecipe", + "scopedEnvironment", + "zeroPrompts", + "nativeExit", + "mcpExit", + ] { + assert_eq!(receipt[field], true, "genuine SDK startup receipt {field}"); + } + println!( + "provider_source_consumer {}", + serde_json::json!({ + "provider":"claude", "case":"default-sdk-fixed-mcp", + "digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&receipt).unwrap())), + }) + ); +} diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 3c697534f..d477df457 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -20,11 +20,15 @@ def docker(*args, check=True): def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--provider",choices=["amplifier","codex"],default="amplifier") + parser.add_argument("--provider",choices=["amplifier","codex","claude"],default="amplifier") parser.add_argument("--test-binary", type=Path, required=True) parser.add_argument("--receipt-dir", type=Path, required=True) args = parser.parse_args() - selected_test = TEST if args.provider == "amplifier" else "managed_mcp_capability::tests::managed_provider_codex_native_typed_inputs_survive_transport" + selected_test = { + "amplifier": TEST, + "codex": "managed_mcp_capability::tests::managed_provider_codex_native_typed_inputs_survive_transport", + "claude": "managed_provider_sdk_tests::managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt", + }[args.provider] binary = args.test_binary.resolve(strict=True) if not binary.is_file() or not re.fullmatch(r"freshell_server-[0-9a-f]{16}", binary.name): raise RuntimeError("source fixture requires its compiled server test artifact") @@ -71,7 +75,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != (18 if args.provider == "amplifier" else 4) or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":4,"claude":1}[args.provider] or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. @@ -86,7 +90,8 @@ def main(): raise RuntimeError("source fixture cleanup ownership changed") docker("rm", "--force", value["Id"]) receipt["ownedCleanup"] = docker("container", "inspect", name, check=False).returncode != 0 - (args.receipt_dir / ("task002-native-source-consumers.json" if args.provider == "amplifier" else "task002-native-codex-source-consumers.json")).write_text(json.dumps(receipt, sort_keys=True) + "\n") + receipt_name = {"amplifier":"task002-native-source-consumers.json", "codex":"task002-native-codex-source-consumers.json", "claude":"task002-native-claude-sdk-consumers.json"}[args.provider] + (args.receipt_dir / receipt_name).write_text(json.dumps(receipt, sort_keys=True) + "\n") if not created or not receipt["ownedCleanup"]: raise RuntimeError("source fixture failed its owned cleanup proof") raise SystemExit(receipt["exitCode"]) From 8b8dc447c30b1e131bc9b06c5b9c4990ef2effe0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 01:52:50 -0700 Subject: [PATCH 076/493] fix: allow initial Claude startup before native identity materializes --- .../src/providers/fresh_agent.rs | 61 ++++++++++++------- 1 file changed, 40 insertions(+), 21 deletions(-) diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index e897b7296..1b3b6bd9b 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -436,22 +436,20 @@ impl FreshAgentTransport for HostedTransport { Some(expected) => Some(self.wait_exact_native(expected).await?), None if matches!(self.provider, FreshProvider::Codex) => Some(session_id), None if matches!(self.provider, FreshProvider::Claude | FreshProvider::Kilroy) => { - let mut receiver = self.native_rx.lock().await; - Some( - tokio::time::timeout(CREATE_TIMEOUT, async { - loop { - if let Some(observed) = receiver.borrow().clone() { - return Ok::<_, String>(observed); - } - receiver - .changed() - .await - .map_err(|_| "provider identity channel closed".to_string())?; - } - }) - .await - .map_err(|_| "provider did not materialize native identity".to_string())??, - ) + // The real SDK starts MCP/configuration before its first user + // turn but publishes system/init identity only with that turn. + // The bridge captures the native identity when it arrives. + let observed = self.native_rx.lock().await.borrow().clone(); + if observed.is_none() { + tracing::info!( + event = "provider.identity.awaiting_first_turn", + stage = "host_create", + runtime_variant = %profile.runtime_variant, + native_identity = "unrecorded", + "provider started before native identity materialization" + ); + } + observed } None => None, }; @@ -1383,7 +1381,7 @@ lines.on('line', line=>{ if(req.type==='create'){ writeFileSync(process.env.FRESHELL_TEST_MCP_RECEIPT,JSON.stringify(req)); process.stdout.write(JSON.stringify({type:'created',requestId:req.requestId,sessionId:'owned-mcp-fixture'})+'\n'); - process.stdout.write(JSON.stringify({type:'sdk.session.init',sessionId:'owned-mcp-fixture',cliSessionId:'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',model:'fixture',cwd:req.cwd,tools:[],slashCommands:[]})+'\n'); + if(req.model!=='deferred-identity')process.stdout.write(JSON.stringify({type:'sdk.session.init',sessionId:'owned-mcp-fixture',cliSessionId:'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa',model:'fixture',cwd:req.cwd,tools:[],slashCommands:[]})+'\n'); } if(req.type==='shutdown')process.exit(0); }); @@ -1409,7 +1407,12 @@ lines.on('line', line=>{ config: vec![], prepared_inputs: None, }; - for selected in [true, false] { + for (selected, deferred_identity, expected_identity) in [ + (true, false, None), + (false, false, None), + (true, true, None), + (true, false, Some("bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb")), + ] { let transport = HostedTransport::new_with_context( FreshProvider::Claude, selected.then_some(&context), @@ -1419,22 +1422,38 @@ lines.on('line', line=>{ provider: FreshProvider::Claude, runtime_variant: "freshclaude".into(), cwd: dir.path().to_string_lossy().into(), - model: None, + model: deferred_identity.then(|| "deferred-identity".into()), effort: None, permission_mode: None, sandbox: None, provider_store_id: "owned-store".into(), - native_session_id: None, + native_session_id: expected_identity.map(str::to_owned), plugins: None, model_selection: None, session_ref: None, provider_launch_context: selected.then_some(context.clone()), provider_secret_references: vec![], }; - transport.start(&profile).await.unwrap(); + let started = + tokio::time::timeout(Duration::from_secs(1), transport.start(&profile)).await; let observed: Value = serde_json::from_slice(&std::fs::read(&receipt).unwrap()).unwrap(); transport.stop().await.unwrap(); + let started = started.expect("Claude create cannot wait for the first model turn"); + if expected_identity.is_some() { + assert!( + started.is_err(), + "an exact resume must reject a different actual native identity" + ); + continue; + } + let started = started.unwrap(); + if deferred_identity { + assert!( + started.native_session_id.is_none(), + "unrecorded native identity must remain unknown" + ); + } assert_eq!( observed["hostedMcp"], selected, "only accepted hosted selection enables the SDK recipe" From 9640c6c1c95fd93d2e7f7c15545be7ad179b9845 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 02:10:58 -0700 Subject: [PATCH 077/493] fix: stage ordinary Claude project resources through the shared graph --- crates/freshell-platform/src/claude_inputs.rs | 35 +++++ crates/freshell-platform/src/lib.rs | 1 + .../src/managed_mcp_capability.rs | 145 ++++++++++++++++-- .../managed_mcp_capability/claude_stage.rs | 89 +++++++++++ .../{codex_stage.rs => ordinary_stage.rs} | 110 +++++++------ 5 files changed, 316 insertions(+), 64 deletions(-) create mode 100644 crates/freshell-platform/src/claude_inputs.rs create mode 100644 crates/freshell-server/src/managed_mcp_capability/claude_stage.rs rename crates/freshell-server/src/managed_mcp_capability/{codex_stage.rs => ordinary_stage.rs} (65%) diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs new file mode 100644 index 000000000..22bf0dcf4 --- /dev/null +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -0,0 +1,35 @@ +//! Accepted ordinary Claude source origins. Payloads remain in private files. +use serde::{Deserialize, Serialize}; +use std::path::{Path, PathBuf}; + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ClaudeInputManifest { + pub version: u8, + pub original_home: PathBuf, + pub cwd: PathBuf, + pub workspace: PathBuf, + pub plugin_dirs: Vec, +} +impl ClaudeInputManifest { + pub fn validate(&self) -> Result<(), &'static str> { + if self.version != 1 || self.plugin_dirs.len() > 8192 { + return Err("accepted Claude discovery context profile exceeded"); + } + for path in [&self.original_home, &self.cwd, &self.workspace] + .into_iter() + .chain(&self.plugin_dirs) + { + if !path.is_absolute() + || path.as_os_str().len() > 64 * 1024 + || path.to_string_lossy().chars().any(char::is_control) + { + return Err("accepted Claude discovery source path is invalid"); + } + } + Ok(()) + } + pub fn runtime_path(&self, logical: &Path) -> PathBuf { + crate::codex_inputs::runtime_path(&self.original_home, logical) + } +} diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 4ee8a457d..409d8c3b9 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -51,6 +51,7 @@ //! injected source ([`Env`], [`FileProbe`], [`path::WslPathResolver`]). Thin `*_live` //! wrappers at the edges perform the real reads and delegate to the pure core. +pub mod claude_inputs; pub mod cli_launch; pub mod clock; pub mod codex_inputs; diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e487271b2..a34732eb2 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -28,12 +28,14 @@ const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; mod amplifier_resources; -mod codex_stage; +mod claude_stage; +mod ordinary_stage; mod source_stage; #[derive(Clone, Copy, Default)] pub(crate) struct OpencodeEphemeralInput<'a> { pub codex_manifest: Option<&'a freshell_platform::codex_inputs::CodexInputManifest>, + pub claude_manifest: Option<&'a freshell_platform::claude_inputs::ClaudeInputManifest>, pub inline_config: Option<&'a str>, pub tui_config_path: Option<&'a str>, pub tui_source: Option<&'a ProviderConfigReference>, @@ -1255,16 +1257,29 @@ fn stage_provider_root( .map_err(|_| "prepared resource descriptor exceeds control profile")?; write_private_bytes(&temporary.join(".freshell/resource-mounts.json"), &metadata)?; } - let codex_resources = if provider == "codex" && opencode_input.launch_context.is_some() { - Some(codex_stage::prepare( - home, - opencode_input.cwd.unwrap_or(home), - opencode_input.codex_manifest, - )?) - } else { - None - }; - if let Some(resources) = &codex_resources { + let ordinary_resources = + if opencode_input.launch_context.is_some() && matches!(provider, "codex" | "claude") { + Some(if provider == "claude" { + claude_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input.claude_manifest, + )? + } else { + ordinary_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input.codex_manifest, + )? + }) + } else { + None + }; + if let Some(resources) = &ordinary_resources { selections.extend(resources.selections.iter().cloned()); } let amplifier_resources = if provider == "amplifier" { @@ -1280,7 +1295,7 @@ fn stage_provider_root( selections.extend(resources.selections.iter().cloned()); } for reference in references { - if codex_resources.is_some() { + if ordinary_resources.is_some() { continue; } if reference.root != ProviderConfigRoot::UserProvider { @@ -1416,14 +1431,14 @@ fn stage_provider_root( opencode_input, )?; } - if let Some(resources) = &codex_resources { + if let Some(resources) = &ordinary_resources { let capability = McpCapabilityReference { grant_id: grant_id.into(), endpoint: "http://host.docker.internal:1".into(), provider_relative_path: CAPABILITY_PATH.into(), host_gateway_address: None, }; - let inputs = codex_stage::publish( + let inputs = ordinary_stage::publish( &temporary, resources, &capability, @@ -1435,7 +1450,7 @@ fn stage_provider_root( &inputs, freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, ) - .map_err(|_| "Codex prepared context exceeds control profile")?, + .map_err(|_| "ordinary prepared context exceeds control profile")?, )?; } allow_controller_stage_read(&temporary)?; @@ -2913,6 +2928,106 @@ except Exception as error: ); } } + #[cfg(unix)] + #[test] + fn managed_provider_claude_complete_stage_carries_project_resources_and_dependencies() { + use freshell_runtime_protocol::{ + PreparedProviderInputs, ProviderLaunchContext, ProviderPreparation, + }; + let control = tempfile::tempdir().unwrap(); + let capabilities = control.path().join("capabilities"); + std::fs::create_dir(&capabilities).unwrap(); + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let skill = workspace.path().join(".claude/skills/probe"); + std::fs::create_dir_all(home.path().join(".claude")).unwrap(); + std::fs::create_dir_all(skill.join("node_modules/local-dependency")).unwrap(); + std::fs::write( + home.path().join(".claude/settings.json"), + r#"{"env":{"CLAUDE_SELECTED_NONCE":"settings-input"}}"#, + ) + .unwrap(); + std::fs::write(home.path().join(".claude/CLAUDE.md"), "global-memory-input").unwrap(); + std::fs::write(skill.join("resource"), "adjacent-resource").unwrap(); + std::fs::write( + skill.join("node_modules/local-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write( + skill.join("probe.mjs"), + r#" +import {readFileSync} from 'node:fs'; +import dep from 'local-dependency'; +process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+':'+dep); +"#, + ) + .unwrap(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let stage = stage_provider_root( + &capabilities, + "grant-claude-complete", + "claude", + home.path(), + &[], + OpencodeEphemeralInput { + cwd: Some(workspace.path()), + workspace: Some(workspace.path()), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")) + .expect("complete Claude source context must be published"), + ) + .unwrap(); + let mount = inputs + .resource_mounts + .iter() + .find(|mount| { + mount.logical_destination + == workspace.path().join(".claude/skills").to_str().unwrap() + }) + .expect("project skills must be selected as a complete resource unit"); + let consumed = std::process::Command::new("node") + .arg( + stage + .join(&mount.stage_relative_path) + .join("probe/probe.mjs"), + ) + .output() + .unwrap(); + assert!( + consumed.status.success(), + "transported project skill must load its installed dependency" + ); + assert_eq!(consumed.stdout, b"adjacent-resource:installed-dependency"); + assert!( + inputs + .resource_mounts + .iter() + .any(|mount| mount.logical_destination + == home.path().join(".claude/CLAUDE.md").to_str().unwrap()), + "global memory remains selected" + ); + assert!( + inputs + .config_binds + .iter() + .any(|bind| bind.logical_destination + == home.path().join(".claude/settings.json").to_str().unwrap()), + "global settings retain declaring origin" + ); + } + #[cfg(unix)] #[test] fn native_state_aliases_are_excluded_without_filtering_dependency_basenames() { diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs new file mode 100644 index 000000000..117fbccf2 --- /dev/null +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -0,0 +1,89 @@ +//! Finite Claude source roots enter the same ordinary resource graph as Codex. +use super::ordinary_stage::{prepare_selected, Preparation}; +use freshell_platform::claude_inputs::ClaudeInputManifest; +use std::path::{Path, PathBuf}; + +pub(super) fn prepare( + home: &Path, + cwd: &Path, + workspace: &Path, + accepted: Option<&ClaudeInputManifest>, +) -> Result { + let manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { + version: 1, + original_home: home.into(), + cwd: cwd.into(), + workspace: workspace.into(), + plugin_dirs: vec![], + }); + manifest.validate().map_err(str::to_string)?; + let mut configurations = Vec::new(); + let mut resources = Vec::new(); + let mut bases: Vec = manifest.cwd.ancestors().map(Path::to_path_buf).collect(); + bases.push(manifest.original_home.clone()); + bases.sort(); + bases.dedup(); + let mut settings_bases = vec![ + manifest.original_home.clone(), + manifest.cwd.clone(), + manifest.workspace.clone(), + ]; + settings_bases.sort(); + settings_bases.dedup(); + for base in settings_bases { + for name in ["settings.json", "settings.local.json"] { + let logical = base.join(".claude").join(name); + if optional(&logical)? { + configurations.push((logical.clone(), canonical(&logical)?)); + } + } + } + for base in bases { + for relative in [ + "CLAUDE.md", + "CLAUDE.local.md", + ".claude/CLAUDE.md", + ".claude/rules", + ".claude/skills", + ".claude/commands", + ".claude/agents", + ".claude/hooks", + ".claude/plugins", + ] { + let logical = base.join(relative); + if optional(&logical)? { + resources.push(logical); + } + } + } + for plugin in &manifest.plugin_dirs { + canonical(plugin)?; // Explicit selections may never be silently omitted. + resources.push(plugin.clone()); + } + prepare_selected( + "claude", + &manifest.cwd, + &manifest, + configurations, + resources, + |path| manifest.runtime_path(path), + ) +} +fn canonical(path: &Path) -> Result { + std::fs::canonicalize(path).map_err(|error| { + format!( + "Claude selected source {} is unavailable: {error}", + path.display() + ) + }) +} +fn optional(path: &Path) -> Result { + match std::fs::symlink_metadata(path) { + Ok(_) => Ok(true), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(error) => Err(format!( + "Claude optional source {} metadata failed: {error}", + path.display() + )), + } +} diff --git a/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs similarity index 65% rename from crates/freshell-server/src/managed_mcp_capability/codex_stage.rs rename to crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs index c17642012..ceb0a7fc9 100644 --- a/crates/freshell-server/src/managed_mcp_capability/codex_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs @@ -1,7 +1,7 @@ -//! Native-selected ordinary Codex inputs use the shared resource copier and +//! Native-selected ordinary inputs use the shared resource copier and //! existing exact mounts. No configuration payload enters callback metadata. use super::{private_directory, write_private_bytes}; -use crate::managed_provider_references::codex::{discover, Discovery, InputKind}; +use crate::managed_provider_references::codex::{discover, InputKind}; use freshell_platform::{codex_inputs::CodexInputManifest, provider_environment::CopySelection}; use freshell_runtime_protocol::*; use std::{ @@ -10,7 +10,7 @@ use std::{ }; pub(super) struct Preparation { - pub discovery: Discovery, + manifest_bytes: Vec, pub selections: Vec, pub binds: Vec, pub mounts: Vec, @@ -28,61 +28,75 @@ pub(super) fn prepare( named_profile: None, }); let discovery = discover(manifest).map_err(str::to_string)?; + let configurations = discovery + .inputs + .iter() + .filter(|input| input.kind == InputKind::Configuration) + .map(|input| (input.logical_path.clone(), input.canonical_source.clone())) + .collect(); + let mut ordinary = discovery.resource_roots.clone(); + for input in &discovery.inputs { + if input.kind != InputKind::Configuration + && !ordinary + .iter() + .any(|root| input.logical_path.starts_with(root)) + { + ordinary.push(input.logical_path.clone()); + } + } + prepare_selected( + "codex", + &discovery.manifest.cwd, + &discovery.manifest, + configurations, + ordinary, + |logical| discovery.manifest.runtime_path(logical), + ) +} + +pub(super) fn prepare_selected( + provider: &str, + cwd: &Path, + manifest: &M, + configurations: Vec<(PathBuf, PathBuf)>, + mut ordinary: Vec, + runtime_path: impl Fn(&Path) -> PathBuf, +) -> Result { let mut result = Preparation { - discovery, + manifest_bytes: bounded_json_bytes(manifest, 64 * 1024 * 1024) + .map_err(|_| "ordinary source metadata budget exceeded")?, selections: Vec::new(), binds: Vec::new(), mounts: Vec::new(), }; let mut files = BTreeMap::::new(); - for input in &result.discovery.inputs { - if input.kind != InputKind::Configuration { - continue; - } + for (logical_path, canonical_source) in configurations { let next = files.len(); - let source = files - .entry(input.canonical_source.clone()) - .or_insert_with(|| { - let relative = format!(".freshell/templates/codex-{next}"); - result.selections.push(CopySelection { - source: input.canonical_source.clone(), - destination_relative: relative.clone().into(), - }); - relative + let source = files.entry(canonical_source.clone()).or_insert_with(|| { + let relative = format!(".freshell/templates/{provider}-{next}"); + result.selections.push(CopySelection { + source: canonical_source.clone(), + destination_relative: relative.clone().into(), }); - for destination in destinations(&result.discovery.manifest, &input.logical_path) { + relative + }); + for destination in destinations(&runtime_path, &logical_path) { let bind = ProviderConfigBind { source_relative_path: source.clone(), logical_destination: destination .to_str() .ok_or("Codex config destination is not UTF-8")? .into(), - effective_destination: effective( - &destination, - &input.canonical_source, - &result.discovery.manifest, - ) - .to_str() - .ok_or("Codex config destination is not UTF-8")? - .into(), + effective_destination: effective(&destination, &canonical_source, cwd) + .to_str() + .ok_or("Codex config destination is not UTF-8")? + .into(), }; if !result.binds.contains(&bind) { result.binds.push(bind); } } } - let mut ordinary = result.discovery.resource_roots.clone(); - for input in &result.discovery.inputs { - if input.kind == InputKind::Configuration { - continue; - } - if !ordinary - .iter() - .any(|root| input.logical_path.starts_with(root)) - { - ordinary.push(input.logical_path.clone()); - } - } ordinary.sort(); ordinary.dedup(); let mut units = BTreeMap::::new(); @@ -111,7 +125,7 @@ pub(super) fn prepare( }); (name, kind) }); - for destination in destinations(&result.discovery.manifest, &logical) { + for destination in destinations(&runtime_path, &logical) { result.mounts.push(ProviderResourceMount { stage_relative_path: format!(".freshell/resource-units/{name}"), provider_relative_path: format!(".freshell/provider-roots/{name}"), @@ -119,7 +133,7 @@ pub(super) fn prepare( .to_str() .ok_or("Codex resource destination is not UTF-8")? .into(), - effective_destination: effective(&destination, &source, &result.discovery.manifest) + effective_destination: effective(&destination, &source, cwd) .to_str() .ok_or("Codex resource destination is not UTF-8")? .into(), @@ -129,17 +143,16 @@ pub(super) fn prepare( } Ok(result) } -fn destinations(manifest: &CodexInputManifest, logical: &Path) -> Vec { - let runtime = manifest.runtime_path(logical); +fn destinations(runtime_path: &impl Fn(&Path) -> PathBuf, logical: &Path) -> Vec { + let runtime = runtime_path(logical); if runtime == logical { vec![runtime] } else { vec![runtime, logical.into()] } } -fn effective(destination: &Path, canonical: &Path, manifest: &CodexInputManifest) -> PathBuf { - if destination.starts_with(&manifest.cwd) && !destination.starts_with("/home/freshell/provider") - { +fn effective(destination: &Path, canonical: &Path, cwd: &Path) -> PathBuf { + if destination.starts_with(cwd) && !destination.starts_with("/home/freshell/provider") { canonical.into() } else { destination.into() @@ -153,15 +166,14 @@ pub(super) fn publish( ) -> Result { use sha2::{Digest, Sha256}; private_directory(&root.join(".freshell"))?; - let bytes = bounded_json_bytes(&preparation.discovery.manifest, 64 * 1024 * 1024) - .map_err(|_| "Codex source metadata budget exceeded")?; - write_private_bytes(&root.join(".freshell/input-manifest.json"), &bytes)?; + let bytes = &preparation.manifest_bytes; + write_private_bytes(&root.join(".freshell/input-manifest.json"), bytes)?; let inputs = PreparedProviderInputs { generation_id: capability.grant_id.clone(), initial_auth_reservation: initial_reservation.map(str::to_string), manifest: ProviderInputManifestReference { relative_path: ".freshell/input-manifest.json".into(), - sha256: format!("{:x}", Sha256::digest(&bytes)), + sha256: format!("{:x}", Sha256::digest(bytes)), source_count: 0, alias_count: 0, recipe_count: 0, From 9904e6296bb38e545d488486f219db1c120c8c63 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 03:45:45 -0700 Subject: [PATCH 078/493] fix: prepare complete Claude inputs for terminal and fresh native launches --- crates/freshell-runtime-protocol/src/lib.rs | 28 ++ .../freshell-server/src/fresh_agent_proxy.rs | 22 ++ .../src/managed_mcp_capability.rs | 197 +++++++++- .../src/managed_provider_launch_tests.rs | 362 ++++++++++++++++++ .../src/managed_provider_references.rs | 1 + .../src/managed_provider_references/claude.rs | 90 +++++ crates/freshell-server/src/managed_runtime.rs | 19 + crates/freshell-session-host/src/main.rs | 2 +- .../src/provider_input_preparation.rs | 79 +++- .../testing/managed-provider-launch-broker.ts | 10 +- .../managed-provider-launch-sandbox.py | 10 +- 11 files changed, 789 insertions(+), 31 deletions(-) create mode 100644 crates/freshell-server/src/managed_provider_references/claude.rs diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index b0cc7313f..ffb7e1c95 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2124,6 +2124,10 @@ pub fn validate_managed_terminal_args(mode: &str, args: &[String]) -> Result<(), let mut index = 0; while index < args.len() { let flag = args[index].as_str(); + if mode == "claude" && flag == "--init-only" { + index += 1; + continue; + } if mode == "codex" && flag == "resume" && index + 1 < args.len() { if !safe_selector(&args[index + 1]) { return Err(invalid()); @@ -3420,6 +3424,30 @@ mod tests { ); } + #[test] + fn claude_native_initialization_argument_retains_selected_plugin_and_rejects_credentials() { + let args = vec![ + "--init-only".into(), + "--plugin-dir".into(), + "/workspace/plugin".into(), + ]; + assert_eq!( + durable_managed_terminal_args("claude", args.clone(), None).unwrap(), + args + ); + assert!(durable_managed_terminal_args( + "claude", + vec![ + "--init-only".into(), + "--api-key".into(), + "private-value".into() + ], + None + ) + .is_err()); + assert!(durable_managed_terminal_args("codex", vec!["--init-only".into()], None).is_err()); + } + #[test] fn managed_claude_shipped_hook_recipe_is_accepted_and_custom_hooks_are_rejected() { use freshell_platform::cli_launch::{claude_settings_json, ProviderTarget}; diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 76eb48602..f034a21fa 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -515,6 +515,28 @@ impl HostedFreshAgentProxy { tracing::warn!(event="fresh_agent.create.provider_inputs_failed",%error); })?; (context.mcp_capability.clone().ok_or(())?, Some(context)) + } else if provider == FreshProvider::Claude { + let home = std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .ok_or(())?; + let context = crate::managed_provider_references::claude::prepare_managed( + crate::managed_provider_references::claude::ClaudePreparation { + client: &self.client, + soul: &soul, + creation_seed: &message.request_id, + home: &home, + cwd: &cwd, + workspace: &workspace, + args: &[], + plugins: message.plugins.as_deref().unwrap_or_default(), + references: &provider_secret_references, + }, + ) + .await + .map_err(|error| { + tracing::warn!(event="fresh_agent.create.provider_inputs_failed",%error); + })?; + (context.mcp_capability.clone().ok_or(())?, Some(context)) } else { let mcp_capability = crate::managed_mcp_capability::issue_mcp_capability( &soul, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index a34732eb2..df6e801f3 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -300,38 +300,72 @@ async fn apply_prepared_callback( ) .map_err(|_| "replacement provider context exceeds control metadata profile".into()); } - if provider == "codex" { + if matches!(provider, "codex" | "claude") { let mut context = callback .prior_context .clone() .ok_or("complete replacement requires prior provider context")?; - let manifest = if context.prepared_inputs.is_some() { - let bytes = - read_prior_manifest_bytes("codex", store, directory, &callback.soul_id, &context)?; - let manifest: freshell_platform::codex_inputs::CodexInputManifest = - serde_json::from_slice(&bytes) - .map_err(|_| "prior Codex source manifest metadata invalid")?; - manifest.validate().map_err(str::to_string)?; - manifest - } else { + if context.prepared_inputs.is_none() { tracing::warn!(event="provider.inputs.preparation.blocked",stage="controller_callback",operation="select_original_sources",provider, soul_id=%callback.soul_id,reason="origin_unknown", "Original source origins are unrecorded; retained inputs were preserved"); - return Err("original Codex source origins are unrecorded; retain the existing inputs and reprepare from their original source before dispatch".into()); - }; + let name = if provider == "codex" { + "Codex" + } else { + "Claude" + }; + return Err(format!("original {name} source origins are unrecorded; retain the existing inputs and reprepare from their original source before dispatch")); + } + let bytes = + read_prior_manifest_bytes(provider, store, directory, &callback.soul_id, &context)?; + let codex_manifest: Option = + if provider == "codex" { + let manifest: freshell_platform::codex_inputs::CodexInputManifest = + serde_json::from_slice(&bytes) + .map_err(|_| "prior Codex source manifest metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + Some(manifest) + } else { + None + }; + let claude_manifest: Option = + if provider == "claude" { + let manifest: freshell_platform::claude_inputs::ClaudeInputManifest = + serde_json::from_slice(&bytes) + .map_err(|_| "prior Claude source manifest metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + Some(manifest) + } else { + None + }; + let (original_home, original_cwd, original_workspace) = + match (&codex_manifest, &claude_manifest) { + (Some(manifest), None) => ( + &manifest.original_home, + &manifest.cwd, + callback.workspace.as_deref(), + ), + (None, Some(manifest)) => ( + &manifest.original_home, + &manifest.cwd, + Some(manifest.workspace.as_path()), + ), + _ => return Err("replacement ordinary source manifest selection invalid".into()), + }; context.config.clear(); let references = callback.provider_secret_references.unwrap_or_default(); let capability = store.issue_replacement_with_inputs( callback.soul_id, directory, endpoint.into(), - Some((provider, &manifest.original_home, &context.config)), + Some((provider, original_home, &context.config)), OpencodeEphemeralInput { launch_context: Some(&context), secret_references: Some(&references), - cwd: Some(&manifest.cwd), - workspace: callback.workspace.as_deref(), - codex_manifest: Some(&manifest), + cwd: Some(original_cwd), + workspace: original_workspace, + codex_manifest: codex_manifest.as_ref(), + claude_manifest: claude_manifest.as_ref(), ..Default::default() }, )?; @@ -341,12 +375,12 @@ async fn apply_prepared_callback( }; reply .validate(provider) - .map_err(|_| "replacement Codex context metadata invalid")?; + .map_err(|_| "replacement ordinary context metadata invalid")?; return freshell_runtime_protocol::bounded_json_bytes( &reply, freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, ) - .map_err(|_| "replacement Codex context exceeds control metadata profile".into()); + .map_err(|_| "replacement ordinary context exceeds control metadata profile".into()); } if provider != "opencode" { let references = provider_source_references(provider, home)?; @@ -1801,12 +1835,12 @@ impl CapabilityStore { provider_relative_path: CAPABILITY_PATH.into(), host_gateway_address: routed_host_ipv4(), }); - if provider == "codex" { + if matches!(provider, "codex" | "claude") { let bytes = source_stage::read_source_view(root, ".freshell/prepared-inputs.json")?; context.prepared_inputs = Some( serde_json::from_str(&bytes) - .map_err(|_| "Codex prepared inputs unavailable")?, + .map_err(|_| "ordinary prepared inputs unavailable")?, ); context.config.clear(); } @@ -4181,6 +4215,129 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[tokio::test] + async fn managed_provider_claude_replacement_retains_original_sources_and_pending_replay() { + use freshell_runtime_protocol::{ + PreparedProviderContext, ProviderLaunchContext, ProviderPreparation, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("original-home"); + let wrong = owned.path().join("current-home"); + let workspace = owned.path().join("workspace"); + for path in [ + &home.join(".claude/skills/probe"), + &wrong.join(".claude"), + &workspace, + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + home.join(".claude/settings.json"), + r#"{"env":{"CLAUDE_SELECTED":"original"}}"#, + ) + .unwrap(); + std::fs::write( + home.join(".claude/skills/probe/resource"), + b"retained-original-resource", + ) + .unwrap(); + std::fs::write( + wrong.join(".claude/settings.json"), + r#"{"env":{"CLAUDE_SELECTED":"wrong"}}"#, + ) + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("claude", &home, &context.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let context = read_prepared_context(&directory, &prior).unwrap(); + assert!(context.prepared_inputs.is_some()); + let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"claude","workspace":workspace,"cwd":workspace,"priorContext":context,"providerSecretReferences":[]})).unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let first = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &wrong, + &client, + ) + .await + .expect("replacement must select recorded original Claude sources"); + std::fs::remove_file(home.join(".claude/settings.json")).unwrap(); + let repeated = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &wrong, + &client, + ) + .await + .unwrap(); + assert_eq!( + serde_json::from_slice::(&first).unwrap(), + serde_json::from_slice::(&repeated).unwrap(), + "pending generation replay must precede original rescanning" + ); + let ready: PreparedProviderContext = serde_json::from_slice(&repeated).unwrap(); + let inputs = ready + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap(); + let mount = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == "/home/freshell/provider/.claude/skills") + .unwrap(); + let stage = staged_provider_root_path( + &directory, + &ready + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap() + .grant_id, + ) + .unwrap(); + assert_eq!( + std::fs::read( + stage + .join(&mount.stage_relative_path) + .join("probe/resource") + ) + .unwrap(), + b"retained-original-resource" + ); + } + #[tokio::test] async fn managed_provider_replacement_complete_codex_metadata_replays_before_rescanning_roots() { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index dfd8a3a55..208bb480a 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -215,6 +215,368 @@ impl RuntimeBackend for BrokerAuthorizedBackend { } } +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prepared_inputs() { + use freshell_supervisor::{ + admission::AdmissionPolicy, + registry::Registry, + service::{serve_control, Supervisor, SupervisorConfig}, + }; + use freshell_terminal::registry::{ManagedTerminalController, ManagedTerminalLaunch}; + use freshell_ws::hosted_fresh_agent::{HostedFreshAgentCommand, HostedFreshAgentGateway}; + use sha2::{Digest, Sha256}; + use std::{os::unix::fs::PermissionsExt, sync::Arc}; + let fixture = PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_FIXTURE").expect("closed Claude launch fixture"), + ); + assert_eq!(fixture.parent(), Some(Path::new("/tmp"))); + assert!(fixture + .file_name() + .unwrap() + .to_str() + .unwrap() + .starts_with("fln-")); + let home = fixture.join("source-home"); + let workspace = fixture.join("workspace"); + let plugin = home.join("selected-plugin"); + let control = fixture.join("scenarios/native/control"); + let runtime = fixture.join("scenarios/n/r"); + for path in [ + &control, + &runtime, + &home.join(".claude"), + &plugin.join(".claude-plugin"), + &plugin.join("hooks"), + &plugin.join("scripts/node_modules/probe-dependency"), + &plugin.join("skills/probe"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + assert!(std::process::Command::new("git") + .args(["init", "--quiet"]) + .arg(&workspace) + .status() + .unwrap() + .success()); + let receipt = workspace.join("claude-consumed.json"); + std::fs::write( + home.join(".claude/CLAUDE.md"), + b"closed-claude-global-memory", + ) + .unwrap(); + let settings = r#"{"env":{"CLAUDE_SELECTED_NONCE":"closed-selected-settings"}}"#; + std::fs::write(home.join(".claude/settings.json"), settings).unwrap(); + std::fs::write(plugin.join("skills/probe/SKILL.md"),"---\nname: closed-plugin-probe\ndescription: selected complete native plugin\n---\nUse the adjacent resources.\n").unwrap(); + std::fs::write(plugin.join("scripts/resource"), b"closed-claude-resource").unwrap(); + std::fs::write( + plugin.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='closed-claude-dependency'\n", + ) + .unwrap(); + std::fs::write( + plugin.join("scripts/run"), + "#!/bin/sh\ncat \"$(dirname \"$0\")/resource\"\n", + ) + .unwrap(); + std::fs::set_permissions( + plugin.join("scripts/run"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + let consumer = r#"const fs=require('node:fs'),path=require('node:path'),crypto=require('node:crypto'),http=require('node:http'); +const hash=v=>crypto.createHash('sha256').update(v).digest('hex'); +let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ + const input=JSON.parse(raw),root=path.resolve(__dirname,'..'); + const endpoint=process.env.FRESHELL_URL,token=process.env.FRESHELL_TOKEN; + const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); + const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,settings:process.env.CLAUDE_SELECTED_NONCE==='closed-selected-settings',authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], + memory:hash(fs.readFileSync(path.join(process.env.HOME,'.claude/CLAUDE.md'))),skill:hash(fs.readFileSync(path.join(root,'skills/probe/SKILL.md'))),resource:hash(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:hash(require('probe-dependency'))}; + fs.writeFileSync(process.argv[2],JSON.stringify(record)); +}); +"#; + std::fs::write(plugin.join("scripts/consume.cjs"), consumer).unwrap(); + let manifest = json!({"name":"closed-selected-plugin","version":"1.0.0","skills":"./skills","hooks":"./hooks/hooks.json"}); + std::fs::write( + plugin.join(".claude-plugin/plugin.json"), + serde_json::to_vec(&manifest).unwrap(), + ) + .unwrap(); + let key_digest = format!("{:x}", Sha256::digest(b"synthetic-claude-onecli-value")); + let hooks = json!({"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":format!("node \"${{CLAUDE_PLUGIN_ROOT}}/scripts/consume.cjs\" {} {}",serde_json::to_string(receipt.to_str().unwrap()).unwrap(),key_digest),"timeout":10}]}]}}); + std::fs::write( + plugin.join("hooks/hooks.json"), + serde_json::to_vec(&hooks).unwrap(), + ) + .unwrap(); + let validated = std::process::Command::new("claude") + .args(["plugin", "validate"]) + .arg(&plugin) + .env("HOME", &home) + .output() + .unwrap(); + assert!( + validated.status.success(), + "native terminal plugin schema prerequisite failed" + ); + let env_file = fixture.join("grants/environment.env"); + std::fs::write( + &env_file, + "ANTHROPIC_API_KEY=synthetic-claude-onecli-value\nANTHROPIC_BASE_URL=http://127.0.0.1:9\n", + ) + .unwrap(); + std::fs::set_permissions(&env_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + std::env::set_var("FRESHELL_MANAGED_CLAUDE_ONECLI_ENV_FILE", &env_file); + let registry = Registry::open( + fixture.join("registry"), + Some( + InstallationId::parse(std::env::var("FRESHELL_PROVIDER_LAUNCH_INSTALLATION").unwrap()) + .unwrap(), + ), + ) + .unwrap(); + std::env::set_var("FRESHELL_MANAGED_PROVIDER_QUALIFICATION", "claude"); + let socket = control.join("control.sock"); + let secret_file = control.join("secret"); + std::fs::write(&secret_file, "closed-synthetic-control-secret").unwrap(); + std::fs::set_permissions(&secret_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + freshell_supervisor::backend::docker::configure_capability_dir(&socket).unwrap(); + let supervisor = Supervisor::new( + registry.clone(), + Arc::new(BrokerAuthorizedBackend { + real: DockerEngineBackend::new(fixture.join("broker.sock")), + control: control.clone(), + handoff: fixture.join("handoff"), + certificate_sentinel: b"synthetic-claude-onecli-value".to_vec(), + }), + SupervisorConfig { + runtime_root: runtime, + control_socket_path: socket.clone(), + host_binary_path: PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_HOST_BINARY").unwrap(), + ), + image_ref: std::env::var("FRESHELL_PROVIDER_LAUNCH_IMAGE").unwrap(), + test_run_id: std::env::var("FRESHELL_PROVIDER_LAUNCH_RUN_ID").unwrap(), + control_secret: "closed-synthetic-control-secret".into(), + lifecycle_log: fixture.join("lifecycle.jsonl"), + admission: AdmissionPolicy::default(), + }, + ) + .unwrap(); + let control_task = tokio::spawn({ + let socket = socket.clone(); + async move { + serve_control(supervisor, &socket).await.unwrap(); + } + }); + tokio::time::timeout(Duration::from_secs(5), async { + while !socket.exists() { + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .unwrap(); + let web = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap(); + let port = web.local_addr().unwrap().port(); + let app = axum::Router::new() + .route("/api/health", axum::routing::get(|| async { "ok" })) + .route( + "/api/mcp", + axum::routing::post(|headers: axum::http::HeaderMap| async move { + if headers + .get("x-auth-token") + .is_some_and(|token| token == "closed-synthetic-web-token") + { + axum::http::StatusCode::OK + } else { + axum::http::StatusCode::UNAUTHORIZED + } + }), + ) + .layer(axum::middleware::from_fn_with_state( + Arc::new("closed-synthetic-web-token".to_string()), + crate::managed_mcp_capability::scoped_auth_middleware, + )); + let web_task = tokio::spawn(async move { axum::serve(web, app).await.unwrap() }); + crate::managed_mcp_capability::configure("0.0.0.0", port, &socket, &secret_file).unwrap(); + crate::managed_mcp_capability::spawn_callback_server() + .await + .unwrap(); + for name in [ + "FRESHELL_MANAGED_RUNTIME_V1", + "FRESHELL_MANAGED_FRESH_AGENT_V1", + ] { + std::env::set_var(name, "1"); + } + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SOCKET", &socket); + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SECRET_FILE", &secret_file); + std::env::set_var("HOME", &home); + let controller = crate::managed_runtime::ServerManagedRuntimeController::from_env() + .await + .unwrap() + .unwrap(); + let cwd = workspace.to_str().unwrap().to_string(); + let terminal = controller + .launch(ManagedTerminalLaunch { + spec: freshell_platform::spawn::SpawnSpec { + program: "claude".into(), + args: vec![ + "--init-only".into(), + "--plugin-dir".into(), + plugin.to_str().unwrap().into(), + ], + env_overrides: BTreeMap::new(), + cwd: Some(cwd.clone()), + cols: 120, + rows: 30, + }, + env: BTreeMap::new(), + terminal_id: "closed-native-terminal".into(), + stream_id: "closed-native-stream".into(), + mode: "claude".into(), + resume_session_id: None, + provider_model: None, + provider_reasoning_effort: None, + provider_sandbox: None, + provider_permission_mode: None, + view_tab_id: None, + view_pane_id: None, + create_request_id: Some("closed-native-terminal-create".into()), + }) + .await + .expect("complete native Claude terminal startup"); + let client = controller.runtime_client(); + async fn consumed(path: &Path) -> Option { + tokio::time::timeout(Duration::from_secs(45), async { + loop { + if let Ok(bytes) = std::fs::read(path) { + if let Ok(value) = serde_json::from_slice(&bytes) { + break value; + } + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await + .ok() + } + let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); + let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"memory":hash(b"closed-claude-global-memory"),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}); + let observed = consumed(&receipt).await; + if observed.is_none() { + let output = client + .read_output( + SoulId::parse(terminal.soul_id.clone()).unwrap(), + 0, + 64 * 1024, + ) + .await + .unwrap(); + let text = output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(); + let text = crate::ai_title::strip_ansi(&text); + let classes = [ + "ENOENT", + "EACCES", + "trust", + "Trust", + "onboarding", + "Sign in", + "API key", + "plugin", + "Plugin", + "settings", + "MCP", + "Permission denied", + "text style", + "theme", + "Theme", + "Welcome", + "Let's", + "Enter", + "Continue", + "terms", + "Token", + "Authentication", + "Setup", + "select", + "Select", + ] + .into_iter() + .filter(|class| text.contains(class)) + .collect::>(); + println!( + "provider_launch_diagnostic {}", + json!({"provider":"claude","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes}) + ); + } + assert_eq!( + observed.expect("native Claude selected plugin must execute its startup hook"), + expected + ); + println!( + "provider_launch_consumer {}", + json!({"provider":"claude","route":"terminal","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + ); + client + .stop(SoulId::parse(terminal.soul_id).unwrap()) + .await + .unwrap(); + std::fs::remove_file(&receipt).unwrap(); + let (broadcast, _) = tokio::sync::broadcast::channel(256); + let mut events = broadcast.subscribe(); + let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( + Some(client.clone()), + Arc::new(broadcast), + ) + .unwrap() + .unwrap(); + Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":"freshclaude","provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; + tokio::time::timeout(Duration::from_secs(45), async { + loop { + let frame: Value = serde_json::from_str(&events.recv().await.unwrap()).unwrap(); + if frame["type"] == "freshAgent.create.failed" { + panic!( + "fresh Claude startup failed classification: {}", + frame["message"] + ); + } + if frame["type"] == "freshAgent.created" { + break; + } + } + }) + .await + .unwrap(); + assert_eq!( + consumed(&receipt) + .await + .expect("genuine default SDK selected plugin must execute its startup hook"), + expected + ); + println!( + "provider_launch_consumer {}", + json!({"provider":"claude","route":"fresh","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + ); + assert_eq!( + std::fs::read_to_string(home.join(".claude/settings.json")).unwrap(), + settings + ); + assert_eq!( + std::fs::read_to_string(plugin.join("scripts/consume.cjs")).unwrap(), + consumer + ); + for view in client.inventory().await.unwrap() { + if view.desired_state == DesiredState::Running { + client.stop(view.soul_id).await.unwrap(); + } + } + web_task.abort(); + control_task.abort(); +} + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn managed_provider_codex_terminal_and_fresh_native_startup_consumes_prepared_inputs() { use freshell_supervisor::{ diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 09c7f73ed..247e18006 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -2,6 +2,7 @@ //! Plans retain approved source metadata and unexpanded templates, never payload bytes. use std::path::{Path, PathBuf}; +pub(crate) mod claude; pub(crate) mod codex; pub(crate) mod environment; pub(crate) mod sources; diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs new file mode 100644 index 000000000..a23471acc --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -0,0 +1,90 @@ +//! Finite original Claude source selection and ordinary first-volume authority. +use freshell_runtime_protocol::*; +use std::path::Path; + +pub(crate) struct ClaudePreparation<'a> { + pub client: &'a freshell_runtime_client::RuntimeClient, + pub soul: &'a SoulId, + pub creation_seed: &'a str, + pub home: &'a Path, + pub cwd: &'a Path, + pub workspace: &'a Path, + pub args: &'a [String], + pub plugins: &'a [String], + pub references: &'a [ProviderSecretReference], +} + +pub(crate) async fn prepare_managed( + request: ClaudePreparation<'_>, +) -> Result { + let mut plugin_dirs = Vec::new(); + let mut path_bytes = 0usize; + let cli = request + .args + .windows(2) + .filter(|pair| pair[0] == "--plugin-dir") + .map(|pair| pair[1].as_str()); + for selected in cli.chain(request.plugins.iter().map(String::as_str)) { + let path = Path::new(selected); + let path = if path.is_absolute() { + path.to_path_buf() + } else { + request.cwd.join(path) + }; + if plugin_dirs.contains(&path) { + continue; + } + path_bytes = path_bytes + .checked_add(path.as_os_str().len()) + .ok_or("Claude plugin source metadata overflow")?; + if plugin_dirs.len() >= 8192 || path_bytes > 8 * 1024 * 1024 { + return Err("Claude plugin source event metadata profile exceeded".into()); + } + plugin_dirs.push(path); + } + let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { + version: 1, + original_home: request.home.into(), + cwd: request.cwd.into(), + workspace: request.workspace.into(), + plugin_dirs, + }; + manifest.validate().map_err(str::to_string)?; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { + mcp_args: vec!["--mcp-config".into(), ".claude/freshell-mcp.json".into()], + }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + let reservation = request + .client + .reserve_provider_inputs( + request.soul.clone(), + "claude".into(), + request.creation_seed.into(), + ) + .await + .map_err(|error| error.to_string())?; + if reservation.provider != "claude" + || &reservation.soul_id != request.soul + || reservation.creation_seed_ref != request.creation_seed + { + return Err("Claude selected resource reservation does not match this creation".into()); + } + crate::managed_mcp_capability::issue_prepared_provider_context( + "claude", + request.soul, + request.home, + &context, + crate::managed_mcp_capability::OpencodeEphemeralInput { + claude_manifest: Some(&manifest), + cwd: Some(request.cwd), + workspace: Some(request.workspace), + secret_references: Some(request.references), + initial_auth_reservation: Some(&reservation.reservation_id), + ..Default::default() + }, + ) +} diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 6598f9ae6..b2cafe1ae 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -254,6 +254,25 @@ impl ManagedTerminalController for ServerManagedRuntimeController { ) .await?; (context.mcp_capability.clone(), Some(context)) + } else if request.mode == "claude" { + let home = std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .ok_or("original Claude HOME is unavailable")?; + let context = crate::managed_provider_references::claude::prepare_managed( + crate::managed_provider_references::claude::ClaudePreparation { + client: &self.client, + soul: &soul_id, + creation_seed: create_key, + home: &home, + cwd: &cwd, + workspace: &workspace, + args: &args, + plugins: &[], + references: &provider_secret_references, + }, + ) + .await?; + (context.mcp_capability.clone(), Some(context)) } else { let mcp_capability = matches!( request.mode.as_str(), diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index ec21acb40..a6cd300b9 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -886,7 +886,7 @@ async fn grant_execution( format!("resolve managed provider credentials: {error}"), ) })?; - if matches!(provider, "opencode" | "codex") { + if matches!(provider, "opencode" | "codex" | "claude") { let generated = provider_input_preparation::prepare( context, &mut secrets, diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 60ea46ae0..12516dadd 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -55,16 +55,22 @@ pub fn prepare( if format!("{:x}", Sha256::digest(&metadata)) != inputs.manifest.sha256 { return Err("prepared source manifest changed before dispatch".into()); } - if context.unwrap().preparation.provider() == "codex" { + let provider = context.unwrap().preparation.provider(); + if matches!(provider, "codex" | "claude") { if !frozen.is_empty() || inputs.manifest.source_count != 0 || inputs.manifest.alias_count != 0 || inputs.manifest.recipe_count != 0 || inputs.indexed_files.is_some() { - return Err( - "Codex ordinary source context carries foreign private reference inputs".into(), - ); + return Err("ordinary source context carries foreign private reference inputs".into()); + } + if provider == "claude" { + let manifest: freshell_platform::claude_inputs::ClaudeInputManifest = + serde_json::from_slice(&metadata) + .map_err(|_| "prepared Claude discovery metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + return Ok(BTreeMap::new()); } let manifest: freshell_platform::codex_inputs::CodexInputManifest = serde_json::from_slice(&metadata) @@ -285,6 +291,71 @@ mod tests { use freshell_platform::opencode_recipe::*; use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + #[test] + fn provider_secret_claude_prepared_manifest_validates_without_original_controller_tree() { + let owned = tempfile::tempdir().unwrap(); + std::fs::create_dir(owned.path().join(".freshell")).unwrap(); + let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { + version: 1, + original_home: "/absent-original-home".into(), + cwd: "/absent-original-workspace".into(), + workspace: "/absent-original-workspace".into(), + plugin_dirs: vec!["/absent-original-plugin".into()], + }; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let mut context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + config: vec![], + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }), + mcp_capability: Some(capability), + }; + let mut secrets = ResolvedProviderSecrets::default(); + let generated=prepare(Some(&context),&mut secrets,&ProviderPathInputs::default(),owned.path(),owned.path()) + .expect("Claude source origins validate from accepted private metadata, without rescanning controller paths"); + assert!(generated.is_empty()); + assert!( + secrets.opencode_plan.is_none(), + "ordinary Claude cannot gain OpenCode source/auth semantics" + ); + context + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .source_count = 1; + assert!( + prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + owned.path(), + owned.path() + ) + .is_err(), + "foreign source recipes cannot enter ordinary Claude metadata" + ); + } + #[test] fn provider_secret_codex_prepared_manifest_retains_native_home_without_original_tree() { let owned = tempfile::tempdir().unwrap(); diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 9ba50aa89..95669b836 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -9,8 +9,10 @@ if (path.dirname(root)!=='/tmp' || !/^fln-[a-f0-9]{16}$/.test(path.basename(root throw new Error('closed provider startup requires its exact owned fixture root') } const configuration=JSON.parse(fs.readFileSync(path.join(root,'broker-configuration.json'),'utf8')) -const image='sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c' -if (configuration.imageRef!==image || !['opencode','codex'].includes(configuration.provider) || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==2) { +const image=configuration.provider==='claude' + ? 'sha256:b09595b8f02dae505e6719d7617305ab8f5ad53a7138b3b1580ac553c0f2663a' + : 'sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c' +if (configuration.imageRef!==image || !['opencode','codex','claude'].includes(configuration.provider) || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==2) { throw new Error('closed provider startup image or finite source set changed') } const gateways=new Set() @@ -19,7 +21,9 @@ const broker=new RestrictedDockerBroker({ runtimeRootPrefix:path.join(root,'scenarios'),allowedHostBinaryPaths:new Set([configuration.hostBinaryPath]), allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), - allowedProviderUserRoots:new Set(configuration.provider==='codex' + allowedProviderUserRoots:new Set(configuration.provider==='claude' + ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-plugin')] + : configuration.provider==='codex' ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] : ['/home/freshell/provider',path.join(root,'.opencode')]), allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json'),path.join(root,'grants/ca.pem')]), diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index f31d22d95..2e80f31b6 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -13,6 +13,8 @@ IMAGE="sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" TEST="managed_provider_launch_tests::managed_provider_opencode_terminal_and_fresh_native_startup_consumes_prepared_inputs" CODEX_TEST="managed_provider_launch_tests::managed_provider_codex_terminal_and_fresh_native_startup_consumes_prepared_inputs" +CLAUDE_TEST="managed_provider_launch_tests::managed_provider_claude_terminal_and_fresh_native_startup_consumes_prepared_inputs" +CLAUDE_IMAGE="sha256:b09595b8f02dae505e6719d7617305ab8f5ad53a7138b3b1580ac553c0f2663a" LABEL="com.freshell.provider-launch-probe" def docker(*args,check=True,timeout=180): @@ -23,14 +25,16 @@ def digest(path): return hashlib.file_digest(stream,"sha256").hexdigest() def main(): + global IMAGE parser=argparse.ArgumentParser(description=__doc__) parser.add_argument("--test-binary",type=Path,required=True) parser.add_argument("--host-binary",type=Path,required=True) parser.add_argument("--broker-binary",type=Path,required=True) parser.add_argument("--receipt-dir",type=Path,required=True) - parser.add_argument("--provider",choices=["opencode","codex"],default="opencode") + parser.add_argument("--provider",choices=["opencode","codex","claude"],default="opencode") args=parser.parse_args() - selected_test=CODEX_TEST if args.provider=="codex" else TEST + selected_test={"opencode":TEST,"codex":CODEX_TEST,"claude":CLAUDE_TEST}[args.provider] + if args.provider=="claude": IMAGE=CLAUDE_IMAGE repo=Path(__file__).resolve().parents[2] test=args.test_binary.resolve(strict=True);host=args.host_binary.resolve(strict=True);broker=args.broker_binary.resolve(strict=True) if test.parent!=repo/"target/debug/deps" or not re.fullmatch(r"freshell_server-[a-f0-9]{16}",test.name) or host!=repo/"target/debug/freshell-session-host" or broker!=repo/"target/managed-provider-launch-broker.mjs" or not args.receipt_dir.is_dir(): @@ -174,7 +178,7 @@ def create(name,*arguments): receipt["exitCode"]=exit_code receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) - expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else {("codex","terminal"),("codex","fresh")} + expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else {(args.provider,"terminal"),(args.provider,"fresh")} observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} if exit_code==0 and (len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes or "test result: ok. 1 passed;" not in outcome.stdout): receipt["exitCode"]=2;receipt["failureClass"]="native_startup_count_mismatch" From 78c8b15280ca73e7bfa777707b40ff854266d91e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 04:13:41 -0700 Subject: [PATCH 079/493] fix: preserve recursive Claude memory imports in managed inputs --- .../src/managed_mcp_capability.rs | 81 ++++- .../managed_mcp_capability/claude_memory.rs | 310 ++++++++++++++++++ .../managed_mcp_capability/claude_stage.rs | 15 + .../src/managed_provider_launch_tests.rs | 127 ++++++- .../testing/managed-provider-launch-broker.ts | 2 +- 5 files changed, 528 insertions(+), 7 deletions(-) create mode 100644 crates/freshell-server/src/managed_mcp_capability/claude_memory.rs diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index df6e801f3..e1fc061e4 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -28,6 +28,7 @@ const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; mod amplifier_resources; +mod claude_memory; mod claude_stage; mod ordinary_stage; mod source_stage; @@ -1302,6 +1303,7 @@ fn stage_provider_root( .or(opencode_input.cwd) .unwrap_or(home), opencode_input.claude_manifest, + &excluded, )? } else { ordinary_stage::prepare( @@ -2981,7 +2983,10 @@ except Exception as error: r#"{"env":{"CLAUDE_SELECTED_NONCE":"settings-input"}}"#, ) .unwrap(); - std::fs::write(home.path().join(".claude/CLAUDE.md"), "global-memory-input").unwrap(); + std::fs::create_dir_all(home.path().join("memory")).unwrap(); + std::fs::write(home.path().join(".claude/CLAUDE.md"), "global-memory-input\n@../first.md\n`@../missing-inline.md`\n```text\n@../missing-fenced.md\n```\n").unwrap(); + std::fs::write(home.path().join("first.md"), "@./memory/final.md\n").unwrap(); + std::fs::write(home.path().join("memory/final.md"), "imported-memory-input").unwrap(); std::fs::write(skill.join("resource"), "adjacent-resource").unwrap(); std::fs::write( skill.join("node_modules/local-dependency/index.js"), @@ -3044,6 +3049,23 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ "transported project skill must load its installed dependency" ); assert_eq!(consumed.stdout, b"adjacent-resource:installed-dependency"); + let imported = inputs + .resource_mounts + .iter() + .find(|mount| { + mount.logical_destination == home.path().join("memory/final.md").to_str().unwrap() + }) + .expect("recursive imports must retain their declaring Markdown origin"); + let imported = std::process::Command::new("node") + .args([ + "-e", + "process.stdout.write(require(\"node:fs\").readFileSync(process.argv[1]))", + ]) + .arg(stage.join(&imported.stage_relative_path)) + .output() + .unwrap(); + assert!(imported.status.success()); + assert_eq!(imported.stdout, b"imported-memory-input"); assert!( inputs .resource_mounts @@ -3062,6 +3084,63 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[cfg(unix)] + #[test] + fn managed_provider_claude_memory_private_alias_is_rejected_before_publication() { + use freshell_runtime_protocol::{ProviderLaunchContext, ProviderPreparation}; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let capabilities = owned.path().join("capabilities"); + for path in [&home.join(".claude"), &workspace, &capabilities] { + std::fs::create_dir_all(path).unwrap(); + } + let private = home.join(".claude/.credentials.json"); + std::fs::write(&private, "synthetic-private-credential-nonce").unwrap(); + std::os::unix::fs::symlink(&private, home.join("ordinary.md")).unwrap(); + std::fs::write(home.join(".claude/CLAUDE.md"), "@../ordinary.md\n").unwrap(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + let error = stage_provider_root( + &capabilities, + "grant-claude-private-alias", + "claude", + &home, + &[], + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + secret_references: Some(&[]), + ..Default::default() + }, + None, + ) + .expect_err("selected ordinary memory cannot expose a credential alias"); + assert!(error.contains("private") && error.contains("ordinary.md")); + assert!(!error.contains("synthetic-private-credential-nonce")); + assert_eq!( + std::fs::read_to_string(private).unwrap(), + "synthetic-private-credential-nonce" + ); + assert!( + !staged_provider_root_path(&capabilities, "grant-claude-private-alias") + .unwrap() + .exists() + ); + assert_eq!( + std::fs::read_dir(owned.path().join("provider-roots")) + .unwrap() + .count(), + 0, + "failed selection must remove its temporary stage" + ); + } + #[cfg(unix)] #[test] fn native_state_aliases_are_excluded_without_filtering_dependency_basenames() { diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs new file mode 100644 index 000000000..c7ebaf490 --- /dev/null +++ b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs @@ -0,0 +1,310 @@ +//! Select ordinary Markdown imports from their declaring logical origins. +use std::{ + collections::{BTreeMap, BTreeSet}, + io::Read, + path::{Component, Path, PathBuf}, + sync::Arc, +}; + +pub(super) fn imports( + home: &Path, + roots: &[PathBuf], + excluded: &[PathBuf], +) -> Result, String> { + let excluded = excluded + .iter() + .map(|path| std::fs::canonicalize(path).unwrap_or_else(|_| path.clone())) + .collect::>(); + let mut graph = ImportGraph { + home, + excluded: &excluded, + active: BTreeSet::new(), + visited: BTreeSet::new(), + sources: BTreeMap::new(), + bytes: 0, + events: 0, + path_bytes: 0, + selected: Vec::new(), + }; + for root in roots { + graph.visit(root, 0)?; + } + Ok(graph.selected) +} + +struct ImportGraph<'a> { + home: &'a Path, + excluded: &'a [PathBuf], + active: BTreeSet, + visited: BTreeSet, + sources: BTreeMap>, + bytes: u64, + events: usize, + path_bytes: usize, + selected: Vec, +} +impl ImportGraph<'_> { + fn visit(&mut self, logical: &Path, depth: usize) -> Result<(), String> { + let logical = normalize(logical); + let error = |cause: &str| format!("Claude memory import {}: {cause}", logical.display()); + if depth > freshell_platform::provider_environment::CopyLimits::default().max_depth { + return Err(error("resource depth profile exceeded")); + } + let source = std::fs::canonicalize(&logical) + .map_err(|cause| error(&format!("selected file unavailable: {cause}")))?; + if self + .excluded + .iter() + .any(|excluded| source.starts_with(excluded)) + { + return Err(error( + "selected private or native-state source cannot become ordinary memory", + )); + } + if self.active.contains(&source) { + return Err(error("cyclic selected memory import")); + } + if self.visited.contains(&logical) { + return Ok(()); + } + if self.visited.len() >= 256 { + return Err(error("reference alias profile exceeded")); + } + let raw = if let Some(raw) = self.sources.get(&source) { + Arc::clone(raw) + } else { + use freshell_platform::provider_input_exchange::{MAX_FILE_BYTES, MAX_TOTAL_BYTES}; + let file = std::fs::File::open(&source) + .map_err(|cause| error(&format!("selected file unreadable: {cause}")))?; + let metadata = file + .metadata() + .map_err(|cause| error(&format!("selected file metadata failed: {cause}")))?; + if !metadata.is_file() || metadata.len() > MAX_FILE_BYTES { + return Err(error("reference source type or per-file profile exceeded")); + } + let next = self + .bytes + .checked_add(metadata.len()) + .ok_or_else(|| error("reference source accounting overflow"))?; + if self.sources.len() >= 128 || next > MAX_TOTAL_BYTES { + return Err(error("reference source aggregate profile exceeded")); + } + let mut raw = String::new(); + file.take(MAX_FILE_BYTES + 1) + .read_to_string(&mut raw) + .map_err(|cause| error(&format!("selected Markdown read failed: {cause}")))?; + let next = self + .bytes + .checked_add(raw.len() as u64) + .ok_or_else(|| error("reference source accounting overflow"))?; + if raw.len() as u64 > MAX_FILE_BYTES || next > MAX_TOTAL_BYTES { + return Err(error("reference source grew beyond its profile")); + } + self.bytes = next; + let raw: Arc = raw.into(); + self.sources.insert(source.clone(), Arc::clone(&raw)); + raw + }; + self.active.insert(source.clone()); + self.visited.insert(logical.clone()); + for selected in paths(&raw) { + let selected = selected.map_err(error)?; + self.events = self + .events + .checked_add(1) + .ok_or_else(|| error("source event accounting overflow"))?; + self.path_bytes = self + .path_bytes + .checked_add(selected.len()) + .ok_or_else(|| error("source path accounting overflow"))?; + if self.events > 8192 || self.path_bytes > 8 * 1024 * 1024 { + return Err(error("source event metadata profile exceeded")); + } + let selected = if let Some(relative) = selected.strip_prefix("~/") { + self.home.join(relative) + } else if Path::new(&selected).is_absolute() { + selected.into() + } else { + logical + .parent() + .ok_or_else(|| error("declaring parent unavailable"))? + .join(selected) + }; + let selected = normalize(&selected); + if selected.as_os_str().len() > 64 * 1024 + || selected.to_string_lossy().chars().any(char::is_control) + { + return Err(error("selected import path profile exceeded")); + } + self.visit(&selected, depth + 1)?; + if !self.selected.contains(&selected) { + self.selected.push(selected); + } + } + self.active.remove(&source); + Ok(()) + } +} + +fn normalize(path: &Path) -> PathBuf { + let mut result = PathBuf::new(); + for component in path.components() { + match component { + Component::CurDir => {} + Component::ParentDir => { + result.pop(); + } + other => result.push(other.as_os_str()), + } + } + result +} + +/// Markdown code fences and matching backtick spans cannot select imports. +/// Quoted tokens are not paths; escaped spaces remain part of one path. +fn paths(raw: &str) -> ImportPaths<'_> { + ImportPaths { + lines: raw.lines(), + line: "", + index: 0, + fence: None, + span: 0, + } +} +struct ImportPaths<'a> { + lines: std::str::Lines<'a>, + line: &'a str, + index: usize, + fence: Option<(u8, usize)>, + span: usize, +} +impl Iterator for ImportPaths<'_> { + type Item = Result; + fn next(&mut self) -> Option { + loop { + if self.index >= self.line.len() { + self.line = self.lines.next()?; + self.index = 0; + let trimmed = self.line.trim_start_matches(' '); + let bytes = trimmed.as_bytes(); + let marker = bytes.first().copied(); + let count = bytes + .iter() + .take_while(|byte| Some(**byte) == marker) + .count(); + if self.line.len() - trimmed.len() <= 3 + && matches!(marker, Some(b'`' | b'~')) + && count >= 3 + { + if let Some((active, width)) = self.fence { + if marker == Some(active) + && count >= width + && trimmed[count..].trim().is_empty() + { + self.fence = None; + } + } else if self.span == 0 { + self.fence = Some((marker.unwrap(), count)); + } + self.index = self.line.len(); + continue; + } + if self.fence.is_some() { + self.index = self.line.len(); + continue; + } + } + let bytes = self.line.as_bytes(); + while self.index < bytes.len() { + if bytes[self.index] == b'`' { + let count = bytes[self.index..] + .iter() + .take_while(|byte| **byte == b'`') + .count(); + if self.span == 0 { + self.span = count; + } else if self.span == count { + self.span = 0; + } + self.index += count; + continue; + } + if self.span == 0 + && bytes[self.index] == b'@' + && (self.index == 0 || bytes[self.index - 1].is_ascii_whitespace()) + { + let start = self.index + 1; + self.index = start; + while self.index < bytes.len() { + if bytes[self.index] == b'\\' && bytes.get(self.index + 1) == Some(&b' ') { + self.index += 2; + } else if bytes[self.index].is_ascii_whitespace() + || matches!(bytes[self.index], b'`' | b'\'' | b'"') + { + break; + } else { + self.index += 1; + } + } + if self.index - start > 64 * 1024 { + return Some(Err("selected import path profile exceeded")); + } + if self.index > start { + return Some(Ok(self.line[start..self.index].replace("\\ ", " "))); + } + } else { + self.index += 1; + } + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn selected_memory_imports_resolve_home_absolute_and_escaped_spaces() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir_all(home.join(".claude")).unwrap(); + let absolute = owned.path().join("absolute.md"); + std::fs::write(&absolute, "absolute-memory").unwrap(); + std::fs::write(home.join("spaced memory.md"), "home-memory").unwrap(); + let memory = home.join(".claude/CLAUDE.md"); + std::fs::write( + &memory, + format!( + "@~/spaced\\ memory.md\n@{}\n`@missing-inline`\n~~~\n@missing-fence\n~~~\n", + absolute.display() + ), + ) + .unwrap(); + let selected = imports(&home, &[memory], &[]).unwrap(); + assert_eq!(selected.len(), 2); + let values = selected + .iter() + .map(|path| std::fs::read_to_string(path).unwrap()) + .collect::>(); + assert_eq!( + values, + BTreeSet::from(["absolute-memory".into(), "home-memory".into()]) + ); + } + + #[test] + fn selected_memory_missing_and_cyclic_imports_fail_without_changing_sources() { + let owned = tempfile::tempdir().unwrap(); + let root = owned.path().join("CLAUDE.md"); + let nested = owned.path().join("nested.md"); + std::fs::write(&root, "@nested.md\n").unwrap(); + let missing = imports(owned.path(), &[root.clone()], &[]).unwrap_err(); + assert!(missing.contains("nested.md") && missing.contains("unavailable")); + std::fs::write(&nested, "@CLAUDE.md\n").unwrap(); + let cycle = imports(owned.path(), &[root.clone()], &[]).unwrap_err(); + assert!(cycle.contains("cyclic")); + assert_eq!(std::fs::read_to_string(root).unwrap(), "@nested.md\n"); + assert_eq!(std::fs::read_to_string(nested).unwrap(), "@CLAUDE.md\n"); + } +} diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 117fbccf2..7ce4fff6f 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -8,6 +8,7 @@ pub(super) fn prepare( cwd: &Path, workspace: &Path, accepted: Option<&ClaudeInputManifest>, + excluded: &[PathBuf], ) -> Result { let manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { version: 1, @@ -19,6 +20,7 @@ pub(super) fn prepare( manifest.validate().map_err(str::to_string)?; let mut configurations = Vec::new(); let mut resources = Vec::new(); + let mut memory = Vec::new(); let mut bases: Vec = manifest.cwd.ancestors().map(Path::to_path_buf).collect(); bases.push(manifest.original_home.clone()); bases.sort(); @@ -52,6 +54,12 @@ pub(super) fn prepare( ] { let logical = base.join(relative); if optional(&logical)? { + if matches!( + relative, + "CLAUDE.md" | "CLAUDE.local.md" | ".claude/CLAUDE.md" + ) { + memory.push(logical.clone()); + } resources.push(logical); } } @@ -60,6 +68,13 @@ pub(super) fn prepare( canonical(plugin)?; // Explicit selections may never be silently omitted. resources.push(plugin.clone()); } + for imported in super::claude_memory::imports(&manifest.original_home, &memory, excluded).inspect_err(|error| { + tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="select_memory_imports",reason=%error,"Selected Claude memory imports could not be prepared"); + })? { + if !resources.iter().any(|root| imported.starts_with(root)) { + resources.push(imported); + } + } prepare_selected( "claude", &manifest.cwd, diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 208bb480a..6a5751872 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -259,11 +259,29 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep .unwrap() .success()); let receipt = workspace.join("claude-consumed.json"); + let instructions_receipt = workspace.join("claude-import-consumed.json"); + let memory = format!( + "closed-claude-global-memory\n@../imported-memory.md\n@../selected-plugin/relative-memory.md\n@~/selected-plugin/home-memory.md\n@{}/absolute-memory.md\n@../selected-plugin/spaced\\ memory.md\n`@../selected-plugin/ignored-inline.md`\n```\n@../selected-plugin/ignored-fence.md\n```\n@\"../selected-plugin/ignored-quoted.md\"\n", + plugin.display() + ).into_bytes(); std::fs::write( - home.join(".claude/CLAUDE.md"), - b"closed-claude-global-memory", + home.join("imported-memory.md"), + b"closed-claude-imported-memory\n@./selected-plugin/nested-memory.md\n", ) .unwrap(); + std::fs::write(home.join(".claude/CLAUDE.md"), &memory).unwrap(); + for name in [ + "relative-memory.md", + "home-memory.md", + "absolute-memory.md", + "spaced memory.md", + "nested-memory.md", + "ignored-inline.md", + "ignored-fence.md", + "ignored-quoted.md", + ] { + std::fs::write(plugin.join(name), format!("closed-native-{name}")).unwrap(); + } let settings = r#"{"env":{"CLAUDE_SELECTED_NONCE":"closed-selected-settings"}}"#; std::fs::write(home.join(".claude/settings.json"), settings).unwrap(); std::fs::write(plugin.join("skills/probe/SKILL.md"),"---\nname: closed-plugin-probe\ndescription: selected complete native plugin\n---\nUse the adjacent resources.\n").unwrap(); @@ -295,6 +313,20 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ }); "#; std::fs::write(plugin.join("scripts/consume.cjs"), consumer).unwrap(); + let instructions_consumer = r#"const fs=require('node:fs'),crypto=require('node:crypto'),path=require('node:path'); +let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ + const input=JSON.parse(raw),name=path.basename(input.file_path||''),event=input.hook_event_name==='InstructionsLoaded'; + const names=['imported-memory.md','relative-memory.md','home-memory.md','absolute-memory.md','spaced memory.md','nested-memory.md']; + const selected=event&&names.includes(name)?{name,digest:crypto.createHash('sha256').update(fs.readFileSync(input.file_path)).digest('hex')}:null; + fs.appendFileSync(process.argv[2]+'.events',JSON.stringify({event,global:input.file_path===process.env.HOME+'/.claude/CLAUDE.md',ignored:event&&name.startsWith('ignored-'),selected})+'\n'); + if(event&&input.file_path===process.env.HOME+'/imported-memory.md')fs.writeFileSync(process.argv[2],JSON.stringify({nativeImport:true,digest:selected.digest})); +}); +"#; + std::fs::write( + plugin.join("scripts/instructions.cjs"), + instructions_consumer, + ) + .unwrap(); let manifest = json!({"name":"closed-selected-plugin","version":"1.0.0","skills":"./skills","hooks":"./hooks/hooks.json"}); std::fs::write( plugin.join(".claude-plugin/plugin.json"), @@ -303,6 +335,8 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ .unwrap(); let key_digest = format!("{:x}", Sha256::digest(b"synthetic-claude-onecli-value")); let hooks = json!({"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":format!("node \"${{CLAUDE_PLUGIN_ROOT}}/scripts/consume.cjs\" {} {}",serde_json::to_string(receipt.to_str().unwrap()).unwrap(),key_digest),"timeout":10}]}]}}); + let mut hooks = hooks; + hooks["hooks"]["InstructionsLoaded"] = json!([{"hooks":[{"type":"command","command":format!("node \"${{CLAUDE_PLUGIN_ROOT}}/scripts/instructions.cjs\" {}",serde_json::to_string(instructions_receipt.to_str().unwrap()).unwrap()),"timeout":10}]}]); std::fs::write( plugin.join("hooks/hooks.json"), serde_json::to_vec(&hooks).unwrap(), @@ -460,7 +494,63 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ .ok() } let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"memory":hash(b"closed-claude-global-memory"),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}); + let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}); + let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); + let mut expected_imports = serde_json::Map::new(); + expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); + for name in [ + "relative-memory.md", + "home-memory.md", + "absolute-memory.md", + "spaced memory.md", + "nested-memory.md", + ] { + expected_imports.insert( + name.into(), + json!(hash(std::fs::read(plugin.join(name)).unwrap().as_slice())), + ); + } + let expected_imports = Value::Object(expected_imports); + let instructions_events = instructions_receipt.with_extension("json.events"); + fn instruction_events(path: &Path) -> Vec { + std::fs::read_to_string(path) + .unwrap_or_default() + .lines() + .map(|line| serde_json::from_str(line).expect("complete native instruction event")) + .collect() + } + async fn imported_graph(path: &Path, expected: &Value) { + let result = tokio::time::timeout(Duration::from_secs(20), async { + loop { + let mut loaded = serde_json::Map::new(); + for event in instruction_events(path) { + assert_ne!( + event["ignored"], + json!(true), + "native code spans/fences/quoted tokens must remain unselected" + ); + if let Some(name) = event["selected"]["name"].as_str() { + loaded.insert(name.into(), event["selected"]["digest"].clone()); + } + } + if Value::Object(loaded.clone()) == *expected { + break; + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await; + assert!( + result.is_ok(), + "native must load the complete original-form Markdown import graph; observed {:?}", + instruction_events(path) + ); + } + let qualified_digest = hash( + serde_json::to_vec(&json!({"startup":expected,"nativeImports":expected_imports})) + .unwrap() + .as_slice(), + ); let observed = consumed(&receipt).await; if observed.is_none() { let output = client @@ -516,15 +606,32 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ observed.expect("native Claude selected plugin must execute its startup hook"), expected ); + let native_import = consumed(&instructions_receipt).await; + if native_import.is_none() { + println!( + "provider_memory_diagnostic {}", + json!({"route":"terminal","hookEvents":std::fs::read_to_string(instructions_receipt.with_extension("json.events")).unwrap_or_default().lines().filter_map(|line|serde_json::from_str::(line).ok()).collect::>() }) + ); + } + assert_eq!( + native_import.expect("native Claude must load selected memory import"), + json!({"nativeImport":true,"digest":hash(&imported_bytes)}) + ); + imported_graph(&instructions_events, &expected_imports).await; println!( "provider_launch_consumer {}", - json!({"provider":"claude","route":"terminal","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + json!({"provider":"claude","route":"terminal","digest":qualified_digest}) ); client .stop(SoulId::parse(terminal.soul_id).unwrap()) .await .unwrap(); + assert!(instruction_events(&instructions_events) + .iter() + .all(|event| event["ignored"] != json!(true))); std::fs::remove_file(&receipt).unwrap(); + std::fs::remove_file(&instructions_receipt).unwrap(); + std::fs::remove_file(&instructions_events).unwrap(); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( @@ -556,9 +663,16 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ .expect("genuine default SDK selected plugin must execute its startup hook"), expected ); + assert_eq!( + consumed(&instructions_receipt) + .await + .expect("native default SDK must load selected memory import"), + json!({"nativeImport":true,"digest":hash(&imported_bytes)}) + ); + imported_graph(&instructions_events, &expected_imports).await; println!( "provider_launch_consumer {}", - json!({"provider":"claude","route":"fresh","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + json!({"provider":"claude","route":"fresh","digest":qualified_digest}) ); assert_eq!( std::fs::read_to_string(home.join(".claude/settings.json")).unwrap(), @@ -573,6 +687,9 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ client.stop(view.soul_id).await.unwrap(); } } + assert!(instruction_events(&instructions_events) + .iter() + .all(|event| event["ignored"] != json!(true))); web_task.abort(); control_task.abort(); } diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 95669b836..9f4353d21 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -22,7 +22,7 @@ const broker=new RestrictedDockerBroker({ allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), allowedProviderUserRoots:new Set(configuration.provider==='claude' - ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-plugin')] + ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-plugin'),path.join(root,'source-home/imported-memory.md')] : configuration.provider==='codex' ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] : ['/home/freshell/provider',path.join(root,'.opencode')]), From d2e34f92f51a87391cd2db7e69c69a615bb7d771 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 04:28:45 -0700 Subject: [PATCH 080/493] fix: share ordinary provider source limits without private payload caps --- crates/freshell-platform/src/claude_inputs.rs | 4 +- crates/freshell-platform/src/lib.rs | 1 + .../src/provider_input_profiles.rs | 6 ++ .../managed_mcp_capability/claude_memory.rs | 64 ++++++++++++++----- .../src/managed_provider_references/claude.rs | 3 +- .../src/managed_provider_references/codex.rs | 21 +++--- 6 files changed, 71 insertions(+), 28 deletions(-) create mode 100644 crates/freshell-platform/src/provider_input_profiles.rs diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index 22bf0dcf4..4c18671e7 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -13,7 +13,9 @@ pub struct ClaudeInputManifest { } impl ClaudeInputManifest { pub fn validate(&self) -> Result<(), &'static str> { - if self.version != 1 || self.plugin_dirs.len() > 8192 { + if self.version != 1 + || self.plugin_dirs.len() > crate::provider_input_profiles::MAX_SOURCE_EVENTS + { return Err("accepted Claude discovery context profile exceeded"); } for path in [&self.original_home, &self.cwd, &self.workspace] diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 409d8c3b9..7b7bc489e 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -69,6 +69,7 @@ pub mod provider_environment; pub mod provider_environment_grants; pub mod provider_exec; pub mod provider_input_exchange; +pub mod provider_input_profiles; pub mod provider_mountpoints; pub mod resume_gate; pub mod spawn; diff --git a/crates/freshell-platform/src/provider_input_profiles.rs b/crates/freshell-platform/src/provider_input_profiles.rs new file mode 100644 index 000000000..d65070724 --- /dev/null +++ b/crates/freshell-platform/src/provider_input_profiles.rs @@ -0,0 +1,6 @@ +//! Shared ordinary source views and selection metadata. Private indexed +//! payloads retain the separate provider_input_exchange profile. +pub const MAX_SOURCE_VIEW_BYTES: u64 = 16 * 1024 * 1024; +pub const MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES: u64 = 64 * 1024 * 1024; +pub const MAX_SOURCE_EVENTS: usize = 8192; +pub const MAX_SOURCE_EVENT_BYTES: usize = 8 * 1024 * 1024; diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs index c7ebaf490..1b3ab515f 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs @@ -1,4 +1,8 @@ //! Select ordinary Markdown imports from their declaring logical origins. +use freshell_platform::provider_input_profiles::{ + MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES, MAX_SOURCE_EVENTS, MAX_SOURCE_EVENT_BYTES, + MAX_SOURCE_VIEW_BYTES, +}; use std::{ collections::{BTreeMap, BTreeSet}, io::Read, @@ -37,7 +41,7 @@ struct ImportGraph<'a> { excluded: &'a [PathBuf], active: BTreeSet, visited: BTreeSet, - sources: BTreeMap>, + sources: BTreeMap>, bytes: u64, events: usize, path_bytes: usize, @@ -67,41 +71,50 @@ impl ImportGraph<'_> { if self.visited.contains(&logical) { return Ok(()); } - if self.visited.len() >= 256 { - return Err(error("reference alias profile exceeded")); + if self.visited.len() >= MAX_SOURCE_EVENTS { + return Err(error("ordinary source event profile exceeded")); + } + let path_bytes = self + .path_bytes + .checked_add(logical.as_os_str().len()) + .and_then(|bytes| bytes.checked_add(source.as_os_str().len())) + .ok_or_else(|| error("ordinary source metadata accounting overflow"))?; + if path_bytes > MAX_SOURCE_EVENT_BYTES { + return Err(error("ordinary source event metadata profile exceeded")); } + self.path_bytes = path_bytes; let raw = if let Some(raw) = self.sources.get(&source) { Arc::clone(raw) } else { - use freshell_platform::provider_input_exchange::{MAX_FILE_BYTES, MAX_TOTAL_BYTES}; let file = std::fs::File::open(&source) .map_err(|cause| error(&format!("selected file unreadable: {cause}")))?; let metadata = file .metadata() .map_err(|cause| error(&format!("selected file metadata failed: {cause}")))?; - if !metadata.is_file() || metadata.len() > MAX_FILE_BYTES { - return Err(error("reference source type or per-file profile exceeded")); + if !metadata.is_file() || metadata.len() > MAX_SOURCE_VIEW_BYTES { + return Err(error("ordinary source type or per-view profile exceeded")); } let next = self .bytes .checked_add(metadata.len()) .ok_or_else(|| error("reference source accounting overflow"))?; - if self.sources.len() >= 128 || next > MAX_TOTAL_BYTES { - return Err(error("reference source aggregate profile exceeded")); + if next > MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES { + return Err(error("ordinary simultaneous source view profile exceeded")); } let mut raw = String::new(); - file.take(MAX_FILE_BYTES + 1) + file.take(MAX_SOURCE_VIEW_BYTES + 1) .read_to_string(&mut raw) .map_err(|cause| error(&format!("selected Markdown read failed: {cause}")))?; let next = self .bytes .checked_add(raw.len() as u64) .ok_or_else(|| error("reference source accounting overflow"))?; - if raw.len() as u64 > MAX_FILE_BYTES || next > MAX_TOTAL_BYTES { - return Err(error("reference source grew beyond its profile")); + if raw.len() as u64 > MAX_SOURCE_VIEW_BYTES || next > MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES + { + return Err(error("ordinary source grew beyond its view profile")); } self.bytes = next; - let raw: Arc = raw.into(); + let raw = Arc::new(raw); self.sources.insert(source.clone(), Arc::clone(&raw)); raw }; @@ -113,11 +126,7 @@ impl ImportGraph<'_> { .events .checked_add(1) .ok_or_else(|| error("source event accounting overflow"))?; - self.path_bytes = self - .path_bytes - .checked_add(selected.len()) - .ok_or_else(|| error("source path accounting overflow"))?; - if self.events > 8192 || self.path_bytes > 8 * 1024 * 1024 { + if self.events > MAX_SOURCE_EVENTS { return Err(error("source event metadata profile exceeded")); } let selected = if let Some(relative) = selected.strip_prefix("~/") { @@ -264,6 +273,27 @@ impl Iterator for ImportPaths<'_> { mod tests { use super::*; + #[test] + fn ordinary_memory_graph_does_not_inherit_private_indexed_payload_count_limit() { + let owned = tempfile::tempdir().unwrap(); + let root = owned.path().join("CLAUDE.md"); + let mut memory = String::new(); + for index in 0..129 { + let name = format!("memory-{index}.md"); + std::fs::write(owned.path().join(&name), format!("ordinary-memory-{index}")).unwrap(); + memory.push_str(&format!("@{name}\n")); + } + std::fs::write(&root, memory).unwrap(); + let selected = imports(owned.path(), &[root], &[]) + .expect("ordinary references retain the declared source event profile"); + assert_eq!(selected.len(), 129); + for path in selected { + assert!(std::fs::read_to_string(path) + .unwrap() + .starts_with("ordinary-memory-")); + } + } + #[test] fn selected_memory_imports_resolve_home_absolute_and_escaped_spaces() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index a23471acc..78c5abdfd 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -1,4 +1,5 @@ //! Finite original Claude source selection and ordinary first-volume authority. +use freshell_platform::provider_input_profiles::{MAX_SOURCE_EVENTS, MAX_SOURCE_EVENT_BYTES}; use freshell_runtime_protocol::*; use std::path::Path; @@ -37,7 +38,7 @@ pub(crate) async fn prepare_managed( path_bytes = path_bytes .checked_add(path.as_os_str().len()) .ok_or("Claude plugin source metadata overflow")?; - if plugin_dirs.len() >= 8192 || path_bytes > 8 * 1024 * 1024 { + if plugin_dirs.len() >= MAX_SOURCE_EVENTS || path_bytes > MAX_SOURCE_EVENT_BYTES { return Err("Claude plugin source event metadata profile exceeded".into()); } plugin_dirs.push(path); diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 1ac48d70e..d834921c2 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -1,4 +1,7 @@ //! Finite Codex typed inputs, resolved per declaring logical TOML layer. +use freshell_platform::provider_input_profiles::{ + MAX_SOURCE_EVENTS, MAX_SOURCE_EVENT_BYTES, MAX_SOURCE_VIEW_BYTES, +}; use std::path::{Path, PathBuf}; #[derive(Clone, Copy, PartialEq, Eq)] @@ -54,7 +57,7 @@ struct Selection<'a> { impl Selection<'_> { fn add(&mut self, logical_path: PathBuf, kind: InputKind) -> Result { - if self.inputs.len() >= 8192 { + if self.inputs.len() >= MAX_SOURCE_EVENTS { return Err("Codex selected input event budget exceeded"); } let canonical_source = std::fs::canonicalize(&logical_path) @@ -69,7 +72,7 @@ impl Selection<'_> { .checked_add(logical_path.as_os_str().len()) .and_then(|bytes| bytes.checked_add(canonical_source.as_os_str().len())) .ok_or("Codex selected input metadata size overflow")?; - if self.metadata_bytes > 8 * 1024 * 1024 { + if self.metadata_bytes > MAX_SOURCE_EVENT_BYTES { return Err("Codex selected input metadata budget exceeded"); } self.inputs.push(SelectedInput { @@ -104,10 +107,10 @@ impl Selection<'_> { let mut raw = String::new(); std::fs::File::open(&canonical) .map_err(|_| "Codex selected configuration unreadable")? - .take(16 * 1024 * 1024 + 1) + .take(MAX_SOURCE_VIEW_BYTES + 1) .read_to_string(&mut raw) .map_err(|_| "Codex selected configuration is unreadable or not UTF-8")?; - if raw.len() > 16 * 1024 * 1024 { + if raw.len() as u64 > MAX_SOURCE_VIEW_BYTES { return Err("Codex selected configuration source view exceeded"); } let value: toml::Value = @@ -127,7 +130,7 @@ impl Selection<'_> { let mut role_paths = Vec::new(); let mut role_path_bytes = 0usize; if let Some(roles) = value.get("agents").and_then(toml::Value::as_table) { - if roles.len() > 8192 - self.inputs.len() { + if roles.len() > MAX_SOURCE_EVENTS - self.inputs.len() { return Err("Codex selected role event budget exceeded"); } for role in roles.values() { @@ -136,7 +139,7 @@ impl Selection<'_> { role_path_bytes = role_path_bytes .checked_add(path.as_os_str().len()) .ok_or("Codex selected role metadata overflow")?; - if role_path_bytes > 8 * 1024 * 1024 - self.metadata_bytes { + if role_path_bytes > MAX_SOURCE_EVENT_BYTES - self.metadata_bytes { return Err("Codex selected role metadata budget exceeded"); } role_paths.push(path); @@ -159,11 +162,11 @@ impl Selection<'_> { } if kind != LayerKind::Role { let declared = role_paths.iter().cloned().collect(); - let remaining_events = 8192usize + let remaining_events = MAX_SOURCE_EVENTS .checked_sub(self.inputs.len()) .and_then(|value| value.checked_sub(role_paths.len())) .ok_or("Codex selected role event budget exceeded")?; - let remaining_metadata = (8 * 1024 * 1024usize) + let remaining_metadata = (MAX_SOURCE_EVENT_BYTES) .checked_sub(self.metadata_bytes) .and_then(|value| value.checked_sub(role_path_bytes)) .ok_or("Codex selected role metadata budget exceeded")?; @@ -576,7 +579,7 @@ fn read_config(path: &Path) -> Result { let mut bytes = Vec::new(); std::fs::File::open(path) .map_err(|_| "Codex selected configuration is unavailable")? - .take(16 * 1024 * 1024 + 1) + .take(MAX_SOURCE_VIEW_BYTES + 1) .read_to_end(&mut bytes) .map_err(|_| "Codex selected configuration is unreadable")?; if bytes.len() > 16 * 1024 * 1024 { From 96e4ff338b245aec6969b1e40446b1e84d9caa41 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 04:38:32 -0700 Subject: [PATCH 081/493] fix: bind ordinary provider inputs with launcher assigned identities --- crates/freshell-supervisor/src/registry.rs | 200 ++++++++++-------- .../src/registry/input_reservation.rs | 8 +- 2 files changed, 114 insertions(+), 94 deletions(-) diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index aa4a14b1d..40b84d158 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -2967,52 +2967,56 @@ mod tests { #[tokio::test] async fn provider_ordinary_initial_reservation_preserves_provider_identity() { for provider in ["codex", "claude", "amplifier"] { - let dir = tempfile::tempdir().unwrap(); - let binary = dir.path().join("synthetic-host"); - std::fs::write(&binary, "synthetic artifact").unwrap(); - let registry = Registry::open(dir.path().join("registry"), None).unwrap(); - let soul = SoulId::new(); - let request = freshell_runtime_protocol::ReserveProviderInputsRequest { - expected_control_epoch: Some(registry.control_epoch()), - soul_id: soul.clone(), - provider: provider.into(), - creation_seed_ref: "seed".into(), - }; - let image = format!("sha256:{}", "a".repeat(64)); - let reserved = registry - .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) - .await - .expect("ordinary selected inputs require their own provider's creation authority"); - let retry = registry - .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) - .await - .unwrap(); - assert_eq!(reserved.id, retry.id); - let mut foreign = request.clone(); - foreign.provider = "opencode".into(); - assert!(matches!( - registry - .reserve_initial_inputs(foreign, image, binary) - .await, - Err(RegistryError::RequestConflict) - )); - let receipt = registry - .record_initial_observation( - reserved, - freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { - daemon_id: DockerDaemonId::new(), - }, - ) - .await - .unwrap(); - assert_eq!(receipt.provider, provider); - let mut launch = prep(soul.clone(), RequestId::new(), "ordinary-reserved-inputs"); - launch.provider = provider.into(); - // Reservation binding protects actual workload preparation, regardless - // of which finite provider's selected ordinary resources it owns. - launch.terminal = Some(serde_json::from_value(serde_json::json!({ + for native_session_id in [None, Some("fixture-native-session".to_string())] { + let dir = tempfile::tempdir().unwrap(); + let binary = dir.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic artifact").unwrap(); + let registry = Registry::open(dir.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let request = freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: provider.into(), + creation_seed_ref: "seed".into(), + }; + let image = format!("sha256:{}", "a".repeat(64)); + let reserved = registry + .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) + .await + .expect( + "ordinary selected inputs require their own provider's creation authority", + ); + let retry = registry + .reserve_initial_inputs(request.clone(), image.clone(), binary.clone()) + .await + .unwrap(); + assert_eq!(reserved.id, retry.id); + let mut foreign = request.clone(); + foreign.provider = "opencode".into(); + assert!(matches!( + registry + .reserve_initial_inputs(foreign, image, binary) + .await, + Err(RegistryError::RequestConflict) + )); + let receipt = registry + .record_initial_observation( + reserved, + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new(), + }, + ) + .await + .unwrap(); + assert_eq!(receipt.provider, provider); + let mut launch = prep(soul.clone(), RequestId::new(), "ordinary-reserved-inputs"); + launch.provider = provider.into(); + launch.native_session_id = native_session_id.clone(); + // Reservation binding protects actual workload preparation, regardless + // of which finite provider's selected ordinary resources it owns. + launch.terminal = Some(serde_json::from_value(serde_json::json!({ "terminalId":"synthetic-terminal", "streamId":"synthetic-stream", "projectKey":"test-project", "mode":provider, - "program":provider,"args":[],"env":{}, "cwd":"/synthetic/workspace", + "program":provider,"args":[],"env":{},"resumeSessionId":native_session_id, "cwd":"/synthetic/workspace", "workspacePath":"/synthetic/workspace","cols":80,"rows":24,"runAsUid":65534,"runAsGid":0, "providerLaunchContext": { "preparation":match provider { @@ -3026,56 +3030,57 @@ mod tests { "manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":0,"aliasCount":0,"recipeCount":0}} } })).unwrap()); - let mut wrong = launch.clone(); - wrong.provider = "opencode".into(); - let workload = wrong.terminal.as_mut().unwrap(); - workload.mode = "opencode".into(); - workload - .provider_launch_context - .as_mut() - .unwrap() - .preparation = freshell_runtime_protocol::ProviderPreparation::Opencode { - project_config: vec![], - tui_config: None, - tui_source: None, - inline_config: false, - }; - let wrong_result = registry.prepare_launch(wrong.clone()).await; - assert!( - matches!(wrong_result, Err(RegistryError::RequestConflict)), - "foreign provider bind outcome: {wrong_result:?}" - ); - let prepared = registry.prepare_launch(launch.clone()).await.unwrap(); - assert_eq!( - registry - .prepare_launch(launch.clone()) - .await + let mut wrong = launch.clone(); + wrong.provider = "opencode".into(); + let workload = wrong.terminal.as_mut().unwrap(); + workload.mode = "opencode".into(); + workload + .provider_launch_context + .as_mut() .unwrap() - .incarnation_id, - prepared.incarnation_id - ); - assert!(matches!(registry.prepare_launch(wrong).await, Err(RegistryError::RequestConflict)), + .preparation = freshell_runtime_protocol::ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }; + let wrong_result = registry.prepare_launch(wrong.clone()).await; + assert!( + matches!(wrong_result, Err(RegistryError::RequestConflict)), + "foreign provider bind outcome: {wrong_result:?}" + ); + let prepared = registry.prepare_launch(launch.clone()).await.unwrap(); + assert_eq!( + registry + .prepare_launch(launch.clone()) + .await + .unwrap() + .incarnation_id, + prepared.incarnation_id + ); + assert!(matches!(registry.prepare_launch(wrong).await, Err(RegistryError::RequestConflict)), "a bound reservation retry cannot change provider even when its caller reuses the payload digest"); - let mut changed_seed = launch; - changed_seed.creation_seed_ref = "changed-creation-seed".into(); - assert!( - matches!( - registry.prepare_launch(changed_seed).await, - Err(RegistryError::RequestConflict) - ), - "a bound reservation retry cannot change its creation seed" - ); - let bound = registry - .bound_input_reservation(prepared.incarnation_id) - .await - .unwrap() - .unwrap(); - assert_eq!(bound.reservation.soul_id, soul); - assert!(registry - .initial_input_reservation(soul, receipt.reservation_id) - .await - .unwrap() - .is_none()); + let mut changed_seed = launch; + changed_seed.creation_seed_ref = "changed-creation-seed".into(); + assert!( + matches!( + registry.prepare_launch(changed_seed).await, + Err(RegistryError::RequestConflict) + ), + "a bound reservation retry cannot change its creation seed" + ); + let bound = registry + .bound_input_reservation(prepared.incarnation_id) + .await + .unwrap() + .unwrap(); + assert_eq!(bound.reservation.soul_id, soul); + assert!(registry + .initial_input_reservation(soul, receipt.reservation_id) + .await + .unwrap() + .is_none()); + } } } @@ -3140,6 +3145,15 @@ mod tests { "sourceCount":0,"aliasCount":0,"recipeCount":0}} } })).expect("the complete prepared context must carry its initial auth reservation")); + let mut existing_native = launch.clone(); + existing_native.native_session_id = Some("preexisting-opencode-session".into()); + assert!( + matches!( + registry.prepare_launch(existing_native).await, + Err(RegistryError::RequestConflict) + ), + "OpenCode initial native-auth authority must remain identity-free" + ); let first = registry.prepare_launch(launch.clone()).await.unwrap(); let replay = registry.prepare_launch(launch.clone()).await.unwrap(); assert_eq!(first.incarnation_id, replay.incarnation_id); diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index bf14b4abf..a1cf022f8 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -220,7 +220,13 @@ pub(super) fn bind_in_transaction( let Some(id) = reservation_reference(input) else { return Ok(()); }; - if input.native_session_id.is_some() { + // Ordinary resources bind to the same provider/soul/creation seed even + // when the launcher has already assigned an exact native identity. + // This does not grant native-auth lookup; OpenCode's initial auth authority + // still requires an identity-free creation, and host resume checks remain. + if input.native_session_id.is_some() + && !matches!(input.provider.as_str(), "codex" | "claude" | "amplifier") + { return Err(RegistryError::RequestConflict); } let changed=tx.execute( From 245740c3784f11e5a1d8b9b357dc9c12e696cf02 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 06:58:27 -0700 Subject: [PATCH 082/493] fix: prepare Amplifier inputs and native writable package state --- .../freshell-platform/src/amplifier_inputs.rs | 31 ++ crates/freshell-platform/src/lib.rs | 1 + .../src/managed_mcp_capability.rs | 434 +++++++++++++++--- .../amplifier_resources.rs | 240 ++++++++++ .../managed_mcp_capability/amplifier_stage.rs | 76 +++ .../src/managed_provider_launch_tests.rs | 431 +++++++++++++++++ .../src/managed_provider_references.rs | 1 + .../managed_provider_references/amplifier.rs | 62 +++ crates/freshell-server/src/managed_runtime.rs | 38 +- crates/freshell-session-host/src/main.rs | 30 +- .../src/provider_input_preparation.rs | 76 ++- .../src/providers/mod.rs | 106 +++++ .../testing/managed-provider-launch-broker.ts | 6 +- .../managed-provider-launch-sandbox.py | 11 +- 14 files changed, 1466 insertions(+), 77 deletions(-) create mode 100644 crates/freshell-platform/src/amplifier_inputs.rs create mode 100644 crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs create mode 100644 crates/freshell-server/src/managed_provider_references/amplifier.rs diff --git a/crates/freshell-platform/src/amplifier_inputs.rs b/crates/freshell-platform/src/amplifier_inputs.rs new file mode 100644 index 000000000..9072317e3 --- /dev/null +++ b/crates/freshell-platform/src/amplifier_inputs.rs @@ -0,0 +1,31 @@ +//! Accepted original Amplifier origins; payloads remain in private stage files. +use serde::{Deserialize, Serialize}; +use std::path::{Path, PathBuf}; + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct AmplifierInputManifest { + pub version: u8, + pub original_home: PathBuf, + pub cwd: PathBuf, + pub workspace: PathBuf, +} +impl AmplifierInputManifest { + pub fn validate(&self) -> Result<(), &'static str> { + if self.version != 1 { + return Err("accepted Amplifier source manifest version unsupported"); + } + for path in [&self.original_home, &self.cwd, &self.workspace] { + if !path.is_absolute() + || path.as_os_str().len() > 64 * 1024 + || path.to_string_lossy().chars().any(char::is_control) + { + return Err("accepted Amplifier discovery source path is invalid"); + } + } + Ok(()) + } + pub fn runtime_path(&self, logical: &Path) -> PathBuf { + crate::codex_inputs::runtime_path(&self.original_home, logical) + } +} diff --git a/crates/freshell-platform/src/lib.rs b/crates/freshell-platform/src/lib.rs index 7b7bc489e..498b0c704 100644 --- a/crates/freshell-platform/src/lib.rs +++ b/crates/freshell-platform/src/lib.rs @@ -51,6 +51,7 @@ //! injected source ([`Env`], [`FileProbe`], [`path::WslPathResolver`]). Thin `*_live` //! wrappers at the edges perform the real reads and delegate to the pure core. +pub mod amplifier_inputs; pub mod claude_inputs; pub mod cli_launch; pub mod clock; diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e1fc061e4..482142d73 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -28,6 +28,7 @@ const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; mod amplifier_resources; +mod amplifier_stage; mod claude_memory; mod claude_stage; mod ordinary_stage; @@ -37,6 +38,7 @@ mod source_stage; pub(crate) struct OpencodeEphemeralInput<'a> { pub codex_manifest: Option<&'a freshell_platform::codex_inputs::CodexInputManifest>, pub claude_manifest: Option<&'a freshell_platform::claude_inputs::ClaudeInputManifest>, + pub amplifier_manifest: Option<&'a freshell_platform::amplifier_inputs::AmplifierInputManifest>, pub inline_config: Option<&'a str>, pub tui_config_path: Option<&'a str>, pub tui_source: Option<&'a ProviderConfigReference>, @@ -301,7 +303,7 @@ async fn apply_prepared_callback( ) .map_err(|_| "replacement provider context exceeds control metadata profile".into()); } - if matches!(provider, "codex" | "claude") { + if matches!(provider, "codex" | "claude" | "amplifier") { let mut context = callback .prior_context .clone() @@ -310,65 +312,83 @@ async fn apply_prepared_callback( tracing::warn!(event="provider.inputs.preparation.blocked",stage="controller_callback",operation="select_original_sources",provider, soul_id=%callback.soul_id,reason="origin_unknown", "Original source origins are unrecorded; retained inputs were preserved"); - let name = if provider == "codex" { - "Codex" - } else { - "Claude" + let name = match provider { + "codex" => "Codex", + "claude" => "Claude", + _ => "Amplifier", }; return Err(format!("original {name} source origins are unrecorded; retain the existing inputs and reprepare from their original source before dispatch")); } let bytes = read_prior_manifest_bytes(provider, store, directory, &callback.soul_id, &context)?; - let codex_manifest: Option = - if provider == "codex" { + enum AcceptedOrigins { + Codex(freshell_platform::codex_inputs::CodexInputManifest), + Claude(freshell_platform::claude_inputs::ClaudeInputManifest), + Amplifier(freshell_platform::amplifier_inputs::AmplifierInputManifest), + } + let origins = match provider { + "codex" => { let manifest: freshell_platform::codex_inputs::CodexInputManifest = serde_json::from_slice(&bytes) .map_err(|_| "prior Codex source manifest metadata invalid")?; manifest.validate().map_err(str::to_string)?; - Some(manifest) - } else { - None - }; - let claude_manifest: Option = - if provider == "claude" { + AcceptedOrigins::Codex(manifest) + } + "claude" => { let manifest: freshell_platform::claude_inputs::ClaudeInputManifest = serde_json::from_slice(&bytes) .map_err(|_| "prior Claude source manifest metadata invalid")?; manifest.validate().map_err(str::to_string)?; - Some(manifest) - } else { - None - }; - let (original_home, original_cwd, original_workspace) = - match (&codex_manifest, &claude_manifest) { - (Some(manifest), None) => ( - &manifest.original_home, - &manifest.cwd, - callback.workspace.as_deref(), - ), - (None, Some(manifest)) => ( - &manifest.original_home, - &manifest.cwd, - Some(manifest.workspace.as_path()), - ), - _ => return Err("replacement ordinary source manifest selection invalid".into()), - }; + AcceptedOrigins::Claude(manifest) + } + "amplifier" => { + let manifest: freshell_platform::amplifier_inputs::AmplifierInputManifest = + serde_json::from_slice(&bytes) + .map_err(|_| "prior Amplifier source manifest metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + AcceptedOrigins::Amplifier(manifest) + } + _ => return Err("replacement ordinary source provider invalid".into()), + }; + let (original_home, mut input) = match &origins { + AcceptedOrigins::Codex(manifest) => ( + &manifest.original_home, + OpencodeEphemeralInput { + cwd: Some(&manifest.cwd), + workspace: callback.workspace.as_deref(), + codex_manifest: Some(manifest), + ..Default::default() + }, + ), + AcceptedOrigins::Claude(manifest) => ( + &manifest.original_home, + OpencodeEphemeralInput { + cwd: Some(&manifest.cwd), + workspace: Some(&manifest.workspace), + claude_manifest: Some(manifest), + ..Default::default() + }, + ), + AcceptedOrigins::Amplifier(manifest) => ( + &manifest.original_home, + OpencodeEphemeralInput { + cwd: Some(&manifest.cwd), + workspace: Some(&manifest.workspace), + amplifier_manifest: Some(manifest), + ..Default::default() + }, + ), + }; context.config.clear(); let references = callback.provider_secret_references.unwrap_or_default(); + input.launch_context = Some(&context); + input.secret_references = Some(&references); let capability = store.issue_replacement_with_inputs( callback.soul_id, directory, endpoint.into(), Some((provider, original_home, &context.config)), - OpencodeEphemeralInput { - launch_context: Some(&context), - secret_references: Some(&references), - cwd: Some(original_cwd), - workspace: original_workspace, - codex_manifest: codex_manifest.as_ref(), - claude_manifest: claude_manifest.as_ref(), - ..Default::default() - }, + input, )?; let reply = freshell_runtime_protocol::PreparedProviderContext { provider_launch_context: read_prepared_context(directory, &capability)?, @@ -1292,29 +1312,40 @@ fn stage_provider_root( .map_err(|_| "prepared resource descriptor exceeds control profile")?; write_private_bytes(&temporary.join(".freshell/resource-mounts.json"), &metadata)?; } - let ordinary_resources = - if opencode_input.launch_context.is_some() && matches!(provider, "codex" | "claude") { - Some(if provider == "claude" { - claude_stage::prepare( - home, - opencode_input.cwd.unwrap_or(home), - opencode_input - .workspace - .or(opencode_input.cwd) - .unwrap_or(home), - opencode_input.claude_manifest, - &excluded, - )? - } else { - ordinary_stage::prepare( - home, - opencode_input.cwd.unwrap_or(home), - opencode_input.codex_manifest, - )? - }) + let ordinary_resources = if opencode_input.launch_context.is_some() + && matches!(provider, "codex" | "claude" | "amplifier") + { + Some(if provider == "amplifier" { + amplifier_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input.amplifier_manifest, + )? + } else if provider == "claude" { + claude_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input.claude_manifest, + &excluded, + )? } else { - None - }; + ordinary_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input.codex_manifest, + )? + }) + } else { + None + }; if let Some(resources) = &ordinary_resources { selections.extend(resources.selections.iter().cloned()); } @@ -1328,7 +1359,22 @@ fn stage_provider_root( None }; if let Some(resources) = &lifier_resources { - selections.extend(resources.selections.iter().cloned()); + for selection in &resources.selections { + let mut selection = selection.clone(); + if ordinary_resources.is_some() { + selection.destination_relative = + amplifier_stage::projected_path(&selection.destination_relative)?; + } + selections.push(selection); + } + if ordinary_resources.is_some() { + let projected = temporary.join(amplifier_stage::PROJECTED); + private_directory(&projected)?; + selections.push(CopySelection { + source: projected, + destination_relative: amplifier_stage::PROJECTED.into(), + }); + } } for reference in references { if ordinary_resources.is_some() { @@ -1449,12 +1495,27 @@ fn stage_provider_root( ); if let Some(resources) = amplifier_resources { if let Some(settings) = resources.rewritten_settings { - replace_staged_private_bytes( - &temporary.join("settings.yaml"), - settings.as_bytes(), - )?; + let relative = if let Some(prepared) = &ordinary_resources { + prepared + .binds + .iter() + .find(|bind| { + bind.logical_destination + == "/home/freshell/provider/.amplifier/settings.yaml" + }) + .map(|bind| bind.source_relative_path.as_str()) + .ok_or("Amplifier projected settings source unavailable")? + } else { + "settings.yaml" + }; + replace_staged_private_bytes(&temporary.join(relative), settings.as_bytes())?; } for (relative, template) in resources.rewritten_resources { + let relative = if ordinary_resources.is_some() { + amplifier_stage::projected_path(&relative)? + } else { + relative + }; replace_staged_private_bytes(&temporary.join(relative), template.as_bytes())?; } } @@ -1837,7 +1898,7 @@ impl CapabilityStore { provider_relative_path: CAPABILITY_PATH.into(), host_gateway_address: routed_host_ipv4(), }); - if matches!(provider, "codex" | "claude") { + if matches!(provider, "codex" | "claude" | "amplifier") { let bytes = source_stage::read_source_view(root, ".freshell/prepared-inputs.json")?; context.prepared_inputs = Some( @@ -2964,6 +3025,117 @@ except Exception as error: ); } } + #[cfg(unix)] + #[test] + fn managed_provider_amplifier_complete_context_records_origins_and_resource_projection() { + use freshell_runtime_protocol::{ + ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let bundle = owned.path().join("selected-bundle"); + let directory = owned.path().join("control/mcp-capabilities"); + for path in [ + &home.join(".amplifier"), + &workspace, + &bundle.join("node_modules/native-dependency"), + &directory, + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + home.join(".amplifier/settings.yaml"), + format!("bundle:\n active: file://{}\n", bundle.display()), + ) + .unwrap(); + std::fs::write( + bundle.join("bundle.yaml"), + "bundle:\n name: selected-native-bundle\n", + ) + .unwrap(); + std::fs::write(bundle.join("resource"), "selected-resource").unwrap(); + std::fs::write( + bundle.join("node_modules/native-dependency/index.js"), + "module.exports='installed-native-dependency'", + ) + .unwrap(); + std::fs::write(bundle.join("consume.mjs"),"import {readFileSync} from 'node:fs'; import dep from 'native-dependency'; process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+':'+dep);").unwrap(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Amplifier { + bundle: "default".into(), + resume_args: vec![], + }, + config: vec![ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "settings.yaml".into(), + provider_relative_path: ".amplifier/settings.yaml".into(), + format: "yaml".into(), + }], + mcp_capability: None, + prepared_inputs: None, + }; + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let capability = store + .issue( + soul, + &directory, + "http://host.docker.internal:4000".into(), + Some(("amplifier", &home, &context.config)), + OpencodeEphemeralInput { + launch_context: Some(&context), + cwd: Some(&workspace), + workspace: Some(&workspace), + secret_references: Some(&[]), + ..Default::default() + }, + None, + ) + .unwrap(); + let ready = read_prepared_context(&directory, &capability).unwrap(); + let inputs = ready + .prepared_inputs + .as_ref() + .expect("Amplifier initial issuance must publish a complete ordinary descriptor"); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let manifest: serde_json::Value = serde_json::from_slice( + &std::fs::read(stage.join(&inputs.manifest.relative_path)).unwrap(), + ) + .unwrap(); + assert_eq!(manifest["originalHome"], home.to_str().unwrap()); + assert_eq!(manifest["cwd"], workspace.to_str().unwrap()); + assert!( + ready.config.is_empty(), + "one accepted descriptor replaces the shallow independent selection" + ); + let resources = inputs + .resource_mounts + .iter() + .find(|mount| { + mount.logical_destination == "/home/freshell/provider/.freshell/resources" + }) + .expect("projected native bundle must be a complete owned resource root"); + let probe = std::fs::read_dir(stage.join(&resources.stage_relative_path)) + .unwrap() + .map(|entry| entry.unwrap().path()) + .find(|path| path.join("consume.mjs").exists()) + .expect("selected complete bundle projection"); + let output = std::process::Command::new("node") + .arg(probe.join("consume.mjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + b"selected-resource:installed-native-dependency" + ); + assert_eq!( + std::fs::read_to_string(home.join(".amplifier/settings.yaml")).unwrap(), + format!("bundle:\n active: file://{}\n", bundle.display()) + ); + } + #[cfg(unix)] #[test] fn managed_provider_claude_complete_stage_carries_project_resources_and_dependencies() { @@ -4417,6 +4589,132 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[tokio::test] + async fn managed_provider_amplifier_replacement_retains_original_sources_and_pending_replay() { + use freshell_runtime_protocol::{ + PreparedProviderContext, ProviderLaunchContext, ProviderPreparation, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("original-home"); + let wrong = owned.path().join("current-home"); + let workspace = owned.path().join("workspace"); + for path in [ + &home.join(".amplifier/skills/probe"), + &wrong.join(".amplifier"), + &workspace, + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + home.join(".amplifier/settings.yaml"), + r#"{"env":{"CLAUDE_SELECTED":"original"}}"#, + ) + .unwrap(); + std::fs::write( + home.join(".amplifier/skills/probe/resource"), + b"retained-original-resource", + ) + .unwrap(); + std::fs::write( + wrong.join(".amplifier/settings.yaml"), + r#"{"env":{"CLAUDE_SELECTED":"wrong"}}"#, + ) + .unwrap(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Amplifier { + bundle: "default".into(), + resume_args: vec![], + }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("amplifier", &home, &context.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let context = read_prepared_context(&directory, &prior).unwrap(); + assert!(context.prepared_inputs.is_some()); + let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"amplifier","workspace":workspace,"cwd":workspace,"priorContext":context,"providerSecretReferences":[]})).unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let first = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &wrong, + &client, + ) + .await + .expect("replacement must select recorded original Amplifier sources"); + std::fs::remove_file(home.join(".amplifier/settings.yaml")).unwrap(); + let repeated = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &wrong, + &client, + ) + .await + .unwrap(); + assert_eq!( + serde_json::from_slice::(&first).unwrap(), + serde_json::from_slice::(&repeated).unwrap(), + "pending generation replay must precede original rescanning" + ); + let ready: PreparedProviderContext = serde_json::from_slice(&repeated).unwrap(); + let inputs = ready + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap(); + let mount = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == "/home/freshell/provider/.amplifier/skills") + .unwrap(); + let stage = staged_provider_root_path( + &directory, + &ready + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap() + .grant_id, + ) + .unwrap(); + assert_eq!( + std::fs::read( + stage + .join(&mount.stage_relative_path) + .join("probe/resource") + ) + .unwrap(), + b"retained-original-resource" + ); + } + #[tokio::test] async fn managed_provider_replacement_complete_codex_metadata_replays_before_rescanning_roots() { diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 55f0393bd..eb7a061d8 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -83,6 +83,7 @@ struct BundleSelection { view_bytes: usize, metadata_bytes: usize, completed: BTreeMap, + modules: BTreeMap, } fn select_local_bundle( source: &str, @@ -245,6 +246,44 @@ fn select_local_bundle( return Err("Amplifier selected bundle requires one YAML document".into()); } let mut bundle_replacements = Vec::new(); + let module_origin = ModuleSourceOrigin { + yaml, + header_offset, + declaring_root: root, + bundle_destination: &relative, + complete_bundle_directory: metadata.is_dir(), + excluded, + }; + if let Some(session) = native_mapping_value(&bundle_documents[0], "session", 0)? { + for kind in ["orchestrator", "context"] { + if let Some(module) = native_mapping_value(session, kind, 0)? { + project_module_source( + module, + &module_origin, + result, + state, + &mut bundle_replacements, + )?; + } + } + } + for kind in ["providers", "tools", "hooks"] { + if let Some(modules) = native_mapping_value(&bundle_documents[0], kind, 0)? { + for module in modules + .data + .as_sequence() + .ok_or("Amplifier selected module list must be a sequence")? + { + project_module_source( + module, + &module_origin, + result, + state, + &mut bundle_replacements, + )?; + } + } + } if let Some(includes) = bundle_documents[0].data.as_mapping_get("includes") { for item in includes .data @@ -390,6 +429,124 @@ fn select_local_bundle( Ok(Some(runtime_uri)) } +#[derive(Clone, Copy)] +struct ModuleSourceOrigin<'a> { + yaml: &'a str, + header_offset: usize, + declaring_root: &'a Path, + bundle_destination: &'a Path, + complete_bundle_directory: bool, + excluded: &'a [PathBuf], +} + +fn project_module_source( + module: &MarkedYamlOwned, + origin: &ModuleSourceOrigin<'_>, + result: &mut ResourcePreparation, + state: &mut BundleSelection, + replacements: &mut Vec<(usize, usize, String)>, +) -> Result<(), String> { + let ModuleSourceOrigin { + yaml, + header_offset, + declaring_root, + bundle_destination, + complete_bundle_directory, + excluded, + } = *origin; + let Some(node) = native_mapping_value(module, "source", 0)? else { + return Ok(()); + }; + let Some(source) = node.data.as_str() else { + return Ok(()); + }; + let local = if let Some(path) = source.strip_prefix("file://") { + path + } else if source.starts_with("./") + || source.starts_with("../") + || Path::new(source).is_absolute() + { + source + } else { + return Ok(()); + }; + if local.is_empty() || local.len() > 64 * 1024 || local.chars().any(char::is_control) { + return Err("Amplifier selected module source path is invalid".into()); + } + let selected = if Path::new(local).is_absolute() { + PathBuf::from(local) + } else { + declaring_root.join(local) + }; + let canonical = std::fs::canonicalize(&selected).map_err(|error| { + format!( + "Amplifier selected module source {} is unavailable: {error}", + selected.display() + ) + })?; + if excluded.iter().any(|path| canonical.starts_with(path)) { + return Err("Amplifier selected module aliases credentials or native state; use the approved private grant source".into()); + } + if !std::fs::metadata(&canonical) + .map_err(|error| format!("Amplifier selected module source metadata failed: {error}"))? + .is_dir() + { + return Err( + "Amplifier selected local module source must be a complete package directory".into(), + ); + } + let runtime = if let Some(runtime) = state.modules.get(&canonical) { + runtime.clone() + } else { + if state.next_id >= 8192 { + return Err("Amplifier selected module event budget exceeded".into()); + } + state.metadata_bytes = state + .metadata_bytes + .checked_add(canonical.as_os_str().len()) + .and_then(|size| size.checked_add(selected.as_os_str().len())) + .ok_or("Amplifier selected module metadata size overflow")?; + if state.metadata_bytes > 8 * 1024 * 1024 { + return Err("Amplifier selected module metadata budget exceeded".into()); + } + let destination = if complete_bundle_directory && canonical.starts_with(declaring_root) { + bundle_destination.join(canonical.strip_prefix(declaring_root).unwrap()) + } else { + let destination = PathBuf::from(format!( + ".freshell/resources/amplifier-module-{}", + state.next_id + )); + result.selections.push(CopySelection { + source: canonical.clone(), + destination_relative: destination.clone(), + }); + destination + }; + state.next_id += 1; + let runtime = format!( + "file://{}", + Path::new("/home/freshell/provider") + .join(destination) + .to_str() + .ok_or("Amplifier projected module path is not UTF-8")? + ); + state.modules.insert(canonical, runtime.clone()); + runtime + }; + let start = header_offset + .checked_add(source_offset(yaml, node.span.start)?) + .ok_or("Amplifier module source offset overflow")?; + let end = header_offset + .checked_add(source_offset(yaml, node.span.end)?) + .ok_or("Amplifier module source offset overflow")?; + replacements.push(( + start, + end, + serde_json::to_string(&runtime).map_err(|_| "Amplifier module source cannot be encoded")?, + )); + Ok(()) +} + fn markdown_frontmatter(raw: &str) -> Option<(&str, usize)> { static HEADER: std::sync::OnceLock = std::sync::OnceLock::new(); let expression = HEADER.get_or_init(|| { @@ -400,6 +557,89 @@ fn markdown_frontmatter(raw: &str) -> Option<(&str, usize)> { Some((header.as_str(), header.start())) } +#[cfg(test)] +mod module_source_tests { + use super::*; + + #[test] + fn amplifier_local_module_source_executes_after_projection_from_unrelated_cwd() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let bundle = owned.path().join("bundle"); + let module = bundle.join("modules/loop-probe"); + let cwd = owned.path().join("unrelated-workspace"); + let stage = owned.path().join("stage"); + for path in [ + &home.join(".amplifier"), + &module.join("node_modules/dep"), + &cwd, + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + home.join(".amplifier/settings.yaml"), + format!("bundle:\n active: file://{}\n", bundle.display()), + ) + .unwrap(); + std::fs::write(bundle.join("bundle.yaml"), "bundle:\n name: selected\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n").unwrap(); + std::fs::write(module.join("resource"), "module-resource").unwrap(); + std::fs::write( + module.join("node_modules/dep/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(module.join("consume.cjs"), "const fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('dep'));").unwrap(); + let prepared = prepare(&home, &cwd, &[]).unwrap(); + freshell_platform::provider_environment::copy_environment( + &prepared.selections, + &stage, + &[], + Default::default(), + ) + .unwrap(); + for (path, bytes) in prepared.rewritten_resources { + std::fs::write(stage.join(path), bytes).unwrap(); + } + let document = std::fs::read_to_string( + stage.join(".freshell/resources/amplifier-local-0/bundle.yaml"), + ) + .unwrap(); + let parsed = MarkedYamlOwned::load_from_str(&document).unwrap(); + let source = parsed[0] + .data + .as_mapping_get("session") + .unwrap() + .data + .as_mapping_get("orchestrator") + .unwrap() + .data + .as_mapping_get("source") + .unwrap() + .data + .as_str() + .unwrap(); + let selected = if let Some(path) = source.strip_prefix("file://") { + stage.join( + Path::new(path) + .strip_prefix("/home/freshell/provider") + .unwrap(), + ) + } else { + cwd.join(source) + }; + let output = std::process::Command::new("node") + .arg(selected.join("consume.cjs")) + .current_dir(&cwd) + .output() + .unwrap(); + assert!( + output.status.success(), + "selected local module must execute independently of launch cwd" + ); + assert_eq!(output.stdout, b"module-resource:installed-dependency"); + } +} + fn native_mapping_value<'a>( node: &'a MarkedYamlOwned, key: &str, diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs new file mode 100644 index 000000000..53fb821c2 --- /dev/null +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs @@ -0,0 +1,76 @@ +//! Existing Amplifier projections share the ordinary manifest/copier boundary. +use super::ordinary_stage::{prepare_selected, Preparation}; +use freshell_platform::amplifier_inputs::AmplifierInputManifest; +use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; +use std::path::Path; + +pub(super) const PROJECTED: &str = ".freshell/resource-units/amplifier-projected"; + +pub(super) fn prepare( + home: &Path, + cwd: &Path, + workspace: &Path, + accepted: Option<&AmplifierInputManifest>, +) -> Result { + let manifest = accepted.cloned().unwrap_or_else(|| AmplifierInputManifest { + version: 1, + original_home: home.into(), + cwd: cwd.into(), + workspace: workspace.into(), + }); + manifest.validate().map_err(str::to_string)?; + let references = super::provider_source_references("amplifier", &manifest.original_home)?; + let mut configurations = Vec::new(); + let mut resources = Vec::new(); + for reference in references { + if reference.relative_path == ".freshell/resources" { + continue; + } + let logical = manifest + .original_home + .join(".amplifier") + .join(&reference.relative_path); + let canonical = std::fs::canonicalize(&logical).map_err(|error| { + format!( + "Amplifier selected source {} unavailable: {error}", + logical.display() + ) + })?; + if std::fs::metadata(&canonical) + .map_err(|error| { + format!( + "Amplifier selected source {} metadata failed: {error}", + logical.display() + ) + })? + .is_dir() + { + resources.push(logical); + } else { + configurations.push((logical, canonical)); + } + } + let mut prepared = prepare_selected( + "amplifier", + &manifest.cwd, + &manifest, + configurations, + resources, + |logical| manifest.runtime_path(logical), + )?; + prepared.mounts.push(ProviderResourceMount { + stage_relative_path: PROJECTED.into(), + provider_relative_path: ".freshell/provider-roots/amplifier-projected".into(), + logical_destination: "/home/freshell/provider/.freshell/resources".into(), + effective_destination: "/home/freshell/provider/.freshell/resources".into(), + kind: ProviderResourceKind::Directory, + }); + Ok(prepared) +} + +pub(super) fn projected_path(path: &Path) -> Result { + Ok(Path::new(PROJECTED).join( + path.strip_prefix(".freshell/resources") + .map_err(|_| "Amplifier projected source path escaped its finite root")?, + )) +} diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 6a5751872..6f2416d3f 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -215,6 +215,437 @@ impl RuntimeBackend for BrokerAuthorizedBackend { } } +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn managed_provider_amplifier_terminal_native_startup_consumes_prepared_inputs() { + use freshell_supervisor::{ + admission::AdmissionPolicy, + registry::Registry, + service::{serve_control, Supervisor, SupervisorConfig}, + }; + use freshell_terminal::registry::{ManagedTerminalController, ManagedTerminalLaunch}; + use sha2::{Digest, Sha256}; + use std::{os::unix::fs::PermissionsExt, sync::Arc}; + let fixture = PathBuf::from(std::env::var_os("FRESHELL_PROVIDER_LAUNCH_FIXTURE").unwrap()); + assert_eq!(fixture.parent(), Some(Path::new("/tmp"))); + assert!(fixture + .file_name() + .unwrap() + .to_str() + .unwrap() + .starts_with("fln-")); + let home = fixture.join("source-home"); + let workspace = fixture.join("workspace"); + let bundle = home.join(".amplifier/bundles/selected"); + let skill = home.join(".amplifier/skills/probe"); + let control = fixture.join("scenarios/native/control"); + let runtime = fixture.join("scenarios/n/r"); + let receipt = workspace.join("amplifier-consumed.json"); + for path in [ + &control, + &runtime, + &bundle.join("modules/context-probe/amplifier_module_context_probe"), + &bundle.join("modules/loop-probe/amplifier_module_loop_probe"), + &bundle.join("modules/hooks-probe/amplifier_module_hooks_probe"), + &skill.join("node_modules/probe-dependency"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + assert!(std::process::Command::new("git") + .args(["init", "--quiet"]) + .arg(&workspace) + .status() + .unwrap() + .success()); + std::fs::write(skill.join("resource"), b"closed-amplifier-resource").unwrap(); + std::fs::write( + skill.join("node_modules/probe-dependency/index.js"), + "module.exports='closed-amplifier-dependency'", + ) + .unwrap(); + std::fs::write(skill.join("run"),"#!/usr/bin/env node\nconst fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('probe-dependency'));").unwrap(); + std::fs::set_permissions(skill.join("run"), std::fs::Permissions::from_mode(0o755)).unwrap(); + let context_module = r#"__amplifier_module_type__='context' +class Context: + def __init__(self): self.messages=[]; self.factory=None + async def add_message(self,message): self.messages.append(message) + async def get_messages(self): return list(self.messages) + async def get_messages_for_request(self,**kwargs): return list(self.messages) + async def set_messages(self,messages): self.messages=list(messages) + async def clear(self): self.messages=[] + async def set_system_prompt_factory(self,factory): self.factory=factory + def get_context_info(self): return {'message_count':len(self.messages),'token_count':0} +async def mount(coordinator,config=None): await coordinator.mount('context',Context()) +"#; + let loop_module = r#"__amplifier_module_type__='orchestrator' +class Loop: + async def execute(self,prompt,context,providers,tools,hooks,**kwargs): raise RuntimeError('model turns are outside this closed startup fixture') +async def mount(coordinator,config=None): await coordinator.mount('orchestrator',Loop()) +"#; + let hook_module = r#"__amplifier_module_type__='hook' +import hashlib,json,os,subprocess,urllib.request +from pathlib import Path +from amplifier_core import HookResult +_consumer=None +async def mount(coordinator,config=None): + global _consumer + config=config or {} + async def consume(event,data): + uv=subprocess.check_output(['uv','--version'],text=True).strip() + output=subprocess.check_output([str(Path.home()/'.amplifier/skills/probe/run')]) + request=urllib.request.Request(os.environ['FRESHELL_URL']+'/api/mcp',data=b'{}',headers={'x-auth-token':os.environ['FRESHELL_TOKEN']}) + with urllib.request.urlopen(request,timeout=5) as response: authenticated=response.status==200 + result={'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest()} + Path(config['receipt']).write_text(json.dumps(result)) + return HookResult(action='continue') + _consumer=consume + for event in ['session:start','session:resume']: coordinator.hooks.register(event,consume,name='closed-native-probe') +async def on_session_ready(coordinator): + await _consumer('on_session_ready',{'session_id':coordinator.session_id}) +"#; + for (name, module) in [ + ("context-probe", context_module), + ("loop-probe", loop_module), + ("hooks-probe", hook_module), + ] { + std::fs::write( + bundle.join(format!( + "modules/{name}/amplifier_module_{}/__init__.py", + name.replace('-', "_") + )), + module, + ) + .unwrap(); + } + let native_id = "41a04bb3-86f0-463d-97c5-f085e8a9ea4b"; + let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); + let bundle_document=format!("bundle:\n name: closed-native-bundle\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); + std::fs::write(bundle.join("bundle.yaml"), &bundle_document).unwrap(); + let settings=format!("bundle:\n active: file://{}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n",bundle.display()); + std::fs::write(home.join(".amplifier/settings.yaml"), &settings).unwrap(); + let env_file = fixture.join("grants/environment.env"); + std::fs::write( + &env_file, + "OPENAI_API_KEY=synthetic-amplifier-onecli-value\nOPENAI_BASE_URL=http://127.0.0.1:9\n", + ) + .unwrap(); + std::fs::set_permissions(&env_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + std::env::set_var("FRESHELL_MANAGED_AMPLIFIER_ONECLI_ENV_FILE", &env_file); + let registry = Registry::open( + fixture.join("registry"), + Some( + InstallationId::parse(std::env::var("FRESHELL_PROVIDER_LAUNCH_INSTALLATION").unwrap()) + .unwrap(), + ), + ) + .unwrap(); + std::env::set_var("FRESHELL_MANAGED_PROVIDER_QUALIFICATION", "amplifier"); + let socket = control.join("control.sock"); + let secret_file = control.join("secret"); + std::fs::write(&secret_file, "closed-synthetic-control-secret").unwrap(); + std::fs::set_permissions(&secret_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + freshell_supervisor::backend::docker::configure_capability_dir(&socket).unwrap(); + let supervisor = Supervisor::new( + registry.clone(), + Arc::new(BrokerAuthorizedBackend { + real: DockerEngineBackend::new(fixture.join("broker.sock")), + control: control.clone(), + handoff: fixture.join("handoff"), + certificate_sentinel: b"synthetic-amplifier-onecli-value".to_vec(), + }), + SupervisorConfig { + runtime_root: runtime, + control_socket_path: socket.clone(), + host_binary_path: PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_HOST_BINARY").unwrap(), + ), + image_ref: std::env::var("FRESHELL_PROVIDER_LAUNCH_IMAGE").unwrap(), + test_run_id: std::env::var("FRESHELL_PROVIDER_LAUNCH_RUN_ID").unwrap(), + control_secret: "closed-synthetic-control-secret".into(), + lifecycle_log: fixture.join("lifecycle.jsonl"), + admission: AdmissionPolicy::default(), + }, + ) + .unwrap(); + let control_task = tokio::spawn({ + let socket = socket.clone(); + async move { + serve_control(supervisor, &socket).await.unwrap(); + } + }); + tokio::time::timeout(Duration::from_secs(5), async { + while !socket.exists() { + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .unwrap(); + let web = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap(); + let port = web.local_addr().unwrap().port(); + let app = axum::Router::new() + .route("/api/health", axum::routing::get(|| async { "ok" })) + .route( + "/api/mcp", + axum::routing::post(|headers: axum::http::HeaderMap| async move { + if headers + .get("x-auth-token") + .is_some_and(|token| token == "closed-synthetic-web-token") + { + axum::http::StatusCode::OK + } else { + axum::http::StatusCode::UNAUTHORIZED + } + }), + ) + .layer(axum::middleware::from_fn_with_state( + Arc::new("closed-synthetic-web-token".to_string()), + crate::managed_mcp_capability::scoped_auth_middleware, + )); + let web_task = tokio::spawn(async move { axum::serve(web, app).await.unwrap() }); + crate::managed_mcp_capability::configure("0.0.0.0", port, &socket, &secret_file).unwrap(); + crate::managed_mcp_capability::spawn_callback_server() + .await + .unwrap(); + for name in [ + "FRESHELL_MANAGED_RUNTIME_V1", + "FRESHELL_MANAGED_FRESH_AGENT_V1", + ] { + std::env::set_var(name, "1"); + } + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SOCKET", &socket); + std::env::set_var("FRESHELL_RUNTIME_CONTROL_SECRET_FILE", &secret_file); + std::env::set_var("HOME", &home); + let controller = crate::managed_runtime::ServerManagedRuntimeController::from_env() + .await + .unwrap() + .unwrap(); + let cwd = workspace.to_str().unwrap().to_string(); + let terminal = controller + .launch(ManagedTerminalLaunch { + spec: freshell_platform::spawn::SpawnSpec { + program: "amplifier".into(), + args: vec![ + "session".into(), + "resume".into(), + "--full-history".into(), + native_id.into(), + ], + env_overrides: BTreeMap::new(), + cwd: Some(cwd), + cols: 120, + rows: 30, + }, + env: BTreeMap::new(), + terminal_id: "closed-native-terminal".into(), + stream_id: "closed-native-stream".into(), + mode: "amplifier".into(), + resume_session_id: Some(native_id.into()), + provider_model: None, + provider_reasoning_effort: None, + provider_sandbox: None, + provider_permission_mode: None, + view_tab_id: None, + view_pane_id: None, + create_request_id: Some("closed-native-terminal-create".into()), + }) + .await + .expect("complete native Amplifier terminal startup"); + let client = controller.runtime_client(); + let observed = tokio::time::timeout(Duration::from_secs(45), async { + loop { + if let Ok(bytes) = std::fs::read(&receipt) { + if let Ok(value) = serde_json::from_slice::(&bytes) { + break value; + } + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await + .ok(); + if observed.is_some() { + tokio::time::sleep(Duration::from_millis(300)).await; + } + let output = client + .read_output( + SoulId::parse(terminal.soul_id.clone()).unwrap(), + 0, + 64 * 1024, + ) + .await + .unwrap(); + if observed.is_none() || output.exited { + let text = crate::ai_title::strip_ansi( + &output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(), + ); + let classes = [ + "ENOENT", + "EACCES", + "Module", + "module", + "provider", + "Provider", + "orchestrator", + "context", + "Error", + "Error:", + "failed", + "Traceback", + "installed", + "Installing", + "dependency", + "missing", + "Permission", + "session", + "Session", + "bundle", + "Bundle", + "setup", + "configure", + "No module named", + "ModuleNotFoundError", + "ImportError", + "TypeError", + "AttributeError", + "ValidationError", + "RuntimeError", + "FileNotFoundError", + "PermissionError", + "ModuleLoadError", + "ModuleValidationError", + "Failed to install", + "Failed to load", + "Failed to mount", + "Failed to initialize", + "Failed to resolve", + "not found", + "not installed", + "pyproject.toml", + "No solution found", + "network", + "Connection", + "loop-probe", + "context-probe", + "hooks-probe", + "provider-vllm", + "orchestrator missing", + "context missing", + "required", + "abstract", + "signature", + "source", + "not found in prepared bundle", + "no source hint", + "activation failed", + "Failed to activate", + "Failed to lazy-activate", + "does not exist", + "entry points or filesystem", + "has no valid Python package", + "failed validation", + "Mount function", + "must be async", + "No pyproject.toml", + "Could not find", + "Cannot find", + "Could not load", + "Module source", + "Source path", + "All configured", + "not available", + "not mounted", + "install dependencies", + "dependency installation", + "No context module", + "No orchestrator module", + "not a valid", + "parse", + "activate", + "resolve", + "validate", + "uv is not installed", + "File not found:", + "No such file or directory", + "Read-only file system", + "Permission denied", + "Failed to install module", + "No providers", + "No context", + "No orchestrator", + "Failed to initialize session", + "validation", + "mount", + "uv", + "pip", + "base_url", + "session_id", + "attribute", + "await", + "Context", + "dict", + "Prompt", + "factory", + ] + .into_iter() + .filter(|class| text.contains(class)) + .collect::>(); + // Emit only finite diagnostic labels; paths, credentials, arbitrary + // exception payloads and provider output never enter the test log. + let labels = "error failed unable cannot could not create remove write read only readonly filesystem permission denied temporary directory file path uv pip install installed package build backend fetch download resolve git clone network connection certificate tls ssl python virtual environment target source module cache lock resource unavailable no such system command subprocess status exit stdout stderr caused by requires version dependency dependencies hooks-mode modes hatchling setuptools attribute object dict await coroutine context set_system_prompt_factory get_context_info config keys data serialize tuple type callable required schema factory prompt" + .split_whitespace().collect::>(); + let classified_errors = text + .lines() + .filter(|line| { + let line = line.to_ascii_lowercase(); + line.contains("error") || line.contains("failed") || line.contains("caused by") + }) + .take(16) + .map(|line| { + line.split(|ch: char| !ch.is_ascii_alphabetic() && ch != '-') + .filter_map(|word| { + labels + .iter() + .copied() + .find(|label| word.eq_ignore_ascii_case(label)) + }) + .take(80) + .collect::>() + }) + .collect::>(); + println!( + "provider_launch_diagnostic {}", + json!({"provider":"amplifier","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes,"classifiedErrors":classified_errors}) + ); + } + let expected = json!({"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency")}); + assert_eq!(observed.expect("real Amplifier must activate selected local module and executable skill before a model turn"),expected); + assert!( + !output.exited, + "native Amplifier must remain usable after the initialization consumer" + ); + println!( + "provider_launch_consumer {}", + json!({"provider":"amplifier","route":"terminal","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + ); + assert_eq!( + std::fs::read_to_string(home.join(".amplifier/settings.yaml")).unwrap(), + settings + ); + assert_eq!( + std::fs::read_to_string(bundle.join("bundle.yaml")).unwrap(), + bundle_document + ); + client + .stop(SoulId::parse(terminal.soul_id).unwrap()) + .await + .unwrap(); + web_task.abort(); + control_task.abort(); +} + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prepared_inputs() { use freshell_supervisor::{ diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 247e18006..1410c8fa2 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -2,6 +2,7 @@ //! Plans retain approved source metadata and unexpanded templates, never payload bytes. use std::path::{Path, PathBuf}; +pub(crate) mod amplifier; pub(crate) mod claude; pub(crate) mod codex; pub(crate) mod environment; diff --git a/crates/freshell-server/src/managed_provider_references/amplifier.rs b/crates/freshell-server/src/managed_provider_references/amplifier.rs new file mode 100644 index 000000000..be2c3ba22 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references/amplifier.rs @@ -0,0 +1,62 @@ +//! Ordinary Amplifier sources use the accepted initial creation authority. +use freshell_runtime_protocol::*; +use std::path::Path; + +pub(crate) struct AmplifierPreparation<'a> { + pub client: &'a freshell_runtime_client::RuntimeClient, + pub soul: &'a SoulId, + pub creation_seed: &'a str, + pub home: &'a Path, + pub cwd: &'a Path, + pub workspace: &'a Path, + pub references: &'a [ProviderSecretReference], +} +pub(crate) async fn prepare_managed( + request: AmplifierPreparation<'_>, +) -> Result { + let manifest = freshell_platform::amplifier_inputs::AmplifierInputManifest { + version: 1, + original_home: request.home.into(), + cwd: request.cwd.into(), + workspace: request.workspace.into(), + }; + manifest.validate().map_err(str::to_string)?; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Amplifier { + bundle: "default".into(), + resume_args: vec![], + }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + let reservation = request + .client + .reserve_provider_inputs( + request.soul.clone(), + "amplifier".into(), + request.creation_seed.into(), + ) + .await + .map_err(|error| error.to_string())?; + if reservation.provider != "amplifier" + || &reservation.soul_id != request.soul + || reservation.creation_seed_ref != request.creation_seed + { + return Err("Amplifier selected resource reservation does not match this creation".into()); + } + crate::managed_mcp_capability::issue_prepared_provider_context( + "amplifier", + request.soul, + request.home, + &context, + crate::managed_mcp_capability::OpencodeEphemeralInput { + amplifier_manifest: Some(&manifest), + cwd: Some(request.cwd), + workspace: Some(request.workspace), + secret_references: Some(request.references), + initial_auth_reservation: Some(&reservation.reservation_id), + ..Default::default() + }, + ) +} diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index b2cafe1ae..5c2cd8159 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -273,6 +273,23 @@ impl ManagedTerminalController for ServerManagedRuntimeController { ) .await?; (context.mcp_capability.clone(), Some(context)) + } else if request.mode == "amplifier" { + let home = std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .ok_or("original Amplifier HOME is unavailable")?; + let context = crate::managed_provider_references::amplifier::prepare_managed( + crate::managed_provider_references::amplifier::AmplifierPreparation { + client: &self.client, + soul: &soul_id, + creation_seed: create_key, + home: &home, + cwd: &cwd, + workspace: &workspace, + references: &provider_secret_references, + }, + ) + .await?; + (context.mcp_capability.clone(), Some(context)) } else { let mcp_capability = matches!( request.mode.as_str(), @@ -775,7 +792,26 @@ fn managed_provider_env( let mut env = managed_env_allowlist(source); // Never persist the web host's PATH into a pinned workload image. It may // contain host-only homes, version-manager shims, or credential helpers. - env.insert("PATH".into(), "/usr/local/bin:/usr/bin:/bin".into()); + let runtime_path = if mode == "amplifier" { + // The pinned image installs Amplifier's native module installer here. + "/opt/uv/bin:/usr/local/bin:/usr/bin:/bin" + } else { + "/usr/local/bin:/usr/bin:/bin" + }; + env.insert("PATH".into(), runtime_path.into()); + if mode == "amplifier" { + // Native module activation may install package dependencies. Its + // interpreter stays in the pinned image; new packages belong to this + // soul's writable HOME and are visible to that same interpreter. + env.insert( + "UV_CONFIG_FILE".into(), + "/home/freshell/provider/.freshell/amplifier/uv.toml".into(), + ); + env.insert( + "PYTHONPATH".into(), + "/home/freshell/provider/.freshell/amplifier/python-packages".into(), + ); + } // Provider state belongs to this soul's Docker volume, not the web // server's inherited HOME. These paths are inside the workload. env.insert("HOME".into(), "/home/freshell/provider".into()); diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index a6cd300b9..8461c8265 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -164,6 +164,7 @@ async fn run() -> Result<(), String> { Some("provider-probe-worker") => providers::run_probe_worker(&args[2..]), Some("refresh-provider-config") => refresh_provider_config_worker(&args[2..]), Some("prepare-provider-features") => providers::run_provider_features_worker(&args[2..]), + Some("prepare-amplifier-state") => providers::run_amplifier_state_worker(&args[2..]), _ => Err("usage: freshell-session-host ...".into()), } } @@ -886,7 +887,7 @@ async fn grant_execution( format!("resolve managed provider credentials: {error}"), ) })?; - if matches!(provider, "opencode" | "codex" | "claude") { + if matches!(provider, "opencode" | "codex" | "claude" | "amplifier") { let generated = provider_input_preparation::prepare( context, &mut secrets, @@ -1215,6 +1216,33 @@ async fn grant_execution( fn prepare_provider_state_for_fresh_launch( terminal: &mut TerminalLaunchSpec, ) -> Result<(), RuntimeError> { + if terminal.mode == "amplifier" + && terminal.provider_bootstrap_files.is_empty() + && terminal + .provider_launch_context + .as_ref() + .is_some_and(|context| context.prepared_inputs.is_some()) + { + // Selected configuration must be available before the native stub + // records the active bundle. All new state is born under the provider + // identity; no pre-existing HOME ownership is acquired by the host. + verify_initialized_provider_home( + Path::new("/home/freshell/provider"), + terminal.run_as_uid, + terminal.run_as_gid, + ) + .and_then(|_| { + prepare_provider_context_for_fresh_launch(terminal, &ProviderConfigPaths::default()) + }) + .and_then(|_| providers::prepare_amplifier_state_as_owner(terminal)) + .map_err(|error| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + format!("prepare managed Amplifier state: {error}"), + ) + })?; + return Ok(()); + } prepare_provider_state_before_bootstrap_with( terminal, |terminal| { diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 12516dadd..5d21e7d6d 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -56,7 +56,7 @@ pub fn prepare( return Err("prepared source manifest changed before dispatch".into()); } let provider = context.unwrap().preparation.provider(); - if matches!(provider, "codex" | "claude") { + if matches!(provider, "codex" | "claude" | "amplifier") { if !frozen.is_empty() || inputs.manifest.source_count != 0 || inputs.manifest.alias_count != 0 @@ -72,6 +72,13 @@ pub fn prepare( manifest.validate().map_err(str::to_string)?; return Ok(BTreeMap::new()); } + if provider == "amplifier" { + let manifest: freshell_platform::amplifier_inputs::AmplifierInputManifest = + serde_json::from_slice(&metadata) + .map_err(|_| "prepared Amplifier discovery metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + return Ok(BTreeMap::new()); + } let manifest: freshell_platform::codex_inputs::CodexInputManifest = serde_json::from_slice(&metadata) .map_err(|_| "prepared Codex discovery metadata invalid")?; @@ -356,6 +363,73 @@ mod tests { ); } + #[test] + fn provider_secret_amplifier_prepared_manifest_validates_without_original_controller_tree() { + let owned = tempfile::tempdir().unwrap(); + std::fs::create_dir(owned.path().join(".freshell")).unwrap(); + let manifest = freshell_platform::amplifier_inputs::AmplifierInputManifest { + version: 1, + original_home: "/absent-original-home".into(), + cwd: "/absent-original-workspace".into(), + workspace: "/absent-original-workspace".into(), + }; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let mut context = ProviderLaunchContext { + preparation: ProviderPreparation::Amplifier { + bundle: "default".into(), + resume_args: vec![], + }, + config: vec![], + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }), + mcp_capability: Some(capability), + }; + let mut secrets = ResolvedProviderSecrets::default(); + let generated=prepare(Some(&context),&mut secrets,&ProviderPathInputs::default(),owned.path(),owned.path()) + .expect("Amplifier source origins validate from accepted private metadata, without rescanning controller paths"); + assert!(generated.is_empty()); + assert!( + secrets.opencode_plan.is_none(), + "ordinary Amplifier cannot gain OpenCode source/auth semantics" + ); + context + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .source_count = 1; + assert!( + prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + owned.path(), + owned.path() + ) + .is_err(), + "foreign source recipes cannot enter ordinary Amplifier metadata" + ); + } + #[test] fn provider_secret_codex_prepared_manifest_retains_native_home_without_original_tree() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 332a1c700..655e1ab4d 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -58,6 +58,108 @@ pub fn prepare_provider_state_before_bootstrap( Ok(ensured.created_paths) } +/// Prepared inputs already have a private, provider-owned HOME. Create native +/// session state as that owner rather than transferring pre-existing inodes. +pub fn prepare_amplifier_state_as_owner( + terminal: &mut freshell_runtime_protocol::TerminalLaunchSpec, +) -> Result<(), String> { + let session_id = terminal + .resume_session_id + .as_deref() + .filter(|id| !id.is_empty()) + .ok_or("managed Amplifier launch requires one exact launcher-assigned session id")?; + let exe = std::env::current_exe().map_err(|error| error.to_string())?; + let output = std::process::Command::new("/usr/bin/setpriv") + .args(crate::provider_identity_args( + terminal.run_as_uid, + terminal.run_as_gid, + )) + .arg(exe) + .arg("prepare-amplifier-state") + .args([session_id, &terminal.cwd, &terminal.terminal_id]) + .stderr(std::process::Stdio::null()) + .output() + .map_err(|_| "prepare Amplifier session state helper failed")?; + if !output.status.success() || output.stdout.len() > 256 * 1024 { + return Err("prepare Amplifier session state helper failed".into()); + } + let recorded_cwd: Option = serde_json::from_slice(&output.stdout) + .map_err(|_| "prepare Amplifier session state response is invalid")?; + if let Some(cwd) = recorded_cwd { + validate_amplifier_state_cwd(&cwd)?; + terminal.cwd = cwd; + } + Ok(()) +} + +fn validate_amplifier_state_cwd(cwd: &str) -> Result<(), String> { + if cwd.len() > 64 * 1024 + || !std::path::Path::new(cwd).is_absolute() + || cwd.chars().any(char::is_control) + { + return Err("Amplifier session working directory is invalid".into()); + } + Ok(()) +} + +pub(crate) fn run_amplifier_state_worker(args: &[String]) -> Result<(), String> { + let [session_id, cwd, terminal_id] = args else { + return Err("invalid Amplifier state preparation arguments".into()); + }; + validate_amplifier_state_cwd(cwd)?; + let ensured = freshell_sessions::amplifier_stub::ensure_session( + std::path::Path::new("/home/freshell/provider/.amplifier"), + session_id, + cwd, + terminal_id, + ) + .map_err(|error| format!("prepare Amplifier session stub: {error}"))?; + if let Some(recorded_cwd) = ensured.working_dir_of_existing.as_deref() { + validate_amplifier_state_cwd(recorded_cwd)?; + } + let response = serde_json::to_vec(&ensured.working_dir_of_existing) + .map_err(|_| "encode Amplifier state response failed")?; + std::io::Write::write_all(&mut std::io::stdout(), &response) + .map_err(|_| "write Amplifier state response failed".into()) +} + +fn prepare_amplifier_package_target() -> Result<(), String> { + use std::os::unix::fs::{DirBuilderExt, OpenOptionsExt}; + let root = std::path::Path::new("/home/freshell/provider/.freshell/amplifier"); + let config = root.join("uv.toml"); + let bytes = + b"[pip]\ntarget = \"/home/freshell/provider/.freshell/amplifier/python-packages\"\n"; + std::fs::DirBuilder::new() + .recursive(true) + .mode(0o700) + .create(root) + .map_err(|_| "create managed Amplifier package configuration directory failed")?; + match std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&config) + { + Ok(mut file) => std::io::Write::write_all(&mut file, bytes) + .map_err(|_| "write managed Amplifier package configuration failed".into()), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + let metadata = std::fs::symlink_metadata(&config) + .map_err(|_| "inspect managed Amplifier package configuration failed")?; + if !metadata.is_file() + || metadata.file_type().is_symlink() + || metadata.len() != bytes.len() as u64 + || std::fs::read(&config) + .map_err(|_| "read managed Amplifier package configuration failed")? + != bytes + { + return Err("managed Amplifier package configuration conflicts with its fixed native target".into()); + } + Ok(()) + } + Err(_) => Err("create managed Amplifier package configuration failed".into()), + } +} + pub async fn prepare_terminal( mut terminal: freshell_runtime_protocol::TerminalLaunchSpec, child_secret_env: &std::collections::BTreeMap, @@ -66,6 +168,9 @@ pub async fn prepare_terminal( let mut child_env = child_secret_env.clone(); prepare_scoped_mcp_environment(terminal.provider_launch_context.as_ref(), &mut child_env) .await?; + if terminal.mode == "amplifier" { + prepare_provider_features(&terminal, "amplifier-packages")?; + } if terminal.mode == "claude" && terminal .provider_launch_context @@ -391,6 +496,7 @@ fn opencode_plan( pub(crate) fn run_provider_features_worker(args: &[String]) -> Result<(), String> { let provider_home = std::path::Path::new("/home/freshell/provider"); match args { + [operation] if operation == "amplifier-packages" => prepare_amplifier_package_target(), [operation] if operation == "claude-mcp" => { use std::os::unix::fs::OpenOptionsExt; let destination = provider_home.join(".claude/freshell-mcp.json"); diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 9f4353d21..441644aa2 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -12,7 +12,7 @@ const configuration=JSON.parse(fs.readFileSync(path.join(root,'broker-configurat const image=configuration.provider==='claude' ? 'sha256:b09595b8f02dae505e6719d7617305ab8f5ad53a7138b3b1580ac553c0f2663a' : 'sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c' -if (configuration.imageRef!==image || !['opencode','codex','claude'].includes(configuration.provider) || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==2) { +if (configuration.imageRef!==image || !['opencode','codex','claude','amplifier'].includes(configuration.provider) || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==(configuration.provider==='amplifier'?1:2)) { throw new Error('closed provider startup image or finite source set changed') } const gateways=new Set() @@ -25,6 +25,8 @@ const broker=new RestrictedDockerBroker({ ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-plugin'),path.join(root,'source-home/imported-memory.md')] : configuration.provider==='codex' ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] + : configuration.provider==='amplifier' + ? ['/home/freshell/provider',path.join(root,'source-home/.amplifier')] : ['/home/freshell/provider',path.join(root,'.opencode')]), allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json'),path.join(root,'grants/ca.pem')]), allowedHostGatewayAddresses:gateways, @@ -43,7 +45,7 @@ const timer=setInterval(()=>{ } const directory=path.join(root,'handoff') const entries=fs.readdirSync(directory).filter((name)=>name.endsWith('.json')) - if (entries.length>2) throw new Error('closed provider fixture exceeds its finite two launch attempts') + if (entries.length>(configuration.provider==='amplifier'?1:2)) throw new Error('closed provider fixture exceeds its finite two launch attempts') for (const name of entries) { if (accepted.has(name)) continue if (!/^[a-zA-Z0-9_-]{1,128}\.json$/.test(name)) throw new Error('closed provider fixture handoff name invalid') diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 2e80f31b6..2163d6990 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -13,6 +13,7 @@ IMAGE="sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c" TEST="managed_provider_launch_tests::managed_provider_opencode_terminal_and_fresh_native_startup_consumes_prepared_inputs" CODEX_TEST="managed_provider_launch_tests::managed_provider_codex_terminal_and_fresh_native_startup_consumes_prepared_inputs" +AMPLIFIER_TEST="managed_provider_launch_tests::managed_provider_amplifier_terminal_native_startup_consumes_prepared_inputs" CLAUDE_TEST="managed_provider_launch_tests::managed_provider_claude_terminal_and_fresh_native_startup_consumes_prepared_inputs" CLAUDE_IMAGE="sha256:b09595b8f02dae505e6719d7617305ab8f5ad53a7138b3b1580ac553c0f2663a" LABEL="com.freshell.provider-launch-probe" @@ -31,9 +32,9 @@ def main(): parser.add_argument("--host-binary",type=Path,required=True) parser.add_argument("--broker-binary",type=Path,required=True) parser.add_argument("--receipt-dir",type=Path,required=True) - parser.add_argument("--provider",choices=["opencode","codex","claude"],default="opencode") + parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") args=parser.parse_args() - selected_test={"opencode":TEST,"codex":CODEX_TEST,"claude":CLAUDE_TEST}[args.provider] + selected_test={"opencode":TEST,"codex":CODEX_TEST,"claude":CLAUDE_TEST,"amplifier":AMPLIFIER_TEST}[args.provider] if args.provider=="claude": IMAGE=CLAUDE_IMAGE repo=Path(__file__).resolve().parents[2] test=args.test_binary.resolve(strict=True);host=args.host_binary.resolve(strict=True);broker=args.broker_binary.resolve(strict=True) @@ -87,7 +88,9 @@ def create(name,*arguments): for name in ["workspace","source-home/.config/opencode","grants","handoff","tmp","scenarios"]: (root/name).mkdir(parents=True,exist_ok=True) initial=[] - for soul in ["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32],"soul-fresh-"+args.provider+"-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]]: + souls=["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32]] + if args.provider!="amplifier": souls.append("soul-fresh-"+args.provider+"-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]) + for soul in souls: volume="freshell-provider-"+hashlib.sha256((installation+"\0"+soul).encode()).hexdigest()[:24] volumes.add(volume) initial.append({"installationId":installation,"soulId":soul,"providerVolumeName":volume,"imageRef":IMAGE,"hostBinaryPath":str(host),"binarySha256":digest(host),"daemonId":info["ID"]}) @@ -178,7 +181,7 @@ def create(name,*arguments): receipt["exitCode"]=exit_code receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) - expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else {(args.provider,"terminal"),(args.provider,"fresh")} + expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} if exit_code==0 and (len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes or "test result: ok. 1 passed;" not in outcome.stdout): receipt["exitCode"]=2;receipt["failureClass"]="native_startup_count_mismatch" From bf468a845ce38f6ecd15ff9390755e953c72002a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 08:03:55 -0700 Subject: [PATCH 083/493] fix: prepare native inputs for existing Kilroy sessions --- .../freshell-agent-runtime/src/host_actor.rs | 2 +- .../src/host_actor_tests.rs | 10 +- crates/freshell-runtime-protocol/src/lib.rs | 48 ++++++++- .../freshell-server/src/fresh_agent_proxy.rs | 51 ++-------- .../src/managed_provider_launch_tests.rs | 97 ++++++++++++++++++- .../src/managed_provider_references/claude.rs | 8 +- crates/freshell-server/src/managed_runtime.rs | 1 + crates/freshell-session-host/src/main.rs | 10 +- crates/freshell-supervisor/src/backend.rs | 5 +- .../freshell-supervisor/src/backend/docker.rs | 2 +- crates/freshell-supervisor/src/recovery.rs | 4 +- crates/freshell-supervisor/src/registry.rs | 72 ++++++++++++++ .../src/registry/input_reservation.rs | 7 +- .../testing/managed-provider-launch-broker.ts | 4 + .../managed-provider-launch-sandbox.py | 10 +- 15 files changed, 270 insertions(+), 61 deletions(-) diff --git a/crates/freshell-agent-runtime/src/host_actor.rs b/crates/freshell-agent-runtime/src/host_actor.rs index 5ab0fec2d..8c2383c83 100644 --- a/crates/freshell-agent-runtime/src/host_actor.rs +++ b/crates/freshell-agent-runtime/src/host_actor.rs @@ -102,7 +102,7 @@ impl FreshAgentProfile { if self .provider_launch_context .as_ref() - .is_some_and(|context| context.validate(self.provider.as_str()).is_err()) + .is_some_and(|context| context.validate(self.provider.native_provider()).is_err()) { return Err(ActorError::InvalidProfile); } diff --git a/crates/freshell-agent-runtime/src/host_actor_tests.rs b/crates/freshell-agent-runtime/src/host_actor_tests.rs index 5bcafe8d1..f39be5972 100644 --- a/crates/freshell-agent-runtime/src/host_actor_tests.rs +++ b/crates/freshell-agent-runtime/src/host_actor_tests.rs @@ -829,7 +829,15 @@ async fn one_state_directory_has_only_one_writer() { #[tokio::test] async fn exact_native_identity_and_profile_are_reused_on_host_actor_recovery() { let dir = tempfile::tempdir().unwrap(); - let initial = profile(FreshProvider::Kilroy, "kilroy-store", None); + let mut initial = profile(FreshProvider::Kilroy, "kilroy-store", None); + initial.provider_launch_context = Some(freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Claude { + mcp_args: Vec::new(), + }, + prepared_inputs: None, + mcp_capability: None, + config: Vec::new(), + }); let actor = FreshAgentHostActor::open(dir.path(), initial, FakeTransport::new("claude-native-9")) .await diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index ffb7e1c95..e2e183442 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -1724,6 +1724,15 @@ mod double_optional { } impl FreshProvider { + /// Native input recipes are shared by the two existing Claude flavors. + /// Runtime identities and reservation owners still use `as_str()`. + pub fn native_provider(&self) -> &'static str { + match self { + Self::Kilroy => "claude", + _ => self.as_str(), + } + } + pub fn as_str(&self) -> &'static str { match self { Self::Claude => "claude", @@ -1880,9 +1889,12 @@ impl FreshAgentLaunchSpec { } } if let Some(context) = &self.provider_launch_context { - context.validate(self.provider.as_str())?; + context.validate(self.provider.native_provider())?; } - validate_provider_secrets(self.provider.as_str(), &self.provider_secret_references)?; + validate_provider_secrets( + self.provider.native_provider(), + &self.provider_secret_references, + )?; if let Some(inputs) = self .provider_launch_context .as_ref() @@ -3812,6 +3824,38 @@ mod tests { )); } + #[test] + fn fresh_kilroy_consumes_claude_recipes_without_changing_runtime_identity() { + let mut launch = fresh_agent_launch(FreshProvider::Kilroy, "kilroy"); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { + mcp_args: Vec::new(), + }, + prepared_inputs: None, + mcp_capability: None, + config: Vec::new(), + }; + assert!(context.validate("kilroy").is_err()); + launch.provider_launch_context = Some(context); + launch.provider_secret_references = vec![ProviderSecretReference { + source_path: "/synthetic/private/claude-environment".into(), + profile: ProviderSecretProfile::ClaudeOnecliEnvironment, + provider_relative_path: None, + }]; + launch.validate().unwrap(); + let retained: FreshAgentLaunchSpec = + serde_json::from_slice(&serde_json::to_vec(&launch).unwrap()).unwrap(); + assert_eq!(retained.provider, FreshProvider::Kilroy); + assert_eq!(retained.session_type, "kilroy"); + assert_eq!(retained.native_session_id.as_deref(), Some("native-one")); + assert_eq!(retained, launch); + launch.provider = FreshProvider::Codex; + launch.session_type = "freshcodex".into(); + assert!(launch.validate().is_err()); + launch.provider_launch_context = None; + assert!(launch.validate().is_err()); + } + #[test] fn fresh_agent_validation_rejects_provider_type_mismatch_and_mixed_workloads() { let agent = fresh_agent_launch(FreshProvider::Codex, "freshclaude"); diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index f034a21fa..34d34ec99 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -18,9 +18,9 @@ use freshell_protocol::{ use freshell_runtime_client::RuntimeClient; use freshell_runtime_protocol::{ AgentEvent, DesiredState, FreshAgentFixtureTransport, FreshAgentLaunchSpec, - FreshAgentRollbackDirection, FreshAgentRollbackMode, FreshProvider, LaunchRequest, - ProviderBootstrapFile, RequestId, RuntimeErrorCode, RuntimeLimits, RuntimeProfile, SoulId, - StopOutcome, ViewIntentKind, ViewIntentRequest, ViewVisibilityIntent, + FreshAgentRollbackDirection, FreshAgentRollbackMode, FreshProvider, LaunchRequest, RequestId, + RuntimeErrorCode, RuntimeLimits, RuntimeProfile, SoulId, StopOutcome, ViewIntentKind, + ViewIntentRequest, ViewVisibilityIntent, }; use freshell_ws::hosted_fresh_agent::{HostedFreshAgentCommand, HostedFreshAgentGateway}; use sha2::{Digest, Sha256}; @@ -453,10 +453,12 @@ impl HostedFreshAgentProxy { .map(|value| value.session_id.clone()); let session_type = session_type_wire(message.session_type); let request_id = stable_request_id(&message.request_id).map_err(|_| ())?; - let provider_bootstrap_files = provider_bootstrap_files(&provider).map_err(|_| ())?; + let provider_bootstrap_files = Vec::new(); let mut provider_secret_references = - crate::managed_provider_bootstrap::named_provider_onecli_references(provider.as_str()) - .map_err(|_| ())?; + crate::managed_provider_bootstrap::named_provider_onecli_references( + provider.native_provider(), + ) + .map_err(|_| ())?; let (mcp_capability, mut provider_launch_context) = if provider == FreshProvider::Opencode { let home = std::env::var_os("HOME") .map(std::path::PathBuf::from) @@ -515,12 +517,13 @@ impl HostedFreshAgentProxy { tracing::warn!(event="fresh_agent.create.provider_inputs_failed",%error); })?; (context.mcp_capability.clone().ok_or(())?, Some(context)) - } else if provider == FreshProvider::Claude { + } else if matches!(provider, FreshProvider::Claude | FreshProvider::Kilroy) { let home = std::env::var_os("HOME") .map(std::path::PathBuf::from) .ok_or(())?; let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { + reservation_provider: provider.as_str(), client: &self.client, soul: &soul, creation_seed: &message.request_id, @@ -1289,40 +1292,6 @@ fn git_stdout(cwd: &Path, args: &[&str]) -> Option { .then(|| String::from_utf8_lossy(&output.stdout).into_owned()) } -fn provider_bootstrap_files( - provider: &FreshProvider, -) -> Result, String> { - let home = std::env::var("HOME") - .ok() - .filter(|value| !value.is_empty()) - .map(PathBuf::from); - let (override_key, fallback, relative) = match provider { - FreshProvider::Kilroy => ( - "FRESHELL_MANAGED_CLAUDE_CREDENTIAL_FILE", - home.map(|value| value.join(".claude/.credentials.json")), - ".claude/.credentials.json", - ), - FreshProvider::Claude | FreshProvider::Codex | FreshProvider::Opencode => { - return Ok(Vec::new()) - } - }; - let candidate = std::env::var(override_key) - .ok() - .filter(|value| !value.is_empty()) - .map(PathBuf::from) - .or(fallback) - .filter(|value| value.is_file()); - let Some(candidate) = candidate else { - return Ok(Vec::new()); - }; - let source_path = std::fs::canonicalize(candidate) - .map_err(|_| "provider credential reference is unreadable".to_string())?; - Ok(vec![ProviderBootstrapFile { - source_path: source_path.to_string_lossy().into_owned(), - provider_relative_path: relative.into(), - }]) -} - #[cfg(test)] #[path = "fresh_agent_proxy_tests.rs"] mod tests; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 6f2416d3f..0a47fb9d8 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -215,6 +215,97 @@ impl RuntimeBackend for BrokerAuthorizedBackend { } } +fn capture_fresh_launch_failure_classes() -> std::sync::Arc>> { + use tracing_subscriber::prelude::*; + #[derive(Clone)] + struct FailureLayer(std::sync::Arc>>); + impl tracing_subscriber::Layer for FailureLayer { + fn on_event( + &self, + event: &tracing::Event<'_>, + _: tracing_subscriber::layer::Context<'_, S>, + ) { + if !event.metadata().target().contains("fresh_agent_proxy") { + return; + } + struct Classes(Vec<&'static str>); + impl tracing::field::Visit for Classes { + fn record_debug( + &mut self, + field: &tracing::field::Field, + value: &dyn std::fmt::Debug, + ) { + if field.name() != "error" { + return; + } + let text = format!("{value:?}"); + for label in [ + "HostUnreachable", + "InvalidRequest", + "ProviderInputsChanged", + "prepare hosted provider config", + "prepare hosted fresh-agent provider state", + "prepare hosted OneCLI provider files", + "open hosted", + "OneCLI", + "profile", + "credential", + "unsafe or mismatched", + "MCP", + "sidecar", + "source", + "manifest", + "copy", + "Permission", + "unavailable", + "initialization", + "timed out", + "Claude", + "kilroy", + "environment", + "grant", + "provider", + "configuration", + "refresh", + "input bind", + "trusted uid/group", + "provider home", + "resource", + "destination", + "projection", + "stage", + "metadata", + "path", + "readable", + "preparation", + "version", + "changed", + "rootless", + "directory", + "filesystem", + "temporary", + "failed to start", + "failed with status", + ] { + if text.contains(label) { + self.0.push(label); + } + } + } + } + let mut classes = Classes(Vec::new()); + event.record(&mut classes); + self.0.lock().unwrap().extend(classes.0); + } + } + let classes = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + tracing::subscriber::set_global_default( + tracing_subscriber::registry().with(FailureLayer(classes.clone())), + ) + .unwrap(); + classes +} + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn managed_provider_amplifier_terminal_native_startup_consumes_prepared_inputs() { use freshell_supervisor::{ @@ -648,6 +739,7 @@ async def on_session_ready(coordinator): #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prepared_inputs() { + let failure_classes = capture_fresh_launch_failure_classes(); use freshell_supervisor::{ admission::AdmissionPolicy, registry::Registry, @@ -1071,11 +1163,14 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ ) .unwrap() .unwrap(); - Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":"freshclaude","provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; + let flavor = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR").unwrap(); + assert!(matches!(flavor.as_str(), "freshclaude" | "kilroy")); + Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":flavor,"provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; tokio::time::timeout(Duration::from_secs(45), async { loop { let frame: Value = serde_json::from_str(&events.recv().await.unwrap()).unwrap(); if frame["type"] == "freshAgent.create.failed" { + println!("provider_launch_failure_classes {}", json!({"provider":"claude","flavor":flavor,"classes":failure_classes.lock().unwrap().clone()})); panic!( "fresh Claude startup failed classification: {}", frame["message"] diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 78c5abdfd..46251b1a2 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -7,6 +7,7 @@ pub(crate) struct ClaudePreparation<'a> { pub client: &'a freshell_runtime_client::RuntimeClient, pub soul: &'a SoulId, pub creation_seed: &'a str, + pub reservation_provider: &'a str, pub home: &'a Path, pub cwd: &'a Path, pub workspace: &'a Path, @@ -18,6 +19,9 @@ pub(crate) struct ClaudePreparation<'a> { pub(crate) async fn prepare_managed( request: ClaudePreparation<'_>, ) -> Result { + if !matches!(request.reservation_provider, "claude" | "kilroy") { + return Err("Claude source preparation requires its exact supported runtime flavor".into()); + } let mut plugin_dirs = Vec::new(); let mut path_bytes = 0usize; let cli = request @@ -63,12 +67,12 @@ pub(crate) async fn prepare_managed( .client .reserve_provider_inputs( request.soul.clone(), - "claude".into(), + request.reservation_provider.into(), request.creation_seed.into(), ) .await .map_err(|error| error.to_string())?; - if reservation.provider != "claude" + if reservation.provider != request.reservation_provider || &reservation.soul_id != request.soul || reservation.creation_seed_ref != request.creation_seed { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 5c2cd8159..4414ff488 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -260,6 +260,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .ok_or("original Claude HOME is unavailable")?; let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { + reservation_provider: "claude", client: &self.client, soul: &soul_id, creation_seed: create_key, diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 8461c8265..c5104da4f 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -869,7 +869,7 @@ async fn grant_execution( .or_else(|| { fresh_agent.as_ref().map(|launch| { ( - launch.provider.as_str(), + launch.provider.native_provider(), launch.provider_secret_references.as_slice(), launch.provider_launch_context.as_ref(), ) @@ -1098,7 +1098,7 @@ async fn grant_execution( } if !exact_resume { prepare_provider_bootstrap_files( - launch.provider.as_str(), + launch.provider.native_provider(), &launch.provider_bootstrap_files, launch.run_as_uid, launch.run_as_gid, @@ -1115,7 +1115,7 @@ async fn grant_execution( })?; prepare_provider_context( launch.provider_launch_context.as_ref(), - launch.provider.as_str(), + launch.provider.native_provider(), &launch.workspace_path, launch.run_as_uid, launch.run_as_gid, @@ -1129,7 +1129,7 @@ async fn grant_execution( } else { refresh_provider_context( launch.provider_launch_context.as_ref(), - launch.provider.as_str(), + launch.provider.native_provider(), &launch.workspace_path, launch.run_as_uid, launch.run_as_gid, @@ -1144,7 +1144,7 @@ async fn grant_execution( ) })?; prepare_provider_files( - launch.provider.as_str(), + launch.provider.native_provider(), &resolved_secrets.provider_files, &resolved_secrets.configuration_files, launch.run_as_uid, diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index a58e204a1..f39cc4b53 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -1080,7 +1080,10 @@ fn runtime_tmpfs(mode: Option<&str>) -> std::collections::BTreeMap Result { - prepared.validate(agent.provider.as_str())?; + prepared.validate(agent.provider.native_provider())?; agent.provider_launch_context = Some(prepared.provider_launch_context); agent.provider_secret_references = prepared.provider_secret_references; } diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 40b84d158..d5910b1e4 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -2964,6 +2964,78 @@ mod tests { ); } + #[tokio::test] + async fn provider_kilroy_ordinary_authority_cannot_bind_the_other_claude_flavor() { + let dir = tempfile::tempdir().unwrap(); + let binary = dir.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic artifact").unwrap(); + let registry = Registry::open(dir.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let request = freshell_runtime_protocol::ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "kilroy".into(), + creation_seed_ref: "seed".into(), + }; + let reservation = registry + .reserve_initial_inputs( + request.clone(), + format!("sha256:{}", "a".repeat(64)), + binary.clone(), + ) + .await + .expect("the existing Kilroy flavor requires its distinct creation authority"); + let mut wrong = request; + wrong.provider = "claude".into(); + assert!(matches!( + registry + .reserve_initial_inputs(wrong, format!("sha256:{}", "a".repeat(64)), binary) + .await, + Err(RegistryError::RequestConflict) + )); + let receipt = registry + .record_initial_observation( + reservation, + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new(), + }, + ) + .await + .unwrap(); + assert_eq!(receipt.provider, "kilroy"); + let mut launch = prep(soul, RequestId::new(), "kilroy-owned-inputs"); + launch.provider = "kilroy".into(); + launch.terminal = None; + launch.fresh_agent = Some(serde_json::from_value(serde_json::json!({ + "sessionId":"synthetic-fresh", "provider":"kilroy", "sessionType":"kilroy", + "runtimeVariant":"kilroy-claude-agent-sdk", "providerStoreId":"test-store", + "cwd":"/synthetic/workspace", "workspacePath":"/synthetic/workspace", "runAsUid":65534,"runAsGid":0, + "providerLaunchContext": { + "preparation":{"claude":{"mcp_args":[]}}, + "mcpCapability":{"grantId":"grant-ordinary", "endpoint":"http://host.docker.internal:3001/api/mcp", "providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":"grant-ordinary", "initialAuthReservation":receipt.reservation_id, + "manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":0,"aliasCount":0,"recipeCount":0}} + } + })).unwrap()); + let mut other = launch.clone(); + other.provider = "claude".into(); + let fresh = other.fresh_agent.as_mut().unwrap(); + fresh.provider = freshell_runtime_protocol::FreshProvider::Claude; + fresh.session_type = "freshclaude".into(); + fresh.runtime_variant = "claude-agent-sdk".into(); + assert!(matches!( + registry.prepare_launch(other).await, + Err(RegistryError::RequestConflict) + )); + let prepared = registry.prepare_launch(launch).await.unwrap(); + let bound = registry + .bound_input_reservation(prepared.incarnation_id) + .await + .unwrap() + .unwrap(); + assert_eq!(bound.reservation.provider, "kilroy"); + } + #[tokio::test] async fn provider_ordinary_initial_reservation_preserves_provider_identity() { for provider in ["codex", "claude", "amplifier"] { diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index a1cf022f8..5a44106e5 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -151,7 +151,7 @@ impl Registry { ) -> Result { if !matches!( request.provider.as_str(), - "opencode" | "codex" | "claude" | "amplifier" + "opencode" | "codex" | "claude" | "kilroy" | "amplifier" ) || request.creation_seed_ref.is_empty() || request.creation_seed_ref.len() > 4096 || request.creation_seed_ref.chars().any(char::is_control) @@ -225,7 +225,10 @@ pub(super) fn bind_in_transaction( // This does not grant native-auth lookup; OpenCode's initial auth authority // still requires an identity-free creation, and host resume checks remain. if input.native_session_id.is_some() - && !matches!(input.provider.as_str(), "codex" | "claude" | "amplifier") + && !matches!( + input.provider.as_str(), + "codex" | "claude" | "kilroy" | "amplifier" + ) { return Err(RegistryError::RequestConflict); } diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 441644aa2..c9aff51c4 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -15,6 +15,10 @@ const image=configuration.provider==='claude' if (configuration.imageRef!==image || !['opencode','codex','claude','amplifier'].includes(configuration.provider) || !Array.isArray(configuration.initialVolumes) || configuration.initialVolumes.length!==(configuration.provider==='amplifier'?1:2)) { throw new Error('closed provider startup image or finite source set changed') } +if (!['freshclaude','kilroy'].includes(configuration.claudeFlavor) + || (configuration.claudeFlavor==='kilroy' && configuration.provider!=='claude')) { + throw new Error('closed provider startup Claude flavor changed') +} const gateways=new Set() const broker=new RestrictedDockerBroker({ realSocketPath:'/run/fixture-docker.sock',proxySocketPath:path.join(root,'broker.sock'), diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 2163d6990..58f63327f 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -33,7 +33,10 @@ def main(): parser.add_argument("--broker-binary",type=Path,required=True) parser.add_argument("--receipt-dir",type=Path,required=True) parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") + parser.add_argument("--claude-flavor",choices=["freshclaude","kilroy"],default="freshclaude") args=parser.parse_args() + if args.provider!="claude" and args.claude_flavor!="freshclaude": + parser.error("the finite Claude flavor requires --provider claude") selected_test={"opencode":TEST,"codex":CODEX_TEST,"claude":CLAUDE_TEST,"amplifier":AMPLIFIER_TEST}[args.provider] if args.provider=="claude": IMAGE=CLAUDE_IMAGE repo=Path(__file__).resolve().parents[2] @@ -89,12 +92,13 @@ def create(name,*arguments): (root/name).mkdir(parents=True,exist_ok=True) initial=[] souls=["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32]] - if args.provider!="amplifier": souls.append("soul-fresh-"+args.provider+"-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]) + fresh_provider="kilroy" if args.provider=="claude" and args.claude_flavor=="kilroy" else args.provider + if args.provider!="amplifier": souls.append("soul-fresh-"+fresh_provider+"-"+hashlib.sha256(b"closed-native-fresh-create").hexdigest()[:32]) for soul in souls: volume="freshell-provider-"+hashlib.sha256((installation+"\0"+soul).encode()).hexdigest()[:24] volumes.add(volume) initial.append({"installationId":installation,"soulId":soul,"providerVolumeName":volume,"imageRef":IMAGE,"hostBinaryPath":str(host),"binarySha256":digest(host),"daemonId":info["ID"]}) - (root/"broker-configuration.json").write_text(json.dumps({"imageRef":IMAGE,"hostBinaryPath":str(host),"runId":run,"provider":args.provider,"initialVolumes":initial})) + (root/"broker-configuration.json").write_text(json.dumps({"imageRef":IMAGE,"hostBinaryPath":str(host),"runId":run,"provider":args.provider,"claudeFlavor":args.claude_flavor,"initialVolumes":initial})) setup=run+"-dependencies" # Install the actual exact published SDK tree into synthetic config # storage. No read-only install bypass, fake SDK or model call. @@ -165,6 +169,7 @@ def create(name,*arguments): "--env",f"RULE_DIR={config}","--env",f"FRESHELL_PROVIDER_LAUNCH_FIXTURE={root}", "--env",f"FRESHELL_PROVIDER_LAUNCH_HOST_BINARY={host}","--env",f"FRESHELL_PROVIDER_LAUNCH_IMAGE={IMAGE}", "--env",f"FRESHELL_PROVIDER_LAUNCH_INSTALLATION={installation}","--env",f"FRESHELL_PROVIDER_LAUNCH_RUN_ID={run}", + "--env",f"FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR={args.claude_flavor}", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_ENV_FILE={root}/grants/environment.env", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE={root}/grants/auth.json", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE={root}/grants/ca.pem", @@ -179,6 +184,7 @@ def create(name,*arguments): exit_code=int(waited.stdout.strip()) outcome=docker("logs",controller,check=False) receipt["exitCode"]=exit_code + if args.provider=="claude": receipt["claudeFlavor"]=args.claude_flavor receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) From 6c045441165f3ad80b9188587b8f884631b4db53 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 08:32:18 -0700 Subject: [PATCH 084/493] fix: retain selected Claude configuration directory origins --- Cargo.lock | 10 ++ crates/freshell-platform/src/claude_inputs.rs | 13 ++ crates/freshell-server/Cargo.toml | 1 + .../freshell-server/src/fresh_agent_proxy.rs | 2 + .../src/managed_mcp_capability.rs | 125 +++++++++++++++++- .../managed_mcp_capability/claude_stage.rs | 30 ++++- .../src/managed_provider_launch_tests.rs | 24 +++- .../src/managed_provider_references/claude.rs | 83 ++++++++++++ crates/freshell-server/src/managed_runtime.rs | 8 ++ .../src/provider_input_preparation.rs | 1 + .../testing/managed-provider-launch-broker.ts | 2 +- 11 files changed, 284 insertions(+), 15 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 787a804c6..56adc81b0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1482,6 +1482,7 @@ dependencies = [ "tower", "tracing", "tracing-subscriber", + "unicode-normalization", "uuid", ] @@ -5686,6 +5687,15 @@ version = "1.0.24" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" +[[package]] +name = "unicode-normalization" +version = "0.1.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5fd4f6878c9cb28d874b009da9e8d183b5abc80117c40bbd187a1fde336be6e8" +dependencies = [ + "tinyvec", +] + [[package]] name = "unicode-segmentation" version = "1.13.3" diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index 4c18671e7..a3a587966 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -9,6 +9,8 @@ pub struct ClaudeInputManifest { pub original_home: PathBuf, pub cwd: PathBuf, pub workspace: PathBuf, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub config_root: Option, pub plugin_dirs: Vec, } impl ClaudeInputManifest { @@ -20,6 +22,7 @@ impl ClaudeInputManifest { } for path in [&self.original_home, &self.cwd, &self.workspace] .into_iter() + .chain(self.config_root.iter()) .chain(&self.plugin_dirs) { if !path.is_absolute() @@ -32,6 +35,16 @@ impl ClaudeInputManifest { Ok(()) } pub fn runtime_path(&self, logical: &Path) -> PathBuf { + if let Some(root) = &self.config_root { + if let Ok(relative) = logical.strip_prefix(root) { + return Path::new("/home/freshell/provider/.claude").join(relative); + } + } crate::codex_inputs::runtime_path(&self.original_home, logical) } + pub fn config_root(&self) -> PathBuf { + self.config_root + .clone() + .unwrap_or_else(|| self.original_home.join(".claude")) + } } diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index 23b6477eb..cdfdd2d60 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -135,6 +135,7 @@ tower = { version = "0.5", features = ["util"] } # `tracing-subscriber`'s own `env-filter` via `matchers`); declared directly # here because `logging.rs` uses it explicitly, not just transitively. regex = "1" +unicode-normalization = "=0.1.25" [target.'cfg(unix)'.dependencies] # `flock(2)` for `settings_store.rs`'s advisory cross-process config lock diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 34d34ec99..b6d219589 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -521,6 +521,7 @@ impl HostedFreshAgentProxy { let home = std::env::var_os("HOME") .map(std::path::PathBuf::from) .ok_or(())?; + let original = |name: &str| std::env::var(name).ok(); let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { reservation_provider: provider.as_str(), @@ -530,6 +531,7 @@ impl HostedFreshAgentProxy { home: &home, cwd: &cwd, workspace: &workspace, + original: &original, args: &[], plugins: message.plugins.as_deref().unwrap_or_default(), references: &provider_secret_references, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 482142d73..31d7e37e7 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1221,6 +1221,13 @@ fn stage_provider_root( .ok_or("managed config root parent unavailable")? .to_owned(); (root, boundary) + } else if let ("claude", Some(manifest)) = (provider, opencode_input.claude_manifest) { + manifest.validate().map_err(str::to_string)?; + let root = manifest.config_root(); + // This is the accepted explicit source root; native permits a + // symlinked config directory outside the original HOME. + let boundary = root.clone(); + (root, boundary) } else { (home.join(provider_dir), home.to_path_buf()) }; @@ -1247,6 +1254,13 @@ fn stage_provider_root( destination_relative: ".freshell/resources".into(), }]; let mut excluded = provider_credential_sources(&source); + if provider == "claude" { + for relative in ["projects", "history.jsonl", "debug"] { + if let Ok(canonical) = std::fs::canonicalize(source.join(relative)) { + excluded.push(canonical); + } + } + } for path in [ ".claude/.credentials.json", ".codex/auth.json", @@ -3256,6 +3270,98 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[cfg(unix)] + #[test] + fn managed_provider_claude_selected_config_root_consumes_resources_and_excludes_default_decoys() + { + use freshell_platform::claude_inputs::ClaudeInputManifest; + use freshell_runtime_protocol::{ + PreparedProviderInputs, ProviderLaunchContext, ProviderPreparation, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let selected = owned.path().join("selected-config"); + let capabilities = owned.path().join("capabilities"); + for root in [ + &home.join(".claude"), + &cwd, + &selected.join("commands"), + &capabilities, + ] { + std::fs::create_dir_all(root).unwrap(); + } + std::fs::write(home.join(".claude/settings.json"), "default-decoy").unwrap(); + std::fs::write(home.join(".claude/CLAUDE.md"), "@../missing-decoy.md").unwrap(); + std::fs::write( + selected.join("settings.json"), + r#"{"env":{"ROOT_NONCE":"selected"}}"#, + ) + .unwrap(); + std::fs::write(selected.join("CLAUDE.md"), "selected-memory").unwrap(); + std::fs::write(selected.join("commands/resource"), "selected-resource").unwrap(); + // Use an ESM consumer, independent of the original cwd and HOME. + std::fs::write(selected.join("commands/probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: Some(selected.clone()), + plugin_dirs: vec![], + }; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let stage = stage_provider_root( + &capabilities, + "grant-claude-selected-root", + "claude", + &home, + &[], + OpencodeEphemeralInput { + launch_context: Some(&context), + claude_manifest: Some(&manifest), + cwd: Some(&cwd), + workspace: Some(&cwd), + ..Default::default() + }, + None, + ) + .expect("unselected default HOME memory must not be traversed"); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let commands = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == "/home/freshell/provider/.claude/commands") + .expect("the selected user commands must enter native writable config HOME"); + let output = std::process::Command::new("node") + .current_dir(&cwd) + .arg(stage.join(&commands.stage_relative_path).join("probe.mjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"selected-resource"); + assert!(inputs.config_binds.iter().any( + |bind| bind.logical_destination == "/home/freshell/provider/.claude/settings.json" + )); + assert!(!inputs + .config_binds + .iter() + .any(|bind| bind.logical_destination + == home.join(".claude/settings.json").to_str().unwrap())); + assert_eq!( + std::fs::read_to_string(home.join(".claude/settings.json")).unwrap(), + "default-decoy" + ); + } + #[cfg(unix)] #[test] fn managed_provider_claude_memory_private_alias_is_rejected_before_publication() { @@ -4473,22 +4579,24 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ }; let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("original-home"); + let selected = owned.path().join("selected-config"); let wrong = owned.path().join("current-home"); let workspace = owned.path().join("workspace"); for path in [ - &home.join(".claude/skills/probe"), + &selected.join("skills/probe"), + &home, &wrong.join(".claude"), &workspace, ] { std::fs::create_dir_all(path).unwrap(); } std::fs::write( - home.join(".claude/settings.json"), + selected.join("settings.json"), r#"{"env":{"CLAUDE_SELECTED":"original"}}"#, ) .unwrap(); std::fs::write( - home.join(".claude/skills/probe/resource"), + selected.join("skills/probe/resource"), b"retained-original-resource", ) .unwrap(); @@ -4507,6 +4615,14 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ config: vec![], prepared_inputs: None, }; + let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { + version: 1, + original_home: home.clone(), + cwd: workspace.clone(), + workspace: workspace.clone(), + config_root: Some(selected.clone()), + plugin_dirs: vec![], + }; let prior = store .issue( soul.clone(), @@ -4514,6 +4630,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ "http://host.docker.internal:4000".into(), Some(("claude", &home, &context.config)), OpencodeEphemeralInput { + claude_manifest: Some(&manifest), cwd: Some(&workspace), workspace: Some(&workspace), launch_context: Some(&context), @@ -4540,7 +4657,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ) .await .expect("replacement must select recorded original Claude sources"); - std::fs::remove_file(home.join(".claude/settings.json")).unwrap(); + std::fs::remove_file(selected.join("settings.json")).unwrap(); let repeated = apply_prepared_callback( &callback, "fixture-control", diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 7ce4fff6f..32ca62c2e 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -15,6 +15,7 @@ pub(super) fn prepare( original_home: home.into(), cwd: cwd.into(), workspace: workspace.into(), + config_root: None, plugin_dirs: vec![], }); manifest.validate().map_err(str::to_string)?; @@ -22,14 +23,31 @@ pub(super) fn prepare( let mut resources = Vec::new(); let mut memory = Vec::new(); let mut bases: Vec = manifest.cwd.ancestors().map(Path::to_path_buf).collect(); - bases.push(manifest.original_home.clone()); bases.sort(); bases.dedup(); - let mut settings_bases = vec![ - manifest.original_home.clone(), - manifest.cwd.clone(), - manifest.workspace.clone(), - ]; + let config_root = manifest.config_root(); + let user_settings = config_root.join("settings.json"); + if optional(&user_settings)? { + configurations.push((user_settings.clone(), canonical(&user_settings)?)); + } + for relative in [ + "CLAUDE.md", + "rules", + "skills", + "commands", + "agents", + "hooks", + "plugins", + ] { + let logical = config_root.join(relative); + if optional(&logical)? { + if relative == "CLAUDE.md" { + memory.push(logical.clone()); + } + resources.push(logical); + } + } + let mut settings_bases = vec![manifest.cwd.clone(), manifest.workspace.clone()]; settings_bases.sort(); settings_bases.dedup(); for base in settings_bases { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 0a47fb9d8..8e9ef03b0 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -760,6 +760,7 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep .unwrap() .starts_with("fln-")); let home = fixture.join("source-home"); + let selected_config = home.join("selected-config"); let workspace = fixture.join("workspace"); let plugin = home.join("selected-plugin"); let control = fixture.join("scenarios/native/control"); @@ -768,6 +769,7 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep &control, &runtime, &home.join(".claude"), + &selected_config, &plugin.join(".claude-plugin"), &plugin.join("hooks"), &plugin.join("scripts/node_modules/probe-dependency"), @@ -792,7 +794,12 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep b"closed-claude-imported-memory\n@./selected-plugin/nested-memory.md\n", ) .unwrap(); - std::fs::write(home.join(".claude/CLAUDE.md"), &memory).unwrap(); + std::fs::write(selected_config.join("CLAUDE.md"), &memory).unwrap(); + std::fs::write( + home.join(".claude/CLAUDE.md"), + "@../missing-unselected-default.md\n", + ) + .unwrap(); for name in [ "relative-memory.md", "home-memory.md", @@ -806,7 +813,12 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep std::fs::write(plugin.join(name), format!("closed-native-{name}")).unwrap(); } let settings = r#"{"env":{"CLAUDE_SELECTED_NONCE":"closed-selected-settings"}}"#; - std::fs::write(home.join(".claude/settings.json"), settings).unwrap(); + std::fs::write(selected_config.join("settings.json"), settings).unwrap(); + std::fs::write( + home.join(".claude/settings.json"), + r#"{"env":{"CLAUDE_SELECTED_NONCE":"unselected-default"}}"#, + ) + .unwrap(); std::fs::write(plugin.join("skills/probe/SKILL.md"),"---\nname: closed-plugin-probe\ndescription: selected complete native plugin\n---\nUse the adjacent resources.\n").unwrap(); std::fs::write(plugin.join("scripts/resource"), b"closed-claude-resource").unwrap(); std::fs::write( @@ -883,6 +895,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .unwrap(); std::fs::set_permissions(&env_file, std::fs::Permissions::from_mode(0o600)).unwrap(); std::env::set_var("FRESHELL_MANAGED_CLAUDE_ONECLI_ENV_FILE", &env_file); + std::env::set_var("CLAUDE_CONFIG_DIR", &selected_config); let registry = Registry::open( fixture.join("registry"), Some( @@ -986,7 +999,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ cols: 120, rows: 30, }, - env: BTreeMap::new(), + env: BTreeMap::from([( + "CLAUDE_CONFIG_DIR".into(), + "../source-home/selected-config".into(), + )]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), mode: "claude".into(), @@ -1201,7 +1217,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ json!({"provider":"claude","route":"fresh","digest":qualified_digest}) ); assert_eq!( - std::fs::read_to_string(home.join(".claude/settings.json")).unwrap(), + std::fs::read_to_string(selected_config.join("settings.json")).unwrap(), settings ); assert_eq!( diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 46251b1a2..c407bb0c6 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -11,6 +11,7 @@ pub(crate) struct ClaudePreparation<'a> { pub home: &'a Path, pub cwd: &'a Path, pub workspace: &'a Path, + pub original: &'a (dyn Fn(&str) -> Option + Sync), pub args: &'a [String], pub plugins: &'a [String], pub references: &'a [ProviderSecretReference], @@ -52,6 +53,11 @@ pub(crate) async fn prepare_managed( original_home: request.home.into(), cwd: request.cwd.into(), workspace: request.workspace.into(), + config_root: Some(selected_config_root( + request.home, + request.cwd, + request.original, + )?), plugin_dirs, }; manifest.validate().map_err(str::to_string)?; @@ -93,3 +99,80 @@ pub(crate) async fn prepare_managed( }, ) } + +fn selected_config_root( + home: &Path, + cwd: &Path, + original: &dyn Fn(&str) -> Option, +) -> Result { + use unicode_normalization::UnicodeNormalization; + let selected = original("CLAUDE_CONFIG_DIR").map(Ok).unwrap_or_else(|| { + home.join(".claude") + .to_str() + .map(str::to_owned) + .ok_or("Claude original configuration directory is not UTF-8") + })?; + if selected.len() > 64 * 1024 || selected.chars().any(char::is_control) { + return Err("Claude selected configuration directory profile is invalid".into()); + } + let mut normalized = String::new(); + for character in selected.nfc() { + if normalized.len() + character.len_utf8() > 64 * 1024 { + return Err("Claude normalized configuration directory profile exceeded".into()); + } + normalized.push(character); + } + let path = Path::new(&normalized); + Ok(freshell_platform::opencode_config::lexical_path( + &if path.is_absolute() { + path.to_path_buf() + } else { + cwd.join(path) + }, + )) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn managed_provider_claude_config_root_keeps_native_cwd_empty_tilde_and_unicode_selection() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let selected = cwd.join("caf\u{e9}"); + for root in [ + &home.join(".claude"), + &cwd, + &selected, + &cwd.join("~/literal"), + ] { + std::fs::create_dir_all(root).unwrap(); + std::fs::write(root.join("resource"), "native-selected-resource").unwrap(); + } + for selector in [ + None, + Some(""), + Some("./caf\u{65}\u{301}"), + Some("~/literal"), + ] { + let original = |_: &str| selector.map(str::to_string); + let root = selected_config_root(&home, &cwd, &original).unwrap(); + assert_eq!( + std::fs::read(root.join("resource")).unwrap(), + b"native-selected-resource" + ); + if selector == Some("") { + assert_eq!(root, cwd); + } + if selector == Some("~/literal") { + assert_eq!(root, cwd.join("~/literal")); + } + } + let invalid = |_: &str| Some("bad\npath".into()); + assert!(selected_config_root(&home, &cwd, &invalid).is_err()); + let oversized = |_: &str| Some("a".repeat(64 * 1024 + 1)); + assert!(selected_config_root(&home, &cwd, &oversized).is_err()); + } +} diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 4414ff488..35a469be2 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -258,6 +258,13 @@ impl ManagedTerminalController for ServerManagedRuntimeController { let home = std::env::var_os("HOME") .map(std::path::PathBuf::from) .ok_or("original Claude HOME is unavailable")?; + let original = |name: &str| { + request + .env + .get(name) + .cloned() + .or_else(|| std::env::var(name).ok()) + }; let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { reservation_provider: "claude", @@ -267,6 +274,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { home: &home, cwd: &cwd, workspace: &workspace, + original: &original, args: &args, plugins: &[], references: &provider_secret_references, diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 5d21e7d6d..831605a1e 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -307,6 +307,7 @@ mod tests { original_home: "/absent-original-home".into(), cwd: "/absent-original-workspace".into(), workspace: "/absent-original-workspace".into(), + config_root: None, plugin_dirs: vec!["/absent-original-plugin".into()], }; let bytes = serde_json::to_vec(&manifest).unwrap(); diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index c9aff51c4..45faec9d2 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -26,7 +26,7 @@ const broker=new RestrictedDockerBroker({ allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), allowedProviderUserRoots:new Set(configuration.provider==='claude' - ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-plugin'),path.join(root,'source-home/imported-memory.md')] + ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-config'),path.join(root,'source-home/selected-plugin'),path.join(root,'source-home/imported-memory.md')] : configuration.provider==='codex' ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] : configuration.provider==='amplifier' From 2ee1f309abaa415bcd9335aad3f072bd960d89e7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 08:48:47 -0700 Subject: [PATCH 085/493] fix: honor Claude filesystem settings source selection --- crates/freshell-platform/src/claude_inputs.rs | 58 ++++++++ crates/freshell-runtime-protocol/src/lib.rs | 48 +++++++ .../src/managed_mcp_capability.rs | 2 + .../managed_mcp_capability/claude_stage.rs | 133 +++++++++++++++--- .../src/managed_provider_references/claude.rs | 8 ++ .../src/provider_input_preparation.rs | 1 + 6 files changed, 228 insertions(+), 22 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index a3a587966..f85c5485e 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -11,12 +11,70 @@ pub struct ClaudeInputManifest { pub workspace: PathBuf, #[serde(default, skip_serializing_if = "Option::is_none")] pub config_root: Option, + #[serde(default = "default_setting_sources")] + pub setting_sources: Vec, pub plugin_dirs: Vec, } + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum ClaudeSettingSource { + User, + Project, + Local, +} + +impl ClaudeSettingSource { + pub fn as_str(self) -> &'static str { + match self { + Self::User => "user", + Self::Project => "project", + Self::Local => "local", + } + } +} + +pub fn parse_setting_sources(value: &str) -> Result, &'static str> { + if value.len() > 1024 * 1024 || value.chars().any(char::is_control) { + return Err("Claude setting-sources argument profile exceeded"); + } + let mut enabled = [false; 3]; + if !value.is_empty() { + for source in value.split(',') { + match source.trim_matches(|c: char| c.is_whitespace() || c == '\u{feff}') { + "user" => enabled[0] = true, + "project" => enabled[1] = true, + "local" => enabled[2] = true, + _ => return Err( + "Claude setting-sources accepts only user, project, local or an empty selector", + ), + } + } + } + Ok(default_setting_sources() + .into_iter() + .enumerate() + .filter_map(|(index, source)| enabled[index].then_some(source)) + .collect()) +} + +pub fn default_setting_sources() -> Vec { + vec![ + ClaudeSettingSource::User, + ClaudeSettingSource::Project, + ClaudeSettingSource::Local, + ] +} impl ClaudeInputManifest { pub fn validate(&self) -> Result<(), &'static str> { if self.version != 1 || self.plugin_dirs.len() > crate::provider_input_profiles::MAX_SOURCE_EVENTS + || self.setting_sources.len() > 3 + || self + .setting_sources + .iter() + .enumerate() + .any(|(index, source)| self.setting_sources[..index].contains(source)) { return Err("accepted Claude discovery context profile exceeded"); } diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index e2e183442..b32ec6566 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2171,6 +2171,7 @@ pub fn validate_managed_terminal_args(mode: &str, args: &[String]) -> Result<(), ("claude", "--session-id" | "--resume") => safe_selector(value), ("claude", "--model" | "--effort" | "--permission-mode") => safe_selector(value), ("claude", "--plugin-dir") => safe_selector(value), + ("claude", "--setting-sources") => freshell_platform::claude_inputs::parse_setting_sources(value).is_ok(), ("codex", "-c") => { matches!( value.as_str(), @@ -2252,6 +2253,26 @@ pub fn durable_managed_terminal_args( let mut index = 0; while index < args.len() { match args[index].as_str() { + flag if flag == "--setting-sources" || flag.starts_with("--setting-sources=") => { + let value = if let Some(value) = flag.strip_prefix("--setting-sources=") { + index += 1; + value + } else { + let value = args.get(index + 1).ok_or_else(invalid)?; + index += 2; + value.as_str() + }; + let selected = freshell_platform::claude_inputs::parse_setting_sources(value) + .map_err(|_| invalid())?; + out.push("--setting-sources".into()); + out.push( + selected + .into_iter() + .map(|source| source.as_str()) + .collect::>() + .join(","), + ); + } "--mcp-config" => { args.get(index + 1).ok_or_else(invalid)?; index += 2; @@ -3427,6 +3448,33 @@ mod tests { } } + #[test] + fn managed_claude_source_selector_survives_native_argument_recipe_and_rejects_unknown_sources() + { + for (input, selected) in [ + ("project,local", "project,local"), + ("", ""), + ("local, user,local", "user,local"), + ] { + let args = vec!["--setting-sources".into(), input.into()]; + let retained = durable_managed_terminal_args("claude", args, None).unwrap(); + assert_eq!(retained, vec!["--setting-sources", selected]); + validate_managed_terminal_args("claude", &retained).unwrap(); + } + let retained = + durable_managed_terminal_args("claude", vec!["--setting-sources=project".into()], None) + .unwrap(); + assert_eq!(retained, vec!["--setting-sources", "project"]); + for input in ["unknown", "project,", "user,secret=value"] { + assert!(durable_managed_terminal_args( + "claude", + vec!["--setting-sources".into(), input.into()], + None + ) + .is_err()); + } + } + #[test] fn claude_provider_native_plugin_argument_survives_durable_recipe() { let args = vec!["--plugin-dir".into(), "/workspace/claude-plugin".into()]; diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 31d7e37e7..7baa5341e 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -3308,6 +3308,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ cwd: cwd.clone(), workspace: cwd.clone(), config_root: Some(selected.clone()), + setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; let context = ProviderLaunchContext { @@ -4621,6 +4622,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ cwd: workspace.clone(), workspace: workspace.clone(), config_root: Some(selected.clone()), + setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; let prior = store diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 32ca62c2e..ef88101a3 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -1,6 +1,6 @@ //! Finite Claude source roots enter the same ordinary resource graph as Codex. use super::ordinary_stage::{prepare_selected, Preparation}; -use freshell_platform::claude_inputs::ClaudeInputManifest; +use freshell_platform::claude_inputs::{ClaudeInputManifest, ClaudeSettingSource}; use std::path::{Path, PathBuf}; pub(super) fn prepare( @@ -16,18 +16,33 @@ pub(super) fn prepare( cwd: cwd.into(), workspace: workspace.into(), config_root: None, + setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }); manifest.validate().map_err(str::to_string)?; let mut configurations = Vec::new(); let mut resources = Vec::new(); let mut memory = Vec::new(); - let mut bases: Vec = manifest.cwd.ancestors().map(Path::to_path_buf).collect(); + let mut bases: Vec = manifest + .cwd + .ancestors() + .filter(|path| path.parent().is_some()) + .map(Path::to_path_buf) + .collect(); bases.sort(); bases.dedup(); let config_root = manifest.config_root(); let user_settings = config_root.join("settings.json"); - if optional(&user_settings)? { + let user_enabled = manifest + .setting_sources + .contains(&ClaudeSettingSource::User); + let project_enabled = manifest + .setting_sources + .contains(&ClaudeSettingSource::Project); + let local_enabled = manifest + .setting_sources + .contains(&ClaudeSettingSource::Local); + if user_enabled && optional(&user_settings)? { configurations.push((user_settings.clone(), canonical(&user_settings)?)); } for relative in [ @@ -39,6 +54,9 @@ pub(super) fn prepare( "hooks", "plugins", ] { + if !user_enabled { + continue; + } let logical = config_root.join(relative); if optional(&logical)? { if relative == "CLAUDE.md" { @@ -47,29 +65,36 @@ pub(super) fn prepare( resources.push(logical); } } - let mut settings_bases = vec![manifest.cwd.clone(), manifest.workspace.clone()]; - settings_bases.sort(); - settings_bases.dedup(); - for base in settings_bases { - for name in ["settings.json", "settings.local.json"] { - let logical = base.join(".claude").join(name); - if optional(&logical)? { - configurations.push((logical.clone(), canonical(&logical)?)); - } + let mut settings = Vec::new(); + if project_enabled { + settings.push(manifest.cwd.join(".claude/settings.json")); + } + if local_enabled { + settings.push(manifest.cwd.join(".claude/settings.local.json")); + settings.push(manifest.workspace.join(".claude/settings.local.json")); + } + settings.sort(); + settings.dedup(); + for logical in settings { + if optional(&logical)? { + configurations.push((logical.clone(), canonical(&logical)?)); } } for base in bases { - for relative in [ - "CLAUDE.md", - "CLAUDE.local.md", - ".claude/CLAUDE.md", - ".claude/rules", - ".claude/skills", - ".claude/commands", - ".claude/agents", - ".claude/hooks", - ".claude/plugins", + for (enabled, relative) in [ + (project_enabled, "CLAUDE.md"), + (local_enabled, "CLAUDE.local.md"), + (project_enabled, ".claude/CLAUDE.md"), + (project_enabled, ".claude/rules"), + (project_enabled, ".claude/skills"), + (project_enabled, ".claude/commands"), + (project_enabled, ".claude/agents"), + (project_enabled, ".claude/hooks"), + (project_enabled, ".claude/plugins"), ] { + if !enabled { + continue; + } let logical = base.join(relative); if optional(&logical)? { if matches!( @@ -120,3 +145,67 @@ fn optional(path: &Path) -> Result { )), } } + +#[cfg(test)] +mod tests { + use super::*; + use freshell_platform::{ + claude_inputs::ClaudeSettingSource, + provider_environment::{copy_environment, CopyLimits}, + }; + + #[test] + fn managed_provider_claude_disabled_sources_do_not_traverse_or_copy_unselected_memory() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let user = home.join(".claude"); + let project = cwd.join(".claude"); + std::fs::create_dir_all(&user).unwrap(); + std::fs::create_dir_all(project.join("commands")).unwrap(); + std::fs::write(user.join("CLAUDE.md"), "@../missing-disabled-user.md").unwrap(); + std::fs::write(user.join("settings.json"), "disabled-user-settings").unwrap(); + std::fs::write(project.join("settings.json"), "selected-project-settings").unwrap(); + std::fs::write( + project.join("commands/resource"), + "selected-project-resource", + ) + .unwrap(); + std::fs::write(project.join("commands/probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); + let mut manifest = ClaudeInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: Some(user.clone()), + setting_sources: vec![ClaudeSettingSource::Project], + plugin_dirs: vec![], + }; + let selected = prepare(&home, &cwd, &cwd, Some(&manifest), &[]) + .expect("disabled user memory must never become a selected import"); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + let commands = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == project.join("commands")) + .unwrap(); + let output = std::process::Command::new("node") + .current_dir(&cwd) + .arg(stage.join(&commands.stage_relative_path).join("probe.mjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"selected-project-resource"); + assert!(!selected + .binds + .iter() + .any(|bind| Path::new(&bind.logical_destination) == user.join("settings.json"))); + std::fs::write(cwd.join("CLAUDE.md"), "@missing-disabled-project.md").unwrap(); + manifest.setting_sources.clear(); + let disabled = prepare(&home, &cwd, &cwd, Some(&manifest), &[]) + .expect("an empty native selector must leave all filesystem sources unselected"); + assert!(disabled.binds.is_empty()); + assert!(disabled.mounts.is_empty()); + } +} diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index c407bb0c6..fdccadc52 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -58,6 +58,14 @@ pub(crate) async fn prepare_managed( request.cwd, request.original, )?), + setting_sources: request + .args + .windows(2) + .rfind(|pair| pair[0] == "--setting-sources") + .map(|pair| freshell_platform::claude_inputs::parse_setting_sources(&pair[1])) + .transpose() + .map_err(str::to_string)? + .unwrap_or_else(freshell_platform::claude_inputs::default_setting_sources), plugin_dirs, }; manifest.validate().map_err(str::to_string)?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 831605a1e..41014c851 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -308,6 +308,7 @@ mod tests { cwd: "/absent-original-workspace".into(), workspace: "/absent-original-workspace".into(), config_root: None, + setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec!["/absent-original-plugin".into()], }; let bytes = serde_json::to_vec(&manifest).unwrap(); From 441e5cc851447122d89de495c3975055e184c4df Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 09:00:44 -0700 Subject: [PATCH 086/493] test: qualify native Claude source selection through managed launch --- .../src/managed_provider_launch_tests.rs | 31 +++++++++++++++++-- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 8e9ef03b0..3ebaf42b4 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -783,6 +783,15 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep .status() .unwrap() .success()); + let disabled_project_memory = workspace.join("CLAUDE.md"); + let disabled_project_settings = workspace.join(".claude/settings.json"); + std::fs::create_dir_all(workspace.join(".claude")).unwrap(); + std::fs::write(&disabled_project_memory, "@missing-disabled-project.md\n").unwrap(); + std::fs::write( + &disabled_project_settings, + r#"{"env":{"CLAUDE_SELECTED_NONCE":"disabled-project-decoy"}}"#, + ) + .unwrap(); let receipt = workspace.join("claude-consumed.json"); let instructions_receipt = workspace.join("claude-import-consumed.json"); let memory = format!( @@ -853,7 +862,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ const input=JSON.parse(raw),name=path.basename(input.file_path||''),event=input.hook_event_name==='InstructionsLoaded'; const names=['imported-memory.md','relative-memory.md','home-memory.md','absolute-memory.md','spaced memory.md','nested-memory.md']; const selected=event&&names.includes(name)?{name,digest:crypto.createHash('sha256').update(fs.readFileSync(input.file_path)).digest('hex')}:null; - fs.appendFileSync(process.argv[2]+'.events',JSON.stringify({event,global:input.file_path===process.env.HOME+'/.claude/CLAUDE.md',ignored:event&&name.startsWith('ignored-'),selected})+'\n'); + fs.appendFileSync(process.argv[2]+'.events',JSON.stringify({event,global:input.file_path===process.env.HOME+'/.claude/CLAUDE.md',project:input.file_path===process.argv[3],ignored:event&&name.startsWith('ignored-'),selected})+'\n'); if(event&&input.file_path===process.env.HOME+'/imported-memory.md')fs.writeFileSync(process.argv[2],JSON.stringify({nativeImport:true,digest:selected.digest})); }); "#; @@ -871,7 +880,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ let key_digest = format!("{:x}", Sha256::digest(b"synthetic-claude-onecli-value")); let hooks = json!({"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":format!("node \"${{CLAUDE_PLUGIN_ROOT}}/scripts/consume.cjs\" {} {}",serde_json::to_string(receipt.to_str().unwrap()).unwrap(),key_digest),"timeout":10}]}]}}); let mut hooks = hooks; - hooks["hooks"]["InstructionsLoaded"] = json!([{"hooks":[{"type":"command","command":format!("node \"${{CLAUDE_PLUGIN_ROOT}}/scripts/instructions.cjs\" {}",serde_json::to_string(instructions_receipt.to_str().unwrap()).unwrap()),"timeout":10}]}]); + hooks["hooks"]["InstructionsLoaded"] = json!([{"hooks":[{"type":"command","command":format!("node \"${{CLAUDE_PLUGIN_ROOT}}/scripts/instructions.cjs\" {} {}",serde_json::to_string(instructions_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(disabled_project_memory.to_str().unwrap()).unwrap()),"timeout":10}]}]); std::fs::write( plugin.join("hooks/hooks.json"), serde_json::to_vec(&hooks).unwrap(), @@ -991,6 +1000,8 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ program: "claude".into(), args: vec![ "--init-only".into(), + "--setting-sources".into(), + "user".into(), "--plugin-dir".into(), plugin.to_str().unwrap().into(), ], @@ -1086,7 +1097,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ ); } let qualified_digest = hash( - serde_json::to_vec(&json!({"startup":expected,"nativeImports":expected_imports})) + serde_json::to_vec(&json!({"startup":expected,"nativeImports":expected_imports,"terminalSettingSources":"user","disabledProjectConsumed":false,"freshSettingSources":"user,project,local"})) .unwrap() .as_slice(), ); @@ -1168,6 +1179,20 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ assert!(instruction_events(&instructions_events) .iter() .all(|event| event["ignored"] != json!(true))); + assert!( + instruction_events(&instructions_events) + .iter() + .all(|event| event["project"] != json!(true)), + "the real native terminal must leave project memory unselected" + ); + assert_eq!( + std::fs::read_to_string(&disabled_project_memory).unwrap(), + "@missing-disabled-project.md\n" + ); + // Fresh SDK exposes the existing all-source default. Remove only these + // terminal-specific owned decoys before its independent launch. + std::fs::remove_file(&disabled_project_memory).unwrap(); + std::fs::remove_file(&disabled_project_settings).unwrap(); std::fs::remove_file(&receipt).unwrap(); std::fs::remove_file(&instructions_receipt).unwrap(); std::fs::remove_file(&instructions_events).unwrap(); From 5db08e376d43ccc5fd5e27e51f85eb4792e7f1cd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 10:40:54 -0700 Subject: [PATCH 087/493] fix: retain selected Claude settings through managed launch --- crates/freshell-platform/src/claude_inputs.rs | 24 +++++ crates/freshell-runtime-protocol/src/lib.rs | 27 ++++-- .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 96 +++++++++++++++++++ .../managed_mcp_capability/claude_stage.rs | 82 ++++++++++++++++ .../src/managed_provider_launch_tests.rs | 22 ++++- .../src/managed_provider_references/claude.rs | 27 ++++++ crates/freshell-server/src/managed_runtime.rs | 11 ++- crates/freshell-session-host/src/main.rs | 2 + .../src/provider_input_preparation.rs | 17 ++++ .../src/provider_secret_resolution.rs | 1 + .../src/providers/mod.rs | 10 +- crates/freshell-session-host/src/pty.rs | 8 +- 13 files changed, 317 insertions(+), 11 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index f85c5485e..f7f7ca6c5 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -2,6 +2,27 @@ use serde::{Deserialize, Serialize}; use std::path::{Path, PathBuf}; +/// Durable argv names this owned selection; its source and declaring origin +/// are recorded in the private accepted manifest, never as raw settings JSON. +pub const MANAGED_SETTINGS_ARGUMENT: &str = ".freshell/claude-selected-settings"; + +pub fn selected_settings_argument(args: &[String]) -> Option<&str> { + let mut selected = None; + let mut index = 0; + while index < args.len() { + if args[index] == "--settings" { + selected = args.get(index + 1).map(String::as_str); + index += 2; + } else { + if let Some(value) = args[index].strip_prefix("--settings=") { + selected = Some(value); + } + index += 1; + } + } + selected +} + #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct ClaudeInputManifest { @@ -11,6 +32,8 @@ pub struct ClaudeInputManifest { pub workspace: PathBuf, #[serde(default, skip_serializing_if = "Option::is_none")] pub config_root: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub settings_file: Option, #[serde(default = "default_setting_sources")] pub setting_sources: Vec, pub plugin_dirs: Vec, @@ -81,6 +104,7 @@ impl ClaudeInputManifest { for path in [&self.original_home, &self.cwd, &self.workspace] .into_iter() .chain(self.config_root.iter()) + .chain(self.settings_file.iter()) .chain(&self.plugin_dirs) { if !path.is_absolute() diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index b32ec6566..d46679dc4 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2163,7 +2163,7 @@ pub fn validate_managed_terminal_args(mode: &str, args: &[String]) -> Result<(), } let value = args.get(index + 1).ok_or_else(invalid)?; let allowed = match (mode, flag) { - ("claude", "--settings") => freshell_platform::cli_launch::is_owned_claude_settings(value), + ("claude", "--settings") => value == freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT || freshell_platform::cli_launch::is_owned_claude_settings(value), ("claude", "--mcp-config") => matches!( value.as_str(), ".claude/freshell-mcp.json" | "/home/freshell/provider/.claude/freshell-mcp.json" @@ -2206,8 +2206,9 @@ pub fn validate_managed_terminal_args(mode: &str, args: &[String]) -> Result<(), } /// Convert the web process's ordinary launch argv into its durable recipe. -/// Claude's web-owned MCP file is omitted and its settings are regenerated -/// from the shipped non-secret recipe before validation. +/// Claude's web-owned MCP file is omitted. Its shipped settings recipe is +/// regenerated; selected user settings use an owned marker resolved only from +/// accepted private inputs at final child argv construction. pub fn durable_managed_terminal_args( mode: &str, args: Vec, @@ -2277,11 +2278,23 @@ pub fn durable_managed_terminal_args( args.get(index + 1).ok_or_else(invalid)?; index += 2; } - "--settings" => { - args.get(index + 1).ok_or_else(invalid)?; + flag if flag == "--settings" || flag.starts_with("--settings=") => { + let value = if let Some(value) = flag.strip_prefix("--settings=") { + index += 1; + value + } else { + let value = args.get(index + 1).ok_or_else(invalid)?; + index += 2; + value.as_str() + }; out.push("--settings".into()); - out.push(claude_settings.ok_or_else(invalid)?.into()); - index += 2; + out.push( + if freshell_platform::cli_launch::is_owned_claude_settings(value) { + claude_settings.ok_or_else(invalid)?.into() + } else { + freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT.into() + }, + ); } _ => { out.push(args[index].clone()); diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index b6d219589..7bf420a8d 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -525,6 +525,7 @@ impl HostedFreshAgentProxy { let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { reservation_provider: provider.as_str(), + selected_settings: None, client: &self.client, soul: &soul, creation_seed: &message.request_id, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 7baa5341e..d2b04bdec 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1543,6 +1543,13 @@ fn stage_provider_root( )?; } if let Some(resources) = &ordinary_resources { + if provider == "claude" { + claude_stage::stage_settings( + &temporary, + resources, + opencode_input.claude_manifest, + )?; + } let capability = McpCapabilityReference { grant_id: grant_id.into(), endpoint: "http://host.docker.internal:1".into(), @@ -3308,6 +3315,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ cwd: cwd.clone(), workspace: cwd.clone(), config_root: Some(selected.clone()), + settings_file: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; @@ -3363,6 +3371,93 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[cfg(unix)] + #[test] + fn managed_provider_claude_explicit_settings_consumer_retains_user_hooks_and_resources() { + use freshell_platform::claude_inputs::ClaudeInputManifest; + use freshell_runtime_protocol::{ + PreparedProviderInputs, ProviderLaunchContext, ProviderPreparation, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let commands = cwd.join(".claude/commands"); + let declaring = cwd.join("selected-origin"); + let capabilities = owned.path().join("capabilities"); + for directory in [&home, &commands, &declaring, &capabilities] { + std::fs::create_dir_all(directory).unwrap(); + } + std::fs::write(commands.join("resource"), "transported-resource").unwrap(); + std::fs::write(commands.join("probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); + let selected = declaring.join("settings.json"); + let raw = r#"{"env":{"SELECTED_NONCE":"selected-explicit"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"node probe.mjs"}]}]}}"#; + std::fs::write(&selected, raw).unwrap(); + let alias = cwd.join("selected.json"); + std::os::unix::fs::symlink(&selected, &alias).unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: None, + settings_file: Some(std::fs::canonicalize(&alias).unwrap()), + setting_sources: freshell_platform::claude_inputs::default_setting_sources(), + plugin_dirs: vec![], + }; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let stage = stage_provider_root( + &capabilities, + "grant-claude-explicit-settings", + "claude", + &home, + &[], + OpencodeEphemeralInput { + launch_context: Some(&context), + claude_manifest: Some(&manifest), + cwd: Some(&cwd), + workspace: Some(&cwd), + ..Default::default() + }, + None, + ) + .unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let settings = inputs + .config_binds + .iter() + .find(|bind| Path::new(&bind.logical_destination) == selected) + .expect( + "the explicitly selected settings file must retain its native declaring origin", + ); + let resources = inputs + .resource_mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == commands) + .unwrap(); + let output = std::process::Command::new("node") + .arg("--input-type=module").arg("-e") + .arg("import {readFileSync} from 'node:fs';import {execSync} from 'node:child_process';const s=JSON.parse(readFileSync(process.argv[1]));const r=execSync(s.hooks.SessionStart[0].hooks[0].command,{cwd:process.argv[2]});process.stdout.write([s.env.SELECTED_NONCE,r.toString(),s.hooks.SessionStart.length,s.hooks.Stop.length].join(':'))") + .arg(stage.join(&settings.source_relative_path)) + .arg(stage.join(&resources.stage_relative_path)) + .output().unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert_eq!(output.stdout, b"selected-explicit:transported-resource:2:1"); + assert_eq!(std::fs::read(&selected).unwrap(), raw.as_bytes()); + assert_eq!(std::fs::read_link(&alias).unwrap(), selected); + } + #[cfg(unix)] #[test] fn managed_provider_claude_memory_private_alias_is_rejected_before_publication() { @@ -4622,6 +4717,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ cwd: workspace.clone(), workspace: workspace.clone(), config_root: Some(selected.clone()), + settings_file: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index ef88101a3..894a47faa 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -16,6 +16,7 @@ pub(super) fn prepare( cwd: cwd.into(), workspace: workspace.into(), config_root: None, + settings_file: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }); @@ -23,6 +24,9 @@ pub(super) fn prepare( let mut configurations = Vec::new(); let mut resources = Vec::new(); let mut memory = Vec::new(); + if let Some(selected) = &manifest.settings_file { + configurations.push((selected.clone(), canonical(selected)?)); + } let mut bases: Vec = manifest .cwd .ancestors() @@ -127,6 +131,83 @@ pub(super) fn prepare( |path| manifest.runtime_path(path), ) } + +// Only append Freshell's two shipped hook groups. Native settings merging, +// source precedence and all other configured values remain the provider's job. +pub(super) fn stage_settings( + root: &Path, + preparation: &Preparation, + manifest: Option<&ClaudeInputManifest>, +) -> Result<(), String> { + use std::io::Read; + let Some(manifest) = manifest else { + return Ok(()); + }; + let Some(selected) = &manifest.settings_file else { + return Ok(()); + }; + let destination = manifest.runtime_path(selected); + let bind = preparation + .binds + .iter() + .find(|bind| Path::new(&bind.logical_destination) == destination) + .ok_or("Claude explicit settings source has no accepted config bind")?; + let path = root.join(&bind.source_relative_path); + let result = (|| { + const LIMIT: usize = 2 * 1024 * 1024; + let mut bytes = Vec::new(); + std::fs::File::open(&path) + .map_err(|_| "read selected settings")? + .take(LIMIT as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "read selected settings")?; + if bytes.len() > LIMIT { + return Err("selected settings exceeds native 2 MiB limit"); + } + let mut settings: serde_json::Value = + serde_json::from_slice(&bytes).map_err(|_| "selected settings is not valid JSON")?; + let object = settings + .as_object_mut() + .ok_or("selected settings must be a JSON object")?; + let hooks = object + .entry("hooks") + .or_insert_with(|| serde_json::json!({})) + .as_object_mut() + .ok_or("selected hooks must be an object")?; + let shipped: serde_json::Value = + serde_json::from_str(&freshell_platform::cli_launch::claude_settings_json( + freshell_platform::cli_launch::ProviderTarget::Unix, + )) + .map_err(|_| "shipped settings recipe invalid")?; + for (event, groups) in shipped["hooks"] + .as_object() + .ok_or("shipped hooks invalid")? + { + let selected = hooks + .entry(event) + .or_insert_with(|| serde_json::json!([])) + .as_array_mut() + .ok_or("selected hook groups must be arrays")?; + for group in groups.as_array().ok_or("shipped hook groups invalid")? { + if !selected.contains(group) { + selected.push(group.clone()); + } + } + } + let bytes = freshell_runtime_protocol::bounded_json_bytes(&settings, LIMIT) + .map_err(|_| "combined settings exceeds native 2 MiB limit")?; + // The copier made this private, physical file in a not-yet-published + // attempt. Writing it retains its executable bits and failed attempts + // are removed by the existing stage transaction. + std::fs::write(&path, bytes).map_err(|_| "write selected settings")?; + Ok(()) + })(); + result.map_err(|reason| { + tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="stage_explicit_settings",path=%selected.display(),reason, + "Selected Claude settings could not be prepared"); + format!("Claude --settings {}: {reason}", selected.display()) + }) +} fn canonical(path: &Path) -> Result { std::fs::canonicalize(path).map_err(|error| { format!( @@ -178,6 +259,7 @@ mod tests { cwd: cwd.clone(), workspace: cwd.clone(), config_root: Some(user.clone()), + settings_file: None, setting_sources: vec![ClaudeSettingSource::Project], plugin_dirs: vec![], }; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 3ebaf42b4..afb3d8c9e 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -851,12 +851,22 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ const input=JSON.parse(raw),root=path.resolve(__dirname,'..'); const endpoint=process.env.FRESHELL_URL,token=process.env.FRESHELL_TOKEN; const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); - const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,settings:process.env.CLAUDE_SELECTED_NONCE==='closed-selected-settings',authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], + const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,settings:['closed-selected-settings','closed-explicit-settings'].includes(process.env.CLAUDE_SELECTED_NONCE),authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], memory:hash(fs.readFileSync(path.join(process.env.HOME,'.claude/CLAUDE.md'))),skill:hash(fs.readFileSync(path.join(root,'skills/probe/SKILL.md'))),resource:hash(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:hash(require('probe-dependency'))}; fs.writeFileSync(process.argv[2],JSON.stringify(record)); }); "#; std::fs::write(plugin.join("scripts/consume.cjs"), consumer).unwrap(); + let flag_receipt = workspace.join("claude-flag-receipt.json"); + std::fs::write(plugin.join("scripts/flag.cjs"), "const fs=require('node:fs'),crypto=require('node:crypto');const hash=v=>crypto.createHash('sha256').update(v).digest('hex');let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{const input=JSON.parse(raw);fs.writeFileSync(process.argv[2],JSON.stringify({flag:input.hook_event_name==='SessionStart'&&process.env.CLAUDE_SELECTED_NONCE==='closed-explicit-settings',resource:hash(require('node:child_process').execFileSync(__dirname+'/run')),dependency:hash(require('probe-dependency'))}))})").unwrap(); + let flag_origin = selected_config.join("explicit-origin"); + std::fs::create_dir(&flag_origin).unwrap(); + let flag_settings = flag_origin.join("settings.json"); + let flag_value = json!({"env":{"CLAUDE_SELECTED_NONCE":"closed-explicit-settings"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":format!("node {} {}",serde_json::to_string(plugin.join("scripts/flag.cjs").to_str().unwrap()).unwrap(),serde_json::to_string(flag_receipt.to_str().unwrap()).unwrap()),"timeout":10}]}]}}); + let flag_bytes = serde_json::to_vec(&flag_value).unwrap(); + std::fs::write(&flag_settings, &flag_bytes).unwrap(); + let flag_alias = selected_config.join("selected-settings.json"); + std::os::unix::fs::symlink(&flag_settings, &flag_alias).unwrap(); let instructions_consumer = r#"const fs=require('node:fs'),crypto=require('node:crypto'),path=require('node:path'); let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ const input=JSON.parse(raw),name=path.basename(input.file_path||''),event=input.hook_event_name==='InstructionsLoaded'; @@ -1002,6 +1012,8 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ "--init-only".into(), "--setting-sources".into(), "user".into(), + "--settings".into(), + "../source-home/selected-config/selected-settings.json".into(), "--plugin-dir".into(), plugin.to_str().unwrap().into(), ], @@ -1156,6 +1168,14 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ observed.expect("native Claude selected plugin must execute its startup hook"), expected ); + assert_eq!( + consumed(&flag_receipt) + .await + .expect("native must execute the explicitly selected flag settings hook"), + json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) + ); + assert_eq!(std::fs::read(&flag_settings).unwrap(), flag_bytes); + assert_eq!(std::fs::read_link(&flag_alias).unwrap(), flag_settings); let native_import = consumed(&instructions_receipt).await; if native_import.is_none() { println!( diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index fdccadc52..ed2193967 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -8,6 +8,7 @@ pub(crate) struct ClaudePreparation<'a> { pub soul: &'a SoulId, pub creation_seed: &'a str, pub reservation_provider: &'a str, + pub selected_settings: Option<&'a str>, pub home: &'a Path, pub cwd: &'a Path, pub workspace: &'a Path, @@ -58,6 +59,7 @@ pub(crate) async fn prepare_managed( request.cwd, request.original, )?), + settings_file: selected_settings_file(request.cwd, request.selected_settings)?, setting_sources: request .args .windows(2) @@ -108,6 +110,31 @@ pub(crate) async fn prepare_managed( ) } +fn selected_settings_file( + cwd: &Path, + selected: Option<&str>, +) -> Result, String> { + let Some(value) = + selected.filter(|value| !freshell_platform::cli_launch::is_owned_claude_settings(value)) + else { + return Ok(None); + }; + if value.trim_start().starts_with('{') { + return Err("Claude inline --settings requires private prepared source transport".into()); + } + let path = Path::new(value); + let logical = if path.is_absolute() { + path.to_path_buf() + } else { + cwd.join(path) + }; + std::fs::canonicalize(&logical).map(Some).map_err(|error| { + tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="select_explicit_settings",path=%logical.display(),cause=%error, + "Selected Claude settings source is unavailable"); + format!("Claude --settings {} is unavailable: {error}", logical.display()) + }) +} + fn selected_config_root( home: &Path, cwd: &Path, diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 35a469be2..e1afc2643 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -178,6 +178,12 @@ impl ManagedTerminalController for ServerManagedRuntimeController { let mut provider_secret_references = { crate::managed_provider_bootstrap::named_provider_onecli_references(&request.mode)? }; + let selected_claude_settings = (request.mode == "claude") + .then(|| { + freshell_platform::claude_inputs::selected_settings_argument(&request.spec.args) + .map(str::to_owned) + }) + .flatten(); let args = managed_provider_args(&request.mode, request.spec.args)?; let request_id = stable_request_id(create_key)?; let opencode_input = crate::managed_mcp_capability::OpencodeEphemeralInput { @@ -268,6 +274,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { reservation_provider: "claude", + selected_settings: selected_claude_settings.as_deref(), client: &self.client, soul: &soul_id, creation_seed: create_key, @@ -1207,7 +1214,9 @@ mod tests { fn managed_claude_strips_web_bound_mcp_tempfile_only() { let args = vec![ "--settings".into(), - "{}".into(), + freshell_platform::cli_launch::claude_settings_json( + freshell_platform::cli_launch::ProviderTarget::Unix, + ), "--mcp-config".into(), "/tmp/freshell-mcp/t1.json".into(), "--session-id".into(), diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index c5104da4f..f48034d45 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1038,6 +1038,7 @@ async fn grant_execution( terminal, &resolved_secrets.environment, resolved_secrets.opencode_plan.as_ref(), + resolved_secrets.claude_settings_path.as_deref(), ) .await .map_err(|error| { @@ -1053,6 +1054,7 @@ async fn grant_execution( &terminal, prepared.child_env, prepared.codex, + prepared.claude_settings, ) .await .map_err(|e| { diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 41014c851..4e9c2cfe6 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -70,6 +70,22 @@ pub fn prepare( serde_json::from_slice(&metadata) .map_err(|_| "prepared Claude discovery metadata invalid")?; manifest.validate().map_err(str::to_string)?; + if let Some(selected) = &manifest.settings_file { + let destination = manifest.runtime_path(selected); + if !inputs + .config_binds + .iter() + .any(|bind| Path::new(&bind.logical_destination) == destination) + { + return Err("selected Claude settings has no accepted config bind".into()); + } + secrets.claude_settings_path = Some( + destination + .to_str() + .ok_or("selected Claude settings destination is not UTF-8")? + .into(), + ); + } return Ok(BTreeMap::new()); } if provider == "amplifier" { @@ -308,6 +324,7 @@ mod tests { cwd: "/absent-original-workspace".into(), workspace: "/absent-original-workspace".into(), config_root: None, + settings_file: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec!["/absent-original-plugin".into()], }; diff --git a/crates/freshell-session-host/src/provider_secret_resolution.rs b/crates/freshell-session-host/src/provider_secret_resolution.rs index 0db90b7c3..f2e62edea 100644 --- a/crates/freshell-session-host/src/provider_secret_resolution.rs +++ b/crates/freshell-session-host/src/provider_secret_resolution.rs @@ -6,6 +6,7 @@ use std::path::Path; const MAX_KEYS_BYTES: u64 = 128 * 1024; #[derive(Default)] pub struct ResolvedProviderSecrets { + pub claude_settings_path: Option, pub opencode_plan: Option, pub environment: BTreeMap, pub provider_files: Vec<(&'static str, Vec)>, diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 655e1ab4d..07d5c6d34 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -29,6 +29,7 @@ pub struct PreparedTerminal { pub terminal: freshell_runtime_protocol::TerminalLaunchSpec, pub codex: Option, pub child_env: std::collections::BTreeMap, + pub claude_settings: Option, } pub fn prepare_provider_state_before_bootstrap( @@ -164,6 +165,7 @@ pub async fn prepare_terminal( mut terminal: freshell_runtime_protocol::TerminalLaunchSpec, child_secret_env: &std::collections::BTreeMap, prepared_opencode_plan: Option<&freshell_platform::opencode_config::OpencodeConfigPlan>, + selected_claude_settings: Option<&str>, ) -> Result { let mut child_env = child_secret_env.clone(); prepare_scoped_mcp_environment(terminal.provider_launch_context.as_ref(), &mut child_env) @@ -265,6 +267,8 @@ pub async fn prepare_terminal( child_env.insert("OPENCODE_CONFIG_CONTENT".into(), config); } } + // Keep the persisted/validated launch recipe separate from the final + // child-only settings argument, just as credential environment is separate. apply_terminal_context(&mut terminal)?; if terminal.mode == "codex" { let mut sidecar_env = terminal.env.clone(); @@ -277,12 +281,14 @@ pub async fn prepare_terminal( terminal, codex: Some(prepared), child_env, + claude_settings: None, }); } Ok(PreparedTerminal { terminal, codex: None, child_env, + claude_settings: selected_claude_settings.map(str::to_owned), }) } @@ -728,7 +734,9 @@ mod provider_secret_context_tests { "OPENCODE_CONFIG_CONTENT".into(), "{\"model\":\"sentinel-private-inline\"}".into(), )]); - let prepared = prepare_terminal(terminal, &secrets, None).await.unwrap(); + let prepared = prepare_terminal(terminal, &secrets, None, None) + .await + .unwrap(); let metadata = serde_json::to_string(&prepared.terminal).unwrap(); assert!( !metadata.contains("sentinel-private-inline"), diff --git a/crates/freshell-session-host/src/pty.rs b/crates/freshell-session-host/src/pty.rs index de4047da7..9173ab578 100644 --- a/crates/freshell-session-host/src/pty.rs +++ b/crates/freshell-session-host/src/pty.rs @@ -48,6 +48,7 @@ impl HostedPty { launch: &TerminalLaunchSpec, child_secret_env: BTreeMap, prepared_codex: Option, + claude_settings: Option, ) -> Result { launch.validate().map_err(|e| e.message)?; let launch_floor_ms = epoch_ms().saturating_sub(500); @@ -92,7 +93,11 @@ impl HostedPty { }); let mut provider_args = crate::provider_identity_args(launch.run_as_uid, launch.run_as_gid); provider_args.push(launch.program.clone()); - provider_args.extend(launch.args.clone()); + for argument in &launch.args { + provider_args.push(if launch.mode == "claude" && argument == freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT { + claude_settings.as_ref().ok_or("selected Claude settings origin is unavailable; retain inputs and reprepare before dispatch")?.clone() + } else { argument.clone() }); + } let spec = SpawnSpec { program: "/usr/bin/setpriv".to_string(), args: provider_args, @@ -744,6 +749,7 @@ mod provider_exec_tests { &launch, environment, None, + None, ) .await; // This regression owns and stops only the unexpectedly dispatched From 7500651a81b77cd7d45a4058437ff0852caca1f6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 11:33:41 -0700 Subject: [PATCH 088/493] fix: transport inline Claude settings as accepted private inputs --- crates/freshell-platform/src/claude_inputs.rs | 53 ++++ .../src/managed_mcp_capability.rs | 227 ++++++++++++++++++ .../managed_mcp_capability/claude_stage.rs | 77 +++--- .../src/managed_provider_launch_tests.rs | 25 +- .../src/managed_provider_references/claude.rs | 21 +- crates/freshell-server/src/managed_runtime.rs | 13 +- crates/freshell-session-host/src/main.rs | 2 +- .../src/provider_input_preparation.rs | 119 ++++++++- .../src/provider_secret_resolution.rs | 2 +- crates/freshell-session-host/src/pty.rs | 4 +- .../managed-provider-launch-sandbox.py | 8 +- 11 files changed, 501 insertions(+), 50 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index f7f7ca6c5..4d091e417 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -5,6 +5,52 @@ use std::path::{Path, PathBuf}; /// Durable argv names this owned selection; its source and declaring origin /// are recorded in the private accepted manifest, never as raw settings JSON. pub const MANAGED_SETTINGS_ARGUMENT: &str = ".freshell/claude-selected-settings"; +pub const INLINE_SETTINGS_SOURCE: &str = ".freshell/source-views/claude-inline-settings.json"; +pub const MAX_NATIVE_SETTINGS_BYTES: usize = 2 * 1024 * 1024; + +pub fn is_inline_settings(value: &str) -> bool { + let value = + value.trim_matches(|character: char| character.is_whitespace() || character == '\u{feff}'); + value.starts_with('{') && value.ends_with('}') +} + +/// Append only the shipped hook groups; retain provider source/tier semantics. +/// Errors never carry configured commands, environment values or JSON snippets. +pub fn settings_with_owned_hooks(raw: &[u8]) -> Result { + if raw.len() > MAX_NATIVE_SETTINGS_BYTES { + return Err("selected settings exceeds native 2 MiB limit"); + } + let mut settings: serde_json::Value = + serde_json::from_slice(raw).map_err(|_| "selected settings is not valid JSON")?; + let object = settings + .as_object_mut() + .ok_or("selected settings must be a JSON object")?; + let hooks = object + .entry("hooks") + .or_insert_with(|| serde_json::json!({})) + .as_object_mut() + .ok_or("selected hooks must be an object")?; + let shipped: serde_json::Value = serde_json::from_str( + &crate::cli_launch::claude_settings_json(crate::cli_launch::ProviderTarget::Unix), + ) + .map_err(|_| "shipped settings recipe invalid")?; + for (event, groups) in shipped["hooks"] + .as_object() + .ok_or("shipped hooks invalid")? + { + let selected = hooks + .entry(event) + .or_insert_with(|| serde_json::json!([])) + .as_array_mut() + .ok_or("selected hook groups must be arrays")?; + for group in groups.as_array().ok_or("shipped hook groups invalid")? { + if !selected.contains(group) { + selected.push(group.clone()); + } + } + } + Ok(settings) +} pub fn selected_settings_argument(args: &[String]) -> Option<&str> { let mut selected = None; @@ -34,6 +80,8 @@ pub struct ClaudeInputManifest { pub config_root: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub settings_file: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub inline_settings_sha256: Option, #[serde(default = "default_setting_sources")] pub setting_sources: Vec, pub plugin_dirs: Vec, @@ -91,6 +139,11 @@ pub fn default_setting_sources() -> Vec { impl ClaudeInputManifest { pub fn validate(&self) -> Result<(), &'static str> { if self.version != 1 + || self.inline_settings_sha256.as_ref().is_some_and(|digest| { + self.settings_file.is_some() + || digest.len() != 64 + || !digest.bytes().all(|byte| byte.is_ascii_hexdigit()) + }) || self.plugin_dirs.len() > crate::provider_input_profiles::MAX_SOURCE_EVENTS || self.setting_sources.len() > 3 || self diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index d2b04bdec..99576b479 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -38,6 +38,7 @@ mod source_stage; pub(crate) struct OpencodeEphemeralInput<'a> { pub codex_manifest: Option<&'a freshell_platform::codex_inputs::CodexInputManifest>, pub claude_manifest: Option<&'a freshell_platform::claude_inputs::ClaudeInputManifest>, + pub claude_inline_settings: Option<&'a str>, pub amplifier_manifest: Option<&'a freshell_platform::amplifier_inputs::AmplifierInputManifest>, pub inline_config: Option<&'a str>, pub tui_config_path: Option<&'a str>, @@ -350,6 +351,14 @@ async fn apply_prepared_callback( } _ => return Err("replacement ordinary source provider invalid".into()), }; + let inline_settings = match &origins { + AcceptedOrigins::Claude(manifest) => manifest + .inline_settings_sha256 + .as_ref() + .map(|digest| read_prior_claude_inline_settings(directory, &context, digest)) + .transpose()?, + _ => None, + }; let (original_home, mut input) = match &origins { AcceptedOrigins::Codex(manifest) => ( &manifest.original_home, @@ -366,6 +375,7 @@ async fn apply_prepared_callback( cwd: Some(&manifest.cwd), workspace: Some(&manifest.workspace), claude_manifest: Some(manifest), + claude_inline_settings: inline_settings.as_deref(), ..Default::default() }, ), @@ -638,6 +648,45 @@ fn read_prior_manifest_bytes( Ok(bytes) } +// Called only after read_prior_manifest_bytes has validated this generation's +// exact soul ownership. Read the one finite accepted private source, never an +// arbitrary path or the controller's present HOME/configuration. +fn read_prior_claude_inline_settings( + directory: &Path, + context: &freshell_runtime_protocol::ProviderLaunchContext, + digest: &str, +) -> Result { + use freshell_platform::claude_inputs::{INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES}; + use std::io::Read; + let capability = context + .mcp_capability + .as_ref() + .ok_or("prior Claude settings generation authority unavailable")?; + let path = + staged_provider_root_path(directory, &capability.grant_id)?.join(INLINE_SETTINGS_SOURCE); + if !std::fs::symlink_metadata(&path) + .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) + || std::fs::canonicalize(&path).map_err(|_| { + "prior Claude inline settings unavailable; retain inputs and reprepare before dispatch" + })? != path + { + return Err("prior Claude inline settings unavailable or not a physical private source; retain inputs and reprepare before dispatch".into()); + } + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "prior Claude inline settings unavailable")? + .take(MAX_NATIVE_SETTINGS_BYTES as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|_| "prior Claude inline settings read failed")?; + if bytes.len() > MAX_NATIVE_SETTINGS_BYTES { + return Err("prior Claude inline --settings exceeds native 2 MiB limit; retain inputs and reprepare before dispatch".into()); + } + if format!("{:x}", sha2::Sha256::digest(&bytes)) != digest { + return Err("prior Claude inline settings identity changed; retain inputs and reprepare before dispatch".into()); + } + String::from_utf8(bytes).map_err(|_| "prior Claude inline settings is not UTF-8".into()) +} + fn read_prior_source_manifest( store: &CapabilityStore, directory: &Path, @@ -1548,6 +1597,7 @@ fn stage_provider_root( &temporary, resources, opencode_input.claude_manifest, + opencode_input.claude_inline_settings, )?; } let capability = McpCapabilityReference { @@ -3316,6 +3366,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ workspace: cwd.clone(), config_root: Some(selected.clone()), settings_file: None, + inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; @@ -3401,6 +3452,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ workspace: cwd.clone(), config_root: None, settings_file: Some(std::fs::canonicalize(&alias).unwrap()), + inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; @@ -3458,6 +3510,115 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ assert_eq!(std::fs::read_link(&alias).unwrap(), selected); } + #[test] + fn managed_provider_claude_inline_native_limit_and_failure_cleanup_preserve_prior_stage() { + use freshell_platform::claude_inputs::{ + ClaudeInputManifest, INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES, + }; + use freshell_runtime_protocol::{ProviderLaunchContext, ProviderPreparation}; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let directory = owned.path().join("capabilities"); + for path in [&home, &cwd, &directory] { + std::fs::create_dir_all(path).unwrap(); + } + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + let mut manifest = ClaudeInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: None, + settings_file: None, + inline_settings_sha256: None, + setting_sources: vec![], + plugin_dirs: vec![], + }; + let body = r#"{"env":{"SELECTED_NONCE":"bounded-selected"}}"#; + let maximum = format!( + "{}{}", + " ".repeat(MAX_NATIVE_SETTINGS_BYTES - body.len()), + body + ); + manifest.inline_settings_sha256 = + Some(format!("{:x}", sha2::Sha256::digest(maximum.as_bytes()))); + let prior = stage_provider_root( + &directory, + "grant-inline-bounded", + "claude", + &home, + &[], + OpencodeEphemeralInput { + launch_context: Some(&context), + claude_manifest: Some(&manifest), + claude_inline_settings: Some(&maximum), + cwd: Some(&cwd), + workspace: Some(&cwd), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage_names = || { + std::fs::read_dir(prior.parent().unwrap()) + .unwrap() + .map(|entry| entry.unwrap().file_name()) + .collect::>() + }; + let accepted_stages = stage_names(); + let consume = || { + let output=std::process::Command::new("node").args(["-e","const s=JSON.parse(require('node:fs').readFileSync(process.argv[1]));process.stdout.write(s.env.SELECTED_NONCE)"]).arg(prior.join(INLINE_SETTINGS_SOURCE)).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"bounded-selected"); + }; + consume(); + let oversized = format!(" {maximum}"); + for (id, raw, reason) in [ + ("grant-inline-oversized", oversized.as_str(), "2 MiB"), + ( + "grant-inline-malformed", + "{private-malformed-nonce}", + "not valid JSON", + ), + ] { + manifest.inline_settings_sha256 = + Some(format!("{:x}", sha2::Sha256::digest(raw.as_bytes()))); + let error = stage_provider_root( + &directory, + id, + "claude", + &home, + &[], + OpencodeEphemeralInput { + launch_context: Some(&context), + claude_manifest: Some(&manifest), + claude_inline_settings: Some(raw), + cwd: Some(&cwd), + workspace: Some(&cwd), + ..Default::default() + }, + None, + ) + .err() + .unwrap(); + assert!(error.contains(reason), "{error}"); + assert!(!error.contains("private-malformed-nonce")); + assert!(!staged_provider_root_path(&directory, id).unwrap().exists()); + consume(); + } + assert_eq!( + stage_names(), + accepted_stages, + "failed attempt temporary stages must be removed without removing prior inputs" + ); + } + #[cfg(unix)] #[test] fn managed_provider_claude_memory_private_alias_is_rejected_before_publication() { @@ -4670,6 +4831,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ #[tokio::test] async fn managed_provider_claude_replacement_retains_original_sources_and_pending_replay() { + use freshell_platform::claude_inputs::INLINE_SETTINGS_SOURCE; use freshell_runtime_protocol::{ PreparedProviderContext, ProviderLaunchContext, ProviderPreparation, }; @@ -4696,6 +4858,12 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ b"retained-original-resource", ) .unwrap(); + std::fs::write( + selected.join("skills/probe/consume.cjs"), + "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource'))", + ) + .unwrap(); + let inline_settings = r#"{"env":{"INLINE_NONCE":"retained-inline"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"node consume.cjs"}]}]}}"#; std::fs::write( wrong.join(".claude/settings.json"), r#"{"env":{"CLAUDE_SELECTED":"wrong"}}"#, @@ -4718,6 +4886,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ workspace: workspace.clone(), config_root: Some(selected.clone()), settings_file: None, + inline_settings_sha256: Some(format!( + "{:x}", + sha2::Sha256::digest(inline_settings.as_bytes()) + )), setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }; @@ -4729,6 +4901,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ Some(("claude", &home, &context.config)), OpencodeEphemeralInput { claude_manifest: Some(&manifest), + claude_inline_settings: Some(inline_settings), cwd: Some(&workspace), workspace: Some(&workspace), launch_context: Some(&context), @@ -4756,6 +4929,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .await .expect("replacement must select recorded original Claude sources"); std::fs::remove_file(selected.join("settings.json")).unwrap(); + let prior_source = staged_provider_root_path(&directory, &prior.grant_id) + .unwrap() + .join(INLINE_SETTINGS_SOURCE); + std::fs::remove_file(&prior_source).unwrap(); let repeated = apply_prepared_callback( &callback, "fixture-control", @@ -4802,6 +4979,56 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(), b"retained-original-resource" ); + let output = std::process::Command::new("node").args(["-e", "const fs=require('node:fs'),s=JSON.parse(fs.readFileSync(process.argv[1]));const r=require('node:child_process').execSync(s.hooks.SessionStart[0].hooks[0].command,{cwd:process.argv[2]});process.stdout.write(s.env.INLINE_NONCE+':'+r)"]) + .arg(stage.join(INLINE_SETTINGS_SOURCE)).arg(stage.join(&mount.stage_relative_path).join("probe")).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"retained-inline:retained-original-resource"); + let accepted = ready + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap(); + let incarnation = IncarnationId::new(); + store + .activate( + &accepted.grant_id, + soul.clone(), + incarnation.clone(), + &directory, + &accepted.endpoint, + ) + .unwrap(); + store + .commit_preparation(&accepted.grant_id, &soul, &incarnation, &directory) + .unwrap(); + // A new generation cannot invent an inline source after its accepted + // private file is lost. Preserve the other accepted data and return a + // reason before creating another grant/stage. + std::fs::remove_file(stage.join(INLINE_SETTINGS_SOURCE)).unwrap(); + let unavailable = serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"claude","workspace":workspace,"cwd":workspace,"priorContext":ready.provider_launch_context,"providerSecretReferences":[]})).unwrap(); + let error = apply_prepared_callback( + &unavailable, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &wrong, + &client, + ) + .await + .err() + .unwrap(); + assert!(error.contains("inline settings unavailable"), "{error}"); + assert!(!error.contains("retained-inline")); + assert_eq!( + std::fs::read( + stage + .join(&mount.stage_relative_path) + .join("probe/resource") + ) + .unwrap(), + b"retained-original-resource" + ); } #[tokio::test] diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 894a47faa..1890a82d2 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -17,6 +17,7 @@ pub(super) fn prepare( workspace: workspace.into(), config_root: None, settings_file: None, + inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], }); @@ -138,11 +139,42 @@ pub(super) fn stage_settings( root: &Path, preparation: &Preparation, manifest: Option<&ClaudeInputManifest>, + inline_settings: Option<&str>, ) -> Result<(), String> { + use freshell_platform::claude_inputs::{ + settings_with_owned_hooks, INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES, + }; use std::io::Read; let Some(manifest) = manifest else { return Ok(()); }; + if let Some(digest) = &manifest.inline_settings_sha256 { + let result: Result<(), String> = (|| { + use sha2::{Digest, Sha256}; + let raw = inline_settings.ok_or("accepted Claude inline settings source is unavailable; retain inputs and reprepare before dispatch")?; + if raw.len() > MAX_NATIVE_SETTINGS_BYTES { + return Err("Claude inline --settings exceeds native 2 MiB limit".into()); + } + if format!("{:x}", Sha256::digest(raw.as_bytes())) != *digest { + return Err("accepted Claude inline settings source identity changed".into()); + } + let merged = settings_with_owned_hooks(raw.as_bytes()).map_err(str::to_string)?; + freshell_runtime_protocol::bounded_json_bytes(&merged, MAX_NATIVE_SETTINGS_BYTES) + .map_err(|_| "combined Claude inline settings exceeds native 2 MiB limit")?; + super::private_directory(&root.join(".freshell/source-views"))?; + super::write_private_bytes(&root.join(INLINE_SETTINGS_SOURCE), raw.as_bytes()) + })(); + return result.inspect_err(|reason| { + tracing::warn!( + event = "provider.inputs.preparation.blocked", + provider = "claude", + stage = "controller", + operation = "stage_inline_settings", + reason, + "Selected Claude inline settings could not be prepared" + ); + }); + } let Some(selected) = &manifest.settings_file else { return Ok(()); }; @@ -153,49 +185,17 @@ pub(super) fn stage_settings( .find(|bind| Path::new(&bind.logical_destination) == destination) .ok_or("Claude explicit settings source has no accepted config bind")?; let path = root.join(&bind.source_relative_path); - let result = (|| { - const LIMIT: usize = 2 * 1024 * 1024; + let result: Result<(), &'static str> = (|| { let mut bytes = Vec::new(); std::fs::File::open(&path) .map_err(|_| "read selected settings")? - .take(LIMIT as u64 + 1) + .take(MAX_NATIVE_SETTINGS_BYTES as u64 + 1) .read_to_end(&mut bytes) .map_err(|_| "read selected settings")?; - if bytes.len() > LIMIT { - return Err("selected settings exceeds native 2 MiB limit"); - } - let mut settings: serde_json::Value = - serde_json::from_slice(&bytes).map_err(|_| "selected settings is not valid JSON")?; - let object = settings - .as_object_mut() - .ok_or("selected settings must be a JSON object")?; - let hooks = object - .entry("hooks") - .or_insert_with(|| serde_json::json!({})) - .as_object_mut() - .ok_or("selected hooks must be an object")?; - let shipped: serde_json::Value = - serde_json::from_str(&freshell_platform::cli_launch::claude_settings_json( - freshell_platform::cli_launch::ProviderTarget::Unix, - )) - .map_err(|_| "shipped settings recipe invalid")?; - for (event, groups) in shipped["hooks"] - .as_object() - .ok_or("shipped hooks invalid")? - { - let selected = hooks - .entry(event) - .or_insert_with(|| serde_json::json!([])) - .as_array_mut() - .ok_or("selected hook groups must be arrays")?; - for group in groups.as_array().ok_or("shipped hook groups invalid")? { - if !selected.contains(group) { - selected.push(group.clone()); - } - } - } - let bytes = freshell_runtime_protocol::bounded_json_bytes(&settings, LIMIT) - .map_err(|_| "combined settings exceeds native 2 MiB limit")?; + let settings = settings_with_owned_hooks(&bytes)?; + let bytes = + freshell_runtime_protocol::bounded_json_bytes(&settings, MAX_NATIVE_SETTINGS_BYTES) + .map_err(|_| "combined settings exceeds native 2 MiB limit")?; // The copier made this private, physical file in a not-yet-published // attempt. Writing it retains its executable bits and failed attempts // are removed by the existing stage transaction. @@ -260,6 +260,7 @@ mod tests { workspace: cwd.clone(), config_root: Some(user.clone()), settings_file: None, + inline_settings_sha256: None, setting_sources: vec![ClaudeSettingSource::Project], plugin_dirs: vec![], }; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index afb3d8c9e..413e2a0b5 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -859,14 +859,35 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ std::fs::write(plugin.join("scripts/consume.cjs"), consumer).unwrap(); let flag_receipt = workspace.join("claude-flag-receipt.json"); std::fs::write(plugin.join("scripts/flag.cjs"), "const fs=require('node:fs'),crypto=require('node:crypto');const hash=v=>crypto.createHash('sha256').update(v).digest('hex');let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{const input=JSON.parse(raw);fs.writeFileSync(process.argv[2],JSON.stringify({flag:input.hook_event_name==='SessionStart'&&process.env.CLAUDE_SELECTED_NONCE==='closed-explicit-settings',resource:hash(require('node:child_process').execFileSync(__dirname+'/run')),dependency:hash(require('probe-dependency'))}))})").unwrap(); + let flag_commands = selected_config.join("commands"); + std::fs::create_dir_all(flag_commands.join("node_modules/probe-dependency")).unwrap(); + for relative in [ + "flag.cjs", + "run", + "resource", + "node_modules/probe-dependency/index.js", + ] { + std::fs::copy( + plugin.join("scripts").join(relative), + flag_commands.join(relative), + ) + .unwrap(); + } let flag_origin = selected_config.join("explicit-origin"); std::fs::create_dir(&flag_origin).unwrap(); let flag_settings = flag_origin.join("settings.json"); - let flag_value = json!({"env":{"CLAUDE_SELECTED_NONCE":"closed-explicit-settings"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":format!("node {} {}",serde_json::to_string(plugin.join("scripts/flag.cjs").to_str().unwrap()).unwrap(),serde_json::to_string(flag_receipt.to_str().unwrap()).unwrap()),"timeout":10}]}]}}); + let flag_value = json!({"env":{"CLAUDE_SELECTED_NONCE":"closed-explicit-settings"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":format!("node \"../source-home/selected-config/commands/flag.cjs\" {}",serde_json::to_string(flag_receipt.to_str().unwrap()).unwrap()),"timeout":10}]}]}}); let flag_bytes = serde_json::to_vec(&flag_value).unwrap(); std::fs::write(&flag_settings, &flag_bytes).unwrap(); let flag_alias = selected_config.join("selected-settings.json"); std::os::unix::fs::symlink(&flag_settings, &flag_alias).unwrap(); + let settings_form = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SETTINGS_FORM") + .unwrap_or_else(|_| "file".into()); + let flag_argument = match settings_form.as_str() { + "file" => "../source-home/selected-config/selected-settings.json".into(), + "inline" => String::from_utf8(flag_bytes.clone()).unwrap(), + _ => panic!("closed fixture settings form must be file or inline"), + }; let instructions_consumer = r#"const fs=require('node:fs'),crypto=require('node:crypto'),path=require('node:path'); let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ const input=JSON.parse(raw),name=path.basename(input.file_path||''),event=input.hook_event_name==='InstructionsLoaded'; @@ -1013,7 +1034,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ "--setting-sources".into(), "user".into(), "--settings".into(), - "../source-home/selected-config/selected-settings.json".into(), + flag_argument, "--plugin-dir".into(), plugin.to_str().unwrap().into(), ], diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index ed2193967..a558ccc81 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -24,6 +24,15 @@ pub(crate) async fn prepare_managed( if !matches!(request.reservation_provider, "claude" | "kilroy") { return Err("Claude source preparation requires its exact supported runtime flavor".into()); } + let inline_settings = request.selected_settings.filter(|value| { + freshell_platform::claude_inputs::is_inline_settings(value) + && !freshell_platform::cli_launch::is_owned_claude_settings(value) + }); + if inline_settings.is_some_and(|value| { + value.len() > freshell_platform::claude_inputs::MAX_NATIVE_SETTINGS_BYTES + }) { + return Err("Claude inline --settings exceeds native 2 MiB limit".into()); + } let mut plugin_dirs = Vec::new(); let mut path_bytes = 0usize; let cli = request @@ -60,6 +69,10 @@ pub(crate) async fn prepare_managed( request.original, )?), settings_file: selected_settings_file(request.cwd, request.selected_settings)?, + inline_settings_sha256: inline_settings.map(|value| { + use sha2::{Digest, Sha256}; + format!("{:x}", Sha256::digest(value.as_bytes())) + }), setting_sources: request .args .windows(2) @@ -101,6 +114,7 @@ pub(crate) async fn prepare_managed( &context, crate::managed_mcp_capability::OpencodeEphemeralInput { claude_manifest: Some(&manifest), + claude_inline_settings: inline_settings, cwd: Some(request.cwd), workspace: Some(request.workspace), secret_references: Some(request.references), @@ -119,8 +133,11 @@ fn selected_settings_file( else { return Ok(None); }; - if value.trim_start().starts_with('{') { - return Err("Claude inline --settings requires private prepared source transport".into()); + if freshell_platform::claude_inputs::is_inline_settings(value) { + return Ok(None); + } + if value.len() > 64 * 1024 || value.chars().any(char::is_control) { + return Err("Claude selected settings path profile is invalid".into()); } let path = Path::new(value); let logical = if path.is_absolute() { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index e1afc2643..72f06f038 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -181,9 +181,18 @@ impl ManagedTerminalController for ServerManagedRuntimeController { let selected_claude_settings = (request.mode == "claude") .then(|| { freshell_platform::claude_inputs::selected_settings_argument(&request.spec.args) - .map(str::to_owned) + .map(|value| { + if value.len() + > freshell_platform::claude_inputs::MAX_NATIVE_SETTINGS_BYTES + { + Err("Claude --settings exceeds native 2 MiB limit") + } else { + Ok(value.to_owned()) + } + }) }) - .flatten(); + .flatten() + .transpose()?; let args = managed_provider_args(&request.mode, request.spec.args)?; let request_id = stable_request_id(create_key)?; let opencode_input = crate::managed_mcp_capability::OpencodeEphemeralInput { diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index f48034d45..f00909d23 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1038,7 +1038,7 @@ async fn grant_execution( terminal, &resolved_secrets.environment, resolved_secrets.opencode_plan.as_ref(), - resolved_secrets.claude_settings_path.as_deref(), + resolved_secrets.claude_settings_argument.as_deref(), ) .await .map_err(|error| { diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 4e9c2cfe6..3b30b438f 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -70,6 +70,25 @@ pub fn prepare( serde_json::from_slice(&metadata) .map_err(|_| "prepared Claude discovery metadata invalid")?; manifest.validate().map_err(str::to_string)?; + if let Some(digest) = &manifest.inline_settings_sha256 { + use freshell_platform::claude_inputs::{ + settings_with_owned_hooks, INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES, + }; + let raw = read_private(stage, INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES).map_err(|reason| format!("Claude inline --settings private source (native 2 MiB limit): {reason}"))?; + if format!("{:x}", Sha256::digest(&raw)) != *digest { + return Err("selected Claude inline settings source changed; retain inputs and reprepare before dispatch".into()); + } + let settings = settings_with_owned_hooks(&raw).map_err(str::to_string)?; + let bytes = freshell_runtime_protocol::bounded_json_bytes( + &settings, + MAX_NATIVE_SETTINGS_BYTES, + ) + .map_err(|_| "combined Claude inline settings exceeds native 2 MiB limit")?; + secrets.claude_settings_argument = Some( + String::from_utf8(bytes) + .map_err(|_| "prepared Claude inline settings is not UTF-8")?, + ); + } if let Some(selected) = &manifest.settings_file { let destination = manifest.runtime_path(selected); if !inputs @@ -79,7 +98,7 @@ pub fn prepare( { return Err("selected Claude settings has no accepted config bind".into()); } - secrets.claude_settings_path = Some( + secrets.claude_settings_argument = Some( destination .to_str() .ok_or("selected Claude settings destination is not UTF-8")? @@ -314,6 +333,103 @@ mod tests { use freshell_platform::opencode_recipe::*; use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + #[test] + fn provider_secret_claude_inline_settings_consumes_accepted_private_source() { + use freshell_platform::claude_inputs::{ClaudeInputManifest, INLINE_SETTINGS_SOURCE}; + let owned = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(owned.path().join(".freshell/source-views")).unwrap(); + let raw = br#"{"env":{"FLAG_NONCE":"inline-selected"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"echo configured"}]}]}}"#; + std::fs::write(owned.path().join(INLINE_SETTINGS_SOURCE), raw).unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + original_home: "/absent-home".into(), + cwd: "/absent-cwd".into(), + workspace: "/absent-cwd".into(), + config_root: None, + settings_file: None, + inline_settings_sha256: Some(format!("{:x}", Sha256::digest(raw))), + setting_sources: vec![], + plugin_dirs: vec![], + }; + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + config: vec![], + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }), + mcp_capability: Some(capability), + }; + let mut secrets = ResolvedProviderSecrets::default(); + prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + owned.path(), + owned.path(), + ) + .unwrap(); + let argument = secrets.claude_settings_argument.as_deref().expect("accepted inline settings must reach the native child argument without a guessed file origin"); + let output = std::process::Command::new("node").args(["-e", "const s=JSON.parse(process.argv[1]);process.stdout.write([s.env.FLAG_NONCE,s.hooks.SessionStart[0].hooks[0].command,s.hooks.SessionStart.length,s.hooks.Stop.length].join(':'))", argument]).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"inline-selected:echo configured:2:1"); + assert!(!serde_json::to_string(&context) + .unwrap() + .contains("inline-selected")); + assert!(secrets.environment.is_empty()); + std::fs::write( + owned.path().join(INLINE_SETTINGS_SOURCE), + b"{\"env\":{\"FLAG_NONCE\":\"changed-private-source\"}}", + ) + .unwrap(); + let mut changed = ResolvedProviderSecrets::default(); + let error = prepare( + Some(&context), + &mut changed, + &ProviderPathInputs::default(), + owned.path(), + owned.path(), + ) + .err() + .unwrap(); + assert!(error.contains("source changed")); + assert!(!error.contains("changed-private-source")); + assert!(changed.claude_settings_argument.is_none()); + std::fs::write( + owned.path().join(INLINE_SETTINGS_SOURCE), + vec![b' '; freshell_platform::claude_inputs::MAX_NATIVE_SETTINGS_BYTES + 1], + ) + .unwrap(); + let mut oversized = ResolvedProviderSecrets::default(); + assert!(prepare( + Some(&context), + &mut oversized, + &ProviderPathInputs::default(), + owned.path(), + owned.path() + ) + .is_err()); + assert!(oversized.claude_settings_argument.is_none()); + } + #[test] fn provider_secret_claude_prepared_manifest_validates_without_original_controller_tree() { let owned = tempfile::tempdir().unwrap(); @@ -325,6 +441,7 @@ mod tests { workspace: "/absent-original-workspace".into(), config_root: None, settings_file: None, + inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec!["/absent-original-plugin".into()], }; diff --git a/crates/freshell-session-host/src/provider_secret_resolution.rs b/crates/freshell-session-host/src/provider_secret_resolution.rs index f2e62edea..11c8c5862 100644 --- a/crates/freshell-session-host/src/provider_secret_resolution.rs +++ b/crates/freshell-session-host/src/provider_secret_resolution.rs @@ -6,7 +6,7 @@ use std::path::Path; const MAX_KEYS_BYTES: u64 = 128 * 1024; #[derive(Default)] pub struct ResolvedProviderSecrets { - pub claude_settings_path: Option, + pub claude_settings_argument: Option, pub opencode_plan: Option, pub environment: BTreeMap, pub provider_files: Vec<(&'static str, Vec)>, diff --git a/crates/freshell-session-host/src/pty.rs b/crates/freshell-session-host/src/pty.rs index 9173ab578..b9555baeb 100644 --- a/crates/freshell-session-host/src/pty.rs +++ b/crates/freshell-session-host/src/pty.rs @@ -93,8 +93,8 @@ impl HostedPty { }); let mut provider_args = crate::provider_identity_args(launch.run_as_uid, launch.run_as_gid); provider_args.push(launch.program.clone()); - for argument in &launch.args { - provider_args.push(if launch.mode == "claude" && argument == freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT { + for (index, argument) in launch.args.iter().enumerate() { + provider_args.push(if launch.mode == "claude" && index > 0 && launch.args[index - 1] == "--settings" && argument == freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT { claude_settings.as_ref().ok_or("selected Claude settings origin is unavailable; retain inputs and reprepare before dispatch")?.clone() } else { argument.clone() }); } diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 58f63327f..91b9e98f8 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -34,9 +34,12 @@ def main(): parser.add_argument("--receipt-dir",type=Path,required=True) parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") parser.add_argument("--claude-flavor",choices=["freshclaude","kilroy"],default="freshclaude") + parser.add_argument("--claude-settings-form",choices=["file","inline"],default="file") args=parser.parse_args() if args.provider!="claude" and args.claude_flavor!="freshclaude": parser.error("the finite Claude flavor requires --provider claude") + if args.provider!="claude" and args.claude_settings_form!="file": + parser.error("the finite Claude settings form requires --provider claude") selected_test={"opencode":TEST,"codex":CODEX_TEST,"claude":CLAUDE_TEST,"amplifier":AMPLIFIER_TEST}[args.provider] if args.provider=="claude": IMAGE=CLAUDE_IMAGE repo=Path(__file__).resolve().parents[2] @@ -170,6 +173,7 @@ def create(name,*arguments): "--env",f"FRESHELL_PROVIDER_LAUNCH_HOST_BINARY={host}","--env",f"FRESHELL_PROVIDER_LAUNCH_IMAGE={IMAGE}", "--env",f"FRESHELL_PROVIDER_LAUNCH_INSTALLATION={installation}","--env",f"FRESHELL_PROVIDER_LAUNCH_RUN_ID={run}", "--env",f"FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR={args.claude_flavor}", + "--env",f"FRESHELL_PROVIDER_LAUNCH_CLAUDE_SETTINGS_FORM={args.claude_settings_form}", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_ENV_FILE={root}/grants/environment.env", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE={root}/grants/auth.json", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE={root}/grants/ca.pem", @@ -184,7 +188,9 @@ def create(name,*arguments): exit_code=int(waited.stdout.strip()) outcome=docker("logs",controller,check=False) receipt["exitCode"]=exit_code - if args.provider=="claude": receipt["claudeFlavor"]=args.claude_flavor + if args.provider=="claude": + receipt["claudeFlavor"]=args.claude_flavor + receipt["claudeSettingsForm"]=args.claude_settings_form receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) From ccfb16e7dd23bfb600da457092ef4dee588f3733 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 12:19:01 -0700 Subject: [PATCH 089/493] fix: retain Claude settings selectors and accepted declaring origins --- crates/freshell-platform/src/claude_inputs.rs | 4 + .../src/managed_mcp_capability.rs | 74 +++++++++++- .../managed_mcp_capability/claude_stage.rs | 22 ++-- .../managed_mcp_capability/ordinary_stage.rs | 5 + .../src/managed_provider_references/claude.rs | 107 +++++++++++++++++- .../src/provider_input_preparation.rs | 51 ++++++++- 6 files changed, 245 insertions(+), 18 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index 4d091e417..4893bee61 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -81,6 +81,8 @@ pub struct ClaudeInputManifest { #[serde(default, skip_serializing_if = "Option::is_none")] pub settings_file: Option, #[serde(default, skip_serializing_if = "Option::is_none")] + pub settings_origin: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] pub inline_settings_sha256: Option, #[serde(default = "default_setting_sources")] pub setting_sources: Vec, @@ -139,6 +141,7 @@ pub fn default_setting_sources() -> Vec { impl ClaudeInputManifest { pub fn validate(&self) -> Result<(), &'static str> { if self.version != 1 + || (self.settings_origin.is_some() && self.settings_file.is_none()) || self.inline_settings_sha256.as_ref().is_some_and(|digest| { self.settings_file.is_some() || digest.len() != 64 @@ -158,6 +161,7 @@ impl ClaudeInputManifest { .into_iter() .chain(self.config_root.iter()) .chain(self.settings_file.iter()) + .chain(self.settings_origin.iter()) .chain(&self.plugin_dirs) { if !path.is_absolute() diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 99576b479..0fc3d66c2 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1596,7 +1596,6 @@ fn stage_provider_root( claude_stage::stage_settings( &temporary, resources, - opencode_input.claude_manifest, opencode_input.claude_inline_settings, )?; } @@ -3366,6 +3365,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ workspace: cwd.clone(), config_root: Some(selected.clone()), settings_file: None, + settings_origin: None, inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], @@ -3451,7 +3451,8 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ cwd: cwd.clone(), workspace: cwd.clone(), config_root: None, - settings_file: Some(std::fs::canonicalize(&alias).unwrap()), + settings_file: Some(alias.clone()), + settings_origin: None, inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], @@ -3494,12 +3495,15 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .iter() .find(|mount| Path::new(&mount.logical_destination) == commands) .unwrap(); + let consumer_script = "import {readFileSync} from 'node:fs';import {execSync} from 'node:child_process';const s=JSON.parse(readFileSync(process.argv[1]));const r=execSync(s.hooks.SessionStart[0].hooks[0].command,{cwd:process.argv[2]});process.stdout.write([s.env.SELECTED_NONCE,r.toString(),s.hooks.SessionStart.length,s.hooks.Stop.length].join(':'))"; let output = std::process::Command::new("node") - .arg("--input-type=module").arg("-e") - .arg("import {readFileSync} from 'node:fs';import {execSync} from 'node:child_process';const s=JSON.parse(readFileSync(process.argv[1]));const r=execSync(s.hooks.SessionStart[0].hooks[0].command,{cwd:process.argv[2]});process.stdout.write([s.env.SELECTED_NONCE,r.toString(),s.hooks.SessionStart.length,s.hooks.Stop.length].join(':'))") + .arg("--input-type=module") + .arg("-e") + .arg(consumer_script) .arg(stage.join(&settings.source_relative_path)) .arg(stage.join(&resources.stage_relative_path)) - .output().unwrap(); + .output() + .unwrap(); assert!( output.status.success(), "{}", @@ -3508,6 +3512,64 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ assert_eq!(output.stdout, b"selected-explicit:transported-resource:2:1"); assert_eq!(std::fs::read(&selected).unwrap(), raw.as_bytes()); assert_eq!(std::fs::read_link(&alias).unwrap(), selected); + let accepted: ClaudeInputManifest = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + let next_source = declaring.join("next-settings.json"); + std::fs::write( + &next_source, + raw.replace("selected-explicit", "selected-next"), + ) + .unwrap(); + std::fs::remove_file(&alias).unwrap(); + std::os::unix::fs::symlink(&next_source, &alias).unwrap(); + let next_stage = stage_provider_root( + &capabilities, + "grant-claude-explicit-next", + "claude", + &home, + &[], + OpencodeEphemeralInput { + launch_context: Some(&context), + claude_manifest: Some(&accepted), + cwd: Some(&cwd), + workspace: Some(&cwd), + ..Default::default() + }, + None, + ) + .unwrap(); + let next_inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(next_stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let next_settings = next_inputs.config_binds.iter().find(|bind| Path::new(&bind.logical_destination)==next_source).expect("new preparation must use the changed original alias and its new native declaring file"); + let next_resources = next_inputs + .resource_mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == commands) + .unwrap(); + let output = std::process::Command::new("node") + .args(["--input-type=module", "-e", consumer_script]) + .arg(next_stage.join(&next_settings.source_relative_path)) + .arg(next_stage.join(&next_resources.stage_relative_path)) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"selected-next:transported-resource:2:1"); + let previous = std::process::Command::new("node") + .args(["--input-type=module", "-e", consumer_script]) + .arg(stage.join(&settings.source_relative_path)) + .arg(stage.join(&resources.stage_relative_path)) + .output() + .unwrap(); + assert!(previous.status.success()); + assert_eq!( + previous.stdout, + b"selected-explicit:transported-resource:2:1" + ); + assert_eq!(std::fs::read(&selected).unwrap(), raw.as_bytes()); } #[test] @@ -3536,6 +3598,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ workspace: cwd.clone(), config_root: None, settings_file: None, + settings_origin: None, inline_settings_sha256: None, setting_sources: vec![], plugin_dirs: vec![], @@ -4886,6 +4949,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ workspace: workspace.clone(), config_root: Some(selected.clone()), settings_file: None, + settings_origin: None, inline_settings_sha256: Some(format!( "{:x}", sha2::Sha256::digest(inline_settings.as_bytes()) diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 1890a82d2..c0685aab8 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -10,13 +10,14 @@ pub(super) fn prepare( accepted: Option<&ClaudeInputManifest>, excluded: &[PathBuf], ) -> Result { - let manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { + let mut manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { version: 1, original_home: home.into(), cwd: cwd.into(), workspace: workspace.into(), config_root: None, settings_file: None, + settings_origin: None, inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec![], @@ -26,7 +27,15 @@ pub(super) fn prepare( let mut resources = Vec::new(); let mut memory = Vec::new(); if let Some(selected) = &manifest.settings_file { - configurations.push((selected.clone(), canonical(selected)?)); + let origin = canonical(selected)?; + if !std::fs::metadata(&origin).is_ok_and(|metadata| metadata.is_file()) { + return Err(format!( + "Claude --settings {} must select a regular file", + selected.display() + )); + } + manifest.settings_origin = Some(origin.clone()); + configurations.push((origin.clone(), origin)); } let mut bases: Vec = manifest .cwd @@ -138,16 +147,14 @@ pub(super) fn prepare( pub(super) fn stage_settings( root: &Path, preparation: &Preparation, - manifest: Option<&ClaudeInputManifest>, inline_settings: Option<&str>, ) -> Result<(), String> { use freshell_platform::claude_inputs::{ settings_with_owned_hooks, INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES, }; use std::io::Read; - let Some(manifest) = manifest else { - return Ok(()); - }; + let manifest: ClaudeInputManifest = serde_json::from_slice(preparation.manifest_bytes()) + .map_err(|_| "accepted Claude discovery metadata invalid")?; if let Some(digest) = &manifest.inline_settings_sha256 { let result: Result<(), String> = (|| { use sha2::{Digest, Sha256}; @@ -175,7 +182,7 @@ pub(super) fn stage_settings( ); }); } - let Some(selected) = &manifest.settings_file else { + let Some(selected) = &manifest.settings_origin else { return Ok(()); }; let destination = manifest.runtime_path(selected); @@ -260,6 +267,7 @@ mod tests { workspace: cwd.clone(), config_root: Some(user.clone()), settings_file: None, + settings_origin: None, inline_settings_sha256: None, setting_sources: vec![ClaudeSettingSource::Project], plugin_dirs: vec![], diff --git a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs index ceb0a7fc9..52d53fea3 100644 --- a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs @@ -15,6 +15,11 @@ pub(super) struct Preparation { pub binds: Vec, pub mounts: Vec, } +impl Preparation { + pub(super) fn manifest_bytes(&self) -> &[u8] { + &self.manifest_bytes + } +} pub(super) fn prepare( home: &Path, cwd: &Path, diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index a558ccc81..b9e24ccce 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -69,6 +69,7 @@ pub(crate) async fn prepare_managed( request.original, )?), settings_file: selected_settings_file(request.cwd, request.selected_settings)?, + settings_origin: None, inline_settings_sha256: inline_settings.map(|value| { use sha2::{Digest, Sha256}; format!("{:x}", Sha256::digest(value.as_bytes())) @@ -140,15 +141,15 @@ fn selected_settings_file( return Err("Claude selected settings path profile is invalid".into()); } let path = Path::new(value); - let logical = if path.is_absolute() { + let logical = freshell_platform::opencode_config::lexical_path(&if path.is_absolute() { path.to_path_buf() } else { cwd.join(path) - }; - std::fs::canonicalize(&logical).map(Some).map_err(|error| { - tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="select_explicit_settings",path=%logical.display(),cause=%error, + }); + std::fs::canonicalize(&logical).map(|_| Some(logical.clone())).map_err(|error| { + tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="select_explicit_settings",field="--settings",cwd=%cwd.display(),cause=%error, "Selected Claude settings source is unavailable"); - format!("Claude --settings {} is unavailable: {error}", logical.display()) + format!("Claude --settings selected file is unavailable: {error}; select an existing settings file or valid inline JSON") }) } @@ -188,6 +189,102 @@ fn selected_config_root( mod tests { use super::*; + #[test] + fn managed_provider_claude_settings_lookup_failure_keeps_private_document_out_of_errors_and_logs( + ) { + use std::sync::{Arc, Mutex}; + use tracing_subscriber::prelude::*; + #[derive(Clone, Default)] + struct Capture(Arc>>); + impl tracing::field::Visit for Capture { + fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) { + self.0 + .lock() + .unwrap() + .insert(field.name().into(), format!("{value:?}")); + } + fn record_str(&mut self, field: &tracing::field::Field, value: &str) { + self.0 + .lock() + .unwrap() + .insert(field.name().into(), value.into()); + } + } + impl tracing_subscriber::Layer for Capture { + fn on_event( + &self, + event: &tracing::Event<'_>, + _: tracing_subscriber::layer::Context<'_, S>, + ) { + event.record(&mut self.clone()); + } + } + let owned = tempfile::tempdir().unwrap(); + let capture = Capture::default(); + let subscriber = tracing_subscriber::registry().with(capture.clone()); + let dispatcher = tracing::Dispatch::new(subscriber); + let raw = r#"{"env":{"PRIVATE_KEY":"synthetic-private-settings-value"}} trailing"#; + let error = tracing::dispatcher::with_default(&dispatcher, || { + selected_settings_file(owned.path(), Some(raw)) + }) + .err() + .unwrap(); + assert!( + !error.contains("synthetic-private-settings-value"), + "failed settings selection must not return raw private input" + ); + let fields = capture.0.lock().unwrap(); + assert!( + !fields.is_empty(), + "selection failure must emit its structured diagnostic" + ); + assert!( + !fields + .values() + .any(|value| value.contains("synthetic-private-settings-value")), + "selection failure must not log raw private input" + ); + assert_eq!( + fields.get("operation").map(String::as_str), + Some("select_explicit_settings") + ); + assert!(fields.contains_key("cause")); + } + + #[cfg(unix)] + #[test] + fn managed_provider_claude_settings_selector_resolves_updated_alias_for_next_preparation() { + let owned = tempfile::tempdir().unwrap(); + let cwd = owned.path().join("workspace"); + std::fs::create_dir(&cwd).unwrap(); + let first = cwd.join("first-settings.json"); + let second = cwd.join("second-settings.json"); + let alias = cwd.join("selected-settings.json"); + std::fs::write(&first, b"first-source-nonce").unwrap(); + std::fs::write(&second, b"next-source-nonce").unwrap(); + std::os::unix::fs::symlink(&first, &alias).unwrap(); + let selected = selected_settings_file(&cwd, Some("selected-settings.json")) + .unwrap() + .unwrap(); + let accepted_snapshot = owned.path().join("accepted-snapshot"); + std::fs::copy( + std::fs::canonicalize(&selected).unwrap(), + &accepted_snapshot, + ) + .unwrap(); + std::fs::remove_file(&alias).unwrap(); + std::os::unix::fs::symlink(&second, &alias).unwrap(); + // Reissuing from accepted original selectors must resolve the next + // original choice, while the accepted prior generation stays intact. + let next_origin = std::fs::canonicalize(&selected).unwrap(); + assert_eq!(std::fs::read(next_origin).unwrap(), b"next-source-nonce"); + assert_eq!( + std::fs::read(&accepted_snapshot).unwrap(), + b"first-source-nonce" + ); + assert_eq!(std::fs::read(&first).unwrap(), b"first-source-nonce"); + } + #[test] fn managed_provider_claude_config_root_keeps_native_cwd_empty_tilde_and_unicode_selection() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 3b30b438f..63db2646e 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -89,7 +89,8 @@ pub fn prepare( .map_err(|_| "prepared Claude inline settings is not UTF-8")?, ); } - if let Some(selected) = &manifest.settings_file { + if manifest.settings_file.is_some() { + let selected = manifest.settings_origin.as_ref().ok_or("selected Claude settings declaring origin is unrecorded; retain inputs and reprepare before dispatch")?; let destination = manifest.runtime_path(selected); if !inputs .config_binds @@ -347,6 +348,7 @@ mod tests { workspace: "/absent-cwd".into(), config_root: None, settings_file: None, + settings_origin: None, inline_settings_sha256: Some(format!("{:x}", Sha256::digest(raw))), setting_sources: vec![], plugin_dirs: vec![], @@ -441,6 +443,7 @@ mod tests { workspace: "/absent-original-workspace".into(), config_root: None, settings_file: None, + settings_origin: None, inline_settings_sha256: None, setting_sources: freshell_platform::claude_inputs::default_setting_sources(), plugin_dirs: vec!["/absent-original-plugin".into()], @@ -497,6 +500,52 @@ mod tests { .is_err(), "foreign source recipes cannot enter ordinary Claude metadata" ); + context + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .source_count = 0; + let mut selected = manifest; + selected.settings_file = Some("/absent-original-home/selected-settings.json".into()); + let bytes = serde_json::to_vec(&selected).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let error = prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + owned.path(), + owned.path(), + ) + .err() + .unwrap(); + assert!(error.contains("declaring origin is unrecorded")); + assert!(error.contains("retain inputs")); + assert!(secrets.claude_settings_argument.is_none()); + selected.settings_origin = Some("/absent-original-home/declaring-settings.json".into()); + let bytes = serde_json::to_vec(&selected).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + context + .prepared_inputs + .as_mut() + .unwrap() + .config_binds + .push(ProviderConfigBind { + source_relative_path: ".freshell/templates/claude-0".into(), + logical_destination: "/home/freshell/provider/declaring-settings.json".into(), + effective_destination: "/home/freshell/provider/declaring-settings.json".into(), + }); + std::fs::create_dir(owned.path().join(".freshell/templates")).unwrap(); + std::fs::write(owned.path().join(".freshell/templates/claude-0"), b"{}").unwrap(); + prepare(Some(&context), &mut secrets, &ProviderPathInputs::default(), owned.path(), owned.path()).expect("a recorded native declaring origin is consumed without reading the unavailable original controller tree"); + assert_eq!( + secrets.claude_settings_argument.as_deref(), + Some("/home/freshell/provider/declaring-settings.json") + ); } #[test] From c62f208797a4b9bf43e99dea55f0dfee5476ee6a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 13:06:41 -0700 Subject: [PATCH 090/493] fix: retain only native last Claude settings selection --- crates/freshell-runtime-protocol/src/lib.rs | 78 +++++++++++++++++-- .../src/managed_provider_launch_tests.rs | 29 +++++-- .../managed-provider-launch-sandbox.py | 2 +- 3 files changed, 93 insertions(+), 16 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index d46679dc4..e29ce8691 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2252,6 +2252,7 @@ pub fn durable_managed_terminal_args( }; let mut out = Vec::with_capacity(args.len()); let mut index = 0; + let mut selected_settings = None; while index < args.len() { match args[index].as_str() { flag if flag == "--setting-sources" || flag.starts_with("--setting-sources=") => { @@ -2287,14 +2288,7 @@ pub fn durable_managed_terminal_args( index += 2; value.as_str() }; - out.push("--settings".into()); - out.push( - if freshell_platform::cli_launch::is_owned_claude_settings(value) { - claude_settings.ok_or_else(invalid)?.into() - } else { - freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT.into() - }, - ); + selected_settings = Some(value); } _ => { out.push(args[index].clone()); @@ -2302,6 +2296,16 @@ pub fn durable_managed_terminal_args( } } } + if let Some(value) = selected_settings { + out.push("--settings".into()); + out.push( + if freshell_platform::cli_launch::is_owned_claude_settings(value) { + claude_settings.ok_or_else(invalid)?.into() + } else { + freshell_platform::claude_inputs::MANAGED_SETTINGS_ARGUMENT.into() + }, + ); + } validate_managed_terminal_args(mode, &out)?; Ok(out) } @@ -3497,6 +3501,64 @@ mod tests { ); } + #[test] + fn managed_claude_duplicate_settings_uses_only_last_native_selection() { + use freshell_platform::claude_inputs::{ + selected_settings_argument, MANAGED_SETTINGS_ARGUMENT, + }; + use freshell_platform::cli_launch::{claude_settings_json, ProviderTarget}; + let shipped = claude_settings_json(ProviderTarget::Unix); + for ignored in [ + "missing-ignored-settings.json", + "{\"env\":{\"PRIVATE_IGNORED\":\"sentinel\"}}", + ] { + for last in [format!("--settings={shipped}"), "--settings".into()] { + let mut args = vec![ + "--settings".into(), + ignored.into(), + "--init-only".into(), + last, + ]; + if args.last().unwrap() == "--settings" { + args.push(shipped.clone()); + } + let retained = + durable_managed_terminal_args("claude", args, Some(&shipped)).unwrap(); + validate_managed_terminal_args("claude", &retained).unwrap(); + assert_eq!( + selected_settings_argument(&retained), + Some(shipped.as_str()) + ); + assert!(!retained.iter().any(|arg| arg == MANAGED_SETTINGS_ARGUMENT), + "an ignored earlier selection must not demand private selected settings at spawn"); + assert!(!serde_json::to_string(&retained) + .unwrap() + .contains("PRIVATE_IGNORED")); + } + } + let retained = durable_managed_terminal_args( + "claude", + vec![ + "--settings".into(), + shipped.clone(), + "--settings=selected-last.json".into(), + ], + Some(&shipped), + ) + .unwrap(); + assert_eq!( + selected_settings_argument(&retained), + Some(MANAGED_SETTINGS_ARGUMENT) + ); + assert_eq!( + retained + .iter() + .filter(|arg| arg.as_str() == "--settings") + .count(), + 1 + ); + } + #[test] fn claude_native_initialization_argument_retains_selected_plugin_and_rejects_credentials() { let args = vec![ diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 413e2a0b5..e2b6a8e18 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -886,7 +886,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ let flag_argument = match settings_form.as_str() { "file" => "../source-home/selected-config/selected-settings.json".into(), "inline" => String::from_utf8(flag_bytes.clone()).unwrap(), - _ => panic!("closed fixture settings form must be file or inline"), + "owned-last" => freshell_platform::cli_launch::claude_settings_json( + freshell_platform::cli_launch::ProviderTarget::Unix, + ), + _ => panic!("closed fixture settings form must be file, inline or owned-last"), }; let instructions_consumer = r#"const fs=require('node:fs'),crypto=require('node:crypto'),path=require('node:path'); let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ @@ -1033,6 +1036,11 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ "--init-only".into(), "--setting-sources".into(), "user".into(), + // Native chooses the last selector. Neither an unavailable + // earlier file nor its private inline document is selected. + "--settings".into(), + "missing-ignored-settings.json".into(), + "--settings={\"env\":{\"CLAUDE_SELECTED_NONCE\":\"ignored-settings-sentinel\"}}".into(), "--settings".into(), flag_argument, "--plugin-dir".into(), @@ -1189,12 +1197,19 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ observed.expect("native Claude selected plugin must execute its startup hook"), expected ); - assert_eq!( - consumed(&flag_receipt) - .await - .expect("native must execute the explicitly selected flag settings hook"), - json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) - ); + if settings_form == "owned-last" { + assert!( + !flag_receipt.exists(), + "ignored user settings hooks must not execute" + ); + } else { + assert_eq!( + consumed(&flag_receipt) + .await + .expect("native must execute the explicitly selected last flag settings hook"), + json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) + ); + } assert_eq!(std::fs::read(&flag_settings).unwrap(), flag_bytes); assert_eq!(std::fs::read_link(&flag_alias).unwrap(), flag_settings); let native_import = consumed(&instructions_receipt).await; diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 91b9e98f8..fd0d69cd0 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -34,7 +34,7 @@ def main(): parser.add_argument("--receipt-dir",type=Path,required=True) parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") parser.add_argument("--claude-flavor",choices=["freshclaude","kilroy"],default="freshclaude") - parser.add_argument("--claude-settings-form",choices=["file","inline"],default="file") + parser.add_argument("--claude-settings-form",choices=["file","inline","owned-last"],default="file") args=parser.parse_args() if args.provider!="claude" and args.claude_flavor!="freshclaude": parser.error("the finite Claude flavor requires --provider claude") From d34d3a8c445d51fa0c06881e113247e73d75de34 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 13:21:25 -0700 Subject: [PATCH 091/493] fix: select requested provider home consistently --- crates/freshell-runtime-protocol/src/lib.rs | 9 +++--- .../src/managed_provider_launch_tests.rs | 27 +++++++++++++--- crates/freshell-server/src/managed_runtime.rs | 32 ++++++++----------- 3 files changed, 41 insertions(+), 27 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index e29ce8691..56cd0a120 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2288,7 +2288,7 @@ pub fn durable_managed_terminal_args( index += 2; value.as_str() }; - selected_settings = Some(value); + selected_settings = Some((out.len(), value)); } _ => { out.push(args[index].clone()); @@ -2296,9 +2296,10 @@ pub fn durable_managed_terminal_args( } } } - if let Some(value) = selected_settings { - out.push("--settings".into()); - out.push( + if let Some((position, value)) = selected_settings { + out.insert(position, "--settings".into()); + out.insert( + position + 1, if freshell_platform::cli_launch::is_owned_claude_settings(value) { claude_settings.ok_or_else(invalid)?.into() } else { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index e2b6a8e18..4371a7d44 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -510,6 +510,14 @@ async def on_session_ready(coordinator): .unwrap() .unwrap(); let cwd = workspace.to_str().unwrap().to_string(); + let ambient_home = fixture.join("ambient-home"); + std::fs::create_dir_all(ambient_home.join(".amplifier")).unwrap(); + std::fs::write( + ambient_home.join(".amplifier/settings.yaml"), + "bundle:\n active: ./missing-ambient-decoy\n", + ) + .unwrap(); + std::env::set_var("HOME", &ambient_home); let terminal = controller .launch(ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { @@ -525,7 +533,7 @@ async def on_session_ready(coordinator): cols: 120, rows: 30, }, - env: BTreeMap::new(), + env: BTreeMap::from([("HOME".into(), home.to_str().unwrap().into())]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), mode: "amplifier".into(), @@ -1028,6 +1036,14 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .unwrap() .unwrap(); let cwd = workspace.to_str().unwrap().to_string(); + let ambient_home = fixture.join("ambient-home"); + std::fs::create_dir_all(ambient_home.join(".claude")).unwrap(); + std::fs::write( + ambient_home.join(".claude/CLAUDE.md"), + "@missing-ambient-decoy.md\n", + ) + .unwrap(); + std::env::set_var("HOME", &ambient_home); let terminal = controller .launch(ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { @@ -1051,10 +1067,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ cols: 120, rows: 30, }, - env: BTreeMap::from([( - "CLAUDE_CONFIG_DIR".into(), - "../source-home/selected-config".into(), - )]), + env: BTreeMap::from([ + ("HOME".into(), home.to_str().unwrap().into()), + ("CLAUDE_CONFIG_DIR".into(), "../source-home/selected-config".into()), + ]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), mode: "claude".into(), @@ -1254,6 +1270,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ std::fs::remove_file(&instructions_events).unwrap(); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); + std::env::set_var("HOME", &home); let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( Some(client.clone()), Arc::new(broadcast), diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 72f06f038..330db27c7 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -201,12 +201,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { ..Default::default() }; let (mcp_capability, provider_launch_context) = if request.mode == "opencode" { - let home = request - .env - .get("HOME") - .map(std::path::PathBuf::from) - .or_else(|| std::env::var_os("HOME").map(std::path::PathBuf::from)) - .ok_or("managed provider HOME unavailable")?; + let home = original_provider_home(&request.env)?; let original = |name: &str| { request .env @@ -241,12 +236,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { Some(prepared.context), ) } else if request.mode == "codex" { - let home = request - .env - .get("HOME") - .map(std::path::PathBuf::from) - .or_else(|| std::env::var_os("HOME").map(std::path::PathBuf::from)) - .ok_or("managed provider HOME unavailable")?; + let home = original_provider_home(&request.env)?; let original = |name: &str| { request .env @@ -270,9 +260,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .await?; (context.mcp_capability.clone(), Some(context)) } else if request.mode == "claude" { - let home = std::env::var_os("HOME") - .map(std::path::PathBuf::from) - .ok_or("original Claude HOME is unavailable")?; + let home = original_provider_home(&request.env)?; let original = |name: &str| { request .env @@ -299,9 +287,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .await?; (context.mcp_capability.clone(), Some(context)) } else if request.mode == "amplifier" { - let home = std::env::var_os("HOME") - .map(std::path::PathBuf::from) - .ok_or("original Amplifier HOME is unavailable")?; + let home = original_provider_home(&request.env)?; let context = crate::managed_provider_references::amplifier::prepare_managed( crate::managed_provider_references::amplifier::AmplifierPreparation { client: &self.client, @@ -810,6 +796,16 @@ fn managed_env_allowlist( .collect() } +fn original_provider_home( + request: &std::collections::BTreeMap, +) -> Result { + request + .get("HOME") + .map(PathBuf::from) + .or_else(|| std::env::var_os("HOME").map(PathBuf::from)) + .ok_or("managed provider HOME unavailable") +} + fn managed_provider_env( mode: &str, source: &std::collections::BTreeMap, From ff19c76de423f952cb8bb6285b676cd7cde8c058 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 13:37:27 -0700 Subject: [PATCH 092/493] fix: retain complete Claude user rule imports --- .../managed_mcp_capability/claude_memory.rs | 83 ++++++++++++++++++- .../managed_mcp_capability/claude_stage.rs | 59 ++++++++++++- .../src/managed_provider_launch_tests.rs | 14 +++- .../testing/managed-provider-launch-broker.ts | 2 +- 4 files changed, 151 insertions(+), 7 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs index 1b3ab515f..1faa4d98f 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs @@ -13,6 +13,7 @@ use std::{ pub(super) fn imports( home: &Path, roots: &[PathBuf], + user_rules: &[PathBuf], excluded: &[PathBuf], ) -> Result, String> { let excluded = excluded @@ -29,10 +30,18 @@ pub(super) fn imports( events: 0, path_bytes: 0, selected: Vec::new(), + rule_directories: BTreeSet::new(), + rule_entries: 0, + rule_path_bytes: 0, }; for root in roots { graph.visit(root, 0)?; } + // Native user rules use the same Markdown import parser, including + // external imports, before conditional rules are activated by the provider. + for root in user_rules { + graph.visit_rules(root, 0)?; + } Ok(graph.selected) } @@ -46,8 +55,74 @@ struct ImportGraph<'a> { events: usize, path_bytes: usize, selected: Vec, + rule_directories: BTreeSet, + rule_entries: usize, + rule_path_bytes: usize, } impl ImportGraph<'_> { + fn visit_rules(&mut self, logical: &Path, depth: usize) -> Result<(), String> { + let limits = freshell_platform::provider_environment::CopyLimits::default(); + let error = + |cause: &str| format!("Claude selected user rules {}: {cause}", logical.display()); + if depth > limits.max_depth { + return Err(error("resource depth profile exceeded")); + } + let source = std::fs::canonicalize(logical) + .map_err(|cause| error(&format!("selected directory unavailable: {cause}")))?; + if self + .excluded + .iter() + .any(|excluded| source.starts_with(excluded)) + { + return Err(error( + "selected private or native-state source cannot become ordinary rules", + )); + } + if self.active.contains(&source) { + return Err(error("cyclic selected rules directory")); + } + if !self.rule_directories.insert(logical.to_owned()) { + return Ok(()); + } + self.active.insert(source.clone()); + let entries = std::fs::read_dir(logical) + .map_err(|cause| error(&format!("selected directory unreadable: {cause}")))?; + let mut paths = Vec::new(); + for entry in entries { + let path = entry + .map_err(|cause| error(&format!("selected directory entry unavailable: {cause}")))? + .path(); + self.rule_entries = self + .rule_entries + .checked_add(1) + .ok_or_else(|| error("resource entry accounting overflow"))?; + self.rule_path_bytes = self + .rule_path_bytes + .checked_add(path.as_os_str().len()) + .ok_or_else(|| error("resource metadata accounting overflow"))?; + if self.rule_entries > limits.max_entries || self.rule_path_bytes > 64 * 1024 * 1024 { + return Err(error("resource entry or metadata profile exceeded")); + } + paths.push(path); + } + paths.sort(); + for path in paths { + let metadata = std::fs::metadata(&path).map_err(|cause| { + error(&format!( + "selected rule {} metadata failed: {cause}", + path.display() + )) + })?; + if metadata.is_dir() { + self.visit_rules(&path, depth + 1)?; + } else if path.extension().is_some_and(|extension| extension == "md") { + self.visit(&path, depth + 1)?; + } + } + self.active.remove(&source); + Ok(()) + } + fn visit(&mut self, logical: &Path, depth: usize) -> Result<(), String> { let logical = normalize(logical); let error = |cause: &str| format!("Claude memory import {}: {cause}", logical.display()); @@ -284,7 +359,7 @@ mod tests { memory.push_str(&format!("@{name}\n")); } std::fs::write(&root, memory).unwrap(); - let selected = imports(owned.path(), &[root], &[]) + let selected = imports(owned.path(), &[root], &[], &[]) .expect("ordinary references retain the declared source event profile"); assert_eq!(selected.len(), 129); for path in selected { @@ -311,7 +386,7 @@ mod tests { ), ) .unwrap(); - let selected = imports(&home, &[memory], &[]).unwrap(); + let selected = imports(&home, &[memory], &[], &[]).unwrap(); assert_eq!(selected.len(), 2); let values = selected .iter() @@ -329,10 +404,10 @@ mod tests { let root = owned.path().join("CLAUDE.md"); let nested = owned.path().join("nested.md"); std::fs::write(&root, "@nested.md\n").unwrap(); - let missing = imports(owned.path(), &[root.clone()], &[]).unwrap_err(); + let missing = imports(owned.path(), &[root.clone()], &[], &[]).unwrap_err(); assert!(missing.contains("nested.md") && missing.contains("unavailable")); std::fs::write(&nested, "@CLAUDE.md\n").unwrap(); - let cycle = imports(owned.path(), &[root.clone()], &[]).unwrap_err(); + let cycle = imports(owned.path(), &[root.clone()], &[], &[]).unwrap_err(); assert!(cycle.contains("cyclic")); assert_eq!(std::fs::read_to_string(root).unwrap(), "@nested.md\n"); assert_eq!(std::fs::read_to_string(nested).unwrap(), "@CLAUDE.md\n"); diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index c0685aab8..0b544002b 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -26,6 +26,7 @@ pub(super) fn prepare( let mut configurations = Vec::new(); let mut resources = Vec::new(); let mut memory = Vec::new(); + let mut user_rules = Vec::new(); if let Some(selected) = &manifest.settings_file { let origin = canonical(selected)?; if !std::fs::metadata(&origin).is_ok_and(|metadata| metadata.is_file()) { @@ -75,6 +76,8 @@ pub(super) fn prepare( if optional(&logical)? { if relative == "CLAUDE.md" { memory.push(logical.clone()); + } else if relative == "rules" { + user_rules.push(logical.clone()); } resources.push(logical); } @@ -125,7 +128,7 @@ pub(super) fn prepare( canonical(plugin)?; // Explicit selections may never be silently omitted. resources.push(plugin.clone()); } - for imported in super::claude_memory::imports(&manifest.original_home, &memory, excluded).inspect_err(|error| { + for imported in super::claude_memory::imports(&manifest.original_home, &memory, &user_rules, excluded).inspect_err(|error| { tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="select_memory_imports",reason=%error,"Selected Claude memory imports could not be prepared"); })? { if !resources.iter().any(|root| imported.starts_with(root)) { @@ -242,6 +245,60 @@ mod tests { provider_environment::{copy_environment, CopyLimits}, }; + #[test] + fn managed_provider_claude_selected_user_rules_keep_external_imports() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let rules = home.join(".claude/rules/nested"); + std::fs::create_dir_all(&rules).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + let rule = rules.join("policy.md"); + let raw = "@../../../rule-memory.md\n`@../../../missing-inline.md`\n```\n@../../../missing-fence.md\n```\n"; + std::fs::write(&rule, raw).unwrap(); + std::fs::write(home.join("rule-memory.md"), "@./adjacent-memory.md\n").unwrap(); + std::fs::write(home.join("adjacent-memory.md"), "selected-rule-resource").unwrap(); + let selected = prepare(&home, &cwd, &cwd, None, &[]).unwrap(); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + let target = Path::new("/home/freshell/provider/adjacent-memory.md"); + let imported = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == target) + .expect("selected native user rule imports must enter the ordinary graph"); + let output = std::process::Command::new("node") + .args([ + "-e", + "process.stdout.write(require('node:fs').readFileSync(process.argv[1]))", + ]) + .arg(stage.join(&imported.stage_relative_path)) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"selected-rule-resource"); + assert_eq!(std::fs::read_to_string(&rule).unwrap(), raw); + // A failed next preparation cannot change the complete prior stage. + std::fs::write(&rule, "@../../../missing-selected-rule.md\n").unwrap(); + let failed = prepare(&home, &cwd, &cwd, None, &[]).err().unwrap(); + assert!(failed.contains("missing-selected-rule.md")); + assert_eq!( + std::fs::read(stage.join(&imported.stage_relative_path)).unwrap(), + b"selected-rule-resource" + ); + std::fs::write(&rule, raw).unwrap(); + #[cfg(unix)] + { + std::os::unix::fs::symlink(&rules, rules.join("cycle")).unwrap(); + let failed = prepare(&home, &cwd, &cwd, None, &[]).err().unwrap(); + assert!(failed.contains("cyclic selected rules directory")); + assert_eq!( + std::fs::read(stage.join(&imported.stage_relative_path)).unwrap(), + b"selected-rule-resource" + ); + } + } + #[test] fn managed_provider_claude_disabled_sources_do_not_traverse_or_copy_unselected_memory() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 4371a7d44..8f585f7ae 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -812,6 +812,10 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep ) .unwrap(); std::fs::write(selected_config.join("CLAUDE.md"), &memory).unwrap(); + std::fs::create_dir_all(selected_config.join("rules/nested")).unwrap(); + let rule = "@../../../rule-memory.md\n`@../../../missing-rule-code.md`\n```\n@../../../missing-rule-fence.md\n```\n"; + std::fs::write(selected_config.join("rules/nested/policy.md"), rule).unwrap(); + std::fs::write(home.join("rule-memory.md"), "closed-selected-rule-memory\n").unwrap(); std::fs::write( home.join(".claude/CLAUDE.md"), "@../missing-unselected-default.md\n", @@ -902,7 +906,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ let instructions_consumer = r#"const fs=require('node:fs'),crypto=require('node:crypto'),path=require('node:path'); let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ const input=JSON.parse(raw),name=path.basename(input.file_path||''),event=input.hook_event_name==='InstructionsLoaded'; - const names=['imported-memory.md','relative-memory.md','home-memory.md','absolute-memory.md','spaced memory.md','nested-memory.md']; + const names=['imported-memory.md','relative-memory.md','home-memory.md','absolute-memory.md','spaced memory.md','nested-memory.md','rule-memory.md']; const selected=event&&names.includes(name)?{name,digest:crypto.createHash('sha256').update(fs.readFileSync(input.file_path)).digest('hex')}:null; fs.appendFileSync(process.argv[2]+'.events',JSON.stringify({event,global:input.file_path===process.env.HOME+'/.claude/CLAUDE.md',project:input.file_path===process.argv[3],ignored:event&&name.startsWith('ignored-'),selected})+'\n'); if(event&&input.file_path===process.env.HOME+'/imported-memory.md')fs.writeFileSync(process.argv[2],JSON.stringify({nativeImport:true,digest:selected.digest})); @@ -1105,6 +1109,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); let mut expected_imports = serde_json::Map::new(); expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); + expected_imports.insert( + "rule-memory.md".into(), + json!(hash(b"closed-selected-rule-memory\n")), + ); for name in [ "relative-memory.md", "home-memory.md", @@ -1227,6 +1235,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ ); } assert_eq!(std::fs::read(&flag_settings).unwrap(), flag_bytes); + assert_eq!( + std::fs::read_to_string(selected_config.join("rules/nested/policy.md")).unwrap(), + rule + ); assert_eq!(std::fs::read_link(&flag_alias).unwrap(), flag_settings); let native_import = consumed(&instructions_receipt).await; if native_import.is_none() { diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 45faec9d2..12cae5a00 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -26,7 +26,7 @@ const broker=new RestrictedDockerBroker({ allowedImageRefs:new Set([image]),allowTerminalWorkloads:true, allowedWorkspaceRoots:new Set([path.join(root,'workspace')]), allowedProviderUserRoots:new Set(configuration.provider==='claude' - ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-config'),path.join(root,'source-home/selected-plugin'),path.join(root,'source-home/imported-memory.md')] + ? ['/home/freshell/provider',path.join(root,'source-home/.claude'),path.join(root,'source-home/selected-config'),path.join(root,'source-home/selected-plugin'),path.join(root,'source-home/imported-memory.md'),path.join(root,'source-home/rule-memory.md')] : configuration.provider==='codex' ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] : configuration.provider==='amplifier' From 4b08ff67c6db8bddde869da303a5eec245ac3749 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:06:41 -0700 Subject: [PATCH 093/493] fix: retain Codex plugins enabled by native default --- .../src/managed_mcp_capability.rs | 78 +++++++++- .../src/managed_provider_references/codex.rs | 142 +++++++++++++++++- .../managed-provider-source-sandbox.py | 2 +- 3 files changed, 218 insertions(+), 4 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 0fc3d66c2..cf3f76aed 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4472,7 +4472,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::create_dir_all(original.join(".codex/agents/nested")).unwrap(); std::fs::create_dir_all(workspace.join(".codex/skills/probe/scripts")).unwrap(); std::fs::create_dir(workspace.join(".git")).unwrap(); - let config = format!("model_instructions_file = 'user.md'\n[[skills.config]]\npath = 'absent-skill'\nenabled = false\n[projects.{}]\ntrust_level = 'trusted'\n[agents.reader]\nconfig_file = 'agents/nested/reader.toml'\n",serde_json::to_string(workspace.to_str().unwrap()).unwrap()); + let config = format!("model_instructions_file = 'user.md'\n[[skills.config]]\npath = 'absent-skill'\nenabled = false\n[projects.{}]\ntrust_level = 'trusted'\n[agents.reader]\nconfig_file = 'agents/nested/reader.toml'\n[features]\nplugins=true\n[plugins.\"probe@fixture\"]\n",serde_json::to_string(workspace.to_str().unwrap()).unwrap()); std::fs::write(original.join(".codex/config.toml"), &config).unwrap(); std::fs::write(original.join(".codex/user.md"), "user-instructions-nonce").unwrap(); std::fs::write( @@ -4509,6 +4509,29 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(); use std::os::unix::fs::PermissionsExt; std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o700)).unwrap(); + let plugin = original.join(".codex/plugins/cache/fixture/probe/1.0.0"); + std::fs::create_dir_all(plugin.join(".codex-plugin")).unwrap(); + std::fs::create_dir_all(plugin.join("skills/probe/scripts/node_modules/probe-dependency")) + .unwrap(); + std::fs::write( + plugin.join(".codex-plugin/plugin.json"), + r#"{"name":"probe","version":"1.0.0","skills":"./skills"}"#, + ) + .unwrap(); + std::fs::write(plugin.join("skills/probe/SKILL.md"), "---\nname: installed-probe\ndescription: Synthetic installed plugin dependency consumer\n---\nRun scripts/run.\n").unwrap(); + std::fs::write( + plugin.join("skills/probe/resource"), + "installed-resource-nonce", + ) + .unwrap(); + std::fs::write( + plugin.join("skills/probe/scripts/node_modules/probe-dependency/index.js"), + "module.exports='installed-dependency-nonce'", + ) + .unwrap(); + let plugin_script = plugin.join("skills/probe/scripts/run"); + std::fs::write(&plugin_script, "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/../resource')+'|'+require('probe-dependency'))\n").unwrap(); + std::fs::set_permissions(&plugin_script, std::fs::Permissions::from_mode(0o700)).unwrap(); let mcp = crate::managed_provider_bootstrap::managed_codex_mcp_args().unwrap(); let context = ProviderLaunchContext { config: vec![], @@ -4680,8 +4703,57 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ found, "native Codex must discover the transported project skill" ); + let installed = skills["data"] + .as_array() + .unwrap() + .iter() + .flat_map(|data| data["skills"].as_array().unwrap()) + .find(|skill| skill["name"] == "probe:installed-probe") + .expect("native Codex must discover an installed plugin with default enabled=true"); + let installed_path = Path::new(installed["path"].as_str().unwrap()); + // The native catalog is complete. Join it before the independent + // executable consumer, as for the existing project resource below. child.kill().await.unwrap(); child.wait().await.unwrap(); + let pids_before = std::fs::read_to_string("/sys/fs/cgroup/pids.current") + .ok() + .and_then(|value| value.trim().parse::().ok()); + let installed_resource = + std::process::Command::new(installed_path.parent().unwrap().join("scripts/run")) + .output() + .unwrap(); + if !installed_resource.status.success() { + use std::os::unix::process::ExitStatusExt; + let stderr = String::from_utf8_lossy(&installed_resource.stderr); + let classes = [ + "MODULE_NOT_FOUND", + "ENOENT", + "EACCES", + "Permission denied", + "node: not found", + "No such file", + "SyntaxError", + "pthread_create", + "uv_thread_create", + "Resource temporarily unavailable", + ] + .into_iter() + .filter(|class| stderr.contains(class)) + .collect::>(); + println!( + "provider_source_diagnostic {}", + serde_json::json!({ + "provider":"codex", "operation":"installed_plugin_helper", "exitCode":installed_resource.status.code(), "signal":installed_resource.status.signal(), "pidsBefore":pids_before, + "stderrBytes":installed_resource.stderr.len(), + "classes":classes + }) + ); + } + assert!(installed_resource.status.success()); + assert_eq!( + installed_resource.stdout, + b"installed-resource-nonce|installed-dependency-nonce" + ); let resource = std::process::Command::new(workspace.join(".codex/skills/probe/scripts/run")) .output() @@ -4703,6 +4775,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ "native-retained-mcp-args", b"node:/opt/freshell-mcp/server.js".as_slice(), ), + ( + "native-installed-plugin-dependency", + installed_resource.stdout.as_slice(), + ), ] { println!( "provider_source_consumer {}", diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index d834921c2..9d288d8d8 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -519,8 +519,13 @@ pub fn discover( // not change the selection. Copy complete installed package/data units. if let Some(plugins) = preceding.get("plugins").and_then(toml::Value::as_table) { for (identity, value) in plugins { - if value.get("enabled").and_then(toml::Value::as_bool) != Some(true) { - continue; + let value = value + .as_table() + .ok_or("Codex installed plugin configuration must be a table")?; + match value.get("enabled") { + None | Some(toml::Value::Boolean(true)) => {} + Some(toml::Value::Boolean(false)) => continue, + _ => return Err("Codex installed plugin enabled field must be a boolean"), } let (plugin, market) = identity .split_once('@') @@ -654,6 +659,139 @@ mod tests { kind: LayerKind::User, } } + + #[test] + fn managed_provider_codex_omitted_plugin_enabled_retains_executable_dependencies() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let cache = home.join(".codex/plugins/cache/fixture/probe"); + let plugin = cache.join("1.0.0"); + std::fs::create_dir_all(plugin.join("scripts/node_modules/probe-dependency")).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write(plugin.join("scripts/resource"), "selected-plugin-resource").unwrap(); + std::fs::write( + plugin.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='installed-plugin-dependency'", + ) + .unwrap(); + std::fs::write(plugin.join("scripts/probe.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource')+'|'+require('probe-dependency'))").unwrap(); + for (index, flag) in ["", "enabled=true\n"].into_iter().enumerate() { + std::fs::write( + home.join(".codex/config.toml"), + format!("[plugins.\"probe@fixture\"]\n{flag}"), + ) + .unwrap(); + let found = discover(CodexInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + codex_home: home.join(".codex"), + named_profile: None, + }) + .unwrap(); + let root = found.resource_roots.iter().find(|root| root.starts_with(&cache)) + .expect("omitted enabled uses native true default and must retain selected installation"); + let stage = owned.path().join(format!("stage-{index}")); + copy_environment( + &[CopySelection { + source: root.clone(), + destination_relative: "plugin".into(), + }], + &stage, + &[], + CopyLimits::default(), + ) + .unwrap(); + let output = std::process::Command::new("node") + .arg( + stage + .join("plugin") + .join(plugin.strip_prefix(root).unwrap()) + .join("scripts/probe.cjs"), + ) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + b"selected-plugin-resource|installed-plugin-dependency" + ); + } + std::fs::write( + home.join(".codex/config.toml"), + "[plugins.\"missing@fixture\"]\nenabled=false\n", + ) + .unwrap(); + let disabled = discover(CodexInputManifest { + version: 1, + original_home: home.clone(), + cwd, + codex_home: home.join(".codex"), + named_profile: None, + }) + .unwrap(); + assert!(!disabled + .resource_roots + .iter() + .any(|root| root.starts_with(home.join(".codex/plugins")))); + } + + #[test] + fn managed_provider_codex_invalid_plugin_enabled_is_an_actionable_selection_error() { + use freshell_platform::codex_inputs::CodexInputManifest; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".codex/plugins/cache/fixture/probe/1.0.0")).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write( + home.join(".codex/config.toml"), + "[plugins.\"probe@fixture\"]\nenabled='PRIVATE_INVALID_FIELD'\n", + ) + .unwrap(); + let error = discover(CodexInputManifest { + version: 1, + original_home: home.clone(), + cwd, + codex_home: home.join(".codex"), + named_profile: None, + }) + .err() + .expect("unsupported native plugin enabled type must block before capability publication"); + assert!(error.contains("enabled")); + assert!(!error.contains("PRIVATE_INVALID_FIELD")); + } + + #[test] + fn managed_provider_codex_invalid_plugin_table_cannot_become_default_enabled() { + use freshell_platform::codex_inputs::CodexInputManifest; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".codex/plugins/cache/fixture/probe/1.0.0")).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write( + home.join(".codex/config.toml"), + "plugins={\"probe@fixture\"='PRIVATE_NOT_TABLE'}\n", + ) + .unwrap(); + let error = discover(CodexInputManifest { + version: 1, + original_home: home.clone(), + cwd, + codex_home: home.join(".codex"), + named_profile: None, + }) + .err() + .expect("an unsupported scalar plugin must never acquire omitted-enabled authority"); + assert!(error.contains("configuration")); + assert!(!error.contains("PRIVATE_NOT_TABLE")); + } #[test] fn managed_provider_codex_skill_path_selectors_do_not_create_resource_authority() { let owned = tempfile::tempdir().unwrap(); diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index d477df457..8a63e6592 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -75,7 +75,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":4,"claude":1}[args.provider] or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":5,"claude":1}[args.provider] or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. From e21731c1606d90cb659aaa5a02cd7548abe3b8a2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:17:21 -0700 Subject: [PATCH 094/493] fix: copy only the native active Codex plugin version --- Cargo.lock | 1 + crates/freshell-server/Cargo.toml | 2 + .../src/managed_mcp_capability.rs | 50 ++++++- .../src/managed_provider_references/codex.rs | 127 +++++++++++++++++- 4 files changed, 177 insertions(+), 3 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 56adc81b0..0e3abc67e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1470,6 +1470,7 @@ dependencies = [ "rusqlite", "saphyr", "saphyr-parser", + "semver", "serde", "serde_json", "sha1", diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index cdfdd2d60..2510a37a2 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -25,6 +25,7 @@ managed-runtime-v1 = [ "dep:toml", "dep:saphyr", "dep:saphyr-parser", + "dep:semver", ] managed-provider-qualification = [ "managed-runtime-v1", @@ -79,6 +80,7 @@ serde_json = { workspace = true } toml = { version = "=0.9.12", optional = true } saphyr = { version = "=0.0.11", default-features = false, optional = true } saphyr-parser = { version = "=0.0.11", optional = true } +semver = { version = "=1.0.28", optional = true } # Boot-scoped identifiers: `srv-` / `boot-`. uuid = { version = "1", features = ["v4"] } # The browser-pane HTTP reverse proxy (`/api/proxy/http/{port}/*`) forwards to a diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index cf3f76aed..f80e8b829 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4509,7 +4509,24 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(); use std::os::unix::fs::PermissionsExt; std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o700)).unwrap(); - let plugin = original.join(".codex/plugins/cache/fixture/probe/1.0.0"); + let cache = original.join(".codex/plugins/cache/fixture/probe"); + for version in ["1.9.0", "1.10.0", "zzz"] { + let inactive = cache.join(version); + std::fs::create_dir_all(inactive.join(".codex-plugin")).unwrap(); + std::fs::create_dir_all(inactive.join("skills/obsolete")).unwrap(); + std::fs::write( + inactive.join(".codex-plugin/plugin.json"), + r#"{"name":"probe","skills":"./skills"}"#, + ) + .unwrap(); + std::fs::write(inactive.join("skills/obsolete/SKILL.md"), "---\nname: obsolete-probe\ndescription: Inactive version control\n---\nInactive resource.\n").unwrap(); + } + std::os::unix::fs::symlink( + "absent-obsolete-target", + cache.join("1.9.0/obsolete-resource"), + ) + .unwrap(); + let plugin = cache.join("local"); std::fs::create_dir_all(plugin.join(".codex-plugin")).unwrap(); std::fs::create_dir_all(plugin.join("skills/probe/scripts/node_modules/probe-dependency")) .unwrap(); @@ -4592,8 +4609,28 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ for unit in &inputs.resource_mounts { let destination = Path::new(&unit.effective_destination); std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); - std::os::unix::fs::symlink(physical.join(&unit.provider_relative_path), destination) + if destination == plugin + || destination == physical.join(".codex/plugins/cache/fixture/probe/local") + { + // Real Docker directory mounts have directory inodes. Native + // Codex intentionally ignores version-directory symlinks. + copy_environment( + &[CopySelection { + source: physical.join(&unit.provider_relative_path), + destination_relative: destination.file_name().unwrap().into(), + }], + destination.parent().unwrap(), + &[], + CopyLimits::default(), + ) .unwrap(); + } else { + std::os::unix::fs::symlink( + physical.join(&unit.provider_relative_path), + destination, + ) + .unwrap(); + } } let version = std::process::Command::new("/usr/local/bin/codex") .arg("--version") @@ -4703,6 +4740,15 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ found, "native Codex must discover the transported project skill" ); + assert!( + !skills["data"] + .as_array() + .unwrap() + .iter() + .flat_map(|data| data["skills"].as_array().unwrap()) + .any(|skill| skill["name"] == "probe:obsolete-probe"), + "native local installation must win over all inactive versions" + ); let installed = skills["data"] .as_array() .unwrap() diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 9d288d8d8..4a48ecf13 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -548,7 +548,7 @@ pub fn discover( "Codex selected installed plugin is unavailable; repair installation and retry", ); } - resources.push(cache); + resources.push(active_plugin_version(&cache)?); let data = manifest .codex_home .join("plugins/data") @@ -572,6 +572,56 @@ pub fn discover( resource_roots: resources, }) } +fn active_plugin_version(cache: &Path) -> Result { + let entries = std::fs::read_dir(cache).map_err(|_| { + "Codex selected installed plugin versions are unreadable; repair installation and retry" + })?; + let mut versions = Vec::new(); + let mut count = 0usize; + let mut metadata_bytes = 0usize; + for entry in entries { + count += 1; + if count > freshell_platform::provider_environment::CopyLimits::default().max_entries { + return Err("Codex installed plugin version entry budget exceeded"); + } + let Ok(entry) = entry else { continue }; + if !entry.file_type().is_ok_and(|kind| kind.is_dir()) { + continue; + } + let Ok(name) = entry.file_name().into_string() else { + continue; + }; + metadata_bytes = metadata_bytes + .checked_add(name.len()) + .ok_or("Codex installed plugin version metadata overflow")?; + if metadata_bytes > 64 * 1024 * 1024 { + return Err("Codex installed plugin version metadata budget exceeded"); + } + if !name.is_empty() + && name != "." + && name != ".." + && name + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || b".+_-".contains(&byte)) + { + versions.push(name); + } + } + versions.sort_by(|left, right| { + match (semver::Version::parse(left), semver::Version::parse(right)) { + (Ok(left), Ok(right)) => left.cmp(&right), + _ => left.cmp(right), + } + }); + let selected = versions + .iter() + .find(|name| name.as_str() == "local") + .or_else(|| versions.last()) + .ok_or( + "Codex selected installed plugin has no active version; repair installation and retry", + )?; + Ok(cache.join(selected)) +} fn exists(path: &Path) -> Result { match std::fs::symlink_metadata(path) { Ok(_) => Ok(true), @@ -741,6 +791,81 @@ mod tests { .any(|root| root.starts_with(home.join(".codex/plugins")))); } + #[test] + fn managed_provider_codex_active_plugin_version_excludes_obsolete_resources() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let cache = home.join(".codex/plugins/cache/fixture/probe"); + std::fs::create_dir_all(&cwd).unwrap(); + for version in ["1.9.0", "1.10.0", "local", "zzz"] { + let package = cache.join(version); + std::fs::create_dir_all(package.join("scripts/node_modules/probe-dependency")).unwrap(); + std::fs::write(package.join("scripts/resource"), version).unwrap(); + std::fs::write( + package.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='version-dependency'", + ) + .unwrap(); + std::fs::write(package.join("scripts/probe.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource')+'|'+require('probe-dependency'))").unwrap(); + } + std::fs::write( + home.join(".codex/config.toml"), + "[plugins.\"probe@fixture\"]\n", + ) + .unwrap(); + #[cfg(unix)] + std::os::unix::fs::symlink( + "absent-obsolete-target", + cache.join("1.9.0/obsolete-resource"), + ) + .unwrap(); + for (index, expected) in ["local", "zzz", "1.10.0"].into_iter().enumerate() { + let found = discover(CodexInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + codex_home: home.join(".codex"), + named_profile: None, + }) + .unwrap(); + let stage = owned.path().join(format!("active-stage-{index}")); + let roots = found + .resource_roots + .iter() + .filter(|root| root.starts_with(&cache)) + .collect::>(); + assert_eq!(roots.len(), 1); + copy_environment( + &[CopySelection { + source: roots[0].clone(), + destination_relative: "plugin".into(), + }], + &stage, + &[], + CopyLimits::default(), + ) + .expect("inactive broken resource must not block the selected complete package"); + let output = std::process::Command::new("node") + .arg(stage.join("plugin/scripts/probe.cjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + format!("{expected}|version-dependency").as_bytes() + ); + // Remove only this test's synthetic, read fixtures after each consumer. + if expected != "1.10.0" { + std::fs::remove_dir_all(cache.join(expected)).unwrap(); + } + } + } + #[test] fn managed_provider_codex_invalid_plugin_enabled_is_an_actionable_selection_error() { use freshell_platform::codex_inputs::CodexInputManifest; From 119797a0445aeb1c5752e1bcaaf56375173429d9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:30:17 -0700 Subject: [PATCH 095/493] fix: retain Codex AgentPlugin native hashed data --- .../src/managed_provider_launch_tests.rs | 56 +++- .../src/managed_provider_references/codex.rs | 263 +++++++++++++++++- 2 files changed, 312 insertions(+), 7 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 8f585f7ae..abbbe0ba4 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -1424,6 +1424,39 @@ let result={};if(message.method==='initialize'){ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n');}); "#; std::fs::write(skill.join("scripts/server.cjs"), server).unwrap(); + let plugin = home.join(".codex/plugins/cache/fixture/probe/local"); + let plugin_data_name = format!("{:x}", Sha256::digest(b"fixture\0probe")); + let plugin_data = home + .join(".codex/plugins/data/agent-plugins") + .join(&plugin_data_name); + let plugin_receipt = workspace.join("codex-installed-consumed.json"); + std::fs::create_dir_all(plugin.join("scripts/node_modules/probe-dependency")).unwrap(); + std::fs::create_dir_all(&plugin_data).unwrap(); + std::fs::write(plugin.join("plugin.json"), r#"{"$schema":"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json","name":"probe"}"#).unwrap(); + std::fs::write( + plugin.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='closed-installed-plugin-dependency'", + ) + .unwrap(); + std::fs::write(plugin_data.join("nonce"), b"closed-hashed-plugin-resource").unwrap(); + std::fs::write(plugin.join("scripts/server.cjs"), r#"const fs=require('node:fs'),crypto=require('node:crypto'); +const digest=value=>crypto.createHash('sha256').update(value).digest('hex'); +require('node:readline').createInterface({input:process.stdin}).on('line',line=>{ + let message;try{message=JSON.parse(line)}catch{return}if(message.id===undefined)return; + let result={};if(message.method==='initialize'){ + const record={initialized:true,data:digest(fs.readFileSync(process.argv[2])),dependency:digest(require('probe-dependency')), + dataIdentity:process.env.PLUGIN_DATA.endsWith('/plugins/data/agent-plugins/'+process.argv[4]), + dataCwd:process.cwd()===process.env.PLUGIN_DATA,root:process.env.PLUGIN_ROOT==='/home/freshell/provider/.codex/plugins/cache/fixture/probe/local'}; + fs.writeFileSync(process.argv[3],JSON.stringify(record)); + result={protocolVersion:'2024-11-05',capabilities:{tools:{}},serverInfo:{name:'closed-installed-data-consumer',version:'1'}}; + }else if(message.method==='tools/list')result={tools:[]};else if(message.method==='resources/list')result={resources:[]};else if(message.method==='prompts/list')result={prompts:[]}; + process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n'); +}); +"#).unwrap(); + std::fs::write(plugin.join("mcp.json"), serde_json::to_vec(&json!({ + "$schema":"https://agent-plugins.org/schemas/1.0.0/mcp.schema.json", + "mcpServers":{"data-probe":{"type":"stdio","command":"node","args":["${PLUGIN_ROOT}/scripts/server.cjs","${PLUGIN_DATA}/nonce",plugin_receipt,plugin_data_name],"cwd":"${PLUGIN_DATA}"}} + })).unwrap()).unwrap(); std::fs::create_dir_all(home.join(".codex/rules")).unwrap(); std::fs::write(home.join(".codex/AGENTS.md"), b"closed-global-instructions").unwrap(); std::fs::write( @@ -1432,6 +1465,7 @@ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n'); ) .unwrap(); let config = format!("model='synthetic-model'\nmodel_provider='fixture'\nmodel_instructions_file='./user.md'\n[model_providers.fixture]\nname='Closed fixture'\nbase_url='http://127.0.0.1:9'\nwire_api='responses'\nrequires_openai_auth=false\n[projects.{}]\ntrust_level='trusted'\n[mcp_servers.probe]\ncommand='node'\nenv_vars=['FRESHELL_URL','FRESHELL_TOKEN','FRESHELL','CODEX_HOME']\nargs=[{},{}]\n", serde_json::to_string(workspace.to_str().unwrap()).unwrap(), serde_json::to_string(skill.join("scripts/server.cjs").to_str().unwrap()).unwrap(), serde_json::to_string(receipt.to_str().unwrap()).unwrap()); + let config = format!("{config}[features]\nplugins=true\n[plugins.\"probe@fixture\"]\n"); std::fs::write(home.join(".codex/config.toml"), &config).unwrap(); std::fs::write(home.join(".codex/user.md"), b"closed-user-input").unwrap(); let project_config = "model_instructions_file='./project.md'\n[agents.worker]\nconfig_file='./agents/worker.toml'\n"; @@ -1621,15 +1655,23 @@ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n'); ); } assert_eq!(consumed_terminal.unwrap(), expected); + let expected_plugin = json!({"initialized":true,"data":hash(b"closed-hashed-plugin-resource"),"dependency":hash(b"closed-installed-plugin-dependency"),"dataIdentity":true,"dataCwd":true,"root":true}); + assert_eq!( + consumed(&plugin_receipt) + .await + .expect("native AgentPlugin MCP must consume transported hashed data"), + expected_plugin + ); println!( "provider_launch_consumer {}", - json!({"provider":"codex","route":"terminal","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + json!({"provider":"codex","route":"terminal","digest":hash(serde_json::to_vec(&json!({"ordinary":expected,"installed":expected_plugin})).unwrap().as_slice())}) ); client .stop(SoulId::parse(terminal.soul_id).unwrap()) .await .unwrap(); std::fs::remove_file(&receipt).unwrap(); + std::fs::remove_file(&plugin_receipt).unwrap(); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( @@ -1653,9 +1695,15 @@ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n'); .await .unwrap(); assert_eq!(consumed(&receipt).await.unwrap(), expected); + assert_eq!( + consumed(&plugin_receipt) + .await + .expect("native fresh AgentPlugin MCP must consume transported hashed data"), + expected_plugin + ); println!( "provider_launch_consumer {}", - json!({"provider":"codex","route":"fresh","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + json!({"provider":"codex","route":"fresh","digest":hash(serde_json::to_vec(&json!({"ordinary":expected,"installed":expected_plugin})).unwrap().as_slice())}) ); assert_eq!( std::fs::read_to_string(home.join(".codex/config.toml")).unwrap(), @@ -1665,6 +1713,10 @@ process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n'); std::fs::read_to_string(workspace.join(".codex/config.toml")).unwrap(), project_config ); + assert_eq!( + std::fs::read(plugin_data.join("nonce")).unwrap(), + b"closed-hashed-plugin-resource" + ); for view in client.inventory().await.unwrap() { if view.desired_state == DesiredState::Running { client.stop(view.soul_id).await.unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 4a48ecf13..5e78df5a6 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -548,11 +548,25 @@ pub fn discover( "Codex selected installed plugin is unavailable; repair installation and retry", ); } - resources.push(active_plugin_version(&cache)?); - let data = manifest - .codex_home - .join("plugins/data") - .join(format!("{plugin}-{market}")); + let active = active_plugin_version(&cache)?; + let agent_plugin = is_agent_plugin(&active)?; + resources.push(active); + let data = if agent_plugin { + use sha2::{Digest, Sha256}; + let mut identity = Sha256::new(); + identity.update(market.as_bytes()); + identity.update([0]); + identity.update(plugin.as_bytes()); + manifest + .codex_home + .join("plugins/data/agent-plugins") + .join(format!("{:x}", identity.finalize())) + } else { + manifest + .codex_home + .join("plugins/data") + .join(format!("{plugin}-{market}")) + }; if exists(&data)? { resources.push(data); } @@ -622,6 +636,44 @@ fn active_plugin_version(cache: &Path) -> Result { )?; Ok(cache.join(selected)) } +fn is_agent_plugin(active: &Path) -> Result { + use std::io::Read; + let path = active.join("plugin.json"); + let metadata = match std::fs::symlink_metadata(&path) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(false), + Err(_) => { + return Err( + "Codex active plugin manifest is inaccessible; repair installation and retry", + ) + } + }; + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err("Codex active root plugin manifest must be a physical regular file; repair installation and retry"); + } + let mut bytes = Vec::new(); + std::fs::File::open(path) + .map_err(|_| "Codex active plugin manifest is unreadable; repair installation and retry")? + .take(MAX_SOURCE_VIEW_BYTES + 1) + .read_to_end(&mut bytes) + .map_err(|_| "Codex active plugin manifest read failed; repair installation and retry")?; + if bytes.len() as u64 > MAX_SOURCE_VIEW_BYTES { + return Err("Codex active plugin manifest source view exceeded"); + } + // Unrelated root JSON uses the native ordered Legacy manifest fallback. + let value = serde_json::from_slice::(&bytes).ok(); + let schema = value + .as_ref() + .and_then(|value| value.get("$schema")) + .and_then(serde_json::Value::as_str); + match schema { + Some("https://agent-plugins.org/schemas/1.0.0/plugin.schema.json") => Ok(true), + Some(schema) if schema.starts_with("https://agent-plugins.org/schemas/") => { + Err("Codex active AgentPlugin schema is unsupported; repair installation and retry") + } + _ => Ok(false), + } +} fn exists(path: &Path) -> Result { match std::fs::symlink_metadata(path) { Ok(_) => Ok(true), @@ -866,6 +918,207 @@ mod tests { } } + #[test] + fn managed_provider_codex_agent_plugin_data_keeps_hashed_native_identity() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let codex_home = home.join(".codex"); + let plugin = codex_home.join("plugins/cache/fixture/probe/local"); + let data = codex_home + .join("plugins/data/agent-plugins") + .join(format!("{:x}", Sha256::digest(b"fixture\0probe"))); + std::fs::create_dir_all(plugin.join("scripts/node_modules/probe-dependency")).unwrap(); + std::fs::create_dir_all(&data).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write( + codex_home.join("config.toml"), + "[plugins.\"probe@fixture\"]\n", + ) + .unwrap(); + std::fs::write(plugin.join("plugin.json"), r#"{"$schema":"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json","name":"probe"}"#).unwrap(); + std::fs::write( + plugin.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='agent-plugin-dependency'", + ) + .unwrap(); + std::fs::write(plugin.join("scripts/probe.cjs"), "process.stdout.write(require('node:fs').readFileSync(process.argv[2])+'|'+require('probe-dependency'))").unwrap(); + std::fs::write(data.join("nonce"), "hashed-data-resource").unwrap(); + #[cfg(unix)] + { + let unselected = codex_home.join("plugins/data/probe-fixture"); + std::fs::create_dir_all(&unselected).unwrap(); + std::os::unix::fs::symlink( + "absent-unselected-target", + unselected.join("unselected-resource"), + ) + .unwrap(); + } + let found = discover(CodexInputManifest { + version: 1, + original_home: home, + cwd, + codex_home: codex_home.clone(), + named_profile: None, + }) + .unwrap(); + let stage = owned.path().join("stage"); + let selections = found + .resource_roots + .iter() + .filter(|root| root.starts_with(codex_home.join("plugins"))) + .map(|root| CopySelection { + source: root.clone(), + destination_relative: root.strip_prefix(&codex_home).unwrap().into(), + }) + .collect::>(); + copy_environment(&selections, &stage, &[], CopyLimits::default()).unwrap(); + let output = std::process::Command::new("node") + .arg( + stage + .join(plugin.strip_prefix(&codex_home).unwrap()) + .join("scripts/probe.cjs"), + ) + .arg( + stage + .join(data.strip_prefix(&codex_home).unwrap()) + .join("nonce"), + ) + .output() + .unwrap(); + assert!( + output.status.success(), + "selected native AgentPlugin data must survive source transport" + ); + assert_eq!( + output.stdout, + b"hashed-data-resource|agent-plugin-dependency" + ); + assert_eq!( + std::fs::read(data.join("nonce")).unwrap(), + b"hashed-data-resource" + ); + } + + #[test] + fn managed_provider_codex_legacy_data_is_selected_by_native_manifest_family() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let codex_home = home.join(".codex"); + let plugin = codex_home.join("plugins/cache/fixture/probe/local"); + let named = codex_home.join("plugins/data/probe-fixture"); + let hashed = codex_home + .join("plugins/data/agent-plugins") + .join(format!("{:x}", Sha256::digest(b"fixture\0probe"))); + for directory in [ + plugin.join(".claude-plugin"), + plugin.join("scripts/node_modules/probe-dependency"), + named.clone(), + hashed.clone(), + cwd.clone(), + ] { + std::fs::create_dir_all(directory).unwrap(); + } + std::fs::write( + codex_home.join("config.toml"), + "[plugins.\"probe@fixture\"]\n", + ) + .unwrap(); + std::fs::write( + plugin.join(".claude-plugin/plugin.json"), + r#"{"name":"probe"}"#, + ) + .unwrap(); + std::fs::write( + plugin.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='legacy-dependency'", + ) + .unwrap(); + std::fs::write(plugin.join("scripts/probe.cjs"), "process.stdout.write(require('node:fs').readFileSync(process.argv[2])+'|'+require('probe-dependency'))").unwrap(); + std::fs::write(named.join("nonce"), "native-named-data").unwrap(); + #[cfg(unix)] + std::os::unix::fs::symlink( + "absent-unselected-target", + hashed.join("unselected-resource"), + ) + .unwrap(); + for (index, unrelated) in [ + r#"{"name":"unrelated"}"#, + "malformed-json", + r#"{"$schema":"https://unrelated.invalid/schema"}"#, + ] + .into_iter() + .enumerate() + { + std::fs::write(plugin.join("plugin.json"), unrelated).unwrap(); + let found = discover(CodexInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + codex_home: codex_home.clone(), + named_profile: None, + }) + .unwrap(); + let stage = owned.path().join(format!("legacy-stage-{index}")); + let selections = found + .resource_roots + .iter() + .filter(|root| root.starts_with(codex_home.join("plugins"))) + .map(|root| CopySelection { + source: root.clone(), + destination_relative: root.strip_prefix(&codex_home).unwrap().into(), + }) + .collect::>(); + copy_environment(&selections, &stage, &[], CopyLimits::default()).unwrap(); + let output = std::process::Command::new("node") + .arg( + stage + .join(plugin.strip_prefix(&codex_home).unwrap()) + .join("scripts/probe.cjs"), + ) + .arg( + stage + .join(named.strip_prefix(&codex_home).unwrap()) + .join("nonce"), + ) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"native-named-data|legacy-dependency"); + } + std::fs::write( + plugin.join("plugin.json"), + r#"{"$schema":"https://agent-plugins.org/schemas/2.0.0/PRIVATE_UNSUPPORTED_SCHEMA"}"#, + ) + .unwrap(); + let error = discover(CodexInputManifest { + version: 1, + original_home: home, + cwd, + codex_home, + named_profile: None, + }) + .err() + .expect( + "unsupported active AgentPlugin schema must not fall back to valid Legacy manifest", + ); + assert!(error.contains("schema")); + assert!(!error.contains("PRIVATE_UNSUPPORTED_SCHEMA")); + assert_eq!( + std::fs::read(named.join("nonce")).unwrap(), + b"native-named-data" + ); + } + #[test] fn managed_provider_codex_invalid_plugin_enabled_is_an_actionable_selection_error() { use freshell_platform::codex_inputs::CodexInputManifest; From f7ca49ec84d295308b1bb25b097fff411c5c097a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 15:19:31 -0700 Subject: [PATCH 096/493] fix: follow native Claude project resource discovery boundaries --- .../managed_mcp_capability/claude_stage.rs | 163 ++++++++++++++++-- .../src/managed_provider_launch_tests.rs | 134 ++++++++++++++ 2 files changed, 283 insertions(+), 14 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 0b544002b..51c5e5159 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -3,6 +3,17 @@ use super::ordinary_stage::{prepare_selected, Preparation}; use freshell_platform::claude_inputs::{ClaudeInputManifest, ClaudeSettingSource}; use std::path::{Path, PathBuf}; +const RESOURCE_DIRECTORIES: &[&str] = &[ + "skills", + "commands", + "agents", + "hooks", + "plugins", + "output-styles", + "workflows", + "routines", +]; + pub(super) fn prepare( home: &Path, cwd: &Path, @@ -60,15 +71,10 @@ pub(super) fn prepare( if user_enabled && optional(&user_settings)? { configurations.push((user_settings.clone(), canonical(&user_settings)?)); } - for relative in [ - "CLAUDE.md", - "rules", - "skills", - "commands", - "agents", - "hooks", - "plugins", - ] { + for relative in ["CLAUDE.md", "rules"] + .into_iter() + .chain(RESOURCE_DIRECTORIES.iter().copied()) + { if !user_enabled { continue; } @@ -103,11 +109,6 @@ pub(super) fn prepare( (local_enabled, "CLAUDE.local.md"), (project_enabled, ".claude/CLAUDE.md"), (project_enabled, ".claude/rules"), - (project_enabled, ".claude/skills"), - (project_enabled, ".claude/commands"), - (project_enabled, ".claude/agents"), - (project_enabled, ".claude/hooks"), - (project_enabled, ".claude/plugins"), ] { if !enabled { continue; @@ -124,6 +125,16 @@ pub(super) fn prepare( } } } + if project_enabled { + for base in project_resource_bases(&manifest)? { + for relative in RESOURCE_DIRECTORIES { + let logical = base.join(".claude").join(relative); + if optional(&logical)? { + resources.push(logical); + } + } + } + } for plugin in &manifest.plugin_dirs { canonical(plugin)?; // Explicit selections may never be silently omitted. resources.push(plugin.clone()); @@ -145,6 +156,35 @@ pub(super) fn prepare( ) } +// The native Markdown resource loader searches upward only until HOME or the +// nearest Git root. Memory has its own root-to-cwd search and must not share +// this stopping rule. Startup cwd is the same accepted cwd for these loaders. +fn project_resource_bases(manifest: &ClaudeInputManifest) -> Result, String> { + let mut bases = Vec::new(); + for base in manifest.cwd.ancestors() { + if base == manifest.original_home { + break; + } + bases.push(base.to_path_buf()); + match std::fs::metadata(base.join(".git")) { + Ok(metadata) if metadata.is_dir() || metadata.is_file() => break, + Ok(_) => {} + Err(error) + if matches!( + error.kind(), + std::io::ErrorKind::NotFound | std::io::ErrorKind::NotADirectory + ) => {} + Err(error) => { + return Err(format!( + "Claude project resource discovery {} metadata failed: {error}", + base.display() + )) + } + } + } + Ok(bases) +} + // Only append Freshell's two shipped hook groups. Native settings merging, // source precedence and all other configured values remain the provider's job. pub(super) fn stage_settings( @@ -245,6 +285,101 @@ mod tests { provider_environment::{copy_environment, CopyLimits}, }; + #[cfg(unix)] + #[test] + fn managed_provider_claude_project_resources_stop_at_git_and_home_without_losing_memory() { + for git_boundary in [true, false] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let boundary = if git_boundary { + home.join("project") + } else { + home.clone() + }; + let cwd = boundary.join("nested"); + let commands = cwd.join(".claude/commands"); + std::fs::create_dir_all(commands.join("node_modules/probe-dependency")).unwrap(); + if git_boundary { + std::fs::create_dir(boundary.join(".git")).unwrap(); + } + let unselected = if git_boundary { + home.clone() + } else { + boundary.clone() + }; + std::fs::create_dir_all(unselected.join(".claude")).unwrap(); + std::os::unix::fs::symlink( + "missing-unselected-commands", + unselected.join(".claude/commands"), + ) + .unwrap(); + std::fs::write(commands.join("resource"), "selected-command-resource").unwrap(); + std::fs::write( + commands.join("node_modules/probe-dependency/index.js"), + "module.exports='selected-installed-dependency'", + ) + .unwrap(); + std::fs::write(commands.join("run.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); + let ancestor_memory = unselected.join("CLAUDE.md"); + std::fs::write(&ancestor_memory, "@./memory-resource.md\n").unwrap(); + std::fs::write( + unselected.join("memory-resource.md"), + "separate-memory-resource", + ) + .unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project], + plugin_dirs: vec![], + }; + let selected = prepare(&home, &cwd, &cwd, Some(&manifest), &[]) + .expect("native-unselected command roots must not block preparation"); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + let mount = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == commands) + .unwrap(); + let output = std::process::Command::new("node") + .arg(stage.join(&mount.stage_relative_path).join("run.cjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + b"selected-command-resource:selected-installed-dependency" + ); + let imported = selected + .mounts + .iter() + .find(|mount| { + Path::new(&mount.logical_destination) + == manifest.runtime_path(&unselected.join("memory-resource.md")) + }) + .expect("the resource boundary must not truncate eager memory imports"); + assert_eq!( + std::fs::read(stage.join(&imported.stage_relative_path)).unwrap(), + b"separate-memory-resource" + ); + assert_eq!( + std::fs::read_to_string(&ancestor_memory).unwrap(), + "@./memory-resource.md\n" + ); + assert_eq!( + std::fs::read_link(unselected.join(".claude/commands")).unwrap(), + Path::new("missing-unselected-commands") + ); + } + } + #[test] fn managed_provider_claude_selected_user_rules_keep_external_imports() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index abbbe0ba4..55bf51d51 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -1280,6 +1280,47 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ std::fs::remove_file(&receipt).unwrap(); std::fs::remove_file(&instructions_receipt).unwrap(); std::fs::remove_file(&instructions_events).unwrap(); + // Native project command discovery stops at this repository root. A + // dangling ancestor command root is deliberately outside that selection. + std::fs::create_dir_all(fixture.join(".claude")).unwrap(); + std::os::unix::fs::symlink( + "missing-unselected-command-root", + fixture.join(".claude/commands"), + ) + .unwrap(); + let project_commands = workspace.join(".claude/commands"); + std::fs::create_dir_all(project_commands.join("scripts/node_modules/probe-dependency")) + .unwrap(); + std::fs::write(project_commands.join("closed-project-command.md"), "---\ndescription: selected project command transport\n---\nUse the adjacent synthetic resources.\n").unwrap(); + std::fs::write( + project_commands.join("scripts/resource"), + "closed-project-command-resource", + ) + .unwrap(); + std::fs::write( + project_commands.join("scripts/node_modules/probe-dependency/index.js"), + "module.exports='closed-project-command-dependency'", + ) + .unwrap(); + std::fs::write( + project_commands.join("scripts/run"), + "#!/bin/sh\ncat \"$(dirname \"$0\")/resource\"\n", + ) + .unwrap(); + std::fs::set_permissions( + project_commands.join("scripts/run"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + let project_receipt = workspace.join("claude-project-command-consumed.json"); + std::fs::write(project_commands.join("scripts/consume.cjs"), "const fs=require('node:fs'),crypto=require('node:crypto');const hash=v=>crypto.createHash('sha256').update(v).digest('hex');fs.writeFileSync(process.argv[2],JSON.stringify({resource:hash(require('node:child_process').execFileSync(__dirname+'/run')),dependency:hash(require('probe-dependency'))}));").unwrap(); + let mut project_settings: Value = serde_json::from_str(settings).unwrap(); + project_settings["hooks"] = json!({"SessionStart":[{"hooks":[{"type":"command","command":format!("node {} {}",serde_json::to_string(project_commands.join("scripts/consume.cjs").to_str().unwrap()).unwrap(),serde_json::to_string(project_receipt.to_str().unwrap()).unwrap()),"timeout":10}]}]}); + std::fs::write( + workspace.join(".claude/settings.json"), + serde_json::to_vec(&project_settings).unwrap(), + ) + .unwrap(); let (broadcast, _) = tokio::sync::broadcast::channel(256); let mut events = broadcast.subscribe(); std::env::set_var("HOME", &home); @@ -1322,6 +1363,11 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ json!({"nativeImport":true,"digest":hash(&imported_bytes)}) ); imported_graph(&instructions_events, &expected_imports).await; + assert_eq!(consumed(&project_receipt).await.expect("native selected project command hook must execute its complete resource and installed dependency"), json!({"resource":hash(b"closed-project-command-resource"),"dependency":hash(b"closed-project-command-dependency")})); + assert_eq!( + std::fs::read_link(fixture.join(".claude/commands")).unwrap(), + Path::new("missing-unselected-command-root") + ); println!( "provider_launch_consumer {}", json!({"provider":"claude","route":"fresh","digest":qualified_digest}) @@ -1334,11 +1380,99 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ std::fs::read_to_string(plugin.join("scripts/consume.cjs")).unwrap(), consumer ); + let catalog_project = fixture.join("catalog-project"); + let catalog_home = fixture.join("catalog-home"); + std::fs::create_dir_all(catalog_project.join(".git")).unwrap(); + std::fs::create_dir_all(&catalog_home).unwrap(); for view in client.inventory().await.unwrap() { if view.desired_state == DesiredState::Running { + let handle = registry + .active_handle_for_soul(view.soul_id.clone()) + .await + .unwrap(); + let inputs = handle + .fresh_agent() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap(); + let commands = inputs.resource_mounts.iter().find(|mount| Path::new(&mount.logical_destination) == project_commands).expect("accepted native project command root must be present in the complete preparation"); + // The hosted SDK intentionally defers publishing its prefetched + // catalog until system/init. With no model turn, consume the real + // native supportedCommands API separately over this accepted copy. + // Stop the hosted process first to avoid overlapping native workers. + freshell_platform::provider_environment::copy_environment( + &[freshell_platform::provider_environment::CopySelection { + source: control + .join("provider-roots") + .join(&inputs.generation_id) + .join(&commands.stage_relative_path), + destination_relative: ".claude/commands".into(), + }], + &catalog_project, + &[], + freshell_platform::provider_environment::CopyLimits::default(), + ) + .unwrap(); client.stop(view.soul_id).await.unwrap(); } } + let catalog_script = fixture.join("catalog-probe.mjs"); + std::fs::write(&catalog_script, r#"import {query} from '/opt/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk/sdk.mjs'; +const abort = new AbortController(); +async function* emptyPrompt() { await new Promise(resolve => abort.signal.addEventListener('abort', resolve, {once:true})); } +let q; +try { + q=query({prompt:emptyPrompt(),options:{abortController:abort,cwd:process.argv[2],settingSources:['project'],env:{PATH:process.env.PATH,TMPDIR:process.env.TMPDIR,HOME:process.argv[3],CLAUDE_CONFIG_DIR:process.argv[3]+'/.claude',ANTHROPIC_API_KEY:'synthetic-catalog-only',ANTHROPIC_BASE_URL:'http://127.0.0.1:9'}}}); + const commands=await q.supportedCommands();process.stdout.write(JSON.stringify({selected:commands.some(command=>command.name==='closed-project-command')})); +} catch(error) { + const message=String(error?.message||''); + const classes=['EROFS','EACCES','ENOENT','Native CLI binary','spawn node','process exited','AbortError'].filter(value=>message.includes(value)); + process.stdout.write(JSON.stringify({failed:true,classes}));process.exitCode=1; +} finally { q?.close();abort.abort(); } +"#).unwrap(); + let catalog_output = tokio::time::timeout( + Duration::from_secs(20), + tokio::process::Command::new("node") + .kill_on_drop(true) + .arg(&catalog_script) + .arg(&catalog_project) + .arg(&catalog_home) + .output(), + ) + .await + .expect("owned model-free native catalog probe deadline") + .unwrap(); + if !catalog_output.status.success() { + let stderr = String::from_utf8_lossy(&catalog_output.stderr); + let probe = serde_json::from_slice::(&catalog_output.stdout).ok(); + let classes: Vec<_> = [ + "ERR_MODULE_NOT_FOUND", + "EACCES", + "ENOENT", + "AbortError", + "close is not a function", + "uv_thread_create", + "SIGABRT", + "Claude Code process exited", + "Unexpected token", + ] + .into_iter() + .filter(|class| stderr.contains(class)) + .collect(); + println!( + "provider_catalog_diagnostic {}", + json!({"code":catalog_output.status.code(),"stderrBytes":catalog_output.stderr.len(),"stdoutBytes":catalog_output.stdout.len(),"classes":classes,"caughtClasses":probe.as_ref().and_then(|value|value.get("classes")),"selectedWritten":catalog_output.stdout.windows(b"\"selected\":true".len()).any(|bytes|bytes==b"\"selected\":true")}) + ); + } + assert!( + catalog_output.status.success(), + "genuine SDK catalog probe must complete without a model turn" + ); + assert_eq!(serde_json::from_slice::(&catalog_output.stdout).unwrap(), json!({"selected":true}), "the genuine default SDK must discover the selected project command through native bounded directory search"); assert!(instruction_events(&instructions_events) .iter() .all(|event| event["ignored"] != json!(true))); From 8fa432b71d8a80ddd6100fd65a07ee1fbfcca6f9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 15:36:00 -0700 Subject: [PATCH 097/493] fix: preserve Codex global instruction override selection --- .../src/managed_provider_references/codex.rs | 155 +++++++++++++++++- 1 file changed, 153 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 5e78df5a6..74595e22d 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -473,9 +473,8 @@ pub fn discover( .map(Path::to_path_buf) .collect::>(); directories.reverse(); - let mut resources = Vec::new(); + let mut resources = global_instruction_resources(&manifest.codex_home)?; for path in [ - manifest.codex_home.join("AGENTS.md"), manifest.codex_home.join("rules"), manifest.codex_home.join("skills"), manifest.original_home.join(".agents/skills"), @@ -586,6 +585,66 @@ pub fn discover( resource_roots: resources, }) } + +// Native global instructions try the override before the default and skip +// nonfiles/empty text. Keep attempted empty files so a refresh can replace an +// older nonempty override, but never make an unread fallback a copy dependency. +fn global_instruction_resources(home: &Path) -> Result, &'static str> { + use std::io::{BufReader, Read}; + let mut resources = Vec::new(); + let limit = freshell_platform::provider_environment::CopyLimits::default().max_file_bytes; + for filename in ["AGENTS.override.md", "AGENTS.md"] { + let path = home.join(filename); + if !exists(&path)? { + continue; + } + let metadata = std::fs::metadata(&path) + .map_err(|_| "Codex global AGENTS instruction source metadata is unavailable")?; + if !metadata.is_file() { + continue; + } + if metadata.len() > limit { + return Err("Codex global AGENTS instruction source exceeds ordinary per-file limit"); + } + let file = std::fs::File::open(&path) + .map_err(|_| "Codex global AGENTS instruction source is unreadable")?; + // Check trim/UTF-8-lossy emptiness with at most one Unicode scalar in + // memory. A non-whitespace prefix selects the file; the common copier + // subsequently enforces complete reads and all shared byte budgets. + let mut scalar = [0_u8; 4]; + let mut length = 0; + let mut nonempty = false; + for (index, byte) in BufReader::new(file.take(limit + 1)).bytes().enumerate() { + if index as u64 >= limit { + return Err( + "Codex global AGENTS instruction source grew beyond ordinary per-file limit", + ); + } + scalar[length] = + byte.map_err(|_| "Codex global AGENTS instruction source read failed")?; + length += 1; + match std::str::from_utf8(&scalar[..length]) { + Ok(text) => { + if text.chars().any(|character| !character.is_whitespace()) { + nonempty = true; + break; + } + length = 0; + } + Err(error) if error.error_len().is_some() || length == scalar.len() => { + nonempty = true; + break; + } + Err(_) => {} + } + } + resources.push(path); + if nonempty || length != 0 { + break; + } + } + Ok(resources) +} fn active_plugin_version(cache: &Path) -> Result { let entries = std::fs::read_dir(cache).map_err(|_| { "Codex selected installed plugin versions are unreadable; repair installation and retry" @@ -762,6 +821,98 @@ mod tests { } } + #[test] + fn managed_provider_codex_global_instruction_override_survives_transport_and_fallback() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + for (override_bytes, nonfile_override) in [ + (Some(b"selected-override".as_slice()), false), + (Some(b" \n\xe2\x80\x83".as_slice()), false), + (Some(b"\xff".as_slice()), false), + (None, false), + (None, true), + ] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let codex_home = home.join("selected-codex"); + std::fs::create_dir_all(&codex_home).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write(codex_home.join("AGENTS.md"), "selected-fallback").unwrap(); + if let Some(bytes) = override_bytes { + std::fs::write(codex_home.join("AGENTS.override.md"), bytes).unwrap(); + } else if nonfile_override { + // A native nonfile override must not activate or traverse an + // otherwise unused resource tree before default fallback. + std::fs::create_dir(codex_home.join("AGENTS.override.md")).unwrap(); + #[cfg(unix)] + std::os::unix::fs::symlink( + "unselected-missing-resource", + codex_home.join("AGENTS.override.md/unused"), + ) + .unwrap(); + } + let manifest = CodexInputManifest { + version: 1, + original_home: home, + cwd, + codex_home: codex_home.clone(), + named_profile: None, + }; + let found = discover(manifest.clone()).unwrap(); + let selections: Vec<_> = found + .resource_roots + .iter() + .filter(|path| path.starts_with(&codex_home)) + .map(|path| CopySelection { + source: path.clone(), + destination_relative: path.strip_prefix(&codex_home).unwrap().into(), + }) + .collect(); + let stage = owned.path().join("stage"); + copy_environment(&selections, &stage, &[], CopyLimits::default()).unwrap(); + let output = std::process::Command::new("node").args(["-e", "const fs=require('node:fs'),path=require('node:path');for(const name of ['AGENTS.override.md','AGENTS.md']){const file=path.join(process.argv[1],name);if(fs.existsSync(file)&&fs.statSync(file).isFile()){const raw=fs.readFileSync(file,'utf8').trim();if(raw){process.stdout.write(raw);break}}}"]).arg(&stage).output().unwrap(); + assert!(output.status.success()); + let expected = match override_bytes { + Some(bytes) if !String::from_utf8_lossy(bytes).trim().is_empty() => { + String::from_utf8_lossy(bytes).trim().to_string() + } + _ => "selected-fallback".into(), + }; + assert_eq!( + String::from_utf8(output.stdout).unwrap(), + expected, + "copied global instructions must retain the native override/fallback order" + ); + assert_eq!( + std::fs::read(codex_home.join("AGENTS.md")).unwrap(), + b"selected-fallback" + ); + #[cfg(unix)] + if override_bytes == Some(b"selected-override".as_slice()) { + std::fs::remove_file(codex_home.join("AGENTS.md")).unwrap(); + std::os::unix::fs::symlink( + "unselected-missing-fallback", + codex_home.join("AGENTS.md"), + ) + .unwrap(); + let found = discover(manifest).unwrap(); + let selections: Vec<_> = found + .resource_roots + .iter() + .filter(|path| path.starts_with(&codex_home)) + .map(|path| CopySelection { + source: path.clone(), + destination_relative: path.strip_prefix(&codex_home).unwrap().into(), + }) + .collect(); + copy_environment(&selections,&owned.path().join("next-stage"),&[],CopyLimits::default()).expect("an effective override must not make the native-unread default file a selected copy dependency"); + } + } + } + #[test] fn managed_provider_codex_omitted_plugin_enabled_retains_executable_dependencies() { use freshell_platform::{ From f0658f94273e177f0397f9ae649f06c5a7d48227 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 15:43:15 -0700 Subject: [PATCH 098/493] fix: copy native-selected Codex project rule files --- .../src/managed_provider_references/codex.rs | 180 +++++++++++++++++- 1 file changed, 178 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 74595e22d..9d00bea90 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -474,8 +474,8 @@ pub fn discover( .collect::>(); directories.reverse(); let mut resources = global_instruction_resources(&manifest.codex_home)?; + add_rule_resources(&manifest.codex_home.join("rules"), &mut resources)?; for path in [ - manifest.codex_home.join("rules"), manifest.codex_home.join("skills"), manifest.original_home.join(".agents/skills"), ] { @@ -500,11 +500,17 @@ pub fn discover( if exists(&skills)? { resources.push(skills); } + let enabled = trusted(&preceding, directory, root); + if enabled { + // Even a missing project TOML leaves an enabled config folder + // from which native Codex discovers immediate rule files. + add_rule_resources(&folder.join("rules"), &mut resources)?; + } let path = folder.join("config.toml"); if exists(&path)? { layers.push(SourceLayer { logical_path: path, - enabled: trusted(&preceding, directory, root), + enabled, kind: LayerKind::Project, }); } @@ -586,6 +592,48 @@ pub fn discover( }) } +fn add_rule_resources(root: &Path, resources: &mut Vec) -> Result<(), &'static str> { + let entries = match std::fs::read_dir(root) { + Ok(entries) => entries, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), + Err(_) => return Err("Codex selected rules directory is unreadable; repair it and retry"), + }; + let max_entries = freshell_platform::provider_environment::CopyLimits::default().max_entries; + let mut entry_count = 0usize; + let mut metadata_bytes = resources + .iter() + .try_fold(0usize, |bytes, path| { + bytes.checked_add(path.as_os_str().len()) + }) + .ok_or("Codex selected resource metadata overflow")?; + for entry in entries { + entry_count += 1; + if entry_count > max_entries { + return Err("Codex selected rules directory entry budget exceeded"); + } + let entry = entry.map_err(|_| "Codex selected rules directory enumeration failed")?; + let kind = entry + .file_type() + .map_err(|_| "Codex selected rules entry metadata unavailable")?; + let path = entry.path(); + // Native collection uses DirEntry type: symlink files are not selected, + // and directories are never recursively activated as policy files. + if kind.is_file() && path.extension() == Some(std::ffi::OsStr::new("rules")) { + if resources.len() >= MAX_SOURCE_EVENTS { + return Err("Codex selected resource event budget exceeded"); + } + metadata_bytes = metadata_bytes + .checked_add(path.as_os_str().len()) + .ok_or("Codex selected resource metadata overflow")?; + if metadata_bytes > MAX_SOURCE_EVENT_BYTES { + return Err("Codex selected resource metadata budget exceeded"); + } + resources.push(path); + } + } + Ok(()) +} + // Native global instructions try the override before the default and skip // nonfiles/empty text. Keep attempted empty files so a refresh can replace an // older nonempty override, but never make an unread fallback a copy dependency. @@ -821,6 +869,134 @@ mod tests { } } + #[test] + fn managed_provider_codex_rules_follow_project_trust_without_requiring_project_toml() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + for trust in ["trusted", "untrusted"] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let root = owned.path().join("project"); + let cwd = root.join("nested"); + let codex_home = home.join("selected-codex"); + std::fs::create_dir_all(&codex_home).unwrap(); + std::fs::create_dir_all(root.join(".git")).unwrap(); + std::fs::create_dir_all(root.join(".codex/rules")).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write( + codex_home.join("config.toml"), + format!( + "[projects.{}]\ntrust_level={trust:?}\n", + serde_json::to_string(root.to_str().unwrap()).unwrap() + ), + ) + .unwrap(); + let rule = root.join(".codex/rules/probe.rules"); + std::fs::write( + &rule, + "prefix_rule(pattern=[\"synthetic-never-run\"],decision=\"forbidden\")\n", + ) + .unwrap(); + let found = discover(CodexInputManifest { + version: 1, + original_home: home, + cwd, + codex_home, + named_profile: None, + }) + .unwrap(); + let stage = owned.path().join("stage"); + let selections = found + .resource_roots + .iter() + .map(|source| CopySelection { + source: source.clone(), + destination_relative: source.strip_prefix(owned.path()).unwrap().into(), + }) + .collect::>(); + copy_environment(&selections, &stage, &[], CopyLimits::default()).unwrap(); + // This executes the copied resource, not a discovery-list/text test. + // The genuine automatic native policy consumer is qualified separately. + let output = std::process::Command::new("node").args(["-e", "const fs=require('node:fs');const p=process.argv[1];process.stdout.write(fs.existsSync(p)?fs.readFileSync(p,'utf8'):'unselected')"]) + .arg(stage.join("project/.codex/rules/probe.rules")).output().unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + if trust == "trusted" { + std::fs::read(&rule).unwrap() + } else { + b"unselected".to_vec() + } + ); + assert!(!root.join(".codex/config.toml").exists()); + assert_eq!( + std::fs::read(&rule).unwrap(), + b"prefix_rule(pattern=[\"synthetic-never-run\"],decision=\"forbidden\")\n" + ); + } + } + + #[test] + fn managed_provider_codex_rules_copy_only_immediate_regular_rule_files() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("project"); + let codex_home = home.join(".codex"); + let rules = codex_home.join("rules"); + std::fs::create_dir_all(rules.join("nested.rules")).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::write(rules.join("probe.rules"), "selected-global-policy").unwrap(); + #[cfg(unix)] + { + std::os::unix::fs::symlink("unselected-missing", rules.join("ignored.txt")).unwrap(); + std::os::unix::fs::symlink("unselected-missing", rules.join("linked.rules")).unwrap(); + std::os::unix::fs::symlink( + "unselected-missing", + rules.join("nested.rules/inner.rules"), + ) + .unwrap(); + } + let found = discover(CodexInputManifest { + version: 1, + original_home: home, + cwd, + codex_home: codex_home.clone(), + named_profile: None, + }) + .unwrap(); + let stage = owned.path().join("stage"); + let selections = found + .resource_roots + .iter() + .map(|source| CopySelection { + source: source.clone(), + destination_relative: source.strip_prefix(&codex_home).unwrap().into(), + }) + .collect::>(); + copy_environment(&selections,&stage,&[],CopyLimits::default()).expect("native-unselected nested, non-rule, and symlink entries must not become copy dependencies"); + let output = std::process::Command::new("node") + .args([ + "-e", + "process.stdout.write(require('node:fs').readFileSync(process.argv[1]))", + ]) + .arg(stage.join("rules/probe.rules")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"selected-global-policy"); + #[cfg(unix)] + assert_eq!( + std::fs::read_link(rules.join("linked.rules")).unwrap(), + Path::new("unselected-missing") + ); + } + #[test] fn managed_provider_codex_global_instruction_override_survives_transport_and_fallback() { use freshell_platform::{ From 72d4b968a05838bab27299b15c8e5fd7ddc8cef1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 16:00:43 -0700 Subject: [PATCH 099/493] test: qualify native Codex automatic rule selection --- .../src/managed_mcp_capability.rs | 339 +++++++++++++++++- 1 file changed, 337 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index f80e8b829..963614495 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4456,6 +4456,340 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .is_dir()); } + #[cfg(feature = "managed-provider-qualification")] + async fn stop_codex_source_child(child: &mut tokio::process::Child) { + let group = child.id().expect("owned native child PID") as libc::pid_t; + assert_eq!( + unsafe { libc::getpgid(group) }, + group, + "only stop the fixture's dedicated native process group" + ); + assert_eq!(unsafe { libc::kill(-group, libc::SIGTERM) }, 0); + match tokio::time::timeout(std::time::Duration::from_secs(5), child.wait()).await { + Ok(result) => { + result.unwrap(); + } + Err(_) => { + assert_eq!(unsafe { libc::kill(-group, libc::SIGKILL) }, 0); + child.wait().await.unwrap(); + } + } + tokio::time::timeout(std::time::Duration::from_secs(5),async { + loop { + // The disposable test executable is PID 1. If the installed + // Node launcher exits first, its native child is adopted here. + // Reap only this owned group, never unrelated children. + let mut status=0; + let reaped=unsafe {libc::waitpid(-group,&mut status,libc::WNOHANG)}; + if reaped>0 { continue; } + if reaped<0 { + assert!(matches!(std::io::Error::last_os_error().raw_os_error(),Some(libc::ECHILD)|Some(libc::EINTR))); + } + if unsafe {libc::kill(-group,0)}==-1 { + assert_eq!(std::io::Error::last_os_error().raw_os_error(),Some(libc::ESRCH)); + break; + } + tokio::time::sleep(std::time::Duration::from_millis(10)).await; + } + }).await.expect("native CLI launcher and its owned native descendants must all exit before the next consumer"); + } + + #[cfg(feature = "managed-provider-qualification")] + async fn qualify_codex_automatic_rules(base: &Path) { + use freshell_platform::provider_environment::{ + copy_environment, CopyLimits, CopySelection, + }; + use freshell_runtime_protocol::{ + ProviderLaunchContext, ProviderPreparation, ProviderResourceKind, + }; + use std::process::Stdio; + use tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt}; + let physical = Path::new("/home/freshell/provider"); + for (case, trust, project_config, poison) in [ + ("trusted-config", Some("trusted"), true, "regular"), + ("trusted-rules-only", Some("trusted"), false, "regular"), + ("untrusted", Some("untrusted"), false, "regular"), + ("unknown", None, false, "regular"), + ("ignored-shapes", Some("trusted"), false, "ignored"), + ("optional-missing", Some("trusted"), false, "missing"), + ] { + let owned = base.join(format!("automatic-rules-{case}")); + let original = owned.join("original-home"); + let workspace = owned.join("workspace"); + let project = workspace.join(".codex"); + let rule = project.join("rules/project-auto-probe.rules"); + std::fs::create_dir_all(original.join(".codex")).unwrap(); + std::fs::create_dir_all(workspace.join(".git")).unwrap(); + std::fs::create_dir_all(&project).unwrap(); + let config = trust + .map(|trust| { + format!( + "[projects.{}]\ntrust_level={trust:?}\n", + serde_json::to_string(workspace.to_str().unwrap()).unwrap() + ) + }) + .unwrap_or_default(); + std::fs::write(original.join(".codex/config.toml"), &config).unwrap(); + if project_config { + std::fs::write( + project.join("config.toml"), + "# synthetic native project layer\n", + ) + .unwrap(); + } + if poison != "missing" { + std::fs::create_dir_all(project.join("rules")).unwrap(); + if poison == "regular" { + std::fs::write(&rule, "prefix_rule(").unwrap(); + } else { + std::fs::create_dir_all(project.join("rules/nested")).unwrap(); + std::fs::write(project.join("rules/nested/ignored.rules"), "prefix_rule(") + .unwrap(); + std::fs::write(project.join("rules/ignored.txt"), "prefix_rule(").unwrap(); + std::os::unix::fs::symlink("ignored.txt", &rule).unwrap(); + } + } + for outside in [ + owned.join(".codex/rules/above.rules"), + workspace.join("below/.codex/rules/below.rules"), + ] { + std::fs::create_dir_all(outside.parent().unwrap()).unwrap(); + std::fs::write(outside, "prefix_rule(").unwrap(); + } + let mcp = crate::managed_provider_bootstrap::managed_codex_mcp_args().unwrap(); + let context = ProviderLaunchContext { + config: vec![], + mcp_capability: None, + prepared_inputs: None, + preparation: ProviderPreparation::Codex { + tui_args: mcp.clone(), + sidecar_args: mcp, + }, + }; + let directory = owned.join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let grant = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", &original, &[])), + OpencodeEphemeralInput { + launch_context: Some(&context), + cwd: Some(&workspace), + workspace: Some(&workspace), + ..Default::default() + }, + None, + ) + .unwrap(); + let prepared = crate::fresh_agent_proxy::context_for_fresh_provider(Some( + read_prepared_context(&directory, &grant).unwrap(), + )) + .unwrap(); + let inputs = prepared.prepared_inputs.as_ref().unwrap(); + let stage = staged_provider_root_path(&directory, &grant.grant_id).unwrap(); + let selections = inputs + .resource_mounts + .iter() + .map(|unit| CopySelection { + source: stage.join(&unit.stage_relative_path), + destination_relative: unit.provider_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&selections, physical, &[], CopyLimits::default()).unwrap(); + let archived_home = owned.join("archived-home"); + let archived_project = owned.join("archived-project"); + std::fs::rename(&original, &archived_home).unwrap(); + std::fs::rename(&project, &archived_project).unwrap(); + std::fs::create_dir_all(&project).unwrap(); + for bind in &inputs.config_binds { + let destination = Path::new(&bind.effective_destination); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + // This is the disposable fixture's own prior HOME projection, + // never a real provider home or another native worker's input. + if std::fs::symlink_metadata(destination).is_ok() { + std::fs::remove_file(destination).unwrap(); + } + std::os::unix::fs::symlink(stage.join(&bind.source_relative_path), destination) + .unwrap(); + } + for unit in &inputs.resource_mounts { + let destination = Path::new(&unit.effective_destination); + assert_eq!(unit.kind, ProviderResourceKind::File); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + // A real exact file bind has a regular inode. Native rule + // discovery deliberately excludes symlink-file adapters. + copy_environment( + &[CopySelection { + source: physical.join(&unit.provider_relative_path), + destination_relative: destination.file_name().unwrap().into(), + }], + destination.parent().unwrap(), + &[], + CopyLimits::default(), + ) + .unwrap(); + } + if trust != Some("trusted") { + // Preserve negative control topology in the native namespace. + // Only positive cases may prove accepted transport; their + // selected originals remain archived and inaccessible here. + copy_environment( + &[CopySelection { + source: archived_project.clone(), + destination_relative: ".codex".into(), + }], + &workspace, + &[], + CopyLimits::default(), + ) + .unwrap(); + } else if poison == "ignored" { + // The ordinary copier follows resource symlinks; restoring + // these native-unselected controls must preserve their type. + std::fs::create_dir_all(project.join("rules/nested")).unwrap(); + for relative in ["rules/nested/ignored.rules", "rules/ignored.txt"] { + std::fs::copy(archived_project.join(relative), project.join(relative)).unwrap(); + } + std::os::unix::fs::symlink("ignored.txt", &rule).unwrap(); + assert_eq!( + std::fs::read_link(&rule).unwrap(), + std::fs::read_link(archived_project.join("rules/project-auto-probe.rules")) + .unwrap() + ); + } + let mut child = tokio::process::Command::new("/usr/local/bin/codex") + .arg("app-server") + .process_group(0) + .current_dir(&workspace) + .env("HOME", physical) + .env("CODEX_HOME", physical.join(".codex")) + .env_remove("OPENAI_API_KEY") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .kill_on_drop(true) + .spawn() + .unwrap(); + let mut stdin = child.stdin.take().unwrap(); + let mut stdout = tokio::io::BufReader::new(child.stdout.take().unwrap()).lines(); + let mut stderr = child.stderr.take().unwrap(); + let mut warnings = Vec::new(); + let mut layers = None; + for (id, method, params) in [ + ( + 1, + "initialize", + serde_json::json!({"clientInfo":{"name":"freshell-fixture","version":"1"},"capabilities":{"experimentalApi":true}}), + ), + ( + 2, + "config/read", + serde_json::json!({"includeLayers":true,"cwd":workspace}), + ), + ] { + stdin + .write_all( + format!( + "{}\n", + serde_json::json!({"id":id,"method":method,"params":params}) + ) + .as_bytes(), + ) + .await + .unwrap(); + let result = tokio::time::timeout(std::time::Duration::from_secs(25), async { + loop { + let Some(line) = stdout.next_line().await.unwrap() else { + let status=child.wait().await.unwrap(); + let mut bytes=Vec::new(); + (&mut stderr).take(64*1024).read_to_end(&mut bytes).await.unwrap(); + let text=String::from_utf8_lossy(&bytes); + let classes=["Permission denied","No such file","MCP","config.toml","model_instructions_file","profile","unknown variant","failed to load configuration","invalid type","ENOENT","EACCES","panic","Failed to load","Error loading","rules","Parse error","not found","exists","agents","symlink","directory","Resource temporarily unavailable","OS can't spawn worker thread","WouldBlock","failed to spawn thread"].into_iter().filter(|class|text.contains(class)).collect::>(); + println!("provider_source_diagnostic {}",serde_json::json!({"provider":"codex","operation":"automatic_rule_startup_failure","case":case,"exitCode":status.code(),"stderrBytes":bytes.len(),"classes":classes})); + panic!("native automatic policy consumer ended before barrier"); + }; + assert!( + line.len() <= 1024 * 1024, + "finite native source fixture response limit" + ); + let response: serde_json::Value = serde_json::from_str(&line).unwrap(); + if response["method"] == "configWarning" { + assert!( + warnings.len() < 16, + "finite native source fixture warning limit" + ); + warnings.push(response["params"].clone()); + } + if response["id"].as_u64() == Some(id) { + assert!( + response.get("error").is_none(), + "native automatic policy request failed" + ); + break response["result"].clone(); + } + } + }) + .await + .expect("native automatic policy response deadline"); + if id == 2 { + layers = Some(result["layers"].clone()); + } + } + // config/read is the ordered barrier after initialize warnings; + // no sleep or quiet-period inference establishes negative cases. + stop_codex_source_child(&mut child).await; + let layers = layers.unwrap(); + let layer = layers + .as_array() + .unwrap() + .iter() + .find(|layer| { + layer["name"]["type"] == "project" + && layer["name"]["dotCodexFolder"].as_str() == project.to_str() + }) + .expect("native project folder layer must survive missing config.toml"); + assert_eq!(layer["disabledReason"].is_null(), trust == Some("trusted")); + let policy = warnings + .iter() + .filter(|warning| { + warning["summary"] == "Error parsing rules; custom rules not applied." + }) + .collect::>(); + let selected = trust == Some("trusted") && poison == "regular"; + assert_eq!( + policy.len(), + usize::from(selected), + "automatic native rule selection case {case}" + ); + if selected { + assert_eq!(policy[0]["path"].as_str(), rule.to_str()); + assert!(policy[0]["details"] + .as_str() + .unwrap() + .contains("Parse error")); + assert_eq!( + std::fs::read(archived_project.join("rules/project-auto-probe.rules")).unwrap(), + b"prefix_rule(" + ); + } + assert_eq!( + std::fs::read_to_string(archived_home.join(".codex/config.toml")).unwrap(), + config + ); + assert_eq!( + archived_project.join("config.toml").exists(), + project_config + ); + println!( + "provider_source_diagnostic {}", + serde_json::json!({"provider":"codex","operation":"automatic_rule_selection","case":case,"selected":selected,"qualified":true}) + ); + } + } + #[cfg(feature = "managed-provider-qualification")] #[tokio::test] async fn managed_provider_codex_native_typed_inputs_survive_transport() { @@ -4643,6 +4977,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); let mut child = tokio::process::Command::new("/usr/local/bin/codex") .arg("app-server") + .process_group(0) .args(match &context.preparation { ProviderPreparation::Codex { sidecar_args, .. } => sidecar_args.clone(), _ => panic!("Codex context missing"), @@ -4759,8 +5094,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let installed_path = Path::new(installed["path"].as_str().unwrap()); // The native catalog is complete. Join it before the independent // executable consumer, as for the existing project resource below. - child.kill().await.unwrap(); - child.wait().await.unwrap(); + stop_codex_source_child(&mut child).await; let pids_before = std::fs::read_to_string("/sys/fs/cgroup/pids.current") .ok() .and_then(|value| value.trim().parse::().ok()); @@ -4810,6 +5144,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::read_to_string(archived.join(".codex/config.toml")).unwrap(), config ); + qualify_codex_automatic_rules(owned.path()).await; for (case, bytes) in [ ( "native-config-read", From bbf47e60e7dee4b12d165694f2e302ede40da030 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 16:23:51 -0700 Subject: [PATCH 100/493] fix: match native Codex canonical and Git trust selection --- .../src/managed_mcp_capability.rs | 137 ++++++++++- .../src/managed_provider_references/codex.rs | 230 +++++++++++++++--- 2 files changed, 325 insertions(+), 42 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 963614495..9cef070c5 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4505,13 +4505,67 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ use std::process::Stdio; use tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt}; let physical = Path::new("/home/freshell/provider"); - for (case, trust, project_config, poison) in [ - ("trusted-config", Some("trusted"), true, "regular"), - ("trusted-rules-only", Some("trusted"), false, "regular"), - ("untrusted", Some("untrusted"), false, "regular"), - ("unknown", None, false, "regular"), - ("ignored-shapes", Some("trusted"), false, "ignored"), - ("optional-missing", Some("trusted"), false, "missing"), + for (case, trust, project_config, poison, topology) in [ + ("trusted-config", Some("trusted"), true, "regular", "plain"), + ( + "trusted-rules-only", + Some("trusted"), + false, + "regular", + "plain", + ), + ("untrusted", Some("untrusted"), false, "regular", "plain"), + ("unknown", None, false, "regular", "plain"), + ("ignored-shapes", Some("trusted"), false, "ignored", "plain"), + ( + "optional-missing", + Some("trusted"), + false, + "missing", + "plain", + ), + ( + "canonical-first-positive", + Some("trusted"), + false, + "regular", + "alias", + ), + ( + "canonical-first-negative", + Some("untrusted"), + false, + "regular", + "alias", + ), + ( + "linked-standard", + Some("trusted"), + false, + "regular", + "linked", + ), + ( + "linked-custom-common", + Some("trusted"), + false, + "regular", + "custom-common", + ), + ( + "linked-unrelated-negative", + Some("untrusted"), + false, + "regular", + "unrelated", + ), + ( + "git-above-project-marker", + Some("trusted"), + false, + "regular", + "ancestor", + ), ] { let owned = base.join(format!("automatic-rules-{case}")); let original = owned.join("original-home"); @@ -4519,16 +4573,72 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let project = workspace.join(".codex"); let rule = project.join("rules/project-auto-probe.rules"); std::fs::create_dir_all(original.join(".codex")).unwrap(); - std::fs::create_dir_all(workspace.join(".git")).unwrap(); + let canonical_workspace = owned.join("canonical-workspace"); + if topology == "alias" { + std::fs::create_dir_all(&canonical_workspace).unwrap(); + std::os::unix::fs::symlink(&canonical_workspace, &workspace).unwrap(); + } + let main = owned.join("main-checkout"); + if topology == "ancestor" { + std::fs::create_dir_all(owned.join(".git")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(workspace.join(".native-project"), "selected marker").unwrap(); + } else if matches!(topology, "linked" | "custom-common" | "unrelated") { + let gitdir = match topology { + "linked" => main.join(".git/worktrees/linked"), + "custom-common" => main.join("native-git-storage/worktrees/linked"), + _ => main.join(".git/unrelated/linked"), + }; + std::fs::create_dir_all(&gitdir).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + workspace.join(".git"), + format!("gitdir:\t{}\n", gitdir.display()), + ) + .unwrap(); + } else { + std::fs::create_dir_all(workspace.join(".git")).unwrap(); + } std::fs::create_dir_all(&project).unwrap(); - let config = trust + let trust_path = if topology == "ancestor" { + &owned + } else if topology == "alias" { + &canonical_workspace + } else if matches!(topology, "linked" | "custom-common" | "unrelated") { + &main + } else { + &workspace + }; + let mut config = trust .map(|trust| { format!( "[projects.{}]\ntrust_level={trust:?}\n", - serde_json::to_string(workspace.to_str().unwrap()).unwrap() + serde_json::to_string(trust_path.to_str().unwrap()).unwrap() ) }) .unwrap_or_default(); + if topology == "ancestor" { + config = format!("project_root_markers=[\".native-project\"]\n{config}"); + } + if topology == "alias" { + let logical_trust = if trust == Some("trusted") { + "untrusted" + } else { + "trusted" + }; + config.push_str(&format!( + "[projects.{}]\ntrust_level={logical_trust:?}\n", + serde_json::to_string(workspace.to_str().unwrap()).unwrap() + )); + } + // The unrelated gitdir negative deliberately has a trusted main + // record; native rejects its hierarchy before considering that key. + if topology == "unrelated" { + config = format!( + "[projects.{}]\ntrust_level=\"trusted\"\n", + serde_json::to_string(main.to_str().unwrap()).unwrap() + ); + } std::fs::write(original.join(".codex/config.toml"), &config).unwrap(); if project_config { std::fs::write( @@ -4641,7 +4751,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ source: archived_project.clone(), destination_relative: ".codex".into(), }], - &workspace, + &std::fs::canonicalize(&workspace).unwrap(), &[], CopyLimits::default(), ) @@ -4765,7 +4875,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ "automatic native rule selection case {case}" ); if selected { - assert_eq!(policy[0]["path"].as_str(), rule.to_str()); + let native_rule = std::fs::canonicalize(&workspace) + .unwrap() + .join(".codex/rules/project-auto-probe.rules"); + assert_eq!(policy[0]["path"].as_str(), native_rule.to_str()); assert!(policy[0]["details"] .as_str() .unwrap() diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 9d00bea90..09ca9479e 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -474,6 +474,8 @@ pub fn discover( .collect::>(); directories.reverse(); let mut resources = global_instruction_resources(&manifest.codex_home)?; + // Git trust fallback is independent of configurable project-root markers. + let git_root = git_trust_root(&manifest.cwd)?; add_rule_resources(&manifest.codex_home.join("rules"), &mut resources)?; for path in [ manifest.codex_home.join("skills"), @@ -500,7 +502,7 @@ pub fn discover( if exists(&skills)? { resources.push(skills); } - let enabled = trusted(&preceding, directory, root); + let enabled = trusted(&preceding, directory, root, git_root.as_deref()); if enabled { // Even a missing project TOML leaves an enabled config folder // from which native Codex discovers immediate rule files. @@ -816,39 +818,94 @@ fn merge(target: &mut toml::Value, value: toml::Value) { *target = value; } } -fn trusted(value: &toml::Value, directory: &Path, root: &Path) -> bool { - let Some(projects) = value.get("projects") else { - return false; +fn git_trust_root(cwd: &Path) -> Result, &'static str> { + use std::io::Read; + let Some(root) = cwd.ancestors().find(|path| path.join(".git").exists()) else { + return Ok(None); }; - let mut paths = vec![directory.to_path_buf()]; - if let Ok(canonical) = std::fs::canonicalize(directory) { - paths.push(canonical); + let marker = root.join(".git"); + if std::fs::metadata(&marker).is_ok_and(|metadata| metadata.is_dir()) { + return Ok(Some(root.into())); } - paths.push(root.to_path_buf()); - if let Ok(canonical) = std::fs::canonicalize(root) { - paths.push(canonical); + // Native lookup ignores unreadable/invalid Git markers. Bound the metadata + // read with the existing source-view profile before any full allocation. + let Ok(file) = std::fs::File::open(&marker) else { + return Ok(None); + }; + let mut bytes = Vec::new(); + if file + .take(MAX_SOURCE_VIEW_BYTES + 1) + .read_to_end(&mut bytes) + .is_err() + { + return Ok(None); } - // Linked worktrees share the root checkout's trust selection. - if let Ok(gitfile) = std::fs::read_to_string(root.join(".git")) { - if let Some(gitdir) = gitfile.trim().strip_prefix("gitdir: ") { - let path = normalize(&root.join(gitdir)); - if let Some(checkout) = path - .ancestors() - .find(|path| path.file_name() == Some(std::ffi::OsStr::new(".git"))) - .and_then(Path::parent) - { - paths.push(checkout.to_path_buf()); - } - } + if bytes.len() as u64 > MAX_SOURCE_VIEW_BYTES { + return Err("Codex Git trust marker exceeds source-view limit; repair marker and retry"); } - for path in paths { - if let Some(trust) = path - .to_str() - .and_then(|key| projects.get(key)) - .and_then(|project| project.get("trust_level")) - .and_then(toml::Value::as_str) - { - return trust == "trusted"; + let Some(gitdir) = std::str::from_utf8(&bytes) + .ok() + .and_then(|raw| raw.trim().strip_prefix("gitdir:")) + .map(str::trim) + .filter(|path| !path.is_empty()) + else { + return Ok(None); + }; + let path = normalize(&root.join(gitdir)); + let Some(worktrees) = path + .parent() + .filter(|path| path.file_name() == Some(std::ffi::OsStr::new("worktrees"))) + else { + return Ok(None); + }; + Ok(worktrees + .parent() + .and_then(Path::parent) + .map(Path::to_path_buf)) +} + +fn trust_lookup_key(path: &Path) -> String { + let key = path.to_string_lossy().into_owned(); + if cfg!(windows) { + key.to_ascii_lowercase() + } else { + key + } +} + +fn project_trust<'a>(projects: &'a toml::Value, key: &str) -> Option<&'a str> { + projects + .get(key) + .or_else(|| { + if !cfg!(windows) { + return None; + } + projects + .as_table()? + .iter() + .filter(|(candidate, _)| candidate.to_ascii_lowercase() == key) + .min_by_key(|(candidate, _)| *candidate) + .map(|(_, value)| value) + })? + .get("trust_level")? + .as_str() +} + +fn trusted(value: &toml::Value, directory: &Path, root: &Path, git_root: Option<&Path>) -> bool { + let Some(projects) = value.get("projects") else { + return false; + }; + for path in [Some(directory), Some(root), git_root] + .into_iter() + .flatten() + { + let canonical = std::fs::canonicalize(path).unwrap_or_else(|_| path.into()); + // Native gives canonical keys precedence, then logical keys, at each + // level. An explicit untrusted match ends the search at that level. + for key in [trust_lookup_key(&canonical), trust_lookup_key(path)] { + if let Some(trust) = project_trust(projects, &key) { + return trust == "trusted"; + } } } false @@ -869,6 +926,119 @@ mod tests { } } + #[cfg(unix)] + #[test] + fn managed_provider_codex_trust_selects_native_canonical_and_git_fallback_resources() { + use freshell_platform::{ + codex_inputs::CodexInputManifest, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + for (case, canonical_trust, logical_trust, git_shape, selected) in [ + ( + "canonical-first-positive", + Some("trusted"), + Some("untrusted"), + "directory", + true, + ), + ( + "canonical-first-negative", + Some("untrusted"), + Some("trusted"), + "directory", + false, + ), + ("logical-fallback", None, Some("trusted"), "directory", true), + ("linked-standard", None, None, "worktrees", true), + ("linked-custom-common", None, None, "custom-common", true), + ("unrelated-git-dir", None, None, "unrelated", false), + ("git-above-project-marker", None, None, "ancestor", true), + ] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let canonical = owned.path().join("canonical-project"); + let logical = owned.path().join("logical-project"); + let main = owned.path().join("main-checkout"); + let codex_home = home.join(".codex"); + std::fs::create_dir_all(&codex_home).unwrap(); + std::fs::create_dir_all(canonical.join("nested")).unwrap(); + std::fs::create_dir_all(canonical.join(".codex/rules")).unwrap(); + std::os::unix::fs::symlink(&canonical, &logical).unwrap(); + let mut config = String::new(); + for (path, trust) in [(&canonical, canonical_trust), (&logical, logical_trust)] { + if let Some(trust) = trust { + config.push_str(&format!( + "[projects.{}]\ntrust_level={trust:?}\n", + serde_json::to_string(path.to_str().unwrap()).unwrap() + )); + } + } + if git_shape == "ancestor" { + std::fs::create_dir_all(owned.path().join(".git")).unwrap(); + std::fs::write(canonical.join(".native-project"), "selected marker").unwrap(); + config.push_str(&format!("project_root_markers=[\".native-project\"]\n[projects.{}]\ntrust_level=\"trusted\"\n",serde_json::to_string(owned.path().to_str().unwrap()).unwrap())); + } else if git_shape == "directory" { + std::fs::create_dir_all(canonical.join(".git")).unwrap(); + } else { + let gitdir = match git_shape { + "worktrees" => main.join(".git/worktrees/linked"), + "custom-common" => main.join("native-git-storage/worktrees/linked"), + _ => main.join(".git/unrelated/linked"), + }; + std::fs::create_dir_all(&gitdir).unwrap(); + // Native accepts whitespace after gitdir:, independent of .git basename. + std::fs::write( + canonical.join(".git"), + format!("gitdir:\t{}\n", gitdir.display()), + ) + .unwrap(); + config.push_str(&format!( + "[projects.{}]\ntrust_level=\"trusted\"\n", + serde_json::to_string(main.to_str().unwrap()).unwrap() + )); + } + std::fs::write(codex_home.join("config.toml"), &config).unwrap(); + let rule = canonical.join(".codex/rules/probe.rules"); + std::fs::write(&rule, case).unwrap(); + let found = discover(CodexInputManifest { + version: 1, + original_home: home, + cwd: logical.join("nested"), + codex_home, + named_profile: None, + }) + .unwrap(); + let stage = owned.path().join("stage"); + let selections = found + .resource_roots + .iter() + .map(|source| CopySelection { + source: source.clone(), + destination_relative: source.strip_prefix(&logical).unwrap().into(), + }) + .collect::>(); + copy_environment(&selections, &stage, &[], CopyLimits::default()).unwrap(); + let output = std::process::Command::new("node") + .args(["-e", "const fs=require('node:fs');const p=process.argv[1];process.stdout.write(fs.existsSync(p)?fs.readFileSync(p,'utf8'):'unselected')"]) + .arg(stage.join(".codex/rules/probe.rules")).output().unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + if selected { + case.as_bytes() + } else { + b"unselected" + }, + "native selection {case}" + ); + assert_eq!(std::fs::read(&rule).unwrap(), case.as_bytes()); + assert_eq!( + std::fs::read_to_string(found.manifest.codex_home.join("config.toml")).unwrap(), + config + ); + } + } + #[test] fn managed_provider_codex_rules_follow_project_trust_without_requiring_project_toml() { use freshell_platform::{ From 7172ea81d4049b858567b09f50a9de218d13b4b2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 16:30:49 -0700 Subject: [PATCH 101/493] test: classify global prepared config by its native source role --- crates/freshell-server/src/managed_mcp_capability.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 9cef070c5..a5a367457 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -7695,7 +7695,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ logical_path: config.clone(), canonical_source: config.clone(), lane: SourceLane::Main, - role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, + role: freshell_platform::opencode_recipe::DiscoveryRole::Global, }], &workspace, &home, From 2b4ddea86b66c191ec2ffd096c622a426e3c9352 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 17:43:34 -0700 Subject: [PATCH 102/493] fix Claude linked source selection and bounded native metadata views --- crates/freshell-platform/src/claude_inputs.rs | 14 + .../src/claude_inputs/git.rs | 397 ++++++++++++++ .../freshell-server/src/fresh_agent_proxy.rs | 2 + .../src/managed_mcp_capability.rs | 252 +++++++++ .../managed_mcp_capability/claude_stage.rs | 507 +++++++++++++++++- .../managed_provider_claude_catalog_probe.mjs | 52 ++ .../src/managed_provider_references/claude.rs | 2 + .../src/managed_provider_sdk_tests.rs | 5 +- crates/freshell-server/src/managed_runtime.rs | 2 + .../src/provider_input_preparation.rs | 6 + 10 files changed, 1222 insertions(+), 17 deletions(-) create mode 100644 crates/freshell-platform/src/claude_inputs/git.rs create mode 100644 crates/freshell-server/src/managed_provider_claude_catalog_probe.mjs diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index 4893bee61..cdc7ae0e0 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -2,6 +2,15 @@ use serde::{Deserialize, Serialize}; use std::path::{Path, PathBuf}; +pub mod git; + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum NativeSourceRuntime { + Terminal, + Sdk, +} + /// Durable argv names this owned selection; its source and declaring origin /// are recorded in the private accepted manifest, never as raw settings JSON. pub const MANAGED_SETTINGS_ARGUMENT: &str = ".freshell/claude-selected-settings"; @@ -73,6 +82,8 @@ pub fn selected_settings_argument(args: &[String]) -> Option<&str> { #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct ClaudeInputManifest { pub version: u8, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub native_source_runtime: Option, pub original_home: PathBuf, pub cwd: PathBuf, pub workspace: PathBuf, @@ -140,6 +151,9 @@ pub fn default_setting_sources() -> Vec { } impl ClaudeInputManifest { pub fn validate(&self) -> Result<(), &'static str> { + if self.native_source_runtime.is_none() { + return Err("Claude native source runtime is unrecorded; retain inputs and reprepare before dispatch"); + } if self.version != 1 || (self.settings_origin.is_some() && self.settings_file.is_none()) || self.inline_settings_sha256.as_ref().is_some_and(|digest| { diff --git a/crates/freshell-platform/src/claude_inputs/git.rs b/crates/freshell-platform/src/claude_inputs/git.rs new file mode 100644 index 000000000..b0e002309 --- /dev/null +++ b/crates/freshell-platform/src/claude_inputs/git.rs @@ -0,0 +1,397 @@ +//! Claude's finite native Git-source selector. This does not share the more +//! forgiving Git metadata resolver used for presentation and workspaces. +use super::NativeSourceRuntime; +use std::path::{Path, PathBuf}; + +#[derive(Debug, Clone)] +pub struct Roots { + pub checkout: PathBuf, + pub canonical: PathBuf, +} + +pub fn roots(cwd: &Path, runtime: NativeSourceRuntime) -> Result, String> { + let cwd = lexical(cwd); + let Some(checkout) = cwd.ancestors().find(|base| candidate(base, runtime)) else { + return Ok(None); + }; + let canonical = linked_root(checkout, runtime)?.unwrap_or_else(|| checkout.into()); + Ok(Some(Roots { + checkout: checkout.into(), + canonical, + })) +} + +fn candidate(base: &Path, runtime: NativeSourceRuntime) -> bool { + let marker = base.join(".git"); + match kind(&marker) { + Kind::File | Kind::Directory => true, + Kind::Link => std::fs::read_link(&marker).ok().is_some_and(|target| { + target.to_str().is_some_and(|raw| { + !raw.contains('\0') + && !unsafe_pointer(raw, base, base, runtime) + && std::fs::metadata(&marker).is_ok_and(|m| m.is_file() || m.is_dir()) + }) + }), + _ => false, + } +} + +fn linked_root(checkout: &Path, runtime: NativeSourceRuntime) -> Result, String> { + let Some(marker) = read_pointer(&checkout.join(".git"))? else { + return Ok(None); + }; + let Some(pointer) = marker + .trim() + .trim_start_matches('\u{feff}') + .strip_prefix("gitdir:") + else { + return Ok(None); + }; + let pointer = pointer.trim(); + if unsafe_pointer(pointer, checkout, checkout, runtime) { + return Ok(None); + } + let gitdir = resolve(checkout, pointer); + let common_file = gitdir.join("commondir"); + if suspect_file(&common_file, &gitdir, runtime) { + return Ok(None); + } + let Some(common) = read_pointer(&common_file)? else { + return Ok(None); + }; + let common = common.trim(); + if unsafe_pointer(common, &gitdir, &gitdir, runtime) { + return Ok(None); + } + let common = resolve(&gitdir, common); + if gitdir.parent().map(lexical) != Some(common.join("worktrees")) { + return Ok(None); + } + let backlink_file = gitdir.join("gitdir"); + if suspect_file(&backlink_file, &gitdir, runtime) { + return Ok(None); + } + let Some(backlink) = read_pointer(&backlink_file)? else { + return Ok(None); + }; + let backlink = backlink.trim(); + if unsafe_network(backlink, checkout, runtime) + || unsafe_traversal(backlink, &gitdir, checkout, runtime) + { + return Ok(None); + } + let Ok(backlink) = std::fs::canonicalize(resolve(&gitdir, backlink)) else { + return Ok(None); + }; + let Ok(real_checkout) = std::fs::canonicalize(checkout) else { + return Ok(None); + }; + if backlink != real_checkout.join(".git") { + return Ok(None); + } + Ok(Some( + if common.file_name().is_some_and(|name| name == ".git") { + common.parent().unwrap_or(&common).into() + } else { + common + }, + )) +} + +fn read_pointer(path: &Path) -> Result, String> { + use std::io::Read; + let Ok(mut file) = std::fs::File::open(path) else { + return Ok(None); + }; + let limit = crate::provider_input_profiles::MAX_SOURCE_VIEW_BYTES; + let Ok(metadata) = file.metadata() else { + return Ok(None); + }; + if !metadata.is_file() { + return Ok(None); + } + let over_limit = || { + format!( + "Claude Git selection metadata {} exceeds the 16 MiB source-view profile; retain inputs and reduce the selected metadata", + path.display() + ) + }; + if metadata.len() > limit { + return Err(over_limit()); + } + let mut bytes = Vec::new(); + bytes + .try_reserve_exact(metadata.len() as usize) + .map_err(|_| over_limit())?; + let mut chunk = [0; 8192]; + loop { + let count = match file.read(&mut chunk) { + Ok(0) => break, + Ok(count) => count, + Err(error) if error.kind() == std::io::ErrorKind::Interrupted => continue, + Err(_) => return Ok(None), + }; + if bytes.len() as u64 + count as u64 > limit { + return Err(over_limit()); + } + bytes.try_reserve_exact(count).map_err(|_| over_limit())?; + bytes.extend_from_slice(&chunk[..count]); + } + let raw = match String::from_utf8(bytes) { + Ok(raw) => raw, + Err(error) => { + let bytes = error.into_bytes(); + lossy_utf8_view(&bytes, limit).map_err(|_| over_limit())? + } + }; + if raw.trim().len() > 64 * 1024 { + return Err(format!( + "Claude Git selection pointer {} exceeds the 64 KiB path profile; retain inputs and repair the selected metadata", + path.display() + )); + } + Ok(Some(raw)) +} + +// Native replacement groups are admitted before allocation. Reserve the exact +// output length so incremental replacement cannot double a view's capacity. +fn lossy_utf8_view(bytes: &[u8], limit: u64) -> Result { + let length = bytes + .utf8_chunks() + .try_fold(0usize, |length, chunk| { + length.checked_add(chunk.valid().len() + if chunk.invalid().is_empty() { 0 } else { 3 }) + }) + .filter(|length| *length as u64 <= limit) + .ok_or(())?; + let mut view = String::new(); + view.try_reserve_exact(length).map_err(|_| ())?; + for chunk in bytes.utf8_chunks() { + view.push_str(chunk.valid()); + if !chunk.invalid().is_empty() { + view.push('\u{fffd}'); + } + } + Ok(view) +} + +#[derive(PartialEq)] +enum Kind { + File, + Directory, + Link, + Absent, + Other, +} +fn kind(path: &Path) -> Kind { + match std::fs::symlink_metadata(path) { + Ok(m) if m.file_type().is_symlink() => Kind::Link, + Ok(m) if m.is_dir() => Kind::Directory, + Ok(m) if m.is_file() => Kind::File, + Err(e) + if matches!( + e.kind(), + std::io::ErrorKind::NotFound | std::io::ErrorKind::NotADirectory + ) => + { + Kind::Absent + } + _ => Kind::Other, + } +} +fn suspect_file(path: &Path, base: &Path, runtime: NativeSourceRuntime) -> bool { + !walk(path, base, 40, runtime) || matches!(kind(path), Kind::Link | Kind::Other) +} +fn unsafe_pointer(raw: &str, base: &Path, anchor: &Path, runtime: NativeSourceRuntime) -> bool { + unsafe_network(raw, base, runtime) || unsafe_traversal(raw, base, anchor, runtime) +} +fn unsafe_traversal(raw: &str, base: &Path, anchor: &Path, runtime: NativeSourceRuntime) -> bool { + let original = if raw.starts_with('/') { + PathBuf::from(raw) + } else { + base.join(raw) + }; + !walk(&original, anchor, 40, runtime) || !walk(&resolve(base, raw), anchor, 40, runtime) +} +fn walk(path: &Path, anchor: &Path, remaining: usize, runtime: NativeSourceRuntime) -> bool { + if remaining == 0 { + return false; + } + let Some(raw) = path.to_str() else { + return false; + }; + let parts = raw.split('/').collect::>(); + let mut current = PathBuf::from("/"); + for (index, component) in parts.iter().enumerate() { + match *component { + "" | "." => continue, + ".." => { + current.pop(); + continue; + } + _ => current.push(component), + } + if network_host(¤t, runtime) + .is_some_and(|host| Some(host) != network_host(anchor, runtime)) + { + return false; + } + match kind(¤t) { + Kind::Other => return false, + Kind::Link => { + let Ok(target) = std::fs::read_link(¤t) else { + return false; + }; + let Some(target) = target.to_str() else { + return false; + }; + let parent = current.parent().unwrap_or(Path::new("/")); + if target.contains('\0') || unsafe_network(target, parent, runtime) { + return false; + } + let mut continued = if target.starts_with('/') { + PathBuf::from(target) + } else { + parent.join(target) + }; + for component in &parts[index + 1..] { + continued.push(component); + } + return walk(&continued, anchor, remaining - 1, runtime); + } + _ => {} + } + } + true +} + +fn unsafe_network(raw: &str, base: &Path, runtime: NativeSourceRuntime) -> bool { + [PathBuf::from(raw), resolve(base, raw)].iter().any(|path| { + let Some(text) = path.to_str() else { + return true; + }; + let bytes = text.as_bytes(); + let slash = |byte: u8| byte == b'/' || byte == b'\\'; + let unc = bytes.len() >= 2 && slash(bytes[0]) && slash(bytes[1]); + let device = + bytes.len() >= 4 && slash(bytes[0]) && &bytes[1..3] == b"??" && slash(bytes[3]); + let normalized_device = text.contains("??") && { + let mut parts = Vec::new(); + for part in text.split(['/', '\\']) { + match part { + "" | "." => {} + ".." => { + parts.pop(); + } + _ => parts.push(part), + } + } + bytes.first().copied().is_some_and(slash) && parts.first() == Some(&"??") + }; + if device || normalized_device { + return true; + } + if unc { + let parts = text + .split(['/', '\\']) + .filter(|p| !p.is_empty()) + .collect::>(); + if parts.iter().any(|part| matches!(*part, "." | "..")) + || parts.first().is_none_or(|part| matches!(*part, "?" | ".")) + || unc_host(text) != base.to_str().and_then(unc_host) + { + return true; + } + } + let normalized = lexical(path); + if runtime == NativeSourceRuntime::Terminal { + let lower = normalized.to_string_lossy().to_lowercase(); + let mut raw_components = Vec::new(); + let mut entered_network = false; + if text.starts_with('/') { + for component in text.split('/') { + match component { + "" | "." => continue, + ".." => { + raw_components.pop(); + } + _ => { + raw_components.push(component); + if raw_components.len() == 1 + && component.eq_ignore_ascii_case("network") + { + entered_network = true; + break; + } + } + } + } + } + if lower == "/net" || entered_network { + return true; + } + } + network_host(&normalized, runtime) + .is_some_and(|host| Some(host) != network_host(base, runtime)) + }) +} +fn unc_host(raw: &str) -> Option { + let bytes = raw.as_bytes(); + if bytes.len() < 2 || !bytes[..2].iter().all(|b| matches!(b, b'/' | b'\\')) { + return None; + } + raw.split(['/', '\\']) + .find(|part| !part.is_empty()) + .map(str::to_ascii_lowercase) +} +fn network_host(path: &Path, runtime: NativeSourceRuntime) -> Option { + let normalized = lexical(path); + let mut parts = normalized.to_str()?.split('/').filter(|p| !p.is_empty()); + let first = parts.next()?; + if first.eq_ignore_ascii_case("net") { + return parts.next().map(str::to_ascii_lowercase); + } + if runtime == NativeSourceRuntime::Terminal + && first.eq_ignore_ascii_case("network") + && parts.next()?.eq_ignore_ascii_case("servers") + { + return parts.next().map(str::to_ascii_lowercase); + } + None +} +pub fn lexical(path: &Path) -> PathBuf { + crate::opencode_config::lexical_path(path) +} +fn resolve(base: &Path, raw: &str) -> PathBuf { + lexical(&if raw.starts_with('/') { + PathBuf::from(raw) + } else { + base.join(raw) + }) +} + +/// Native eager-memory containment is inclusive and case folded, even on +/// Linux. Its /private aliases are lexical, rather than filesystem realpaths. +pub fn contained(candidate: &Path, root: &Path) -> bool { + let normalize = |path: &Path| { + let mut value = lexical(path) + .to_string_lossy() + .to_lowercase() + .replace('\u{0131}', "i") + .replace('\u{017f}', "s"); + if value.starts_with("/private/var/") { + value = value.replacen("/private/var/", "/var/", 1); + } + if value == "/private/tmp" || value.starts_with("/private/tmp/") { + value = value.replacen("/private/tmp", "/tmp", 1); + } + PathBuf::from(value) + }; + normalize(candidate) + .strip_prefix(normalize(root)) + .is_ok_and(|relative| { + !relative + .to_string_lossy() + .split(['/', '\\']) + .any(|part| part == "..") + }) +} diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 7bf420a8d..7cec277ba 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -524,6 +524,8 @@ impl HostedFreshAgentProxy { let original = |name: &str| std::env::var(name).ok(); let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { + native_source_runtime: + freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, reservation_provider: provider.as_str(), selected_settings: None, client: &self.client, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index a5a367457..a08fb3b0c 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -34,6 +34,9 @@ mod claude_stage; mod ordinary_stage; mod source_stage; +#[cfg(all(test, unix, feature = "managed-provider-qualification"))] +pub(crate) use tests::qualify_claude_linked_sources; + #[derive(Clone, Copy, Default)] pub(crate) struct OpencodeEphemeralInput<'a> { pub codex_manifest: Option<&'a freshell_platform::codex_inputs::CodexInputManifest>, @@ -3360,6 +3363,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::write(selected.join("commands/probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); let manifest = ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.clone(), cwd: cwd.clone(), workspace: cwd.clone(), @@ -3447,6 +3453,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::os::unix::fs::symlink(&selected, &alias).unwrap(); let manifest = ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.clone(), cwd: cwd.clone(), workspace: cwd.clone(), @@ -3593,6 +3602,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ }; let mut manifest = ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.clone(), cwd: cwd.clone(), workspace: cwd.clone(), @@ -4456,6 +4468,243 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .is_dir()); } + #[cfg(feature = "managed-provider-qualification")] + pub(crate) async fn qualify_claude_linked_sources(base: &Path) { + use freshell_platform::{ + claude_inputs::{ClaudeInputManifest, ClaudeSettingSource, NativeSourceRuntime}, + provider_environment::{copy_environment, CopyLimits, CopySelection}, + }; + use std::os::unix::fs::{symlink, PermissionsExt}; + use tokio::io::{AsyncBufReadExt, BufReader}; + let script = base.join("linked-catalog-probe.mjs"); + std::fs::write( + &script, + include_str!("managed_provider_claude_catalog_probe.mjs"), + ) + .unwrap(); + for ( + case, + empty_checkout, + nested_commands, + custom_common, + broken_backlink, + pointer_symlink, + ) in [ + ("missing", false, false, false, false, false), + ("empty-checkout", true, false, false, false, false), + ("nested", false, true, false, false, false), + ("custom-common", false, false, true, false, false), + ("wrong-backlink", false, true, false, true, false), + ("pointer-symlink", false, true, false, false, true), + ] { + let fixture = base.join("linked-sources").join(case); + let home = fixture.join("source-home"); + let main = fixture.join("main"); + let common = if custom_common { + main.clone() + } else { + main.join(".git") + }; + let checkout = fixture.join("checkout"); + let cwd = checkout.join("nested"); + let gitdir = common.join("worktrees/fixture"); + for directory in [&home, &cwd, &gitdir] { + std::fs::create_dir_all(directory).unwrap(); + } + std::fs::write( + checkout.join(".git"), + format!("gitdir: {}\n", gitdir.display()), + ) + .unwrap(); + std::fs::write(gitdir.join("commondir"), "../..\n").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", checkout.join(".git").display()), + ) + .unwrap(); + if broken_backlink { + let other = fixture.join("other-checkout"); + std::fs::create_dir(&other).unwrap(); + std::fs::write(other.join(".git"), "unrelated").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", other.join(".git").display()), + ) + .unwrap(); + } + if pointer_symlink { + std::fs::rename(gitdir.join("commondir"), gitdir.join("ordinary-pointer")).unwrap(); + symlink("ordinary-pointer", gitdir.join("commondir")).unwrap(); + } + let main_commands = main.join(".claude/commands"); + let main_name = format!("selected-main-{case}"); + std::fs::create_dir_all(main_commands.join("node_modules/probe-dependency")).unwrap(); + std::fs::write(main_commands.join(format!("{main_name}.md")), format!("---\ndescription: {case} native source nonce\n---\nComplete selected command resource.\n")).unwrap(); + std::fs::write(main_commands.join("resource"), format!("{case}-resource")).unwrap(); + std::fs::write( + main_commands.join("node_modules/probe-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(main_commands.join("run.cjs"), "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); + std::fs::set_permissions( + main_commands.join("run.cjs"), + std::fs::Permissions::from_mode(0o750), + ) + .unwrap(); + if empty_checkout { + std::fs::create_dir_all(checkout.join(".claude/commands")).unwrap(); + } + let nested_name = format!("selected-nested-{case}"); + if nested_commands { + std::fs::create_dir_all(cwd.join(".claude/commands")).unwrap(); + std::fs::write( + cwd.join(".claude/commands") + .join(format!("{nested_name}.md")), + "---\ndescription: Nested native source nonce\n---\nNested command.\n", + ) + .unwrap(); + } + let manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(NativeSourceRuntime::Sdk), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: checkout.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project], + plugin_dirs: vec![], + }; + let selected = + claude_stage::prepare(&home, &cwd, &checkout, Some(&manifest), &[]).unwrap(); + let stage = fixture.join("controller-copy"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + let host_copy = fixture.join("host-copy"); + let host_sources = selected + .mounts + .iter() + .map(|mount| CopySelection { + source: stage.join(&mount.stage_relative_path), + destination_relative: mount.provider_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&host_sources, &host_copy, &[], CopyLimits::default()).unwrap(); + let mut originals = Vec::new(); + for mount in &selected.mounts { + let logical = Path::new(&mount.logical_destination); + assert!( + logical.starts_with(&fixture), + "native catalog qualifier selects only its fixture" + ); + let archive = logical.with_file_name(format!( + "{}-original", + logical.file_name().unwrap().to_str().unwrap() + )); + std::fs::rename(logical, &archive).unwrap(); + copy_environment( + &[CopySelection { + source: host_copy.join(&mount.provider_relative_path), + destination_relative: logical.file_name().unwrap().into(), + }], + logical.parent().unwrap(), + &[], + CopyLimits::default(), + ) + .unwrap(); + originals.push((logical.to_path_buf(), archive)); + } + let expected_main = !empty_checkout && !broken_backlink && !pointer_symlink; + if expected_main { + let output = std::process::Command::new(main_commands.join("run.cjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + format!("{case}-resource:installed-dependency").as_bytes() + ); + assert_eq!( + std::fs::metadata(main_commands.join("run.cjs")) + .unwrap() + .permissions() + .mode() + & 0o111, + 0o110 + ); + } + let mut child = tokio::process::Command::new("node") + .arg(&script) + .arg(&cwd) + .arg(&main_name) + .arg(&nested_name) + .current_dir(&cwd) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .env("HOME", "/home/freshell/provider") + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::null()) + .process_group(0) + .kill_on_drop(true) + .spawn() + .unwrap(); + let mut stdout = BufReader::new(child.stdout.take().unwrap()).lines(); + let received = + tokio::time::timeout(std::time::Duration::from_secs(25), stdout.next_line()).await; + // No native worker may overlap the next case or original restoration. + stop_codex_source_child(&mut child).await; + let line = received + .expect("bounded genuine native supportedCommands response") + .unwrap() + .expect("native catalog receipt"); + let receipt: serde_json::Value = serde_json::from_str(&line).unwrap(); + assert_eq!( + receipt["failed"], false, + "classified model-free catalog failed for {case}" + ); + assert_eq!(receipt["uid"], 65534); + assert_eq!(receipt["gid"], 0); + assert_eq!(receipt["defaultChild"], true); + assert_eq!(receipt["zeroPrompts"], true); + assert_eq!( + receipt["mainSelected"], expected_main, + "native main catalog selection {case}" + ); + assert_eq!( + receipt["nestedSelected"], nested_commands, + "native nested catalog selection {case}" + ); + for (logical, archive) in originals { + assert!(logical.starts_with(&fixture)); + if logical.is_dir() { + std::fs::remove_dir_all(&logical).unwrap(); + } else { + std::fs::remove_file(&logical).unwrap(); + } + std::fs::rename(archive, logical).unwrap(); + } + assert_eq!( + std::fs::read_to_string(main_commands.join("resource")).unwrap(), + format!("{case}-resource") + ); + assert_eq!( + std::fs::metadata(main_commands.join("run.cjs")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o750 + ); + println!( + "provider_source_selection {}", + serde_json::json!({"provider":"claude","case":case,"uid":65534,"gid":0,"ownedGroupJoined":true,"digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&receipt).unwrap()))}) + ); + } + } + #[cfg(feature = "managed-provider-qualification")] async fn stop_codex_source_child(child: &mut tokio::process::Child) { let group = child.id().expect("owned native child PID") as libc::pid_t; @@ -5514,6 +5763,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ }; let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.clone(), cwd: workspace.clone(), workspace: workspace.clone(), diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 51c5e5159..ec705e2a1 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -23,6 +23,9 @@ pub(super) fn prepare( ) -> Result { let mut manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.into(), cwd: cwd.into(), workspace: workspace.into(), @@ -49,14 +52,13 @@ pub(super) fn prepare( manifest.settings_origin = Some(origin.clone()); configurations.push((origin.clone(), origin)); } - let mut bases: Vec = manifest - .cwd + let native_cwd = std::fs::canonicalize(&manifest.cwd).unwrap_or_else(|_| manifest.cwd.clone()); + let mut bases: Vec = native_cwd .ancestors() .filter(|path| path.parent().is_some()) .map(Path::to_path_buf) .collect(); - bases.sort(); - bases.dedup(); + bases.reverse(); let config_root = manifest.config_root(); let user_settings = config_root.join("settings.json"); let user_enabled = manifest @@ -68,6 +70,26 @@ pub(super) fn prepare( let local_enabled = manifest .setting_sources .contains(&ClaudeSettingSource::Local); + let roots = if project_enabled { + freshell_platform::claude_inputs::git::roots( + &native_cwd, + manifest + .native_source_runtime + .ok_or("Claude native source runtime is unrecorded")?, + ) + .inspect_err(|reason| { + tracing::warn!( + event = "provider.inputs.preparation.blocked", + provider = "claude", + stage = "controller", + operation = "select_native_git_roots", + reason, + "Claude native Git source selection could not be prepared" + ); + })? + } else { + None + }; if user_enabled && optional(&user_settings)? { configurations.push((user_settings.clone(), canonical(&user_settings)?)); } @@ -104,11 +126,18 @@ pub(super) fn prepare( } } for base in bases { + let excluded_project = roots.as_ref().is_some_and(|roots| { + use freshell_platform::claude_inputs::git::contained; + roots.checkout != roots.canonical + && contained(&roots.checkout, &roots.canonical) + && contained(&base, &roots.canonical) + && !contained(&base, &roots.checkout) + }); for (enabled, relative) in [ - (project_enabled, "CLAUDE.md"), + (project_enabled && !excluded_project, "CLAUDE.md"), (local_enabled, "CLAUDE.local.md"), - (project_enabled, ".claude/CLAUDE.md"), - (project_enabled, ".claude/rules"), + (project_enabled && !excluded_project, ".claude/CLAUDE.md"), + (project_enabled && !excluded_project, ".claude/rules"), ] { if !enabled { continue; @@ -126,13 +155,33 @@ pub(super) fn prepare( } } if project_enabled { - for base in project_resource_bases(&manifest)? { - for relative in RESOURCE_DIRECTORIES { - let logical = base.join(".claude").join(relative); - if optional(&logical)? { - resources.push(logical); - } - } + for relative in RESOURCE_DIRECTORIES { + let directories = if matches!(*relative, "hooks" | "plugins") { + project_resource_bases(&manifest)? + .iter() + .map(|base| base.join(".claude").join(relative)) + .filter_map(|path| match optional(&path) { + Ok(true) => Some(Ok(path)), + Ok(false) => None, + Err(error) => Some(Err(error)), + }) + .collect::, String>>()? + } else { + project_resource_directories(&native_cwd, roots.as_ref(), relative).inspect_err( + |reason| { + tracing::warn!( + event = "provider.inputs.preparation.blocked", + provider = "claude", + stage = "controller", + operation = "select_project_resources", + directory = *relative, + reason, + "Selected Claude project resources could not be prepared" + ); + }, + )? + }; + resources.extend(directories); } } for plugin in &manifest.plugin_dirs { @@ -156,6 +205,64 @@ pub(super) fn prepare( ) } +// The ordinary native initializer sets projectRoot to realpath(launch cwd). +// The Markdown loader chooses fallback independently for each category; +// requested workspace is not another source root. +fn project_resource_directories( + cwd: &Path, + roots: Option<&freshell_platform::claude_inputs::git::Roots>, + relative: &str, +) -> Result, String> { + use freshell_platform::claude_inputs::git; + let mut directories = Vec::new(); + for base in cwd.ancestors() { + if git::lexical(base) == Path::new("/home/freshell/provider") { + break; + } + let directory = base.join(".claude").join(relative); + if native_resource_exists(&directory)? { + directories.push(directory); + } + if roots.is_some_and(|roots| roots.checkout == base) { + break; + } + } + if let Some(roots) = roots { + if roots.checkout != roots.canonical { + let checkout = roots.checkout.join(".claude").join(relative); + if !directories.contains(&checkout) { + let fallback = roots.canonical.join(".claude").join(relative); + if !directories.contains(&fallback) && native_resource_exists(&fallback)? { + directories.push(fallback); + } + } + } + } + Ok(directories) +} +fn native_resource_exists(path: &Path) -> Result { + match std::fs::metadata(path) { + Ok(_) => Ok(true), + Err(error) + if matches!( + error.kind(), + std::io::ErrorKind::NotFound | std::io::ErrorKind::NotADirectory + ) => + { + Ok(false) + } + #[cfg(unix)] + Err(error) if error.raw_os_error() == Some(libc::ENFILE) => { + tracing::warn!(event="provider.inputs.resource_scan.skipped",provider="claude",stage="controller",operation="stat_project_directory",path=%path.display(),cause=%error,"Native Claude resource directory scan exhausted the file table"); + Ok(false) + } + Err(error) => Err(format!( + "Claude project resource discovery {} metadata failed: {error}", + path.display() + )), + } +} + // The native Markdown resource loader searches upward only until HOME or the // nearest Git root. Memory has its own root-to-cwd search and must not share // this stopping rule. Startup cwd is the same accepted cwd for these loaders. @@ -178,7 +285,7 @@ fn project_resource_bases(manifest: &ClaudeInputManifest) -> Result return Err(format!( "Claude project resource discovery {} metadata failed: {error}", base.display() - )) + )); } } } @@ -329,6 +436,9 @@ mod tests { .unwrap(); let manifest = ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.clone(), cwd: cwd.clone(), workspace: cwd.clone(), @@ -380,6 +490,370 @@ mod tests { } } + #[cfg(unix)] + #[test] + fn managed_provider_claude_linked_resources_select_fallback_per_native_directory() { + use std::os::unix::fs::PermissionsExt; + for (case, checkout_commands, nested_commands, checkout_agents, custom_common) in [ + ("missing", false, false, false, false), + ("empty-commands", true, false, false, false), + ("nested-commands", false, true, false, false), + ("independent-agents", true, true, false, true), + ("empty-agents", false, false, true, false), + ] { + let owned = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::Builder::new() + .prefix("claude-linked-resources-") + .tempdir_in(root) + .unwrap(), + None => tempfile::tempdir().unwrap(), + }; + let home = owned.path().join("home"); + let main = owned.path().join("main"); + let common = if custom_common { + main.clone() + } else { + main.join(".git") + }; + let checkout = owned.path().join("checkout"); + let cwd = checkout.join("nested"); + let gitdir = common.join("worktrees/fixture"); + for path in [&home, &cwd, &gitdir] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + checkout.join(".git"), + format!("gitdir: {}\n", gitdir.display()), + ) + .unwrap(); + std::fs::write(gitdir.join("commondir"), "../..\n").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", checkout.join(".git").display()), + ) + .unwrap(); + for category in ["commands", "agents"] { + let root = main.join(".claude").join(category); + std::fs::create_dir_all(root.join("node_modules/probe-dependency")).unwrap(); + std::fs::write(root.join("resource"), format!("{case}-{category}")).unwrap(); + std::fs::write( + root.join("node_modules/probe-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + let executable = root.join("run.cjs"); + std::fs::write(&executable, "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); + std::fs::set_permissions(&executable, std::fs::Permissions::from_mode(0o750)) + .unwrap(); + } + if checkout_commands { + std::fs::create_dir_all(checkout.join(".claude/commands")).unwrap(); + } + if checkout_agents { + std::fs::create_dir_all(checkout.join(".claude/agents")).unwrap(); + } + if nested_commands { + std::fs::create_dir_all(cwd.join(".claude/commands")).unwrap(); + } + let manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: checkout.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project], + plugin_dirs: vec![], + }; + let selected = prepare(&home, &cwd, &checkout, Some(&manifest), &[]).unwrap(); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + for (category, suppressed) in + [("commands", checkout_commands), ("agents", checkout_agents)] + { + let root = main.join(".claude").join(category); + let mount = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == root); + assert_eq!( + mount.is_none(), + suppressed, + "{case}: native fallback for {category}" + ); + if let Some(mount) = mount { + let executable = stage.join(&mount.stage_relative_path).join("run.cjs"); + let output = std::process::Command::new(&executable).output().unwrap(); + assert!( + output.status.success(), + "{case}: selected {category} executable" + ); + assert_eq!( + output.stdout, + format!("{case}-{category}:installed-dependency").as_bytes() + ); + assert_eq!( + std::fs::metadata(&executable).unwrap().permissions().mode() & 0o111, + 0o110 + ); + } + assert_eq!( + std::fs::read_to_string(root.join("resource")).unwrap(), + format!("{case}-{category}") + ); + assert_eq!( + std::fs::metadata(root.join("run.cjs")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o750 + ); + } + assert_eq!( + selected + .mounts + .iter() + .any(|mount| Path::new(&mount.logical_destination) + == cwd.join(".claude/commands")), + nested_commands + ); + } + } + + #[test] + fn managed_provider_claude_git_lossy_source_view_limit_preserves_prior_copied_resources() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let commands = cwd.join(".claude/commands"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&commands).unwrap(); + std::fs::write(commands.join("resource"), "retained-selected-resource").unwrap(); + std::fs::write( + commands.join("consume.cjs"), + "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource'))", + ) + .unwrap(); + std::fs::write(cwd.join(".git"), "ordinary native checkout marker\n").unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(freshell_platform::claude_inputs::NativeSourceRuntime::Sdk), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project], + plugin_dirs: vec![], + }; + let prior = prepare(&home, &cwd, &cwd, Some(&manifest), &[]).unwrap(); + let stage = owned.path().join("prior-copy"); + copy_environment(&prior.selections, &stage, &[], CopyLimits::default()).unwrap(); + let mount = prior + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == commands) + .unwrap(); + let consume = || { + let output = std::process::Command::new("node") + .arg(stage.join(&mount.stage_relative_path).join("consume.cjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"retained-selected-resource"); + }; + consume(); + let mut expanded = b"gitdir: ".to_vec(); + expanded.resize(6 * 1024 * 1024, 0xff); + std::fs::write(cwd.join(".git"), &expanded).unwrap(); + let error = prepare(&home, &cwd, &cwd, Some(&manifest), &[]) + .err() + .unwrap(); + assert!( + error.contains("16 MiB source-view"), + "UTF-8 replacement expansion must be admitted before allocating a transformed view: {error}" + ); + assert!( + !error.contains('\u{fffd}'), + "the rejected source must not enter errors" + ); + consume(); + assert_eq!(std::fs::read(cwd.join(".git")).unwrap(), expanded); + assert_eq!( + std::fs::read_to_string(commands.join("resource")).unwrap(), + "retained-selected-resource" + ); + // Native trimming may consume a complete admitted view. One extra + // input byte must fail without disturbing the already accepted copy. + let maximum = freshell_platform::provider_input_profiles::MAX_SOURCE_VIEW_BYTES as usize; + for (size, admitted) in [(maximum - 1, true), (maximum, true), (maximum + 1, false)] { + let source = vec![b' '; size]; + std::fs::write(cwd.join(".git"), &source).unwrap(); + let next = prepare(&home, &cwd, &cwd, Some(&manifest), &[]); + if admitted { + let next = next.unwrap(); + let destination = owned.path().join(format!("admitted-{size}")); + copy_environment(&next.selections, &destination, &[], CopyLimits::default()) + .unwrap(); + let mount = next + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == commands) + .unwrap(); + let output = std::process::Command::new("node") + .arg( + destination + .join(&mount.stage_relative_path) + .join("consume.cjs"), + ) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"retained-selected-resource"); + } else { + assert!(next.err().unwrap().contains("16 MiB source-view")); + } + consume(); + assert_eq!(std::fs::read(cwd.join(".git")).unwrap(), source); + } + } + + #[cfg(unix)] + #[test] + fn managed_provider_claude_nested_linked_memory_retains_local_and_excludes_main_project_imports( + ) { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let main = owned.path().join("main"); + let intermediate = main.join("checkouts"); + let checkout = intermediate.join("linked"); + let cwd = checkout.join("nested"); + let gitdir = main.join(".git/worktrees/linked"); + for path in [&home, &cwd, &gitdir] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + checkout.join(".git"), + format!("gitdir: {}\n", gitdir.display()), + ) + .unwrap(); + std::fs::write(gitdir.join("commondir"), "../..\n").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", checkout.join(".git").display()), + ) + .unwrap(); + for ancestor in [&main, &intermediate] { + std::fs::create_dir_all(ancestor.join(".claude/rules")).unwrap(); + std::fs::write(ancestor.join("CLAUDE.md"), "@missing-main-project.md\n").unwrap(); + std::fs::write( + ancestor.join(".claude/CLAUDE.md"), + "@../missing-main-dotclaude.md\n", + ) + .unwrap(); + std::fs::write( + ancestor.join(".claude/rules/unselected.md"), + "@missing-unselected-rule.md\n", + ) + .unwrap(); + std::fs::write(ancestor.join("CLAUDE.local.md"), "@local-resource.md\n").unwrap(); + std::fs::write( + ancestor.join("local-resource.md"), + "selected-local-ancestor", + ) + .unwrap(); + } + std::fs::write(owned.path().join("CLAUDE.md"), "@above-main.md\n").unwrap(); + std::fs::write(owned.path().join("above-main.md"), "selected-above-main").unwrap(); + std::fs::write(checkout.join("CLAUDE.md"), "@checkout-resource.md\n").unwrap(); + std::fs::write( + checkout.join("checkout-resource.md"), + "selected-checkout-project", + ) + .unwrap(); + let mut manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(freshell_platform::claude_inputs::NativeSourceRuntime::Sdk), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: checkout.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project, ClaudeSettingSource::Local], + plugin_dirs: vec![], + }; + for local_enabled in [true, false] { + if !local_enabled { + manifest.setting_sources = vec![ClaudeSettingSource::Project]; + } + let selected = prepare(&home, &cwd, &checkout, Some(&manifest), &[]) + .expect("native-excluded main project memory must not resolve its broken imports"); + let stage = owned.path().join(if local_enabled { + "with-local" + } else { + "without-local" + }); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + for (path, nonce) in [ + (owned.path().join("above-main.md"), "selected-above-main"), + ( + checkout.join("checkout-resource.md"), + "selected-checkout-project", + ), + ] { + let mount = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == path) + .unwrap(); + let output = std::process::Command::new("node") + .args([ + "-e", + "process.stdout.write(require('node:fs').readFileSync(process.argv[1]))", + ]) + .arg(stage.join(&mount.stage_relative_path)) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, nonce.as_bytes()); + } + for ancestor in [&main, &intermediate] { + let local = selected.mounts.iter().find(|mount| { + Path::new(&mount.logical_destination) == ancestor.join("local-resource.md") + }); + assert_eq!(local.is_some(), local_enabled); + if let Some(local) = local { + let output = std::process::Command::new("node").args(["-e", "process.stdout.write(require('node:fs').readFileSync(process.argv[1]))"]).arg(stage.join(&local.stage_relative_path)).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"selected-local-ancestor"); + } + assert!(!selected + .mounts + .iter() + .any(|mount| Path::new(&mount.logical_destination) + == ancestor.join("CLAUDE.md") + || Path::new(&mount.logical_destination) + == ancestor.join(".claude/CLAUDE.md") + || Path::new(&mount.logical_destination) + == ancestor.join(".claude/rules"))); + assert_eq!( + std::fs::read_to_string(ancestor.join("CLAUDE.md")).unwrap(), + "@missing-main-project.md\n" + ); + } + } + } + #[test] fn managed_provider_claude_selected_user_rules_keep_external_imports() { let owned = tempfile::tempdir().unwrap(); @@ -454,6 +928,9 @@ mod tests { std::fs::write(project.join("commands/probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); let mut manifest = ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: home.clone(), cwd: cwd.clone(), workspace: cwd.clone(), diff --git a/crates/freshell-server/src/managed_provider_claude_catalog_probe.mjs b/crates/freshell-server/src/managed_provider_claude_catalog_probe.mjs new file mode 100644 index 000000000..9b4a0e607 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_claude_catalog_probe.mjs @@ -0,0 +1,52 @@ +import { spawn } from 'node:child_process' +import { query } from '/opt/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk/sdk.mjs' + +// The Rust owner terminates and joins this dedicated process group after the +// bounded receipt. Keep the real query alive; never submit a user message. +const abort = new AbortController() +let prompts = 0, defaultChild = false +async function* emptyPrompt() { + await new Promise(resolve => abort.signal.addEventListener('abort', resolve, { once: true })) +} +let q +try { + q = query({ prompt: emptyPrompt(), options: { + cwd: process.argv[2], settingSources: ['project'], abortController: abort, + env: { PATH: process.env.PATH, HOME: process.env.HOME, + ANTHROPIC_API_KEY: 'synthetic-catalog-only', ANTHROPIC_BASE_URL: 'http://127.0.0.1:9' }, + spawnClaudeCodeProcess: options => { + defaultChild = options.command.includes('@anthropic-ai/claude-agent-sdk-linux-x64/') + && options.command.endsWith('/claude') + const child = spawn(options.command, options.args, { + cwd: options.cwd, env: options.env, stdio: ['pipe', 'pipe', 'pipe'], + }) + child.stderr.resume() + const write = child.stdin.write + let pending = '' + child.stdin.write = function(chunk, ...args) { + pending += chunk.toString() + let boundary + while ((boundary = pending.indexOf('\n')) >= 0) { + const frame = pending.slice(0, boundary) + pending = pending.slice(boundary + 1) + try { if (JSON.parse(frame).type === 'user') prompts++ } catch { /* non-JSON transport */ } + } + if (pending.length > 1024 * 1024) throw new Error('fixture-native-input-profile') + return write.call(this, chunk, ...args) + } + return child + }, + } }) + const commands = await q.supportedCommands() + process.stdout.write(JSON.stringify({ + failed: false, uid: process.geteuid(), gid: process.getegid(), defaultChild, + zeroPrompts: prompts === 0, + mainSelected: commands.some(command => command.name === process.argv[3]), + nestedSelected: commands.some(command => command.name === process.argv[4]), + }) + '\n') +} catch { + process.stdout.write(JSON.stringify({ failed: true, stage: 'native-catalog' }) + '\n') +} +// The finite owner stops the actual native child and all inherited-group +// descendants, then waits/reaps before it restores originals or starts a case. +await new Promise(() => {}) diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index b9e24ccce..1cb7acae0 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -7,6 +7,7 @@ pub(crate) struct ClaudePreparation<'a> { pub client: &'a freshell_runtime_client::RuntimeClient, pub soul: &'a SoulId, pub creation_seed: &'a str, + pub native_source_runtime: freshell_platform::claude_inputs::NativeSourceRuntime, pub reservation_provider: &'a str, pub selected_settings: Option<&'a str>, pub home: &'a Path, @@ -60,6 +61,7 @@ pub(crate) async fn prepare_managed( } let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { version: 1, + native_source_runtime: Some(request.native_source_runtime), original_home: request.home.into(), cwd: request.cwd.into(), workspace: request.workspace.into(), diff --git a/crates/freshell-server/src/managed_provider_sdk_tests.rs b/crates/freshell-server/src/managed_provider_sdk_tests.rs index 9e3133af2..9b0ccd8e1 100644 --- a/crates/freshell-server/src/managed_provider_sdk_tests.rs +++ b/crates/freshell-server/src/managed_provider_sdk_tests.rs @@ -1,8 +1,8 @@ //! Genuine default SDK startup qualification in the closed provider sandbox. use sha2::{Digest, Sha256}; -#[test] -fn managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt() { +#[tokio::test] +async fn managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt() { let root = std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") .expect("native SDK qualification requires its owned executable sandbox"); let fixture = tempfile::Builder::new() @@ -57,4 +57,5 @@ fn managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt() "digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&receipt).unwrap())), }) ); + crate::managed_mcp_capability::qualify_claude_linked_sources(fixture.path()).await; } diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 330db27c7..08f325f09 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -270,6 +270,8 @@ impl ManagedTerminalController for ServerManagedRuntimeController { }; let context = crate::managed_provider_references::claude::prepare_managed( crate::managed_provider_references::claude::ClaudePreparation { + native_source_runtime: + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, reservation_provider: "claude", selected_settings: selected_claude_settings.as_deref(), client: &self.client, diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 63db2646e..8f748d0b2 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -343,6 +343,9 @@ mod tests { std::fs::write(owned.path().join(INLINE_SETTINGS_SOURCE), raw).unwrap(); let manifest = ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: "/absent-home".into(), cwd: "/absent-cwd".into(), workspace: "/absent-cwd".into(), @@ -438,6 +441,9 @@ mod tests { std::fs::create_dir(owned.path().join(".freshell")).unwrap(); let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { version: 1, + native_source_runtime: Some( + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ), original_home: "/absent-original-home".into(), cwd: "/absent-original-workspace".into(), workspace: "/absent-original-workspace".into(), From dd3de8cbf9d2123641dee2dfac3f2be6978f88c2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 18:08:23 -0700 Subject: [PATCH 103/493] match native Claude Git pointer whitespace selection --- .../src/claude_inputs/git.rs | 34 +++-- .../src/managed_mcp_capability.rs | 82 +++++++++++- .../managed_mcp_capability/claude_stage.rs | 124 ++++++++++++++++++ 3 files changed, 224 insertions(+), 16 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs/git.rs b/crates/freshell-platform/src/claude_inputs/git.rs index b0e002309..6fbf151db 100644 --- a/crates/freshell-platform/src/claude_inputs/git.rs +++ b/crates/freshell-platform/src/claude_inputs/git.rs @@ -40,14 +40,10 @@ fn linked_root(checkout: &Path, runtime: NativeSourceRuntime) -> Result Result Result Result, String> { lossy_utf8_view(&bytes, limit).map_err(|_| over_limit())? } }; - if raw.trim().len() > 64 * 1024 { + if native_trim(&raw).len() > 64 * 1024 { return Err(format!( "Claude Git selection pointer {} exceeds the 64 KiB path profile; retain inputs and repair the selected metadata", path.display() @@ -153,6 +149,26 @@ fn read_pointer(path: &Path) -> Result, String> { Ok(Some(raw)) } +// Both pinned native runtimes use String.trim: BOM is whitespace, NEL is not. +fn native_trim(raw: &str) -> &str { + raw.trim_matches(|character| { + matches!( + character, + '\u{0009}'..='\u{000d}' + | '\u{0020}' + | '\u{00a0}' + | '\u{1680}' + | '\u{2000}'..='\u{200a}' + | '\u{2028}' + | '\u{2029}' + | '\u{202f}' + | '\u{205f}' + | '\u{3000}' + | '\u{feff}' + ) + }) +} + // Native replacement groups are admitted before allocation. Reserve the exact // output length so incremental replacement cannot double a view's capacity. fn lossy_utf8_view(bytes: &[u8], limit: u64) -> Result { diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index a08fb3b0c..ce07dbb96 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4489,13 +4489,68 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ custom_common, broken_backlink, pointer_symlink, + pointer_padding, ) in [ - ("missing", false, false, false, false, false), - ("empty-checkout", true, false, false, false, false), - ("nested", false, true, false, false, false), - ("custom-common", false, false, true, false, false), - ("wrong-backlink", false, true, false, true, false), - ("pointer-symlink", false, true, false, false, true), + ("missing", false, false, false, false, false, None), + ("empty-checkout", true, false, false, false, false, None), + ("nested", false, true, false, false, false, None), + ("custom-common", false, false, true, false, false, None), + ("wrong-backlink", false, true, false, true, false, None), + ("pointer-symlink", false, true, false, false, true, None), + ( + "bom-marker", + false, + false, + false, + false, + false, + Some(("marker", "\u{feff} \t", true)), + ), + ( + "bom-common", + false, + false, + false, + false, + false, + Some(("common", "\u{feff} \t", true)), + ), + ( + "bom-backlink", + false, + false, + false, + false, + false, + Some(("backlink", "\u{feff} \t", true)), + ), + ( + "nel-marker", + false, + true, + false, + false, + false, + Some(("marker", "\u{85}", false)), + ), + ( + "nel-common", + false, + true, + false, + false, + false, + Some(("common", "\u{85}", false)), + ), + ( + "nel-backlink", + false, + true, + false, + false, + false, + Some(("backlink", "\u{85}", false)), + ), ] { let fixture = base.join("linked-sources").join(case); let home = fixture.join("source-home"); @@ -4522,6 +4577,16 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ format!("{}\n", checkout.join(".git").display()), ) .unwrap(); + if let Some((field, padding, _)) = pointer_padding { + let path = match field { + "marker" => checkout.join(".git"), + "common" => gitdir.join("commondir"), + "backlink" => gitdir.join("gitdir"), + _ => unreachable!(), + }; + let raw = std::fs::read_to_string(&path).unwrap(); + std::fs::write(path, format!("{padding}{raw}{padding}")).unwrap(); + } if broken_backlink { let other = fixture.join("other-checkout"); std::fs::create_dir(&other).unwrap(); @@ -4616,7 +4681,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(); originals.push((logical.to_path_buf(), archive)); } - let expected_main = !empty_checkout && !broken_backlink && !pointer_symlink; + let expected_main = !empty_checkout + && !broken_backlink + && !pointer_symlink + && pointer_padding.is_none_or(|(_, _, selected)| selected); if expected_main { let output = std::process::Command::new(main_commands.join("run.cjs")) .output() diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index ec705e2a1..b8f771a80 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -626,6 +626,130 @@ mod tests { } } + #[cfg(unix)] + #[test] + fn managed_provider_claude_git_native_trim_preserves_selected_and_unselected_resources() { + use freshell_platform::claude_inputs::NativeSourceRuntime; + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + for field in ["marker", "common", "backlink"] { + for (case, padding, main_selected) in + [("bom", "\u{feff} \t", true), ("nel", "\u{85}", false)] + { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let main = owned.path().join("main"); + let checkout = owned.path().join("checkout"); + let cwd = checkout.join("nested"); + let gitdir = main.join(".git/worktrees/fixture"); + for directory in [&home, &cwd, &gitdir] { + std::fs::create_dir_all(directory).unwrap(); + } + let sources = [ + ( + "marker", + checkout.join(".git"), + format!("gitdir: {}", gitdir.display()), + ), + ("common", gitdir.join("commondir"), "../..".into()), + ( + "backlink", + gitdir.join("gitdir"), + checkout.join(".git").to_str().unwrap().to_string(), + ), + ] + .map(|(kind, path, raw)| { + let raw = if kind == field { + format!("{padding}{raw}{padding}\n") + } else { + format!("{raw}\n") + }; + std::fs::write(&path, &raw).unwrap(); + (path, raw) + }); + for (root, nonce) in [(&cwd, "cwd-source"), (&main, "canonical-source")] { + let commands = root.join(".claude/commands"); + std::fs::create_dir_all(commands.join("node_modules/probe-dependency")) + .unwrap(); + std::fs::write(commands.join("resource"), nonce).unwrap(); + std::fs::write( + commands.join("node_modules/probe-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(commands.join("consume.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); + } + let poison = main.join(".claude/commands/unselected-broken-link"); + if !main_selected { + std::os::unix::fs::symlink("missing-native-unselected-resource", &poison) + .unwrap(); + } + let manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(runtime), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: checkout.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project], + plugin_dirs: vec![], + }; + let selected = prepare(&home, &cwd, &checkout, Some(&manifest), &[]).unwrap(); + let canonical = main.join(".claude/commands"); + assert_eq!( + selected + .mounts + .iter() + .any(|mount| Path::new(&mount.logical_destination) == canonical), + main_selected, + "{runtime:?}/{field}/{case}: native-effective canonical command selection" + ); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()) + .unwrap(); + for (root, nonce, enabled) in [ + (&cwd, "cwd-source", true), + (&main, "canonical-source", main_selected), + ] { + if !enabled { + continue; + } + let commands = root.join(".claude/commands"); + let mount = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == commands) + .unwrap(); + let output = std::process::Command::new("node") + .arg(stage.join(&mount.stage_relative_path).join("consume.cjs")) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + format!("{nonce}:installed-dependency").as_bytes() + ); + assert_eq!( + std::fs::read_to_string(commands.join("resource")).unwrap(), + nonce + ); + } + for (path, raw) in sources { + assert_eq!(std::fs::read_to_string(path).unwrap(), raw); + } + if !main_selected { + assert_eq!( + std::fs::read_link(poison).unwrap(), + Path::new("missing-native-unselected-resource") + ); + } + } + } + } + } + #[test] fn managed_provider_claude_git_lossy_source_view_limit_preserves_prior_copied_resources() { let owned = tempfile::tempdir().unwrap(); From 896c86734b4a30111458861bf9d2e625c3fd82f9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 19:01:25 -0700 Subject: [PATCH 104/493] Keep provider preparation failures before grant consumption --- crates/freshell-session-host/src/main.rs | 184 +++++++++++++++++- .../src/providers/mod.rs | 58 ++++-- 2 files changed, 223 insertions(+), 19 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index f00909d23..99891e514 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -926,11 +926,16 @@ async fn grant_execution( None }; - persisted.max_control_epoch = control_epoch; - persisted.max_execution_generation = execution_generation; - persisted.grant_id = Some(grant_id.clone()); - persisted.worker_pid = None; - write_state(&state.state_dir, &persisted).map_err(registry_like_error)?; + // Selection-affecting projection and provider preparation can still fail + // safely. Persist consumption only at the native dispatch boundary; once + // spawning is attempted, uncertain delivery must never authorize a replay. + let consume_grant = |persisted: &mut PersistedHostState| { + persisted.max_control_epoch = control_epoch; + persisted.max_execution_generation = execution_generation; + persisted.grant_id = Some(grant_id.clone()); + persisted.worker_pid = None; + write_state(&state.state_dir, persisted).map_err(registry_like_error) + }; let (pid, evidence, native_session_id) = match (fixture, terminal, fresh_agent) { (Some(fixture), None, None) => { @@ -965,6 +970,7 @@ async fn grant_execution( .arg(&resume.provider_session.native_session_id); } } + consume_grant(&mut persisted)?; let child = command.spawn().map_err(|e| { RuntimeError::new( RuntimeErrorCode::HostUnreachable, @@ -1047,6 +1053,10 @@ async fn grant_execution( format!("prepare managed provider: {error}"), ) })?; + consume_grant(&mut persisted)?; + let prepared = providers::start_terminal_sidecar(prepared) + .await + .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; let terminal = prepared.terminal; let hosted = HostedPty::spawn( &state.state_dir, @@ -1170,6 +1180,7 @@ async fn grant_execution( ) .await .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; + consume_grant(&mut persisted)?; freshell_platform::managed_child_secrets::install(resolved_secrets.environment) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; let actor = providers::open_hosted_fresh_agent(&state.state_dir, launch.clone()) @@ -4635,6 +4646,169 @@ mod tests { assert_eq!(persisted.worker_launch_count, 0); } + #[tokio::test] + async fn provider_secret_projection_failure_keeps_grant_retryable_and_prior_inputs() { + for route in ["terminal", "fresh"] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("provider-home"); + std::fs::create_dir_all(home.join(".config/opencode")).unwrap(); + let prior = home.join(".config/opencode/opencode.json"); + std::fs::write(&prior, b"prior-accepted-source").unwrap(); + let context = freshell_runtime_protocol::ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Opencode { + project_config: vec![freshell_runtime_protocol::ProviderConfigReference { + root: freshell_runtime_protocol::ProviderConfigRoot::Workspace, + relative_path: "opencode.json".into(), + provider_relative_path: ".config/opencode/opencode.json".into(), + format: "json".into(), + }], + tui_config: None, + tui_source: None, + inline_config: false, + }, + prepared_inputs: None, + mcp_capability: None, + config: Vec::new(), + }; + context.validate("opencode").unwrap(); + let mut state = test_host_state(); + Arc::get_mut(&mut state).unwrap().provider_config_paths = ProviderConfigPaths { + private_mount: owned.path().join("private"), + provider_home: home, + }; + let cwd = owned.path().to_str().unwrap(); + let uid = unsafe { libc::geteuid() }; + let gid = unsafe { libc::getegid() }; + let session_type = if route == "terminal" { + "opencode" + } else { + "freshopencode" + }; + let runtime_variant = if route == "terminal" { + "managed_terminal_pty" + } else { + "opencode" + }; + let resume: ResumeSpec = serde_json::from_value(serde_json::json!({ + "providerSession":{"provider":"opencode","providerStoreId":"synthetic-store","nativeSessionId":"ses_projection_retry"}, + "mode":session_type,"runtimeVariant":runtime_variant,"program":"/usr/bin/true","resumeArgv":[], + "providerHome":"/home/freshell/provider","cwd":cwd,"workspacePath":cwd, + "identityProvenance":"provider_observed","durablePosition":{},"creationSeedRef":"synthetic-seed", + "checkpointRevision":0,"allocationState":"verified_durable","evidenceRevision":1,"neverDispatched":false, + })).unwrap(); + let terminal = (route == "terminal").then(|| serde_json::from_value(serde_json::json!({ + "terminalId":"synthetic-projection","streamId":"synthetic-stream","mode":"opencode","program":"/usr/bin/true","args":[],"env":{}, + "cwd":cwd,"workspacePath":cwd,"runAsUid":uid,"runAsGid":gid,"cols":80,"rows":24,"projectKey":"synthetic", + "providerLaunchContext":context, + })).unwrap()); + let fresh = (route == "fresh").then(|| serde_json::from_value(serde_json::json!({ + "sessionId":"synthetic-projection","provider":"opencode","sessionType":session_type,"runtimeVariant":runtime_variant, + "providerStoreId":"synthetic-store","cwd":cwd,"workspacePath":cwd,"runAsUid":uid,"runAsGid":gid, + "providerLaunchContext":context, + })).unwrap()); + let grant = GrantId::new(); + for attempt in 0..2 { + let error = grant_execution( + &state, + GrantExecutionRequest { + soul_id: SoulId::new(), + control_epoch: 1, + execution_generation: 1, + grant_id: grant.clone(), + fixture: None, + provider_path_inputs: Default::default(), + terminal: terminal.clone(), + fresh_agent: fresh.clone(), + resume_spec: Some(resume.clone()), + }, + ) + .await + .unwrap_err(); + assert_eq!( + error.code, + RuntimeErrorCode::HostUnreachable, + "{route}/{attempt}: retry must reach the source preparation failure" + ); + assert!( + error + .message + .starts_with("select .config/opencode/opencode.json:"), + "{route}: actual common-copy failure: {}", + error.message + ); + let persisted = state.persisted.lock().await; + assert!( + persisted.grant_id.is_none(), + "{route}: failed projection cannot consume the execution grant" + ); + assert_eq!(persisted.worker_launch_count, 0); + assert!(persisted.worker_pid.is_none()); + assert_eq!(std::fs::read(&prior).unwrap(), b"prior-accepted-source"); + } + assert!(state.pty.lock().await.is_none()); + assert!(state.fresh_agent.lock().await.is_none()); + } + } + + #[tokio::test] + async fn provider_secret_spawn_attempt_remains_consumed_after_launch_failure() { + for provider in ["opencode", "codex"] { + let owned = tempfile::tempdir().unwrap(); + let cwd = owned.path().to_str().unwrap(); + let program = owned.path().join("missing-owned-launcher"); + let terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"synthetic-spawn","streamId":"synthetic-stream","mode":provider, + "program":program,"args":[],"env":{"FRESHELL_OPENCODE_REBIND":"0"},"cwd":cwd,"workspacePath":cwd, + "runAsUid":unsafe { libc::geteuid() },"runAsGid":unsafe { libc::getegid() }, + "cols":80,"rows":24,"projectKey":"synthetic", + })) + .unwrap(); + let resume: ResumeSpec = serde_json::from_value(serde_json::json!({ + "providerSession":{"provider":provider,"providerStoreId":"synthetic-store","nativeSessionId":"ses_synthetic_spawn"}, + "mode":provider,"runtimeVariant":"managed_terminal_pty","program":program,"resumeArgv":[], + "environment":{"FRESHELL_OPENCODE_REBIND":"0"}, + "providerHome":"/home/freshell/provider","cwd":cwd,"workspacePath":cwd, + "identityProvenance":"provider_observed","durablePosition":{},"creationSeedRef":"synthetic-seed", + "checkpointRevision":0,"allocationState":"verified_durable","evidenceRevision":1,"neverDispatched":false, + })).unwrap(); + let state = test_host_state(); + let grant = GrantId::new(); + for (attempt, expected) in [ + RuntimeErrorCode::HostUnreachable, + RuntimeErrorCode::StaleExecutionGrant, + ] + .into_iter() + .enumerate() + { + let error = grant_execution( + &state, + GrantExecutionRequest { + soul_id: SoulId::new(), + control_epoch: 1, + execution_generation: 1, + grant_id: grant.clone(), + fixture: None, + provider_path_inputs: Default::default(), + terminal: Some(terminal.clone()), + fresh_agent: None, + resume_spec: Some(resume.clone()), + }, + ) + .await + .unwrap_err(); + assert_eq!( + error.code, expected, + "{provider}/{attempt}: failed native dispatch remains fenced" + ); + let persisted = state.persisted.lock().await; + assert_eq!(persisted.grant_id.as_ref(), Some(&grant)); + assert_eq!(persisted.worker_launch_count, 0); + assert!(persisted.worker_pid.is_none()); + } + assert!(state.pty.lock().await.is_none()); + } + } + #[tokio::test] async fn dispatch_rejects_wrong_role_before_any_host_mutation() { let state = test_host_state(); diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 07d5c6d34..9c9b05976 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -270,20 +270,6 @@ pub async fn prepare_terminal( // Keep the persisted/validated launch recipe separate from the final // child-only settings argument, just as credential environment is separate. apply_terminal_context(&mut terminal)?; - if terminal.mode == "codex" { - let mut sidecar_env = terminal.env.clone(); - sidecar_env.extend(child_env.clone()); - let prepared = codex::prepare(&terminal, &sidecar_env).await?; - let mut args = prepared.remote_args.to_vec(); - args.extend(terminal.args); - terminal.args = args; - return Ok(PreparedTerminal { - terminal, - codex: Some(prepared), - child_env, - claude_settings: None, - }); - } Ok(PreparedTerminal { terminal, codex: None, @@ -292,6 +278,23 @@ pub async fn prepare_terminal( }) } +/// Codex's app-server is native execution. The host calls this only after +/// consuming the grant, alongside the terminal process dispatch. +pub async fn start_terminal_sidecar( + mut prepared: PreparedTerminal, +) -> Result { + if prepared.terminal.mode == "codex" { + let mut sidecar_env = prepared.terminal.env.clone(); + sidecar_env.extend(prepared.child_env.clone()); + let codex = codex::prepare(&prepared.terminal, &sidecar_env).await?; + let mut args = codex.remote_args.to_vec(); + args.extend(prepared.terminal.args); + prepared.terminal.args = args; + prepared.codex = Some(codex); + } + Ok(prepared) +} + async fn prepare_scoped_mcp_environment( context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, child_env: &mut std::collections::BTreeMap, @@ -722,6 +725,33 @@ mod provider_secret_context_tests { McpCapabilityReference, ProviderLaunchContext, ProviderPreparation, }; + #[tokio::test] + async fn provider_secret_codex_preparation_defers_native_dispatch() { + let owned = tempfile::tempdir().unwrap(); + // A native process could not start from this cwd. Preparation must + // still finish without dispatching it before the grant is consumed. + let cwd = owned.path().join("missing-native-cwd"); + let terminal = serde_json::from_value(serde_json::json!({ + "terminalId":"synthetic-codex", "streamId":"synthetic-stream", "mode":"codex", "program":"codex", + "args":["--model","synthetic-model"],"cwd":cwd,"workspacePath":owned.path(), + "runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"synthetic", + "env":{} + })).unwrap(); + let secrets = std::collections::BTreeMap::from([( + "SYNTHETIC_CREDENTIAL".into(), + "synthetic-child-only-value".into(), + )]); + let prepared = prepare_terminal(terminal, &secrets, None, None) + .await + .expect("source preparation cannot dispatch the Codex app-server"); + assert_eq!(prepared.terminal.args, ["--model", "synthetic-model"]); + assert!(prepared.codex.is_none()); + assert_eq!(prepared.child_env, secrets); + assert!(!serde_json::to_string(&prepared.terminal) + .unwrap() + .contains("synthetic-child-only-value")); + } + #[tokio::test] async fn provider_secret_inline_configuration_stays_only_in_actual_child_environment() { let owned = tempfile::tempdir().unwrap(); From 46922560e75ce9522274adc1ab0c282025117271 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 20:23:19 -0700 Subject: [PATCH 105/493] Add finite actual-process Claude source observations --- crates/freshell-platform/src/claude_inputs.rs | 1 + .../src/claude_inputs/git.rs | 3 +- .../src/claude_inputs/observation.rs | 562 ++++++++++++++++++ .../src/managed_mcp_capability.rs | 208 ++++++- ...managed_provider_claude_local_consumer.cjs | 28 + .../managed_provider_claude_local_probe.mjs | 59 ++ .../src/managed_provider_sdk_tests.rs | 1 + .../src/claude_source_observation.rs | 118 ++++ crates/freshell-session-host/src/main.rs | 2 + .../tests/claude_source_observation.rs | 149 +++++ 10 files changed, 1129 insertions(+), 2 deletions(-) create mode 100644 crates/freshell-platform/src/claude_inputs/observation.rs create mode 100644 crates/freshell-server/src/managed_provider_claude_local_consumer.cjs create mode 100644 crates/freshell-server/src/managed_provider_claude_local_probe.mjs create mode 100644 crates/freshell-session-host/src/claude_source_observation.rs create mode 100644 crates/freshell-session-host/tests/claude_source_observation.rs diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index cdc7ae0e0..d3e29d0fd 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -3,6 +3,7 @@ use serde::{Deserialize, Serialize}; use std::path::{Path, PathBuf}; pub mod git; +pub mod observation; #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] diff --git a/crates/freshell-platform/src/claude_inputs/git.rs b/crates/freshell-platform/src/claude_inputs/git.rs index 6fbf151db..8635cd43b 100644 --- a/crates/freshell-platform/src/claude_inputs/git.rs +++ b/crates/freshell-platform/src/claude_inputs/git.rs @@ -3,7 +3,8 @@ use super::NativeSourceRuntime; use std::path::{Path, PathBuf}; -#[derive(Debug, Clone)] +#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct Roots { pub checkout: PathBuf, pub canonical: PathBuf, diff --git a/crates/freshell-platform/src/claude_inputs/observation.rs b/crates/freshell-platform/src/claude_inputs/observation.rs new file mode 100644 index 000000000..5c70f8796 --- /dev/null +++ b/crates/freshell-platform/src/claude_inputs/observation.rs @@ -0,0 +1,562 @@ +//! Finite Claude source observations made by the actual reader process. +//! The caller owns namespace/attempt qualification; this reader accepts no +//! claimed uid and never reads settings payloads or credentials. +use super::{git, ClaudeSettingSource, NativeSourceRuntime}; +use serde::{Deserialize, Serialize}; +#[cfg(unix)] +use std::os::unix::fs::MetadataExt; +use std::path::{Path, PathBuf}; + +pub const MAX_REQUEST_BYTES: usize = 256 * 1024; +pub const MAX_REPLY_BYTES: usize = 1024 * 1024; + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ObservationRequest { + pub version: u8, + pub request_id: String, + pub native_source_runtime: NativeSourceRuntime, + pub cwd: PathBuf, + pub home: PathBuf, + pub setting_sources: Vec, + pub selector_manifest_sha256: String, + pub plan_revision: u8, +} + +impl ObservationRequest { + pub fn validate(&self) -> Result<(), String> { + if self.version != 1 + || self.plan_revision > 2 + || self.request_id.len() != 36 + || !self.request_id.bytes().enumerate().all(|(index, byte)| { + if matches!(index, 8 | 13 | 18 | 23) { + byte == b'-' + } else { + byte.is_ascii_hexdigit() + } + }) + || self.selector_manifest_sha256.len() != 64 + || !self + .selector_manifest_sha256 + .bytes() + .all(|byte| byte.is_ascii_hexdigit()) + { + return Err( + "Claude observation request binding is invalid; retain inputs and reprepare".into(), + ); + } + validate_sources(&self.setting_sources)?; + validate_path(&self.cwd)?; + validate_path(&self.home)?; + Ok(()) + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ObservationReply { + pub request_sha256: String, + pub observation: NativeObservation, +} + +impl ObservationReply { + pub fn validate_for_request( + &self, + request: &ObservationRequest, + request_sha256: &str, + ) -> Result<(), String> { + request.validate()?; + self.observation.validate()?; + let observed = &self.observation; + if request_sha256.len() != 64 + || !request_sha256.bytes().all(|byte| byte.is_ascii_hexdigit()) + || self.request_sha256 != request_sha256 + || observed.native_source_runtime != request.native_source_runtime + || observed.setting_sources != request.setting_sources + || observed.cwd != request.cwd + || observed.home.as_ref() != Some(&request.home) + || observed.uid != observed.euid + || observed.gid != observed.egid + { + return Err("Claude source observation does not match the selected native request; retain inputs and reprepare".into()); + } + Ok(()) + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct NativeObservation { + pub version: u8, + pub native_source_runtime: NativeSourceRuntime, + pub setting_sources: Vec, + pub uid: u32, + pub euid: u32, + pub gid: u32, + pub egid: u32, + pub cwd: PathBuf, + pub home: Option, + pub roots: Option, + pub local_decision: LocalDecision, + pub root_uid: Option, + pub git_uid: Option, + pub claude_uid: Option, + pub local_settings: Vec, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum LocalDecision { + Disabled, + NoRoot, + CwdRoot, + HomeUnavailable, + HomeRoot, + RootUnavailable, + RootOwnerMismatch, + GitUnavailable, + GitOwnerMismatch, + ClaudeUnavailable, + ClaudeOwnerMismatch, + QualifiedCanonical, +} + +#[cfg(unix)] +pub fn observe( + cwd: &Path, + home: &Path, + runtime: NativeSourceRuntime, + setting_sources: &[ClaudeSettingSource], +) -> Result { + validate_sources(setting_sources)?; + validate_path(cwd)?; + validate_path(home)?; + let cwd = std::fs::canonicalize(cwd) + .map_err(|_| "Claude native launch cwd is unavailable; retain inputs and repair it")?; + if !std::fs::metadata(&cwd).is_ok_and(|metadata| metadata.is_dir()) { + return Err( + "Claude native launch cwd is not a directory; retain inputs and repair it".into(), + ); + } + let local_enabled = setting_sources.contains(&ClaudeSettingSource::Local); + let roots = if local_enabled || setting_sources.contains(&ClaudeSettingSource::Project) { + git::roots(&cwd, runtime)? + } else { + None + }; + let home = std::fs::canonicalize(home).ok(); + let mut observation = NativeObservation { + version: 1, + native_source_runtime: runtime, + setting_sources: setting_sources.to_vec(), + uid: unsafe { libc::getuid() }, + euid: unsafe { libc::geteuid() }, + gid: unsafe { libc::getgid() }, + egid: unsafe { libc::getegid() }, + local_decision: LocalDecision::Disabled, + root_uid: None, + git_uid: None, + claude_uid: None, + local_settings: if local_enabled { + vec![cwd.join(".claude/settings.local.json")] + } else { + Vec::new() + }, + cwd, + home, + roots, + }; + if local_enabled { + observation.local_decision = select_local(&mut observation); + } + if observation.local_decision == LocalDecision::QualifiedCanonical { + observation.local_settings.push( + observation + .roots + .as_ref() + .unwrap() + .canonical + .join(".claude/settings.local.json"), + ); + } + observation.validate()?; + Ok(observation) +} + +fn validate_sources(sources: &[ClaudeSettingSource]) -> Result<(), String> { + if sources.len() > 3 + || sources + .iter() + .enumerate() + .any(|(index, source)| sources[..index].contains(source)) + { + return Err( + "Claude source observation flags are invalid; retain inputs and reprepare".into(), + ); + } + Ok(()) +} + +fn validate_path(path: &Path) -> Result<(), String> { + if !path.is_absolute() + || path.as_os_str().len() > 64 * 1024 + || path + .to_str() + .is_none_or(|raw| raw.chars().any(char::is_control)) + { + return Err("Claude source observation path profile exceeded; retain inputs and repair the selected path".into()); + } + Ok(()) +} + +impl NativeObservation { + pub fn validate(&self) -> Result<(), String> { + validate_sources(&self.setting_sources)?; + let local_enabled = self.setting_sources.contains(&ClaudeSettingSource::Local); + if self.version != 1 + || self.local_settings.len() > 2 + || local_enabled != (self.local_decision != LocalDecision::Disabled) + || local_enabled != !self.local_settings.is_empty() + || (!self.local_settings.is_empty() + && self.local_settings[0] != self.cwd.join(".claude/settings.local.json")) + || (!local_enabled + && (self.root_uid.is_some() || self.git_uid.is_some() || self.claude_uid.is_some())) + || (!local_enabled + && !self.setting_sources.contains(&ClaudeSettingSource::Project) + && self.roots.is_some()) + || !self.decision_evidence_matches() + || (self.local_decision == LocalDecision::QualifiedCanonical) + != (self.local_settings.len() == 2) + || (self.local_settings.len() == 2 + && self.roots.as_ref().is_none_or(|roots| { + roots.canonical == self.cwd + || self.local_settings[1] + != roots.canonical.join(".claude/settings.local.json") + })) + { + return Err( + "Claude source observation metadata is invalid; retain inputs and reprepare".into(), + ); + } + for path in std::iter::once(&self.cwd) + .chain(self.home.iter()) + .chain( + self.roots + .iter() + .flat_map(|roots| [&roots.checkout, &roots.canonical]), + ) + .chain(&self.local_settings) + { + validate_path(path)?; + } + Ok(()) + } + + fn decision_evidence_matches(&self) -> bool { + let ownership_absent = + self.root_uid.is_none() && self.git_uid.is_none() && self.claude_uid.is_none(); + match self.local_decision { + LocalDecision::Disabled => ownership_absent, + LocalDecision::NoRoot => self.roots.is_none() && ownership_absent, + LocalDecision::CwdRoot => { + self.roots + .as_ref() + .is_some_and(|roots| roots.canonical == self.cwd) + && ownership_absent + } + decision => { + let Some(roots) = &self.roots else { + return false; + }; + if roots.canonical == self.cwd { + return false; + } + match decision { + LocalDecision::HomeUnavailable => self.home.is_none() && ownership_absent, + LocalDecision::HomeRoot => { + self.home.as_ref() == Some(&roots.canonical) && ownership_absent + } + _ => { + if self + .home + .as_ref() + .is_none_or(|home| home == &roots.canonical) + { + return false; + } + match decision { + LocalDecision::RootUnavailable => ownership_absent, + LocalDecision::RootOwnerMismatch => { + self.root_uid.is_some_and(|uid| uid != self.uid) + && self.git_uid.is_none() + && self.claude_uid.is_none() + } + LocalDecision::GitUnavailable => { + self.root_uid == Some(self.uid) + && self.git_uid.is_none() + && self.claude_uid.is_none() + } + LocalDecision::GitOwnerMismatch => { + self.root_uid == Some(self.uid) + && self.git_uid.is_some_and(|uid| uid != self.uid) + && self.claude_uid.is_none() + } + LocalDecision::ClaudeUnavailable => { + self.root_uid == Some(self.uid) + && self.git_uid == Some(self.uid) + && self.claude_uid.is_none() + } + LocalDecision::ClaudeOwnerMismatch => { + self.root_uid == Some(self.uid) + && self.git_uid == Some(self.uid) + && self.claude_uid.is_some_and(|uid| uid != self.uid) + } + LocalDecision::QualifiedCanonical => { + self.root_uid == Some(self.uid) + && self.git_uid == Some(self.uid) + && self.claude_uid.is_none_or(|uid| uid == self.uid) + } + _ => false, + } + } + } + } + } + } +} + +#[cfg(unix)] +fn select_local(observation: &mut NativeObservation) -> LocalDecision { + let Some(roots) = &observation.roots else { + return LocalDecision::NoRoot; + }; + let root = &roots.canonical; + if root == &observation.cwd { + return LocalDecision::CwdRoot; + } + let Some(home) = &observation.home else { + return LocalDecision::HomeUnavailable; + }; + if root == home { + return LocalDecision::HomeRoot; + } + let Ok(metadata) = std::fs::metadata(root) else { + return LocalDecision::RootUnavailable; + }; + observation.root_uid = Some(metadata.uid()); + if metadata.uid() != observation.uid { + return LocalDecision::RootOwnerMismatch; + } + let Ok(metadata) = std::fs::symlink_metadata(root.join(".git")) else { + return LocalDecision::GitUnavailable; + }; + observation.git_uid = Some(metadata.uid()); + if metadata.uid() != observation.uid { + return LocalDecision::GitOwnerMismatch; + } + match std::fs::symlink_metadata(root.join(".claude")) { + Ok(metadata) => { + observation.claude_uid = Some(metadata.uid()); + if metadata.uid() != observation.uid { + return LocalDecision::ClaudeOwnerMismatch; + } + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return LocalDecision::ClaudeUnavailable, + } + LocalDecision::QualifiedCanonical +} + +#[cfg(all(test, unix))] +mod tests { + use super::*; + + #[test] + fn claude_native_observation_reply_cannot_be_reused_for_another_selection() { + let owned = tempfile::tempdir().unwrap(); + let request = ObservationRequest { + version: 1, + request_id: "01234567-89ab-4cde-8fab-0123456789ab".into(), + native_source_runtime: NativeSourceRuntime::Terminal, + cwd: owned.path().into(), + home: owned.path().into(), + setting_sources: vec![ClaudeSettingSource::Local], + selector_manifest_sha256: "a".repeat(64), + plan_revision: 0, + }; + let reply = ObservationReply { + request_sha256: "b".repeat(64), + observation: observe( + &request.cwd, + &request.home, + request.native_source_runtime, + &request.setting_sources, + ) + .unwrap(), + }; + reply + .validate_for_request(&request, &"b".repeat(64)) + .unwrap(); + for field in [ + "request_digest", + "runtime", + "cwd", + "home", + "sources", + "identity", + ] { + let mut changed = reply.clone(); + match field { + "request_digest" => changed.request_sha256 = "c".repeat(64), + "runtime" => changed.observation.native_source_runtime = NativeSourceRuntime::Sdk, + "cwd" => { + changed.observation.cwd = owned.path().join("other"); + changed.observation.local_settings = + vec![changed.observation.cwd.join(".claude/settings.local.json")]; + } + "home" => changed.observation.home = Some(owned.path().join("other-home")), + "sources" => { + changed.observation.setting_sources.clear(); + changed.observation.local_settings.clear(); + changed.observation.local_decision = LocalDecision::Disabled; + } + "identity" => changed.observation.euid = changed.observation.uid.wrapping_add(1), + _ => unreachable!(), + } + assert!( + changed + .validate_for_request(&request, &"b".repeat(64)) + .is_err(), + "different {field} must not qualify the selected native source request" + ); + } + } + + #[test] + fn claude_native_observation_rejects_non_directory_cwd_without_changing_prior_inputs() { + let owned = tempfile::tempdir().unwrap(); + let prior = owned.path().join("prior-accepted-inputs"); + std::fs::write(&prior, b"prior-accepted-inputs").unwrap(); + let error = observe( + &prior, + owned.path(), + NativeSourceRuntime::Terminal, + &super::super::default_setting_sources(), + ) + .expect_err("a file cannot be accepted as the native launch cwd"); + assert!(error.contains("launch cwd is not a directory")); + assert_eq!(std::fs::read(&prior).unwrap(), b"prior-accepted-inputs"); + } + + #[test] + fn claude_native_observation_consumes_cwd_then_qualified_canonical_local_settings() { + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let main = owned.path().join("canonical"); + let checkout = owned.path().join("checkout"); + let cwd = checkout.join("nested"); + let gitdir = main.join(".git/worktrees/fixture"); + for path in [&home, &gitdir, &cwd.join(".claude"), &main.join(".claude")] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + checkout.join(".git"), + format!("gitdir: {}\n", gitdir.display()), + ) + .unwrap(); + std::fs::write(gitdir.join("commondir"), "../..\n").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", checkout.join(".git").display()), + ) + .unwrap(); + std::fs::write( + cwd.join(".claude/settings.local.json"), + br#"{"env":{"SELECTED":"cwd","CWD_ONLY":"retained"}}"#, + ) + .unwrap(); + std::fs::write( + main.join(".claude/settings.local.json"), + br#"{"env":{"SELECTED":"canonical"}}"#, + ) + .unwrap(); + let observed = observe( + &cwd, + &home, + runtime, + &super::super::default_setting_sources(), + ) + .unwrap(); + assert_eq!(observed.uid, unsafe { libc::getuid() }); + assert_eq!(observed.euid, unsafe { libc::geteuid() }); + assert_eq!(observed.roots.as_ref().unwrap().canonical, main); + let mut consumed = serde_json::Map::new(); + for path in &observed.local_settings { + let settings: serde_json::Value = + serde_json::from_slice(&std::fs::read(path).unwrap()).unwrap(); + consumed.extend(settings["env"].as_object().unwrap().clone()); + } + assert_eq!( + consumed["SELECTED"], "canonical", + "canonical local settings override cwd local settings" + ); + assert_eq!(consumed["CWD_ONLY"], "retained"); + assert_eq!( + observed.local_settings, + [ + cwd.join(".claude/settings.local.json"), + main.join(".claude/settings.local.json") + ] + ); + } + } + + #[test] + fn claude_native_observation_disabled_sources_leave_unselected_git_metadata_untouched() { + let owned = tempfile::tempdir().unwrap(); + let marker = owned.path().join(".git"); + let file = std::fs::File::create(&marker).unwrap(); + let bytes = crate::provider_input_profiles::MAX_SOURCE_VIEW_BYTES + 1; + file.set_len(bytes).unwrap(); + let observed = observe(owned.path(), owned.path(), NativeSourceRuntime::Sdk, &[]) + .expect("disabled project/local sources cannot select oversized Git metadata"); + assert!(observed.roots.is_none()); + assert!(observed.local_settings.is_empty()); + assert_eq!(std::fs::metadata(&marker).unwrap().len(), bytes); + } + + #[test] + fn claude_native_observation_rejects_incomplete_or_contradictory_qualified_metadata() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let root = owned.path().join("root"); + let cwd = root.join("nested"); + for path in [&home, &root.join(".git"), &cwd] { + std::fs::create_dir_all(path).unwrap(); + } + let accepted = observe( + &cwd, + &home, + NativeSourceRuntime::Terminal, + &[ClaudeSettingSource::Local], + ) + .unwrap(); + assert_eq!(accepted.local_decision, LocalDecision::QualifiedCanonical); + for field in ["rootUid", "gitUid", "uid", "home", "roots"] { + let mut transport = serde_json::to_value(&accepted).unwrap(); + if field == "uid" { + transport[field] = serde_json::json!(accepted.uid.wrapping_add(1)); + } else { + transport[field] = serde_json::Value::Null; + } + let reply: NativeObservation = serde_json::from_value(transport).unwrap(); + assert!(reply.validate().is_err(), "incomplete or contradictory {field} must block instead of qualifying a canonical source"); + } + assert!(accepted.validate().is_ok()); + assert_eq!( + std::fs::metadata(root.join(".git")).unwrap().uid(), + unsafe { libc::getuid() } + ); + } +} diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index ce07dbb96..945781de4 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -35,7 +35,7 @@ mod ordinary_stage; mod source_stage; #[cfg(all(test, unix, feature = "managed-provider-qualification"))] -pub(crate) use tests::qualify_claude_linked_sources; +pub(crate) use tests::{qualify_claude_linked_sources, qualify_claude_local_settings}; #[derive(Clone, Copy, Default)] pub(crate) struct OpencodeEphemeralInput<'a> { @@ -4468,6 +4468,212 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .is_dir()); } + #[cfg(feature = "managed-provider-qualification")] + pub(crate) async fn qualify_claude_local_settings(base: &Path) { + use freshell_platform::claude_inputs::{ + observation::observe, ClaudeSettingSource, NativeSourceRuntime, + }; + use std::os::unix::fs::PermissionsExt; + use tokio::io::{AsyncBufReadExt, AsyncReadExt, BufReader}; + assert_eq!(unsafe { libc::getuid() }, 65534); + assert_eq!(unsafe { libc::getgid() }, 0); + let probe = base.join("local-settings-probe.mjs"); + let consumer = base.join("local-settings-consumer.cjs"); + std::fs::write( + &probe, + include_str!("managed_provider_claude_local_probe.mjs"), + ) + .unwrap(); + std::fs::write( + &consumer, + include_str!("managed_provider_claude_local_consumer.cjs"), + ) + .unwrap(); + for case in [ + "qualified-linked", + "cwd-root", + "home-root", + "unreadable-marker", + "project-only", + "disabled", + ] { + let fixture = base.join(format!("local-{case}")); + let main = fixture.join("canonical"); + let checkout = fixture.join("checkout"); + let cwd = if case == "cwd-root" { + main.clone() + } else if case == "home-root" { + main.join("nested") + } else { + checkout.join("nested") + }; + let home = if case == "home-root" { + main.clone() + } else { + fixture.join("home") + }; + let gitdir = main.join(".git/worktrees/fixture"); + for directory in [ + &gitdir, + &home, + &cwd.join(".claude"), + &main.join(".claude"), + &checkout.join(".claude"), + ] { + std::fs::create_dir_all(directory).unwrap(); + } + std::fs::write( + checkout.join(".git"), + format!("gitdir: {}\n", gitdir.display()), + ) + .unwrap(); + std::fs::write(gitdir.join("commondir"), "../..\n").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", checkout.join(".git").display()), + ) + .unwrap(); + let documents = [ + ( + main.join(".claude/settings.local.json"), + serde_json::json!({"env":{"FRESHELL_LOCAL_SELECTED_NONCE":format!("{case}-canonical")}}), + ), + ( + checkout.join(".claude/settings.local.json"), + serde_json::json!({"env":{"FRESHELL_LOCAL_SELECTED_NONCE":format!("{case}-checkout")}}), + ), + ( + cwd.join(".claude/settings.local.json"), + serde_json::json!({"env":{"FRESHELL_LOCAL_SELECTED_NONCE":format!("{case}-cwd"),"FRESHELL_LOCAL_CWD_ONLY_NONCE":format!("{case}-retained")}}), + ), + ]; + for (path, document) in &documents { + std::fs::write(path, serde_json::to_vec(document).unwrap()).unwrap(); + } + let prior = documents + .iter() + .map(|(path, _)| (path.clone(), std::fs::read(path).unwrap())) + .collect::>(); + if case == "unreadable-marker" { + std::fs::set_permissions(checkout.join(".git"), std::fs::Permissions::from_mode(0)) + .unwrap(); + } + let sources = match case { + "project-only" => vec![ClaudeSettingSource::Project], + "disabled" => vec![], + _ => vec![ClaudeSettingSource::Local], + }; + let observation = observe(&cwd, &home, NativeSourceRuntime::Sdk, &sources).unwrap(); + observation.validate().unwrap(); + assert_eq!(observation.uid, 65534); + assert_eq!(observation.euid, 65534); + assert_eq!(observation.gid, 0); + assert_eq!(observation.egid, 0); + if case == "unreadable-marker" { + assert_eq!(observation.roots.as_ref().unwrap().canonical, checkout); + } + let mut selected = serde_json::Map::new(); + for candidate in &observation.local_settings { + if candidate.is_file() { + let value: serde_json::Value = + serde_json::from_slice(&std::fs::read(candidate).unwrap()).unwrap(); + selected.extend(value["env"].as_object().unwrap().clone()); + } + } + let expected = |name: &str| { + format!( + "{:x}", + Sha256::digest( + selected + .get(name) + .and_then(serde_json::Value::as_str) + .unwrap_or("") + .as_bytes() + ) + ) + }; + let consumed = fixture.join("native-consumed.json"); + let selector = sources + .iter() + .map(|source| source.as_str()) + .collect::>() + .join(","); + let mut child = tokio::process::Command::new("node") + .arg(&probe) + .arg(&cwd) + .arg(selector) + .arg(&consumer) + .arg(&consumed) + .current_dir(&cwd) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .env("HOME", &home) + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::null()) + .process_group(0) + .kill_on_drop(true) + .spawn() + .unwrap(); + let mut output = BufReader::new(child.stdout.take().unwrap()).take(4097); + let mut line = Vec::new(); + let received = tokio::time::timeout( + std::time::Duration::from_secs(25), + output.read_until(b'\n', &mut line), + ) + .await; + stop_codex_source_child(&mut child).await; + received + .expect("native local-settings consumption must complete within the finite budget") + .unwrap(); + assert!( + line.len() <= 4096 && line.last() == Some(&b'\n'), + "native metadata receipt must be complete and bounded" + ); + let receipt: serde_json::Value = serde_json::from_slice(&line).unwrap(); + assert_eq!( + receipt["failed"], false, + "classified native settings consumption failed: {case}" + ); + for field in ["defaultChild", "zeroPrompts"] { + assert_eq!(receipt[field], true, "native local receipt {field}: {case}"); + } + for field in ["uid", "consumerUid"] { + assert_eq!(receipt[field], 65534); + } + for field in ["gid", "consumerGid"] { + assert_eq!(receipt[field], 0); + } + assert_eq!( + receipt["selectedSha256"], + expected("FRESHELL_LOCAL_SELECTED_NONCE"), + "native selected local ordering: {case}" + ); + assert_eq!( + receipt["cwdOnlySha256"], + expected("FRESHELL_LOCAL_CWD_ONLY_NONCE"), + "native cwd-only local retention: {case}" + ); + if case == "unreadable-marker" { + std::fs::set_permissions( + checkout.join(".git"), + std::fs::Permissions::from_mode(0o644), + ) + .unwrap(); + } + for (path, bytes) in prior { + assert!( + std::fs::read(path).unwrap() == bytes, + "native settings consumer must retain its original inputs" + ); + } + println!( + "provider_native_local_selection {}", + serde_json::json!({"provider":"claude","case":case,"uid":65534,"gid":0,"ownedGroupJoined":true,"zeroPrompts":true,"digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&receipt).unwrap()))}) + ); + } + } + #[cfg(feature = "managed-provider-qualification")] pub(crate) async fn qualify_claude_linked_sources(base: &Path) { use freshell_platform::{ diff --git a/crates/freshell-server/src/managed_provider_claude_local_consumer.cjs b/crates/freshell-server/src/managed_provider_claude_local_consumer.cjs new file mode 100644 index 000000000..5eea63f04 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_claude_local_consumer.cjs @@ -0,0 +1,28 @@ +const fs = require('node:fs') +const crypto = require('node:crypto') +const readline = require('node:readline') +const hash = value => crypto.createHash('sha256').update(value || '').digest('hex') +fs.writeFileSync(process.argv[2], JSON.stringify({ + selectedSha256: hash(process.env.FRESHELL_LOCAL_SELECTED_NONCE), + cwdOnlySha256: hash(process.env.FRESHELL_LOCAL_CWD_ONLY_NONCE), + consumerUid: process.geteuid(), consumerGid: process.getegid(), +})) +readline.createInterface({ input: process.stdin }).on('line', line => { + let request + try { request = JSON.parse(line) } catch { return } + if (request.id === undefined) return + let result + switch (request.method) { + case 'initialize': + result = { protocolVersion: '2024-11-05', capabilities: { tools: {} }, + serverInfo: { name: 'closed-local-observer', version: '1.0.0' } } + break + case 'tools/list': result = { tools: [] }; break + case 'ping': result = {}; break + default: + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, + error: { code: -32601, message: 'unsupported fixture method' } }) + '\n') + return + } + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n') +}) diff --git a/crates/freshell-server/src/managed_provider_claude_local_probe.mjs b/crates/freshell-server/src/managed_provider_claude_local_probe.mjs new file mode 100644 index 000000000..f929d7ab5 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_claude_local_probe.mjs @@ -0,0 +1,59 @@ +import { spawn } from 'node:child_process' +import { existsSync, readFileSync } from 'node:fs' +import { query } from '/opt/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk/sdk.mjs' + +// The Rust fixture owns, terminates and joins this dedicated process group. +// The actual native MCP child records only hashes of the synthetic settings. +const [cwd, sources, consumer, consumed] = process.argv.slice(2) +const abort = new AbortController() +let prompts = 0, defaultChild = false, q +async function* emptyPrompt() { + await new Promise(resolve => abort.signal.addEventListener('abort', resolve, { once: true })) +} +let receipt = { failed: true, stage: 'native-local-startup' } +try { + q = query({ prompt: emptyPrompt(), options: { + cwd, settingSources: sources ? sources.split(',') : [], abortController: abort, + env: { PATH: process.env.PATH, HOME: process.env.HOME, + ANTHROPIC_API_KEY: 'synthetic-local-only', ANTHROPIC_BASE_URL: 'http://127.0.0.1:9' }, + mcpServers: { localObserver: { command: '/usr/local/bin/node', args: [consumer, consumed] } }, + spawnClaudeCodeProcess: options => { + defaultChild = options.command.includes('@anthropic-ai/claude-agent-sdk-linux-x64/') + && options.command.endsWith('/claude') + const child = spawn(options.command, options.args, { + cwd: options.cwd, env: options.env, stdio: ['pipe', 'pipe', 'pipe'], + }) + child.stderr.resume() + const write = child.stdin.write + let pending = '' + child.stdin.write = function(chunk, ...args) { + pending += chunk.toString() + let boundary + while ((boundary = pending.indexOf('\n')) >= 0) { + const frame = pending.slice(0, boundary) + pending = pending.slice(boundary + 1) + try { if (JSON.parse(frame).type === 'user') prompts++ } catch {} + } + if (pending.length > 1024 * 1024) throw new Error('fixture-native-input-profile') + return write.call(this, chunk, ...args) + } + return child + }, + } }) + await q.initializationResult() + for (let index = 0; index < 100; index++) { + const statuses = await q.mcpServerStatus() + if (statuses.some(status => status.name === 'localObserver' && status.status === 'connected') + && existsSync(consumed)) { + const selected = JSON.parse(readFileSync(consumed, 'utf8')) + receipt = { failed: false, defaultChild, zeroPrompts: prompts === 0, + uid: process.geteuid(), gid: process.getegid(), ...selected } + break + } + await new Promise(resolve => setTimeout(resolve, 100)) + } +} catch { + receipt = { failed: true, stage: 'native-local-startup', zeroPrompts: prompts === 0 } +} +process.stdout.write(JSON.stringify(receipt) + '\n') +await new Promise(() => {}) diff --git a/crates/freshell-server/src/managed_provider_sdk_tests.rs b/crates/freshell-server/src/managed_provider_sdk_tests.rs index 9b0ccd8e1..c30c7569f 100644 --- a/crates/freshell-server/src/managed_provider_sdk_tests.rs +++ b/crates/freshell-server/src/managed_provider_sdk_tests.rs @@ -58,4 +58,5 @@ async fn managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_pro }) ); crate::managed_mcp_capability::qualify_claude_linked_sources(fixture.path()).await; + crate::managed_mcp_capability::qualify_claude_local_settings(fixture.path()).await; } diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs new file mode 100644 index 000000000..9b12a0c60 --- /dev/null +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -0,0 +1,118 @@ +//! One finite metadata request from an owned parent. No provider is dispatched; +//! request and reply bodies use private pipes, never command arguments or logs. +use freshell_platform::claude_inputs::observation::{ + observe, ObservationReply, ObservationRequest, MAX_REPLY_BYTES, MAX_REQUEST_BYTES, +}; +use sha2::{Digest, Sha256}; +use std::io::{Read, Write}; + +pub(crate) fn serve_stdio() -> Result<(), String> { + exchange(&mut std::io::stdin().lock(), &mut std::io::stdout().lock()) +} + +fn exchange(input: &mut impl Read, output: &mut impl Write) -> Result<(), String> { + let mut length = [0; 4]; + input + .read_exact(&mut length) + .map_err(|_| "Claude observation request header unavailable")?; + let length = u32::from_be_bytes(length) as usize; + if length > MAX_REQUEST_BYTES { + return Err( + "Claude observation request exceeds the metadata profile; retain inputs and reprepare" + .into(), + ); + } + let mut raw = Vec::new(); + raw.try_reserve_exact(length) + .map_err(|_| "Claude observation request allocation unavailable")?; + raw.resize(length, 0); + input + .read_exact(&mut raw) + .map_err(|_| "Claude observation request incomplete")?; + let request: ObservationRequest = serde_json::from_slice(&raw) + .map_err(|_| "Claude observation request is invalid; retain inputs and reprepare")?; + request.validate()?; + let cwd = std::env::current_dir().map_err(|_| { + "Claude observation native launch cwd is unavailable; retain inputs and repair it" + })?; + let home = std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .ok_or("Claude observation native HOME is unavailable; retain inputs and repair it")?; + if cwd != request.cwd || home != request.home { + return Err("Claude observation native launch cwd or HOME changed; retain inputs and reprepare before dispatch".into()); + } + let observation = observe( + &cwd, + &home, + request.native_source_runtime, + &request.setting_sources, + )?; + let reply = ObservationReply { + request_sha256: format!("{:x}", Sha256::digest(&raw)), + observation, + }; + let bytes = + freshell_runtime_protocol::bounded_json_bytes(&reply, MAX_REPLY_BYTES).map_err(|_| { + "Claude observation reply exceeds the metadata profile; retain inputs and reprepare" + })?; + output + .write_all(&(bytes.len() as u32).to_be_bytes()) + .map_err(|_| "Claude observation response header unavailable")?; + output + .write_all(&bytes) + .map_err(|_| "Claude observation response incomplete")?; + output + .flush() + .map_err(|_| "Claude observation response incomplete")?; + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::io::{Cursor, Error, ErrorKind}; + + struct HeaderOnly { + header: Cursor<[u8; 4]>, + body_reads: usize, + } + impl Read for HeaderOnly { + fn read(&mut self, bytes: &mut [u8]) -> std::io::Result { + if self.header.position() == 4 { + self.body_reads += 1; + return Err(Error::new(ErrorKind::Other, "body must remain unread")); + } + self.header.read(bytes) + } + } + + #[test] + fn claude_observation_rejects_oversized_header_before_reading_or_allocating_body() { + for length in [MAX_REQUEST_BYTES as u32 + 1, u32::MAX] { + let mut input = HeaderOnly { + header: Cursor::new(length.to_be_bytes()), + body_reads: 0, + }; + let mut output = Vec::new(); + let error = exchange(&mut input, &mut output).unwrap_err(); + assert!(error.contains("exceeds the metadata profile")); + assert_eq!(input.body_reads, 0); + assert!(output.is_empty()); + } + } + + #[test] + fn claude_observation_incomplete_request_never_emits_partial_evidence_or_private_input() { + let private = b"PRIVATE_INCOMPLETE_METADATA_SENTINEL"; + let mut input = Vec::from((private.len() as u32 + 1).to_be_bytes()); + input.extend(private); + let mut output = Vec::new(); + let error = exchange(&mut Cursor::new(input), &mut output).unwrap_err(); + assert!(error.contains("request incomplete")); + assert!(!error + .as_bytes() + .windows(private.len()) + .any(|window| window == private)); + assert!(output.is_empty()); + } +} diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 99891e514..8588e408b 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1,3 +1,4 @@ +mod claude_source_observation; mod command_journal; mod control; mod output_journal; @@ -117,6 +118,7 @@ async fn run() -> Result<(), String> { match args.get(1).map(String::as_str) { Some("serve") => serve(&args[2..]).await, Some("worker") => worker(&args[2..]).await, + Some("observe-claude-sources") if args.len() == 2 => claude_source_observation::serve_stdio(), Some("native-auth-only") if args.len() == 2 => { verify_private_helper_namespace( Path::new("/home/freshell/provider"), diff --git a/crates/freshell-session-host/tests/claude_source_observation.rs b/crates/freshell-session-host/tests/claude_source_observation.rs new file mode 100644 index 000000000..d9091266c --- /dev/null +++ b/crates/freshell-session-host/tests/claude_source_observation.rs @@ -0,0 +1,149 @@ +#![cfg(unix)] + +use serde_json::{json, Value}; +use sha2::{Digest, Sha256}; +use std::{os::unix::process::CommandExt, path::Path, process::Stdio, time::Duration}; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; + +async fn read_metadata(cwd: &Path, home: &Path, request: &[u8]) -> Result, &'static str> { + let mut command = tokio::process::Command::new(env!("CARGO_BIN_EXE_freshell-session-host")); + command + .arg("observe-claude-sources") + .current_dir(cwd) + .env_clear() + .env("HOME", home) + .env("PATH", "/usr/bin:/bin") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .kill_on_drop(true); + command.as_std_mut().process_group(0); + let mut child = command + .spawn() + .map_err(|_| "owned reader could not start")?; + let mut input = child.stdin.take().unwrap(); + let mut output = child.stdout.take().unwrap(); + let operation = async { + input + .write_u32(request.len() as u32) + .await + .map_err(|_| "request header failed")?; + input + .write_all(request) + .await + .map_err(|_| "request body failed")?; + drop(input); + let length = output + .read_u32() + .await + .map_err(|_| "response header unavailable")? as usize; + if length > 1024 * 1024 { + return Err("response exceeds metadata profile"); + } + let mut bytes = vec![0; length]; + output + .read_exact(&mut bytes) + .await + .map_err(|_| "response body incomplete")?; + let mut trailing = [0]; + if output + .read(&mut trailing) + .await + .map_err(|_| "completion unavailable")? + != 0 + { + return Err("reader emitted an extra frame"); + } + Ok(bytes) + }; + let result = tokio::time::timeout(Duration::from_secs(10), operation).await; + if !matches!(result, Ok(Ok(_))) { + let _ = child.kill().await; + } + let status = tokio::time::timeout(Duration::from_secs(5), child.wait()) + .await + .map_err(|_| "owned reader join timed out")? + .map_err(|_| "owned reader join failed")?; + if !status.success() { + return Err("reader failed without complete evidence"); + } + result.map_err(|_| "reader timed out")? +} + +#[tokio::test] +async fn session_host_claude_observer_returns_bound_metadata_without_consuming_settings() { + let owned = tempfile::tempdir().unwrap(); + let root = owned.path().join("root"); + let home = owned.path().join("home"); + let cwd = root.join("nested"); + for path in [ + &home, + &root.join(".git"), + &root.join(".claude"), + &cwd.join(".claude"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + let poison = b"PRIVATE_METADATA_ONLY_SETTINGS_SENTINEL: invalid JSON"; + let selected = root.join(".claude/settings.local.json"); + std::fs::write(&selected, poison).unwrap(); + for runtime in ["terminal", "sdk"] { + let request = serde_json::to_vec(&json!({ + "version":1,"requestId":uuid::Uuid::new_v4().to_string(), + "nativeSourceRuntime":runtime,"cwd":cwd,"home":home,"settingSources":["local"], + "selectorManifestSha256":"a".repeat(64),"planRevision":0 + })) + .unwrap(); + let bytes = read_metadata(&cwd, &home, &request) + .await + .expect("the production reader must return complete bound metadata"); + assert!(!bytes.windows(poison.len()).any(|window| window == poison)); + let reply: Value = serde_json::from_slice(&bytes).unwrap(); + assert_eq!( + reply["requestSha256"], + format!("{:x}", Sha256::digest(&request)) + ); + let observed: freshell_platform::claude_inputs::observation::NativeObservation = + serde_json::from_value(reply["observation"].clone()).unwrap(); + observed.validate().unwrap(); + assert_eq!(observed.uid, unsafe { libc::getuid() }); + assert_eq!(observed.euid, unsafe { libc::geteuid() }); + assert_eq!(observed.gid, unsafe { libc::getgid() }); + assert_eq!(observed.egid, unsafe { libc::getegid() }); + assert_eq!( + observed.local_settings, + [cwd.join(".claude/settings.local.json"), selected.clone()] + ); + assert!( + std::fs::read(&selected).unwrap() == poison, + "metadata observation must retain the private settings file" + ); + } +} + +#[tokio::test] +async fn session_host_claude_observer_rejects_a_different_native_cwd_or_home_and_retains_inputs() { + let owned = tempfile::tempdir().unwrap(); + let cwd = owned.path().join("actual-cwd"); + let other_cwd = owned.path().join("other-cwd"); + let home = owned.path().join("actual-home"); + let other_home = owned.path().join("other-home"); + for path in [&cwd, &other_cwd, &home, &other_home] { + std::fs::create_dir(path).unwrap(); + } + let prior = home.join("accepted-prior-inputs"); + std::fs::write(&prior, b"immutable-prior-inputs").unwrap(); + for role in ["cwd", "home"] { + let request = serde_json::to_vec(&json!({ + "version":1,"requestId":uuid::Uuid::new_v4().to_string(),"nativeSourceRuntime":"terminal", + "cwd":if role == "cwd" { &other_cwd } else { &cwd }, + "home":if role == "home" { &other_home } else { &home }, + "settingSources":["local"],"selectorManifestSha256":"b".repeat(64),"planRevision":0 + })).unwrap(); + assert!( + read_metadata(&cwd, &home, &request).await.is_err(), + "a request for another native {role} must not become source-selection evidence" + ); + assert_eq!(std::fs::read(&prior).unwrap(), b"immutable-prior-inputs"); + } +} From 965614ee753831e4645243b3a3fb588de518036b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 21:00:51 -0700 Subject: [PATCH 106/493] Observe Claude source metadata through a scoped private helper --- crates/freshell-runtime-client/src/lib.rs | 16 + crates/freshell-runtime-protocol/src/lib.rs | 77 ++++ .../src/claude_source_observation.rs | 17 + crates/freshell-session-host/src/main.rs | 7 + .../tests/claude_source_observation.rs | 33 +- crates/freshell-supervisor/src/backend.rs | 36 ++ .../src/backend/claude_source_observation.rs | 418 ++++++++++++++++++ .../claude_source_observation_tests.rs | 258 +++++++++++ .../src/claude_source_observation.rs | 371 ++++++++++++++++ .../src/native_auth_lookup_tests.rs | 80 ++++ crates/freshell-supervisor/src/registry.rs | 1 + .../src/registry/input_reservation.rs | 2 +- crates/freshell-supervisor/src/service.rs | 10 + 13 files changed, 1324 insertions(+), 2 deletions(-) create mode 100644 crates/freshell-supervisor/src/backend/claude_source_observation.rs create mode 100644 crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs create mode 100644 crates/freshell-supervisor/src/claude_source_observation.rs diff --git a/crates/freshell-runtime-client/src/lib.rs b/crates/freshell-runtime-client/src/lib.rs index daee99f3e..72cb2fe99 100644 --- a/crates/freshell-runtime-client/src/lib.rs +++ b/crates/freshell-runtime-client/src/lib.rs @@ -117,6 +117,22 @@ impl RuntimeClient { _ => Err(ClientError::UnexpectedResult), } } + pub async fn observe_claude_sources( + &self, + mut request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, + ) -> Result { + request.expected_control_epoch = Some(self.current_epoch().await?); + match self + .request( + RequestId::new(), + AdminCommand::ObserveClaudeSources(request), + ) + .await? + { + AdminResult::ClaudeSourceObservation(reply) => Ok(reply), + _ => Err(ClientError::UnexpectedResult), + } + } pub async fn from_secret_file( socket_path: impl Into, secret_file: impl Into, diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 56cd0a120..d9fc3c7ac 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2799,6 +2799,7 @@ pub enum AdminCommand { Health, ReserveProviderInputs(ReserveProviderInputsRequest), NativeAuthLookup(NativeAuthLookupRequest), + ObserveClaudeSources(ObserveClaudeSourcesRequest), BindProviderPathInputs(BindProviderPathInputsRequest), Launch(Box), Stop(StopRequest), @@ -2880,6 +2881,81 @@ pub struct NativeAuthLookupRequest { pub keys: Vec, } +/// Claude source selection has its own purpose and eligibility. Creation +/// absence authorizes an empty initial HOME, never source or native-store absence. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)] +pub enum ClaudeSourceAuthority { + InitialReservation { + #[serde(rename = "reservationId")] + reservation_id: String, + }, + RecoveryAttempt { + #[serde(rename = "attemptId")] + attempt_id: String, + }, +} + +pub type ClaudeSourceObservation = freshell_platform::claude_inputs::observation::ObservationReply; + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ObserveClaudeSourcesRequest { + pub expected_control_epoch: Option, + pub soul_id: SoulId, + pub provider: String, + pub authority: ClaudeSourceAuthority, + pub workspace_path: String, + pub git_common_dir: Option, + pub candidate: Option, + pub observation_request: freshell_platform::claude_inputs::observation::ObservationRequest, +} + +impl ObserveClaudeSourcesRequest { + pub fn validate(&self) -> Result<(), RuntimeError> { + let invalid = || { + RuntimeError::new(RuntimeErrorCode::InvalidRequest, + "Claude source request has invalid selection or authority; retain inputs and reprepare") + }; + if !matches!(self.provider.as_str(), "claude" | "kilroy") + || self.observation_request.home != std::path::Path::new("/home/freshell/provider") + { + return Err(invalid()); + } + let id = match &self.authority { + ClaudeSourceAuthority::InitialReservation { reservation_id } => reservation_id, + ClaudeSourceAuthority::RecoveryAttempt { attempt_id } => attempt_id, + }; + if id.len() != 36 || uuid::Uuid::parse_str(id).is_err() { + return Err(invalid()); + } + self.observation_request.validate().map_err(|_| invalid())?; + for path in std::iter::once(&self.workspace_path).chain(self.git_common_dir.iter()) { + if path.len() > 64 * 1024 + || path.chars().any(char::is_control) + || !std::path::Path::new(path).is_absolute() + { + return Err(invalid()); + } + } + if !self + .observation_request + .cwd + .starts_with(&self.workspace_path) + { + return Err(invalid()); + } + if let Some(candidate) = &self.candidate { + candidate.validate("claude")?; + if candidate.provider_launch_context.prepared_inputs.is_none() { + return Err(invalid()); + } + } + bounded_json_bytes(self, MAX_CONTROL_FRAME_BYTES).map_err(|_| invalid())?; + Ok(()) + } +} + #[derive(Clone, Serialize, Deserialize)] #[serde(transparent)] pub struct NativeAuthValues(pub std::collections::BTreeMap); @@ -3007,6 +3083,7 @@ pub struct LaunchResult { pub enum AdminResult { ProviderInputReservation(ProviderInputReservation), NativeAuthLookup(NativeAuthValues), + ClaudeSourceObservation(ClaudeSourceObservation), ProviderPathInputsBound, Health { control_epoch: u64, diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index 9b12a0c60..585f21f78 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -10,6 +10,23 @@ pub(crate) fn serve_stdio() -> Result<(), String> { exchange(&mut std::io::stdin().lock(), &mut std::io::stdout().lock()) } +pub(crate) fn serve_private() -> Result<(), String> { + let mut input = + std::os::unix::net::UnixStream::connect("/run/freshell-source-exchange/reply.sock") + .map_err(|_| "Claude observation private exchange unavailable")?; + let timeout = Some(std::time::Duration::from_secs(15)); + input + .set_read_timeout(timeout) + .map_err(|_| "Claude observation private timeout unavailable")?; + input + .set_write_timeout(timeout) + .map_err(|_| "Claude observation private timeout unavailable")?; + let mut output = input + .try_clone() + .map_err(|_| "Claude observation private exchange unavailable")?; + exchange(&mut input, &mut output) +} + fn exchange(input: &mut impl Read, output: &mut impl Write) -> Result<(), String> { let mut length = [0; 4]; input diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 8588e408b..165d1a8ce 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -119,6 +119,13 @@ async fn run() -> Result<(), String> { Some("serve") => serve(&args[2..]).await, Some("worker") => worker(&args[2..]).await, Some("observe-claude-sources") if args.len() == 2 => claude_source_observation::serve_stdio(), + Some("observe-claude-sources-private") if args.len() == 2 => { + verify_private_helper_namespace( + Path::new("/home/freshell/provider"), + Path::new("/run/freshell-source-exchange/reply.sock"), + )?; + claude_source_observation::serve_private() + } Some("native-auth-only") if args.len() == 2 => { verify_private_helper_namespace( Path::new("/home/freshell/provider"), diff --git a/crates/freshell-session-host/tests/claude_source_observation.rs b/crates/freshell-session-host/tests/claude_source_observation.rs index d9091266c..46776c366 100644 --- a/crates/freshell-session-host/tests/claude_source_observation.rs +++ b/crates/freshell-session-host/tests/claude_source_observation.rs @@ -6,9 +6,18 @@ use std::{os::unix::process::CommandExt, path::Path, process::Stdio, time::Durat use tokio::io::{AsyncReadExt, AsyncWriteExt}; async fn read_metadata(cwd: &Path, home: &Path, request: &[u8]) -> Result, &'static str> { + read_operation(cwd, home, request, "observe-claude-sources").await +} + +async fn read_operation( + cwd: &Path, + home: &Path, + request: &[u8], + operation: &str, +) -> Result, &'static str> { let mut command = tokio::process::Command::new(env!("CARGO_BIN_EXE_freshell-session-host")); command - .arg("observe-claude-sources") + .arg(operation) .current_dir(cwd) .env_clear() .env("HOME", home) @@ -70,6 +79,28 @@ async fn read_metadata(cwd: &Path, home: &Path, request: &[u8]) -> Result Result { + Err(BackendError::Unavailable( + "Claude native source reader unavailable; retain inputs and reprepare".into(), + )) + } async fn observe_initial_auth_volume( &self, _reservation: &crate::registry::OwnedInputReservation, @@ -572,6 +594,20 @@ fn runtime_host_environment( #[async_trait] impl RuntimeBackend for DockerEngineBackend { + async fn observe_claude_sources( + &self, + owner: &ClaudeSourceOwner, + request: &freshell_runtime_protocol::ObserveClaudeSourcesRequest, + test_run_id: &str, + ) -> Result { + self.read_claude_source_helper( + owner, + request, + test_run_id, + std::time::Duration::from_secs(25), + ) + .await + } async fn observe_initial_auth_volume( &self, reservation: &crate::registry::OwnedInputReservation, diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs new file mode 100644 index 000000000..c0a1ad274 --- /dev/null +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -0,0 +1,418 @@ +//! Fixed Claude reader in the owned provider namespace; metadata uses private +//! IPC. Helper cleanup never stops a provider workload or creates its HOME. +use super::*; +use freshell_platform::claude_inputs::observation::{ + ObservationReply, MAX_REPLY_BYTES, MAX_REQUEST_BYTES, +}; +use freshell_runtime_protocol::ObserveClaudeSourcesRequest; +use std::time::Duration; +use tokio::net::UnixListener; + +fn fail(reason: &'static str) -> BackendError { + BackendError::Unavailable(reason.into()) +} + +fn owned_helper(actual: &Value, expected: &Value) -> bool { + input_publisher::owned_helper(actual, expected) + && actual["Config"]["WorkingDir"] == expected["WorkingDir"] + && [ + "NetworkMode", + "ReadonlyRootfs", + "CapDrop", + "SecurityOpt", + "Memory", + "MemorySwap", + "NanoCpus", + "PidsLimit", + "Tmpfs", + ] + .iter() + .all(|key| actual["HostConfig"][key] == expected["HostConfig"][key]) + && actual["HostConfig"]["CapAdd"] + .as_array() + .is_none_or(|caps| caps.is_empty()) +} + +async fn exchange(listener: &UnixListener, raw: &[u8]) -> Result { + let (mut stream, _) = listener + .accept() + .await + .map_err(|_| fail("Claude source exchange unavailable"))?; + stream + .write_u32(raw.len() as u32) + .await + .map_err(|_| fail("Claude source request unavailable"))?; + stream + .write_all(raw) + .await + .map_err(|_| fail("Claude source request incomplete"))?; + let length = stream + .read_u32() + .await + .map_err(|_| fail("Claude source reply unavailable"))? as usize; + if length > MAX_REPLY_BYTES { + return Err(fail("Claude source metadata reply exceeds profile")); + } + let mut bytes = Vec::new(); + bytes + .try_reserve_exact(length) + .map_err(|_| fail("Claude source metadata allocation unavailable"))?; + bytes.resize(length, 0); + stream + .read_exact(&mut bytes) + .await + .map_err(|_| fail("Claude source metadata reply incomplete"))?; + let mut trailing = [0]; + if stream + .read(&mut trailing) + .await + .map_err(|_| fail("Claude source reply completion unavailable"))? + != 0 + { + return Err(fail("Claude source helper emitted extra metadata")); + } + serde_json::from_slice(&bytes).map_err(|_| fail("Claude source metadata reply invalid")) +} + +impl DockerEngineBackend { + pub(super) async fn read_claude_source_helper( + &self, + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + test_run_id: &str, + budget: Duration, + ) -> Result { + request + .validate() + .map_err(|_| fail("Claude source request invalid"))?; + if request.candidate.is_some() { + return Err(fail("Claude candidate source topology is not yet qualified; retain inputs and reprepare")); + } + let name = format!("freshell-claude-source-{}", uuid::Uuid::new_v4()); + let mut attempted = false; + let mut expected = None::; + let mut directory = None::; + let operation = async { + let (daemon, rootless) = self + .daemon_info() + .await + .map_err(|_| fail("Claude source daemon unavailable"))?; + if !rootless { + return Err(fail( + "Claude source reader requires the qualified rootless daemon", + )); + } + let (installation, soul, image, binary, retained) = match owner { + ClaudeSourceOwner::Initial { + reservation, + observation, + } => { + if reservation.provider != request.provider + || self.observe_initial_volume(reservation).await? != *observation + { + return Err(fail("Claude source creation absence changed")); + } + ( + &reservation.installation_id, + &reservation.soul_id, + reservation.image_ref.as_str(), + reservation.binary.as_path(), + None, + ) + } + ClaudeSourceOwner::Retained { handle } => { + let provider = handle + .terminal() + .map(|spec| spec.mode.as_str()) + .or_else(|| handle.fresh_agent().map(|spec| spec.provider.as_str())); + let workspace = handle + .terminal() + .map(|spec| spec.workspace_path.as_str()) + .or_else(|| { + handle + .fresh_agent() + .map(|spec| spec.workspace_path.as_str()) + }); + if provider != Some(request.provider.as_str()) + || workspace != Some(request.workspace_path.as_str()) + || handle.daemon_id() != &daemon + || !matches!( + self.inspect(handle).await?.state, + BackendRuntimeState::Created + | BackendRuntimeState::Exited + | BackendRuntimeState::Dead + | BackendRuntimeState::Missing + ) + { + return Err(fail( + "Claude source requires its exact quiescent retained owner", + )); + } + let response = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", handle.provider_volume_name()), + None, + 64 * 1024, + ) + .await?; + let volume: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("Claude retained HOME metadata invalid"))?; + if response.status != 200 + || volume["Name"] != handle.provider_volume_name() + || volume["Driver"] != "local" + { + return Err(fail("Claude retained HOME is unavailable; preserve native stores and repair the source")); + } + ( + handle.installation_id(), + handle.soul_id(), + handle.image_ref(), + handle.host_binary_path(), + Some(handle.provider_volume_name()), + ) + } + }; + if soul != &request.soul_id { + return Err(fail("Claude source soul ownership changed")); + } + let binary = canonical_regular_file(binary) + .map_err(|_| fail("Claude source reader artifact unavailable"))?; + let digest = crate::registry::binary_digest(&binary) + .map_err(|_| fail("Claude source reader artifact changed"))?; + if let ClaudeSourceOwner::Initial { reservation, .. } = owner { + if digest != reservation.binary_sha256 { + return Err(fail("Claude reserved reader artifact changed")); + } + } + let workspace = canonical_dir(Path::new(&request.workspace_path)) + .map_err(|_| fail("Claude source workspace unavailable"))?; + let cwd = canonical_dir(&request.observation_request.cwd) + .map_err(|_| fail("Claude source native cwd unavailable"))?; + if workspace != Path::new(&request.workspace_path) + || cwd != request.observation_request.cwd + || !cwd.starts_with(&workspace) + { + return Err(fail("Claude source workload topology changed")); + } + let common = request + .git_common_dir + .as_deref() + .map(|path| { + let canonical = canonical_dir(Path::new(path)) + .map_err(|_| fail("Claude source Git common directory unavailable"))?; + if canonical != Path::new(path) { + return Err(fail("Claude source Git common topology changed")); + } + Ok(canonical) + }) + .transpose()?; + directory = Some(native_auth::ExchangeDirectory::create_input()?); + let directory = directory.as_ref().unwrap(); + let socket = directory.0.join("reply.sock"); + let listener = UnixListener::bind(&socket) + .map_err(|_| fail("Claude source private exchange bind failed"))?; + directory.mode(&socket, 0o660)?; + let mut mounts = vec![ + json!({"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}), + json!({"Type":"bind","Source":workspace,"Target":workspace,"ReadOnly":true}), + json!({"Type":"bind","Source":directory.0,"Target":"/run/freshell-source-exchange","ReadOnly":true}), + ]; + if let Some(common) = common.filter(|common| !common.starts_with(&workspace)) { + mounts.push(json!({"Type":"bind","Source":common,"Target":common,"ReadOnly":true})); + } + let tmpfs = if let Some(volume) = retained { + mounts.push(json!({"Type":"volume","Source":volume,"Target":"/home/freshell/provider","ReadOnly":true,"VolumeOptions":{"NoCopy":true}})); + json!({}) + } else { + json!({"/home/freshell/provider":"rw,nosuid,nodev,noexec,size=16m,uid=65534,gid=0,mode=0700"}) + }; + let body = json!({ + "Image":image,"User":"65534:0","Tty":false,"WorkingDir":cwd, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["observe-claude-sources-private"], + "Labels":{"com.freshell.installation-id":installation,"com.freshell.soul-id":soul, + "com.freshell.runtime-test-run-id":test_run_id,"com.freshell.source-request-id":request.observation_request.request_id, + "com.freshell.source-binary-sha256":digest}, + "Env":self.private_helper_environment(image,&["HOME=/home/freshell/provider"]).await?, + "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"SecurityOpt":["no-new-privileges"], + "Memory":256*1024*1024,"MemorySwap":256*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, + "LogConfig":{"Type":"none","Config":{}},"Tmpfs":tmpfs,"Mounts":mounts} + }); + let raw = freshell_runtime_protocol::bounded_json_bytes( + &request.observation_request, + MAX_REQUEST_BYTES, + ) + .map_err(|_| fail("Claude source request exceeds metadata profile"))?; + expected = Some(body.clone()); + attempted = true; + let created = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/create?name={name}"), + Some(&body), + 64 * 1024, + ) + .await + .map_err(|_| fail("Claude source helper creation acknowledgement unavailable"))?; + if created.status == 409 { + attempted = false; + } + if created.status != 201 { + return Err(fail("Claude source helper creation failed")); + } + let created: Value = serde_json::from_slice(&created.body) + .map_err(|_| fail("Claude source helper creation invalid"))?; + let id = created["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|b| b.is_ascii_hexdigit())) + .ok_or_else(|| fail("Claude source helper identity unavailable"))?; + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{id}/json"), + None, + 1024 * 1024, + ) + .await?; + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("Claude source helper ownership invalid"))?; + if inspected.status != 200 + || actual["Id"] != id + || !owned_helper(&actual, &body) + || actual["State"]["Running"] != false + { + return Err(fail( + "Claude source helper namespace/topology ownership changed", + )); + } + let supervision = async { + let started = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/start"), + None, + 64 * 1024, + ) + .await?; + if started.status != 204 { + return Err(fail("Claude source helper start failed")); + } + let waited = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/wait?condition=not-running"), + None, + 64 * 1024, + ) + .await?; + let value: Value = serde_json::from_slice(&waited.body) + .map_err(|_| fail("Claude source helper completion invalid"))?; + if waited.status != 200 || value["StatusCode"] != 0 { + return Err(fail( + "Claude source helper did not return complete evidence", + )); + } + Ok::<_, BackendError>(()) + }; + let (reply, ()) = tokio::try_join!(exchange(&listener, &raw), supervision)?; + reply + .validate_for_request( + &request.observation_request, + &format!("{:x}", Sha256::digest(&raw)), + ) + .map_err(|_| fail("Claude source reply selection binding changed"))?; + if ( + reply.observation.uid, + reply.observation.euid, + reply.observation.gid, + reply.observation.egid, + ) != (65534, 65534, 0, 0) + { + return Err(fail("Claude source native reader identity changed")); + } + if self.daemon_id().await? != daemon + || crate::registry::binary_digest(&binary) + .map_err(|_| fail("Claude source reader artifact unavailable"))? + != digest + { + return Err(fail("Claude source daemon or reader artifact changed")); + } + if let ClaudeSourceOwner::Initial { + reservation, + observation, + } = owner + { + if self.observe_initial_volume(reservation).await? != *observation { + return Err(fail("Claude source initial HOME presence changed")); + } + } + Ok(reply) + }; + let result = tokio::time::timeout(budget, operation) + .await + .unwrap_or_else(|_| { + Err(fail( + "Claude source helper timed out; retain inputs and reprepare", + )) + }); + if attempted { + let cleanup = async { + let inspection = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 1024 * 1024, + ) + .await?; + if inspection.status == 404 { + return Ok(()); + } + let actual: Value = serde_json::from_slice(&inspection.body) + .map_err(|_| fail("Claude source cleanup ownership invalid"))?; + if inspection.status != 200 + || !expected + .as_ref() + .is_some_and(|expected| owned_helper(&actual, expected)) + { + return Err(fail( + "Claude source cleanup ownership changed; no provider workload was stopped", + )); + } + let removed = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/containers/{name}?force=1"), + None, + 64 * 1024, + ) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail("Claude source helper cleanup failed")); + } + let absent = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 64 * 1024, + ) + .await?; + if absent.status != 404 { + return Err(fail("Claude source helper cleanup not confirmed")); + } + Ok(()) + }; + tokio::time::timeout(Duration::from_secs(5), cleanup) + .await + .map_err(|_| fail("Claude source helper cleanup timed out"))??; + } + tracing::debug!(event="provider.inputs.native_source_helper.settled",soul_id=%request.soul_id,helper=%name,succeeded=result.is_ok(), + "Claude source helper settled after owned cleanup"); + result + } +} + +#[cfg(test)] +#[path = "claude_source_observation_tests.rs"] +mod tests; diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs new file mode 100644 index 000000000..b75ce9fcf --- /dev/null +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -0,0 +1,258 @@ +use super::*; +use freshell_platform::claude_inputs::observation::{LocalDecision, NativeObservation}; +use freshell_runtime_protocol::{ + ClaudeSourceAuthority, InitialAuthVolumeObservation, ReserveProviderInputsRequest, +}; +use std::sync::{Arc, Mutex}; +use tokio::{net::UnixListener, task::JoinSet}; + +#[tokio::test] +async fn provider_claude_source_helper_uses_private_bound_ipc_without_creating_stable_home() { + for mode in [ + "success", + "wrong_uid", + "wrong_digest", + "oversized_reply", + "failed_exit", + "lost_create_ack", + "cleanup_mismatch", + "timeout", + "nonrootless", + "existing_initial_home", + ] { + qualify_transport(mode).await; + } +} + +async fn qualify_transport(mode: &'static str) { + let temp = tempfile::tempdir().unwrap(); + let workspace = temp.path().join("workspace"); + std::fs::create_dir(&workspace).unwrap(); + let private = "PRIVATE_CLAUDE_SOURCE_SETTINGS_SENTINEL"; + std::fs::create_dir(workspace.join(".claude")).unwrap(); + std::fs::write(workspace.join(".claude/settings.local.json"), private).unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, b"synthetic helper artifact").unwrap(); + let registry = crate::registry::Registry::open(temp.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let image = format!("sha256:{}", "a".repeat(64)); + let reservation = registry + .reserve_initial_inputs( + ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "claude".into(), + creation_seed_ref: "synthetic-seed".into(), + }, + image, + binary, + ) + .await + .unwrap(); + let daemon = DockerDaemonId::new(); + let owner = ClaudeSourceOwner::Initial { + reservation: reservation.clone(), + observation: InitialAuthVolumeObservation::Absent { + daemon_id: daemon.clone(), + }, + }; + let request: ObserveClaudeSourcesRequest = serde_json::from_value(json!({ + "expectedControlEpoch":registry.control_epoch(), "soulId":soul, "provider":"claude", + "authority":{"kind":"initial_reservation","reservationId":reservation.id}, + "workspacePath":workspace,"gitCommonDir":null,"candidate":null, + "observationRequest":{"version":1,"requestId":uuid::Uuid::new_v4().to_string(), + "nativeSourceRuntime":"terminal","cwd":workspace,"home":"/home/freshell/provider", + "settingSources":["local"],"selectorManifestSha256":"b".repeat(64),"planRevision":0} + })) + .unwrap(); + assert!(matches!( + request.authority, + ClaudeSourceAuthority::InitialReservation { .. } + )); + let requests = Arc::new(Mutex::new(Vec::<(String, Vec)>::new())); + let helper_bodies = Arc::new(Mutex::new(Vec::::new())); + let listener = UnixListener::bind(temp.path().join("docker.sock")).unwrap(); + let seen = requests.clone(); + let bodies = helper_bodies.clone(); + let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); + let server = tokio::spawn(async move { + let mut jobs = JoinSet::new(); + loop { + tokio::select! { + _ = &mut stopped => break, + accepted = listener.accept() => { + let (mut stream, _) = accepted.unwrap(); + let (seen, bodies, daemon) = (seen.clone(), bodies.clone(), daemon.clone()); + jobs.spawn(async move { + let mut header = Vec::new(); + let mut byte = [0]; + while !header.ends_with(b"\r\n\r\n") { + stream.read_exact(&mut byte).await.unwrap(); + header.push(byte[0]); + assert!(header.len() <= 32 * 1024); + } + let header = String::from_utf8(header).unwrap(); + let length = header.lines().find_map(|line|line.strip_prefix("Content-Length: ")) + .unwrap().trim().parse::().unwrap(); + assert!(length <= freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES); + let mut raw = vec![0; length]; + stream.read_exact(&mut raw).await.unwrap(); + let line = header.lines().next().unwrap().to_string(); + seen.lock().unwrap().push((line.clone(), raw.clone())); + let (status, response) = if line.contains("/info ") { + (200, json!({"ID":daemon,"SecurityOptions":if mode=="nonrootless" {vec![]} else {vec!["name=rootless"]}})) + } else if line.starts_with("GET") && line.contains("/volumes/") { + (if mode=="existing_initial_home" {200} else {404}, json!({})) + } else if line.starts_with("GET") && line.contains("/images/") { + (200, json!({"Config":{"Env":["ANTHROPIC_API_KEY=PRIVATE_IMAGE_CREDENTIAL_SENTINEL","PATH=/usr/bin"]}})) + } else if line.starts_with("POST") && line.contains("/containers/create?") { + bodies.lock().unwrap().push(serde_json::from_slice(&raw).unwrap()); + (if mode=="lost_create_ack" {503} else {201}, json!({"Id":"b".repeat(64)})) + } else if line.starts_with("GET") && line.contains("/containers/") && line.contains("/json ") { + match bodies.lock().unwrap().last().cloned() { + Some(mut body) => { + if mode=="cleanup_mismatch" && line.contains("/freshell-claude-source-") {body["User"]=json!("foreign-identity");} + (200, json!({"Id":"b".repeat(64),"Config":body,"HostConfig":body["HostConfig"],"State":{"Running":false}})) + }, + None => (404, json!({})), + } + } else if line.starts_with("POST") && line.contains("/start ") { + let body = bodies.lock().unwrap().last().unwrap().clone(); + let exchange = body["HostConfig"]["Mounts"].as_array().unwrap().iter() + .find(|mount|mount["Target"]=="/run/freshell-source-exchange").unwrap()["Source"].as_str().unwrap().to_string(); + let mut channel = UnixStream::connect(Path::new(&exchange).join("reply.sock")).await.unwrap(); + if mode=="timeout" { + tokio::time::sleep(Duration::from_secs(10)).await; + return; + } + let length = channel.read_u32().await.unwrap() as usize; + assert!(length <= freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES); + let mut raw = vec![0; length]; + channel.read_exact(&mut raw).await.unwrap(); + let request: freshell_platform::claude_inputs::observation::ObservationRequest = serde_json::from_slice(&raw).unwrap(); + let mut reply = ObservationReply { + request_sha256: format!("{:x}", Sha256::digest(&raw)), + observation: NativeObservation { + version:1,native_source_runtime:request.native_source_runtime, + setting_sources:request.setting_sources,uid:65534,euid:65534,gid:0,egid:0, + local_settings:vec![request.cwd.join(".claude/settings.local.json")], + cwd:request.cwd,home:Some(request.home),roots:None, + local_decision:LocalDecision::NoRoot,root_uid:None,git_uid:None,claude_uid:None, + }, + }; + if mode=="wrong_uid" {reply.observation.uid=65535;reply.observation.euid=65535;} + if mode=="wrong_digest" {reply.request_sha256="c".repeat(64);} + if mode=="oversized_reply" { + channel.write_u32(freshell_platform::claude_inputs::observation::MAX_REPLY_BYTES as u32+1).await.unwrap(); + channel.shutdown().await.unwrap(); + (204, json!({})) + } else { + let raw = serde_json::to_vec(&reply).unwrap(); + channel.write_u32(raw.len() as u32).await.unwrap(); + channel.write_all(&raw).await.unwrap(); + channel.shutdown().await.unwrap(); + (204, json!({})) + } + } else if line.starts_with("POST") && line.contains("/wait?") { + (200, json!({"StatusCode":if mode=="failed_exit" {1} else {0}})) + } else if line.starts_with("DELETE") && line.contains("/containers/") { + // Readback must prove the owned helper is gone. + bodies.lock().unwrap().clear(); + (204, json!({})) + } else { (500, json!({})) }; + let response = response.to_string(); + stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{response}",response.len()).as_bytes()).await.unwrap(); + }); + } + result = jobs.join_next(), if !jobs.is_empty() => { result.unwrap().unwrap(); } + } + } + jobs.shutdown().await; + }); + let backend = DockerEngineBackend::new(temp.path().join("docker.sock")); + let result = backend + .read_claude_source_helper( + &owner, + &request, + "synthetic-source-run", + Duration::from_millis(250), + ) + .await; + let _ = stop.send(()); + tokio::time::timeout(Duration::from_secs(5), server) + .await + .unwrap() + .unwrap(); + assert_eq!( + result.is_ok(), + mode == "success", + "{mode}: unavailable, contradictory or unjoined evidence cannot be accepted" + ); + if let Ok(reply) = result { + let raw = freshell_runtime_protocol::bounded_json_bytes( + &request.observation_request, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .unwrap(); + reply + .validate_for_request( + &request.observation_request, + &format!("{:x}", Sha256::digest(&raw)), + ) + .unwrap(); + } + let seen = requests.lock().unwrap(); + assert!(seen + .iter() + .all(|(line, raw)| !line.contains("/volumes/create") + && !line.contains("/logs?") + && !raw + .windows(private.len()) + .any(|bytes| bytes == private.as_bytes()) + && !raw + .windows(b"PRIVATE_IMAGE_CREDENTIAL_SENTINEL".len()) + .any(|bytes| bytes == b"PRIVATE_IMAGE_CREDENTIAL_SENTINEL"))); + let create = seen + .iter() + .find(|(line, _)| line.contains("/containers/create?")); + if matches!(mode, "nonrootless" | "existing_initial_home") { + assert!( + create.is_none(), + "unqualified creation namespace must not create a helper or native workload" + ); + } else { + let created: Value = serde_json::from_slice(&create.unwrap().1).unwrap(); + assert_eq!(created["User"], "65534:0"); + assert_eq!(created["Cmd"], json!(["observe-claude-sources-private"])); + assert_eq!( + created["WorkingDir"], + json!(request.observation_request.cwd) + ); + assert_eq!(created["HostConfig"]["NetworkMode"], "none"); + assert_eq!(created["HostConfig"]["LogConfig"]["Type"], "none"); + assert_eq!(created["HostConfig"]["CapDrop"], json!(["ALL"])); + assert_eq!(created["HostConfig"]["ReadonlyRootfs"], true); + assert_eq!(created["HostConfig"]["PidsLimit"], 32); + assert_eq!(created["HostConfig"]["Memory"], 256 * 1024 * 1024); + assert_eq!(created["HostConfig"]["NanoCpus"], 500_000_000); + assert!(created["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .all(|mount| mount["Type"] == "bind" && mount["ReadOnly"] == true)); + } + if mode == "cleanup_mismatch" { + assert!( + !seen.iter().any(|(line, _)| line.starts_with("DELETE")), + "changed ownership cannot authorize deleting a helper" + ); + } else { + assert!( + helper_bodies.lock().unwrap().is_empty(), + "owned helper cleanup must complete before returning metadata/error" + ); + } + assert!( + std::fs::read(workspace.join(".claude/settings.local.json")).unwrap() == private.as_bytes() + ); +} diff --git a/crates/freshell-supervisor/src/claude_source_observation.rs b/crates/freshell-supervisor/src/claude_source_observation.rs new file mode 100644 index 000000000..dee072874 --- /dev/null +++ b/crates/freshell-supervisor/src/claude_source_observation.rs @@ -0,0 +1,371 @@ +//! Claude-purpose source authority. No lifecycle lock is reacquired by the +//! nested recovery callback, and no registry/map lock crosses helper IPC. +use super::*; +use crate::backend::ClaudeSourceOwner; +use freshell_runtime_protocol::{ClaudeSourceAuthority, ObserveClaudeSourcesRequest}; + +impl Supervisor { + pub(super) async fn observe_claude_sources( + &self, + request: ObserveClaudeSourcesRequest, + ) -> Result { + request.validate()?; + let denied = || { + RuntimeError::new(RuntimeErrorCode::InvalidRequest, + "Claude source observation requires its current creation or recovery attempt; retain inputs and reprepare") + }; + let unavailable = || { + RuntimeError::new(RuntimeErrorCode::HostUnreachable, + "Claude native source evidence is unavailable; retain inputs and repair the source before dispatch") + }; + let mut recovery = None; + let owner = match &request.authority { + ClaudeSourceAuthority::InitialReservation { reservation_id } => { + let (reservation, expected) = self + .registry + .initial_input_reservation(request.soul_id.clone(), reservation_id.clone()) + .await + .map_err(map_registry)? + .ok_or_else(denied)?; + if reservation.provider != request.provider + || reservation.image_ref != self.config.image_ref + || reservation.binary != self.config.host_binary_path + { + return Err(denied()); + } + let observation = tokio::time::timeout( + Duration::from_secs(10), + self.backend.observe_initial_auth_volume(&reservation), + ) + .await + .map_err(|_| unavailable())? + .map_err(|_| unavailable())?; + if observation != expected { + return Err(denied()); + } + ClaudeSourceOwner::Initial { + reservation, + observation, + } + } + ClaudeSourceAuthority::RecoveryAttempt { attempt_id } => { + // This scope already belongs to the callback owner. Eligibility + // here is Claude/Kilroy, independently of OpenCode auth lookup. + let authority = { + let authorities = self + .auth_lookup_authorities + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + authorities + .get(attempt_id) + .filter(|authority| authority.expires > Instant::now()) + .cloned() + .ok_or_else(denied)? + }; + if authority.start.context.soul_id != request.soul_id + || authority.start.context.provider != request.provider + || authority.start.context.prior_handle.image_ref() != self.config.image_ref + || authority.start.context.prior_handle.host_binary_path() + != self.config.host_binary_path + || !self + .registry + .validate_auth_lookup_attempt(authority.start.clone()) + .await + .map_err(map_registry)? + { + return Err(denied()); + } + let handle = authority.start.context.prior_handle.clone(); + recovery = Some(authority.start); + ClaudeSourceOwner::Retained { handle } + } + }; + let raw = freshell_runtime_protocol::bounded_json_bytes( + &request.observation_request, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .map_err(|_| denied())?; + let digest = format!("{:x}", Sha256::digest(&raw)); + // The backend owns a 25-second operation and five-second finite cleanup. + // This outer deadline permits that cleanup to finish before returning. + let reply = tokio::time::timeout( + Duration::from_secs(35), + self.backend + .observe_claude_sources(&owner, &request, &self.config.test_run_id), + ) + .await + .map_err(|_| unavailable())? + .map_err(|_| unavailable())?; + reply + .validate_for_request(&request.observation_request, &digest) + .map_err(|_| unavailable())?; + let observed = &reply.observation; + if (observed.uid, observed.euid, observed.gid, observed.egid) != (65534, 65534, 0, 0) { + return Err(unavailable()); + } + match &request.authority { + ClaudeSourceAuthority::InitialReservation { reservation_id } => { + let current = self + .registry + .initial_input_reservation(request.soul_id.clone(), reservation_id.clone()) + .await + .map_err(map_registry)? + .ok_or_else(denied)?; + let ClaudeSourceOwner::Initial { + reservation, + observation, + } = &owner + else { + return Err(denied()); + }; + if current.0.binary_sha256 != reservation.binary_sha256 || ¤t.1 != observation + { + return Err(denied()); + } + } + ClaudeSourceAuthority::RecoveryAttempt { attempt_id } => { + let start = recovery.ok_or_else(denied)?; + let live_scope = { + let authorities = self + .auth_lookup_authorities + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + authorities.get(attempt_id).is_some_and(|authority| { + authority.expires > Instant::now() + && authority.start.attempt_id == start.attempt_id + && authority.start.context.soul_id == request.soul_id + }) + }; + if !live_scope + || !self + .registry + .validate_auth_lookup_attempt(start) + .await + .map_err(map_registry)? + { + return Err(denied()); + } + } + } + tracing::debug!(event="provider.inputs.native_source_observation.settled",provider=%request.provider, + soul_id=%request.soul_id,plan_revision=request.observation_request.plan_revision, + local_sources=observed.local_settings.len(),uid=observed.uid,gid=observed.gid, + "Qualified Claude native source observation settled"); + Ok(reply) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::backend::*; + use crate::registry::*; + use async_trait::async_trait; + use freshell_platform::claude_inputs::observation::{ + LocalDecision, NativeObservation, ObservationReply, + }; + use freshell_runtime_protocol::*; + + struct ReplyBackend(StdMutex>>); + #[async_trait] + impl RuntimeBackend for ReplyBackend { + async fn observe_claude_sources( + &self, + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + _: &str, + ) -> Result { + let ClaudeSourceOwner::Retained { handle } = owner else { + panic!("requires retained fixture owner"); + }; + assert_eq!(handle.soul_id(), &request.soul_id); + let action = self.0.lock().unwrap().take(); + if let Some(action) = action { + action(); + } + // This is a synthetic transport reply, never native UID evidence. + let selected = &request.observation_request; + let raw = bounded_json_bytes( + selected, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .unwrap(); + Ok(ObservationReply { + request_sha256: format!("{:x}", Sha256::digest(raw)), + observation: NativeObservation { + version: 1, + native_source_runtime: selected.native_source_runtime, + setting_sources: selected.setting_sources.clone(), + uid: 65534, + euid: 65534, + gid: 0, + egid: 0, + cwd: selected.cwd.clone(), + home: Some(selected.home.clone()), + roots: None, + local_decision: LocalDecision::NoRoot, + root_uid: None, + git_uid: None, + claude_uid: None, + local_settings: vec![selected.cwd.join(".claude/settings.local.json")], + }, + }) + } + async fn create_stopped( + &self, + _: &CreateRuntimeSpec, + ) -> Result { + unreachable!() + } + async fn start_host(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + async fn inspect(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + unreachable!() + } + async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + async fn verify_empty(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn list_known( + &self, + _: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + unreachable!() + } + async fn read_limits(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + } + + #[tokio::test] + async fn provider_claude_nested_source_reply_requires_live_scope_after_private_ipc() { + for mode in ["live", "dropped", "expired"] { + let owned = tempfile::tempdir().unwrap(); + let binary = owned.path().join("host"); + std::fs::write(&binary, b"synthetic helper artifact").unwrap(); + let prior = owned.path().join("accepted-prior-inputs"); + std::fs::write(&prior, b"immutable accepted prior generation").unwrap(); + let registry = Registry::open(owned.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let limits = RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + }; + let prepared = registry + .prepare_launch(LaunchPreparation { + soul_id: soul.clone(), + provider: "claude".into(), + provider_store_id: "synthetic-store".into(), + native_session_id: None, + creation_seed_ref: "synthetic-seed".into(), + request_id: RequestId::new(), + payload_digest: "synthetic-digest".into(), + requested_limits: limits, + profile: RuntimeProfile::Custom, + project_key: "synthetic-workspace".into(), + fixture: None, + terminal: None, + fresh_agent: None, + view_intent: None, + admission: crate::admission::AdmissionPolicy::default(), + }) + .await + .unwrap(); + let image = format!("sha256:{}", "a".repeat(64)); + registry + .commit_created( + prepared.incarnation_id, + BackendCreatedRecord { + daemon_id: DockerDaemonId::new(), + container_id: "owned-source-provider-never-stopped".into(), + image_ref: image.clone(), + runtime_dir: owned.path().into(), + host_binary_path: binary.clone(), + immutable_config_digest: "synthetic-digest".into(), + }, + ) + .await + .unwrap(); + let start = registry + .begin_recovery( + soul.clone(), + RecoveryTrigger::ManualRetry, + RecoveryPath::NativeResume, + ) + .await + .unwrap(); + let backend = Arc::new(ReplyBackend(StdMutex::new(None))); + let supervisor = Supervisor::new( + registry, + backend.clone(), + SupervisorConfig { + runtime_root: owned.path().join("runtime"), + control_socket_path: owned.path().join("control.sock"), + host_binary_path: binary, + image_ref: image, + test_run_id: "claude-source-scope".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: owned.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let lock = supervisor.lifecycle_lock(&soul).await; + let _held = lock.lock().await; + let scope = supervisor.auth_lookup_scope(&start); + if mode != "live" { + let authorities = supervisor.auth_lookup_authorities.clone(); + let key = scope.key.clone(); + *backend.0.lock().unwrap() = Some(Box::new(move || { + let mut map = authorities.lock().unwrap(); + if mode == "dropped" { + map.remove(&key); + } else { + map.get_mut(&key).unwrap().expires = Instant::now(); + } + })); + } + let request: ObserveClaudeSourcesRequest=serde_json::from_value(serde_json::json!({ + "expectedControlEpoch":supervisor.registry.control_epoch(),"soulId":soul,"provider":"claude", + "authority":{"kind":"recovery_attempt","attemptId":scope.key}, + "workspacePath":owned.path(),"gitCommonDir":null,"candidate":null, + "observationRequest":{"version":1,"requestId":uuid::Uuid::new_v4().to_string(),"nativeSourceRuntime":"sdk", + "cwd":owned.path(),"home":"/home/freshell/provider","settingSources":["local"], + "selectorManifestSha256":"b".repeat(64),"planRevision":0} + })).unwrap(); + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::ObserveClaudeSources(request), + ); + envelope.auth = Some("synthetic-control-secret".into()); + let result = tokio::time::timeout( + Duration::from_millis(250), + supervisor.dispatch(envelope), + ) + .await + .expect("nested Claude source lookup must not reacquire the held soul lifecycle lock") + .result; + assert_eq!( + result.is_ok(), + mode == "live", + "{mode}: changed scoped authority after IPC cannot qualify source evidence" + ); + if let Err(error) = result { + assert_eq!(error.code, RuntimeErrorCode::InvalidRequest); + } + assert!(std::fs::read(&prior).unwrap() == b"immutable accepted prior generation"); + } + } +} diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index 14c4c6517..aa5924f55 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -12,6 +12,74 @@ use std::{ struct AuthBackend(AtomicUsize); +#[tokio::test] +async fn provider_claude_creation_source_operation_requires_reader_evidence_after_real_absence() { + let temp = tempfile::tempdir().unwrap(); + let binary = temp.path().join("synthetic-host"); + std::fs::write(&binary, "synthetic helper artifact").unwrap(); + let registry = Registry::open(temp.path().join("registry"), None).unwrap(); + let backend = Arc::new(AuthBackend(AtomicUsize::new(0))); + let secret = "synthetic-control-secret"; + let supervisor = Supervisor::new( + registry, + backend.clone(), + SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: binary, + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "claude-source-authority".into(), + control_secret: secret.into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let soul = SoulId::new(); + let mut reserve = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::ReserveProviderInputs(ReserveProviderInputsRequest { + expected_control_epoch: Some(supervisor.registry.control_epoch()), + soul_id: soul.clone(), + provider: "claude".into(), + creation_seed_ref: "synthetic-creation-seed".into(), + }), + ); + reserve.auth = Some(secret.into()); + let AdminResult::ProviderInputReservation(receipt) = + supervisor.dispatch(reserve).await.result.unwrap() + else { + panic!("unexpected creation receipt"); + }; + let command: AdminCommand = serde_json::from_value(serde_json::json!({ + "method":"observe_claude_sources", "params": { + "expectedControlEpoch":supervisor.registry.control_epoch(), "soulId":soul, "provider":"claude", + "authority":{"kind":"initial_reservation","reservationId":receipt.reservation_id}, + "workspacePath":temp.path(), "gitCommonDir":null, "candidate":null, + "observationRequest": { + "version":1,"requestId":uuid::Uuid::new_v4().to_string(),"nativeSourceRuntime":"terminal", + "cwd":temp.path(),"home":"/home/freshell/provider","settingSources":["local"], + "selectorManifestSha256":"b".repeat(64),"planRevision":0 + } + } + })).expect("Claude source selection requires a real fixed-reader operation, independently of OpenCode auth lookup"); + let mut lookup = Envelope::new(RequestId::new(), ControlRole::Web, command); + lookup.auth = Some(secret.into()); + let result = supervisor.dispatch(lookup).await.result; + assert_eq!(result.unwrap_err().code, RuntimeErrorCode::HostUnreachable, + "proved initial volume absence must not substitute for unavailable source-selection evidence"); + assert_eq!( + backend.0.load(Ordering::SeqCst), + 3, + "reserve absence, reobserve absence, then invoke the distinct Claude fixed reader" + ); + assert!( + supervisor.registry.inventory().await.unwrap().is_empty(), + "source observation must not create a workload, grant or native conversation" + ); +} + #[tokio::test] async fn provider_ordinary_initial_reservation_cannot_authorize_opencode_auth_lookup() { let temp = tempfile::tempdir().unwrap(); @@ -328,6 +396,18 @@ async fn provider_frozen_path_binding_is_idempotent_private_and_generation_scope #[async_trait] impl RuntimeBackend for AuthBackend { + async fn observe_claude_sources( + &self, + owner: &ClaudeSourceOwner, + _: &ObserveClaudeSourcesRequest, + _: &str, + ) -> Result { + assert!(matches!(owner, ClaudeSourceOwner::Initial { .. })); + self.0.fetch_add(1, Ordering::SeqCst); + Err(BackendError::Unavailable( + "synthetic fixed reader unavailable".into(), + )) + } async fn observe_initial_auth_volume( &self, reservation: &OwnedInputReservation, diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index d5910b1e4..74cb6d760 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -19,6 +19,7 @@ use std::{ pub const SCHEMA_VERSION: u32 = 9; mod input_reservation; +pub(crate) use input_reservation::binary_digest; pub use input_reservation::{BoundInputReservation, OwnedInputReservation}; const ACTIVE_STATES: &str = "'prepared','created','starting','running','stopping'"; diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index 5a44106e5..4d160603c 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -46,7 +46,7 @@ pub(super) fn create_table(conn: &Connection) -> Result<(), RegistryError> { )?; Ok(()) } -fn binary_digest(binary: &Path) -> Result { +pub(crate) fn binary_digest(binary: &Path) -> Result { use std::io::Read; use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; let metadata = std::fs::symlink_metadata(binary)?; diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index c14fe7fd4..e6a56b058 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -77,6 +77,8 @@ struct AuthLookupScope { key: String, } +#[path = "claude_source_observation.rs"] +mod claude_source_observation; #[cfg(test)] #[path = "native_auth_lookup_tests.rs"] mod native_auth_lookup_tests; @@ -290,6 +292,14 @@ impl Supervisor { .await .map(AdminResult::NativeAuthLookup) } + AdminCommand::ObserveClaudeSources(request) => { + self.registry + .assert_epoch(request.expected_control_epoch) + .map_err(map_registry)?; + self.observe_claude_sources(request) + .await + .map(AdminResult::ClaudeSourceObservation) + } AdminCommand::Health => Ok(AdminResult::Health { control_epoch: self.registry.control_epoch(), installation_id: self.registry.installation_id().clone(), From 084261083f93fc4983ab9d53f6a4e3c38b5b065d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 21:17:47 -0700 Subject: [PATCH 107/493] Stage Claude local settings in observed native order --- .../managed_mcp_capability/claude_stage.rs | 148 +++++++++++++++++- 1 file changed, 140 insertions(+), 8 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index b8f771a80..cccdec0e5 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -1,6 +1,8 @@ //! Finite Claude source roots enter the same ordinary resource graph as Codex. use super::ordinary_stage::{prepare_selected, Preparation}; -use freshell_platform::claude_inputs::{ClaudeInputManifest, ClaudeSettingSource}; +use freshell_platform::claude_inputs::{ + observation::NativeObservation, ClaudeInputManifest, ClaudeSettingSource, +}; use std::path::{Path, PathBuf}; const RESOURCE_DIRECTORIES: &[&str] = &[ @@ -14,12 +16,48 @@ const RESOURCE_DIRECTORIES: &[&str] = &[ "routines", ]; +pub(super) fn prepare_with_native_observation( + home: &Path, + cwd: &Path, + workspace: &Path, + accepted: &ClaudeInputManifest, + observation: &NativeObservation, + excluded: &[PathBuf], +) -> Result { + observation.validate()?; + if Some(observation.native_source_runtime) != accepted.native_source_runtime + || observation.setting_sources != accepted.setting_sources + || std::fs::canonicalize(cwd).ok().as_ref() != Some(&observation.cwd) + { + return Err("Claude native source observation does not match the selected launch; retain inputs and reprepare".into()); + } + prepare_selected_sources( + home, + cwd, + workspace, + Some(accepted), + Some(observation), + excluded, + ) +} + pub(super) fn prepare( home: &Path, cwd: &Path, workspace: &Path, accepted: Option<&ClaudeInputManifest>, excluded: &[PathBuf], +) -> Result { + prepare_selected_sources(home, cwd, workspace, accepted, None, excluded) +} + +fn prepare_selected_sources( + home: &Path, + cwd: &Path, + workspace: &Path, + accepted: Option<&ClaudeInputManifest>, + observation: Option<&NativeObservation>, + excluded: &[PathBuf], ) -> Result { let mut manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { version: 1, @@ -52,7 +90,11 @@ pub(super) fn prepare( manifest.settings_origin = Some(origin.clone()); configurations.push((origin.clone(), origin)); } - let native_cwd = std::fs::canonicalize(&manifest.cwd).unwrap_or_else(|_| manifest.cwd.clone()); + let native_cwd = observation + .map(|observed| observed.cwd.clone()) + .unwrap_or_else(|| { + std::fs::canonicalize(&manifest.cwd).unwrap_or_else(|_| manifest.cwd.clone()) + }); let mut bases: Vec = native_cwd .ancestors() .filter(|path| path.parent().is_some()) @@ -70,7 +112,9 @@ pub(super) fn prepare( let local_enabled = manifest .setting_sources .contains(&ClaudeSettingSource::Local); - let roots = if project_enabled { + let roots = if let Some(observed) = observation { + observed.roots.clone() + } else if project_enabled { freshell_platform::claude_inputs::git::roots( &native_cwd, manifest @@ -112,14 +156,20 @@ pub(super) fn prepare( } let mut settings = Vec::new(); if project_enabled { - settings.push(manifest.cwd.join(".claude/settings.json")); + settings.push(native_cwd.join(".claude/settings.json")); } if local_enabled { - settings.push(manifest.cwd.join(".claude/settings.local.json")); - settings.push(manifest.workspace.join(".claude/settings.local.json")); + if let Some(observed) = observation { + settings.extend(observed.local_settings.iter().cloned()); + } else { + settings.push(manifest.cwd.join(".claude/settings.local.json")); + settings.push(manifest.workspace.join(".claude/settings.local.json")); + } + } + if observation.is_none() { + settings.sort(); + settings.dedup(); } - settings.sort(); - settings.dedup(); for logical in settings { if optional(&logical)? { configurations.push((logical.clone(), canonical(&logical)?)); @@ -392,6 +442,88 @@ mod tests { provider_environment::{copy_environment, CopyLimits}, }; + #[cfg(unix)] + #[test] + fn managed_provider_claude_observed_local_order_consumes_cwd_then_canonical_and_leaves_workspace_poison( + ) { + use freshell_platform::claude_inputs::{observation::observe, NativeSourceRuntime}; + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let canonical_root = owned.path().join("canonical"); + let workspace = owned.path().join("checkout"); + let cwd = workspace.join("nested"); + let gitdir = canonical_root.join(".git/worktrees/fixture"); + for path in [ + &home, + &gitdir, + &cwd.join(".claude"), + &workspace.join(".claude"), + &canonical_root.join(".claude"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + workspace.join(".git"), + format!("gitdir: {}\n", gitdir.display()), + ) + .unwrap(); + std::fs::write(gitdir.join("commondir"), "../..\n").unwrap(); + std::fs::write( + gitdir.join("gitdir"), + format!("{}\n", workspace.join(".git").display()), + ) + .unwrap(); + let current = cwd.join(".claude/settings.local.json"); + let canonical = canonical_root.join(".claude/settings.local.json"); + let unselected = workspace.join(".claude/settings.local.json"); + std::fs::write( + ¤t, + br#"{"env":{"SELECTED":"cwd","CWD_ONLY":"retained"}}"#, + ) + .unwrap(); + std::fs::write(&canonical, br#"{"env":{"SELECTED":"canonical"}}"#).unwrap(); + let poison = b"NATIVE_UNSELECTED_WORKSPACE_POISON: invalid JSON"; + std::fs::write(&unselected, poison).unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(runtime), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: workspace.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Local], + plugin_dirs: vec![], + }; + let observed = observe(&cwd, &home, runtime, &manifest.setting_sources).unwrap(); + assert_eq!(observed.uid, unsafe { libc::getuid() }); + assert_eq!(observed.local_settings, [current, canonical]); + let selected=prepare_with_native_observation(&home,&cwd,&workspace,&manifest,&observed,&[]) + .expect("native-selected local settings must prepare without the unrelated workspace document"); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + stage_settings(&stage, &selected, None) + .expect("unselected malformed workspace settings must not be read or staged"); + let files = selected + .binds + .iter() + .map(|bind| stage.join(&bind.source_relative_path)) + .collect::>(); + let output=std::process::Command::new("node").args(["-e", "const fs=require('node:fs');let env={};for(const file of process.argv.slice(1)){Object.assign(env,JSON.parse(fs.readFileSync(file,'utf8')).env||{})};process.stdout.write(env.SELECTED+':'+env.CWD_ONLY)"]) + .args(files).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"canonical:retained"); + assert!(std::fs::read(&unselected).unwrap() == poison); + assert!(!selected + .selections + .iter() + .any(|copy| copy.source == unselected)); + } + } + #[cfg(unix)] #[test] fn managed_provider_claude_project_resources_stop_at_git_and_home_without_losing_memory() { From ef141934e83860cc89025ce502ea6250026c90f5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 21:41:36 -0700 Subject: [PATCH 108/493] Allow scoped Claude source readers through the native test broker --- scripts/testing/runtime-test-broker.ts | 107 +++++++++++++++++- .../testing/runtime-test-broker.test.ts | 100 +++++++++++++++- 2 files changed, 204 insertions(+), 3 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 101efd322..2f064e6d6 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -169,6 +169,7 @@ export class RestrictedDockerBroker { private readonly knownContainerIds = new Set() private readonly receiptsById = new Map() private readonly historyHelpers = new Map() + private readonly claudeSourceHelpers = new Map; containerId?: string }>() private readonly events: BrokerEvent[] = [] private server?: http.Server private stopFailuresRemaining = 0 @@ -249,7 +250,7 @@ export class RestrictedDockerBroker { } receiptIds(): Set { - return new Set([...this.knownContainerIds, ...[...this.historyHelpers.values(),...this.inputPublishers.values()].flatMap((helper) => helper.containerId ? [helper.containerId] : [])]) + return new Set([...this.knownContainerIds, ...[...this.historyHelpers.values(),...this.inputPublishers.values(),...this.claudeSourceHelpers.values()].flatMap((helper) => helper.containerId ? [helper.containerId] : [])]) } eventsSnapshot(): BrokerEvent[] { @@ -375,6 +376,33 @@ export class RestrictedDockerBroker { } const helperTarget = url.match(/^\/v1\.47\/containers\/([^/?]+)(?:\/(start|wait|logs|json))?(?:\?.*)?$/) + const sourceEntry = helperTarget && [...this.claudeSourceHelpers].find(([name, helper]) => ( + helperTarget[1] === name || helperTarget[1] === helper.containerId + )) + if (sourceEntry) { + const [name, helper] = sourceEntry + const targetName = helperTarget![1] + const exactId = targetName === helper.containerId + const deleting = method === 'DELETE' && url === `${DOCKER_API_PREFIX}/containers/${targetName}?force=1` + if (deleting) { + const inspected = await this.forward({ method: 'GET', url: `${DOCKER_API_PREFIX}/containers/${targetName}/json`, headers: {} } as IncomingMessage, Buffer.alloc(0)) + let actual: Record = {} + try { actual = JSON.parse(inspected.body.toString()) } catch {} + if (inspected.statusCode !== 404 && (inspected.statusCode !== 200 || !this.ownedClaudeSourceHelper(actual, helper.expected))) { + this.block(response, method, url, 403, 'Claude source helper cleanup ownership changed', true) + return + } + } + if (deleting + || (method === 'GET' && url === `${DOCKER_API_PREFIX}/containers/${targetName}/json`) + || (method === 'POST' && exactId && [ + `${DOCKER_API_PREFIX}/containers/${targetName}/start`, + `${DOCKER_API_PREFIX}/containers/${targetName}/wait?condition=not-running`, + ].includes(url))) { + await this.forwardAndReply(request, response, body, { destructive: method !== 'GET', containerId: helper.containerId, helperName: name }) + return + } + } const publisher=helperTarget && [...this.inputPublishers].find(([name,publisher])=>helperTarget[1]===name || helperTarget[1]===publisher.containerId) if (publisher) { const [name,owned]=publisher @@ -430,6 +458,30 @@ export class RestrictedDockerBroker { if (method === 'POST' && url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`)) { const name = new URL(url, 'http://docker').searchParams.get('name') ?? '' + if (name.startsWith('freshell-claude-source-')) { + if (!/^freshell-claude-source-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) + || !this.validateClaudeSourceHelper(body) || this.claudeSourceHelpers.has(name)) { + this.block(response, method, url, 403, 'Claude source helper must match its exact owned fixed reader', false) + return + } + const helper: { expected: Record; containerId?: string } = { expected: JSON.parse(body.toString()) } + this.claudeSourceHelpers.set(name, helper) + try { + const forwarded = await this.forward(request, body) + if (forwarded.statusCode === 201) { + const parsed = JSON.parse(forwarded.body.toString()) + if (typeof parsed.Id !== 'string' || !/^[a-f0-9]{64}$/.test(parsed.Id) || this.receiptIds().has(parsed.Id)) throw new Error('invalid Claude source helper identity') + helper.containerId = parsed.Id + } + this.record({ method, url, decision: 'forward', destructive: false, unsafeAttempt: false, helperName: name, containerId: helper.containerId, requestDigest: sha256(body) }) + replyDocker(response, forwarded) + } catch { + this.record({ method, url, decision: 'forward', reason: 'Claude source helper acknowledgement unavailable', destructive: false, unsafeAttempt: false, helperName: name, requestDigest: sha256(body) }) + response.statusCode = 503 + response.end(JSON.stringify({ message: 'Claude source helper acknowledgement unavailable; exact named ownership inspection required' })) + } + return + } const mountpointHelper=/^freshell-mountpoints-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name) || mountpointHelper) { const input=mountpointHelper ? this.validateMountpointHelper(body) : this.validateInputPublisher(name,body) @@ -691,6 +743,59 @@ export class RestrictedDockerBroker { return { ok: true, incarnationId, soulId, imageRef, runtimeDir, hostBinaryPath, ...(providerVolumeName ? { providerVolumeName } : {}), ...(workspacePath ? { workspacePath } : {}) } } + private validateClaudeSourceHelper(body: Buffer): boolean { + let parsed: Record + try { parsed = JSON.parse(body.toString()) } catch { return false } + const labels = parsed.Labels ?? {} + const owner = [...this.initialProviderVolumes.values()].find((input) => input.installationId === labels['com.freshell.installation-id'] + && input.soulId === labels['com.freshell.soul-id'] && input.imageRef === parsed.Image + && input.binarySha256 === labels['com.freshell.source-binary-sha256']) + if (!owner || parsed.User !== '65534:0' || parsed.Tty !== false + || JSON.stringify(parsed.Entrypoint) !== JSON.stringify(['/runtime/freshell-session-host']) + || JSON.stringify(parsed.Cmd) !== JSON.stringify(['observe-claude-sources-private']) + || !privateHelperEnvironment(parsed.Env, true) + || !/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(labels['com.freshell.source-request-id']) + || !equalRecord(labels, { + 'com.freshell.installation-id': owner.installationId, 'com.freshell.soul-id': owner.soulId, + 'com.freshell.runtime-test-run-id': this.policy.testRunId, + 'com.freshell.source-request-id': labels['com.freshell.source-request-id'], + 'com.freshell.source-binary-sha256': owner.binarySha256, + })) return false + const host = parsed.HostConfig ?? {} + if (Object.keys(parsed).some((key) => !['Image','User','Tty','WorkingDir','Labels','Entrypoint','Cmd','Env','HostConfig'].includes(key)) + || Object.keys(host).some((key) => !['NetworkMode','ReadonlyRootfs','CapDrop','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','LogConfig','Tmpfs','Mounts'].includes(key)) + || host.NetworkMode !== 'none' || host.ReadonlyRootfs !== true + || JSON.stringify(host.CapDrop) !== JSON.stringify(['ALL']) || JSON.stringify(host.SecurityOpt) !== JSON.stringify(['no-new-privileges']) + || host.Memory !== 256 * 1024 * 1024 || host.MemorySwap !== 256 * 1024 * 1024 || host.NanoCpus !== 500_000_000 || host.PidsLimit !== 32 + || !equalRecord(host.LogConfig, { Type: 'none', Config: host.LogConfig?.Config }) || Object.keys(host.LogConfig?.Config ?? {}).length + || !equalRecord(host.Tmpfs, { '/home/freshell/provider': 'rw,nosuid,nodev,noexec,size=16m,uid=65534,gid=0,mode=0700' })) return false + const mounts = host.Mounts + if (!Array.isArray(mounts) || mounts.length !== 3 + || mounts.some((mount) => !mount || Object.keys(mount).some((key) => !['Type','Source','Target','ReadOnly'].includes(key)))) return false + const binary = mounts.find((mount) => mount.Type === 'bind' && mount.Source === owner.hostBinaryPath && mount.Target === '/runtime/freshell-session-host' && mount.ReadOnly === true) + const workspace = mounts.find((mount) => mount.Type === 'bind' && mount.Source === mount.Target && this.policy.allowedWorkspaceRoots?.has(mount.Source) && mount.ReadOnly === true) + const exchange = mounts.find((mount) => mount.Type === 'bind' && mount.Target === '/run/freshell-source-exchange' && mount.ReadOnly === true + && typeof mount.Source === 'string' && path.dirname(mount.Source) === os.tmpdir() + && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + return !!binary && !!workspace && !!exchange && typeof parsed.WorkingDir === 'string' + && path.normalize(parsed.WorkingDir) === parsed.WorkingDir + && (parsed.WorkingDir === workspace.Source || isStrictDescendant(parsed.WorkingDir, workspace.Source)) + } + + private ownedClaudeSourceHelper(actual: Record, expected: Record): boolean { + const config = actual.Config ?? {}, host = actual.HostConfig ?? {} + const same = (actual: unknown, expected: unknown) => JSON.stringify(actual) === JSON.stringify(expected) + return ['Image','User','Tty','WorkingDir'].every((key) => config[key] === expected[key]) + && ['Entrypoint','Cmd'].every((key) => same(config[key], expected[key])) + && ownedHelperLabels(config.Labels, expected.Labels) + && Array.isArray(config.Env) && same([...config.Env].sort(), [...expected.Env].sort()) + && ['NetworkMode','ReadonlyRootfs','CapDrop','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','Tmpfs'].every((key) => same(host[key], expected.HostConfig[key])) + && (host.CapAdd?.length ?? 0) === 0 && host.LogConfig?.Type === 'none' + && Array.isArray(host.Mounts) && host.Mounts.length === expected.HostConfig.Mounts.length + && expected.HostConfig.Mounts.every((wanted: Record) => host.Mounts.some((mount: Record) => + ['Type','Source','Target','ReadOnly'].every((key) => mount[key] === wanted[key]) && !mount.VolumeOptions)) + } + private validateInputPublisher(name:string,body:Buffer):PreparedInputAuthorization | undefined { let parsed:Record try {parsed=JSON.parse(body.toString())} catch {return} diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index e02f22095..fd0885c38 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -274,6 +274,101 @@ function authHelperBody(): any { return body } +const sourceName = 'freshell-claude-source-11111111-1111-4111-8111-111111111111' +function sourceAuthorization() { + const installationId = 'installation-one', soulId = 'soul-initial' + return { + installationId, soulId, + providerVolumeName: `freshell-provider-${createHash('sha256').update(`${installationId}\0${soulId}`).digest('hex').slice(0, 24)}`, + imageRef: 'sha256:fixture-image', hostBinaryPath: '/tmp/owned/freshell-session-host', + binarySha256: 'a'.repeat(64), daemonId: 'fixture-daemon', + } +} +function sourceHelperBody(): any { + const owner = sourceAuthorization() + return { + Image: owner.imageRef, User: '65534:0', Tty: false, WorkingDir: '/workspace/nested', + Entrypoint: ['/runtime/freshell-session-host'], Cmd: ['observe-claude-sources-private'], + Env: ['PATH=', 'HOME=/home/freshell/provider'], + Labels: { + 'com.freshell.installation-id': owner.installationId, + 'com.freshell.soul-id': owner.soulId, + 'com.freshell.runtime-test-run-id': 'run-one', + 'com.freshell.source-request-id': '22222222-2222-4222-8222-222222222222', + 'com.freshell.source-binary-sha256': owner.binarySha256, + }, + HostConfig: { + NetworkMode: 'none', ReadonlyRootfs: true, CapDrop: ['ALL'], SecurityOpt: ['no-new-privileges'], + Memory: 256 * 1024 * 1024, MemorySwap: 256 * 1024 * 1024, NanoCpus: 500_000_000, PidsLimit: 32, + LogConfig: { Type: 'none', Config: {} }, + Tmpfs: { '/home/freshell/provider': 'rw,nosuid,nodev,noexec,size=16m,uid=65534,gid=0,mode=0700' }, + Mounts: [ + { Type: 'bind', Source: owner.hostBinaryPath, Target: '/runtime/freshell-session-host', ReadOnly: true }, + { Type: 'bind', Source: '/workspace', Target: '/workspace', ReadOnly: true }, + { Type: 'bind', Source: '/tmp/freshell-inputs-33333333-3333-4333-8333-333333333333', Target: '/run/freshell-source-exchange', ReadOnly: true }, + ], + }, + } +} + +describe('scoped Claude source observer broker', () => { + it('allows the reserved fixed reader and exact private lifecycle without creating initial HOME or exposing logs', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance }) => { + const body = sourceHelperBody() + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, body)).status).toBe(403) + expect(forwarded).toHaveLength(0) + instance.authorizeInitialProviderVolume(sourceAuthorization()) + for (const mutate of [ + (value: any) => { value.User = '0:0' }, + (value: any) => { value.Cmd.push('PRIVATE_SOURCE_SENTINEL') }, + (value: any) => { value.Env.push('SECRET=PRIVATE_SOURCE_SENTINEL') }, + (value: any) => { value.WorkingDir = '/outside' }, + (value: any) => { value.Labels['com.freshell.soul-id'] = 'foreign' }, + (value: any) => { value.Labels['com.freshell.source-binary-sha256'] = 'b'.repeat(64) }, + (value: any) => { value.HostConfig.CapAdd = ['SYS_ADMIN'] }, + (value: any) => { value.HostConfig.NetworkMode = 'host' }, + (value: any) => { value.HostConfig.Tmpfs['/home/freshell/provider'] = 'rw,size=128m' }, + (value: any) => { value.HostConfig.Mounts[1].ReadOnly = false }, + (value: any) => { value.HostConfig.Mounts.push({ Type: 'bind', Source: '/operator-home', Target: '/outside', ReadOnly: true }) }, + (value: any) => { value.HostConfig.LogConfig.Type = 'json-file' }, + ]) { + const changed = structuredClone(body); mutate(changed) + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, changed)).status).toBe(403) + expect(forwarded).toHaveLength(0) + } + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, body)).status).toBe(201) + expect(instance.receiptIds().has(helperId)).toBe(true) + expect((await request('GET', `/v1.47/containers/${helperId}/json`)).status).toBe(200) + expect((await request('POST', `/v1.47/containers/${helperId}/start`)).status).toBe(204) + expect((await request('POST', `/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200) + const before = forwarded.length + for (const [method, url] of [ + ['GET', `/v1.47/containers/${helperId}/logs?stdout=1&stderr=1`], + ['POST', `/v1.47/containers/${helperId}/stop`], + ['POST', `/v1.47/containers/${sourceName}/start`], + ['POST', `/v1.47/containers/${helperId}/exec`], + ]) { + expect((await request(method, url)).status).toBe(403) + expect(forwarded).toHaveLength(before) + } + expect((await request('DELETE', `/v1.47/containers/${sourceName}?force=1`)).status).toBe(204) + expect(forwarded.some((operation) => operation.url === '/v1.47/volumes/create')).toBe(false) + expect(JSON.stringify(instance.eventsSnapshot())).not.toContain('PRIVATE_SOURCE_SENTINEL') + }) + }) + it('retains exact-name cleanup after lost acknowledgement and refuses changed source helper ownership', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance, loseHelperAck, replaceHelperInspection }) => { + instance.authorizeInitialProviderVolume(sourceAuthorization()) + loseHelperAck() + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, sourceHelperBody())).status).toBe(503) + expect((await request('GET', `/v1.47/containers/${sourceName}/json`)).status).toBe(200) + replaceHelperInspection() + expect((await request('DELETE', `/v1.47/containers/${sourceName}?force=1`)).status).toBe(403) + expect(forwarded.filter((operation) => operation.method === 'DELETE')).toEqual([]) + }) + }) +}) + describe('receipt-owned auth exchange HTTP routes', () => { it('allows only the fixed private helper, lifecycle and ownership inspection with Docker logs disabled', async () => { await withHttpBroker(async ({ request, forwarded }) => { @@ -475,8 +570,9 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise if (req.method==='GET' && req.url?.startsWith('/v1.47/volumes/freshell-inputs-') && indexedVolumeMetadata) { res.statusCode=200;res.end(JSON.stringify(indexedVolumeMetadata));return; } - if (req.method==='GET' && req.url?.includes('-publisher/json')) { - const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]==='publish-provider-inputs')!.body; + if (req.method==='GET' && (req.url?.includes('-publisher/json') || req.url?.includes('/freshell-claude-source-'))) { + const command=req.url?.includes('/freshell-claude-source-')?'observe-claude-sources-private':'publish-provider-inputs'; + const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]===command)!.body; const inspected=structuredClone(config); inspected.Labels['org.opencontainers.image.version']='24.04'; for (const mount of inspected.HostConfig.Mounts) if (mount.ReadOnly===false) delete mount.ReadOnly; From 8134a6edcfc2fecebc6bb15b0f4a7c443dfc8678 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:00:47 -0700 Subject: [PATCH 109/493] Qualify initial Claude source readers in the native startup fixture --- .../src/managed_provider_launch_tests.rs | 114 +++++++++++++++++- 1 file changed, 113 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 55bf51d51..a72505997 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -138,6 +138,16 @@ impl BrokerAuthorizedBackend { } #[async_trait] impl RuntimeBackend for BrokerAuthorizedBackend { + async fn observe_claude_sources( + &self, + owner: &freshell_supervisor::backend::ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + test_run_id: &str, + ) -> Result { + self.real + .observe_claude_sources(owner, request, test_run_id) + .await + } async fn observe_initial_auth_volume( &self, reservation: &OwnedInputReservation, @@ -1039,6 +1049,91 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .await .unwrap() .unwrap(); + async fn qualify_private_source_reader( + client: &freshell_runtime_client::RuntimeClient, + soul: SoulId, + provider: &str, + seed: &str, + workspace: &Path, + runtime: freshell_platform::claude_inputs::NativeSourceRuntime, + ) { + use freshell_platform::claude_inputs::{observation::LocalDecision, ClaudeSettingSource}; + use std::os::unix::fs::MetadataExt; + let cwd = workspace.join("source-observer/nested"); + std::fs::create_dir_all(cwd.join(".claude")).unwrap(); + let local = cwd.join(".claude/settings.local.json"); + std::fs::write(&local, b"{}").unwrap(); + let reservation = client + .reserve_provider_inputs(soul.clone(), provider.into(), seed.into()) + .await + .expect("same initial creation must reserve absent HOME before source observation"); + let request = ObserveClaudeSourcesRequest { + expected_control_epoch: None, + soul_id: soul, + provider: provider.into(), + authority: ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation.reservation_id, + }, + workspace_path: workspace.to_str().unwrap().into(), + git_common_dir: None, + candidate: None, + observation_request: + freshell_platform::claude_inputs::observation::ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: runtime, + cwd: cwd.clone(), + home: "/home/freshell/provider".into(), + setting_sources: vec![ClaudeSettingSource::Local], + selector_manifest_sha256: "a".repeat(64), + plan_revision: 0, + }, + }; + let reply = client.observe_claude_sources(request).await.expect( + "actual scoped private source reader must finish and clean up before native creation", + ); + let observed = reply.observation; + assert_eq!( + (observed.uid, observed.euid, observed.gid, observed.egid), + (65534, 65534, 0, 0) + ); + assert_eq!(observed.native_source_runtime, runtime); + assert_eq!(observed.cwd, cwd); + assert_eq!( + observed.home.as_deref(), + Some(Path::new("/home/freshell/provider")) + ); + let owner = std::fs::metadata(workspace).unwrap().uid(); + assert_ne!( + owner, 65534, + "fixture ownership must exercise the actual reader UID predicate" + ); + assert_eq!(observed.root_uid, Some(owner)); + assert_eq!(observed.local_decision, LocalDecision::RootOwnerMismatch); + let roots = observed.roots.unwrap(); + assert_eq!(roots.checkout, workspace); + assert_eq!(roots.canonical, workspace); + assert_eq!(observed.local_settings, [local.clone()]); + assert_eq!(std::fs::read(&local).unwrap(), b"{}"); + println!( + "provider_source_observer {}", + json!({"runtime":runtime,"uid":observed.uid,"gid":observed.gid,"decision":observed.local_decision,"localCount":observed.local_settings.len(),"sourceBytesPreserved":true}) + ); + } + let client = controller.runtime_client(); + qualify_private_source_reader( + &client, + SoulId::parse(format!( + "soul-terminal-{}", + &format!("{:x}", Sha256::digest(b"closed-native-terminal-create"))[..32] + )) + .unwrap(), + "claude", + "closed-native-terminal-create", + &workspace, + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ) + .await; let cwd = workspace.to_str().unwrap().to_string(); let ambient_home = fixture.join("ambient-home"); std::fs::create_dir_all(ambient_home.join(".claude")).unwrap(); @@ -1089,7 +1184,6 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ }) .await .expect("complete native Claude terminal startup"); - let client = controller.runtime_client(); async fn consumed(path: &Path) -> Option { tokio::time::timeout(Duration::from_secs(45), async { loop { @@ -1332,6 +1426,24 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .unwrap(); let flavor = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR").unwrap(); assert!(matches!(flavor.as_str(), "freshclaude" | "kilroy")); + let source_provider = if flavor == "kilroy" { + "kilroy" + } else { + "claude" + }; + qualify_private_source_reader( + &client, + SoulId::parse(format!( + "soul-fresh-{source_provider}-{}", + &format!("{:x}", Sha256::digest(b"closed-native-fresh-create"))[..32] + )) + .unwrap(), + source_provider, + "closed-native-fresh-create", + &workspace, + freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, + ) + .await; Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":flavor,"provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; tokio::time::timeout(Duration::from_secs(45), async { loop { From 09d40345f719ea5db5c5962b021ddd4f204bd2e1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:20:38 -0700 Subject: [PATCH 110/493] Observe native Claude sources before initial input preparation --- .../freshell-server/src/fresh_agent_proxy.rs | 1 + .../src/managed_mcp_capability.rs | 38 ++- .../src/managed_provider_references/claude.rs | 234 ++++++++++++++++++ crates/freshell-server/src/managed_runtime.rs | 1 + 4 files changed, 264 insertions(+), 10 deletions(-) diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 7cec277ba..4d19fed92 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -527,6 +527,7 @@ impl HostedFreshAgentProxy { native_source_runtime: freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, reservation_provider: provider.as_str(), + git_common_dir: git_common_dir(&workspace).as_deref(), selected_settings: None, client: &self.client, soul: &soul, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 945781de4..5eb9b8031 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -41,6 +41,8 @@ pub(crate) use tests::{qualify_claude_linked_sources, qualify_claude_local_setti pub(crate) struct OpencodeEphemeralInput<'a> { pub codex_manifest: Option<&'a freshell_platform::codex_inputs::CodexInputManifest>, pub claude_manifest: Option<&'a freshell_platform::claude_inputs::ClaudeInputManifest>, + pub claude_observation: + Option<&'a freshell_platform::claude_inputs::observation::NativeObservation>, pub claude_inline_settings: Option<&'a str>, pub amplifier_manifest: Option<&'a freshell_platform::amplifier_inputs::AmplifierInputManifest>, pub inline_config: Option<&'a str>, @@ -1392,16 +1394,32 @@ fn stage_provider_root( opencode_input.amplifier_manifest, )? } else if provider == "claude" { - claude_stage::prepare( - home, - opencode_input.cwd.unwrap_or(home), - opencode_input - .workspace - .or(opencode_input.cwd) - .unwrap_or(home), - opencode_input.claude_manifest, - &excluded, - )? + if let Some(observation) = opencode_input.claude_observation { + claude_stage::prepare_with_native_observation( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input + .claude_manifest + .ok_or("Claude native source observation lacks original selectors")?, + observation, + &excluded, + )? + } else { + claude_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input.claude_manifest, + &excluded, + )? + } } else { ordinary_stage::prepare( home, diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 1cb7acae0..4c1dd58f6 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -13,6 +13,7 @@ pub(crate) struct ClaudePreparation<'a> { pub home: &'a Path, pub cwd: &'a Path, pub workspace: &'a Path, + pub git_common_dir: Option<&'a Path>, pub original: &'a (dyn Fn(&str) -> Option + Sync), pub args: &'a [String], pub plugins: &'a [String], @@ -110,6 +111,62 @@ pub(crate) async fn prepare_managed( { return Err("Claude selected resource reservation does not match this creation".into()); } + let observation_request = freshell_platform::claude_inputs::observation::ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: request.native_source_runtime, + cwd: std::fs::canonicalize(request.cwd) + .map_err(|_| "Claude native cwd is unavailable; retain inputs and reprepare")?, + home: "/home/freshell/provider".into(), + setting_sources: manifest.setting_sources.clone(), + selector_manifest_sha256: { + use sha2::{Digest, Sha256}; + let bytes = bounded_json_bytes( + &manifest, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .map_err(|_| "Claude selector metadata exceeds its finite profile")?; + format!("{:x}", Sha256::digest(bytes)) + }, + plan_revision: 0, + }; + let observation_request_sha256 = { + use sha2::{Digest, Sha256}; + let bytes = bounded_json_bytes( + &observation_request, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .map_err(|_| "Claude source request exceeds its finite profile")?; + format!("{:x}", Sha256::digest(bytes)) + }; + let observation = request + .client + .observe_claude_sources(ObserveClaudeSourcesRequest { + expected_control_epoch: None, + soul_id: request.soul.clone(), + provider: request.reservation_provider.into(), + authority: ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation.reservation_id.clone(), + }, + workspace_path: request + .workspace + .to_str() + .ok_or("Claude workspace is not UTF-8; retain inputs and reprepare")? + .into(), + git_common_dir: request + .git_common_dir + .map(|path| { + path.to_str().map(str::to_owned).ok_or( + "Claude git common directory is not UTF-8; retain inputs and reprepare", + ) + }) + .transpose()?, + candidate: None, + observation_request: observation_request.clone(), + }) + .await + .map_err(|_| "Claude native source evidence is unavailable; retain inputs and reprepare")?; + observation.validate_for_request(&observation_request, &observation_request_sha256)?; crate::managed_mcp_capability::issue_prepared_provider_context( "claude", request.soul, @@ -117,6 +174,7 @@ pub(crate) async fn prepare_managed( &context, crate::managed_mcp_capability::OpencodeEphemeralInput { claude_manifest: Some(&manifest), + claude_observation: Some(&observation.observation), claude_inline_settings: inline_settings, cwd: Some(request.cwd), workspace: Some(request.workspace), @@ -191,6 +249,182 @@ fn selected_config_root( mod tests { use super::*; + #[cfg(unix)] + #[tokio::test] + async fn managed_provider_claude_initial_source_failure_blocks_before_stage_and_dispatch() { + use freshell_supervisor::{ + backend::{ + BackendCreated, BackendError, BackendInspection, ClaudeSourceOwner, + CreateRuntimeSpec, RuntimeBackend, + }, + registry::{OwnedInputReservation, OwnedRuntimeHandle, Registry}, + service::{serve_control, Supervisor, SupervisorConfig}, + }; + use std::sync::{ + atomic::{AtomicUsize, Ordering}, + Arc, + }; + + struct UnavailableSources { + daemon: DockerDaemonId, + calls: AtomicUsize, + } + #[async_trait::async_trait] + impl RuntimeBackend for UnavailableSources { + async fn observe_initial_auth_volume( + &self, + _: &OwnedInputReservation, + ) -> Result { + Ok(InitialAuthVolumeObservation::Absent { + daemon_id: self.daemon.clone(), + }) + } + async fn observe_claude_sources( + &self, + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + _: &str, + ) -> Result + { + assert!(matches!(owner, ClaudeSourceOwner::Initial { .. })); + assert_eq!(request.provider, "claude"); + assert_eq!( + request.observation_request.native_source_runtime, + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal + ); + assert_eq!( + request.observation_request.setting_sources, + vec![freshell_platform::claude_inputs::ClaudeSettingSource::Local] + ); + self.calls.fetch_add(1, Ordering::SeqCst); + Err(BackendError::Unavailable( + "synthetic fixed reader unavailable".into(), + )) + } + async fn create_stopped( + &self, + _: &CreateRuntimeSpec, + ) -> Result { + unreachable!() + } + async fn start_host(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + async fn inspect( + &self, + _: &OwnedRuntimeHandle, + ) -> Result { + unreachable!() + } + async fn request_stop( + &self, + _: &OwnedRuntimeHandle, + _: u64, + ) -> Result<(), BackendError> { + unreachable!() + } + async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + async fn verify_empty(&self, _: &OwnedRuntimeHandle) -> Result { + unreachable!() + } + async fn list_known( + &self, + _: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + unreachable!() + } + async fn read_limits( + &self, + _: &OwnedRuntimeHandle, + ) -> Result { + unreachable!() + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + unreachable!() + } + } + let owned = tempfile::tempdir().unwrap(); + let binary = owned.path().join("host"); + std::fs::write(&binary, "synthetic immutable host artifact").unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".claude")).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let selected = home.join(".claude/settings.json"); + let private = br#"{"env":{"SYNTHETIC_PRIVATE":"preserve-original-source"}}"#; + std::fs::write(&selected, private).unwrap(); + let registry = Registry::open(owned.path().join("registry"), None).unwrap(); + let backend = Arc::new(UnavailableSources { + daemon: DockerDaemonId::new(), + calls: AtomicUsize::new(0), + }); + let socket = owned.path().join("control.sock"); + let supervisor = Supervisor::new( + registry.clone(), + backend.clone(), + SupervisorConfig { + runtime_root: owned.path().join("runtime"), + control_socket_path: socket.clone(), + host_binary_path: binary, + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "synthetic-source-unavailable".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: owned.path().join("lifecycle.jsonl"), + admission: freshell_supervisor::admission::AdmissionPolicy::default(), + }, + ) + .unwrap(); + let task = tokio::spawn({ + let socket = socket.clone(); + async move { serve_control(supervisor, &socket).await } + }); + let result = tokio::time::timeout(std::time::Duration::from_secs(5), async { + while !socket.exists() { + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + let client = freshell_runtime_client::RuntimeClient::new( + socket.clone(), + "synthetic-control-secret", + ); + let soul = SoulId::new(); + prepare_managed(ClaudePreparation { + client: &client, + soul: &soul, + creation_seed: "synthetic-initial-source", + native_source_runtime: + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + reservation_provider: "claude", + selected_settings: None, + home: &home, + cwd: &workspace, + workspace: &workspace, + git_common_dir: None, + original: &|_| None, + args: &["--setting-sources".into(), "local".into()], + plugins: &[], + references: &[], + }) + .await + }) + .await; + task.abort(); + assert!( + task.await.unwrap_err().is_cancelled(), + "join the owned control service before assertions" + ); + let result = + result.expect("initial preparation must return within its finite test deadline"); + assert_eq!(backend.calls.load(Ordering::SeqCst), 1, + "authoritative source evidence must be requested before private staging or grant publication"); + let error = result.unwrap_err(); + assert!(error.contains("retain inputs and reprepare"), "{error}"); + assert!(!error.contains("preserve-original-source")); + assert!(registry.inventory().await.unwrap().is_empty()); + assert_eq!(std::fs::read(selected).unwrap(), private); + } + #[test] fn managed_provider_claude_settings_lookup_failure_keeps_private_document_out_of_errors_and_logs( ) { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 08f325f09..87060cc6c 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -273,6 +273,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { native_source_runtime: freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, reservation_provider: "claude", + git_common_dir: git_common_dir.as_deref(), selected_settings: selected_claude_settings.as_deref(), client: &self.client, soul: &soul_id, From 76e1d52b5bc20f09b16cdc664fb36cf8719949d0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:25:16 -0700 Subject: [PATCH 111/493] Reject changes to accepted Claude native source selection --- .../src/claude_inputs/observation.rs | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/crates/freshell-platform/src/claude_inputs/observation.rs b/crates/freshell-platform/src/claude_inputs/observation.rs index 5c70f8796..174c79a1b 100644 --- a/crates/freshell-platform/src/claude_inputs/observation.rs +++ b/crates/freshell-platform/src/claude_inputs/observation.rs @@ -210,6 +210,16 @@ fn validate_path(path: &Path) -> Result<(), String> { } impl NativeObservation { + /// The final provider namespace must retain the selected native source view. + pub fn require_same_selection(&self, current: &Self) -> Result<(), String> { + self.validate()?; + current.validate()?; + if self != current { + return Err("Claude native source selection changed before dispatch; retain inputs and reprepare".into()); + } + Ok(()) + } + pub fn validate(&self) -> Result<(), String> { validate_sources(&self.setting_sources)?; let local_enabled = self.setting_sources.contains(&ClaudeSettingSource::Local); @@ -448,6 +458,44 @@ mod tests { assert_eq!(std::fs::read(&prior).unwrap(), b"prior-accepted-inputs"); } + #[test] + fn claude_native_final_selection_blocks_changed_git_view_before_consuming_sources() { + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let root = owned.path().join("root"); + let cwd = root.join("nested"); + for path in [ + &home, + &root.join(".git"), + &root.join(".claude"), + &cwd.join(".claude"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + let prior = root.join(".claude/settings.local.json"); + let bytes = br#"{"env":{"SELECTED":"canonical-private-synthetic"}}"#; + std::fs::write(&prior, bytes).unwrap(); + std::fs::write(cwd.join(".claude/settings.local.json"), b"{}").unwrap(); + let accepted = observe(&cwd, &home, runtime, &[ClaudeSettingSource::Local]).unwrap(); + assert_eq!(accepted.local_decision, LocalDecision::QualifiedCanonical); + let same = observe(&cwd, &home, runtime, &[ClaudeSettingSource::Local]).unwrap(); + accepted + .require_same_selection(&same) + .expect("unchanged actual namespace remains dispatchable"); + std::fs::rename(root.join(".git"), root.join("prior-git")).unwrap(); + let changed = observe(&cwd, &home, runtime, &[ClaudeSettingSource::Local]).unwrap(); + assert_eq!(changed.local_decision, LocalDecision::NoRoot); + assert_eq!(changed.local_settings.len(), 1); + let error = accepted.require_same_selection(&changed).expect_err( + "a final namespace selecting different native settings must block before dispatch", + ); + assert!(error.contains("retain inputs and reprepare")); + assert!(!error.contains("canonical-private-synthetic")); + assert_eq!(std::fs::read(prior).unwrap(), bytes); + } + } + #[test] fn claude_native_observation_consumes_cwd_then_qualified_canonical_local_settings() { for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { From c6fccd02f49265884d02fe8474d68d465150c68c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:59:15 -0700 Subject: [PATCH 112/493] Verify accepted Claude source plans before native dispatch --- crates/freshell-platform/src/claude_inputs.rs | 37 +++ .../src/claude_inputs/observation.rs | 35 ++- .../src/managed_mcp_capability.rs | 5 + .../managed_mcp_capability/claude_stage.rs | 8 + .../src/managed_provider_references/claude.rs | 32 ++- .../src/claude_source_observation.rs | 130 +++++++++++ crates/freshell-session-host/src/main.rs | 28 +++ .../src/provider_input_preparation.rs | 216 ++++++++++++++++++ 8 files changed, 483 insertions(+), 8 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs.rs b/crates/freshell-platform/src/claude_inputs.rs index d3e29d0fd..d21be082c 100644 --- a/crates/freshell-platform/src/claude_inputs.rs +++ b/crates/freshell-platform/src/claude_inputs.rs @@ -99,6 +99,16 @@ pub struct ClaudeInputManifest { #[serde(default = "default_setting_sources")] pub setting_sources: Vec, pub plugin_dirs: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub accepted_native_plan: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct AcceptedNativeSourcePlan { + pub selector_manifest_sha256: String, + pub plan_revision: u8, + pub observation: observation::NativeObservation, } #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -151,6 +161,14 @@ pub fn default_setting_sources() -> Vec { ] } impl ClaudeInputManifest { + /// Selection precedes file-origin enrichment and the accepted native plan. + pub fn original_selectors(&self) -> Self { + let mut selected = self.clone(); + selected.settings_origin = None; + selected.accepted_native_plan = None; + selected + } + pub fn validate(&self) -> Result<(), &'static str> { if self.native_source_runtime.is_none() { return Err("Claude native source runtime is unrecorded; retain inputs and reprepare before dispatch"); @@ -172,6 +190,25 @@ impl ClaudeInputManifest { { return Err("accepted Claude discovery context profile exceeded"); } + if let Some(plan) = &self.accepted_native_plan { + if plan.plan_revision > 2 + || plan.selector_manifest_sha256.len() != 64 + || !plan + .selector_manifest_sha256 + .bytes() + .all(|byte| byte.is_ascii_hexdigit()) + || Some(plan.observation.native_source_runtime) != self.native_source_runtime + || plan.observation.setting_sources != self.setting_sources + || plan.observation.home.is_none() + || plan.observation.uid != plan.observation.euid + || plan.observation.gid != plan.observation.egid + || plan.observation.validate().is_err() + { + return Err( + "accepted Claude native source plan is incomplete; retain inputs and reprepare", + ); + } + } for path in [&self.original_home, &self.cwd, &self.workspace] .into_iter() .chain(self.config_root.iter()) diff --git a/crates/freshell-platform/src/claude_inputs/observation.rs b/crates/freshell-platform/src/claude_inputs/observation.rs index 174c79a1b..329214bec 100644 --- a/crates/freshell-platform/src/claude_inputs/observation.rs +++ b/crates/freshell-platform/src/claude_inputs/observation.rs @@ -214,7 +214,16 @@ impl NativeObservation { pub fn require_same_selection(&self, current: &Self) -> Result<(), String> { self.validate()?; current.validate()?; - if self != current { + if self.native_source_runtime != current.native_source_runtime + || self.setting_sources != current.setting_sources + || (self.uid, self.euid, self.gid, self.egid) + != (current.uid, current.euid, current.gid, current.egid) + || self.cwd != current.cwd + || self.home != current.home + || self.roots != current.roots + || self.local_decision != current.local_decision + || self.local_settings != current.local_settings + { return Err("Claude native source selection changed before dispatch; retain inputs and reprepare".into()); } Ok(()) @@ -496,6 +505,30 @@ mod tests { } } + #[test] + fn claude_native_final_selection_accepts_provider_owned_directory_without_changing_local_predicate( + ) { + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let root = owned.path().join("root"); + let cwd = root.join("nested"); + for directory in [&home, &root.join(".git"), &cwd] { + std::fs::create_dir_all(directory).unwrap(); + } + let accepted = observe(&cwd, &home, runtime, &[ClaudeSettingSource::Local]).unwrap(); + assert_eq!(accepted.local_decision, LocalDecision::QualifiedCanonical); + assert!(accepted.claude_uid.is_none()); + std::fs::create_dir(root.join(".claude")).unwrap(); + let current = observe(&cwd, &home, runtime, &[ClaudeSettingSource::Local]).unwrap(); + assert_eq!(current.claude_uid, Some(unsafe { libc::getuid() })); + assert_eq!(current.local_decision, accepted.local_decision); + assert_eq!(current.local_settings, accepted.local_settings); + accepted.require_same_selection(¤t) + .expect("an owned empty directory leaves the native absent-or-owned predicate and ordered candidates unchanged"); + } + } + #[test] fn claude_native_observation_consumes_cwd_then_qualified_canonical_local_settings() { for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 5eb9b8031..e504893f7 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -3381,6 +3381,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::write(selected.join("commands/probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -3471,6 +3472,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::os::unix::fs::symlink(&selected, &alias).unwrap(); let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -3620,6 +3622,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ }; let mut manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -4856,6 +4859,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ } let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some(NativeSourceRuntime::Sdk), original_home: home.clone(), cwd: cwd.clone(), @@ -6055,6 +6059,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ }; let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index cccdec0e5..140d2d0dc 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -61,6 +61,7 @@ fn prepare_selected_sources( ) -> Result { let mut manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -487,6 +488,7 @@ mod tests { std::fs::write(&unselected, poison).unwrap(); let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some(runtime), original_home: home.clone(), cwd: cwd.clone(), @@ -568,6 +570,7 @@ mod tests { .unwrap(); let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -689,6 +692,7 @@ mod tests { } let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -817,6 +821,7 @@ mod tests { } let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some(runtime), original_home: home.clone(), cwd: cwd.clone(), @@ -899,6 +904,7 @@ mod tests { std::fs::write(cwd.join(".git"), "ordinary native checkout marker\n").unwrap(); let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some(freshell_platform::claude_inputs::NativeSourceRuntime::Sdk), original_home: home.clone(), cwd: cwd.clone(), @@ -1037,6 +1043,7 @@ mod tests { .unwrap(); let mut manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some(freshell_platform::claude_inputs::NativeSourceRuntime::Sdk), original_home: home.clone(), cwd: cwd.clone(), @@ -1184,6 +1191,7 @@ mod tests { std::fs::write(project.join("commands/probe.mjs"), "import {readFileSync} from 'node:fs';process.stdout.write(readFileSync(new URL('./resource',import.meta.url)))").unwrap(); let mut manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 4c1dd58f6..4a202297b 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -60,8 +60,9 @@ pub(crate) async fn prepare_managed( } plugin_dirs.push(path); } - let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { + let mut manifest = freshell_platform::claude_inputs::ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some(request.native_source_runtime), original_home: request.home.into(), cwd: request.cwd.into(), @@ -121,11 +122,8 @@ pub(crate) async fn prepare_managed( setting_sources: manifest.setting_sources.clone(), selector_manifest_sha256: { use sha2::{Digest, Sha256}; - let bytes = bounded_json_bytes( - &manifest, - freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, - ) - .map_err(|_| "Claude selector metadata exceeds its finite profile")?; + let bytes = bounded_json_bytes(&manifest, 64 * 1024 * 1024) + .map_err(|_| "Claude selector metadata exceeds its finite profile")?; format!("{:x}", Sha256::digest(bytes)) }, plan_revision: 0, @@ -167,6 +165,12 @@ pub(crate) async fn prepare_managed( .await .map_err(|_| "Claude native source evidence is unavailable; retain inputs and reprepare")?; observation.validate_for_request(&observation_request, &observation_request_sha256)?; + manifest.accepted_native_plan = + Some(freshell_platform::claude_inputs::AcceptedNativeSourcePlan { + selector_manifest_sha256: observation_request.selector_manifest_sha256, + plan_revision: observation_request.plan_revision, + observation: observation.observation.clone(), + }); crate::managed_mcp_capability::issue_prepared_provider_context( "claude", request.soul, @@ -361,6 +365,20 @@ mod tests { calls: AtomicUsize::new(0), }); let socket = owned.path().join("control.sock"); + let plugins = (0..5000) + .map(|index| { + workspace + .join(format!("selected-plugin-{index:04}-{}", "x".repeat(40))) + .to_str() + .unwrap() + .to_owned() + }) + .collect::>(); + assert!( + plugins.iter().map(String::len).sum::() + > freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES + ); + assert!(plugins.iter().map(String::len).sum::() < MAX_SOURCE_EVENT_BYTES); let supervisor = Supervisor::new( registry.clone(), backend.clone(), @@ -403,7 +421,7 @@ mod tests { git_common_dir: None, original: &|_| None, args: &["--setting-sources".into(), "local".into()], - plugins: &[], + plugins: &plugins, references: &[], }) .await diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index 585f21f78..ce6833d54 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -6,6 +6,136 @@ use freshell_platform::claude_inputs::observation::{ use sha2::{Digest, Sha256}; use std::io::{Read, Write}; +pub(crate) async fn verify_final( + context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, + stage: &std::path::Path, + cwd: &std::path::Path, + home: &std::path::Path, + uid: u32, + gid: u32, + runtime: freshell_platform::claude_inputs::NativeSourceRuntime, +) -> Result<(), String> { + use freshell_runtime_protocol::{ + bounded_json_bytes, read_frame_with_limit, write_frame_with_limit, + }; + use tokio::io::AsyncReadExt; + let Some((request, expected)) = crate::provider_input_preparation::final_claude_source_request( + context, stage, cwd, home, runtime, + )? + else { + return Ok(()); + }; + if (expected.uid, expected.euid, expected.gid, expected.egid) != (uid, uid, gid, gid) { + return Err("Claude actual provider identity differs from accepted inputs; retain inputs and reprepare".into()); + } + let raw = bounded_json_bytes(&request, MAX_REQUEST_BYTES) + .map_err(|_| "Claude final source request exceeds its finite metadata profile")?; + let request_sha256 = format!("{:x}", Sha256::digest(raw)); + let executable = std::env::current_exe().map_err(|_| { + "Claude final source reader artifact unavailable; retain inputs and reprepare" + })?; + let mut command = tokio::process::Command::new("/usr/bin/setpriv"); + command + .args(crate::provider_identity_args(uid, gid)) + .arg(executable) + .arg("observe-claude-sources") + .current_dir(cwd) + .env_clear() + .env("HOME", home) + .env("PATH", "/usr/bin:/bin") + .stdin(std::process::Stdio::piped()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::null()) + .kill_on_drop(true); + let started = std::time::Instant::now(); + let mut child = command + .spawn() + .map_err(|_| "Claude final source reader could not start; retain inputs and reprepare")?; + let reader_pid = child.id(); + tracing::info!( + event = "provider.inputs.native_final.started", + provider = "claude", + stage = "session_host", + reader_pid, + uid, + gid, + "Observing final native provider source selection" + ); + let exchange = async { + let mut input = child + .stdin + .take() + .ok_or("Claude final source pipe unavailable")?; + let mut output = child + .stdout + .take() + .ok_or("Claude final source pipe unavailable")?; + write_frame_with_limit(&mut input, &request, MAX_REQUEST_BYTES) + .await + .map_err(|_| "Claude final source request incomplete")?; + drop(input); + let reply: ObservationReply = read_frame_with_limit(&mut output, MAX_REPLY_BYTES) + .await + .map_err(|_| { + "Claude final source evidence unavailable or exceeds its finite profile" + })?; + let mut trailing = [0]; + if output + .read(&mut trailing) + .await + .map_err(|_| "Claude final reader completion unavailable")? + != 0 + { + return Err("Claude final reader emitted extra evidence"); + } + Ok(reply) + }; + let exchange = tokio::time::timeout(std::time::Duration::from_secs(20), exchange).await; + let cleanup = async { + if !matches!(&exchange, Ok(Ok(_))) { + child + .kill() + .await + .map_err(|_| "Claude final source reader termination failed")?; + } + child + .wait() + .await + .map_err(|_| "Claude final source reader join failed") + }; + let status = tokio::time::timeout(std::time::Duration::from_secs(5), cleanup).await; + let result = (|| { + let status = status.map_err(|_| "Claude final source reader cleanup timed out")??; + let reply = exchange.map_err(|_| "Claude final source reader timed out")??; + if !status.success() { + return Err("Claude final source reader failed without complete evidence".to_string()); + } + reply.validate_for_request(&request, &request_sha256)?; + expected.require_same_selection(&reply.observation) + })(); + match &result { + Ok(()) => tracing::info!( + event = "provider.inputs.native_final.completed", + provider = "claude", + stage = "session_host", + reader_pid, + elapsed_ms = started.elapsed().as_millis() as u64, + "Final native provider source selection agrees" + ), + Err(reason) => { + tracing::warn!(event="provider.inputs.native_final.blocked",provider="claude",stage="session_host", + reader_pid,elapsed_ms=started.elapsed().as_millis() as u64,cause=%reason,"Final native provider source selection unavailable or changed") + } + } + result.map_err(|reason| { + if reason.contains("retain inputs and reprepare") { + reason + } else { + format!("{reason}; retain inputs and reprepare") + } + }) +} + pub(crate) fn serve_stdio() -> Result<(), String> { exchange(&mut std::io::stdin().lock(), &mut std::io::stdout().lock()) } diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 165d1a8ce..caab6c3fb 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1062,6 +1062,20 @@ async fn grant_execution( format!("prepare managed provider: {error}"), ) })?; + claude_source_observation::verify_final( + prepared.terminal.provider_launch_context.as_ref(), + &state + .provider_config_paths + .private_mount + .join("user-provider"), + Path::new(&prepared.terminal.cwd), + &state.provider_config_paths.provider_home, + prepared.terminal.run_as_uid, + prepared.terminal.run_as_gid, + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ) + .await + .map_err(|error| RuntimeError::new(RuntimeErrorCode::ProviderInputsChanged, error))?; consume_grant(&mut persisted)?; let prepared = providers::start_terminal_sidecar(prepared) .await @@ -1189,6 +1203,20 @@ async fn grant_execution( ) .await .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; + claude_source_observation::verify_final( + launch.provider_launch_context.as_ref(), + &state + .provider_config_paths + .private_mount + .join("user-provider"), + Path::new(&launch.cwd), + &state.provider_config_paths.provider_home, + launch.run_as_uid, + launch.run_as_gid, + freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, + ) + .await + .map_err(|error| RuntimeError::new(RuntimeErrorCode::ProviderInputsChanged, error))?; consume_grant(&mut persisted)?; freshell_platform::managed_child_secrets::install(resolved_secrets.environment) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 8f748d0b2..e79e307fd 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -31,6 +31,69 @@ fn read_private(root: &Path, relative: &str, limit: usize) -> Result, St } Ok(bytes) } + +pub(crate) fn final_claude_source_request( + context: Option<&ProviderLaunchContext>, + stage: &Path, + cwd: &Path, + home: &Path, + runtime: freshell_platform::claude_inputs::NativeSourceRuntime, +) -> Result< + Option<( + freshell_platform::claude_inputs::observation::ObservationRequest, + freshell_platform::claude_inputs::observation::NativeObservation, + )>, + String, +> { + use freshell_platform::claude_inputs::{observation::ObservationRequest, ClaudeInputManifest}; + use sha2::{Digest, Sha256}; + let Some(context) = context.filter(|context| context.preparation.provider() == "claude") else { + return Ok(None); + }; + let Some(inputs) = context.prepared_inputs.as_ref() else { + return Ok(None); + }; + context.validate("claude").map_err(|error| error.message)?; + let bytes = read_private(stage, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + if format!("{:x}", Sha256::digest(&bytes)) != inputs.manifest.sha256 { + return Err("Claude prepared generation changed; retain inputs and reprepare".into()); + } + let manifest: ClaudeInputManifest = serde_json::from_slice(&bytes) + .map_err(|_| "Claude prepared generation is invalid; retain inputs and reprepare")?; + manifest.validate().map_err(str::to_string)?; + let plan = manifest + .accepted_native_plan + .as_ref() + .ok_or("Claude native source evidence is unrecorded; retain inputs and reprepare")?; + let selectors = freshell_runtime_protocol::bounded_json_bytes( + &manifest.original_selectors(), + 64 * 1024 * 1024, + ) + .map_err(|_| "Claude original selectors exceed the private profile")?; + if plan.selector_manifest_sha256 != format!("{:x}", Sha256::digest(selectors)) + || plan.observation.native_source_runtime != runtime + || plan.observation.cwd != cwd + || plan.observation.home.as_deref() != Some(home) + { + return Err( + "Claude original selection or native launch view changed; retain inputs and reprepare" + .into(), + ); + } + let request = ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: runtime, + cwd: cwd.into(), + home: home.into(), + setting_sources: manifest.setting_sources, + selector_manifest_sha256: plan.selector_manifest_sha256.clone(), + plan_revision: plan.plan_revision, + }; + request.validate()?; + Ok(Some((request, plan.observation.clone()))) +} + pub fn prepare( context: Option<&ProviderLaunchContext>, secrets: &mut ResolvedProviderSecrets, @@ -334,6 +397,157 @@ mod tests { use freshell_platform::opencode_recipe::*; use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + + #[tokio::test] + async fn provider_secret_claude_final_source_gate_blocks_unrecorded_generation_and_keeps_private_inputs( + ) { + let owned = tempfile::tempdir().unwrap(); + std::fs::create_dir(owned.path().join(".freshell")).unwrap(); + let manifest = serde_json::json!({ + "version":1,"nativeSourceRuntime":"terminal", + "originalHome":owned.path(),"cwd":owned.path(),"workspace":owned.path(), + "settingSources":["local"],"pluginDirs":[], + }); + let bytes = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(owned.path().join(".freshell/input-manifest.json"), &bytes).unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let context: ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"claude":{"mcp_args":[]}}, + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000", + "providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{ + "relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&bytes)), + "sourceCount":0,"aliasCount":0,"recipeCount":0}} + })).unwrap(); + let error = final_claude_source_request(Some(&context), owned.path(), owned.path(), owned.path(), + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal) + .expect_err("a prepared Claude generation without actual source evidence must block before dispatch"); + assert!(error.contains("retain inputs and reprepare"), "{error}"); + let error = crate::claude_source_observation::verify_final( + Some(&context), + owned.path(), + owned.path(), + owned.path(), + unsafe { libc::getuid() }, + unsafe { libc::getgid() }, + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + ) + .await + .expect_err("the actual final host gate cannot bypass unrecorded native source evidence"); + assert!(error.contains("retain inputs and reprepare"), "{error}"); + assert_eq!( + std::fs::read(owned.path().join(".freshell/input-manifest.json")).unwrap(), + bytes + ); + assert!(final_claude_source_request( + None, + owned.path(), + owned.path(), + owned.path(), + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal + ) + .unwrap() + .is_none()); + } + #[test] + fn provider_secret_claude_final_source_request_keeps_enriched_origin_and_frozen_original_selectors( + ) { + use freshell_platform::claude_inputs::{ + observation::observe, AcceptedNativeSourcePlan, ClaudeInputManifest, + ClaudeSettingSource, NativeSourceRuntime, + }; + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = tempfile::tempdir().unwrap(); + let original_home = owned.path().join("original-home"); + let home = owned.path().join("provider-home"); + let workspace = owned.path().join("workspace"); + let cwd = workspace.join("nested"); + let stage = owned.path().join("stage"); + for directory in [&original_home, &home, &cwd, &stage.join(".freshell")] { + std::fs::create_dir_all(directory).unwrap(); + } + let selected = original_home.join("selected.json"); + let selected_bytes = br#"{"env":{"SYNTHETIC_PRIVATE":"frozen-original-selector"}}"#; + std::fs::write(&selected, selected_bytes).unwrap(); + let mut manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(runtime), + original_home: original_home.clone(), + cwd: cwd.clone(), + workspace: workspace.clone(), + config_root: None, + settings_file: Some(selected.clone()), + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Local], + plugin_dirs: vec![], + accepted_native_plan: None, + }; + let original = freshell_runtime_protocol::bounded_json_bytes( + &manifest.original_selectors(), + 64 * 1024 * 1024, + ) + .unwrap(); + let observed = observe(&cwd, &home, runtime, &manifest.setting_sources).unwrap(); + assert_eq!(observed.uid, unsafe { libc::getuid() }); + manifest.settings_origin = Some(std::fs::canonicalize(&selected).unwrap()); + manifest.accepted_native_plan = Some(AcceptedNativeSourcePlan { + selector_manifest_sha256: format!("{:x}", Sha256::digest(original)), + plan_revision: 0, + observation: observed.clone(), + }); + let bytes = + freshell_runtime_protocol::bounded_json_bytes(&manifest, 64 * 1024 * 1024).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let mut context: ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"claude":{"mcp_args":[]}}, + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000", + "providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{ + "relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&bytes)), + "sourceCount":0,"aliasCount":0,"recipeCount":0}} + })).unwrap(); + std::fs::rename(&original_home, owned.path().join("preserved-original-home")).unwrap(); + let (request, expected) = + final_claude_source_request(Some(&context), &stage, &cwd, &home, runtime) + .unwrap() + .expect("accepted selection does not rescan unavailable original sources"); + let actual = observe( + &request.cwd, + &request.home, + request.native_source_runtime, + &request.setting_sources, + ) + .unwrap(); + expected.require_same_selection(&actual).unwrap(); + assert_eq!(expected, observed); + assert!(final_claude_source_request( + Some(&context), + &stage, + &workspace, + &home, + runtime + ) + .is_err()); + // A new descriptor SHA cannot authorize changing frozen original selectors. + manifest.cwd = workspace.clone(); + let changed = + freshell_runtime_protocol::bounded_json_bytes(&manifest, 64 * 1024 * 1024).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &changed).unwrap(); + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(changed)); + let error = final_claude_source_request(Some(&context), &stage, &cwd, &home, runtime) + .unwrap_err(); + assert!(error.contains("retain inputs and reprepare")); + assert!(!error.contains("frozen-original-selector")); + assert_eq!( + std::fs::read(owned.path().join("preserved-original-home/selected.json")).unwrap(), + selected_bytes + ); + } + } + #[test] fn provider_secret_claude_inline_settings_consumes_accepted_private_source() { use freshell_platform::claude_inputs::{ClaudeInputManifest, INLINE_SETTINGS_SOURCE}; @@ -343,6 +557,7 @@ mod tests { std::fs::write(owned.path().join(INLINE_SETTINGS_SOURCE), raw).unwrap(); let manifest = ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), @@ -441,6 +656,7 @@ mod tests { std::fs::create_dir(owned.path().join(".freshell")).unwrap(); let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { version: 1, + accepted_native_plan: None, native_source_runtime: Some( freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ), From 836a8e00a1911d1efb810a6c573bb6c38eb1459a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 23:25:46 -0700 Subject: [PATCH 113/493] Qualify final Claude source mismatch before grant consumption --- .../src/managed_provider_launch_tests.rs | 136 +++++++++++++++++- .../managed-provider-launch-sandbox.py | 12 +- 2 files changed, 142 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index a72505997..954e51801 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -184,7 +184,27 @@ impl RuntimeBackend for BrokerAuthorizedBackend { spec: &CreateRuntimeSpec, ) -> Result { self.authorize(spec).await?; - self.real.create_stopped(spec).await + let created = self.real.create_stopped(spec).await?; + if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD").as_deref() + == Ok("changed-git") + { + let (route, cwd) = if let Some(terminal) = &spec.terminal { + ("terminal", terminal.cwd.as_str()) + } else { + ("fresh", spec.fresh_agent.as_ref().unwrap().cwd.as_str()) + }; + // Native discovery recognizes the closest filesystem Git marker. + // Introduce a closer root only after accepted-source observation + // and mount validation. All original files and declared workspace + // and common-directory mount paths remain untouched. + std::fs::create_dir(Path::new(cwd).join(".git")).unwrap(); + std::fs::write( + self.handoff.join(format!("source-guard-{route}.state")), + spec.runtime_dir.to_str().unwrap(), + ) + .unwrap(); + } + Ok(created) } async fn start_host(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { self.real.start_host(handle).await @@ -250,6 +270,7 @@ fn capture_fresh_launch_failure_classes() -> std::sync::Arcraw+=b);process.stdin.on('end',async()=>{ std::os::unix::fs::symlink(&flag_settings, &flag_alias).unwrap(); let settings_form = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SETTINGS_FORM") .unwrap_or_else(|_| "file".into()); + let source_guard = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD") + .unwrap_or_else(|_| "unchanged".into()); + assert!(matches!(source_guard.as_str(), "unchanged" | "changed-git")); + let changed_git = source_guard == "changed-git"; let flag_argument = match settings_form.as_str() { + "file" if changed_git => flag_alias.to_str().unwrap().into(), "file" => "../source-home/selected-config/selected-settings.json".into(), "inline" => String::from_utf8(flag_bytes.clone()).unwrap(), "owned-last" => freshell_platform::cli_launch::claude_settings_json( @@ -1134,7 +1160,14 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ) .await; - let cwd = workspace.to_str().unwrap().to_string(); + let cwd = if changed_git { + workspace.join("source-observer/nested") + } else { + workspace.clone() + } + .to_str() + .unwrap() + .to_string(); let ambient_home = fixture.join("ambient-home"); std::fs::create_dir_all(ambient_home.join(".claude")).unwrap(); std::fs::write( @@ -1150,7 +1183,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ args: vec![ "--init-only".into(), "--setting-sources".into(), - "user".into(), + if changed_git { "user,local" } else { "user" }.into(), // Native chooses the last selector. Neither an unavailable // earlier file nor its private inline document is selected. "--settings".into(), @@ -1168,7 +1201,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ }, env: BTreeMap::from([ ("HOME".into(), home.to_str().unwrap().into()), - ("CLAUDE_CONFIG_DIR".into(), "../source-home/selected-config".into()), + ("CLAUDE_CONFIG_DIR".into(), if changed_git { selected_config.to_str().unwrap().into() } else { "../source-home/selected-config".into() }), ]), terminal_id: "closed-native-terminal".into(), stream_id: "closed-native-stream".into(), @@ -1182,8 +1215,101 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ view_pane_id: None, create_request_id: Some("closed-native-terminal-create".into()), }) + .await; + if changed_git { + let error = terminal.expect_err("actual final source mismatch must block terminal startup"); + println!( + "provider_source_guard_diagnostic {}", + json!({"selectionChanged":error.contains("Claude native source selection changed"),"commonDir":error.contains("git common dir"),"hostUnreachable":error.contains("host") }) + ); + assert!( + error.contains("Claude native source selection changed"), + "final native source mismatch must be the cause, not a fixture or preparation failure" + ); + fn unconsumed_host(handoff: &Path, route: &str, receipt: &Path) { + let runtime = + std::fs::read_to_string(handoff.join(format!("source-guard-{route}.state"))) + .unwrap(); + let state: Value = serde_json::from_slice( + &std::fs::read(Path::new(&runtime).join("host-state.json")).unwrap(), + ) + .unwrap(); + assert!(state["grantId"].is_null()); + assert!(state["workerPid"].is_null()); + assert_eq!(state["workerLaunchCount"], json!(0)); + assert!(!receipt.exists(), "no native startup consumer may execute"); + println!( + "provider_source_guard {}", + json!({"provider":"claude","route":route,"grantUnconsumed":true,"workerLaunchCount":0,"nativeNotStarted":true}) + ); + } + let handoff = fixture.join("handoff"); + unconsumed_host(&handoff, "terminal", &receipt); + std::fs::remove_dir(Path::new(&cwd).join(".git")).unwrap(); + // These owned terminal-only decoys are outside the fresh route's + // intended selection; remove them exactly as in the positive case. + std::fs::remove_file(&disabled_project_memory).unwrap(); + std::fs::remove_file(&disabled_project_settings).unwrap(); + let (broadcast, _) = tokio::sync::broadcast::channel(256); + let mut events = broadcast.subscribe(); + std::env::set_var("HOME", &home); + let proxy = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( + Some(client.clone()), + Arc::new(broadcast), + ) + .unwrap() + .unwrap(); + let flavor = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR").unwrap(); + let source_provider = if flavor == "kilroy" { + "kilroy" + } else { + "claude" + }; + qualify_private_source_reader( + &client, + SoulId::parse(format!( + "soul-fresh-{source_provider}-{}", + &format!("{:x}", Sha256::digest(b"closed-native-fresh-create"))[..32] + )) + .unwrap(), + source_provider, + "closed-native-fresh-create", + &workspace, + freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, + ) + .await; + Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":flavor,"provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; + tokio::time::timeout(Duration::from_secs(45), async { + loop { + let frame: Value = serde_json::from_str(&events.recv().await.unwrap()).unwrap(); + assert_ne!(frame["type"], json!("freshAgent.created")); + if frame["type"] == "freshAgent.create.failed" { + break; + } + } + }) .await - .expect("complete native Claude terminal startup"); + .unwrap(); + assert!( + failure_classes + .lock() + .unwrap() + .contains(&"Claude native source selection changed"), + "fresh failure must come from the actual final native source selection check" + ); + unconsumed_host(&handoff, "fresh", &receipt); + std::fs::remove_dir(Path::new(&cwd).join(".git")).unwrap(); + assert_eq!(std::fs::read(&flag_settings).unwrap(), flag_bytes); + assert_eq!(std::fs::read_link(&flag_alias).unwrap(), flag_settings); + assert!(!flag_receipt.exists()); + assert!(!instructions_receipt.exists()); + web_task.abort(); + control_task.abort(); + assert!(web_task.await.unwrap_err().is_cancelled()); + assert!(control_task.await.unwrap_err().is_cancelled()); + return; + } + let terminal = terminal.expect("complete native Claude terminal startup"); async fn consumed(path: &Path) -> Option { tokio::time::timeout(Duration::from_secs(45), async { loop { diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index fd0d69cd0..3c25877b4 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -35,11 +35,14 @@ def main(): parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") parser.add_argument("--claude-flavor",choices=["freshclaude","kilroy"],default="freshclaude") parser.add_argument("--claude-settings-form",choices=["file","inline","owned-last"],default="file") + parser.add_argument("--claude-source-guard",choices=["unchanged","changed-git"],default="unchanged") args=parser.parse_args() if args.provider!="claude" and args.claude_flavor!="freshclaude": parser.error("the finite Claude flavor requires --provider claude") if args.provider!="claude" and args.claude_settings_form!="file": parser.error("the finite Claude settings form requires --provider claude") + if args.provider!="claude" and args.claude_source_guard!="unchanged": + parser.error("the finite Claude source guard requires --provider claude") selected_test={"opencode":TEST,"codex":CODEX_TEST,"claude":CLAUDE_TEST,"amplifier":AMPLIFIER_TEST}[args.provider] if args.provider=="claude": IMAGE=CLAUDE_IMAGE repo=Path(__file__).resolve().parents[2] @@ -174,6 +177,7 @@ def create(name,*arguments): "--env",f"FRESHELL_PROVIDER_LAUNCH_INSTALLATION={installation}","--env",f"FRESHELL_PROVIDER_LAUNCH_RUN_ID={run}", "--env",f"FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR={args.claude_flavor}", "--env",f"FRESHELL_PROVIDER_LAUNCH_CLAUDE_SETTINGS_FORM={args.claude_settings_form}", + "--env",f"FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD={args.claude_source_guard}", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_ENV_FILE={root}/grants/environment.env", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE={root}/grants/auth.json", "--env",f"FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE={root}/grants/ca.pem", @@ -191,11 +195,17 @@ def create(name,*arguments): if args.provider=="claude": receipt["claudeFlavor"]=args.claude_flavor receipt["claudeSettingsForm"]=args.claude_settings_form + receipt["claudeSourceGuard"]=args.claude_source_guard receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} - if exit_code==0 and (len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes or "test result: ok. 1 passed;" not in outcome.stdout): + route_mismatch=len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes + if args.claude_source_guard=="changed-git": + receipt["blockedRoutes"]=[json.loads(line.split("provider_source_guard ",1)[1]) for line in outcome.stdout.splitlines() if "provider_source_guard " in line] + blocked={(item.get("provider"),item.get("route")) for item in receipt["blockedRoutes"]} + route_mismatch=bool(receipt["consumedRoutes"]) or len(receipt["blockedRoutes"])!=2 or blocked!={("claude","terminal"),("claude","fresh")} or any(item.get("grantUnconsumed") is not True or item.get("workerLaunchCount")!=0 or item.get("nativeNotStarted") is not True for item in receipt["blockedRoutes"]) + if exit_code==0 and (route_mismatch or "test result: ok. 1 passed;" not in outcome.stdout): receipt["exitCode"]=2;receipt["failureClass"]="native_startup_count_mismatch" (root/"broker-complete").write_text("ok") deadline=time.monotonic()+10 From e9b56566188d4da1c9ad6b2d92f581e0a1a35c87 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 7 Oct 2026 23:42:04 -0700 Subject: [PATCH 114/493] Join final Claude source readers within the finite exchange --- .../src/claude_source_observation.rs | 269 +++++++++++++++--- 1 file changed, 224 insertions(+), 45 deletions(-) diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index ce6833d54..877a1f767 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -15,10 +15,7 @@ pub(crate) async fn verify_final( gid: u32, runtime: freshell_platform::claude_inputs::NativeSourceRuntime, ) -> Result<(), String> { - use freshell_runtime_protocol::{ - bounded_json_bytes, read_frame_with_limit, write_frame_with_limit, - }; - use tokio::io::AsyncReadExt; + use freshell_runtime_protocol::bounded_json_bytes; let Some((request, expected)) = crate::provider_input_preparation::final_claude_source_request( context, stage, cwd, home, runtime, )? @@ -61,6 +58,47 @@ pub(crate) async fn verify_final( gid, "Observing final native provider source selection" ); + let result = read_final_child( + &mut child, + &request, + &request_sha256, + std::time::Duration::from_secs(20), + std::time::Duration::from_secs(5), + ) + .await + .and_then(|reply| expected.require_same_selection(&reply.observation)); + match &result { + Ok(()) => tracing::info!( + event = "provider.inputs.native_final.completed", + provider = "claude", + stage = "session_host", + reader_pid, + elapsed_ms = started.elapsed().as_millis() as u64, + "Final native provider source selection agrees" + ), + Err(reason) => { + tracing::warn!(event="provider.inputs.native_final.blocked",provider="claude",stage="session_host", + reader_pid,elapsed_ms=started.elapsed().as_millis() as u64,cause=%reason,"Final native provider source selection unavailable or changed") + } + } + result.map_err(|reason| { + if reason.contains("retain inputs and reprepare") { + reason + } else { + format!("{reason}; retain inputs and reprepare") + } + }) +} + +async fn read_final_child( + child: &mut tokio::process::Child, + request: &ObservationRequest, + request_sha256: &str, + exchange_timeout: std::time::Duration, + cleanup_timeout: std::time::Duration, +) -> Result { + use freshell_runtime_protocol::{read_frame_with_limit, write_frame_with_limit}; + use tokio::io::AsyncReadExt; let exchange = async { let mut input = child .stdin @@ -70,7 +108,7 @@ pub(crate) async fn verify_final( .stdout .take() .ok_or("Claude final source pipe unavailable")?; - write_frame_with_limit(&mut input, &request, MAX_REQUEST_BYTES) + write_frame_with_limit(&mut input, request, MAX_REQUEST_BYTES) .await .map_err(|_| "Claude final source request incomplete")?; drop(input); @@ -88,52 +126,43 @@ pub(crate) async fn verify_final( { return Err("Claude final reader emitted extra evidence"); } - Ok(reply) - }; - let exchange = tokio::time::timeout(std::time::Duration::from_secs(20), exchange).await; - let cleanup = async { - if !matches!(&exchange, Ok(Ok(_))) { - child - .kill() - .await - .map_err(|_| "Claude final source reader termination failed")?; - } - child + // A complete pipe reply is not process completion. Keep its owned + // process join inside the exchange deadline so a reader that closes + // stdout but remains alive enters the same bounded kill/wait cleanup. + let status = child .wait() .await - .map_err(|_| "Claude final source reader join failed") + .map_err(|_| "Claude final source reader completion unavailable")?; + Ok((reply, status)) }; - let status = tokio::time::timeout(std::time::Duration::from_secs(5), cleanup).await; - let result = (|| { - let status = status.map_err(|_| "Claude final source reader cleanup timed out")??; - let reply = exchange.map_err(|_| "Claude final source reader timed out")??; - if !status.success() { - return Err("Claude final source reader failed without complete evidence".to_string()); - } - reply.validate_for_request(&request, &request_sha256)?; - expected.require_same_selection(&reply.observation) - })(); - match &result { - Ok(()) => tracing::info!( - event = "provider.inputs.native_final.completed", - provider = "claude", - stage = "session_host", - reader_pid, - elapsed_ms = started.elapsed().as_millis() as u64, - "Final native provider source selection agrees" - ), + let exchange = tokio::time::timeout(exchange_timeout, exchange).await; + let completed = match exchange { + Ok(result) => result.map_err(str::to_string), + Err(_) => Err("Claude final source reader timed out".to_string()), + }; + let (reply, status) = match completed { + Ok(completed) => completed, Err(reason) => { - tracing::warn!(event="provider.inputs.native_final.blocked",provider="claude",stage="session_host", - reader_pid,elapsed_ms=started.elapsed().as_millis() as u64,cause=%reason,"Final native provider source selection unavailable or changed") + let cleanup = async { + child + .start_kill() + .map_err(|_| "Claude final source reader termination failed")?; + child + .wait() + .await + .map_err(|_| "Claude final source reader join failed") + }; + tokio::time::timeout(cleanup_timeout, cleanup) + .await + .map_err(|_| "Claude final source reader cleanup timed out")??; + return Err(reason); } + }; + if !status.success() { + return Err("Claude final source reader failed without complete evidence".to_string()); } - result.map_err(|reason| { - if reason.contains("retain inputs and reprepare") { - reason - } else { - format!("{reason}; retain inputs and reprepare") - } - }) + reply.validate_for_request(request, request_sha256)?; + Ok(reply) } pub(crate) fn serve_stdio() -> Result<(), String> { @@ -219,6 +248,156 @@ mod tests { use super::*; use std::io::{Cursor, Error, ErrorKind}; + struct ReaderCase { + result: Result, + joined_before_test_cleanup: bool, + elapsed: std::time::Duration, + } + + async fn reader_case(mode: &str) -> ReaderCase { + use freshell_platform::claude_inputs::NativeSourceRuntime; + use std::os::unix::fs::PermissionsExt; + let root = tempfile::tempdir().unwrap(); + let cwd = root.path().join("cwd"); + let home = root.path().join("home"); + std::fs::create_dir(&cwd).unwrap(); + std::fs::create_dir(&home).unwrap(); + let request = ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: NativeSourceRuntime::Terminal, + cwd: cwd.clone(), + home: home.clone(), + setting_sources: vec![], + selector_manifest_sha256: "a".repeat(64), + plan_revision: 0, + }; + let raw = + freshell_runtime_protocol::bounded_json_bytes(&request, MAX_REQUEST_BYTES).unwrap(); + let request_sha256 = format!("{:x}", Sha256::digest(raw)); + let mut reply = ObservationReply { + request_sha256: request_sha256.clone(), + observation: observe(&cwd, &home, NativeSourceRuntime::Terminal, &[]).unwrap(), + }; + if mode == "wrong-binding" { + reply.request_sha256 = "b".repeat(64); + } else if mode == "wrong-runtime" { + reply.observation.native_source_runtime = NativeSourceRuntime::Sdk; + } + let mut bytes = serde_json::to_vec(&reply).unwrap(); + if matches!(mode, "near-cap" | "oversized") { + bytes.resize(MAX_REPLY_BYTES, b' '); + } else if mode == "invalid" { + bytes = b"PRIVATE_READER_BODY_SENTINEL".to_vec(); + } + let body = root.path().join("reply.bin"); + std::fs::write(&body, bytes).unwrap(); + std::fs::set_permissions(&body, std::fs::Permissions::from_mode(0o600)).unwrap(); + let script = root.path().join("reader.cjs"); + std::fs::write(&script, r#"const fs=require('node:fs'); +const mode=process.argv[2],body=fs.readFileSync(process.argv[3]); +process.stdin.on('data',()=>{}).on('end',()=>{ + const header=Buffer.alloc(4);header.writeUInt32BE(body.length+(mode==='oversized'||mode==='short-body'?1:0)); + if(mode==='stall'){setInterval(()=>{},10000);return;} + if(mode==='oversized'){process.stdout.write(header);setInterval(()=>{},10000);return;} + if(mode==='short-header'){process.stdout.write(header.subarray(0,2));return;} + const frame=Buffer.concat([header,body]); + if(mode==='eof-stall'){process.stdout.write(frame,()=>{fs.closeSync(1);setInterval(()=>{},10000)});return;} + process.stdout.write(mode==='extra'?Buffer.concat([frame,Buffer.from([1])]):frame); + if(mode==='nonzero')process.exitCode=6; +}); +"#).unwrap(); + let mut child = tokio::process::Command::new("node") + .arg(&script) + .arg(mode) + .arg(&body) + .current_dir(&cwd) + .env_clear() + .env("HOME", &home) + .env("PATH", std::env::var_os("PATH").unwrap()) + .stdin(std::process::Stdio::piped()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::null()) + .kill_on_drop(true) + .spawn() + .unwrap(); + let started = std::time::Instant::now(); + let result = read_final_child( + &mut child, + &request, + &request_sha256, + std::time::Duration::from_secs(2), + std::time::Duration::from_millis(200), + ) + .await; + let elapsed = started.elapsed(); + let joined_before_test_cleanup = child.id().is_none(); + // Join every owned fault child before any assertion can panic. This + // fallback records a failed production join; it cannot make it pass. + if !joined_before_test_cleanup { + tokio::time::timeout(std::time::Duration::from_secs(2), async { + child.kill().await.unwrap(); + child.wait().await.unwrap(); + }) + .await + .expect("test-owned reader cleanup must finish"); + } + ReaderCase { + result, + joined_before_test_cleanup, + elapsed, + } + } + + #[tokio::test] + async fn claude_final_reader_real_pipe_bounds_binding_failures_and_joins() { + for mode in [ + "valid", + "near-cap", + "oversized", + "short-header", + "short-body", + "invalid", + "extra", + "nonzero", + "wrong-binding", + "wrong-runtime", + "stall", + ] { + let case = reader_case(mode).await; + assert!( + case.joined_before_test_cleanup, + "owned reader must already be joined for {mode}" + ); + assert_eq!( + case.result.is_ok(), + matches!(mode, "valid" | "near-cap"), + "actual reader result for {mode}" + ); + assert!( + case.elapsed < std::time::Duration::from_secs(3), + "finite reader exchange for {mode}" + ); + if let Err(error) = case.result { + assert!(!error.contains("PRIVATE_READER_BODY_SENTINEL")); + } + } + } + + #[tokio::test] + async fn claude_final_reader_complete_reply_without_process_exit_terminates_and_joins() { + let case = reader_case("eof-stall").await; + assert!( + case.result.is_err(), + "a reply alone cannot establish process completion" + ); + assert!( + case.joined_before_test_cleanup, + "complete reply plus EOF must not leave the owned reader running after cleanup timeout" + ); + assert!(case.elapsed < std::time::Duration::from_secs(3)); + } + struct HeaderOnly { header: Cursor<[u8; 4]>, body_reads: usize, From b246f4de539b7f3330779bcb5fa20b2746ea4cbe Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 00:25:24 -0700 Subject: [PATCH 115/493] Measure effective native group membership for Claude source agreement --- .../src/claude_inputs/observation.rs | 110 ++++++++++++++++++ .../src/managed_provider_launch_tests.rs | 21 +++- .../claude_source_observation_tests.rs | 1 + .../src/claude_source_observation.rs | 1 + 4 files changed, 127 insertions(+), 6 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs/observation.rs b/crates/freshell-platform/src/claude_inputs/observation.rs index 329214bec..ef66de336 100644 --- a/crates/freshell-platform/src/claude_inputs/observation.rs +++ b/crates/freshell-platform/src/claude_inputs/observation.rs @@ -9,6 +9,7 @@ use std::path::{Path, PathBuf}; pub const MAX_REQUEST_BYTES: usize = 256 * 1024; pub const MAX_REPLY_BYTES: usize = 1024 * 1024; +const MAX_GROUP_IDS: usize = MAX_REPLY_BYTES / std::mem::size_of::(); #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] @@ -94,6 +95,7 @@ pub struct NativeObservation { pub euid: u32, pub gid: u32, pub egid: u32, + pub supplementary_groups: Vec, pub cwd: PathBuf, pub home: Option, pub roots: Option, @@ -153,6 +155,7 @@ pub fn observe( euid: unsafe { libc::geteuid() }, gid: unsafe { libc::getgid() }, egid: unsafe { libc::getegid() }, + supplementary_groups: supplementary_groups()?, local_decision: LocalDecision::Disabled, root_uid: None, git_uid: None, @@ -183,6 +186,40 @@ pub fn observe( Ok(observation) } +#[cfg(unix)] +fn supplementary_groups() -> Result, String> { + let count = unsafe { libc::getgroups(0, std::ptr::null_mut()) }; + let length = usize::try_from(count) + .map_err(|_| "Claude actual native groups unavailable; retain inputs and reprepare")?; + if length > MAX_GROUP_IDS { + return Err( + "Claude actual native groups exceed the metadata profile; retain inputs and reprepare" + .into(), + ); + } + let mut groups = Vec::::new(); + groups.try_reserve_exact(length).map_err(|_| { + "Claude actual native groups allocation unavailable; retain inputs and reprepare" + })?; + groups.resize(length, 0); + let read = unsafe { libc::getgroups(count, groups.as_mut_ptr()) }; + let read = usize::try_from(read).map_err(|_| { + "Claude actual native groups changed or unavailable; retain inputs and reprepare" + })?; + if read > length { + return Err( + "Claude actual native groups changed before observation; retain inputs and reprepare" + .into(), + ); + } + groups.truncate(read); + // Group order and duplicate memberships do not change native access. + // Keep the measured supplementary set separate from primary GID evidence. + groups.sort_unstable(); + groups.dedup(); + Ok(groups) +} + fn validate_sources(sources: &[ClaudeSettingSource]) -> Result<(), String> { if sources.len() > 3 || sources @@ -218,6 +255,7 @@ impl NativeObservation { || self.setting_sources != current.setting_sources || (self.uid, self.euid, self.gid, self.egid) != (current.uid, current.euid, current.gid, current.egid) + || self.effective_groups() != current.effective_groups() || self.cwd != current.cwd || self.home != current.home || self.roots != current.roots @@ -229,10 +267,23 @@ impl NativeObservation { Ok(()) } + fn effective_groups(&self) -> Vec { + let mut groups = self.supplementary_groups.clone(); + if let Err(index) = groups.binary_search(&self.egid) { + groups.insert(index, self.egid); + } + groups + } + pub fn validate(&self) -> Result<(), String> { validate_sources(&self.setting_sources)?; let local_enabled = self.setting_sources.contains(&ClaudeSettingSource::Local); if self.version != 1 + || self.supplementary_groups.len() > MAX_GROUP_IDS + || self + .supplementary_groups + .windows(2) + .any(|pair| pair[0] >= pair[1]) || self.local_settings.len() > 2 || local_enabled != (self.local_decision != LocalDecision::Disabled) || local_enabled != !self.local_settings.is_empty() @@ -390,6 +441,65 @@ fn select_local(observation: &mut NativeObservation) -> LocalDecision { mod tests { use super::*; + #[test] + fn claude_native_observation_changed_supplementary_group_blocks_agreement() { + let owned = tempfile::tempdir().unwrap(); + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let accepted = observe(owned.path(), owned.path(), runtime, &[]).unwrap(); + accepted.require_same_selection(&accepted).unwrap(); + let mut current = accepted.clone(); + let added = (0..u32::MAX) + .find(|group| { + *group != accepted.egid && !accepted.supplementary_groups.contains(group) + }) + .unwrap(); + current.supplementary_groups.push(added); + current.supplementary_groups.sort_unstable(); + assert!(accepted.require_same_selection(¤t).is_err(), "same UID/primary GID cannot establish identical native file access when an additional group changes"); + let mut incomplete = serde_json::to_value(&accepted).unwrap(); + incomplete + .as_object_mut() + .unwrap() + .remove("supplementaryGroups"); + assert!(serde_json::from_value::(incomplete).is_err(), "old observations without measured groups must require reprepare, not invent an empty group set"); + } + } + + #[test] + fn claude_native_observation_rejects_incomplete_or_unbounded_group_evidence() { + let owned = tempfile::tempdir().unwrap(); + let accepted = observe( + owned.path(), + owned.path(), + NativeSourceRuntime::Terminal, + &[], + ) + .unwrap(); + for groups in [vec![1, 1], vec![2, 1], vec![0; MAX_GROUP_IDS + 1]] { + let mut changed = accepted.clone(); + changed.supplementary_groups = groups; + assert!(changed.validate().is_err()); + assert!(accepted.require_same_selection(&changed).is_err()); + } + } + + #[test] + fn claude_native_observation_primary_group_membership_is_equivalent_when_repeated() { + let owned = tempfile::tempdir().unwrap(); + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let mut without = observe(owned.path(), owned.path(), runtime, &[]).unwrap(); + without + .supplementary_groups + .retain(|group| *group != without.egid); + let mut repeated = without.clone(); + repeated.supplementary_groups.push(repeated.egid); + repeated.supplementary_groups.sort_unstable(); + assert_ne!(without.supplementary_groups, repeated.supplementary_groups); + without.require_same_selection(&repeated).expect("retaining the same primary group as a supplementary member must not change native group access"); + repeated.require_same_selection(&without).unwrap(); + } + } + #[test] fn claude_native_observation_reply_cannot_be_reused_for_another_selection() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 954e51801..f9d7ac666 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -894,7 +894,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ const input=JSON.parse(raw),root=path.resolve(__dirname,'..'); const endpoint=process.env.FRESHELL_URL,token=process.env.FRESHELL_TOKEN; const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); - const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,settings:['closed-selected-settings','closed-explicit-settings'].includes(process.env.CLAUDE_SELECTED_NONCE),authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], + const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,nativeIdentity:{uid:process.getuid(),euid:process.geteuid(),gid:process.getgid(),egid:process.getegid(),groups:[...new Set([...process.getgroups(),process.getegid()])].sort((a,b)=>a-b)},settings:['closed-selected-settings','closed-explicit-settings'].includes(process.env.CLAUDE_SELECTED_NONCE),authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], memory:hash(fs.readFileSync(path.join(process.env.HOME,'.claude/CLAUDE.md'))),skill:hash(fs.readFileSync(path.join(root,'skills/probe/SKILL.md'))),resource:hash(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:hash(require('probe-dependency'))}; fs.writeFileSync(process.argv[2],JSON.stringify(record)); }); @@ -1082,7 +1082,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ seed: &str, workspace: &Path, runtime: freshell_platform::claude_inputs::NativeSourceRuntime, - ) { + ) -> Value { use freshell_platform::claude_inputs::{observation::LocalDecision, ClaudeSettingSource}; use std::os::unix::fs::MetadataExt; let cwd = workspace.join("source-observer/nested"); @@ -1143,11 +1143,16 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ assert_eq!(std::fs::read(&local).unwrap(), b"{}"); println!( "provider_source_observer {}", - json!({"runtime":runtime,"uid":observed.uid,"gid":observed.gid,"decision":observed.local_decision,"localCount":observed.local_settings.len(),"sourceBytesPreserved":true}) + json!({"runtime":runtime,"uid":observed.uid,"gid":observed.gid,"supplementaryGroups":observed.supplementary_groups,"decision":observed.local_decision,"localCount":observed.local_settings.len(),"sourceBytesPreserved":true}) ); + let mut groups = observed.supplementary_groups.clone(); + groups.push(observed.egid); + groups.sort_unstable(); + groups.dedup(); + json!({"uid":observed.uid,"euid":observed.euid,"gid":observed.gid,"egid":observed.egid,"groups":groups}) } let client = controller.runtime_client(); - qualify_private_source_reader( + let terminal_identity = qualify_private_source_reader( &client, SoulId::parse(format!( "soul-terminal-{}", @@ -1325,7 +1330,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .ok() } let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}); + let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":terminal_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}); let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); let mut expected_imports = serde_json::Map::new(); expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); @@ -1557,7 +1562,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ } else { "claude" }; - qualify_private_source_reader( + let fresh_identity = qualify_private_source_reader( &client, SoulId::parse(format!( "soul-fresh-{source_provider}-{}", @@ -1570,6 +1575,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, ) .await; + assert_eq!( + fresh_identity, terminal_identity, + "both actual source readers must match the real native consumer identity" + ); Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":flavor,"provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; tokio::time::timeout(Duration::from_secs(45), async { loop { diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index b75ce9fcf..da4ca1e3d 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -135,6 +135,7 @@ async fn qualify_transport(mode: &'static str) { observation: NativeObservation { version:1,native_source_runtime:request.native_source_runtime, setting_sources:request.setting_sources,uid:65534,euid:65534,gid:0,egid:0, + supplementary_groups:vec![], local_settings:vec![request.cwd.join(".claude/settings.local.json")], cwd:request.cwd,home:Some(request.home),roots:None, local_decision:LocalDecision::NoRoot,root_uid:None,git_uid:None,claude_uid:None, diff --git a/crates/freshell-supervisor/src/claude_source_observation.rs b/crates/freshell-supervisor/src/claude_source_observation.rs index dee072874..0da9877c0 100644 --- a/crates/freshell-supervisor/src/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/claude_source_observation.rs @@ -200,6 +200,7 @@ mod tests { euid: 65534, gid: 0, egid: 0, + supplementary_groups: vec![], cwd: selected.cwd.clone(), home: Some(selected.home.clone()), roots: None, From ddd64fddf0a5be53fdf1ba369573ff1e2faf9b60 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 00:36:06 -0700 Subject: [PATCH 116/493] Keep unavailable Claude source evidence distinct from selection changes --- .../src/claude_source_observation.rs | 129 +++++++++++++++--- crates/freshell-session-host/src/main.rs | 6 +- .../src/provider_input_preparation.rs | 6 +- 3 files changed, 117 insertions(+), 24 deletions(-) diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index 877a1f767..f18372bb8 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -3,6 +3,7 @@ use freshell_platform::claude_inputs::observation::{ observe, ObservationReply, ObservationRequest, MAX_REPLY_BYTES, MAX_REQUEST_BYTES, }; +use freshell_runtime_protocol::{RuntimeError, RuntimeErrorCode}; use sha2::{Digest, Sha256}; use std::io::{Read, Write}; @@ -14,22 +15,33 @@ pub(crate) async fn verify_final( uid: u32, gid: u32, runtime: freshell_platform::claude_inputs::NativeSourceRuntime, -) -> Result<(), String> { +) -> Result<(), RuntimeError> { use freshell_runtime_protocol::bounded_json_bytes; let Some((request, expected)) = crate::provider_input_preparation::final_claude_source_request( context, stage, cwd, home, runtime, - )? + ) + .map_err(|reason| final_source_error(RuntimeErrorCode::HostUnreachable, reason))? else { return Ok(()); }; if (expected.uid, expected.euid, expected.gid, expected.egid) != (uid, uid, gid, gid) { - return Err("Claude actual provider identity differs from accepted inputs; retain inputs and reprepare".into()); + return Err(final_source_error( + RuntimeErrorCode::ProviderInputsChanged, + "Claude actual provider identity differs from accepted inputs", + )); } - let raw = bounded_json_bytes(&request, MAX_REQUEST_BYTES) - .map_err(|_| "Claude final source request exceeds its finite metadata profile")?; + let raw = bounded_json_bytes(&request, MAX_REQUEST_BYTES).map_err(|_| { + final_source_error( + RuntimeErrorCode::HostUnreachable, + "Claude final source request exceeds its finite metadata profile", + ) + })?; let request_sha256 = format!("{:x}", Sha256::digest(raw)); let executable = std::env::current_exe().map_err(|_| { - "Claude final source reader artifact unavailable; retain inputs and reprepare" + final_source_error( + RuntimeErrorCode::HostUnreachable, + "Claude final source reader artifact unavailable", + ) })?; let mut command = tokio::process::Command::new("/usr/bin/setpriv"); command @@ -45,9 +57,12 @@ pub(crate) async fn verify_final( .stderr(std::process::Stdio::null()) .kill_on_drop(true); let started = std::time::Instant::now(); - let mut child = command - .spawn() - .map_err(|_| "Claude final source reader could not start; retain inputs and reprepare")?; + let mut child = command.spawn().map_err(|_| { + final_source_error( + RuntimeErrorCode::HostUnreachable, + "Claude final source reader could not start", + ) + })?; let reader_pid = child.id(); tracing::info!( event = "provider.inputs.native_final.started", @@ -58,15 +73,15 @@ pub(crate) async fn verify_final( gid, "Observing final native provider source selection" ); - let result = read_final_child( + let result = verify_final_child( &mut child, &request, &request_sha256, + &expected, std::time::Duration::from_secs(20), std::time::Duration::from_secs(5), ) - .await - .and_then(|reply| expected.require_same_selection(&reply.observation)); + .await; match &result { Ok(()) => tracing::info!( event = "provider.inputs.native_final.completed", @@ -78,16 +93,22 @@ pub(crate) async fn verify_final( ), Err(reason) => { tracing::warn!(event="provider.inputs.native_final.blocked",provider="claude",stage="session_host", - reader_pid,elapsed_ms=started.elapsed().as_millis() as u64,cause=%reason,"Final native provider source selection unavailable or changed") + reader_pid,elapsed_ms=started.elapsed().as_millis() as u64,code=?reason.code,cause=%reason.message,"Final native provider source selection unavailable or changed") } } - result.map_err(|reason| { + result +} + +fn final_source_error(code: RuntimeErrorCode, reason: impl Into) -> RuntimeError { + let reason = reason.into(); + RuntimeError::new( + code, if reason.contains("retain inputs and reprepare") { reason } else { format!("{reason}; retain inputs and reprepare") - } - }) + }, + ) } async fn read_final_child( @@ -165,6 +186,33 @@ async fn read_final_child( Ok(reply) } +async fn verify_final_child( + child: &mut tokio::process::Child, + request: &ObservationRequest, + request_sha256: &str, + expected: &freshell_platform::claude_inputs::observation::NativeObservation, + exchange_timeout: std::time::Duration, + cleanup_timeout: std::time::Duration, +) -> Result<(), RuntimeError> { + // Transport, binding and cleanup failures provide no selection evidence. + // Only two complete validated observations can establish a source change. + let reply = read_final_child( + child, + request, + request_sha256, + exchange_timeout, + cleanup_timeout, + ) + .await + .map_err(|reason| final_source_error(RuntimeErrorCode::HostUnreachable, reason))?; + expected + .validate() + .map_err(|reason| final_source_error(RuntimeErrorCode::HostUnreachable, reason))?; + expected + .require_same_selection(&reply.observation) + .map_err(|reason| final_source_error(RuntimeErrorCode::ProviderInputsChanged, reason)) +} + pub(crate) fn serve_stdio() -> Result<(), String> { exchange(&mut std::io::stdin().lock(), &mut std::io::stdout().lock()) } @@ -249,7 +297,7 @@ mod tests { use std::io::{Cursor, Error, ErrorKind}; struct ReaderCase { - result: Result, + result: Result<(), RuntimeError>, joined_before_test_cleanup: bool, elapsed: std::time::Duration, } @@ -279,10 +327,19 @@ mod tests { request_sha256: request_sha256.clone(), observation: observe(&cwd, &home, NativeSourceRuntime::Terminal, &[]).unwrap(), }; + let expected = reply.observation.clone(); if mode == "wrong-binding" { reply.request_sha256 = "b".repeat(64); } else if mode == "wrong-runtime" { reply.observation.native_source_runtime = NativeSourceRuntime::Sdk; + } else if mode == "changed-selection" { + let added = (0..u32::MAX) + .find(|group| { + *group != expected.egid && !expected.supplementary_groups.contains(group) + }) + .unwrap(); + reply.observation.supplementary_groups.push(added); + reply.observation.supplementary_groups.sort_unstable(); } let mut bytes = serde_json::to_vec(&reply).unwrap(); if matches!(mode, "near-cap" | "oversized") { @@ -322,10 +379,11 @@ process.stdin.on('data',()=>{}).on('end',()=>{ .spawn() .unwrap(); let started = std::time::Instant::now(); - let result = read_final_child( + let result = verify_final_child( &mut child, &request, &request_sha256, + &expected, std::time::Duration::from_secs(2), std::time::Duration::from_millis(200), ) @@ -379,7 +437,40 @@ process.stdin.on('data',()=>{}).on('end',()=>{ "finite reader exchange for {mode}" ); if let Err(error) = case.result { - assert!(!error.contains("PRIVATE_READER_BODY_SENTINEL")); + assert!(!error.message.contains("PRIVATE_READER_BODY_SENTINEL")); + } + } + } + + #[tokio::test] + async fn claude_final_reader_classifies_only_complete_proven_selection_changes() { + for mode in [ + "valid", + "changed-selection", + "invalid", + "wrong-binding", + "wrong-runtime", + "eof-stall", + ] { + let case = reader_case(mode).await; + assert!( + case.joined_before_test_cleanup, + "actual private fault reader must be joined for {mode}" + ); + if mode == "valid" { + case.result.unwrap(); + } else { + let error = case.result.unwrap_err(); + let expected = if mode == "changed-selection" { + RuntimeErrorCode::ProviderInputsChanged + } else { + RuntimeErrorCode::HostUnreachable + }; + assert_eq!( + error.code, expected, + "unavailable evidence cannot authorize source replan for {mode}" + ); + assert!(!error.message.contains("PRIVATE_READER_BODY_SENTINEL")); } } } diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index caab6c3fb..0cce52804 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1074,8 +1074,7 @@ async fn grant_execution( prepared.terminal.run_as_gid, freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, ) - .await - .map_err(|error| RuntimeError::new(RuntimeErrorCode::ProviderInputsChanged, error))?; + .await?; consume_grant(&mut persisted)?; let prepared = providers::start_terminal_sidecar(prepared) .await @@ -1215,8 +1214,7 @@ async fn grant_execution( launch.run_as_gid, freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, ) - .await - .map_err(|error| RuntimeError::new(RuntimeErrorCode::ProviderInputsChanged, error))?; + .await?; consume_grant(&mut persisted)?; freshell_platform::managed_child_secrets::install(resolved_secrets.environment) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index e79e307fd..eb75648e7 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -434,7 +434,11 @@ mod tests { ) .await .expect_err("the actual final host gate cannot bypass unrecorded native source evidence"); - assert!(error.contains("retain inputs and reprepare"), "{error}"); + assert_eq!(error.code, RuntimeErrorCode::HostUnreachable); + assert!( + error.message.contains("retain inputs and reprepare"), + "{error:?}" + ); assert_eq!( std::fs::read(owned.path().join(".freshell/input-manifest.json")).unwrap(), bytes From b6524f5711fb9df2f1823ff6a280b1992da7f94d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 00:41:18 -0700 Subject: [PATCH 117/493] Confirm owned mountpoint helper absence before releasing preparation --- .../src/backend/provider_mountpoints.rs | 184 ++++++++++++++---- 1 file changed, 148 insertions(+), 36 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index 3e55f1acd..4666a75e1 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -73,6 +73,58 @@ fn selections(mounts: &docker::TerminalMounts) -> Result, Backen } impl DockerEngineBackend { + async fn cleanup_provider_mountpoint_helper( + &self, + name: &str, + body: &Value, + ) -> Result<(), BackendError> { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 1024 * 1024, + ) + .await?; + if inspected.status == 404 { + return Ok(()); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("provider mountpoint cleanup metadata invalid"))?; + let id = actual["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) + .ok_or_else(|| fail("provider mountpoint cleanup identity invalid"))?; + if inspected.status != 200 || !input_publisher::owned_helper(&actual, body) { + return Err(fail("provider mountpoint cleanup ownership changed")); + } + let removed = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/containers/{id}?force=1"), + None, + 64 * 1024, + ) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail("provider mountpoint helper cleanup failed")); + } + // An acknowledged delete does not release a candidate's mounts until + // the exact inspected helper is confirmed absent. + let absent = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{id}/json"), + None, + 64 * 1024, + ) + .await?; + if absent.status != 404 { + return Err(fail("provider mountpoint helper cleanup not confirmed")); + } + Ok(()) + } + pub(super) async fn prepare_provider_mountpoints( &self, spec: &CreateRuntimeSpec, @@ -226,42 +278,12 @@ impl DockerEngineBackend { .await .unwrap_or_else(|_| Err(fail("provider mountpoint preparation timed out"))); if attempted { - let cleanup = async { - let inspected = self - .request_bounded( - "GET", - &format!("{DOCKER_API}/containers/{name}/json"), - None, - 1024 * 1024, - ) - .await?; - if inspected.status == 404 { - return Ok(()); - } - let actual: Value = serde_json::from_slice(&inspected.body) - .map_err(|_| fail("provider mountpoint cleanup metadata invalid"))?; - let id = actual["Id"] - .as_str() - .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) - .ok_or_else(|| fail("provider mountpoint cleanup identity invalid"))?; - if inspected.status != 200 || !input_publisher::owned_helper(&actual, &body) { - return Err(fail("provider mountpoint cleanup ownership changed")); - } - let removed = self - .request( - "DELETE", - &format!("{DOCKER_API}/containers/{id}?force=1"), - None, - ) - .await?; - if !matches!(removed.status, 204 | 404) { - return Err(fail("provider mountpoint helper cleanup failed")); - } - Ok(()) - }; - tokio::time::timeout(Duration::from_secs(5), cleanup) - .await - .map_err(|_| fail("provider mountpoint cleanup timed out"))??; + tokio::time::timeout( + Duration::from_secs(5), + self.cleanup_provider_mountpoint_helper(&name, &body), + ) + .await + .map_err(|_| fail("provider mountpoint cleanup timed out"))??; } tracing::debug!(soul_id=%spec.soul_id,entries=selected.len(),succeeded=result.is_ok(),"runtime.provider_mountpoints_settled"); result @@ -275,6 +297,96 @@ mod tests { ProviderConfigBind, ProviderResourceKind, ProviderResourceMount, }; + #[tokio::test] + async fn provider_mountpoint_cleanup_requires_confirmed_absence_after_delete() { + for mode in [ + "present-after-delete", + "removed", + "already-absent", + "foreign", + "lost-delete-ack", + ] { + let temp = tempfile::tempdir().unwrap(); + let body = json!({"Image":"synthetic-image","User":"0:0","Tty":false, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["prepare-provider-mountpoints"], + "Labels":{"com.freshell.soul-id":"synthetic-owner"},"Env":[], + "HostConfig":{"Mounts":[],"LogConfig":{"Type":"none"}}}); + let seen = std::sync::Arc::new(std::sync::Mutex::new(Vec::::new())); + let requests = seen.clone(); + let expected = body.clone(); + let listener = tokio::net::UnixListener::bind(temp.path().join("docker.sock")).unwrap(); + let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); + let server = tokio::spawn(async move { + let mut deleted = false; + loop { + let accepted = tokio::select! { + _ = &mut stopped => break, + accepted = listener.accept() => accepted, + }; + let (mut stream, _) = accepted.unwrap(); + let mut header = Vec::new(); + let mut byte = [0]; + while !header.ends_with(b"\r\n\r\n") { + stream.read_exact(&mut byte).await.unwrap(); + header.push(byte[0]); + assert!(header.len() <= 32 * 1024); + } + let header = String::from_utf8(header).unwrap(); + let line = header.lines().next().unwrap().to_owned(); + requests.lock().unwrap().push(line.clone()); + let (status, value) = if line.starts_with("DELETE") { + deleted = true; + (if mode == "lost-delete-ack" { 503 } else { 204 }, json!({})) + } else if mode == "already-absent" || (deleted && mode == "removed") { + (404, json!({})) + } else { + let mut actual = expected.clone(); + if mode == "foreign" { + actual["User"] = json!("foreign"); + } + ( + 200, + json!({"Id":"a".repeat(64),"Config":actual,"HostConfig":actual["HostConfig"]}), + ) + }; + let raw = value.to_string(); + stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{raw}", raw.len()).as_bytes()).await.unwrap(); + } + }); + let backend = DockerEngineBackend::new(temp.path().join("docker.sock")); + let result = tokio::time::timeout( + Duration::from_secs(2), + backend.cleanup_provider_mountpoint_helper("freshell-mountpoints-synthetic", &body), + ) + .await + .unwrap(); + let _ = stop.send(()); + tokio::time::timeout(Duration::from_secs(2), server) + .await + .unwrap() + .unwrap(); + let requests = seen.lock().unwrap(); + assert_eq!( + result.is_ok(), + matches!(mode, "removed" | "already-absent"), + "delete acknowledgement cannot prove cleanup for {mode}" + ); + if mode == "removed" { + assert_eq!( + requests.len(), + 3, + "owned deletion must be followed by exact absence inspection" + ); + } + if matches!(mode, "foreign" | "already-absent") { + assert!( + !requests.iter().any(|line| line.starts_with("DELETE")), + "cleanup must not delete foreign or already absent objects" + ); + } + } + } + #[test] fn provider_mountpoints_precreate_private_leaf_inside_external_selected_root() { let mounts = docker::TerminalMounts { From 31d1379d2d7ab009028e1470724fc6e89e2b96fb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 01:30:56 -0700 Subject: [PATCH 118/493] Add bounded ordinary Claude candidate projection helper --- .../src/claude_inputs/observation.rs | 38 ++ crates/freshell-runtime-protocol/src/lib.rs | 131 +++++ .../src/managed_provider_launch_tests.rs | 1 + .../src/managed_provider_references/claude.rs | 1 + .../src/claude_source_observation.rs | 531 ++++++++++++++++++ crates/freshell-session-host/src/main.rs | 3 + .../src/provider_input_preparation.rs | 3 +- .../tests/claude_source_observation.rs | 51 ++ 8 files changed, 758 insertions(+), 1 deletion(-) diff --git a/crates/freshell-platform/src/claude_inputs/observation.rs b/crates/freshell-platform/src/claude_inputs/observation.rs index ef66de336..d2a977569 100644 --- a/crates/freshell-platform/src/claude_inputs/observation.rs +++ b/crates/freshell-platform/src/claude_inputs/observation.rs @@ -22,6 +22,22 @@ pub struct ObservationRequest { pub setting_sources: Vec, pub selector_manifest_sha256: String, pub plan_revision: u8, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub candidate_binding: Option, +} + +/// Ordinary candidate identity covered by the private observation request +/// digest. Authority material is hashed; no credential payload enters this +/// reader contract. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct CandidateBinding { + pub soul_id: String, + pub generation_id: String, + pub authority_sha256: String, + pub publication_sha256: String, + pub manifest_sha256: String, + pub prepared_result_sha256: String, } impl ObservationRequest { @@ -49,6 +65,27 @@ impl ObservationRequest { validate_sources(&self.setting_sources)?; validate_path(&self.cwd)?; validate_path(&self.home)?; + if let Some(binding) = &self.candidate_binding { + if [binding.soul_id.as_str(), binding.generation_id.as_str()] + .iter() + .any(|id| id.is_empty() || id.len() > 160 || id.chars().any(char::is_control)) + || [ + &binding.authority_sha256, + &binding.publication_sha256, + &binding.manifest_sha256, + &binding.prepared_result_sha256, + ] + .iter() + .any(|digest| { + digest.len() != 64 || !digest.bytes().all(|byte| byte.is_ascii_hexdigit()) + }) + { + return Err( + "Claude candidate observation binding is invalid; retain inputs and reprepare" + .into(), + ); + } + } Ok(()) } } @@ -512,6 +549,7 @@ mod tests { setting_sources: vec![ClaudeSettingSource::Local], selector_manifest_sha256: "a".repeat(64), plan_revision: 0, + candidate_binding: None, }; let reply = ObservationReply { request_sha256: "b".repeat(64), diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index d9fc3c7ac..e1b440a30 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2911,6 +2911,137 @@ pub struct ObserveClaudeSourcesRequest { pub observation_request: freshell_platform::claude_inputs::observation::ObservationRequest, } +/// Selection-only input to the fixed Claude candidate projection helper. +/// Ordinary documents stay on its verified private stage mount; credentials, +/// secret references and native provider stores are not projection inputs. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ClaudeCandidateProjectionRequest { + pub source_request: freshell_platform::claude_inputs::observation::ObservationRequest, + pub workspace_path: String, + pub provider_launch_context: ProviderLaunchContext, + pub publication: ClaudeCandidatePublication, +} + +/// Credential-free issuer metadata published beside a complete ordinary +/// Claude result. It qualifies the published generation, not a provider HOME. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ClaudeCandidatePublication { + pub version: u8, + pub soul_id: SoulId, + pub provider: String, + pub authority_sha256: String, + pub initial_reservation_id: Option, + pub registry_attempt_id: Option, + pub intent_revision: u64, + pub prior_generation_id: Option, + pub generation_id: String, + pub manifest_sha256: String, + pub prepared_context_sha256: String, + pub prepared_result_sha256: String, + pub selector_manifest_sha256: String, + pub plan_revision: u8, + pub native_source_runtime: freshell_platform::claude_inputs::NativeSourceRuntime, + pub workspace_path: String, + pub cwd: std::path::PathBuf, + pub git_common_dir: Option, +} + +impl ClaudeCandidateProjectionRequest { + pub fn validate(&self) -> Result<(), RuntimeError> { + use sha2::Digest; + let invalid = || { + RuntimeError::new(RuntimeErrorCode::InvalidRequest, + "Claude candidate projection has incomplete ordinary selection metadata; retain inputs and reprepare") + }; + self.source_request.validate().map_err(|_| invalid())?; + self.provider_launch_context.validate("claude")?; + let inputs = self + .provider_launch_context + .prepared_inputs + .as_ref() + .ok_or_else(invalid)?; + if self.workspace_path.len() > 64 * 1024 + || self.workspace_path.chars().any(char::is_control) + || !std::path::Path::new(&self.workspace_path).is_absolute() + || !self.source_request.cwd.starts_with(&self.workspace_path) + || inputs.manifest.source_count != 0 + || inputs.manifest.alias_count != 0 + || inputs.manifest.recipe_count != 0 + || inputs.indexed_files.is_some() + { + return Err(invalid()); + } + let source = &self.source_request; + let binding = source.candidate_binding.as_ref().ok_or_else(invalid)?; + let published = &self.publication; + let publication_bytes = + bounded_json_bytes(published, MAX_CONTROL_FRAME_BYTES).map_err(|_| invalid())?; + let context_bytes = + bounded_json_bytes(&self.provider_launch_context, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| invalid())?; + let initial = published.initial_reservation_id.as_ref(); + if published.version != 1 + || !matches!(published.provider.as_str(), "claude" | "kilroy") + || (published.provider == "kilroy" + && source.native_source_runtime + != freshell_platform::claude_inputs::NativeSourceRuntime::Sdk) + || published.soul_id.as_str() != binding.soul_id + || published.generation_id != binding.generation_id + || published.generation_id != inputs.generation_id + || published.manifest_sha256 != binding.manifest_sha256 + || published.manifest_sha256 != inputs.manifest.sha256 + || published.prepared_result_sha256 != binding.prepared_result_sha256 + || published.authority_sha256 != binding.authority_sha256 + || published.workspace_path != self.workspace_path + || published.cwd != source.cwd + || published.native_source_runtime != source.native_source_runtime + || published.selector_manifest_sha256 != source.selector_manifest_sha256 + || published.plan_revision != source.plan_revision + || published.intent_revision == 0 + || published.soul_id.as_str().len() > 160 + || published.soul_id.as_str().chars().any(char::is_control) + || published.soul_id.as_str().is_empty() + || format!("{:x}", Sha256::digest(publication_bytes)) != binding.publication_sha256 + || format!("{:x}", Sha256::digest(context_bytes)) != published.prepared_context_sha256 + || if let Some(reservation) = initial { + published.intent_revision != 1 + || uuid::Uuid::parse_str(reservation).is_err() + || inputs.initial_auth_reservation.as_ref() != initial + || published.registry_attempt_id.is_some() + || published.prior_generation_id.is_some() + } else { + published + .registry_attempt_id + .as_ref() + .is_none_or(|attempt| RecoveryAttemptId::parse(attempt.as_str()).is_err()) + || published + .prior_generation_id + .as_ref() + .is_none_or(|generation| { + !generation.starts_with("grant-") + || uuid::Uuid::parse_str(&generation[6..]).is_err() + }) + } + || published.git_common_dir.as_ref().is_some_and(|path| { + path.len() > 64 * 1024 + || path.chars().any(char::is_control) + || !std::path::Path::new(path).is_absolute() + }) + { + return Err(invalid()); + } + bounded_json_bytes( + source, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .map_err(|_| invalid())?; + bounded_json_bytes(self, MAX_CONTROL_FRAME_BYTES).map_err(|_| invalid())?; + Ok(()) + } +} + impl ObserveClaudeSourcesRequest { pub fn validate(&self) -> Result<(), RuntimeError> { let invalid = || { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index f9d7ac666..96c84f08b 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -1113,6 +1113,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ setting_sources: vec![ClaudeSettingSource::Local], selector_manifest_sha256: "a".repeat(64), plan_revision: 0, + candidate_binding: None, }, }; let reply = client.observe_claude_sources(request).await.expect( diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 4a202297b..0d74f51c3 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -127,6 +127,7 @@ pub(crate) async fn prepare_managed( format!("{:x}", Sha256::digest(bytes)) }, plan_revision: 0, + candidate_binding: None, }; let observation_request_sha256 = { use sha2::{Digest, Sha256}; diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index f18372bb8..e12c333b5 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -217,6 +217,106 @@ pub(crate) fn serve_stdio() -> Result<(), String> { exchange(&mut std::io::stdin().lock(), &mut std::io::stdout().lock()) } +fn project_candidate( + request: &freshell_runtime_protocol::ClaudeCandidateProjectionRequest, + stage: &std::path::Path, + home: &std::path::Path, + identity: (u32, u32), +) -> Result<(), String> { + let unavailable = |operation: &'static str| { + tracing::warn!( + event = "provider.inputs.candidate_projection.blocked", + provider = "claude", + stage = "candidate", + operation, + "Candidate ordinary projection unavailable" + ); + format!("Claude candidate {operation} unavailable; retain inputs and reprepare") + }; + request.validate().map_err(|_| unavailable("metadata"))?; + let published = crate::provider_input_preparation::read_private( + stage, + ".freshell/claude-candidate-binding.json", + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| unavailable("publication"))?; + let binding = request + .source_request + .candidate_binding + .as_ref() + .ok_or_else(|| unavailable("publication"))?; + if format!("{:x}", Sha256::digest(&published)) != binding.publication_sha256 { + return Err(unavailable("publication")); + } + crate::verify_initialized_provider_home(home, identity.0, identity.1) + .map_err(|_| unavailable("initialized_home"))?; + let source = &request.source_request; + let Some((accepted, expected)) = + crate::provider_input_preparation::final_claude_source_request( + Some(&request.provider_launch_context), + stage, + &source.cwd, + home, + source.native_source_runtime, + ) + .map_err(|_| unavailable("accepted_manifest"))? + else { + return Err(unavailable("accepted_manifest")); + }; + if source.home != home + || source.selector_manifest_sha256 != accepted.selector_manifest_sha256 + || source.plan_revision != accepted.plan_revision + || source.setting_sources != accepted.setting_sources + || (expected.uid, expected.euid, expected.gid, expected.egid) + != (identity.0, identity.0, identity.1, identity.1) + || std::fs::canonicalize(&request.workspace_path) + .ok() + .as_deref() + != Some(std::path::Path::new(&request.workspace_path)) + || std::fs::canonicalize(&source.cwd).ok().as_ref() != Some(&source.cwd) + { + return Err(unavailable("selection_binding")); + } + // Reuse the workload's selected copier and stale-leaf transaction. This + // never resolves secrets, probes a provider, or reads original HOME again. + crate::refresh_provider_config_references( + &request.provider_launch_context, + "claude", + std::path::Path::new(&request.workspace_path), + stage, + home, + identity.0, + identity.1, + ) + .map_err(|_| unavailable("ordinary_refresh")) +} + +fn read_candidate_request( + input: &mut impl Read, +) -> Result { + let mut header = [0; 4]; + input + .read_exact(&mut header) + .map_err(|_| "Claude candidate private request header unavailable")?; + let length = u32::from_be_bytes(header) as usize; + if length > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err("Claude candidate private request exceeds the metadata profile; retain inputs and reprepare".into()); + } + let mut raw = Vec::new(); + raw.try_reserve_exact(length) + .map_err(|_| "Claude candidate private request allocation unavailable")?; + raw.resize(length, 0); + input + .read_exact(&mut raw) + .map_err(|_| "Claude candidate private request incomplete")?; + let request: freshell_runtime_protocol::ClaudeCandidateProjectionRequest = + serde_json::from_slice(&raw).map_err(|_| "Claude candidate private request invalid")?; + request + .validate() + .map_err(|_| "Claude candidate private request metadata unavailable")?; + Ok(request) +} + pub(crate) fn serve_private() -> Result<(), String> { let mut input = std::os::unix::net::UnixStream::connect("/run/freshell-source-exchange/reply.sock") @@ -234,6 +334,99 @@ pub(crate) fn serve_private() -> Result<(), String> { exchange(&mut input, &mut output) } +pub(crate) fn serve_candidate_private() -> Result<(), String> { + use std::{ + os::unix::fs::{FileTypeExt, MetadataExt}, + path::Path, + }; + let home = Path::new("/home/freshell/provider"); + let socket = Path::new("/run/freshell-source-exchange/reply.sock"); + let unavailable = + || "Claude candidate private helper namespace mapping is unavailable".to_string(); + if unsafe { libc::getuid() } != 0 + || unsafe { libc::geteuid() } != 0 + || unsafe { libc::getgid() } != 0 + || unsafe { libc::getegid() } != 0 + { + return Err(unavailable()); + } + crate::verify_initialized_provider_home(home, 65534, 0).map_err(|_| unavailable())?; + let directory = + std::fs::symlink_metadata(socket.parent().unwrap()).map_err(|_| unavailable())?; + let endpoint = std::fs::symlink_metadata(socket).map_err(|_| unavailable())?; + if !directory.is_dir() + || directory.uid() != 0 + || directory.gid() != 0 + || directory.mode() & 0o777 != 0o710 + || !endpoint.file_type().is_socket() + || endpoint.uid() != 0 + || endpoint.gid() != 0 + || endpoint.mode() & 0o777 != 0o660 + { + return Err(unavailable()); + } + let mut stream = std::os::unix::net::UnixStream::connect(socket) + .map_err(|_| "Claude candidate private exchange unavailable")?; + let timeout = Some(std::time::Duration::from_secs(15)); + stream + .set_read_timeout(timeout) + .map_err(|_| "Claude candidate private timeout unavailable")?; + stream + .set_write_timeout(timeout) + .map_err(|_| "Claude candidate private timeout unavailable")?; + let request = read_candidate_request(&mut stream)?; + if request.source_request.home != home + || std::env::current_dir().ok().as_ref() != Some(&request.source_request.cwd) + || std::env::var_os("HOME") + .map(std::path::PathBuf::from) + .as_deref() + != Some(home) + { + return Err( + "Claude candidate actual launch selection unavailable; retain inputs and reprepare" + .into(), + ); + } + let raw = + freshell_runtime_protocol::bounded_json_bytes(&request.source_request, MAX_REQUEST_BYTES) + .map_err(|_| "Claude candidate source request exceeds metadata profile")?; + project_candidate( + &request, + Path::new("/run/freshell-candidate-inputs"), + home, + (65534, 0), + )?; + // Match the existing provider mountpoint helper's identity transition. The + // source evidence below measures this process after the transition. + if unsafe { libc::setgroups(0, std::ptr::null()) } != 0 + || unsafe { libc::setgid(0) } != 0 + || unsafe { libc::setuid(65534) } != 0 + { + return Err("Claude candidate provider identity transition unavailable".into()); + } + crate::verify_private_helper_namespace(home, socket).map_err(|_| unavailable())?; + if request.source_request.native_source_runtime + == freshell_platform::claude_inputs::NativeSourceRuntime::Terminal + { + crate::providers::run_provider_features_worker(&["claude-mcp".into()]) + .map_err(|_| "Claude candidate ordinary feature preparation unavailable")?; + } + let source = &request.source_request; + let observation = observe( + &source.cwd, + home, + source.native_source_runtime, + &source.setting_sources, + )?; + write_reply( + &ObservationReply { + request_sha256: format!("{:x}", Sha256::digest(raw)), + observation, + }, + &mut stream, + ) +} + fn exchange(input: &mut impl Read, output: &mut impl Write) -> Result<(), String> { let mut length = [0; 4]; input @@ -275,6 +468,10 @@ fn exchange(input: &mut impl Read, output: &mut impl Write) -> Result<(), String request_sha256: format!("{:x}", Sha256::digest(&raw)), observation, }; + write_reply(&reply, output) +} + +fn write_reply(reply: &ObservationReply, output: &mut impl Write) -> Result<(), String> { let bytes = freshell_runtime_protocol::bounded_json_bytes(&reply, MAX_REPLY_BYTES).map_err(|_| { "Claude observation reply exceeds the metadata profile; retain inputs and reprepare" @@ -296,6 +493,302 @@ mod tests { use super::*; use std::io::{Cursor, Error, ErrorKind}; + #[test] + fn claude_candidate_projection_runs_selected_dependency_and_frozen_config_consumer() { + use freshell_platform::claude_inputs::{ + AcceptedNativeSourcePlan, ClaudeInputManifest, ClaudeSettingSource, NativeSourceRuntime, + }; + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeCandidateProjectionRequest, ProviderLaunchContext, + }; + use std::os::unix::fs::PermissionsExt; + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = tempfile::tempdir().unwrap(); + let workspace = owned.path().join("workspace"); + let original = owned.path().join("original"); + let stage = owned.path().join("stage"); + let home = owned.path().join("candidate-home"); + for path in [&workspace, &original, &stage, &home] { + std::fs::create_dir(path).unwrap(); + } + std::fs::set_permissions(&home, std::fs::Permissions::from_mode(0o700)).unwrap(); + crate::verify_initialized_provider_home(&home, unsafe { libc::geteuid() }, unsafe { libc::getegid() }).expect("synthetic candidate HOME must match the existing initialized identity before testing projection"); + std::fs::create_dir_all(stage.join(".freshell")).unwrap(); + std::fs::create_dir_all(stage.join(".claude")).unwrap(); + std::fs::write( + stage.join(".claude/settings.json"), + r#"{"selected":"frozen-original"}"#, + ) + .unwrap(); + std::fs::create_dir_all(home.join(".claude")).unwrap(); + let retained_auth = home.join(".claude/.credentials.json"); + std::fs::write(&retained_auth, b"PRIVATE_RETAINED_AUTH_SENTINEL").unwrap(); + std::fs::create_dir_all(home.join(".freshell/provider-roots/unit-0")).unwrap(); + let unit = stage.join(".freshell/resource-units/unit-0"); + std::fs::create_dir_all(unit.join("node_modules/selected-dependency")).unwrap(); + std::fs::write( + unit.join("node_modules/selected-dependency/index.js"), + "module.exports=value=>'dependency:'+value", + ) + .unwrap(); + std::fs::write(unit.join("consumer.cjs"), r#"const fs=require('node:fs');const selected=JSON.parse(fs.readFileSync(process.argv[2],'utf8'));console.log(require('selected-dependency')(selected.selected));"#).unwrap(); + let mut manifest: ClaudeInputManifest = serde_json::from_value(serde_json::json!({ + "version":1,"nativeSourceRuntime":runtime,"originalHome":original, + "cwd":workspace,"workspace":workspace,"settingSources":["user"],"pluginDirs":[] + })) + .unwrap(); + let selector_sha256 = format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024).unwrap() + ) + ); + let observation = + observe(&workspace, &home, runtime, &[ClaudeSettingSource::User]).unwrap(); + manifest.accepted_native_plan = Some(AcceptedNativeSourcePlan { + selector_manifest_sha256: selector_sha256.clone(), + plan_revision: 0, + observation, + }); + let raw = serde_json::to_vec(&manifest).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &raw).unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let reservation = uuid::Uuid::new_v4().to_string(); + let context: ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"claude":{"mcp_args":[]}}, + "config":[{"root":"user_provider","relativePath":".claude/settings.json","providerRelativePath":".claude/settings.json","format":"json"}], + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"initialAuthReservation":reservation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&raw)),"sourceCount":0,"aliasCount":0,"recipeCount":0}, + "resourceMounts":[{"stageRelativePath":".freshell/resource-units/unit-0","providerRelativePath":".freshell/provider-roots/unit-0","logicalDestination":workspace.join("selected-plugin"),"effectiveDestination":workspace.join("selected-plugin"),"kind":"directory"}]} + })).unwrap(); + let publication = freshell_runtime_protocol::ClaudeCandidatePublication { + version: 1, + soul_id: freshell_runtime_protocol::SoulId::new(), + provider: "claude".into(), + authority_sha256: "d".repeat(64), + initial_reservation_id: Some(reservation), + registry_attempt_id: None, + intent_revision: 1, + prior_generation_id: None, + generation_id: generation.clone(), + manifest_sha256: context + .prepared_inputs + .as_ref() + .unwrap() + .manifest + .sha256 + .clone(), + prepared_context_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes( + &context, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + ) + .unwrap() + ) + ), + prepared_result_sha256: "e".repeat(64), + selector_manifest_sha256: selector_sha256.clone(), + plan_revision: 0, + native_source_runtime: runtime, + workspace_path: workspace.to_str().unwrap().into(), + cwd: workspace.clone(), + git_common_dir: None, + }; + let publication_bytes = bounded_json_bytes( + &publication, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .unwrap(); + std::fs::write( + stage.join(".freshell/claude-candidate-binding.json"), + &publication_bytes, + ) + .unwrap(); + let request = ClaudeCandidateProjectionRequest { + source_request: ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: runtime, + cwd: workspace.clone(), + home: home.clone(), + setting_sources: vec![ClaudeSettingSource::User], + selector_manifest_sha256: selector_sha256, + plan_revision: 0, + candidate_binding: Some( + freshell_platform::claude_inputs::observation::CandidateBinding { + soul_id: publication.soul_id.as_str().into(), + generation_id: generation, + authority_sha256: publication.authority_sha256.clone(), + publication_sha256: format!("{:x}", Sha256::digest(&publication_bytes)), + manifest_sha256: publication.manifest_sha256.clone(), + prepared_result_sha256: publication.prepared_result_sha256.clone(), + }, + ), + }, + workspace_path: workspace.to_str().unwrap().into(), + provider_launch_context: context, + publication, + }; + let mut raw_request = freshell_runtime_protocol::bounded_json_bytes( + &request, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .unwrap(); + // Exercise the inclusive transport boundary with a valid request; + // padding is insignificant JSON whitespace, not extra selection. + raw_request.resize(freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, b' '); + let mut frame = Vec::from((raw_request.len() as u32).to_be_bytes()); + frame.extend(raw_request); + let request = read_candidate_request(&mut Cursor::new(frame)).expect( + "valid bounded private candidate frame must reach the actual projection consumer", + ); + std::fs::remove_dir(&original).unwrap(); + for fault in [ + "selector", + "revision", + "runtime", + "manifest", + "indexed-source", + "home", + "publication-soul", + "publication-generation", + "publication-authority", + "publication-result", + "publication-stage", + "publication-context", + "publication-missing-binding", + ] { + let mut altered = request.clone(); + match fault { + "selector" => altered.source_request.selector_manifest_sha256 = "b".repeat(64), + "revision" => altered.source_request.plan_revision = 2, + "runtime" => { + altered.source_request.native_source_runtime = + if runtime == NativeSourceRuntime::Terminal { + NativeSourceRuntime::Sdk + } else { + NativeSourceRuntime::Terminal + } + } + "manifest" => { + altered + .provider_launch_context + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .sha256 = "c".repeat(64) + } + "indexed-source" => { + altered + .provider_launch_context + .prepared_inputs + .as_mut() + .unwrap() + .manifest + .source_count = 1 + } + "home" => altered.source_request.home = owned.path().join("foreign-home"), + "publication-soul" => { + altered.publication.soul_id = freshell_runtime_protocol::SoulId::new() + } + "publication-generation" => { + altered.publication.generation_id = + format!("grant-{}", uuid::Uuid::new_v4()) + } + "publication-authority" => { + altered.publication.authority_sha256 = "f".repeat(64) + } + "publication-result" => { + altered.publication.prepared_result_sha256 = "f".repeat(64) + } + "publication-stage" => { + altered.publication.soul_id = freshell_runtime_protocol::SoulId::new(); + let binding = altered.source_request.candidate_binding.as_mut().unwrap(); + binding.soul_id = altered.publication.soul_id.as_str().into(); + binding.publication_sha256 = format!( + "{:x}", + Sha256::digest( + bounded_json_bytes( + &altered.publication, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + ) + .unwrap() + ) + ); + altered.validate().expect( + "a coherent substituted envelope must reach independent stage readback", + ); + } + "publication-context" => { + altered.provider_launch_context.config[0].provider_relative_path = + ".claude/another-selected-settings.json".into() + } + "publication-missing-binding" => { + altered.source_request.candidate_binding = None + } + _ => unreachable!(), + } + assert!( + project_candidate(&altered, &stage, &home, unsafe { + (libc::geteuid(), libc::getegid()) + }) + .is_err(), + "incomplete or mismatched candidate must block for {fault}" + ); + assert!( + !home.join(".claude/settings.json").exists(), + "blocked candidate must not project settings for {fault}" + ); + assert!( + !home + .join(".freshell/provider-roots/unit-0/consumer.cjs") + .exists(), + "blocked candidate must not project executable resources for {fault}" + ); + assert!( + !home.join(crate::PROVIDER_PROJECTION_MANIFEST).exists(), + "blocked candidate must not commit a projection for {fault}" + ); + assert_eq!( + std::fs::read(&retained_auth).unwrap(), + b"PRIVATE_RETAINED_AUTH_SENTINEL" + ); + } + project_candidate(&request, &stage, &home, unsafe { + (libc::geteuid(), libc::getegid()) + }) + .unwrap(); + let output = std::process::Command::new("node") + .arg(home.join(".freshell/provider-roots/unit-0/consumer.cjs")) + .arg(home.join(".claude/settings.json")) + .current_dir(&workspace) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + assert!( + output.status.success(), + "selected staged dependency/config consumer must execute" + ); + assert_eq!(output.stdout, b"dependency:frozen-original\n"); + assert_eq!( + std::fs::read(&retained_auth).unwrap(), + b"PRIVATE_RETAINED_AUTH_SENTINEL" + ); + assert_eq!( + std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + raw + ); + assert!( + !original.exists(), + "pure candidate projection must not rescan frozen original HOME" + ); + } + } + struct ReaderCase { result: Result<(), RuntimeError>, joined_before_test_cleanup: bool, @@ -319,6 +812,7 @@ mod tests { setting_sources: vec![], selector_manifest_sha256: "a".repeat(64), plan_revision: 0, + candidate_binding: None, }; let raw = freshell_runtime_protocol::bounded_json_bytes(&request, MAX_REQUEST_BYTES).unwrap(); @@ -503,6 +997,43 @@ process.stdin.on('data',()=>{}).on('end',()=>{ } } + #[test] + fn claude_candidate_rejects_oversized_header_before_reading_or_allocating_body() { + for length in [ + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES as u32 + 1, + u32::MAX, + ] { + let mut input = HeaderOnly { + header: Cursor::new(length.to_be_bytes()), + body_reads: 0, + }; + let error = read_candidate_request(&mut input).unwrap_err(); + assert!(error.contains("exceeds the metadata profile")); + assert_eq!(input.body_reads, 0); + } + } + + #[test] + fn claude_candidate_incomplete_or_invalid_frame_never_discloses_private_body() { + let private = b"PRIVATE_CANDIDATE_BODY_SENTINEL"; + let mut incomplete = Vec::from((private.len() as u32 + 1).to_be_bytes()); + incomplete.extend(private); + let mut invalid = Vec::from((private.len() as u32).to_be_bytes()); + invalid.extend(private); + for (frame, expected) in [ + (vec![0, 0, 0], "header unavailable"), + (incomplete, "request incomplete"), + (invalid, "request invalid"), + ] { + let error = read_candidate_request(&mut Cursor::new(frame)).unwrap_err(); + assert!(error.contains(expected), "{error}"); + assert!(!error + .as_bytes() + .windows(private.len()) + .any(|window| window == private)); + } + } + #[test] fn claude_observation_rejects_oversized_header_before_reading_or_allocating_body() { for length in [MAX_REQUEST_BYTES as u32 + 1, u32::MAX] { diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 0cce52804..b7f14a1b4 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -126,6 +126,9 @@ async fn run() -> Result<(), String> { )?; claude_source_observation::serve_private() } + Some("project-claude-candidate-private") if args.len() == 2 => { + claude_source_observation::serve_candidate_private() + } Some("native-auth-only") if args.len() == 2 => { verify_private_helper_namespace( Path::new("/home/freshell/provider"), diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index eb75648e7..5992dc8b1 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -3,7 +3,7 @@ use crate::provider_secret_resolution::ResolvedProviderSecrets; use freshell_runtime_protocol::{ProviderLaunchContext, ProviderPathInputs}; use std::{collections::BTreeMap, path::Path}; -fn read_private(root: &Path, relative: &str, limit: usize) -> Result, String> { +pub(crate) fn read_private(root: &Path, relative: &str, limit: usize) -> Result, String> { use std::io::Read; if std::path::Path::new(relative) .components() @@ -89,6 +89,7 @@ pub(crate) fn final_claude_source_request( setting_sources: manifest.setting_sources, selector_manifest_sha256: plan.selector_manifest_sha256.clone(), plan_revision: plan.plan_revision, + candidate_binding: None, }; request.validate()?; Ok(Some((request, plan.observation.clone()))) diff --git a/crates/freshell-session-host/tests/claude_source_observation.rs b/crates/freshell-session-host/tests/claude_source_observation.rs index 46776c366..3e3d97edd 100644 --- a/crates/freshell-session-host/tests/claude_source_observation.rs +++ b/crates/freshell-session-host/tests/claude_source_observation.rs @@ -101,6 +101,57 @@ async fn session_host_claude_private_observer_rejects_the_wrong_actual_namespace assert!(std::fs::read(&prior).unwrap() == b"immutable prior generation"); } +#[tokio::test] +async fn session_host_claude_candidate_private_helper_checks_actual_identity_before_request() { + assert_ne!(unsafe { libc::geteuid() }, 0); + let owned = tempfile::tempdir().unwrap(); + let prior = owned.path().join("accepted-prior-inputs"); + std::fs::write(&prior, b"immutable prior generation").unwrap(); + let mut child = tokio::process::Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("project-claude-candidate-private") + .current_dir(owned.path()) + .env_clear() + .env("HOME", owned.path()) + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .kill_on_drop(true) + .spawn() + .unwrap(); + let mut output = child.stdout.take().unwrap(); + let mut error = child.stderr.take().unwrap(); + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let completed = tokio::time::timeout(Duration::from_secs(10), async { + tokio::try_join!( + output.read_to_end(&mut stdout), + error.read_to_end(&mut stderr), + child.wait() + ) + }) + .await; + if !matches!(completed, Ok(Ok(_))) { + child.kill().await.unwrap(); + tokio::time::timeout(Duration::from_secs(5), child.wait()) + .await + .unwrap() + .unwrap(); + panic!("owned candidate helper did not complete within its finite control deadline"); + } + let (_, _, status) = completed.unwrap().unwrap(); + assert!(!status.success()); + assert!( + stdout.is_empty(), + "wrong actual identity must not emit candidate evidence" + ); + let stderr = String::from_utf8(stderr).unwrap(); + assert!( + stderr.contains("candidate private helper namespace mapping is unavailable"), + "{stderr}" + ); + assert_eq!(std::fs::read(prior).unwrap(), b"immutable prior generation"); +} + #[tokio::test] async fn session_host_claude_observer_returns_bound_metadata_without_consuming_settings() { let owned = tempfile::tempdir().unwrap(); From fa3a8079fbe4410a482fd96efee8152a851fc00c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 01:50:57 -0700 Subject: [PATCH 119/493] Bind initial Claude candidate publication to issuer ownership --- crates/freshell-runtime-protocol/src/lib.rs | 44 +++ .../src/managed_mcp_capability.rs | 346 +++++++++++++++++- .../src/managed_provider_references/claude.rs | 2 + 3 files changed, 391 insertions(+), 1 deletion(-) diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index e1b440a30..e35d30fd1 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -3043,6 +3043,50 @@ impl ClaudeCandidateProjectionRequest { } impl ObserveClaudeSourcesRequest { + pub fn candidate_projection( + &self, + publication: ClaudeCandidatePublication, + ) -> Result { + use sha2::Digest; + self.validate()?; + let unavailable = || { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "Claude candidate publication binding unavailable; retain inputs and reprepare", + ) + }; + let candidate = self.candidate.as_ref().ok_or_else(unavailable)?; + let authority = bounded_json_bytes(&self.authority, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| unavailable())?; + let result = + bounded_json_bytes(candidate, MAX_CONTROL_FRAME_BYTES).map_err(|_| unavailable())?; + if publication.soul_id != self.soul_id + || publication.provider != self.provider + || publication.workspace_path != self.workspace_path + || publication.git_common_dir != self.git_common_dir + || publication.authority_sha256 != format!("{:x}", Sha256::digest(authority)) + || publication.prepared_result_sha256 != format!("{:x}", Sha256::digest(result)) + || match &self.authority { + ClaudeSourceAuthority::InitialReservation { reservation_id } => { + publication.initial_reservation_id.as_ref() != Some(reservation_id) + } + ClaudeSourceAuthority::RecoveryAttempt { .. } => { + publication.initial_reservation_id.is_some() + } + } + { + return Err(unavailable()); + } + let projection = ClaudeCandidateProjectionRequest { + source_request: self.observation_request.clone(), + workspace_path: self.workspace_path.clone(), + provider_launch_context: candidate.provider_launch_context.clone(), + publication, + }; + projection.validate()?; + Ok(projection) + } + pub fn validate(&self) -> Result<(), RuntimeError> { let invalid = || { RuntimeError::new(RuntimeErrorCode::InvalidRequest, diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e504893f7..4bb9bf9f5 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -44,6 +44,8 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub claude_observation: Option<&'a freshell_platform::claude_inputs::observation::NativeObservation>, pub claude_inline_settings: Option<&'a str>, + pub claude_reservation_provider: Option<&'a str>, + pub claude_git_common_dir: Option<&'a Path>, pub amplifier_manifest: Option<&'a freshell_platform::amplifier_inputs::AmplifierInputManifest>, pub inline_config: Option<&'a str>, pub tui_config_path: Option<&'a str>, @@ -1883,6 +1885,121 @@ fn publish_prepared_context_metadata( Ok(()) } +fn publish_claude_candidate_binding( + root: &Path, + soul: &SoulId, + provider: &str, + context: &freshell_runtime_protocol::ProviderLaunchContext, + input: OpencodeEphemeralInput<'_>, +) -> Result<(), String> { + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeCandidatePublication, ClaudeSourceAuthority, + PreparedProviderContext, MAX_CONTROL_FRAME_BYTES, + }; + if provider != "claude" { + return Ok(()); + } + let Some(manifest) = input.claude_manifest else { + return Ok(()); + }; + let Some(plan) = &manifest.accepted_native_plan else { + return Ok(()); + }; + let unavailable = + || "Claude candidate issuer authority unavailable; retain inputs and reprepare".to_string(); + let reservation = input.initial_auth_reservation.ok_or_else(unavailable)?; + if reservation.len() != 36 || uuid::Uuid::parse_str(reservation).is_err() { + return Err(unavailable()); + } + let flavor = input.claude_reservation_provider.unwrap_or(provider); + if !matches!(flavor, "claude" | "kilroy") + || (flavor == "kilroy" + && plan.observation.native_source_runtime + != freshell_platform::claude_inputs::NativeSourceRuntime::Sdk) + || manifest.native_source_runtime != Some(plan.observation.native_source_runtime) + || plan.observation.setting_sources != manifest.setting_sources + { + return Err(unavailable()); + } + let inputs = context.prepared_inputs.as_ref().ok_or_else(unavailable)?; + if inputs.initial_auth_reservation.as_deref() != Some(reservation) + || inputs.indexed_files.is_some() + || inputs.manifest.source_count != 0 + || inputs.manifest.alias_count != 0 + || inputs.manifest.recipe_count != 0 + { + return Err(unavailable()); + } + let selectors = bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024) + .map_err(|_| unavailable())?; + if format!("{:x}", Sha256::digest(selectors)) != plan.selector_manifest_sha256 { + return Err(unavailable()); + } + let workspace = std::fs::canonicalize(input.workspace.ok_or_else(unavailable)?) + .map_err(|_| unavailable())?; + let cwd = + std::fs::canonicalize(input.cwd.ok_or_else(unavailable)?).map_err(|_| unavailable())?; + if cwd != plan.observation.cwd || !cwd.starts_with(&workspace) { + return Err(unavailable()); + } + let authority = ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation.into(), + }; + let result = PreparedProviderContext { + provider_launch_context: context.clone(), + provider_secret_references: input.secret_references.ok_or_else(unavailable)?.to_vec(), + }; + result.validate("claude").map_err(|_| unavailable())?; + let publication = ClaudeCandidatePublication { + version: 1, + soul_id: soul.clone(), + provider: flavor.into(), + authority_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| unavailable())? + ) + ), + initial_reservation_id: Some(reservation.into()), + registry_attempt_id: None, + intent_revision: 1, + prior_generation_id: None, + generation_id: inputs.generation_id.clone(), + manifest_sha256: inputs.manifest.sha256.clone(), + prepared_context_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(context, MAX_CONTROL_FRAME_BYTES).map_err(|_| unavailable())? + ) + ), + prepared_result_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&result, MAX_CONTROL_FRAME_BYTES).map_err(|_| unavailable())? + ) + ), + selector_manifest_sha256: plan.selector_manifest_sha256.clone(), + plan_revision: plan.plan_revision, + native_source_runtime: plan.observation.native_source_runtime, + workspace_path: workspace.to_str().ok_or_else(unavailable)?.into(), + cwd, + git_common_dir: input + .claude_git_common_dir + .map(|path| { + std::fs::canonicalize(path) + .map_err(|_| unavailable()) + .and_then(|path| path.to_str().map(str::to_owned).ok_or_else(unavailable)) + }) + .transpose()?, + }; + let bytes = + bounded_json_bytes(&publication, MAX_CONTROL_FRAME_BYTES).map_err(|_| unavailable())?; + let path = root.join(".freshell/claude-candidate-binding.json"); + write_private_bytes(&path, &bytes)?; + allow_controller_stage_read(&path) +} + impl CapabilityStore { fn renew_active_at(&self, now: u64, active: &HashSet<(SoulId, IncarnationId)>) { for grant in self.grants.lock().unwrap().values_mut() { @@ -2003,7 +2120,16 @@ impl CapabilityStore { provider, &context, opencode_input.secret_references, - ); + ) + .and_then(|()| { + publish_claude_candidate_binding( + root, + &soul_id, + provider, + &context, + opencode_input, + ) + }); if let Err(error) = result { let _ = std::fs::remove_dir_all(root); return Err(error); @@ -3227,6 +3353,224 @@ except Exception as error: ); } + #[cfg(unix)] + #[test] + fn managed_provider_claude_issuer_publication_gates_owned_dependency_consumer() { + use freshell_platform::claude_inputs::{ + observation::{observe, CandidateBinding, ObservationRequest}, + AcceptedNativeSourcePlan, ClaudeInputManifest, ClaudeSettingSource, + NativeSourceRuntime, + }; + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeCandidatePublication, ClaudeSourceAuthority, + ObserveClaudeSourcesRequest, PreparedProviderContext, ProviderLaunchContext, + ProviderPreparation, MAX_CONTROL_FRAME_BYTES, + }; + for (flavor, runtime) in [ + ("claude", NativeSourceRuntime::Terminal), + ("claude", NativeSourceRuntime::Sdk), + ("kilroy", NativeSourceRuntime::Sdk), + ] { + let owned = tempfile::tempdir().unwrap(); + let directory = owned.path().join("capabilities"); + let home = owned.path().join("original-home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir(&directory).unwrap(); + std::fs::create_dir(&workspace).unwrap(); + let skill = home.join(".claude/skills/probe"); + std::fs::create_dir_all(skill.join("node_modules/selected-dependency")).unwrap(); + std::fs::write( + skill.join("node_modules/selected-dependency/index.js"), + "module.exports='issued-frozen-dependency'", + ) + .unwrap(); + std::fs::write( + skill.join("consumer.cjs"), + "process.stdout.write(require('selected-dependency'))", + ) + .unwrap(); + let mut manifest: ClaudeInputManifest = serde_json::from_value(serde_json::json!({"version":1,"nativeSourceRuntime":runtime,"originalHome":home,"cwd":workspace,"workspace":workspace,"configRoot":home.join(".claude"),"settingSources":["user"],"pluginDirs":[]})).unwrap(); + let selector = format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024).unwrap() + ) + ); + let observation = + observe(&workspace, &home, runtime, &[ClaudeSettingSource::User]).unwrap(); + manifest.accepted_native_plan = Some(AcceptedNativeSourcePlan { + selector_manifest_sha256: selector.clone(), + plan_revision: 0, + observation: observation.clone(), + }); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let reservation = uuid::Uuid::new_v4().to_string(); + let capability = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("claude", &home, &[])), + OpencodeEphemeralInput { + claude_manifest: Some(&manifest), + claude_observation: Some(&observation), + claude_reservation_provider: Some(flavor), + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + secret_references: Some(&[]), + initial_auth_reservation: Some(&reservation), + ..Default::default() + }, + None, + ) + .unwrap(); + let context = read_prepared_context(&directory, &capability).unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let published = std::fs::read(stage.join(".freshell/claude-candidate-binding.json")) + .expect("issued candidate must publish its ownership binding before any consumer can be selected"); + let publication: ClaudeCandidatePublication = + serde_json::from_slice(&published).unwrap(); + let ready = PreparedProviderContext { + provider_launch_context: context.clone(), + provider_secret_references: vec![], + }; + let request = ObserveClaudeSourcesRequest { + expected_control_epoch: None, + soul_id: soul, + provider: flavor.into(), + authority: ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation, + }, + workspace_path: workspace.to_str().unwrap().into(), + git_common_dir: None, + candidate: Some(ready.clone()), + observation_request: ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: runtime, + cwd: workspace.clone(), + home: "/home/freshell/provider".into(), + setting_sources: vec![ClaudeSettingSource::User], + selector_manifest_sha256: selector, + plan_revision: 0, + candidate_binding: Some(CandidateBinding { + soul_id: publication.soul_id.as_str().into(), + generation_id: capability.grant_id.clone(), + authority_sha256: publication.authority_sha256.clone(), + publication_sha256: format!("{:x}", Sha256::digest(&published)), + manifest_sha256: context + .prepared_inputs + .as_ref() + .unwrap() + .manifest + .sha256 + .clone(), + prepared_result_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&ready, MAX_CONTROL_FRAME_BYTES).unwrap() + ) + ), + }), + }, + }; + for fault in [ + "soul", + "reservation", + "authority-purpose", + "flavor", + "common-dir", + "result", + ] { + let mut wrong = request.clone(); + match fault { + "soul" => wrong.soul_id = SoulId::new(), + "reservation" => { + wrong.authority = ClaudeSourceAuthority::InitialReservation { + reservation_id: uuid::Uuid::new_v4().to_string(), + } + } + "authority-purpose" => { + wrong.authority = ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: uuid::Uuid::new_v4().to_string(), + } + } + "flavor" => { + wrong.provider = if flavor == "claude" { + "kilroy" + } else { + "claude" + } + .into() + } + "common-dir" => { + wrong.git_common_dir = Some( + workspace + .join("different-common-dir") + .to_str() + .unwrap() + .into(), + ) + } + "result" => { + wrong + .candidate + .as_mut() + .unwrap() + .provider_launch_context + .mcp_capability + .as_mut() + .unwrap() + .endpoint = "http://host.docker.internal:4999".into() + } + _ => unreachable!(), + } + wrong + .validate() + .expect("coherent outer substitutions must reach the ownership/digest gate"); + assert!( + wrong.candidate_projection(publication.clone()).is_err(), + "changed {fault} must block before selecting an executable consumer" + ); + } + let projection = request + .candidate_projection(publication) + .expect("published owner binding must permit the owned ordinary consumer"); + std::fs::remove_dir_all(&home).unwrap(); + let selected = projection + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap() + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == "/home/freshell/provider/.claude/skills") + .expect("selected skills must retain their provider destination"); + let output = std::process::Command::new("node") + .arg( + stage + .join(&selected.stage_relative_path) + .join("probe/consumer.cjs"), + ) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .current_dir(&workspace) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"issued-frozen-dependency"); + assert!(!home.exists()); + } + } + #[cfg(unix)] #[test] fn managed_provider_claude_complete_stage_carries_project_resources_and_dependencies() { diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 0d74f51c3..87404f3df 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -181,6 +181,8 @@ pub(crate) async fn prepare_managed( claude_manifest: Some(&manifest), claude_observation: Some(&observation.observation), claude_inline_settings: inline_settings, + claude_reservation_provider: Some(request.reservation_provider), + claude_git_common_dir: request.git_common_dir, cwd: Some(request.cwd), workspace: Some(request.workspace), secret_references: Some(request.references), From 8393db3fb6a45c434d0e01a780d2c0a09930bdab Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 02:39:40 -0700 Subject: [PATCH 120/493] Bind Claude recovery publication to the saved source attempt --- crates/freshell-runtime-protocol/src/lib.rs | 40 +++ .../src/managed_mcp_capability.rs | 279 +++++++++++++++--- crates/freshell-supervisor/src/backend.rs | 1 + .../src/backend/claude_source_observation.rs | 2 +- .../src/claude_source_observation.rs | 128 +++++++- crates/freshell-supervisor/src/service.rs | 78 +++++ 6 files changed, 467 insertions(+), 61 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index e35d30fd1..0a669d187 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2948,6 +2948,46 @@ pub struct ClaudeCandidatePublication { pub git_common_dir: Option, } +/// Finite provenance supplied by the authenticated recovery callback owner. +/// The scope key itself is confined to the existing private callback channel. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ClaudeRecoverySourceBinding { + pub provider: String, + pub git_common_dir: Option, + pub authority_sha256: String, + pub registry_attempt_id: RecoveryAttemptId, + pub intent_revision: u64, + pub prior_generation_id: String, +} + +impl ClaudeRecoverySourceBinding { + pub fn validate(&self) -> Result<(), RuntimeError> { + if !matches!(self.provider.as_str(), "claude" | "kilroy") + || self.authority_sha256.len() != 64 + || !self + .authority_sha256 + .bytes() + .all(|byte| byte.is_ascii_hexdigit()) + || RecoveryAttemptId::parse(self.registry_attempt_id.as_str()).is_err() + || self.intent_revision == 0 + || !self.prior_generation_id.starts_with("grant-") + || uuid::Uuid::parse_str(&self.prior_generation_id[6..]).is_err() + || self.git_common_dir.as_ref().is_some_and(|path| { + path.len() > 64 * 1024 + || path.chars().any(char::is_control) + || !std::path::Path::new(path).is_absolute() + }) + { + return Err(RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "Claude recovery source binding is unavailable; retain inputs and reprepare", + )); + } + Ok(()) + } +} + impl ClaudeCandidateProjectionRequest { pub fn validate(&self) -> Result<(), RuntimeError> { use sha2::Digest; diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 4bb9bf9f5..42d5e8bb7 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -46,6 +46,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub claude_inline_settings: Option<&'a str>, pub claude_reservation_provider: Option<&'a str>, pub claude_git_common_dir: Option<&'a Path>, + pub claude_recovery_source: Option<&'a freshell_runtime_protocol::ClaudeRecoverySourceBinding>, pub amplifier_manifest: Option<&'a freshell_platform::amplifier_inputs::AmplifierInputManifest>, pub inline_config: Option<&'a str>, pub tui_config_path: Option<&'a str>, @@ -270,6 +271,7 @@ struct CapabilityCallback { tui_source: Option, workspace: Option, auth_lookup_authority: Option, + claude_recovery_source: Option, cwd: Option, provider_secret_references: Option>, prior_context: Option, @@ -298,6 +300,40 @@ async fn apply_prepared_callback( .provider .as_deref() .ok_or("missing managed MCP provider")?; + if let Some(binding) = callback.claude_recovery_source.as_ref() { + binding + .validate() + .map_err(|_| "Claude callback source provenance unavailable")?; + let authority = freshell_runtime_protocol::ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: callback + .auth_lookup_authority + .as_ref() + .ok_or("Claude callback source scope unavailable")? + .clone(), + }; + let prior = callback + .prior_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + .ok_or("Claude callback prior generation unavailable")?; + if provider != "claude" + || prior.generation_id != binding.prior_generation_id + || format!( + "{:x}", + Sha256::digest( + freshell_runtime_protocol::bounded_json_bytes( + &authority, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + ) + .map_err(|_| "Claude callback source scope profile exceeded")? + ) + ) != binding.authority_sha256 + { + return Err( + "Claude callback source provenance changed; retain inputs and reprepare".into(), + ); + } + } if let Some(reply) = store.pending_prepared_context( &callback.soul_id, directory, @@ -383,6 +419,16 @@ async fn apply_prepared_callback( workspace: Some(&manifest.workspace), claude_manifest: Some(manifest), claude_inline_settings: inline_settings.as_deref(), + claude_reservation_provider: callback + .claude_recovery_source + .as_ref() + .map(|binding| binding.provider.as_str()), + claude_recovery_source: callback.claude_recovery_source.as_ref(), + claude_git_common_dir: callback + .claude_recovery_source + .as_ref() + .and_then(|binding| binding.git_common_dir.as_deref()) + .map(Path::new), ..Default::default() }, ), @@ -1907,10 +1953,6 @@ fn publish_claude_candidate_binding( }; let unavailable = || "Claude candidate issuer authority unavailable; retain inputs and reprepare".to_string(); - let reservation = input.initial_auth_reservation.ok_or_else(unavailable)?; - if reservation.len() != 36 || uuid::Uuid::parse_str(reservation).is_err() { - return Err(unavailable()); - } let flavor = input.claude_reservation_provider.unwrap_or(provider); if !matches!(flavor, "claude" | "kilroy") || (flavor == "kilroy" @@ -1922,7 +1964,7 @@ fn publish_claude_candidate_binding( return Err(unavailable()); } let inputs = context.prepared_inputs.as_ref().ok_or_else(unavailable)?; - if inputs.initial_auth_reservation.as_deref() != Some(reservation) + if inputs.initial_auth_reservation.as_deref() != input.initial_auth_reservation || inputs.indexed_files.is_some() || inputs.manifest.source_count != 0 || inputs.manifest.alias_count != 0 @@ -1942,8 +1984,55 @@ fn publish_claude_candidate_binding( if cwd != plan.observation.cwd || !cwd.starts_with(&workspace) { return Err(unavailable()); } - let authority = ClaudeSourceAuthority::InitialReservation { - reservation_id: reservation.into(), + let ( + authority_sha256, + initial_reservation_id, + registry_attempt_id, + intent_revision, + prior_generation_id, + ) = match (input.initial_auth_reservation, input.claude_recovery_source) { + (Some(reservation), None) => { + if reservation.len() != 36 || uuid::Uuid::parse_str(reservation).is_err() { + return Err(unavailable()); + } + let authority = ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation.into(), + }; + ( + format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| unavailable())? + ) + ), + Some(reservation.into()), + None, + 1, + None, + ) + } + (None, Some(recovery)) => { + recovery.validate().map_err(|_| unavailable())?; + let prior = input + .launch_context + .and_then(|context| context.prepared_inputs.as_ref()) + .ok_or_else(unavailable)?; + if prior.generation_id != recovery.prior_generation_id + || inputs.generation_id == recovery.prior_generation_id + || recovery.provider != flavor + { + return Err(unavailable()); + } + ( + recovery.authority_sha256.clone(), + None, + Some(recovery.registry_attempt_id.clone()), + recovery.intent_revision, + Some(recovery.prior_generation_id.clone()), + ) + } + _ => return Err(unavailable()), }; let result = PreparedProviderContext { provider_launch_context: context.clone(), @@ -1954,17 +2043,11 @@ fn publish_claude_candidate_binding( version: 1, soul_id: soul.clone(), provider: flavor.into(), - authority_sha256: format!( - "{:x}", - Sha256::digest( - bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES) - .map_err(|_| unavailable())? - ) - ), - initial_reservation_id: Some(reservation.into()), - registry_attempt_id: None, - intent_revision: 1, - prior_generation_id: None, + authority_sha256, + initial_reservation_id, + registry_attempt_id, + intent_revision, + prior_generation_id, generation_id: inputs.generation_id.clone(), manifest_sha256: inputs.manifest.sha256.clone(), prepared_context_sha256: format!( @@ -2282,15 +2365,31 @@ impl CapabilityStore { }); } let retained = recoverable_input_path(directory, &soul_id)?; - let prior_stage = self - .grants - .lock() - .unwrap() - .iter() - .find(|(_, grant)| grant.soul_id == soul_id) - .map(|(grant_id, _)| staged_provider_root_path(directory, grant_id)) - .transpose()? - .or_else(|| retained.is_dir().then_some(retained)); + let prior_stage = if let Some(recovery) = input.claude_recovery_source { + recovery.validate().map_err(|error| error.message)?; + if !self + .grants + .lock() + .unwrap() + .get(&recovery.prior_generation_id) + .is_some_and(|grant| grant.soul_id == soul_id) + { + return Err("Claude exact prior generation ownership unavailable; retain inputs and reprepare".into()); + } + Some(staged_provider_root_path( + directory, + &recovery.prior_generation_id, + )?) + } else { + self.grants + .lock() + .unwrap() + .iter() + .find(|(_, grant)| grant.soul_id == soul_id) + .map(|(grant_id, _)| staged_provider_root_path(directory, grant_id)) + .transpose()? + .or_else(|| retained.is_dir().then_some(retained)) + }; let replacement = self.issue( soul_id.clone(), directory, @@ -3544,29 +3643,111 @@ except Exception as error: let projection = request .candidate_projection(publication) .expect("published owner binding must permit the owned ordinary consumer"); + let recovery_authority = ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: uuid::Uuid::new_v4().to_string(), + }; + let recovery_binding = freshell_runtime_protocol::ClaudeRecoverySourceBinding { + provider: flavor.into(), + git_common_dir: None, + authority_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&recovery_authority, MAX_CONTROL_FRAME_BYTES).unwrap() + ) + ), + registry_attempt_id: freshell_runtime_protocol::RecoveryAttemptId::new(), + intent_revision: 7, + prior_generation_id: capability.grant_id.clone(), + }; + let mut revised = manifest.clone(); + revised.accepted_native_plan.as_mut().unwrap().plan_revision = 1; + let replacement = store.issue_replacement_with_inputs( + request.soul_id.clone(),&directory,"http://host.docker.internal:4000".into(),Some(("claude",&home,&[])),OpencodeEphemeralInput { + claude_manifest:Some(&revised),claude_observation:Some(&observation),claude_reservation_provider:Some(flavor),claude_recovery_source:Some(&recovery_binding),cwd:Some(&workspace),workspace:Some(&workspace),launch_context:Some(&context),secret_references:Some(&[]),..Default::default() + } + ).expect("current recovery provenance must permit publication without replacing the prior lease"); + assert_ne!(replacement.grant_id, capability.grant_id); + assert!( + store + .grants + .lock() + .unwrap() + .contains_key(&capability.grant_id), + "prior lease must survive candidate issuance" + ); + let replacement_context = read_prepared_context(&directory, &replacement).unwrap(); + let replacement_stage = + staged_provider_root_path(&directory, &replacement.grant_id).unwrap(); + let replacement_publication_bytes = + std::fs::read(replacement_stage.join(".freshell/claude-candidate-binding.json")) + .unwrap(); + let replacement_publication: ClaudeCandidatePublication = + serde_json::from_slice(&replacement_publication_bytes).unwrap(); + let replacement_ready = PreparedProviderContext { + provider_launch_context: replacement_context.clone(), + provider_secret_references: vec![], + }; + let mut replacement_request = request.clone(); + replacement_request.authority = recovery_authority; + replacement_request.candidate = Some(replacement_ready.clone()); + replacement_request.observation_request.plan_revision = 1; + replacement_request.observation_request.candidate_binding = Some(CandidateBinding { + soul_id: request.soul_id.as_str().into(), + generation_id: replacement.grant_id.clone(), + authority_sha256: recovery_binding.authority_sha256.clone(), + publication_sha256: format!("{:x}", Sha256::digest(&replacement_publication_bytes)), + manifest_sha256: replacement_context + .prepared_inputs + .as_ref() + .unwrap() + .manifest + .sha256 + .clone(), + prepared_result_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&replacement_ready, MAX_CONTROL_FRAME_BYTES).unwrap() + ) + ), + }); + let mut stale_scope = replacement_request.clone(); + stale_scope.authority = ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: uuid::Uuid::new_v4().to_string(), + }; + assert!(stale_scope + .candidate_projection(replacement_publication.clone()) + .is_err()); + let replacement_projection = replacement_request.candidate_projection(replacement_publication).expect("replacement publication must bind its current authority and exact prior generation"); std::fs::remove_dir_all(&home).unwrap(); - let selected = projection - .provider_launch_context - .prepared_inputs - .as_ref() - .unwrap() - .resource_mounts - .iter() - .find(|mount| mount.logical_destination == "/home/freshell/provider/.claude/skills") - .expect("selected skills must retain their provider destination"); - let output = std::process::Command::new("node") - .arg( - stage - .join(&selected.stage_relative_path) - .join("probe/consumer.cjs"), - ) - .env_clear() - .env("PATH", "/usr/local/bin:/usr/bin:/bin") - .current_dir(&workspace) - .output() - .unwrap(); - assert!(output.status.success()); - assert_eq!(output.stdout, b"issued-frozen-dependency"); + for (stage, projection) in [ + (&stage, &projection), + (&replacement_stage, &replacement_projection), + ] { + let selected = projection + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap() + .resource_mounts + .iter() + .find(|mount| { + mount.logical_destination == "/home/freshell/provider/.claude/skills" + }) + .expect("selected skills must retain their provider destination"); + let output = std::process::Command::new("node") + .arg( + stage + .join(&selected.stage_relative_path) + .join("probe/consumer.cjs"), + ) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .current_dir(&workspace) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"issued-frozen-dependency"); + } assert!(!home.exists()); } } diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 0c15db5e3..3694ece91 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -130,6 +130,7 @@ pub enum ClaudeSourceOwner { }, Retained { handle: OwnedRuntimeHandle, + recovery_source: Option, }, } diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index c0a1ad274..d82396ad3 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -120,7 +120,7 @@ impl DockerEngineBackend { None, ) } - ClaudeSourceOwner::Retained { handle } => { + ClaudeSourceOwner::Retained { handle, .. } => { let provider = handle .terminal() .map(|spec| spec.mode.as_str()) diff --git a/crates/freshell-supervisor/src/claude_source_observation.rs b/crates/freshell-supervisor/src/claude_source_observation.rs index 0da9877c0..30df4749f 100644 --- a/crates/freshell-supervisor/src/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/claude_source_observation.rs @@ -76,8 +76,29 @@ impl Supervisor { return Err(denied()); } let handle = authority.start.context.prior_handle.clone(); + let recovery_source = if request.candidate.is_some() { + let context = handle + .terminal() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + handle + .fresh_agent() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }) + .ok_or_else(denied)?; + Some(self.claude_recovery_source_binding( + &authority.start, + attempt_id, + context, + )?) + } else { + None + }; recovery = Some(authority.start); - ClaudeSourceOwner::Retained { handle } + ClaudeSourceOwner::Retained { + handle, + recovery_source, + } } }; let raw = freshell_runtime_protocol::bounded_json_bytes( @@ -166,7 +187,11 @@ mod tests { }; use freshell_runtime_protocol::*; - struct ReplyBackend(StdMutex>>); + struct ReplyBackend( + StdMutex>>, + StdMutex>, + std::sync::atomic::AtomicUsize, + ); #[async_trait] impl RuntimeBackend for ReplyBackend { async fn observe_claude_sources( @@ -175,10 +200,31 @@ mod tests { request: &ObserveClaudeSourcesRequest, _: &str, ) -> Result { - let ClaudeSourceOwner::Retained { handle } = owner else { + self.2.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let ClaudeSourceOwner::Retained { + handle, + recovery_source, + } = owner + else { panic!("requires retained fixture owner"); }; assert_eq!(handle.soul_id(), &request.soul_id); + if let Some(expected) = self.1.lock().unwrap().as_ref() { + let supplied = recovery_source.as_ref().ok_or_else(|| { + BackendError::Unavailable("current registry source provenance missing".into()) + })?; + if supplied.provider != expected.provider + || supplied.registry_attempt_id != expected.registry_attempt_id + || supplied.intent_revision != expected.intent_revision + || supplied.prior_generation_id != expected.prior_generation_id + || supplied.authority_sha256 != expected.authority_sha256 + || supplied.git_common_dir != expected.git_common_dir + { + return Err(BackendError::Unavailable( + "current registry source provenance changed".into(), + )); + } + } let action = self.0.lock().unwrap().take(); if let Some(action) = action { action(); @@ -249,7 +295,15 @@ mod tests { #[tokio::test] async fn provider_claude_nested_source_reply_requires_live_scope_after_private_ipc() { - for mode in ["live", "dropped", "expired"] { + for mode in [ + "live", + "dropped", + "expired", + "candidate", + "candidate-dropped", + "candidate-expired", + "candidate-unrecorded-prior", + ] { let owned = tempfile::tempdir().unwrap(); let binary = owned.path().join("host"); std::fs::write(&binary, b"synthetic helper artifact").unwrap(); @@ -263,6 +317,15 @@ mod tests { swap_bytes: 0, pids_max: 32, }; + let prior_generation = format!("grant-{}", uuid::Uuid::new_v4()); + let prior_context:ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"claude":{"mcp_args":[]}},"config":[], + "mcpCapability":{"grantId":prior_generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":prior_generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":0,"aliasCount":0,"recipeCount":0}} + })).unwrap(); + let terminal = (mode.starts_with("candidate")).then(||serde_json::from_value::(serde_json::json!({ + "terminalId":"source-provenance","streamId":"source-stream","mode":"claude","program":"claude","args":[],"cwd":owned.path(),"workspacePath":owned.path(),"runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"synthetic-workspace","providerLaunchContext":if mode=="candidate-unrecorded-prior" {None} else {Some(&prior_context)} + })).unwrap()); let prepared = registry .prepare_launch(LaunchPreparation { soul_id: soul.clone(), @@ -276,7 +339,7 @@ mod tests { profile: RuntimeProfile::Custom, project_key: "synthetic-workspace".into(), fixture: None, - terminal: None, + terminal, fresh_agent: None, view_intent: None, admission: crate::admission::AdmissionPolicy::default(), @@ -306,7 +369,11 @@ mod tests { ) .await .unwrap(); - let backend = Arc::new(ReplyBackend(StdMutex::new(None))); + let backend = Arc::new(ReplyBackend( + StdMutex::new(None), + StdMutex::new(None), + std::sync::atomic::AtomicUsize::new(0), + )); let supervisor = Supervisor::new( registry, backend.clone(), @@ -325,19 +392,22 @@ mod tests { let lock = supervisor.lifecycle_lock(&soul).await; let _held = lock.lock().await; let scope = supervisor.auth_lookup_scope(&start); - if mode != "live" { + if matches!( + mode, + "dropped" | "expired" | "candidate-dropped" | "candidate-expired" + ) { let authorities = supervisor.auth_lookup_authorities.clone(); let key = scope.key.clone(); *backend.0.lock().unwrap() = Some(Box::new(move || { let mut map = authorities.lock().unwrap(); - if mode == "dropped" { + if mode.ends_with("dropped") { map.remove(&key); } else { map.get_mut(&key).unwrap().expires = Instant::now(); } })); } - let request: ObserveClaudeSourcesRequest=serde_json::from_value(serde_json::json!({ + let mut request: ObserveClaudeSourcesRequest=serde_json::from_value(serde_json::json!({ "expectedControlEpoch":supervisor.registry.control_epoch(),"soulId":soul,"provider":"claude", "authority":{"kind":"recovery_attempt","attemptId":scope.key}, "workspacePath":owned.path(),"gitCommonDir":null,"candidate":null, @@ -345,6 +415,41 @@ mod tests { "cwd":owned.path(),"home":"/home/freshell/provider","settingSources":["local"], "selectorManifestSha256":"b".repeat(64),"planRevision":0} })).unwrap(); + if mode.starts_with("candidate") { + let source_binding = ClaudeRecoverySourceBinding { + provider: "claude".into(), + git_common_dir: None, + authority_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&request.authority, MAX_CONTROL_FRAME_BYTES) + .unwrap() + ) + ), + registry_attempt_id: start.attempt_id.clone(), + intent_revision: start.context.intent_revision, + prior_generation_id: prior_generation, + }; + let mut candidate = prior_context.clone(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + candidate.mcp_capability.as_mut().unwrap().grant_id = generation.clone(); + candidate.prepared_inputs.as_mut().unwrap().generation_id = generation.clone(); + request.candidate = Some(PreparedProviderContext { + provider_launch_context: candidate, + provider_secret_references: vec![], + }); + request.observation_request.candidate_binding = Some( + freshell_platform::claude_inputs::observation::CandidateBinding { + soul_id: soul.as_str().into(), + generation_id: generation, + authority_sha256: source_binding.authority_sha256.clone(), + publication_sha256: "c".repeat(64), + manifest_sha256: "a".repeat(64), + prepared_result_sha256: "d".repeat(64), + }, + ); + *backend.1.lock().unwrap() = Some(source_binding); + } let mut envelope = Envelope::new( RequestId::new(), ControlRole::Web, @@ -358,10 +463,11 @@ mod tests { .await .expect("nested Claude source lookup must not reacquire the held soul lifecycle lock") .result; + assert_eq!(backend.2.load(std::sync::atomic::Ordering::SeqCst),usize::from(mode!="candidate-unrecorded-prior"),"unrecorded prior inputs must block before the backend; valid provenance reaches it once"); assert_eq!( result.is_ok(), - mode == "live", - "{mode}: changed scoped authority after IPC cannot qualify source evidence" + matches!(mode,"live"|"candidate"), + "{mode}: changed scoped authority after IPC cannot qualify source evidence: {result:?}" ); if let Err(error) = result { assert_eq!(error.code, RuntimeErrorCode::InvalidRequest); diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index e6a56b058..c2e19336f 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -61,6 +61,7 @@ struct McpCapabilityRequest<'a> { tui_source: Option<&'a freshell_runtime_protocol::ProviderConfigReference>, workspace: Option<&'a str>, auth_lookup_authority: Option<&'a str>, + claude_recovery_source: Option<&'a freshell_runtime_protocol::ClaudeRecoverySourceBinding>, cwd: Option<&'a str>, prior_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, provider_secret_references: Option<&'a [freshell_runtime_protocol::ProviderSecretReference]>, @@ -2005,6 +2006,7 @@ impl Supervisor { tui_source: None, workspace: None, auth_lookup_authority: None, + claude_recovery_source: None, cwd: None, prior_context: None, provider_secret_references: None, @@ -2103,6 +2105,73 @@ impl Supervisor { } } + fn claude_recovery_source_binding( + &self, + start: &crate::registry::RecoveryStart, + scope_key: &str, + context: &freshell_runtime_protocol::ProviderLaunchContext, + ) -> Result { + use sha2::Digest; + let unavailable = || { + RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "Claude prior source generation unavailable; retain inputs and reprepare", + ) + }; + let saved = start + .context + .prior_handle + .terminal() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + start + .context + .prior_handle + .fresh_agent() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }) + .ok_or_else(unavailable)?; + context.validate("claude")?; + let inputs = context.prepared_inputs.as_ref().ok_or_else(unavailable)?; + if saved + .prepared_inputs + .as_ref() + .map(|inputs| inputs.generation_id.as_str()) + != Some(inputs.generation_id.as_str()) + { + return Err(unavailable()); + } + let authority = freshell_runtime_protocol::ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: scope_key.into(), + }; + let raw = freshell_runtime_protocol::bounded_json_bytes( + &authority, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| unavailable())?; + let binding = freshell_runtime_protocol::ClaudeRecoverySourceBinding { + provider: start.context.provider.clone(), + git_common_dir: start + .context + .prior_handle + .terminal() + .and_then(|spec| spec.git_common_dir.clone()) + .or_else(|| { + start + .context + .prior_handle + .fresh_agent() + .and_then(|spec| spec.git_common_dir.clone()) + }), + authority_sha256: format!("{:x}", sha2::Sha256::digest(raw)), + registry_attempt_id: start.attempt_id.clone(), + intent_revision: start.context.intent_revision, + prior_generation_id: inputs.generation_id.clone(), + }; + binding.validate()?; + Ok(binding) + } + pub(crate) async fn issue_replacement_mcp_capability( &self, start: &crate::registry::RecoveryStart, @@ -2119,6 +2188,13 @@ impl Supervisor { _ => None, }; let scope = self.auth_lookup_scope(start); + let claude_recovery_source = if provider == "claude" + && matches!(start.context.provider.as_str(), "claude" | "kilroy") + { + Some(self.claude_recovery_source_binding(start, &scope.key, context)?) + } else { + None + }; let path_inputs = self.bound_provider_path_inputs( &start.context.soul_id, context @@ -2136,6 +2212,7 @@ impl Supervisor { tui_source, workspace: Some(workspace), auth_lookup_authority: Some(&scope.key), + claude_recovery_source: claude_recovery_source.as_ref(), cwd: Some(cwd), prior_context: Some(context), provider_secret_references: Some(references), @@ -2174,6 +2251,7 @@ impl Supervisor { "tuiSource": request.tui_source, "workspace": request.workspace, "authLookupAuthority": request.auth_lookup_authority, + "claudeRecoverySource": request.claude_recovery_source, "cwd":request.cwd, "priorContext":request.prior_context, "providerSecretReferences":request.provider_secret_references, From 84d77a0f654c61fc95cf917e6c7cf71bf1ccf3db Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 02:50:07 -0700 Subject: [PATCH 121/493] Qualify ordinary Claude candidate files before helper creation --- .../src/backend/claude_source_observation.rs | 224 +++++++++++++ .../claude_source_observation_tests.rs | 312 ++++++++++++++++++ .../freshell-supervisor/src/backend/docker.rs | 46 +++ 3 files changed, 582 insertions(+) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index d82396ad3..7a78949d9 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -12,6 +12,219 @@ fn fail(reason: &'static str) -> BackendError { BackendError::Unavailable(reason.into()) } +fn qualify_candidate_stage( + root: &Path, + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, +) -> Result { + use freshell_platform::claude_inputs::ClaudeInputManifest; + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeCandidatePublication, MAX_CONTROL_FRAME_BYTES, + }; + let blocked = + || fail("Claude candidate publication changed or unavailable; retain inputs and reprepare"); + request.validate().map_err(|_| blocked())?; + let binding = request + .observation_request + .candidate_binding + .as_ref() + .ok_or_else(blocked)?; + let publication_bytes = read_candidate_metadata( + root, + ".freshell/claude-candidate-binding.json", + MAX_CONTROL_FRAME_BYTES, + )?; + if format!("{:x}", Sha256::digest(&publication_bytes)) != binding.publication_sha256 { + return Err(blocked()); + } + let publication: ClaudeCandidatePublication = + serde_json::from_slice(&publication_bytes).map_err(|_| blocked())?; + let projection = request + .candidate_projection(publication) + .map_err(|_| blocked())?; + let published = &projection.publication; + match owner { + ClaudeSourceOwner::Initial { reservation, .. } => { + if reservation.soul_id != request.soul_id + || reservation.provider != request.provider + || published.initial_reservation_id.as_deref() != Some(reservation.id.as_str()) + || published.registry_attempt_id.is_some() + || published.prior_generation_id.is_some() + || published.intent_revision != 1 + { + return Err(blocked()); + } + } + ClaudeSourceOwner::Retained { + handle, + recovery_source, + } => { + let expected = recovery_source.as_ref().ok_or_else(blocked)?; + expected.validate().map_err(|_| blocked())?; + let context = handle + .terminal() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + handle + .fresh_agent() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }); + if handle.soul_id() != &request.soul_id + || published.initial_reservation_id.is_some() + || published.provider != expected.provider + || published.git_common_dir != expected.git_common_dir + || published.authority_sha256 != expected.authority_sha256 + || published.registry_attempt_id.as_ref() != Some(&expected.registry_attempt_id) + || published.intent_revision != expected.intent_revision + || published.prior_generation_id.as_deref() + != Some(expected.prior_generation_id.as_str()) + || context + .and_then(|context| context.prepared_inputs.as_ref()) + .map(|inputs| inputs.generation_id.as_str()) + != Some(expected.prior_generation_id.as_str()) + { + return Err(blocked()); + } + } + } + let inputs = projection + .provider_launch_context + .prepared_inputs + .as_ref() + .ok_or_else(blocked)?; + for (relative, digest) in [ + ( + ".freshell/prepared-context.json", + published.prepared_context_sha256.as_str(), + ), + ( + ".freshell/prepared-result.json", + published.prepared_result_sha256.as_str(), + ), + ] { + let bytes = read_candidate_metadata(root, relative, MAX_CONTROL_FRAME_BYTES)?; + if format!("{:x}", Sha256::digest(bytes)) != digest { + return Err(blocked()); + } + } + let manifest_bytes = + read_candidate_metadata(root, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + if format!("{:x}", Sha256::digest(&manifest_bytes)) != published.manifest_sha256 { + return Err(blocked()); + } + let manifest: ClaudeInputManifest = + serde_json::from_slice(&manifest_bytes).map_err(|_| blocked())?; + manifest.validate().map_err(|_| blocked())?; + let plan = manifest.accepted_native_plan.as_ref().ok_or_else(blocked)?; + let selectors = bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024) + .map_err(|_| blocked())?; + let source = &request.observation_request; + if plan.selector_manifest_sha256 != format!("{:x}", Sha256::digest(selectors)) + || plan.selector_manifest_sha256 != source.selector_manifest_sha256 + || plan.plan_revision != source.plan_revision + || manifest.workspace != Path::new(&request.workspace_path) + || manifest.setting_sources != source.setting_sources + || plan.observation.setting_sources != source.setting_sources + || plan.observation.native_source_runtime != source.native_source_runtime + || plan.observation.cwd != source.cwd + || plan.observation.home.as_deref() != Some(source.home.as_path()) + || ( + plan.observation.uid, + plan.observation.euid, + plan.observation.gid, + plan.observation.egid, + ) != (65534, 65534, 0, 0) + { + return Err(blocked()); + } + docker::ordinary_claude_candidate_mounts( + Path::new(&request.workspace_path), + request.git_common_dir.as_deref().map(Path::new), + root, + &projection.provider_launch_context, + ) + .map_err(|_| blocked())?; + Ok(projection) +} + +fn read_candidate_metadata( + root: &Path, + relative: &str, + limit: usize, +) -> Result, BackendError> { + use std::io::Read; + use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; + let blocked = || { + fail("Claude candidate private metadata unavailable or exceeds profile; retain inputs and reprepare") + }; + if !Path::new(relative) + .components() + .all(|part| matches!(part, std::path::Component::Normal(_))) + || canonical_dir(root).map_err(|_| blocked())? != root + { + return Err(blocked()); + } + let path = root.join(relative); + let before = std::fs::symlink_metadata(&path).map_err(|_| blocked())?; + if !before.is_file() + || before.file_type().is_symlink() + || before.len() > limit as u64 + || std::fs::canonicalize(&path).map_err(|_| blocked())? != path + { + return Err(blocked()); + } + let mut file = std::fs::OpenOptions::new() + .read(true) + .custom_flags(libc::O_NOFOLLOW) + .open(&path) + .map_err(|_| blocked())?; + let opened = file.metadata().map_err(|_| blocked())?; + if (opened.dev(), opened.ino(), opened.len()) != (before.dev(), before.ino(), before.len()) { + return Err(blocked()); + } + let mut bytes = Vec::new(); + bytes + .try_reserve_exact(before.len() as usize) + .map_err(|_| blocked())?; + let mut chunk = [0u8; 64 * 1024]; + loop { + let count = file.read(&mut chunk).map_err(|_| blocked())?; + if count == 0 { + break; + } + let next = bytes + .len() + .checked_add(count) + .filter(|length| *length <= limit) + .ok_or_else(blocked)?; + if next > bytes.capacity() { + bytes.try_reserve_exact(count).map_err(|_| blocked())?; + } + bytes.extend_from_slice(&chunk[..count]); + } + let after = std::fs::symlink_metadata(&path).map_err(|_| blocked())?; + if bytes.len() > limit + || after.file_type().is_symlink() + || ( + after.dev(), + after.ino(), + after.len(), + after.mtime(), + after.mtime_nsec(), + ) != ( + before.dev(), + before.ino(), + before.len(), + before.mtime(), + before.mtime_nsec(), + ) + || bytes.len() as u64 != before.len() + { + return Err(blocked()); + } + Ok(bytes) +} + fn owned_helper(actual: &Value, expected: &Value) -> bool { input_publisher::owned_helper(actual, expected) && actual["Config"]["WorkingDir"] == expected["WorkingDir"] @@ -86,6 +299,17 @@ impl DockerEngineBackend { .validate() .map_err(|_| fail("Claude source request invalid"))?; if request.candidate.is_some() { + let grant = request + .candidate + .as_ref() + .and_then(|candidate| candidate.provider_launch_context.mcp_capability.as_ref()) + .ok_or_else(|| { + fail("Claude candidate grant unavailable; retain inputs and reprepare") + })?; + let root = docker::staged_provider_root_for_grant(&grant.grant_id).map_err(|_| { + fail("Claude candidate stage unavailable; retain inputs and reprepare") + })?; + qualify_candidate_stage(&root, owner, request)?; return Err(fail("Claude candidate source topology is not yet qualified; retain inputs and reprepare")); } let name = format!("freshell-claude-source-{}", uuid::Uuid::new_v4()); diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index da4ca1e3d..8a948e02f 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -6,6 +6,318 @@ use freshell_runtime_protocol::{ use std::sync::{Arc, Mutex}; use tokio::{net::UnixListener, task::JoinSet}; +#[test] +fn provider_claude_private_candidate_metadata_enforces_inclusive_limits() { + use std::io::Write; + for limit in [ + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + 64 * 1024 * 1024, + ] { + let owned = tempfile::tempdir().unwrap(); + let file = owned.path().join("metadata"); + let mut writer = std::fs::File::create(&file).unwrap(); + let chunk = [b'x'; 64 * 1024]; + for _ in 0..limit / chunk.len() { + writer.write_all(&chunk).unwrap(); + } + writer.sync_all().unwrap(); + let bytes = read_candidate_metadata(owned.path(), "metadata", limit) + .expect("existing inclusive metadata limit must remain usable"); + assert_eq!(bytes.len(), limit); + assert!(bytes.iter().all(|byte| *byte == b'x')); + drop(bytes); + writer.write_all(b"x").unwrap(); + writer.sync_all().unwrap(); + assert!( + read_candidate_metadata(owned.path(), "metadata", limit).is_err(), + "one byte over cap must block before loading the body" + ); + assert!(read_candidate_metadata(owned.path(), "../metadata", limit).is_err()); + std::fs::rename(&file, owned.path().join("physical")).unwrap(); + std::os::unix::fs::symlink(owned.path().join("physical"), &file).unwrap(); + assert!( + read_candidate_metadata(owned.path(), "metadata", limit + 1).is_err(), + "a link cannot become the qualified private metadata file" + ); + } +} + +#[tokio::test] +async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() { + use freshell_platform::claude_inputs::{ + AcceptedNativeSourcePlan, ClaudeInputManifest, ClaudeSettingSource, NativeSourceRuntime, + }; + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeCandidatePublication, PreparedProviderContext, + ProviderLaunchContext, MAX_CONTROL_FRAME_BYTES, + }; + for (flavor, runtime) in [ + ("claude", NativeSourceRuntime::Terminal), + ("claude", NativeSourceRuntime::Sdk), + ("kilroy", NativeSourceRuntime::Sdk), + ] { + for mode in [ + "valid", + "missing-resource", + "wrong-soul", + "wrong-reservation", + "wrong-owner", + "changed-selectors", + "wrong-plan-uid", + "wrong-home", + "wrong-cwd", + "missing-plan", + "changed-publication", + "changed-result", + "changed-context", + "changed-manifest", + "symlink-publication", + "oversized-publication", + ] { + let owned = tempfile::tempdir().unwrap(); + let workspace = owned.path().join("workspace"); + let original = owned.path().join("original-home"); + let stage = owned.path().join("stage"); + std::fs::create_dir(&workspace).unwrap(); + std::fs::create_dir(&original).unwrap(); + std::fs::create_dir_all(stage.join(".freshell")).unwrap(); + let resource = stage.join(".freshell/resource-units/unit-0/probe"); + std::fs::create_dir_all(resource.join("node_modules/frozen-dependency")).unwrap(); + std::fs::write( + resource.join("node_modules/frozen-dependency/index.js"), + "module.exports='qualified-frozen-dependency'", + ) + .unwrap(); + std::fs::write( + resource.join("consumer.cjs"), + "process.stdout.write(require('frozen-dependency'))", + ) + .unwrap(); + let binary = owned.path().join("synthetic-host"); + std::fs::write(&binary, b"synthetic artifact").unwrap(); + let registry = + crate::registry::Registry::open(owned.path().join("registry"), None).unwrap(); + let soul = SoulId::new(); + let mut reservation = registry + .reserve_initial_inputs( + ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: flavor.into(), + creation_seed_ref: "synthetic-seed".into(), + }, + format!("sha256:{}", "a".repeat(64)), + binary, + ) + .await + .unwrap(); + let mut manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(runtime), + original_home: original.clone(), + cwd: workspace.clone(), + workspace: workspace.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::User], + plugin_dirs: vec![], + accepted_native_plan: None, + }; + let selector = format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024).unwrap() + ) + ); + let mut observation = NativeObservation { + version: 1, + native_source_runtime: runtime, + setting_sources: manifest.setting_sources.clone(), + uid: 65534, + euid: 65534, + gid: 0, + egid: 0, + supplementary_groups: vec![], + cwd: workspace.clone(), + home: Some("/home/freshell/provider".into()), + roots: None, + local_settings: vec![], + local_decision: LocalDecision::Disabled, + root_uid: None, + git_uid: None, + claude_uid: None, + }; + if mode == "wrong-plan-uid" { + observation.uid = 65535; + observation.euid = 65535; + } + if mode == "wrong-home" { + observation.home = Some(original.clone()); + } + if mode == "wrong-cwd" { + std::fs::create_dir(workspace.join("nested")).unwrap(); + observation.cwd = workspace.join("nested"); + } + manifest.accepted_native_plan = Some(AcceptedNativeSourcePlan { + selector_manifest_sha256: selector.clone(), + plan_revision: 0, + observation, + }); + if mode == "changed-selectors" { + manifest.original_home = owned.path().join("other-original-home"); + } + if mode == "missing-plan" { + manifest.accepted_native_plan = None; + } + manifest.validate().expect( + "synthetic fixture must have valid native plan shape before physical qualification", + ); + let manifest_bytes = bounded_json_bytes(&manifest, 64 * 1024 * 1024).unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let reservation_id = if mode == "wrong-reservation" { + uuid::Uuid::new_v4().to_string() + } else { + reservation.id.clone() + }; + let candidate_soul = if mode == "wrong-soul" { + SoulId::new() + } else { + soul + }; + let context:ProviderLaunchContext=serde_json::from_value(json!({ + "preparation":{"claude":{"mcp_args":[]}},"config":[], + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"initialAuthReservation":reservation_id, + "manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&manifest_bytes)),"sourceCount":0,"aliasCount":0,"recipeCount":0}, + "resourceMounts":[{"stageRelativePath":".freshell/resource-units/unit-0","providerRelativePath":".freshell/provider-roots/unit-0","logicalDestination":"/home/freshell/provider/.claude/skills","effectiveDestination":"/home/freshell/provider/.claude/skills","kind":"directory"}]} + })).unwrap(); + let ready = PreparedProviderContext { + provider_launch_context: context.clone(), + provider_secret_references: vec![], + }; + let result_bytes = bounded_json_bytes(&ready, MAX_CONTROL_FRAME_BYTES).unwrap(); + let context_bytes = bounded_json_bytes(&context, MAX_CONTROL_FRAME_BYTES).unwrap(); + let authority = ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation_id.clone(), + }; + let publication = ClaudeCandidatePublication { + version: 1, + soul_id: candidate_soul.clone(), + provider: flavor.into(), + authority_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES).unwrap() + ) + ), + initial_reservation_id: Some(reservation_id), + registry_attempt_id: None, + intent_revision: 1, + prior_generation_id: None, + generation_id: generation.clone(), + manifest_sha256: format!("{:x}", Sha256::digest(&manifest_bytes)), + prepared_context_sha256: format!("{:x}", Sha256::digest(&context_bytes)), + prepared_result_sha256: format!("{:x}", Sha256::digest(&result_bytes)), + selector_manifest_sha256: selector.clone(), + plan_revision: 0, + native_source_runtime: runtime, + workspace_path: workspace.to_str().unwrap().into(), + cwd: workspace.clone(), + git_common_dir: None, + }; + let publication_bytes = + bounded_json_bytes(&publication, MAX_CONTROL_FRAME_BYTES).unwrap(); + let request:ObserveClaudeSourcesRequest=serde_json::from_value(json!({ + "expectedControlEpoch":registry.control_epoch(),"soulId":candidate_soul,"provider":flavor,"authority":authority,"workspacePath":workspace,"gitCommonDir":null,"candidate":ready, + "observationRequest":{"version":1,"requestId":uuid::Uuid::new_v4().to_string(),"nativeSourceRuntime":runtime,"cwd":workspace,"home":"/home/freshell/provider","settingSources":["user"],"selectorManifestSha256":selector,"planRevision":0, + "candidateBinding":{"soulId":candidate_soul,"generationId":generation,"authoritySha256":publication.authority_sha256,"publicationSha256":format!("{:x}",Sha256::digest(&publication_bytes)),"manifestSha256":publication.manifest_sha256,"preparedResultSha256":publication.prepared_result_sha256}} + })).unwrap(); + request.candidate_projection(publication).expect("coherent wire fixture must pass metadata shape checks before physical/owner qualification"); + std::fs::write(stage.join(".freshell/input-manifest.json"), manifest_bytes).unwrap(); + std::fs::write(stage.join(".freshell/prepared-context.json"), context_bytes).unwrap(); + std::fs::write(stage.join(".freshell/prepared-result.json"), result_bytes).unwrap(); + std::fs::write( + stage.join(".freshell/claude-candidate-binding.json"), + publication_bytes, + ) + .unwrap(); + let private = b"PRIVATE_UNUSED_CAPABILITY_SENTINEL"; + std::fs::write(stage.join(".freshell/mcp-capability.json"), private).unwrap(); + std::fs::remove_dir(&original).unwrap(); + if mode == "missing-resource" { + std::fs::remove_dir_all(stage.join(".freshell/resource-units/unit-0")).unwrap(); + } + if mode == "wrong-owner" { + reservation.id = uuid::Uuid::new_v4().to_string(); + } + for (condition, file) in [ + ("changed-publication", "claude-candidate-binding.json"), + ("changed-result", "prepared-result.json"), + ("changed-context", "prepared-context.json"), + ("changed-manifest", "input-manifest.json"), + ] { + if mode == condition { + std::fs::write(stage.join(".freshell").join(file), private).unwrap(); + } + } + if mode == "symlink-publication" { + let path = stage.join(".freshell/claude-candidate-binding.json"); + std::fs::rename(&path, owned.path().join("escaped-publication")).unwrap(); + std::os::unix::fs::symlink(owned.path().join("escaped-publication"), path).unwrap(); + } + if mode == "oversized-publication" { + std::fs::OpenOptions::new() + .write(true) + .open(stage.join(".freshell/claude-candidate-binding.json")) + .unwrap() + .set_len(MAX_CONTROL_FRAME_BYTES as u64 + 1) + .unwrap(); + } + let owner = ClaudeSourceOwner::Initial { + reservation, + observation: InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::new(), + }, + }; + let qualified = qualify_candidate_stage(&stage, &owner, &request); + assert_eq!(qualified.is_ok(),mode=="valid","{flavor}/{runtime:?}/{mode}: physical issuer and current owner must qualify before consumer selection: {:?}",qualified.as_ref().err()); + if let Ok(projection) = qualified { + let selected = &projection + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap() + .resource_mounts[0]; + let output = std::process::Command::new("node") + .arg( + stage + .join(&selected.stage_relative_path) + .join("probe/consumer.cjs"), + ) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .current_dir(&workspace) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"qualified-frozen-dependency"); + } else { + assert!(!qualified + .unwrap_err() + .to_string() + .contains(std::str::from_utf8(private).unwrap())); + } + assert!(!original.exists()); + assert_eq!( + std::fs::read(stage.join(".freshell/mcp-capability.json")).unwrap(), + private + ); + } + } +} + #[tokio::test] async fn provider_claude_source_helper_uses_private_bound_ipc_without_creating_stable_home() { for mode in [ diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 4cb5ab265..1ae29ae0e 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -246,6 +246,52 @@ struct PreparedInputMounts { indexed_files: Option, } +/// The ordinary subset of the workload's existing prepared mount plan. This +/// does not resolve credentials, read a capability file, or consult user HOME. +pub(super) fn ordinary_claude_candidate_mounts( + workspace: &Path, + git_common_dir: Option<&Path>, + stage: &Path, + context: &ProviderLaunchContext, +) -> Result { + context.validate("claude").map_err(|error| error.message)?; + let inputs = context + .prepared_inputs + .as_ref() + .ok_or("Claude candidate ordinary inputs unavailable")?; + if inputs.indexed_files.is_some() + || inputs.manifest.source_count != 0 + || inputs.manifest.alias_count != 0 + || inputs.manifest.recipe_count != 0 + { + return Err("Claude candidate contains unavailable ordinary file selections".into()); + } + if canonical_dir(workspace, "candidate workspace")? != workspace + || canonical_dir(stage, "candidate stage")? != stage + { + return Err("Claude candidate ordinary namespace changed".into()); + } + if let Some(common) = git_common_dir { + if canonical_dir(common, "candidate common directory")? != common { + return Err("Claude candidate ordinary common directory changed".into()); + } + } + let prepared = prepared_input_mounts(workspace, Some(stage), Some(inputs))?; + Ok(TerminalMounts { + workspace: workspace.into(), + git_common_dir: git_common_dir.map(Path::to_path_buf), + provider_bootstrap_files: vec![], + provider_secret_files: vec![], + indexed_secret_sources: Default::default(), + provider_user_root: Some(stage.into()), + mcp_capability_file: None, + input_manifest: prepared.input_manifest, + config_binds: prepared.config_binds, + resource_mounts: prepared.resource_mounts, + indexed_files: None, + }) +} + fn prepared_input_mounts( workspace: &Path, root: Option<&Path>, From 230bbce9d38b31ccbe62866a23c482abec8076af Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 03:06:10 -0700 Subject: [PATCH 122/493] Share the owned mountpoint helper with bounded candidate preparation --- .../src/backend/provider_mountpoints.rs | 315 +++++++++++++++++- 1 file changed, 302 insertions(+), 13 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index 4666a75e1..1e81aa806 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -8,6 +8,34 @@ fn fail(cause: &'static str) -> BackendError { BackendError::Unavailable(cause.into()) } +/// A target whose exact volume and daemon were checked by the owning caller. +/// This is the fixed mountpoint helper's input, never a workload launch spec. +pub(super) struct MountpointHelperTarget<'a> { + pub image: &'a str, + pub volume: &'a str, + pub binary: &'a Path, + pub daemon: &'a DockerDaemonId, + pub labels: Value, +} + +fn owned_mountpoint_helper(actual: &Value, expected: &Value) -> bool { + input_publisher::owned_helper(actual, expected) + && [ + "NetworkMode", + "ReadonlyRootfs", + "CapDrop", + "CapAdd", + "SecurityOpt", + "Memory", + "MemorySwap", + "NanoCpus", + "PidsLimit", + "Tmpfs", + ] + .iter() + .all(|field| actual["HostConfig"][field] == expected["HostConfig"][field]) +} + fn selections(mounts: &docker::TerminalMounts) -> Result, BackendError> { let home = Path::new("/home/freshell/provider"); let mut selected = BTreeMap::::new(); @@ -73,7 +101,7 @@ fn selections(mounts: &docker::TerminalMounts) -> Result, Backen } impl DockerEngineBackend { - async fn cleanup_provider_mountpoint_helper( + pub(super) async fn cleanup_provider_mountpoint_helper( &self, name: &str, body: &Value, @@ -95,7 +123,7 @@ impl DockerEngineBackend { .as_str() .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) .ok_or_else(|| fail("provider mountpoint cleanup identity invalid"))?; - if inspected.status != 200 || !input_publisher::owned_helper(&actual, body) { + if inspected.status != 200 || !owned_mountpoint_helper(&actual, body) { return Err(fail("provider mountpoint cleanup ownership changed")); } let removed = self @@ -136,8 +164,6 @@ impl DockerEngineBackend { if selected.is_empty() { return Ok(()); } - let request = freshell_runtime_protocol::bounded_json_bytes(&selected, 1024 * 1024) - .map_err(|_| fail("provider mountpoint metadata profile exceeded"))?; if spec.initial_input_reservation.is_none() { let prior = spec.prior_provider_owner.as_ref().ok_or_else(|| fail("provider mountpoints require a bound initial reservation or retained registry owner"))?; if prior.soul_id() != &spec.soul_id @@ -180,27 +206,69 @@ impl DockerEngineBackend { if self.daemon_id().await? != *daemon { return Err(fail("provider mountpoint daemon changed")); } + let mut owned = None; + let target = MountpointHelperTarget { + image: &spec.image_ref, + volume: &spec.provider_volume_name, + binary, + daemon, + labels: json!({"com.freshell.installation-id":spec.installation_id,"com.freshell.soul-id":spec.soul_id,"com.freshell.incarnation-id":spec.incarnation_id}), + }; + self.run_provider_mountpoint_helper( + target, + selected, + (Duration::from_secs(30), Duration::from_secs(5)), + &mut owned, + ) + .await + } + + pub(super) async fn run_provider_mountpoint_helper( + &self, + target: MountpointHelperTarget<'_>, + selected: Vec, + budgets: (Duration, Duration), + owned: &mut Option<(String, Value)>, + ) -> Result<(), BackendError> { + if owned.is_some() { + return Err(fail( + "provider mountpoint previous helper cleanup remains unconfirmed", + )); + } + let deadline = tokio::time::Instant::now() + budgets.0; + let request = freshell_runtime_protocol::bounded_json_bytes(&selected, 1024 * 1024) + .map_err(|_| fail("provider mountpoint metadata profile exceeded"))?; let directory = native_auth::ExchangeDirectory::create_input()?; let socket = directory.0.join("reply.sock"); let listener = UnixListener::bind(&socket) .map_err(|_| fail("provider mountpoint exchange bind failed"))?; directory.mode(&socket, 0o660)?; let name = format!("freshell-mountpoints-{}", uuid::Uuid::new_v4()); + let environment = + tokio::time::timeout_at(deadline, self.private_helper_environment(target.image, &[])) + .await + .map_err(|_| fail("provider mountpoint image lookup timed out"))??; + if tokio::time::Instant::now() >= deadline { + return Err(fail( + "provider mountpoint preparation timed out before create", + )); + } let body = json!({ - "Image":spec.image_ref,"User":"0:0","Tty":false, - "Labels":{"com.freshell.installation-id":spec.installation_id,"com.freshell.soul-id":spec.soul_id,"com.freshell.incarnation-id":spec.incarnation_id}, + "Image":target.image,"User":"0:0","Tty":false, + "Labels":target.labels, "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["prepare-provider-mountpoints"], - "Env":self.private_helper_environment(&spec.image_ref,&[]).await?, + "Env":environment, "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"CapAdd":["CHOWN","SETGID","SETUID"], "SecurityOpt":["no-new-privileges"],"Memory":128*1024*1024,"MemorySwap":128*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, "LogConfig":{"Type":"none","Config":{}}, - "Mounts":[{"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, - {"Type":"volume","Source":spec.provider_volume_name,"Target":"/home/freshell/provider","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}, + "Mounts":[{"Type":"bind","Source":target.binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, + {"Type":"volume","Source":target.volume,"Target":"/home/freshell/provider","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}, {"Type":"bind","Source":directory.0,"Target":"/run/freshell-input-exchange","ReadOnly":true}]} }); let mut attempted = false; let operation = async { attempted = true; + *owned = Some((name.clone(), body.clone())); let created = self .request_bounded( "POST", @@ -211,6 +279,7 @@ impl DockerEngineBackend { .await?; if created.status == 409 { attempted = false; + *owned = None; } if created.status != 201 { return Err(fail("provider mountpoint helper creation failed")); @@ -221,6 +290,25 @@ impl DockerEngineBackend { .as_str() .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) .ok_or_else(|| fail("provider mountpoint helper identity invalid"))?; + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{id}/json"), + None, + 1024 * 1024, + ) + .await?; + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("provider mountpoint helper inspection invalid"))?; + if inspected.status != 200 + || actual["Id"] != id + || actual["State"]["Running"] != false + || !owned_mountpoint_helper(&actual, &body) + { + return Err(fail( + "provider mountpoint helper namespace changed before start", + )); + } let started = self .request("POST", &format!("{DOCKER_API}/containers/{id}/start"), None) .await?; @@ -267,25 +355,31 @@ impl DockerEngineBackend { Ok(()) }; tokio::try_join!(exchange, wait)?; - if self.daemon_id().await? != *daemon { + if self.daemon_id().await? != *target.daemon { return Err(fail( "provider mountpoint daemon changed before main create", )); } Ok(()) }; - let result = tokio::time::timeout(Duration::from_secs(30), operation) + let result = tokio::time::timeout_at(deadline, operation) .await .unwrap_or_else(|_| Err(fail("provider mountpoint preparation timed out"))); if attempted { tokio::time::timeout( - Duration::from_secs(5), + budgets.1, self.cleanup_provider_mountpoint_helper(&name, &body), ) .await .map_err(|_| fail("provider mountpoint cleanup timed out"))??; + *owned = None; } - tracing::debug!(soul_id=%spec.soul_id,entries=selected.len(),succeeded=result.is_ok(),"runtime.provider_mountpoints_settled"); + tracing::debug!( + event = "runtime.provider_mountpoints.settled", + entries = selected.len(), + succeeded = result.is_ok(), + "Provider mountpoint preparation settled after owned cleanup" + ); result } } @@ -297,6 +391,201 @@ mod tests { ProviderConfigBind, ProviderResourceKind, ProviderResourceMount, }; + #[tokio::test] + async fn provider_mountpoint_candidate_target_joins_owned_preparation_and_cleanup() { + use std::sync::{Arc, Mutex}; + use tokio::task::JoinSet; + for mode in [ + "prepared", + "empty", + "lost-create-ack", + "wrong-caps", + "cleanup-still-present", + "timeout", + "cancel-private-exchange", + "slow-image", + ] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("scratch-home"); + std::fs::create_dir(&home).unwrap(); + let selected = if mode == "empty" { + vec![] + } else { + vec![ + Mountpoint { + relative_path: ".freshell/provider-roots/unit-0".into(), + directory: true, + }, + Mountpoint { + relative_path: ".claude/settings.json".into(), + directory: false, + }, + ] + }; + let daemon = DockerDaemonId::new(); + let binary = owned.path().join("host"); + std::fs::write(&binary, b"synthetic helper artifact").unwrap(); + let listener = UnixListener::bind(owned.path().join("docker.sock")).unwrap(); + let expected = Arc::new(Mutex::new(None::)); + let requests = Arc::new(Mutex::new(Vec::::new())); + let removed = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let (body, seen, deleted, fixture_home, fixture_daemon) = ( + expected.clone(), + requests.clone(), + removed.clone(), + home.clone(), + daemon.clone(), + ); + let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); + let server = tokio::spawn(async move { + let mut jobs = JoinSet::new(); + let private_jobs = Arc::new(Mutex::new(Vec::>::new())); + loop { + tokio::select! { + _=&mut stopped=>break, + accepted=listener.accept()=>{ + let (mut stream,_)=accepted.unwrap(); + let (body,seen,deleted,home,daemon,private_jobs)=(body.clone(),seen.clone(),deleted.clone(),fixture_home.clone(),fixture_daemon.clone(),private_jobs.clone()); + jobs.spawn(async move { + let mut header=Vec::new(); let mut byte=[0]; + while !header.ends_with(b"\r\n\r\n") {stream.read_exact(&mut byte).await.unwrap();header.push(byte[0]);assert!(header.len()<=32*1024);} + let header=String::from_utf8(header).unwrap(); + let line=header.lines().next().unwrap().to_string(); + let length=header.lines().find_map(|line|line.strip_prefix("Content-Length: ")).unwrap().trim().parse::().unwrap(); + assert!(length<=1024*1024);let mut raw=vec![0;length];stream.read_exact(&mut raw).await.unwrap(); + seen.lock().unwrap().push(line.clone()); + let (status,value)=if line.contains("/images/") {if mode=="slow-image" {tokio::time::sleep(Duration::from_secs(10)).await;} (200,json!({"Config":{"Env":["PATH=/usr/bin","ANTHROPIC_API_KEY=PRIVATE_UNUSED_IMAGE_CREDENTIAL"]}}))} + else if line.contains("/info ") {(200,json!({"ID":daemon,"SecurityOptions":["name=rootless"]}))} + else if line.contains("/containers/create?") { *body.lock().unwrap()=Some(serde_json::from_slice(&raw).unwrap());(if mode=="lost-create-ack" {503} else {201},json!({"Id":"c".repeat(64)}))} + else if line.starts_with("GET") && line.contains("/containers/") { + if deleted.load(std::sync::atomic::Ordering::SeqCst) && mode!="cleanup-still-present" {(404,json!({}))} + else { + let mut expected=body.lock().unwrap().as_ref().unwrap().clone(); + if mode=="wrong-caps" {expected["HostConfig"]["CapAdd"]=json!(["SYS_ADMIN"]);} + (200,json!({"Id":"c".repeat(64),"Config":expected,"HostConfig":expected["HostConfig"],"State":{"Running":false}})) + } + } + else if line.contains("/start ") { + let expected=body.lock().unwrap().as_ref().unwrap().clone(); + let exchange=expected["HostConfig"]["Mounts"].as_array().unwrap().iter().find(|mount|mount["Target"]=="/run/freshell-input-exchange").unwrap()["Source"].as_str().unwrap().to_owned(); + private_jobs.lock().unwrap().push(tokio::spawn(async move { + let mut channel=UnixStream::connect(Path::new(&exchange).join("reply.sock")).await.unwrap(); + if matches!(mode,"timeout"|"cancel-private-exchange") {tokio::time::sleep(Duration::from_secs(10)).await;return;} + let length=channel.read_u32().await.unwrap() as usize; assert!(length<=1024*1024); + let mut raw=vec![0;length];channel.read_exact(&mut raw).await.unwrap(); + let selected:Vec=serde_json::from_slice(&raw).unwrap(); + for entry in selected { + let path=home.join(entry.relative_path);std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + if entry.directory {std::fs::create_dir_all(path).unwrap();} else {std::fs::File::create(path).unwrap();} + } + channel.write_all(b"ok").await.unwrap(); + })); + (204,json!({})) + } + else if line.contains("/wait?") {(200,json!({"StatusCode":0}))} + else if line.starts_with("DELETE") {deleted.store(true,std::sync::atomic::Ordering::SeqCst);(204,json!({}))} + else {(500,json!({}))}; + let raw=value.to_string();let response=format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{raw}",raw.len()); + let _=stream.write_all(response.as_bytes()).await; + }); + }, + result=jobs.join_next(),if !jobs.is_empty()=>{result.unwrap().unwrap();} + } + } + jobs.shutdown().await; + let workers = std::mem::take(&mut *private_jobs.lock().unwrap()); + for worker in workers { + worker.abort(); + if let Err(error) = worker.await { + assert!(error.is_cancelled()); + } + } + }); + let backend = DockerEngineBackend::new(owned.path().join("docker.sock")); + let mut helper = None; + let outcome=tokio::time::timeout(if mode=="cancel-private-exchange" {Duration::from_millis(50)} else {Duration::from_secs(3)},backend.run_provider_mountpoint_helper( + MountpointHelperTarget {image:"synthetic-image",volume:"owned-candidate-volume",binary:&binary,daemon:&daemon,labels:json!({"com.freshell.soul-id":"synthetic-owner","com.freshell.source-request-id":"synthetic-request"})},selected,(Duration::from_millis(250),Duration::from_millis(250)),&mut helper + )).await; + if mode == "cancel-private-exchange" { + assert!(outcome.is_err()); + let (name, body) = helper + .as_ref() + .expect("canceled private exchange must retain exact helper ownership"); + tokio::time::timeout( + Duration::from_millis(250), + backend.cleanup_provider_mountpoint_helper(name, body), + ) + .await + .unwrap() + .unwrap(); + helper = None; + } + let _ = stop.send(()); + tokio::time::timeout(Duration::from_secs(2), server) + .await + .unwrap() + .unwrap(); + if mode == "cancel-private-exchange" { + assert!(helper.is_none()); + assert!(removed.load(std::sync::atomic::Ordering::SeqCst)); + continue; + } + let result = outcome + .expect("preparation and cleanup must both be bounded, including image lookup"); + assert_eq!( + result.is_ok(), + matches!(mode, "prepared" | "empty"), + "{mode}: candidate preparation must settle with exact owned absence: {result:?}" + ); + if matches!(mode, "prepared" | "empty" | "lost-create-ack" | "timeout") { + assert!(helper.is_none()); + assert!(removed.load(std::sync::atomic::Ordering::SeqCst)); + } + if matches!(mode, "wrong-caps" | "cleanup-still-present") { + assert!( + helper.is_some(), + "unconfirmed ownership or absence cannot clear cleanup tracking" + ); + } + if mode == "slow-image" { + assert!(helper.is_none()); + assert!(!requests + .lock() + .unwrap() + .iter() + .any(|line| line.contains("/containers/create?"))); + continue; + } + if mode == "prepared" { + assert!(home.join(".freshell/provider-roots/unit-0").is_dir()); + assert!(home.join(".claude/settings.json").is_file()); + } + if mode == "wrong-caps" { + assert!( + !requests + .lock() + .unwrap() + .iter() + .any(|line| line.contains("/start ") || line.starts_with("DELETE")), + "changed identity/caps cannot authorize helper start or deletion" + ); + } + let body = expected.lock().unwrap(); + let body = body + .as_ref() + .expect("even an empty candidate needs initial HOME preparation"); + assert_eq!( + body["HostConfig"]["CapAdd"], + json!(["CHOWN", "SETGID", "SETUID"]) + ); + assert_eq!( + body["HostConfig"]["Mounts"][1]["Source"], + "owned-candidate-volume" + ); + assert!(!body.to_string().contains("PRIVATE_UNUSED_IMAGE_CREDENTIAL")); + } + } + #[tokio::test] async fn provider_mountpoint_cleanup_requires_confirmed_absence_after_delete() { for mode in [ From 31c1ce8278c95562e28f7a70d3701d25b6dabf3c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 03:34:31 -0700 Subject: [PATCH 123/493] Prepare initial Claude candidates in owned ordinary disk backing --- .../src/backend/claude_source_observation.rs | 620 +++++++++++++++++- .../claude_source_observation_tests.rs | 298 +++++++++ .../src/backend/provider_mountpoints.rs | 2 +- 3 files changed, 916 insertions(+), 4 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index 7a78949d9..5aea6c2e3 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -12,6 +12,545 @@ fn fail(reason: &'static str) -> BackendError { BackendError::Unavailable(reason.into()) } +/// Ordinary candidate files use disk backing, separately from indexed secrets. +/// The owning reader retains this identity through helper and volume cleanup. +struct CandidateScratch { + name: String, + labels: Value, + daemon: DockerDaemonId, + attempted: bool, +} + +impl CandidateScratch { + fn owns(&self, actual: &Value) -> bool { + actual["Name"] == self.name + && actual["Driver"] == "local" + && (actual["Options"].is_null() + || actual["Options"] + .as_object() + .is_some_and(|options| options.is_empty())) + && actual["Labels"] == self.labels + } +} + +/// An empty mount skeleton exposes only the selected ordinary staged files. +/// Resource payloads remain on their existing disk backing; they are not copied +/// into the controller's private exchange or indexed-secret volume. +struct CandidateInputView(PathBuf, bool); + +impl CandidateInputView { + fn create( + stage: &Path, + exchange: &native_auth::ExchangeDirectory, + mounts: &docker::TerminalMounts, + ) -> Result<(Self, Vec), BackendError> { + use std::os::unix::fs::PermissionsExt; + let blocked = || fail("Claude candidate ordinary input view unavailable"); + let root = exchange.0.join("ordinary-stage"); + std::fs::create_dir(&root).map_err(|_| blocked())?; + let view = Self(root, false); + let mut selected = BTreeMap::::new(); + for relative in [ + ".freshell/claude-candidate-binding.json", + ".freshell/prepared-context.json", + ".freshell/prepared-result.json", + ] { + selected.insert(relative.into(), (stage.join(relative), false)); + } + for file in mounts + .input_manifest + .iter() + .chain(mounts.config_binds.iter().map(|bind| &bind.source)) + { + selected.insert( + file.strip_prefix(stage).map_err(|_| blocked())?.into(), + (file.clone(), false), + ); + } + for resource in &mounts.resource_mounts { + selected.insert( + resource.stage_relative_path.clone().into(), + ( + stage.join(&resource.stage_relative_path), + resource.kind == freshell_runtime_protocol::ProviderResourceKind::Directory, + ), + ); + } + let mut bindings = vec![ + json!({"Type":"bind","Source":view.0,"Target":"/run/freshell-candidate-inputs","ReadOnly":true}), + ]; + for (relative, (source, directory)) in selected { + if !relative + .components() + .all(|part| matches!(part, std::path::Component::Normal(_))) + { + return Err(blocked()); + } + let target = view.0.join(&relative); + std::fs::create_dir_all(target.parent().ok_or_else(blocked)?).map_err(|_| blocked())?; + if directory { + std::fs::create_dir_all(&target).map_err(|_| blocked())?; + } else { + std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&target) + .map_err(|_| blocked())?; + std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o640)) + .map_err(|_| blocked())?; + } + let mut ancestor = if directory { + target.as_path() + } else { + target.parent().unwrap() + }; + loop { + std::fs::set_permissions(ancestor, std::fs::Permissions::from_mode(0o750)) + .map_err(|_| blocked())?; + if ancestor == view.0 { + break; + } + ancestor = ancestor.parent().ok_or_else(blocked)?; + } + bindings.push(json!({"Type":"bind","Source":source,"Target":Path::new("/run/freshell-candidate-inputs").join(relative),"ReadOnly":true})); + } + Ok((view, bindings)) + } + + fn cleanup(&mut self) -> Result<(), BackendError> { + std::fs::remove_dir_all(&self.0) + .map_err(|_| fail("Claude candidate ordinary input view cleanup failed"))?; + if !matches!(std::fs::symlink_metadata(&self.0), Err(error) if error.kind()==std::io::ErrorKind::NotFound) + { + return Err(fail( + "Claude candidate ordinary input view cleanup not confirmed", + )); + } + self.1 = true; + Ok(()) + } +} + +impl Drop for CandidateInputView { + fn drop(&mut self) { + if !self.1 && std::fs::remove_dir_all(&self.0).is_err() { + tracing::warn!( + event = "provider.inputs.claude_candidate_view.cleanup_failed", + "Owned ordinary input mount skeleton cleanup failed" + ); + } + } +} + +impl DockerEngineBackend { + async fn read_claude_candidate_helper( + &self, + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + root: &Path, + test_run_id: &str, + budget: Duration, + ) -> Result { + use freshell_runtime_protocol::{bounded_json_bytes, MAX_CONTROL_FRAME_BYTES}; + let deadline = tokio::time::Instant::now() + budget; + let name = format!("freshell-claude-candidate-reader-{}", uuid::Uuid::new_v4()); + let mut scratch = None::; + let mut mountpoint_helper = None::<(String, Value)>; + let mut expected = None::; + let mut attempted = false; + let mut directory = None::; + let mut input_view = None::; + let operation = async { + let projection = qualify_candidate_stage(root, owner, request)?; + let raw = bounded_json_bytes(&projection, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| fail("Claude candidate projection exceeds metadata profile"))?; + let (daemon, rootless) = self.daemon_info().await?; + if !rootless { + return Err(fail( + "Claude candidate reader requires the qualified rootless daemon", + )); + } + let (reservation, observation) = match owner { + ClaudeSourceOwner::Initial { reservation, observation } => (reservation, observation), + // Retained candidates need ordinary metadata from their exact + // prior HOME. An empty scratch HOME is not equivalent to it. + ClaudeSourceOwner::Retained { .. } => return Err(fail("Claude retained candidate ordinary topology unavailable; retain inputs and reprepare")), + }; + if reservation.soul_id != request.soul_id + || reservation.provider != request.provider + || self.observe_initial_volume(reservation).await? != *observation + || !matches!(observation, freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { daemon_id } if daemon_id == &daemon) + { + return Err(fail("Claude candidate creation absence changed")); + } + let binary = canonical_regular_file(&reservation.binary) + .map_err(|_| fail("Claude candidate reader artifact unavailable"))?; + let digest = crate::registry::binary_digest(&binary) + .map_err(|_| fail("Claude candidate reader artifact unavailable"))?; + if digest != reservation.binary_sha256 { + return Err(fail("Claude candidate reserved reader artifact changed")); + } + let workspace = canonical_dir(Path::new(&request.workspace_path)) + .map_err(|_| fail("Claude candidate workspace unavailable"))?; + let cwd = canonical_dir(&request.observation_request.cwd) + .map_err(|_| fail("Claude candidate cwd unavailable"))?; + if workspace != Path::new(&request.workspace_path) + || cwd != request.observation_request.cwd + || !cwd.starts_with(&workspace) + { + return Err(fail("Claude candidate workload topology changed")); + } + let mounts = docker::ordinary_claude_candidate_mounts( + &workspace, + request.git_common_dir.as_deref().map(Path::new), + root, + &projection.provider_launch_context, + ) + .map_err(|_| fail("Claude candidate ordinary mounts unavailable"))?; + let labels = json!({"com.freshell.installation-id":reservation.installation_id, + "com.freshell.soul-id":request.soul_id,"com.freshell.runtime-test-run-id":test_run_id, + "com.freshell.source-request-id":request.observation_request.request_id, + "com.freshell.source-binary-sha256":digest, + "com.freshell.input-generation":projection.publication.generation_id, + "com.freshell.source-reservation-id":reservation.id, + "com.freshell.claude-candidate":"ordinary"}); + scratch = Some(CandidateScratch { + name: format!("freshell-claude-candidate-{}", uuid::Uuid::new_v4()), + labels: labels.clone(), + daemon: daemon.clone(), + attempted: false, + }); + let backing = scratch.as_mut().unwrap(); + self.create_claude_candidate_scratch(backing).await?; + self.run_provider_mountpoint_helper( + provider_mountpoints::MountpointHelperTarget { + image: &reservation.image_ref, + volume: &backing.name, + binary: &binary, + daemon: &daemon, + labels: labels.clone(), + }, + provider_mountpoints::selections(&mounts)?, + ( + deadline.saturating_duration_since(tokio::time::Instant::now()), + Duration::from_secs(5), + ), + &mut mountpoint_helper, + ) + .await?; + directory = Some(native_auth::ExchangeDirectory::create_input()?); + let exchange_directory = directory.as_ref().unwrap(); + let socket = exchange_directory.0.join("reply.sock"); + let listener = UnixListener::bind(&socket) + .map_err(|_| fail("Claude candidate exchange unavailable"))?; + exchange_directory.mode(&socket, 0o660)?; + let (view, view_bindings) = + CandidateInputView::create(root, exchange_directory, &mounts)?; + input_view = Some(view); + let mut descriptors = vec![ + json!({"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}), + json!({"Type":"bind","Source":workspace,"Target":workspace,"ReadOnly":true}), + json!({"Type":"bind","Source":exchange_directory.0,"Target":"/run/freshell-source-exchange","ReadOnly":true}), + json!({"Type":"volume","Source":backing.name,"Target":"/home/freshell/provider","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}), + ]; + descriptors.extend(view_bindings); + if let Some(common) = mounts + .git_common_dir + .as_ref() + .filter(|common| !common.starts_with(&workspace)) + { + descriptors + .push(json!({"Type":"bind","Source":common,"Target":common,"ReadOnly":true})); + } + descriptors.extend(prepared_inputs::mount_descriptors(&mounts, &backing.name)?); + let body = json!({"Image":reservation.image_ref,"User":"0:0","Tty":false,"WorkingDir":cwd, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["project-claude-candidate-private"],"Labels":labels, + "Env":self.private_helper_environment(&reservation.image_ref,&["HOME=/home/freshell/provider"]).await?, + "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"CapAdd":["CHOWN","SETGID","SETUID"], + "SecurityOpt":["no-new-privileges"],"Memory":256*1024*1024,"MemorySwap":256*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, + "LogConfig":{"Type":"none","Config":{}},"Tmpfs":{},"Mounts":descriptors}}); + expected = Some(body.clone()); + attempted = true; + let created = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/create?name={name}"), + Some(&body), + 64 * 1024, + ) + .await?; + if created.status == 409 { + attempted = false; + } + if created.status != 201 { + return Err(fail( + "Claude candidate reader creation acknowledgement unavailable", + )); + } + let created: Value = serde_json::from_slice(&created.body) + .map_err(|_| fail("Claude candidate reader identity invalid"))?; + let id = created["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) + .ok_or_else(|| fail("Claude candidate reader identity unavailable"))?; + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{id}/json"), + None, + 1024 * 1024, + ) + .await?; + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("Claude candidate reader metadata invalid"))?; + if inspected.status != 200 + || actual["Id"] != id + || actual["State"]["Running"] != false + || !owned_candidate_helper(&actual, &body) + { + return Err(fail( + "Claude candidate reader namespace changed before start", + )); + } + let volume_reply = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", backing.name), + None, + 64 * 1024, + ) + .await?; + let volume: Value = serde_json::from_slice(&volume_reply.body) + .map_err(|_| fail("Claude candidate ordinary volume metadata invalid"))?; + if volume_reply.status != 200 { + return Err(fail("Claude candidate ordinary backing disappeared")); + } + verify_candidate_mount_reports(&actual, &body, backing, &volume, &mounts)?; + let supervision = async { + let started = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/start"), + None, + 64 * 1024, + ) + .await?; + if started.status != 204 { + return Err(fail("Claude candidate reader start failed")); + } + let waited = self + .request_bounded( + "POST", + &format!("{DOCKER_API}/containers/{id}/wait?condition=not-running"), + None, + 64 * 1024, + ) + .await?; + let value: Value = serde_json::from_slice(&waited.body) + .map_err(|_| fail("Claude candidate reader completion invalid"))?; + if waited.status != 200 || value["StatusCode"] != 0 { + return Err(fail( + "Claude candidate reader did not return complete evidence", + )); + } + Ok::<_, BackendError>(()) + }; + let (reply, ()) = tokio::try_join!(exchange(&listener, &raw), supervision)?; + let source_raw = bounded_json_bytes(&request.observation_request, MAX_REQUEST_BYTES) + .map_err(|_| fail("Claude candidate source metadata exceeds profile"))?; + reply + .validate_for_request( + &request.observation_request, + &format!("{:x}", Sha256::digest(source_raw)), + ) + .map_err(|_| fail("Claude candidate source selection binding changed"))?; + if ( + reply.observation.uid, + reply.observation.euid, + reply.observation.gid, + reply.observation.egid, + ) != (65534, 65534, 0, 0) + || self.daemon_id().await? != daemon + || crate::registry::binary_digest(&binary) + .map_err(|_| fail("Claude candidate reader artifact unavailable"))? + != digest + || self.observe_initial_volume(reservation).await? != *observation + { + return Err(fail("Claude candidate actual namespace changed")); + } + qualify_candidate_stage(root, owner, request)?; + Ok(reply) + }; + let result = tokio::time::timeout_at(deadline, operation) + .await + .unwrap_or_else(|_| { + Err(fail( + "Claude candidate reader timed out; retain inputs and reprepare", + )) + }); + // One cleanup reserve covers both helpers and the ordinary backing. + // Any uncertain helper removal prevents deleting its still-mounted HOME. + let cleanup = async { + if let Some((helper, body)) = mountpoint_helper.as_ref() { + self.cleanup_provider_mountpoint_helper(helper, body) + .await?; + mountpoint_helper = None; + } + if attempted { + self.cleanup_claude_candidate_reader(&name, expected.as_ref().unwrap()) + .await?; + attempted = false; + } + if let Some(backing) = scratch.as_mut() { + self.cleanup_claude_candidate_scratch(backing).await?; + } + if let Some(view) = input_view.as_mut() { + view.cleanup()?; + } + Ok::<_, BackendError>(()) + }; + tokio::time::timeout(Duration::from_secs(5), cleanup) + .await + .map_err(|_| fail("Claude candidate owned cleanup timed out"))??; + tracing::debug!(event="provider.inputs.claude_candidate_helper.settled",soul_id=%request.soul_id,succeeded=result.is_ok(),"Claude candidate reader settled after owned cleanup"); + result + } + + async fn cleanup_claude_candidate_reader( + &self, + name: &str, + expected: &Value, + ) -> Result<(), BackendError> { + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{name}/json"), + None, + 1024 * 1024, + ) + .await?; + if inspected.status == 404 { + return Ok(()); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("Claude candidate cleanup metadata invalid"))?; + let id = actual["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|byte| byte.is_ascii_hexdigit())) + .ok_or_else(|| fail("Claude candidate cleanup identity invalid"))?; + if inspected.status != 200 || !owned_candidate_helper(&actual, expected) { + return Err(fail("Claude candidate cleanup ownership changed")); + } + let removed = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/containers/{id}?force=1"), + None, + 64 * 1024, + ) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail("Claude candidate helper cleanup failed")); + } + let absent = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/containers/{id}/json"), + None, + 64 * 1024, + ) + .await?; + if absent.status != 404 { + return Err(fail("Claude candidate helper cleanup not confirmed")); + } + Ok(()) + } + + async fn create_claude_candidate_scratch( + &self, + scratch: &mut CandidateScratch, + ) -> Result<(), BackendError> { + if scratch.attempted || self.daemon_id().await? != scratch.daemon { + return Err(fail("Claude candidate scratch namespace changed")); + } + let absent = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", scratch.name), + None, + 64 * 1024, + ) + .await?; + if absent.status != 404 { + return Err(fail("Claude candidate scratch name is already present")); + } + // Retain ownership on a lost acknowledgement. A conflict is foreign. + scratch.attempted = true; + let created = self.request_bounded("POST", &format!("{DOCKER_API}/volumes/create"), Some(&json!({"Name":scratch.name,"Driver":"local","Labels":scratch.labels,"DriverOpts":{}})), 64 * 1024).await?; + if created.status == 409 { + scratch.attempted = false; + } + if !matches!(created.status, 200 | 201) { + return Err(fail( + "Claude candidate scratch creation acknowledgement unavailable", + )); + } + let inspected = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", scratch.name), + None, + 64 * 1024, + ) + .await?; + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("Claude candidate scratch metadata invalid"))?; + if inspected.status != 200 + || !scratch.owns(&actual) + || self.daemon_id().await? != scratch.daemon + { + return Err(fail("Claude candidate ordinary scratch ownership changed")); + } + Ok(()) + } + + async fn cleanup_claude_candidate_scratch( + &self, + scratch: &mut CandidateScratch, + ) -> Result<(), BackendError> { + if !scratch.attempted { + return Ok(()); + } + if self.daemon_id().await? != scratch.daemon { + return Err(fail("Claude candidate scratch cleanup daemon changed")); + } + let path = format!("{DOCKER_API}/volumes/{}", scratch.name); + let inspected = self.request_bounded("GET", &path, None, 64 * 1024).await?; + if inspected.status == 404 { + scratch.attempted = false; + return Ok(()); + } + let actual: Value = serde_json::from_slice(&inspected.body) + .map_err(|_| fail("Claude candidate scratch cleanup metadata invalid"))?; + if inspected.status != 200 || !scratch.owns(&actual) { + return Err(fail("Claude candidate scratch cleanup ownership changed")); + } + // No force: a helper that still holds this volume makes cleanup fail. + let removed = self + .request_bounded("DELETE", &path, None, 64 * 1024) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail("Claude candidate scratch cleanup failed")); + } + let absent = self.request_bounded("GET", &path, None, 64 * 1024).await?; + if absent.status != 404 || self.daemon_id().await? != scratch.daemon { + return Err(fail("Claude candidate scratch cleanup not confirmed")); + } + scratch.attempted = false; + Ok(()) + } +} + fn qualify_candidate_stage( root: &Path, owner: &ClaudeSourceOwner, @@ -225,13 +764,14 @@ fn read_candidate_metadata( Ok(bytes) } -fn owned_helper(actual: &Value, expected: &Value) -> bool { +fn owned_helper_profile(actual: &Value, expected: &Value) -> bool { input_publisher::owned_helper(actual, expected) && actual["Config"]["WorkingDir"] == expected["WorkingDir"] && [ "NetworkMode", "ReadonlyRootfs", "CapDrop", + "CapAdd", "SecurityOpt", "Memory", "MemorySwap", @@ -241,11 +781,84 @@ fn owned_helper(actual: &Value, expected: &Value) -> bool { ] .iter() .all(|key| actual["HostConfig"][key] == expected["HostConfig"][key]) +} + +fn owned_helper(actual: &Value, expected: &Value) -> bool { + owned_helper_profile(actual, expected) && actual["HostConfig"]["CapAdd"] .as_array() .is_none_or(|caps| caps.is_empty()) } +fn owned_candidate_helper(actual: &Value, expected: &Value) -> bool { + expected["User"] == "0:0" + && expected["Cmd"] == json!(["project-claude-candidate-private"]) + && expected["HostConfig"]["CapAdd"] == json!(["CHOWN", "SETGID", "SETUID"]) + && owned_helper_profile(actual, expected) +} + +fn verify_candidate_mount_reports( + actual: &Value, + expected: &Value, + backing: &CandidateScratch, + volume: &Value, + mounts: &docker::TerminalMounts, +) -> Result<(), BackendError> { + let blocked = || fail("Claude candidate reported mount topology changed"); + if !backing.owns(volume) || !volume["Mountpoint"].is_string() { + return Err(blocked()); + } + let declared = expected["HostConfig"]["Mounts"] + .as_array() + .ok_or_else(blocked)?; + let reported = actual["Mounts"].as_array().ok_or_else(blocked)?; + if declared.len() != reported.len() { + return Err(blocked()); + } + for mount in declared { + let reports = reported + .iter() + .filter(|report| report["Destination"] == mount["Target"]) + .collect::>(); + if reports.len() != 1 { + return Err(blocked()); + } + let report = reports[0]; + if report["Type"] != mount["Type"] + || report["RW"].as_bool() != mount["ReadOnly"].as_bool().map(|readonly| !readonly) + || (mount["Type"] == "bind" && report["Source"] != mount["Source"]) + || (mount["Type"] == "volume" + && (report["Name"] != backing.name || report["Source"] != volume["Mountpoint"])) + { + return Err(blocked()); + } + } + let selected = prepared_inputs::mount_descriptors(mounts, &backing.name)?; + let targets = selected + .iter() + .map(|mount| mount["Target"].clone()) + .collect::>(); + let mut ordinary = actual.clone(); + ordinary["HostConfig"]["Mounts"] = json!(actual["HostConfig"]["Mounts"] + .as_array() + .ok_or_else(blocked)? + .iter() + .filter(|mount| targets.contains(&mount["Target"])) + .cloned() + .collect::>()); + ordinary["Mounts"] = json!(reported + .iter() + .filter(|mount| targets.contains(&mount["Destination"])) + .cloned() + .collect::>()); + prepared_inputs::verify( + &ordinary, + mounts, + &backing.name, + &BTreeMap::from([(backing.name.clone(), volume.clone())]), + ) +} + async fn exchange(listener: &UnixListener, raw: &[u8]) -> Result { let (mut stream, _) = listener .accept() @@ -309,8 +922,9 @@ impl DockerEngineBackend { let root = docker::staged_provider_root_for_grant(&grant.grant_id).map_err(|_| { fail("Claude candidate stage unavailable; retain inputs and reprepare") })?; - qualify_candidate_stage(&root, owner, request)?; - return Err(fail("Claude candidate source topology is not yet qualified; retain inputs and reprepare")); + return self + .read_claude_candidate_helper(owner, request, &root, test_run_id, budget) + .await; } let name = format!("freshell-claude-source-{}", uuid::Uuid::new_v4()); let mut attempted = false; diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 8a948e02f..73118fed0 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -6,6 +6,124 @@ use freshell_runtime_protocol::{ use std::sync::{Arc, Mutex}; use tokio::{net::UnixListener, task::JoinSet}; +#[tokio::test] +async fn provider_claude_candidate_scratch_owns_disk_backing_and_confirmed_cleanup() { + for mode in [ + "success", + "lost-ack", + "existing", + "conflict", + "wrong-labels", + "driver-options", + "wrong-daemon", + "still-present", + ] { + let owned = tempfile::tempdir().unwrap(); + let daemon = DockerDaemonId::new(); + let labels = json!({"com.freshell.soul-id":"synthetic-soul", "com.freshell.source-request-id":"synthetic-request"}); + let mut scratch = CandidateScratch { + name: "freshell-claude-candidate-synthetic".into(), + labels: labels.clone(), + daemon: daemon.clone(), + attempted: false, + }; + let present = Arc::new(Mutex::new(mode == "existing")); + let calls = Arc::new(Mutex::new(Vec::<(String, Value)>::new())); + let listener = UnixListener::bind(owned.path().join("docker.sock")).unwrap(); + let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); + let (state, seen) = (present.clone(), calls.clone()); + let server = tokio::spawn(async move { + let mut jobs = JoinSet::new(); + loop { + tokio::select! { + _=&mut stopped=>break, + accepted=listener.accept()=>{ + let (mut stream,_)=accepted.unwrap(); + let (state,seen,labels,daemon)=(state.clone(),seen.clone(),labels.clone(),daemon.clone()); + jobs.spawn(async move { + let mut header=Vec::new();let mut byte=[0]; + while !header.ends_with(b"\r\n\r\n") {stream.read_exact(&mut byte).await.unwrap();header.push(byte[0]);assert!(header.len()<=32*1024);} + let header=String::from_utf8(header).unwrap();let line=header.lines().next().unwrap().to_string(); + let length=header.lines().find_map(|line|line.strip_prefix("Content-Length: ")).unwrap().trim().parse::().unwrap(); + assert!(length<=64*1024);let mut raw=vec![0;length];stream.read_exact(&mut raw).await.unwrap(); + let body=if raw.is_empty() {Value::Null} else {serde_json::from_slice(&raw).unwrap()}; + seen.lock().unwrap().push((line.clone(),body.clone())); + let (status,value)=if line.contains("/info ") {(200,json!({"ID":if mode=="wrong-daemon" {DockerDaemonId::new()} else {daemon},"SecurityOptions":["name=rootless"]}))} + else if line.starts_with("POST") && line.contains("/volumes/create ") { + *state.lock().unwrap()=true; + (if mode=="lost-ack" {503} else if mode=="conflict" {409} else {201},json!({"Name":body["Name"]})) + } else if line.starts_with("GET") && line.contains("/volumes/") { + if *state.lock().unwrap() {(200,json!({"Name":"freshell-claude-candidate-synthetic","Driver":"local","Options":if mode=="driver-options" {json!({"type":"tmpfs","o":"size=64m"})} else {json!({})},"Labels":if mode=="wrong-labels" {json!({"foreign":"owner"})} else {labels}}))} else {(404,json!({}))} + } else if line.starts_with("DELETE") && line.contains("/volumes/") {if mode!="still-present" {*state.lock().unwrap()=false;}(204,json!({}))} + else {(500,json!({}))}; + let response=value.to_string();stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{response}",response.len()).as_bytes()).await.unwrap(); + }); + } + result=jobs.join_next(),if !jobs.is_empty()=>{result.unwrap().unwrap();} + } + } + jobs.shutdown().await; + }); + let backend = DockerEngineBackend::new(owned.path().join("docker.sock")); + let created = tokio::time::timeout( + Duration::from_secs(2), + backend.create_claude_candidate_scratch(&mut scratch), + ) + .await + .unwrap(); + let cleaned = tokio::time::timeout( + Duration::from_secs(2), + backend.cleanup_claude_candidate_scratch(&mut scratch), + ) + .await + .unwrap(); + let _ = stop.send(()); + tokio::time::timeout(Duration::from_secs(2), server) + .await + .unwrap() + .unwrap(); + assert_eq!( + created.is_ok(), + matches!(mode, "success" | "still-present"), + "{mode}: only absent, exact ordinary backing can be accepted: {created:?}" + ); + assert_eq!( + cleaned.is_ok(), + matches!( + mode, + "success" | "lost-ack" | "existing" | "conflict" | "wrong-daemon" + ), + "{mode}: cleanup must prove exact absence or retain uncertain ownership: {cleaned:?}" + ); + let seen = calls.lock().unwrap(); + let deletes = seen + .iter() + .filter(|(line, _)| line.starts_with("DELETE")) + .count(); + assert_eq!( + deletes, + usize::from(matches!(mode, "success" | "lost-ack" | "still-present")), + "{mode}: no foreign or unqualified volume removal" + ); + if let Some((_, body)) = seen + .iter() + .find(|(line, _)| line.contains("/volumes/create ")) + { + assert_eq!(body["Driver"], "local"); + assert!( + body["DriverOpts"] + .as_object() + .is_none_or(|options| options.is_empty()), + "ordinary resource profile cannot be lowered to secret tmpfs size" + ); + } + assert_eq!( + scratch.attempted, + matches!(mode, "wrong-labels" | "driver-options" | "still-present") + ); + } +} + #[test] fn provider_claude_private_candidate_metadata_enforces_inclusive_limits() { use std::io::Write; @@ -284,6 +402,7 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() let qualified = qualify_candidate_stage(&stage, &owner, &request); assert_eq!(qualified.is_ok(),mode=="valid","{flavor}/{runtime:?}/{mode}: physical issuer and current owner must qualify before consumer selection: {:?}",qualified.as_ref().err()); if let Ok(projection) = qualified { + qualify_candidate_helper_transport(&owner, &request, &stage).await; let selected = &projection .provider_launch_context .prepared_inputs @@ -569,3 +688,182 @@ async fn qualify_transport(mode: &'static str) { std::fs::read(workspace.join(".claude/settings.local.json")).unwrap() == private.as_bytes() ); } + +async fn qualify_candidate_helper_transport( + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + stage: &Path, +) { + for mode in [ + "success", + "wrong-caps", + "lost-projection-ack", + "projection-timeout", + "volume-still-present", + "wrong-mount-report", + "wrong-resource-subpath", + ] { + let owned = tempfile::tempdir().unwrap(); + let daemon = match owner { + ClaudeSourceOwner::Initial { + observation: InitialAuthVolumeObservation::Absent { daemon_id }, + .. + } => daemon_id.clone(), + _ => panic!("this fixture qualifies an initial candidate"), + }; + let calls = Arc::new(Mutex::new(Vec::<(String, Value)>::new())); + let containers = Arc::new(Mutex::new(BTreeMap::::new())); + let volume = Arc::new(Mutex::new(None::)); + let listener = UnixListener::bind(owned.path().join("docker.sock")).unwrap(); + let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); + let (seen, bodies, backing) = (calls.clone(), containers.clone(), volume.clone()); + let server = tokio::spawn(async move { + let mut jobs = JoinSet::new(); + let private_jobs = Arc::new(Mutex::new(Vec::>::new())); + loop { + tokio::select! { + _=&mut stopped=>break, + accepted=listener.accept()=>{ + let (mut stream,_)=accepted.unwrap(); + let (seen,bodies,backing,daemon,private_jobs)=(seen.clone(),bodies.clone(),backing.clone(),daemon.clone(),private_jobs.clone()); + jobs.spawn(async move { + let mut header=Vec::new();let mut byte=[0]; + while !header.ends_with(b"\r\n\r\n") {stream.read_exact(&mut byte).await.unwrap();header.push(byte[0]);assert!(header.len()<=32*1024);} + let header=String::from_utf8(header).unwrap();let line=header.lines().next().unwrap().to_string(); + let length=header.lines().find_map(|line|line.strip_prefix("Content-Length: ")).unwrap().trim().parse::().unwrap(); + assert!(length<=1024*1024);let mut raw=vec![0;length];stream.read_exact(&mut raw).await.unwrap(); + let body=if raw.is_empty(){Value::Null}else{serde_json::from_slice(&raw).unwrap()}; + seen.lock().unwrap().push((line.clone(),body.clone())); + let (status,value)=if line.contains("/info "){(200,json!({"ID":daemon,"SecurityOptions":["name=rootless"]}))} + else if line.contains("/images/"){(200,json!({"Config":{"Env":["PATH=/usr/bin","ANTHROPIC_API_KEY=PRIVATE_IMAGE_CREDENTIAL_SENTINEL"]}}))} + else if line.starts_with("POST") && line.contains("/volumes/create "){let actual=json!({"Name":body["Name"],"Driver":"local","Options":{},"Labels":body["Labels"],"Mountpoint":"/synthetic/ordinary-volume"});*backing.lock().unwrap()=Some(actual.clone());(201,actual)} + else if line.starts_with("GET") && line.contains("/volumes/"){match backing.lock().unwrap().clone(){Some(v) if line.contains(v["Name"].as_str().unwrap())=>(200,v),_=>(404,json!({}))}} + else if line.starts_with("DELETE") && line.contains("/volumes/"){assert!(bodies.lock().unwrap().is_empty(),"candidate helpers must be gone before volume deletion");if mode!="volume-still-present" {*backing.lock().unwrap()=None;}(204,json!({}))} + else if line.contains("/containers/create?"){ + if body["Cmd"]==json!(["project-claude-candidate-private"]) {let root=body["HostConfig"]["Mounts"].as_array().unwrap().iter().find(|m|m["Target"]=="/run/freshell-candidate-inputs").unwrap()["Source"].as_str().unwrap();assert!(!Path::new(root).join(".freshell/mcp-capability.json").exists(),"ordinary candidate input root must not expose capability credentials");} + let name=line.split("name=").nth(1).unwrap().split_whitespace().next().unwrap().to_string();let id=if body["Cmd"]==json!(["prepare-provider-mountpoints"]) {"a".repeat(64)} else {"b".repeat(64)};bodies.lock().unwrap().insert(name,json!({"Id":id,"Config":body,"HostConfig":body["HostConfig"],"State":{"Running":false}}));(if mode=="lost-projection-ack" && body["Cmd"]==json!(["project-claude-candidate-private"]) {503}else{201},json!({"Id":id}))} + else if line.starts_with("GET") && line.contains("/containers/"){let key=line.split("/containers/").nth(1).unwrap().split('/').next().unwrap();let found=bodies.lock().unwrap().iter().find(|(name,v)|*name==key || v["Id"]==key).map(|(_,v)|v.clone());match found {Some(mut v)=>{ + let reports=v["HostConfig"]["Mounts"].as_array().unwrap().iter().map(|m|json!({"Type":m["Type"],"Destination":m["Target"],"RW":!m["ReadOnly"].as_bool().unwrap(),"Source":if m["Type"]=="volume" {json!("/synthetic/ordinary-volume")} else {m["Source"].clone()},"Name":if m["Type"]=="volume" {m["Source"].clone()} else {Value::Null}})).collect::>(); + v["Mounts"]=json!(reports); + if key.len()==64 && v["Config"]["Cmd"]==json!(["project-claude-candidate-private"]) { + if mode=="wrong-caps" {v["HostConfig"]["CapAdd"]=json!(["SYS_ADMIN"]);} + if mode=="wrong-mount-report" {v["Mounts"][0]["RW"]=json!(true);} + if mode=="wrong-resource-subpath" {let mount=v["HostConfig"]["Mounts"].as_array_mut().unwrap().iter_mut().find(|m|m["VolumeOptions"]["Subpath"].is_string()).unwrap();mount["VolumeOptions"]["Subpath"]=json!("foreign/ordinary");} + } (200,v)},None=>(404,json!({}))}} + else if line.starts_with("DELETE") && line.contains("/containers/"){let key=line.split("/containers/").nth(1).unwrap().split('?').next().unwrap();let name=bodies.lock().unwrap().iter().find(|(name,v)|*name==key || v["Id"]==key).map(|(name,_)|name.clone());if let Some(name)=name{bodies.lock().unwrap().remove(&name);}(204,json!({}))} + else if line.contains("/start "){let id=line.split("/containers/").nth(1).unwrap().split('/').next().unwrap();let v=bodies.lock().unwrap().values().find(|v|v["Id"]==id).unwrap().clone();private_jobs.lock().unwrap().push(tokio::spawn(async move { + let is_mountpoint=v["Config"]["Cmd"]==json!(["prepare-provider-mountpoints"]); + let target=if is_mountpoint {"/run/freshell-input-exchange"} else {"/run/freshell-source-exchange"}; + let path=v["HostConfig"]["Mounts"].as_array().unwrap().iter().find(|m|m["Target"]==target).unwrap()["Source"].as_str().unwrap(); + let mut channel=UnixStream::connect(Path::new(path).join("reply.sock")).await.unwrap(); + if !is_mountpoint && mode=="projection-timeout" {tokio::time::sleep(Duration::from_secs(10)).await;return;} + let length=channel.read_u32().await.unwrap() as usize;assert!(length<=1024*1024);let mut raw=vec![0;length];channel.read_exact(&mut raw).await.unwrap(); + if is_mountpoint {let _:Vec=serde_json::from_slice(&raw).unwrap();channel.write_all(b"ok").await.unwrap();} + else {let p:freshell_runtime_protocol::ClaudeCandidateProjectionRequest=serde_json::from_slice(&raw).unwrap();p.validate().unwrap();let source=p.source_request;let canonical=freshell_runtime_protocol::bounded_json_bytes(&source,MAX_REQUEST_BYTES).unwrap();let reply=ObservationReply {request_sha256:format!("{:x}",Sha256::digest(canonical)),observation:NativeObservation {version:1,native_source_runtime:source.native_source_runtime,setting_sources:source.setting_sources,uid:65534,euid:65534,gid:0,egid:0,supplementary_groups:vec![],cwd:source.cwd,home:Some(source.home),roots:None,local_settings:vec![],local_decision:LocalDecision::Disabled,root_uid:None,git_uid:None,claude_uid:None}};let raw=serde_json::to_vec(&reply).unwrap();channel.write_u32(raw.len() as u32).await.unwrap();channel.write_all(&raw).await.unwrap();channel.shutdown().await.unwrap();} + }));(204,json!({}))} + else if line.contains("/wait?"){(200,json!({"StatusCode":0}))}else{(500,json!({}))}; + let response=value.to_string();let _=stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{response}",response.len()).as_bytes()).await; + }); + } + result=jobs.join_next(),if !jobs.is_empty()=>{result.unwrap().unwrap();} + } + } + jobs.shutdown().await; + let workers = std::mem::take(&mut *private_jobs.lock().unwrap()); + for worker in workers { + if !worker.is_finished() { + worker.abort(); + } + match worker.await { + Ok(()) => {} + Err(e) if e.is_cancelled() => {} + Err(e) => panic!("private fixture worker failed: {e}"), + } + } + }); + let backend = DockerEngineBackend::new(owned.path().join("docker.sock")); + let result = backend + .read_claude_candidate_helper( + owner, + request, + stage, + "synthetic-source-run", + Duration::from_millis(250), + ) + .await; + let _ = stop.send(()); + tokio::time::timeout(Duration::from_secs(3), server) + .await + .unwrap() + .unwrap(); + assert_eq!( + result.is_ok(), + mode == "success", + "{mode}: candidate projection and confirmed owned cleanup gate evidence: {result:?}" + ); + assert!( + containers.lock().unwrap().is_empty(), + "{mode}: both helpers must be removed" + ); + assert_eq!( + volume.lock().unwrap().is_some(), + mode == "volume-still-present" + ); + let seen = calls.lock().unwrap(); + if matches!( + mode, + "wrong-caps" | "wrong-mount-report" | "wrong-resource-subpath" + ) { + assert!( + !seen + .iter() + .any(|(line, _)| line + .contains(&format!("/containers/{}/start ", "b".repeat(64)))), + "{mode}: changed candidate topology must block before projection starts" + ); + } + for (line, body) in seen.iter() { + assert!(!line.contains("/logs")); + assert!(!body + .to_string() + .contains("PRIVATE_IMAGE_CREDENTIAL_SENTINEL")); + if body["Cmd"] == json!(["project-claude-candidate-private"]) { + let root = body["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .find(|mount| mount["Target"] == "/run/freshell-candidate-inputs") + .unwrap()["Source"] + .as_str() + .unwrap(); + assert!( + !Path::new(root).exists(), + "{mode}: owned ordinary mount skeleton must be gone after reader settlement" + ); + } + } + if let Some((_, body)) = seen + .iter() + .find(|(_, body)| body["Cmd"] == json!(["project-claude-candidate-private"])) + { + assert_eq!(body["User"], "0:0"); + assert_eq!( + body["HostConfig"]["CapAdd"], + json!(["CHOWN", "SETGID", "SETUID"]) + ); + assert_eq!(body["HostConfig"]["NetworkMode"], "none"); + assert_eq!(body["HostConfig"]["ReadonlyRootfs"], true); + assert!( + body["HostConfig"]["Tmpfs"].is_null() || body["HostConfig"]["Tmpfs"] == json!({}) + ); + assert_eq!(body["WorkingDir"], json!(request.observation_request.cwd)); + assert!(body["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .any(|m| m["Target"] == "/run/freshell-candidate-inputs" + && m["Source"] != json!(stage) + && m["ReadOnly"] == true)); + } + } +} diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index 1e81aa806..f97d9c42b 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -36,7 +36,7 @@ fn owned_mountpoint_helper(actual: &Value, expected: &Value) -> bool { .all(|field| actual["HostConfig"][field] == expected["HostConfig"][field]) } -fn selections(mounts: &docker::TerminalMounts) -> Result, BackendError> { +pub(super) fn selections(mounts: &docker::TerminalMounts) -> Result, BackendError> { let home = Path::new("/home/freshell/provider"); let mut selected = BTreeMap::::new(); let mut add = |path: &Path, directory: bool| -> Result<(), BackendError> { From 28bffc669c583e7174c6c0d854c93690f035dec8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 03:49:06 -0700 Subject: [PATCH 124/493] Bind Claude candidate requests to complete frozen issuer metadata --- .../src/managed_mcp_capability.rs | 259 ++++++++++++++++++ 1 file changed, 259 insertions(+) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 42d5e8bb7..b0b9b9452 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -2650,6 +2650,188 @@ fn read_prepared_context( result } +// These are selected issuer metadata, never original HOME or capability values. +// Check physical size before reserving, and never allocate a growth sentinel. +fn read_claude_candidate_metadata( + root: &Path, + relative: &str, + limit: usize, +) -> Result, String> { + use std::io::Read; + let blocked = || { + "Claude frozen candidate private metadata unavailable or exceeds profile; retain inputs and reprepare".to_string() + }; + if !Path::new(relative) + .components() + .all(|part| matches!(part, std::path::Component::Normal(_))) + { + return Err(blocked()); + } + let path = root.join(relative); + let before = std::fs::symlink_metadata(&path).map_err(|_| blocked())?; + if !before.is_file() + || before.file_type().is_symlink() + || before.len() > limit as u64 + || std::fs::canonicalize(&path).map_err(|_| blocked())? != path + { + return Err(blocked()); + } + let mut options = std::fs::OpenOptions::new(); + options.read(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.custom_flags(libc::O_NOFOLLOW); + } + let mut file = options.open(&path).map_err(|_| blocked())?; + let opened = file.metadata().map_err(|_| blocked())?; + let modified = before.modified().map_err(|_| blocked())?; + if opened.len() != before.len() || opened.modified().map_err(|_| blocked())? != modified { + return Err(blocked()); + } + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + if (opened.dev(), opened.ino()) != (before.dev(), before.ino()) { + return Err(blocked()); + } + } + let length = usize::try_from(before.len()).map_err(|_| blocked())?; + let mut bytes = Vec::new(); + bytes.try_reserve_exact(length).map_err(|_| blocked())?; + bytes.resize(length, 0); + file.read_exact(&mut bytes).map_err(|_| blocked())?; + let mut extra = [0]; + if file.read(&mut extra).map_err(|_| blocked())? != 0 { + return Err(blocked()); + } + let after = std::fs::symlink_metadata(&path).map_err(|_| blocked())?; + if after.file_type().is_symlink() + || after.len() != before.len() + || after.modified().map_err(|_| blocked())? != modified + { + return Err(blocked()); + } + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + if (after.dev(), after.ino()) != (before.dev(), before.ino()) { + return Err(blocked()); + } + } + Ok(bytes) +} + +fn bind_claude_candidate_request( + store: &CapabilityStore, + directory: &Path, + request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, +) -> Result { + use freshell_platform::claude_inputs::{observation::CandidateBinding, ClaudeInputManifest}; + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeCandidatePublication, PreparedProviderContext, + MAX_CONTROL_FRAME_BYTES, + }; + let blocked = || { + "Claude frozen candidate request unavailable or changed; retain inputs and reprepare" + .to_string() + }; + let context = &request + .candidate + .as_ref() + .ok_or_else(blocked)? + .provider_launch_context; + let grant = context.mcp_capability.as_ref().ok_or_else(blocked)?; + let generation = grant.grant_id.clone(); + let inputs = context.prepared_inputs.as_ref().ok_or_else(blocked)?; + if inputs.generation_id != generation + || !store + .grants + .lock() + .unwrap() + .get(&generation) + .is_some_and(|grant| grant.soul_id == request.soul_id) + { + return Err(blocked()); + } + let root = staged_provider_root_path(directory, &generation).map_err(|_| blocked())?; + let published = read_claude_candidate_metadata( + &root, + ".freshell/claude-candidate-binding.json", + MAX_CONTROL_FRAME_BYTES, + )?; + let publication: ClaudeCandidatePublication = + serde_json::from_slice(&published).map_err(|_| blocked())?; + let context_bytes = read_claude_candidate_metadata( + &root, + ".freshell/prepared-context.json", + MAX_CONTROL_FRAME_BYTES, + )?; + let ready_bytes = read_claude_candidate_metadata( + &root, + ".freshell/prepared-result.json", + MAX_CONTROL_FRAME_BYTES, + )?; + let ready: PreparedProviderContext = + serde_json::from_slice(&ready_bytes).map_err(|_| blocked())?; + let manifest_bytes = + read_claude_candidate_metadata(&root, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + let manifest: ClaudeInputManifest = + serde_json::from_slice(&manifest_bytes).map_err(|_| blocked())?; + manifest.validate().map_err(|_| blocked())?; + let plan = manifest.accepted_native_plan.as_ref().ok_or_else(blocked)?; + if format!("{:x}", Sha256::digest(&context_bytes)) != publication.prepared_context_sha256 + || format!("{:x}", Sha256::digest(&ready_bytes)) != publication.prepared_result_sha256 + || format!("{:x}", Sha256::digest(&manifest_bytes)) != publication.manifest_sha256 + || publication.manifest_sha256 != inputs.manifest.sha256 + || bounded_json_bytes(context, MAX_CONTROL_FRAME_BYTES).map_err(|_| blocked())? + != context_bytes + || bounded_json_bytes(&ready.provider_launch_context, MAX_CONTROL_FRAME_BYTES) + .map_err(|_| blocked())? + != context_bytes + || bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024) + .map(|bytes| format!("{:x}", Sha256::digest(bytes))) + .map_err(|_| blocked())? + != publication.selector_manifest_sha256 + || request.observation_request.setting_sources != plan.observation.setting_sources + || request.observation_request.native_source_runtime + != plan.observation.native_source_runtime + || request.observation_request.cwd != plan.observation.cwd + || request.observation_request.selector_manifest_sha256 != plan.selector_manifest_sha256 + || request.observation_request.plan_revision != plan.plan_revision + { + return Err(blocked()); + } + let binding = CandidateBinding { + soul_id: publication.soul_id.as_str().into(), + generation_id: generation, + authority_sha256: publication.authority_sha256.clone(), + publication_sha256: format!("{:x}", Sha256::digest(published)), + manifest_sha256: publication.manifest_sha256.clone(), + prepared_result_sha256: publication.prepared_result_sha256.clone(), + }; + if request + .observation_request + .candidate_binding + .as_ref() + .is_some_and(|prior| { + bounded_json_bytes(prior, MAX_CONTROL_FRAME_BYTES).ok() + != bounded_json_bytes(&binding, MAX_CONTROL_FRAME_BYTES).ok() + }) + { + return Err(blocked()); + } + let mut bound = request; + bound.candidate = Some(ready); + bound.observation_request.candidate_binding = Some(binding); + bound + .candidate_projection(publication) + .map_err(|_| blocked())?; + bounded_json_bytes(&bound, MAX_CONTROL_FRAME_BYTES).map_err(|_| blocked())?; + tracing::debug!(event="provider.inputs.claude_candidate.request_bound",soul_id=%bound.soul_id,plan_revision=bound.observation_request.plan_revision,"Frozen Claude candidate request bound to its issuer"); + Ok(bound) +} + /// A web-server restart restores only grants whose exact incarnation remains /// live in the supervisor inventory. Recoverable souls retain their private /// OpenCode inputs before stale scopes and incarnation stages are revoked. @@ -3452,6 +3634,32 @@ except Exception as error: ); } + #[cfg(unix)] + #[test] + fn managed_provider_claude_candidate_private_metadata_keeps_inclusive_limits() { + for limit in [ + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + 64 * 1024 * 1024, + ] { + let owned = tempfile::tempdir().unwrap(); + let path = owned.path().join("metadata"); + let file = std::fs::File::create(&path).unwrap(); + file.set_len(limit as u64).unwrap(); + let bytes = read_claude_candidate_metadata(owned.path(), "metadata", limit) + .expect("supported inclusive metadata limit must remain readable"); + assert_eq!(bytes.len(), limit); + drop(bytes); + file.set_len(limit as u64 + 1).unwrap(); + assert!(read_claude_candidate_metadata(owned.path(), "metadata", limit).is_err()); + assert!( + read_claude_candidate_metadata(owned.path(), "../metadata", limit + 1).is_err() + ); + std::fs::rename(&path, owned.path().join("escaped")).unwrap(); + std::os::unix::fs::symlink(owned.path().join("escaped"), &path).unwrap(); + assert!(read_claude_candidate_metadata(owned.path(), "metadata", limit + 1).is_err()); + } + } + #[cfg(unix)] #[test] fn managed_provider_claude_issuer_publication_gates_owned_dependency_consumer() { @@ -3719,6 +3927,57 @@ except Exception as error: .is_err()); let replacement_projection = replacement_request.candidate_projection(replacement_publication).expect("replacement publication must bind its current authority and exact prior generation"); std::fs::remove_dir_all(&home).unwrap(); + for candidate in [&request, &replacement_request] { + let bound = bind_claude_candidate_request(&store, &directory, candidate.clone()) + .expect("candidate request must bind the exact frozen issuer after original HOME disappears"); + let capability = bound + .candidate + .as_ref() + .unwrap() + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap(); + let root = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let publication = serde_json::from_slice( + &std::fs::read(root.join(".freshell/claude-candidate-binding.json")).unwrap(), + ) + .unwrap(); + bound.candidate_projection(publication).expect( + "production request must retain complete issuer/result/authority binding", + ); + for relative in [ + ".freshell/claude-candidate-binding.json", + ".freshell/prepared-context.json", + ".freshell/prepared-result.json", + ".freshell/input-manifest.json", + ] { + let path = root.join(relative); + let bytes = std::fs::read(&path).unwrap(); + let private = b"PRIVATE_CHANGED_CANDIDATE_METADATA_SENTINEL"; + std::fs::write(&path, private).unwrap(); + let error=bind_claude_candidate_request(&store,&directory,candidate.clone()) + .expect_err("changed frozen metadata must block before an executable consumer is selected"); + assert!(!error.contains(std::str::from_utf8(private).unwrap())); + std::fs::write(&path, bytes).unwrap(); + } + let mut wrong = candidate.clone(); + wrong.observation_request.setting_sources = vec![ClaudeSettingSource::Local]; + assert!(bind_claude_candidate_request(&store, &directory, wrong).is_err()); + let mut wrong = candidate.clone(); + wrong + .observation_request + .candidate_binding + .as_mut() + .unwrap() + .publication_sha256 = "0".repeat(64); + assert!(bind_claude_candidate_request(&store, &directory, wrong).is_err()); + assert_eq!( + store.grants.lock().unwrap().len(), + 2, + "failed request binding must preserve both existing generations" + ); + } for (stage, projection) in [ (&stage, &projection), (&replacement_stage, &replacement_projection), From 8cf90dc1637c32d39ff9f166d3af31da80189ac2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 04:04:49 -0700 Subject: [PATCH 125/493] Compare initial Claude candidates before accepting launch preparation --- .../src/managed_mcp_capability.rs | 13 + .../src/managed_provider_references/claude.rs | 377 +++++++++++++++++- 2 files changed, 372 insertions(+), 18 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index b0b9b9452..247d4a970 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -2832,6 +2832,19 @@ fn bind_claude_candidate_request( Ok(bound) } +pub(crate) fn bind_prepared_claude_candidate( + request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, +) -> Result { + bind_claude_candidate_request(store(), &capability_dir()?, request) +} + +pub(crate) fn abort_prepared_claude_candidate( + soul: &SoulId, + generation: &str, +) -> Result<(), String> { + store().abort_preparation(generation, soul, &capability_dir()?) +} + /// A web-server restart restores only grants whose exact incarnation remains /// live in the supervisor inventory. Recoverable souls retain their private /// OpenCode inputs before stale scopes and incarnation stages are revoked. diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 87404f3df..800648452 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -172,24 +172,121 @@ pub(crate) async fn prepare_managed( plan_revision: observation_request.plan_revision, observation: observation.observation.clone(), }); - crate::managed_mcp_capability::issue_prepared_provider_context( - "claude", - request.soul, - request.home, - &context, - crate::managed_mcp_capability::OpencodeEphemeralInput { - claude_manifest: Some(&manifest), - claude_observation: Some(&observation.observation), - claude_inline_settings: inline_settings, - claude_reservation_provider: Some(request.reservation_provider), - claude_git_common_dir: request.git_common_dir, - cwd: Some(request.cwd), - workspace: Some(request.workspace), - secret_references: Some(request.references), - initial_auth_reservation: Some(&reservation.reservation_id), - ..Default::default() - }, - ) + // Each candidate is private and unbound. Only complete, cleanup-confirmed + // source evidence can either accept it or authorize discarding it for the + // next original-selector reprepare. An uncertain reader result retains it. + for revision in 0..=2 { + let plan = manifest + .accepted_native_plan + .as_ref() + .ok_or("Claude accepted native plan unavailable")?; + if plan.plan_revision != revision { + return Err( + "Claude predispatch plan revision changed; retain inputs and reprepare".into(), + ); + } + let prepared = crate::managed_mcp_capability::issue_prepared_provider_context( + "claude", + request.soul, + request.home, + &context, + crate::managed_mcp_capability::OpencodeEphemeralInput { + claude_manifest: Some(&manifest), + claude_observation: Some(&plan.observation), + claude_inline_settings: inline_settings, + claude_reservation_provider: Some(request.reservation_provider), + claude_git_common_dir: request.git_common_dir, + cwd: Some(request.cwd), + workspace: Some(request.workspace), + secret_references: Some(request.references), + initial_auth_reservation: Some(&reservation.reservation_id), + ..Default::default() + }, + )?; + let generation = prepared + .prepared_inputs + .as_ref() + .ok_or("Claude prepared generation unavailable")? + .generation_id + .clone(); + let source = freshell_platform::claude_inputs::observation::ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: plan.observation.native_source_runtime, + cwd: plan.observation.cwd.clone(), + home: "/home/freshell/provider".into(), + setting_sources: plan.observation.setting_sources.clone(), + selector_manifest_sha256: plan.selector_manifest_sha256.clone(), + plan_revision: revision, + candidate_binding: None, + }; + let candidate = crate::managed_mcp_capability::bind_prepared_claude_candidate( + ObserveClaudeSourcesRequest { + expected_control_epoch: None, + soul_id: request.soul.clone(), + provider: request.reservation_provider.into(), + authority: ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation.reservation_id.clone(), + }, + workspace_path: request + .workspace + .to_str() + .ok_or("Claude workspace unavailable")? + .into(), + git_common_dir: request + .git_common_dir + .map(|path| { + path.to_str() + .map(str::to_owned) + .ok_or("Claude common directory unavailable") + }) + .transpose()?, + candidate: Some(PreparedProviderContext { + provider_launch_context: prepared.clone(), + provider_secret_references: vec![], + }), + observation_request: source, + }, + ); + let candidate = match candidate { + Ok(candidate) => candidate, + Err(error) => { + crate::managed_mcp_capability::abort_prepared_claude_candidate( + request.soul, + &generation, + )?; + return Err(error); + } + }; + let source = candidate.observation_request.clone(); + let reply = match request.client.observe_claude_sources(candidate).await { + Ok(reply) => reply, + Err(_) => { + tracing::warn!(event="provider.inputs.claude_candidate.evidence_unavailable",soul_id=%request.soul,generation_id=%generation,plan_revision=revision, + "Claude candidate evidence unavailable; retain its private inputs before reconciliation"); + return Err("Claude candidate native source evidence unavailable; retain inputs and reprepare".into()); + } + }; + match accept_candidate_observation( + manifest.accepted_native_plan.as_mut().unwrap(), + &source, + &reply, + ) { + Ok(true) => return Ok(prepared), + Ok(false) => crate::managed_mcp_capability::abort_prepared_claude_candidate( + request.soul, + &generation, + )?, + Err(error) => { + crate::managed_mcp_capability::abort_prepared_claude_candidate( + request.soul, + &generation, + )?; + return Err(error); + } + } + } + Err("Claude native source selection exhausted two predispatch replans; retain inputs and reprepare".into()) } fn selected_settings_file( @@ -220,6 +317,73 @@ fn selected_settings_file( }) } +fn accept_candidate_observation( + plan: &mut freshell_platform::claude_inputs::AcceptedNativeSourcePlan, + source: &freshell_platform::claude_inputs::observation::ObservationRequest, + reply: &freshell_platform::claude_inputs::observation::ObservationReply, +) -> Result { + use sha2::{Digest, Sha256}; + plan.observation.validate()?; + let raw = bounded_json_bytes( + source, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .map_err(|_| { + "Claude candidate request exceeds its finite profile; retain inputs and reprepare" + })?; + reply.validate_for_request(source, &format!("{:x}", Sha256::digest(raw)))?; + if source.candidate_binding.is_none() + || source.plan_revision != plan.plan_revision + || source.selector_manifest_sha256 != plan.selector_manifest_sha256 + || source.native_source_runtime != plan.observation.native_source_runtime + || source.setting_sources != plan.observation.setting_sources + || source.cwd != plan.observation.cwd + || plan.observation.home.as_ref() != Some(&source.home) + { + return Err( + "Claude candidate accepted plan changed or is incomplete; retain inputs and reprepare" + .into(), + ); + } + let identity = + |observation: &freshell_platform::claude_inputs::observation::NativeObservation| { + let mut groups = observation.supplementary_groups.clone(); + groups.push(observation.egid); + groups.sort_unstable(); + groups.dedup(); + ( + observation.uid, + observation.euid, + observation.gid, + observation.egid, + groups, + ) + }; + if identity(&plan.observation) != identity(&reply.observation) { + return Err( + "Claude candidate actual reader identity changed; retain inputs and reprepare".into(), + ); + } + if plan + .observation + .require_same_selection(&reply.observation) + .is_ok() + { + return Ok(true); + } + if plan.plan_revision >= 2 { + return Err("Claude native source selection changed after two predispatch replans; retain inputs and reprepare".into()); + } + plan.plan_revision += 1; + plan.observation = reply.observation.clone(); + tracing::debug!( + event = "provider.inputs.claude_candidate.replan_required", + plan_revision = plan.plan_revision, + "Claude candidate source selection requires predispatch reprepare" + ); + Ok(false) +} + fn selected_config_root( home: &Path, cwd: &Path, @@ -256,6 +420,183 @@ fn selected_config_root( mod tests { use super::*; + #[test] + fn managed_provider_claude_candidate_comparison_bounds_replans_and_blocks_incomplete_evidence() + { + use freshell_platform::claude_inputs::{ + git::Roots, + observation::{ + CandidateBinding, LocalDecision, NativeObservation, ObservationReply, + ObservationRequest, + }, + AcceptedNativeSourcePlan, ClaudeSettingSource, NativeSourceRuntime, + }; + use sha2::{Digest, Sha256}; + let cwd = std::path::PathBuf::from("/workspace/nested"); + let initial = NativeObservation { + version: 1, + native_source_runtime: NativeSourceRuntime::Terminal, + setting_sources: vec![ClaudeSettingSource::Project, ClaudeSettingSource::Local], + uid: 65534, + euid: 65534, + gid: 0, + egid: 0, + supplementary_groups: vec![], + cwd: cwd.clone(), + home: Some("/home/freshell/provider".into()), + roots: Some(Roots { + checkout: "/workspace".into(), + canonical: "/workspace".into(), + }), + local_settings: vec![cwd.join(".claude/settings.local.json")], + local_decision: LocalDecision::RootOwnerMismatch, + root_uid: Some(0), + git_uid: None, + claude_uid: None, + }; + initial.validate().unwrap(); + let make_source = |plan: &AcceptedNativeSourcePlan| ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: plan.observation.native_source_runtime, + cwd: cwd.clone(), + home: "/home/freshell/provider".into(), + setting_sources: plan.observation.setting_sources.clone(), + selector_manifest_sha256: plan.selector_manifest_sha256.clone(), + plan_revision: plan.plan_revision, + candidate_binding: Some(CandidateBinding { + soul_id: "synthetic-soul".into(), + generation_id: format!("grant-{}", uuid::Uuid::new_v4()), + authority_sha256: "a".repeat(64), + publication_sha256: "b".repeat(64), + manifest_sha256: "c".repeat(64), + prepared_result_sha256: "d".repeat(64), + }), + }; + let make_reply = + |source: &ObservationRequest, observation: NativeObservation| ObservationReply { + request_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes( + source, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES + ) + .unwrap() + ) + ), + observation, + }; + let original = AcceptedNativeSourcePlan { + selector_manifest_sha256: "e".repeat(64), + plan_revision: 0, + observation: initial.clone(), + }; + let mut plan = original.clone(); + let source = make_source(&plan); + let mut same = initial.clone(); + same.supplementary_groups = vec![0]; + assert!( + accept_candidate_observation(&mut plan, &source, &make_reply(&source, same)).unwrap(), + "same effective provider identity and source selection must accept without rebuilding" + ); + assert_eq!(plan.observation, original.observation); + assert_eq!(plan.plan_revision, original.plan_revision); + assert_eq!( + plan.selector_manifest_sha256, + original.selector_manifest_sha256 + ); + for fault in [ + "digest", + "uid", + "groups", + "flags", + "revision", + "missing-binding", + ] { + let mut plan = original.clone(); + let mut source = make_source(&plan); + let mut reply = make_reply(&source, initial.clone()); + match fault { + "digest" => reply.request_sha256 = "f".repeat(64), + "uid" => { + reply.observation.uid = 65535; + reply.observation.euid = 65535; + } + "groups" => reply.observation.supplementary_groups = vec![7], + "flags" => reply.observation.setting_sources = vec![ClaudeSettingSource::Local], + "revision" => source.plan_revision = 1, + "missing-binding" => source.candidate_binding = None, + _ => unreachable!(), + } + assert!( + accept_candidate_observation(&mut plan, &source, &reply).is_err(), + "{fault}: incomplete or changed namespace evidence must block without replanning" + ); + assert_eq!( + plan.observation, original.observation, + "{fault}: accepted inputs must remain intact" + ); + assert_eq!(plan.plan_revision, original.plan_revision); + assert_eq!( + plan.selector_manifest_sha256, + original.selector_manifest_sha256 + ); + } + let mut plan = original.clone(); + for revision in 1..=3 { + let source = make_source(&plan); + let mut changed = initial.clone(); + match revision { + 1 => { + changed.roots = Some(Roots { + checkout: cwd.clone(), + canonical: cwd.clone(), + }); + changed.local_decision = LocalDecision::CwdRoot; + changed.root_uid = None; + } + 2 => { + changed.roots = None; + changed.local_decision = LocalDecision::NoRoot; + changed.root_uid = None; + } + 3 => {} + _ => unreachable!(), + } + changed.validate().unwrap(); + let before = plan.clone(); + let result = accept_candidate_observation( + &mut plan, + &source, + &make_reply(&source, changed.clone()), + ); + if revision <= 2 { + assert!( + !result.unwrap(), + "changed selection must request a predispatch rebuild" + ); + assert_eq!(plan.plan_revision, revision); + assert_eq!(plan.observation, changed); + assert_eq!( + plan.selector_manifest_sha256, + original.selector_manifest_sha256 + ); + } else { + assert!( + result.is_err(), + "third changed selection must stop at the two-replan limit" + ); + assert_eq!(plan.observation, before.observation); + assert_eq!(plan.plan_revision, before.plan_revision); + assert_eq!( + plan.selector_manifest_sha256, + before.selector_manifest_sha256 + ); + } + } + } + #[cfg(unix)] #[tokio::test] async fn managed_provider_claude_initial_source_failure_blocks_before_stage_and_dispatch() { From c0a7bfe527b9172fc8f7fec422e3b81e4d8ea0b2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 04:16:07 -0700 Subject: [PATCH 126/493] Bind Claude scratch backing to its qualified request identity --- .../src/backend/claude_source_observation.rs | 5 ++++- .../src/backend/claude_source_observation_tests.rs | 8 ++++++-- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index 5aea6c2e3..c18d022b6 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -215,7 +215,10 @@ impl DockerEngineBackend { "com.freshell.source-reservation-id":reservation.id, "com.freshell.claude-candidate":"ordinary"}); scratch = Some(CandidateScratch { - name: format!("freshell-claude-candidate-{}", uuid::Uuid::new_v4()), + name: format!( + "freshell-claude-candidate-{}", + request.observation_request.request_id + ), labels: labels.clone(), daemon: daemon.clone(), attempted: false, diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 73118fed0..16a52d1ad 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -717,6 +717,10 @@ async fn qualify_candidate_helper_transport( let listener = UnixListener::bind(owned.path().join("docker.sock")).unwrap(); let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); let (seen, bodies, backing) = (calls.clone(), containers.clone(), volume.clone()); + let expected_scratch = format!( + "freshell-claude-candidate-{}", + request.observation_request.request_id + ); let server = tokio::spawn(async move { let mut jobs = JoinSet::new(); let private_jobs = Arc::new(Mutex::new(Vec::>::new())); @@ -725,7 +729,7 @@ async fn qualify_candidate_helper_transport( _=&mut stopped=>break, accepted=listener.accept()=>{ let (mut stream,_)=accepted.unwrap(); - let (seen,bodies,backing,daemon,private_jobs)=(seen.clone(),bodies.clone(),backing.clone(),daemon.clone(),private_jobs.clone()); + let (seen,bodies,backing,daemon,private_jobs,expected_scratch)=(seen.clone(),bodies.clone(),backing.clone(),daemon.clone(),private_jobs.clone(),expected_scratch.clone()); jobs.spawn(async move { let mut header=Vec::new();let mut byte=[0]; while !header.ends_with(b"\r\n\r\n") {stream.read_exact(&mut byte).await.unwrap();header.push(byte[0]);assert!(header.len()<=32*1024);} @@ -736,7 +740,7 @@ async fn qualify_candidate_helper_transport( seen.lock().unwrap().push((line.clone(),body.clone())); let (status,value)=if line.contains("/info "){(200,json!({"ID":daemon,"SecurityOptions":["name=rootless"]}))} else if line.contains("/images/"){(200,json!({"Config":{"Env":["PATH=/usr/bin","ANTHROPIC_API_KEY=PRIVATE_IMAGE_CREDENTIAL_SENTINEL"]}}))} - else if line.starts_with("POST") && line.contains("/volumes/create "){let actual=json!({"Name":body["Name"],"Driver":"local","Options":{},"Labels":body["Labels"],"Mountpoint":"/synthetic/ordinary-volume"});*backing.lock().unwrap()=Some(actual.clone());(201,actual)} + else if line.starts_with("POST") && line.contains("/volumes/create "){assert_eq!(body["Name"],expected_scratch,"private candidate request must identify its exact scratch backing before creation");let actual=json!({"Name":body["Name"],"Driver":"local","Options":{},"Labels":body["Labels"],"Mountpoint":"/synthetic/ordinary-volume"});*backing.lock().unwrap()=Some(actual.clone());(201,actual)} else if line.starts_with("GET") && line.contains("/volumes/"){match backing.lock().unwrap().clone(){Some(v) if line.contains(v["Name"].as_str().unwrap())=>(200,v),_=>(404,json!({}))}} else if line.starts_with("DELETE") && line.contains("/volumes/"){assert!(bodies.lock().unwrap().is_empty(),"candidate helpers must be gone before volume deletion");if mode!="volume-still-present" {*backing.lock().unwrap()=None;}(204,json!({}))} else if line.contains("/containers/create?"){ From d944efa110a4ed2dd9f109699a4da3588c525472 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 04:39:19 -0700 Subject: [PATCH 127/493] Recover exact Claude helper identity after lost create acknowledgement --- scripts/testing/runtime-test-broker.ts | 17 +++++++- .../testing/runtime-test-broker.test.ts | 40 +++++++++++++++++-- 2 files changed, 53 insertions(+), 4 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 2f064e6d6..d07b30a23 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -384,6 +384,22 @@ export class RestrictedDockerBroker { const targetName = helperTarget![1] const exactId = targetName === helper.containerId const deleting = method === 'DELETE' && url === `${DOCKER_API_PREFIX}/containers/${targetName}?force=1` + if (method === 'GET' && url === `${DOCKER_API_PREFIX}/containers/${targetName}/json`) { + const inspected = await this.forward(request, body) + let actual: Record = {} + try { actual = JSON.parse(inspected.body.toString()) } catch {} + // A successful create may lose its acknowledgement. Adopt only the full + // identity returned by the exact reserved name and fixed owned profile. + if (!helper.containerId && targetName === name && inspected.statusCode === 200 + && typeof actual.Id === 'string' && /^[a-f0-9]{64}$/.test(actual.Id) + && !this.receiptIds().has(actual.Id) && this.ownedClaudeSourceHelper(actual, helper.expected)) { + helper.containerId = actual.Id + } + this.record({ method, url, decision: 'forward', destructive: false, unsafeAttempt: false, + containerId: helper.containerId, helperName: name }) + replyDocker(response, inspected) + return + } if (deleting) { const inspected = await this.forward({ method: 'GET', url: `${DOCKER_API_PREFIX}/containers/${targetName}/json`, headers: {} } as IncomingMessage, Buffer.alloc(0)) let actual: Record = {} @@ -394,7 +410,6 @@ export class RestrictedDockerBroker { } } if (deleting - || (method === 'GET' && url === `${DOCKER_API_PREFIX}/containers/${targetName}/json`) || (method === 'POST' && exactId && [ `${DOCKER_API_PREFIX}/containers/${targetName}/start`, `${DOCKER_API_PREFIX}/containers/${targetName}/wait?condition=not-running`, diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index fd0885c38..847932f46 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -367,6 +367,36 @@ describe('scoped Claude source observer broker', () => { expect(forwarded.filter((operation) => operation.method === 'DELETE')).toEqual([]) }) }) + it('adopts the full owned identity after a lost source-helper acknowledgement and confirms absence', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance, loseHelperAck }) => { + instance.authorizeInitialProviderVolume(sourceAuthorization()) + loseHelperAck() + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, sourceHelperBody())).status).toBe(503) + expect(instance.receiptIds().has(helperId)).toBe(false) + const inspected = await request('GET', `/v1.47/containers/${sourceName}/json`) + expect(inspected.status).toBe(200) + expect(JSON.parse(inspected.body).Id).toBe(helperId) + expect(instance.receiptIds().has(helperId)).toBe(true) + expect((await request('DELETE', `/v1.47/containers/${helperId}?force=1`)).status).toBe(204) + expect((await request('GET', `/v1.47/containers/${helperId}/json`)).status).toBe(404) + expect(forwarded.filter((operation) => operation.method === 'DELETE')).toEqual([ + { method: 'DELETE', url: `/v1.47/containers/${helperId}?force=1`, body: undefined }, + ]) + }) + }) + it('leaves a foreign inspected helper unowned after a lost acknowledgement', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance, loseHelperAck, replaceHelperInspection }) => { + instance.authorizeInitialProviderVolume(sourceAuthorization()) + loseHelperAck() + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, sourceHelperBody())).status).toBe(503) + replaceHelperInspection() + expect((await request('GET', `/v1.47/containers/${sourceName}/json`)).status).toBe(200) + expect(instance.receiptIds().has(helperId)).toBe(false) + expect((await request('DELETE', `/v1.47/containers/${helperId}?force=1`)).status).toBe(403) + expect((await request('DELETE', `/v1.47/containers/${sourceName}?force=1`)).status).toBe(403) + expect(forwarded.filter((operation) => operation.method === 'DELETE')).toEqual([]) + }) + }) }) describe('receipt-owned auth exchange HTTP routes', () => { @@ -557,6 +587,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise let helperAckId = helperId let indexedVolumeMetadata:unknown let foreignHelper=false + let sourceHelperDeleted = false const upstream = http.createServer(async (req, res) => { const chunks: Buffer[] = [] for await (const chunk of req) chunks.push(Buffer.from(chunk)) @@ -570,13 +601,15 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise if (req.method==='GET' && req.url?.startsWith('/v1.47/volumes/freshell-inputs-') && indexedVolumeMetadata) { res.statusCode=200;res.end(JSON.stringify(indexedVolumeMetadata));return; } - if (req.method==='GET' && (req.url?.includes('-publisher/json') || req.url?.includes('/freshell-claude-source-'))) { - const command=req.url?.includes('/freshell-claude-source-')?'observe-claude-sources-private':'publish-provider-inputs'; + if (req.method==='GET' && (req.url?.includes('-publisher/json') || req.url?.includes('/freshell-claude-source-') + || (req.url===`/v1.47/containers/${helperId}/json` && forwarded.some((operation)=>operation.body?.Cmd?.[0]==='observe-claude-sources-private')))) { + if (sourceHelperDeleted) { res.statusCode=404;res.end('{}');return } + const command=req.url?.includes('/freshell-claude-source-') || req.url===`/v1.47/containers/${helperId}/json`?'observe-claude-sources-private':'publish-provider-inputs'; const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]===command)!.body; const inspected=structuredClone(config); inspected.Labels['org.opencontainers.image.version']='24.04'; for (const mount of inspected.HostConfig.Mounts) if (mount.ReadOnly===false) delete mount.ReadOnly; - res.statusCode=200;res.end(JSON.stringify({Config:foreignHelper?{...inspected,User:'0:0'}:inspected,HostConfig:inspected.HostConfig}));return; + res.statusCode=200;res.end(JSON.stringify({Id:helperId,Config:foreignHelper?{...inspected,User:'0:0'}:inspected,HostConfig:inspected.HostConfig}));return; } if (req.url?.startsWith('/v1.47/containers/create?')) { if (body.Entrypoint?.[0] === '/runtime/freshell-session-host' && loseAck) { @@ -588,6 +621,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise return } if (req.method === 'DELETE' || req.url?.endsWith('/start') || req.url?.endsWith('/update')) { + if (req.method==='DELETE' && (req.url===`/v1.47/containers/${helperId}?force=1` || req.url===`/v1.47/containers/${sourceName}?force=1`)) sourceHelperDeleted=true res.statusCode = 204 res.end() return From 1772c79e4225201f382dffcb0e1d0f25ace0f2c4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 05:21:47 -0700 Subject: [PATCH 128/493] Authorize fixed initial Claude candidates in the closed native broker --- scripts/testing/runtime-test-broker.ts | 180 +++++++++++++++++- .../testing/runtime-test-broker.test.ts | 177 ++++++++++++++++- 2 files changed, 351 insertions(+), 6 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index d07b30a23..2f3109de1 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -151,6 +151,30 @@ export type InitialProviderVolumeAuthorization = { daemonId:string } +/** Metadata for the fixed initial Claude candidate, before a workload exists. */ +export type ClaudeCandidateAuthorization = InitialProviderVolumeAuthorization & { + reservationId: string + requestId: string + generationId: string + stageRoot: string + workspacePath: string + cwd: string + gitCommonDir?: string + manifestRelativePath: string + configurationBinds: { sourceRelativePath: string; effectiveDestination: string }[] + resourceMounts: { stageRelativePath: string; providerRelativePath: string; effectiveDestination: string; kind: 'file' | 'directory' }[] +} + +function candidateVolumeName(input: ClaudeCandidateAuthorization): string { + return `freshell-claude-candidate-${input.requestId}` +} +function candidateLabels(input: ClaudeCandidateAuthorization, runId: string): Record { + return { 'com.freshell.installation-id': input.installationId, 'com.freshell.soul-id': input.soulId, + 'com.freshell.runtime-test-run-id': runId, 'com.freshell.source-request-id': input.requestId, + 'com.freshell.source-binary-sha256': input.binarySha256, 'com.freshell.input-generation': input.generationId, + 'com.freshell.source-reservation-id': input.reservationId, 'com.freshell.claude-candidate': 'ordinary' } +} + type DockerResponse = { statusCode: number headers: http.IncomingHttpHeaders @@ -158,6 +182,8 @@ type DockerResponse = { } export class RestrictedDockerBroker { + private readonly claudeCandidates = new Map() + private readonly candidateVolumes = new Set() private readonly initialProviderVolumes = new Map() private readonly preparedInputs = new Map() private readonly inputVolumes = new Set() @@ -176,6 +202,46 @@ export class RestrictedDockerBroker { constructor(private readonly policy: RestrictedDockerBrokerPolicy) {} + authorizeClaudeCandidate(input: ClaudeCandidateAuthorization): void { + const initial = this.initialProviderVolumes.get(input.providerVolumeName) + const uuid = (value: string) => /^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(value) + const fixture = path.dirname(this.policy.runtimeRootPrefix) + const relative = path.relative(fixture, input.stageRoot) + const destination = (value: string) => typeof value === 'string' && path.isAbsolute(value) + && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) + && (value === input.workspacePath || isStrictDescendant(value, input.workspacePath) + || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) + const unit = (value: string, prefix: string, file = false) => typeof value === 'string' + && new RegExp(`^${prefix}/[a-zA-Z0-9_-]{1,128}${file ? '/source' : ''}$`).test(value) + if (!initial || !['installationId','soulId','imageRef','hostBinaryPath','binarySha256','daemonId'].every((key) => input[key as keyof InitialProviderVolumeAuthorization] === initial[key as keyof InitialProviderVolumeAuthorization]) + || !uuid(input.requestId) || !uuid(input.reservationId) || !input.generationId.startsWith('grant-') || !uuid(input.generationId.slice(6)) + || relative !== `scenarios/${relative.split('/')[1]}/control/provider-roots/${input.generationId}` + || !/^[a-zA-Z0-9_-]{1,128}$/.test(relative.split('/')[1] ?? '') + || !this.policy.allowedWorkspaceRoots?.has(input.workspacePath) || !destination(input.cwd) + || !(input.cwd === input.workspacePath || isStrictDescendant(input.cwd, input.workspacePath)) + || (input.gitCommonDir !== undefined && (!path.isAbsolute(input.gitCommonDir) || path.normalize(input.gitCommonDir) !== input.gitCommonDir + || /[\x00-\x1f\x7f]/.test(input.gitCommonDir) || !(input.gitCommonDir === input.workspacePath || isStrictDescendant(input.gitCommonDir, input.workspacePath) || isStrictDescendant(input.gitCommonDir, fixture)))) + || input.manifestRelativePath !== '.freshell/input-manifest.json' + || !Array.isArray(input.configurationBinds) || !Array.isArray(input.resourceMounts) + || Buffer.byteLength(JSON.stringify(input)) > 1024 * 1024) throw new Error('initial Claude candidate is outside this fixture request') + const targets = new Set() + const claim = (value: string) => { if (!destination(value) || targets.has(value)) throw new Error('Claude candidate selected target is invalid'); targets.add(value) } + for (const bind of input.configurationBinds) { + if (!unit(bind.sourceRelativePath, '\\.freshell/templates')) throw new Error('Claude candidate template source is invalid') + claim(bind.effectiveDestination) + } + for (const mount of input.resourceMounts) { + if (!['file','directory'].includes(mount.kind) + || !unit(mount.stageRelativePath, '\\.freshell/resource-units', mount.kind === 'file') + || !unit(mount.providerRelativePath, '\\.freshell/provider-roots', mount.kind === 'file')) throw new Error('Claude candidate resource source is invalid') + claim(mount.effectiveDestination) + } + const key = candidateVolumeName(input), previous = this.claudeCandidates.get(key) + if (previous && JSON.stringify(previous) !== JSON.stringify(input)) throw new Error('Claude candidate request changed') + if (!previous && this.claudeCandidates.size >= 6) throw new Error('closed Claude fixture exceeds its two routes and two replans') + this.claudeCandidates.set(key, structuredClone(input)) + } + authorizeInitialProviderVolume(input:InitialProviderVolumeAuthorization):void { const stable=`freshell-provider-${createHash('sha256').update(`${input.installationId}\0${input.soulId}`).digest('hex').slice(0,24)}` if (input.providerVolumeName!==stable || !/^[a-zA-Z0-9_-]{1,128}$/.test(input.installationId) @@ -324,6 +390,24 @@ export class RestrictedDockerBroker { await this.forwardAndReply(request, response, body, { destructive: false }) return } + const candidateVolume = url.match(/^\/v1\.47\/volumes\/(freshell-claude-candidate-[a-f0-9-]{36})$/)?.[1] + const candidate = candidateVolume && this.claudeCandidates.get(candidateVolume) + if (method === 'GET' && candidate) { + await this.forwardAndReply(request, response, body, { destructive: false }) + return + } + if (method === 'DELETE' && candidate && this.candidateVolumes.has(candidateVolume!)) { + const inspected = await this.forward({ method: 'GET', url, headers: {} } as IncomingMessage, Buffer.alloc(0)) + let actual: Record = {} + try { actual = JSON.parse(inspected.body.toString()) } catch {} + if (inspected.statusCode !== 404 && (inspected.statusCode !== 200 || actual.Name !== candidateVolume || actual.Driver !== 'local' + || (actual.Options != null && !equalRecord(actual.Options, {})) || !equalRecord(actual.Labels, candidateLabels(candidate, this.policy.testRunId)))) { + this.block(response, method, url, 403, 'Claude candidate ordinary disk ownership changed', true) + return + } + await this.forwardAndReply(request, response, body, { destructive: true }) + return + } const indexedVolume=url.match(/^\/v1\.47\/volumes\/(freshell-inputs-[a-f0-9]{24})$/)?.[1] const input=indexedVolume && [...this.preparedInputs.values()].find((candidate)=>candidate.indexedFiles?.volumeName===indexedVolume) if (method==='GET' && input) { @@ -345,6 +429,17 @@ export class RestrictedDockerBroker { if (method==='POST' && url===`${DOCKER_API_PREFIX}/volumes/create`) { let parsed:Record={} try {parsed=JSON.parse(body.toString())} catch {} + const candidate = this.claudeCandidates.get(parsed.Name) + if (candidate) { + if (Object.keys(parsed).length !== 4 || parsed.Driver !== 'local' || !equalRecord(parsed.DriverOpts, {}) + || !equalRecord(parsed.Labels, candidateLabels(candidate, this.policy.testRunId))) { + this.block(response, method, url, 403, 'Claude candidate requires its exact ordinary disk backing', true) + return + } + this.candidateVolumes.add(parsed.Name) // Retain exact ownership on a lost acknowledgement. + await this.forwardAndReply(request, response, body, { destructive: false }) + return + } const initial=this.initialProviderVolumes.get(parsed.Name) if (initial) { const labels=parsed.Labels ?? {} @@ -473,9 +568,17 @@ export class RestrictedDockerBroker { if (method === 'POST' && url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`)) { const name = new URL(url, 'http://docker').searchParams.get('name') ?? '' - if (name.startsWith('freshell-claude-source-')) { - if (!/^freshell-claude-source-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) - || !this.validateClaudeSourceHelper(body) || this.claudeSourceHelpers.has(name)) { + const candidateReader = /^freshell-claude-candidate-reader-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) + const mountpointHelper = /^freshell-mountpoints-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) + let candidateMountpoints = false + if (mountpointHelper) { + try { candidateMountpoints = JSON.parse(body.toString()).Labels?.['com.freshell.claude-candidate'] === 'ordinary' } catch {} + } + if (name.startsWith('freshell-claude-source-') || candidateReader || candidateMountpoints) { + const valid = candidateReader || candidateMountpoints + ? this.validateClaudeCandidateHelper(body, candidateReader ? 'reader' : 'mountpoints') + : /^freshell-claude-source-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) && this.validateClaudeSourceHelper(body) + if (!valid || this.claudeSourceHelpers.has(name)) { this.block(response, method, url, 403, 'Claude source helper must match its exact owned fixed reader', false) return } @@ -497,7 +600,6 @@ export class RestrictedDockerBroker { } return } - const mountpointHelper=/^freshell-mountpoints-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(name) if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name) || mountpointHelper) { const input=mountpointHelper ? this.validateMountpointHelper(body) : this.validateInputPublisher(name,body) if (!input || this.inputPublishers.has(name)) { @@ -758,6 +860,62 @@ export class RestrictedDockerBroker { return { ok: true, incarnationId, soulId, imageRef, runtimeDir, hostBinaryPath, ...(providerVolumeName ? { providerVolumeName } : {}), ...(workspacePath ? { workspacePath } : {}) } } + private validateClaudeCandidateHelper(body: Buffer, role: 'reader' | 'mountpoints'): boolean { + let parsed: Record + try { parsed = JSON.parse(body.toString()) } catch { return false } + const input = [...this.claudeCandidates.values()].find((input) => equalRecord(parsed.Labels, candidateLabels(input, this.policy.testRunId))) + const reader = role === 'reader', host = parsed.HostConfig ?? {} + if (!input || !this.candidateVolumes.has(candidateVolumeName(input)) || parsed.Image !== input.imageRef || parsed.User !== '0:0' || parsed.Tty !== false + || JSON.stringify(parsed.Entrypoint) !== JSON.stringify(['/runtime/freshell-session-host']) + || JSON.stringify(parsed.Cmd) !== JSON.stringify([reader ? 'project-claude-candidate-private' : 'prepare-provider-mountpoints']) + || !privateHelperEnvironment(parsed.Env, reader) + || (reader ? parsed.WorkingDir !== input.cwd : parsed.WorkingDir !== undefined) + || Object.keys(parsed).some((key) => !['Image','User','Tty','Labels','Entrypoint','Cmd','Env','HostConfig',...(reader ? ['WorkingDir'] : [])].includes(key)) + || Object.keys(host).some((key) => !['NetworkMode','ReadonlyRootfs','CapDrop','CapAdd','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','LogConfig','Mounts',...(reader ? ['Tmpfs'] : [])].includes(key)) + || host.NetworkMode !== 'none' || host.ReadonlyRootfs !== true + || JSON.stringify(host.CapDrop) !== JSON.stringify(['ALL']) || JSON.stringify(host.CapAdd) !== JSON.stringify(['CHOWN','SETGID','SETUID']) + || JSON.stringify(host.SecurityOpt) !== JSON.stringify(['no-new-privileges']) + || host.Memory !== (reader ? 256 : 128) * 1024 * 1024 || host.MemorySwap !== host.Memory || host.NanoCpus !== 500_000_000 || host.PidsLimit !== 32 + || !equalRecord(host.LogConfig, { Type: 'none', Config: host.LogConfig?.Config }) || Object.keys(host.LogConfig?.Config ?? {}).length + || (reader ? !equalRecord(host.Tmpfs, {}) : host.Tmpfs !== undefined)) return false + const mounts = host.Mounts + if (!Array.isArray(mounts)) return false + const exchangeTarget = reader ? '/run/freshell-source-exchange' : '/run/freshell-input-exchange' + const exchange = mounts.find((mount) => mount?.Type === 'bind' && mount.Target === exchangeTarget && mount.ReadOnly === true + && typeof mount.Source === 'string' && path.dirname(mount.Source) === os.tmpdir() + && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + if (!exchange) return false + const expected: Record[] = [ + { Type: 'bind', Source: input.hostBinaryPath, Target: '/runtime/freshell-session-host', ReadOnly: true }, + { Type: 'volume', Source: candidateVolumeName(input), Target: '/home/freshell/provider', ReadOnly: false, VolumeOptions: { NoCopy: true } }, + { Type: 'bind', Source: exchange.Source, Target: exchangeTarget, ReadOnly: true }, + ] + if (reader) { + expected.push({ Type: 'bind', Source: input.workspacePath, Target: input.workspacePath, ReadOnly: true }, + { Type: 'bind', Source: path.join(exchange.Source, 'ordinary-stage'), Target: '/run/freshell-candidate-inputs', ReadOnly: true }) + const selected = new Set(['.freshell/claude-candidate-binding.json','.freshell/prepared-context.json','.freshell/prepared-result.json',input.manifestRelativePath, + ...input.configurationBinds.map((bind) => bind.sourceRelativePath),...input.resourceMounts.map((mount) => mount.stageRelativePath)]) + for (const relative of selected) expected.push({ Type: 'bind', Source: path.join(input.stageRoot, relative), Target: path.join('/run/freshell-candidate-inputs', relative), ReadOnly: true }) + if (input.gitCommonDir && input.gitCommonDir !== input.workspacePath && !isStrictDescendant(input.gitCommonDir, input.workspacePath)) { + expected.push({ Type: 'bind', Source: input.gitCommonDir, Target: input.gitCommonDir, ReadOnly: true }) + } + for (const bind of input.configurationBinds) expected.push({ Type: 'bind', Source: path.join(input.stageRoot, bind.sourceRelativePath), Target: bind.effectiveDestination, ReadOnly: true }) + for (const mount of input.resourceMounts) expected.push({ Type: 'volume', Source: candidateVolumeName(input), Target: mount.effectiveDestination, + ReadOnly: mount.kind === 'file', VolumeOptions: { NoCopy: true, Subpath: mount.providerRelativePath } }) + } + if (mounts.length !== expected.length) return false + const remaining = new Map(expected.map((mount) => [mount.Target, mount])) + for (const mount of mounts) { + if (!mount || typeof mount !== 'object' || Array.isArray(mount)) return false + const wanted = remaining.get(mount.Target) + if (!wanted || Object.keys(mount).some((key) => !['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key)) + || ['Type','Source','Target','ReadOnly'].some((key) => mount[key] !== wanted[key])) return false + if (wanted.Type === 'volume' ? !equalRecord(mount.VolumeOptions, wanted.VolumeOptions) : mount.VolumeOptions !== undefined) return false + remaining.delete(mount.Target) + } + return remaining.size === 0 + } + private validateClaudeSourceHelper(body: Buffer): boolean { let parsed: Record try { parsed = JSON.parse(body.toString()) } catch { return false } @@ -800,6 +958,20 @@ export class RestrictedDockerBroker { private ownedClaudeSourceHelper(actual: Record, expected: Record): boolean { const config = actual.Config ?? {}, host = actual.HostConfig ?? {} const same = (actual: unknown, expected: unknown) => JSON.stringify(actual) === JSON.stringify(expected) + if (expected.Labels?.['com.freshell.claude-candidate'] === 'ordinary') { + return ['Image','User','Tty'].every((key) => config[key] === expected[key]) + && (expected.WorkingDir === undefined || config.WorkingDir === expected.WorkingDir) + && ['Entrypoint','Cmd'].every((key) => same(config[key], expected[key])) + && ownedHelperLabels(config.Labels, expected.Labels) + && Array.isArray(config.Env) && same([...config.Env].sort(), [...expected.Env].sort()) + && ['NetworkMode','ReadonlyRootfs','CapDrop','CapAdd','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit'].every((key) => same(host[key], expected.HostConfig[key])) + && same(host.Tmpfs ?? {}, expected.HostConfig.Tmpfs ?? {}) && host.LogConfig?.Type === 'none' + && Array.isArray(host.Mounts) && host.Mounts.length === expected.HostConfig.Mounts.length + && expected.HostConfig.Mounts.every((wanted: Record) => host.Mounts.some((mount: Record) => + ['Type','Source','Target'].every((key) => mount[key] === wanted[key]) && (mount.ReadOnly ?? false) === wanted.ReadOnly + && (wanted.Type === 'volume' ? mount.VolumeOptions?.NoCopy === true && !mount.VolumeOptions.DriverConfig + && (mount.VolumeOptions.Subpath ?? '') === (wanted.VolumeOptions.Subpath ?? '') : mount.VolumeOptions === undefined))) + } return ['Image','User','Tty','WorkingDir'].every((key) => config[key] === expected[key]) && ['Entrypoint','Cmd'].every((key) => same(config[key], expected[key])) && ownedHelperLabels(config.Labels, expected.Labels) diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 847932f46..76fa66da2 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -5,7 +5,7 @@ import http from 'node:http' import path from 'node:path' import { describe, expect, it } from 'vitest' -import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' +import { RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' const runtimeDir = '/tmp/owned/r/scenario/x/incarnation-one' const actorKey = createHash('sha256').update('installation-one\0soul-one').digest('hex') @@ -311,6 +311,155 @@ function sourceHelperBody(): any { } } +function candidateAuthorization(): ClaudeCandidateAuthorization { + const generationId = 'grant-44444444-4444-4444-8444-444444444444' + return { ...sourceAuthorization(), generationId, + reservationId: '11111111-1111-4111-8111-111111111111', + requestId: '22222222-2222-4222-8222-222222222222', + stageRoot: `/tmp/owned/scenarios/claude/control/provider-roots/${generationId}`, + workspacePath: '/workspace', cwd: '/workspace/nested', manifestRelativePath: '.freshell/input-manifest.json', + configurationBinds: [{ sourceRelativePath: '.freshell/templates/source-0', effectiveDestination: '/workspace/.claude/settings.json' }], + resourceMounts: [{ stageRelativePath: '.freshell/resource-units/unit-0', providerRelativePath: '.freshell/provider-roots/unit-0', + effectiveDestination: '/workspace/.claude/skills', kind: 'directory' }], + } +} +function candidateLabels(input: ClaudeCandidateAuthorization) { + return { 'com.freshell.installation-id': input.installationId, 'com.freshell.soul-id': input.soulId, + 'com.freshell.runtime-test-run-id': 'run-one', 'com.freshell.source-request-id': input.requestId, + 'com.freshell.source-binary-sha256': input.binarySha256, 'com.freshell.input-generation': input.generationId, + 'com.freshell.source-reservation-id': input.reservationId, 'com.freshell.claude-candidate': 'ordinary' } +} +function candidateHelperBody(input: ClaudeCandidateAuthorization, role: 'mountpoints' | 'reader'): any { + const volume = `freshell-claude-candidate-${input.requestId}` + const exchange = '/tmp/freshell-inputs-33333333-3333-4333-8333-333333333333' + const body = sourceHelperBody() + body.User = '0:0'; body.Labels = candidateLabels(input) + body.Cmd = [role === 'reader' ? 'project-claude-candidate-private' : 'prepare-provider-mountpoints'] + body.HostConfig.CapAdd = ['CHOWN', 'SETGID', 'SETUID'] + body.HostConfig.Memory = body.HostConfig.MemorySwap = (role === 'reader' ? 256 : 128) * 1024 * 1024 + body.HostConfig.Mounts = [ + { Type: 'bind', Source: input.hostBinaryPath, Target: '/runtime/freshell-session-host', ReadOnly: true }, + { Type: 'volume', Source: volume, Target: '/home/freshell/provider', ReadOnly: false, VolumeOptions: { NoCopy: true } }, + { Type: 'bind', Source: exchange, Target: role === 'reader' ? '/run/freshell-source-exchange' : '/run/freshell-input-exchange', ReadOnly: true }, + ] + if (role === 'mountpoints') { + delete body.WorkingDir; delete body.HostConfig.Tmpfs; body.Env = ['PATH='] + return body + } + body.WorkingDir = input.cwd; body.HostConfig.Tmpfs = {} + body.HostConfig.Mounts.push( + { Type: 'bind', Source: input.workspacePath, Target: input.workspacePath, ReadOnly: true }, + { Type: 'bind', Source: `${exchange}/ordinary-stage`, Target: '/run/freshell-candidate-inputs', ReadOnly: true }, + ) + const selected = new Set(['.freshell/claude-candidate-binding.json', '.freshell/prepared-context.json', '.freshell/prepared-result.json', input.manifestRelativePath, + ...input.configurationBinds.map((bind) => bind.sourceRelativePath), ...input.resourceMounts.map((mount) => mount.stageRelativePath)]) + for (const relative of selected) body.HostConfig.Mounts.push({ Type: 'bind', Source: `${input.stageRoot}/${relative}`, Target: `/run/freshell-candidate-inputs/${relative}`, ReadOnly: true }) + for (const bind of input.configurationBinds) body.HostConfig.Mounts.push({ Type: 'bind', Source: `${input.stageRoot}/${bind.sourceRelativePath}`, Target: bind.effectiveDestination, ReadOnly: true }) + for (const mount of input.resourceMounts) body.HostConfig.Mounts.push({ Type: 'volume', Source: volume, Target: mount.effectiveDestination, ReadOnly: mount.kind === 'file', VolumeOptions: { NoCopy: true, Subpath: mount.providerRelativePath } }) + return body +} + +describe('fixed initial Claude candidate broker', () => { + it('owns only the exact request disk backing and confirms non-force cleanup', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance }) => { + const input = candidateAuthorization(), name = `freshell-claude-candidate-${input.requestId}` + const url = `/v1.47/volumes/${name}` + expect((await request('GET', url)).status).toBe(403) + expect(forwarded).toEqual([]) + instance.authorizeInitialProviderVolume(sourceAuthorization()) + instance.authorizeClaudeCandidate(input) + expect((await request('GET', url)).status).toBe(404) + const body = { Name: name, Driver: 'local', DriverOpts: {}, Labels: candidateLabels(input) } + const before = forwarded.length + for (const mutation of ['foreign-name','tmpfs','labels','extra']) { + const changed: any = structuredClone(body) + if (mutation === 'foreign-name') changed.Name = 'freshell-claude-candidate-33333333-3333-4333-8333-333333333333' + if (mutation === 'tmpfs') changed.DriverOpts = { type: 'tmpfs', device: 'tmpfs', o: 'size=64m' } + if (mutation === 'labels') changed.Labels['com.freshell.soul-id'] = 'foreign' + if (mutation === 'extra') changed.Privileged = true + expect((await request('POST', '/v1.47/volumes/create', changed)).status, mutation).toBe(403) + expect(forwarded).toHaveLength(before) + } + expect((await request('POST', '/v1.47/volumes/create', body)).status).toBe(201) + expect((await request('GET', url)).status).toBe(200) + const beforeForce = forwarded.length + expect((await request('DELETE', `${url}?force=1`)).status).toBe(403) + expect(forwarded).toHaveLength(beforeForce) + expect((await request('DELETE', url)).status).toBe(204) + expect((await request('GET', url)).status).toBe(404) + }) + }) + it('retains attempted disk ownership after a lost acknowledgement and refuses a foreign readback', async () => { + for (const foreign of [false, true]) { + await withHttpBroker(async ({ request, forwarded, broker: instance, loseCandidateVolumeAck, replaceCandidateVolumeInspection }) => { + const input = candidateAuthorization(), name = `freshell-claude-candidate-${input.requestId}` + const url = `/v1.47/volumes/${name}` + instance.authorizeInitialProviderVolume(sourceAuthorization()) + instance.authorizeClaudeCandidate(input) + loseCandidateVolumeAck() + expect((await request('POST', '/v1.47/volumes/create', { Name: name, Driver: 'local', DriverOpts: {}, Labels: candidateLabels(input) })).status).toBe(503) + expect((await request('GET', url)).status).toBe(200) + if (foreign) replaceCandidateVolumeInspection() + expect((await request('DELETE', url)).status).toBe(foreign ? 403 : 204) + expect(forwarded.filter((operation) => operation.method === 'DELETE')).toHaveLength(foreign ? 0 : 1) + if (!foreign) expect((await request('GET', url)).status).toBe(404) + }) + } + }) + it('permits only the two fixed candidate helpers with selected ordinary inputs and exact lost-ack cleanup', async () => { + for (const role of ['mountpoints', 'reader'] as const) { + for (const lost of [false, true]) { + await withHttpBroker(async ({ request, forwarded, broker: instance, loseHelperAck }) => { + const input = candidateAuthorization(), volume = `freshell-claude-candidate-${input.requestId}` + const name = `${role === 'reader' ? 'freshell-claude-candidate-reader' : 'freshell-mountpoints'}-11111111-1111-4111-8111-111111111111` + instance.authorizeInitialProviderVolume(sourceAuthorization()); instance.authorizeClaudeCandidate(input) + expect((await request('POST', '/v1.47/volumes/create', { Name: volume, Driver: 'local', DriverOpts: {}, Labels: candidateLabels(input) })).status).toBe(201) + const body = candidateHelperBody(input, role), before = forwarded.length + for (const mutation of ['uid','command','credential','caps','network','memory','logs','reservation','generation','request','mount','readonly','volume']) { + const changed = structuredClone(body) + if (mutation === 'uid') changed.User = '65534:0' + if (mutation === 'command') changed.Cmd.push('PRIVATE_CANDIDATE_SENTINEL') + if (mutation === 'credential') changed.Env.push('SECRET=PRIVATE_CANDIDATE_SENTINEL') + if (mutation === 'caps') changed.HostConfig.CapAdd.push('DAC_OVERRIDE') + if (mutation === 'network') changed.HostConfig.NetworkMode = 'bridge' + if (mutation === 'memory') changed.HostConfig.Memory *= 2 + if (mutation === 'logs') changed.HostConfig.LogConfig.Type = 'json-file' + if (mutation === 'reservation') changed.Labels['com.freshell.source-reservation-id'] = '33333333-3333-4333-8333-333333333333' + if (mutation === 'generation') changed.Labels['com.freshell.input-generation'] = 'grant-33333333-3333-4333-8333-333333333333' + if (mutation === 'request') changed.Labels['com.freshell.source-request-id'] = '33333333-3333-4333-8333-333333333333' + if (mutation === 'mount') changed.HostConfig.Mounts.push({ Type: 'bind', Source: `${input.stageRoot}/.freshell/mcp-capability.json`, Target: '/outside', ReadOnly: true }) + if (mutation === 'readonly') changed.HostConfig.Mounts[0].ReadOnly = false + if (mutation === 'volume') changed.HostConfig.Mounts[1].Source = sourceAuthorization().providerVolumeName + expect((await request('POST', `/v1.47/containers/create?name=${name}`, changed)).status, `${role}/${mutation}`).toBe(403) + expect(forwarded).toHaveLength(before) + } + if (lost) loseHelperAck() + expect((await request('POST', `/v1.47/containers/create?name=${name}`, body)).status).toBe(lost ? 503 : 201) + expect((await request('GET', `/v1.47/containers/${name}/json`)).status).toBe(200) + expect(instance.receiptIds().has(helperId)).toBe(true) + if (!lost) { + expect((await request('POST', `/v1.47/containers/${helperId}/start`)).status).toBe(204) + expect((await request('POST', `/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200) + } + const beforeControls = forwarded.length + for (const [method, url] of [ + ['GET', `/v1.47/containers/${helperId}/logs?stdout=1&stderr=1`], ['POST', `/v1.47/containers/${helperId}/exec`], + ['POST', `/v1.47/containers/${helperId}/stop`], ['POST', `/v1.47/containers/${name}/start`], + ]) { + expect((await request(method, url)).status).toBe(403) + expect(forwarded).toHaveLength(beforeControls) + } + expect((await request('DELETE', `/v1.47/containers/${helperId}?force=1`)).status).toBe(204) + expect((await request('GET', `/v1.47/containers/${helperId}/json`)).status).toBe(404) + expect((await request('DELETE', `/v1.47/volumes/${volume}`)).status).toBe(204) + expect((await request('GET', `/v1.47/volumes/${volume}`)).status).toBe(404) + expect(JSON.stringify(instance.eventsSnapshot())).not.toContain('PRIVATE_CANDIDATE_SENTINEL') + }) + } + } + }) +}) + describe('scoped Claude source observer broker', () => { it('allows the reserved fixed reader and exact private lifecycle without creating initial HOME or exposing logs', async () => { await withHttpBroker(async ({ request, forwarded, broker: instance }) => { @@ -442,6 +591,8 @@ type BrokerHttpFixture = { loseHelperAck(): void returnHelperId(id: string): void replaceHelperInspection():void + loseCandidateVolumeAck(): void + replaceCandidateVolumeInspection(): void } function publisherAuthorization():PreparedInputAuthorization { @@ -588,6 +739,9 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise let indexedVolumeMetadata:unknown let foreignHelper=false let sourceHelperDeleted = false + let candidateVolumeMetadata: any + let candidateVolumeAckUnavailable = false + let candidateHelperDeleted = false const upstream = http.createServer(async (req, res) => { const chunks: Buffer[] = [] for await (const chunk of req) chunks.push(Buffer.from(chunk)) @@ -595,12 +749,29 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise const body = raw ? JSON.parse(raw) : undefined forwarded.push({ method: req.method!, url: req.url!, body }) if (req.url==='/v1.47/volumes/create') { + if (body.Name.startsWith('freshell-claude-candidate-')) { + candidateVolumeMetadata={Name:body.Name,Driver:body.Driver,Options:body.DriverOpts,Labels:body.Labels} + res.statusCode=candidateVolumeAckUnavailable?503:201;res.end(JSON.stringify(candidateVolumeMetadata));return + } indexedVolumeMetadata={Name:body.Name,Driver:body.Driver,Options:body.DriverOpts,Labels:body.Labels}; res.statusCode=201;res.end(JSON.stringify(indexedVolumeMetadata));return; } + if (req.url?.startsWith('/v1.47/volumes/freshell-claude-candidate-')) { + if (req.method==='DELETE') {candidateVolumeMetadata=undefined;res.statusCode=204;res.end();return} + res.statusCode=candidateVolumeMetadata?200:404;res.end(JSON.stringify(candidateVolumeMetadata ?? {}));return + } if (req.method==='GET' && req.url?.startsWith('/v1.47/volumes/freshell-inputs-') && indexedVolumeMetadata) { res.statusCode=200;res.end(JSON.stringify(indexedVolumeMetadata));return; } + const candidateCreation = forwarded.findLast((operation) => operation.url.includes('/containers/create?') && operation.body?.Labels?.['com.freshell.claude-candidate']==='ordinary') + if (req.method==='GET' && candidateCreation && (req.url===`/v1.47/containers/${helperId}/json` + || req.url===`${candidateCreation.url.replace('/containers/create?name=', '/containers/')}/json`)) { + if (candidateHelperDeleted) {res.statusCode=404;res.end('{}');return} + const inspected=structuredClone(candidateCreation.body) + inspected.Labels['org.opencontainers.image.version']='24.04' + for (const mount of inspected.HostConfig.Mounts) if (mount.ReadOnly===false) delete mount.ReadOnly + res.statusCode=200;res.end(JSON.stringify({Id:helperId,Config:foreignHelper?{...inspected,User:'foreign'}:inspected,HostConfig:inspected.HostConfig}));return + } if (req.method==='GET' && (req.url?.includes('-publisher/json') || req.url?.includes('/freshell-claude-source-') || (req.url===`/v1.47/containers/${helperId}/json` && forwarded.some((operation)=>operation.body?.Cmd?.[0]==='observe-claude-sources-private')))) { if (sourceHelperDeleted) { res.statusCode=404;res.end('{}');return } @@ -621,6 +792,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise return } if (req.method === 'DELETE' || req.url?.endsWith('/start') || req.url?.endsWith('/update')) { + if (req.method==='DELETE' && candidateCreation && req.url===`/v1.47/containers/${helperId}?force=1`) candidateHelperDeleted=true if (req.method==='DELETE' && (req.url===`/v1.47/containers/${helperId}?force=1` || req.url===`/v1.47/containers/${sourceName}?force=1`)) sourceHelperDeleted=true res.statusCode = 204 res.end() @@ -654,7 +826,8 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise try { await new Promise((resolve) => upstream.listen(socket, resolve)) await instance.start() - await run({ request, forwarded, broker: instance, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true} }) + await run({ request, forwarded, broker: instance, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true}, + loseCandidateVolumeAck:()=>{candidateVolumeAckUnavailable=true},replaceCandidateVolumeInspection:()=>{candidateVolumeMetadata.Labels['com.freshell.soul-id']='foreign'} }) } finally { await instance.close() await new Promise((resolve) => upstream.close(() => resolve())) From ec7825fdf2f5015d213dc2b4a7e844ffeabbee95 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 06:04:29 -0700 Subject: [PATCH 129/493] Accept Docker empty tmpfs metadata for fixed Claude candidate helpers --- .../src/backend/claude_source_observation.rs | 9 ++++++++- .../src/backend/claude_source_observation_tests.rs | 8 ++++++-- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index c18d022b6..60c024134 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -768,6 +768,9 @@ fn read_candidate_metadata( } fn owned_helper_profile(actual: &Value, expected: &Value) -> bool { + let empty_tmpfs = |value: &Value| { + value.is_null() || value.as_object().is_some_and(|mounts| mounts.is_empty()) + }; input_publisher::owned_helper(actual, expected) && actual["Config"]["WorkingDir"] == expected["WorkingDir"] && [ @@ -780,10 +783,14 @@ fn owned_helper_profile(actual: &Value, expected: &Value) -> bool { "MemorySwap", "NanoCpus", "PidsLimit", - "Tmpfs", ] .iter() .all(|key| actual["HostConfig"][key] == expected["HostConfig"][key]) + // Docker reports an empty create-time map as null. Both mean no + // tmpfs mount; a populated map still requires exact equality. + && (actual["HostConfig"]["Tmpfs"] == expected["HostConfig"]["Tmpfs"] + || (empty_tmpfs(&actual["HostConfig"]["Tmpfs"]) + && empty_tmpfs(&expected["HostConfig"]["Tmpfs"]))) } fn owned_helper(actual: &Value, expected: &Value) -> bool { diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 16a52d1ad..c5c39ee9e 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -696,6 +696,8 @@ async fn qualify_candidate_helper_transport( ) { for mode in [ "success", + "docker-empty-tmpfs", + "unexpected-tmpfs", "wrong-caps", "lost-projection-ack", "projection-timeout", @@ -749,7 +751,9 @@ async fn qualify_candidate_helper_transport( else if line.starts_with("GET") && line.contains("/containers/"){let key=line.split("/containers/").nth(1).unwrap().split('/').next().unwrap();let found=bodies.lock().unwrap().iter().find(|(name,v)|*name==key || v["Id"]==key).map(|(_,v)|v.clone());match found {Some(mut v)=>{ let reports=v["HostConfig"]["Mounts"].as_array().unwrap().iter().map(|m|json!({"Type":m["Type"],"Destination":m["Target"],"RW":!m["ReadOnly"].as_bool().unwrap(),"Source":if m["Type"]=="volume" {json!("/synthetic/ordinary-volume")} else {m["Source"].clone()},"Name":if m["Type"]=="volume" {m["Source"].clone()} else {Value::Null}})).collect::>(); v["Mounts"]=json!(reports); + if mode=="docker-empty-tmpfs" && v["Config"]["Cmd"]==json!(["project-claude-candidate-private"]) {v["HostConfig"]["Tmpfs"]=Value::Null;} if key.len()==64 && v["Config"]["Cmd"]==json!(["project-claude-candidate-private"]) { + if mode=="unexpected-tmpfs" {v["HostConfig"]["Tmpfs"]=json!({"/foreign":"rw,size=1m"});} if mode=="wrong-caps" {v["HostConfig"]["CapAdd"]=json!(["SYS_ADMIN"]);} if mode=="wrong-mount-report" {v["Mounts"][0]["RW"]=json!(true);} if mode=="wrong-resource-subpath" {let mount=v["HostConfig"]["Mounts"].as_array_mut().unwrap().iter_mut().find(|m|m["VolumeOptions"]["Subpath"].is_string()).unwrap();mount["VolumeOptions"]["Subpath"]=json!("foreign/ordinary");} @@ -802,7 +806,7 @@ async fn qualify_candidate_helper_transport( .unwrap(); assert_eq!( result.is_ok(), - mode == "success", + matches!(mode, "success" | "docker-empty-tmpfs"), "{mode}: candidate projection and confirmed owned cleanup gate evidence: {result:?}" ); assert!( @@ -816,7 +820,7 @@ async fn qualify_candidate_helper_transport( let seen = calls.lock().unwrap(); if matches!( mode, - "wrong-caps" | "wrong-mount-report" | "wrong-resource-subpath" + "unexpected-tmpfs" | "wrong-caps" | "wrong-mount-report" | "wrong-resource-subpath" ) { assert!( !seen From b6c9b83128d42ee7af4f17b0ad82a50573ca83a4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 06:18:39 -0700 Subject: [PATCH 130/493] Project fixed Claude candidates on provider-owned disk --- .../src/claude_source_observation.rs | 98 ++++++++++++++++--- crates/freshell-session-host/src/main.rs | 10 +- 2 files changed, 91 insertions(+), 17 deletions(-) diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index e12c333b5..043260cd3 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -217,22 +217,24 @@ pub(crate) fn serve_stdio() -> Result<(), String> { exchange(&mut std::io::stdin().lock(), &mut std::io::stdout().lock()) } -fn project_candidate( +fn candidate_projection_unavailable(operation: &'static str) -> String { + tracing::warn!( + event = "provider.inputs.candidate_projection.blocked", + provider = "claude", + stage = "candidate", + operation, + "Candidate ordinary projection unavailable" + ); + format!("Claude candidate {operation} unavailable; retain inputs and reprepare") +} + +fn validate_candidate_projection( request: &freshell_runtime_protocol::ClaudeCandidateProjectionRequest, stage: &std::path::Path, home: &std::path::Path, identity: (u32, u32), ) -> Result<(), String> { - let unavailable = |operation: &'static str| { - tracing::warn!( - event = "provider.inputs.candidate_projection.blocked", - provider = "claude", - stage = "candidate", - operation, - "Candidate ordinary projection unavailable" - ); - format!("Claude candidate {operation} unavailable; retain inputs and reprepare") - }; + let unavailable = candidate_projection_unavailable; request.validate().map_err(|_| unavailable("metadata"))?; let published = crate::provider_input_preparation::read_private( stage, @@ -277,6 +279,15 @@ fn project_candidate( { return Err(unavailable("selection_binding")); } + Ok(()) +} + +fn refresh_candidate_projection( + request: &freshell_runtime_protocol::ClaudeCandidateProjectionRequest, + stage: &std::path::Path, + home: &std::path::Path, + identity: (u32, u32), +) -> Result<(), String> { // Reuse the workload's selected copier and stale-leaf transaction. This // never resolves secrets, probes a provider, or reads original HOME again. crate::refresh_provider_config_references( @@ -288,7 +299,18 @@ fn project_candidate( identity.0, identity.1, ) - .map_err(|_| unavailable("ordinary_refresh")) + .map_err(|_| candidate_projection_unavailable("ordinary_refresh")) +} + +#[cfg(test)] +fn project_candidate( + request: &freshell_runtime_protocol::ClaudeCandidateProjectionRequest, + stage: &std::path::Path, + home: &std::path::Path, + identity: (u32, u32), +) -> Result<(), String> { + validate_candidate_projection(request, stage, home, identity)?; + refresh_candidate_projection(request, stage, home, identity) } fn read_candidate_request( @@ -390,7 +412,7 @@ pub(crate) fn serve_candidate_private() -> Result<(), String> { let raw = freshell_runtime_protocol::bounded_json_bytes(&request.source_request, MAX_REQUEST_BYTES) .map_err(|_| "Claude candidate source request exceeds metadata profile")?; - project_candidate( + validate_candidate_projection( &request, Path::new("/run/freshell-candidate-inputs"), home, @@ -405,6 +427,15 @@ pub(crate) fn serve_candidate_private() -> Result<(), String> { return Err("Claude candidate provider identity transition unavailable".into()); } crate::verify_private_helper_namespace(home, socket).map_err(|_| unavailable())?; + // Private publication is validated by the fixed host above. Ordinary + // projection now runs as the same provider identity that will read it, + // using its owned disk transaction instead of read-only ambient /tmp. + refresh_candidate_projection( + &request, + Path::new("/run/freshell-candidate-inputs"), + home, + (65534, 0), + )?; if request.source_request.native_source_runtime == freshell_platform::claude_inputs::NativeSourceRuntime::Terminal { @@ -757,10 +788,32 @@ mod tests { b"PRIVATE_RETAINED_AUTH_SENTINEL" ); } - project_candidate(&request, &stage, &home, unsafe { + // A private subprocess changes only its own ambient temporary + // root. The selected frozen inputs and provider disk stay usable. + let blocked_ambient_temp = + std::env::var_os("FRESHELL_TEST_CANDIDATE_DISK_TEMP").is_some(); + let previous_temp = std::env::var_os("TMPDIR"); + if blocked_ambient_temp { + std::env::set_var("TMPDIR", &retained_auth); + } + let projection_result = project_candidate(&request, &stage, &home, unsafe { (libc::geteuid(), libc::getegid()) - }) - .unwrap(); + }); + if blocked_ambient_temp { + match previous_temp { + Some(value) => std::env::set_var("TMPDIR", value), + None => std::env::remove_var("TMPDIR"), + } + } + projection_result.expect("frozen ordinary projection must use provider disk even when ambient temporary storage is unavailable"); + assert!( + std::fs::read_dir(&home).unwrap().all(|entry| !entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with(".freshell-provider-refresh-")), + "owned disk transaction must be removed before accepting projection" + ); let output = std::process::Command::new("node") .arg(home.join(".freshell/provider-roots/unit-0/consumer.cjs")) .arg(home.join(".claude/settings.json")) @@ -789,6 +842,19 @@ mod tests { } } + #[test] + fn claude_candidate_projection_uses_provider_disk_without_ambient_temporary_storage() { + let result = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "claude_source_observation::tests::claude_candidate_projection_runs_selected_dependency_and_frozen_config_consumer", "--nocapture", "--test-threads=1"]) + .env("FRESHELL_TEST_CANDIDATE_DISK_TEMP", "1") + .status() + .unwrap(); + assert!( + result.success(), + "the isolated frozen consumer must succeed with ambient temporary storage unavailable" + ); + } + struct ReaderCase { result: Result<(), RuntimeError>, joined_before_test_cleanup: bool, diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index b7f14a1b4..fbdc049df 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -2001,7 +2001,15 @@ fn refresh_provider_config_references( run_as_gid: u32, ) -> Result<(), String> { context.validate(provider).map_err(|error| error.message)?; - let stage = tempfile::tempdir().map_err(|error| error.to_string())?; + let stage = + if run_as_uid == unsafe { libc::geteuid() } && run_as_gid == unsafe { libc::getegid() } { + tempfile::Builder::new() + .prefix(".freshell-provider-refresh-") + .tempdir_in(provider_home) + } else { + tempfile::tempdir() + } + .map_err(|error| error.to_string())?; let projection = stage.path().join("projection"); std::fs::create_dir(&projection).map_err(|error| error.to_string())?; copy_provider_config_references_with_ownership( From 87d1b0df479570163a5d30418ed5eb8e61b35876 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 06:18:53 -0700 Subject: [PATCH 131/493] Qualify request-bound initial Claude candidate helpers in native sandbox --- .../src/managed_provider_launch_tests.rs | 130 +++++++++++++++++- .../testing/managed-provider-launch-broker.ts | 18 ++- .../managed-provider-launch-sandbox.py | 38 ++++- 3 files changed, 181 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 96c84f08b..b5f765925 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -60,6 +60,117 @@ struct BrokerAuthorizedBackend { certificate_sentinel: Vec, } impl BrokerAuthorizedBackend { + async fn authorize_claude_candidate( + &self, + owner: &freshell_supervisor::backend::ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + ) -> Result<(), BackendError> { + use std::io::Read; + let Some(candidate) = request.candidate.as_ref() else { + return Ok(()); + }; + let invalid = || { + BackendError::InvalidConfig("closed initial Claude candidate handoff invalid".into()) + }; + let freshell_supervisor::backend::ClaudeSourceOwner::Initial { observation, .. } = owner + else { + // The retained reader stays fail-closed until its actual prior + // ordinary topology is qualified. Do not invent initial authority. + return Ok(()); + }; + let InitialAuthVolumeObservation::Absent { daemon_id } = observation; + let ClaudeSourceAuthority::InitialReservation { reservation_id } = &request.authority + else { + return Err(invalid()); + }; + request.validate().map_err(|_| invalid())?; + let inputs = candidate + .provider_launch_context + .prepared_inputs + .as_ref() + .ok_or_else(invalid)?; + let binding = request + .observation_request + .candidate_binding + .as_ref() + .ok_or_else(invalid)?; + if binding.soul_id != request.soul_id.as_str() + || binding.generation_id != inputs.generation_id + || inputs.initial_auth_reservation.as_deref() != Some(reservation_id.as_str()) + || inputs.indexed_files.is_some() + || inputs.manifest.source_count != 0 + || inputs.manifest.alias_count != 0 + || inputs.manifest.recipe_count != 0 + { + return Err(invalid()); + } + let root = self.handoff.parent().ok_or_else(invalid)?; + // The wrapper's closed source configuration contains only artifact and + // owner metadata. Bound the physical file before allocating its frame. + let mut file = + std::fs::File::open(root.join("broker-configuration.json")).map_err(|_| invalid())?; + let length = file.metadata().map_err(|_| invalid())?.len(); + if length > MAX_CONTROL_FRAME_BYTES as u64 { + return Err(invalid()); + } + let mut bytes = vec![0; length as usize]; + file.read_exact(&mut bytes).map_err(|_| invalid())?; + let mut growth = [0; 1]; + if file.read(&mut growth).map_err(|_| invalid())? != 0 { + return Err(invalid()); + } + let configuration: Value = serde_json::from_slice(&bytes).map_err(|_| invalid())?; + let sources = configuration["initialVolumes"] + .as_array() + .ok_or_else(invalid)?; + let mut matching = sources + .iter() + .filter(|source| source["soulId"] == request.soul_id.as_str()); + let source = matching.next().ok_or_else(invalid)?; + if matching.next().is_some() || source["daemonId"] != daemon_id.as_str() { + return Err(invalid()); + } + let value = json!({ + "installationId":source["installationId"],"soulId":request.soul_id, + "providerVolumeName":source["providerVolumeName"],"imageRef":source["imageRef"], + "hostBinaryPath":source["hostBinaryPath"],"binarySha256":source["binarySha256"],"daemonId":source["daemonId"], + "reservationId":reservation_id,"requestId":request.observation_request.request_id,"generationId":inputs.generation_id, + "stageRoot":self.control.join("provider-roots").join(&inputs.generation_id), + "workspacePath":request.workspace_path,"cwd":request.observation_request.cwd, + "manifestRelativePath":inputs.manifest.relative_path, + "configurationBinds":inputs.config_binds.iter().map(|bind|json!({"sourceRelativePath":bind.source_relative_path,"effectiveDestination":bind.effective_destination})).collect::>(), + "resourceMounts":inputs.resource_mounts.iter().map(|mount|json!({"stageRelativePath":mount.stage_relative_path,"providerRelativePath":mount.provider_relative_path,"effectiveDestination":mount.effective_destination,"kind":mount.kind})).collect::>(), + }); + let mut value = value; + if let Some(common) = &request.git_common_dir { + value["gitCommonDir"] = json!(common); + } + let metadata = + bounded_json_bytes(&value, MAX_CONTROL_FRAME_BYTES).map_err(|_| invalid())?; + let path = root + .join("candidate-handoff") + .join(format!("{}.json", request.observation_request.request_id)); + let temporary = path.with_extension("pending"); + std::fs::write(&temporary, metadata).map_err(|_| invalid())?; + std::fs::rename(&temporary, &path).map_err(|_| invalid())?; + assert_private_sentinels_absent( + &root.join("candidate-handoff"), + &self.certificate_sentinel, + ); + tokio::time::timeout(Duration::from_secs(10), async { + while !path.with_extension("accepted").is_file() { + tokio::time::sleep(Duration::from_millis(20)).await; + } + }) + .await + .map_err(|_| { + BackendError::Unavailable( + "closed Claude candidate broker authorization timed out".into(), + ) + })?; + Ok(()) + } + async fn authorize(&self, spec: &CreateRuntimeSpec) -> Result<(), BackendError> { let invalid = || BackendError::InvalidConfig("closed native fixture metadata handoff invalid".into()); @@ -144,9 +255,24 @@ impl RuntimeBackend for BrokerAuthorizedBackend { request: &ObserveClaudeSourcesRequest, test_run_id: &str, ) -> Result { - self.real + self.authorize_claude_candidate(owner, request).await?; + let reply = self + .real .observe_claude_sources(owner, request, test_run_id) - .await + .await?; + if let Some(binding) = &request.observation_request.candidate_binding { + println!( + "provider_candidate_observer {}", + json!({ + "requestId":request.observation_request.request_id,"generationId":binding.generation_id, + "soulId":request.soul_id,"planRevision":request.observation_request.plan_revision, + "runtime":request.observation_request.native_source_runtime,"uid":reply.observation.uid, + "gid":reply.observation.gid,"supplementaryGroups":reply.observation.supplementary_groups, + "backendCompleteAfterCleanup":true, + }) + ); + } + Ok(reply) } async fn observe_initial_auth_volume( &self, diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 12cae5a00..5e525c0c2 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -2,7 +2,7 @@ //! the native startup fixture receives this broker's restricted proxy socket. import fs from 'node:fs' import path from 'node:path' -import { RestrictedDockerBroker, type PreparedInputAuthorization } from './runtime-test-broker.js' +import { RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from './runtime-test-broker.js' const root=process.env.FRESHELL_PROVIDER_LAUNCH_FIXTURE ?? '' if (path.dirname(root)!=='/tmp' || !/^fln-[a-f0-9]{16}$/.test(path.basename(root))) { @@ -39,6 +39,7 @@ const broker=new RestrictedDockerBroker({ for (const authorization of configuration.initialVolumes) broker.authorizeInitialProviderVolume(authorization) await broker.start() const accepted=new Set() +const candidates=new Map() const timer=setInterval(()=>{ try { const gatewayFile=path.join(root,'controller-gateway.json') @@ -65,11 +66,26 @@ const timer=setInterval(()=>{ accepted.add(name) fs.writeFileSync(file.slice(0,-'.json'.length)+'.accepted','ok',{mode:0o600}) } + const candidateDirectory=path.join(root,'candidate-handoff') + const candidateEntries=fs.readdirSync(candidateDirectory).filter((name)=>name.endsWith('.json')) + if (candidateEntries.length>6 || (configuration.provider!=='claude' && candidateEntries.length)) throw new Error('closed candidate handoff exceeds two routes and two replans') + for (const name of candidateEntries) { + if (candidates.has(name)) continue + if (!/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}\.json$/.test(name)) throw new Error('closed candidate handoff request identity invalid') + const file=path.join(candidateDirectory,name),stat=fs.lstatSync(file) + if (!stat.isFile() || stat.isSymbolicLink() || stat.size>1024*1024) throw new Error('closed candidate handoff profile exceeded') + const input=JSON.parse(fs.readFileSync(file,'utf8')) as ClaudeCandidateAuthorization + if (name!==`${input.requestId}.json`) throw new Error('closed candidate handoff request changed') + broker.authorizeClaudeCandidate(input) + candidates.set(name,input) + fs.writeFileSync(file.slice(0,-'.json'.length)+'.accepted','ok',{mode:0o600}) + } if (fs.existsSync(path.join(root,'broker-complete'))) { clearInterval(timer) void broker.close().then(()=>{ fs.writeFileSync(path.join(root,'broker-receipt.json'),JSON.stringify({ events:broker.eventsSnapshot(),receipts:broker.receipts(),unsafeAttempts:broker.unsafeAttempts(),acceptedPreparations:accepted.size, + authorizedCandidates:[...candidates.values()], })+'\n',{mode:0o600}) }) } diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 3c25877b4..1d05b1fdb 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -74,6 +74,15 @@ def remove(name): if labels.get("com.freshell.soul-id")==metadata["soulId"] and labels.get("com.freshell.incarnation-id")==metadata["incarnationId"]: known_helpers.append(metadata) fixed_helper=value["Config"].get("Image")==IMAGE and value["Config"].get("Cmd") in [["publish-provider-inputs"],["prepare-provider-mountpoints"]] and bool(known_helpers) + candidate_helper=value["Config"].get("Image")==IMAGE and value["Config"].get("Cmd") in [["project-claude-candidate-private"],["prepare-provider-mountpoints"]] and labels.get("com.freshell.runtime-test-run-id")==run and labels.get("com.freshell.installation-id")==installation and labels.get("com.freshell.claude-candidate")=="ordinary" + if candidate_helper: + host_config=value["HostConfig"] + receipt.setdefault("candidateHelperInspections",[]).append({ + "id":value["Id"],"image":value["Config"]["Image"],"user":value["Config"]["User"], + "command":value["Config"]["Cmd"],"labels":labels,"workingDirectory":value["Config"].get("WorkingDir"), + "running":value["State"].get("Running"),"exitCode":value["State"].get("ExitCode"), + "hostSafety":{key:host_config.get(key) for key in ["NetworkMode","ReadonlyRootfs","CapDrop","CapAdd","SecurityOpt","Memory","MemorySwap","NanoCpus","PidsLimit","Tmpfs","LogConfig","Mounts"]}, + }) if fixed_helper: receipt.setdefault("helperInspections",[]).append({ "id":value["Id"],"image":value["Config"]["Image"],"user":value["Config"]["User"], @@ -94,7 +103,7 @@ def create(name,*arguments): root=Path(temporary).with_name("fln-"+uuid.uuid4().hex[:16]) Path(temporary).rename(root) try: - for name in ["workspace","source-home/.config/opencode","grants","handoff","tmp","scenarios"]: + for name in ["workspace","source-home/.config/opencode","grants","handoff","candidate-handoff","tmp","scenarios"]: (root/name).mkdir(parents=True,exist_ok=True) initial=[] souls=["soul-terminal-"+hashlib.sha256(b"closed-native-terminal-create").hexdigest()[:32]] @@ -197,6 +206,7 @@ def create(name,*arguments): receipt["claudeSettingsForm"]=args.claude_settings_form receipt["claudeSourceGuard"]=args.claude_source_guard receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] + receipt["candidateObservers"]=[json.loads(line.split("provider_candidate_observer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_candidate_observer " in line] print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} @@ -215,6 +225,16 @@ def create(name,*arguments): evidence=json.loads((root/"broker-receipt.json").read_text()) receipt["broker"]=evidence for value in evidence["receipts"]:owned.append(value["containerId"]) + if exit_code==0 and args.provider=="claude" and args.claude_source_guard=="unchanged": + candidates=evidence["authorizedCandidates"] + observers=receipt["candidateObservers"] + authorized={(value["requestId"],value["generationId"],value["soulId"]) for value in candidates} + observed={(value.get("requestId"),value.get("generationId"),value.get("soulId")) for value in observers} + # Authorization alone is not source acceptance. Require + # real backend replies after its owned helper cleanup, + # plus the native consumer and terminal test above. + if not 2<=len(candidates)<=6 or len(authorized)!=len(candidates) or observed!=authorized or len(observers)!=len(candidates) or any(value.get("uid")!=65534 or value.get("gid")!=0 or value.get("backendCompleteAfterCleanup") is not True for value in observers): + receipt["exitCode"]=2;receipt["failureClass"]="candidate_observer_count_mismatch" else: receipt["exitCode"]=2;receipt["failureClass"]="broker_receipt_unavailable" finally: @@ -228,6 +248,13 @@ def create(name,*arguments): for candidate in (root/"handoff").glob("*.json"): value=json.loads(candidate.read_text()) if value.get("indexedFiles"):volumes.add(value["indexedFiles"]["volumeName"]) + candidates=[json.loads(candidate.read_text()) for candidate in (root/"candidate-handoff").glob("*.json")] + if len(candidates)>6: + raise RuntimeError("closed candidate cleanup exceeds its finite request set") + for candidate in candidates: + if not re.fullmatch(r"[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}",candidate["requestId"]): + raise RuntimeError("closed candidate cleanup request identity invalid") + volumes.add("freshell-claude-candidate-"+candidate["requestId"]) for volume in volumes: inspected=docker("volume","inspect",volume,check=False) if inspected.returncode:continue @@ -245,7 +272,14 @@ def create(name,*arguments): labels={"com.freshell.managed-inputs":"true","com.freshell.installation-id":installation, "com.freshell.soul-id":prepared["soulId"],"com.freshell.incarnation-id":prepared["incarnationId"], "com.freshell.input-generation":prepared["generationId"]} - if labels is None or value.get("Driver")!="local" or value.get("Labels")!=labels: + for candidate in candidates: + if volume=="freshell-claude-candidate-"+candidate["requestId"]: + source=next(item for item in initial if item["soulId"]==candidate["soulId"]) + labels={"com.freshell.installation-id":installation,"com.freshell.soul-id":candidate["soulId"], + "com.freshell.runtime-test-run-id":run,"com.freshell.source-request-id":candidate["requestId"], + "com.freshell.source-binary-sha256":source["binarySha256"],"com.freshell.input-generation":candidate["generationId"], + "com.freshell.source-reservation-id":candidate["reservationId"],"com.freshell.claude-candidate":"ordinary"} + if labels is None or value.get("Driver")!="local" or value.get("Options") not in (None,{}) or value.get("Labels")!=labels: raise RuntimeError("closed startup volume cleanup ownership changed") docker("volume","rm",volume) receipt["ownedCleanup"]=all(docker("container","inspect",name,check=False).returncode!=0 for name in owned) and all(docker("volume","inspect",volume,check=False).returncode!=0 for volume in volumes) From 992092c522a8b5db98b98a1fad8bbe0ff2cf54bc Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 07:44:00 -0700 Subject: [PATCH 132/493] Stage ordinary provider refreshes on provider-owned disk with bounded cleanup --- .../src/managed_provider_launch_tests.rs | 9 +- crates/freshell-session-host/src/main.rs | 396 +++++++++++++++++- .../tests/provider_config_refresh.rs | 155 +++++++ .../managed-provider-launch-sandbox.py | 5 +- 4 files changed, 546 insertions(+), 19 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index b5f765925..4668078bf 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -999,6 +999,10 @@ async fn managed_provider_claude_terminal_and_fresh_native_startup_consumes_prep .unwrap(); std::fs::write(plugin.join("skills/probe/SKILL.md"),"---\nname: closed-plugin-probe\ndescription: selected complete native plugin\n---\nUse the adjacent resources.\n").unwrap(); std::fs::write(plugin.join("scripts/resource"), b"closed-claude-resource").unwrap(); + // Ordinary selected resources must use their disk budget, not the + // workload's deliberately small temporary-memory filesystem. + let disk_resource = vec![0x5a; 128 * 1024 * 1024]; + std::fs::write(plugin.join("scripts/disk-resource"), &disk_resource).unwrap(); std::fs::write( plugin.join("scripts/node_modules/probe-dependency/index.js"), "module.exports='closed-claude-dependency'\n", @@ -1021,7 +1025,8 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ const endpoint=process.env.FRESHELL_URL,token=process.env.FRESHELL_TOKEN; const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,nativeIdentity:{uid:process.getuid(),euid:process.geteuid(),gid:process.getgid(),egid:process.getegid(),groups:[...new Set([...process.getgroups(),process.getegid()])].sort((a,b)=>a-b)},settings:['closed-selected-settings','closed-explicit-settings'].includes(process.env.CLAUDE_SELECTED_NONCE),authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], - memory:hash(fs.readFileSync(path.join(process.env.HOME,'.claude/CLAUDE.md'))),skill:hash(fs.readFileSync(path.join(root,'skills/probe/SKILL.md'))),resource:hash(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:hash(require('probe-dependency'))}; + refreshStageAbsent:fs.readdirSync(process.env.HOME).every(name=>!name.startsWith('.freshell-provider-refresh-')), + memory:hash(fs.readFileSync(path.join(process.env.HOME,'.claude/CLAUDE.md'))),skill:hash(fs.readFileSync(path.join(root,'skills/probe/SKILL.md'))),resource:hash(require('node:child_process').execFileSync(path.join(__dirname,'run'))),diskResource:hash(fs.readFileSync(path.join(__dirname,'disk-resource'))),dependency:hash(require('probe-dependency'))}; fs.writeFileSync(process.argv[2],JSON.stringify(record)); }); "#; @@ -1457,7 +1462,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .ok() } let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":terminal_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}); + let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":terminal_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"refreshStageAbsent":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"diskResource":hash(&disk_resource),"dependency":hash(b"closed-claude-dependency")}); let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); let mut expected_imports = serde_json::Map::new(); expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index fbdc049df..961bb6e0a 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -2001,15 +2001,39 @@ fn refresh_provider_config_references( run_as_gid: u32, ) -> Result<(), String> { context.validate(provider).map_err(|error| error.message)?; - let stage = - if run_as_uid == unsafe { libc::geteuid() } && run_as_gid == unsafe { libc::getegid() } { - tempfile::Builder::new() - .prefix(".freshell-provider-refresh-") - .tempdir_in(provider_home) - } else { - tempfile::tempdir() - } + if run_as_uid != unsafe { libc::geteuid() } || run_as_gid != unsafe { libc::getegid() } { + return refresh_provider_config_as_provider( + context, + provider, + workspace, + user_provider_root, + provider_home, + run_as_uid, + run_as_gid, + ); + } + let stage = tempfile::Builder::new() + .prefix(".freshell-provider-refresh-") + .tempdir_in(provider_home) .map_err(|error| error.to_string())?; + refresh_provider_config_in_stage( + context, + provider, + workspace, + user_provider_root, + provider_home, + stage, + ) +} + +fn refresh_provider_config_in_stage( + context: &freshell_runtime_protocol::ProviderLaunchContext, + provider: &str, + workspace: &Path, + user_provider_root: &Path, + provider_home: &Path, + stage: tempfile::TempDir, +) -> Result<(), String> { let projection = stage.path().join("projection"); std::fs::create_dir(&projection).map_err(|error| error.to_string())?; copy_provider_config_references_with_ownership( @@ -2055,19 +2079,59 @@ fn refresh_provider_config_references( .map_err(|_| "resource mountpoint metadata profile exceeded")?; std::fs::write(stage.path().join("resource-mountpoints.json"), mounted) .map_err(|_| "resource mountpoint metadata write failed")?; - allow_provider_read_stage(stage.path(), run_as_gid)?; - if run_as_uid == unsafe { libc::geteuid() } && run_as_gid == unsafe { libc::getegid() } { - return apply_staged_provider_config(stage.path(), provider_home); - } + allow_provider_read_stage(stage.path(), unsafe { libc::getegid() })?; + apply_staged_provider_config(stage.path(), provider_home) +} + +fn refresh_provider_config_as_provider( + context: &freshell_runtime_protocol::ProviderLaunchContext, + provider: &str, + workspace: &Path, + user_provider_root: &Path, + provider_home: &Path, + uid: u32, + gid: u32, +) -> Result<(), String> { + use std::io::{Seek, Write}; + // Only bounded control metadata uses temporary memory. The child performs + // the complete ordinary transaction on disk as the actual provider UID. + let bytes = freshell_runtime_protocol::bounded_json_bytes( + context, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| "provider-owned refresh control metadata exceeds its profile")?; + let mut input = tempfile::tempfile().map_err(|_| "provider-owned refresh input unavailable")?; + input + .write_all(&(bytes.len() as u32).to_be_bytes()) + .and_then(|()| input.write_all(&bytes)) + .and_then(|()| input.rewind()) + .map_err(|_| "provider-owned refresh input incomplete")?; let exe = std::env::current_exe().map_err(|error| error.to_string())?; - let status = std::process::Command::new("/usr/bin/setpriv") - .args(provider_identity_args(run_as_uid, run_as_gid)) + let transaction = uuid::Uuid::new_v4().to_string(); + let stage = provider_home.join(format!(".freshell-provider-refresh-{transaction}")); + let mut child = std::process::Command::new("/usr/bin/setpriv") + .args(provider_identity_args(uid, gid)) .arg(exe) .arg("refresh-provider-config") - .arg(stage.path()) + .arg("--prepare") + .arg(provider) + .arg(workspace) + .arg(user_provider_root) .arg(provider_home) - .status() + .arg(&transaction) + .env_clear() + .stdin(std::process::Stdio::from(input)) + .stdout(std::process::Stdio::null()) + .spawn() .map_err(|error| format!("provider-owned config refresh failed to start: {error}"))?; + let status = wait_for_provider_config_refresh( + &mut child, + provider, + uid, + gid, + std::time::Duration::from_secs(25), + &stage, + )?; if status.success() { Ok(()) } else { @@ -2077,6 +2141,93 @@ fn refresh_provider_config_references( } } +fn wait_for_provider_config_refresh( + child: &mut std::process::Child, + provider: &str, + uid: u32, + gid: u32, + timeout: std::time::Duration, + stage: &Path, +) -> Result { + let deadline = std::time::Instant::now() + timeout; + loop { + match child.try_wait() { + Ok(Some(status)) => { + cleanup_provider_refresh_stage(stage, uid, gid)?; + return Ok(status); + } + Ok(None) if std::time::Instant::now() < deadline => { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + Err(_) => { + tracing::warn!(event="provider.inputs.refresh.completion_unavailable", provider, stage="session_host", "Provider-owned ordinary refresh wait ownership was not confirmed; retain inputs and repair"); + return Err("provider-owned config refresh completion unavailable; retain inputs and repair".into()); + } + Ok(None) => { + // The restricted root host has no CAP_KILL. Signal only this + // unreaped child through its existing provider identity. + let pid = child.id().to_string(); + let termination = + run_provider_refresh_cleanup(uid, gid, "/usr/bin/kill", &["-KILL", "--", &pid]); + let cleanup_deadline = + std::time::Instant::now() + std::time::Duration::from_secs(5); + loop { + match child.try_wait() { + Ok(Some(_)) => { + cleanup_provider_refresh_stage(stage, uid, gid)?; + break; + } + Ok(None) if std::time::Instant::now() < cleanup_deadline => { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + _ => { + tracing::warn!(event="provider.inputs.refresh.cleanup_failed", provider, stage="session_host", worker_pid=%pid, termination_succeeded=termination.is_ok(), "Provider-owned ordinary refresh join was not confirmed"); + return Err("provider-owned config refresh cleanup unavailable; retain inputs and repair".into()); + } + } + } + return Err( + "provider-owned config refresh timed out; retain inputs and retry".into(), + ); + } + } + } +} + +fn cleanup_provider_refresh_stage(stage: &Path, uid: u32, gid: u32) -> Result<(), String> { + let stage = stage + .to_str() + .ok_or("provider refresh transaction path is not UTF-8")?; + // This path is one newly generated transaction, never an accepted provider + // projection. The UID worker is joined before its directory can be removed. + run_provider_refresh_cleanup(uid, gid, "/usr/bin/rm", &["-rf", "--", stage]).map_err(|_| { + "provider-owned refresh transaction cleanup unavailable; retain inputs and repair".into() + }) +} + +fn run_provider_refresh_cleanup( + uid: u32, + gid: u32, + program: &str, + args: &[&str], +) -> Result<(), String> { + let status = std::process::Command::new("/usr/bin/setpriv") + .args(provider_identity_args(uid, gid)) + .args(["/usr/bin/timeout", "--kill-after=1s", "4s", program]) + .args(args) + .env_clear() + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .status() + .map_err(|_| "provider refresh cleanup failed to start")?; + if status.success() { + Ok(()) + } else { + Err("provider refresh cleanup failed or timed out".into()) + } +} + fn provider_identity_args(uid: u32, gid: u32) -> Vec { let group_option = if uid == unsafe { libc::geteuid() } && gid == unsafe { libc::getegid() } { "--keep-groups" @@ -2123,6 +2274,59 @@ fn allow_provider_read_stage(path: &Path, gid: u32) -> Result<(), String> { } fn refresh_provider_config_worker(args: &[String]) -> Result<(), String> { + if args.len() == 6 && args[0] == "--prepare" { + use std::io::Read; + let mut input = std::io::stdin().lock(); + let mut header = [0; 4]; + input + .read_exact(&mut header) + .map_err(|_| "provider refresh control header unavailable")?; + let length = u32::from_be_bytes(header) as usize; + if length > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { + return Err("provider refresh control metadata exceeds its profile".into()); + } + let mut bytes = Vec::new(); + bytes + .try_reserve_exact(length) + .map_err(|_| "provider refresh control allocation unavailable")?; + bytes.resize(length, 0); + input + .read_exact(&mut bytes) + .map_err(|_| "provider refresh control input incomplete")?; + let mut trailing = [0]; + if input + .read(&mut trailing) + .map_err(|_| "provider refresh control completion unavailable")? + != 0 + { + return Err("provider refresh control input has trailing metadata".into()); + } + let context: freshell_runtime_protocol::ProviderLaunchContext = + serde_json::from_slice(&bytes) + .map_err(|_| "provider refresh control metadata invalid")?; + context.validate(&args[1]).map_err(|error| error.message)?; + let transaction = + uuid::Uuid::parse_str(&args[5]).map_err(|_| "provider refresh transaction invalid")?; + if transaction.get_version_num() != 4 || transaction.to_string() != args[5] { + return Err("provider refresh transaction invalid".into()); + } + let name = format!(".freshell-provider-refresh-{transaction}"); + let stage = tempfile::Builder::new() + .prefix(&name) + .rand_bytes(0) + .tempdir_in(&args[4]) + .map_err(|error| { + format!("provider refresh transaction creation unavailable: {error}") + })?; + return refresh_provider_config_in_stage( + &context, + &args[1], + Path::new(&args[2]), + Path::new(&args[3]), + Path::new(&args[4]), + stage, + ); + } if args.len() != 2 { return Err("refresh-provider-config requires stage and provider home".into()); } @@ -4281,6 +4485,166 @@ mod tests { assert!(event["elapsed_ms"].as_u64().is_some()); } + #[test] + fn provider_config_worker_timeout_removes_owned_stage_after_join() { + use super::*; + use std::time::{Duration, Instant}; + const STAGE_ENV: &str = "FRESHELL_TEST_PROVIDER_REFRESH_STAGE"; + if let Some(stage) = std::env::var_os(STAGE_ENV) { + let stage = PathBuf::from(stage); + std::fs::create_dir(&stage).unwrap(); + std::fs::write(stage.join("partial"), b"selected ordinary resource").unwrap(); + std::thread::sleep(Duration::from_secs(30)); + return; + } + let root = tempfile::tempdir().unwrap(); + let stage = root.path().join(format!( + ".freshell-provider-refresh-{}", + uuid::Uuid::new_v4() + )); + let retained = root.path().join("retained-state"); + std::fs::write(&retained, b"previous accepted projection").unwrap(); + let mut child = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "tests::provider_config_worker_timeout_removes_owned_stage_after_join", + ]) + .env(STAGE_ENV, &stage) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .spawn() + .unwrap(); + let ready_deadline = Instant::now() + Duration::from_secs(2); + while !stage.join("partial").is_file() { + if Instant::now() >= ready_deadline { + let _ = child.kill(); + let _ = child.wait(); + panic!("owned worker did not create its transaction stage"); + } + std::thread::sleep(Duration::from_millis(5)); + } + let started = Instant::now(); + let error = wait_for_provider_config_refresh( + &mut child, + "claude", + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + Duration::from_millis(30), + &stage, + ) + .unwrap_err(); + assert!(error.contains("timed out"), "{error}"); + assert!( + child.try_wait().unwrap().is_some(), + "owned worker must be joined" + ); + assert!(started.elapsed() < Duration::from_secs(6)); + assert_eq!( + std::fs::read(retained).unwrap(), + b"previous accepted projection" + ); + assert!( + !stage.exists(), + "joined timed-out worker left an ordinary transaction stage" + ); + } + + #[test] + fn provider_config_worker_unconfirmed_termination_preserves_stage() { + use super::*; + use std::time::{Duration, Instant}; + let root = tempfile::tempdir().unwrap(); + let stage = root.path().join(format!( + ".freshell-provider-refresh-{}", + uuid::Uuid::new_v4() + )); + let retained = root.path().join("retained-state"); + std::fs::write(&retained, b"previous accepted projection").unwrap(); + let mut child = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "tests::provider_config_worker_timeout_removes_owned_stage_after_join", + ]) + .env("FRESHELL_TEST_PROVIDER_REFRESH_STAGE", &stage) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .spawn() + .unwrap(); + let ready_deadline = Instant::now() + Duration::from_secs(2); + while !stage.join("partial").is_file() { + if Instant::now() >= ready_deadline { + let _ = child.kill(); + let _ = child.wait(); + panic!("owned worker did not create its transaction stage"); + } + std::thread::sleep(Duration::from_millis(5)); + } + let started = Instant::now(); + // The strict sandbox user cannot adopt this different UID. Exercise a + // real failed signal operation, rather than mocking a joined worker. + let outcome = wait_for_provider_config_refresh( + &mut child, + "claude", + unsafe { libc::geteuid() } + 1, + unsafe { libc::getegid() }, + Duration::from_millis(30), + &stage, + ); + let elapsed = started.elapsed(); + let worker_alive = child.try_wait().unwrap().is_none(); + let stage_retained = stage.join("partial").is_file(); + let accepted = std::fs::read(&retained).unwrap(); + // The test itself owns the exact child and can signal it as its actual + // UID. Join before assertions even when the production outcome is wrong. + if worker_alive { + child.kill().unwrap(); + } + child.wait().unwrap(); + let error = outcome.unwrap_err(); + assert!(error.contains("cleanup unavailable"), "{error}"); + assert!(worker_alive, "unconfirmed worker must remain observable"); + assert!(stage_retained, "unconfirmed worker inputs must be retained"); + assert_eq!(accepted, b"previous accepted projection"); + assert!( + elapsed < Duration::from_secs(6), + "uncertain cleanup must settle finitely" + ); + } + + #[test] + fn provider_config_worker_lost_wait_ownership_returns_incomplete_evidence() { + use super::*; + let root = tempfile::tempdir().unwrap(); + let stage = root.path().join(format!( + ".freshell-provider-refresh-{}", + uuid::Uuid::new_v4() + )); + std::fs::create_dir(&stage).unwrap(); + std::fs::write(stage.join("retained"), b"uncertain ordinary transaction").unwrap(); + let mut child = std::process::Command::new("/usr/bin/true").spawn().unwrap(); + let mut exit = 0; + // Reap only the child this test created, through another real waiter. + // The Child handle can no longer prove that its old PID is owned. + assert_eq!( + unsafe { libc::waitpid(child.id() as libc::pid_t, &mut exit, 0) }, + child.id() as libc::pid_t + ); + let error = wait_for_provider_config_refresh( + &mut child, + "claude", + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + std::time::Duration::from_millis(30), + &stage, + ) + .unwrap_err(); + assert!(error.contains("completion unavailable"), "{error}"); + assert_eq!( + std::fs::read(stage.join("retained")).unwrap(), + b"uncertain ordinary transaction" + ); + } + #[test] fn provider_identity_command_uses_clear_groups_only_when_switching_identity() { let uid = unsafe { libc::geteuid() }; diff --git a/crates/freshell-session-host/tests/provider_config_refresh.rs b/crates/freshell-session-host/tests/provider_config_refresh.rs index 9cd67ebff..99b01e2e7 100644 --- a/crates/freshell-session-host/tests/provider_config_refresh.rs +++ b/crates/freshell-session-host/tests/provider_config_refresh.rs @@ -151,3 +151,158 @@ fn provider_config_spawned_helper_emits_safe_jsonl_for_success_and_typed_failure marker.as_bytes() ); } + +#[cfg(unix)] +#[test] +fn provider_config_preparation_worker_consumes_bounded_private_control_and_cleans_failures() { + use freshell_runtime_protocol::*; + use std::{ + fs, + io::{Seek, Write}, + os::unix::fs::PermissionsExt, + process::{Command, Stdio}, + }; + let root = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(directory) => tempfile::tempdir_in(directory), + None => tempfile::tempdir(), + } + .unwrap(); + let home = root.path().join("home"); + let source = root.path().join("source"); + fs::create_dir(&home).unwrap(); + fs::create_dir_all(source.join("plugin/node_modules/selected-dependency")).unwrap(); + fs::write( + source.join("plugin/node_modules/selected-dependency/index.js"), + "module.exports='installed-dependency';", + ) + .unwrap(); + fs::write(source.join("plugin/resource"), "ordinary-resource").unwrap(); + let executable = source.join("plugin/command"); + fs::write(&executable, "#!/usr/bin/env node\nconst fs=require('node:fs');process.stdout.write(require('selected-dependency')+':'+fs.readFileSync(require('node:path').join(__dirname,'resource'),'utf8'));\n").unwrap(); + fs::set_permissions(&executable, fs::Permissions::from_mode(0o751)).unwrap(); + fs::write(home.join("retained"), b"accepted-provider-state").unwrap(); + let context = ProviderLaunchContext { + prepared_inputs: None, + mcp_capability: Some(McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }), + config: vec![ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "plugin".into(), + provider_relative_path: ".config/opencode/plugin".into(), + format: "directory".into(), + }], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + }; + context.validate("opencode").unwrap(); + let encoded = serde_json::to_vec(&context).unwrap(); + let marker = "synthetic-private-control-payload-marker"; + let run = |frame: &[u8]| { + let mut input = tempfile::tempfile_in(root.path()).unwrap(); + input.write_all(frame).unwrap(); + input.rewind().unwrap(); + let transaction = uuid::Uuid::new_v4().to_string(); + let output = Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("refresh-provider-config") + .arg("--prepare") + .arg("opencode") + .arg(root.path()) + .arg(&source) + .arg(&home) + .arg(&transaction) + .stdin(Stdio::from(input)) + .output() + .unwrap(); + assert!( + output.stdout.is_empty(), + "private worker must preserve protocol stdout" + ); + assert!( + !String::from_utf8_lossy(&output.stderr).contains(marker), + "private control contents entered worker logs" + ); + assert!( + !home + .join(format!(".freshell-provider-refresh-{transaction}")) + .exists(), + "worker left its transaction after completion" + ); + assert_eq!( + fs::read(home.join("retained")).unwrap(), + b"accepted-provider-state" + ); + output + }; + let frame = |payload: &[u8]| { + let mut frame = (payload.len() as u32).to_be_bytes().to_vec(); + frame.extend_from_slice(payload); + frame + }; + for length in [MAX_CONTROL_FRAME_BYTES - 1, MAX_CONTROL_FRAME_BYTES] { + let mut payload = encoded.clone(); + payload.resize(length, b' '); + let output = run(&frame(&payload)); + assert!( + output.status.success(), + "bounded complete control must prepare selected inputs" + ); + let copied = home.join(".config/opencode/plugin/command"); + assert_eq!( + fs::metadata(&copied).unwrap().permissions().mode() & 0o111, + fs::metadata(&executable).unwrap().permissions().mode() & 0o111, + "every original execute bit must survive both copying stages" + ); + let executed = Command::new(copied).output().unwrap(); + assert!(executed.status.success(), "copied command must execute"); + assert_eq!(executed.stdout, b"installed-dependency:ordinary-resource"); + } + let accepted = fs::read(home.join(".config/opencode/plugin/command")).unwrap(); + let mut growth = frame(&encoded); + growth.extend_from_slice(marker.as_bytes()); + let failures = [ + (vec![0, 0], "header unavailable"), + ( + ((MAX_CONTROL_FRAME_BYTES + 1) as u32) + .to_be_bytes() + .to_vec(), + "exceeds its profile", + ), + ( + [100u32.to_be_bytes().as_slice(), b"{}"].concat(), + "input incomplete", + ), + (growth, "trailing metadata"), + (frame(marker.as_bytes()), "metadata invalid"), + ]; + for (input, reason) in failures { + let output = run(&input); + assert!(!output.status.success(), "invalid control was accepted"); + assert!( + String::from_utf8_lossy(&output.stderr).contains(reason), + "worker failure reason was not actionable" + ); + assert_eq!( + fs::read(home.join(".config/opencode/plugin/command")).unwrap(), + accepted + ); + } + fs::remove_dir_all(source.join("plugin")).unwrap(); + let missing = run(&frame(&encoded)); + assert!( + !missing.status.success(), + "missing selected resource must block preparation" + ); + assert!(String::from_utf8_lossy(&missing.stderr).contains("provider.environment.copy.failed")); + assert_eq!( + fs::read(home.join(".config/opencode/plugin/command")).unwrap(), + accepted + ); +} diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 1d05b1fdb..1cbb48a8e 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -197,7 +197,10 @@ def create(name,*arguments): if not gateway: raise RuntimeError("closed controller has no owned bridge gateway") (root/"controller-gateway.json").write_text(json.dumps({"address":gateway})) - waited=docker("wait",controller) + # The Claude fixture consumes an ordinary file at the full128MiB + # boundary through two independent controller/native routes. + # Keep a finite overall wait separate from each helper's deadline. + waited=docker("wait",controller,timeout=360 if args.provider=="claude" else 180) exit_code=int(waited.stdout.strip()) outcome=docker("logs",controller,check=False) receipt["exitCode"]=exit_code From 8431dc67b11cf8d2a2d3076824191799bdb76d37 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 07:57:48 -0700 Subject: [PATCH 133/493] Select Claude ordinary project inputs from the current launch directory --- .../managed_mcp_capability/claude_stage.rs | 166 ++++++++++++++++-- 1 file changed, 156 insertions(+), 10 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 140d2d0dc..5935d1998 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -91,11 +91,11 @@ fn prepare_selected_sources( manifest.settings_origin = Some(origin.clone()); configurations.push((origin.clone(), origin)); } + // The manifest retains original explicit selector origins. Project discovery + // and projection follow the current launch directory on every preparation. let native_cwd = observation .map(|observed| observed.cwd.clone()) - .unwrap_or_else(|| { - std::fs::canonicalize(&manifest.cwd).unwrap_or_else(|_| manifest.cwd.clone()) - }); + .unwrap_or_else(|| std::fs::canonicalize(cwd).unwrap_or_else(|_| cwd.to_path_buf())); let mut bases: Vec = native_cwd .ancestors() .filter(|path| path.parent().is_some()) @@ -163,8 +163,8 @@ fn prepare_selected_sources( if let Some(observed) = observation { settings.extend(observed.local_settings.iter().cloned()); } else { - settings.push(manifest.cwd.join(".claude/settings.local.json")); - settings.push(manifest.workspace.join(".claude/settings.local.json")); + settings.push(cwd.join(".claude/settings.local.json")); + settings.push(workspace.join(".claude/settings.local.json")); } } if observation.is_none() { @@ -208,7 +208,7 @@ fn prepare_selected_sources( if project_enabled { for relative in RESOURCE_DIRECTORIES { let directories = if matches!(*relative, "hooks" | "plugins") { - project_resource_bases(&manifest)? + project_resource_bases(&manifest.original_home, &native_cwd)? .iter() .map(|base| base.join(".claude").join(relative)) .filter_map(|path| match optional(&path) { @@ -248,7 +248,7 @@ fn prepare_selected_sources( } prepare_selected( "claude", - &manifest.cwd, + cwd, &manifest, configurations, resources, @@ -317,10 +317,10 @@ fn native_resource_exists(path: &Path) -> Result { // The native Markdown resource loader searches upward only until HOME or the // nearest Git root. Memory has its own root-to-cwd search and must not share // this stopping rule. Startup cwd is the same accepted cwd for these loaders. -fn project_resource_bases(manifest: &ClaudeInputManifest) -> Result, String> { +fn project_resource_bases(home: &Path, cwd: &Path) -> Result, String> { let mut bases = Vec::new(); - for base in manifest.cwd.ancestors() { - if base == manifest.original_home { + for base in cwd.ancestors() { + if base == home { break; } bases.push(base.to_path_buf()); @@ -443,6 +443,152 @@ mod tests { provider_environment::{copy_environment, CopyLimits}, }; + #[cfg(unix)] + #[test] + fn managed_provider_claude_resume_selects_current_project_and_keeps_explicit_origins() { + use freshell_platform::claude_inputs::NativeSourceRuntime; + use std::os::unix::fs::PermissionsExt; + for runtime in [NativeSourceRuntime::Terminal, NativeSourceRuntime::Sdk] { + let owned = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::Builder::new() + .prefix("claude-resume-") + .tempdir_in(root) + .unwrap(), + None => tempfile::tempdir().unwrap(), + }; + let home = owned.path().join("home"); + let original = owned.path().join("original"); + let resumed = owned.path().join("resumed"); + let plugin = original.join("explicit-plugin"); + for path in [ + &home, + &plugin, + &original.join(".claude/hooks"), + &resumed.join(".claude/hooks/node_modules/probe-dependency"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + let settings = original.join("explicit-settings.json"); + std::fs::write(&settings, r#"{"env":{"EXPLICIT_ORIGIN":"original"}}"#).unwrap(); + std::fs::write(plugin.join("resource"), "original-plugin-resource").unwrap(); + std::fs::write( + original.join(".claude/hooks/resource"), + "wrong-original-project", + ) + .unwrap(); + std::fs::write( + resumed.join(".claude/settings.local.json"), + r#"{"env":{"LOCAL_ORIGIN":"resumed"}}"#, + ) + .unwrap(); + let hooks = resumed.join(".claude/hooks"); + std::fs::write(hooks.join("resource"), "resumed-project-resource").unwrap(); + std::fs::write( + hooks.join("node_modules/probe-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(hooks.join("run.cjs"), "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); + std::fs::set_permissions( + hooks.join("run.cjs"), + std::fs::Permissions::from_mode(0o751), + ) + .unwrap(); + let manifest = ClaudeInputManifest { + version: 1, + accepted_native_plan: None, + native_source_runtime: Some(runtime), + original_home: home.clone(), + cwd: original.clone(), + workspace: original.clone(), + config_root: None, + settings_file: Some(settings.clone()), + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![ClaudeSettingSource::Project, ClaudeSettingSource::Local], + plugin_dirs: vec![plugin.clone()], + }; + let selected = prepare(&home, &resumed, &resumed, Some(&manifest), &[]).unwrap(); + let stage = owned.path().join("stage"); + copy_environment(&selected.selections, &stage, &[], CopyLimits::default()).unwrap(); + let current = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == hooks) + .expect("resume must transport hooks from its current project directory"); + let explicit = selected + .binds + .iter() + .find(|bind| Path::new(&bind.logical_destination) == settings) + .unwrap(); + let local = selected + .binds + .iter() + .find(|bind| { + Path::new(&bind.logical_destination) + == resumed.join(".claude/settings.local.json") + }) + .expect("resume must select local settings at its current directory"); + let plugin_mount = selected + .mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == plugin) + .unwrap(); + assert!(!selected.mounts.iter().any( + |mount| Path::new(&mount.logical_destination) == original.join(".claude/hooks") + )); + std::fs::remove_dir_all(&original).unwrap(); + std::fs::remove_dir_all(&resumed).unwrap(); + let output = std::process::Command::new( + stage.join(¤t.stage_relative_path).join("run.cjs"), + ) + .output() + .unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert_eq!( + output.stdout, + b"resumed-project-resource:installed-dependency" + ); + assert_ne!( + std::fs::metadata(stage.join(¤t.stage_relative_path).join("run.cjs")) + .unwrap() + .permissions() + .mode() + & 0o111, + 0 + ); + let explicit_value: serde_json::Value = serde_json::from_slice( + &std::fs::read(stage.join(&explicit.source_relative_path)).unwrap(), + ) + .unwrap(); + let local_value: serde_json::Value = serde_json::from_slice( + &std::fs::read(stage.join(&local.source_relative_path)).unwrap(), + ) + .unwrap(); + assert_eq!(explicit_value["env"]["EXPLICIT_ORIGIN"], "original"); + assert_eq!(local_value["env"]["LOCAL_ORIGIN"], "resumed"); + assert_eq!( + std::fs::read( + stage + .join(&plugin_mount.stage_relative_path) + .join("resource") + ) + .unwrap(), + b"original-plugin-resource" + ); + let retained: ClaudeInputManifest = + serde_json::from_slice(selected.manifest_bytes()).unwrap(); + assert_eq!(retained.cwd, manifest.cwd); + assert_eq!(retained.workspace, manifest.workspace); + assert_eq!(retained.settings_file, manifest.settings_file); + assert_eq!(retained.plugin_dirs, manifest.plugin_dirs); + } + } + #[cfg(unix)] #[test] fn managed_provider_claude_observed_local_order_consumes_cwd_then_canonical_and_leaves_workspace_poison( From d7b636e9c7a6d4b0fbf03513f7c77ad177000202 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:10:25 -0700 Subject: [PATCH 134/493] Keep direct Claude resource execution tests stable during transient writer close --- .../managed_mcp_capability/claude_stage.rs | 69 +++++++++++++++++-- 1 file changed, 63 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 5935d1998..2d07bc4d4 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -443,6 +443,65 @@ mod tests { provider_environment::{copy_environment, CopyLimits}, }; + #[cfg(unix)] + fn execute_copied_script(path: &Path) -> std::io::Result { + // A parallel fork can briefly inherit a writer before close-on-exec. + // ETXTBSY creates no child, so retry only that transient spawn error; + // still execute the actual copied script and fail every other error. + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); + loop { + match std::process::Command::new(path).output() { + Err(error) + if error.raw_os_error() == Some(libc::ETXTBSY) + && std::time::Instant::now() < deadline => + { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + result => return result, + } + } + } + + #[cfg(unix)] + #[test] + fn managed_provider_claude_copied_script_exec_waits_only_for_transient_owned_writer() { + use std::os::unix::fs::PermissionsExt; + let root = std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") + .unwrap_or_else(|| std::env::temp_dir().into()); + let owned = tempfile::tempdir_in(root).unwrap(); + let script = owned.path().join("run.sh"); + std::fs::write(&script, "#!/bin/sh\nprintf 'executed-complete-script'\n").unwrap(); + std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o700)).unwrap(); + let writer = std::fs::OpenOptions::new() + .write(true) + .open(&script) + .unwrap(); + let release = std::thread::spawn(move || { + std::thread::sleep(std::time::Duration::from_millis(50)); + drop(writer); + }); + let result = execute_copied_script(&script); + release.join().unwrap(); + let output = result + .expect("a transient inherited writer cannot make executable resource coverage flaky"); + assert!(output.status.success()); + assert_eq!(output.stdout, b"executed-complete-script"); + let writer = std::fs::OpenOptions::new() + .write(true) + .open(&script) + .unwrap(); + let started = std::time::Instant::now(); + let result = execute_copied_script(&script); + drop(writer); + assert_eq!(result.unwrap_err().raw_os_error(), Some(libc::ETXTBSY)); + assert!(started.elapsed() < std::time::Duration::from_secs(2)); + std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o600)).unwrap(); + assert_eq!( + execute_copied_script(&script).unwrap_err().kind(), + std::io::ErrorKind::PermissionDenied + ); + } + #[cfg(unix)] #[test] fn managed_provider_claude_resume_selects_current_project_and_keeps_explicit_origins() { @@ -539,11 +598,9 @@ mod tests { )); std::fs::remove_dir_all(&original).unwrap(); std::fs::remove_dir_all(&resumed).unwrap(); - let output = std::process::Command::new( - stage.join(¤t.stage_relative_path).join("run.cjs"), - ) - .output() - .unwrap(); + let output = + execute_copied_script(&stage.join(¤t.stage_relative_path).join("run.cjs")) + .unwrap(); assert!( output.status.success(), "{}", @@ -870,7 +927,7 @@ mod tests { ); if let Some(mount) = mount { let executable = stage.join(&mount.stage_relative_path).join("run.cjs"); - let output = std::process::Command::new(&executable).output().unwrap(); + let output = execute_copied_script(&executable).unwrap(); assert!( output.status.success(), "{case}: selected {category} executable" From 30b0bc0ca71ac980c648b6c177768421207cf9d2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:10:37 -0700 Subject: [PATCH 135/493] Qualify current Claude recovery source authority before pending callback replay --- .../src/managed_mcp_capability.rs | 297 ++++++++++++++++++ 1 file changed, 297 insertions(+) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 247d4a970..661555a61 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -333,6 +333,88 @@ async fn apply_prepared_callback( "Claude callback source provenance changed; retain inputs and reprepare".into(), ); } + // A replayed pending result is data, not authority for a new callback. + // The supervisor validates the current registry attempt before and + // after the private reader; no capability-store lock crosses this IPC. + let context = callback.prior_context.as_ref().unwrap(); + let manifest: freshell_platform::claude_inputs::ClaudeInputManifest = + serde_json::from_slice(&read_prior_manifest_bytes( + "claude", + store, + directory, + &callback.soul_id, + context, + )?) + .map_err(|_| "prior Claude source manifest metadata invalid")?; + manifest.validate().map_err(str::to_string)?; + let source = freshell_platform::claude_inputs::observation::ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: manifest + .native_source_runtime + .ok_or("Claude native source runtime unavailable")?, + cwd: std::fs::canonicalize( + callback + .cwd + .as_ref() + .ok_or("Claude current launch directory unavailable")?, + ) + .map_err(|_| "Claude current launch directory unavailable")?, + home: "/home/freshell/provider".into(), + setting_sources: manifest.setting_sources.clone(), + selector_manifest_sha256: format!( + "{:x}", + Sha256::digest( + freshell_runtime_protocol::bounded_json_bytes( + &manifest.original_selectors(), + 64 * 1024 * 1024 + ) + .map_err(|_| "Claude selector metadata exceeds its finite profile")? + ) + ), + plan_revision: 0, + candidate_binding: None, + }; + let digest = format!( + "{:x}", + Sha256::digest( + freshell_runtime_protocol::bounded_json_bytes( + &source, + freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES, + ) + .map_err(|_| "Claude source request exceeds its finite profile")? + ) + ); + let reply = client + .observe_claude_sources(freshell_runtime_protocol::ObserveClaudeSourcesRequest { + expected_control_epoch: None, + soul_id: callback.soul_id.clone(), + provider: binding.provider.clone(), + authority, + workspace_path: callback + .workspace + .as_ref() + .and_then(|path| path.to_str()) + .ok_or("Claude current workspace unavailable")? + .into(), + git_common_dir: binding.git_common_dir.clone(), + candidate: None, + observation_request: source.clone(), + }) + .await + .map_err(|_| { + "Claude current native source evidence unavailable; retain inputs and reprepare" + })?; + reply.validate_for_request(&source, &digest)?; + if ( + reply.observation.uid, + reply.observation.euid, + reply.observation.gid, + reply.observation.egid, + ) != (65534, 65534, 0, 0) + { + return Err("Claude current native source evidence identity changed; retain inputs and reprepare".into()); + } } if let Some(reply) = store.pending_prepared_context( &callback.soul_id, @@ -6804,6 +6886,221 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[tokio::test] + async fn managed_provider_claude_pending_replay_requires_current_source_authority() { + use freshell_platform::claude_inputs::{ClaudeInputManifest, NativeSourceRuntime}; + use freshell_runtime_protocol::{ + bounded_json_bytes, ClaudeRecoverySourceBinding, ClaudeSourceAuthority, + ProviderLaunchContext, ProviderPreparation, RecoveryAttemptId, MAX_CONTROL_FRAME_BYTES, + }; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let directory = owned.path().join("control/mcp-capabilities"); + for path in [&home.join(".claude"), &cwd, &directory] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + home.join(".claude/settings.json"), + r#"{"env":{"ORDINARY_SOURCE":"retained"}}"#, + ) + .unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let endpoint = "http://host.docker.internal:4000"; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + config: vec![], + mcp_capability: None, + prepared_inputs: None, + }; + let manifest = ClaudeInputManifest { + version: 1, + native_source_runtime: Some(NativeSourceRuntime::Terminal), + original_home: home.clone(), + cwd: cwd.clone(), + workspace: cwd.clone(), + config_root: None, + settings_file: None, + settings_origin: None, + inline_settings_sha256: None, + setting_sources: vec![], + plugin_dirs: vec![], + accepted_native_plan: None, + }; + let input = OpencodeEphemeralInput { + claude_manifest: Some(&manifest), + cwd: Some(&cwd), + workspace: Some(&cwd), + launch_context: Some(&context), + secret_references: Some(&[]), + ..Default::default() + }; + let prior = store + .issue( + soul.clone(), + &directory, + endpoint.into(), + Some(("claude", &home, &[])), + input, + None, + ) + .unwrap(); + let prior_context = read_prepared_context(&directory, &prior).unwrap(); + let replacement = store + .issue_replacement_with_inputs( + soul.clone(), + &directory, + endpoint.into(), + Some(("claude", &home, &[])), + OpencodeEphemeralInput { + launch_context: Some(&prior_context), + ..input + }, + ) + .unwrap(); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + let pending_stage = staged_provider_root_path(&directory, &replacement.grant_id).unwrap(); + let prior_before = + std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(); + let pending_before = + std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(); + let scope = uuid::Uuid::new_v4().to_string(); + let authority = ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: scope.clone(), + }; + let binding = ClaudeRecoverySourceBinding { + provider: "claude".into(), + git_common_dir: None, + authority_sha256: format!( + "{:x}", + Sha256::digest(bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES).unwrap()) + ), + registry_attempt_id: RecoveryAttemptId::new(), + intent_revision: 2, + prior_generation_id: prior.grant_id.clone(), + }; + let callback = serde_json::to_vec(&serde_json::json!({ + "action":"issue", "controlSecret":"fixture-control", "soulId":soul, + "provider":"claude", "cwd":cwd, "workspace":cwd, "priorContext":prior_context, + "providerSecretReferences":[], "authLookupAuthority":scope, "claudeRecoverySource":binding, + })).unwrap(); + // The ordinary pending generation is intact, but there is no current + // supervisor scope available to qualify it. No replay may grant authority. + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unavailable.sock"), + "fixture-control", + ); + let result = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + endpoint, + &home, + &client, + ) + .await; + assert!( + result.is_err(), + "pending replay must first obtain current native source authority" + ); + assert!(result.unwrap_err().contains("native source evidence")); + assert_eq!( + std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(), + prior_before + ); + assert_eq!( + std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(), + pending_before + ); + assert_eq!(store.grants.lock().unwrap().len(), 2); + for fault in ["digest", "identity", "none"] { + use freshell_platform::claude_inputs::observation::{ + LocalDecision, NativeObservation, ObservationReply, + }; + use freshell_runtime_protocol::{ + read_frame, write_frame, AdminCommand, AdminReply, AdminResult, Envelope, + InstallationId, + }; + let socket_path = owned.path().join(format!("source-{fault}.sock")); + let listener = tokio::net::UnixListener::bind(&socket_path).unwrap(); + let expected_scope = scope.clone(); + let expected_soul = soul.clone(); + let expected_cwd = cwd.clone(); + let reader = tokio::spawn(async move { + tokio::time::timeout(std::time::Duration::from_secs(5), async move { + for operation in 0..2 { + let (mut socket, _) = listener.accept().await.unwrap(); + let request: Envelope = read_frame(&mut socket).await.unwrap(); + assert_eq!(request.auth.as_deref(), Some("fixture-control")); + let result = match (operation, request.body) { + (0, AdminCommand::Health) => AdminResult::Health { + control_epoch: 1, installation_id: InstallationId::parse("installation-fixture").unwrap(), + }, + (1, AdminCommand::ObserveClaudeSources(request)) => { + assert_eq!(request.expected_control_epoch, Some(1)); + assert_eq!(request.soul_id, expected_soul); + assert!(matches!(request.authority, ClaudeSourceAuthority::RecoveryAttempt { attempt_id } if attempt_id == expected_scope)); + assert_eq!(request.observation_request.cwd, expected_cwd); + assert!(request.candidate.is_none()); + let selected = request.observation_request; + let raw = bounded_json_bytes(&selected, freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES).unwrap(); + // This synthetic reply qualifies IPC validation, not an actual native reader. + AdminResult::ClaudeSourceObservation(ObservationReply { + request_sha256: if fault == "digest" { "0".repeat(64) } else { format!("{:x}", Sha256::digest(raw)) }, + observation: NativeObservation { + version: 1, native_source_runtime: selected.native_source_runtime, + setting_sources: selected.setting_sources, uid: if fault == "identity" { 0 } else { 65534 }, + euid: if fault == "identity" { 0 } else { 65534 }, gid: 0, egid: 0, + supplementary_groups: vec![], cwd: selected.cwd, home: Some(selected.home), + roots: None, local_decision: LocalDecision::Disabled, + root_uid: None, git_uid: None, claude_uid: None, local_settings: vec![], + }, + }) + }, + _ => panic!("unexpected scoped source callback request"), + }; + write_frame(&mut socket, &AdminReply { request_id: request.request_id, result: Ok(result) }).await.unwrap(); + } + }).await.unwrap(); + }); + let client = + freshell_runtime_client::RuntimeClient::new(socket_path, "fixture-control"); + let result = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + endpoint, + &home, + &client, + ) + .await; + reader.await.unwrap(); + if fault == "none" { + assert_eq!( + serde_json::from_slice::(&result.unwrap()).unwrap(), + serde_json::from_slice::(&pending_before).unwrap() + ); + } else { + assert!( + result.is_err(), + "invalid {fault} must not grant pending replay authority" + ); + } + assert_eq!( + std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(), + prior_before + ); + assert_eq!( + std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(), + pending_before + ); + assert_eq!(store.grants.lock().unwrap().len(), 2); + } + } + #[tokio::test] async fn managed_provider_claude_replacement_retains_original_sources_and_pending_replay() { use freshell_platform::claude_inputs::INLINE_SETTINGS_SOURCE; From 05b5de8a19fe74965ba1ebf9a5c67e47843f878c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:15:19 -0700 Subject: [PATCH 136/493] Use the declared Claude resume directory in replacement input callbacks --- .../src/managed_mcp_capability.rs | 54 +++++++++++++++++-- 1 file changed, 50 insertions(+), 4 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 661555a61..a9617cbb9 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -497,8 +497,12 @@ async fn apply_prepared_callback( AcceptedOrigins::Claude(manifest) => ( &manifest.original_home, OpencodeEphemeralInput { - cwd: Some(&manifest.cwd), - workspace: Some(&manifest.workspace), + cwd: Some(callback.cwd.as_deref().ok_or( + "Claude current launch directory unavailable; retain inputs and reprepare", + )?), + workspace: Some(callback.workspace.as_deref().ok_or( + "Claude current workspace unavailable; retain inputs and reprepare", + )?), claude_manifest: Some(manifest), claude_inline_settings: inline_settings.as_deref(), claude_reservation_provider: callback @@ -7107,19 +7111,41 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ use freshell_runtime_protocol::{ PreparedProviderContext, ProviderLaunchContext, ProviderPreparation, }; - let owned = tempfile::tempdir().unwrap(); + let owned = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::tempdir_in(root).unwrap(), + None => tempfile::tempdir().unwrap(), + }; let home = owned.path().join("original-home"); let selected = owned.path().join("selected-config"); let wrong = owned.path().join("current-home"); let workspace = owned.path().join("workspace"); + let resume_cwd = workspace.join("resume"); + let resumed_hooks = resume_cwd.join(".claude/hooks"); for path in [ &selected.join("skills/probe"), &home, &wrong.join(".claude"), &workspace, + &resumed_hooks.join("node_modules/resume-dependency"), ] { std::fs::create_dir_all(path).unwrap(); } + std::fs::write(resumed_hooks.join("resource"), "resumed-hook-resource").unwrap(); + std::fs::write( + resumed_hooks.join("node_modules/resume-dependency/index.js"), + "module.exports='resumed-installed-dependency'", + ) + .unwrap(); + std::fs::write(resumed_hooks.join("consume.cjs"), "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('resume-dependency'))").unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions( + resumed_hooks.join("consume.cjs"), + std::fs::Permissions::from_mode(0o750), + ) + .unwrap(); + } std::fs::write( selected.join("settings.json"), r#"{"env":{"CLAUDE_SELECTED":"original"}}"#, @@ -7189,7 +7215,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(); let context = read_prepared_context(&directory, &prior).unwrap(); assert!(context.prepared_inputs.is_some()); - let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"claude","workspace":workspace,"cwd":workspace,"priorContext":context,"providerSecretReferences":[]})).unwrap(); + let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"claude","workspace":workspace,"cwd":resume_cwd,"priorContext":context,"providerSecretReferences":[]})).unwrap(); let client = freshell_runtime_client::RuntimeClient::new( owned.path().join("unused.sock"), "fixture-control", @@ -7260,6 +7286,26 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .arg(stage.join(INLINE_SETTINGS_SOURCE)).arg(stage.join(&mount.stage_relative_path).join("probe")).output().unwrap(); assert!(output.status.success()); assert_eq!(output.stdout, b"retained-inline:retained-original-resource"); + let hook_mount = inputs + .resource_mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == resumed_hooks) + .expect( + "recovery callback must transport project hooks from its actual resume directory", + ); + std::fs::remove_dir_all(&resume_cwd).unwrap(); + let output = std::process::Command::new( + stage + .join(&hook_mount.stage_relative_path) + .join("consume.cjs"), + ) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + b"resumed-hook-resource:resumed-installed-dependency" + ); let accepted = ready .provider_launch_context .mcp_capability From 157e561b3186cb37d7711cc27e889d16965b78f2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:25:23 -0700 Subject: [PATCH 137/493] Carry current Claude recovery source observations into ordinary preparation --- .../src/managed_mcp_capability.rs | 115 +++++++++++++++--- 1 file changed, 100 insertions(+), 15 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index a9617cbb9..98a3301e1 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -300,6 +300,7 @@ async fn apply_prepared_callback( .provider .as_deref() .ok_or("missing managed MCP provider")?; + let mut current_claude_plan = None; if let Some(binding) = callback.claude_recovery_source.as_ref() { binding .validate() @@ -415,6 +416,11 @@ async fn apply_prepared_callback( { return Err("Claude current native source evidence identity changed; retain inputs and reprepare".into()); } + current_claude_plan = Some(freshell_platform::claude_inputs::AcceptedNativeSourcePlan { + selector_manifest_sha256: source.selector_manifest_sha256, + plan_revision: 0, + observation: reply.observation, + }); } if let Some(reply) = store.pending_prepared_context( &callback.soul_id, @@ -461,9 +467,12 @@ async fn apply_prepared_callback( AcceptedOrigins::Codex(manifest) } "claude" => { - let manifest: freshell_platform::claude_inputs::ClaudeInputManifest = + let mut manifest: freshell_platform::claude_inputs::ClaudeInputManifest = serde_json::from_slice(&bytes) .map_err(|_| "prior Claude source manifest metadata invalid")?; + if let Some(plan) = ¤t_claude_plan { + manifest.accepted_native_plan = Some(plan.clone()); + } manifest.validate().map_err(str::to_string)?; AcceptedOrigins::Claude(manifest) } @@ -504,6 +513,7 @@ async fn apply_prepared_callback( "Claude current workspace unavailable; retain inputs and reprepare", )?), claude_manifest: Some(manifest), + claude_observation: current_claude_plan.as_ref().map(|plan| &plan.observation), claude_inline_settings: inline_settings.as_deref(), claude_reservation_provider: callback .claude_recovery_source @@ -6892,7 +6902,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ #[tokio::test] async fn managed_provider_claude_pending_replay_requires_current_source_authority() { - use freshell_platform::claude_inputs::{ClaudeInputManifest, NativeSourceRuntime}; + use freshell_platform::claude_inputs::{ + ClaudeInputManifest, ClaudeSettingSource, NativeSourceRuntime, + }; use freshell_runtime_protocol::{ bounded_json_bytes, ClaudeRecoverySourceBinding, ClaudeSourceAuthority, ProviderLaunchContext, ProviderPreparation, RecoveryAttemptId, MAX_CONTROL_FRAME_BYTES, @@ -6900,10 +6912,25 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let cwd = owned.path().join("workspace"); + let resumed = cwd.join("resume"); + let plugin = owned.path().join("selected-plugin"); let directory = owned.path().join("control/mcp-capabilities"); - for path in [&home.join(".claude"), &cwd, &directory] { + for path in [ + &home.join(".claude"), + &cwd.join(".claude"), + &directory, + &plugin.join("node_modules/probe-dependency"), + ] { std::fs::create_dir_all(path).unwrap(); } + std::fs::write(cwd.join(".claude/settings.local.json"), "{}").unwrap(); + std::fs::write(plugin.join("resource"), "selected-plugin-resource").unwrap(); + std::fs::write( + plugin.join("node_modules/probe-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(plugin.join("consume.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); std::fs::write( home.join(".claude/settings.json"), r#"{"env":{"ORDINARY_SOURCE":"retained"}}"#, @@ -6928,8 +6955,8 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ settings_file: None, settings_origin: None, inline_settings_sha256: None, - setting_sources: vec![], - plugin_dirs: vec![], + setting_sources: vec![ClaudeSettingSource::Local], + plugin_dirs: vec![plugin.clone()], accepted_native_plan: None, }; let input = OpencodeEphemeralInput { @@ -7019,7 +7046,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ pending_before ); assert_eq!(store.grants.lock().unwrap().len(), 2); - for fault in ["digest", "identity", "none"] { + for fault in ["digest", "identity", "none", "replacement"] { use freshell_platform::claude_inputs::observation::{ LocalDecision, NativeObservation, ObservationReply, }; @@ -7027,11 +7054,32 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ read_frame, write_frame, AdminCommand, AdminReply, AdminResult, Envelope, InstallationId, }; + let current_callback = if fault == "replacement" { + store + .abort_preparation(&replacement.grant_id, &soul, &directory) + .unwrap(); + std::fs::create_dir_all(resumed.join(".claude")).unwrap(); + std::fs::write( + cwd.join(".claude/settings.local.json"), + "invalid unselected workspace-local settings", + ) + .unwrap(); + std::fs::write(resumed.join(".claude/settings.local.json"), r#"{"env":{"LOCAL_SELECTED":"resumed"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"node consume.cjs"}]}]}}"#).unwrap(); + let mut value: serde_json::Value = serde_json::from_slice(&callback).unwrap(); + value["cwd"] = serde_json::to_value(&resumed).unwrap(); + serde_json::to_vec(&value).unwrap() + } else { + callback.clone() + }; let socket_path = owned.path().join(format!("source-{fault}.sock")); let listener = tokio::net::UnixListener::bind(&socket_path).unwrap(); let expected_scope = scope.clone(); let expected_soul = soul.clone(); - let expected_cwd = cwd.clone(); + let expected_cwd = if fault == "replacement" { + resumed.clone() + } else { + cwd.clone() + }; let reader = tokio::spawn(async move { tokio::time::timeout(std::time::Duration::from_secs(5), async move { for operation in 0..2 { @@ -7057,9 +7105,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ version: 1, native_source_runtime: selected.native_source_runtime, setting_sources: selected.setting_sources, uid: if fault == "identity" { 0 } else { 65534 }, euid: if fault == "identity" { 0 } else { 65534 }, gid: 0, egid: 0, - supplementary_groups: vec![], cwd: selected.cwd, home: Some(selected.home), - roots: None, local_decision: LocalDecision::Disabled, - root_uid: None, git_uid: None, claude_uid: None, local_settings: vec![], + supplementary_groups: vec![], local_settings: vec![selected.cwd.join(".claude/settings.local.json")], cwd: selected.cwd, home: Some(selected.home), + roots: None, local_decision: LocalDecision::NoRoot, + root_uid: None, git_uid: None, claude_uid: None, }, }) }, @@ -7072,7 +7120,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let client = freshell_runtime_client::RuntimeClient::new(socket_path, "fixture-control"); let result = apply_prepared_callback( - &callback, + ¤t_callback, "fixture-control", &store, &directory, @@ -7087,6 +7135,41 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ serde_json::from_slice::(&result.unwrap()).unwrap(), serde_json::from_slice::(&pending_before).unwrap() ); + } else if fault == "replacement" { + let ready: freshell_runtime_protocol::PreparedProviderContext = serde_json::from_slice(&result.expect("qualified current local selection must not stage unselected workspace-local poison")).unwrap(); + let context = &ready.provider_launch_context; + let inputs = context.prepared_inputs.as_ref().unwrap(); + let stage = staged_provider_root_path(&directory, &inputs.generation_id).unwrap(); + let settings = inputs + .config_binds + .iter() + .find(|bind| { + Path::new(&bind.logical_destination) + == resumed.join(".claude/settings.local.json") + }) + .unwrap(); + let resource = inputs + .resource_mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == plugin) + .unwrap(); + std::fs::remove_dir_all(&plugin).unwrap(); + let output = std::process::Command::new("node").args(["-e", "const fs=require('node:fs'),s=JSON.parse(fs.readFileSync(process.argv[1]));process.stdout.write(s.env.LOCAL_SELECTED+':'+require('node:child_process').execSync(s.hooks.SessionStart[0].hooks[0].command,{cwd:process.argv[2]}))"]) + .arg(stage.join(&settings.source_relative_path)).arg(stage.join(&resource.stage_relative_path)).output().unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, + b"resumed:selected-plugin-resource:installed-dependency" + ); + assert!(!inputs + .config_binds + .iter() + .any(|bind| Path::new(&bind.logical_destination) + == cwd.join(".claude/settings.local.json"))); + assert!( + !pending_stage.exists(), + "deliberately aborted private generation stays removed" + ); } else { assert!( result.is_err(), @@ -7097,10 +7180,12 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(), prior_before ); - assert_eq!( - std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(), - pending_before - ); + if fault != "replacement" { + assert_eq!( + std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(), + pending_before + ); + } assert_eq!(store.grants.lock().unwrap().len(), 2); } } From e7cbbfb46c07492526a99f58c72b2150f13727d3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:37:12 -0700 Subject: [PATCH 138/493] Confirm frozen Claude recovery candidates before returning prepared inputs --- .../src/managed_mcp_capability.rs | 218 +++++++++++++++--- .../src/managed_provider_references/claude.rs | 2 +- 2 files changed, 188 insertions(+), 32 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 98a3301e1..49726a82a 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -429,6 +429,10 @@ async fn apply_prepared_callback( provider, callback.prior_context.as_ref(), )? { + if let Some(plan) = ¤t_claude_plan { + confirm_claude_callback_candidate(&callback, store, directory, client, plan, &reply) + .await?; + } return freshell_runtime_protocol::bounded_json_bytes( &reply, freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, @@ -539,11 +543,14 @@ async fn apply_prepared_callback( ), }; context.config.clear(); - let references = callback.provider_secret_references.unwrap_or_default(); + let references = callback + .provider_secret_references + .clone() + .unwrap_or_default(); input.launch_context = Some(&context); input.secret_references = Some(&references); let capability = store.issue_replacement_with_inputs( - callback.soul_id, + callback.soul_id.clone(), directory, endpoint.into(), Some((provider, original_home, &context.config)), @@ -556,6 +563,10 @@ async fn apply_prepared_callback( reply .validate(provider) .map_err(|_| "replacement ordinary context metadata invalid")?; + if let Some(plan) = ¤t_claude_plan { + confirm_claude_callback_candidate(&callback, store, directory, client, plan, &reply) + .await?; + } return freshell_runtime_protocol::bounded_json_bytes( &reply, freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, @@ -747,6 +758,75 @@ async fn apply_prepared_callback( .map_err(|_| "replacement provider context exceeds control metadata profile".into()) } +async fn confirm_claude_callback_candidate( + callback: &CapabilityCallback, + store: &CapabilityStore, + directory: &Path, + client: &freshell_runtime_client::RuntimeClient, + plan: &freshell_platform::claude_inputs::AcceptedNativeSourcePlan, + ready: &freshell_runtime_protocol::PreparedProviderContext, +) -> Result<(), String> { + use freshell_platform::claude_inputs::observation::ObservationRequest; + use freshell_runtime_protocol::{ClaudeSourceAuthority, ObserveClaudeSourcesRequest}; + let binding = callback + .claude_recovery_source + .as_ref() + .ok_or("Claude current candidate recovery provenance unavailable")?; + let source = ObservationRequest { + version: 1, + request_id: uuid::Uuid::new_v4().to_string(), + native_source_runtime: plan.observation.native_source_runtime, + cwd: plan.observation.cwd.clone(), + home: "/home/freshell/provider".into(), + setting_sources: plan.observation.setting_sources.clone(), + selector_manifest_sha256: plan.selector_manifest_sha256.clone(), + plan_revision: plan.plan_revision, + candidate_binding: None, + }; + let request = bind_claude_candidate_request( + store, + directory, + ObserveClaudeSourcesRequest { + expected_control_epoch: None, + soul_id: callback.soul_id.clone(), + provider: binding.provider.clone(), + authority: ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: callback + .auth_lookup_authority + .as_ref() + .ok_or("Claude current candidate source scope unavailable")? + .clone(), + }, + workspace_path: callback + .workspace + .as_ref() + .and_then(|path| path.to_str()) + .ok_or("Claude current candidate workspace unavailable")? + .into(), + git_common_dir: binding.git_common_dir.clone(), + candidate: Some(ready.clone()), + observation_request: source, + }, + )?; + let source = request.observation_request.clone(); + let reply = client.observe_claude_sources(request).await.map_err(|_| { + tracing::warn!(event="provider.inputs.claude_candidate.evidence_unavailable",soul_id=%callback.soul_id, + plan_revision=plan.plan_revision, + "Claude recovery candidate evidence unavailable; both private generations retained"); + "Claude current candidate native source evidence unavailable; retain inputs and reprepare" + })?; + let mut compared = plan.clone(); + if crate::managed_provider_references::claude::accept_candidate_observation( + &mut compared, + &source, + &reply, + )? { + Ok(()) + } else { + Err("Claude current candidate source selection changed; retain inputs and reprepare".into()) + } +} + fn read_prior_manifest_bytes( provider: &str, store: &CapabilityStore, @@ -6903,7 +6983,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ #[tokio::test] async fn managed_provider_claude_pending_replay_requires_current_source_authority() { use freshell_platform::claude_inputs::{ - ClaudeInputManifest, ClaudeSettingSource, NativeSourceRuntime, + observation::{LocalDecision, NativeObservation}, + AcceptedNativeSourcePlan, ClaudeInputManifest, ClaudeSettingSource, + NativeSourceRuntime, }; use freshell_runtime_protocol::{ bounded_json_bytes, ClaudeRecoverySourceBinding, ClaudeSourceAuthority, @@ -6945,7 +7027,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ mcp_capability: None, prepared_inputs: None, }; - let manifest = ClaudeInputManifest { + let mut manifest = ClaudeInputManifest { version: 1, native_source_runtime: Some(NativeSourceRuntime::Terminal), original_home: home.clone(), @@ -6959,8 +7041,40 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ plugin_dirs: vec![plugin.clone()], accepted_native_plan: None, }; + let original_observation = NativeObservation { + version: 1, + native_source_runtime: NativeSourceRuntime::Terminal, + setting_sources: manifest.setting_sources.clone(), + uid: 65534, + euid: 65534, + gid: 0, + egid: 0, + supplementary_groups: vec![], + cwd: cwd.clone(), + home: Some("/home/freshell/provider".into()), + roots: None, + local_decision: LocalDecision::NoRoot, + root_uid: None, + git_uid: None, + claude_uid: None, + local_settings: vec![cwd.join(".claude/settings.local.json")], + }; + manifest.accepted_native_plan = Some(AcceptedNativeSourcePlan { + selector_manifest_sha256: format!( + "{:x}", + Sha256::digest( + bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024).unwrap() + ) + ), + plan_revision: 0, + observation: original_observation.clone(), + }); + let initial_reservation = uuid::Uuid::new_v4().to_string(); let input = OpencodeEphemeralInput { claude_manifest: Some(&manifest), + claude_observation: Some(&original_observation), + claude_reservation_provider: Some("claude"), + initial_auth_reservation: Some(&initial_reservation), cwd: Some(&cwd), workspace: Some(&cwd), launch_context: Some(&context), @@ -6978,6 +7092,21 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ) .unwrap(); let prior_context = read_prepared_context(&directory, &prior).unwrap(); + let scope = uuid::Uuid::new_v4().to_string(); + let authority = ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: scope.clone(), + }; + let binding = ClaudeRecoverySourceBinding { + provider: "claude".into(), + git_common_dir: None, + authority_sha256: format!( + "{:x}", + Sha256::digest(bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES).unwrap()) + ), + registry_attempt_id: RecoveryAttemptId::new(), + intent_revision: 2, + prior_generation_id: prior.grant_id.clone(), + }; let replacement = store .issue_replacement_with_inputs( soul.clone(), @@ -6986,6 +7115,8 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ Some(("claude", &home, &[])), OpencodeEphemeralInput { launch_context: Some(&prior_context), + initial_auth_reservation: None, + claude_recovery_source: Some(&binding), ..input }, ) @@ -6996,21 +7127,6 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(); let pending_before = std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(); - let scope = uuid::Uuid::new_v4().to_string(); - let authority = ClaudeSourceAuthority::RecoveryAttempt { - attempt_id: scope.clone(), - }; - let binding = ClaudeRecoverySourceBinding { - provider: "claude".into(), - git_common_dir: None, - authority_sha256: format!( - "{:x}", - Sha256::digest(bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES).unwrap()) - ), - registry_attempt_id: RecoveryAttemptId::new(), - intent_revision: 2, - prior_generation_id: prior.grant_id.clone(), - }; let callback = serde_json::to_vec(&serde_json::json!({ "action":"issue", "controlSecret":"fixture-control", "soulId":soul, "provider":"claude", "cwd":cwd, "workspace":cwd, "priorContext":prior_context, @@ -7046,7 +7162,14 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ pending_before ); assert_eq!(store.grants.lock().unwrap().len(), 2); - for fault in ["digest", "identity", "none", "replacement"] { + let mut replacement_snapshot = None; + for fault in [ + "digest", + "identity", + "none", + "replacement", + "candidate_digest", + ] { use freshell_platform::claude_inputs::observation::{ LocalDecision, NativeObservation, ObservationReply, }; @@ -7068,6 +7191,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let mut value: serde_json::Value = serde_json::from_slice(&callback).unwrap(); value["cwd"] = serde_json::to_value(&resumed).unwrap(); serde_json::to_vec(&value).unwrap() + } else if fault == "candidate_digest" { + let mut value: serde_json::Value = serde_json::from_slice(&callback).unwrap(); + value["cwd"] = serde_json::to_value(&resumed).unwrap(); + serde_json::to_vec(&value).unwrap() } else { callback.clone() }; @@ -7075,32 +7202,40 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let listener = tokio::net::UnixListener::bind(&socket_path).unwrap(); let expected_scope = scope.clone(); let expected_soul = soul.clone(); - let expected_cwd = if fault == "replacement" { + let expected_cwd = if matches!(fault, "replacement" | "candidate_digest") { resumed.clone() } else { cwd.clone() }; + let (settled, mut stop_reader) = tokio::sync::oneshot::channel(); let reader = tokio::spawn(async move { tokio::time::timeout(std::time::Duration::from_secs(5), async move { - for operation in 0..2 { - let (mut socket, _) = listener.accept().await.unwrap(); + let mut observations = Vec::new(); + loop { + let (mut socket, _) = tokio::select! { + _ = &mut stop_reader => break, + accepted = listener.accept() => accepted.unwrap(), + }; let request: Envelope = read_frame(&mut socket).await.unwrap(); assert_eq!(request.auth.as_deref(), Some("fixture-control")); - let result = match (operation, request.body) { - (0, AdminCommand::Health) => AdminResult::Health { + let result = match request.body { + AdminCommand::Health => AdminResult::Health { control_epoch: 1, installation_id: InstallationId::parse("installation-fixture").unwrap(), }, - (1, AdminCommand::ObserveClaudeSources(request)) => { + AdminCommand::ObserveClaudeSources(request) => { assert_eq!(request.expected_control_epoch, Some(1)); assert_eq!(request.soul_id, expected_soul); assert!(matches!(request.authority, ClaudeSourceAuthority::RecoveryAttempt { attempt_id } if attempt_id == expected_scope)); assert_eq!(request.observation_request.cwd, expected_cwd); - assert!(request.candidate.is_none()); + let candidate = request.candidate.is_some(); + observations.push(candidate); + assert!(observations.len() <= 4, "only one baseline plus three finite candidate revisions are permitted"); + assert_eq!(request.candidate.is_some(), request.observation_request.candidate_binding.is_some()); let selected = request.observation_request; let raw = bounded_json_bytes(&selected, freshell_platform::claude_inputs::observation::MAX_REQUEST_BYTES).unwrap(); // This synthetic reply qualifies IPC validation, not an actual native reader. AdminResult::ClaudeSourceObservation(ObservationReply { - request_sha256: if fault == "digest" { "0".repeat(64) } else { format!("{:x}", Sha256::digest(raw)) }, + request_sha256: if fault == "digest" || (fault == "candidate_digest" && candidate) { "0".repeat(64) } else { format!("{:x}", Sha256::digest(raw)) }, observation: NativeObservation { version: 1, native_source_runtime: selected.native_source_runtime, setting_sources: selected.setting_sources, uid: if fault == "identity" { 0 } else { 65534 }, @@ -7115,7 +7250,8 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ }; write_frame(&mut socket, &AdminReply { request_id: request.request_id, result: Ok(result) }).await.unwrap(); } - }).await.unwrap(); + observations + }).await.unwrap() }); let client = freshell_runtime_client::RuntimeClient::new(socket_path, "fixture-control"); @@ -7129,7 +7265,17 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ &client, ) .await; - reader.await.unwrap(); + let _ = settled.send(()); + let observations = reader.await.unwrap(); + if matches!(fault, "none" | "replacement" | "candidate_digest") { + assert_eq!(observations, vec![false, true], "both pending replay and new replacement must confirm their actual candidate before any consumer"); + } else { + assert_eq!( + observations, + vec![false], + "failed current source evidence must stop before candidate work" + ); + } if fault == "none" { assert_eq!( serde_json::from_slice::(&result.unwrap()).unwrap(), @@ -7170,6 +7316,16 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ !pending_stage.exists(), "deliberately aborted private generation stays removed" ); + let path = stage.join(".freshell/prepared-result.json"); + let bytes = std::fs::read(&path).unwrap(); + replacement_snapshot = Some((path, bytes)); + } else if fault == "candidate_digest" { + assert!( + result.is_err(), + "incomplete candidate evidence cannot return a prepared result" + ); + let (path, bytes) = replacement_snapshot.as_ref().unwrap(); + assert_eq!(&std::fs::read(path).unwrap(), bytes); } else { assert!( result.is_err(), @@ -7180,7 +7336,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(), prior_before ); - if fault != "replacement" { + if matches!(fault, "digest" | "identity" | "none") { assert_eq!( std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(), pending_before diff --git a/crates/freshell-server/src/managed_provider_references/claude.rs b/crates/freshell-server/src/managed_provider_references/claude.rs index 800648452..d93a58366 100644 --- a/crates/freshell-server/src/managed_provider_references/claude.rs +++ b/crates/freshell-server/src/managed_provider_references/claude.rs @@ -317,7 +317,7 @@ fn selected_settings_file( }) } -fn accept_candidate_observation( +pub(crate) fn accept_candidate_observation( plan: &mut freshell_platform::claude_inputs::AcceptedNativeSourcePlan, source: &freshell_platform::claude_inputs::observation::ObservationRequest, reply: &freshell_platform::claude_inputs::observation::ObservationReply, From 01a8ab7defc3fb22a6bc9e67405eed45e6516714 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:52:02 -0700 Subject: [PATCH 139/493] Compare Claude recovery candidates with their frozen source selections --- .../src/managed_mcp_capability.rs | 40 +++++++++++++++---- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 49726a82a..18f734b29 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -783,7 +783,7 @@ async fn confirm_claude_callback_candidate( plan_revision: plan.plan_revision, candidate_binding: None, }; - let request = bind_claude_candidate_request( + let (request, mut frozen_plan) = bind_claude_candidate_request_and_plan( store, directory, ObserveClaudeSourcesRequest { @@ -815,9 +815,11 @@ async fn confirm_claude_callback_candidate( "Claude recovery candidate evidence unavailable; both private generations retained"); "Claude current candidate native source evidence unavailable; retain inputs and reprepare" })?; - let mut compared = plan.clone(); + // A replay's files were selected by its frozen manifest, which may differ + // from the newly observed baseline. Two current observations agreeing on + // drift cannot authorize an older selection without a deliberate replan. if crate::managed_provider_references::claude::accept_candidate_observation( - &mut compared, + &mut frozen_plan, &source, &reply, )? { @@ -2903,6 +2905,20 @@ fn bind_claude_candidate_request( directory: &Path, request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, ) -> Result { + bind_claude_candidate_request_and_plan(store, directory, request).map(|(request, _)| request) +} + +fn bind_claude_candidate_request_and_plan( + store: &CapabilityStore, + directory: &Path, + request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, +) -> Result< + ( + freshell_runtime_protocol::ObserveClaudeSourcesRequest, + freshell_platform::claude_inputs::AcceptedNativeSourcePlan, + ), + String, +> { use freshell_platform::claude_inputs::{observation::CandidateBinding, ClaudeInputManifest}; use freshell_runtime_protocol::{ bounded_json_bytes, ClaudeCandidatePublication, PreparedProviderContext, @@ -3005,7 +3021,7 @@ fn bind_claude_candidate_request( .map_err(|_| blocked())?; bounded_json_bytes(&bound, MAX_CONTROL_FRAME_BYTES).map_err(|_| blocked())?; tracing::debug!(event="provider.inputs.claude_candidate.request_bound",soul_id=%bound.soul_id,plan_revision=bound.observation_request.plan_revision,"Frozen Claude candidate request bound to its issuer"); - Ok(bound) + Ok((bound, plan.clone())) } pub(crate) fn bind_prepared_claude_candidate( @@ -7167,6 +7183,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ "digest", "identity", "none", + "pending_selection_drift", "replacement", "candidate_digest", ] { @@ -7240,8 +7257,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ version: 1, native_source_runtime: selected.native_source_runtime, setting_sources: selected.setting_sources, uid: if fault == "identity" { 0 } else { 65534 }, euid: if fault == "identity" { 0 } else { 65534 }, gid: 0, egid: 0, - supplementary_groups: vec![], local_settings: vec![selected.cwd.join(".claude/settings.local.json")], cwd: selected.cwd, home: Some(selected.home), - roots: None, local_decision: LocalDecision::NoRoot, + supplementary_groups: vec![], local_settings: vec![selected.cwd.join(".claude/settings.local.json")], cwd: selected.cwd.clone(), home: Some(selected.home), + roots: (fault == "pending_selection_drift").then(|| freshell_platform::claude_inputs::git::Roots { checkout: selected.cwd.clone(), canonical: selected.cwd }), + local_decision: if fault == "pending_selection_drift" { LocalDecision::CwdRoot } else { LocalDecision::NoRoot }, root_uid: None, git_uid: None, claude_uid: None, }, }) @@ -7267,7 +7285,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .await; let _ = settled.send(()); let observations = reader.await.unwrap(); - if matches!(fault, "none" | "replacement" | "candidate_digest") { + if matches!( + fault, + "none" | "replacement" | "candidate_digest" | "pending_selection_drift" + ) { assert_eq!(observations, vec![false, true], "both pending replay and new replacement must confirm their actual candidate before any consumer"); } else { assert_eq!( @@ -7336,7 +7357,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(), prior_before ); - if matches!(fault, "digest" | "identity" | "none") { + if matches!( + fault, + "digest" | "identity" | "none" | "pending_selection_drift" + ) { assert_eq!( std::fs::read(pending_stage.join(".freshell/prepared-result.json")).unwrap(), pending_before From b43317ac84f53cb664cbffc7b459d11ef74f5545 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 08:58:26 -0700 Subject: [PATCH 140/493] Preserve frozen Claude plan revisions during scoped recovery replay --- .../src/managed_mcp_capability.rs | 26 ++++++++++++++++--- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 18f734b29..37f96483c 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -807,11 +807,12 @@ async fn confirm_claude_callback_candidate( candidate: Some(ready.clone()), observation_request: source, }, + true, )?; let source = request.observation_request.clone(); let reply = client.observe_claude_sources(request).await.map_err(|_| { tracing::warn!(event="provider.inputs.claude_candidate.evidence_unavailable",soul_id=%callback.soul_id, - plan_revision=plan.plan_revision, + plan_revision=source.plan_revision, "Claude recovery candidate evidence unavailable; both private generations retained"); "Claude current candidate native source evidence unavailable; retain inputs and reprepare" })?; @@ -2905,13 +2906,15 @@ fn bind_claude_candidate_request( directory: &Path, request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, ) -> Result { - bind_claude_candidate_request_and_plan(store, directory, request).map(|(request, _)| request) + bind_claude_candidate_request_and_plan(store, directory, request, false) + .map(|(request, _)| request) } fn bind_claude_candidate_request_and_plan( store: &CapabilityStore, directory: &Path, - request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, + mut request: freshell_runtime_protocol::ObserveClaudeSourcesRequest, + use_frozen_revision: bool, ) -> Result< ( freshell_runtime_protocol::ObserveClaudeSourcesRequest, @@ -2954,6 +2957,12 @@ fn bind_claude_candidate_request_and_plan( )?; let publication: ClaudeCandidatePublication = serde_json::from_slice(&published).map_err(|_| blocked())?; + if use_frozen_revision { + // A fresh baseline has no prepared revision yet. Same-scope replay + // retains the issuer's frozen count; all publication/manifest hashes + // and recovery authority are requalified below before any IPC. + request.observation_request.plan_revision = publication.plan_revision; + } let context_bytes = read_claude_candidate_metadata( &root, ".freshell/prepared-context.json", @@ -6998,6 +7007,12 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ #[tokio::test] async fn managed_provider_claude_pending_replay_requires_current_source_authority() { + for frozen_revision in 0..=2 { + qualify_claude_pending_replay_source_authority(frozen_revision).await; + } + } + + async fn qualify_claude_pending_replay_source_authority(frozen_revision: u8) { use freshell_platform::claude_inputs::{ observation::{LocalDecision, NativeObservation}, AcceptedNativeSourcePlan, ClaudeInputManifest, ClaudeSettingSource, @@ -7082,7 +7097,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ bounded_json_bytes(&manifest.original_selectors(), 64 * 1024 * 1024).unwrap() ) ), - plan_revision: 0, + plan_revision: frozen_revision, observation: original_observation.clone(), }); let initial_reservation = uuid::Uuid::new_v4().to_string(); @@ -7245,6 +7260,9 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ assert!(matches!(request.authority, ClaudeSourceAuthority::RecoveryAttempt { attempt_id } if attempt_id == expected_scope)); assert_eq!(request.observation_request.cwd, expected_cwd); let candidate = request.candidate.is_some(); + assert_eq!(request.observation_request.plan_revision, + if candidate && !matches!(fault, "replacement" | "candidate_digest") { frozen_revision } else { 0 }, + "same-scope pending replay must retain its frozen revision; new preparation starts at zero"); observations.push(candidate); assert!(observations.len() <= 4, "only one baseline plus three finite candidate revisions are permitted"); assert_eq!(request.candidate.is_some(), request.observation_request.candidate_binding.is_some()); From aecb3ca16446a5eaeef9ace4f4b81a66fce1ea38 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 09:53:56 -0700 Subject: [PATCH 141/493] Observe retained Claude sources through their frozen ordinary aliases --- .../src/backend/claude_source_observation.rs | 131 +++++++++- .../claude_source_observation_tests.rs | 232 ++++++++++++++++-- 2 files changed, 347 insertions(+), 16 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index 60c024134..1632685e1 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -12,6 +12,39 @@ fn fail(reason: &'static str) -> BackendError { BackendError::Unavailable(reason.into()) } +fn retained_claude_context( + handle: &OwnedRuntimeHandle, +) -> Result<&freshell_runtime_protocol::ProviderLaunchContext, BackendError> { + let context = handle + .terminal() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + handle + .fresh_agent() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }) + .ok_or_else(|| { + fail("Claude retained ordinary context unavailable; retain inputs and reprepare") + })?; + context.validate("claude").map_err(|_| { + fail("Claude retained ordinary context changed; retain inputs and reprepare") + })?; + let inputs = context.prepared_inputs.as_ref().ok_or_else(|| { + fail("Claude retained ordinary generation unavailable; retain inputs and reprepare") + })?; + if context + .mcp_capability + .as_ref() + .map(|grant| grant.grant_id.as_str()) + != Some(inputs.generation_id.as_str()) + { + return Err(fail( + "Claude retained ordinary generation changed; retain inputs and reprepare", + )); + } + Ok(context) +} + /// Ordinary candidate files use disk backing, separately from indexed secrets. /// The owning reader retains this identity through helper and volume cleanup. struct CandidateScratch { @@ -807,6 +840,49 @@ fn owned_candidate_helper(actual: &Value, expected: &Value) -> bool { && owned_helper_profile(actual, expected) } +fn verify_retained_claude_mount_reports( + actual: &Value, + expected: &Value, + volume: &Value, +) -> Result<(), BackendError> { + let blocked = || { + tracing::warn!( + event = "provider.inputs.claude_baseline.blocked", + stage = "retained_source_view", + operation = "verify_reported_mounts", + cause = "incomplete_or_changed_topology", + "Claude retained ordinary source evidence unavailable" + ); + fail("Claude retained source reported mount topology changed; retain inputs and reprepare") + }; + let declared = expected["HostConfig"]["Mounts"] + .as_array() + .ok_or_else(blocked)?; + let reported = actual["Mounts"].as_array().ok_or_else(blocked)?; + if declared.len() != reported.len() || !volume["Mountpoint"].is_string() { + return Err(blocked()); + } + for mount in declared { + let mut matches = reported + .iter() + .filter(|report| report["Destination"] == mount["Target"]); + let report = matches.next().ok_or_else(blocked)?; + if matches.next().is_some() + || mount["ReadOnly"] != true + || report["RW"] != false + || report["Type"] != mount["Type"] + || (mount["Type"] == "bind" && report["Source"] != mount["Source"]) + || (mount["Type"] == "volume" + && (mount["Source"] != volume["Name"] + || report["Name"] != volume["Name"] + || report["Source"] != volume["Mountpoint"])) + { + return Err(blocked()); + } + } + Ok(()) +} + fn verify_candidate_mount_reports( actual: &Value, expected: &Value, @@ -936,11 +1012,40 @@ impl DockerEngineBackend { .read_claude_candidate_helper(owner, request, &root, test_run_id, budget) .await; } + let prior_stage = match owner { + ClaudeSourceOwner::Initial { .. } => None, + ClaudeSourceOwner::Retained { handle, .. } => { + let context = retained_claude_context(handle)?; + let grant = context.mcp_capability.as_ref().unwrap(); + Some(docker::staged_provider_root_for_grant(&grant.grant_id).map_err(|_| { + fail("Claude retained frozen ordinary stage unavailable; retain inputs and reprepare") + })?) + } + }; + self.read_claude_baseline_helper( + owner, + request, + test_run_id, + budget, + prior_stage.as_deref(), + ) + .await + } + + async fn read_claude_baseline_helper( + &self, + owner: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + test_run_id: &str, + budget: Duration, + prior_stage: Option<&Path>, + ) -> Result { let name = format!("freshell-claude-source-{}", uuid::Uuid::new_v4()); let mut attempted = false; let mut expected = None::; let mut directory = None::; let operation = async { + let mut retained_volume = None::; let (daemon, rootless) = self .daemon_info() .await @@ -1012,6 +1117,7 @@ impl DockerEngineBackend { { return Err(fail("Claude retained HOME is unavailable; preserve native stores and repair the source")); } + retained_volume = Some(volume); ( handle.installation_id(), handle.soul_id(), @@ -1066,11 +1172,31 @@ impl DockerEngineBackend { json!({"Type":"bind","Source":workspace,"Target":workspace,"ReadOnly":true}), json!({"Type":"bind","Source":directory.0,"Target":"/run/freshell-source-exchange","ReadOnly":true}), ]; - if let Some(common) = common.filter(|common| !common.starts_with(&workspace)) { + if let Some(common) = common + .as_ref() + .filter(|common| !common.starts_with(&workspace)) + { mounts.push(json!({"Type":"bind","Source":common,"Target":common,"ReadOnly":true})); } let tmpfs = if let Some(volume) = retained { mounts.push(json!({"Type":"volume","Source":volume,"Target":"/home/freshell/provider","ReadOnly":true,"VolumeOptions":{"NoCopy":true}})); + let ClaudeSourceOwner::Retained { handle, .. } = owner else { + return Err(fail("Claude retained ordinary owner unavailable")); + }; + let ordinary = docker::ordinary_claude_candidate_mounts( + &workspace, + common.as_deref(), + prior_stage.ok_or_else(|| fail("Claude retained frozen ordinary stage unavailable; retain inputs and reprepare"))?, + retained_claude_context(handle)?, + ).map_err(|_| fail("Claude retained frozen ordinary sources unavailable; retain inputs and reprepare"))?; + let mut aliases = prepared_inputs::mount_descriptors(&ordinary, volume)?; + // The workload's directories are writable. This observation + // reproduces their read view without allowing recovery lookup + // to mutate the retained ordinary or native provider store. + for alias in &mut aliases { + alias["ReadOnly"] = json!(true); + } + mounts.extend(aliases); json!({}) } else { json!({"/home/freshell/provider":"rw,nosuid,nodev,noexec,size=16m,uid=65534,gid=0,mode=0700"}) @@ -1133,6 +1259,9 @@ impl DockerEngineBackend { "Claude source helper namespace/topology ownership changed", )); } + if let Some(volume) = &retained_volume { + verify_retained_claude_mount_reports(&actual, &body, volume)?; + } let supervision = async { let started = self .request_bounded( diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index c5c39ee9e..28d1181fb 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -455,6 +455,22 @@ async fn provider_claude_source_helper_uses_private_bound_ipc_without_creating_s } } +#[tokio::test] +async fn provider_claude_retained_baseline_consumes_frozen_aliases_without_native_data() { + for mode in [ + "retained-success", + "retained-terminal-success", + "retained-writable-report", + "retained-wrong-bind-source", + "retained-missing-report", + "retained-duplicate-report", + "retained-missing-template", + "retained-changed-manifest", + ] { + qualify_transport(mode).await; + } +} + async fn qualify_transport(mode: &'static str) { let temp = tempfile::tempdir().unwrap(); let workspace = temp.path().join("workspace"); @@ -487,24 +503,139 @@ async fn qualify_transport(mode: &'static str) { daemon_id: daemon.clone(), }, }; + let retained = mode.starts_with("retained-"); + let stage = temp.path().join("prior-stage"); + let native_home = temp.path().join("native-home"); + let resource_relative = ".freshell/provider-roots/frozen-hook"; + let consumed = Arc::new(Mutex::new(None::>)); + let owner = if retained { + use freshell_runtime_protocol::{IncarnationId, InstallationId, LaunchNonce}; + std::fs::create_dir(workspace.join(".claude/hooks")).unwrap(); + std::fs::write( + workspace.join(".claude/hooks/unselected-decoy"), + b"PRIVATE_UNSELECTED_WORKSPACE_HOOK_SENTINEL", + ) + .unwrap(); + std::fs::create_dir_all(stage.join(".freshell/resource-units/frozen-hook")).unwrap(); + std::fs::create_dir_all(stage.join(".freshell/templates")).unwrap(); + std::fs::create_dir_all(native_home.join(".claude/projects")).unwrap(); + std::fs::write( + native_home.join(".claude/projects/native-history.jsonl"), + b"PRIVATE_NATIVE_HISTORY_SENTINEL", + ) + .unwrap(); + std::fs::write( + native_home.join(".claude/.credentials.json"), + b"PRIVATE_NATIVE_CREDENTIAL_SENTINEL", + ) + .unwrap(); + let manifest = b"{}"; + std::fs::write(stage.join(".freshell/input-manifest.json"), manifest).unwrap(); + std::fs::write( + stage.join(".freshell/templates/frozen-settings"), + r#"{"env":{"ORDINARY_MARKER":"frozen-config"}}"#, + ) + .unwrap(); + for root in [ + native_home.join(resource_relative), + stage.join(".freshell/resource-units/frozen-hook"), + ] { + std::fs::create_dir_all(root.join("node_modules/frozen-dependency")).unwrap(); + std::fs::write(root.join("resource"), b"frozen-resource").unwrap(); + std::fs::write( + root.join("node_modules/frozen-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(root.join("consume.cjs"), "const fs=require('node:fs');process.stdout.write(JSON.parse(fs.readFileSync(process.argv[2])).env.ORDINARY_MARKER+':'+fs.readFileSync(__dirname+'/resource')+':'+require('frozen-dependency'));").unwrap(); + } + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let context: freshell_runtime_protocol::ProviderLaunchContext = serde_json::from_value(json!({ + "preparation":{"claude":{"mcp_args":[]}}, "config":[], + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation, + "manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(manifest)),"sourceCount":0,"aliasCount":0,"recipeCount":0}, + "configBinds":[{"sourceRelativePath":".freshell/templates/frozen-settings","logicalDestination":workspace.join(".claude/settings.local.json"),"effectiveDestination":workspace.join(".claude/settings.local.json")}], + "resourceMounts":[{"stageRelativePath":".freshell/resource-units/frozen-hook","providerRelativePath":resource_relative,"logicalDestination":workspace.join(".claude/hooks"),"effectiveDestination":workspace.join(".claude/hooks"),"kind":"directory"}]} + })).unwrap(); + context.validate("claude").unwrap(); + if mode == "retained-missing-template" { + std::fs::remove_file(stage.join(".freshell/templates/frozen-settings")).unwrap(); + } else if mode == "retained-changed-manifest" { + std::fs::write( + stage.join(".freshell/input-manifest.json"), + b"changed ordinary manifest", + ) + .unwrap(); + } + let (terminal, agent) = if mode == "retained-terminal-success" { + (Some(serde_json::from_value(json!({ + "terminalId":"synthetic-terminal", "streamId":"synthetic-stream", "mode":"claude", "program":"claude", "args":[], "env":{}, + "cwd":workspace,"workspacePath":workspace,"runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"synthetic-project","providerLaunchContext":context + })).unwrap()),None) + } else { + (None,Some(serde_json::from_value(json!({ + "sessionId":"synthetic-session", "provider":"claude", "sessionType":"freshclaude", "runtimeVariant":"claude", "providerStoreId":"synthetic-store", + "cwd":workspace, "workspacePath":workspace, "runAsUid":65534, "runAsGid":0, "providerLaunchContext":context + })).unwrap())) + }; + ClaudeSourceOwner::Retained { + handle: OwnedRuntimeHandle::from_registry( + InstallationId::new(), + reservation.soul_id.clone(), + IncarnationId::new(), + LaunchNonce::new(), + daemon.clone(), + "owned-native-container-must-not-stop".into(), + reservation.image_ref.clone(), + temp.path().into(), + reservation.binary.clone(), + "synthetic-owned-config-digest".into(), + freshell_runtime_protocol::RuntimeLimits { + cpu_milli: 500, + memory_bytes: 256 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + }, + None, + terminal, + agent, + "owned-retained-home".into(), + ), + recovery_source: None, + } + } else { + owner + }; let request: ObserveClaudeSourcesRequest = serde_json::from_value(json!({ "expectedControlEpoch":registry.control_epoch(), "soulId":soul, "provider":"claude", "authority":{"kind":"initial_reservation","reservationId":reservation.id}, "workspacePath":workspace,"gitCommonDir":null,"candidate":null, "observationRequest":{"version":1,"requestId":uuid::Uuid::new_v4().to_string(), - "nativeSourceRuntime":"terminal","cwd":workspace,"home":"/home/freshell/provider", + "nativeSourceRuntime":if retained && mode!="retained-terminal-success" {"sdk"} else {"terminal"},"cwd":workspace,"home":"/home/freshell/provider", "settingSources":["local"],"selectorManifestSha256":"b".repeat(64),"planRevision":0} })) .unwrap(); + let mut request = request; + if retained { + request.authority = ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: uuid::Uuid::new_v4().to_string(), + }; + } assert!(matches!( request.authority, ClaudeSourceAuthority::InitialReservation { .. } + | ClaudeSourceAuthority::RecoveryAttempt { .. } )); let requests = Arc::new(Mutex::new(Vec::<(String, Vec)>::new())); let helper_bodies = Arc::new(Mutex::new(Vec::::new())); let listener = UnixListener::bind(temp.path().join("docker.sock")).unwrap(); let seen = requests.clone(); let bodies = helper_bodies.clone(); + let consumer = consumed.clone(); + let retained_root = native_home.clone(); + let settings_target = workspace.join(".claude/settings.local.json"); + let hooks_target = workspace.join(".claude/hooks"); let (stop, mut stopped) = tokio::sync::oneshot::channel::<()>(); let server = tokio::spawn(async move { let mut jobs = JoinSet::new(); @@ -513,7 +644,7 @@ async fn qualify_transport(mode: &'static str) { _ = &mut stopped => break, accepted = listener.accept() => { let (mut stream, _) = accepted.unwrap(); - let (seen, bodies, daemon) = (seen.clone(), bodies.clone(), daemon.clone()); + let (seen, bodies, daemon, consumer, retained_root, settings_target, hooks_target) = (seen.clone(), bodies.clone(), daemon.clone(), consumer.clone(), retained_root.clone(), settings_target.clone(), hooks_target.clone()); jobs.spawn(async move { let mut header = Vec::new(); let mut byte = [0]; @@ -533,7 +664,8 @@ async fn qualify_transport(mode: &'static str) { let (status, response) = if line.contains("/info ") { (200, json!({"ID":daemon,"SecurityOptions":if mode=="nonrootless" {vec![]} else {vec!["name=rootless"]}})) } else if line.starts_with("GET") && line.contains("/volumes/") { - (if mode=="existing_initial_home" {200} else {404}, json!({})) + if retained { (200,json!({"Name":"owned-retained-home","Driver":"local","Options":{},"Mountpoint":retained_root})) } + else { (if mode=="existing_initial_home" {200} else {404}, json!({})) } } else if line.starts_with("GET") && line.contains("/images/") { (200, json!({"Config":{"Env":["ANTHROPIC_API_KEY=PRIVATE_IMAGE_CREDENTIAL_SENTINEL","PATH=/usr/bin"]}})) } else if line.starts_with("POST") && line.contains("/containers/create?") { @@ -543,12 +675,30 @@ async fn qualify_transport(mode: &'static str) { match bodies.lock().unwrap().last().cloned() { Some(mut body) => { if mode=="cleanup_mismatch" && line.contains("/freshell-claude-source-") {body["User"]=json!("foreign-identity");} - (200, json!({"Id":"b".repeat(64),"Config":body,"HostConfig":body["HostConfig"],"State":{"Running":false}})) + let mut reports=body["HostConfig"]["Mounts"].as_array().unwrap().iter().map(|mount| { + json!({"Type":mount["Type"],"Destination":mount["Target"],"Source":if mount["Type"]=="volume" {json!(retained_root)} else {mount["Source"].clone()},"Name":if mount["Type"]=="volume" {mount["Source"].clone()} else {Value::Null},"RW":if mode=="retained-writable-report" && mount["Target"]==json!(hooks_target) {Some(true)} else {mount["ReadOnly"].as_bool().map(|ro|!ro)}}) + }).collect::>(); + if mode=="retained-wrong-bind-source" { + reports.iter_mut().find(|report|report["Destination"]==json!(settings_target)).unwrap()["Source"]=json!("/synthetic/foreign-config"); + } else if mode=="retained-missing-report" { reports.pop(); } + else if mode=="retained-duplicate-report" { let report=reports.iter().find(|report|report["Destination"]==json!(hooks_target)).unwrap().clone();reports.pop();reports.push(report); } + (200, json!({"Id":"b".repeat(64),"Config":body,"HostConfig":body["HostConfig"],"Mounts":reports,"State":{"Running":false}})) }, None => (404, json!({})), } } else if line.starts_with("POST") && line.contains("/start ") { let body = bodies.lock().unwrap().last().unwrap().clone(); + if retained { + let mounts=body["HostConfig"]["Mounts"].as_array().unwrap(); + let config=mounts.iter().find(|mount|mount["Target"]==json!(settings_target) && mount["Type"]=="bind" && mount["ReadOnly"]==true); + let resource=mounts.iter().find(|mount|mount["Target"]==json!(hooks_target) && mount["Type"]=="volume" && mount["Source"]=="owned-retained-home" && mount["ReadOnly"]==true); + if let (Some(config),Some(resource))=(config,resource) { + let root=retained_root.join(resource["VolumeOptions"]["Subpath"].as_str().unwrap()); + let output=std::process::Command::new("node").arg(root.join("consume.cjs")).arg(config["Source"].as_str().unwrap()).output().unwrap(); + assert!(output.status.success(),"selected ordinary consumer failed"); + *consumer.lock().unwrap()=Some(output.stdout); + } + } let exchange = body["HostConfig"]["Mounts"].as_array().unwrap().iter() .find(|mount|mount["Target"]=="/run/freshell-source-exchange").unwrap()["Source"].as_str().unwrap().to_string(); let mut channel = UnixStream::connect(Path::new(&exchange).join("reply.sock")).await.unwrap(); @@ -602,14 +752,26 @@ async fn qualify_transport(mode: &'static str) { jobs.shutdown().await; }); let backend = DockerEngineBackend::new(temp.path().join("docker.sock")); - let result = backend - .read_claude_source_helper( - &owner, - &request, - "synthetic-source-run", - Duration::from_millis(250), - ) - .await; + let result = if retained { + backend + .read_claude_baseline_helper( + &owner, + &request, + "synthetic-source-run", + Duration::from_secs(3), + Some(&stage), + ) + .await + } else { + backend + .read_claude_source_helper( + &owner, + &request, + "synthetic-source-run", + Duration::from_millis(250), + ) + .await + }; let _ = stop.send(()); tokio::time::timeout(Duration::from_secs(5), server) .await @@ -617,7 +779,7 @@ async fn qualify_transport(mode: &'static str) { .unwrap(); assert_eq!( result.is_ok(), - mode == "success", + mode == "success" || matches!(mode, "retained-success" | "retained-terminal-success"), "{mode}: unavailable, contradictory or unjoined evidence cannot be accepted" ); if let Ok(reply) = result { @@ -647,7 +809,13 @@ async fn qualify_transport(mode: &'static str) { let create = seen .iter() .find(|(line, _)| line.contains("/containers/create?")); - if matches!(mode, "nonrootless" | "existing_initial_home") { + if matches!( + mode, + "nonrootless" + | "existing_initial_home" + | "retained-missing-template" + | "retained-changed-manifest" + ) { assert!( create.is_none(), "unqualified creation namespace must not create a helper or native workload" @@ -671,7 +839,10 @@ async fn qualify_transport(mode: &'static str) { .as_array() .unwrap() .iter() - .all(|mount| mount["Type"] == "bind" && mount["ReadOnly"] == true)); + .all( + |mount| (mount["Type"] == "bind" || (retained && mount["Type"] == "volume")) + && mount["ReadOnly"] == true + )); } if mode == "cleanup_mismatch" { assert!( @@ -687,6 +858,37 @@ async fn qualify_transport(mode: &'static str) { assert!( std::fs::read(workspace.join(".claude/settings.local.json")).unwrap() == private.as_bytes() ); + if retained { + if matches!(mode, "retained-success" | "retained-terminal-success") { + assert_eq!(consumed.lock().unwrap().as_deref(),Some(b"frozen-config:frozen-resource:installed-dependency".as_slice()),"retained baseline must consume the prior frozen config/resource aliases, leaving the workspace decoy unselected"); + } else { + assert!( + consumed.lock().unwrap().is_none(), + "incomplete retained topology must block before any consumer" + ); + assert!(!seen + .iter() + .any(|(line, _)| line.starts_with("POST") && line.contains("/start "))); + } + assert_eq!( + std::fs::read(workspace.join(".claude/hooks/unselected-decoy")).unwrap(), + b"PRIVATE_UNSELECTED_WORKSPACE_HOOK_SENTINEL" + ); + assert_eq!( + std::fs::read(native_home.join(".claude/projects/native-history.jsonl")).unwrap(), + b"PRIVATE_NATIVE_HISTORY_SENTINEL" + ); + assert_eq!( + std::fs::read(native_home.join(".claude/.credentials.json")).unwrap(), + b"PRIVATE_NATIVE_CREDENTIAL_SENTINEL" + ); + assert!(seen.iter().all(|(_, raw)| !raw + .windows(b"PRIVATE_NATIVE_HISTORY_SENTINEL".len()) + .any(|bytes| bytes == b"PRIVATE_NATIVE_HISTORY_SENTINEL") + && !raw + .windows(b"PRIVATE_NATIVE_CREDENTIAL_SENTINEL".len()) + .any(|bytes| bytes == b"PRIVATE_NATIVE_CREDENTIAL_SENTINEL"))); + } } async fn qualify_candidate_helper_transport( From 5f029c41324412d3215831d6872decf634617863 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 10:12:27 -0700 Subject: [PATCH 142/493] Reject oversized frozen recovery metadata before reading payloads --- .../src/managed_mcp_capability.rs | 254 ++++++++++-------- 1 file changed, 148 insertions(+), 106 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 37f96483c..09b7623f4 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -858,23 +858,11 @@ fn read_prior_manifest_bytes( return Err("prior provider source context belongs to another owner".into()); } let root = staged_provider_root_path(directory, &capability.grant_id)?; - let path = root.join(&inputs.manifest.relative_path); - if !std::fs::symlink_metadata(&path) - .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) - || std::fs::canonicalize(&path).map_err(|_| "prior source manifest unavailable")? != path - { - return Err("prior source manifest must be a physical private file".into()); - } - use std::io::Read; - let mut bytes = Vec::new(); - std::fs::File::open(path) - .map_err(|_| "prior source manifest unavailable")? - .take(64 * 1024 * 1024 + 1) - .read_to_end(&mut bytes) - .map_err(|_| "prior source manifest read failed")?; - if bytes.len() > 64 * 1024 * 1024 - || format!("{:x}", sha2::Sha256::digest(&bytes)) != inputs.manifest.sha256 - { + let bytes = read_prepared_metadata(&root, &inputs.manifest.relative_path, 64 * 1024 * 1024) + .map_err(|_| { + "prior source manifest unavailable or exceeds profile; retain inputs and reprepare" + })?; + if format!("{:x}", sha2::Sha256::digest(&bytes)) != inputs.manifest.sha256 { return Err("prior source manifest profile or identity changed".into()); } Ok(bytes) @@ -889,30 +877,13 @@ fn read_prior_claude_inline_settings( digest: &str, ) -> Result { use freshell_platform::claude_inputs::{INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES}; - use std::io::Read; let capability = context .mcp_capability .as_ref() .ok_or("prior Claude settings generation authority unavailable")?; - let path = - staged_provider_root_path(directory, &capability.grant_id)?.join(INLINE_SETTINGS_SOURCE); - if !std::fs::symlink_metadata(&path) - .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) - || std::fs::canonicalize(&path).map_err(|_| { - "prior Claude inline settings unavailable; retain inputs and reprepare before dispatch" - })? != path - { - return Err("prior Claude inline settings unavailable or not a physical private source; retain inputs and reprepare before dispatch".into()); - } - let mut bytes = Vec::new(); - std::fs::File::open(path) - .map_err(|_| "prior Claude inline settings unavailable")? - .take(MAX_NATIVE_SETTINGS_BYTES as u64 + 1) - .read_to_end(&mut bytes) - .map_err(|_| "prior Claude inline settings read failed")?; - if bytes.len() > MAX_NATIVE_SETTINGS_BYTES { - return Err("prior Claude inline --settings exceeds native 2 MiB limit; retain inputs and reprepare before dispatch".into()); - } + let root = staged_provider_root_path(directory, &capability.grant_id)?; + let bytes = read_prepared_metadata(&root, INLINE_SETTINGS_SOURCE, MAX_NATIVE_SETTINGS_BYTES) + .map_err(|_| "prior Claude inline settings unavailable or exceeds native 2 MiB limit; retain inputs and reprepare before dispatch")?; if format!("{:x}", sha2::Sha256::digest(&bytes)) != digest { return Err("prior Claude inline settings identity changed; retain inputs and reprepare before dispatch".into()); } @@ -931,41 +902,10 @@ fn read_prior_source_manifest( context .validate("opencode") .map_err(|_| "prior provider source context metadata invalid")?; - let Some(inputs) = context.prepared_inputs.as_ref() else { + let Some(_) = context.prepared_inputs.as_ref() else { return Ok(None); }; - let capability = context - .mcp_capability - .as_ref() - .ok_or("prior source context lacks generation authority")?; - let owned = store - .grants - .lock() - .unwrap() - .get(&capability.grant_id) - .is_some_and(|grant| grant.soul_id == *soul); - if !owned { - return Err("prior provider source context belongs to another owner".into()); - } - let root = staged_provider_root_path(directory, &capability.grant_id)?; - let path = root.join(&inputs.manifest.relative_path); - if !std::fs::symlink_metadata(&path) - .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) - { - return Err("prior source manifest must be a physical private file".into()); - } - use std::io::Read; - let mut bytes = Vec::new(); - std::fs::File::open(path) - .map_err(|_| "prior source manifest unavailable")? - .take(64 * 1024 * 1024 + 1) - .read_to_end(&mut bytes) - .map_err(|_| "prior source manifest read failed")?; - if bytes.len() > 64 * 1024 * 1024 - || format!("{:x}", sha2::Sha256::digest(&bytes)) != inputs.manifest.sha256 - { - return Err("prior source manifest profile or identity changed".into()); - } + let bytes = read_prior_manifest_bytes("opencode", store, directory, soul, context)?; let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice(&bytes).map_err(|_| "prior source manifest metadata invalid")?; if manifest.version != 1 @@ -2470,16 +2410,8 @@ impl CapabilityStore { if !path.exists() { return Ok(None); } - use std::io::Read; - let mut bytes = Vec::new(); - std::fs::File::open(path) - .map_err(|_| "pending prepared result unavailable")? - .take((freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + 1) as u64) - .read_to_end(&mut bytes) - .map_err(|_| "pending prepared result read failed")?; - if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { - return Err("pending prepared result exceeds metadata profile".into()); - } + let bytes = read_prepared_metadata(&stage, ".freshell/prepared-result.json", freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES) + .map_err(|_| "pending prepared result unavailable or exceeds metadata profile; retain inputs and reprepare")?; let result: freshell_runtime_protocol::PreparedProviderContext = serde_json::from_slice(&bytes) .map_err(|_| "pending prepared result metadata invalid")?; @@ -2807,17 +2739,9 @@ fn read_prepared_context( capability: &McpCapabilityReference, ) -> Result { let result = (|| { - use std::io::Read; let stage = staged_provider_root_path(&directory, &capability.grant_id)?; - let mut bytes = Vec::new(); - std::fs::File::open(stage.join(".freshell/prepared-context.json")) - .map_err(|_| "complete prepared provider context unavailable")? - .take((freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES + 1) as u64) - .read_to_end(&mut bytes) - .map_err(|_| "complete prepared provider context read failed")?; - if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { - return Err("complete prepared provider context exceeds metadata profile".into()); - } + let bytes = read_prepared_metadata(&stage, ".freshell/prepared-context.json", freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES) + .map_err(|_| "complete prepared provider context unavailable or exceeds metadata profile; retain inputs and reprepare")?; let context: freshell_runtime_protocol::ProviderLaunchContext = serde_json::from_slice(&bytes) .map_err(|_| "complete prepared provider context metadata invalid")?; @@ -2831,14 +2755,10 @@ fn read_prepared_context( // These are selected issuer metadata, never original HOME or capability values. // Check physical size before reserving, and never allocate a growth sentinel. -fn read_claude_candidate_metadata( - root: &Path, - relative: &str, - limit: usize, -) -> Result, String> { +fn read_prepared_metadata(root: &Path, relative: &str, limit: usize) -> Result, String> { use std::io::Read; let blocked = || { - "Claude frozen candidate private metadata unavailable or exceeds profile; retain inputs and reprepare".to_string() + "Selected provider private metadata unavailable or exceeds profile; retain inputs and reprepare".to_string() }; if !Path::new(relative) .components() @@ -2950,7 +2870,7 @@ fn bind_claude_candidate_request_and_plan( return Err(blocked()); } let root = staged_provider_root_path(directory, &generation).map_err(|_| blocked())?; - let published = read_claude_candidate_metadata( + let published = read_prepared_metadata( &root, ".freshell/claude-candidate-binding.json", MAX_CONTROL_FRAME_BYTES, @@ -2963,12 +2883,12 @@ fn bind_claude_candidate_request_and_plan( // and recovery authority are requalified below before any IPC. request.observation_request.plan_revision = publication.plan_revision; } - let context_bytes = read_claude_candidate_metadata( + let context_bytes = read_prepared_metadata( &root, ".freshell/prepared-context.json", MAX_CONTROL_FRAME_BYTES, )?; - let ready_bytes = read_claude_candidate_metadata( + let ready_bytes = read_prepared_metadata( &root, ".freshell/prepared-result.json", MAX_CONTROL_FRAME_BYTES, @@ -2976,7 +2896,7 @@ fn bind_claude_candidate_request_and_plan( let ready: PreparedProviderContext = serde_json::from_slice(&ready_bytes).map_err(|_| blocked())?; let manifest_bytes = - read_claude_candidate_metadata(&root, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + read_prepared_metadata(&root, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; let manifest: ClaudeInputManifest = serde_json::from_slice(&manifest_bytes).map_err(|_| blocked())?; manifest.validate().map_err(|_| blocked())?; @@ -3194,6 +3114,130 @@ pub(crate) async fn scoped_auth_middleware( mod tests { use super::*; + #[cfg(target_os = "linux")] + #[test] + fn managed_provider_recovery_rejects_oversized_manifest_before_payload_read() { + const CHILD: &str = "FRESHELL_TEST_RECOVERY_SPARSE_MANIFEST_CHILD"; + if std::env::var_os(CHILD).is_none() { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "managed_mcp_capability::tests::managed_provider_recovery_rejects_oversized_manifest_before_payload_read", "--nocapture"]) + .env(CHILD, "1") + .output() + .unwrap(); + assert!( + String::from_utf8_lossy(&output.stdout) + .contains("recovery_sparse_manifest_reader_ready"), + "allocation fixture must execute its child reader: {}", + String::from_utf8_lossy(&output.stdout) + ); + eprintln!( + "recovery_sparse_manifest_child_result status={} oversized_payload_read={}", + output.status, + String::from_utf8_lossy(&output.stderr) + .contains("oversized manifest payload was read") + ); + assert!(output.status.success(), "oversized recovery metadata must reject before reading payload and preserve its generation: {}", output.status); + return; + } + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let directory = owned.path().join("control/mcp-capabilities"); + for path in [&home.join(".codex"), &workspace, &directory] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write(home.join(".codex/config.toml"), "model = 'frozen-model'\n").unwrap(); + let native = home.join(".codex/native-history.jsonl"); + std::fs::write(&native, "synthetic-native-store-preserved").unwrap(); + let context = crate::managed_provider_bootstrap::provider_launch_context_from_home( + "codex", &workspace, &home, None, + ) + .unwrap() + .unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let grant = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("codex", &home, &context.config)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let accepted = read_prepared_context(&directory, &grant).unwrap(); + let original = + read_prior_manifest_bytes("codex", &store, &directory, &soul, &accepted).unwrap(); + let manifest: freshell_platform::codex_inputs::CodexInputManifest = + serde_json::from_slice(&original).unwrap(); + manifest.validate().unwrap(); + let stage = staged_provider_root_path(&directory, &grant.grant_id).unwrap(); + let saved_context = std::fs::read(stage.join(".freshell/prepared-context.json")).unwrap(); + let path = stage.join( + &accepted + .prepared_inputs + .as_ref() + .unwrap() + .manifest + .relative_path, + ); + let mut sparse = std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(&path) + .unwrap(); + sparse.set_len(64 * 1024 * 1024 + 1).unwrap(); + let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + sparse + .set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + std::io::Read::read_exact(&mut sparse, &mut [0u8; 1]).unwrap(); + assert_ne!( + std::fs::metadata(&path).unwrap().accessed().unwrap(), + old_access, + "fixture filesystem must report actual payload reads" + ); + sparse + .set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + drop(sparse); + let pages: u64 = std::fs::read_to_string("/proc/self/statm") + .unwrap() + .split_whitespace() + .next() + .unwrap() + .parse() + .unwrap(); + let page_size = unsafe { libc::sysconf(libc::_SC_PAGESIZE) } as u64; + let limit = libc::rlimit { + rlim_cur: pages * page_size + 8 * 1024 * 1024, + rlim_max: pages * page_size + 8 * 1024 * 1024, + }; + println!("recovery_sparse_manifest_reader_ready"); + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_AS, &limit) }, 0); + assert!(read_prior_manifest_bytes("codex", &store, &directory, &soul, &accepted).is_err()); + assert_eq!( + std::fs::metadata(&path).unwrap().accessed().unwrap(), + old_access, + "oversized manifest payload was read before its physical bound was rejected" + ); + assert!(store.grants.lock().unwrap().contains_key(&grant.grant_id)); + assert_eq!( + std::fs::read(stage.join(".freshell/prepared-context.json")).unwrap(), + saved_context + ); + assert_eq!( + std::fs::read_to_string(native).unwrap(), + "synthetic-native-store-preserved" + ); + } + #[cfg(all(unix, feature = "managed-provider-qualification"))] #[test] fn managed_provider_amplifier_native_local_bundle_survives_transport() { @@ -3859,18 +3903,16 @@ except Exception as error: let path = owned.path().join("metadata"); let file = std::fs::File::create(&path).unwrap(); file.set_len(limit as u64).unwrap(); - let bytes = read_claude_candidate_metadata(owned.path(), "metadata", limit) + let bytes = read_prepared_metadata(owned.path(), "metadata", limit) .expect("supported inclusive metadata limit must remain readable"); assert_eq!(bytes.len(), limit); drop(bytes); file.set_len(limit as u64 + 1).unwrap(); - assert!(read_claude_candidate_metadata(owned.path(), "metadata", limit).is_err()); - assert!( - read_claude_candidate_metadata(owned.path(), "../metadata", limit + 1).is_err() - ); + assert!(read_prepared_metadata(owned.path(), "metadata", limit).is_err()); + assert!(read_prepared_metadata(owned.path(), "../metadata", limit + 1).is_err()); std::fs::rename(&path, owned.path().join("escaped")).unwrap(); std::os::unix::fs::symlink(owned.path().join("escaped"), &path).unwrap(); - assert!(read_claude_candidate_metadata(owned.path(), "metadata", limit + 1).is_err()); + assert!(read_prepared_metadata(owned.path(), "metadata", limit + 1).is_err()); } } From 9640bfbd13a2ea5967422b9d48ef9a5bbbf7e042 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 10:20:53 -0700 Subject: [PATCH 143/493] Qualify Claude recovery in its current workspace while retaining source origins --- .../src/backend/claude_source_observation.rs | 6 +- .../claude_source_observation_tests.rs | 129 ++++++++++++++++-- 2 files changed, 120 insertions(+), 15 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index 1632685e1..dfe75e06f 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -697,7 +697,11 @@ fn qualify_candidate_stage( if plan.selector_manifest_sha256 != format!("{:x}", Sha256::digest(selectors)) || plan.selector_manifest_sha256 != source.selector_manifest_sha256 || plan.plan_revision != source.plan_revision - || manifest.workspace != Path::new(&request.workspace_path) + // Recovery retains creation-time selector origins. Its current + // projection workspace is bound by the issuer and recovery owner + // above, rather than replacing the frozen selector workspace. + || (matches!(owner, ClaudeSourceOwner::Initial { .. }) + && manifest.workspace != Path::new(&request.workspace_path)) || manifest.setting_sources != source.setting_sources || plan.observation.setting_sources != source.setting_sources || plan.observation.native_source_runtime != source.native_source_runtime diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 28d1181fb..2c83b7177 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -176,6 +176,7 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() ] { for mode in [ "valid", + "retained-current-workspace", "missing-resource", "wrong-soul", "wrong-reservation", @@ -198,6 +199,9 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() let stage = owned.path().join("stage"); std::fs::create_dir(&workspace).unwrap(); std::fs::create_dir(&original).unwrap(); + let retained = mode == "retained-current-workspace"; + let prior_workspace = owned.path().join("original-workspace"); + std::fs::create_dir(&prior_workspace).unwrap(); std::fs::create_dir_all(stage.join(".freshell")).unwrap(); let resource = stage.join(".freshell/resource-units/unit-0/probe"); std::fs::create_dir_all(resource.join("node_modules/frozen-dependency")).unwrap(); @@ -208,7 +212,11 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() .unwrap(); std::fs::write( resource.join("consumer.cjs"), - "process.stdout.write(require('frozen-dependency'))", + if retained { + "process.stdout.write(require('frozen-dependency')+':'+process.cwd())" + } else { + "process.stdout.write(require('frozen-dependency'))" + }, ) .unwrap(); let binary = owned.path().join("synthetic-host"); @@ -233,8 +241,16 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() version: 1, native_source_runtime: Some(runtime), original_home: original.clone(), - cwd: workspace.clone(), - workspace: workspace.clone(), + cwd: if retained { + prior_workspace.clone() + } else { + workspace.clone() + }, + workspace: if retained { + prior_workspace.clone() + } else { + workspace.clone() + }, config_root: None, settings_file: None, settings_origin: None, @@ -304,21 +320,36 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() } else { soul }; - let context:ProviderLaunchContext=serde_json::from_value(json!({ + let mut context:ProviderLaunchContext=serde_json::from_value(json!({ "preparation":{"claude":{"mcp_args":[]}},"config":[], "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, "preparedInputs":{"generationId":generation,"initialAuthReservation":reservation_id, "manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&manifest_bytes)),"sourceCount":0,"aliasCount":0,"recipeCount":0}, "resourceMounts":[{"stageRelativePath":".freshell/resource-units/unit-0","providerRelativePath":".freshell/provider-roots/unit-0","logicalDestination":"/home/freshell/provider/.claude/skills","effectiveDestination":"/home/freshell/provider/.claude/skills","kind":"directory"}]} })).unwrap(); + if retained { + context + .prepared_inputs + .as_mut() + .unwrap() + .initial_auth_reservation = None; + } let ready = PreparedProviderContext { provider_launch_context: context.clone(), provider_secret_references: vec![], }; let result_bytes = bounded_json_bytes(&ready, MAX_CONTROL_FRAME_BYTES).unwrap(); let context_bytes = bounded_json_bytes(&context, MAX_CONTROL_FRAME_BYTES).unwrap(); - let authority = ClaudeSourceAuthority::InitialReservation { - reservation_id: reservation_id.clone(), + let attempt = freshell_runtime_protocol::RecoveryAttemptId::new(); + let prior_generation = format!("grant-{}", uuid::Uuid::new_v4()); + let authority = if retained { + ClaudeSourceAuthority::RecoveryAttempt { + attempt_id: uuid::Uuid::new_v4().to_string(), + } + } else { + ClaudeSourceAuthority::InitialReservation { + reservation_id: reservation_id.clone(), + } }; let publication = ClaudeCandidatePublication { version: 1, @@ -330,10 +361,10 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() bounded_json_bytes(&authority, MAX_CONTROL_FRAME_BYTES).unwrap() ) ), - initial_reservation_id: Some(reservation_id), - registry_attempt_id: None, - intent_revision: 1, - prior_generation_id: None, + initial_reservation_id: if retained { None } else { Some(reservation_id) }, + registry_attempt_id: retained.then_some(attempt.clone()), + intent_revision: if retained { 2 } else { 1 }, + prior_generation_id: retained.then_some(prior_generation.clone()), generation_id: generation.clone(), manifest_sha256: format!("{:x}", Sha256::digest(&manifest_bytes)), prepared_context_sha256: format!("{:x}", Sha256::digest(&context_bytes)), @@ -352,7 +383,7 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() "observationRequest":{"version":1,"requestId":uuid::Uuid::new_v4().to_string(),"nativeSourceRuntime":runtime,"cwd":workspace,"home":"/home/freshell/provider","settingSources":["user"],"selectorManifestSha256":selector,"planRevision":0, "candidateBinding":{"soulId":candidate_soul,"generationId":generation,"authoritySha256":publication.authority_sha256,"publicationSha256":format!("{:x}",Sha256::digest(&publication_bytes)),"manifestSha256":publication.manifest_sha256,"preparedResultSha256":publication.prepared_result_sha256}} })).unwrap(); - request.candidate_projection(publication).expect("coherent wire fixture must pass metadata shape checks before physical/owner qualification"); + request.candidate_projection(publication.clone()).expect("coherent wire fixture must pass metadata shape checks before physical/owner qualification"); std::fs::write(stage.join(".freshell/input-manifest.json"), manifest_bytes).unwrap(); std::fs::write(stage.join(".freshell/prepared-context.json"), context_bytes).unwrap(); std::fs::write(stage.join(".freshell/prepared-result.json"), result_bytes).unwrap(); @@ -399,10 +430,72 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() daemon_id: DockerDaemonId::new(), }, }; + let owner = if retained { + use freshell_runtime_protocol::{IncarnationId, InstallationId, LaunchNonce}; + let ClaudeSourceOwner::Initial { + reservation, + observation: InitialAuthVolumeObservation::Absent { daemon_id }, + } = owner + else { + unreachable!() + }; + let mut prior_context = context.clone(); + prior_context.mcp_capability.as_mut().unwrap().grant_id = prior_generation.clone(); + prior_context + .prepared_inputs + .as_mut() + .unwrap() + .generation_id = prior_generation.clone(); + prior_context.validate("claude").unwrap(); + let (terminal, agent) = if runtime == NativeSourceRuntime::Terminal { + (Some(serde_json::from_value(json!({"terminalId":"retained-terminal","streamId":"retained-stream","mode":"claude","program":"claude","args":[],"env":{},"cwd":prior_workspace,"workspacePath":prior_workspace,"runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"retained-project","providerLaunchContext":prior_context})).unwrap()),None) + } else { + (None,Some(serde_json::from_value(json!({"sessionId":"retained-session","provider":flavor,"sessionType":if flavor=="kilroy" {"kilroy"} else {"freshclaude"},"runtimeVariant":"claude","providerStoreId":"retained-store","cwd":prior_workspace,"workspacePath":prior_workspace,"runAsUid":65534,"runAsGid":0,"providerLaunchContext":prior_context})).unwrap())) + }; + ClaudeSourceOwner::Retained { + handle: OwnedRuntimeHandle::from_registry( + InstallationId::new(), + reservation.soul_id, + IncarnationId::new(), + LaunchNonce::new(), + daemon_id, + "retained-owned-container".into(), + reservation.image_ref, + owned.path().into(), + reservation.binary, + "retained-config-digest".into(), + freshell_runtime_protocol::RuntimeLimits { + cpu_milli: 500, + memory_bytes: 256 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + }, + None, + terminal, + agent, + "retained-owned-home".into(), + ), + recovery_source: Some(freshell_runtime_protocol::ClaudeRecoverySourceBinding { + provider: flavor.into(), + git_common_dir: None, + authority_sha256: publication.authority_sha256, + registry_attempt_id: attempt, + intent_revision: 2, + prior_generation_id: prior_generation, + }), + } + } else { + owner + }; + if retained { + std::fs::remove_dir(&prior_workspace).unwrap(); + } let qualified = qualify_candidate_stage(&stage, &owner, &request); - assert_eq!(qualified.is_ok(),mode=="valid","{flavor}/{runtime:?}/{mode}: physical issuer and current owner must qualify before consumer selection: {:?}",qualified.as_ref().err()); + assert_eq!(qualified.is_ok(),mode=="valid" || retained,"{flavor}/{runtime:?}/{mode}: physical issuer and current owner must qualify before consumer selection: {:?}",qualified.as_ref().err()); if let Ok(projection) = qualified { - qualify_candidate_helper_transport(&owner, &request, &stage).await; + if !retained { + qualify_candidate_helper_transport(&owner, &request, &stage).await; + } let selected = &projection .provider_launch_context .prepared_inputs @@ -421,7 +514,15 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() .output() .unwrap(); assert!(output.status.success()); - assert_eq!(output.stdout, b"qualified-frozen-dependency"); + if retained { + assert_eq!( + output.stdout, + format!("qualified-frozen-dependency:{}", workspace.display()).as_bytes() + ); + assert!(!prior_workspace.exists()); + } else { + assert_eq!(output.stdout, b"qualified-frozen-dependency"); + } } else { assert!(!qualified .unwrap_err() From ca7b76c81aa9e67fb09f371b32480cd56204f061 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 10:38:39 -0700 Subject: [PATCH 144/493] Bound host private source reads before allocation and reject changed files --- .../src/provider_input_preparation.rs | 139 +++++++++++++++++- 1 file changed, 132 insertions(+), 7 deletions(-) diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 5992dc8b1..18b9a3083 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -16,18 +16,90 @@ pub(crate) fn read_private(root: &Path, relative: &str, limit: usize) -> Result< std::fs::symlink_metadata(&path).map_err(|_| "prepared private source unavailable")?; if !metadata.is_file() || metadata.file_type().is_symlink() + || metadata.len() > limit as u64 || std::fs::canonicalize(&path).map_err(|_| "prepared private source unavailable")? != path { - return Err("prepared private source must be a physical stage file".into()); + return Err( + "prepared private source must be a physical stage file within its byte profile".into(), + ); + } + let mut options = std::fs::OpenOptions::new(); + options.read(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.custom_flags(libc::O_NOFOLLOW); + } + let mut file = options + .open(&path) + .map_err(|_| "prepared private source unreadable")?; + let opened = file + .metadata() + .map_err(|_| "prepared private source unavailable")?; + let modified = metadata + .modified() + .map_err(|_| "prepared private source metadata unavailable")?; + if opened.len() != metadata.len() + || opened + .modified() + .map_err(|_| "prepared private source metadata unavailable")? + != modified + { + return Err( + "prepared private source changed before reading; retain inputs and reprepare".into(), + ); + } + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + if (opened.dev(), opened.ino()) != (metadata.dev(), metadata.ino()) { + return Err( + "prepared private source inode changed before reading; retain inputs and reprepare" + .into(), + ); + } } + let length = usize::try_from(metadata.len()) + .map_err(|_| "prepared private source byte profile exceeded")?; let mut bytes = Vec::new(); - std::fs::File::open(path) - .map_err(|_| "prepared private source unreadable")? - .take(limit as u64 + 1) - .read_to_end(&mut bytes) + bytes + .try_reserve_exact(length) + .map_err(|_| "prepared private source allocation unavailable")?; + bytes.resize(length, 0); + file.read_exact(&mut bytes) .map_err(|_| "prepared private source read failed")?; - if bytes.len() > limit { - return Err("prepared source-view or filesystem metadata profile exceeded".into()); + let mut extra = [0]; + if file + .read(&mut extra) + .map_err(|_| "prepared private source completion unavailable")? + != 0 + { + return Err( + "prepared private source grew during reading; retain inputs and reprepare".into(), + ); + } + let after = std::fs::symlink_metadata(&path) + .map_err(|_| "prepared private source unavailable after reading")?; + if after.file_type().is_symlink() + || after.len() != metadata.len() + || after + .modified() + .map_err(|_| "prepared private source metadata unavailable")? + != modified + { + return Err( + "prepared private source changed during reading; retain inputs and reprepare".into(), + ); + } + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + if (after.dev(), after.ino()) != (metadata.dev(), metadata.ino()) { + return Err( + "prepared private source inode changed during reading; retain inputs and reprepare" + .into(), + ); + } } Ok(bytes) } @@ -395,6 +467,59 @@ pub fn prepare( #[cfg(test)] mod tests { use super::*; + + #[cfg(target_os = "linux")] + #[test] + fn provider_secret_private_reader_rejects_oversize_before_payload_read() { + use std::io::Read; + let owned = tempfile::tempdir().unwrap(); + let private = owned.path().join("metadata"); + let native = owned.path().join("native-history.jsonl"); + std::fs::write(&native, b"synthetic-native-store-preserved").unwrap(); + let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + for limit in [ + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + freshell_platform::claude_inputs::MAX_NATIVE_SETTINGS_BYTES, + 64 * 1024 * 1024, + ] { + let mut file = std::fs::OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(true) + .open(&private) + .unwrap(); + file.set_len(limit as u64).unwrap(); + let accepted = read_private(owned.path(), "metadata", limit).unwrap(); + assert_eq!( + accepted.len(), + limit, + "inclusive physical file limit must remain supported" + ); + drop(accepted); + file.set_len(limit as u64 + 1).unwrap(); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + file.read_exact(&mut [0u8; 1]).unwrap(); + assert_ne!( + std::fs::metadata(&private).unwrap().accessed().unwrap(), + old_access, + "fixture must report actual reads" + ); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + assert!(read_private(owned.path(), "metadata", limit).is_err()); + assert_eq!( + std::fs::metadata(&private).unwrap().accessed().unwrap(), + old_access, + "host must reject oversized private metadata before reading its payload" + ); + assert_eq!( + std::fs::read(&native).unwrap(), + b"synthetic-native-store-preserved" + ); + } + } use freshell_platform::opencode_recipe::*; use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; From e433cbd642b0ae9c551022543c3743ae0f6fea99 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 11:23:44 -0700 Subject: [PATCH 145/493] Reconstruct retained Claude candidates from frozen ordinary inputs and metadata --- .../src/claude_source_observation.rs | 492 ++++++++++++++++++ 1 file changed, 492 insertions(+) diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index 043260cd3..8d0ccf797 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -302,6 +302,304 @@ fn refresh_candidate_projection( .map_err(|_| candidate_projection_unavailable("ordinary_refresh")) } +fn seed_retained_ordinary_candidate( + prior_generation: &str, + stage: &std::path::Path, + retained_home: &std::path::Path, + home: &std::path::Path, + current: &freshell_runtime_protocol::ProviderLaunchContext, + identity: (u32, u32), +) -> Result<(), String> { + use std::{ + collections::BTreeSet, + os::unix::fs::{MetadataExt, PermissionsExt}, + path::{Path, PathBuf}, + }; + let blocked = || candidate_projection_unavailable("retained_ordinary_topology"); + crate::verify_initialized_provider_home(retained_home, identity.0, identity.1) + .map_err(|_| blocked())?; + crate::verify_initialized_provider_home(home, identity.0, identity.1).map_err(|_| blocked())?; + let raw = crate::provider_input_preparation::read_private( + stage, + ".freshell/prepared-context.json", + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .map_err(|_| blocked())?; + let prior: freshell_runtime_protocol::ProviderLaunchContext = + serde_json::from_slice(&raw).map_err(|_| blocked())?; + prior.validate("claude").map_err(|_| blocked())?; + current.validate("claude").map_err(|_| blocked())?; + let inputs = prior.prepared_inputs.as_ref().ok_or_else(blocked)?; + if inputs.generation_id != prior_generation + || inputs.indexed_files.is_some() + || inputs.manifest.source_count != 0 + || inputs.manifest.alias_count != 0 + || inputs.manifest.recipe_count != 0 + { + return Err(blocked()); + } + let manifest = crate::provider_input_preparation::read_private( + stage, + &inputs.manifest.relative_path, + 64 * 1024 * 1024, + ) + .map_err(|_| blocked())?; + if format!("{:x}", Sha256::digest(&manifest)) != inputs.manifest.sha256 { + return Err(blocked()); + } + drop(manifest); + drop(raw); + let protected: BTreeSet = current + .prepared_inputs + .iter() + .flat_map(|inputs| { + inputs + .config_binds + .iter() + .map(|bind| bind.effective_destination.as_str()) + .chain( + inputs + .resource_mounts + .iter() + .map(|unit| unit.effective_destination.as_str()), + ) + }) + .filter_map(|path| { + Path::new(path) + .strip_prefix("/home/freshell/provider") + .ok() + .map(Path::to_path_buf) + }) + .collect(); + struct Metadata { + path: PathBuf, + source: std::fs::Metadata, + link: Option, + native_sibling: bool, + } + let root_uid = unsafe { libc::geteuid() }; + // The helper keeps its saved root identity only during private preparation. + // Filesystem reads and the common copier use the provider's effective UID; + // the existing permanent transition still precedes source observation. + if unsafe { libc::seteuid(identity.0) } != 0 { + return Err(blocked()); + } + let capture = (|| { + let raw = crate::provider_input_preparation::read_private( + retained_home, + crate::PROVIDER_PROJECTION_MANIFEST, + 64 * 1024 * 1024, + ) + .map_err(|_| blocked())?; + let tracked: BTreeSet = serde_json::from_slice(&raw).map_err(|_| blocked())?; + drop(raw); + if tracked.len() + > freshell_platform::provider_environment::CopyLimits::default().max_entries + { + return Err(blocked()); + } + let ordinary: Vec = crate::provider_config_references(&prior) + .iter() + .map(|reference| PathBuf::from(&reference.provider_relative_path)) + .chain( + inputs + .resource_mounts + .iter() + .map(|unit| PathBuf::from(&unit.provider_relative_path)), + ) + .collect(); + if tracked.iter().any(|entry| { + let entry = Path::new(entry); + entry.as_os_str().is_empty() + || entry + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + || !ordinary + .iter() + .any(|selected| entry.starts_with(selected) || selected.starts_with(entry)) + }) { + return Err(blocked()); + } + let mut paths: BTreeSet = tracked.iter().map(PathBuf::from).collect(); + paths.insert(PathBuf::new()); + for name in [ + "", + "skills", + "commands", + "agents", + "hooks", + "plugins", + "output-styles", + "workflows", + "routines", + "rules", + ] { + paths.insert(Path::new(".claude").join(name)); + } + for context in [&prior, current] { + for target in context.prepared_inputs.iter().flat_map(|inputs| { + inputs + .config_binds + .iter() + .map(|bind| bind.effective_destination.as_str()) + .chain( + inputs + .resource_mounts + .iter() + .map(|unit| unit.effective_destination.as_str()), + ) + }) { + if let Ok(relative) = Path::new(target).strip_prefix("/home/freshell/provider") { + for ancestor in relative.ancestors() { + paths.insert(ancestor.into()); + } + } + } + } + let mut metadata = Vec::new(); + let mut bytes = 0usize; + for relative in paths { + if relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + || relative.components().count() > 64 + { + return Err(blocked()); + } + if !relative.as_os_str().is_empty() + && protected.iter().any(|alias| relative.starts_with(alias)) + { + continue; + } + let source = retained_home.join(&relative); + let observed = match std::fs::symlink_metadata(&source) { + Ok(value) => value, + Err(error) + if matches!( + error.kind(), + std::io::ErrorKind::NotFound | std::io::ErrorKind::NotADirectory + ) => + { + continue + } + Err(_) => return Err(blocked()), + }; + if !relative.as_os_str().is_empty() + && !std::fs::canonicalize(source.parent().ok_or_else(blocked)?) + .map_err(|_| blocked())? + .starts_with(retained_home) + { + return Err(blocked()); + } + let link = if observed.file_type().is_symlink() { + Some(std::fs::read_link(&source).map_err(|_| blocked())?) + } else { + None + }; + if !observed.is_dir() && !observed.is_file() && link.is_none() { + return Err(blocked()); + } + let native_sibling = if observed.is_dir() { + let mut unknown = false; + for entry in std::fs::read_dir(&source).map_err(|_| blocked())? { + let entry = entry.map_err(|_| blocked())?; + let child = relative.join(entry.file_name()); + if child.to_str().is_none_or(|child| !tracked.contains(child)) { + unknown = true; + } + } + unknown + } else { + false + }; + bytes = bytes + .checked_add( + relative.as_os_str().len() + + std::mem::size_of::() + + link.as_ref().map_or(0, |path| path.as_os_str().len()), + ) + .filter(|bytes| *bytes <= 64 * 1024 * 1024) + .ok_or_else(blocked)?; + metadata.try_reserve(1).map_err(|_| blocked())?; + metadata.push(Metadata { + path: relative, + source: observed, + link, + native_sibling, + }); + } + crate::copy_provider_config_references_with_ownership( + &prior, "claude", stage, stage, home, home, identity, + ) + .map_err(|_| blocked())?; + Ok::<_, String>(metadata) + })(); + let restored = unsafe { libc::seteuid(root_uid) } == 0; + if !restored { + return Err(candidate_projection_unavailable( + "retained_identity_restore", + )); + } + let mut metadata = capture?; + // Give only these owned scratch parents temporary access while restoring + // their recorded metadata. The read-only retained HOME is never changed. + for entry in metadata.iter().filter(|entry| entry.source.is_dir()) { + let destination = home.join(&entry.path); + std::fs::create_dir_all(&destination).map_err(|_| blocked())?; + crate::set_owner(&destination, root_uid, identity.1).map_err(|_| blocked())?; + std::fs::set_permissions(&destination, std::fs::Permissions::from_mode(0o700)) + .map_err(|_| blocked())?; + if entry.native_sibling { + let marker = destination.join(format!( + ".freshell-retained-presence-{}", + uuid::Uuid::new_v4() + )); + std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(marker) + .map_err(|_| blocked())?; + } + } + metadata.sort_by_key(|entry| std::cmp::Reverse(entry.path.components().count())); + for entry in metadata { + let destination = home.join(&entry.path); + if let Some(target) = entry.link { + if std::fs::symlink_metadata(&destination).is_ok() { + std::fs::remove_file(&destination).map_err(|_| blocked())?; + } + std::os::unix::fs::symlink(target, &destination).map_err(|_| blocked())?; + } else if entry.source.is_file() { + if !destination.exists() { + std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&destination) + .map_err(|_| blocked())?; + } + crate::set_owner(&destination, root_uid, identity.1).map_err(|_| blocked())?; + std::fs::set_permissions( + &destination, + std::fs::Permissions::from_mode(entry.source.mode() & 0o7777), + ) + .map_err(|_| blocked())?; + } else { + std::fs::set_permissions( + &destination, + std::fs::Permissions::from_mode(entry.source.mode() & 0o7777), + ) + .map_err(|_| blocked())?; + } + use std::os::unix::ffi::OsStrExt; + let name = + std::ffi::CString::new(destination.as_os_str().as_bytes()).map_err(|_| blocked())?; + if unsafe { libc::lchown(name.as_ptr(), entry.source.uid(), entry.source.gid()) } != 0 { + return Err(blocked()); + } + } + Ok(()) +} + #[cfg(test)] fn project_candidate( request: &freshell_runtime_protocol::ClaudeCandidateProjectionRequest, @@ -412,6 +710,16 @@ pub(crate) fn serve_candidate_private() -> Result<(), String> { let raw = freshell_runtime_protocol::bounded_json_bytes(&request.source_request, MAX_REQUEST_BYTES) .map_err(|_| "Claude candidate source request exceeds metadata profile")?; + if let Some(prior) = request.publication.prior_generation_id.as_deref() { + seed_retained_ordinary_candidate( + prior, + Path::new("/run/freshell-retained-inputs"), + Path::new("/run/freshell-retained-home"), + home, + &request.provider_launch_context, + (65534, 0), + )?; + } validate_candidate_projection( &request, Path::new("/run/freshell-candidate-inputs"), @@ -524,6 +832,190 @@ mod tests { use super::*; use std::io::{Cursor, Error, ErrorKind}; + #[test] + fn claude_retained_candidate_seed_preserves_metadata_and_executes_only_frozen_ordinary_inputs() + { + use std::os::unix::fs::{MetadataExt, PermissionsExt}; + let owned = tempfile::tempdir().unwrap(); + let stage = owned.path().join("stage"); + let retained = owned.path().join("retained-home"); + let home = owned.path().join("candidate-home"); + let workspace = owned.path().join("workspace"); + for path in [&stage, &retained, &home, &workspace] { + std::fs::create_dir(path).unwrap(); + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).unwrap(); + } + std::fs::create_dir_all( + stage.join(".freshell/resource-units/unit-0/node_modules/frozen-dependency"), + ) + .unwrap(); + std::fs::create_dir_all(stage.join(".claude")).unwrap(); + std::fs::write( + stage.join(".claude/old-settings.json"), + br#"{"ordinary":"frozen"}"#, + ) + .unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), b"{}").unwrap(); + std::fs::write( + stage.join(".freshell/resource-units/unit-0/resource"), + b"frozen-resource", + ) + .unwrap(); + std::fs::write( + stage.join(".freshell/resource-units/unit-0/node_modules/frozen-dependency/index.js"), + "module.exports='installed-dependency'", + ) + .unwrap(); + std::fs::write(stage.join(".freshell/resource-units/unit-0/consumer.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('frozen-dependency'))").unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let context: freshell_runtime_protocol::ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"claude":{"mcp_args":[]}}, "config":[{"root":"user_provider","relativePath":".claude/old-settings.json","providerRelativePath":".claude/old-settings.json","format":"json"}], + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(b"{}")),"sourceCount":0,"aliasCount":0,"recipeCount":0}, + "resourceMounts":[{"stageRelativePath":".freshell/resource-units/unit-0","providerRelativePath":".freshell/provider-roots/unit-0","logicalDestination":workspace.join("selected-plugin"),"effectiveDestination":workspace.join("selected-plugin"),"kind":"directory"}]} + })).unwrap(); + context.validate("claude").unwrap(); + std::fs::write( + stage.join(".freshell/prepared-context.json"), + freshell_runtime_protocol::bounded_json_bytes( + &context, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .unwrap(), + ) + .unwrap(); + let identity = unsafe { (libc::geteuid(), libc::getegid()) }; + std::fs::create_dir_all(retained.join(".freshell/provider-roots/unit-0")).unwrap(); + crate::copy_provider_config_references_with_ownership( + &context, "claude", &workspace, &stage, &retained, &retained, identity, + ) + .unwrap(); + std::fs::create_dir_all(retained.join(".claude/projects")).unwrap(); + let history = retained.join(".claude/projects/history.jsonl"); + let auth = retained.join(".claude/.credentials.json"); + std::fs::write(&history, b"PRIVATE_NATIVE_HISTORY_SENTINEL").unwrap(); + std::fs::write(&auth, b"PRIVATE_NATIVE_CREDENTIAL_SENTINEL").unwrap(); + std::fs::set_permissions( + retained.join(".claude"), + std::fs::Permissions::from_mode(0o751), + ) + .unwrap(); + let metadata = std::fs::metadata(retained.join(".claude")).unwrap(); + let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + for path in [&history, &auth] { + std::fs::File::open(path) + .unwrap() + .set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + } + let mut current = context.clone(); + current.config.clear(); + let next_generation = format!("grant-{}", uuid::Uuid::new_v4()); + current.prepared_inputs.as_mut().unwrap().generation_id = next_generation.clone(); + current.mcp_capability.as_mut().unwrap().grant_id = next_generation.clone(); + seed_retained_ordinary_candidate(&generation,&stage,&retained,&home,¤t,identity).expect("retained seed must reconstruct frozen ordinary inputs and actual directory metadata without reading native payloads"); + let seeded = std::fs::metadata(home.join(".claude")).unwrap(); + assert_eq!( + (seeded.uid(), seeded.gid(), seeded.mode() & 0o777), + (metadata.uid(), metadata.gid(), metadata.mode() & 0o777) + ); + let output = std::process::Command::new("node") + .arg(home.join(".freshell/provider-roots/unit-0/consumer.cjs")) + .current_dir(&workspace) + .env_clear() + .env("PATH", "/usr/local/bin:/usr/bin:/bin") + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"frozen-resource:installed-dependency"); + crate::refresh_provider_config_references( + ¤t, "claude", &workspace, &stage, &home, identity.0, identity.1, + ) + .unwrap(); + assert!(!home.join(".claude/old-settings.json").exists()); + assert!(home.join(".claude").is_dir(),"native siblings in retained HOME keep the category parent present after stale ordinary removal"); + assert!(!home.join(".claude/projects").exists()); + assert!(!home.join(".claude/.credentials.json").exists()); + for path in [&history, &auth] { + assert_eq!( + std::fs::metadata(path).unwrap().accessed().unwrap(), + old_access, + "native payload must remain unread" + ); + } + assert_eq!( + std::fs::read(&history).unwrap(), + b"PRIVATE_NATIVE_HISTORY_SENTINEL" + ); + assert_eq!( + std::fs::read(&auth).unwrap(), + b"PRIVATE_NATIVE_CREDENTIAL_SENTINEL" + ); + let unchanged = std::fs::metadata(retained.join(".claude")).unwrap(); + assert_eq!( + (unchanged.uid(), unchanged.gid(), unchanged.mode() & 0o777), + (metadata.uid(), metadata.gid(), metadata.mode() & 0o777) + ); + let damaged = owned.path().join("damaged-candidate"); + std::fs::create_dir(&damaged).unwrap(); + std::fs::set_permissions(&damaged, std::fs::Permissions::from_mode(0o700)).unwrap(); + let ledger = retained.join(crate::PROVIDER_PROJECTION_MANIFEST); + let mut entries: std::collections::BTreeSet = + serde_json::from_slice(&std::fs::read(&ledger).unwrap()).unwrap(); + entries.insert(".claude/.credentials.json".into()); + std::fs::write(&ledger, serde_json::to_vec(&entries).unwrap()).unwrap(); + assert!( + seed_retained_ordinary_candidate( + &generation, + &stage, + &retained, + &damaged, + ¤t, + identity + ) + .is_err(), + "damaged ordinary projection metadata must not recreate a native credential path" + ); + assert!(!damaged.join(".claude/.credentials.json").exists()); + + let without_user = owned.path().join("retained-without-user-sources"); + let absent_candidate = owned.path().join("candidate-without-user-sources"); + for path in [&without_user, &absent_candidate] { + std::fs::create_dir(path).unwrap(); + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).unwrap(); + } + std::fs::create_dir_all(without_user.join(".freshell/provider-roots/unit-0")).unwrap(); + crate::copy_provider_config_references_with_ownership( + ¤t, + "claude", + &workspace, + &stage, + &without_user, + &without_user, + identity, + ) + .unwrap(); + std::fs::write( + stage.join(".freshell/prepared-context.json"), + freshell_runtime_protocol::bounded_json_bytes( + ¤t, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .unwrap(), + ) + .unwrap(); + seed_retained_ordinary_candidate( + &next_generation, + &stage, + &without_user, + &absent_candidate, + ¤t, + identity, + ) + .expect("known-absent optional user sources must remain absent"); + assert!(!absent_candidate.join(".claude").exists()); + } + #[test] fn claude_candidate_projection_runs_selected_dependency_and_frozen_config_consumer() { use freshell_platform::claude_inputs::{ From f26ee7acf321dd14eb8c011a3f66ce37da9cbfd0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 11:42:07 -0700 Subject: [PATCH 146/493] Qualify retained Claude candidates with their owned frozen inputs and HOME --- .../src/backend/claude_source_observation.rs | 274 +++++++++++++++--- .../claude_source_observation_tests.rs | 133 +++++++-- 2 files changed, 340 insertions(+), 67 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index dfe75e06f..7c38340ec 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -76,18 +76,32 @@ impl CandidateInputView { stage: &Path, exchange: &native_auth::ExchangeDirectory, mounts: &docker::TerminalMounts, + retained: bool, ) -> Result<(Self, Vec), BackendError> { use std::os::unix::fs::PermissionsExt; let blocked = || fail("Claude candidate ordinary input view unavailable"); - let root = exchange.0.join("ordinary-stage"); + let root = exchange.0.join(if retained { + "retained-stage" + } else { + "ordinary-stage" + }); + let namespace = if retained { + "/run/freshell-retained-inputs" + } else { + "/run/freshell-candidate-inputs" + }; std::fs::create_dir(&root).map_err(|_| blocked())?; let view = Self(root, false); let mut selected = BTreeMap::::new(); - for relative in [ - ".freshell/claude-candidate-binding.json", - ".freshell/prepared-context.json", - ".freshell/prepared-result.json", - ] { + for relative in std::iter::once(".freshell/prepared-context.json").chain( + (!retained) + .then_some([ + ".freshell/claude-candidate-binding.json", + ".freshell/prepared-result.json", + ]) + .into_iter() + .flatten(), + ) { selected.insert(relative.into(), (stage.join(relative), false)); } for file in mounts @@ -109,9 +123,8 @@ impl CandidateInputView { ), ); } - let mut bindings = vec![ - json!({"Type":"bind","Source":view.0,"Target":"/run/freshell-candidate-inputs","ReadOnly":true}), - ]; + let mut bindings = + vec![json!({"Type":"bind","Source":view.0,"Target":namespace,"ReadOnly":true})]; for (relative, (source, directory)) in selected { if !relative .components() @@ -145,7 +158,7 @@ impl CandidateInputView { } ancestor = ancestor.parent().ok_or_else(blocked)?; } - bindings.push(json!({"Type":"bind","Source":source,"Target":Path::new("/run/freshell-candidate-inputs").join(relative),"ReadOnly":true})); + bindings.push(json!({"Type":"bind","Source":source,"Target":Path::new(namespace).join(relative),"ReadOnly":true})); } Ok((view, bindings)) } @@ -176,6 +189,53 @@ impl Drop for CandidateInputView { } impl DockerEngineBackend { + async fn retained_claude_volume( + &self, + handle: &OwnedRuntimeHandle, + request: &ObserveClaudeSourcesRequest, + daemon: &DockerDaemonId, + ) -> Result { + let provider = handle + .terminal() + .map(|spec| spec.mode.as_str()) + .or_else(|| handle.fresh_agent().map(|spec| spec.provider.as_str())); + if handle.soul_id() != &request.soul_id + || provider != Some(request.provider.as_str()) + || handle.daemon_id() != daemon + || !matches!( + self.inspect(handle).await?.state, + BackendRuntimeState::Created + | BackendRuntimeState::Exited + | BackendRuntimeState::Dead + | BackendRuntimeState::Missing + ) + { + return Err(fail( + "Claude candidate requires its exact quiescent retained owner", + )); + } + let response = self + .request_bounded( + "GET", + &format!("{DOCKER_API}/volumes/{}", handle.provider_volume_name()), + None, + 64 * 1024, + ) + .await?; + let volume: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("Claude retained HOME metadata invalid"))?; + if response.status != 200 + || volume["Name"] != handle.provider_volume_name() + || volume["Driver"] != "local" + || !volume["Mountpoint"].is_string() + { + return Err(fail( + "Claude retained HOME is unavailable; preserve native stores and repair the source", + )); + } + Ok(volume) + } + async fn read_claude_candidate_helper( &self, owner: &ClaudeSourceOwner, @@ -183,6 +243,7 @@ impl DockerEngineBackend { root: &Path, test_run_id: &str, budget: Duration, + prior_stage: Option<&Path>, ) -> Result { use freshell_runtime_protocol::{bounded_json_bytes, MAX_CONTROL_FRAME_BYTES}; let deadline = tokio::time::Instant::now() + budget; @@ -193,6 +254,7 @@ impl DockerEngineBackend { let mut attempted = false; let mut directory = None::; let mut input_view = None::; + let mut retained_view = None::; let operation = async { let projection = qualify_candidate_stage(root, owner, request)?; let raw = bounded_json_bytes(&projection, MAX_CONTROL_FRAME_BYTES) @@ -203,25 +265,47 @@ impl DockerEngineBackend { "Claude candidate reader requires the qualified rootless daemon", )); } - let (reservation, observation) = match owner { - ClaudeSourceOwner::Initial { reservation, observation } => (reservation, observation), - // Retained candidates need ordinary metadata from their exact - // prior HOME. An empty scratch HOME is not equivalent to it. - ClaudeSourceOwner::Retained { .. } => return Err(fail("Claude retained candidate ordinary topology unavailable; retain inputs and reprepare")), + let mut retained_volume = None; + let (installation, image, binary_path, reservation_id) = match owner { + ClaudeSourceOwner::Initial { + reservation, + observation, + } => { + if reservation.soul_id != request.soul_id + || reservation.provider != request.provider + || self.observe_initial_volume(reservation).await? != *observation + || !matches!(observation, freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { daemon_id } if daemon_id == &daemon) + { + return Err(fail("Claude candidate creation absence changed")); + } + ( + reservation.installation_id.as_str(), + reservation.image_ref.as_str(), + reservation.binary.as_path(), + Some(reservation.id.as_str()), + ) + } + ClaudeSourceOwner::Retained { handle, .. } => { + retained_volume = Some( + self.retained_claude_volume(handle, request, &daemon) + .await?, + ); + ( + handle.installation_id().as_str(), + handle.image_ref(), + handle.host_binary_path(), + None, + ) + } }; - if reservation.soul_id != request.soul_id - || reservation.provider != request.provider - || self.observe_initial_volume(reservation).await? != *observation - || !matches!(observation, freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { daemon_id } if daemon_id == &daemon) - { - return Err(fail("Claude candidate creation absence changed")); - } - let binary = canonical_regular_file(&reservation.binary) + let binary = canonical_regular_file(binary_path) .map_err(|_| fail("Claude candidate reader artifact unavailable"))?; let digest = crate::registry::binary_digest(&binary) .map_err(|_| fail("Claude candidate reader artifact unavailable"))?; - if digest != reservation.binary_sha256 { - return Err(fail("Claude candidate reserved reader artifact changed")); + if let ClaudeSourceOwner::Initial { reservation, .. } = owner { + if digest != reservation.binary_sha256 { + return Err(fail("Claude candidate reserved reader artifact changed")); + } } let workspace = canonical_dir(Path::new(&request.workspace_path)) .map_err(|_| fail("Claude candidate workspace unavailable"))?; @@ -240,12 +324,12 @@ impl DockerEngineBackend { &projection.provider_launch_context, ) .map_err(|_| fail("Claude candidate ordinary mounts unavailable"))?; - let labels = json!({"com.freshell.installation-id":reservation.installation_id, + let labels = json!({"com.freshell.installation-id":installation, "com.freshell.soul-id":request.soul_id,"com.freshell.runtime-test-run-id":test_run_id, "com.freshell.source-request-id":request.observation_request.request_id, "com.freshell.source-binary-sha256":digest, "com.freshell.input-generation":projection.publication.generation_id, - "com.freshell.source-reservation-id":reservation.id, + "com.freshell.source-reservation-id":reservation_id, "com.freshell.claude-candidate":"ordinary"}); scratch = Some(CandidateScratch { name: format!( @@ -260,7 +344,7 @@ impl DockerEngineBackend { self.create_claude_candidate_scratch(backing).await?; self.run_provider_mountpoint_helper( provider_mountpoints::MountpointHelperTarget { - image: &reservation.image_ref, + image, volume: &backing.name, binary: &binary, daemon: &daemon, @@ -281,7 +365,7 @@ impl DockerEngineBackend { .map_err(|_| fail("Claude candidate exchange unavailable"))?; exchange_directory.mode(&socket, 0o660)?; let (view, view_bindings) = - CandidateInputView::create(root, exchange_directory, &mounts)?; + CandidateInputView::create(root, exchange_directory, &mounts, false)?; input_view = Some(view); let mut descriptors = vec![ json!({"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}), @@ -290,6 +374,20 @@ impl DockerEngineBackend { json!({"Type":"volume","Source":backing.name,"Target":"/home/freshell/provider","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}), ]; descriptors.extend(view_bindings); + if let ClaudeSourceOwner::Retained { handle, .. } = owner { + let stage = prior_stage.ok_or_else(||fail("Claude retained frozen ordinary stage unavailable; retain inputs and reprepare"))?; + let ordinary = qualify_retained_claude_stage( + stage, + handle, + &workspace, + mounts.git_common_dir.as_deref(), + )?; + let (view, bindings) = + CandidateInputView::create(stage, exchange_directory, &ordinary, true)?; + retained_view = Some(view); + descriptors.extend(bindings); + descriptors.push(json!({"Type":"volume","Source":handle.provider_volume_name(),"Target":"/run/freshell-retained-home","ReadOnly":true,"VolumeOptions":{"NoCopy":true}})); + } if let Some(common) = mounts .git_common_dir .as_ref() @@ -299,9 +397,9 @@ impl DockerEngineBackend { .push(json!({"Type":"bind","Source":common,"Target":common,"ReadOnly":true})); } descriptors.extend(prepared_inputs::mount_descriptors(&mounts, &backing.name)?); - let body = json!({"Image":reservation.image_ref,"User":"0:0","Tty":false,"WorkingDir":cwd, + let body = json!({"Image":image,"User":"0:0","Tty":false,"WorkingDir":cwd, "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["project-claude-candidate-private"],"Labels":labels, - "Env":self.private_helper_environment(&reservation.image_ref,&["HOME=/home/freshell/provider"]).await?, + "Env":self.private_helper_environment(image,&["HOME=/home/freshell/provider"]).await?, "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"CapAdd":["CHOWN","SETGID","SETUID"], "SecurityOpt":["no-new-privileges"],"Memory":256*1024*1024,"MemorySwap":256*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, "LogConfig":{"Type":"none","Config":{}},"Tmpfs":{},"Mounts":descriptors}}); @@ -361,7 +459,14 @@ impl DockerEngineBackend { if volume_reply.status != 200 { return Err(fail("Claude candidate ordinary backing disappeared")); } - verify_candidate_mount_reports(&actual, &body, backing, &volume, &mounts)?; + verify_candidate_mount_reports( + &actual, + &body, + backing, + &volume, + &mounts, + retained_volume.as_ref(), + )?; let supervision = async { let started = self .request_bounded( @@ -410,10 +515,37 @@ impl DockerEngineBackend { || crate::registry::binary_digest(&binary) .map_err(|_| fail("Claude candidate reader artifact unavailable"))? != digest - || self.observe_initial_volume(reservation).await? != *observation { return Err(fail("Claude candidate actual namespace changed")); } + match owner { + ClaudeSourceOwner::Initial { + reservation, + observation, + } => { + if self.observe_initial_volume(reservation).await? != *observation { + return Err(fail("Claude candidate creation absence changed")); + } + } + ClaudeSourceOwner::Retained { handle, .. } => { + if Some( + self.retained_claude_volume(handle, request, &daemon) + .await?, + ) != retained_volume + { + return Err(fail( + "Claude retained HOME metadata changed during candidate projection", + )); + } + qualify_retained_claude_stage( + prior_stage + .ok_or_else(|| fail("Claude retained frozen stage unavailable"))?, + handle, + &workspace, + mounts.git_common_dir.as_deref(), + )?; + } + } qualify_candidate_stage(root, owner, request)?; Ok(reply) }; @@ -443,6 +575,9 @@ impl DockerEngineBackend { if let Some(view) = input_view.as_mut() { view.cleanup()?; } + if let Some(view) = retained_view.as_mut() { + view.cleanup()?; + } Ok::<_, BackendError>(()) }; tokio::time::timeout(Duration::from_secs(5), cleanup) @@ -587,6 +722,36 @@ impl DockerEngineBackend { } } +fn qualify_retained_claude_stage( + stage: &Path, + handle: &OwnedRuntimeHandle, + workspace: &Path, + common: Option<&Path>, +) -> Result { + use freshell_runtime_protocol::{bounded_json_bytes, MAX_CONTROL_FRAME_BYTES}; + let blocked = || { + fail("Claude retained frozen ordinary stage changed or unavailable; retain inputs and reprepare") + }; + let context = retained_claude_context(handle)?; + let expected = bounded_json_bytes(context, MAX_CONTROL_FRAME_BYTES).map_err(|_| blocked())?; + if read_candidate_metadata( + stage, + ".freshell/prepared-context.json", + MAX_CONTROL_FRAME_BYTES, + )? != expected + { + return Err(blocked()); + } + let inputs = context.prepared_inputs.as_ref().ok_or_else(blocked)?; + let manifest = + read_candidate_metadata(stage, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + if format!("{:x}", Sha256::digest(&manifest)) != inputs.manifest.sha256 { + return Err(blocked()); + } + docker::ordinary_claude_candidate_mounts(workspace, common, stage, context) + .map_err(|_| blocked()) +} + fn qualify_candidate_stage( root: &Path, owner: &ClaudeSourceOwner, @@ -893,6 +1058,7 @@ fn verify_candidate_mount_reports( backing: &CandidateScratch, volume: &Value, mounts: &docker::TerminalMounts, + retained: Option<&Value>, ) -> Result<(), BackendError> { let blocked = || fail("Claude candidate reported mount topology changed"); if !backing.owns(volume) || !volume["Mountpoint"].is_string() { @@ -914,11 +1080,22 @@ fn verify_candidate_mount_reports( return Err(blocked()); } let report = reports[0]; + let expected_volume = if mount["Type"] == "volume" { + if mount["Source"] == backing.name { + Some(volume) + } else { + retained.filter(|volume| volume["Name"] == mount["Source"]) + } + } else { + None + }; if report["Type"] != mount["Type"] || report["RW"].as_bool() != mount["ReadOnly"].as_bool().map(|readonly| !readonly) || (mount["Type"] == "bind" && report["Source"] != mount["Source"]) || (mount["Type"] == "volume" - && (report["Name"] != backing.name || report["Source"] != volume["Mountpoint"])) + && !expected_volume.is_some_and(|volume| { + report["Name"] == volume["Name"] && report["Source"] == volume["Mountpoint"] + })) { return Err(blocked()); } @@ -1001,6 +1178,16 @@ impl DockerEngineBackend { request .validate() .map_err(|_| fail("Claude source request invalid"))?; + let prior_stage = match owner { + ClaudeSourceOwner::Initial { .. } => None, + ClaudeSourceOwner::Retained { handle, .. } => { + let context = retained_claude_context(handle)?; + let grant = context.mcp_capability.as_ref().unwrap(); + Some(docker::staged_provider_root_for_grant(&grant.grant_id).map_err(|_| { + fail("Claude retained frozen ordinary stage unavailable; retain inputs and reprepare") + })?) + } + }; if request.candidate.is_some() { let grant = request .candidate @@ -1013,19 +1200,16 @@ impl DockerEngineBackend { fail("Claude candidate stage unavailable; retain inputs and reprepare") })?; return self - .read_claude_candidate_helper(owner, request, &root, test_run_id, budget) + .read_claude_candidate_helper( + owner, + request, + &root, + test_run_id, + budget, + prior_stage.as_deref(), + ) .await; } - let prior_stage = match owner { - ClaudeSourceOwner::Initial { .. } => None, - ClaudeSourceOwner::Retained { handle, .. } => { - let context = retained_claude_context(handle)?; - let grant = context.mcp_capability.as_ref().unwrap(); - Some(docker::staged_provider_root_for_grant(&grant.grant_id).map_err(|_| { - fail("Claude retained frozen ordinary stage unavailable; retain inputs and reprepare") - })?) - } - }; self.read_claude_baseline_helper( owner, request, diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 2c83b7177..a3cc8f175 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -493,9 +493,7 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() let qualified = qualify_candidate_stage(&stage, &owner, &request); assert_eq!(qualified.is_ok(),mode=="valid" || retained,"{flavor}/{runtime:?}/{mode}: physical issuer and current owner must qualify before consumer selection: {:?}",qualified.as_ref().err()); if let Ok(projection) = qualified { - if !retained { - qualify_candidate_helper_transport(&owner, &request, &stage).await; - } + qualify_candidate_helper_transport(&owner, &request, &stage).await; let selected = &projection .provider_launch_context .prepared_inputs @@ -997,7 +995,7 @@ async fn qualify_candidate_helper_transport( request: &ObserveClaudeSourcesRequest, stage: &Path, ) { - for mode in [ + let mut modes = vec![ "success", "docker-empty-tmpfs", "unexpected-tmpfs", @@ -1007,15 +1005,55 @@ async fn qualify_candidate_helper_transport( "volume-still-present", "wrong-mount-report", "wrong-resource-subpath", - ] { + ]; + if matches!(owner, ClaudeSourceOwner::Retained { .. }) { + modes.extend([ + "writable-retained-home", + "wrong-retained-volume", + "changed-prior-context", + ]); + } + for mode in modes { let owned = tempfile::tempdir().unwrap(); let daemon = match owner { ClaudeSourceOwner::Initial { observation: InitialAuthVolumeObservation::Absent { daemon_id }, .. } => daemon_id.clone(), - _ => panic!("this fixture qualifies an initial candidate"), + ClaudeSourceOwner::Retained { handle, .. } => handle.daemon_id().clone(), + }; + let retained_name = match owner { + ClaudeSourceOwner::Retained { handle, .. } => { + Some(handle.provider_volume_name().to_string()) + } + _ => None, }; + let prior_stage = owned.path().join("prior-stage"); + if let ClaudeSourceOwner::Retained { handle, .. } = owner { + use freshell_platform::provider_environment::{copy_environment, CopySelection}; + let context = retained_claude_context(handle).unwrap(); + let inputs = context.prepared_inputs.as_ref().unwrap(); + let selections = std::iter::once(CopySelection { + source: stage.join(&inputs.manifest.relative_path), + destination_relative: inputs.manifest.relative_path.clone().into(), + }) + .chain(inputs.resource_mounts.iter().map(|unit| CopySelection { + source: stage.join(&unit.stage_relative_path), + destination_relative: unit.stage_relative_path.clone().into(), + })) + .collect::>(); + std::fs::create_dir(&prior_stage).unwrap(); + copy_environment(&selections, &prior_stage, &[], Default::default()).unwrap(); + std::fs::write( + prior_stage.join(".freshell/prepared-context.json"), + freshell_runtime_protocol::bounded_json_bytes( + context, + freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + ) + .unwrap(), + ) + .unwrap(); + } let calls = Arc::new(Mutex::new(Vec::<(String, Value)>::new())); let containers = Arc::new(Mutex::new(BTreeMap::::new())); let volume = Arc::new(Mutex::new(None::)); @@ -1026,7 +1064,9 @@ async fn qualify_candidate_helper_transport( "freshell-claude-candidate-{}", request.observation_request.request_id ); + let retained_name_for_server = retained_name.clone(); let server = tokio::spawn(async move { + let retained_name = retained_name_for_server; let mut jobs = JoinSet::new(); let private_jobs = Arc::new(Mutex::new(Vec::>::new())); loop { @@ -1034,7 +1074,7 @@ async fn qualify_candidate_helper_transport( _=&mut stopped=>break, accepted=listener.accept()=>{ let (mut stream,_)=accepted.unwrap(); - let (seen,bodies,backing,daemon,private_jobs,expected_scratch)=(seen.clone(),bodies.clone(),backing.clone(),daemon.clone(),private_jobs.clone(),expected_scratch.clone()); + let (seen,bodies,backing,daemon,private_jobs,expected_scratch,retained_name)=(seen.clone(),bodies.clone(),backing.clone(),daemon.clone(),private_jobs.clone(),expected_scratch.clone(),retained_name.clone()); jobs.spawn(async move { let mut header=Vec::new();let mut byte=[0]; while !header.ends_with(b"\r\n\r\n") {stream.read_exact(&mut byte).await.unwrap();header.push(byte[0]);assert!(header.len()<=32*1024);} @@ -1046,13 +1086,13 @@ async fn qualify_candidate_helper_transport( let (status,value)=if line.contains("/info "){(200,json!({"ID":daemon,"SecurityOptions":["name=rootless"]}))} else if line.contains("/images/"){(200,json!({"Config":{"Env":["PATH=/usr/bin","ANTHROPIC_API_KEY=PRIVATE_IMAGE_CREDENTIAL_SENTINEL"]}}))} else if line.starts_with("POST") && line.contains("/volumes/create "){assert_eq!(body["Name"],expected_scratch,"private candidate request must identify its exact scratch backing before creation");let actual=json!({"Name":body["Name"],"Driver":"local","Options":{},"Labels":body["Labels"],"Mountpoint":"/synthetic/ordinary-volume"});*backing.lock().unwrap()=Some(actual.clone());(201,actual)} - else if line.starts_with("GET") && line.contains("/volumes/"){match backing.lock().unwrap().clone(){Some(v) if line.contains(v["Name"].as_str().unwrap())=>(200,v),_=>(404,json!({}))}} + else if line.starts_with("GET") && line.contains("/volumes/"){if retained_name.as_ref().is_some_and(|name|line.contains(name)) {(200,json!({"Name":retained_name,"Driver":"local","Options":{},"Mountpoint":"/synthetic/retained-home"}))} else {match backing.lock().unwrap().clone(){Some(v) if line.contains(v["Name"].as_str().unwrap())=>(200,v),_=>(404,json!({}))}}} else if line.starts_with("DELETE") && line.contains("/volumes/"){assert!(bodies.lock().unwrap().is_empty(),"candidate helpers must be gone before volume deletion");if mode!="volume-still-present" {*backing.lock().unwrap()=None;}(204,json!({}))} else if line.contains("/containers/create?"){ if body["Cmd"]==json!(["project-claude-candidate-private"]) {let root=body["HostConfig"]["Mounts"].as_array().unwrap().iter().find(|m|m["Target"]=="/run/freshell-candidate-inputs").unwrap()["Source"].as_str().unwrap();assert!(!Path::new(root).join(".freshell/mcp-capability.json").exists(),"ordinary candidate input root must not expose capability credentials");} let name=line.split("name=").nth(1).unwrap().split_whitespace().next().unwrap().to_string();let id=if body["Cmd"]==json!(["prepare-provider-mountpoints"]) {"a".repeat(64)} else {"b".repeat(64)};bodies.lock().unwrap().insert(name,json!({"Id":id,"Config":body,"HostConfig":body["HostConfig"],"State":{"Running":false}}));(if mode=="lost-projection-ack" && body["Cmd"]==json!(["project-claude-candidate-private"]) {503}else{201},json!({"Id":id}))} else if line.starts_with("GET") && line.contains("/containers/"){let key=line.split("/containers/").nth(1).unwrap().split('/').next().unwrap();let found=bodies.lock().unwrap().iter().find(|(name,v)|*name==key || v["Id"]==key).map(|(_,v)|v.clone());match found {Some(mut v)=>{ - let reports=v["HostConfig"]["Mounts"].as_array().unwrap().iter().map(|m|json!({"Type":m["Type"],"Destination":m["Target"],"RW":!m["ReadOnly"].as_bool().unwrap(),"Source":if m["Type"]=="volume" {json!("/synthetic/ordinary-volume")} else {m["Source"].clone()},"Name":if m["Type"]=="volume" {m["Source"].clone()} else {Value::Null}})).collect::>(); + let reports=v["HostConfig"]["Mounts"].as_array().unwrap().iter().map(|m|json!({"Type":m["Type"],"Destination":m["Target"],"RW":!m["ReadOnly"].as_bool().unwrap(),"Source":if m["Type"]=="volume" {json!(if m["Target"]=="/run/freshell-retained-home" {"/synthetic/retained-home"} else {"/synthetic/ordinary-volume"})} else {m["Source"].clone()},"Name":if m["Type"]=="volume" {m["Source"].clone()} else {Value::Null}})).collect::>(); v["Mounts"]=json!(reports); if mode=="docker-empty-tmpfs" && v["Config"]["Cmd"]==json!(["project-claude-candidate-private"]) {v["HostConfig"]["Tmpfs"]=Value::Null;} if key.len()==64 && v["Config"]["Cmd"]==json!(["project-claude-candidate-private"]) { @@ -1060,6 +1100,8 @@ async fn qualify_candidate_helper_transport( if mode=="wrong-caps" {v["HostConfig"]["CapAdd"]=json!(["SYS_ADMIN"]);} if mode=="wrong-mount-report" {v["Mounts"][0]["RW"]=json!(true);} if mode=="wrong-resource-subpath" {let mount=v["HostConfig"]["Mounts"].as_array_mut().unwrap().iter_mut().find(|m|m["VolumeOptions"]["Subpath"].is_string()).unwrap();mount["VolumeOptions"]["Subpath"]=json!("foreign/ordinary");} + if mode=="writable-retained-home" {v["Mounts"].as_array_mut().unwrap().iter_mut().find(|m|m["Destination"]=="/run/freshell-retained-home").unwrap()["RW"]=json!(true);} + if mode=="wrong-retained-volume" {v["Mounts"].as_array_mut().unwrap().iter_mut().find(|m|m["Destination"]=="/run/freshell-retained-home").unwrap()["Name"]=json!("foreign-provider-home");} } (200,v)},None=>(404,json!({}))}} else if line.starts_with("DELETE") && line.contains("/containers/"){let key=line.split("/containers/").nth(1).unwrap().split('?').next().unwrap();let name=bodies.lock().unwrap().iter().find(|(name,v)|*name==key || v["Id"]==key).map(|(name,_)|name.clone());if let Some(name)=name{bodies.lock().unwrap().remove(&name);}(204,json!({}))} else if line.contains("/start "){let id=line.split("/containers/").nth(1).unwrap().split('/').next().unwrap();let v=bodies.lock().unwrap().values().find(|v|v["Id"]==id).unwrap().clone();private_jobs.lock().unwrap().push(tokio::spawn(async move { @@ -1070,7 +1112,30 @@ async fn qualify_candidate_helper_transport( if !is_mountpoint && mode=="projection-timeout" {tokio::time::sleep(Duration::from_secs(10)).await;return;} let length=channel.read_u32().await.unwrap() as usize;assert!(length<=1024*1024);let mut raw=vec![0;length];channel.read_exact(&mut raw).await.unwrap(); if is_mountpoint {let _:Vec=serde_json::from_slice(&raw).unwrap();channel.write_all(b"ok").await.unwrap();} - else {let p:freshell_runtime_protocol::ClaudeCandidateProjectionRequest=serde_json::from_slice(&raw).unwrap();p.validate().unwrap();let source=p.source_request;let canonical=freshell_runtime_protocol::bounded_json_bytes(&source,MAX_REQUEST_BYTES).unwrap();let reply=ObservationReply {request_sha256:format!("{:x}",Sha256::digest(canonical)),observation:NativeObservation {version:1,native_source_runtime:source.native_source_runtime,setting_sources:source.setting_sources,uid:65534,euid:65534,gid:0,egid:0,supplementary_groups:vec![],cwd:source.cwd,home:Some(source.home),roots:None,local_settings:vec![],local_decision:LocalDecision::Disabled,root_uid:None,git_uid:None,claude_uid:None}};let raw=serde_json::to_vec(&reply).unwrap();channel.write_u32(raw.len() as u32).await.unwrap();channel.write_all(&raw).await.unwrap();channel.shutdown().await.unwrap();} + else { + let p:freshell_runtime_protocol::ClaudeCandidateProjectionRequest=serde_json::from_slice(&raw).unwrap(); + p.validate().unwrap(); + if let Some(generation) = p.publication.prior_generation_id.as_deref() { + let mounts = v["HostConfig"]["Mounts"].as_array().unwrap(); + let context = mounts.iter().find(|m|m["Target"]=="/run/freshell-retained-inputs/.freshell/prepared-context.json").unwrap()["Source"].as_str().unwrap(); + let prior:freshell_runtime_protocol::ProviderLaunchContext = serde_json::from_slice(&std::fs::read(context).unwrap()).unwrap(); + assert_eq!(prior.prepared_inputs.as_ref().unwrap().generation_id,generation); + let unit = mounts.iter().find(|m|m["Target"]=="/run/freshell-retained-inputs/.freshell/resource-units/unit-0").unwrap()["Source"].as_str().unwrap(); + let mut consumer = std::process::Command::new("node").arg(Path::new(unit).join("probe/consumer.cjs")).current_dir(&p.workspace_path).env_clear().env("PATH","/usr/local/bin:/usr/bin:/bin").stdout(std::process::Stdio::piped()).stderr(std::process::Stdio::null()).spawn().unwrap(); + let started = std::time::Instant::now(); + let completed = loop { + if let Some(status) = consumer.try_wait().unwrap() { break status; } + if started.elapsed() >= Duration::from_secs(3) { consumer.kill().unwrap();consumer.wait().unwrap();panic!("owned frozen consumer timed out after finite joined cleanup"); } + std::thread::sleep(Duration::from_millis(5)); + }; + assert!(completed.success()); + let mut output = Vec::new(); + std::io::Read::read_to_end(&mut consumer.stdout.take().unwrap(),&mut output).unwrap(); + assert_eq!(output,format!("qualified-frozen-dependency:{}",p.workspace_path).as_bytes()); + if mode=="changed-prior-context" { std::fs::write(context,b"{}").unwrap(); } + } + let source=p.source_request;let canonical=freshell_runtime_protocol::bounded_json_bytes(&source,MAX_REQUEST_BYTES).unwrap();let reply=ObservationReply {request_sha256:format!("{:x}",Sha256::digest(canonical)),observation:NativeObservation {version:1,native_source_runtime:source.native_source_runtime,setting_sources:source.setting_sources,uid:65534,euid:65534,gid:0,egid:0,supplementary_groups:vec![],cwd:source.cwd,home:Some(source.home),roots:None,local_settings:vec![],local_decision:LocalDecision::Disabled,root_uid:None,git_uid:None,claude_uid:None}};let raw=serde_json::to_vec(&reply).unwrap();channel.write_u32(raw.len() as u32).await.unwrap();channel.write_all(&raw).await.unwrap();channel.shutdown().await.unwrap(); + } }));(204,json!({}))} else if line.contains("/wait?"){(200,json!({"StatusCode":0}))}else{(500,json!({}))}; let response=value.to_string();let _=stream.write_all(format!("HTTP/1.1 {status} fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{response}",response.len()).as_bytes()).await; @@ -1082,10 +1147,13 @@ async fn qualify_candidate_helper_transport( jobs.shutdown().await; let workers = std::mem::take(&mut *private_jobs.lock().unwrap()); for worker in workers { - if !worker.is_finished() { + if mode == "projection-timeout" && !worker.is_finished() { worker.abort(); } - match worker.await { + match tokio::time::timeout(Duration::from_secs(5), worker) + .await + .expect("owned private worker must finish and join") + { Ok(()) => {} Err(e) if e.is_cancelled() => {} Err(e) => panic!("private fixture worker failed: {e}"), @@ -1100,6 +1168,7 @@ async fn qualify_candidate_helper_transport( stage, "synthetic-source-run", Duration::from_millis(250), + retained_name.as_ref().map(|_| prior_stage.as_path()), ) .await; let _ = stop.send(()); @@ -1123,7 +1192,12 @@ async fn qualify_candidate_helper_transport( let seen = calls.lock().unwrap(); if matches!( mode, - "unexpected-tmpfs" | "wrong-caps" | "wrong-mount-report" | "wrong-resource-subpath" + "unexpected-tmpfs" + | "wrong-caps" + | "wrong-mount-report" + | "wrong-resource-subpath" + | "writable-retained-home" + | "wrong-retained-volume" ) { assert!( !seen @@ -1135,22 +1209,37 @@ async fn qualify_candidate_helper_transport( } for (line, body) in seen.iter() { assert!(!line.contains("/logs")); + if let Some(retained) = retained_name.as_ref() { + assert!( + !line.starts_with("DELETE") || !line.contains(retained), + "candidate cleanup must preserve the exact retained native HOME" + ); + } assert!(!body .to_string() .contains("PRIVATE_IMAGE_CREDENTIAL_SENTINEL")); if body["Cmd"] == json!(["project-claude-candidate-private"]) { - let root = body["HostConfig"]["Mounts"] - .as_array() - .unwrap() - .iter() - .find(|mount| mount["Target"] == "/run/freshell-candidate-inputs") - .unwrap()["Source"] - .as_str() - .unwrap(); - assert!( + for target in [ + "/run/freshell-candidate-inputs", + "/run/freshell-retained-inputs", + ] { + let root = body["HostConfig"]["Mounts"] + .as_array() + .unwrap() + .iter() + .find(|mount| mount["Target"] == target); + let Some(root) = root else { + assert!( + target == "/run/freshell-retained-inputs" && retained_name.is_none() + ); + continue; + }; + let root = root["Source"].as_str().unwrap(); + assert!( !Path::new(root).exists(), "{mode}: owned ordinary mount skeleton must be gone after reader settlement" ); + } } } if let Some((_, body)) = seen From 081f46f8c4244d73ed0e63adbeaaeabdba97a879 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 11:54:19 -0700 Subject: [PATCH 147/493] Use string recovery metadata for retained Claude helper labels --- .../src/backend/claude_source_observation.rs | 8 ++++++-- .../src/backend/claude_source_observation_tests.rs | 6 +++++- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation.rs b/crates/freshell-supervisor/src/backend/claude_source_observation.rs index 7c38340ec..f46efee7f 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation.rs @@ -324,13 +324,17 @@ impl DockerEngineBackend { &projection.provider_launch_context, ) .map_err(|_| fail("Claude candidate ordinary mounts unavailable"))?; - let labels = json!({"com.freshell.installation-id":installation, + let mut labels = json!({"com.freshell.installation-id":installation, "com.freshell.soul-id":request.soul_id,"com.freshell.runtime-test-run-id":test_run_id, "com.freshell.source-request-id":request.observation_request.request_id, "com.freshell.source-binary-sha256":digest, "com.freshell.input-generation":projection.publication.generation_id, - "com.freshell.source-reservation-id":reservation_id, "com.freshell.claude-candidate":"ordinary"}); + if let Some(reservation_id) = reservation_id { + labels["com.freshell.source-reservation-id"] = json!(reservation_id); + } else if let Some(attempt) = &projection.publication.registry_attempt_id { + labels["com.freshell.source-recovery-attempt-id"] = json!(attempt); + } scratch = Some(CandidateScratch { name: format!( "freshell-claude-candidate-{}", diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index a3cc8f175..f205ccb11 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -1085,7 +1085,11 @@ async fn qualify_candidate_helper_transport( seen.lock().unwrap().push((line.clone(),body.clone())); let (status,value)=if line.contains("/info "){(200,json!({"ID":daemon,"SecurityOptions":["name=rootless"]}))} else if line.contains("/images/"){(200,json!({"Config":{"Env":["PATH=/usr/bin","ANTHROPIC_API_KEY=PRIVATE_IMAGE_CREDENTIAL_SENTINEL"]}}))} - else if line.starts_with("POST") && line.contains("/volumes/create "){assert_eq!(body["Name"],expected_scratch,"private candidate request must identify its exact scratch backing before creation");let actual=json!({"Name":body["Name"],"Driver":"local","Options":{},"Labels":body["Labels"],"Mountpoint":"/synthetic/ordinary-volume"});*backing.lock().unwrap()=Some(actual.clone());(201,actual)} + else if line.starts_with("POST") && line.contains("/volumes/create "){ + assert_eq!(body["Name"],expected_scratch,"private candidate request must identify its exact scratch backing before creation"); + if serde_json::from_value::>(body["Labels"].clone()).is_err() { (400,json!({"message":"volume labels require string metadata"})) } + else { let actual=json!({"Name":body["Name"],"Driver":"local","Options":{},"Labels":body["Labels"],"Mountpoint":"/synthetic/ordinary-volume"});*backing.lock().unwrap()=Some(actual.clone());(201,actual) } + } else if line.starts_with("GET") && line.contains("/volumes/"){if retained_name.as_ref().is_some_and(|name|line.contains(name)) {(200,json!({"Name":retained_name,"Driver":"local","Options":{},"Mountpoint":"/synthetic/retained-home"}))} else {match backing.lock().unwrap().clone(){Some(v) if line.contains(v["Name"].as_str().unwrap())=>(200,v),_=>(404,json!({}))}}} else if line.starts_with("DELETE") && line.contains("/volumes/"){assert!(bodies.lock().unwrap().is_empty(),"candidate helpers must be gone before volume deletion");if mode!="volume-still-present" {*backing.lock().unwrap()=None;}(204,json!({}))} else if line.contains("/containers/create?"){ From 8a1681336b5d3e2894f6dc1676791a2105b835a3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 12:37:54 -0700 Subject: [PATCH 148/493] Authorize retained Claude candidates through selected frozen inputs in the closed fixture --- scripts/testing/runtime-test-broker.ts | 48 ++++++++++++------ .../testing/runtime-test-broker.test.ts | 49 ++++++++++++++++++- 2 files changed, 82 insertions(+), 15 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 2f3109de1..40719ee02 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -151,7 +151,7 @@ export type InitialProviderVolumeAuthorization = { daemonId:string } -/** Metadata for the fixed initial Claude candidate, before a workload exists. */ +/** Selected ordinary metadata for the fixed initial or retained Claude candidate. */ export type ClaudeCandidateAuthorization = InitialProviderVolumeAuthorization & { reservationId: string requestId: string @@ -163,6 +163,7 @@ export type ClaudeCandidateAuthorization = InitialProviderVolumeAuthorization & manifestRelativePath: string configurationBinds: { sourceRelativePath: string; effectiveDestination: string }[] resourceMounts: { stageRelativePath: string; providerRelativePath: string; effectiveDestination: string; kind: 'file' | 'directory' }[] + retained?: Pick & { attemptId: string } } function candidateVolumeName(input: ClaudeCandidateAuthorization): string { @@ -172,7 +173,7 @@ function candidateLabels(input: ClaudeCandidateAuthorization, runId: string): Re return { 'com.freshell.installation-id': input.installationId, 'com.freshell.soul-id': input.soulId, 'com.freshell.runtime-test-run-id': runId, 'com.freshell.source-request-id': input.requestId, 'com.freshell.source-binary-sha256': input.binarySha256, 'com.freshell.input-generation': input.generationId, - 'com.freshell.source-reservation-id': input.reservationId, 'com.freshell.claude-candidate': 'ordinary' } + ...(input.retained ? { 'com.freshell.source-recovery-attempt-id': input.retained.attemptId } : { 'com.freshell.source-reservation-id': input.reservationId }), 'com.freshell.claude-candidate': 'ordinary' } } type DockerResponse = { @@ -224,17 +225,29 @@ export class RestrictedDockerBroker { || input.manifestRelativePath !== '.freshell/input-manifest.json' || !Array.isArray(input.configurationBinds) || !Array.isArray(input.resourceMounts) || Buffer.byteLength(JSON.stringify(input)) > 1024 * 1024) throw new Error('initial Claude candidate is outside this fixture request') - const targets = new Set() - const claim = (value: string) => { if (!destination(value) || targets.has(value)) throw new Error('Claude candidate selected target is invalid'); targets.add(value) } - for (const bind of input.configurationBinds) { - if (!unit(bind.sourceRelativePath, '\\.freshell/templates')) throw new Error('Claude candidate template source is invalid') - claim(bind.effectiveDestination) - } - for (const mount of input.resourceMounts) { - if (!['file','directory'].includes(mount.kind) - || !unit(mount.stageRelativePath, '\\.freshell/resource-units', mount.kind === 'file') - || !unit(mount.providerRelativePath, '\\.freshell/provider-roots', mount.kind === 'file')) throw new Error('Claude candidate resource source is invalid') - claim(mount.effectiveDestination) + if (input.retained && (typeof input.retained.attemptId !== 'string' || !input.retained.attemptId.startsWith('recovery-') || !uuid(input.retained.attemptId.slice(9)) + || typeof input.retained.generationId !== 'string' || !input.retained.generationId.startsWith('grant-') || !uuid(input.retained.generationId.slice(6)) + || input.retained.generationId === input.generationId || input.retained.stageRoot !== path.join(path.dirname(input.stageRoot),input.retained.generationId) + || !Array.isArray(input.retained.configurationBinds) || !Array.isArray(input.retained.resourceMounts))) throw new Error('retained Claude candidate is outside its frozen generation') + for (const profile of [input,...(input.retained ? [input.retained] : [])]) { + const targets = new Set() + // Retained logical origins are metadata only: the helper receives their + // frozen stage files, rather than mounts at those original destinations. + const claim = (value: string) => { + const valid = profile === input ? destination(value) : typeof value === 'string' && path.isAbsolute(value) && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) + if (!valid || targets.has(value)) throw new Error('Claude candidate selected target is invalid') + targets.add(value) + } + for (const bind of profile.configurationBinds) { + if (!unit(bind.sourceRelativePath, '\\.freshell/templates')) throw new Error('Claude candidate template source is invalid') + claim(bind.effectiveDestination) + } + for (const mount of profile.resourceMounts) { + if (!['file','directory'].includes(mount.kind) + || !unit(mount.stageRelativePath, '\\.freshell/resource-units', mount.kind === 'file') + || !unit(mount.providerRelativePath, '\\.freshell/provider-roots', mount.kind === 'file')) throw new Error('Claude candidate resource source is invalid') + claim(mount.effectiveDestination) + } } const key = candidateVolumeName(input), previous = this.claudeCandidates.get(key) if (previous && JSON.stringify(previous) !== JSON.stringify(input)) throw new Error('Claude candidate request changed') @@ -509,7 +522,7 @@ export class RestrictedDockerBroker { `${DOCKER_API_PREFIX}/containers/${targetName}/start`, `${DOCKER_API_PREFIX}/containers/${targetName}/wait?condition=not-running`, ].includes(url))) { - await this.forwardAndReply(request, response, body, { destructive: method !== 'GET', containerId: helper.containerId, helperName: name }) + await this.forwardAndReply(request, response, body, { destructive: true, containerId: helper.containerId, helperName: name }) return } } @@ -896,6 +909,13 @@ export class RestrictedDockerBroker { const selected = new Set(['.freshell/claude-candidate-binding.json','.freshell/prepared-context.json','.freshell/prepared-result.json',input.manifestRelativePath, ...input.configurationBinds.map((bind) => bind.sourceRelativePath),...input.resourceMounts.map((mount) => mount.stageRelativePath)]) for (const relative of selected) expected.push({ Type: 'bind', Source: path.join(input.stageRoot, relative), Target: path.join('/run/freshell-candidate-inputs', relative), ReadOnly: true }) + if (input.retained) { + expected.push({ Type:'volume',Source:input.providerVolumeName,Target:'/run/freshell-retained-home',ReadOnly:true,VolumeOptions:{NoCopy:true} }, + { Type:'bind',Source:path.join(exchange.Source,'retained-stage'),Target:'/run/freshell-retained-inputs',ReadOnly:true }) + const selected = new Set(['.freshell/prepared-context.json',input.manifestRelativePath, + ...input.retained.configurationBinds.map((bind) => bind.sourceRelativePath),...input.retained.resourceMounts.map((unit) => unit.stageRelativePath)]) + for (const relative of selected) expected.push({Type:'bind',Source:path.join(input.retained.stageRoot,relative),Target:path.join('/run/freshell-retained-inputs',relative),ReadOnly:true}) + } if (input.gitCommonDir && input.gitCommonDir !== input.workspacePath && !isStrictDescendant(input.gitCommonDir, input.workspacePath)) { expected.push({ Type: 'bind', Source: input.gitCommonDir, Target: input.gitCommonDir, ReadOnly: true }) } diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 76fa66da2..1c6d2e393 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -327,7 +327,7 @@ function candidateLabels(input: ClaudeCandidateAuthorization) { return { 'com.freshell.installation-id': input.installationId, 'com.freshell.soul-id': input.soulId, 'com.freshell.runtime-test-run-id': 'run-one', 'com.freshell.source-request-id': input.requestId, 'com.freshell.source-binary-sha256': input.binarySha256, 'com.freshell.input-generation': input.generationId, - 'com.freshell.source-reservation-id': input.reservationId, 'com.freshell.claude-candidate': 'ordinary' } + ...(input.retained ? { 'com.freshell.source-recovery-attempt-id': input.retained.attemptId } : { 'com.freshell.source-reservation-id': input.reservationId }), 'com.freshell.claude-candidate': 'ordinary' } } function candidateHelperBody(input: ClaudeCandidateAuthorization, role: 'mountpoints' | 'reader'): any { const volume = `freshell-claude-candidate-${input.requestId}` @@ -356,10 +356,57 @@ function candidateHelperBody(input: ClaudeCandidateAuthorization, role: 'mountpo for (const relative of selected) body.HostConfig.Mounts.push({ Type: 'bind', Source: `${input.stageRoot}/${relative}`, Target: `/run/freshell-candidate-inputs/${relative}`, ReadOnly: true }) for (const bind of input.configurationBinds) body.HostConfig.Mounts.push({ Type: 'bind', Source: `${input.stageRoot}/${bind.sourceRelativePath}`, Target: bind.effectiveDestination, ReadOnly: true }) for (const mount of input.resourceMounts) body.HostConfig.Mounts.push({ Type: 'volume', Source: volume, Target: mount.effectiveDestination, ReadOnly: mount.kind === 'file', VolumeOptions: { NoCopy: true, Subpath: mount.providerRelativePath } }) + if (input.retained) { + body.HostConfig.Mounts.push( + { Type: 'volume', Source: input.providerVolumeName, Target: '/run/freshell-retained-home', ReadOnly: true, VolumeOptions: { NoCopy: true } }, + { Type: 'bind', Source: `${exchange}/retained-stage`, Target: '/run/freshell-retained-inputs', ReadOnly: true }, + ) + const selected = new Set(['.freshell/prepared-context.json',input.manifestRelativePath, + ...input.retained.configurationBinds.map((bind) => bind.sourceRelativePath), ...input.retained.resourceMounts.map((unit) => unit.stageRelativePath)]) + for (const relative of selected) body.HostConfig.Mounts.push({ Type: 'bind', Source: `${input.retained.stageRoot}/${relative}`, Target: `/run/freshell-retained-inputs/${relative}`, ReadOnly: true }) + } return body } describe('fixed initial Claude candidate broker', () => { + it('qualifies retained candidates only through selected frozen inputs and read-only owned HOME', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance }) => { + const input = candidateAuthorization(), generationId = 'grant-55555555-5555-4555-8555-555555555555' + input.retained = { generationId, stageRoot: `/tmp/owned/scenarios/claude/control/provider-roots/${generationId}`, + attemptId: 'recovery-66666666-6666-4666-8666-666666666666', configurationBinds: structuredClone(input.configurationBinds), resourceMounts: structuredClone(input.resourceMounts) } + instance.authorizeInitialProviderVolume(sourceAuthorization()) + for (const mutation of ['generation','stage','credential-source']) { + const changed = structuredClone(input) + if (mutation === 'generation') changed.retained!.generationId = input.generationId + if (mutation === 'stage') changed.retained!.stageRoot = `/outside/${generationId}` + if (mutation === 'credential-source') changed.retained!.configurationBinds[0].sourceRelativePath = '.freshell/mcp-capability.json' + expect(() => instance.authorizeClaudeCandidate(changed),mutation).toThrow() + } + instance.authorizeClaudeCandidate(input) + const volume = `freshell-claude-candidate-${input.requestId}` + expect((await request('POST','/v1.47/volumes/create',{Name:volume,Driver:'local',DriverOpts:{},Labels:candidateLabels(input)})).status).toBe(201) + const name = 'freshell-claude-candidate-reader-11111111-1111-4111-8111-111111111111' + const body = candidateHelperBody(input,'reader'), before = forwarded.length + for (const mutation of ['retained-write','retained-name','retained-stage','retained-credential','attempt','extra-native-alias']) { + const changed = structuredClone(body) + if (mutation === 'retained-write') changed.HostConfig.Mounts.find((mount:any) => mount.Target === '/run/freshell-retained-home').ReadOnly = false + if (mutation === 'retained-name') changed.HostConfig.Mounts.find((mount:any) => mount.Target === '/run/freshell-retained-home').Source = 'foreign-provider-home' + if (mutation === 'retained-stage') changed.HostConfig.Mounts.find((mount:any) => mount.Target === '/run/freshell-retained-inputs/.freshell/prepared-context.json').Source = `${input.stageRoot}/.freshell/prepared-context.json` + if (mutation === 'retained-credential') changed.HostConfig.Mounts.push({Type:'bind',Source:`${input.retained.stageRoot}/.freshell/mcp-capability.json`,Target:'/run/freshell-retained-inputs/.freshell/mcp-capability.json',ReadOnly:true}) + if (mutation === 'attempt') changed.Labels['com.freshell.source-recovery-attempt-id'] = 'recovery-77777777-7777-4777-8777-777777777777' + if (mutation === 'extra-native-alias') changed.HostConfig.Mounts.push({Type:'volume',Source:input.providerVolumeName,Target:'/history',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'.claude/projects'}}) + expect((await request('POST',`/v1.47/containers/create?name=${name}`,changed)).status,mutation).toBe(403) + expect(forwarded).toHaveLength(before) + } + expect((await request('POST',`/v1.47/containers/create?name=${name}`,body)).status).toBe(201) + expect((await request('POST',`/v1.47/containers/${helperId}/start`)).status).toBe(204) + expect((await request('POST',`/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200) + expect((await request('DELETE',`/v1.47/containers/${helperId}?force=1`)).status).toBe(204) + expect((await request('GET',`/v1.47/containers/${helperId}/json`)).status).toBe(404) + expect((await request('DELETE',`/v1.47/volumes/${volume}`)).status).toBe(204) + expect(forwarded.some((operation) => operation.method === 'DELETE' && operation.url.includes(input.providerVolumeName))).toBe(false) + }) + }) it('owns only the exact request disk backing and confirms non-force cleanup', async () => { await withHttpBroker(async ({ request, forwarded, broker: instance }) => { const input = candidateAuthorization(), name = `freshell-claude-candidate-${input.requestId}` From 84c3802e50cc8b4f41b2c6785cf5d2ffdfc8a4dc Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 13:02:51 -0700 Subject: [PATCH 149/493] Qualify retained Claude source readers through frozen ordinary aliases --- scripts/testing/runtime-test-broker.ts | 33 +++++++++++++++---- .../testing/runtime-test-broker.test.ts | 32 ++++++++++++++++++ 2 files changed, 59 insertions(+), 6 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 40719ee02..ee54b835f 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -960,17 +960,36 @@ export class RestrictedDockerBroker { || host.NetworkMode !== 'none' || host.ReadonlyRootfs !== true || JSON.stringify(host.CapDrop) !== JSON.stringify(['ALL']) || JSON.stringify(host.SecurityOpt) !== JSON.stringify(['no-new-privileges']) || host.Memory !== 256 * 1024 * 1024 || host.MemorySwap !== 256 * 1024 * 1024 || host.NanoCpus !== 500_000_000 || host.PidsLimit !== 32 - || !equalRecord(host.LogConfig, { Type: 'none', Config: host.LogConfig?.Config }) || Object.keys(host.LogConfig?.Config ?? {}).length - || !equalRecord(host.Tmpfs, { '/home/freshell/provider': 'rw,nosuid,nodev,noexec,size=16m,uid=65534,gid=0,mode=0700' })) return false + || !equalRecord(host.LogConfig, { Type: 'none', Config: host.LogConfig?.Config }) || Object.keys(host.LogConfig?.Config ?? {}).length) return false const mounts = host.Mounts - if (!Array.isArray(mounts) || mounts.length !== 3 - || mounts.some((mount) => !mount || Object.keys(mount).some((key) => !['Type','Source','Target','ReadOnly'].includes(key)))) return false + if (!Array.isArray(mounts) + || mounts.some((mount) => !mount || Object.keys(mount).some((key) => !['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key)))) return false const binary = mounts.find((mount) => mount.Type === 'bind' && mount.Source === owner.hostBinaryPath && mount.Target === '/runtime/freshell-session-host' && mount.ReadOnly === true) const workspace = mounts.find((mount) => mount.Type === 'bind' && mount.Source === mount.Target && this.policy.allowedWorkspaceRoots?.has(mount.Source) && mount.ReadOnly === true) const exchange = mounts.find((mount) => mount.Type === 'bind' && mount.Target === '/run/freshell-source-exchange' && mount.ReadOnly === true && typeof mount.Source === 'string' && path.dirname(mount.Source) === os.tmpdir() && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) - return !!binary && !!workspace && !!exchange && typeof parsed.WorkingDir === 'string' + if (!binary || !workspace || !exchange || [binary,workspace,exchange].some(mount => mount.VolumeOptions !== undefined)) return false + const home = mounts.find(mount => mount.Target === '/home/freshell/provider') + if (home) { + if (!equalRecord(host.Tmpfs, {}) || home.Type !== 'volume' || home.Source !== owner.providerVolumeName + || home.ReadOnly !== true || !equalRecord(home.VolumeOptions,{NoCopy:true})) return false + // Match a previously authorized complete ordinary generation. No native + // store path, arbitrary alias, or original controller source is admitted. + const profiles = [...this.claudeCandidates.values()].filter(input => input.installationId === owner.installationId + && input.soulId === owner.soulId && input.providerVolumeName === owner.providerVolumeName) + const matches = profiles.some(input => { + const expected = [binary,workspace,exchange,home, + ...input.configurationBinds.map(bind => ({Type:'bind',Source:path.join(input.stageRoot,bind.sourceRelativePath),Target:bind.effectiveDestination,ReadOnly:true})), + ...input.resourceMounts.map(unit => ({Type:'volume',Source:owner.providerVolumeName,Target:unit.effectiveDestination,ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:unit.providerRelativePath}})), + ] + return mounts.length === expected.length && new Set(mounts.map(mount => mount.Target)).size === mounts.length + && expected.every(wanted => mounts.some(actual => ['Type','Source','Target','ReadOnly'].every(key => actual[key] === wanted[key]) + && (wanted.Type === 'volume' ? equalRecord(actual.VolumeOptions,wanted.VolumeOptions) : actual.VolumeOptions === undefined))) + }) + if (!matches) return false + } else if (mounts.length !== 3 || !equalRecord(host.Tmpfs, { '/home/freshell/provider': 'rw,nosuid,nodev,noexec,size=16m,uid=65534,gid=0,mode=0700' })) return false + return typeof parsed.WorkingDir === 'string' && path.normalize(parsed.WorkingDir) === parsed.WorkingDir && (parsed.WorkingDir === workspace.Source || isStrictDescendant(parsed.WorkingDir, workspace.Source)) } @@ -1000,7 +1019,9 @@ export class RestrictedDockerBroker { && (host.CapAdd?.length ?? 0) === 0 && host.LogConfig?.Type === 'none' && Array.isArray(host.Mounts) && host.Mounts.length === expected.HostConfig.Mounts.length && expected.HostConfig.Mounts.every((wanted: Record) => host.Mounts.some((mount: Record) => - ['Type','Source','Target','ReadOnly'].every((key) => mount[key] === wanted[key]) && !mount.VolumeOptions)) + ['Type','Source','Target','ReadOnly'].every((key) => mount[key] === wanted[key]) + && (wanted.Type === 'volume' ? mount.VolumeOptions?.NoCopy === true && !mount.VolumeOptions.DriverConfig + && (mount.VolumeOptions.Subpath ?? '') === (wanted.VolumeOptions.Subpath ?? '') : mount.VolumeOptions === undefined))) } private validateInputPublisher(name:string,body:Buffer):PreparedInputAuthorization | undefined { diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 1c6d2e393..44fdccd5e 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -369,6 +369,38 @@ function candidateHelperBody(input: ClaudeCandidateAuthorization, role: 'mountpo } describe('fixed initial Claude candidate broker', () => { + it('observes retained sources through the exact frozen ordinary aliases and read-only owned HOME', async () => { + await withHttpBroker(async ({request, forwarded, broker: instance}) => { + const input = candidateAuthorization(), body = sourceHelperBody() + instance.authorizeInitialProviderVolume(sourceAuthorization()) + instance.authorizeClaudeCandidate(input) + body.HostConfig.Tmpfs = {} + body.HostConfig.Mounts.push( + {Type:'volume',Source:input.providerVolumeName,Target:'/home/freshell/provider',ReadOnly:true,VolumeOptions:{NoCopy:true}}, + ...input.configurationBinds.map(bind => ({Type:'bind',Source:`${input.stageRoot}/${bind.sourceRelativePath}`,Target:bind.effectiveDestination,ReadOnly:true})), + ...input.resourceMounts.map(unit => ({Type:'volume',Source:input.providerVolumeName,Target:unit.effectiveDestination,ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:unit.providerRelativePath}})), + ) + const before = forwarded.length + for (const mutation of ['home-write','foreign-home','alias-write','alias-subpath','alias-source','missing-alias','native-history']) { + const changed = structuredClone(body) + if (mutation === 'home-write') changed.HostConfig.Mounts[3].ReadOnly = false + if (mutation === 'foreign-home') changed.HostConfig.Mounts[3].Source = 'foreign-home' + if (mutation === 'alias-write') changed.HostConfig.Mounts.at(-1).ReadOnly = false + if (mutation === 'alias-subpath') changed.HostConfig.Mounts.at(-1).VolumeOptions.Subpath = '.claude/projects' + if (mutation === 'alias-source') changed.HostConfig.Mounts[4].Source = `${input.stageRoot}/.freshell/mcp-capability.json` + if (mutation === 'missing-alias') changed.HostConfig.Mounts.pop() + if (mutation === 'native-history') changed.HostConfig.Mounts.push({Type:'volume',Source:input.providerVolumeName,Target:'/history',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'.claude/projects'}}) + expect((await request('POST',`/v1.47/containers/create?name=${sourceName}`,changed)).status,mutation).toBe(403) + expect(forwarded).toHaveLength(before) + } + expect((await request('POST',`/v1.47/containers/create?name=${sourceName}`,body)).status).toBe(201) + expect((await request('GET',`/v1.47/containers/${helperId}/json`)).status).toBe(200) + expect((await request('POST',`/v1.47/containers/${helperId}/start`)).status).toBe(204) + expect((await request('POST',`/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200) + expect((await request('DELETE',`/v1.47/containers/${helperId}?force=1`)).status).toBe(204) + expect((await request('GET',`/v1.47/containers/${helperId}/json`)).status).toBe(404) + }) + }) it('qualifies retained candidates only through selected frozen inputs and read-only owned HOME', async () => { await withHttpBroker(async ({ request, forwarded, broker: instance }) => { const input = candidateAuthorization(), generationId = 'grant-55555555-5555-4555-8555-555555555555' From da7a2e7a4d1fe412e9fe4742a3491fac929cd1e9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 14:47:35 -0700 Subject: [PATCH 150/493] Retain frozen Claude sources through fresh recovery preparation --- .../src/managed_mcp_capability.rs | 277 +++++++++++++- .../managed_mcp_capability/claude_memory.rs | 81 +++- .../managed_mcp_capability/claude_stage.rs | 359 +++++++++++++++++- .../src/managed_provider_launch_tests.rs | 269 ++++++++++++- .../managed-provider-launch-sandbox.py | 20 +- scripts/testing/runtime-test-broker.ts | 3 +- .../testing/runtime-test-broker.test.ts | 2 + 7 files changed, 951 insertions(+), 60 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 09b7623f4..acf2e6d9d 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1561,7 +1561,22 @@ fn stage_provider_root( opencode_input.amplifier_manifest, )? } else if provider == "claude" { - if let Some(observation) = opencode_input.claude_observation { + if let Some(prior_stage) = prior_stage { + claude_stage::prepare_retained( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input + .claude_manifest + .ok_or("Claude retained inputs lack original selectors")?, + opencode_input.claude_observation, + &excluded, + prior_stage, + )? + } else if let Some(observation) = opencode_input.claude_observation { claude_stage::prepare_with_native_observation( home, opencode_input.cwd.unwrap_or(home), @@ -7679,6 +7694,266 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[tokio::test] + async fn managed_provider_claude_fresh_replacement_uses_frozen_origins_and_current_project() { + use freshell_platform::claude_inputs::{ + ClaudeInputManifest, NativeSourceRuntime, INLINE_SETTINGS_SOURCE, + }; + use freshell_runtime_protocol::{ + PreparedProviderContext, ProviderLaunchContext, ProviderPreparation, + }; + for form in ["file", "inline"] { + let owned = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::tempdir_in(root).unwrap(), + None => tempfile::tempdir().unwrap(), + }; + let home = owned.path().join("original-home"); + let config = home.join("selected-config"); + let plugin = home.join("selected-plugin"); + let workspace = owned.path().join("workspace"); + let original_cwd = workspace.join("original"); + let resume_cwd = workspace.join("resume"); + let hooks = resume_cwd.join(".claude/hooks"); + for path in [ + &config.join("skills/probe/node_modules/probe-dependency"), + &plugin, + &original_cwd.join(".claude/skills/old-project"), + &hooks, + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + config.join("settings.json"), + r#"{"env":{"SELECTED_USER":"frozen"}}"#, + ) + .unwrap(); + let explicit = config.join("explicit.json"); + let settings = r#"{"env":{"SELECTED_EXPLICIT":"frozen"},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"node consume.cjs"}]}]}}"#; + std::fs::write(&explicit, settings).unwrap(); + std::fs::write(config.join("CLAUDE.md"), "@../imported-memory.md\n").unwrap(); + std::fs::write(home.join("imported-memory.md"), "frozen-memory-import").unwrap(); + std::fs::write(config.join("skills/probe/resource"), "frozen-resource").unwrap(); + std::fs::write( + config.join("skills/probe/node_modules/probe-dependency/index.js"), + "module.exports='frozen-dependency'", + ) + .unwrap(); + std::fs::write(config.join("skills/probe/consume.cjs"), "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8')+':'+require('probe-dependency'))").unwrap(); + std::fs::write(plugin.join("resource"), "frozen-plugin").unwrap(); + std::fs::write(plugin.join("consume.cjs"), "process.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8'))").unwrap(); + std::fs::write( + original_cwd.join(".claude/skills/old-project/poison"), + "unselected-old-project", + ) + .unwrap(); + std::fs::write(hooks.join("resource"), "current-resume-hook").unwrap(); + std::fs::write(hooks.join("consume.cjs"), "#!/usr/bin/env node\nprocess.stdout.write(require('node:fs').readFileSync(__dirname+'/resource','utf8'))").unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + for path in [ + config.join("skills/probe/consume.cjs"), + hooks.join("consume.cjs"), + ] { + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o750)).unwrap(); + } + } + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Claude { mcp_args: vec![] }, + mcp_capability: None, + config: vec![], + prepared_inputs: None, + }; + let manifest = ClaudeInputManifest { + version: 1, + accepted_native_plan: None, + native_source_runtime: Some(NativeSourceRuntime::Terminal), + original_home: home.clone(), + cwd: original_cwd.clone(), + workspace: workspace.clone(), + config_root: Some(config.clone()), + settings_file: (form == "file").then(|| explicit.clone()), + settings_origin: None, + inline_settings_sha256: (form == "inline") + .then(|| format!("{:x}", sha2::Sha256::digest(settings.as_bytes()))), + setting_sources: freshell_platform::claude_inputs::default_setting_sources(), + plugin_dirs: vec![plugin.clone()], + }; + let prior = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("claude", &home, &context.config)), + OpencodeEphemeralInput { + claude_manifest: Some(&manifest), + claude_inline_settings: (form == "inline").then_some(settings), + cwd: Some(&original_cwd), + workspace: Some(&workspace), + launch_context: Some(&context), + ..Default::default() + }, + None, + ) + .unwrap(); + let prior_context = read_prepared_context(&directory, &prior).unwrap(); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + let prior_before = + std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(); + std::fs::rename(&home, owned.path().join("unavailable-original-home")).unwrap(); + let callback = serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"claude","workspace":workspace,"cwd":resume_cwd,"priorContext":prior_context,"providerSecretReferences":[]})).unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + owned.path().join("unused.sock"), + "fixture-control", + ); + let bytes = apply_prepared_callback(&callback, "fixture-control", &store, &directory, "http://host.docker.internal:4000", &owned.path().join("wrong-home"), &client).await + .expect("fresh replacement must use frozen selected origins when original HOME is unavailable"); + let ready: PreparedProviderContext = serde_json::from_slice(&bytes).unwrap(); + let inputs = ready + .provider_launch_context + .prepared_inputs + .as_ref() + .unwrap(); + let stage = staged_provider_root_path(&directory, &inputs.generation_id).unwrap(); + assert_ne!(inputs.generation_id, prior.grant_id); + assert_eq!( + std::fs::read(prior_stage.join(".freshell/prepared-context.json")).unwrap(), + prior_before + ); + assert!(!inputs + .resource_mounts + .iter() + .any(|mount| Path::new(&mount.logical_destination).starts_with(&original_cwd))); + let skill = inputs + .resource_mounts + .iter() + .find(|mount| mount.logical_destination == "/home/freshell/provider/.claude/skills") + .unwrap(); + let consume = stage + .join(&skill.stage_relative_path) + .join("probe/consume.cjs"); + let output = std::process::Command::new(&consume).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"frozen-resource:frozen-dependency"); + let plugin_mount = inputs + .resource_mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == plugin) + .unwrap(); + let output = std::process::Command::new("node") + .arg( + stage + .join(&plugin_mount.stage_relative_path) + .join("consume.cjs"), + ) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"frozen-plugin"); + let memory = inputs + .resource_mounts + .iter() + .find(|mount| { + Path::new(&mount.logical_destination) == home.join("imported-memory.md") + }) + .unwrap(); + assert_eq!( + std::fs::read(stage.join(&memory.stage_relative_path)).unwrap(), + b"frozen-memory-import" + ); + let hook = inputs + .resource_mounts + .iter() + .find(|mount| Path::new(&mount.logical_destination) == hooks) + .unwrap(); + let output = std::process::Command::new( + stage.join(&hook.stage_relative_path).join("consume.cjs"), + ) + .output() + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"current-resume-hook"); + let settings_path = if form == "inline" { + stage.join(INLINE_SETTINGS_SOURCE) + } else { + let bind = inputs + .config_binds + .iter() + .find(|bind| { + Path::new(&bind.logical_destination) == manifest.runtime_path(&explicit) + }) + .unwrap(); + stage.join(&bind.source_relative_path) + }; + let output = std::process::Command::new("node").args(["-e", "const fs=require('node:fs'),s=JSON.parse(fs.readFileSync(process.argv[1]));process.stdout.write(s.env.SELECTED_EXPLICIT+':'+require('node:child_process').execSync(s.hooks.SessionStart[0].hooks[0].command,{cwd:process.argv[2]}))"]) + .arg(settings_path).arg(consume.parent().unwrap()).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"frozen:frozen-resource:frozen-dependency"); + let accepted = ready + .provider_launch_context + .mcp_capability + .as_ref() + .unwrap(); + let incarnation = IncarnationId::new(); + store + .activate( + &accepted.grant_id, + soul.clone(), + incarnation.clone(), + &directory, + &accepted.endpoint, + ) + .unwrap(); + store + .commit_preparation(&accepted.grant_id, &soul, &incarnation, &directory) + .unwrap(); + let context_before = + std::fs::read(stage.join(".freshell/prepared-context.json")).unwrap(); + let stages_before = std::fs::read_dir(stage.parent().unwrap()) + .unwrap() + .map(|entry| entry.unwrap().file_name()) + .collect::>(); + std::fs::remove_file(stage.join(&memory.stage_relative_path)).unwrap(); + let callback = serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"claude","workspace":workspace,"cwd":resume_cwd,"priorContext":ready.provider_launch_context,"providerSecretReferences":[]})).unwrap(); + let error = apply_prepared_callback( + &callback, + "fixture-control", + &store, + &directory, + "http://host.docker.internal:4000", + &owned.path().join("wrong-home"), + &client, + ) + .await + .err() + .expect("lost selected frozen ordinary source must block another fresh generation"); + assert!( + error.contains("frozen selected source unavailable"), + "{error}" + ); + assert!(!error.contains("frozen-memory-import")); + assert_eq!( + std::fs::read(stage.join(".freshell/prepared-context.json")).unwrap(), + context_before + ); + assert_eq!( + std::fs::read_dir(stage.parent().unwrap()) + .unwrap() + .map(|entry| entry.unwrap().file_name()) + .collect::>(), + stages_before, + "failed generation must leave no temporary or published replacement" + ); + let output = std::process::Command::new(&consume).output().unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"frozen-resource:frozen-dependency"); + } + } + #[tokio::test] async fn managed_provider_amplifier_replacement_retains_original_sources_and_pending_replay() { use freshell_runtime_protocol::{ diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs index 1faa4d98f..e1811fe21 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs @@ -16,6 +16,19 @@ pub(super) fn imports( user_rules: &[PathBuf], excluded: &[PathBuf], ) -> Result, String> { + imports_with_frozen(home, roots, user_rules, &[], &[], excluded, None) + .map(|(fresh, frozen)| fresh.into_iter().chain(frozen).collect()) +} + +pub(super) fn imports_with_frozen( + home: &Path, + roots: &[PathBuf], + user_rules: &[PathBuf], + frozen_roots: &[PathBuf], + frozen_rules: &[PathBuf], + excluded: &[PathBuf], + frozen_source: Option<&dyn Fn(&Path) -> Result>, +) -> Result<(Vec, Vec), String> { let excluded = excluded .iter() .map(|path| std::fs::canonicalize(path).unwrap_or_else(|_| path.clone())) @@ -33,16 +46,28 @@ pub(super) fn imports( rule_directories: BTreeSet::new(), rule_entries: 0, rule_path_bytes: 0, + frozen_source, }; + for root in frozen_roots { + graph.visit(root, 0, true)?; + } + for root in frozen_rules { + graph.visit_rules(root, 0, true)?; + } for root in roots { - graph.visit(root, 0)?; + graph.visit(root, 0, false)?; } // Native user rules use the same Markdown import parser, including // external imports, before conditional rules are activated by the provider. for root in user_rules { - graph.visit_rules(root, 0)?; + graph.visit_rules(root, 0, false)?; } - Ok(graph.selected) + let (frozen, fresh): (Vec<_>, Vec<_>) = + graph.selected.into_iter().partition(|(_, frozen)| *frozen); + Ok(( + fresh.into_iter().map(|(path, _)| path).collect(), + frozen.into_iter().map(|(path, _)| path).collect(), + )) } struct ImportGraph<'a> { @@ -54,25 +79,36 @@ struct ImportGraph<'a> { bytes: u64, events: usize, path_bytes: usize, - selected: Vec, + selected: Vec<(PathBuf, bool)>, rule_directories: BTreeSet, rule_entries: usize, rule_path_bytes: usize, + frozen_source: Option<&'a dyn Fn(&Path) -> Result>, } impl ImportGraph<'_> { - fn visit_rules(&mut self, logical: &Path, depth: usize) -> Result<(), String> { + fn source(&self, logical: &Path, frozen: bool) -> Result { + if frozen { + self.frozen_source + .ok_or("Claude frozen source view unavailable")?(logical) + } else { + std::fs::canonicalize(logical) + .map_err(|cause| format!("selected source unavailable: {cause}")) + } + } + fn visit_rules(&mut self, logical: &Path, depth: usize, frozen: bool) -> Result<(), String> { let limits = freshell_platform::provider_environment::CopyLimits::default(); let error = |cause: &str| format!("Claude selected user rules {}: {cause}", logical.display()); if depth > limits.max_depth { return Err(error("resource depth profile exceeded")); } - let source = std::fs::canonicalize(logical) - .map_err(|cause| error(&format!("selected directory unavailable: {cause}")))?; + let source = self + .source(logical, frozen) + .map_err(|cause| error(&cause))?; if self .excluded .iter() - .any(|excluded| source.starts_with(excluded)) + .any(|excluded| source.starts_with(excluded) || logical.starts_with(excluded)) { return Err(error( "selected private or native-state source cannot become ordinary rules", @@ -85,13 +121,14 @@ impl ImportGraph<'_> { return Ok(()); } self.active.insert(source.clone()); - let entries = std::fs::read_dir(logical) + let entries = std::fs::read_dir(&source) .map_err(|cause| error(&format!("selected directory unreadable: {cause}")))?; let mut paths = Vec::new(); for entry in entries { - let path = entry + let entry = entry .map_err(|cause| error(&format!("selected directory entry unavailable: {cause}")))? - .path(); + .file_name(); + let path = logical.join(entry); self.rule_entries = self .rule_entries .checked_add(1) @@ -107,34 +144,36 @@ impl ImportGraph<'_> { } paths.sort(); for path in paths { - let metadata = std::fs::metadata(&path).map_err(|cause| { + let physical = self.source(&path, frozen).map_err(|cause| error(&cause))?; + let metadata = std::fs::metadata(&physical).map_err(|cause| { error(&format!( "selected rule {} metadata failed: {cause}", path.display() )) })?; if metadata.is_dir() { - self.visit_rules(&path, depth + 1)?; + self.visit_rules(&path, depth + 1, frozen)?; } else if path.extension().is_some_and(|extension| extension == "md") { - self.visit(&path, depth + 1)?; + self.visit(&path, depth + 1, frozen)?; } } self.active.remove(&source); Ok(()) } - fn visit(&mut self, logical: &Path, depth: usize) -> Result<(), String> { + fn visit(&mut self, logical: &Path, depth: usize, frozen: bool) -> Result<(), String> { let logical = normalize(logical); let error = |cause: &str| format!("Claude memory import {}: {cause}", logical.display()); if depth > freshell_platform::provider_environment::CopyLimits::default().max_depth { return Err(error("resource depth profile exceeded")); } - let source = std::fs::canonicalize(&logical) - .map_err(|cause| error(&format!("selected file unavailable: {cause}")))?; + let source = self + .source(&logical, frozen) + .map_err(|cause| error(&cause))?; if self .excluded .iter() - .any(|excluded| source.starts_with(excluded)) + .any(|excluded| source.starts_with(excluded) || logical.starts_with(excluded)) { return Err(error( "selected private or native-state source cannot become ordinary memory", @@ -220,9 +259,9 @@ impl ImportGraph<'_> { { return Err(error("selected import path profile exceeded")); } - self.visit(&selected, depth + 1)?; - if !self.selected.contains(&selected) { - self.selected.push(selected); + self.visit(&selected, depth + 1, frozen)?; + if !self.selected.iter().any(|(path, _)| path == &selected) { + self.selected.push((selected, frozen)); } } self.active.remove(&source); diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 2d07bc4d4..2d0765459 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -5,6 +5,239 @@ use freshell_platform::claude_inputs::{ }; use std::path::{Path, PathBuf}; +struct FrozenSources { + root: PathBuf, + inputs: freshell_runtime_protocol::PreparedProviderInputs, +} +impl FrozenSources { + fn load(root: &Path, accepted: &ClaudeInputManifest) -> Result { + use freshell_runtime_protocol::{ProviderLaunchContext, MAX_CONTROL_FRAME_BYTES}; + use sha2::{Digest, Sha256}; + let bytes = super::read_prepared_metadata( + root, + ".freshell/prepared-context.json", + MAX_CONTROL_FRAME_BYTES, + )?; + let context: ProviderLaunchContext = + serde_json::from_slice(&bytes).map_err(|_| "Claude frozen context invalid")?; + let capability = context + .mcp_capability + .as_ref() + .ok_or("Claude frozen generation unavailable")?; + let inputs = context + .prepared_inputs + .ok_or("Claude frozen inputs unavailable")?; + inputs + .validate(Some(capability)) + .map_err(|error| error.message)?; + if root.file_name().and_then(|name| name.to_str()) != Some(inputs.generation_id.as_str()) { + return Err("Claude frozen source generation changed".into()); + } + let bytes = + super::read_prepared_metadata(root, &inputs.manifest.relative_path, 64 * 1024 * 1024)?; + if format!("{:x}", Sha256::digest(&bytes)) != inputs.manifest.sha256 { + return Err("Claude frozen source manifest identity changed".into()); + } + let prior: ClaudeInputManifest = + serde_json::from_slice(&bytes).map_err(|_| "Claude frozen source manifest invalid")?; + prior.validate().map_err(str::to_string)?; + if prior.original_home != accepted.original_home + || prior.config_root != accepted.config_root + || prior.settings_file != accepted.settings_file + || prior.settings_origin != accepted.settings_origin + || prior.inline_settings_sha256 != accepted.inline_settings_sha256 + || prior.plugin_dirs != accepted.plugin_dirs + || prior.setting_sources != accepted.setting_sources + || prior.native_source_runtime != accepted.native_source_runtime + { + return Err("Claude frozen original selectors changed".into()); + } + Ok(Self { + root: root.into(), + inputs, + }) + } + fn selected(&self, logical: &Path) -> Result, String> { + let mut selected: Option<(usize, PathBuf)> = None; + for bind in &self.inputs.config_binds { + if logical == Path::new(&bind.logical_destination) + || logical == Path::new(&bind.effective_destination) + { + selected = Some((usize::MAX, self.root.join(&bind.source_relative_path))); + } + } + for mount in &self.inputs.resource_mounts { + for destination in [&mount.logical_destination, &mount.effective_destination] { + let destination = Path::new(destination); + if let Ok(suffix) = logical.strip_prefix(destination) { + let length = destination.as_os_str().len(); + let physical = self.root.join(&mount.stage_relative_path); + let physical = if suffix.as_os_str().is_empty() { + physical + } else { + physical.join(suffix) + }; + if let Some((prior_length, prior_path)) = &selected { + if *prior_length == length && *prior_path != physical { + return Err("Claude frozen selected source is ambiguous".into()); + } + if *prior_length > length { + continue; + } + } + selected = Some((length, physical)); + } + } + } + selected + .map(|(_, path)| { + std::fs::canonicalize(path) + .map_err(|_| { + "Claude frozen selected source unavailable; retain inputs and reprepare" + .to_string() + }) + .and_then(|path| { + if path.starts_with(&self.root) { + Ok(path) + } else { + Err("Claude frozen selected source escaped its ordinary stage".into()) + } + }) + }) + .transpose() + } + fn source(&self, logical: &Path) -> Result { + self.selected(logical)?.ok_or( + "Claude source has no selected frozen ordinary alias; retain inputs and reprepare" + .into(), + ) + } + fn append( + &self, + prepared: &mut Preparation, + manifest: &ClaudeInputManifest, + configs: &[PathBuf], + resources: &[PathBuf], + ) -> Result<(), String> { + // Keep the existing typed unit layout. Current project units get names + // beyond the prior generation's range, preserving frozen aliases. + fn next_id<'a>( + paths: impl Iterator, + prefix: &str, + ) -> Result { + paths + .filter_map(|path| path.strip_prefix(prefix)) + .filter_map(|suffix| suffix.split('/').next()?.parse::().ok()) + .max() + .map_or(Ok(0), |id| { + id.checked_add(1) + .ok_or("Claude current unit name overflow".into()) + }) + } + let next_config = next_id( + self.inputs + .config_binds + .iter() + .map(|bind| bind.source_relative_path.as_str()), + ".freshell/templates/claude-", + )?; + let next_resource = next_id( + self.inputs + .resource_mounts + .iter() + .map(|mount| mount.stage_relative_path.as_str()), + ".freshell/resource-units/root-", + )?; + let renamed = |relative: &str| -> Result { + for (prefix, offset) in [ + (".freshell/templates/claude-", next_config), + (".freshell/resource-units/root-", next_resource), + (".freshell/provider-roots/root-", next_resource), + ] { + if let Some(suffix) = relative.strip_prefix(prefix) { + let (number, file) = suffix + .split_once('/') + .map_or((suffix, ""), |(number, file)| (number, file)); + let number: usize = number + .parse() + .map_err(|_| "Claude current unit name invalid")?; + let number = number + .checked_add(offset) + .ok_or("Claude current unit name overflow")?; + return Ok(if file.is_empty() { + format!("{prefix}{number}") + } else { + format!("{prefix}{number}/{file}") + }); + } + } + Err("Claude current unit source layout invalid".into()) + }; + for selection in &mut prepared.selections { + selection.destination_relative = renamed( + selection + .destination_relative + .to_str() + .ok_or("Claude current unit path invalid")?, + )? + .into(); + } + for bind in &mut prepared.binds { + bind.source_relative_path = renamed(&bind.source_relative_path)?; + } + for mount in &mut prepared.mounts { + mount.stage_relative_path = renamed(&mount.stage_relative_path)?; + mount.provider_relative_path = renamed(&mount.provider_relative_path)?; + } + let selected = |logical: &str, paths: &[PathBuf]| { + paths.iter().any(|path| { + Path::new(logical) == path || Path::new(logical) == manifest.runtime_path(path) + }) + }; + for bind in &self.inputs.config_binds { + if selected(&bind.logical_destination, configs) && !prepared.binds.contains(bind) { + prepared.binds.push(bind.clone()); + } + } + for mount in &self.inputs.resource_mounts { + if selected(&mount.logical_destination, resources) && !prepared.mounts.contains(mount) { + prepared.mounts.push(mount.clone()); + } + } + for relative in prepared + .binds + .iter() + .map(|bind| &bind.source_relative_path) + .chain( + prepared + .mounts + .iter() + .map(|mount| &mount.stage_relative_path), + ) + { + if !prepared + .selections + .iter() + .any(|selection| selection.destination_relative == Path::new(relative)) + { + let source = self.root.join(relative); + let canonical = std::fs::canonicalize(&source) + .map_err(|_| "Claude frozen ordinary unit unavailable")?; + if !canonical.starts_with(&self.root) { + return Err("Claude frozen ordinary unit escaped its stage".into()); + } + prepared + .selections + .push(freshell_platform::provider_environment::CopySelection { + source, + destination_relative: relative.into(), + }); + } + } + Ok(()) + } +} + const RESOURCE_DIRECTORIES: &[&str] = &[ "skills", "commands", @@ -38,6 +271,7 @@ pub(super) fn prepare_with_native_observation( Some(accepted), Some(observation), excluded, + None, ) } @@ -48,7 +282,37 @@ pub(super) fn prepare( accepted: Option<&ClaudeInputManifest>, excluded: &[PathBuf], ) -> Result { - prepare_selected_sources(home, cwd, workspace, accepted, None, excluded) + prepare_selected_sources(home, cwd, workspace, accepted, None, excluded, None) +} + +pub(super) fn prepare_retained( + home: &Path, + cwd: &Path, + workspace: &Path, + accepted: &ClaudeInputManifest, + observation: Option<&NativeObservation>, + excluded: &[PathBuf], + prior_stage: &Path, +) -> Result { + if let Some(observation) = observation { + observation.validate()?; + if Some(observation.native_source_runtime) != accepted.native_source_runtime + || observation.setting_sources != accepted.setting_sources + || std::fs::canonicalize(cwd).ok().as_ref() != Some(&observation.cwd) + { + return Err("Claude native source observation does not match retained launch".into()); + } + } + let frozen = FrozenSources::load(prior_stage, accepted)?; + prepare_selected_sources( + home, + cwd, + workspace, + Some(accepted), + observation, + excluded, + Some(&frozen), + ) } fn prepare_selected_sources( @@ -58,6 +322,7 @@ fn prepare_selected_sources( accepted: Option<&ClaudeInputManifest>, observation: Option<&NativeObservation>, excluded: &[PathBuf], + frozen: Option<&FrozenSources>, ) -> Result { let mut manifest = accepted.cloned().unwrap_or_else(|| ClaudeInputManifest { version: 1, @@ -80,16 +345,36 @@ fn prepare_selected_sources( let mut resources = Vec::new(); let mut memory = Vec::new(); let mut user_rules = Vec::new(); + let mut frozen_configurations = Vec::new(); + let mut frozen_resources = Vec::new(); + let mut frozen_memory = Vec::new(); + let mut frozen_rules = Vec::new(); if let Some(selected) = &manifest.settings_file { - let origin = canonical(selected)?; - if !std::fs::metadata(&origin).is_ok_and(|metadata| metadata.is_file()) { + let origin = if frozen.is_some() { + manifest + .settings_origin + .clone() + .ok_or("Claude frozen settings origin unrecorded")? + } else { + canonical(selected)? + }; + let physical = if let Some(frozen) = frozen { + frozen.source(&origin)? + } else { + origin.clone() + }; + if !std::fs::metadata(&physical).is_ok_and(|metadata| metadata.is_file()) { return Err(format!( "Claude --settings {} must select a regular file", selected.display() )); } manifest.settings_origin = Some(origin.clone()); - configurations.push((origin.clone(), origin)); + if frozen.is_some() { + frozen_configurations.push(origin); + } else { + configurations.push((origin.clone(), origin)); + } } // The manifest retains original explicit selector origins. Project discovery // and projection follow the current launch directory on every preparation. @@ -135,8 +420,19 @@ fn prepare_selected_sources( } else { None }; - if user_enabled && optional(&user_settings)? { - configurations.push((user_settings.clone(), canonical(&user_settings)?)); + let original_optional = |path: &Path| { + if let Some(frozen) = frozen { + frozen.selected(path).map(|source| source.is_some()) + } else { + optional(path) + } + }; + if user_enabled && original_optional(&user_settings)? { + if frozen.is_some() { + frozen_configurations.push(user_settings.clone()); + } else { + configurations.push((user_settings.clone(), canonical(&user_settings)?)); + } } for relative in ["CLAUDE.md", "rules"] .into_iter() @@ -146,7 +442,16 @@ fn prepare_selected_sources( continue; } let logical = config_root.join(relative); - if optional(&logical)? { + if original_optional(&logical)? { + if frozen.is_some() { + if relative == "CLAUDE.md" { + frozen_memory.push(logical.clone()); + } else if relative == "rules" { + frozen_rules.push(logical.clone()); + } + frozen_resources.push(logical); + continue; + } if relative == "CLAUDE.md" { memory.push(logical.clone()); } else if relative == "rules" { @@ -236,24 +541,52 @@ fn prepare_selected_sources( } } for plugin in &manifest.plugin_dirs { - canonical(plugin)?; // Explicit selections may never be silently omitted. - resources.push(plugin.clone()); + if let Some(frozen) = frozen { + frozen.source(plugin)?; + frozen_resources.push(plugin.clone()); + } else { + canonical(plugin)?; + resources.push(plugin.clone()); + } } - for imported in super::claude_memory::imports(&manifest.original_home, &memory, &user_rules, excluded).inspect_err(|error| { + let frozen_resolver = |path: &Path| { + frozen + .ok_or("Claude frozen source view unavailable".to_string())? + .source(path) + }; + let (imports, frozen_imports) = super::claude_memory::imports_with_frozen(&manifest.original_home, &memory, &user_rules, &frozen_memory, &frozen_rules, excluded, frozen.map(|_| &frozen_resolver as &dyn Fn(&Path) -> Result)).inspect_err(|error| { tracing::warn!(event="provider.inputs.preparation.blocked",provider="claude",stage="controller",operation="select_memory_imports",reason=%error,"Selected Claude memory imports could not be prepared"); - })? { + })?; + for imported in imports { if !resources.iter().any(|root| imported.starts_with(root)) { resources.push(imported); } } - prepare_selected( + for imported in frozen_imports { + if !frozen_resources + .iter() + .any(|root| imported.starts_with(root)) + { + frozen_resources.push(imported); + } + } + let mut prepared = prepare_selected( "claude", cwd, &manifest, configurations, resources, |path| manifest.runtime_path(path), - ) + )?; + if let Some(frozen) = frozen { + frozen.append( + &mut prepared, + &manifest, + &frozen_configurations, + &frozen_resources, + )?; + } + Ok(prepared) } // The ordinary native initializer sets projectRoot to realpath(launch cwd). diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 4668078bf..cc777375d 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -69,20 +69,8 @@ impl BrokerAuthorizedBackend { let Some(candidate) = request.candidate.as_ref() else { return Ok(()); }; - let invalid = || { - BackendError::InvalidConfig("closed initial Claude candidate handoff invalid".into()) - }; - let freshell_supervisor::backend::ClaudeSourceOwner::Initial { observation, .. } = owner - else { - // The retained reader stays fail-closed until its actual prior - // ordinary topology is qualified. Do not invent initial authority. - return Ok(()); - }; - let InitialAuthVolumeObservation::Absent { daemon_id } = observation; - let ClaudeSourceAuthority::InitialReservation { reservation_id } = &request.authority - else { - return Err(invalid()); - }; + let invalid = + || BackendError::InvalidConfig("closed Claude candidate handoff invalid".into()); request.validate().map_err(|_| invalid())?; let inputs = candidate .provider_launch_context @@ -96,7 +84,6 @@ impl BrokerAuthorizedBackend { .ok_or_else(invalid)?; if binding.soul_id != request.soul_id.as_str() || binding.generation_id != inputs.generation_id - || inputs.initial_auth_reservation.as_deref() != Some(reservation_id.as_str()) || inputs.indexed_files.is_some() || inputs.manifest.source_count != 0 || inputs.manifest.alias_count != 0 @@ -104,6 +91,59 @@ impl BrokerAuthorizedBackend { { return Err(invalid()); } + let (daemon_id, reservation_id, retained) = match owner { + freshell_supervisor::backend::ClaudeSourceOwner::Initial { observation, .. } => { + let InitialAuthVolumeObservation::Absent { daemon_id } = observation; + let ClaudeSourceAuthority::InitialReservation { reservation_id } = + &request.authority + else { + return Err(invalid()); + }; + if inputs.initial_auth_reservation.as_deref() != Some(reservation_id.as_str()) { + return Err(invalid()); + } + (daemon_id, reservation_id.clone(), None) + } + freshell_supervisor::backend::ClaudeSourceOwner::Retained { + handle, + recovery_source, + } => { + let source = recovery_source.as_ref().ok_or_else(invalid)?; + source.validate().map_err(|_| invalid())?; + if !matches!( + request.authority, + ClaudeSourceAuthority::RecoveryAttempt { .. } + ) || handle.soul_id() != &request.soul_id + || inputs.initial_auth_reservation.is_some() + { + return Err(invalid()); + } + let prior = handle + .terminal() + .and_then(|spec| spec.provider_launch_context.as_ref()) + .or_else(|| { + handle + .fresh_agent() + .and_then(|spec| spec.provider_launch_context.as_ref()) + }) + .and_then(|context| context.prepared_inputs.as_ref()) + .ok_or_else(invalid)?; + if prior.generation_id != source.prior_generation_id + || prior.generation_id == inputs.generation_id + { + return Err(invalid()); + } + // Original creation metadata is carried only by this closed + // fixture. The real authority remains its registry recovery + // attempt and private callback scope, never a new reservation. + let reservation = prior.initial_auth_reservation.clone().ok_or_else(invalid)?; + let retained = json!({"attemptId":source.registry_attempt_id,"generationId":prior.generation_id, + "stageRoot":self.control.join("provider-roots").join(&prior.generation_id), + "configurationBinds":prior.config_binds.iter().map(|bind|json!({"sourceRelativePath":bind.source_relative_path,"effectiveDestination":bind.effective_destination})).collect::>(), + "resourceMounts":prior.resource_mounts.iter().map(|mount|json!({"stageRelativePath":mount.stage_relative_path,"providerRelativePath":mount.provider_relative_path,"effectiveDestination":mount.effective_destination,"kind":mount.kind})).collect::>()}); + (handle.daemon_id(), reservation, Some(retained)) + } + }; let root = self.handoff.parent().ok_or_else(invalid)?; // The wrapper's closed source configuration contains only artifact and // owner metadata. Bound the physical file before allocating its frame. @@ -142,6 +182,9 @@ impl BrokerAuthorizedBackend { "resourceMounts":inputs.resource_mounts.iter().map(|mount|json!({"stageRelativePath":mount.stage_relative_path,"providerRelativePath":mount.provider_relative_path,"effectiveDestination":mount.effective_destination,"kind":mount.kind})).collect::>(), }); let mut value = value; + if let Some(retained) = retained { + value["retained"] = retained; + } if let Some(common) = &request.git_common_dir { value["gitCommonDir"] = json!(common); } @@ -256,10 +299,46 @@ impl RuntimeBackend for BrokerAuthorizedBackend { test_run_id: &str, ) -> Result { self.authorize_claude_candidate(owner, request).await?; - let reply = self + let reply = match self .real .observe_claude_sources(owner, request, test_run_id) - .await?; + .await + { + Ok(reply) => reply, + Err(error) => { + let message = error.to_string(); + let classes = [ + "private", + "peer", + "identity", + "reply", + "digest", + "retained", + "frozen", + "manifest", + "metadata", + "topology", + "mount", + "report", + "actual", + "changed", + "timeout", + "unavailable", + "cleanup", + "ownership", + "baseline", + "candidate", + ] + .into_iter() + .filter(|class| message.contains(class)) + .collect::>(); + println!( + "provider_source_reader_failure {}", + json!({"candidate":request.candidate.is_some(),"classes":classes,"messageBytes":message.len()}) + ); + return Err(error); + } + }; if let Some(binding) = &request.observation_request.candidate_binding { println!( "provider_candidate_observer {}", @@ -271,6 +350,14 @@ impl RuntimeBackend for BrokerAuthorizedBackend { "backendCompleteAfterCleanup":true, }) ); + } else if matches!( + owner, + freshell_supervisor::backend::ClaudeSourceOwner::Retained { .. } + ) { + println!( + "provider_retained_source_observer {}", + json!({"uid":reply.observation.uid,"gid":reply.observation.gid,"supplementaryGroups":reply.observation.supplementary_groups,"rootUid":reply.observation.root_uid,"decision":reply.observation.local_decision,"localCount":reply.observation.local_settings.len(),"backendCompleteAfterCleanup":true}) + ); } Ok(reply) } @@ -381,7 +468,9 @@ fn capture_fresh_launch_failure_classes() -> std::sync::Arc, _: tracing_subscriber::layer::Context<'_, S>, ) { - if !event.metadata().target().contains("fresh_agent_proxy") { + if !event.metadata().target().contains("fresh_agent_proxy") + && !event.metadata().target().contains("managed_mcp_capability") + { return; } struct Classes(Vec<&'static str>); @@ -443,6 +532,19 @@ fn capture_fresh_launch_failure_classes() -> std::sync::Arcraw+=b);process.stdin.on('end',async()=>{ .unwrap_or_else(|_| "file".into()); let source_guard = std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD") .unwrap_or_else(|_| "unchanged".into()); - assert!(matches!(source_guard.as_str(), "unchanged" | "changed-git")); + assert!(matches!( + source_guard.as_str(), + "unchanged" | "changed-git" | "retained" + )); let changed_git = source_guard == "changed-git"; let flag_argument = match settings_form.as_str() { "file" if changed_git => flag_alias.to_str().unwrap().into(), @@ -1613,6 +1718,120 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ "provider_launch_consumer {}", json!({"provider":"claude","route":"terminal","digest":qualified_digest}) ); + if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD").as_deref() == Ok("retained") { + let soul = SoulId::parse(terminal.soul_id.clone()).unwrap(); + let prior = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + let prior_generation = prior + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id + .clone(); + let real = DockerEngineBackend::new(fixture.join("broker.sock")); + real.force_stop(&prior) + .await + .expect("stop only the registry-owned disposable runtime"); + assert!(real.verify_empty(&prior).await.unwrap()); + let hidden_home = fixture.join("source-home-unavailable"); + std::fs::rename(&home, &hidden_home).unwrap(); + for path in [&receipt, &instructions_receipt, &instructions_events] { + std::fs::remove_file(path).unwrap(); + } + if settings_form != "owned-last" { + std::fs::remove_file(&flag_receipt).unwrap(); + } + let recovered = tokio::time::timeout( + Duration::from_secs(120), + client.recover(soul.clone(), RecoveryTrigger::HostUnreachable), + ) + .await + .expect("finite actual retained recovery") + .expect("authenticated supervisor recovery reply"); + println!( + "provider_recovery_outcome {}", + json!({"outcome":recovered.outcome,"priorGeneration":prior_generation,"probe":recovered.probe.as_ref().map(|probe|match probe { RecoveryProbe::ResumeReady {..} => "native_resume", RecoveryProbe::PristineSeedReady {..} => "pristine_seed", _ => "other" })}) + ); + if let Some(RecoveryProbe::Blocked { + reason, retry_hint, .. + }) = &recovered.probe + { + let repair = retry_hint.repair.as_deref().unwrap_or(""); + let classes = [ + "Claude", + "source", + "selection", + "changed", + "observation", + "identity", + "frozen", + "ordinary", + "manifest", + "metadata", + "HOME", + "scope", + "grant", + "missing", + "unavailable", + "reprepare", + "budget", + "native", + "store", + "provider", + "callback", + "checkpoint", + "permission", + "configuration", + "revision", + ] + .into_iter() + .filter(|class| repair.contains(class)) + .collect::>(); + println!( + "provider_recovery_diagnostic {}", + json!({"reason":reason,"repairClasses":classes,"repairBytes":repair.len(),"callbackFailureClasses":failure_classes.lock().unwrap().clone()}) + ); + } + assert_eq!(recovered.outcome, RecoveryOutcome::Replaced, "actual retained callback must replace the stopped owned runtime after original ordinary inputs disappear"); + let current = registry.active_handle_for_soul(soul).await.unwrap(); + let generation = ¤t + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id; + assert_ne!(generation, &prior_generation); + assert_eq!(current.provider_volume_name(), prior.provider_volume_name()); + assert_eq!( + consumed(&receipt) + .await + .expect("recovered real native startup hook"), + expected + ); + if settings_form != "owned-last" { + assert_eq!( + consumed(&flag_receipt) + .await + .expect("recovered explicitly selected settings hook"), + json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) + ); + } + imported_graph(&instructions_events, &expected_imports).await; + assert!(!home.exists()); + println!( + "provider_recovery_consumer {}", + json!({"provider":"claude","route":"terminal","digest":qualified_digest,"sameOwnedHome":true,"priorGeneration":prior_generation,"generationId":generation,"originalOrdinaryUnavailable":true,"outcome":recovered.outcome}) + ); + std::fs::rename(&hidden_home, &home).unwrap(); + } client .stop(SoulId::parse(terminal.soul_id).unwrap()) .await @@ -1630,6 +1849,16 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ std::fs::read_to_string(&disabled_project_memory).unwrap(), "@missing-disabled-project.md\n" ); + if source_guard == "retained" { + // This closed scenario spends its two runtime attempts on the original + // terminal and its replacement. The unchanged scenario below retains + // its independent terminal and fresh SDK coverage. + web_task.abort(); + control_task.abort(); + assert!(web_task.await.unwrap_err().is_cancelled()); + assert!(control_task.await.unwrap_err().is_cancelled()); + return; + } // Fresh SDK exposes the existing all-source default. Remove only these // terminal-specific owned decoys before its independent launch. std::fs::remove_file(&disabled_project_memory).unwrap(); @@ -1857,6 +2086,8 @@ try { .all(|event| event["ignored"] != json!(true))); web_task.abort(); control_task.abort(); + assert!(web_task.await.unwrap_err().is_cancelled()); + assert!(control_task.await.unwrap_err().is_cancelled()); } #[tokio::test(flavor = "multi_thread", worker_threads = 4)] diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 1cbb48a8e..8ff05d9cc 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -35,7 +35,7 @@ def main(): parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") parser.add_argument("--claude-flavor",choices=["freshclaude","kilroy"],default="freshclaude") parser.add_argument("--claude-settings-form",choices=["file","inline","owned-last"],default="file") - parser.add_argument("--claude-source-guard",choices=["unchanged","changed-git"],default="unchanged") + parser.add_argument("--claude-source-guard",choices=["unchanged","changed-git","retained"],default="unchanged") args=parser.parse_args() if args.provider!="claude" and args.claude_flavor!="freshclaude": parser.error("the finite Claude flavor requires --provider claude") @@ -75,9 +75,10 @@ def remove(name): known_helpers.append(metadata) fixed_helper=value["Config"].get("Image")==IMAGE and value["Config"].get("Cmd") in [["publish-provider-inputs"],["prepare-provider-mountpoints"]] and bool(known_helpers) candidate_helper=value["Config"].get("Image")==IMAGE and value["Config"].get("Cmd") in [["project-claude-candidate-private"],["prepare-provider-mountpoints"]] and labels.get("com.freshell.runtime-test-run-id")==run and labels.get("com.freshell.installation-id")==installation and labels.get("com.freshell.claude-candidate")=="ordinary" - if candidate_helper: + source_helper=value["Config"].get("Image")==IMAGE and value["Config"].get("Cmd")==["observe-claude-sources-private"] and labels.get("com.freshell.runtime-test-run-id")==run and labels.get("com.freshell.installation-id")==installation + if candidate_helper or source_helper: host_config=value["HostConfig"] - receipt.setdefault("candidateHelperInspections",[]).append({ + receipt.setdefault("sourceHelperInspections" if source_helper else "candidateHelperInspections",[]).append({ "id":value["Id"],"image":value["Config"]["Image"],"user":value["Config"]["User"], "command":value["Config"]["Cmd"],"labels":labels,"workingDirectory":value["Config"].get("WorkingDir"), "running":value["State"].get("Running"),"exitCode":value["State"].get("ExitCode"), @@ -210,10 +211,16 @@ def create(name,*arguments): receipt["claudeSourceGuard"]=args.claude_source_guard receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] receipt["candidateObservers"]=[json.loads(line.split("provider_candidate_observer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_candidate_observer " in line] + if args.claude_source_guard=="retained": + receipt["recoveryConsumers"]=[json.loads(line.split("provider_recovery_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_recovery_consumer " in line] + recovered=receipt["recoveryConsumers"] + route_mismatch_recovery=len(recovered)!=1 or any(value.get("outcome")!="replaced" or value.get("sameOwnedHome") is not True or value.get("originalOrdinaryUnavailable") is not True or value.get("generationId")==value.get("priorGeneration") for value in recovered) print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) + if args.claude_source_guard=="retained":expected_routes={("claude","terminal")} observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} route_mismatch=len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes + if args.claude_source_guard=="retained":route_mismatch=route_mismatch or route_mismatch_recovery if args.claude_source_guard=="changed-git": receipt["blockedRoutes"]=[json.loads(line.split("provider_source_guard ",1)[1]) for line in outcome.stdout.splitlines() if "provider_source_guard " in line] blocked={(item.get("provider"),item.get("route")) for item in receipt["blockedRoutes"]} @@ -228,7 +235,7 @@ def create(name,*arguments): evidence=json.loads((root/"broker-receipt.json").read_text()) receipt["broker"]=evidence for value in evidence["receipts"]:owned.append(value["containerId"]) - if exit_code==0 and args.provider=="claude" and args.claude_source_guard=="unchanged": + if exit_code==0 and args.provider=="claude" and args.claude_source_guard!="changed-git": candidates=evidence["authorizedCandidates"] observers=receipt["candidateObservers"] authorized={(value["requestId"],value["generationId"],value["soulId"]) for value in candidates} @@ -266,6 +273,7 @@ def create(name,*arguments): labels=None for prepared in preparations: if volume==prepared["providerVolumeName"]: + if not prepared.get("initialAuthReservation"):continue source=next(item for item in initial if item["providerVolumeName"]==volume) labels={"com.freshell.installation-id":installation,"com.freshell.soul-id":prepared["soulId"], "freshell.input.reservation":prepared["initialAuthReservation"],"freshell.input.incarnation":prepared["incarnationId"], @@ -281,7 +289,9 @@ def create(name,*arguments): labels={"com.freshell.installation-id":installation,"com.freshell.soul-id":candidate["soulId"], "com.freshell.runtime-test-run-id":run,"com.freshell.source-request-id":candidate["requestId"], "com.freshell.source-binary-sha256":source["binarySha256"],"com.freshell.input-generation":candidate["generationId"], - "com.freshell.source-reservation-id":candidate["reservationId"],"com.freshell.claude-candidate":"ordinary"} + "com.freshell.claude-candidate":"ordinary"} + if candidate.get("retained"):labels["com.freshell.source-recovery-attempt-id"]=candidate["retained"]["attemptId"] + else:labels["com.freshell.source-reservation-id"]=candidate["reservationId"] if labels is None or value.get("Driver")!="local" or value.get("Options") not in (None,{}) or value.get("Labels")!=labels: raise RuntimeError("closed startup volume cleanup ownership changed") docker("volume","rm",volume) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index ee54b835f..88508dfab 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -1015,7 +1015,8 @@ export class RestrictedDockerBroker { && ['Entrypoint','Cmd'].every((key) => same(config[key], expected[key])) && ownedHelperLabels(config.Labels, expected.Labels) && Array.isArray(config.Env) && same([...config.Env].sort(), [...expected.Env].sort()) - && ['NetworkMode','ReadonlyRootfs','CapDrop','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','Tmpfs'].every((key) => same(host[key], expected.HostConfig[key])) + && ['NetworkMode','ReadonlyRootfs','CapDrop','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit'].every((key) => same(host[key], expected.HostConfig[key])) + && same(host.Tmpfs ?? {}, expected.HostConfig.Tmpfs ?? {}) && (host.CapAdd?.length ?? 0) === 0 && host.LogConfig?.Type === 'none' && Array.isArray(host.Mounts) && host.Mounts.length === expected.HostConfig.Mounts.length && expected.HostConfig.Mounts.every((wanted: Record) => host.Mounts.some((mount: Record) => diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 44fdccd5e..f3992962a 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -858,6 +858,8 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]===command)!.body; const inspected=structuredClone(config); inspected.Labels['org.opencontainers.image.version']='24.04'; + // Docker reports an empty Tmpfs object as null on the retained reader. + if (inspected.HostConfig.Tmpfs && Object.keys(inspected.HostConfig.Tmpfs).length === 0) inspected.HostConfig.Tmpfs = null; for (const mount of inspected.HostConfig.Mounts) if (mount.ReadOnly===false) delete mount.ReadOnly; res.statusCode=200;res.end(JSON.stringify({Id:helperId,Config:foreignHelper?{...inspected,User:'0:0'}:inspected,HostConfig:inspected.HostConfig}));return; } From c20c0e662d3441172067f40dbe7bcf3d5404d2b2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 14:59:56 -0700 Subject: [PATCH 151/493] Bound provider projection metadata before reading and publication --- crates/freshell-session-host/src/main.rs | 212 ++++++++++++++++++----- 1 file changed, 170 insertions(+), 42 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 961bb6e0a..d7ceccee9 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1915,6 +1915,27 @@ fn private_bound_destinations( } const PROVIDER_PROJECTION_MANIFEST: &str = ".freshell-config-projection.json"; +// Ordinary disk metadata shares the resource-manifest profile. The control +// pipe's 1 MiB limit does not reduce these provider-filesystem views. +const MAX_PROJECTION_METADATA_BYTES: usize = 64 * 1024 * 1024; + +fn read_projection_metadata(root: &Path, relative: &str) -> Result>, String> { + match std::fs::symlink_metadata(root.join(relative)) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(_) => { + return Err("provider projection metadata unavailable; retain inputs and repair".into()) + } + Ok(_) => {} + } + crate::provider_input_preparation::read_private(root, relative, MAX_PROJECTION_METADATA_BYTES) + .map(Some) + .map_err(|reason| { + tracing::warn!(event="provider.inputs.projection_metadata.blocked",stage="session_host", + operation="read",relative_path=relative,permitted_bytes=MAX_PROJECTION_METADATA_BYTES, + cause=%reason,"Provider projection metadata unavailable or exceeds its profile"); + format!("provider projection metadata {relative}: {reason} (64 MiB limit); retain inputs and repair") + }) +} fn write_projection_manifest( provider_home: &Path, @@ -1925,11 +1946,20 @@ fn write_projection_manifest( "{PROVIDER_PROJECTION_MANIFEST}.tmp-{}", std::process::id() )); - std::fs::write( - &temporary, - serde_json::to_vec(entries).map_err(|error| error.to_string())?, - ) - .map_err(|error| error.to_string())?; + let bytes = + freshell_runtime_protocol::bounded_json_bytes(entries, MAX_PROJECTION_METADATA_BYTES) + .map_err(|_| { + tracing::warn!( + event = "provider.inputs.projection_metadata.blocked", + stage = "session_host", + operation = "serialize", + relative_path = PROVIDER_PROJECTION_MANIFEST, + permitted_bytes = MAX_PROJECTION_METADATA_BYTES, + "Provider projection metadata exceeds its profile" + ); + "provider projection metadata exceeds 64 MiB; retain inputs and repair".to_owned() + })?; + std::fs::write(&temporary, bytes).map_err(|error| error.to_string())?; set_mode(&temporary, 0o600)?; std::fs::rename(&temporary, &manifest).map_err(|error| error.to_string())?; Ok(manifest) @@ -1938,10 +1968,10 @@ fn write_projection_manifest( fn read_projection_manifest( provider_home: &Path, ) -> Result, String> { - match std::fs::read(provider_home.join(PROVIDER_PROJECTION_MANIFEST)) { - Ok(bytes) => serde_json::from_slice(&bytes).map_err(|error| error.to_string()), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(Default::default()), - Err(error) => Err(error.to_string()), + match read_projection_metadata(provider_home, PROVIDER_PROJECTION_MANIFEST)? { + Some(bytes) => serde_json::from_slice(&bytes) + .map_err(|_| "provider projection metadata invalid".into()), + None => Ok(Default::default()), } } @@ -2058,23 +2088,28 @@ fn refresh_provider_config_in_stage( ); std::fs::write( stage.path().join("manifest.json"), - serde_json::to_vec(&destinations).map_err(|error| error.to_string())?, + freshell_runtime_protocol::bounded_json_bytes(&destinations, MAX_PROJECTION_METADATA_BYTES) + .map_err(|_| "provider projection destinations exceed 64 MiB")?, ) .map_err(|error| error.to_string())?; std::fs::write( stage.path().join("provider.json"), - serde_json::to_vec(provider).map_err(|error| error.to_string())?, + freshell_runtime_protocol::bounded_json_bytes(&provider, MAX_PROJECTION_METADATA_BYTES) + .map_err(|_| "provider projection identity exceeds 64 MiB")?, ) .map_err(|error| error.to_string())?; std::fs::write( stage.path().join("private-mountpoints.json"), - serde_json::to_vec(&private_bound_destinations(context)) - .map_err(|_| "private mountpoint metadata invalid")?, + freshell_runtime_protocol::bounded_json_bytes( + &private_bound_destinations(context), + MAX_PROJECTION_METADATA_BYTES, + ) + .map_err(|_| "private mountpoint metadata exceeds 64 MiB")?, ) .map_err(|_| "private mountpoint metadata write failed")?; let mounted = freshell_runtime_protocol::bounded_json_bytes( &resource_mount_roots(context), - freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, + MAX_PROJECTION_METADATA_BYTES, ) .map_err(|_| "resource mountpoint metadata profile exceeded")?; std::fs::write(stage.path().join("resource-mountpoints.json"), mounted) @@ -2335,19 +2370,19 @@ fn refresh_provider_config_worker(args: &[String]) -> Result<(), String> { fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<(), String> { let destinations: Vec = serde_json::from_slice( - &std::fs::read(stage.join("manifest.json")).map_err(|error| error.to_string())?, + &read_projection_metadata(stage, "manifest.json")? + .ok_or("provider projection destinations unavailable")?, ) - .map_err(|error| error.to_string())?; + .map_err(|_| "provider projection destinations invalid")?; let projection = stage.join("projection"); let previous = read_projection_manifest(provider_home)?; let current = read_projection_manifest(&projection)?; let protected: std::collections::BTreeSet = - match std::fs::read(stage.join("private-mountpoints.json")) { - Ok(bytes) => { + match read_projection_metadata(stage, "private-mountpoints.json")? { + Some(bytes) => { serde_json::from_slice(&bytes).map_err(|_| "private mountpoint metadata invalid")? } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Default::default(), - Err(_) => return Err("private mountpoint metadata unavailable".into()), + None => Default::default(), }; if protected.iter().any(|relative| { Path::new(relative).as_os_str().is_empty() @@ -2408,10 +2443,10 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() }, ) .collect::>(); - let provider = match std::fs::read(stage.join("provider.json")) { - Ok(bytes) => serde_json::from_slice::(&bytes).map_err(|error| error.to_string())?, - Err(error) if error.kind() == std::io::ErrorKind::NotFound => "unknown".into(), - Err(error) => return Err(error.to_string()), + let provider = match read_projection_metadata(stage, "provider.json")? { + Some(bytes) => serde_json::from_slice::(&bytes) + .map_err(|_| "provider projection identity invalid")?, + None => "unknown".into(), }; if !matches!( provider.as_str(), @@ -2419,24 +2454,12 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() ) { return Err("provider config refresh provider is unsupported".into()); } - let mounted: Vec = { - use std::io::Read; - match std::fs::File::open(stage.join("resource-mountpoints.json")) { - Ok(file) => { - let mut bytes = Vec::new(); - file.take(freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES as u64 + 1) - .read_to_end(&mut bytes) - .map_err(|_| "resource mountpoint metadata read failed")?; - if bytes.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES { - return Err("resource mountpoint metadata profile exceeded".into()); - } - serde_json::from_slice(&bytes) - .map_err(|_| "resource mountpoint metadata invalid")? - } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Vec::new(), - Err(_) => return Err("resource mountpoint metadata unavailable".into()), - } - }; + let mounted: Vec = + match read_projection_metadata(stage, "resource-mountpoints.json")? { + Some(bytes) => serde_json::from_slice(&bytes) + .map_err(|_| "resource mountpoint metadata invalid")?, + None => Vec::new(), + }; if mounted.iter().any(|path| { !path.starts_with(".freshell/provider-roots") || path @@ -4117,6 +4140,111 @@ mod tests { ); } + #[test] + fn provider_config_projection_metadata_writer_preserves_prior_on_oversize() { + let home = tempfile::tempdir().unwrap(); + let accepted = ["skills/prior".to_owned()].into_iter().collect(); + let path = write_projection_manifest(home.path(), &accepted).unwrap(); + let before = std::fs::read(&path).unwrap(); + let oversized = ["x".repeat(64 * 1024 * 1024)].into_iter().collect(); + assert!(write_projection_manifest(home.path(), &oversized).is_err()); + assert_eq!(std::fs::read(path).unwrap(), before); + assert_eq!( + std::fs::read_dir(home.path()).unwrap().count(), + 1, + "failed metadata publication must not leave a temporary file" + ); + } + + #[cfg(unix)] + #[test] + fn provider_config_projection_metadata_rejects_oversize_before_read_and_publication() { + use std::io::Read; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let stage = owned.path().join("stage"); + std::fs::create_dir_all(home.join("skills")).unwrap(); + std::fs::create_dir_all(stage.join("projection/skills")).unwrap(); + let fixtures = [ + ( + home.join(PROVIDER_PROJECTION_MANIFEST), + br#"["skills","skills/old"]"#.as_slice(), + ), + ( + stage.join("projection").join(PROVIDER_PROJECTION_MANIFEST), + br#"["skills","skills/new"]"#.as_slice(), + ), + (stage.join("manifest.json"), br#"["skills"]"#.as_slice()), + (stage.join("private-mountpoints.json"), b"[]".as_slice()), + (stage.join("resource-mountpoints.json"), b"[]".as_slice()), + (stage.join("provider.json"), br#""claude""#.as_slice()), + ]; + for (path, bytes) in &fixtures { + std::fs::write(path, bytes).unwrap(); + } + std::fs::write(home.join("skills/old"), b"prior-projection-nonce").unwrap(); + std::fs::write(home.join("skills/sibling"), b"native-sibling-nonce").unwrap(); + std::fs::write( + stage.join("projection/skills/new"), + b"current-projection-nonce", + ) + .unwrap(); + for (path, original) in &fixtures { + let mut file = std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(path) + .unwrap(); + file.set_len(64 * 1024 * 1024 + 1).unwrap(); + let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + file.read_exact(&mut [0]).unwrap(); + assert_ne!( + file.metadata().unwrap().accessed().unwrap(), + old_access, + "fixture filesystem must observe payload reads" + ); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + drop(file); + assert!(apply_staged_provider_config(&stage, &home).is_err()); + assert_eq!( + std::fs::metadata(path).unwrap().accessed().unwrap(), + old_access, + "oversized ordinary metadata payload was read: {}", + path.display() + ); + assert_eq!( + std::fs::read(home.join("skills/old")).unwrap(), + b"prior-projection-nonce" + ); + assert_eq!( + std::fs::read(home.join("skills/sibling")).unwrap(), + b"native-sibling-nonce" + ); + assert!(!home.join("skills/new").exists()); + std::fs::write(path, original).unwrap(); + } + // The disk profile is 64 MiB, independently of the private control + // frame. JSON whitespace remains an admitted ordinary metadata view. + let resource_metadata = stage.join("resource-mountpoints.json"); + let mut padded = vec![b' '; 64 * 1024 * 1024]; + padded[..2].copy_from_slice(b"[]"); + std::fs::write(resource_metadata, &padded).unwrap(); + drop(padded); + apply_staged_provider_config(&stage, &home).unwrap(); + assert!(!home.join("skills/old").exists()); + assert_eq!( + std::fs::read(home.join("skills/new")).unwrap(), + b"current-projection-nonce" + ); + assert_eq!( + std::fs::read(home.join("skills/sibling")).unwrap(), + b"native-sibling-nonce" + ); + } + #[cfg(unix)] #[test] fn provider_config_failed_staged_resource_preserves_previous_projection_and_retry() { From c40aa8fbf4c6559cf6fb9d8ef8eb112a0c93e97f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 15:10:16 -0700 Subject: [PATCH 152/493] Preserve provider preparation journals and leases on failed writes --- .../src/managed_mcp_capability.rs | 456 ++++++++++++++++-- 1 file changed, 410 insertions(+), 46 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index acf2e6d9d..50e3f85f8 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -27,6 +27,7 @@ use tokio::io::{AsyncReadExt, AsyncWriteExt}; const CAPABILITY_PATH: &str = ".freshell/mcp-capability.json"; const GRANT_LIFETIME_SECS: u64 = 30 * 24 * 60 * 60; const RENEWAL_WINDOW_SECS: u64 = 7 * 24 * 60 * 60; +const MAX_CAPABILITY_METADATA_BYTES: usize = 128 * 1024; mod amplifier_resources; mod amplifier_stage; mod claude_memory; @@ -1342,7 +1343,6 @@ fn cleanup_unrecoverable_inputs( } fn protected_preparation_ids(directory: &Path) -> Result, String> { - use std::io::Read; let mut protected = HashSet::new(); for entry in std::fs::read_dir(directory).map_err(|_| "provider preparation inventory unavailable")? @@ -1358,19 +1358,7 @@ fn protected_preparation_ids(directory: &Path) -> Result, String { continue; } - let mut bytes = Vec::new(); - let Ok(file) = std::fs::File::open(&path) else { - continue; - }; - file.take(128 * 1024 + 1) - .read_to_end(&mut bytes) - .map_err(|_| "provider preparation metadata read failed")?; - if bytes.len() > 128 * 1024 { - return Err("provider preparation metadata profile limit exceeded".into()); - } - let Ok(stored) = serde_json::from_slice::(&bytes) else { - continue; - }; + let stored = read_stored_capability(directory, id)?; if let Some(preparation) = stored.preparation { if preparation.phase != InputPreparationPhase::Committed || !preparation.prior_grant_ids.is_empty() @@ -1962,6 +1950,28 @@ struct StoredCapabilityFile { preparation: Option, } +fn read_stored_capability(directory: &Path, id: &str) -> Result { + grant_path(directory, id)?; + let bytes = read_prepared_metadata(directory, &format!("{id}.json"), MAX_CAPABILITY_METADATA_BYTES) + .map_err(|_| { + tracing::warn!(event="provider.inputs.preparation.journal_blocked",stage="controller", + operation="read",grant_id=id,permitted_bytes=MAX_CAPABILITY_METADATA_BYTES, + "Provider preparation ownership journal unavailable; retain inputs and reconcile"); + "provider preparation ownership journal unavailable or exceeds 128 KiB; retain inputs and reconcile" + })?; + serde_json::from_slice(&bytes).map_err(|_| { + tracing::warn!( + event = "provider.inputs.preparation.journal_blocked", + stage = "controller", + operation = "decode", + grant_id = id, + "Provider preparation ownership journal incomplete; retain inputs and reconcile" + ); + "provider preparation ownership journal invalid or incomplete; retain inputs and reconcile" + .into() + }) +} + fn restored_grant_at( stored: StoredCapabilityFile, incarnation_id: IncarnationId, @@ -1997,26 +2007,48 @@ fn restored_grant( } fn write_capability(path: &Path, grant: &Grant) -> Result<(), String> { - let bytes = serde_json::to_vec(&CapabilityFile { - endpoint: &grant.endpoint, - scope: &grant.token, - soul_id: &grant.soul_id, - incarnation_id: grant.incarnation_id.as_ref(), - expires_at: grant.expires_at, - preparation: grant.preparation.as_ref(), + let result: Result<(), String> = (|| { + let bytes = freshell_runtime_protocol::bounded_json_bytes( + &CapabilityFile { + endpoint: &grant.endpoint, + scope: &grant.token, + soul_id: &grant.soul_id, + incarnation_id: grant.incarnation_id.as_ref(), + expires_at: grant.expires_at, + preparation: grant.preparation.as_ref(), + }, + MAX_CAPABILITY_METADATA_BYTES, + ) + .map_err(|_| { + "provider preparation ownership journal exceeds 128 KiB; retain inputs and reconcile" + })?; + let parent = path + .parent() + .ok_or("provider preparation journal parent unavailable")?; + // A partial write must never erase the prior generation's ownership. The + // temporary file is private and drops itself on every pre-publication error. + let mut file = tempfile::Builder::new() + .prefix(".freshell-grant-") + .tempfile_in(parent) + .map_err(|error| error.to_string())?; + file.write_all(&bytes).map_err(|error| error.to_string())?; + file.as_file() + .sync_all() + .map_err(|error| error.to_string())?; + file.persist(path) + .map_err(|error| error.error.to_string())?; + #[cfg(unix)] + std::fs::File::open(parent) + .and_then(|directory| directory.sync_all()) + .map_err(|error| error.to_string())?; + Ok(()) + })(); + result.map_err(|reason| { + tracing::warn!(event="provider.inputs.preparation.journal_blocked",stage="controller", + operation="publish",soul_id=%grant.soul_id,cause=%reason, + "Provider preparation journal publication failed; retain inputs and reconcile"); + format!("provider preparation journal publication failed: {reason}; retain inputs and reconcile") }) - .map_err(|error| error.to_string())?; - let mut options = OpenOptions::new(); - options.write(true).create(true).truncate(true); - #[cfg(unix)] - { - use std::os::unix::fs::OpenOptionsExt; - options.mode(0o600); - } - let mut file = options.open(path).map_err(|error| error.to_string())?; - file.write_all(&bytes).map_err(|error| error.to_string())?; - file.sync_all().map_err(|error| error.to_string())?; - Ok(()) } fn publish_prepared_context_metadata( @@ -2569,7 +2601,8 @@ impl CapabilityStore { "provider input preparation commit has no matching accepted incarnation".into(), ); } - let Some(preparation) = grant.preparation.as_mut() else { + let mut updated = grant.clone(); + let Some(preparation) = updated.preparation.as_mut() else { return Ok(()); }; if preparation.phase == InputPreparationPhase::Pending { @@ -2578,7 +2611,8 @@ impl CapabilityStore { let changed = preparation.phase != InputPreparationPhase::Committed; preparation.phase = InputPreparationPhase::Committed; let prior = preparation.prior_grant_ids.clone(); - write_capability(&grant.path, grant)?; + write_capability(&updated.path, &updated)?; + *grant = updated; (prior, changed) }; for prior_id in prior { @@ -2588,10 +2622,12 @@ impl CapabilityStore { let grant = grants .get_mut(grant_id) .ok_or("committed provider preparation disappeared")?; - if let Some(preparation) = grant.preparation.as_mut() { + let mut updated = grant.clone(); + if let Some(preparation) = updated.preparation.as_mut() { preparation.prior_grant_ids.clear(); } - write_capability(&grant.path, grant)?; + write_capability(&updated.path, &updated)?; + *grant = updated; if changed { tracing::info!(event="provider.inputs.preparation.committed",grant_id, soul_id=%soul_id,incarnation_id=%incarnation_id); @@ -3021,12 +3057,7 @@ pub(crate) async fn rehydrate_running_capabilities( continue; } } - let Ok(file) = std::fs::read(&path) else { - continue; - }; - let Ok(stored) = serde_json::from_slice::(&file) else { - continue; - }; + let stored = read_stored_capability(&directory, &grant_id)?; if protected.contains(&grant_id) { let current = stored.endpoint == endpoint()? && !stored.scope.is_empty(); store().grants.lock().unwrap().insert( @@ -3129,6 +3160,327 @@ pub(crate) async fn scoped_auth_middleware( mod tests { use super::*; + #[cfg(target_os = "linux")] + #[test] + fn managed_mcp_preparation_journal_partial_write_preserves_prior_and_startup_ownership() { + const CHILD: &str = "FRESHELL_TEST_PREPARATION_PARTIAL_WRITE_CHILD"; + if std::env::var_os(CHILD).is_none() { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "managed_mcp_capability::tests::managed_mcp_preparation_journal_partial_write_preserves_prior_and_startup_ownership", "--nocapture"]) + .env(CHILD, "1").output().unwrap(); + assert!( + String::from_utf8_lossy(&output.stdout) + .contains("preparation_partial_write_fixture_ready"), + "the isolated partial-write fixture must execute" + ); + eprintln!( + "preparation_partial_write_child status={} prior_ownership_erased={}", + output.status, + String::from_utf8_lossy(&output.stderr) + .contains("failed journal publication erased prior accepted ownership") + ); + assert!( + output.status.success(), + "partial journal write did not preserve prior ownership: {}", + output.status + ); + return; + } + let owned = tempfile::tempdir().unwrap(); + let directory = owned.path().join("mcp-capabilities"); + std::fs::create_dir(&directory).unwrap(); + let path = grant_path(&directory, "grant-pending").unwrap(); + let prior = staged_provider_root_path(&directory, "grant-prior").unwrap(); + let pending = staged_provider_root_path(&directory, "grant-pending").unwrap(); + let orphan = staged_provider_root_path(&directory, "grant-orphan").unwrap(); + for root in [&prior, &pending, &orphan] { + std::fs::create_dir_all(root).unwrap(); + } + std::fs::write(prior.join("usable-resource"), b"prior-resource-nonce").unwrap(); + let mut grant = Grant { + soul_id: SoulId::new(), + incarnation_id: None, + token: "synthetic-prior-scope".into(), + expires_at: 1, + endpoint: "http://host.docker.internal:4000".into(), + path: path.clone(), + preparation: Some(InputPreparation { + phase: InputPreparationPhase::Pending, + prior_grant_ids: vec!["grant-prior".into()], + }), + }; + write_capability(&path, &grant).unwrap(); + let before = std::fs::read(&path).unwrap(); + grant.token = "synthetic-replacement-scope".into(); + grant.incarnation_id = Some(IncarnationId::new()); + grant.preparation.as_mut().unwrap().phase = InputPreparationPhase::Bound; + let mut previous = libc::rlimit { + rlim_cur: 0, + rlim_max: 0, + }; + assert_eq!( + unsafe { libc::getrlimit(libc::RLIMIT_FSIZE, &mut previous) }, + 0 + ); + let restricted = libc::rlimit { + rlim_cur: 16, + rlim_max: previous.rlim_max, + }; + println!("preparation_partial_write_fixture_ready"); + let old_signal = unsafe { libc::signal(libc::SIGXFSZ, libc::SIG_IGN) }; + assert_eq!( + unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &restricted) }, + 0 + ); + let result = write_capability(&path, &grant); + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &previous) }, 0); + unsafe { + libc::signal(libc::SIGXFSZ, old_signal); + } + assert!( + result.is_err(), + "fixture must force a real partial write failure" + ); + assert!( + std::fs::read(&path).unwrap() == before, + "failed journal publication erased prior accepted ownership" + ); + assert_eq!( + std::fs::read_dir(&directory).unwrap().count(), + 1, + "failed journal publication left a temporary generation" + ); + cleanup_orphaned_provider_roots(&directory).unwrap(); + assert!(prior.join("usable-resource").is_file()); + assert!(pending.is_dir()); + assert!(!orphan.exists()); + } + + #[cfg(unix)] + #[test] + fn managed_mcp_preparation_inventory_rejects_oversize_before_payload_read() { + use std::io::Read; + let owned = tempfile::tempdir().unwrap(); + let path = grant_path(owned.path(), "grant-pending").unwrap(); + let mut file = std::fs::OpenOptions::new() + .create_new(true) + .read(true) + .write(true) + .open(&path) + .unwrap(); + file.set_len(128 * 1024 + 1).unwrap(); + let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + file.read_exact(&mut [0]).unwrap(); + assert_ne!( + file.metadata().unwrap().accessed().unwrap(), + old_access, + "fixture filesystem must observe payload reads" + ); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + drop(file); + assert!(protected_preparation_ids(owned.path()).is_err()); + assert_eq!( + std::fs::metadata(path).unwrap().accessed().unwrap(), + old_access, + "oversized preparation inventory payload was read" + ); + } + + #[cfg(target_os = "linux")] + #[test] + fn managed_mcp_preparation_failed_commit_replays_bound_generation_until_durable_commit() { + const CHILD: &str = "FRESHELL_TEST_PREPARATION_COMMIT_FAILURE_CHILD"; + if std::env::var_os(CHILD).is_none() { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "managed_mcp_capability::tests::managed_mcp_preparation_failed_commit_replays_bound_generation_until_durable_commit", "--nocapture"]) + .env(CHILD, "1").output().unwrap(); + assert!(String::from_utf8_lossy(&output.stdout) + .contains("preparation_commit_failure_fixture_ready")); + eprintln!( + "preparation_commit_failure_child status={} third_preparation_authorized={}", + output.status, + String::from_utf8_lossy(&output.stderr) + .contains("failed commit authorized a third preparation") + ); + assert!( + output.status.success(), + "failed commit lost its bound generation: {}", + output.status + ); + return; + } + let owned = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let endpoint = "http://host.docker.internal:4000"; + let first = store + .issue( + soul.clone(), + owned.path(), + endpoint.into(), + None, + Default::default(), + None, + ) + .unwrap(); + let first_incarnation = IncarnationId::new(); + store + .activate( + &first.grant_id, + soul.clone(), + first_incarnation.clone(), + owned.path(), + endpoint, + ) + .unwrap(); + store + .commit_preparation(&first.grant_id, &soul, &first_incarnation, owned.path()) + .unwrap(); + let second = store + .issue_replacement( + soul.clone(), + owned.path(), + endpoint.into(), + None, + None, + None, + ) + .unwrap(); + let incarnation = IncarnationId::new(); + store + .activate( + &second.grant_id, + soul.clone(), + incarnation.clone(), + owned.path(), + endpoint, + ) + .unwrap(); + let second_path = grant_path(owned.path(), &second.grant_id).unwrap(); + let accepted = std::fs::read(&second_path).unwrap(); + let prior_scope = store + .grants + .lock() + .unwrap() + .get(&first.grant_id) + .unwrap() + .token + .clone(); + let mut previous = libc::rlimit { + rlim_cur: 0, + rlim_max: 0, + }; + assert_eq!( + unsafe { libc::getrlimit(libc::RLIMIT_FSIZE, &mut previous) }, + 0 + ); + let restricted = libc::rlimit { + rlim_cur: 16, + rlim_max: previous.rlim_max, + }; + println!("preparation_commit_failure_fixture_ready"); + let old_signal = unsafe { libc::signal(libc::SIGXFSZ, libc::SIG_IGN) }; + assert_eq!( + unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &restricted) }, + 0 + ); + let result = store.commit_preparation(&second.grant_id, &soul, &incarnation, owned.path()); + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &previous) }, 0); + unsafe { + libc::signal(libc::SIGXFSZ, old_signal); + } + assert!(result.is_err()); + assert!(std::fs::read(&second_path).unwrap() == accepted); + assert!( + store.authenticates(&prior_scope), + "failed commit retired the prior lease" + ); + let repeated = store + .issue_replacement( + soul.clone(), + owned.path(), + endpoint.into(), + None, + None, + None, + ) + .unwrap(); + assert!( + repeated.grant_id == second.grant_id, + "failed commit authorized a third preparation" + ); + assert_eq!(std::fs::read_dir(owned.path()).unwrap().count(), 2); + store + .commit_preparation(&second.grant_id, &soul, &incarnation, owned.path()) + .unwrap(); + assert!(!store.authenticates(&prior_scope)); + assert!(!grant_path(owned.path(), &first.grant_id).unwrap().exists()); + assert_eq!(std::fs::read_dir(owned.path()).unwrap().count(), 1); + } + + #[test] + fn managed_mcp_preparation_inventory_accepts_exact_profile_and_tracks_pending_prior() { + let owned = tempfile::tempdir().unwrap(); + let path = grant_path(owned.path(), "grant-pending").unwrap(); + let grant = Grant { + soul_id: SoulId::new(), + incarnation_id: None, + token: "synthetic-scope".into(), + expires_at: 1, + endpoint: "http://host.docker.internal:4000".into(), + path: path.clone(), + preparation: Some(InputPreparation { + phase: InputPreparationPhase::Pending, + prior_grant_ids: vec!["grant-prior".into()], + }), + }; + write_capability(&path, &grant).unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + assert_eq!( + std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, + 0o600 + ); + } + let mut bytes = std::fs::read(&path).unwrap(); + bytes.resize(128 * 1024, b' '); + std::fs::write(&path, bytes).unwrap(); + assert_eq!( + protected_preparation_ids(owned.path()).unwrap(), + ["grant-pending".to_owned(), "grant-prior".to_owned()] + .into_iter() + .collect() + ); + } + + #[test] + fn managed_mcp_preparation_incomplete_journal_blocks_startup_cleanup() { + let owned = tempfile::tempdir().unwrap(); + let directory = owned.path().join("mcp-capabilities"); + std::fs::create_dir(&directory).unwrap(); + let prior = staged_provider_root_path(&directory, "grant-prior-unrecorded").unwrap(); + let pending = staged_provider_root_path(&directory, "grant-pending").unwrap(); + for root in [&prior, &pending] { + std::fs::create_dir_all(root).unwrap(); + } + std::fs::write(prior.join("usable-resource"), b"retained-resource").unwrap(); + std::fs::write( + grant_path(&directory, "grant-pending").unwrap(), + b"{\"preparation\":", + ) + .unwrap(); + assert!( + cleanup_orphaned_provider_roots(&directory).is_err(), + "incomplete ownership cannot authorize orphan cleanup" + ); + assert!(prior.join("usable-resource").is_file()); + assert!(pending.is_dir()); + } + #[cfg(target_os = "linux")] #[test] fn managed_provider_recovery_rejects_oversized_manifest_before_payload_read() { @@ -4994,9 +5346,21 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ let orphan = staged_provider_root_path(&capabilities, "grant-orphan").unwrap(); std::fs::create_dir_all(&active).unwrap(); std::fs::create_dir_all(&orphan).unwrap(); - std::fs::write( - grant_path(&capabilities, "grant-active").unwrap(), - "running grant", + let path = grant_path(&capabilities, "grant-active").unwrap(); + write_capability( + &path, + &Grant { + soul_id: SoulId::new(), + incarnation_id: Some(IncarnationId::new()), + token: "synthetic-scope".into(), + expires_at: 1, + endpoint: "http://host.docker.internal:4000".into(), + path: path.clone(), + preparation: Some(InputPreparation { + phase: InputPreparationPhase::Committed, + prior_grant_ids: vec![], + }), + }, ) .unwrap(); cleanup_orphaned_provider_roots(&capabilities).unwrap(); From ba24c7f0e4ef54e16c913b6aedc9bac0ec0201f7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 16:14:17 -0700 Subject: [PATCH 153/493] Use current provider preparation during fresh recovery --- .../freshell-agent-runtime/src/host_actor.rs | 9 + .../src/host_actor_tests.rs | 153 +++- .../src/managed_provider_launch_tests.rs | 835 +++++++++++------- .../managed-provider-launch-sandbox.py | 8 +- 4 files changed, 675 insertions(+), 330 deletions(-) diff --git a/crates/freshell-agent-runtime/src/host_actor.rs b/crates/freshell-agent-runtime/src/host_actor.rs index 8c2383c83..5fe187545 100644 --- a/crates/freshell-agent-runtime/src/host_actor.rs +++ b/crates/freshell-agent-runtime/src/host_actor.rs @@ -422,9 +422,18 @@ impl FreshAgentHostActor { if expected_profile.native_session_id.is_none() { expected_profile.native_session_id = loaded.profile.native_session_id.clone(); } + // The host validates a fresh preparation for each incarnation. + // Its grant and resource generation may change while the durable + // conversation and every other launch setting remain the same. + let current_preparation = expected_profile.provider_launch_context.take(); + if current_preparation.is_some() { + expected_profile.provider_launch_context = + loaded.profile.provider_launch_context.clone(); + } if loaded.profile != expected_profile { return Err(ActorError::NativeIdentityMismatch); } + loaded.profile.provider_launch_context = current_preparation; for command in loaded.commands.values_mut() { if matches!(command.state, CommandState::Dispatching) { command.state = CommandState::Ambiguous; diff --git a/crates/freshell-agent-runtime/src/host_actor_tests.rs b/crates/freshell-agent-runtime/src/host_actor_tests.rs index f39be5972..abc19dc67 100644 --- a/crates/freshell-agent-runtime/src/host_actor_tests.rs +++ b/crates/freshell-agent-runtime/src/host_actor_tests.rs @@ -261,6 +261,7 @@ impl FreshAgentTransport for OperationTransport { struct FakeTransport { native: String, starts: AtomicUsize, + start_profiles: std::sync::Mutex>, dispatches: AtomicUsize, decisions: AtomicUsize, stops: AtomicUsize, @@ -359,6 +360,7 @@ impl FakeTransport { Arc::new(Self { native: native.into(), starts: AtomicUsize::new(0), + start_profiles: std::sync::Mutex::new(Vec::new()), dispatches: AtomicUsize::new(0), decisions: AtomicUsize::new(0), stops: AtomicUsize::new(0), @@ -372,6 +374,7 @@ impl FakeTransport { ..Self { native: native.into(), starts: AtomicUsize::new(0), + start_profiles: std::sync::Mutex::new(Vec::new()), dispatches: AtomicUsize::new(0), decisions: AtomicUsize::new(0), stops: AtomicUsize::new(0), @@ -384,8 +387,9 @@ impl FakeTransport { #[async_trait] impl FreshAgentTransport for FakeTransport { - async fn start(&self, _profile: &FreshAgentProfile) -> Result { + async fn start(&self, profile: &FreshAgentProfile) -> Result { self.starts.fetch_add(1, Ordering::SeqCst); + self.start_profiles.lock().unwrap().push(profile.clone()); Ok(TransportStart { native_session_id: Some(self.native.clone()), }) @@ -887,6 +891,153 @@ async fn exact_native_identity_and_profile_are_reused_on_host_actor_recovery() { assert!(matches!(wrong, Err(ActorError::NativeIdentityMismatch))); } +#[tokio::test] +async fn current_provider_preparation_reopens_the_same_fresh_conversation() { + use freshell_runtime_protocol::{ + McpCapabilityReference, ProviderLaunchContext, ProviderPreparation, + }; + for provider in [ + FreshProvider::Claude, + FreshProvider::Kilroy, + FreshProvider::Codex, + FreshProvider::Opencode, + ] { + let preparation = match provider { + FreshProvider::Claude | FreshProvider::Kilroy => ProviderPreparation::Claude { + mcp_args: Vec::new(), + }, + FreshProvider::Codex => ProviderPreparation::Codex { + tui_args: Vec::new(), + sidecar_args: Vec::new(), + }, + FreshProvider::Opencode => ProviderPreparation::Opencode { + project_config: Vec::new(), + tui_config: None, + tui_source: None, + inline_config: false, + }, + }; + let dir = tempfile::tempdir().unwrap(); + let mut initial = profile(provider, "same-store", None); + initial.provider_launch_context = Some(ProviderLaunchContext { + preparation, + prepared_inputs: None, + config: Vec::new(), + mcp_capability: Some(McpCapabilityReference { + grant_id: "grant-prior".into(), + endpoint: "http://host.docker.internal:9".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }), + }); + let actor = + FreshAgentHostActor::open(dir.path(), initial, FakeTransport::new("same-native")) + .await + .unwrap(); + let mut refreshed = actor.profile().await; + drop(actor); + refreshed + .provider_launch_context + .as_mut() + .unwrap() + .mcp_capability + .as_mut() + .unwrap() + .grant_id = "grant-replacement".into(); + let transport = FakeTransport::new("same-native"); + let actor = FreshAgentHostActor::open(dir.path(), refreshed.clone(), transport.clone()) + .await + .expect("current prepared grant must not change the durable conversation identity"); + assert_eq!( + transport.start_profiles.lock().unwrap().as_slice(), + &[refreshed.clone()] + ); + assert_eq!(actor.profile().await, refreshed); + drop(actor); + // The new preparation must also survive a cold actor reopen. + let actor = FreshAgentHostActor::open( + dir.path(), + refreshed.clone(), + FakeTransport::new("same-native"), + ) + .await + .unwrap(); + assert_eq!(actor.profile().await, refreshed); + drop(actor); + let before = fs::read(dir.path().join("fresh-agent-state.json")).unwrap(); + let mut next_preparation = refreshed.clone(); + next_preparation + .provider_launch_context + .as_mut() + .unwrap() + .mcp_capability + .as_mut() + .unwrap() + .grant_id = "grant-unconfirmed".into(); + let wrong_native = FakeTransport::new("wrong-native-from-provider"); + let result = + FreshAgentHostActor::open(dir.path(), next_preparation.clone(), wrong_native.clone()) + .await; + assert!(matches!(result, Err(ActorError::NativeIdentityMismatch))); + assert_eq!( + wrong_native.start_profiles.lock().unwrap().as_slice(), + &[next_preparation] + ); + assert!( + fs::read(dir.path().join("fresh-agent-state.json")).unwrap() == before, + "failed native identity verification must preserve the accepted preparation" + ); + for changed in [ + "native", + "model", + "effort", + "cwd", + "plugins", + "store", + "runtime", + "permission", + "sandbox", + "secret-reference", + "context-absent", + ] { + let mut wrong = refreshed.clone(); + match changed { + "native" => wrong.native_session_id = Some("different-native".into()), + "model" => wrong.model = Some("different-model".into()), + "effort" => wrong.effort = Some("high".into()), + "cwd" => wrong.cwd = "/different-workspace".into(), + "plugins" => wrong.plugins = Some(vec!["/different-plugin".into()]), + "store" => wrong.provider_store_id = "different-store".into(), + "runtime" => wrong.runtime_variant = "different-runtime".into(), + "permission" => wrong.permission_mode = Some("different-permission".into()), + "sandbox" => wrong.sandbox = Some("different-sandbox".into()), + "secret-reference" => wrong.provider_secret_references.push(freshell_runtime_protocol::ProviderSecretReference { + source_path: "/synthetic/different-private-reference".into(), + profile: match wrong.provider { + FreshProvider::Claude | FreshProvider::Kilroy => freshell_runtime_protocol::ProviderSecretProfile::ClaudeOnecliEnvironment, + FreshProvider::Codex => freshell_runtime_protocol::ProviderSecretProfile::CodexOnecliEnvironment, + FreshProvider::Opencode => freshell_runtime_protocol::ProviderSecretProfile::OpencodeOnecliEnvironment, + }, + provider_relative_path: None, + }), + "context-absent" => wrong.provider_launch_context = None, + _ => unreachable!(), + } + let transport = FakeTransport::new("same-native"); + let result = FreshAgentHostActor::open(dir.path(), wrong, transport.clone()).await; + assert!( + matches!(result, Err(ActorError::NativeIdentityMismatch)), + "{changed}" + ); + assert_eq!(transport.starts.load(Ordering::SeqCst), 0, "{changed}"); + assert!( + fs::read(dir.path().join("fresh-agent-state.json")).unwrap() == before, + "{changed} must preserve the prior journal" + ); + } + } +} + #[tokio::test] async fn independent_souls_have_independent_transports_and_bounded_journals() { let root = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index cc777375d..7bfa7c0b6 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -1163,9 +1163,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ .unwrap_or_else(|_| "unchanged".into()); assert!(matches!( source_guard.as_str(), - "unchanged" | "changed-git" | "retained" + "unchanged" | "changed-git" | "retained" | "retained-fresh" )); let changed_git = source_guard == "changed-git"; + let retained_fresh = source_guard == "retained-fresh"; let flag_argument = match settings_form.as_str() { "file" if changed_git => flag_alias.to_str().unwrap().into(), "file" => "../source-home/selected-config/selected-settings.json".into(), @@ -1389,17 +1390,43 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ json!({"uid":observed.uid,"euid":observed.euid,"gid":observed.gid,"egid":observed.egid,"groups":groups}) } let client = controller.runtime_client(); - let terminal_identity = qualify_private_source_reader( - &client, - SoulId::parse(format!( + let source_provider = + if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_FLAVOR").as_deref() == Ok("kilroy") { + "kilroy" + } else { + "claude" + }; + let initial_seed = if retained_fresh { + "closed-native-fresh-create" + } else { + "closed-native-terminal-create" + }; + let initial_soul = if retained_fresh { + format!( + "soul-fresh-{source_provider}-{}", + &format!("{:x}", Sha256::digest(initial_seed.as_bytes()))[..32] + ) + } else { + format!( "soul-terminal-{}", - &format!("{:x}", Sha256::digest(b"closed-native-terminal-create"))[..32] - )) - .unwrap(), - "claude", - "closed-native-terminal-create", + &format!("{:x}", Sha256::digest(initial_seed.as_bytes()))[..32] + ) + }; + let source_identity = qualify_private_source_reader( + &client, + SoulId::parse(initial_soul).unwrap(), + if retained_fresh { + source_provider + } else { + "claude" + }, + initial_seed, &workspace, - freshell_platform::claude_inputs::NativeSourceRuntime::Terminal, + if retained_fresh { + freshell_platform::claude_inputs::NativeSourceRuntime::Sdk + } else { + freshell_platform::claude_inputs::NativeSourceRuntime::Terminal + }, ) .await; let cwd = if changed_git { @@ -1418,7 +1445,86 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ ) .unwrap(); std::env::set_var("HOME", &ambient_home); - let terminal = controller + async fn consumed(path: &Path) -> Option { + tokio::time::timeout(Duration::from_secs(45), async { + loop { + if let Ok(bytes) = std::fs::read(path) { + if let Ok(value) = serde_json::from_slice(&bytes) { + break value; + } + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await + .ok() + } + let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); + let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":source_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"refreshStageAbsent":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"diskResource":hash(&disk_resource),"dependency":hash(b"closed-claude-dependency")}); + let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); + let mut expected_imports = serde_json::Map::new(); + expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); + expected_imports.insert( + "rule-memory.md".into(), + json!(hash(b"closed-selected-rule-memory\n")), + ); + for name in [ + "relative-memory.md", + "home-memory.md", + "absolute-memory.md", + "spaced memory.md", + "nested-memory.md", + ] { + expected_imports.insert( + name.into(), + json!(hash(std::fs::read(plugin.join(name)).unwrap().as_slice())), + ); + } + let expected_imports = Value::Object(expected_imports); + let instructions_events = instructions_receipt.with_extension("json.events"); + fn instruction_events(path: &Path) -> Vec { + std::fs::read_to_string(path) + .unwrap_or_default() + .lines() + .map(|line| serde_json::from_str(line).expect("complete native instruction event")) + .collect() + } + async fn imported_graph(path: &Path, expected: &Value) { + let result = tokio::time::timeout(Duration::from_secs(20), async { + loop { + let mut loaded = serde_json::Map::new(); + for event in instruction_events(path) { + assert_ne!( + event["ignored"], + json!(true), + "native code spans/fences/quoted tokens must remain unselected" + ); + if let Some(name) = event["selected"]["name"].as_str() { + loaded.insert(name.into(), event["selected"]["digest"].clone()); + } + } + if Value::Object(loaded.clone()) == *expected { + break; + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await; + assert!( + result.is_ok(), + "native must load the complete original-form Markdown import graph; observed {:?}", + instruction_events(path) + ); + } + let qualified_digest = hash( + serde_json::to_vec(&json!({"startup":expected,"nativeImports":expected_imports,"terminalSettingSources":"user","disabledProjectConsumed":false,"freshSettingSources":"user,project,local"})) + .unwrap() + .as_slice(), + ); + let terminal = if retained_fresh { + None + } else { + Some(controller .launch(ManagedTerminalLaunch { spec: freshell_platform::spawn::SpawnSpec { program: "claude".into(), @@ -1457,9 +1563,12 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ view_pane_id: None, create_request_id: Some("closed-native-terminal-create".into()), }) - .await; + .await) + }; if changed_git { - let error = terminal.expect_err("actual final source mismatch must block terminal startup"); + let error = terminal + .unwrap() + .expect_err("actual final source mismatch must block terminal startup"); println!( "provider_source_guard_diagnostic {}", json!({"selectionChanged":error.contains("Claude native source selection changed"),"commonDir":error.contains("git common dir"),"hostUnreachable":error.contains("host") }) @@ -1549,323 +1658,253 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ control_task.abort(); assert!(web_task.await.unwrap_err().is_cancelled()); assert!(control_task.await.unwrap_err().is_cancelled()); - return; - } - let terminal = terminal.expect("complete native Claude terminal startup"); - async fn consumed(path: &Path) -> Option { - tokio::time::timeout(Duration::from_secs(45), async { - loop { - if let Ok(bytes) = std::fs::read(path) { - if let Ok(value) = serde_json::from_slice(&bytes) { - break value; - } - } - tokio::time::sleep(Duration::from_millis(30)).await; - } - }) - .await - .ok() - } - let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":terminal_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"refreshStageAbsent":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"diskResource":hash(&disk_resource),"dependency":hash(b"closed-claude-dependency")}); - let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); - let mut expected_imports = serde_json::Map::new(); - expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); - expected_imports.insert( - "rule-memory.md".into(), - json!(hash(b"closed-selected-rule-memory\n")), - ); - for name in [ - "relative-memory.md", - "home-memory.md", - "absolute-memory.md", - "spaced memory.md", - "nested-memory.md", - ] { - expected_imports.insert( - name.into(), - json!(hash(std::fs::read(plugin.join(name)).unwrap().as_slice())), - ); - } - let expected_imports = Value::Object(expected_imports); - let instructions_events = instructions_receipt.with_extension("json.events"); - fn instruction_events(path: &Path) -> Vec { - std::fs::read_to_string(path) - .unwrap_or_default() - .lines() - .map(|line| serde_json::from_str(line).expect("complete native instruction event")) - .collect() - } - async fn imported_graph(path: &Path, expected: &Value) { - let result = tokio::time::timeout(Duration::from_secs(20), async { - loop { - let mut loaded = serde_json::Map::new(); - for event in instruction_events(path) { - assert_ne!( - event["ignored"], - json!(true), - "native code spans/fences/quoted tokens must remain unselected" - ); - if let Some(name) = event["selected"]["name"].as_str() { - loaded.insert(name.into(), event["selected"]["digest"].clone()); - } - } - if Value::Object(loaded.clone()) == *expected { - break; - } - tokio::time::sleep(Duration::from_millis(30)).await; - } - }) - .await; - assert!( - result.is_ok(), - "native must load the complete original-form Markdown import graph; observed {:?}", - instruction_events(path) - ); - } - let qualified_digest = hash( - serde_json::to_vec(&json!({"startup":expected,"nativeImports":expected_imports,"terminalSettingSources":"user","disabledProjectConsumed":false,"freshSettingSources":"user,project,local"})) - .unwrap() - .as_slice(), - ); - let observed = consumed(&receipt).await; - if observed.is_none() { - let output = client - .read_output( - SoulId::parse(terminal.soul_id.clone()).unwrap(), - 0, - 64 * 1024, - ) - .await - .unwrap(); - let text = output - .frames - .iter() - .map(|frame| frame.data.as_str()) - .collect::(); - let text = crate::ai_title::strip_ansi(&text); - let classes = [ - "ENOENT", - "EACCES", - "trust", - "Trust", - "onboarding", - "Sign in", - "API key", - "plugin", - "Plugin", - "settings", - "MCP", - "Permission denied", - "text style", - "theme", - "Theme", - "Welcome", - "Let's", - "Enter", - "Continue", - "terms", - "Token", - "Authentication", - "Setup", - "select", - "Select", - ] - .into_iter() - .filter(|class| text.contains(class)) - .collect::>(); - println!( - "provider_launch_diagnostic {}", - json!({"provider":"claude","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes}) - ); - } - assert_eq!( - observed.expect("native Claude selected plugin must execute its startup hook"), - expected - ); - if settings_form == "owned-last" { - assert!( - !flag_receipt.exists(), - "ignored user settings hooks must not execute" - ); - } else { - assert_eq!( - consumed(&flag_receipt) - .await - .expect("native must execute the explicitly selected last flag settings hook"), - json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) - ); - } - assert_eq!(std::fs::read(&flag_settings).unwrap(), flag_bytes); - assert_eq!( - std::fs::read_to_string(selected_config.join("rules/nested/policy.md")).unwrap(), - rule - ); - assert_eq!(std::fs::read_link(&flag_alias).unwrap(), flag_settings); - let native_import = consumed(&instructions_receipt).await; - if native_import.is_none() { - println!( - "provider_memory_diagnostic {}", - json!({"route":"terminal","hookEvents":std::fs::read_to_string(instructions_receipt.with_extension("json.events")).unwrap_or_default().lines().filter_map(|line|serde_json::from_str::(line).ok()).collect::>() }) - ); - } - assert_eq!( - native_import.expect("native Claude must load selected memory import"), - json!({"nativeImport":true,"digest":hash(&imported_bytes)}) - ); - imported_graph(&instructions_events, &expected_imports).await; - println!( - "provider_launch_consumer {}", - json!({"provider":"claude","route":"terminal","digest":qualified_digest}) - ); - if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD").as_deref() == Ok("retained") { - let soul = SoulId::parse(terminal.soul_id.clone()).unwrap(); - let prior = registry.active_handle_for_soul(soul.clone()).await.unwrap(); - let prior_generation = prior - .terminal() - .unwrap() - .provider_launch_context - .as_ref() - .unwrap() - .prepared_inputs - .as_ref() - .unwrap() - .generation_id - .clone(); - let real = DockerEngineBackend::new(fixture.join("broker.sock")); - real.force_stop(&prior) - .await - .expect("stop only the registry-owned disposable runtime"); - assert!(real.verify_empty(&prior).await.unwrap()); - let hidden_home = fixture.join("source-home-unavailable"); - std::fs::rename(&home, &hidden_home).unwrap(); - for path in [&receipt, &instructions_receipt, &instructions_events] { - std::fs::remove_file(path).unwrap(); - } - if settings_form != "owned-last" { - std::fs::remove_file(&flag_receipt).unwrap(); - } - let recovered = tokio::time::timeout( - Duration::from_secs(120), - client.recover(soul.clone(), RecoveryTrigger::HostUnreachable), - ) - .await - .expect("finite actual retained recovery") - .expect("authenticated supervisor recovery reply"); - println!( - "provider_recovery_outcome {}", - json!({"outcome":recovered.outcome,"priorGeneration":prior_generation,"probe":recovered.probe.as_ref().map(|probe|match probe { RecoveryProbe::ResumeReady {..} => "native_resume", RecoveryProbe::PristineSeedReady {..} => "pristine_seed", _ => "other" })}) - ); - if let Some(RecoveryProbe::Blocked { - reason, retry_hint, .. - }) = &recovered.probe - { - let repair = retry_hint.repair.as_deref().unwrap_or(""); - let classes = [ - "Claude", - "source", - "selection", - "changed", - "observation", - "identity", - "frozen", - "ordinary", - "manifest", - "metadata", - "HOME", - "scope", - "grant", - "missing", - "unavailable", - "reprepare", - "budget", - "native", - "store", - "provider", - "callback", - "checkpoint", - "permission", - "configuration", - "revision", + return; + } + if let Some(terminal) = terminal { + let terminal = terminal.expect("complete native Claude terminal startup"); + let observed = consumed(&receipt).await; + if observed.is_none() { + let output = client + .read_output( + SoulId::parse(terminal.soul_id.clone()).unwrap(), + 0, + 64 * 1024, + ) + .await + .unwrap(); + let text = output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(); + let text = crate::ai_title::strip_ansi(&text); + let classes = [ + "ENOENT", + "EACCES", + "trust", + "Trust", + "onboarding", + "Sign in", + "API key", + "plugin", + "Plugin", + "settings", + "MCP", + "Permission denied", + "text style", + "theme", + "Theme", + "Welcome", + "Let's", + "Enter", + "Continue", + "terms", + "Token", + "Authentication", + "Setup", + "select", + "Select", ] .into_iter() - .filter(|class| repair.contains(class)) + .filter(|class| text.contains(class)) .collect::>(); println!( - "provider_recovery_diagnostic {}", - json!({"reason":reason,"repairClasses":classes,"repairBytes":repair.len(),"callbackFailureClasses":failure_classes.lock().unwrap().clone()}) + "provider_launch_diagnostic {}", + json!({"provider":"claude","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes}) ); } - assert_eq!(recovered.outcome, RecoveryOutcome::Replaced, "actual retained callback must replace the stopped owned runtime after original ordinary inputs disappear"); - let current = registry.active_handle_for_soul(soul).await.unwrap(); - let generation = ¤t - .terminal() - .unwrap() - .provider_launch_context - .as_ref() - .unwrap() - .prepared_inputs - .as_ref() - .unwrap() - .generation_id; - assert_ne!(generation, &prior_generation); - assert_eq!(current.provider_volume_name(), prior.provider_volume_name()); assert_eq!( - consumed(&receipt) - .await - .expect("recovered real native startup hook"), + observed.expect("native Claude selected plugin must execute its startup hook"), expected ); - if settings_form != "owned-last" { + if settings_form == "owned-last" { + assert!( + !flag_receipt.exists(), + "ignored user settings hooks must not execute" + ); + } else { assert_eq!( consumed(&flag_receipt) .await - .expect("recovered explicitly selected settings hook"), + .expect("native must execute the explicitly selected last flag settings hook"), json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) ); } + assert_eq!(std::fs::read(&flag_settings).unwrap(), flag_bytes); + assert_eq!( + std::fs::read_to_string(selected_config.join("rules/nested/policy.md")).unwrap(), + rule + ); + assert_eq!(std::fs::read_link(&flag_alias).unwrap(), flag_settings); + let native_import = consumed(&instructions_receipt).await; + if native_import.is_none() { + println!( + "provider_memory_diagnostic {}", + json!({"route":"terminal","hookEvents":std::fs::read_to_string(instructions_receipt.with_extension("json.events")).unwrap_or_default().lines().filter_map(|line|serde_json::from_str::(line).ok()).collect::>() }) + ); + } + assert_eq!( + native_import.expect("native Claude must load selected memory import"), + json!({"nativeImport":true,"digest":hash(&imported_bytes)}) + ); imported_graph(&instructions_events, &expected_imports).await; - assert!(!home.exists()); println!( - "provider_recovery_consumer {}", - json!({"provider":"claude","route":"terminal","digest":qualified_digest,"sameOwnedHome":true,"priorGeneration":prior_generation,"generationId":generation,"originalOrdinaryUnavailable":true,"outcome":recovered.outcome}) + "provider_launch_consumer {}", + json!({"provider":"claude","route":"terminal","digest":qualified_digest}) ); - std::fs::rename(&hidden_home, &home).unwrap(); - } - client - .stop(SoulId::parse(terminal.soul_id).unwrap()) - .await - .unwrap(); - assert!(instruction_events(&instructions_events) - .iter() - .all(|event| event["ignored"] != json!(true))); - assert!( - instruction_events(&instructions_events) + if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD").as_deref() + == Ok("retained") + { + let soul = SoulId::parse(terminal.soul_id.clone()).unwrap(); + let prior = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + let prior_generation = prior + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id + .clone(); + let real = DockerEngineBackend::new(fixture.join("broker.sock")); + real.force_stop(&prior) + .await + .expect("stop only the registry-owned disposable runtime"); + assert!(real.verify_empty(&prior).await.unwrap()); + let hidden_home = fixture.join("source-home-unavailable"); + std::fs::rename(&home, &hidden_home).unwrap(); + for path in [&receipt, &instructions_receipt, &instructions_events] { + std::fs::remove_file(path).unwrap(); + } + if settings_form != "owned-last" { + std::fs::remove_file(&flag_receipt).unwrap(); + } + let recovered = tokio::time::timeout( + Duration::from_secs(120), + client.recover(soul.clone(), RecoveryTrigger::HostUnreachable), + ) + .await + .expect("finite actual retained recovery") + .expect("authenticated supervisor recovery reply"); + println!( + "provider_recovery_outcome {}", + json!({"outcome":recovered.outcome,"priorGeneration":prior_generation,"probe":recovered.probe.as_ref().map(|probe|match probe { RecoveryProbe::ResumeReady {..} => "native_resume", RecoveryProbe::PristineSeedReady {..} => "pristine_seed", _ => "other" })}) + ); + if let Some(RecoveryProbe::Blocked { + reason, retry_hint, .. + }) = &recovered.probe + { + let repair = retry_hint.repair.as_deref().unwrap_or(""); + let classes = [ + "Claude", + "source", + "selection", + "changed", + "observation", + "identity", + "frozen", + "ordinary", + "manifest", + "metadata", + "HOME", + "scope", + "grant", + "missing", + "unavailable", + "reprepare", + "budget", + "native", + "store", + "provider", + "callback", + "checkpoint", + "permission", + "configuration", + "revision", + ] + .into_iter() + .filter(|class| repair.contains(class)) + .collect::>(); + println!( + "provider_recovery_diagnostic {}", + json!({"reason":reason,"repairClasses":classes,"repairBytes":repair.len(),"callbackFailureClasses":failure_classes.lock().unwrap().clone()}) + ); + } + assert_eq!(recovered.outcome, RecoveryOutcome::Replaced, "actual retained callback must replace the stopped owned runtime after original ordinary inputs disappear"); + let current = registry.active_handle_for_soul(soul).await.unwrap(); + let generation = ¤t + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id; + assert_ne!(generation, &prior_generation); + assert_eq!(current.provider_volume_name(), prior.provider_volume_name()); + assert_eq!( + consumed(&receipt) + .await + .expect("recovered real native startup hook"), + expected + ); + if settings_form != "owned-last" { + assert_eq!( + consumed(&flag_receipt) + .await + .expect("recovered explicitly selected settings hook"), + json!({"flag":true,"resource":hash(b"closed-claude-resource"),"dependency":hash(b"closed-claude-dependency")}) + ); + } + imported_graph(&instructions_events, &expected_imports).await; + assert!(!home.exists()); + println!( + "provider_recovery_consumer {}", + json!({"provider":"claude","route":"terminal","digest":qualified_digest,"sameOwnedHome":true,"priorGeneration":prior_generation,"generationId":generation,"originalOrdinaryUnavailable":true,"outcome":recovered.outcome}) + ); + std::fs::rename(&hidden_home, &home).unwrap(); + } + client + .stop(SoulId::parse(terminal.soul_id).unwrap()) + .await + .unwrap(); + assert!(instruction_events(&instructions_events) .iter() - .all(|event| event["project"] != json!(true)), - "the real native terminal must leave project memory unselected" - ); - assert_eq!( - std::fs::read_to_string(&disabled_project_memory).unwrap(), - "@missing-disabled-project.md\n" - ); - if source_guard == "retained" { - // This closed scenario spends its two runtime attempts on the original - // terminal and its replacement. The unchanged scenario below retains - // its independent terminal and fresh SDK coverage. - web_task.abort(); - control_task.abort(); - assert!(web_task.await.unwrap_err().is_cancelled()); - assert!(control_task.await.unwrap_err().is_cancelled()); - return; + .all(|event| event["ignored"] != json!(true))); + assert!( + instruction_events(&instructions_events) + .iter() + .all(|event| event["project"] != json!(true)), + "the real native terminal must leave project memory unselected" + ); + assert_eq!( + std::fs::read_to_string(&disabled_project_memory).unwrap(), + "@missing-disabled-project.md\n" + ); + if source_guard == "retained" { + // This closed scenario spends its two runtime attempts on the original + // terminal and its replacement. The unchanged scenario below retains + // its independent terminal and fresh SDK coverage. + web_task.abort(); + control_task.abort(); + assert!(web_task.await.unwrap_err().is_cancelled()); + assert!(control_task.await.unwrap_err().is_cancelled()); + return; + } } // Fresh SDK exposes the existing all-source default. Remove only these // terminal-specific owned decoys before its independent launch. std::fs::remove_file(&disabled_project_memory).unwrap(); std::fs::remove_file(&disabled_project_settings).unwrap(); - std::fs::remove_file(&receipt).unwrap(); - std::fs::remove_file(&instructions_receipt).unwrap(); - std::fs::remove_file(&instructions_events).unwrap(); + if !retained_fresh { + std::fs::remove_file(&receipt).unwrap(); + std::fs::remove_file(&instructions_receipt).unwrap(); + std::fs::remove_file(&instructions_events).unwrap(); + } // Native project command discovery stops at this repository root. A // dangling ancestor command root is deliberately outside that selection. std::fs::create_dir_all(fixture.join(".claude")).unwrap(); @@ -1923,21 +1962,25 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ } else { "claude" }; - let fresh_identity = qualify_private_source_reader( - &client, - SoulId::parse(format!( - "soul-fresh-{source_provider}-{}", - &format!("{:x}", Sha256::digest(b"closed-native-fresh-create"))[..32] - )) - .unwrap(), - source_provider, - "closed-native-fresh-create", - &workspace, - freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, - ) - .await; + let fresh_identity = if retained_fresh { + source_identity.clone() + } else { + qualify_private_source_reader( + &client, + SoulId::parse(format!( + "soul-fresh-{source_provider}-{}", + &format!("{:x}", Sha256::digest(b"closed-native-fresh-create"))[..32] + )) + .unwrap(), + source_provider, + "closed-native-fresh-create", + &workspace, + freshell_platform::claude_inputs::NativeSourceRuntime::Sdk, + ) + .await + }; assert_eq!( - fresh_identity, terminal_identity, + fresh_identity, source_identity, "both actual source readers must match the real native consumer identity" ); Arc::clone(&proxy).dispatch(HostedFreshAgentCommand::Create(serde_json::from_value(json!({"requestId":"closed-native-fresh-create","sessionType":flavor,"provider":"claude","cwd":cwd,"plugins":[plugin.to_str().unwrap()]})).unwrap())).await; @@ -1946,10 +1989,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ let frame: Value = serde_json::from_str(&events.recv().await.unwrap()).unwrap(); if frame["type"] == "freshAgent.create.failed" { println!("provider_launch_failure_classes {}", json!({"provider":"claude","flavor":flavor,"classes":failure_classes.lock().unwrap().clone()})); - panic!( - "fresh Claude startup failed classification: {}", - frame["message"] - ); + panic!("fresh Claude startup failed; only the safe failure classes are reported"); } if frame["type"] == "freshAgent.created" { break; @@ -1988,16 +2028,157 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ std::fs::read_to_string(plugin.join("scripts/consume.cjs")).unwrap(), consumer ); + if retained_fresh { + let soul = SoulId::parse(format!( + "soul-fresh-{source_provider}-{}", + &format!("{:x}", Sha256::digest(b"closed-native-fresh-create"))[..32] + )) + .unwrap(); + let prior = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + let prior_generation = prior + .fresh_agent() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id + .clone(); + let real = DockerEngineBackend::new(fixture.join("broker.sock")); + real.force_stop(&prior) + .await + .expect("stop only the registry-owned disposable SDK runtime"); + assert!(real.verify_empty(&prior).await.unwrap()); + let hidden_home = fixture.join("source-home-unavailable"); + std::fs::rename(&home, &hidden_home).unwrap(); + for path in [ + &receipt, + &instructions_receipt, + &instructions_events, + &project_receipt, + ] { + std::fs::remove_file(path).unwrap(); + } + let recovered = tokio::time::timeout( + Duration::from_secs(120), + client.recover(soul.clone(), RecoveryTrigger::HostUnreachable), + ) + .await + .expect("finite actual retained SDK recovery") + .expect("authenticated supervisor recovery reply"); + println!( + "provider_recovery_outcome {}", + json!({"outcome":recovered.outcome,"priorGeneration":prior_generation,"probe":recovered.probe.as_ref().map(|probe|match probe {RecoveryProbe::ResumeReady {..} => "native_resume", RecoveryProbe::PristineSeedReady {..} => "pristine_seed", _ => "other"}),"callbackFailureClasses":failure_classes.lock().unwrap().clone()}) + ); + if let Some(RecoveryProbe::Blocked { + reason, retry_hint, .. + }) = &recovered.probe + { + let repair = retry_hint.repair.as_deref().unwrap_or(""); + let classes = [ + "Claude", + "source", + "selection", + "changed", + "observation", + "identity", + "frozen", + "ordinary", + "manifest", + "metadata", + "HOME", + "scope", + "grant", + "missing", + "unavailable", + "reprepare", + "budget", + "native", + "store", + "provider", + "callback", + "checkpoint", + "permission", + "configuration", + "revision", + "healthy", + "host", + "worker", + "activation", + "ready", + "timed out", + ] + .into_iter() + .filter(|class| repair.contains(class)) + .collect::>(); + println!( + "provider_recovery_diagnostic {}", + json!({"reason":reason,"repairClasses":classes,"repairBytes":repair.len()}) + ); + } + assert_eq!(recovered.outcome, RecoveryOutcome::Replaced, "actual retained SDK callback must replace the stopped owned runtime after original ordinary inputs disappear"); + assert!( + matches!( + recovered.probe, + Some(RecoveryProbe::PristineSeedReady { .. }) + ), + "empty SDK fixture must use the actual pristine seed recovery path" + ); + let current = registry.active_handle_for_soul(soul).await.unwrap(); + let generation = ¤t + .fresh_agent() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id; + assert_ne!(generation, &prior_generation); + assert_eq!(current.provider_volume_name(), prior.provider_volume_name()); + assert_eq!( + consumed(&receipt) + .await + .expect("recovered genuine SDK startup hook"), + expected + ); + assert_eq!( + consumed(&instructions_receipt) + .await + .expect("recovered SDK native memory import"), + json!({"nativeImport":true,"digest":hash(&imported_bytes)}) + ); + imported_graph(&instructions_events, &expected_imports).await; + assert_eq!( + consumed(&project_receipt) + .await + .expect("recovered SDK current project command hook"), + json!({"resource":hash(b"closed-project-command-resource"),"dependency":hash(b"closed-project-command-dependency")}) + ); + assert!(!home.exists()); + println!( + "provider_recovery_consumer {}", + json!({"provider":"claude","route":"fresh","digest":qualified_digest,"sameOwnedHome":true,"priorGeneration":prior_generation,"generationId":generation,"originalOrdinaryUnavailable":true,"outcome":recovered.outcome}) + ); + std::fs::rename(&hidden_home, &home).unwrap(); + } let catalog_project = fixture.join("catalog-project"); let catalog_home = fixture.join("catalog-home"); std::fs::create_dir_all(catalog_project.join(".git")).unwrap(); std::fs::create_dir_all(&catalog_home).unwrap(); + let mut catalog_owners = 0; for view in client.inventory().await.unwrap() { - if view.desired_state == DesiredState::Running { + if view.desired_state == DesiredState::Running && view.launch_state == LaunchState::Running + { let handle = registry .active_handle_for_soul(view.soul_id.clone()) .await .unwrap(); + assert_eq!(handle.incarnation_id(), &view.incarnation_id); + catalog_owners += 1; let inputs = handle .fresh_agent() .unwrap() @@ -2028,6 +2209,10 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ client.stop(view.soul_id).await.unwrap(); } } + assert_eq!( + catalog_owners, 1, + "catalog must consume exactly the active SDK preparation" + ); let catalog_script = fixture.join("catalog-probe.mjs"); std::fs::write(&catalog_script, r#"import {query} from '/opt/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk/sdk.mjs'; const abort = new AbortController(); diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 8ff05d9cc..37821bb64 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -35,7 +35,7 @@ def main(): parser.add_argument("--provider",choices=["opencode","codex","claude","amplifier"],default="opencode") parser.add_argument("--claude-flavor",choices=["freshclaude","kilroy"],default="freshclaude") parser.add_argument("--claude-settings-form",choices=["file","inline","owned-last"],default="file") - parser.add_argument("--claude-source-guard",choices=["unchanged","changed-git","retained"],default="unchanged") + parser.add_argument("--claude-source-guard",choices=["unchanged","changed-git","retained","retained-fresh"],default="unchanged") args=parser.parse_args() if args.provider!="claude" and args.claude_flavor!="freshclaude": parser.error("the finite Claude flavor requires --provider claude") @@ -211,16 +211,16 @@ def create(name,*arguments): receipt["claudeSourceGuard"]=args.claude_source_guard receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] receipt["candidateObservers"]=[json.loads(line.split("provider_candidate_observer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_candidate_observer " in line] - if args.claude_source_guard=="retained": + if args.claude_source_guard in {"retained","retained-fresh"}: receipt["recoveryConsumers"]=[json.loads(line.split("provider_recovery_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_recovery_consumer " in line] recovered=receipt["recoveryConsumers"] route_mismatch_recovery=len(recovered)!=1 or any(value.get("outcome")!="replaced" or value.get("sameOwnedHome") is not True or value.get("originalOrdinaryUnavailable") is not True or value.get("generationId")==value.get("priorGeneration") for value in recovered) print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) - if args.claude_source_guard=="retained":expected_routes={("claude","terminal")} + if args.claude_source_guard in {"retained","retained-fresh"}:expected_routes={("claude","fresh" if args.claude_source_guard=="retained-fresh" else "terminal")} observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} route_mismatch=len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes - if args.claude_source_guard=="retained":route_mismatch=route_mismatch or route_mismatch_recovery + if args.claude_source_guard in {"retained","retained-fresh"}:route_mismatch=route_mismatch or route_mismatch_recovery if args.claude_source_guard=="changed-git": receipt["blockedRoutes"]=[json.loads(line.split("provider_source_guard ",1)[1]) for line in outcome.stdout.splitlines() if "provider_source_guard " in line] blocked={(item.get("provider"),item.get("route")) for item in receipt["blockedRoutes"]} From e5ac8d39fc91b075e870cab52537694a118b3a96 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 16:40:28 -0700 Subject: [PATCH 154/493] Remove incomplete provider projection metadata on publication failure --- crates/freshell-session-host/src/main.rs | 132 +++++++++++++++++++++-- 1 file changed, 125 insertions(+), 7 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index d7ceccee9..58cc83707 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1941,11 +1941,8 @@ fn write_projection_manifest( provider_home: &Path, entries: &std::collections::BTreeSet, ) -> Result { + use std::io::Write; let manifest = provider_home.join(PROVIDER_PROJECTION_MANIFEST); - let temporary = provider_home.join(format!( - "{PROVIDER_PROJECTION_MANIFEST}.tmp-{}", - std::process::id() - )); let bytes = freshell_runtime_protocol::bounded_json_bytes(entries, MAX_PROJECTION_METADATA_BYTES) .map_err(|_| { @@ -1959,9 +1956,27 @@ fn write_projection_manifest( ); "provider projection metadata exceeds 64 MiB; retain inputs and repair".to_owned() })?; - std::fs::write(&temporary, bytes).map_err(|error| error.to_string())?; - set_mode(&temporary, 0o600)?; - std::fs::rename(&temporary, &manifest).map_err(|error| error.to_string())?; + let published: Result<(), std::io::Error> = (|| { + // Keep incomplete metadata private and remove it on every write or + // rename failure. Only the complete file replaces the prior manifest. + let mut temporary = tempfile::Builder::new() + .prefix(".freshell-config-projection-") + .tempfile_in(provider_home)?; + temporary.write_all(&bytes)?; + temporary.as_file().sync_all()?; + temporary.persist(&manifest).map_err(|error| error.error)?; + #[cfg(unix)] + std::fs::File::open(provider_home)?.sync_all()?; + Ok(()) + })(); + published.map_err(|cause| { + tracing::warn!(event="provider.inputs.projection_metadata.blocked",stage="session_host", + operation="publish",relative_path=PROVIDER_PROJECTION_MANIFEST,cause=%cause, + "Provider projection metadata publication failed; retain inputs and repair"); + format!( + "provider projection metadata publication failed: {cause}; retain inputs and repair" + ) + })?; Ok(manifest) } @@ -4156,6 +4171,109 @@ mod tests { ); } + #[cfg(target_os = "linux")] + #[test] + fn provider_config_projection_metadata_writer_cleans_failed_publication() { + const CHILD: &str = "FRESHELL_TEST_PROJECTION_WRITE_FAILURE_CHILD"; + const TEST: &str = + "tests::provider_config_projection_metadata_writer_cleans_failed_publication"; + if std::env::var_os(CHILD).is_none() { + let mut failed = Vec::new(); + for case in ["write", "rename"] { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", TEST, "--nocapture"]) + .env(CHILD, case) + .output() + .unwrap(); + assert!(String::from_utf8_lossy(&output.stdout) + .contains("projection_write_failure_fixture_ready")); + eprintln!( + "projection_write_failure_child_result {}", + serde_json::json!({ + "case":case, + "success":output.status.success(), + "temporaryMetadataRemaining":String::from_utf8_lossy(&output.stderr) + .contains("left temporary metadata") + }) + ); + if !output.status.success() { + failed.push(case); + } + } + assert!(failed.is_empty(), "projection metadata failure must preserve prior metadata and remove owned temporary files: {failed:?}"); + return; + } + use std::os::unix::fs::PermissionsExt; + let home = tempfile::tempdir().unwrap(); + let accepted = ["skills/prior".to_owned()].into_iter().collect(); + let manifest = write_projection_manifest(home.path(), &accepted).unwrap(); + let before = std::fs::read(&manifest).unwrap(); + let incoming = [ + "skills/new-resource".to_owned(), + "skills/next-resource".to_owned(), + ] + .into_iter() + .collect(); + println!("projection_write_failure_fixture_ready"); + if std::env::var(CHILD).unwrap() == "write" { + let mut previous = libc::rlimit { + rlim_cur: 0, + rlim_max: 0, + }; + assert_eq!( + unsafe { libc::getrlimit(libc::RLIMIT_FSIZE, &mut previous) }, + 0 + ); + let restricted = libc::rlimit { + rlim_cur: 16, + rlim_max: previous.rlim_max, + }; + let old_signal = unsafe { libc::signal(libc::SIGXFSZ, libc::SIG_IGN) }; + assert_eq!( + unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &restricted) }, + 0 + ); + let result = write_projection_manifest(home.path(), &incoming); + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &previous) }, 0); + unsafe { + libc::signal(libc::SIGXFSZ, old_signal); + } + assert!(result.is_err()); + assert!(std::fs::read(&manifest).unwrap() == before); + assert_eq!( + std::fs::read_dir(home.path()).unwrap().count(), + 1, + "failed publication left temporary metadata" + ); + } + + // A rename failure must clean the fully written temporary too, without + // replacing the occupied destination or its provider-owned contents. + std::fs::remove_file(&manifest).unwrap(); + std::fs::create_dir(&manifest).unwrap(); + let sibling = manifest.join("provider-owned"); + std::fs::write(&sibling, b"owned-metadata-sibling-nonce").unwrap(); + assert!(write_projection_manifest(home.path(), &incoming).is_err()); + assert_eq!( + std::fs::read_dir(home.path()).unwrap().count(), + 1, + "failed rename left temporary metadata" + ); + assert_eq!( + std::fs::read(&sibling).unwrap(), + b"owned-metadata-sibling-nonce" + ); + std::fs::remove_file(sibling).unwrap(); + std::fs::remove_dir(&manifest).unwrap(); + write_projection_manifest(home.path(), &incoming).unwrap(); + assert_eq!(read_projection_manifest(home.path()).unwrap(), incoming); + assert_eq!( + std::fs::metadata(&manifest).unwrap().permissions().mode() & 0o777, + 0o600 + ); + assert_eq!(std::fs::read_dir(home.path()).unwrap().count(), 1); + } + #[cfg(unix)] #[test] fn provider_config_projection_metadata_rejects_oversize_before_read_and_publication() { From 68d35441849b139979f3268cde2810165179462f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 17:15:04 -0700 Subject: [PATCH 155/493] Retain bound and unconfirmed provider preparations on abort --- .../src/managed_mcp_capability.rs | 350 +++++++++++++++++- 1 file changed, 334 insertions(+), 16 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 50e3f85f8..02cb665f2 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -2641,23 +2641,69 @@ impl CapabilityStore { soul_id: &SoulId, directory: &Path, ) -> Result<(), String> { - if let Some(grant) = self.grants.lock().unwrap().get(grant_id) { - if &grant.soul_id != soul_id { - return Err("provider input preparation belongs to another soul".into()); + let result = (|| { + let mut grants = self.grants.lock().unwrap(); + let pending = |owner: &SoulId, + incarnation: Option<&IncarnationId>, + preparation: Option<&InputPreparation>| { + if owner != soul_id { + return Err("provider input preparation belongs to another soul".into()); + } + if incarnation.is_some() + || preparation.is_none_or(|p| p.phase != InputPreparationPhase::Pending) + { + return Err("provider input preparation is bound or unconfirmed; retain inputs and reconcile before cleanup".into()); + } + Ok::<(), String>(()) + }; + if let Some(grant) = grants.get(grant_id) { + pending( + &grant.soul_id, + grant.incarnation_id.as_ref(), + grant.preparation.as_ref(), + )?; } - if grant - .preparation - .as_ref() - .is_some_and(|p| p.phase == InputPreparationPhase::Committed) - { - return Err( - "provider input preparation is committed; reconcile before cleanup".into(), - ); + let path = grant_path(directory, grant_id)?; + match std::fs::symlink_metadata(&path) { + Ok(_) => { + let stored = read_stored_capability(directory, grant_id)?; + pending( + &stored.soul_id, + stored.incarnation_id.as_ref(), + stored.preparation.as_ref(), + )?; + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + let stage = staged_provider_root_path(directory, grant_id)?; + if !grants.contains_key(grant_id) + && std::fs::symlink_metadata(stage) + .is_err_and(|error| error.kind() == std::io::ErrorKind::NotFound) + { + return Ok(()); + } + return Err( + "provider input preparation ownership journal is missing; retain inputs and reconcile before cleanup".into(), + ); + } + Err(_) => { + return Err( + "provider input preparation ownership journal unavailable; retain inputs and reconcile before cleanup".into(), + ); + } } - } - self.revoke(grant_id, directory)?; - tracing::info!(event="provider.inputs.preparation.aborted",grant_id,soul_id=%soul_id); - Ok(()) + // No activation may bind this pending generation between the ownership + // check and its removal. These private filesystem operations await no IPC. + Self::remove_preparation_files(grant_id, directory)?; + grants.remove(grant_id); + tracing::info!(event="provider.inputs.preparation.aborted",grant_id,soul_id=%soul_id); + Ok(()) + })(); + result.map_err(|reason: String| { + tracing::warn!(event="provider.inputs.preparation.abort_blocked",stage="controller", + operation="abort",grant_id,soul_id=%soul_id,cause=%reason, + "Provider preparation abort unconfirmed; retain inputs and reconcile"); + reason + }) } fn activate( @@ -2701,6 +2747,12 @@ impl CapabilityStore { } fn revoke(&self, grant_id: &str, directory: &Path) -> Result<(), String> { + Self::remove_preparation_files(grant_id, directory)?; + self.grants.lock().unwrap().remove(grant_id); + Ok(()) + } + + fn remove_preparation_files(grant_id: &str, directory: &Path) -> Result<(), String> { let path = grant_path(directory, grant_id)?; let staged = staged_provider_root_path(directory, grant_id)?; match std::fs::remove_file(path) { @@ -2713,7 +2765,6 @@ impl CapabilityStore { Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), Err(error) => Err(error.to_string()), }?; - self.grants.lock().unwrap().remove(grant_id); Ok(()) } @@ -5680,6 +5731,273 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .is_err()); } + #[test] + fn managed_mcp_generation_abort_preserves_bound_inputs_after_warm_and_cold_delivery() { + let control = tempfile::tempdir().unwrap(); + let directory = control.path().join("mcp-capabilities"); + std::fs::create_dir(&directory).unwrap(); + let home = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let endpoint = "http://host.docker.internal:4000"; + let prior = store + .issue( + soul.clone(), + &directory, + endpoint.into(), + Some(("codex", home.path(), &[])), + Default::default(), + None, + ) + .unwrap(); + let prior_incarnation = IncarnationId::new(); + store + .activate( + &prior.grant_id, + soul.clone(), + prior_incarnation.clone(), + &directory, + endpoint, + ) + .unwrap(); + store + .commit_preparation(&prior.grant_id, &soul, &prior_incarnation, &directory) + .unwrap(); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + std::fs::write(prior_stage.join("usable-resource"), b"prior-input-nonce").unwrap(); + let replacement = store + .issue_replacement( + soul.clone(), + &directory, + endpoint.into(), + Some(("codex", home.path(), &[])), + None, + Some(home.path()), + ) + .unwrap(); + let incarnation = IncarnationId::new(); + store + .activate( + &replacement.grant_id, + soul.clone(), + incarnation.clone(), + &directory, + endpoint, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &replacement.grant_id).unwrap(); + std::fs::write(stage.join("usable-resource"), b"bound-input-nonce").unwrap(); + let path = grant_path(&directory, &replacement.grant_id).unwrap(); + let journal = std::fs::read(&path).unwrap(); + let (prior_scope, current_scope) = { + let grants = store.grants.lock().unwrap(); + ( + grants[&prior.grant_id].token.clone(), + grants[&replacement.grant_id].token.clone(), + ) + }; + // Empty controller memory is not evidence that this preparation was + // never delivered. Its bounded durable record still belongs to a host. + let cold = + CapabilityStore::default().abort_preparation(&replacement.grant_id, &soul, &directory); + let warm = store.abort_preparation(&replacement.grant_id, &soul, &directory); + eprintln!( + "bound_preparation_abort {}", + serde_json::json!({ + "coldAbortSucceeded":cold.is_ok(), "warmAbortSucceeded":warm.is_ok(), + "currentInputsRetained":stage.is_dir(), + "priorInputsRetained":prior_stage.is_dir() + }) + ); + assert!( + cold.is_err(), + "cold abort erased bound preparation ownership" + ); + assert!( + warm.is_err(), + "warm abort erased bound preparation ownership" + ); + assert!(std::fs::read(&path).unwrap() == journal); + assert_eq!( + std::fs::read(stage.join("usable-resource")).unwrap(), + b"bound-input-nonce" + ); + assert_eq!( + std::fs::read(prior_stage.join("usable-resource")).unwrap(), + b"prior-input-nonce" + ); + assert!(store.authenticates(&prior_scope)); + assert!(store.authenticates(¤t_scope)); + let repeated = store + .issue_replacement( + soul.clone(), + &directory, + endpoint.into(), + Some(("codex", home.path(), &[])), + None, + Some(home.path()), + ) + .unwrap(); + assert_eq!( + repeated.grant_id, replacement.grant_id, + "uncertain bound delivery cannot authorize another generation" + ); + let restored = read_stored_capability(&directory, &replacement.grant_id).unwrap(); + assert_eq!(restored.incarnation_id.as_ref(), Some(&incarnation)); + assert!(matches!( + restored.preparation.unwrap().phase, + InputPreparationPhase::Bound + )); + } + + #[test] + fn managed_mcp_generation_abort_cold_pending_requires_complete_matching_ownership() { + use std::io::Read; + let mut failed = Vec::new(); + for case in [ + "pending", + "missing", + "malformed", + "oversize", + "other_soul", + "assigned_pending", + "unrecorded", + ] { + let control = tempfile::tempdir().unwrap(); + let directory = control.path().join("mcp-capabilities"); + std::fs::create_dir(&directory).unwrap(); + let home = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let endpoint = "http://host.docker.internal:4000"; + let prior = store + .issue( + soul.clone(), + &directory, + endpoint.into(), + Some(("codex", home.path(), &[])), + Default::default(), + None, + ) + .unwrap(); + let incarnation = IncarnationId::new(); + store + .activate( + &prior.grant_id, + soul.clone(), + incarnation.clone(), + &directory, + endpoint, + ) + .unwrap(); + store + .commit_preparation(&prior.grant_id, &soul, &incarnation, &directory) + .unwrap(); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + std::fs::write(prior_stage.join("usable-resource"), b"prior-input-nonce").unwrap(); + let pending = store + .issue_replacement( + soul.clone(), + &directory, + endpoint.into(), + Some(("codex", home.path(), &[])), + None, + Some(home.path()), + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &pending.grant_id).unwrap(); + std::fs::write(stage.join("usable-resource"), b"pending-input-nonce").unwrap(); + let path = grant_path(&directory, &pending.grant_id).unwrap(); + let before = std::fs::read(&path).unwrap(); + let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + match case { + "missing" => std::fs::remove_file(&path).unwrap(), + "malformed" => std::fs::write(&path, b"{\"preparation\":").unwrap(), + "oversize" => { + let mut file = std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(&path) + .unwrap(); + file.set_len(128 * 1024 + 1).unwrap(); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + file.read_exact(&mut [0]).unwrap(); + assert_ne!( + file.metadata().unwrap().accessed().unwrap(), + old_access, + "fixture filesystem must observe ownership payload reads" + ); + file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) + .unwrap(); + } + "assigned_pending" | "unrecorded" => { + let mut stored: serde_json::Value = serde_json::from_slice(&before).unwrap(); + if case == "assigned_pending" { + stored["incarnationId"] = + serde_json::to_value(IncarnationId::new()).unwrap(); + } else { + stored.as_object_mut().unwrap().remove("preparation"); + } + std::fs::write(&path, serde_json::to_vec(&stored).unwrap()).unwrap(); + } + _ => {} + } + let requested_soul = if case == "other_soul" { + SoulId::new() + } else { + soul + }; + let modified = if matches!(case, "assigned_pending" | "unrecorded") { + Some(std::fs::read(&path).unwrap()) + } else { + None + }; + let cold = CapabilityStore::default(); + let result = cold.abort_preparation(&pending.grant_id, &requested_soul, &directory); + let prior_retained = std::fs::read(prior_stage.join("usable-resource")) + .is_ok_and(|bytes| bytes == b"prior-input-nonce"); + let current_retained = std::fs::read(stage.join("usable-resource")) + .is_ok_and(|bytes| bytes == b"pending-input-nonce"); + eprintln!( + "cold_preparation_abort {}", + serde_json::json!({ + "case":case,"abortSucceeded":result.is_ok(), + "currentInputsRetained":current_retained,"priorInputsRetained":prior_retained + }) + ); + let journal_retained = match case { + "pending" | "missing" => !path.exists(), + "malformed" => { + std::fs::read(&path).is_ok_and(|bytes| bytes == b"{\"preparation\":") + } + "oversize" => std::fs::metadata(&path).is_ok_and(|metadata| { + metadata.len() == 128 * 1024 + 1 && metadata.accessed().unwrap() == old_access + }), + "assigned_pending" | "unrecorded" => { + std::fs::read(&path).is_ok_and(|bytes| Some(bytes) == modified) + } + _ => std::fs::read(&path).is_ok_and(|bytes| bytes == before), + }; + let expected_success = case == "pending"; + if result.is_ok() != expected_success + || !prior_retained + || !journal_retained + || current_retained == expected_success + { + failed.push(case); + } + if case == "pending" { + assert!( + cold.abort_preparation(&pending.grant_id, &requested_soul, &directory) + .is_ok(), + "completed undispatched abort must be idempotent" + ); + } + } + assert!(failed.is_empty(), "cold pending abort changed unconfirmed ownership or rejected an undispatched preparation: {failed:?}"); + } + #[test] fn opencode_private_stage_survives_reissue_without_durable_secret_bytes() { let control = tempfile::tempdir().unwrap(); From 007e7ba82ec0124b602638935521563979f73d21 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 17:22:12 -0700 Subject: [PATCH 156/493] Qualify managed exec capacity with actual kernel and stack limits --- crates/freshell-platform/src/provider_exec.rs | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/crates/freshell-platform/src/provider_exec.rs b/crates/freshell-platform/src/provider_exec.rs index 7b58b83c0..99d489d60 100644 --- a/crates/freshell-platform/src/provider_exec.rs +++ b/crates/freshell-platform/src/provider_exec.rs @@ -274,4 +274,135 @@ mod tests { Err(ExecError::Interpreter) ); } + + #[test] + fn provider_exec_profile_executes_actual_kernel_capacity_and_checks_process_stack() { + const CHILD: &str = "FRESHELL_EXEC_CAPACITY_CHILD"; + if std::env::var_os(CHILD).is_none() { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "provider_exec::tests::provider_exec_profile_executes_actual_kernel_capacity_and_checks_process_stack", + "--nocapture", + ]) + .env_clear() + .env(CHILD, "1") + .output() + .unwrap(); + assert!( + output.status.success(), + "isolated actual exec/stack qualification failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + let observed: serde_json::Value = serde_json::from_slice(&output.stderr).unwrap(); + println!("provider_exec_capacity {observed}"); + return; + } + let mut limits = std::mem::MaybeUninit::::uninit(); + assert_eq!( + unsafe { libc::getrlimit(libc::RLIMIT_STACK, limits.as_mut_ptr()) }, + 0 + ); + let original = unsafe { limits.assume_init() }; + assert!(original.rlim_max >= MIN_STACK_BYTES); + let selected = libc::rlimit { + rlim_cur: MIN_STACK_BYTES, + rlim_max: original.rlim_max, + }; + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_STACK, &selected) }, 0); + let native = process_profile().unwrap(); + assert_eq!(native.soft_stack_bytes, MIN_STACK_BYTES); + let string_bytes = native.page_bytes.checked_mul(32).unwrap(); + let command = |arguments: Vec, environment: Option<(&str, String)>| { + let mut command = std::process::Command::new("/bin/true"); + command.env_clear().args(arguments); + if let Some((name, value)) = environment { + command.env(name, value); + } + command + }; + let mut at_string_limit = command(vec!["x".repeat(string_bytes - 1)], None); + check_command(&at_string_limit, false, &[]).unwrap(); + assert!(at_string_limit.status().unwrap().success()); + let mut over_string_limit = command(vec!["x".repeat(string_bytes)], None); + assert_eq!( + check_command(&over_string_limit, false, &[]), + Err(ExecError::String) + ); + assert_eq!( + over_string_limit.status().unwrap_err().raw_os_error(), + Some(libc::E2BIG) + ); + let name = "FRESHELL_EXEC_CAPACITY_VALUE"; + let value_bytes = string_bytes - name.len() - 2; + let mut at_environment_limit = command(vec![], Some((name, "x".repeat(value_bytes)))); + check_command(&at_environment_limit, false, &[]).unwrap(); + assert!(at_environment_limit.status().unwrap().success()); + let mut over_environment_limit = command(vec![], Some((name, "x".repeat(value_bytes + 1)))); + assert_eq!( + check_command(&over_environment_limit, false, &[]), + Err(ExecError::String) + ); + assert_eq!( + over_environment_limit.status().unwrap_err().raw_os_error(), + Some(libc::E2BIG) + ); + + // Find the accepted total through the public command boundary rather + // than reproducing the internal NUL/interpreter/pointer accounting. + let arguments = |bytes: usize| { + (0..8) + .map(|index| "x".repeat(bytes / 8 + usize::from(index < bytes % 8))) + .collect::>() + }; + let mut accepted = 0; + let mut rejected = MAX_EXEC_BYTES.min(8 * (string_bytes - 1)); + assert_eq!( + check_command(&command(arguments(rejected), None), false, &[]), + Err(ExecError::Total) + ); + while accepted + 1 < rejected { + let midpoint = accepted + (rejected - accepted) / 2; + match check_command(&command(arguments(midpoint), None), false, &[]) { + Ok(()) => accepted = midpoint, + Err(ExecError::Total) => rejected = midpoint, + other => panic!("unexpected finite aggregate qualification: {other:?}"), + } + } + assert!(accepted > MAX_EXEC_BYTES - native.page_bytes); + let mut at_total_limit = command(arguments(accepted), None); + check_command(&at_total_limit, false, &[]).unwrap(); + assert!(at_total_limit.status().unwrap().success()); + assert_eq!( + check_command(&command(arguments(rejected), None), false, &[]), + Err(ExecError::Total) + ); + let too_small = libc::rlimit { + rlim_cur: MIN_STACK_BYTES - 1, + ..selected + }; + assert_eq!( + unsafe { libc::setrlimit(libc::RLIMIT_STACK, &too_small) }, + 0 + ); + assert_eq!( + process_profile().unwrap().soft_stack_bytes, + MIN_STACK_BYTES - 1 + ); + assert_eq!( + check_command(&command(vec![], None), false, &[]), + Err(ExecError::Stack) + ); + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_STACK, &original) }, 0); + eprintln!( + "{}", + serde_json::json!({ + "severity":"info", "stage":"exec_capacity", "pageBytes":native.page_bytes, + "queriedStackBytes":native.soft_stack_bytes, "kernelStringBytesWithNul":string_bytes, + "acceptedArgPayloadBytes":accepted, "rejectedArgPayloadBytes":rejected, + "singleStringKernelE2big":true, "completeEnvironmentKernelE2big":true, + "stackBelowMinimumRejected":true + }) + ); + } } From 8e1f94d94fae6fa0f035056d7726d157fa90aed7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 17:28:39 -0700 Subject: [PATCH 157/493] Restore existing grant ownership for delayed activation callbacks --- .../src/managed_mcp_capability.rs | 201 ++++++++++++++++-- 1 file changed, 182 insertions(+), 19 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 02cb665f2..1bbd642fc 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -2716,31 +2716,40 @@ impl CapabilityStore { ) -> Result<(), String> { let path = grant_path(directory, grant_id)?; let mut grants = self.grants.lock().unwrap(); - if let Some(existing) = grants.get(grant_id) { - if existing.soul_id != soul_id || existing.endpoint != endpoint { - return Err("managed MCP grant identity mismatch".into()); - } - if existing.incarnation_id.as_ref() == Some(&incarnation_id) { - return Ok(()); + // A delayed callback cannot create authority after an undispatched + // preparation was aborted. Cold memory must recover the existing + // bounded ownership record, including its prior-generation graph. + let existing = if let Some(existing) = grants.get(grant_id) { + existing.clone() + } else { + let stored = read_stored_capability(directory, grant_id)?; + Grant { + soul_id: stored.soul_id, + incarnation_id: stored.incarnation_id, + token: stored.scope, + expires_at: stored.expires_at, + endpoint: stored.endpoint, + path: path.clone(), + preparation: stored.preparation, } + }; + if existing.soul_id != soul_id || existing.endpoint != endpoint { + return Err("managed MCP grant identity mismatch".into()); + } + if existing.incarnation_id.as_ref() == Some(&incarnation_id) { + grants.entry(grant_id.into()).or_insert(existing); + return Ok(()); } - let mut preparation = grants - .get(grant_id) - .and_then(|grant| grant.preparation.clone()); - if let Some(preparation) = preparation.as_mut() { + let mut grant = existing; + if let Some(preparation) = grant.preparation.as_mut() { if preparation.phase != InputPreparationPhase::Committed { preparation.phase = InputPreparationPhase::Bound; } } - let grant = Grant { - preparation, - soul_id, - incarnation_id: Some(incarnation_id), - token: new_secret(), - expires_at: now_secs() + GRANT_LIFETIME_SECS, - endpoint: endpoint.into(), - path: path.clone(), - }; + grant.incarnation_id = Some(incarnation_id); + grant.token = new_secret(); + grant.expires_at = now_secs() + GRANT_LIFETIME_SECS; + grant.path = path.clone(); write_capability(&path, &grant)?; grants.insert(grant_id.into(), grant); Ok(()) @@ -5998,6 +6007,160 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ assert!(failed.is_empty(), "cold pending abort changed unconfirmed ownership or rejected an undispatched preparation: {failed:?}"); } + #[test] + fn managed_mcp_activation_callback_preserves_cold_ownership_and_rejects_aborted_delivery() { + let mut failed = Vec::new(); + for case in [ + "warm_aborted", + "cold_aborted", + "warm_pending", + "cold_pending", + "cold_bound", + "cold_wrong_soul", + "cold_wrong_endpoint", + ] { + let owned = tempfile::tempdir().unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let endpoint = "http://host.docker.internal:4000"; + let prior = store + .issue( + soul.clone(), + owned.path(), + endpoint.into(), + None, + Default::default(), + None, + ) + .unwrap(); + let prior_incarnation = IncarnationId::new(); + store + .activate( + &prior.grant_id, + soul.clone(), + prior_incarnation.clone(), + owned.path(), + endpoint, + ) + .unwrap(); + store + .commit_preparation(&prior.grant_id, &soul, &prior_incarnation, owned.path()) + .unwrap(); + let prior_stage = staged_provider_root_path(owned.path(), &prior.grant_id).unwrap(); + std::fs::create_dir_all(&prior_stage).unwrap(); + std::fs::write(prior_stage.join("usable-resource"), b"prior-callback-input").unwrap(); + let current = store + .issue_replacement( + soul.clone(), + owned.path(), + endpoint.into(), + None, + None, + None, + ) + .unwrap(); + let current_stage = staged_provider_root_path(owned.path(), ¤t.grant_id).unwrap(); + std::fs::create_dir_all(¤t_stage).unwrap(); + std::fs::write( + current_stage.join("usable-resource"), + b"current-callback-input", + ) + .unwrap(); + let incarnation = IncarnationId::new(); + if case == "cold_bound" { + store + .activate( + ¤t.grant_id, + soul.clone(), + incarnation.clone(), + owned.path(), + endpoint, + ) + .unwrap(); + } + let aborted = case.ends_with("aborted"); + if aborted { + store + .abort_preparation(¤t.grant_id, &soul, owned.path()) + .unwrap(); + } + let path = grant_path(owned.path(), ¤t.grant_id).unwrap(); + let before = std::fs::read(&path).ok(); + let cold = CapabilityStore::default(); + let target = if case.starts_with("warm") { + &store + } else { + &cold + }; + let requested_soul = if case == "cold_wrong_soul" { + SoulId::new() + } else { + soul.clone() + }; + let requested_endpoint = if case == "cold_wrong_endpoint" { + "http://host.docker.internal:4001" + } else { + endpoint + }; + let bytes = serde_json::to_vec(&serde_json::json!({ + "action":"activate", "controlSecret":"closed-synthetic-control-secret", + "soulId":requested_soul, "grantId":current.grant_id, "incarnationId":incarnation + })) + .unwrap(); + let result = apply_callback( + &bytes, + "closed-synthetic-control-secret", + target, + owned.path(), + requested_endpoint, + owned.path(), + ); + let should_accept = matches!(case, "warm_pending" | "cold_pending" | "cold_bound"); + let stored = read_stored_capability(owned.path(), ¤t.grant_id).ok(); + let graph_retained = stored.as_ref().is_some_and(|stored| { + stored.soul_id == soul + && stored.endpoint == endpoint + && stored.incarnation_id.as_ref() == Some(&incarnation) + && stored.preparation.as_ref().is_some_and(|preparation| { + preparation.phase == InputPreparationPhase::Bound + && preparation.prior_grant_ids == [prior.grant_id.clone()] + }) + && target.authenticates(&stored.scope) + }); + let journal_unchanged = std::fs::read(&path).ok() == before; + let prior_retained = std::fs::read(prior_stage.join("usable-resource")) + .is_ok_and(|bytes| bytes == b"prior-callback-input"); + let current_retained = std::fs::read(current_stage.join("usable-resource")) + .is_ok_and(|bytes| bytes == b"current-callback-input"); + let memory_absent = !target + .grants + .lock() + .unwrap() + .contains_key(¤t.grant_id); + eprintln!( + "preparation_activation_callback {}", + serde_json::json!({ + "case":case,"callbackAccepted":result.is_ok(),"priorGraphRetained":graph_retained, + "journalUnchanged":journal_unchanged,"priorInputsRetained":prior_retained, + "currentInputsRetained":current_retained,"memoryAbsent":memory_absent + }) + ); + if result.is_ok() != should_accept + || !prior_retained + || current_retained == aborted + || (should_accept && !graph_retained) + || (!should_accept && (!journal_unchanged || !memory_absent)) + || (case == "cold_bound" && !journal_unchanged) + { + failed.push(case); + } + } + assert!( + failed.is_empty(), + "activation changed cold ownership or recreated aborted preparation: {failed:?}" + ); + } + #[test] fn opencode_private_stage_survives_reissue_without_durable_secret_bytes() { let control = tempfile::tempdir().unwrap(); From fec2c6de73bf9a6b48f7fb009a565c7ab282b840 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 17:53:48 -0700 Subject: [PATCH 158/493] Accept exact managed indexed-volume options during native cleanup --- scripts/testing/managed-provider-launch-sandbox.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 37821bb64..a4ce8202f 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -17,6 +17,7 @@ CLAUDE_TEST="managed_provider_launch_tests::managed_provider_claude_terminal_and_fresh_native_startup_consumes_prepared_inputs" CLAUDE_IMAGE="sha256:b09595b8f02dae505e6719d7617305ab8f5ad53a7138b3b1580ac553c0f2663a" LABEL="com.freshell.provider-launch-probe" +INDEXED_OPTIONS={"type":"tmpfs","device":"tmpfs","o":"size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec"} def docker(*args,check=True,timeout=180): return subprocess.run(["docker",*args],text=True,capture_output=True,timeout=timeout,check=check) @@ -271,6 +272,7 @@ def create(name,*arguments): value=json.loads(inspected.stdout)[0] preparations=[json.loads(candidate.read_text()) for candidate in (root/"handoff").glob("*.json")] labels=None + options=None for prepared in preparations: if volume==prepared["providerVolumeName"]: if not prepared.get("initialAuthReservation"):continue @@ -283,6 +285,7 @@ def create(name,*arguments): labels={"com.freshell.managed-inputs":"true","com.freshell.installation-id":installation, "com.freshell.soul-id":prepared["soulId"],"com.freshell.incarnation-id":prepared["incarnationId"], "com.freshell.input-generation":prepared["generationId"]} + options=INDEXED_OPTIONS for candidate in candidates: if volume=="freshell-claude-candidate-"+candidate["requestId"]: source=next(item for item in initial if item["soulId"]==candidate["soulId"]) @@ -292,7 +295,8 @@ def create(name,*arguments): "com.freshell.claude-candidate":"ordinary"} if candidate.get("retained"):labels["com.freshell.source-recovery-attempt-id"]=candidate["retained"]["attemptId"] else:labels["com.freshell.source-reservation-id"]=candidate["reservationId"] - if labels is None or value.get("Driver")!="local" or value.get("Options") not in (None,{}) or value.get("Labels")!=labels: + valid_options=value.get("Options")==options if options is not None else value.get("Options") in (None,{}) + if labels is None or value.get("Driver")!="local" or not valid_options or value.get("Labels")!=labels: raise RuntimeError("closed startup volume cleanup ownership changed") docker("volume","rm",volume) receipt["ownedCleanup"]=all(docker("container","inspect",name,check=False).returncode!=0 for name in owned) and all(docker("volume","inspect",volume,check=False).returncode!=0 for volume in volumes) From 62b6f377b71a40e42623a16ed17b180dd2013874 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:02:51 -0700 Subject: [PATCH 159/493] Qualify full default provider copy budgets across both stages --- .../src/provider_environment.rs | 191 ++++++++++++++++++ 1 file changed, 191 insertions(+) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 0aebe9819..568129fd6 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -1552,6 +1552,197 @@ mod tests { } } + #[test] + fn default_copy_entry_capacity_survives_both_stages_and_rejects_one_more() { + let root = executable_fixture(); + let source = root.path().join("source"); + fs::create_dir(&source).unwrap(); + for index in 0..99_999 { + fs::write(source.join(format!("f{index:05}")), []).unwrap(); + } + fs::write(source.join("f99998"), b"entry-capacity-nonce").unwrap(); + let (first, second, home) = copy_default_twice(&source, root.path()); + assert_eq!((first.entries, second.entries), (100_000, 100_000)); + assert_eq!((first.bytes, second.bytes), (20, 20)); + assert_eq!( + fs::read(home.join("resource/f99998")).unwrap(), + b"entry-capacity-nonce" + ); + fs::write(source.join("extra"), []).unwrap(); + let rejected = copy_environment( + &[selection(&source, "resource")], + &home, + &[], + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(rejected.limit_kind, Some("entries")); + assert_eq!( + (rejected.observed, rejected.permitted), + (Some(100_001), Some(100_000)) + ); + assert_eq!( + fs::read(home.join("resource/f99998")).unwrap(), + b"entry-capacity-nonce" + ); + assert!(!home.join("resource/extra").exists()); + assert_eq!(fs::read_dir(&home).unwrap().count(), 1); + } + + fn copy_default_twice(source: &Path, root: &Path) -> (CopyReport, CopyReport, PathBuf) { + let stage = root.join("stage"); + let first = copy_environment( + &[selection(source, "resource")], + &stage, + &[], + CopyLimits::default(), + ) + .unwrap(); + let home = root.join("home"); + let second = copy_environment( + &[selection(&stage.join("resource"), "resource")], + &home, + &[], + CopyLimits::default(), + ) + .unwrap(); + (first, second, home) + } + + fn verify_capacity_payloads(home: &Path) { + for index in 0..4 { + let mut input = File::open(home.join(format!("resource/payload{index}"))).unwrap(); + let mut bytes = 0u64; + let mut actual = [0u8; 64 * 1024]; + loop { + let length = input.read(&mut actual).unwrap(); + if length == 0 { + break; + } + assert!(actual[..length].iter().all(|byte| *byte == 0x5a)); + bytes += length as u64; + } + assert_eq!(bytes, 128 * 1024 * 1024); + } + } + + fn mark_capacity_sources_unread(source: &Path) -> std::time::SystemTime { + let before = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + for index in 0..4 { + File::options() + .write(true) + .open(source.join(format!("payload{index}"))) + .unwrap() + .set_times(fs::FileTimes::new().set_accessed(before)) + .unwrap(); + } + before + } + + fn verify_capacity_sources_unread(source: &Path, before: std::time::SystemTime) { + for index in 0..4 { + assert_eq!( + fs::metadata(source.join(format!("payload{index}"))) + .unwrap() + .accessed() + .unwrap(), + before + ); + } + } + + #[test] + fn default_copy_byte_capacity_reads_both_complete_stages_and_rejects_overflow() { + let root = executable_fixture(); + let source = root.path().join("source"); + fs::create_dir(&source).unwrap(); + let block = [0x5au8; 64 * 1024]; + for index in 0..4 { + let mut file = File::create(source.join(format!("payload{index}"))).unwrap(); + for _ in 0..2048 { + file.write_all(&block).unwrap(); + } + } + let (first, second, home) = copy_default_twice(&source, root.path()); + assert_eq!( + (first.bytes, second.bytes), + (512 * 1024 * 1024, 512 * 1024 * 1024) + ); + verify_capacity_payloads(&home); + fs::write(source.join("extra"), b"x").unwrap(); + let unread = mark_capacity_sources_unread(&source); + let total = copy_environment( + &[selection(&source, "resource")], + &home, + &[], + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(total.limit_kind, Some("total_bytes")); + assert_eq!( + (total.observed, total.permitted), + (Some(512 * 1024 * 1024 + 1), Some(512 * 1024 * 1024)) + ); + assert!(!home.join("resource/extra").exists()); + verify_capacity_sources_unread(&source, unread); + verify_capacity_payloads(&home); + fs::remove_file(source.join("extra")).unwrap(); + OpenOptions::new() + .append(true) + .open(source.join("payload0")) + .unwrap() + .write_all(b"x") + .unwrap(); + let unread = mark_capacity_sources_unread(&source); + let file = copy_environment( + &[selection(&source, "resource")], + &home, + &[], + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(file.limit_kind, Some("file_bytes")); + assert_eq!( + (file.observed, file.permitted), + (Some(128 * 1024 * 1024 + 1), Some(128 * 1024 * 1024)) + ); + verify_capacity_sources_unread(&source, unread); + verify_capacity_payloads(&home); + assert_eq!(fs::read_dir(&home).unwrap().count(), 1); + } + + #[test] + fn default_copy_depth_capacity_survives_both_stages_and_rejects_one_more() { + let root = executable_fixture(); + let source = root.path().join("source"); + fs::create_dir(&source).unwrap(); + let mut leaf = source.clone(); + let mut relative = PathBuf::new(); + for _ in 0..63 { + leaf.push("d"); + relative.push("d"); + fs::create_dir(&leaf).unwrap(); + } + fs::write(leaf.join("nonce"), b"depth-capacity-nonce").unwrap(); + let (_, _, home) = copy_default_twice(&source, root.path()); + let published = home.join("resource").join(&relative).join("nonce"); + assert_eq!(fs::read(&published).unwrap(), b"depth-capacity-nonce"); + fs::create_dir(leaf.join("d")).unwrap(); + fs::write(leaf.join("d/extra"), b"x").unwrap(); + let error = copy_environment( + &[selection(&source, "resource")], + &home, + &[], + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(error.limit_kind, Some("depth")); + assert_eq!((error.observed, error.permitted), (Some(65), Some(64))); + assert_eq!(fs::read(published).unwrap(), b"depth-capacity-nonce"); + assert!(!home.join("resource").join(relative).join("d").exists()); + assert_eq!(fs::read_dir(&home).unwrap().count(), 1); + } + #[cfg(unix)] #[test] fn concrete_credential_identities_exclude_aliases_but_keep_package_auth() { From a5d1068a2ab8e25d4d28c7d88c65e84291ca4b77 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:14:41 -0700 Subject: [PATCH 160/493] Qualify full indexed input publication capacity over private IPC --- .../src/provider_input_exchange.rs | 180 ++++++++++++++++++ 1 file changed, 180 insertions(+) diff --git a/crates/freshell-platform/src/provider_input_exchange.rs b/crates/freshell-platform/src/provider_input_exchange.rs index 946faa969..fdcf5586b 100644 --- a/crates/freshell-platform/src/provider_input_exchange.rs +++ b/crates/freshell-platform/src/provider_input_exchange.rs @@ -369,4 +369,184 @@ mod tests { }; assert_eq!(bytes.len(), MAX_CHUNK_BYTES); } + + #[test] + fn provider_input_exchange_actual_capacity_and_overflow_preserve_prior_generation() { + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let prior = owned.path().join("prior"); + std::fs::create_dir(&prior).unwrap(); + std::fs::write(prior.join("nonce"), b"prior-indexed-generation").unwrap(); + for (case, ids, payloads, expected_success) in [ + ("exact", 128u16, vec![MAX_FILE_BYTES, MAX_FILE_BYTES], true), + ("file-over", 1, vec![MAX_FILE_BYTES + 1], false), + ( + "total-over", + 3, + vec![MAX_FILE_BYTES, MAX_FILE_BYTES, 1], + false, + ), + ("source-over", 129, vec![], false), + ] { + let root = owned.path().join(case); + std::fs::create_dir(&root).unwrap(); + let socket = owned.path().join(format!("{case}.sock")); + let listener = UnixListener::bind(&socket).unwrap(); + listener.set_nonblocking(true).unwrap(); + let outcome = std::thread::scope(|scope| { + let worker = scope.spawn(|| serve(&root, &socket)); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5); + let (mut stream, _) = loop { + match listener.accept() { + Ok(pair) => break pair, + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + assert!( + std::time::Instant::now() < deadline, + "private capacity publisher did not connect" + ); + std::thread::yield_now(); + } + Err(_) => panic!("private capacity accept failed"), + } + }; + stream + .set_read_timeout(Some(std::time::Duration::from_secs(20))) + .unwrap(); + stream + .set_write_timeout(Some(std::time::Duration::from_secs(20))) + .unwrap(); + let acknowledged = |stream: &mut std::os::unix::net::UnixStream, + request: Request, + expected: Acknowledgement| { + write_request(stream, &request).unwrap(); + let mut length = [0; 4]; + if stream.read_exact(&mut length).is_err() { + return false; + } + let length = u32::from_be_bytes(length) as usize; + assert!(length < MAX_FRAME_BYTES); + let mut response = vec![0; length]; + stream.read_exact(&mut response).unwrap(); + serde_json::from_slice::(&response).unwrap() == expected + }; + let mut accepted = acknowledged( + &mut stream, + Request::Begin { + source_ids: (0..ids).collect(), + }, + Acknowledgement::Begun, + ); + for id in 0..ids { + if !accepted { + break; + } + accepted = acknowledged( + &mut stream, + Request::Open { source_id: id }, + Acknowledgement::Opened, + ); + let mut remaining = payloads.get(id as usize).copied().unwrap_or(0); + while accepted && remaining != 0 { + let length = remaining.min(MAX_CHUNK_BYTES as u64) as usize; + accepted = acknowledged( + &mut stream, + Request::Chunk { + bytes: vec![0x5a; length], + }, + Acknowledgement::Written, + ); + remaining -= length as u64; + } + if accepted { + accepted = + acknowledged(&mut stream, Request::Finish, Acknowledgement::Finished); + } + } + if accepted { + accepted = acknowledged(&mut stream, Request::Seal, Acknowledgement::Populated); + assert!( + !worker.is_finished(), + "exact full generation must remain leased until release" + ); + if accepted { + accepted = + acknowledged(&mut stream, Request::Release, Acknowledgement::Released); + } + } + drop(stream); + let result = worker.join().unwrap(); + (accepted, result) + }); + assert_eq!(outcome.0, expected_success, "capacity case {case}"); + assert_eq!(outcome.1.is_ok(), expected_success, "capacity case {case}"); + if expected_success { + assert_eq!(std::fs::read_dir(root.join("files")).unwrap().count(), 128); + for id in 0..128 { + let path = root.join(format!("files/source-{id}")); + assert_eq!( + std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, + 0o600 + ); + let mut input = std::fs::File::open(path).unwrap(); + let mut bytes = 0u64; + let mut block = [0; MAX_CHUNK_BYTES]; + loop { + let length = input.read(&mut block).unwrap(); + if length == 0 { + break; + } + assert!(block[..length].iter().all(|byte| *byte == 0x5a)); + bytes += length as u64; + } + assert_eq!(bytes, payloads.get(id as usize).copied().unwrap_or(0)); + } + } else { + assert!( + outcome.1.unwrap_err().len() < 128, + "failure remains a fixed diagnostic without payload" + ); + match case { + "file-over" => { + assert_eq!( + std::fs::metadata(root.join("files/source-0")) + .unwrap() + .len(), + MAX_FILE_BYTES + ); + } + "total-over" => { + for id in 0..2 { + assert_eq!( + std::fs::metadata(root.join(format!("files/source-{id}"))) + .unwrap() + .len(), + MAX_FILE_BYTES + ); + } + assert_eq!( + std::fs::metadata(root.join("files/source-2")) + .unwrap() + .len(), + 0, + "over-budget byte must not be written" + ); + } + "source-over" => assert!( + !root.join("files").exists(), + "source-count overflow must precede private-file allocation" + ), + _ => unreachable!(), + } + } + assert_eq!( + std::fs::read(prior.join("nonce")).unwrap(), + b"prior-indexed-generation" + ); + std::fs::remove_dir_all(&root).unwrap(); + assert!( + !root.exists(), + "failed attempt's indexed files are removable without prior-generation mutation" + ); + } + } } From 9c38ca4d5cac71da8e1f605348cc90eee753681f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:19:50 -0700 Subject: [PATCH 161/493] Qualify selected environment limits in actual guarded children --- .../src/provider_inputs.rs | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 9ecd3095b..2759b8576 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -470,6 +470,115 @@ mod tests { assert!(!format!("{received:?}").contains("PRIVATE_PATH_INPUT_SENTINEL")); } + #[cfg(target_os = "linux")] + #[test] + fn provider_frozen_inputs_actual_maximum_profile_reaches_guarded_child() { + const MAXIMUM_VALUE_NAME: &str = "FRESHELL_TEST_MAXIMUM_PROVIDER_VALUE"; + let marker = format!("N{:0127}", 0); + if std::env::var_os(&marker).is_some() || std::env::var_os(MAXIMUM_VALUE_NAME).is_some() { + let variables = std::env::vars().collect::>(); + let maximum_value_case = variables.contains_key(MAXIMUM_VALUE_NAME); + let names = if maximum_value_case { 1 } else { 128 }; + let value_bytes = if maximum_value_case { 64 * 1024 } else { 1920 }; + assert_eq!(variables.len(), names); + assert!(variables + .values() + .all(|value| value.len() == value_bytes && value.bytes().all(|byte| byte == b'x'))); + if !maximum_value_case { + assert!(variables.keys().all(|name| name.len() == 128)); + } + let total = variables + .iter() + .map(|(name, value)| name.len() + value.len()) + .sum::(); + assert_eq!( + total, + if maximum_value_case { + MAXIMUM_VALUE_NAME.len() + 64 * 1024 + } else { + 256 * 1024 + } + ); + assert!(ProviderPathInputs::new(variables).is_ok()); + println!( + "frozen_input_actual_environment {}", + serde_json::json!({"names":names,"valueBytes":value_bytes,"totalBytes":total}) + ); + return; + } + + let exact = (0..128) + .map(|id| (format!("N{id:0127}"), "x".repeat(1920))) + .collect::>(); + let full = ProviderPathInputs::new(exact.clone()).unwrap(); + let mut over = exact; + over.get_mut(&marker).unwrap().push('x'); + assert!( + ProviderPathInputs::new(over).is_err(), + "one extra shared byte must fail before dispatch" + ); + assert!(ProviderPathInputs::new(std::collections::BTreeMap::from([( + format!("N{}", "0".repeat(128)), + "x".into() + )])) + .is_err()); + let maximum_value = ProviderPathInputs::new(std::collections::BTreeMap::from([( + MAXIMUM_VALUE_NAME.into(), + "x".repeat(64 * 1024), + )])) + .unwrap(); + assert!(ProviderPathInputs::new(std::collections::BTreeMap::from([( + MAXIMUM_VALUE_NAME.into(), + "x".repeat(64 * 1024 + 1) + )])) + .is_err()); + + for inputs in [full, maximum_value] { + let encoded = + crate::bounded_json_bytes(&inputs, crate::MAX_CONTROL_FRAME_BYTES).unwrap(); + let received: ProviderPathInputs = serde_json::from_slice(&encoded).unwrap(); + let mut command = std::process::Command::new(std::env::current_exe().unwrap()); + command.env_clear().envs(received.iter()).args([ + "--exact", + "provider_inputs::tests::provider_frozen_inputs_actual_maximum_profile_reaches_guarded_child", + "--nocapture", + ]); + freshell_platform::provider_exec::check_command(&command, false, &[]).unwrap(); + let output = command.output().unwrap(); + assert!( + output.status.success(), + "isolated exact-profile child failed" + ); + let diagnostic = String::from_utf8(output.stdout).unwrap(); + let receipt = diagnostic + .lines() + .find_map(|line| { + line.split_once("frozen_input_actual_environment ") + .map(|(_, receipt)| receipt) + }) + .expect("exact selected profile must be consumed in the actual child"); + let value: serde_json::Value = serde_json::from_str(receipt).unwrap(); + assert_eq!(value["names"], received.iter().count()); + assert_eq!( + value["valueBytes"], + received.iter().map(|(_, value)| value.len()).max().unwrap() + ); + assert_eq!( + value.as_object().unwrap().len(), + 3, + "child diagnostics carry numbers only" + ); + assert_eq!( + value["totalBytes"], + received + .iter() + .map(|(name, value)| name.len() + value.len()) + .sum::() + ); + println!("frozen_input_actual_environment {value}"); + } + } + fn prepared() -> PreparedProviderInputs { PreparedProviderInputs { generation_id: "grant-synthetic".into(), From f05a1bcdadf4cf15804a28fa09b8e6935053ae94 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:27:16 -0700 Subject: [PATCH 162/493] Enable journal tempfile dependency for managed production builds --- crates/freshell-server/Cargo.toml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index 2510a37a2..8a50070f8 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -26,6 +26,7 @@ managed-runtime-v1 = [ "dep:saphyr", "dep:saphyr-parser", "dep:semver", + "dep:tempfile", ] managed-provider-qualification = [ "managed-runtime-v1", @@ -81,6 +82,8 @@ toml = { version = "=0.9.12", optional = true } saphyr = { version = "=0.0.11", default-features = false, optional = true } saphyr-parser = { version = "=0.0.11", optional = true } semver = { version = "=1.0.28", optional = true } +# Managed preparation journals use private temporary files before atomic publication. +tempfile = { version = "3", optional = true } # Boot-scoped identifiers: `srv-` / `boot-`. uuid = { version = "1", features = ["v4"] } # The browser-pane HTTP reverse proxy (`/api/proxy/http/{port}/*`) forwards to a From a91f65e4c977ec9fe1c6bbe8a2ddd62870e8e673 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:38:21 -0700 Subject: [PATCH 163/493] Reject oversized accepted source views before reading --- .../managed_mcp_capability/source_stage.rs | 55 +++++++++++++++++-- 1 file changed, 50 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index e8df5216f..9b5bb02f9 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1,7 +1,10 @@ //! Stage source views in the grant's private filesystem before publication. use super::{private_directory, write_private_bytes}; use crate::managed_provider_references::sources::{PlannedSourceOccurrence, SourceLane}; -use freshell_platform::opencode_recipe::{SelectedSourceFile, SourceManifest, SourceView}; +use freshell_platform::{ + opencode_recipe::{SelectedSourceFile, SourceManifest, SourceView}, + provider_input_profiles::MAX_SOURCE_VIEW_BYTES, +}; use std::{collections::BTreeMap, path::Path}; /// Selected discovery roots are complete writable units, independent of the @@ -99,19 +102,24 @@ pub(super) fn read_source_view(root: &Path, relative: &str) -> Result MAX_SOURCE_VIEW_BYTES { + return Err("accepted source view exceeds 16 MiB".into()); + } let mut raw = String::new(); std::fs::File::open(path) .map_err(|_| "accepted source view is unavailable")? - .take(16 * 1024 * 1024 + 1) + .take(MAX_SOURCE_VIEW_BYTES + 1) .read_to_string(&mut raw) .map_err(|_| "accepted source view read failed")?; - if raw.len() > 16 * 1024 * 1024 { + if raw.len() as u64 > MAX_SOURCE_VIEW_BYTES { return Err("accepted source view exceeds 16 MiB".into()); } Ok(raw) @@ -518,3 +526,40 @@ pub(super) fn describe( super::allow_controller_stage_read(&root.join(".freshell/prepared-inputs.json"))?; Ok(inputs) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn accepted_source_view_rejects_actual_oversize_before_reading_or_prior_mutation() { + let owned = tempfile::tempdir().unwrap(); + let root = owned.path().join("stage"); + std::fs::create_dir(&root).unwrap(); + let path = root.join("view"); + let file = std::fs::File::create(&path).unwrap(); + file.set_len(16 * 1024 * 1024 + 1).unwrap(); + let before = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + file.set_times(std::fs::FileTimes::new().set_accessed(before)) + .unwrap(); + let prior = owned.path().join("prior"); + std::fs::write(&prior, b"prior-accepted-source-view").unwrap(); + let result = read_source_view(&root, "view"); + assert!(result.is_err()); + assert_eq!( + std::fs::metadata(&path).unwrap().accessed().unwrap(), + before, + "oversized stage view must be rejected from metadata before source bytes are read" + ); + assert_eq!( + std::fs::read(&prior).unwrap(), + b"prior-accepted-source-view" + ); + assert_eq!(std::fs::read_dir(&root).unwrap().count(), 1); + file.set_len(16 * 1024 * 1024).unwrap(); + assert_eq!( + read_source_view(&root, "view").unwrap().len(), + 16 * 1024 * 1024 + ); + } +} From 0cb2d0263da0612fc1a8e34c2f6f4a6e0aa868e2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:42:58 -0700 Subject: [PATCH 164/493] Bound retained original source views before reading --- .../managed_provider_references/sources.rs | 97 ++++++++++++++++++- 1 file changed, 92 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 48cf575df..302786b3b 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1,5 +1,8 @@ //! Ordered native source occurrences for managed OpenCode preparation. pub use freshell_platform::opencode_recipe::{DiscoveryRole, DiscoveryRoot}; +use freshell_platform::provider_input_profiles::{ + MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES, MAX_SOURCE_VIEW_BYTES, +}; use std::path::{Path, PathBuf}; #[derive(Clone, Copy, PartialEq, Eq)] @@ -289,7 +292,7 @@ fn add_source( .map_err(|_| "selected configuration source is unavailable, dangling or cyclic")?; let metadata = std::fs::metadata(&canonical_source) .map_err(|_| "selected configuration source metadata failed")?; - if !metadata.is_file() || metadata.len() > 16 * 1024 * 1024 { + if !metadata.is_file() || metadata.len() > MAX_SOURCE_VIEW_BYTES { return Err("selected configuration source is not a bounded regular file"); } sources.push(SourceOccurrence { @@ -315,7 +318,7 @@ pub(crate) struct SourceViewBudget { } impl SourceViewBudget { pub fn charge(&mut self, view: &str) -> Result<(), &'static str> { - if view.len() > 16 * 1024 * 1024 { + if view.len() as u64 > MAX_SOURCE_VIEW_BYTES { return Err("source view exceeds 16 MiB"); } let identity = (view.as_ptr() as usize, view.len()); @@ -326,7 +329,7 @@ impl SourceViewBudget { .bytes .checked_add(view.len()) .ok_or("source-view byte count overflow")?; - if bytes > 64 * 1024 * 1024 { + if bytes as u64 > MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES { return Err("simultaneous retained source views exceed 64 MiB"); } self.views.insert(identity); @@ -375,10 +378,15 @@ pub fn read_opencode_sources( let view = if let Some(view) = canonical.get(&source.canonical_source) { view.clone() } else { - let remaining = (64 * 1024 * 1024usize) + let remaining = (MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES as usize) .checked_sub(retained) .ok_or_else(|| fail("source-view byte count overflow"))?; - let limit = remaining.min(16 * 1024 * 1024); + let limit = remaining.min(MAX_SOURCE_VIEW_BYTES as usize); + let metadata = std::fs::metadata(&source.canonical_source) + .map_err(|_| fail("selected source read failed"))?; + if !metadata.is_file() || metadata.len() > limit as u64 { + return Err(fail("selected source exceeds source-view limit")); + } let mut raw = String::new(); std::fs::File::open(&source.canonical_source) .map_err(|_| fail("selected source read failed"))? @@ -618,6 +626,85 @@ pub fn plan_loaded_opencode_sources_relocated( mod tests { use super::*; use sha2::{Digest, Sha256}; + + #[test] + fn managed_provider_references_actual_source_file_growth_rejects_before_reading() { + let owned = tempfile::tempdir().unwrap(); + let path = owned.path().join("opencode.json"); + let file = std::fs::File::create(&path).unwrap(); + let mut sources = Vec::new(); + add_source( + &mut sources, + path.clone(), + SourceLane::Main, + DiscoveryRole::Workspace, + ) + .unwrap(); + file.set_len(16 * 1024 * 1024 + 1).unwrap(); + let unread = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + file.set_times(std::fs::FileTimes::new().set_accessed(unread)) + .unwrap(); + assert!(read_opencode_sources(sources).is_err()); + assert_eq!( + file.metadata().unwrap().accessed().unwrap(), + unread, + "growth beyond the actual per-source limit must fail before reading source bytes" + ); + } + + #[test] + fn managed_provider_references_actual_aggregate_view_limit_rejects_before_extra_read() { + let owned = tempfile::tempdir().unwrap(); + let mut paths = Vec::new(); + for ordinal in 0..4 { + let path = owned.path().join(format!("source-{ordinal}.json")); + std::fs::File::create(&path) + .unwrap() + .set_len(16 * 1024 * 1024) + .unwrap(); + paths.push(path); + } + let occurrences = |paths: &[PathBuf]| { + paths + .iter() + .map(|path| SourceOccurrence { + logical_path: path.clone(), + canonical_source: path.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }) + .collect() + }; + let mut repeated = paths.clone(); + repeated.push(paths[0].clone()); + let loaded = read_opencode_sources(occurrences(&repeated)).unwrap(); + assert_eq!( + retained_source_view_bytes(&loaded).unwrap(), + 64 * 1024 * 1024 + ); + assert!(std::sync::Arc::ptr_eq(&loaded[0].view, &loaded[4].view)); + assert!(loaded.iter().all(|source| { + source.view.len() == 16 * 1024 * 1024 + && source.view.as_bytes().iter().all(|byte| *byte == 0) + })); + drop(loaded); + let extra = owned.path().join("extra.json"); + let extra_file = std::fs::File::create(&extra).unwrap(); + extra_file.set_len(1).unwrap(); + let unread = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + extra_file + .set_times(std::fs::FileTimes::new().set_accessed(unread)) + .unwrap(); + paths.push(extra); + assert!(read_opencode_sources(occurrences(&paths)).is_err()); + assert_eq!( + extra_file.metadata().unwrap().accessed().unwrap(), + unread, + "the extra view must be rejected before reading beyond 64 MiB" + ); + assert_eq!(std::fs::read_dir(owned.path()).unwrap().count(), 5); + } + fn digest(path: &Path) -> String { format!("{:x}", Sha256::digest(std::fs::read(path).unwrap())) } From c354f62daef78f71855a7fd5c2bf5097dc888a59 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:47:53 -0700 Subject: [PATCH 165/493] Exercise actual reference alias and substitution capacities --- .../src/managed_provider_references.rs | 151 +++++++++++++++++- 1 file changed, 150 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 1410c8fa2..0df817d14 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -430,10 +430,17 @@ process.stdout.write(JSON.stringify(receipts)); evaluate_generated_recipe(recipe, env).unwrap(), ); } + consume_published(runtime, &child) + } + + fn consume_published( + runtime: &Path, + child: &std::collections::BTreeMap, + ) -> Vec { let result = std::process::Command::new("node") .arg("-e") .arg(CONSUMER) - .arg(&source) + .arg(runtime.join("source.json")) .arg(runtime) .arg(serde_json::to_string(&child).unwrap()) .output() @@ -446,6 +453,148 @@ process.stdout.write(JSON.stringify(receipts)); serde_json::from_slice(&result.stdout).unwrap() } + #[cfg(unix)] + #[test] + fn managed_provider_references_actual_maximum_sources_aliases_reach_child_and_overflow_preserves_projection( + ) { + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("opencode.json"); + let runtime = owned.path().join("runtime"); + let mut fields = serde_json::Map::new(); + let mut expected = Vec::new(); + for ordinal in 0..128 { + let name = format!("payload-{ordinal:03}"); + let payload = format!(" selected-reference-nonce-{ordinal:03}\nquote\"slash\\ "); + std::fs::write(owned.path().join(&name), &payload).unwrap(); + for suffix in ["a", "b"] { + let alias = format!("alias-{ordinal:03}-{suffix}"); + std::os::unix::fs::symlink(&name, owned.path().join(&alias)).unwrap(); + fields.insert( + format!("field-{ordinal:03}-{suffix}"), + serde_json::Value::String(format!("{{file:{alias}}}")), + ); + expected.push(digest(payload.trim())); + } + } + let raw = serde_json::Value::Object(fields.clone()).to_string(); + std::fs::write(&source, &raw).unwrap(); + let resolve = |raw: &str| { + resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: owned.path(), + original_home: owned.path(), + }, + &|_| None, + ) + }; + let plan = resolve(&raw).unwrap(); + assert_eq!(plan.selections.len(), 128); + assert_eq!(plan.occurrences.len(), 256); + assert_eq!( + plan.occurrences + .iter() + .map(|occurrence| &occurrence.approved_alias) + .collect::>() + .len(), + 256 + ); + assert_eq!(consume(&plan, &runtime), expected); + let prior_source = std::fs::read(runtime.join("source.json")).unwrap(); + std::os::unix::fs::symlink("payload-000", owned.path().join("alias-overflow")).unwrap(); + let mut alias_overflow = fields.clone(); + alias_overflow.insert( + "overflow".into(), + serde_json::Value::String("{file:alias-overflow}".into()), + ); + let error = resolve(&serde_json::Value::Object(alias_overflow).to_string()) + .err() + .unwrap(); + assert_eq!(error.cause, "configuration alias count exceeds 256"); + std::fs::write(owned.path().join("source-overflow"), b"PRIVATE_OVERFLOW").unwrap(); + fields.insert( + "overflow".into(), + serde_json::Value::String("{file:source-overflow}".into()), + ); + let error = resolve(&serde_json::Value::Object(fields).to_string()) + .err() + .unwrap(); + assert_eq!(error.cause, "configuration reference count exceeds 128"); + assert!(!error.to_string().contains("PRIVATE_OVERFLOW")); + assert_eq!(std::fs::read(&source).unwrap(), raw.as_bytes()); + assert_eq!( + std::fs::read(runtime.join("source.json")).unwrap(), + prior_source + ); + assert_eq!(consume_published(&runtime, &Default::default()), expected); + assert_eq!( + std::fs::read_dir(runtime.join(".freshell/references")) + .unwrap() + .count(), + 128 + ); + } + + #[test] + fn managed_provider_references_actual_maximum_file_events_reach_child_and_overflow_preserves_projection( + ) { + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("opencode.json"); + let runtime = owned.path().join("runtime"); + std::fs::write(owned.path().join("rules"), b" event-capacity-nonce ").unwrap(); + let raw = serde_json::Value::Object( + (0..8192) + .map(|ordinal| { + ( + format!("field-{ordinal:04}"), + serde_json::Value::String("{file:rules}".into()), + ) + }) + .collect(), + ) + .to_string(); + std::fs::write(&source, &raw).unwrap(); + let resolve = |raw: &str| { + resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: owned.path(), + original_home: owned.path(), + }, + &|_| None, + ) + }; + let plan = resolve(&raw).unwrap(); + assert_eq!(plan.substitution_events, 8192); + assert_eq!(plan.occurrences.len(), 8192); + assert_eq!(plan.selections.len(), 1); + let expected = vec![digest("event-capacity-nonce"); 8192]; + assert_eq!(consume(&plan, &runtime), expected); + let overflow = format!( + "{},\"overflow\":\"{{file:rules}}\"}}", + &raw[..raw.len() - 1] + ); + let error = resolve(&overflow).err().unwrap(); + assert_eq!( + error.cause, + "configuration source, expansion or event profile limit exceeded" + ); + assert_eq!(std::fs::read(&source).unwrap(), raw.as_bytes()); + assert_eq!( + std::fs::read(runtime.join("source.json")).unwrap(), + plan.template.as_bytes() + ); + assert_eq!(consume_published(&runtime, &Default::default()), expected); + assert_eq!( + std::fs::read_dir(runtime.join(".freshell/references")) + .unwrap() + .count(), + 1 + ); + } + #[test] fn managed_provider_references_child_consumes_original_absolute_relative_home_spaces_and_symlink( ) { From 08f440ec827ef559fe4eb883387907bf6c000a0d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 18:54:54 -0700 Subject: [PATCH 166/493] Check Codex source bounds before reading --- .../src/managed_provider_references/codex.rs | 171 +++++++++++++++++- 1 file changed, 170 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_references/codex.rs b/crates/freshell-server/src/managed_provider_references/codex.rs index 09ca9479e..e04117d41 100644 --- a/crates/freshell-server/src/managed_provider_references/codex.rs +++ b/crates/freshell-server/src/managed_provider_references/codex.rs @@ -104,6 +104,13 @@ impl Selection<'_> { return Err("Codex role configuration cycle detected"); } let result = (|| { + if std::fs::metadata(&canonical) + .map_err(|_| "Codex selected configuration unreadable")? + .len() + > MAX_SOURCE_VIEW_BYTES + { + return Err("Codex selected configuration source view exceeded"); + } let mut raw = String::new(); std::fs::File::open(&canonical) .map_err(|_| "Codex selected configuration unreadable")? @@ -760,6 +767,9 @@ fn is_agent_plugin(active: &Path) -> Result { if !metadata.is_file() || metadata.file_type().is_symlink() { return Err("Codex active root plugin manifest must be a physical regular file; repair installation and retry"); } + if metadata.len() > MAX_SOURCE_VIEW_BYTES { + return Err("Codex active plugin manifest source view exceeded"); + } let mut bytes = Vec::new(); std::fs::File::open(path) .map_err(|_| "Codex active plugin manifest is unreadable; repair installation and retry")? @@ -792,13 +802,21 @@ fn exists(path: &Path) -> Result { } fn read_config(path: &Path) -> Result { use std::io::Read; + let metadata = + std::fs::metadata(path).map_err(|_| "Codex selected configuration is unavailable")?; + if !metadata.is_file() { + return Err("Codex selected configuration is unavailable"); + } + if metadata.len() > MAX_SOURCE_VIEW_BYTES { + return Err("Codex selected configuration source view exceeded"); + } let mut bytes = Vec::new(); std::fs::File::open(path) .map_err(|_| "Codex selected configuration is unavailable")? .take(MAX_SOURCE_VIEW_BYTES + 1) .read_to_end(&mut bytes) .map_err(|_| "Codex selected configuration is unreadable")?; - if bytes.len() > 16 * 1024 * 1024 { + if bytes.len() as u64 > MAX_SOURCE_VIEW_BYTES { return Err("Codex selected configuration source view exceeded"); } let raw = @@ -926,6 +944,157 @@ mod tests { } } + #[test] + fn managed_provider_codex_actual_source_read_limits_reject_before_reading() { + use std::io::Write; + let owned = tempfile::tempdir().unwrap(); + let unread = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + let mut read_oversized = Vec::new(); + for kind in ["selection", "configuration", "plugin"] { + let root = owned.path().join(kind); + std::fs::create_dir(&root).unwrap(); + let path = root.join(if kind == "plugin" { + "plugin.json" + } else { + "config.toml" + }); + let file = std::fs::File::create(&path).unwrap(); + file.set_len(16 * 1024 * 1024 + 1).unwrap(); + file.set_times(std::fs::FileTimes::new().set_accessed(unread)) + .unwrap(); + let rejected = match kind { + "selection" => { + select_inputs(&[layer(path.clone(), true)], owned.path(), None).is_err() + } + "configuration" => read_config(&path).is_err(), + "plugin" => is_agent_plugin(&root).is_err(), + _ => unreachable!(), + }; + assert!(rejected); + if file.metadata().unwrap().accessed().unwrap() != unread { + read_oversized.push(kind); + } + } + assert!( + read_oversized.is_empty(), + "oversized readers accessed source bytes: {read_oversized:?}" + ); + for kind in ["selection", "configuration", "plugin"] { + let root = owned.path().join(kind); + let path = root.join(if kind == "plugin" { + "plugin.json" + } else { + "config.toml" + }); + let prefix: &[u8] = if kind == "plugin" { + br#"{"$schema":"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json"}"# + } else { + b"# exact source profile\n" + }; + let mut file = std::fs::File::create(&path).unwrap(); + file.write_all(prefix).unwrap(); + let chunk = vec![b' '; 64 * 1024]; + let mut remaining = 16 * 1024 * 1024 - prefix.len(); + while remaining != 0 { + let size = remaining.min(chunk.len()); + file.write_all(&chunk[..size]).unwrap(); + remaining -= size; + } + match kind { + "selection" => assert_eq!( + select_inputs(&[layer(path, true)], owned.path(), None) + .unwrap() + .len(), + 1 + ), + "configuration" => { + assert!(read_config(&path).unwrap().as_table().unwrap().is_empty()) + } + "plugin" => assert!(is_agent_plugin(&root).unwrap()), + _ => unreachable!(), + } + } + } + + #[cfg(unix)] + #[test] + fn managed_provider_codex_actual_event_count_and_metadata_capacity() { + let owned = tempfile::tempdir().unwrap(); + let root = owned.path().join("a".repeat(200)).join("b".repeat(200)); + std::fs::create_dir_all(&root).unwrap(); + let length = 512 - root.as_os_str().len() - 1; + assert!((8..255).contains(&length)); + let configuration = root.join("c".repeat(length)); + let role = root.join("r".repeat(length)); + let alias = root.join("s".repeat(length + 1)); + std::fs::write(&role, "# actual selected role\n").unwrap(); + std::os::unix::fs::symlink(role.file_name().unwrap(), &alias).unwrap(); + assert_eq!(configuration.as_os_str().len(), 512); + assert_eq!(role.as_os_str().len(), 512); + assert_eq!(alias.as_os_str().len(), 513); + let document = |count: usize, extra_alias: bool| { + (0..count) + .map(|ordinal| { + let path = if extra_alias && ordinal + 1 == count { + &alias + } else { + &role + }; + format!( + "[agents.role_{ordinal:04}]\nconfig_file={}\n", + serde_json::to_string(path.to_str().unwrap()).unwrap() + ) + }) + .collect::() + }; + let raw = document(8191, false); + std::fs::write(&configuration, &raw).unwrap(); + let selected = + select_inputs(&[layer(configuration.clone(), true)], owned.path(), None).unwrap(); + assert_eq!(selected.len(), 8192); + assert_eq!( + selected + .iter() + .map(|input| input.logical_path.as_os_str().len() + + input.canonical_source.as_os_str().len()) + .sum::(), + 8 * 1024 * 1024 + ); + assert!(selected + .iter() + .all(|input| input.kind == InputKind::Configuration)); + assert!(selected[1..] + .iter() + .all(|input| input.logical_path == role && input.canonical_source == role)); + let metadata_overflow = root.join("m".repeat(length)); + std::fs::write(&metadata_overflow, document(8191, true)).unwrap(); + assert_eq!( + select_inputs(&[layer(metadata_overflow, true)], owned.path(), None).err(), + Some("Codex selected input metadata budget exceeded") + ); + let event_overflow = root.join("e".repeat(length)); + std::fs::write(&event_overflow, document(8192, false)).unwrap(); + let unread = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + std::fs::File::open(&role) + .unwrap() + .set_times(std::fs::FileTimes::new().set_accessed(unread)) + .unwrap(); + assert_eq!( + select_inputs(&[layer(event_overflow, true)], owned.path(), None).err(), + Some("Codex selected role event budget exceeded") + ); + assert_eq!( + std::fs::metadata(&role).unwrap().accessed().unwrap(), + unread + ); + assert_eq!(std::fs::read(&configuration).unwrap(), raw.as_bytes()); + assert_eq!( + selected.len(), + 8192, + "failed selections leave the prior retained selection intact" + ); + } + #[cfg(unix)] #[test] fn managed_provider_codex_trust_selects_native_canonical_and_git_fallback_resources() { From 042833973ba2e14dbb26617ac6e5119700a512e6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 19:14:34 -0700 Subject: [PATCH 167/493] test(managed-provider): exercise actual merged input environment limits --- .../src/provider_input_preparation.rs | 207 ++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 18b9a3083..e15211719 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -524,6 +524,213 @@ mod tests { use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + #[cfg(target_os = "linux")] + #[test] + fn provider_secret_actual_merged_profile_reaches_guarded_child() { + const GENERATED: &str = "FRESHELL_CONFIG_REFERENCE_0"; + const RECEIPT: &str = "HOST_MERGED_PROFILE="; + if std::env::var_os(GENERATED).is_some() { + let received: BTreeMap = std::env::vars().collect(); + ProviderPathInputs::new(received.clone()).unwrap(); + let parsed: serde_json::Value = serde_json::from_str(&received[GENERATED]).unwrap(); + assert!(parsed["value"] + .as_str() + .unwrap() + .contains("{file:/home/freshell/provider/.freshell/references/0}")); + let total: usize = received + .iter() + .map(|(name, value)| name.len() + value.len()) + .sum(); + println!( + "{RECEIPT}{}", + serde_json::json!({"names":received.len(),"total":total,"maxValue":received.values().map(String::len).max().unwrap()}) + ); + return; + } + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir(&home).unwrap(); + let destination = "/home/freshell/provider/.freshell/references/0"; + let total_value_limit = 256 * 1024 - 127 * (128 + 1920) - GENERATED.len(); + let mut prior = Vec::new(); + for (ordinal, frozen_count, input_bytes, generated_bytes, accepted) in [ + (0, 127, 1920, total_value_limit, true), + (1, 127, 1920, total_value_limit + 1, false), + (2, 128, 16, 128, false), + (3, 1, 16, 64 * 1024, true), + (4, 1, 16, 64 * 1024 + 1, false), + ] { + let stage = owned.path().join(format!("attempt-{ordinal}")); + std::fs::create_dir_all(stage.join(".freshell/source-views")).unwrap(); + let controller = owned.path().join(format!("original-{ordinal}")); + std::fs::create_dir(&controller).unwrap(); + std::fs::write(controller.join("rules"), b"PRIVATE_MERGED_PROFILE_PAYLOAD").unwrap(); + let input = format!( + "{}{{file:rules}}", + "x".repeat(input_bytes - "{file:rules}".len()) + ); + let frozen = ProviderPathInputs::new( + (0..frozen_count) + .map(|index| (format!("N{index:0127}"), input.clone())) + .collect(), + ) + .unwrap(); + let names = frozen + .iter() + .map(|(name, _)| name.to_owned()) + .collect::>(); + let expanded = input.replace("{file:rules}", &format!("{{file:{destination}}}")); + let minimum = serde_json::json!({"value":expanded}).to_string().len(); + let generated_raw = serde_json::json!({"value":format!("{{env:{}}}{}",names[0],"p".repeat(generated_bytes - minimum))}).to_string(); + let shared_raw = serde_json::Value::Object( + names + .iter() + .enumerate() + .map(|(index, name)| { + ( + format!("field-{index:03}"), + serde_json::Value::String(format!("{{env:{name}}}")), + ) + }) + .collect(), + ) + .to_string(); + let approved = controller.join("rules"); + let occurrence = |index| ApprovedFileOccurrence { + ordinal: index, + approved_alias: approved.clone(), + runtime_destination: destination.into(), + }; + let sources = (0..3) + .map(|index| { + let raw = if index == 0 { + &generated_raw + } else { + &shared_raw + }; + let relative = format!(".freshell/source-views/source-{index}"); + std::fs::write(stage.join(&relative), raw).unwrap(); + SourceView { + logical_path: controller.join(if index == 0 { + "generated.json" + } else { + "shared.json" + }), + discovery_role: DiscoveryRole::Workspace, + native_destination: None, + lane: if index == 2 { + SourceLane::Tui + } else { + SourceLane::Main + }, + unexpanded_relative_path: relative, + template_relative_path: format!(".freshell/templates/source-{index}"), + template_sha256: None, + generated_name: (index == 0).then(|| GENERATED.into()), + occurrences: if index == 0 { + vec![occurrence(0)] + } else { + (0..frozen_count).map(occurrence).collect() + }, + referenced_variables: if index == 0 { + vec![names[0].clone()] + } else { + names.clone() + }, + path_variables: if index == 0 { + vec![names[0].clone()] + } else { + names.clone() + }, + original_form: index != 0, + } + }) + .collect(); + let manifest = SourceManifest { + version: 1, + cwd: controller, + original_home: home.clone(), + global_config_root: None, + sources, + files: vec![SelectedSourceFile { + source_id: 0, + canonical_source: approved.clone(), + }], + frozen_inputs: names, + user_owns_freshell_mcp: true, + native_source_flags: vec![], + resource_roots: vec![], + rebind_enabled: true, + inline_edits: Default::default(), + }; + let bytes = bounded_json_bytes(&manifest, 64 * 1024 * 1024).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let context:ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"opencode":{"project_config":[],"tui_config":null,"tui_source":null,"inline_config":false}}, + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&bytes)),"sourceCount":1,"aliasCount":0,"recipeCount":1}, + "indexedFiles":{"volumeName":indexed_provider_volume_name(&SoulId::new(),&generation),"aliases":[]}} + })).unwrap(); + let mut secrets = ResolvedProviderSecrets::default(); + let result = prepare(Some(&context), &mut secrets, &frozen, &stage, &home); + if accepted { + let environment = result.unwrap(); + assert_eq!(environment.len(), frozen_count + 1); + assert_eq!(environment[GENERATED].len(), generated_bytes); + assert!(frozen + .iter() + .all(|(name, value)| environment.get(name).map(String::as_str) == Some(value))); + let binary = std::env::current_exe().unwrap(); + let args: Vec = vec!["--exact".into(),"provider_input_preparation::tests::provider_secret_actual_merged_profile_reaches_guarded_child".into(),"--nocapture".into()]; + let mut command = std::process::Command::new(binary); + command.args(&args).env_clear().envs(&environment); + freshell_platform::provider_exec::check_command(&command, false, &[]).unwrap(); + let output = command.output().unwrap(); + assert!( + output.status.success(), + "guarded merged-profile child failed" + ); + let stdout = String::from_utf8(output.stdout).unwrap(); + let receipt: serde_json::Value = serde_json::from_str( + stdout + .lines() + .find_map(|line| line.split_once(RECEIPT).map(|(_, value)| value)) + .unwrap(), + ) + .unwrap(); + let total = environment + .iter() + .map(|(name, value)| name.len() + value.len()) + .sum::(); + assert_eq!( + receipt, + serde_json::json!({"names":environment.len(),"total":total,"maxValue":environment.values().map(String::len).max().unwrap()}) + ); + println!("HOST_MERGED_PROFILE_QUALIFIED={receipt}"); + if ordinal == 0 { + assert_eq!(total, 256 * 1024); + prior = (0..3) + .map(|index| stage.join(format!(".freshell/source-views/source-{index}"))) + .chain([stage.join(".freshell/input-manifest.json"), approved]) + .map(|path| { + let bytes = std::fs::read(&path).unwrap(); + (path, bytes) + }) + .collect(); + } + } else { + let error = result.err().unwrap(); + assert!(error.contains("profile") || error.contains("64 KiB")); + assert!(!error.contains("PRIVATE_MERGED_PROFILE_PAYLOAD")); + assert!(secrets.opencode_plan.is_none()); + for (path, bytes) in &prior { + assert_eq!(std::fs::read(path).unwrap(), *bytes); + } + } + } + } + #[tokio::test] async fn provider_secret_claude_final_source_gate_blocks_unrecorded_generation_and_keeps_private_inputs( ) { From 237c8b25393e79285a7919134ac94bde9f19ed83 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 19:27:23 -0700 Subject: [PATCH 168/493] test(managed-provider): exercise both stages at metadata capacity --- .../src/provider_environment.rs | 168 ++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 568129fd6..2d115bdd3 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -1589,6 +1589,174 @@ mod tests { assert_eq!(fs::read_dir(&home).unwrap().count(), 1); } + #[test] + fn default_copy_actual_metadata_capacity_preserves_both_stages_on_one_byte_overflow() { + fn destination(length: usize) -> PathBuf { + assert!((402..=656).contains(&length)); + PathBuf::from("a".repeat(200)) + .join("b".repeat(199)) + .join("c".repeat(length - 401)) + } + fn name(index: usize, length: usize) -> String { + let prefix = format!("f{index:06}"); + assert!((prefix.len()..=255).contains(&length)); + format!("{prefix}{}", "x".repeat(length - prefix.len())) + } + fn verify(directory: &Path, names: &[String]) { + assert_eq!(fs::read_dir(directory).unwrap().count(), names.len()); + for (index, name) in names.iter().enumerate() { + assert_eq!( + fs::read(directory.join(name)).unwrap(), + (index as u64).to_le_bytes() + ); + } + } + fn verify_projection(root: &Path, relative: &Path, alias_length: usize, names: &[String]) { + let alias = root.join("z".repeat(alias_length)); + assert!(fs::symlink_metadata(&alias) + .unwrap() + .file_type() + .is_symlink()); + assert_eq!( + fs::canonicalize(&alias).unwrap(), + fs::canonicalize(root.join(relative)).unwrap() + ); + verify(&root.join(relative), names); + verify(&alias, names); + assert_eq!(fs::read_dir(root).unwrap().count(), 2); + } + fn mark_unread(directory: &Path, names: &[String]) -> std::time::SystemTime { + let before = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); + for name in names { + File::options() + .write(true) + .open(directory.join(name)) + .unwrap() + .set_times(fs::FileTimes::new().set_accessed(before)) + .unwrap(); + } + before + } + fn verify_unread(directory: &Path, names: &[String], before: std::time::SystemTime) { + for name in names { + assert_eq!( + fs::metadata(directory.join(name)) + .unwrap() + .accessed() + .unwrap(), + before + ); + } + } + fn selections(source: &Path, relative: &Path, alias_length: usize) -> [CopySelection; 2] { + [ + CopySelection { + source: source.into(), + destination_relative: relative.into(), + }, + selection(source, &"z".repeat(alias_length)), + ] + } + + let root = executable_fixture(); + let mut source = root.path().join("s".repeat(200)).join("t".repeat(200)); + let remaining = 500usize.checked_sub(source.as_os_str().len() + 1).unwrap(); + assert!((1..=255).contains(&remaining)); + source.push("u".repeat(remaining)); + fs::create_dir_all(&source).unwrap(); + assert_eq!(source.as_os_str().len(), 500); + assert_eq!(fs::canonicalize(&source).unwrap().as_os_str().len(), 500); + let first_relative = destination(500); + assert_eq!(first_relative.as_os_str().len(), 500); + // Both graphs retain a 1000-byte source/destination path pair, 600 + // bytes of connecting parents, and a 508-byte selected directory alias + // (including the retained separator on its mapped directory target). + // Ten adjustable physical filenames calibrate the real graph at 64 MiB. + let limit = 64 * 1024 * 1024usize; + let base = 2 * 1000 + 600 + 508; + let full_count = (limit - base - 10 * 1002 - 2 * 160) / 1402; + let mut extra = (limit - base - full_count * 1402 - 10 * 1002) / 2; + let mut names: Vec<_> = (0..full_count).map(|index| name(index, 200)).collect(); + for index in 0..10 { + let length = (extra - (9 - index) * 7).min(255); + names.push(name(full_count + index, length)); + extra -= length; + } + assert_eq!(extra, 0); + assert_eq!( + base + names + .iter() + .map(|name| 1002 + 2 * name.len()) + .sum::(), + limit + ); + for (index, name) in names.iter().enumerate() { + fs::write(source.join(name), (index as u64).to_le_bytes()).unwrap(); + assert_eq!( + fs::canonicalize(source.join(name)) + .unwrap() + .as_os_str() + .len(), + 501 + name.len() + ); + } + let stage = root.path().join("stage"); + let first = copy_environment( + &selections(&source, &first_relative, 7), + &stage, + &[], + CopyLimits::default(), + ) + .unwrap(); + assert_eq!( + (first.entries, first.bytes), + (names.len() + 4, (8 * names.len()) as u64) + ); + let staged_source = stage.join(&first_relative); + verify_projection(&stage, &first_relative, 7, &names); + let second_relative = destination(500 - stage.as_os_str().len() - 1); + let second_alias_length = 507 - second_relative.as_os_str().len(); + let home = root.path().join("home"); + let second = copy_environment( + &selections(&staged_source, &second_relative, second_alias_length), + &home, + &[], + CopyLimits::default(), + ) + .unwrap(); + assert_eq!((first.entries, first.bytes), (second.entries, second.bytes)); + verify_projection(&home, &second_relative, second_alias_length, &names); + // Only the logical alias grows by one byte. The underlying file names + // and contents remain unchanged; the two rejected attempts must not + // open any regular source file or alter either previous publication. + for (source, destination, relative, alias_length) in [ + (&source, &stage, &first_relative, 7), + (&staged_source, &home, &second_relative, second_alias_length), + ] { + let before = mark_unread(source, &names); + let failure = copy_environment( + &selections(source, relative, alias_length + 1), + destination, + &[], + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(failure.limit_kind, Some("metadata_bytes")); + assert_eq!( + (failure.observed, failure.permitted), + (Some(limit as u64 + 1), Some(limit as u64)) + ); + verify_unread(source, &names, before); + verify_projection(destination, relative, alias_length, &names); + assert!(!destination.join("z".repeat(alias_length + 1)).exists()); + } + assert_eq!(fs::read_dir(root.path()).unwrap().count(), 3); + println!( + "RESOURCE_METADATA_CAPACITY={}", + serde_json::json!({"bytes":limit,"entries":first.entries,"fileBytes":first.bytes,"copies":2,"oneByteOverRejected":2}) + ); + } + fn copy_default_twice(source: &Path, root: &Path) -> (CopyReport, CopyReport, PathBuf) { let stage = root.join("stage"); let first = copy_environment( From 7e573df3fbb96605905bea9bd166c8abf5bd41d8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 8 Oct 2026 20:19:23 -0700 Subject: [PATCH 169/493] Check publisher source capacity before payload reads --- .../src/backend/input_publisher.rs | 123 +++++++++++++++--- 1 file changed, 105 insertions(+), 18 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index a58826716..87a4891fc 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -88,6 +88,39 @@ fn environment(value: &Value) -> Option> { Some(result) } +async fn open_approved_source( + path: &Path, + remaining: u64, +) -> Result, BackendError> { + if std::fs::canonicalize(path).map_err(|_| fail("approved input source unavailable"))? != path { + return Err(fail("approved input source selection changed")); + } + let limit = MAX_FILE_BYTES.min(remaining); + let metadata = tokio::fs::metadata(path) + .await + .map_err(|_| fail("approved input source metadata failed"))?; + if !metadata.is_file() { + return Err(fail("approved input source is not regular")); + } + if metadata.len() > limit { + return Err(fail("approved input source actual bytes exceed profile")); + } + let file = tokio::fs::File::open(path) + .await + .map_err(|_| fail("approved input source read failed"))?; + let opened = file + .metadata() + .await + .map_err(|_| fail("approved input source metadata failed"))?; + if !opened.is_file() { + return Err(fail("approved input source is not regular")); + } + if opened.len() > limit { + return Err(fail("approved input source actual bytes exceed profile")); + } + Ok(file.take(limit + 1)) +} + pub(super) fn owned_helper(actual: &Value, expected: &Value) -> bool { let mounts_match = actual["HostConfig"]["Mounts"] .as_array() @@ -455,25 +488,8 @@ impl DockerEngineBackend { .await?; let mut total = 0u64; for (source_id, path) in sources.iter().enumerate() { - if std::fs::canonicalize(path) - .map_err(|_| fail("approved input source unavailable"))? - != *path - { - return Err(fail("approved input source selection changed")); - } - let file = tokio::fs::File::open(path) - .await - .map_err(|_| fail("approved input source read failed"))?; - if !file - .metadata() - .await - .map_err(|_| fail("approved input source metadata failed"))? - .is_file() - { - return Err(fail("approved input source is not regular")); - } let mut file = - file.take(MAX_FILE_BYTES.min(MAX_TOTAL_BYTES.saturating_sub(total)) + 1); + open_approved_source(path, MAX_TOTAL_BYTES.saturating_sub(total)).await?; request( &mut stream, &Request::Open { @@ -691,6 +707,76 @@ impl DockerEngineBackend { mod tests { use super::*; + #[tokio::test] + async fn provider_input_publisher_rechecks_actual_capacity_before_payload_read() { + use std::os::unix::fs::MetadataExt; + use std::time::{Duration, UNIX_EPOCH}; + + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("approved-source"); + for (bytes, remaining) in [ + (MAX_FILE_BYTES + 1, MAX_TOTAL_BYTES), + (MAX_FILE_BYTES, MAX_FILE_BYTES - 1), + ] { + let file = std::fs::File::create(&source).unwrap(); + file.set_len(bytes).unwrap(); + file.set_times( + std::fs::FileTimes::new().set_accessed(UNIX_EPOCH + Duration::from_secs(1)), + ) + .unwrap(); + drop(file); + let result = open_approved_source(&source, remaining).await; + let rejected = match result { + Err(error) => { + assert!(error.to_string().contains("exceed")); + true + } + Ok(mut reader) => { + let mut byte = [0]; + reader.read_exact(&mut byte).await.unwrap(); + false + } + }; + assert!( + rejected, + "fresh oversized payload must be rejected before reading" + ); + assert_eq!(std::fs::metadata(&source).unwrap().atime(), 1); + } + + let file = std::fs::File::create(&source).unwrap(); + file.set_len(MAX_FILE_BYTES).unwrap(); + drop(file); + let mut reader = open_approved_source(&source, MAX_FILE_BYTES).await.unwrap(); + let mut bytes = [0; MAX_CHUNK_BYTES]; + let mut observed = 0u64; + loop { + let count = reader.read(&mut bytes).await.unwrap(); + if count == 0 { + break; + } + assert!(bytes[..count].iter().all(|byte| *byte == 0)); + observed += count as u64; + } + assert_eq!(observed, MAX_FILE_BYTES); + + std::fs::write(&source, []).unwrap(); + let mut reader = open_approved_source(&source, 0).await.unwrap(); + assert_eq!(reader.read(&mut [0]).await.unwrap(), 0); + + // Growth after acceptance remains bounded by the same one-byte + // lookahead used by the publisher's streaming count checks. + let mut reader = open_approved_source(&source, 1).await.unwrap(); + std::fs::OpenOptions::new() + .write(true) + .open(&source) + .unwrap() + .set_len(3) + .unwrap(); + assert_eq!(reader.read(&mut bytes).await.unwrap(), 2); + assert_eq!(reader.read(&mut bytes).await.unwrap(), 0); + } + #[test] fn provider_input_helper_accepts_docker_inherited_labels_and_omitted_false_without_losing_owner_fences( ) { @@ -976,5 +1062,6 @@ mod tests { .is_ok()); drop(backend); server.abort(); + let _ = server.await; } } From eeb0fe06865c26c3093eb28ada1073436eb3b9f9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:42:44 -0700 Subject: [PATCH 170/493] fix: prepare private provider input parent on exact resume --- crates/freshell-session-host/src/main.rs | 98 +++++++++++++++++++----- 1 file changed, 80 insertions(+), 18 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 58cc83707..c644a5c2b 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -2017,14 +2017,7 @@ fn refresh_provider_context( let Some(context) = context else { return Ok(()); }; - let private_mount = &paths.private_mount; - let user_root = private_mount.join("user-provider"); - if provider_config_references(context).iter().any(|reference| { - reference.root == freshell_runtime_protocol::ProviderConfigRoot::UserProvider - }) && !user_root.is_dir() - { - return Err("provider user config mount is unavailable".into()); - } + let user_root = prepare_provider_input_mount(context, paths)?; refresh_provider_config_references( context, provider, @@ -2544,17 +2537,10 @@ fn prepare_provider_context_for_fresh_launch( paths, ) } -fn prepare_provider_context_at( - context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, - provider: &str, - workspace: &str, - run_as_uid: u32, - run_as_gid: u32, +fn prepare_provider_input_mount( + context: &freshell_runtime_protocol::ProviderLaunchContext, paths: &ProviderConfigPaths, -) -> Result<(), String> { - let Some(context) = context else { - return Ok(()); - }; +) -> Result { let user_root = paths.private_mount.join("user-provider"); if provider_config_references(context).iter().any(|reference| { reference.root == freshell_runtime_protocol::ProviderConfigRoot::UserProvider @@ -2589,6 +2575,21 @@ fn prepare_provider_context_at( })?; set_mode(&paths.private_mount, 0o710)?; } + Ok(user_root) +} + +fn prepare_provider_context_at( + context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, + provider: &str, + workspace: &str, + run_as_uid: u32, + run_as_gid: u32, + paths: &ProviderConfigPaths, +) -> Result<(), String> { + let Some(context) = context else { + return Ok(()); + }; + let user_root = prepare_provider_input_mount(context, paths)?; refresh_provider_config_references( context, provider, @@ -4418,6 +4419,67 @@ mod tests { ); } + #[cfg(unix)] + #[test] + fn provider_config_resume_validates_private_input_parent_before_copy() { + use freshell_runtime_protocol::{ + ProviderConfigReference, ProviderConfigRoot, ProviderLaunchContext, ProviderPreparation, + }; + let root = tempfile::tempdir().unwrap(); + let real_private = root.path().join("real-private"); + let private = root.path().join("private"); + let home = root.path().join("home"); + std::fs::create_dir_all(real_private.join("user-provider")).unwrap(); + std::fs::create_dir(&home).unwrap(); + std::fs::write( + real_private.join("user-provider/settings.json"), + b"new-config", + ) + .unwrap(); + std::fs::create_dir(home.join(".claude")).unwrap(); + std::fs::write(home.join(".claude/settings.json"), b"accepted-config").unwrap(); + std::os::unix::fs::symlink(&real_private, &private).unwrap(); + let context = ProviderLaunchContext { + prepared_inputs: None, + preparation: ProviderPreparation::Claude { + mcp_args: Vec::new(), + }, + mcp_capability: None, + config: vec![ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "settings.json".into(), + provider_relative_path: ".claude/settings.json".into(), + format: "json".into(), + }], + }; + let error = refresh_provider_context( + Some(&context), + "claude", + root.path().to_str().unwrap(), + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + &ProviderConfigPaths { + private_mount: private, + provider_home: home.clone(), + }, + ) + .expect_err("resume must validate the same physical private parent as initial preparation"); + assert!(error.contains("trusted physical directory")); + assert_eq!( + std::fs::read(home.join(".claude/settings.json")).unwrap(), + b"accepted-config" + ); + assert_eq!( + std::fs::read(real_private.join("user-provider/settings.json")).unwrap(), + b"new-config" + ); + assert!(std::fs::read_dir(&home).unwrap().all(|entry| !entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with(".freshell-provider-refresh-"))); + } + #[test] fn provider_config_exact_resume_refreshes_changes_and_removes_stale_projection() { use freshell_runtime_protocol::{ From deb576cf6673a1c4d7594defd94e2ccc9a52b959 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:43:02 -0700 Subject: [PATCH 171/493] fix: retain accepted Amplifier resources when original inputs are unavailable --- .../src/managed_mcp_capability.rs | 97 ++++++++++++++++--- .../managed_mcp_capability/amplifier_stage.rs | 80 ++++++++++++++- .../managed_mcp_capability/ordinary_stage.rs | 2 +- 3 files changed, 165 insertions(+), 14 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 1bbd642fc..26b3b2f69 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1535,19 +1535,37 @@ fn stage_provider_root( .map_err(|_| "prepared resource descriptor exceeds control profile")?; write_private_bytes(&temporary.join(".freshell/resource-mounts.json"), &metadata)?; } + let retained_amplifier = + provider == "amplifier" && canonical_root.is_none() && prior_stage.is_some(); let ordinary_resources = if opencode_input.launch_context.is_some() && matches!(provider, "codex" | "claude" | "amplifier") { Some(if provider == "amplifier" { - amplifier_stage::prepare( - home, - opencode_input.cwd.unwrap_or(home), - opencode_input - .workspace - .or(opencode_input.cwd) - .unwrap_or(home), - opencode_input.amplifier_manifest, - )? + if retained_amplifier { + tracing::info!( + event = "provider.inputs.preparation.retained", + stage = "controller", + provider, + reason = "original_root_unavailable", + "Reusing the accepted Amplifier ordinary resource generation" + ); + amplifier_stage::prepare_retained( + opencode_input + .amplifier_manifest + .ok_or("Amplifier retained original origins unavailable")?, + prior_stage.ok_or("Amplifier retained ordinary stage unavailable")?, + )? + } else { + amplifier_stage::prepare( + home, + opencode_input.cwd.unwrap_or(home), + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + opencode_input.amplifier_manifest, + )? + } } else if provider == "claude" { if let Some(prior_stage) = prior_stage { claude_stage::prepare_retained( @@ -1603,7 +1621,7 @@ fn stage_provider_root( if let Some(resources) = &ordinary_resources { selections.extend(resources.selections.iter().cloned()); } - let amplifier_resources = if provider == "amplifier" { + let amplifier_resources = if provider == "amplifier" && !retained_amplifier { Some(amplifier_resources::prepare( home, opencode_input.cwd.unwrap_or(home), @@ -8815,9 +8833,13 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ] { std::fs::create_dir_all(path).unwrap(); } + let bundle = home.join(".amplifier/bundles/selected"); + std::fs::create_dir_all(&bundle).unwrap(); + std::fs::write(bundle.join("bundle.yaml"), "bundle:\n name: retained\n").unwrap(); + std::fs::write(bundle.join("resource"), b"retained-bundle-resource").unwrap(); std::fs::write( home.join(".amplifier/settings.yaml"), - r#"{"env":{"CLAUDE_SELECTED":"original"}}"#, + format!("bundle:\n active: file://{}\n", bundle.display()), ) .unwrap(); std::fs::write( @@ -8860,6 +8882,8 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(); let context = read_prepared_context(&directory, &prior).unwrap(); assert!(context.prepared_inputs.is_some()); + let prior_inputs = context.prepared_inputs.clone().unwrap(); + std::fs::rename(&home, owned.path().join("unavailable-home")).unwrap(); let callback=serde_json::to_vec(&serde_json::json!({"action":"issue","controlSecret":"fixture-control","soulId":soul,"provider":"amplifier","workspace":workspace,"cwd":workspace,"priorContext":context,"providerSecretReferences":[]})).unwrap(); let client = freshell_runtime_client::RuntimeClient::new( owned.path().join("unused.sock"), @@ -8876,7 +8900,12 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ) .await .expect("replacement must select recorded original Amplifier sources"); - std::fs::remove_file(home.join(".amplifier/settings.yaml")).unwrap(); + std::fs::remove_file( + owned + .path() + .join("unavailable-home/.amplifier/settings.yaml"), + ) + .unwrap(); let repeated = apply_prepared_callback( &callback, "fixture-control", @@ -8923,6 +8952,50 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .unwrap(), b"retained-original-resource" ); + assert_eq!(inputs.config_binds, prior_inputs.config_binds); + assert_eq!(inputs.resource_mounts, prior_inputs.resource_mounts); + assert_ne!(inputs.generation_id, prior_inputs.generation_id); + let projected = inputs + .resource_mounts + .iter() + .find(|mount| { + mount.logical_destination == "/home/freshell/provider/.freshell/resources" + }) + .unwrap(); + assert_eq!( + std::fs::read( + stage + .join(&projected.stage_relative_path) + .join("amplifier-local-0/resource") + ) + .unwrap(), + b"retained-bundle-resource" + ); + assert!(!home.exists()); + let prior_stage = staged_provider_root_path(&directory, &prior.grant_id).unwrap(); + let manifest = prior_stage.join(&prior_inputs.manifest.relative_path); + std::fs::write(&manifest, b"{}").unwrap(); + assert!(amplifier_stage::prepare_retained( + &freshell_platform::amplifier_inputs::AmplifierInputManifest { + version: 1, + original_home: home.clone(), + cwd: workspace.clone(), + workspace, + }, + &prior_stage, + ) + .err() + .expect("changed retained manifest must block preparation") + .contains("manifest changed")); + assert_eq!( + std::fs::read( + stage + .join(&projected.stage_relative_path) + .join("amplifier-local-0/resource") + ) + .unwrap(), + b"retained-bundle-resource" + ); } #[tokio::test] diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs index 53fb821c2..ae33cad0f 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs @@ -1,11 +1,89 @@ //! Existing Amplifier projections share the ordinary manifest/copier boundary. use super::ordinary_stage::{prepare_selected, Preparation}; -use freshell_platform::amplifier_inputs::AmplifierInputManifest; +use freshell_platform::{ + amplifier_inputs::AmplifierInputManifest, provider_environment::CopySelection, +}; use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; use std::path::Path; pub(super) const PROJECTED: &str = ".freshell/resource-units/amplifier-projected"; +/// The accepted ordinary stage remains authoritative when its original root +/// is unavailable. Preserve its exact selected routes and complete resource +/// closure; native session state and credential files never enter this stage. +pub(super) fn prepare_retained( + accepted: &AmplifierInputManifest, + prior_stage: &Path, +) -> Result { + use freshell_runtime_protocol::{ProviderLaunchContext, MAX_CONTROL_FRAME_BYTES}; + use sha2::{Digest, Sha256}; + accepted.validate().map_err(str::to_string)?; + let context: ProviderLaunchContext = serde_json::from_slice(&super::read_prepared_metadata( + prior_stage, + ".freshell/prepared-context.json", + MAX_CONTROL_FRAME_BYTES, + )?) + .map_err(|_| "Amplifier retained context invalid; retain inputs and reprepare")?; + context + .validate("amplifier") + .map_err(|error| error.message)?; + let inputs = context + .prepared_inputs + .ok_or("Amplifier retained inputs unavailable")?; + if prior_stage.file_name().and_then(|name| name.to_str()) != Some(&inputs.generation_id) { + return Err("Amplifier retained generation changed".into()); + } + let manifest_bytes = super::read_prepared_metadata( + prior_stage, + &inputs.manifest.relative_path, + 64 * 1024 * 1024, + )?; + if format!("{:x}", Sha256::digest(&manifest_bytes)) != inputs.manifest.sha256 { + return Err("Amplifier retained source manifest changed".into()); + } + let prior: AmplifierInputManifest = serde_json::from_slice(&manifest_bytes) + .map_err(|_| "Amplifier retained source manifest invalid")?; + prior.validate().map_err(str::to_string)?; + if prior.original_home != accepted.original_home + || prior.cwd != accepted.cwd + || prior.workspace != accepted.workspace + { + return Err("Amplifier retained original origins changed".into()); + } + let mut selections = Vec::new(); + let mut paths = std::collections::BTreeSet::new(); + for relative in inputs + .config_binds + .iter() + .map(|bind| &bind.source_relative_path) + .chain( + inputs + .resource_mounts + .iter() + .map(|mount| &mount.stage_relative_path), + ) + { + if !paths.insert(relative) { + continue; + } + let source = std::fs::canonicalize(prior_stage.join(relative)) + .map_err(|_| "Amplifier retained selected source unavailable")?; + if !source.starts_with(prior_stage) { + return Err("Amplifier retained selected source escaped its stage".into()); + } + selections.push(CopySelection { + source, + destination_relative: relative.into(), + }); + } + Ok(Preparation { + manifest_bytes, + selections, + binds: inputs.config_binds, + mounts: inputs.resource_mounts, + }) +} + pub(super) fn prepare( home: &Path, cwd: &Path, diff --git a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs index 52d53fea3..86466e0c9 100644 --- a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs @@ -10,7 +10,7 @@ use std::{ }; pub(super) struct Preparation { - manifest_bytes: Vec, + pub(super) manifest_bytes: Vec, pub selections: Vec, pub binds: Vec, pub mounts: Vec, From d9d66e18bdba6c590b5ac0dd34f6b12af91c0f7c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:10:29 -0700 Subject: [PATCH 172/493] fix: inspect accepted provider mounts without republishing sources --- crates/freshell-supervisor/src/backend.rs | 12 +- .../freshell-supervisor/src/backend/docker.rs | 240 +++++++++++++++++- 2 files changed, 242 insertions(+), 10 deletions(-) diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 3694ece91..22504e6b0 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -311,8 +311,8 @@ impl DockerEngineBackend { } verify_inspect_config(handle, &value)?; let mounts = match (handle.terminal(), handle.fresh_agent()) { - (Some(terminal), None) => Some(docker::terminal_mounts(terminal)), - (None, Some(agent)) => Some(docker::fresh_agent_mounts(agent)), + (Some(terminal), None) => Some(docker::terminal_mounts_for_adoption(terminal)), + (None, Some(agent)) => Some(docker::fresh_agent_mounts_for_adoption(agent)), _ => None, } .transpose() @@ -1434,13 +1434,13 @@ fn verify_inspect_config(handle: &OwnedRuntimeHandle, value: &Value) -> Result<( )); } if let Some(terminal) = handle.terminal() { - let expected = - docker::terminal_mounts(terminal).map_err(BackendError::OwnershipMismatch)?; + let expected = docker::terminal_mounts_for_adoption(terminal) + .map_err(BackendError::OwnershipMismatch)?; verify_workload_mounts(mounts, &expected)?; } if let Some(agent) = handle.fresh_agent() { - let expected = - docker::fresh_agent_mounts(agent).map_err(BackendError::OwnershipMismatch)?; + let expected = docker::fresh_agent_mounts_for_adoption(agent) + .map_err(BackendError::OwnershipMismatch)?; verify_workload_mounts(mounts, &expected)?; } Ok(()) diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 1ae29ae0e..e2cf3d40d 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -52,7 +52,26 @@ pub struct ResolvedProviderConfigBind { pub reference: freshell_runtime_protocol::ProviderConfigBind, } +#[derive(Clone, Copy, PartialEq, Eq)] +enum MountPurpose { + Create, + Adopt, +} + pub fn terminal_mounts(spec: &TerminalLaunchSpec) -> Result { + terminal_mounts_for(spec, MountPurpose::Create) +} + +pub(super) fn terminal_mounts_for_adoption( + spec: &TerminalLaunchSpec, +) -> Result { + terminal_mounts_for(spec, MountPurpose::Adopt) +} + +fn terminal_mounts_for( + spec: &TerminalLaunchSpec, + purpose: MountPurpose, +) -> Result { workload_mounts( &spec.workspace_path, &spec.cwd, @@ -61,10 +80,24 @@ pub fn terminal_mounts(spec: &TerminalLaunchSpec) -> Result Result { + fresh_agent_mounts_for(spec, MountPurpose::Create) +} + +pub(super) fn fresh_agent_mounts_for_adoption( + spec: &FreshAgentLaunchSpec, +) -> Result { + fresh_agent_mounts_for(spec, MountPurpose::Adopt) +} + +fn fresh_agent_mounts_for( + spec: &FreshAgentLaunchSpec, + purpose: MountPurpose, +) -> Result { workload_mounts( &spec.workspace_path, &spec.cwd, @@ -73,6 +106,7 @@ pub fn fresh_agent_mounts(spec: &FreshAgentLaunchSpec) -> Result, provider: &str, + purpose: MountPurpose, ) -> Result { if let Some(context) = context { context.validate(provider).map_err(|error| error.message)?; @@ -121,6 +156,11 @@ fn workload_mounts( } let mut provider_bootstrap_files = Vec::with_capacity(bootstrap_files.len()); for file in bootstrap_files { + if purpose == MountPurpose::Adopt { + let source = accepted_source_path(&file.source_path)?; + provider_bootstrap_files.push(source); + continue; + } let source = std::fs::canonicalize(&file.source_path) .map_err(|error| format!("provider bootstrap file: {error}"))?; if !source.is_file() || source != Path::new(&file.source_path) { @@ -136,6 +176,14 @@ fn workload_mounts( .is_some_and(|inputs| inputs.indexed_files.is_some()); let mut source_id = 0u16; for (index, secret) in secret_references.iter().enumerate() { + if purpose == MountPurpose::Adopt { + provider_secret_files.push(accepted_source_path(&secret.source_path)?); + if indexed && secret.profile.is_configuration_file() { + indexed_secret_sources.insert(index, source_id); + source_id += 1; + } + continue; + } let source = std::fs::canonicalize(&secret.source_path) .map_err(|error| format!("provider secret reference: {error}"))?; if !source.is_file() || source != Path::new(&secret.source_path) { @@ -164,9 +212,10 @@ fn workload_mounts( } let provider_user_root = if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { - Some(approved_staged_provider_root( - &capability_directory()?, + Some(staged_provider_root_for( + &capability_directory_for(purpose)?, &capability.grant_id, + purpose, )?) } else if context.is_some() { let home = std::env::var_os("HOME"); @@ -186,7 +235,7 @@ fn workload_mounts( let mcp_capability_file = context .and_then(|context| context.mcp_capability.as_ref()) .map(|capability| { - let directory = capability_directory()?; + let directory = capability_directory_for(purpose)?; if !capability.grant_id.starts_with("grant-") || capability.grant_id.len() > 128 || !capability @@ -197,6 +246,9 @@ fn workload_mounts( return Err("invalid managed MCP grant id".to_string()); } let source = directory.join(format!("{}.json", capability.grant_id)); + if purpose == MountPurpose::Adopt { + return Ok(source); + } let canonical = std::fs::canonicalize(&source) .map_err(|error| format!("managed MCP grant unavailable: {error}"))?; if canonical != source || !canonical.is_file() { @@ -218,10 +270,12 @@ fn workload_mounts( Ok(canonical) }) .transpose()?; - let prepared = prepared_input_mounts( + let prepared = prepared_input_mounts_for( &workspace, provider_user_root.as_deref(), context.and_then(|context| context.prepared_inputs.as_ref()), + purpose, + context.and_then(|context| context.mcp_capability.as_ref()), )?; Ok(TerminalMounts { workspace, @@ -296,10 +350,40 @@ fn prepared_input_mounts( workspace: &Path, root: Option<&Path>, inputs: Option<&freshell_runtime_protocol::PreparedProviderInputs>, +) -> Result { + prepared_input_mounts_for(workspace, root, inputs, MountPurpose::Create, None) +} + +fn prepared_input_mounts_for( + workspace: &Path, + root: Option<&Path>, + inputs: Option<&freshell_runtime_protocol::PreparedProviderInputs>, + purpose: MountPurpose, + capability: Option<&freshell_runtime_protocol::McpCapabilityReference>, ) -> Result { let mut prepared = PreparedInputMounts::default(); if let Some(inputs) = inputs { let root = root.ok_or("prepared provider stage is unavailable")?; + if purpose == MountPurpose::Adopt { + inputs.validate(capability).map_err(|error| error.message)?; + // Existing mounts are verified against these accepted descriptors + // and Docker's actual immutable configuration below. Missing or + // changed source bytes cannot authorize a new launch, but do not + // change the identity of an already mounted, healthy workload. + return Ok(PreparedInputMounts { + input_manifest: Some(root.join(&inputs.manifest.relative_path)), + config_binds: inputs + .config_binds + .iter() + .map(|reference| ResolvedProviderConfigBind { + source: root.join(&reference.source_relative_path), + reference: reference.clone(), + }) + .collect(), + resource_mounts: inputs.resource_mounts.clone(), + indexed_files: inputs.indexed_files.clone(), + }); + } let manifest = canonical_stage_file(root, &inputs.manifest.relative_path)?; verify_input_manifest_digest(&manifest, &inputs.manifest.sha256)?; for reference in &inputs.config_binds { @@ -554,14 +638,135 @@ mod prepared_stage_tests { b"{\"model\":\"original\"}" ); } + + #[test] + fn provider_prepared_adoption_keeps_accepted_routes_when_source_authority_is_lost() { + let (owned, stage, workspace, mut inputs) = fixture(); + let capability = freshell_runtime_protocol::McpCapabilityReference { + grant_id: inputs.generation_id.clone(), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let expected = prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).unwrap(); + std::fs::rename(&stage, owned.path().join("unavailable-stage")).unwrap(); + let adopted = prepared_input_mounts_for( + &workspace, + Some(&stage), + Some(&inputs), + MountPurpose::Adopt, + Some(&capability), + ) + .expect("accepted mounted descriptors do not require source republication"); + assert_eq!(adopted.input_manifest, expected.input_manifest); + assert_eq!(adopted.config_binds, expected.config_binds); + assert_eq!(adopted.resource_mounts, expected.resource_mounts); + assert!( + prepared_input_mounts(&workspace, Some(&stage), Some(&inputs)).is_err(), + "missing authority must still block creating a new runtime" + ); + inputs.config_binds[0].source_relative_path = "../unapproved".into(); + assert!(prepared_input_mounts_for( + &workspace, + Some(&stage), + Some(&inputs), + MountPurpose::Adopt, + Some(&capability) + ) + .is_err()); + assert_eq!( + std::fs::read( + owned + .path() + .join("unavailable-stage/.freshell/templates/source-0") + ) + .unwrap(), + b"{\"model\":\"transported\"}" + ); + } + + #[cfg(unix)] + #[test] + fn provider_prepared_adoption_terminal_and_fresh_use_the_same_accepted_mounts() { + const CHILD: &str = "FRESHELL_ADOPTION_MOUNT_TEST_CHILD"; + if std::env::var_os(CHILD).is_none() { + let status = std::process::Command::new("/usr/bin/timeout") + .args(["--kill-after=1s", "20s"]) + .arg(std::env::current_exe().unwrap()) + .args(["--exact", "backend::docker::prepared_stage_tests::provider_prepared_adoption_terminal_and_fresh_use_the_same_accepted_mounts", "--nocapture"]) + .env(CHILD, "1").status().unwrap(); + assert!(status.success(), "isolated shared adoption behavior failed"); + return; + } + let (owned, stage, workspace, inputs) = fixture(); + let capabilities = owned.path().join("mcp-capabilities"); + let selected = owned + .path() + .join("provider-roots") + .join(&inputs.generation_id); + std::fs::create_dir(&capabilities).unwrap(); + std::fs::create_dir(selected.parent().unwrap()).unwrap(); + std::fs::rename(stage, &selected).unwrap(); + let capability = freshell_runtime_protocol::McpCapabilityReference { + grant_id: inputs.generation_id.clone(), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let grant = capabilities.join(format!("{}.json", capability.grant_id)); + std::fs::write(&grant, b"{}").unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&grant, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + std::env::set_var("FRESHELL_RUNTIME_CAPABILITY_DIR", &capabilities); + let context = ProviderLaunchContext { + preparation: freshell_runtime_protocol::ProviderPreparation::Claude { + mcp_args: vec![], + }, + config: vec![], + mcp_capability: Some(capability), + prepared_inputs: Some(inputs), + }; + let mut terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"adoption-terminal", "streamId":"stream", "mode":"claude", "program":"claude", "args":[], + "cwd":workspace,"workspacePath":workspace,"projectKey":"project", "runAsUid":65534,"runAsGid":0,"cols":80,"rows":24 + })).unwrap(); + terminal.provider_launch_context = Some(context.clone()); + let mut fresh: FreshAgentLaunchSpec = serde_json::from_value(serde_json::json!({ + "sessionId":"adoption-fresh", "provider":"claude", "sessionType":"freshclaude", "runtimeVariant":"claude-sdk", + "providerStoreId":"store", "cwd":workspace,"workspacePath":workspace,"runAsUid":65534,"runAsGid":0 + })).unwrap(); + fresh.provider_launch_context = Some(context); + let expected = terminal_mounts(&terminal).unwrap(); + assert_eq!(fresh_agent_mounts(&fresh).unwrap(), expected); + std::fs::rename(&selected, owned.path().join("unavailable-selected")).unwrap(); + std::fs::rename(&grant, grant.with_extension("unavailable")).unwrap(); + assert_eq!(terminal_mounts_for_adoption(&terminal).unwrap(), expected); + assert_eq!(fresh_agent_mounts_for_adoption(&fresh).unwrap(), expected); + assert!(terminal_mounts(&terminal).is_err()); + assert!(fresh_agent_mounts(&fresh).is_err()); + } } fn capability_directory() -> Result { + capability_directory_for(MountPurpose::Create) +} + +fn capability_directory_for(purpose: MountPurpose) -> Result { let directory = CAPABILITY_DIR .get() .cloned() .or_else(|| std::env::var_os("FRESHELL_RUNTIME_CAPABILITY_DIR").map(PathBuf::from)) .ok_or("managed MCP capability directory is unavailable")?; + if purpose == MountPurpose::Adopt { + return accepted_source_path( + directory + .to_str() + .ok_or("managed capability path is not UTF-8")?, + ); + } std::fs::canonicalize(&directory) .map_err(|error| format!("managed MCP capability directory unavailable: {error}")) } @@ -571,6 +776,14 @@ pub(crate) fn staged_provider_root_for_grant(grant_id: &str) -> Result Result { + staged_provider_root_for(directory, grant_id, MountPurpose::Create) +} + +fn staged_provider_root_for( + directory: &Path, + grant_id: &str, + purpose: MountPurpose, +) -> Result { if !grant_id.starts_with("grant-") || grant_id.len() > 128 || !grant_id @@ -584,6 +797,9 @@ fn approved_staged_provider_root(directory: &Path, grant_id: &str) -> Result Result Result { + let path = Path::new(raw); + if !path.is_absolute() + || path.components().any(|part| { + matches!( + part, + std::path::Component::CurDir | std::path::Component::ParentDir + ) + }) + { + return Err("accepted mount source must be an absolute normal path".into()); + } + reject_management_path(path)?; + Ok(path.into()) +} + fn user_provider_config_referenced(context: &ProviderLaunchContext) -> bool { context .config From 8a7299da8d3630dc1faaf870eec4a3145d370322 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:19:25 -0700 Subject: [PATCH 173/493] test: qualify native Amplifier conversation recovery and frozen inputs --- .../src/managed_provider_launch_tests.rs | 602 ++++++++++++++++-- .../testing/managed-provider-launch-broker.ts | 4 +- .../managed-provider-launch-sandbox.py | 29 +- 3 files changed, 586 insertions(+), 49 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 7bfa7c0b6..3f5b71ddc 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -16,6 +16,33 @@ use std::{ time::Duration, }; +// Read only the fixture's actual native process (or its bounded hook ancestry). +// argv/environment bytes stay transient; only profile compliance is reported. +const NATIVE_EXEC_PROFILE: &str = r#" +function nativeExecProfile(target) { + let pid=target==='self'?process.pid:process.ppid; + if(target==='claude') { + let found=false; + for(let depth=0;depth<16&&pid>1;depth++) { + if(fs.readlinkSync('/proc/'+pid+'/exe').split('/').pop()==='claude') {found=true;break;} + const status=fs.readFileSync('/proc/'+pid+'/status','utf8'); + pid=Number(status.match(/^PPid:\s+(\d+)/m)[1]); + } + if(!found)throw Error('closed native hook ancestry unavailable'); + } + const prefix='/proc/'+pid+'/',argv=fs.readFileSync(prefix+'cmdline'),env=fs.readFileSync(prefix+'environ'); + const strings=[...argv.toString().split('\0').filter(Boolean),...env.toString().split('\0').filter(Boolean)]; + const stack=fs.readFileSync(prefix+'limits','utf8').match(/^Max stack size\s+(\S+)/m)[1]; + const result={stackAtLeast8MiB:stack==='unlimited'||Number(stack)>=8388608,aggregateWithin1MiB:argv.length+env.length+8*(strings.length+2)+Buffer.byteLength(fs.readlinkSync(prefix+'exe'))+1<=1048576,completeStringsWithin32Pages:strings.every(value=>Buffer.byteLength(value)+1<=131072)}; + if(Object.values(result).some(value=>!value))throw Error('closed final native execution profile exceeded'); + return result; +} +"#; + +fn native_exec_profile() -> Value { + json!({"stackAtLeast8MiB":true,"aggregateWithin1MiB":true,"completeStringsWithin32Pages":true}) +} + fn assert_private_sentinels_absent(root: &Path, certificate_sentinel: &[u8]) { for entry in std::fs::read_dir(root).unwrap() { let entry = entry.unwrap(); @@ -615,16 +642,22 @@ async fn managed_provider_amplifier_terminal_native_startup_consumes_prepared_in std::fs::write(skill.join("run"),"#!/usr/bin/env node\nconst fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('probe-dependency'));").unwrap(); std::fs::set_permissions(skill.join("run"), std::fs::Permissions::from_mode(0o755)).unwrap(); let context_module = r#"__amplifier_module_type__='context' +import hashlib,json +from pathlib import Path class Context: - def __init__(self): self.messages=[]; self.factory=None + def __init__(self,session_id,config): self.messages=[]; self.factory=None; self.session_id=session_id; self.config=config async def add_message(self,message): self.messages.append(message) async def get_messages(self): return list(self.messages) async def get_messages_for_request(self,**kwargs): return list(self.messages) - async def set_messages(self,messages): self.messages=list(messages) + async def set_messages(self,messages): + self.messages=list(messages) + pair=[{'role':m.get('role'),'content':m.get('content')} for m in self.messages if m.get('role') in ['user','assistant']] + value={'sessionId':self.session_id,'roles':[m['role'] for m in pair],'messageCount':len(pair),'digest':hashlib.sha256(json.dumps(pair,separators=(',',':'),sort_keys=True).encode()).hexdigest()} + Path(self.config['conversation_receipt']).write_text(json.dumps(value)) async def clear(self): self.messages=[] async def set_system_prompt_factory(self,factory): self.factory=factory def get_context_info(self): return {'message_count':len(self.messages),'token_count':0} -async def mount(coordinator,config=None): await coordinator.mount('context',Context()) +async def mount(coordinator,config=None): await coordinator.mount('context',Context(coordinator.session_id,config or {})) "#; let loop_module = r#"__amplifier_module_type__='orchestrator' class Loop: @@ -632,9 +665,10 @@ class Loop: async def mount(coordinator,config=None): await coordinator.mount('orchestrator',Loop()) "#; let hook_module = r#"__amplifier_module_type__='hook' -import hashlib,json,os,subprocess,urllib.request +import hashlib,json,os,resource,secrets,subprocess,urllib.request from pathlib import Path from amplifier_core import HookResult +from amplifier_app_cli.session_store import SessionStore _consumer=None async def mount(coordinator,config=None): global _consumer @@ -644,7 +678,22 @@ async def mount(coordinator,config=None): output=subprocess.check_output([str(Path.home()/'.amplifier/skills/probe/run')]) request=urllib.request.Request(os.environ['FRESHELL_URL']+'/api/mcp',data=b'{}',headers={'x-auth-token':os.environ['FRESHELL_TOKEN']}) with urllib.request.urlopen(request,timeout=5) as response: authenticated=response.status==200 - result={'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest()} + seed_receipt=Path(config['receipt']).with_suffix('.seed.json') + if not seed_receipt.exists(): + store=SessionStore() + transcript,metadata=store.load(coordinator.session_id) + if transcript: raise RuntimeError('closed seed setup requires an empty native stub') + pair=[{'role':'user','content':secrets.token_hex(16)},{'role':'assistant','content':secrets.token_hex(16)}] + store.save(coordinator.session_id,pair,metadata) + persisted,persisted_metadata=store.load(coordinator.session_id) + if persisted!=pair or persisted_metadata.get('session_id')!=coordinator.session_id: raise RuntimeError('closed native seed verification failed') + value={'sessionId':coordinator.session_id,'roles':[m['role'] for m in pair],'messageCount':len(pair),'digest':hashlib.sha256(json.dumps(pair,separators=(',',':'),sort_keys=True).encode()).hexdigest()} + seed_receipt.write_text(json.dumps(value)) + argv=Path('/proc/self/cmdline').read_bytes(); env=Path('/proc/self/environ').read_bytes(); strings=[value for value in (argv+env).split(b'\x00') if value] + stack=resource.getrlimit(resource.RLIMIT_STACK)[0] + profile={'stackAtLeast8MiB':stack==resource.RLIM_INFINITY or stack>=8388608,'aggregateWithin1MiB':len(argv)+len(env)+8*(len(strings)+2)+len(os.readlink('/proc/self/exe').encode())+1<=1048576,'completeStringsWithin32Pages':all(len(value)+1<=131072 for value in strings)} + if not all(profile.values()): raise RuntimeError('closed final native execution profile exceeded') + result={'finalExecProfile':profile,'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest()} Path(config['receipt']).write_text(json.dumps(result)) return HookResult(action='continue') _consumer=consume @@ -668,7 +717,8 @@ async def on_session_ready(coordinator): } let native_id = "41a04bb3-86f0-463d-97c5-f085e8a9ea4b"; let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let bundle_document=format!("bundle:\n name: closed-native-bundle\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); + let conversation_receipt = workspace.join("amplifier-conversation.json"); + let bundle_document=format!("bundle:\n name: closed-native-bundle\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); std::fs::write(bundle.join("bundle.yaml"), &bundle_document).unwrap(); let settings=format!("bundle:\n active: file://{}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n",bundle.display()); std::fs::write(home.join(".amplifier/settings.yaml"), &settings).unwrap(); @@ -694,28 +744,26 @@ async def on_session_ready(coordinator): std::fs::write(&secret_file, "closed-synthetic-control-secret").unwrap(); std::fs::set_permissions(&secret_file, std::fs::Permissions::from_mode(0o600)).unwrap(); freshell_supervisor::backend::docker::configure_capability_dir(&socket).unwrap(); - let supervisor = Supervisor::new( - registry.clone(), - Arc::new(BrokerAuthorizedBackend { - real: DockerEngineBackend::new(fixture.join("broker.sock")), - control: control.clone(), - handoff: fixture.join("handoff"), - certificate_sentinel: b"synthetic-amplifier-onecli-value".to_vec(), - }), - SupervisorConfig { - runtime_root: runtime, - control_socket_path: socket.clone(), - host_binary_path: PathBuf::from( - std::env::var_os("FRESHELL_PROVIDER_LAUNCH_HOST_BINARY").unwrap(), - ), - image_ref: std::env::var("FRESHELL_PROVIDER_LAUNCH_IMAGE").unwrap(), - test_run_id: std::env::var("FRESHELL_PROVIDER_LAUNCH_RUN_ID").unwrap(), - control_secret: "closed-synthetic-control-secret".into(), - lifecycle_log: fixture.join("lifecycle.jsonl"), - admission: AdmissionPolicy::default(), - }, - ) - .unwrap(); + let backend = Arc::new(BrokerAuthorizedBackend { + real: DockerEngineBackend::new(fixture.join("broker.sock")), + control: control.clone(), + handoff: fixture.join("handoff"), + certificate_sentinel: b"synthetic-amplifier-onecli-value".to_vec(), + }); + let supervisor_config = SupervisorConfig { + runtime_root: runtime, + control_socket_path: socket.clone(), + host_binary_path: PathBuf::from( + std::env::var_os("FRESHELL_PROVIDER_LAUNCH_HOST_BINARY").unwrap(), + ), + image_ref: std::env::var("FRESHELL_PROVIDER_LAUNCH_IMAGE").unwrap(), + test_run_id: std::env::var("FRESHELL_PROVIDER_LAUNCH_RUN_ID").unwrap(), + control_secret: "closed-synthetic-control-secret".into(), + lifecycle_log: fixture.join("lifecycle.jsonl"), + admission: AdmissionPolicy::default(), + }; + let supervisor = + Supervisor::new(registry.clone(), backend.clone(), supervisor_config.clone()).unwrap(); let control_task = tokio::spawn({ let socket = socket.clone(); async move { @@ -978,7 +1026,7 @@ async def on_session_ready(coordinator): json!({"provider":"amplifier","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes,"classifiedErrors":classified_errors}) ); } - let expected = json!({"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency")}); + let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency")}); assert_eq!(observed.expect("real Amplifier must activate selected local module and executable skill before a model turn"),expected); assert!( !output.exited, @@ -986,7 +1034,7 @@ async def on_session_ready(coordinator): ); println!( "provider_launch_consumer {}", - json!({"provider":"amplifier","route":"terminal","digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) + json!({"provider":"amplifier","route":"terminal","finalExecProfile":native_exec_profile(),"digest":hash(serde_json::to_vec(&expected).unwrap().as_slice())}) ); assert_eq!( std::fs::read_to_string(home.join(".amplifier/settings.yaml")).unwrap(), @@ -996,12 +1044,461 @@ async def on_session_ready(coordinator): std::fs::read_to_string(bundle.join("bundle.yaml")).unwrap(), bundle_document ); + let soul = SoulId::parse(terminal.soul_id.clone()).unwrap(); + let prior = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + let prior_generation = prior + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id + .clone(); + let hidden_home = fixture.join("source-home-unavailable"); + std::fs::rename(&home, &hidden_home).unwrap(); + let healthy = client + .recover(soul.clone(), RecoveryTrigger::HostUnreachable) + .await + .unwrap(); + assert_eq!(healthy.outcome, RecoveryOutcome::Reattached); + let attached = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + assert_eq!(attached.incarnation_id(), prior.incarnation_id()); + assert_eq!(attached.container_id(), prior.container_id()); + assert_eq!( + attached.provider_volume_name(), + prior.provider_volume_name() + ); + // Reopen real production registry/supervisor/controller instances, keeping + // the same provider-owned volume and frozen preparation authority. Joining + // the old control listener cannot stop the independently owned native CLI. + drop(controller); + control_task.abort(); + let _ = control_task.await; + drop(registry); + let registry = Registry::open( + fixture.join("registry"), + Some( + InstallationId::parse(std::env::var("FRESHELL_PROVIDER_LAUNCH_INSTALLATION").unwrap()) + .unwrap(), + ), + ) + .unwrap(); + let supervisor = Supervisor::new(registry.clone(), backend, supervisor_config).unwrap(); + std::fs::remove_file(&socket).unwrap(); + let control_task = tokio::spawn({ + let socket = socket.clone(); + async move { + serve_control(supervisor, &socket).await.unwrap(); + } + }); + tokio::time::timeout(Duration::from_secs(5), async { + while !socket.exists() { + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .unwrap(); + let _cold_controller = crate::managed_runtime::ServerManagedRuntimeController::from_env() + .await + .unwrap() + .unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + socket.clone(), + "closed-synthetic-control-secret", + ); + let cold = client + .recover(soul.clone(), RecoveryTrigger::HostUnreachable) + .await + .unwrap(); + assert_eq!(cold.outcome, RecoveryOutcome::Reattached); + let cold_handle = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + assert_eq!(cold_handle.incarnation_id(), prior.incarnation_id()); + assert_eq!(cold_handle.container_id(), prior.container_id()); + assert_eq!( + cold_handle.provider_volume_name(), + prior.provider_volume_name() + ); + assert_eq!( + cold_handle + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id, + prior_generation + ); + let real = DockerEngineBackend::new(fixture.join("broker.sock")); + real.force_stop(&prior) + .await + .expect("stop only exact registry-owned Amplifier fixture runtime"); + assert!(real.verify_empty(&prior).await.unwrap()); + std::fs::remove_file(&receipt).unwrap(); + let recovered = tokio::time::timeout( + Duration::from_secs(120), + client.recover(soul.clone(), RecoveryTrigger::HostUnreachable), + ) + .await + .expect("finite actual Amplifier native recovery") + .unwrap(); + println!( + "provider_recovery_outcome {}", + json!({"provider":"amplifier","outcome":recovered.outcome,"probe":recovered.probe.as_ref().map(|probe|match probe {RecoveryProbe::ResumeReady {..}=>"native_resume",RecoveryProbe::PristineSeedReady {..}=>"pristine_seed",_=>"other"})}) + ); + if let Some(RecoveryProbe::Blocked { + reason, retry_hint, .. + }) = &recovered.probe + { + let repair = retry_hint.repair.as_deref().unwrap_or(""); + let labels = [ + "manifest", + "copy", + "symlink", + "changed", + "ordinary", + "projection", + "bundle", + "module", + "mount", + "native", + "session", + "store", + "helper", + "metadata", + "cwd", + "credential", + "bootstrap", + "runtime", + "image", + "missing", + "unavailable", + "frozen", + "already exists", + "conflict", + "private", + "reference", + "failed", + "read", + "write", + "input", + "selection", + "permission", + "host", + "grant", + "socket", + "closed", + "auth", + "secret", + "environment", + "capability", + "lease", + "volume", + "scope", + "ownership", + ]; + println!( + "provider_recovery_diagnostic {}", + json!({"provider":"amplifier","reason":reason,"repairBytes":repair.len(),"repairClasses":labels.into_iter().filter(|label|repair.to_ascii_lowercase().contains(label)).collect::>()}) + ); + for line in std::fs::read_to_string(fixture.join("lifecycle.jsonl")) + .unwrap() + .lines() + { + if let Ok(event) = serde_json::from_str::(line) { + if event["event"] == "supervisor.activation_failed" + || event["event"] == "supervisor.recovery.activation_failed" + { + println!( + "provider_activation_diagnostic {}", + json!({"event":event["event"],"stage":event["data"]["stage"],"errorCode":event["data"]["errorCode"],"errorClass":event["data"]["errorClass"]}) + ); + } + } + } + } + assert_eq!(recovered.outcome, RecoveryOutcome::Replaced); + assert!( + matches!(recovered.probe, Some(RecoveryProbe::ResumeReady { .. })), + "existing completed conversation must use genuine native resume" + ); + let current = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + assert_ne!(current.incarnation_id(), prior.incarnation_id()); + assert_eq!(current.provider_volume_name(), prior.provider_volume_name()); + let generation = ¤t + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id; + assert_ne!(generation, &prior_generation); + let expected_conversation: Value = serde_json::from_slice( + &std::fs::read(receipt.with_extension("seed.json")) + .expect("native writer's completed pair receipt"), + ) + .unwrap(); + assert_eq!(expected_conversation["sessionId"], native_id); + assert_eq!(expected_conversation["roles"], json!(["user", "assistant"])); + assert_eq!(expected_conversation["messageCount"], 2); + assert_eq!(expected_conversation["digest"].as_str().unwrap().len(), 64); + let restored = tokio::time::timeout(Duration::from_secs(45), async { + loop { + if let Ok(bytes) = std::fs::read(&conversation_receipt) { + if let Ok(value) = serde_json::from_slice::(&bytes) { + break value; + } + } + tokio::time::sleep(Duration::from_millis(30)).await; + } + }) + .await + .ok(); + if restored.is_none() { + let output = client + .read_output(soul.clone(), 0, 64 * 1024) + .await + .unwrap(); + let text = crate::ai_title::strip_ansi( + &output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::(), + ); + let labels = [ + "Error", + "failed", + "Traceback", + "No such file", + "Permission denied", + "not found", + "bundle", + "Bundle", + "source", + "Source", + "context", + "Context", + "session", + "Session", + "ModuleNotFoundError", + "ModuleLoadError", + "ModuleValidationError", + "FileNotFoundError", + "RuntimeError", + "TypeError", + "AttributeError", + "not installed", + "not mounted", + "missing", + "resolve", + "activate", + "initialize", + "load", + "transcript", + "history", + "No module named", + "Failed to activate", + "Could not find", + "does not exist", + "Invalid", + "uv", + "read-only", + "reference", + "settings", + ]; + println!( + "provider_conversation_diagnostic {}", + json!({"provider":"amplifier","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"nativeStartupConsumerExists":receipt.exists(),"originalBundleMentioned":text.contains(bundle.to_str().unwrap()),"projectedBundleMentioned":text.contains("/home/freshell/provider/.freshell/resources/amplifier-local-0"),"fileSourceMissing":text.contains("File source not found"),"classes":labels.into_iter().filter(|label|text.contains(label)).collect::>() }) + ); + } + assert_eq!( + restored.expect( + "genuine native resume must restore the completed pair into its actual context" + ), + expected_conversation + ); + assert!(!home.exists()); + println!( + "provider_recovery_consumer {}", + json!({"provider":"amplifier","route":"terminal","outcome":recovered.outcome,"sameOwnedHome":true,"originalOrdinaryUnavailable":true,"priorGeneration":prior_generation,"generationId":generation,"nativeSessionId":native_id,"messageCount":2,"conversationDigest":expected_conversation["digest"],"healthyReattachSameRuntime":true,"coldProductionConstructorsReopened":true}) + ); + let accepted_inputs = current + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap(); + let accepted_manifest = control + .join("provider-roots") + .join(generation) + .join(&accepted_inputs.manifest.relative_path); + assert!( + !client + .read_output(soul.clone(), 0, 64 * 1024) + .await + .unwrap() + .exited, + "replacement native process must remain usable before authority loss" + ); + std::fs::rename( + &accepted_manifest, + accepted_manifest.with_extension("unavailable"), + ) + .unwrap(); + let still_live = client + .recover(soul.clone(), RecoveryTrigger::HostUnreachable) + .await + .unwrap(); + if still_live.outcome != RecoveryOutcome::Reattached { + let output = client.read_output(soul.clone(), 0, 64 * 1024).await.ok(); + let text = crate::ai_title::strip_ansi( + &output + .as_ref() + .map(|output| { + output + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::() + }) + .unwrap_or_default(), + ); + let labels = [ + "Error", + "failed", + "Traceback", + "AttributeError", + "TypeError", + "KeyError", + "get_messages", + "get_context_info", + "set_messages", + "session", + "context", + "Context", + "bundle", + "source", + "manifest", + "token", + "message", + "dict", + "list", + "Permission denied", + "not found", + ]; + let probe = match &still_live.probe { + Some(RecoveryProbe::Blocked { path, reason, .. }) => { + json!({"path":path,"reason":reason}) + } + _ => json!(null), + }; + let inspection = match real.inspect(¤t).await { + Ok(value) => json!({"state":format!("{:?}",value.state)}), + Err(error) => { + let text = error.to_string(); + let labels = [ + "manifest", + "metadata", + "prepared", + "provider", + "source", + "unavailable", + "changed", + "missing", + "ownership", + "mount", + "file", + ]; + json!({"errorClasses":labels.into_iter().filter(|label|text.to_ascii_lowercase().contains(label)).collect::>()}) + } + }; + println!( + "provider_environment_authority_diagnostic {}", + json!({"provider":"amplifier","outcome":still_live.outcome,"probe":probe,"inspection":inspection,"outputAvailable":output.is_some(),"exited":output.as_ref().map(|value|value.exited),"exitCode":output.as_ref().and_then(|value|value.exit_code),"outputBytes":text.len(),"classes":labels.into_iter().filter(|label|text.contains(label)).collect::>() }) + ); + } + assert_eq!(still_live.outcome, RecoveryOutcome::Reattached); + assert_eq!( + registry + .active_handle_for_soul(soul.clone()) + .await + .unwrap() + .container_id(), + current.container_id() + ); + real.force_stop(¤t) + .await + .expect("stop only the exact replacement fixture runtime"); + assert!(real.verify_empty(¤t).await.unwrap()); + let blocked = tokio::time::timeout( + Duration::from_secs(120), + client.recover(soul.clone(), RecoveryTrigger::HostUnreachable), + ) + .await + .unwrap() + .unwrap(); + assert_eq!(blocked.outcome, RecoveryOutcome::Blocked); + assert!( + matches!(blocked.probe, Some(RecoveryProbe::Blocked { reason, .. }) if reason != freshell_runtime_protocol::RecoveryBlockReason::StoreMissing) + ); + let retained = registry + .owned_handle(current.incarnation_id().clone()) + .await + .unwrap(); + assert_eq!(retained.incarnation_id(), current.incarnation_id()); + assert_eq!( + retained.provider_volume_name(), + current.provider_volume_name() + ); + assert_eq!( + retained + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id, + *generation + ); + assert_eq!( + std::fs::read_dir(fixture.join("handoff")) + .unwrap() + .filter_map(Result::ok) + .filter(|entry| entry + .path() + .extension() + .is_some_and(|extension| extension == "json")) + .count(), + 2 + ); + println!( + "provider_environment_authority_consumer {}", + json!({"provider":"amplifier","outcome":blocked.outcome,"healthyReattachSameRuntime":true,"acceptedGenerationRetained":true,"nativeNotRedispatched":true,"nativeStoreAbsenceNotInferred":true}) + ); + std::fs::rename(&hidden_home, &home).unwrap(); client .stop(SoulId::parse(terminal.soul_id).unwrap()) .await .unwrap(); web_task.abort(); control_task.abort(); + let _ = web_task.await; + let _ = control_task.await; } #[tokio::test(flavor = "multi_thread", worker_threads = 4)] @@ -1126,13 +1623,17 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ const input=JSON.parse(raw),root=path.resolve(__dirname,'..'); const endpoint=process.env.FRESHELL_URL,token=process.env.FRESHELL_TOKEN; const authenticated=await new Promise(resolve=>{if(!endpoint||!token)return resolve(false);const req=http.request(new URL('/api/mcp',endpoint),{method:'POST',headers:{'x-auth-token':token}},res=>{res.resume();resolve(res.statusCode===200)});req.on('error',()=>resolve(false));req.end('{}')}); - const record={hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,nativeIdentity:{uid:process.getuid(),euid:process.geteuid(),gid:process.getgid(),egid:process.getegid(),groups:[...new Set([...process.getgroups(),process.getegid()])].sort((a,b)=>a-b)},settings:['closed-selected-settings','closed-explicit-settings'].includes(process.env.CLAUDE_SELECTED_NONCE),authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], + const record={finalExecProfile:nativeExecProfile('claude'),hook:input.hook_event_name==='SessionStart',physicalHome:process.env.HOME,nativeIdentity:{uid:process.getuid(),euid:process.geteuid(),gid:process.getgid(),egid:process.getegid(),groups:[...new Set([...process.getgroups(),process.getegid()])].sort((a,b)=>a-b)},settings:['closed-selected-settings','closed-explicit-settings'].includes(process.env.CLAUDE_SELECTED_NONCE),authenticated,scopedAlias:process.env.FRESHELL==='1',onecli:hash(process.env.ANTHROPIC_API_KEY||'')===process.argv[3], refreshStageAbsent:fs.readdirSync(process.env.HOME).every(name=>!name.startsWith('.freshell-provider-refresh-')), memory:hash(fs.readFileSync(path.join(process.env.HOME,'.claude/CLAUDE.md'))),skill:hash(fs.readFileSync(path.join(root,'skills/probe/SKILL.md'))),resource:hash(require('node:child_process').execFileSync(path.join(__dirname,'run'))),diskResource:hash(fs.readFileSync(path.join(__dirname,'disk-resource'))),dependency:hash(require('probe-dependency'))}; fs.writeFileSync(process.argv[2],JSON.stringify(record)); }); "#; - std::fs::write(plugin.join("scripts/consume.cjs"), consumer).unwrap(); + std::fs::write( + plugin.join("scripts/consume.cjs"), + format!("{NATIVE_EXEC_PROFILE}{consumer}"), + ) + .unwrap(); let flag_receipt = workspace.join("claude-flag-receipt.json"); std::fs::write(plugin.join("scripts/flag.cjs"), "const fs=require('node:fs'),crypto=require('node:crypto');const hash=v=>crypto.createHash('sha256').update(v).digest('hex');let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{const input=JSON.parse(raw);fs.writeFileSync(process.argv[2],JSON.stringify({flag:input.hook_event_name==='SessionStart'&&process.env.CLAUDE_SELECTED_NONCE==='closed-explicit-settings',resource:hash(require('node:child_process').execFileSync(__dirname+'/run')),dependency:hash(require('probe-dependency'))}))})").unwrap(); let flag_commands = selected_config.join("commands"); @@ -1460,7 +1961,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ .ok() } let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":source_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"refreshStageAbsent":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"diskResource":hash(&disk_resource),"dependency":hash(b"closed-claude-dependency")}); + let expected = json!({"finalExecProfile":native_exec_profile(),"hook":true,"physicalHome":"/home/freshell/provider","nativeIdentity":source_identity,"settings":true,"authenticated":true,"scopedAlias":true,"onecli":true,"refreshStageAbsent":true,"memory":hash(&memory),"skill":hash(std::fs::read(plugin.join("skills/probe/SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-claude-resource"),"diskResource":hash(&disk_resource),"dependency":hash(b"closed-claude-dependency")}); let imported_bytes = std::fs::read(home.join("imported-memory.md")).unwrap(); let mut expected_imports = serde_json::Map::new(); expected_imports.insert("imported-memory.md".into(), json!(hash(&imported_bytes))); @@ -1750,7 +2251,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ imported_graph(&instructions_events, &expected_imports).await; println!( "provider_launch_consumer {}", - json!({"provider":"claude","route":"terminal","digest":qualified_digest}) + json!({"provider":"claude","route":"terminal","finalExecProfile":native_exec_profile(),"digest":qualified_digest}) ); if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD").as_deref() == Ok("retained") @@ -2018,7 +2519,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ ); println!( "provider_launch_consumer {}", - json!({"provider":"claude","route":"fresh","digest":qualified_digest}) + json!({"provider":"claude","route":"fresh","finalExecProfile":native_exec_profile(),"digest":qualified_digest}) ); assert_eq!( std::fs::read_to_string(selected_config.join("settings.json")).unwrap(), @@ -2344,7 +2845,7 @@ let result={};if(message.method==='initialize'){ const nativeArgv=fs.readFileSync('/proc/'+process.ppid+'/cmdline','utf8').split('\0'); const retainedMcpRecipe=nativeArgv.includes('mcp_servers.freshell.command="node"')&&nativeArgv.includes('mcp_servers.freshell.args=["/opt/freshell-mcp/server.js"]')&&nativeArgv.some(arg=>arg.startsWith('mcp_servers.freshell.env_vars=')); const policy=JSON.parse(require('node:child_process').execFileSync('codex',['execpolicy','check','--rules',path.join(process.env.CODEX_HOME,'rules/probe.rules'),'closed-fixture-never-run'],{encoding:'utf8'})); - const record={globalInstructions:digest(fs.readFileSync(path.join(process.env.CODEX_HOME,'AGENTS.md'))),nativeRuleForbidden:policy.decision==='forbidden',physicalHome:process.env.HOME,initialized:true,authenticated,retainedMcpRecipe,scopedAlias:process.env.FRESHELL==='1',endpointPresent:!!endpoint,tokenPresent:!!token, + const record={finalExecProfile:nativeExecProfile('parent'),globalInstructions:digest(fs.readFileSync(path.join(process.env.CODEX_HOME,'AGENTS.md'))),nativeRuleForbidden:policy.decision==='forbidden',physicalHome:process.env.HOME,initialized:true,authenticated,retainedMcpRecipe,scopedAlias:process.env.FRESHELL==='1',endpointPresent:!!endpoint,tokenPresent:!!token, user:digest(fs.readFileSync(path.join(process.env.CODEX_HOME,'user.md'))),project:digest(fs.readFileSync(path.join(process.cwd(),'.codex/project.md'))), role:digest(fs.readFileSync(path.join(process.cwd(),'.codex/agents/role.md'))),skill:digest(fs.readFileSync(path.join(__dirname,'../SKILL.md'))), resource:digest(require('node:child_process').execFileSync(path.join(__dirname,'run'))),dependency:digest(require('probe-dependency'))}; @@ -2352,7 +2853,11 @@ let result={};if(message.method==='initialize'){ }else if(message.method==='tools/list')result={tools:[]};else if(message.method==='resources/list')result={resources:[]};else if(message.method==='prompts/list')result={prompts:[]}; process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:message.id,result})+'\n');}); "#; - std::fs::write(skill.join("scripts/server.cjs"), server).unwrap(); + std::fs::write( + skill.join("scripts/server.cjs"), + format!("{NATIVE_EXEC_PROFILE}{server}"), + ) + .unwrap(); let plugin = home.join(".codex/plugins/cache/fixture/probe/local"); let plugin_data_name = format!("{:x}", Sha256::digest(b"fixture\0probe")); let plugin_data = home @@ -2537,7 +3042,7 @@ require('node:readline').createInterface({input:process.stdin}).on('line',line=> .map_err(|_| "native Codex must initialize its configured MCP consumer") } let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); - let expected = json!({"globalInstructions":hash(b"closed-global-instructions"),"nativeRuleForbidden":true,"physicalHome":"/home/freshell/provider","initialized":true,"authenticated":true,"retainedMcpRecipe":true,"scopedAlias":true,"endpointPresent":true,"tokenPresent":true,"user":hash(b"closed-user-input"),"project":hash(b"closed-project-input"),"role":hash(b"closed-role-input"),"skill":hash(std::fs::read(skill.join("SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-codex-resource"),"dependency":hash(b"closed-codex-dependency")}); + let expected = json!({"finalExecProfile":native_exec_profile(),"globalInstructions":hash(b"closed-global-instructions"),"nativeRuleForbidden":true,"physicalHome":"/home/freshell/provider","initialized":true,"authenticated":true,"retainedMcpRecipe":true,"scopedAlias":true,"endpointPresent":true,"tokenPresent":true,"user":hash(b"closed-user-input"),"project":hash(b"closed-project-input"),"role":hash(b"closed-role-input"),"skill":hash(std::fs::read(skill.join("SKILL.md")).unwrap().as_slice()),"resource":hash(b"closed-codex-resource"),"dependency":hash(b"closed-codex-dependency")}); let consumed_terminal = consumed(&receipt).await; if consumed_terminal.is_err() { let output = client @@ -2593,7 +3098,7 @@ require('node:readline').createInterface({input:process.stdin}).on('line',line=> ); println!( "provider_launch_consumer {}", - json!({"provider":"codex","route":"terminal","digest":hash(serde_json::to_vec(&json!({"ordinary":expected,"installed":expected_plugin})).unwrap().as_slice())}) + json!({"provider":"codex","route":"terminal","finalExecProfile":native_exec_profile(),"digest":hash(serde_json::to_vec(&json!({"ordinary":expected,"installed":expected_plugin})).unwrap().as_slice())}) ); client .stop(SoulId::parse(terminal.soul_id).unwrap()) @@ -2632,7 +3137,7 @@ require('node:readline').createInterface({input:process.stdin}).on('line',line=> ); println!( "provider_launch_consumer {}", - json!({"provider":"codex","route":"fresh","digest":hash(serde_json::to_vec(&json!({"ordinary":expected,"installed":expected_plugin})).unwrap().as_slice())}) + json!({"provider":"codex","route":"fresh","finalExecProfile":native_exec_profile(),"digest":hash(serde_json::to_vec(&json!({"ordinary":expected,"installed":expected_plugin})).unwrap().as_slice())}) ); assert_eq!( std::fs::read_to_string(home.join(".codex/config.toml")).unwrap(), @@ -2896,14 +3401,19 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ "selected-root-resource", ) .unwrap(); - std::fs::write(resource_root.join("plugins/selected/index.mjs"), r#"import fs from 'node:fs';import crypto from 'node:crypto';import dependency from 'fixture-dependency'; + let selected_root_plugin = r#"import fs from 'node:fs';import crypto from 'node:crypto';import dependency from 'fixture-dependency'; const home=import.meta.url.startsWith('file:///home/freshell/provider/.opencode/'); export default {id:'closed-writable-selected-root-'+(home?'home':'workspace'),async server(input,options){return {async config(config){ const root=new URL('../../',import.meta.url); const path=new URL('provider-owned-write',root); fs.writeFileSync(path,'selected-root-write'); - fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({physicalHome:process.env.HOME,privateRule:crypto.createHash('sha256').update(config.agent.rootScope.prompt).digest('hex'),dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); + fs.writeFileSync(home?options.homeReceipt:options.receipt,JSON.stringify({finalExecProfile:nativeExecProfile('self'),physicalHome:process.env.HOME,privateRule:crypto.createHash('sha256').update(config.agent.rootScope.prompt).digest('hex'),dependency:crypto.createHash('sha256').update(dependency).digest('hex'),resource:crypto.createHash('sha256').update(fs.readFileSync(new URL('./resource',import.meta.url))).digest('hex'),write:crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex')})); }}}}; -"#).unwrap(); +"#; + std::fs::write( + resource_root.join("plugins/selected/index.mjs"), + format!("{NATIVE_EXEC_PROFILE}{selected_root_plugin}"), + ) + .unwrap(); let root_config = serde_json::to_vec( &json!({"agent":{"rootScope":{"prompt":"{file:./private-root-rule}"}}, "plugin":[["./plugins/selected/index.mjs", {"receipt":root_receipt, "homeReceipt":home_root_receipt}]]}), ) @@ -3101,7 +3611,7 @@ export default {{id:'closed-directory-discovery',async server(){{ } }; assert_eq!(terminal_receipt, expected); - let expected_root = json!({"physicalHome":"/home/freshell/provider","privateRule":format!("{:x}",Sha256::digest(b"shared-home-private-rule")),"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); + let expected_root = json!({"finalExecProfile":native_exec_profile(),"physicalHome":"/home/freshell/provider","privateRule":format!("{:x}",Sha256::digest(b"shared-home-private-rule")),"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); assert_eq!( read_receipt(&home_root_receipt).await.unwrap(), @@ -3217,7 +3727,7 @@ export default {{id:'closed-directory-discovery',async server(){{ assert_eq!(std::fs::read(&explicit_config).unwrap(), explicit_bytes); println!( "provider_launch_consumer {}", - json!({"provider":"opencode","route":"terminal","workspaceScope":"default-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) + json!({"provider":"opencode","route":"terminal","finalExecProfile":native_exec_profile(),"workspaceScope":"default-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&expected).unwrap()))}) ); client .stop(SoulId::parse(terminal.soul_id).unwrap()) @@ -3342,7 +3852,7 @@ export default {{id:'closed-directory-discovery',async server(){{ .expect("native plugin must stop the fresh request before the model loop"); println!( "provider_launch_consumer {}", - json!({"provider":"opencode","route":"fresh","workspaceScope":"workspace-under-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&fresh_expected).unwrap()))}) + json!({"provider":"opencode","route":"fresh","finalExecProfile":native_exec_profile(),"workspaceScope":"workspace-under-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&fresh_expected).unwrap()))}) ); // Scan only run-owned durable/staged surfaces; referenced payloads belong // in private indexed files, not ordinary resource/configuration trees. diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 5e525c0c2..af18de2a2 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -50,7 +50,9 @@ const timer=setInterval(()=>{ } const directory=path.join(root,'handoff') const entries=fs.readdirSync(directory).filter((name)=>name.endsWith('.json')) - if (entries.length>(configuration.provider==='amplifier'?1:2)) throw new Error('closed provider fixture exceeds its finite two launch attempts') + // Amplifier uses one exact soul for initial seed establishment and its + // genuine existing-conversation replacement; other fixtures have two routes. + if (entries.length>2) throw new Error('closed provider fixture exceeds its finite two launch attempts') for (const name of entries) { if (accepted.has(name)) continue if (!/^[a-zA-Z0-9_-]{1,128}\.json$/.test(name)) throw new Error('closed provider fixture handoff name invalid') diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index a4ce8202f..d0b69a631 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -95,6 +95,26 @@ def remove(name): }) if labels.get(LABEL)!=run and labels.get("com.freshell.runtime-test-run-id")!=run and not fixed_helper: raise RuntimeError("closed startup container cleanup ownership changed") + # Inspect only this failed fixture's exact broker-recorded runtime. + # Provider/host log bytes remain transient; evidence is finite labels. + runtime_ids={item["containerId"] for item in receipt.get("broker",{}).get("receipts",[])} + if receipt.get("exitCode",0)!=0 and value["Id"] in runtime_ids: + output=docker("logs","--tail","64",value["Id"],check=False) + text=output.stdout+output.stderr + classes=[label for label in ["Operation not permitted","Permission denied","Read-only file system","No such file or directory","Directory not empty","symlink","source","destination","chown","ownership","group","mount","file","existing","metadata","manifest","unsafe","selected","stage","provider.environment.copy.failed"] if label.lower() in text.lower()] + copy_failures=[] + for line in text.splitlines(): + try:event=json.loads(line) + except (ValueError,TypeError):continue + if event.get("event")!="provider.environment.copy.failed":continue + # Only closed operation and error-kind enums leave transient logs. + # Neither paths nor arbitrary corrective text enter receipts. + operations={"plan","select","resolve","inspect","read_directory","set_private_mode","create_stage","resolve_selection","inspect_destination","create_stage_parent","create_directory","open","create_file","read","write","create_link","publish","publish_directory","publish_leaf","inspect_mounted_file","open_mounted_file","backup_mounted_file","restore_mounted_file","publish_mounted_file","select_resource_scope","resolve_lookup_context","resolve_package_scope","resolve_selected_alias","inspect_link_target"} + operation=event.get("operation") + cause=re.search(r"\((\w+), OS (?:Some\((\d+)\)|None)\)",event.get("corrective_message", "")) + kinds={"NotFound","PermissionDenied","AlreadyExists","NotADirectory","IsADirectory","DirectoryNotEmpty","ReadOnlyFilesystem","InvalidInput","InvalidData","Unsupported","Other","StorageFull","TooManyLinks","FilesystemLoop"} + copy_failures.append({"operation":operation if operation in operations else "unclassified","cause":cause.group(1) if cause and cause.group(1) in kinds else "unclassified","osCode":int(cause.group(2)) if cause and cause.group(2) else None,"limit":event.get("limit_kind") if event.get("limit_kind") in {"entries","file_bytes","total_bytes","depth"} else None}) + receipt.setdefault("failedRuntimeDiagnostics",[]).append({"id":value["Id"],"bytes":len(text.encode()),"classes":classes,"copyFailures":copy_failures,"privateInputLeak":any(marker in text for marker in ["synthetic-amplifier-onecli-value","synthetic-onecli-shadowed","AUTH_PREFIX"] )}) docker("rm","--force",value["Id"]) def create(name,*arguments): owned.append(name) @@ -212,16 +232,21 @@ def create(name,*arguments): receipt["claudeSourceGuard"]=args.claude_source_guard receipt["consumedRoutes"]=[json.loads(line.split("provider_launch_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_launch_consumer " in line] receipt["candidateObservers"]=[json.loads(line.split("provider_candidate_observer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_candidate_observer " in line] - if args.claude_source_guard in {"retained","retained-fresh"}: + if args.provider=="amplifier" or args.claude_source_guard in {"retained","retained-fresh"}: receipt["recoveryConsumers"]=[json.loads(line.split("provider_recovery_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_recovery_consumer " in line] recovered=receipt["recoveryConsumers"] route_mismatch_recovery=len(recovered)!=1 or any(value.get("outcome")!="replaced" or value.get("sameOwnedHome") is not True or value.get("originalOrdinaryUnavailable") is not True or value.get("generationId")==value.get("priorGeneration") for value in recovered) + if args.provider=="amplifier": + receipt["authorityConsumers"]=[json.loads(line.split("provider_environment_authority_consumer ",1)[1]) for line in outcome.stdout.splitlines() if "provider_environment_authority_consumer " in line] + authority=receipt["authorityConsumers"] + route_mismatch_recovery=route_mismatch_recovery or any(value.get("messageCount")!=2 or value.get("nativeSessionId")!="41a04bb3-86f0-463d-97c5-f085e8a9ea4b" or not re.fullmatch(r"[a-f0-9]{64}",value.get("conversationDigest","")) or value.get("coldProductionConstructorsReopened") is not True or value.get("healthyReattachSameRuntime") is not True for value in recovered) + route_mismatch_recovery=route_mismatch_recovery or len(authority)!=1 or any(value.get("outcome")!="blocked" or any(value.get(field) is not True for field in ["healthyReattachSameRuntime","acceptedGenerationRetained","nativeNotRedispatched","nativeStoreAbsenceNotInferred"]) for value in authority) print(outcome.stdout,end="",flush=True);print(outcome.stderr,end="",flush=True) expected_routes={("opencode","terminal"),("opencode","fresh"),("opencode","terminal-tui")} if args.provider=="opencode" else ({(args.provider,"terminal")} if args.provider=="amplifier" else {(args.provider,"terminal"),(args.provider,"fresh")}) if args.claude_source_guard in {"retained","retained-fresh"}:expected_routes={("claude","fresh" if args.claude_source_guard=="retained-fresh" else "terminal")} observed_routes={(item.get("provider"),item.get("route")) for item in receipt["consumedRoutes"]} route_mismatch=len(receipt["consumedRoutes"])!=len(expected_routes) or observed_routes!=expected_routes - if args.claude_source_guard in {"retained","retained-fresh"}:route_mismatch=route_mismatch or route_mismatch_recovery + if args.provider=="amplifier" or args.claude_source_guard in {"retained","retained-fresh"}:route_mismatch=route_mismatch or route_mismatch_recovery if args.claude_source_guard=="changed-git": receipt["blockedRoutes"]=[json.loads(line.split("provider_source_guard ",1)[1]) for line in outcome.stdout.splitlines() if "provider_source_guard " in line] blocked={(item.get("provider"),item.get("route")) for item in receipt["blockedRoutes"]} From 7da87d13042d12b9e99975144cef1a462375b816 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:54:44 -0700 Subject: [PATCH 174/493] test: observe pinned Claude native process variants and preserve fixture bytes --- .../src/managed_provider_launch_tests.rs | 35 ++++++++++++++----- 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 3f5b71ddc..cb12dffd1 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -20,22 +20,34 @@ use std::{ // argv/environment bytes stay transient; only profile compliance is reported. const NATIVE_EXEC_PROFILE: &str = r#" function nativeExecProfile(target) { + let operation='select_native_process'; + const ancestry=[]; + try { let pid=target==='self'?process.pid:process.ppid; if(target==='claude') { let found=false; for(let depth=0;depth<16&&pid>1;depth++) { - if(fs.readlinkSync('/proc/'+pid+'/exe').split('/').pop()==='claude') {found=true;break;} + const executable=fs.readlinkSync('/proc/'+pid+'/exe').split('/').pop(); + const comm=fs.readFileSync('/proc/'+pid+'/comm','utf8').trim(); + ancestry.push({executable:executable==='claude'?'claude':executable.includes('claude')?'claude_variant':['node','bash','sh','timeout','freshell-session-host'].includes(executable)?executable:'other',comm:comm==='claude'?'claude':comm.includes('claude')?'claude_variant':['node','bash','sh','timeout','freshell-sessio'].includes(comm)?comm:'other'}); + if(comm==='claude'&&executable.includes('claude')) {found=true;break;} const status=fs.readFileSync('/proc/'+pid+'/status','utf8'); pid=Number(status.match(/^PPid:\s+(\d+)/m)[1]); } if(!found)throw Error('closed native hook ancestry unavailable'); } - const prefix='/proc/'+pid+'/',argv=fs.readFileSync(prefix+'cmdline'),env=fs.readFileSync(prefix+'environ'); + const prefix='/proc/'+pid+'/'; + operation='read_native_argv'; const argv=fs.readFileSync(prefix+'cmdline'); + operation='read_native_environment'; const env=fs.readFileSync(prefix+'environ'); const strings=[...argv.toString().split('\0').filter(Boolean),...env.toString().split('\0').filter(Boolean)]; - const stack=fs.readFileSync(prefix+'limits','utf8').match(/^Max stack size\s+(\S+)/m)[1]; + operation='read_native_stack'; const stack=fs.readFileSync(prefix+'limits','utf8').match(/^Max stack size\s+(\S+)/m)[1]; const result={stackAtLeast8MiB:stack==='unlimited'||Number(stack)>=8388608,aggregateWithin1MiB:argv.length+env.length+8*(strings.length+2)+Buffer.byteLength(fs.readlinkSync(prefix+'exe'))+1<=1048576,completeStringsWithin32Pages:strings.every(value=>Buffer.byteLength(value)+1<=131072)}; if(Object.values(result).some(value=>!value))throw Error('closed final native execution profile exceeded'); return result; + } catch(error) { + if(target==='claude')fs.writeFileSync(process.argv[2]+'.profile-failure.json',JSON.stringify({operation,code:['ENOENT','EACCES','EPERM'].includes(error.code)?error.code:error.message==='closed native hook ancestry unavailable'?'NATIVE_NOT_FOUND':error instanceof TypeError?'TYPE_ERROR':'unclassified',ancestry})); + throw Error('closed native profile observation failed'); + } } "#; @@ -1629,11 +1641,8 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',async()=>{ fs.writeFileSync(process.argv[2],JSON.stringify(record)); }); "#; - std::fs::write( - plugin.join("scripts/consume.cjs"), - format!("{NATIVE_EXEC_PROFILE}{consumer}"), - ) - .unwrap(); + let consumer = format!("{NATIVE_EXEC_PROFILE}{consumer}"); + std::fs::write(plugin.join("scripts/consume.cjs"), &consumer).unwrap(); let flag_receipt = workspace.join("claude-flag-receipt.json"); std::fs::write(plugin.join("scripts/flag.cjs"), "const fs=require('node:fs'),crypto=require('node:crypto');const hash=v=>crypto.createHash('sha256').update(v).digest('hex');let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{const input=JSON.parse(raw);fs.writeFileSync(process.argv[2],JSON.stringify({flag:input.hook_event_name==='SessionStart'&&process.env.CLAUDE_SELECTED_NONCE==='closed-explicit-settings',resource:hash(require('node:child_process').execFileSync(__dirname+'/run')),dependency:hash(require('probe-dependency'))}))})").unwrap(); let flag_commands = selected_config.join("commands"); @@ -2213,6 +2222,16 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ "provider_launch_diagnostic {}", json!({"provider":"claude","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes}) ); + let failure = std::fs::read(format!( + "{}.profile-failure.json", + receipt.to_str().unwrap() + )) + .ok() + .and_then(|bytes| serde_json::from_slice::(&bytes).ok()); + println!( + "provider_native_profile_diagnostic {}", + json!({"observed":failure.is_some(),"failure":failure}) + ); } assert_eq!( observed.expect("native Claude selected plugin must execute its startup hook"), From 8a8ebde75b3fb21157f1b10e8b01c9b7e86b9540 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 02:57:28 -0700 Subject: [PATCH 175/493] fix: join Claude source readers through the provider identity --- .../src/claude_source_observation.rs | 79 +++++++++++++++++-- .../managed-provider-source-sandbox.py | 16 ++-- 2 files changed, 83 insertions(+), 12 deletions(-) diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index 8d0ccf797..84822fb82 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -78,6 +78,7 @@ pub(crate) async fn verify_final( &request, &request_sha256, &expected, + (uid, gid), std::time::Duration::from_secs(20), std::time::Duration::from_secs(5), ) @@ -115,6 +116,7 @@ async fn read_final_child( child: &mut tokio::process::Child, request: &ObservationRequest, request_sha256: &str, + identity: (u32, u32), exchange_timeout: std::time::Duration, cleanup_timeout: std::time::Duration, ) -> Result { @@ -165,9 +167,39 @@ async fn read_final_child( Ok(completed) => completed, Err(reason) => { let cleanup = async { - child - .start_kill() - .map_err(|_| "Claude final source reader termination failed")?; + // Restricted root cannot signal a provider-UID child without + // CAP_KILL. The unreaped child keeps this exact PID owned; + // use the same identity as ordinary refresh cleanup. + if child + .try_wait() + .map_err(|_| "Claude final source reader join failed")? + .is_none() + { + let pid = child + .id() + .ok_or("Claude final source reader identity unavailable")? + .to_string(); + let status = tokio::process::Command::new("/usr/bin/setpriv") + .args(crate::provider_identity_args(identity.0, identity.1)) + .args(["/usr/bin/kill", "-KILL", "--", &pid]) + .env_clear() + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .status() + .await + .map_err(|_| "Claude final source reader termination failed")?; + // A concurrent exit can make kill report failure. Join + // first: complete ownership settlement is authoritative. + if !status.success() + && child + .try_wait() + .map_err(|_| "Claude final source reader join failed")? + .is_none() + { + return Err("Claude final source reader termination failed"); + } + } child .wait() .await @@ -191,6 +223,7 @@ async fn verify_final_child( request: &ObservationRequest, request_sha256: &str, expected: &freshell_platform::claude_inputs::observation::NativeObservation, + identity: (u32, u32), exchange_timeout: std::time::Duration, cleanup_timeout: std::time::Duration, ) -> Result<(), RuntimeError> { @@ -200,6 +233,7 @@ async fn verify_final_child( child, request, request_sha256, + identity, exchange_timeout, cleanup_timeout, ) @@ -1361,6 +1395,23 @@ mod tests { let home = root.path().join("home"); std::fs::create_dir(&cwd).unwrap(); std::fs::create_dir(&home).unwrap(); + let deployed_identity = std::env::var_os("FRESHELL_TEST_RESTRICTED_READER").is_some(); + let (uid, gid) = if deployed_identity { + assert_eq!(unsafe { libc::geteuid() }, 0); + let status = std::fs::read_to_string("/proc/self/status").unwrap(); + let capabilities = status + .lines() + .find_map(|line| line.strip_prefix("CapEff:\t")) + .unwrap(); + let capabilities = u64::from_str_radix(capabilities, 16).unwrap(); + assert_eq!(capabilities, (1 << 0) | (1 << 6) | (1 << 7)); + for path in [root.path(), &cwd, &home] { + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755)).unwrap(); + } + (65534, 0) + } else { + (unsafe { libc::geteuid() }, unsafe { libc::getegid() }) + }; let request = ObservationRequest { version: 1, request_id: uuid::Uuid::new_v4().to_string(), @@ -1401,10 +1452,15 @@ mod tests { } let body = root.path().join("reply.bin"); std::fs::write(&body, bytes).unwrap(); - std::fs::set_permissions(&body, std::fs::Permissions::from_mode(0o600)).unwrap(); + std::fs::set_permissions( + &body, + std::fs::Permissions::from_mode(if deployed_identity { 0o644 } else { 0o600 }), + ) + .unwrap(); let script = root.path().join("reader.cjs"); std::fs::write(&script, r#"const fs=require('node:fs'); const mode=process.argv[2],body=fs.readFileSync(process.argv[3]); +if(process.getuid()!==Number(process.argv[4]))process.exit(7); process.stdin.on('data',()=>{}).on('end',()=>{ const header=Buffer.alloc(4);header.writeUInt32BE(body.length+(mode==='oversized'||mode==='short-body'?1:0)); if(mode==='stall'){setInterval(()=>{},10000);return;} @@ -1416,10 +1472,13 @@ process.stdin.on('data',()=>{}).on('end',()=>{ if(mode==='nonzero')process.exitCode=6; }); "#).unwrap(); - let mut child = tokio::process::Command::new("node") + let mut child = tokio::process::Command::new("/usr/bin/setpriv") + .args(crate::provider_identity_args(uid, gid)) + .arg("node") .arg(&script) .arg(mode) .arg(&body) + .arg(uid.to_string()) .current_dir(&cwd) .env_clear() .env("HOME", &home) @@ -1436,6 +1495,7 @@ process.stdin.on('data',()=>{}).on('end',()=>{ &request, &request_sha256, &expected, + (uid, gid), std::time::Duration::from_secs(2), std::time::Duration::from_millis(200), ) @@ -1446,7 +1506,13 @@ process.stdin.on('data',()=>{}).on('end',()=>{ // fallback records a failed production join; it cannot make it pass. if !joined_before_test_cleanup { tokio::time::timeout(std::time::Duration::from_secs(2), async { - child.kill().await.unwrap(); + crate::run_provider_refresh_cleanup( + uid, + gid, + "/usr/bin/kill", + &["-KILL", "--", &child.id().unwrap().to_string()], + ) + .unwrap(); child.wait().await.unwrap(); }) .await @@ -1473,6 +1539,7 @@ process.stdin.on('data',()=>{}).on('end',()=>{ "wrong-binding", "wrong-runtime", "stall", + "eof-stall", ] { let case = reader_case(mode).await; assert!( diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 8a63e6592..080a76d65 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -20,7 +20,7 @@ def docker(*args, check=True): def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--provider",choices=["amplifier","codex","claude"],default="amplifier") + parser.add_argument("--provider",choices=["amplifier","codex","claude","claude-reader"],default="amplifier") parser.add_argument("--test-binary", type=Path, required=True) parser.add_argument("--receipt-dir", type=Path, required=True) args = parser.parse_args() @@ -28,9 +28,11 @@ def main(): "amplifier": TEST, "codex": "managed_mcp_capability::tests::managed_provider_codex_native_typed_inputs_survive_transport", "claude": "managed_provider_sdk_tests::managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt", + "claude-reader": "claude_source_observation::tests::claude_final_reader_real_pipe_bounds_binding_failures_and_joins", }[args.provider] binary = args.test_binary.resolve(strict=True) - if not binary.is_file() or not re.fullmatch(r"freshell_server-[0-9a-f]{16}", binary.name): + artifact_prefix = "freshell_session_host" if args.provider == "claude-reader" else "freshell_server" + if not binary.is_file() or not re.fullmatch(artifact_prefix + r"-[0-9a-f]{16}", binary.name): raise RuntimeError("source fixture requires its compiled server test artifact") expected_directory = Path(__file__).resolve().parents[2] / "target/debug/deps" if binary.parent != expected_directory.resolve(): @@ -52,14 +54,16 @@ def main(): created = False receipt = {"image": IMAGE, "testArtifactSha256": digest, "test": selected_test, "runId": name, "rootless": True} + reader = args.provider == "claude-reader" + identity_args = ["--cap-add", "CHOWN", "--cap-add", "SETGID", "--cap-add", "SETUID", "--env", "FRESHELL_TEST_RESTRICTED_READER=1"] if reader else [] try: docker("create", "--name", name, "--label", f"{LABEL}={name}", - "--network", "none", "--read-only", "--user", "65534:0", "--cap-drop", "ALL", + "--network", "none", "--read-only", "--user", "0:0" if reader else "65534:0", "--cap-drop", "ALL", *identity_args, "--security-opt", "no-new-privileges:true", "--pids-limit", "128", "--memory", "2g", "--mount", f"type=bind,src={binary},dst=/runtime/source-consumer-tests,readonly", "--tmpfs", "/fixture:rw,exec,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", "--tmpfs", f"/home/freshell/provider:rw,{'exec' if args.provider == 'codex' else 'noexec'},nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", - "--tmpfs", "/tmp:rw,nosuid,nodev,noexec,size=256m,uid=65534,gid=0,mode=0700", + "--tmpfs", f"/tmp:rw,nosuid,nodev,noexec,size=256m,uid={0 if reader else 65534},gid=0,mode={'1777' if reader else '0700'}", "--env", "HOME=/home/freshell/provider", "--env", "AMPLIFIER_HOME=/home/freshell/provider/.amplifier", "--env", "FRESHELL_TEST_EXECUTABLE_TMP_DIR=/fixture", @@ -75,7 +79,7 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":5,"claude":1}[args.provider] or "test result: ok. 1 passed;" not in result.stdout): + if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":5,"claude":1,"claude-reader":0}[args.provider] or "test result: ok. 1 passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. @@ -90,7 +94,7 @@ def main(): raise RuntimeError("source fixture cleanup ownership changed") docker("rm", "--force", value["Id"]) receipt["ownedCleanup"] = docker("container", "inspect", name, check=False).returncode != 0 - receipt_name = {"amplifier":"task002-native-source-consumers.json", "codex":"task002-native-codex-source-consumers.json", "claude":"task002-native-claude-sdk-consumers.json"}[args.provider] + receipt_name = {"amplifier":"task002-native-source-consumers.json", "codex":"task002-native-codex-source-consumers.json", "claude":"task002-native-claude-sdk-consumers.json", "claude-reader":"task002-repair13-native-claude-reader.json"}[args.provider] (args.receipt_dir / receipt_name).write_text(json.dumps(receipt, sort_keys=True) + "\n") if not created or not receipt["ownedCleanup"]: raise RuntimeError("source fixture failed its owned cleanup proof") From 1735d36c8b6f977f568900b2feb3f7e50651565d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 03:06:06 -0700 Subject: [PATCH 176/493] fix: bound simultaneous OpenCode source transformations before allocation --- .../freshell-platform/src/opencode_config.rs | 91 +++++-- .../src/managed_provider_references.rs | 239 +++++++++++------- .../managed_provider_references/sources.rs | 78 ++++++ 3 files changed, 291 insertions(+), 117 deletions(-) diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index d848f88fa..d1c033402 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -563,11 +563,11 @@ struct SubstitutionSegment<'a> { const MAX_CONFIG_VIEW_BYTES: usize = 16 * 1024 * 1024; const MAX_CONFIG_EVENTS: usize = 8192; -fn bounded_append(out: &mut String, piece: &str) -> Result<(), TransformError> { +fn bounded_append(out: &mut String, piece: &str, available: usize) -> Result<(), TransformError> { if out .len() .checked_add(piece.len()) - .is_none_or(|n| n > MAX_CONFIG_VIEW_BYTES) + .is_none_or(|n| n > MAX_CONFIG_VIEW_BYTES || n > available) { return Err(TransformError::ExpansionLimit); } @@ -600,8 +600,31 @@ fn outer_file_end(raw: &str, start: usize) -> Option { pub fn transform_file_tokens( raw: &str, env: &dyn Fn(&str) -> Option, - mut file: impl FnMut(FileToken<'_>) -> Result, + file: impl FnMut(FileToken<'_>) -> Result, ) -> Result> { + transform_file_tokens_inner(raw, env, file, 2 * MAX_CONFIG_VIEW_BYTES, true) + .map(|(text, _)| text.expect("rendered transform owns output")) +} + +/// Inspect the same pinned substitution pass without retaining its private +/// expanded output. `available` accounts for every source view the caller +/// already holds; check each in-progress expansion before appending bytes. +pub fn inspect_file_tokens( + raw: &str, + env: &dyn Fn(&str) -> Option, + available: usize, + file: impl FnMut(FileToken<'_>) -> Result, +) -> Result> { + transform_file_tokens_inner(raw, env, file, available, false).map(|(_, bytes)| bytes) +} + +fn transform_file_tokens_inner( + raw: &str, + env: &dyn Fn(&str) -> Option, + mut file: impl FnMut(FileToken<'_>) -> Result, + available: usize, + render: bool, +) -> Result<(Option, usize), TokenTransformError> { use TokenTransformError::{Bound, Selection}; if raw.len() > MAX_CONFIG_VIEW_BYTES { return Err(Bound(TransformError::SourceLimit)); @@ -630,7 +653,7 @@ pub fn transform_file_tokens( .last() .map_or(0, |segment: &SubstitutionSegment<'_>| segment.original.end); let begin = expanded.len(); - bounded_append(&mut expanded, &raw[literal_start..start]).map_err(Bound)?; + bounded_append(&mut expanded, &raw[literal_start..start], available).map_err(Bound)?; segments.push(SubstitutionSegment { expanded: begin..expanded.len(), original: literal_start..start, @@ -638,7 +661,7 @@ pub fn transform_file_tokens( }); let begin = expanded.len(); if let Some(value) = env(name) { - bounded_append(&mut expanded, &value).map_err(Bound)?; + bounded_append(&mut expanded, &value, available).map_err(Bound)?; } segments.push(SubstitutionSegment { expanded: begin..expanded.len(), @@ -647,17 +670,45 @@ pub fn transform_file_tokens( }); cursor = close + 1; } - let literal_start = segments - .last() - .map_or(0, |segment: &SubstitutionSegment<'_>| segment.original.end); - let begin = expanded.len(); - bounded_append(&mut expanded, &raw[literal_start..]).map_err(Bound)?; - segments.push(SubstitutionSegment { - expanded: begin..expanded.len(), - original: literal_start..raw.len(), - variable: None, - }); - let mut out = String::new(); + // A source without substitutions is already a complete accepted view. + // Borrow it instead of allocating an identical expansion at capacity. + let expanded = if segments.is_empty() { + segments.push(SubstitutionSegment { + expanded: 0..raw.len(), + original: 0..raw.len(), + variable: None, + }); + std::borrow::Cow::Borrowed(raw) + } else { + let literal_start = segments.last().unwrap().original.end; + let begin = expanded.len(); + bounded_append(&mut expanded, &raw[literal_start..], available).map_err(Bound)?; + segments.push(SubstitutionSegment { + expanded: begin..expanded.len(), + original: literal_start..raw.len(), + variable: None, + }); + std::borrow::Cow::Owned(expanded) + }; + let expanded_bytes = if matches!(expanded, std::borrow::Cow::Owned(_)) { + expanded.len() + } else { + 0 + }; + let mut out = render.then(String::new); + let mut out_bytes = 0usize; + let mut append = |piece: &str| { + out_bytes = out_bytes + .checked_add(piece.len()) + .ok_or(TransformError::ExpansionLimit)?; + if out_bytes > MAX_CONFIG_VIEW_BYTES { + return Err(TransformError::ExpansionLimit); + } + if let Some(out) = &mut out { + bounded_append(out, piece, available.saturating_sub(expanded_bytes))?; + } + Ok(()) + }; let mut cursor = 0; let mut scan = 0; let mut ordinal = 0; @@ -709,13 +760,13 @@ pub fn transform_file_tokens( ordinal, }) .map_err(Selection)?; - bounded_append(&mut out, &expanded[cursor..start]).map_err(Bound)?; - bounded_append(&mut out, &replacement).map_err(Bound)?; + append(&expanded[cursor..start]).map_err(Bound)?; + append(&replacement).map_err(Bound)?; cursor = end; ordinal += 1; } - bounded_append(&mut out, &expanded[cursor..]).map_err(Bound)?; - Ok(out) + append(&expanded[cursor..]).map_err(Bound)?; + Ok((out, out_bytes)) } #[derive(Debug)] diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 0df817d14..7d5e70f0b 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -119,12 +119,14 @@ pub(super) fn resolve_file_references_from_view( ReferenceProjection { relocated_files: &std::collections::BTreeMap::new(), original_form: false, + available_view_bytes: (64 * 1024 * 1024usize).saturating_sub(raw.len()), }, ) } pub(super) struct ReferenceProjection<'a> { pub relocated_files: &'a std::collections::BTreeMap, pub original_form: bool, + pub available_view_bytes: usize, } pub(super) fn resolve_file_references_from_view_relocated( raw: &str, @@ -136,7 +138,7 @@ pub(super) fn resolve_file_references_from_view_relocated( projection: ReferenceProjection<'_>, ) -> Result { use freshell_platform::opencode_config::{ - file_reference_path, transform_file_tokens, TokenTransformError, + file_reference_path, inspect_file_tokens, TokenTransformError, }; let mut selections: Vec = Vec::new(); let mut rewritten = Vec::new(); @@ -167,59 +169,61 @@ pub(super) fn resolve_file_references_from_view_relocated( substitution_events.set(substitution_events.get() + 1); env(name) }; - let transformed = transform_file_tokens(raw, &lookup, |token| { - let names: Vec = token - .variables - .iter() - .map(|name| name.to_string()) - .collect(); - path_variables.extend(names.iter().cloned()); - substitution_events.set(substitution_events.get() + 1); - if substitution_events.get() > 8192 { - return Err(fail( - token.ordinal, - names, - "source substitution event count exceeds 8192", - )); - } - generated |= !token.direct_expression; - let path = file_reference_path( - token.path, - origin.declaring_config, - origin.cwd, - origin.original_home, - ); - let physical_source = projection.relocated_files.get(&path).unwrap_or(&path); - let canonical = std::fs::canonicalize(physical_source).map_err(|_| { - fail( - token.ordinal, - names.clone(), - "referenced file is missing, inaccessible or cyclic", - ) - })?; - let metadata = std::fs::metadata(&canonical).map_err(|_| { - fail( - token.ordinal, - names.clone(), - "referenced file metadata unavailable", - ) - })?; - if !metadata.is_file() || metadata.len() > 16 * 1024 * 1024 { - return Err(fail( - token.ordinal, - names, - "referenced target must be a regular file at most 16 MiB", - )); - } - std::fs::File::open(&canonical).map_err(|_| { - fail( - token.ordinal, - names.clone(), - "referenced file is unreadable", - ) - })?; - let destination = - if let Some(selection) = selections.iter().find(|s| s.canonical_source == canonical) { + let transformed_length = + inspect_file_tokens(raw, &lookup, projection.available_view_bytes, |token| { + let names: Vec = token + .variables + .iter() + .map(|name| name.to_string()) + .collect(); + path_variables.extend(names.iter().cloned()); + substitution_events.set(substitution_events.get() + 1); + if substitution_events.get() > 8192 { + return Err(fail( + token.ordinal, + names, + "source substitution event count exceeds 8192", + )); + } + generated |= !token.direct_expression; + let path = file_reference_path( + token.path, + origin.declaring_config, + origin.cwd, + origin.original_home, + ); + let physical_source = projection.relocated_files.get(&path).unwrap_or(&path); + let canonical = std::fs::canonicalize(physical_source).map_err(|_| { + fail( + token.ordinal, + names.clone(), + "referenced file is missing, inaccessible or cyclic", + ) + })?; + let metadata = std::fs::metadata(&canonical).map_err(|_| { + fail( + token.ordinal, + names.clone(), + "referenced file metadata unavailable", + ) + })?; + if !metadata.is_file() || metadata.len() > 16 * 1024 * 1024 { + return Err(fail( + token.ordinal, + names, + "referenced target must be a regular file at most 16 MiB", + )); + } + std::fs::File::open(&canonical).map_err(|_| { + fail( + token.ordinal, + names.clone(), + "referenced file is unreadable", + ) + })?; + let destination = if let Some(selection) = + selections.iter().find(|s| s.canonical_source == canonical) + { selection.runtime_destination.clone() } else { if selections.len() >= 128 { @@ -252,52 +256,63 @@ pub(super) fn resolve_file_references_from_view_relocated( }); destination }; - if aliases.insert(path.clone()) && aliases.len() > 256 { - return Err(fail( - token.ordinal, - names, - "configuration alias count exceeds 256", - )); - } - occurrences.push(ApprovedFileOccurrence { - ordinal: token.ordinal, - approved_alias: path, - runtime_destination: destination.clone(), - }); - let replacement = format!("{{file:{destination}}}"); - let kind = if token.path.starts_with("~/") { - NativeAliasKind::Home - } else if Path::new(token.path).is_absolute() { - NativeAliasKind::Absolute - } else { - NativeAliasKind::Relative - }; - // Only literal source-independent paths can receive an identical edit - // at a shared inode. Relative paths and loader-specific interpolation - // retain native evaluation and their approved occurrence aliases. - let selective = token.direct_expression - && token.variables.is_empty() - && !matches!(kind, NativeAliasKind::Relative); - if !projection.original_form || selective { - rewritten.push((token.original_span, replacement.clone())); - } else { - native_alias_kinds.insert(token.ordinal, kind); - } - Ok(replacement) - }) - .map_err(|error| match error { - TokenTransformError::Selection(error) => error, - TokenTransformError::Bound(_) => fail( - 0, - Vec::new(), - "configuration source, expansion or event profile limit exceeded", - ), - })?; + if aliases.insert(path.clone()) && aliases.len() > 256 { + return Err(fail( + token.ordinal, + names, + "configuration alias count exceeds 256", + )); + } + occurrences.push(ApprovedFileOccurrence { + ordinal: token.ordinal, + approved_alias: path, + runtime_destination: destination.clone(), + }); + let replacement = format!("{{file:{destination}}}"); + let kind = if token.path.starts_with("~/") { + NativeAliasKind::Home + } else if Path::new(token.path).is_absolute() { + NativeAliasKind::Absolute + } else { + NativeAliasKind::Relative + }; + // Only literal source-independent paths can receive an identical edit + // at a shared inode. Relative paths and loader-specific interpolation + // retain native evaluation and their approved occurrence aliases. + let selective = token.direct_expression + && token.variables.is_empty() + && !matches!(kind, NativeAliasKind::Relative); + if !projection.original_form || selective { + rewritten.push((token.original_span, replacement.clone())); + } else { + native_alias_kinds.insert(token.ordinal, kind); + } + Ok(replacement) + }) + .map_err(|error| match error { + TokenTransformError::Selection(error) => error, + TokenTransformError::Bound(_) => fail( + 0, + Vec::new(), + "configuration source, expansion or event profile limit exceeded", + ), + })?; if let Some(cause) = input_error.get() { return Err(fail(0, Vec::new(), cause)); } - let transformed_length = transformed.len(); - drop(transformed); + // The transient expanded inspection view is gone before retaining any + // template/recipe. Charge complete output views before allocating them. + let check_new_views = |bytes: usize| { + if bytes > projection.available_view_bytes { + Err(fail( + 0, + Vec::new(), + "simultaneous retained source views exceed 64 MiB", + )) + } else { + Ok(()) + } + }; let (template, recipes) = if generated && !projection.original_form { let generated_name = freshell_platform::opencode_config::generated_config_reference_name(recipe_ordinal); @@ -315,6 +330,11 @@ pub(super) fn resolve_file_references_from_view_relocated( "generated configuration name collides with an original child input", )); } + check_new_views( + raw.len() + .checked_add(generated_name.len() + 6) + .ok_or_else(|| fail(0, Vec::new(), "source-view size overflow"))?, + )?; ( format!("{{env:{generated_name}}}"), vec![GeneratedFileRecipe { @@ -327,6 +347,31 @@ pub(super) fn resolve_file_references_from_view_relocated( }], ) } else { + if rewritten.is_empty() { + let source = source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned())); + return Ok(ResolvedFileReferencePlan { + template: source.clone(), + unexpanded_direct_source: Some(source), + selections, + referenced_variables: variables.into_inner().into_iter().collect(), + path_variables: path_variables.into_iter().collect(), + recipes: Vec::new(), + occurrences, + substitution_events: substitution_events.get(), + native_alias_kinds, + }); + } + let mut template_bytes = raw.len(); + for (span, replacement) in &rewritten { + template_bytes = template_bytes + .checked_sub(span.len()) + .and_then(|bytes| bytes.checked_add(replacement.len())) + .ok_or_else(|| fail(0, Vec::new(), "source-view size overflow"))?; + } + if template_bytes > 16 * 1024 * 1024 { + return Err(fail(0, Vec::new(), "source view exceeds 16 MiB")); + } + check_new_views(template_bytes)?; let mut template = String::new(); let mut cursor = 0; for (span, replacement) in rewritten { diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 302786b3b..acd3058f5 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -500,6 +500,20 @@ pub fn plan_loaded_opencode_sources_relocated( let mut variables = std::collections::BTreeSet::new(); let mut event_count = 0usize; let mut view_budget = SourceViewBudget::default(); + // All unprocessed originals are still live while the first source is + // transformed. Include them, plus retained outputs from earlier sources. + for loaded in &sources { + view_budget + .charge(&loaded.view) + .map_err(|cause| super::ReferenceError { + operation: "read", + field: "OpenCode source", + source_ordinal: 0, + declaring_source: loaded.source.logical_path.clone(), + variables: vec![], + cause, + })?; + } for (ordinal, loaded) in sources.into_iter().enumerate() { let source = loaded.source; let original_form = source.lane != SourceLane::Inline @@ -563,6 +577,8 @@ pub fn plan_loaded_opencode_sources_relocated( super::ReferenceProjection { relocated_files, original_form, + available_view_bytes: MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES as usize + - view_budget.bytes, }, )?; if original_form { @@ -588,6 +604,14 @@ pub fn plan_loaded_opencode_sources_relocated( .checked_add(references.recipes.len()) .ok_or_else(|| fail("source recipe count overflow"))?; view_budget.charge(&references.template).map_err(fail)?; + if references.unexpanded_direct_source.is_none() { + // Generated recipes own a replacement raw view. Its original Arc + // cannot be shared by another occurrence (those use original form). + let identity = (loaded.view.as_ptr() as usize, loaded.view.len()); + if view_budget.views.remove(&identity) { + view_budget.bytes -= loaded.view.len(); + } + } view_budget .charge(references.unexpanded_source()) .map_err(fail)?; @@ -709,6 +733,60 @@ mod tests { format!("{:x}", Sha256::digest(std::fs::read(path).unwrap())) } #[test] + fn managed_provider_references_simultaneous_transform_views_preserve_inclusive_capacity() { + let owned = tempfile::tempdir().unwrap(); + let mut paths = Vec::new(); + let limit = MAX_SOURCE_VIEW_BYTES as usize; + let suffix = "{env:VALUE}"; + let document = |tail: &str| { + format!( + "{{\"description\":\"{}{tail}\"}}", + "x".repeat(limit - tail.len() - 18) + ) + }; + for ordinal in 0..4 { + let path = owned.path().join(format!("source-{ordinal}.json")); + let raw = document(if ordinal == 0 { suffix } else { "" }); + assert_eq!(raw.len(), limit); + assert!(serde_json::from_str::(&raw).is_ok()); + std::fs::write(&path, raw).unwrap(); + paths.push(path); + } + let sources = |count: usize| { + paths[..count] + .iter() + .map(|path| SourceOccurrence { + logical_path: path.clone(), + canonical_source: path.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }) + .collect() + }; + let lookup = |_: SourceLane, _: &Path, name: &str| { + (name == "VALUE").then(|| "x".repeat(suffix.len())) + }; + let at_capacity = plan_opencode_sources(sources(3), owned.path(), owned.path(), &lookup) + .expect( + "48 MiB retained originals plus one complete 16 MiB expansion fits inclusively", + ); + assert_eq!(at_capacity.len(), 3); + drop(at_capacity); + let exhausted = plan_opencode_sources(sources(4), owned.path(), owned.path(), &lookup); + assert!( + exhausted.is_err(), + "64 MiB retained originals cannot allocate another complete expanded source view" + ); + std::fs::write(&paths[0], document("")).unwrap(); + let no_op = plan_opencode_sources(sources(4), owned.path(), owned.path(), &|_, _, _| None) + .expect("four distinct no-op 16 MiB sources keep the full inclusive 64 MiB capacity"); + assert_eq!(no_op.len(), 4); + assert!(no_op.iter().all(|source| std::ptr::eq( + source.references.template.as_str(), + source.references.unexpanded_source() + ))); + } + #[test] fn managed_provider_references_shared_large_source_uses_unique_view_budget() { let owned = tempfile::tempdir().unwrap(); let shared = owned.path().join("shared.json"); From b02bd2215b326321fafdbeeb722748a96f12c69f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 03:51:16 -0700 Subject: [PATCH 177/493] fix: transport explicitly configured OpenCode external resources --- Cargo.lock | 1 + crates/freshell-server/Cargo.toml | 2 + .../src/managed_mcp_capability.rs | 9 + .../managed_mcp_capability/source_stage.rs | 258 +++++++++++- .../src/managed_provider_launch_tests.rs | 114 +++++ .../src/managed_provider_references.rs | 1 + .../opencode_resources.rs | 389 ++++++++++++++++++ .../managed_provider_references/sources.rs | 30 +- .../testing/managed-provider-launch-broker.ts | 2 +- 9 files changed, 797 insertions(+), 9 deletions(-) create mode 100644 crates/freshell-server/src/managed_provider_references/opencode_resources.rs diff --git a/Cargo.lock b/Cargo.lock index 0e3abc67e..05138c927 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1462,6 +1462,7 @@ dependencies = [ "freshell-terminal", "freshell-ws", "futures-util", + "glob", "hyper", "hyper-util", "libc", diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index 8a50070f8..3fd72c348 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -27,6 +27,7 @@ managed-runtime-v1 = [ "dep:saphyr-parser", "dep:semver", "dep:tempfile", + "dep:glob", ] managed-provider-qualification = [ "managed-runtime-v1", @@ -84,6 +85,7 @@ saphyr-parser = { version = "=0.0.11", optional = true } semver = { version = "=1.0.28", optional = true } # Managed preparation journals use private temporary files before atomic publication. tempfile = { version = "3", optional = true } +glob = { version = "0.3.3", optional = true } # Boot-scoped identifiers: `srv-` / `boot-`. uuid = { version = "1", features = ["v4"] } # The browser-pane HTTP reverse proxy (`/api/proxy/http/{port}/*`) forwards to a diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 26b3b2f69..afd050a92 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1510,6 +1510,15 @@ fn stage_provider_root( .map(|reference| PathBuf::from(&reference.source_path)), ); if let Some(plan) = opencode_input.source_plan { + // Indexed payloads remain exclusively in the private input volume, + // even when an ordinary package scope also contains their files. + excluded.extend(plan.iter().flat_map(|source| { + source + .references + .selections + .iter() + .map(|selection| selection.canonical_source.clone()) + })); let units = source_stage::resource_units( home, &source, diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 9b5bb02f9..6c02ac627 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -7,6 +7,214 @@ use freshell_platform::{ }; use std::{collections::BTreeMap, path::Path}; +#[cfg(test)] +mod configured_resource_tests { + use super::*; + use crate::managed_provider_references::sources::*; + use freshell_platform::provider_environment::{copy_environment, CopySelection}; + #[test] + fn managed_provider_configured_external_resources_fail_before_publication_when_unavailable() { + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let config = owned.path().join("opencode.json"); + let resource = owned.path().join("unavailable-resource.mjs"); + for field in ["plugin", "instructions"] { + for unreadable in [false, true] { + if unreadable { + std::fs::write(&resource, "resource").unwrap(); + std::fs::set_permissions(&resource, std::fs::Permissions::from_mode(0)) + .unwrap(); + } + let raw = serde_json::to_string(&serde_json::json!({field:[resource]})).unwrap(); + std::fs::write(&config, &raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + owned.path(), + owned.path(), + &|_, _, _| None, + ); + if unreadable { + std::fs::set_permissions(&resource, std::fs::Permissions::from_mode(0o600)) + .unwrap(); + std::fs::remove_file(&resource).unwrap(); + } + let error = plan + .err() + .expect("selected unavailable ordinary resource must prevent preparation"); + assert_eq!( + error.field, + if field == "plugin" { + "OpenCode plugin" + } else { + "OpenCode instructions" + } + ); + assert_eq!(error.declaring_source, config); + assert!(error + .cause + .contains(if unreadable { "unreadable" } else { "missing" })); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + assert!(!owned.path().join("stage").exists()); + } + } + } + #[test] + fn managed_provider_configured_resources_keep_field_and_relative_instruction_origins() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace/nested"); + let global = owned.path().join("xdg/opencode"); + let plugin = global.join("plugins/selected"); + for directory in [ + &home, + &cwd, + &plugin, + &cwd.join("rules"), + &cwd.parent().unwrap().join("rules"), + ] { + std::fs::create_dir_all(directory).unwrap(); + } + std::fs::write(plugin.join("index.mjs"), "export default ()=>({});").unwrap(); + std::fs::write( + cwd.join("rules/selected.md"), + "nearest selected instruction", + ) + .unwrap(); + std::fs::write( + cwd.parent().unwrap().join("rules/unselected.md"), + "ancestor must not be selected", + ) + .unwrap(); + let config = global.join("opencode.jsonc"); + let raw = r#"{ // retain raw native spellings and unrelated options +"plugin":[["file:///home/freshell/provider/.config/opencode/plugins/selected/index.mjs",{"instructions":["/unselected/missing"],"path":"{env:UNRELATED}"}]], +"instructions":["{env:RULE_PATTERN}"],}"#; + std::fs::write(&config, raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Global, + }], + &cwd, + &home, + &|_, _, name| match name { + "RULE_PATTERN" => Some("rules/*.md".into()), + "UNRELATED" => Some("ordinary option value".into()), + _ => None, + }, + ) + .unwrap(); + assert_eq!( + plan[0].configured_resources, + vec![cwd.join("rules/selected.md")] + ); + assert_eq!(plan[0].references.path_variables, vec!["RULE_PATTERN"]); + assert_eq!(plan[0].references.unexpanded_source(), raw); + } + #[test] + fn managed_provider_configured_external_resources_execute_complete_transport() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let package = owned.path().join("external/package"); + for path in [&home, &workspace, &package.join("node_modules/dependency")] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write(package.join("package.json"), r#"{"type":"module"}"#).unwrap(); + std::fs::write(package.join("helper.mjs"), "export default 'adjacent';").unwrap(); + std::fs::write(package.join("resource"), "resource").unwrap(); + std::fs::write( + package.join("node_modules/dependency/package.json"), + r#"{"type":"module","exports":"./index.mjs"}"#, + ) + .unwrap(); + std::fs::write( + package.join("node_modules/dependency/index.mjs"), + "export default 'dependency';", + ) + .unwrap(); + std::fs::write(package.join("index.mjs"), "import fs from 'node:fs';import helper from './helper.mjs';import dependency from 'dependency';console.log(helper+':'+dependency+':'+fs.readFileSync(new URL('./resource',import.meta.url),'utf8'));").unwrap(); + let instructions = owned.path().join("external/instructions.md"); + std::fs::write(&instructions, "instruction-resource").unwrap(); + let config = home.join("opencode.jsonc"); + let raw = format!("{{ // preserve comments\n\"plugin\":[[{},{{\"unrelated\":\"/unselected/path\"}}]],\"instructions\":[{}],}}", serde_json::to_string(&format!("file://{}",package.join("index.mjs").display())).unwrap(), serde_json::to_string(&instructions).unwrap()); + std::fs::write(&config, &raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Home, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + assert!( + units.iter().any(|(source, _)| source == &package), + "ordinary file-URL plugin must select its complete package" + ); + assert!( + units.iter().any(|(source, _)| source == &instructions), + "ordinary instruction file must be transported" + ); + let stage = owned.path().join("stage"); + copy_environment( + &units + .iter() + .map(|(source, mount)| CopySelection { + source: source.clone(), + destination_relative: mount.stage_relative_path.clone().into(), + }) + .collect::>(), + &stage, + &[], + Default::default(), + ) + .unwrap(); + let plugin = units.iter().find(|(source, _)| source == &package).unwrap(); + let hidden = owned.path().join("hidden-original-resources"); + std::fs::rename(owned.path().join("external"), &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg(stage.join(&plugin.1.stage_relative_path).join("index.mjs")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, owned.path().join("external")).unwrap(); + assert!( + output.status.success(), + "transported dependency and adjacent imports must execute" + ); + assert_eq!(output.stdout, b"adjacent:dependency:resource\n"); + let instruction = units + .iter() + .find(|(source, _)| source == &instructions) + .unwrap(); + assert_eq!( + std::fs::read(stage.join(&instruction.1.stage_relative_path)).unwrap(), + b"instruction-resource" + ); + assert_eq!(std::fs::read_to_string(config).unwrap(), raw); + } +} + /// Selected discovery roots are complete writable units, independent of the /// private rewritten configuration leaves mounted inside them. pub(super) fn resource_units( @@ -52,15 +260,53 @@ pub(super) fn resource_units( } roots.insert((parent.to_path_buf(), source.source.role), canonical); } + for source in plan { + for logical in &source.configured_resources { + // The mounted workspace already supplies its native paths. External + // paths and relocated HOME inputs need complete owned projections. + if logical.starts_with(workspace) { + continue; + } + let mut roles = + vec![crate::managed_provider_references::sources::DiscoveryRole::Workspace]; + if logical.starts_with(global) { + roles.push(crate::managed_provider_references::sources::DiscoveryRole::Global); + } else if logical.starts_with(home) { + roles.push(crate::managed_provider_references::sources::DiscoveryRole::Home); + } + for role in roles { + if roots + .keys() + .any(|(root, root_role)| *root_role == role && logical.starts_with(root)) + { + continue; + } + let canonical = std::fs::canonicalize(&logical) + .map_err(|_| "selected OpenCode configured resource is unavailable")?; + roots.insert((logical.clone(), role), canonical); + } + } + } let mut units = Vec::new(); let mut identities = BTreeMap::new(); for ((logical, role), canonical) in roots { let next = identities.len(); let ordinal = *identities.entry(canonical.clone()).or_insert(next); - let destination = - crate::managed_provider_references::sources::runtime_directory_destination( + let kind = if canonical.is_file() { + ProviderResourceKind::File + } else { + ProviderResourceKind::Directory + }; + let name = if kind == ProviderResourceKind::File { + format!("root-{ordinal}/source") + } else { + format!("root-{ordinal}") + }; + let destination = freshell_platform::opencode_config::lexical_path( + &crate::managed_provider_references::sources::runtime_directory_destination( home, global, &logical, role, - ); + ), + ); let effective = if destination.starts_with("/home/freshell/provider") || !logical.starts_with(workspace) { @@ -76,8 +322,8 @@ pub(super) fn resource_units( units.push(( canonical, ProviderResourceMount { - stage_relative_path: format!(".freshell/resource-units/root-{ordinal}"), - provider_relative_path: format!(".freshell/provider-roots/root-{ordinal}"), + stage_relative_path: format!(".freshell/resource-units/{name}"), + provider_relative_path: format!(".freshell/provider-roots/{name}"), logical_destination: destination .to_str() .ok_or("resource destination is not UTF-8")? @@ -86,7 +332,7 @@ pub(super) fn resource_units( .to_str() .ok_or("resource destination is not UTF-8")? .into(), - kind: ProviderResourceKind::Directory, + kind, }, )); } diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index cb12dffd1..71d736a8c 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3375,7 +3375,58 @@ export default {{id:'closed-shared-{label}',async tui(api,options){{ } let cwd = nested.to_string_lossy().into_owned(); let original_config = fixture.join("source-home/.config/opencode/opencode.json"); + // These ordinary native fields select resources outside every implicit + // HOME, workspace and control root. The plugin needs its adjacent scope + // and installed parent lookup, and reads the selected instruction itself. + let external = fixture.join("scenarios/external-native-resources"); + let package = external.join("plugin-package"); + std::fs::create_dir_all(package.join("node_modules/closed-child")).unwrap(); + std::fs::create_dir_all(external.join("node_modules/closed-parent")).unwrap(); + for (base, name, value) in [ + (&package, "closed-child", "native-child-dependency"), + (&external, "closed-parent", "native-parent-dependency"), + ] { + let dependency = base.join("node_modules").join(name); + std::fs::write( + dependency.join("package.json"), + br#"{"type":"module","exports":"./index.mjs"}"#, + ) + .unwrap(); + std::fs::write( + dependency.join("index.mjs"), + format!("export default {};", serde_json::to_string(value).unwrap()), + ) + .unwrap(); + } + std::fs::write( + package.join("package.json"), + br#"{"type":"module","exports":"./index.mjs"}"#, + ) + .unwrap(); + std::fs::write( + package.join("adjacent.mjs"), + "export default 'native-adjacent-import';", + ) + .unwrap(); + std::fs::write(package.join("resource.txt"), "native-adjacent-resource").unwrap(); + let external_instruction = external.join("ordinary-instructions.md"); + std::fs::write(&external_instruction, "native-external-instruction").unwrap(); + let external_receipt = fixture.join("workspace/native-external-consumed.json"); + let external_plugin = r#"import fs from 'node:fs';import crypto from 'node:crypto'; +import adjacent from './adjacent.mjs';import child from 'closed-child';import parent from 'closed-parent'; +export default async (_input,options)=>({config:async(config)=>{ + const digest=value=>crypto.createHash('sha256').update(value).digest('hex'); + fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction)})); +}}); +"#; + std::fs::write(package.join("index.mjs"), external_plugin).unwrap(); + let mut original: Value = + serde_json::from_slice(&std::fs::read(&original_config).unwrap()).unwrap(); + original["plugin"].as_array_mut().unwrap().push(json!([reqwest::Url::from_file_path(package.join("index.mjs")).unwrap().to_string(), {"receipt":external_receipt,"instruction":external_instruction}])); + original["instructions"] = json!([external_instruction]); + std::fs::write(&original_config, serde_json::to_vec(&original).unwrap()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); + let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true}); let explicit_config = fixture.join("workspace/selected-config.json"); let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; std::fs::write(&explicit_config, explicit_bytes).unwrap(); @@ -3560,6 +3611,44 @@ export default {{id:'closed-directory-discovery',async server(){{ view_pane_id: None, create_request_id: Some("closed-native-terminal-create".into()), }; + // Fail through the actual terminal controller before any Docker handoff. + // Restore every owned fixture before checking the result, including RED. + for (field, selected) in [ + ("OpenCode plugin", package.join("index.mjs")), + ("OpenCode instructions", external_instruction.clone()), + ] { + for unreadable in [false, true] { + let hidden = selected.with_extension("closed-unavailable"); + if unreadable { + std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0)).unwrap(); + } else { + std::fs::rename(&selected, &hidden).unwrap(); + } + let failed = controller.launch(request.clone()).await; + if unreadable { + std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0o644)) + .unwrap(); + } else { + std::fs::rename(&hidden, &selected).unwrap(); + } + let error = failed + .err() + .expect("selected unavailable resource must block the actual launch"); + assert!( + error.contains(field) + && error.contains(if unreadable { "unreadable" } else { "missing" }) + ); + assert_eq!( + std::fs::read_dir(fixture.join("handoff")).unwrap().count(), + 0 + ); + assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); + println!( + "provider_resource_selection_failure {}", + json!({"field":field,"cause":if unreadable {"unreadable"} else {"missing"},"beforeDockerHandoff":true,"originalsRestored":true}) + ); + } + } let terminal = controller .launch(request) .await @@ -3630,6 +3719,18 @@ export default {{id:'closed-directory-discovery',async server(){{ } }; assert_eq!(terminal_receipt, expected); + assert_eq!( + read_receipt(&external_receipt).await.unwrap(), + expected_external + ); + assert_eq!( + std::fs::read_to_string(package.join("index.mjs")).unwrap(), + external_plugin + ); + assert_eq!( + std::fs::read(&external_instruction).unwrap(), + b"native-external-instruction" + ); let expected_root = json!({"finalExecProfile":native_exec_profile(),"physicalHome":"/home/freshell/provider","privateRule":format!("{:x}",Sha256::digest(b"shared-home-private-rule")),"dependency":format!("{:x}",Sha256::digest(b"selected-root-dependency")),"resource":format!("{:x}",Sha256::digest(b"selected-root-resource")),"write":format!("{:x}",Sha256::digest(b"selected-root-write"))}); assert_eq!(read_receipt(&root_receipt).await.unwrap(), expected_root); assert_eq!( @@ -3755,6 +3856,7 @@ export default {{id:'closed-directory-discovery',async server(){{ std::fs::remove_file(&receipt).unwrap(); std::fs::remove_file(&shared_receipt).unwrap(); std::fs::remove_file(&root_receipt).unwrap(); + std::fs::remove_file(&external_receipt).unwrap(); std::fs::remove_file(&home_root_receipt).unwrap(); std::fs::remove_file(&directory_receipt).unwrap(); std::fs::remove_file(shared_lane_receipt.with_extension("root.main")).unwrap(); @@ -3816,6 +3918,18 @@ export default {{id:'closed-directory-discovery',async server(){{ .dispatch(HostedFreshAgentCommand::Send(send)) .await; assert_eq!(read_receipt(&receipt).await.unwrap(), fresh_expected); + assert_eq!( + read_receipt(&external_receipt).await.unwrap(), + expected_external + ); + assert_eq!( + std::fs::read_to_string(package.join("index.mjs")).unwrap(), + external_plugin + ); + assert_eq!( + std::fs::read(&external_instruction).unwrap(), + b"native-external-instruction" + ); assert_eq!( read_receipt(&shared_lane_receipt.with_extension("root.main")) .await diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 7d5e70f0b..ba0ab8760 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -6,6 +6,7 @@ pub(crate) mod amplifier; pub(crate) mod claude; pub(crate) mod codex; pub(crate) mod environment; +pub(crate) mod opencode_resources; pub(crate) mod sources; pub struct FileSourceOrigin<'a> { diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs new file mode 100644 index 000000000..35d8d0d38 --- /dev/null +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -0,0 +1,389 @@ +//! The two native ordinary resource fields. Keep raw documents and every +//! unrelated string intact; these selections are paths, never private payloads. +use super::sources::PlannedSourceOccurrence; +use std::path::{Path, PathBuf}; + +#[derive(Clone, Copy)] +enum Token { + String(usize, usize), + Punctuation(u8), + Other, +} +struct Cursor<'a> { + raw: &'a str, + at: usize, +} +impl Cursor<'_> { + fn next(&mut self) -> Option { + let bytes = self.raw.as_bytes(); + loop { + while bytes.get(self.at).is_some_and(u8::is_ascii_whitespace) { + self.at += 1; + } + if bytes.get(self.at..self.at + 2) == Some(b"//") { + while bytes.get(self.at).is_some_and(|b| *b != b'\n') { + self.at += 1; + } + } else if bytes.get(self.at..self.at + 2) == Some(b"/*") { + self.at += 2; + while self.at < bytes.len() && bytes.get(self.at..self.at + 2) != Some(b"*/") { + self.at += 1; + } + self.at = (self.at + 2).min(bytes.len()); + } else { + break; + } + } + let byte = *bytes.get(self.at)?; + let start = self.at; + self.at += 1; + if byte == b'"' { + while let Some(byte) = bytes.get(self.at) { + self.at += 1; + if *byte == b'\\' { + self.at = (self.at + 1).min(bytes.len()); + } else if *byte == b'"' { + return Some(Token::String(start, self.at)); + } + } + None + } else if b"{}[]:,".contains(&byte) { + Some(Token::Punctuation(byte)) + } else { + while bytes + .get(self.at) + .is_some_and(|b| !b.is_ascii_whitespace() && !b"{}[]:,".contains(b)) + { + self.at += 1; + } + Some(Token::Other) + } + } +} + +// A finite field visitor rather than decoded/serialized document rewriting. +// Plugin tuple options (and nested unrelated fields) are never path schemas. +fn fields(raw: &str) -> Result, &'static str> { + let mut cursor = Cursor { raw, at: 0 }; + let mut depth = 0usize; + let mut result = Vec::new(); + while let Some(token) = cursor.next() { + match token { + Token::Punctuation(b'{' | b'[') => depth += 1, + Token::Punctuation(b'}' | b']') => depth = depth.saturating_sub(1), + Token::String(start, end) if depth == 1 && end - start <= 64 => { + let key: String = serde_json::from_str(&raw[start..end]) + .map_err(|_| "invalid selected resource field")?; + let field = match key.as_str() { + "plugin" => "OpenCode plugin", + "instructions" => "OpenCode instructions", + _ => continue, + }; + let mut array = Cursor { raw, at: cursor.at }; + if !matches!(array.next(), Some(Token::Punctuation(b':'))) + || !matches!(array.next(), Some(Token::Punctuation(b'['))) + { + continue; + } + let mut array_depth = 1usize; + let mut tuple_first = false; + while array_depth != 0 { + match array.next().ok_or("incomplete selected resource field")? { + Token::Punctuation(b'[') => { + array_depth += 1; + tuple_first = array_depth == 2; + } + Token::Punctuation(b'{') => { + array_depth += 1; + tuple_first = false; + } + Token::Punctuation(b']' | b'}') => { + array_depth -= 1; + tuple_first = false; + } + Token::String(start, end) + if array_depth == 1 + || (field == "OpenCode plugin" + && array_depth == 2 + && tuple_first) => + { + tuple_first = false; + if result.len() >= 100_000 || end - start > 64 * 1024 { + return Err("configured resource selection profile exceeded"); + } + result.push(( + field, + serde_json::from_str(&raw[start..end]) + .map_err(|_| "invalid selected resource path")?, + )); + } + _ if array_depth == 2 => tuple_first = false, + _ => {} + } + } + cursor = array; + } + _ => {} + } + } + Ok(result) +} + +fn home_path(path: &str, home: &Path) -> PathBuf { + if let Some(relative) = path.strip_prefix("~/") { + home.join(relative) + } else { + path.into() + } +} +pub struct ResourceError { + pub field: &'static str, + pub cause: &'static str, +} +fn checked(path: &Path, field: &'static str, _source: &Path) -> Result { + let fail = |cause| ResourceError { field, cause }; + let canonical = std::fs::canonicalize(path) + .map_err(|_| fail("selected path is missing, inaccessible, dangling or cyclic"))?; + let metadata = + std::fs::metadata(&canonical).map_err(|_| fail("selected path metadata unavailable"))?; + if metadata.is_file() { + std::fs::File::open(&canonical).map_err(|_| fail("selected file is unreadable"))?; + } else if metadata.is_dir() { + std::fs::read_dir(&canonical).map_err(|_| fail("selected directory is unreadable"))?; + } else { + return Err(fail("selected resource has an unsupported type")); + } + Ok(canonical) +} + +/// Return complete ordinary units and native installed ancestor lookup units. +/// Exact mounts retain native field spellings and their declaring origins. +pub fn select( + source: &PlannedSourceOccurrence, + home: &Path, + global: Option<&Path>, + lookup: &dyn Fn(&str) -> Option, +) -> Result<(Vec, Vec), ResourceError> { + let declaring = &source.source.logical_path; + let fail = |field: &'static str, cause: &'static str| ResourceError { field, cause }; + let mut result = std::collections::BTreeSet::new(); + let mut resource_path_bytes = 0usize; + let mut insert = |path: PathBuf, field| { + if !result.contains(&path) { + let bytes = path.as_os_str().len(); + if result.len() >= 100_000 + || bytes > (64 * 1024 * 1024usize).saturating_sub(resource_path_bytes) + { + return Err(fail( + field, + "configured resource selection exceeds shared entry or metadata profile", + )); + } + resource_path_bytes += bytes; + result.insert(path); + } + Ok(()) + }; + let mut variables = std::collections::BTreeSet::new(); + for (field, spelling) in fields(source.references.unexpanded_source()) + .map_err(|cause| fail("OpenCode resource", cause))? + { + // Private file-token payloads already use the accepted grant plan. + if spelling.contains("{file:") { + continue; + } + let names = std::cell::RefCell::new(std::collections::BTreeSet::new()); + let introduced_file = std::cell::Cell::new(false); + let invalid_input = std::cell::Cell::new(false); + let spelling = freshell_platform::opencode_config::transform_file_tokens( + &spelling, + &|name| { + names.borrow_mut().insert(name.to_owned()); + let value = lookup(name); + if value.as_ref().is_some_and(|value| value.len() > 64 * 1024) { + invalid_input.set(true); + return None; + } + value + }, + |_| { + introduced_file.set(true); + Ok::<_, ()>(String::new()) + }, + ) + .map_err(|_| { + fail( + field, + "selected path substitution exceeds its finite profile", + ) + })?; + if invalid_input.get() || spelling.len() > 64 * 1024 { + return Err(fail(field, "selected path input exceeds 64 KiB")); + } + if introduced_file.get() { + continue; + } + if spelling.starts_with("http://") || spelling.starts_with("https://") { + continue; + } + let mut path = if field == "OpenCode plugin" { + if spelling.starts_with("file:") { + reqwest::Url::parse(&spelling) + .ok() + .and_then(|url| url.to_file_path().ok()) + .ok_or_else(|| fail(field, "local file URL is invalid"))? + } else if spelling.starts_with(['/', '.']) || spelling.starts_with("~/") { + home_path(&spelling, home) + } else { + continue; + } // Native package identity remains native. + } else { + home_path(&spelling, home) + }; + let runtime_home = Path::new("/home/freshell/provider"); + if field == "OpenCode plugin" + && [ + freshell_platform::opencode_plugin::rebind_plugin_path(runtime_home), + freshell_platform::opencode_plugin::rebind_plugin_package_path(runtime_home), + ] + .contains(&path) + { + continue; + } + if let Ok(relative) = path.strip_prefix(runtime_home.join(".config/opencode")) { + path = global + .unwrap_or(&home.join(".config/opencode")) + .join(relative); + } else if let Ok(relative) = path.strip_prefix(runtime_home) { + path = home.join(relative); + } + if field == "OpenCode instructions" && path.to_string_lossy().contains(['*', '?', '[']) { + let candidates = if path.is_absolute() { + vec![path] + } else { + source + .instance_cwd + .ancestors() + .map(|base| base.join(&path)) + .collect() + }; + let mut matched = false; + for candidate in candidates { + let pattern = candidate + .to_str() + .ok_or_else(|| fail(field, "instruction glob is not UTF-8"))?; + for selected in + glob::glob(pattern).map_err(|_| fail(field, "instruction glob is invalid"))? + { + let selected = + selected.map_err(|_| fail(field, "instruction glob traversal failed"))?; + let canonical = checked(&selected, field, declaring)?; + if !canonical.is_file() { + return Err(fail(field, "selected instruction is not a regular file")); + } + insert(selected, field)?; + matched = true; + } + if matched { + break; + } + } + if !matched { + return Err(fail( + field, + "selected instruction glob has no available files", + )); + } + variables.extend(names.into_inner()); + continue; + } + if !path.is_absolute() { + if field == "OpenCode plugin" { + path = declaring + .parent() + .unwrap_or(&source.instance_cwd) + .join(path); + } else { + path = source + .instance_cwd + .ancestors() + .map(|base| base.join(&path)) + .find(|candidate| std::fs::symlink_metadata(candidate).is_ok()) + .unwrap_or_else(|| source.instance_cwd.join(path)); + } + } + let canonical = checked(&path, field, declaring)?; + variables.extend(names.into_inner()); + if field == "OpenCode instructions" { + if !canonical.is_file() { + return Err(fail(field, "selected instruction is not a regular file")); + } + insert(path, field)?; + continue; + } + // The native global plugin discovery unit is already selected in the + // ordinary global projection, including its installed dependencies. + // Do not widen it to the entire config package (private siblings belong + // to the indexed/grant path, not ordinary package transport). + if global.is_some_and(|global| path.starts_with(global.join("plugins"))) { + continue; + } + let mut scope = if canonical.is_dir() { + path.clone() + } else { + path.parent() + .ok_or_else(|| fail(field, "plugin has no containing scope"))? + .to_owned() + }; + if canonical.is_file() { + for ancestor in scope.ancestors() { + if ancestor == home || ancestor == Path::new("/") { + break; + } + if ancestor.join("package.json").is_file() { + scope = ancestor.to_owned(); + break; + } + } + } + if scope == home || scope == Path::new("/") { + return Err(fail(field, "select a bounded plugin or package directory")); + } + insert(scope.clone(), field)?; + for ancestor in scope.parent().into_iter().flat_map(Path::ancestors) { + if ancestor == Path::new("/") { + break; + } + for name in [ + "node_modules", + "package.json", + "package-lock.json", + "bun.lock", + "bun.lockb", + "pnpm-lock.yaml", + ] { + let candidate = ancestor.join(name); + match std::fs::symlink_metadata(&candidate) { + Ok(_) => { + checked(&candidate, field, declaring)?; + insert(candidate, field)?; + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => { + return Err(fail( + field, + "installed plugin lookup context is unavailable", + )) + } + } + } + if ancestor == home { + break; + } + } + } + Ok(( + result.into_iter().collect(), + variables.into_iter().collect(), + )) +} diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index acd3058f5..6b00fe501 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -309,6 +309,8 @@ pub struct PlannedSourceOccurrence { pub references: super::ResolvedFileReferencePlan, pub original_form: bool, pub requires_config_bind: bool, + pub instance_cwd: PathBuf, + pub configured_resources: Vec, } #[derive(Default)] @@ -485,6 +487,11 @@ pub fn plan_loaded_opencode_sources_relocated( lookup: &dyn Fn(SourceLane, &Path, &str) -> Option, relocated_files: &std::collections::BTreeMap, ) -> Result, super::ReferenceError> { + let global = sources + .iter() + .find(|source| source.source.role == DiscoveryRole::Global) + .and_then(|source| source.source.logical_path.parent()) + .map(Path::to_owned); let original_names = source_environment_names(&sources)?; let mut canonical_occurrences = std::collections::BTreeMap::new(); for source in &sources { @@ -636,12 +643,31 @@ pub fn plan_loaded_opencode_sources_relocated( "combined source recipe, alias, variable or event limit exceeded", )); } - result.push(PlannedSourceOccurrence { + let mut planned = PlannedSourceOccurrence { source, references, original_form, requires_config_bind: loaded.requires_config_bind, - }); + instance_cwd: cwd.into(), + configured_resources: Vec::new(), + }; + let (resources, names) = + super::opencode_resources::select(&planned, home, global.as_deref(), &|name| { + lookup(planned.source.lane, &planned.source.logical_path, name) + }) + .map_err(|error| super::ReferenceError { + operation: "select", + field: error.field, + source_ordinal: ordinal, + declaring_source: planned.source.logical_path.clone(), + variables: vec![], + cause: error.cause, + })?; + planned.configured_resources = resources; + planned.references.path_variables.extend(names); + planned.references.path_variables.sort(); + planned.references.path_variables.dedup(); + result.push(planned); } Ok(result) } diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index af18de2a2..d106afa8d 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -31,7 +31,7 @@ const broker=new RestrictedDockerBroker({ ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] : configuration.provider==='amplifier' ? ['/home/freshell/provider',path.join(root,'source-home/.amplifier')] - : ['/home/freshell/provider',path.join(root,'.opencode')]), + : ['/home/freshell/provider',path.join(root,'.opencode'),path.join(root,'scenarios/external-native-resources')]), allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json'),path.join(root,'grants/ca.pem')]), allowedHostGatewayAddresses:gateways, testRunId:configuration.runId,logPath:path.join(root,'broker.jsonl'), From 19081517d0016f93b8e1090cd0d789c4ee3a3dca Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 04:27:55 -0700 Subject: [PATCH 178/493] fix: reconcile indexed provider publications across cold backends --- .../src/managed_provider_launch_tests.rs | 98 ++- crates/freshell-supervisor/src/backend.rs | 18 + .../src/backend/input_publisher.rs | 700 +++++++++++++++++- crates/freshell-supervisor/src/service.rs | 15 + .../testing/managed-provider-launch-broker.ts | 1 + scripts/testing/runtime-test-broker.ts | 60 +- .../testing/runtime-test-broker.test.ts | 67 +- 7 files changed, 895 insertions(+), 64 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 71d736a8c..88e48f34d 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -58,7 +58,12 @@ fn native_exec_profile() -> Value { fn assert_private_sentinels_absent(root: &Path, certificate_sentinel: &[u8]) { for entry in std::fs::read_dir(root).unwrap() { let entry = entry.unwrap(); - let metadata = entry.file_type().unwrap(); + let metadata = entry.file_type().unwrap_or_else(|error| { + panic!( + "owned privacy scan metadata {}: {error}", + entry.path().display() + ) + }); if metadata.is_dir() { assert_private_sentinels_absent(&entry.path(), certificate_sentinel); } else if metadata.is_file() { @@ -94,11 +99,15 @@ fn assert_private_sentinels_absent(root: &Path, certificate_sentinel: &[u8]) { /// added operation delivers metadata to the closed fixture's trusted broker. struct BrokerAuthorizedBackend { real: DockerEngineBackend, + cold_backend: std::sync::OnceLock, control: PathBuf, handoff: PathBuf, certificate_sentinel: Vec, } impl BrokerAuthorizedBackend { + fn real(&self) -> &DockerEngineBackend { + self.cold_backend.get().unwrap_or(&self.real) + } async fn authorize_claude_candidate( &self, owner: &freshell_supervisor::backend::ClaudeSourceOwner, @@ -339,7 +348,7 @@ impl RuntimeBackend for BrokerAuthorizedBackend { ) -> Result { self.authorize_claude_candidate(owner, request).await?; let reply = match self - .real + .real() .observe_claude_sources(owner, request, test_run_id) .await { @@ -404,13 +413,32 @@ impl RuntimeBackend for BrokerAuthorizedBackend { &self, reservation: &OwnedInputReservation, ) -> Result { - self.real.observe_initial_auth_volume(reservation).await + self.real().observe_initial_auth_volume(reservation).await } async fn commit_prepared_inputs( &self, handle: &OwnedRuntimeHandle, ) -> Result<(), BackendError> { - self.real.commit_prepared_inputs(handle).await + if handle.terminal().is_some_and(|v| v.mode == "opencode") { + // The accepted workload now holds the tmpfs. A second fresh + // constructor releases the exact old publisher without its map. + let cold = DockerEngineBackend::new(self.handoff.parent().unwrap().join("broker.sock")); + cold.commit_prepared_inputs(handle).await?; + println!( + "provider_indexed_cold_lifecycle {}", + json!({"phase":"accepted-release","freshConstructor":true,"workloadVolumePreserved":true}) + ); + } + self.real().commit_prepared_inputs(handle).await + } + async fn retire_prior_prepared_inputs( + &self, + current: &OwnedRuntimeHandle, + prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + self.real() + .retire_prior_prepared_inputs(current, prior) + .await } async fn read_native_auth( &self, @@ -418,7 +446,7 @@ impl RuntimeBackend for BrokerAuthorizedBackend { keys: &[String], binary: &Path, ) -> Result, BackendError> { - self.real.read_native_auth(handle, keys, binary).await + self.real().read_native_auth(handle, keys, binary).await } async fn read_native_history( &self, @@ -427,7 +455,7 @@ impl RuntimeBackend for BrokerAuthorizedBackend { id: &str, binary: &Path, ) -> Result { - self.real + self.real() .read_native_history(handle, provider, id, binary) .await } @@ -436,7 +464,28 @@ impl RuntimeBackend for BrokerAuthorizedBackend { spec: &CreateRuntimeSpec, ) -> Result { self.authorize(spec).await?; - let created = self.real.create_stopped(spec).await?; + if spec.terminal.as_ref().is_some_and(|v| v.mode == "opencode") + && self.cold_backend.get().is_none() + { + // The finite broker fails BEFORE forwarding main create, after the + // real publisher seals its tmpfs. Drop every old backend reference + // then construct a fresh production backend, not a cloned map. + let old = DockerEngineBackend::new(self.handoff.parent().unwrap().join("broker.sock")); + let failed = old.create_stopped(spec).await; + assert!(matches!(failed,Err(BackendError::DockerHttp{status:503,..})),"closed cold-boundary failure must occur after actual publication and before workload create"); + drop(old); + assert!(self + .cold_backend + .set(DockerEngineBackend::new( + self.handoff.parent().unwrap().join("broker.sock") + )) + .is_ok()); + println!( + "provider_indexed_cold_lifecycle {}", + json!({"phase":"population-before-create","freshConstructor":true,"oldBackendDropped":true,"noWorkloadDispatch":true}) + ); + } + let created = self.real().create_stopped(spec).await?; if std::env::var("FRESHELL_PROVIDER_LAUNCH_CLAUDE_SOURCE_GUARD").as_deref() == Ok("changed-git") { @@ -459,41 +508,41 @@ impl RuntimeBackend for BrokerAuthorizedBackend { Ok(created) } async fn start_host(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { - self.real.start_host(handle).await + self.real().start_host(handle).await } async fn inspect( &self, handle: &OwnedRuntimeHandle, ) -> Result { - self.real.inspect(handle).await + self.real().inspect(handle).await } async fn request_stop( &self, handle: &OwnedRuntimeHandle, grace: u64, ) -> Result<(), BackendError> { - self.real.request_stop(handle, grace).await + self.real().request_stop(handle, grace).await } async fn force_stop(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { - self.real.force_stop(handle).await + self.real().force_stop(handle).await } async fn verify_empty(&self, handle: &OwnedRuntimeHandle) -> Result { - self.real.verify_empty(handle).await + self.real().verify_empty(handle).await } async fn list_known( &self, handles: &[OwnedRuntimeHandle], ) -> Result, BackendError> { - self.real.list_known(handles).await + self.real().list_known(handles).await } async fn read_limits( &self, handle: &OwnedRuntimeHandle, ) -> Result { - self.real.read_limits(handle).await + self.real().read_limits(handle).await } async fn enable_long_lived(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { - self.real.enable_long_lived(handle).await + self.real().enable_long_lived(handle).await } } @@ -758,6 +807,7 @@ async def on_session_ready(coordinator): freshell_supervisor::backend::docker::configure_capability_dir(&socket).unwrap(); let backend = Arc::new(BrokerAuthorizedBackend { real: DockerEngineBackend::new(fixture.join("broker.sock")), + cold_backend: Default::default(), control: control.clone(), handoff: fixture.join("handoff"), certificate_sentinel: b"synthetic-amplifier-onecli-value".to_vec(), @@ -1752,6 +1802,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ registry.clone(), Arc::new(BrokerAuthorizedBackend { real: DockerEngineBackend::new(fixture.join("broker.sock")), + cold_backend: Default::default(), control: control.clone(), handoff: fixture.join("handoff"), certificate_sentinel: b"synthetic-claude-onecli-value".to_vec(), @@ -2948,6 +2999,7 @@ require('node:readline').createInterface({input:process.stdin}).on('line',line=> registry.clone(), Arc::new(BrokerAuthorizedBackend { real: DockerEngineBackend::new(fixture.join("broker.sock")), + cold_backend: Default::default(), control: control.clone(), handoff: fixture.join("handoff"), certificate_sentinel: b"unselected-private-sentinel".to_vec(), @@ -3220,6 +3272,7 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr assert!(certificate_sentinel.len() >= 32); let backend = Arc::new(BrokerAuthorizedBackend { real: DockerEngineBackend::new(fixture.join("broker.sock")), + cold_backend: Default::default(), control: control.clone(), handoff: fixture.join("handoff"), certificate_sentinel: certificate_sentinel.clone(), @@ -3279,12 +3332,12 @@ async fn managed_provider_opencode_terminal_and_fresh_native_startup_consumes_pr let Ok((mut stream, _)) = web.accept().await else { break; }; - tokio::spawn(async move { + { use tokio::io::{AsyncReadExt, AsyncWriteExt}; let mut bytes = [0; 4096]; let _ = stream.read(&mut bytes).await; let _=stream.write_all(b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: 13\r\nConnection: close\r\n\r\n{\"config\":{}}").await; - }); + } } }); crate::managed_mcp_capability::configure("0.0.0.0", port, &socket, &secret_file).unwrap(); @@ -3987,15 +4040,18 @@ export default {{id:'closed-directory-discovery',async server(){{ "provider_launch_consumer {}", json!({"provider":"opencode","route":"fresh","finalExecProfile":native_exec_profile(),"workspaceScope":"workspace-under-home","digest":format!("{:x}",Sha256::digest(serde_json::to_vec(&fresh_expected).unwrap()))}) ); - // Scan only run-owned durable/staged surfaces; referenced payloads belong - // in private indexed files, not ordinary resource/configuration trees. - assert_private_sentinels_absent(&fixture.join("registry"), &certificate_sentinel); - assert_private_sentinels_absent(&fixture.join("handoff"), &certificate_sentinel); for view in client.inventory().await.unwrap() { if view.desired_state == DesiredState::Running { client.stop(view.soul_id).await.unwrap(); } } web_task.abort(); + let _ = web_task.await; control_task.abort(); + let _ = control_task.await; + // Each prepared root is also scanned before publication. Join owned + // writers before this final complete scan so retiring SQLite/temp entries + // cannot disappear between enumeration and reading. No entry is omitted. + assert_private_sentinels_absent(&fixture.join("registry"), &certificate_sentinel); + assert_private_sentinels_absent(&fixture.join("handoff"), &certificate_sentinel); } diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 22504e6b0..3d52e7336 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -160,6 +160,15 @@ pub trait RuntimeBackend: Send + Sync { ) -> Result<(), BackendError> { Ok(()) } + /// Called only after accepted grant/registry commit and exact native + /// verification when a resume requires it. No pre-dispatch retirement. + async fn retire_prior_prepared_inputs( + &self, + _current: &OwnedRuntimeHandle, + _prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + Ok(()) + } async fn read_native_auth( &self, _handle: &OwnedRuntimeHandle, @@ -632,11 +641,20 @@ impl RuntimeBackend for DockerEngineBackend { handle.soul_id(), handle.incarnation_id(), &inputs.generation_id, + &json!({"com.freshell.managed-inputs":"true","com.freshell.installation-id":handle.installation_id(),"com.freshell.soul-id":handle.soul_id(),"com.freshell.incarnation-id":handle.incarnation_id(),"com.freshell.input-generation":inputs.generation_id}), + handle.image_ref(),handle.host_binary_path(), ) .await?; } Ok(()) } + async fn retire_prior_prepared_inputs( + &self, + current: &OwnedRuntimeHandle, + prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + self.retire_prior_indexed_inputs(current, prior).await + } async fn read_native_auth( &self, handle: &OwnedRuntimeHandle, diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index 87a4891fc..90a0b6254 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -77,6 +77,35 @@ fn labels(spec: &CreateRuntimeSpec, generation: &str) -> Value { "com.freshell.input-generation":generation}) } +fn filter_query(value: Value) -> String { + serde_json::to_vec(&value) + .expect("finite Docker filter") + .into_iter() + .map(|byte| format!("%{byte:02X}")) + .collect() +} +fn helper_body( + image: &str, + ownership: &Value, + binary: &Path, + volume: &str, + directory: &Path, +) -> Value { + json!({ + "Image":image,"User":"65534:0","Tty":false,"Labels":ownership, + "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["publish-provider-inputs"],"Env":[], + "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"SecurityOpt":["no-new-privileges"], + "Memory":128*1024*1024,"MemorySwap":128*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, + "LogConfig":{"Type":"none","Config":{}}, + "Mounts":[{"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, + {"Type":"volume","Source":volume,"Target":"/run/freshell-input-files","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}, + {"Type":"bind","Source":directory,"Target":"/run/freshell-input-exchange","ReadOnly":true}]} + }) +} +fn volume_options() -> Value { + json!({"type":"tmpfs","device":"tmpfs","o":prepared_inputs::INDEXED_VOLUME_OPTIONS}) +} + fn environment(value: &Value) -> Option> { let mut result = BTreeMap::new(); for entry in value.as_array()? { @@ -228,6 +257,237 @@ impl DockerEngineBackend { .map(|(key, value)| format!("{key}={value}")) .collect()) } + async fn input_ownership_request(&self, path: &str) -> Result { + tokio::time::timeout( + Duration::from_secs(5), + self.request_bounded("GET", path, None, 1024 * 1024), + ) + .await + .map_err(|_| fail("input ownership inspection timed out; preserve preparation"))? + } + async fn verify_no_input_workloads( + &self, + volume: &str, + helper: Option<&str>, + ) -> Result<(), BackendError> { + let filters = filter_query(json!({"volume":[volume]})); + let response = self + .input_ownership_request(&format!( + "{DOCKER_API}/containers/json?all=1&filters={filters}" + )) + .await + .map_err(|_| fail("input generation users unavailable; preserve preparation"))?; + if response.status != 200 { + return Err(fail( + "input generation users unavailable; preserve preparation", + )); + } + let values: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("input generation users invalid; preserve preparation"))?; + let values = values + .as_array() + .ok_or_else(|| fail("input generation users invalid; preserve preparation"))?; + for value in values { + let id = value["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|b| b.is_ascii_hexdigit())) + .ok_or_else(|| { + fail("input generation user identity invalid; preserve preparation") + })?; + if Some(id) != helper { + return Err(fail("input generation has an uncertain workload reference; preserve dispatched ownership")); + } + } + Ok(()) + } + async fn check_input_generation_capacity( + &self, + spec: &CreateRuntimeSpec, + current: &str, + ) -> Result<(), BackendError> { + let filters = filter_query( + json!({"label":["com.freshell.managed-inputs=true",format!("com.freshell.installation-id={}",spec.installation_id),format!("com.freshell.soul-id={}",spec.soul_id)]}), + ); + let response = self + .input_ownership_request(&format!("{DOCKER_API}/volumes?filters={filters}")) + .await + .map_err(|_| fail("input generation ownership inventory unavailable"))?; + if response.status != 200 { + return Err(fail("input generation ownership inventory unavailable")); + } + let value: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("input generation ownership inventory invalid"))?; + let empty = Vec::new(); + let volumes = if value["Volumes"].is_null() { + &empty + } else { + value["Volumes"] + .as_array() + .ok_or_else(|| fail("input generation ownership inventory invalid"))? + }; + let mut names = std::collections::BTreeSet::new(); + for volume in volumes { + let generation = volume["Labels"]["com.freshell.input-generation"] + .as_str() + .filter(|v| { + v.len() <= 128 + && v.starts_with("grant-") + && v.bytes() + .all(|b| b.is_ascii_alphanumeric() || b == b'-' || b == b'_') + }) + .ok_or_else(|| fail("input generation inventory ownership changed"))?; + let incarnation = volume["Labels"]["com.freshell.incarnation-id"] + .as_str() + .ok_or_else(|| fail("input generation inventory ownership changed"))?; + IncarnationId::parse(incarnation.to_owned()) + .map_err(|_| fail("input generation inventory owner invalid"))?; + let expected = + freshell_runtime_protocol::indexed_provider_volume_name(&spec.soul_id, generation); + let mut ownership = labels(spec, generation); + ownership["com.freshell.incarnation-id"] = json!(incarnation); + if volume["Name"] != expected + || volume["Labels"] != ownership + || volume["Driver"] != "local" + || volume["Options"] != volume_options() + || !names.insert(expected) + { + return Err(fail( + "input generation inventory ownership changed; preserve volumes", + )); + } + if names.len() > 2 { + return Err(fail("input generations exceed one active and one pending; reconcile exact ownership")); + } + } + if names.len() == 2 && !names.contains(current) { + return Err(fail( + "one active and one pending input generation already exist; preserve and reconcile", + )); + } + Ok(()) + } + async fn inspect_cold_publisher( + &self, + volume: &str, + ownership: &Value, + image: &str, + binary: &Path, + ) -> Result, BackendError> { + let name = format!("{volume}-publisher"); + let response = self + .input_ownership_request(&format!("{DOCKER_API}/containers/{name}/json")) + .await + .map_err(|_| fail("cold input publisher inspection unavailable"))?; + if response.status == 404 { + return Ok(None); + } + if response.status != 200 { + return Err(fail("cold input publisher inspection unavailable")); + } + let actual: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("cold input publisher metadata invalid"))?; + let mounts = actual["HostConfig"]["Mounts"] + .as_array() + .ok_or_else(|| fail("cold input publisher mounts invalid"))?; + let directory = mounts + .iter() + .find(|m| m["Target"] == "/run/freshell-input-exchange") + .and_then(|m| m["Source"].as_str()) + .map(PathBuf::from) + .ok_or_else(|| fail("cold input publisher exchange invalid"))?; + let suffix = directory + .file_name() + .and_then(|v| v.to_str()) + .and_then(|v| v.strip_prefix("freshell-inputs-")) + .ok_or_else(|| fail("cold input publisher exchange ownership changed"))?; + if directory.parent() != Some(std::env::temp_dir().as_path()) + || uuid::Uuid::parse_str(suffix) + .ok() + .is_none_or(|id| id.get_version_num() != 4 || id.to_string() != suffix) + { + return Err(fail("cold input publisher exchange ownership changed")); + } + let mut expected = helper_body(image, ownership, binary, volume, &directory); + expected["Env"] = serde_json::to_value( + tokio::time::timeout( + Duration::from_secs(5), + self.private_helper_environment(image, &[]), + ) + .await + .map_err(|_| fail("cold input publisher image inspection timed out"))??, + ) + .map_err(|_| fail("cold input publisher environment invalid"))?; + let host = &actual["HostConfig"]; + let id = actual["Id"] + .as_str() + .filter(|id| id.len() == 64 && id.bytes().all(|b| b.is_ascii_hexdigit())) + .ok_or_else(|| fail("cold input publisher identity invalid"))?; + if actual["Name"] != format!("/{name}") + || !owned_helper(&actual, &expected) + || [ + "NetworkMode", + "ReadonlyRootfs", + "CapDrop", + "SecurityOpt", + "Memory", + "MemorySwap", + "NanoCpus", + "PidsLimit", + ] + .iter() + .any(|key| host[key] != expected["HostConfig"][key]) + || host["Privileged"].as_bool().unwrap_or(false) + || !host["PidMode"].as_str().unwrap_or("").is_empty() + || ["CapAdd", "Devices", "Binds", "VolumesFrom"] + .iter() + .any(|key| host[key].as_array().is_some_and(|v| !v.is_empty())) + { + return Err(fail( + "cold input publisher fixed ownership changed; preserve helper", + )); + } + Ok(Some((expected, id.to_owned()))) + } + async fn cleanup_cold_publisher( + &self, + volume: &str, + expected: &Value, + ) -> Result<(), BackendError> { + self.cleanup_input_helper(&format!("{volume}-publisher"), expected) + .await?; + // Only the exact private socket and its empty directory are retired. + // Never recurse or remove a changed path inferred from Docker metadata. + use std::os::unix::fs::{MetadataExt, PermissionsExt}; + let directory = PathBuf::from( + expected["HostConfig"]["Mounts"][2]["Source"] + .as_str() + .ok_or_else(|| fail("cold input publisher exchange unavailable"))?, + ); + if let Ok(meta) = std::fs::symlink_metadata(&directory) { + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + if meta.is_dir() + && !meta.file_type().is_symlink() + && meta.uid() == uid + && meta.gid() == gid + && meta.permissions().mode() & 0o777 == 0o710 + { + let socket = directory.join("reply.sock"); + if std::fs::symlink_metadata(&socket).is_ok_and(|m| { + m.uid() == uid + && m.gid() == gid + && m.permissions().mode() & 0o777 == 0o660 + && std::os::unix::fs::FileTypeExt::is_socket(&m.file_type()) + }) { + let _ = std::fs::remove_file(socket); + } + let _ = std::fs::remove_dir(directory); + } else { + tracing::warn!("runtime.input_publication_cold_exchange_preserved"); + } + } + Ok(()) + } + pub(super) async fn abort_indexed_publication( &self, spec: &CreateRuntimeSpec, @@ -252,20 +512,36 @@ impl DockerEngineBackend { .input_publishers .lock() .map_err(|_| fail("input publisher ownership unavailable"))?; - let Some(publisher) = held.get(&files.volume_name) else { - return Ok(()); - }; - if publisher.soul != spec.soul_id.as_str() - || publisher.incarnation != spec.incarnation_id.as_str() - || publisher.context != *context.unwrap() - { - return Err(fail("input publisher abort belongs to another preparation")); + if let Some(publisher) = held.get(&files.volume_name) { + if publisher.soul != spec.soul_id.as_str() + || publisher.incarnation != spec.incarnation_id.as_str() + || publisher.context != *context.unwrap() + { + return Err(fail("input publisher abort belongs to another preparation")); + } } } + let ownership = labels(spec, &inputs.generation_id); + let cold = self + .inspect_cold_publisher( + &files.volume_name, + &ownership, + &spec.image_ref, + &spec.host_binary_path, + ) + .await?; + self.verify_no_input_workloads( + &files.volume_name, + cold.as_ref().map(|(_, id)| id.as_str()), + ) + .await?; self.release_indexed_publication( &spec.soul_id, &spec.incarnation_id, &inputs.generation_id, + &ownership, + &spec.image_ref, + &spec.host_binary_path, ) .await?; self.cleanup_input_volume( @@ -374,18 +650,14 @@ impl DockerEngineBackend { directory.mode(&socket, 0o660)?; let name = format!("{}-publisher", files.volume_name); let ownership = labels(spec, &inputs.generation_id); - let mut body = json!({ - "Image":spec.image_ref,"User":"65534:0","Tty":false,"Labels":ownership, - "Entrypoint":["/runtime/freshell-session-host"],"Cmd":["publish-provider-inputs"],"Env":[], - "HostConfig":{"NetworkMode":"none","ReadonlyRootfs":true,"CapDrop":["ALL"],"SecurityOpt":["no-new-privileges"], - "Memory":128*1024*1024,"MemorySwap":128*1024*1024,"NanoCpus":500_000_000,"PidsLimit":32, - "LogConfig":{"Type":"none","Config":{}}, - "Mounts":[{"Type":"bind","Source":binary,"Target":"/runtime/freshell-session-host","ReadOnly":true}, - {"Type":"volume","Source":files.volume_name,"Target":"/run/freshell-input-files","ReadOnly":false,"VolumeOptions":{"NoCopy":true}}, - {"Type":"bind","Source":directory.0,"Target":"/run/freshell-input-exchange","ReadOnly":true}]} - }); - let options = - json!({"type":"tmpfs","device":"tmpfs","o":prepared_inputs::INDEXED_VOLUME_OPTIONS}); + let mut body = helper_body( + &spec.image_ref, + &ownership, + binary, + &files.volume_name, + &directory.0, + ); + let options = volume_options(); let mut volume_attempted = false; let mut helper_attempted = false; let operation = async { @@ -403,9 +675,38 @@ impl DockerEngineBackend { ) .await .map_err(|_| fail("input publisher expected-volume query failed"))?; - if prior.status != 404 { + self.check_input_generation_capacity(spec, &files.volume_name) + .await?; + if prior.status == 200 { + let metadata: Value = serde_json::from_slice(&prior.body) + .map_err(|_| fail("input volume cold metadata invalid"))?; + if metadata["Name"] != files.volume_name + || metadata["Driver"] != "local" + || metadata["Labels"] != ownership + || metadata["Options"] != options + { + return Err(fail( + "input volume cold ownership changed; preserve preparation", + )); + } + let cold = self + .inspect_cold_publisher(&files.volume_name, &ownership, &spec.image_ref, binary) + .await?; + self.verify_no_input_workloads( + &files.volume_name, + cold.as_ref().map(|(_, id)| id.as_str()), + ) + .await?; + if let Some((expected, _)) = cold { + self.cleanup_cold_publisher(&files.volume_name, &expected) + .await?; + } + self.cleanup_input_volume(&files.volume_name, &ownership, &options) + .await?; + tracing::info!(soul_id=%spec.soul_id,generation_id=%inputs.generation_id,"runtime.input_publication_cold_repopulation"); + } else if prior.status != 404 { return Err(fail( - "input publisher volume already exists; reconcile before preparation", + "input volume cold ownership unavailable; preserve preparation", )); } volume_attempted = true; @@ -657,11 +958,110 @@ impl DockerEngineBackend { .map_err(|_| fail("input volume cleanup timed out"))? } + pub(super) async fn retire_prior_indexed_inputs( + &self, + current: &OwnedRuntimeHandle, + prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + fn context( + handle: &OwnedRuntimeHandle, + ) -> Option<&freshell_runtime_protocol::ProviderLaunchContext> { + handle + .terminal() + .and_then(|v| v.provider_launch_context.as_ref()) + .or_else(|| { + handle + .fresh_agent() + .and_then(|v| v.provider_launch_context.as_ref()) + }) + } + let Some(inputs) = context(prior).and_then(|v| v.prepared_inputs.as_ref()) else { + return Ok(()); + }; + let Some(files) = inputs.indexed_files.as_ref() else { + return Ok(()); + }; + let next = context(current) + .and_then(|v| v.prepared_inputs.as_ref()) + .ok_or_else(|| fail("accepted current input generation unavailable; preserve prior"))?; + if current.installation_id() != prior.installation_id() + || current.soul_id() != prior.soul_id() + || current.incarnation_id() == prior.incarnation_id() + { + return Err(fail( + "prior input retirement owner changed; preserve generation", + )); + } + if next.generation_id == inputs.generation_id { + return Ok(()); + } + if files.volume_name + != freshell_runtime_protocol::indexed_provider_volume_name( + prior.soul_id(), + &inputs.generation_id, + ) + { + return Err(fail("prior input generation identity changed")); + } + let ownership = json!({"com.freshell.managed-inputs":"true","com.freshell.installation-id":prior.installation_id(),"com.freshell.soul-id":prior.soul_id(),"com.freshell.incarnation-id":prior.incarnation_id(),"com.freshell.input-generation":inputs.generation_id}); + let operation = async { + let state = self.inspect(prior).await?.state; + match state { + BackendRuntimeState::Missing => {} + BackendRuntimeState::Created + | BackendRuntimeState::Exited + | BackendRuntimeState::Dead => { + // Docker refuses removal if this exact proved object becomes + // running again. Retirement never forces a workload stop. + let removed = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/containers/{}?force=0", prior.container_id()), + None, + 64 * 1024, + ) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail( + "prior input workload removal uncertain; preserve generation", + )); + } + } + _ => { + return Err(fail( + "prior input workload remains live or uncertain; preserve generation", + )) + } + } + self.release_indexed_publication( + prior.soul_id(), + prior.incarnation_id(), + &inputs.generation_id, + &ownership, + prior.image_ref(), + prior.host_binary_path(), + ) + .await?; + self.verify_no_input_workloads(&files.volume_name, None) + .await?; + self.cleanup_input_volume(&files.volume_name, &ownership, &volume_options()) + .await?; + tracing::info!(soul_id=%prior.soul_id(),prior_incarnation_id=%prior.incarnation_id(),generation_id=%inputs.generation_id,"runtime.input_publication_prior_retired"); + Ok(()) + }; + tokio::time::timeout(Duration::from_secs(20), operation) + .await + .map_err(|_| fail("prior input retirement timed out; preserve uncertain generation"))? + } + pub(super) async fn release_indexed_publication( &self, soul: &SoulId, incarnation: &IncarnationId, generation: &str, + ownership: &Value, + image: &str, + binary: &Path, ) -> Result<(), BackendError> { let volume = freshell_runtime_protocol::indexed_provider_volume_name(soul, generation); let held = self @@ -670,6 +1070,13 @@ impl DockerEngineBackend { .map_err(|_| fail("input publisher ownership unavailable"))? .remove(&volume); let Some(mut held) = held else { + if let Some((expected, _)) = self + .inspect_cold_publisher(&volume, ownership, image, binary) + .await? + { + self.cleanup_cold_publisher(&volume, &expected).await?; + tracing::info!(soul_id=%soul,generation_id=generation,"runtime.input_publication_cold_release"); + } return Ok(()); }; if held.soul != soul.as_str() || held.incarnation != incarnation.as_str() { @@ -815,15 +1222,64 @@ mod tests { #[tokio::test] async fn provider_input_publisher_streams_only_private_ipc_and_pins_tmpfs_before_workload_creation( ) { - publication_fixture(false).await; + publication_fixture(PublicationCase::WarmCommit).await; } #[tokio::test] async fn provider_input_publisher_abort_removes_only_the_pending_owned_generation() { - publication_fixture(true).await; + publication_fixture(PublicationCase::WarmAbort).await; } - async fn publication_fixture(abort: bool) { + #[tokio::test] + async fn provider_input_publisher_cold_backend_repopulates_confirmed_undispatched_generation() { + publication_fixture(PublicationCase::ColdRetry).await; + } + #[tokio::test] + async fn provider_input_publisher_cold_abort_removes_exact_helper_and_volume() { + publication_fixture(PublicationCase::ColdAbort).await; + } + #[tokio::test] + async fn provider_input_publisher_cold_backend_preserves_uncertain_workload_reference() { + publication_fixture(PublicationCase::ColdUncertain).await; + } + #[derive(Clone, Copy)] + enum PublicationCase { + WarmCommit, + WarmAbort, + ColdRetry, + ColdAbort, + ColdUncertain, + ColdAcceptedRelease, + ColdCapacity, + ColdRetirement, + } + #[tokio::test] + async fn provider_input_publisher_cold_accepted_release_preserves_workload_volume() { + publication_fixture(PublicationCase::ColdAcceptedRelease).await; + } + #[tokio::test] + async fn provider_input_publisher_cold_inventory_bounds_one_active_and_one_pending() { + publication_fixture(PublicationCase::ColdCapacity).await; + } + #[tokio::test] + async fn provider_input_publisher_cold_retirement_removes_only_the_prior_accepted_generation() { + publication_fixture(PublicationCase::ColdRetirement).await; + } + async fn publication_fixture(case: PublicationCase) { + let abort = matches!( + case, + PublicationCase::WarmAbort + | PublicationCase::ColdAbort + | PublicationCase::ColdUncertain + ); + let cold = !matches!( + case, + PublicationCase::WarmCommit | PublicationCase::WarmAbort + ); + let uncertain = matches!(case, PublicationCase::ColdUncertain); + let accepted = matches!(case, PublicationCase::ColdAcceptedRelease); + let capacity = matches!(case, PublicationCase::ColdCapacity); + let retirement = matches!(case, PublicationCase::ColdRetirement); let owned = tempfile::tempdir().unwrap(); let source = owned.path().join("approved-source"); std::fs::write(&source, b"PRIVATE_PUBLISHER_SENTINEL").unwrap(); @@ -901,17 +1357,23 @@ mod tests { None::>>, )); let active_helper = helper.clone(); + let workload = Arc::new(Mutex::new(false)); + let workload_for_server = workload.clone(); + let over_capacity = Arc::new(Mutex::new(false)); + let over_capacity_for_server = over_capacity.clone(); let private_root = volume_root.clone(); let created_for_server = created.clone(); let server = tokio::spawn(async move { loop { let (mut stream, _) = listener.accept().await.unwrap(); - let (seen, created, volume, helper, private_root) = ( + let (seen, created, volume, helper, private_root, workload, over_capacity) = ( seen.clone(), created_for_server.clone(), volume.clone(), active_helper.clone(), private_root.clone(), + workload_for_server.clone(), + over_capacity_for_server.clone(), ); tokio::spawn(async move { let mut header = Vec::new(); @@ -932,9 +1394,44 @@ mod tests { stream.read_exact(&mut body).await.unwrap(); let request = header.lines().next().unwrap().to_string(); seen.lock().unwrap().push((request.clone(), body.clone())); - let (status, reply) = if request.contains("/images/") { + let (status, reply) = if request.contains("/info ") { + (200,json!({"ID":"input-publisher-daemon","SecurityOptions":["name=rootless"]}).to_string()) + } else if request.contains(&format!("/containers/{}/json ", "c".repeat(64))) { + (404, String::new()) + } else if request.contains("/images/") { (200,json!({"Config":{"Env":["PATH=/image/path","INHERITED_TOKEN=PRIVATE_IMAGE_SENTINEL"]}}).to_string()) + } else if request.contains("/volumes?filters=") { + let mut values = volume.lock().unwrap().iter().cloned().collect::>(); + if *over_capacity.lock().unwrap() { + for generation in ["grant-extra-one", "grant-extra-two"] { + let mut extra = values[0].clone(); + let soul = SoulId::parse( + extra["Labels"]["com.freshell.soul-id"].as_str().unwrap(), + ) + .unwrap(); + extra["Name"] = + json!(freshell_runtime_protocol::indexed_provider_volume_name( + &soul, generation + )); + extra["Labels"]["com.freshell.input-generation"] = + json!(generation); + values.push(extra); + } + } + (200, json!({"Volumes":values}).to_string()) + } else if request.contains("/containers/json?") { + let mut users = Vec::new(); + if created.lock().unwrap().is_some() { + users.push(json!({"Id":"b".repeat(64)})); + } + if *workload.lock().unwrap() { + users.push(json!({"Id":"c".repeat(64)})); + } + (200, serde_json::to_string(&users).unwrap()) } else if request.contains("/volumes/create ") { + if private_root.join("files").exists() { + std::fs::remove_dir_all(private_root.join("files")).unwrap(); + } let mut value: Value = serde_json::from_slice(&body).unwrap(); value["Mountpoint"] = json!("/owned/daemon/volume"); value["Options"] = value["DriverOpts"].clone(); @@ -970,11 +1467,15 @@ mod tests { })); (204, String::new()) } else if request.contains("/json ") { - let mut config = created.lock().unwrap().clone().unwrap(); - // Docker inspection includes normalized default fields. - config["HostConfig"]["Mounts"][0]["Consistency"] = json!("default"); - (200,json!({"Config":config,"HostConfig":config["HostConfig"],"State":{"Status":"running"}}).to_string()) + match created.lock().unwrap().clone() { + Some(mut config) => { + config["HostConfig"]["Mounts"][0]["Consistency"] = json!("default"); + (200,json!({"Id":"b".repeat(64),"Name":format!("/{}-publisher",config["HostConfig"]["Mounts"][1]["Source"].as_str().unwrap()),"Config":config,"HostConfig":config["HostConfig"],"State":{"Status":"running"}}).to_string()) + } + None => (404, String::new()), + } } else if request.starts_with("DELETE ") { + *created.lock().unwrap() = None; (204, String::new()) } else { panic!("unexpected owned Docker request {request}"); @@ -997,6 +1498,134 @@ mod tests { !helper.lock().unwrap().as_ref().unwrap().is_finished(), "publisher must pin completed tmpfs before workload creation" ); + if cold { + let expected_helper = created.lock().unwrap().clone().unwrap(); + drop(backend); + // The original controller disappears. Its helper sees EOF and is + // joined before any fresh constructor attempts reconciliation. + assert!(helper + .lock() + .unwrap() + .take() + .unwrap() + .join() + .unwrap() + .is_err()); + let fresh = DockerEngineBackend::new(&socket); + *workload.lock().unwrap() = uncertain || accepted; + *over_capacity.lock().unwrap() = capacity; + std::fs::write(&source, b"FRESH_APPROVED_PUBLISHER_BYTES").unwrap(); + let result = if retirement { + let make_handle = |candidate: &CreateRuntimeSpec| { + crate::registry::OwnedRuntimeHandle::from_registry( + candidate.installation_id.clone(), + candidate.soul_id.clone(), + candidate.incarnation_id.clone(), + freshell_runtime_protocol::LaunchNonce::new(), + DockerDaemonId::parse("input-publisher-daemon").unwrap(), + "c".repeat(64), + candidate.image_ref.clone(), + candidate.runtime_dir.clone(), + binary.clone(), + digest_expected(&ExpectedConfig::new( + candidate, + &binary, + &candidate.runtime_dir, + )) + .unwrap(), + candidate.limits, + None, + candidate.terminal.clone(), + None, + candidate.provider_volume_name.clone(), + ) + }; + let prior = make_handle(&spec); + let mut next = spec.clone(); + next.incarnation_id = IncarnationId::parse("incarnation-current").unwrap(); + let context = next + .terminal + .as_mut() + .unwrap() + .provider_launch_context + .as_mut() + .unwrap(); + context.prepared_inputs.as_mut().unwrap().generation_id = "grant-current".into(); + context + .prepared_inputs + .as_mut() + .unwrap() + .indexed_files + .as_mut() + .unwrap() + .volume_name = freshell_runtime_protocol::indexed_provider_volume_name( + &spec.soul_id, + "grant-current", + ); + fresh + .retire_prior_prepared_inputs(&make_handle(&next), &prior) + .await + } else if accepted { + fresh + .release_indexed_publication( + &spec.soul_id, + &spec.incarnation_id, + generation, + &labels(&spec, generation), + &spec.image_ref, + &binary, + ) + .await + } else if abort { + fresh.abort_indexed_publication(&spec).await + } else { + fresh + .begin_indexed_publication(&spec, &mounts, &binary) + .await + }; + let observed_deletes = captured + .lock() + .unwrap() + .iter() + .filter(|(request, _)| request.starts_with("DELETE ")) + .count(); + let progressed = result.is_ok() + && if accepted { + observed_deletes == 1 + && std::fs::read(volume_root.join("files/source-0")).unwrap() + == b"PRIVATE_PUBLISHER_SENTINEL" + } else if abort || retirement { + observed_deletes == 2 + } else { + std::fs::read(volume_root.join("files/source-0")) + .is_ok_and(|bytes| bytes == b"FRESH_APPROVED_PUBLISHER_BYTES") + }; + let preserved = (uncertain || capacity) + && result.is_err() + && observed_deletes == 0 + && std::fs::read(volume_root.join("files/source-0")).unwrap() + == b"PRIVATE_PUBLISHER_SENTINEL"; + *workload.lock().unwrap() = false; + *over_capacity.lock().unwrap() = false; + // Always settle exact fixture ownership before asserting RED. + if !abort && progressed && !accepted { + fresh.abort_indexed_publication(&spec).await.unwrap(); + } else { + fresh + .cleanup_input_helper(&format!("{volume_name}-publisher"), &expected_helper) + .await + .unwrap(); + fresh.cleanup_input_volume(&volume_name,&labels(&spec,generation),&json!({"type":"tmpfs","device":"tmpfs","o":prepared_inputs::INDEXED_VOLUME_OPTIONS})).await.unwrap(); + } + if let Some(thread) = helper.lock().unwrap().take() { + assert!(thread.join().unwrap().is_ok()); + } + drop(fresh); + server.abort(); + let _ = server.await; + assert!(if uncertain || capacity {preserved} else {progressed},"fresh backend must reconcile only exact undispatched ownership; result={result:?}, deletes={observed_deletes}"); + return; + } // A lost workload-create acknowledgement retries the same frozen // preparation. It must reuse the pinned bytes, even after the source // disappears, rather than publish a second generation. @@ -1037,7 +1666,14 @@ mod tests { backend.abort_indexed_publication(&spec).await.unwrap(); } else { backend - .release_indexed_publication(&spec.soul_id, &spec.incarnation_id, generation) + .release_indexed_publication( + &spec.soul_id, + &spec.incarnation_id, + generation, + &labels(&spec, generation), + &spec.image_ref, + &spec.host_binary_path, + ) .await .unwrap(); } diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index c2e19336f..29ac91d46 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -2066,6 +2066,21 @@ impl Supervisor { .commit_prepared_inputs(&handle) .await .map_err(map_backend)?; + // Both callers are after HostGrantAccepted and registry commit. + // Initial/pristine activation calls here only without resume_spec; + // exact resumes reach here from recovery only AFTER native identity + // verification, queued-input replay, and recovery-success commit. + if let Some(prior) = self + .registry + .prior_provider_owner(incarnation.clone()) + .await + .map_err(map_registry)? + { + self.backend + .retire_prior_prepared_inputs(&handle, &prior) + .await + .map_err(map_backend)?; + } } if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { self.commit_provider_path_inputs(&workload.soul_id, &inputs.generation_id); diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index d106afa8d..58aabb6f3 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -36,6 +36,7 @@ const broker=new RestrictedDockerBroker({ allowedHostGatewayAddresses:gateways, testRunId:configuration.runId,logPath:path.join(root,'broker.jsonl'), }) +if (configuration.provider==='opencode') broker.failNextInputPublicationBeforeCreate() for (const authorization of configuration.initialVolumes) broker.authorizeInitialProviderVolume(authorization) await broker.start() const accepted=new Set() diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 88508dfab..97cd17c6e 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -85,6 +85,7 @@ function preparedMountsMatch(actual: unknown[], input: PreparedInputAuthorizatio export type BrokerReceipt = { containerId: string + installationId?: string createdAt: string requestDigest: string incarnationId: string @@ -200,6 +201,7 @@ export class RestrictedDockerBroker { private readonly events: BrokerEvent[] = [] private server?: http.Server private stopFailuresRemaining = 0 + private inputPublicationBeforeCreateFailure = false constructor(private readonly policy: RestrictedDockerBrokerPolicy) {} @@ -344,6 +346,8 @@ export class RestrictedDockerBroker { return this.events.filter((event) => event.destructive && event.containerId === containerId) } + failNextInputPublicationBeforeCreate(): void { this.inputPublicationBeforeCreateFailure = true } + failNextStop(count = 1): void { this.stopFailuresRemaining = Math.max(0, count) } @@ -397,6 +401,40 @@ export class RestrictedDockerBroker { return } + // Cold publisher inspection is confined to one accepted generation or + // one installation/soul's indexed labels. Never expose an unfiltered list. + if (method === 'GET' && body.length === 0) { + const parsed = new URL(url, 'http://docker') + let filters: Record = {} + try { const value = JSON.parse(parsed.searchParams.get('filters') ?? '{}'); if (value && typeof value === 'object' && !Array.isArray(value)) filters = value } catch {} + const keys = [...parsed.searchParams.keys()] + const users = parsed.pathname === `${DOCKER_API_PREFIX}/containers/json` + && keys.length === 2 && parsed.searchParams.getAll('filters').length === 1 && parsed.searchParams.getAll('all').length === 1 && parsed.searchParams.get('all') === '1' + && Object.keys(filters).length === 1 && Array.isArray(filters.volume) && filters.volume.length === 1 + && [...this.preparedInputs.values()].some(input => input.indexedFiles?.volumeName === (filters.volume as unknown[])[0]) + const inventory = parsed.pathname === `${DOCKER_API_PREFIX}/volumes` + && keys.length === 1 && parsed.searchParams.getAll('filters').length === 1 && Object.keys(filters).length === 1 + && Array.isArray(filters.label) && filters.label.length === 3 && filters.label.every(value => typeof value === 'string') + && [...this.preparedInputs.values()].some(input => JSON.stringify([...filters.label as string[]].sort()) === JSON.stringify([ + 'com.freshell.managed-inputs=true', `com.freshell.installation-id=${input.installationId}`, `com.freshell.soul-id=${input.soulId}`, + ].sort())) + if (users || inventory) { await this.forwardAndReply(request,response,body,{destructive:false}); return } + } + const retiring = url.match(/^\/v1\.47\/containers\/([a-f0-9]{64})\?force=0$/)?.[1] + const prior = retiring && this.receiptsById.get(retiring) + if (method === 'DELETE' && prior && body.length === 0) { + const inspected = await this.forward({ method:'GET',url:`${DOCKER_API_PREFIX}/containers/${retiring}/json`,headers:{} } as IncomingMessage,Buffer.alloc(0)) + let actual: Record = {};try {actual=JSON.parse(inspected.body.toString())} catch {} + const labels = {project:'freshell','com.freshell.managed':'true','com.freshell.installation-id':prior.installationId, + 'com.freshell.soul-id':prior.soulId,'com.freshell.incarnation-id':prior.incarnationId,'com.freshell.runtime-test-run-id':this.policy.testRunId} + if (inspected.statusCode !== 404 && (inspected.statusCode !== 200 || actual.Id !== retiring || actual.Config?.Image !== prior.imageRef + || !ownedHelperLabels(actual.Config?.Labels,labels) || actual.State?.Running !== false || actual.State?.Paused === true + || actual.State?.Restarting === true || !['created','exited','dead'].includes(actual.State?.Status))) { + this.block(response,method,url,403,'prior indexed generation workload is live or ownership changed',true);return + } + await this.forwardAndReply(request,response,body,{destructive:true,containerId:retiring});return + } + const volume = url.match(/^\/v1\.47\/volumes\/(freshell-provider-[a-f0-9]{24})$/)?.[1] if (method === 'GET' && volume && (this.initialProviderVolumes.has(volume) || this.receipts().some((receipt) => receipt.providerVolumeName === volume))) { @@ -555,7 +593,9 @@ export class RestrictedDockerBroker { || (method==='POST' && exactId && owned.expected.Cmd?.[0]==='prepare-provider-mountpoints' && url===`${DOCKER_API_PREFIX}/containers/${owned.containerId}/wait?condition=not-running`) || (method==='DELETE' && url===`${DOCKER_API_PREFIX}/containers/${helperTarget![1]}?force=1`)) { - await this.forwardAndReply(request,response,body,{destructive:method!=='GET',containerId:owned.containerId,helperName:name}) + const forwarded=await this.forward(request,body) + this.record({method,url,decision:'forward',destructive:method!=='GET',unsafeAttempt:false,containerId:owned.containerId,helperName:name}) + replyDocker(response,forwarded) return } } @@ -615,7 +655,13 @@ export class RestrictedDockerBroker { } if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name) || mountpointHelper) { const input=mountpointHelper ? this.validateMountpointHelper(body) : this.validateInputPublisher(name,body) - if (!input || this.inputPublishers.has(name)) { + let previousAbsent=true + if (input && this.inputPublishers.has(name)) { + const previous=await this.forward({method:'GET',url:`${DOCKER_API_PREFIX}/containers/${name}/json`,headers:{}} as IncomingMessage,Buffer.alloc(0)) + previousAbsent=previous.statusCode===404 + this.record({method:'GET',url:`${DOCKER_API_PREFIX}/containers/${name}/json`,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name}) + } + if (!input || !previousAbsent) { this.block(response,method,url,403,'indexed publisher must match an owned fixed preparation',false) return } @@ -672,6 +718,11 @@ export class RestrictedDockerBroker { this.block(response, method, url, 403, validation.reason, true, undefined) return } + if (this.inputPublicationBeforeCreateFailure && [...this.inputPublishers.values()].some(publisher => publisher.input.incarnationId === validation.incarnationId && publisher.expected.Cmd?.[0] === 'publish-provider-inputs')) { + this.inputPublicationBeforeCreateFailure = false + this.record({method,url,decision:'inject_failure',reason:'indexed population before main create',destructive:false,unsafeAttempt:false}) + response.statusCode=503;response.end(JSON.stringify({message:'closed indexed publication before main create'}));return + } const forwarded = await this.forward(request, body) if (forwarded.statusCode === 201) { const parsed = JSON.parse(forwarded.body.toString('utf8')) as { Id?: unknown } @@ -683,6 +734,7 @@ export class RestrictedDockerBroker { this.knownContainerIds.add(containerId) this.receiptsById.set(containerId, { containerId, + installationId: validation.installationId, createdAt: new Date().toISOString(), requestDigest: sha256(body), incarnationId: validation.incarnationId, @@ -731,7 +783,7 @@ export class RestrictedDockerBroker { } private validateCreate(body: Buffer): - | { ok: true; incarnationId: string; soulId: string; imageRef: string; runtimeDir: string; hostBinaryPath: string; providerVolumeName?: string; workspacePath?: string } + | { ok: true; installationId: string; incarnationId: string; soulId: string; imageRef: string; runtimeDir: string; hostBinaryPath: string; providerVolumeName?: string; workspacePath?: string } | { ok: false; reason: string } { let parsed: Record try { @@ -870,7 +922,7 @@ export class RestrictedDockerBroker { && input.soulId === soulId && input.incarnationId === incarnationId && input.imageRef === imageRef && input.hostBinaryPath === hostBinaryPath && input.providerVolumeName === providerVolumeName); if ((host.Mounts?.length || prepared.length) && !prepared.some((input)=>preparedMountsMatch(host.Mounts ?? [],input))) return { ok: false, reason: 'typed mounts differ from this attempt owned prepared input set' } - return { ok: true, incarnationId, soulId, imageRef, runtimeDir, hostBinaryPath, ...(providerVolumeName ? { providerVolumeName } : {}), ...(workspacePath ? { workspacePath } : {}) } + return { ok: true, installationId, incarnationId, soulId, imageRef, runtimeDir, hostBinaryPath, ...(providerVolumeName ? { providerVolumeName } : {}), ...(workspacePath ? { workspacePath } : {}) } } private validateClaudeCandidateHelper(body: Buffer, role: 'reader' | 'mountpoints'): boolean { diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index f3992962a..1ee8394f0 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -3,7 +3,7 @@ import os from 'node:os' import fs from 'node:fs/promises' import http from 'node:http' import path from 'node:path' -import { describe, expect, it } from 'vitest' +import { describe, expect, it, vi } from 'vitest' import { RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' @@ -223,12 +223,13 @@ describe('managed provider host gateway', () => { const validate = (body: Buffer): { ok: boolean } => ( broker() as unknown as { validateCreate(body: Buffer): { ok: boolean } } ).validateCreate(body) - const address = Object.values(os.networkInterfaces()).flatMap((interfaces) => interfaces ?? []) - .find((network) => network.family === 'IPv4' && !network.internal)?.address - expect(address).toBeDefined() + const address = '192.0.2.10' + const interfaces = vi.spyOn(os,'networkInterfaces').mockReturnValue({fixture:[{address,netmask:'255.255.255.0',family:'IPv4',mac:'00:00:00:00:00:00',internal:false,cidr:address+'/24'}]}) + try { expect(validate(createBody([...ordinaryBinds, actorBind], undefined, [`host.docker.internal:${address}`])).ok).toBe(true) expect(validate(createBody([...ordinaryBinds, actorBind], undefined, ['host.docker.internal:203.0.113.7'])).ok).toBe(false) + } finally { interfaces.mockRestore() } }) }) @@ -664,6 +665,7 @@ describe('receipt-owned auth exchange HTTP routes', () => { type ForwardedRequest = { method: string; url: string; body: any } type BrokerHttpFixture = { + setRuntimeRunning:(running:boolean)=>void request(method: string, url: string, body?: unknown): Promise<{ status: number; body: string }> forwarded: ForwardedRequest[] broker: RestrictedDockerBroker @@ -768,6 +770,51 @@ describe('fixed indexed publisher broker',()=>{ expect(forwarded.filter((request)=>request.method==='DELETE')).toEqual([]); }); }); + it('retires only the exact stopped receipt and never forces a live or foreign workload',async()=>{ + await withHttpBroker(async({request,forwarded,setRuntimeRunning})=>{ + expect((await request('POST','/v1.47/containers/create?name=owned-runtime',createBody([...ordinaryBinds,actorBind]))).status).toBe(201); + const before=forwarded.length; + expect((await request('DELETE',`/v1.47/containers/${ownedId}?force=1`)).status).toBe(403); + expect((await request('DELETE',`/v1.47/containers/${'f'.repeat(64)}?force=0`)).status).toBe(403); + expect(forwarded).toHaveLength(before); + setRuntimeRunning(true); + expect((await request('DELETE',`/v1.47/containers/${ownedId}?force=0`)).status).toBe(403); + expect(forwarded.filter(value=>value.method==='DELETE')).toEqual([]); + setRuntimeRunning(false); + expect((await request('DELETE',`/v1.47/containers/${ownedId}?force=0`)).status).toBe(204); + expect(forwarded.filter(value=>value.method==='DELETE').map(value=>value.url)).toEqual([`/v1.47/containers/${ownedId}?force=0`]); + }); + }); + it('fails once at the indexed population boundary before forwarding any workload creation',async()=>{ + await withHttpBroker(async({request,forwarded,broker:instance})=>{ + const input=publisherAuthorization();instance.authorizePreparedInputs(input); + expect((await request('POST','/v1.47/volumes/create',{Name:input.indexedFiles!.volumeName,Driver:'local',DriverOpts:indexedOptions,Labels:publisherLabels(input)})).status).toBe(201); + expect((await request('POST',`/v1.47/containers/create?name=${input.indexedFiles!.volumeName}-publisher`,publisherBody(input))).status).toBe(201); + const body=JSON.parse(createBody([...ordinaryBinds,actorBind]).toString()); + body.HostConfig.Mounts=[{Type:'volume',Source:input.indexedFiles!.volumeName,Target:'/workspace/input',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'files/source-0'}},{Type:'volume',Source:input.indexedFiles!.volumeName,Target:'/run/freshell-secrets/provider-0',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'files/source-0'}}]; + instance.failNextInputPublicationBeforeCreate();const before=forwarded.length; + expect((await request('POST','/v1.47/containers/create?name=owned-runtime',body)).status).toBe(503);expect(forwarded).toHaveLength(before); + expect((await request('POST','/v1.47/containers/create?name=owned-runtime-retry',body)).status).toBe(201); + expect((await request('DELETE',`/v1.47/containers/${helperId}?force=1`)).status).toBe(204); + expect((await request('GET',`/v1.47/containers/${helperId}/json`)).status).toBe(404); + expect((await request('POST',`/v1.47/containers/create?name=${input.indexedFiles!.volumeName}-publisher`,publisherBody(input))).status).toBe(201); + expect(instance.eventsSnapshot().filter(value=>value.decision==='inject_failure')).toHaveLength(1); + }); + }); + it('allows only exact generation users and soul-scoped indexed inventories for cold reconciliation',async()=>{ + await withHttpBroker(async({request,forwarded,broker:instance})=>{ + const input=publisherAuthorization();instance.authorizePreparedInputs(input); + const users=(volume:string)=>`/v1.47/containers/json?all=1&filters=${encodeURIComponent(JSON.stringify({volume:[volume]}))}`; + const inventory=(soul:string)=>`/v1.47/volumes?filters=${encodeURIComponent(JSON.stringify({label:['com.freshell.managed-inputs=true',`com.freshell.installation-id=${input.installationId}`,`com.freshell.soul-id=${soul}`]}))}`; + const before=forwarded.length; + for(const url of ['/v1.47/containers/json?all=1','/v1.47/volumes',users('freshell-inputs-ffffffffffffffffffffffff'),inventory('foreign'),users(input.indexedFiles!.volumeName)+'&limit=100',inventory(input.soulId)+'&filters=%7B%7D']) { + expect((await request('GET',url)).status).toBe(403);expect(forwarded).toHaveLength(before); + } + expect((await request('GET',users(input.indexedFiles!.volumeName))).status).toBe(200); + expect((await request('GET',inventory(input.soulId))).status).toBe(200); + expect(forwarded.slice(before).map(value=>value.url)).toEqual([users(input.indexedFiles!.volumeName),inventory(input.soulId)]); + }); + }); it('limits tmpfs and publisher operations to the exact preparation and refuses extra host access',async()=>{ await withHttpBroker(async({request,forwarded,broker:instance})=>{ const input=publisherAuthorization(); @@ -814,6 +861,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise const proxy = path.join(dir, 'broker.sock') const forwarded: ForwardedRequest[] = [] let loseAck = false + let runtimeRunning=false let helperAckId = helperId let indexedVolumeMetadata:unknown let foreignHelper=false @@ -842,6 +890,10 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise if (req.method==='GET' && req.url?.startsWith('/v1.47/volumes/freshell-inputs-') && indexedVolumeMetadata) { res.statusCode=200;res.end(JSON.stringify(indexedVolumeMetadata));return; } + if (req.method==='GET' && req.url===`/v1.47/containers/${ownedId}/json`) { + const creation=forwarded.findLast(value=>value.url.includes('/containers/create?') && !value.body?.Entrypoint)!; + res.statusCode=200;res.end(JSON.stringify({Id:ownedId,Config:creation.body,HostConfig:creation.body.HostConfig,State:{Running:runtimeRunning,Status:runtimeRunning?'running':'exited',Paused:false,Restarting:false}}));return; + } const candidateCreation = forwarded.findLast((operation) => operation.url.includes('/containers/create?') && operation.body?.Labels?.['com.freshell.claude-candidate']==='ordinary') if (req.method==='GET' && candidateCreation && (req.url===`/v1.47/containers/${helperId}/json` || req.url===`${candidateCreation.url.replace('/containers/create?name=', '/containers/')}/json`)) { @@ -852,9 +904,9 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise res.statusCode=200;res.end(JSON.stringify({Id:helperId,Config:foreignHelper?{...inspected,User:'foreign'}:inspected,HostConfig:inspected.HostConfig}));return } if (req.method==='GET' && (req.url?.includes('-publisher/json') || req.url?.includes('/freshell-claude-source-') - || (req.url===`/v1.47/containers/${helperId}/json` && forwarded.some((operation)=>operation.body?.Cmd?.[0]==='observe-claude-sources-private')))) { + || (req.url===`/v1.47/containers/${helperId}/json` && forwarded.some((operation)=>['observe-claude-sources-private','publish-provider-inputs'].includes(operation.body?.Cmd?.[0]))))) { if (sourceHelperDeleted) { res.statusCode=404;res.end('{}');return } - const command=req.url?.includes('/freshell-claude-source-') || req.url===`/v1.47/containers/${helperId}/json`?'observe-claude-sources-private':'publish-provider-inputs'; + const command=req.url?.includes('-publisher/json') || (req.url===`/v1.47/containers/${helperId}/json` && forwarded.some(operation=>operation.body?.Cmd?.[0]==='publish-provider-inputs'))?'publish-provider-inputs':'observe-claude-sources-private'; const config=forwarded.findLast((request)=>request.url.includes('/containers/create?') && request.body?.Cmd?.[0]===command)!.body; const inspected=structuredClone(config); inspected.Labels['org.opencontainers.image.version']='24.04'; @@ -864,6 +916,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise res.statusCode=200;res.end(JSON.stringify({Id:helperId,Config:foreignHelper?{...inspected,User:'0:0'}:inspected,HostConfig:inspected.HostConfig}));return; } if (req.url?.startsWith('/v1.47/containers/create?')) { + if(body.Cmd?.[0]==='publish-provider-inputs') sourceHelperDeleted=false if (body.Entrypoint?.[0] === '/runtime/freshell-session-host' && loseAck) { req.socket.destroy() return @@ -907,7 +960,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise try { await new Promise((resolve) => upstream.listen(socket, resolve)) await instance.start() - await run({ request, forwarded, broker: instance, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true}, + await run({ request, forwarded, broker: instance, setRuntimeRunning:(running)=>{runtimeRunning=running}, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true}, loseCandidateVolumeAck:()=>{candidateVolumeAckUnavailable=true},replaceCandidateVolumeInspection:()=>{candidateVolumeMetadata.Labels['com.freshell.soul-id']='foreign'} }) } finally { await instance.close() From b8e25d2486e97f5b33d10879dc870454ccd1df10 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 04:35:16 -0700 Subject: [PATCH 179/493] fix: release only provider preparations with indexed files --- crates/freshell-supervisor/src/backend.rs | 5 +- .../src/backend/input_publisher.rs | 46 +++++++++++++++++++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 3d52e7336..7bdac280d 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -636,7 +636,10 @@ impl RuntimeBackend for DockerEngineBackend { .fresh_agent() .and_then(|spec| spec.provider_launch_context.as_ref()) }); - if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { + if let Some(inputs) = context + .and_then(|context| context.prepared_inputs.as_ref()) + .filter(|inputs| inputs.indexed_files.is_some()) + { self.release_indexed_publication( handle.soul_id(), handle.incarnation_id(), diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index 90a0b6254..e1599d51f 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -1114,6 +1114,52 @@ impl DockerEngineBackend { mod tests { use super::*; + #[tokio::test] + async fn provider_input_publisher_commit_without_indexed_descriptor_needs_no_daemon() { + let owned = tempfile::tempdir().unwrap(); + let terminal: TerminalLaunchSpec = serde_json::from_value(json!({ + "terminalId":"terminal-nonindexed","streamId":"stream-nonindexed","mode":"codex","program":"codex", + "cwd":"/fixture/workspace","workspacePath":"/fixture/workspace","runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"project-nonindexed", + "providerLaunchContext":{ + "preparation":{"codex":{"tui_args":[],"sidecar_args":[]}}, + "mcpCapability":{"grantId":"grant-nonindexed","endpoint":"http://host.docker.internal:3344","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":"grant-nonindexed","manifest":{"relativePath":".freshell/input-manifest.json","sha256":"a".repeat(64),"sourceCount":0,"aliasCount":0,"recipeCount":0}} + } + })).unwrap(); + terminal.validate().unwrap(); + let handle = crate::registry::OwnedRuntimeHandle::from_registry( + InstallationId::parse("installation-nonindexed").unwrap(), + SoulId::parse("soul-nonindexed").unwrap(), + IncarnationId::parse("incarnation-nonindexed").unwrap(), + freshell_runtime_protocol::LaunchNonce::new(), + DockerDaemonId::parse("daemon-nonindexed").unwrap(), + "c".repeat(64), + format!("sha256:{}", "a".repeat(64)), + owned.path().join("runtime"), + owned.path().join("session-host"), + "d".repeat(64), + RuntimeLimits { + cpu_milli: 500, + memory_bytes: 128 * 1024 * 1024, + swap_bytes: 0, + pids_max: 64, + }, + None, + Some(terminal), + None, + "freshell-provider-nonindexed".into(), + ); + // There is intentionally no daemon/socket: ordinary prepared inputs + // never created an indexed publisher, so their commit must be inert. + let backend = DockerEngineBackend::new(owned.path().join("absent-daemon.sock")); + let committed = backend.commit_prepared_inputs(&handle).await; + drop(backend); + assert!( + committed.is_ok(), + "nonindexed commit must not inspect or retire a publisher: {committed:?}" + ); + } + #[tokio::test] async fn provider_input_publisher_rechecks_actual_capacity_before_payload_read() { use std::os::unix::fs::MetadataExt; From 5fee96d20bddf6a434d9edbf54eb1a50c513abd6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 05:06:51 -0700 Subject: [PATCH 180/493] test: share bounded execution for copied provider scripts --- .../src/managed_mcp_capability.rs | 42 +++++++++++++++---- .../managed_mcp_capability/claude_stage.rs | 21 +--------- 2 files changed, 36 insertions(+), 27 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index afd050a92..940f7f627 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -3243,6 +3243,25 @@ pub(crate) async fn scoped_auth_middleware( next.run(request).await } +#[cfg(all(test, unix))] +fn execute_copied_script(path: &Path) -> std::io::Result { + // A parallel fork can briefly inherit a writer before close-on-exec. + // ETXTBSY creates no child, so retry only that transient spawn error; + // still execute the actual copied script and fail every other error. + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); + loop { + match std::process::Command::new(path).output() { + Err(error) + if error.raw_os_error() == Some(libc::ETXTBSY) + && std::time::Instant::now() < deadline => + { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + result => return result, + } + } +} + #[cfg(test)] mod tests { use super::*; @@ -4164,8 +4183,7 @@ except Exception as error: .unwrap(); assert!(plugin.status.success(), "dependency lookup: {plugin:?}"); assert_eq!(plugin.stdout, b"dependency-nonce\n"); - let script = Command::new(home.join("scripts/run")) - .output() + let script = execute_copied_script(&home.join("scripts/run")) .expect("two-stage executable must retain execute permission"); assert!(script.status.success(), "{script:?}"); assert_eq!(script.stdout, b"resource-nonce\n"); @@ -4174,7 +4192,7 @@ except Exception as error: #[cfg(unix)] #[test] fn managed_provider_two_stage_script_retains_execute_permission() { - use std::{os::unix::fs::PermissionsExt, process::Command}; + use std::os::unix::fs::PermissionsExt; let root = executable_fixture_directory(); let source = root.path().join("source"); std::fs::create_dir(&source).unwrap(); @@ -4185,11 +4203,19 @@ except Exception as error: let home = root.path().join("home"); copy_provider_source(&source, &source, &stage, 0, &mut 0).unwrap(); copy_provider_source(&stage, &stage, &home, 0, &mut 0).unwrap(); - assert!(Command::new(home.join("run")) - .output() - .expect("two-stage script spawn") - .status - .success()); + let writer = std::fs::OpenOptions::new() + .write(true) + .open(home.join("run")) + .unwrap(); + let release = std::thread::spawn(move || { + std::thread::sleep(std::time::Duration::from_millis(50)); + drop(writer); + }); + let result = execute_copied_script(&home.join("run")); + release.join().unwrap(); + let script = result.expect("two-stage script spawn after exact transient writer closes"); + assert!(script.status.success()); + assert_eq!(script.stdout, b"resource-nonce"); } #[cfg(unix)] diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs index 2d0765459..656029c5a 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_stage.rs @@ -770,31 +770,14 @@ fn optional(path: &Path) -> Result { #[cfg(test)] mod tests { + #[cfg(unix)] + use super::super::execute_copied_script; use super::*; use freshell_platform::{ claude_inputs::ClaudeSettingSource, provider_environment::{copy_environment, CopyLimits}, }; - #[cfg(unix)] - fn execute_copied_script(path: &Path) -> std::io::Result { - // A parallel fork can briefly inherit a writer before close-on-exec. - // ETXTBSY creates no child, so retry only that transient spawn error; - // still execute the actual copied script and fail every other error. - let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); - loop { - match std::process::Command::new(path).output() { - Err(error) - if error.raw_os_error() == Some(libc::ETXTBSY) - && std::time::Instant::now() < deadline => - { - std::thread::sleep(std::time::Duration::from_millis(5)); - } - result => return result, - } - } - } - #[cfg(unix)] #[test] fn managed_provider_claude_copied_script_exec_waits_only_for_transient_owned_writer() { From 2ec752f0d17aa475e15daf2afa62b849ef46d174 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:10:54 -0700 Subject: [PATCH 181/493] fix: transport configured external OpenCode skill directories --- .../managed_mcp_capability/source_stage.rs | 138 +++++++++++++++++ .../src/managed_provider_launch_tests.rs | 45 +++++- .../opencode_resources.rs | 145 ++++++++++++------ 3 files changed, 279 insertions(+), 49 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 6c02ac627..f2152b0ce 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -118,6 +118,144 @@ mod configured_resource_tests { assert_eq!(plan[0].references.path_variables, vec!["RULE_PATTERN"]); assert_eq!(plan[0].references.unexpanded_source(), raw); } + #[test] + fn managed_provider_configured_skill_paths_execute_whole_directory_after_both_copies() { + use std::os::unix::fs::PermissionsExt; + let owned = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::tempdir_in(root).unwrap(), + None => tempfile::tempdir().unwrap(), + }; + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace/nested"); + let skill_root = owned.path().join("external-skills"); + let skill = skill_root.join("probe"); + for path in [&home, &cwd, &skill.join("scripts/node_modules/dependency")] { + std::fs::create_dir_all(path).unwrap(); + } + let skill_document = "---\nname: external-probe\ndescription: external selected skill\n---\nRun the adjacent executable.\n"; + std::fs::write(skill.join("SKILL.md"), skill_document).unwrap(); + std::fs::write(skill.join("scripts/resource"), "skill-resource").unwrap(); + std::fs::write( + skill.join("scripts/node_modules/dependency/index.js"), + "module.exports='skill-dependency';", + ) + .unwrap(); + std::fs::write(skill.join("scripts/run"), "#!/usr/bin/env node\nconst fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('dependency'));").unwrap(); + std::fs::set_permissions( + skill.join("scripts/run"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + let home_skill_root = home.join("selected-skills"); + std::os::unix::fs::symlink(&skill_root, &home_skill_root).unwrap(); + let config = home.join("opencode.jsonc"); + for spelling in [ + skill_root.to_str().unwrap(), + "../../external-skills", + "~/selected-skills", + ] { + let raw = format!("{{ // native selected field\n\"skills\":{{\"paths\":[{}],\"urls\":[\"https://example.invalid/skills\"]}},\"unrelated\":{{\"skills\":{{\"paths\":[\"/missing-unselected\"]}}}}}}", serde_json::to_string(spelling).unwrap()); + std::fs::write(&config, &raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Home, + }], + &cwd, + &home, + &|_, _, _| None, + ) + .unwrap(); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &owned.path().join("workspace"), + &plan, + None, + ) + .unwrap(); + let unit = units.iter().find(|(source, _)| source == &skill_root) + .unwrap_or_else(|| panic!("configured external skills.paths selects its whole directory for {spelling}; selected {:?}", plan[0].configured_resources)); + let stage = owned.path().join("controller-stage"); + let host = owned.path().join("provider-home"); + copy_environment( + &[CopySelection { + source: unit.0.clone(), + destination_relative: unit.1.stage_relative_path.clone().into(), + }], + &stage, + &[], + Default::default(), + ) + .unwrap(); + copy_environment( + &[CopySelection { + source: stage.join(&unit.1.stage_relative_path), + destination_relative: unit.1.provider_relative_path.clone().into(), + }], + &host, + &[], + Default::default(), + ) + .unwrap(); + let projected = host.join(&unit.1.provider_relative_path).join("probe"); + let hidden = owned.path().join("hidden-original-skills"); + std::fs::rename(&skill_root, &hidden).unwrap(); + let output = std::process::Command::new(projected.join("scripts/run")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, &skill_root).unwrap(); + assert!(output.status.success(), "copied selected skill executable must consume installed dependencies and adjacent resources"); + assert_eq!(output.stdout, b"skill-resource:skill-dependency"); + assert_eq!( + std::fs::read_to_string(projected.join("SKILL.md")).unwrap(), + skill_document + ); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + } + } + + #[test] + fn managed_provider_configured_skill_paths_reject_unavailable_or_non_directory_selection() { + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let config = owned.path().join("opencode.json"); + let missing = owned.path().join("missing"); + let file = owned.path().join("file"); + let unreadable = owned.path().join("unreadable"); + let cyclic = owned.path().join("cyclic"); + std::fs::write(&file, "not a directory").unwrap(); + std::fs::create_dir(&unreadable).unwrap(); + std::fs::set_permissions(&unreadable, std::fs::Permissions::from_mode(0)).unwrap(); + std::os::unix::fs::symlink(&cyclic, &cyclic).unwrap(); + for selected in [&missing, &file, &unreadable, &cyclic] { + let raw = serde_json::json!({"skills":{"paths":[selected],"urls":["https://example.invalid/skills"]}}).to_string(); + std::fs::write(&config, &raw).unwrap(); + let result = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + owned.path(), + owned.path(), + &|_, _, _| None, + ); + let error = result + .err() + .expect("selected skill directory must be available before publication"); + assert_eq!(error.field, "OpenCode skills.paths"); + assert_eq!(error.declaring_source, config); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + } + std::fs::set_permissions(&unreadable, std::fs::Permissions::from_mode(0o700)).unwrap(); + } + #[test] fn managed_provider_configured_external_resources_execute_complete_transport() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 88e48f34d..098191c1e 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3464,12 +3464,40 @@ export default {{id:'closed-shared-{label}',async tui(api,options){{ std::fs::write(package.join("resource.txt"), "native-adjacent-resource").unwrap(); let external_instruction = external.join("ordinary-instructions.md"); std::fs::write(&external_instruction, "native-external-instruction").unwrap(); + let selected_skills = external.join("selected-skills"); + let selected_skill = selected_skills.join("probe"); + std::fs::create_dir_all(selected_skill.join("scripts/node_modules/skill-dependency")).unwrap(); + std::fs::write(selected_skill.join("SKILL.md"), "---\nname: closed-external-skill\ndescription: selected external skill\n---\nExecute the adjacent synthetic resources.\n").unwrap(); + std::fs::write( + selected_skill.join("scripts/resource"), + "native-skill-resource", + ) + .unwrap(); + std::fs::write( + selected_skill.join("scripts/node_modules/skill-dependency/index.js"), + "module.exports='native-skill-dependency';", + ) + .unwrap(); + std::fs::write(selected_skill.join("scripts/run"), "#!/usr/bin/env node\nconst fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('skill-dependency'));\n").unwrap(); + std::fs::set_permissions( + selected_skill.join("scripts/run"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); let external_receipt = fixture.join("workspace/native-external-consumed.json"); - let external_plugin = r#"import fs from 'node:fs';import crypto from 'node:crypto'; + let external_plugin = r#"import fs from 'node:fs';import crypto from 'node:crypto';import path from 'node:path';import {execFileSync} from 'node:child_process'; import adjacent from './adjacent.mjs';import child from 'closed-child';import parent from 'closed-parent'; -export default async (_input,options)=>({config:async(config)=>{ +export default async (input,options)=>({config:async(config)=>{ const digest=value=>crypto.createHash('sha256').update(value).digest('hex'); - fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction)})); + setTimeout(async()=>{ + try { + const response=await input.client.app.skills(); + const skill=response.data.find(skill=>skill.name==='closed-external-skill'); + if(!skill)throw Error('selected native skill unavailable'); + const output=execFileSync(path.join(path.dirname(skill.location),'scripts/run')); + fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction),nativeSkill:true,skillContent:digest(skill.content),skillExecution:digest(output)})); + } catch(error) {fs.writeFileSync(options.receipt+'.failure',JSON.stringify({operation:'native_skill_consumer',type:error instanceof TypeError?'TypeError':error.code==='ENOENT'?'ENOENT':'native_skill_unavailable'}));} + },0); }}); "#; std::fs::write(package.join("index.mjs"), external_plugin).unwrap(); @@ -3477,9 +3505,10 @@ export default async (_input,options)=>({config:async(config)=>{ serde_json::from_slice(&std::fs::read(&original_config).unwrap()).unwrap(); original["plugin"].as_array_mut().unwrap().push(json!([reqwest::Url::from_file_path(package.join("index.mjs")).unwrap().to_string(), {"receipt":external_receipt,"instruction":external_instruction}])); original["instructions"] = json!([external_instruction]); + original["skills"] = json!({"paths":[selected_skills,"../../scenarios/external-native-resources/selected-skills"]}); std::fs::write(&original_config, serde_json::to_vec(&original).unwrap()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); - let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true}); + let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency"))}); let explicit_config = fixture.join("workspace/selected-config.json"); let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; std::fs::write(&explicit_config, explicit_bytes).unwrap(); @@ -3669,6 +3698,7 @@ export default {{id:'closed-directory-discovery',async server(){{ for (field, selected) in [ ("OpenCode plugin", package.join("index.mjs")), ("OpenCode instructions", external_instruction.clone()), + ("OpenCode skills.paths", selected_skills.clone()), ] { for unreadable in [false, true] { let hidden = selected.with_extension("closed-unavailable"); @@ -3679,8 +3709,11 @@ export default {{id:'closed-directory-discovery',async server(){{ } let failed = controller.launch(request.clone()).await; if unreadable { - std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0o644)) - .unwrap(); + std::fs::set_permissions( + &selected, + std::fs::Permissions::from_mode(if selected.is_dir() { 0o755 } else { 0o644 }), + ) + .unwrap(); } else { std::fs::rename(&hidden, &selected).unwrap(); } diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 35d8d0d38..e545bbcbd 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1,4 +1,4 @@ -//! The two native ordinary resource fields. Keep raw documents and every +//! Finite native ordinary resource fields. Keep raw documents and every //! unrelated string intact; these selections are paths, never private payloads. use super::sources::PlannedSourceOccurrence; use std::path::{Path, PathBuf}; @@ -74,54 +74,58 @@ fn fields(raw: &str) -> Result, &'static str> { Token::String(start, end) if depth == 1 && end - start <= 64 => { let key: String = serde_json::from_str(&raw[start..end]) .map_err(|_| "invalid selected resource field")?; - let field = match key.as_str() { - "plugin" => "OpenCode plugin", - "instructions" => "OpenCode instructions", - _ => continue, - }; - let mut array = Cursor { raw, at: cursor.at }; - if !matches!(array.next(), Some(Token::Punctuation(b':'))) - || !matches!(array.next(), Some(Token::Punctuation(b'['))) - { + let mut selected = Cursor { raw, at: cursor.at }; + if !matches!(selected.next(), Some(Token::Punctuation(b':'))) { continue; } - let mut array_depth = 1usize; - let mut tuple_first = false; - while array_depth != 0 { - match array.next().ok_or("incomplete selected resource field")? { - Token::Punctuation(b'[') => { - array_depth += 1; - tuple_first = array_depth == 2; - } - Token::Punctuation(b'{') => { - array_depth += 1; - tuple_first = false; - } - Token::Punctuation(b']' | b'}') => { - array_depth -= 1; - tuple_first = false; - } - Token::String(start, end) - if array_depth == 1 - || (field == "OpenCode plugin" - && array_depth == 2 - && tuple_first) => - { - tuple_first = false; - if result.len() >= 100_000 || end - start > 64 * 1024 { - return Err("configured resource selection profile exceeded"); + match key.as_str() { + "plugin" | "instructions" + if matches!(selected.next(), Some(Token::Punctuation(b'['))) => + { + let field = if key == "plugin" { + "OpenCode plugin" + } else { + "OpenCode instructions" + }; + resource_array(&mut selected, field, &mut result)?; + cursor = selected; + } + "skills" if matches!(selected.next(), Some(Token::Punctuation(b'{'))) => { + let mut skills_depth = 1usize; + while skills_depth != 0 { + match selected.next().ok_or("incomplete selected skills field")? { + Token::Punctuation(b'{' | b'[') => skills_depth += 1, + Token::Punctuation(b'}' | b']') => skills_depth -= 1, + Token::String(start, end) + if skills_depth == 1 && end - start <= 64 => + { + let key: String = serde_json::from_str(&raw[start..end]) + .map_err(|_| "invalid selected skills field")?; + if key != "paths" { + continue; + } + let mut paths = Cursor { + raw, + at: selected.at, + }; + if matches!(paths.next(), Some(Token::Punctuation(b':'))) + && matches!(paths.next(), Some(Token::Punctuation(b'['))) + { + resource_array( + &mut paths, + "OpenCode skills.paths", + &mut result, + )?; + selected = paths; + } + } + _ => {} } - result.push(( - field, - serde_json::from_str(&raw[start..end]) - .map_err(|_| "invalid selected resource path")?, - )); } - _ if array_depth == 2 => tuple_first = false, - _ => {} + cursor = selected; } + _ => {} } - cursor = array; } _ => {} } @@ -129,6 +133,47 @@ fn fields(raw: &str) -> Result, &'static str> { Ok(result) } +fn resource_array( + cursor: &mut Cursor<'_>, + field: &'static str, + result: &mut Vec<(&'static str, String)>, +) -> Result<(), &'static str> { + let mut depth = 1usize; + let mut tuple_first = false; + while depth != 0 { + match cursor.next().ok_or("incomplete selected resource field")? { + Token::Punctuation(b'[') => { + depth += 1; + tuple_first = depth == 2; + } + Token::Punctuation(b'{') => { + depth += 1; + tuple_first = false; + } + Token::Punctuation(b']' | b'}') => { + depth -= 1; + tuple_first = false; + } + Token::String(start, end) + if depth == 1 || (field == "OpenCode plugin" && depth == 2 && tuple_first) => + { + tuple_first = false; + if result.len() >= 100_000 || end - start > 64 * 1024 { + return Err("configured resource selection profile exceeded"); + } + result.push(( + field, + serde_json::from_str(&cursor.raw[start..end]) + .map_err(|_| "invalid selected resource path")?, + )); + } + _ if depth == 2 => tuple_first = false, + _ => {} + } + } + Ok(()) +} + fn home_path(path: &str, home: &Path) -> PathBuf { if let Some(relative) = path.strip_prefix("~/") { home.join(relative) @@ -303,6 +348,8 @@ pub fn select( .parent() .unwrap_or(&source.instance_cwd) .join(path); + } else if field == "OpenCode skills.paths" { + path = source.instance_cwd.join(path); } else { path = source .instance_cwd @@ -312,6 +359,11 @@ pub fn select( .unwrap_or_else(|| source.instance_cwd.join(path)); } } + if field == "OpenCode skills.paths" { + // Native instance-cwd resolution normalizes .. before deciding + // whether the selected directory is supplied by the workspace. + path = freshell_platform::opencode_config::lexical_path(&path); + } let canonical = checked(&path, field, declaring)?; variables.extend(names.into_inner()); if field == "OpenCode instructions" { @@ -321,6 +373,13 @@ pub fn select( insert(path, field)?; continue; } + if field == "OpenCode skills.paths" { + if !canonical.is_dir() { + return Err(fail(field, "selected skill path is not a directory")); + } + insert(path, field)?; + continue; + } // The native global plugin discovery unit is already selected in the // ordinary global projection, including its installed dependencies. // Do not widen it to the entire config package (private siblings belong From 938f33c76b18512967c6fd04dc33a2c51defeaa1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:14:47 -0700 Subject: [PATCH 182/493] fix: select Amplifier top-level native local bundle paths --- .../amplifier_resources.rs | 203 +++++++++++++++++- .../src/managed_provider_launch_tests.rs | 5 +- 2 files changed, 203 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index eb7a061d8..dbcd68cec 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -55,9 +55,6 @@ pub(super) fn prepare( let Some(source) = active.data.as_str() else { return Err("Amplifier bundle.active source has an unsupported type".into()); }; - if !source.starts_with("file://") { - return Ok(result); - } let mut state = BundleSelection { view_bytes: raw.len(), ..Default::default() @@ -561,6 +558,206 @@ fn markdown_frontmatter(raw: &str) -> Option<(&str, usize)> { mod module_source_tests { use super::*; + #[test] + fn managed_provider_amplifier_top_local_forms_consume_modules_context_and_dependencies_after_both_copies( + ) { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let bundle = owned.path().join("external-bundle"); + let module = bundle.join("modules/probe"); + for path in [ + &home.join(".amplifier"), + &cwd, + &module.join("node_modules/dependency"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + let raw_bundle = "bundle: {name: selected}\ncontext:\n include: [context.md]\nsession:\n orchestrator:\n module: probe\n source: ./modules/probe\n"; + std::fs::write(bundle.join("bundle.yaml"), raw_bundle).unwrap(); + std::fs::write(bundle.join("context.md"), "selected-context").unwrap(); + std::fs::write(module.join("resource"), "module-resource").unwrap(); + std::fs::write( + module.join("node_modules/dependency/index.js"), + "module.exports='installed-dependency';", + ) + .unwrap(); + std::fs::write(module.join("consume.cjs"), "const fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('dependency')+':'+fs.readFileSync(process.argv[2],'utf8'));").unwrap(); + for (ordinal, source) in [ + bundle.join("bundle.yaml").display().to_string(), + "../external-bundle/bundle.yaml".into(), + "./../external-bundle/bundle.yaml".into(), + ] + .iter() + .enumerate() + { + let raw_settings = format!( + "bundle:\n active: {}\n", + serde_json::to_string(source).unwrap() + ); + let settings = home.join(".amplifier/settings.yaml"); + std::fs::write(&settings, &raw_settings).unwrap(); + let prepared = prepare(&home, &cwd, &[]).unwrap(); + let staged_settings = prepared + .rewritten_settings + .as_ref() + .expect("selected top-level local bundle is projected"); + let projected_source = MarkedYamlOwned::load_from_str(staged_settings).unwrap()[0] + .data + .as_mapping_get("bundle") + .unwrap() + .data + .as_mapping_get("active") + .unwrap() + .data + .as_str() + .unwrap() + .to_owned(); + let stage = owned.path().join(format!("stage-{ordinal}")); + let host = owned.path().join(format!("host-{ordinal}")); + freshell_platform::provider_environment::copy_environment( + &prepared.selections, + &stage, + &[], + Default::default(), + ) + .unwrap(); + for (relative, raw) in &prepared.rewritten_resources { + std::fs::write(stage.join(relative), raw).unwrap(); + } + let selections = prepared + .selections + .iter() + .map(|selection| CopySelection { + source: stage.join(&selection.destination_relative), + destination_relative: selection.destination_relative.clone(), + }) + .collect::>(); + freshell_platform::provider_environment::copy_environment( + &selections, + &host, + &[], + Default::default(), + ) + .unwrap(); + let relative = Path::new(projected_source.strip_prefix("file://").unwrap()) + .strip_prefix("/home/freshell/provider") + .unwrap(); + let projected_bundle = host.join(relative); + let projected_raw = std::fs::read_to_string(&projected_bundle).unwrap(); + let document = MarkedYamlOwned::load_from_str(&projected_raw).unwrap(); + let module_source = document[0] + .data + .as_mapping_get("session") + .unwrap() + .data + .as_mapping_get("orchestrator") + .unwrap() + .data + .as_mapping_get("source") + .unwrap() + .data + .as_str() + .unwrap(); + let module = host.join( + Path::new(module_source.strip_prefix("file://").unwrap()) + .strip_prefix("/home/freshell/provider") + .unwrap(), + ); + let context = document[0] + .data + .as_mapping_get("context") + .unwrap() + .data + .as_mapping_get("include") + .unwrap() + .data + .as_sequence() + .unwrap()[0] + .data + .as_str() + .unwrap(); + let hidden = owned.path().join("hidden-original-bundle"); + std::fs::rename(&bundle, &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg(module.join("consume.cjs")) + .arg(projected_bundle.parent().unwrap().join(context)) + .current_dir(&cwd) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, &bundle).unwrap(); + assert!( + output.status.success(), + "selected top bundle modules and context must work with originals unavailable" + ); + assert_eq!( + output.stdout, + b"module-resource:installed-dependency:selected-context" + ); + assert_eq!(std::fs::read_to_string(settings).unwrap(), raw_settings); + assert_eq!( + std::fs::read_to_string(bundle.join("bundle.yaml")).unwrap(), + raw_bundle + ); + } + } + + #[test] + fn managed_provider_amplifier_top_local_forms_block_missing_and_unreadable_sources_and_preserve_native_names( + ) { + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + std::fs::create_dir_all(home.join(".amplifier")).unwrap(); + std::fs::create_dir(&cwd).unwrap(); + let settings = home.join(".amplifier/settings.yaml"); + let selected = owned.path().join("bundle.yaml"); + for source in [ + selected.display().to_string(), + "../bundle.yaml".into(), + "./../bundle.yaml".into(), + ] { + for unreadable in [false, true] { + if unreadable { + std::fs::write(&selected, "bundle: {name: selected}\n").unwrap(); + std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0)) + .unwrap(); + } + let raw = format!( + "bundle:\n active: {}\n", + serde_json::to_string(&source).unwrap() + ); + std::fs::write(&settings, &raw).unwrap(); + let result = prepare(&home, &cwd, &[]); + if unreadable { + std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0o600)) + .unwrap(); + std::fs::remove_file(&selected).unwrap(); + } + let error = result + .err() + .expect("unavailable selected top-level local bundle must block preparation"); + assert!(error.contains(if unreadable { "unreadable" } else { "missing" })); + assert_eq!(std::fs::read_to_string(&settings).unwrap(), raw); + } + } + for name in [ + "registered-bundle", + "git+https://example.invalid/bundle", + "https://example.invalid/bundle", + ] { + let raw = format!("bundle:\n active: {name}\n"); + std::fs::write(&settings, &raw).unwrap(); + let prepared = prepare(&home, &cwd, &[]).unwrap(); + assert!(prepared.selections.is_empty()); + assert!(prepared.rewritten_settings.is_none()); + assert_eq!(std::fs::read_to_string(&settings).unwrap(), raw); + } + } + #[test] fn amplifier_local_module_source_executes_after_projection_from_unrelated_cwd() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 098191c1e..6f36a4cca 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -673,7 +673,7 @@ async fn managed_provider_amplifier_terminal_native_startup_consumes_prepared_in .starts_with("fln-")); let home = fixture.join("source-home"); let workspace = fixture.join("workspace"); - let bundle = home.join(".amplifier/bundles/selected"); + let bundle = fixture.join("scenarios/external-amplifier-bundle"); let skill = home.join(".amplifier/skills/probe"); let control = fixture.join("scenarios/native/control"); let runtime = fixture.join("scenarios/n/r"); @@ -681,6 +681,7 @@ async fn managed_provider_amplifier_terminal_native_startup_consumes_prepared_in for path in [ &control, &runtime, + &home.join(".amplifier"), &bundle.join("modules/context-probe/amplifier_module_context_probe"), &bundle.join("modules/loop-probe/amplifier_module_loop_probe"), &bundle.join("modules/hooks-probe/amplifier_module_hooks_probe"), @@ -781,7 +782,7 @@ async def on_session_ready(coordinator): let conversation_receipt = workspace.join("amplifier-conversation.json"); let bundle_document=format!("bundle:\n name: closed-native-bundle\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); std::fs::write(bundle.join("bundle.yaml"), &bundle_document).unwrap(); - let settings=format!("bundle:\n active: file://{}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n",bundle.display()); + let settings = "bundle:\n active: ../scenarios/external-amplifier-bundle/bundle.yaml\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n".to_owned(); std::fs::write(home.join(".amplifier/settings.yaml"), &settings).unwrap(); let env_file = fixture.join("grants/environment.env"); std::fs::write( From 1912dc662c2bab405e2fff2f263cb277a271b065 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:32:39 -0700 Subject: [PATCH 183/493] fix(supervisor): retry accepted provider preparation completion --- .../src/provider_preparation_tests.rs | 513 ++++++++++++++++++ crates/freshell-supervisor/src/recovery.rs | 36 +- crates/freshell-supervisor/src/registry.rs | 124 ++++- crates/freshell-supervisor/src/service.rs | 79 ++- 4 files changed, 726 insertions(+), 26 deletions(-) create mode 100644 crates/freshell-supervisor/src/provider_preparation_tests.rs diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs new file mode 100644 index 000000000..2f870482c --- /dev/null +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -0,0 +1,513 @@ +//! Owning service/recovery tests. Synthetic daemon and authenticated host IPC +//! retain exact generation ownership across real registry/supervisor reopen. +use super::*; +use crate::backend::*; +use crate::registry::OwnedRuntimeHandle; +use async_trait::async_trait; +use freshell_runtime_protocol::*; +use std::collections::BTreeSet; +use std::sync::{Arc, Mutex}; +use tokio::net::UnixListener; + +#[derive(Default)] +struct Daemon { + generations: BTreeSet, + publishers: BTreeSet, + running: BTreeSet, + events: Vec, + fail: Option<&'static str>, + grants: usize, + host_tasks: Vec>, +} +struct Backend(Arc>, std::path::PathBuf); + +fn generation(handle: &OwnedRuntimeHandle) -> String { + handle + .terminal() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap() + .generation_id + .clone() +} + +fn context(soul: &SoulId, ordinal: usize) -> ProviderLaunchContext { + let generation_id = format!("grant-{ordinal}"); + ProviderLaunchContext { + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + config: vec![], + mcp_capability: Some(McpCapabilityReference { + grant_id: generation_id.clone(), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }), + prepared_inputs: Some(PreparedProviderInputs { + generation_id: generation_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 1, + alias_count: 1, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: Some(IndexedProviderFiles { + volume_name: indexed_provider_volume_name(soul, &generation_id), + aliases: vec![IndexedProviderFileAlias { + source_id: 0, + logical_destination: "/home/freshell/provider/.freshell/references/0".into(), + effective_destination: "/home/freshell/provider/.freshell/references/0".into(), + }], + }), + }), + } +} + +fn references() -> Vec { + vec![ProviderSecretReference { + profile: ProviderSecretProfile::OpencodeConfigurationFile, + source_path: "/synthetic/approved-source".into(), + provider_relative_path: Some(".freshell/references/0".into()), + }] +} + +fn limits() -> RuntimeLimits { + RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + } +} + +#[async_trait] +impl RuntimeBackend for Backend { + async fn create_stopped( + &self, + spec: &CreateRuntimeSpec, + ) -> Result { + let inputs = spec + .terminal + .as_ref() + .unwrap() + .provider_launch_context + .as_ref() + .unwrap() + .prepared_inputs + .as_ref() + .unwrap(); + let mut state = self.0.lock().unwrap(); + if !state.generations.contains(&inputs.generation_id) && state.generations.len() >= 2 { + return Err(BackendError::Unavailable( + "one active and one pending input generation already exist; preserve and reconcile" + .into(), + )); + } + state.generations.insert(inputs.generation_id.clone()); + state.publishers.insert(inputs.generation_id.clone()); + state + .events + .push(format!("create:{}", inputs.generation_id)); + Ok(BackendCreated { + daemon_id: DockerDaemonId::parse("synthetic-daemon").unwrap(), + container_id: format!( + "{:x}", + sha2::Sha256::digest(spec.incarnation_id.as_str().as_bytes()) + ), + immutable_config_digest: "a".repeat(64), + }) + } + async fn start_host(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { + let listener = UnixListener::bind(handle.runtime_dir().join("host.sock")).unwrap(); + let secret = std::fs::read(handle.runtime_dir().join("secret")).unwrap(); + let id = handle.incarnation_id().clone(); + self.0.lock().unwrap().running.insert(id.clone()); + let state = self.0.clone(); + let task = tokio::spawn(async move { + let boot = HostBootId::new(); + loop { + let (mut socket, _) = listener.accept().await.unwrap(); + let envelope: Envelope = read_frame(&mut socket).await.unwrap(); + let result = match envelope.body { + HostCommand::Hello { + challenge, + incarnation_id, + } => HostResult::Hello { + host_boot_id: boot.clone(), + proof: host_proof(&secret, &challenge, &boot, &incarnation_id), + capabilities: vec![], + effective_limits: limits(), + }, + HostCommand::GrantExecution { .. } => { + state.lock().unwrap().grants += 1; + HostResult::GrantAccepted { + host_boot_id: boot.clone(), + worker_pid: 1234, + worker_launch_count: 1, + fixture_evidence: serde_json::json!({}), + native_session_id: Some("ses_exact".into()), + } + } + HostCommand::Status { .. } => HostResult::Status { + host_boot_id: boot.clone(), + worker_pid: Some(1234), + worker_launch_count: 1, + max_control_epoch: 100, + max_execution_generation: 100, + fixture_evidence: serde_json::json!({}), + exited: false, + native_session_id: Some("ses_exact".into()), + }, + HostCommand::ProbeRecovery { + resume_spec: Some(spec), + .. + } => HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { + evidence_revision: spec.evidence_revision, + resume_spec: spec, + }), + command => panic!("unexpected synthetic host command {command:?}"), + }; + write_frame( + &mut socket, + &HostReply { + request_id: envelope.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + }); + self.0.lock().unwrap().host_tasks.push(task); + Ok(()) + } + async fn commit_prepared_inputs( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + let registry_path = self.1.join("runtime.sqlite3"); + let conn = rusqlite::Connection::open(registry_path).unwrap(); + let (running, live): (String, String) = conn.query_row("SELECT i.launch_state,s.recovery_state FROM incarnations i JOIN souls s ON s.soul_id=i.soul_id WHERE i.incarnation_id=?1", [handle.incarnation_id().as_str()], |row| Ok((row.get(0)?, row.get(1)?))).unwrap(); + assert_eq!(running, "running"); + assert_eq!( + live, "live", + "resumed B cannot complete before verified durable recovery success" + ); + let mut state = self.0.lock().unwrap(); + let generation = generation(handle); + state.events.push(format!("release:{generation}")); + if state.fail == Some("publisher") { + state.fail = None; + return Err(BackendError::Unavailable( + "synthetic publisher transient".into(), + )); + } + state.publishers.remove(&generation); + Ok(()) + } + async fn retire_prior_prepared_inputs( + &self, + current: &OwnedRuntimeHandle, + prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + let mut state = self.0.lock().unwrap(); + assert!( + !state.running.contains(prior.incarnation_id()), + "never stop/delete uncertain live prior ownership" + ); + assert!(state.running.contains(current.incarnation_id())); + state.events.push(format!( + "retire:{}:{}", + generation(current), + generation(prior) + )); + if state.fail == Some("retire") { + state.fail = None; + return Err(BackendError::Unavailable( + "synthetic retirement transient".into(), + )); + } + state.generations.remove(&generation(prior)); + state.publishers.remove(&generation(prior)); + Ok(()) + } + async fn inspect(&self, h: &OwnedRuntimeHandle) -> Result { + Ok(BackendInspection { + state: if self.0.lock().unwrap().running.contains(h.incarnation_id()) { + BackendRuntimeState::Running + } else { + BackendRuntimeState::Exited + }, + configured_limits: limits(), + }) + } + async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + panic!("healthy retry must not stop a workload") + } + async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + panic!("healthy retry must not stop a workload") + } + async fn verify_empty(&self, h: &OwnedRuntimeHandle) -> Result { + Ok(!self.0.lock().unwrap().running.contains(h.incarnation_id())) + } + async fn list_known( + &self, + handles: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + let mut result = vec![]; + for h in handles { + result.push((h.incarnation_id().clone(), self.inspect(h).await?)); + } + Ok(result) + } + async fn read_limits(&self, _: &OwnedRuntimeHandle) -> Result { + Ok(limits()) + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + Ok(()) + } +} + +#[tokio::test] +async fn provider_accepted_completion_retries_after_registry_success_on_fresh_healthy_startup() { + for failure in ["controller", "publisher", "retire"] { + let temp = tempfile::tempdir().unwrap(); + let root = temp.path().join("registry"); + let registry = Registry::open(&root, None).unwrap(); + let soul = SoulId::new(); + let daemon = Arc::new(Mutex::new(Daemon::default())); + let callback = UnixListener::bind(temp.path().join("mcp.sock")).unwrap(); + let callback_daemon = daemon.clone(); + let callback_soul = soul.clone(); + let callback_task = tokio::spawn(async move { + let mut next = 1; + loop { + let (mut socket, _) = callback.accept().await.unwrap(); + let mut raw = vec![]; + socket.read_to_end(&mut raw).await.unwrap(); + let request: serde_json::Value = serde_json::from_slice(&raw).unwrap(); + let response = if request["action"] == "issue" { + next += 1; + serde_json::to_vec(&PreparedProviderContext { + provider_launch_context: context(&callback_soul, next), + provider_secret_references: references(), + }) + .unwrap() + } else { + let mut state = callback_daemon.lock().unwrap(); + state.events.push(format!( + "{}:{}", + request["action"].as_str().unwrap(), + request["grantId"].as_str().unwrap() + )); + if request["action"] == "commit" && state.fail == Some("controller") { + state.fail = None; + b"failed".to_vec() + } else { + b"ok".to_vec() + } + }; + socket.write_all(&response).await.unwrap(); + } + }); + let config = SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: temp.path().join("synthetic-host"), + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "provider-completion".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }; + std::fs::write(&config.host_binary_path, "synthetic-host-artifact").unwrap(); + let supervisor = Supervisor::new( + registry.clone(), + Arc::new(Backend(daemon.clone(), root.clone())), + config.clone(), + ) + .unwrap(); + let terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({"terminalId":"terminal","streamId":"stream","mode":"opencode","program":"opencode","args":[],"env":{"HOME":"/home/freshell/provider"},"cwd":temp.path(),"runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"project","workspacePath":temp.path(),"providerLaunchContext":context(&soul,1),"providerSecretReferences":references()})).unwrap(); + let preparation = LaunchPreparation { + soul_id: soul.clone(), + provider: "opencode".into(), + provider_store_id: "store-exact".into(), + native_session_id: None, + creation_seed_ref: "seed".into(), + request_id: RequestId::new(), + payload_digest: "test-digest".into(), + requested_limits: limits(), + profile: RuntimeProfile::Custom, + project_key: "project".into(), + fixture: None, + terminal: Some(terminal.clone()), + fresh_agent: None, + view_intent: None, + admission: crate::admission::AdmissionPolicy::default(), + }; + let prepared = registry.prepare_launch(preparation.clone()).await.unwrap(); + daemon.lock().unwrap().fail = Some("publisher"); + assert!(supervisor + .activate_prepared( + prepared.clone(), + LaunchWorkload { + soul_id: soul.clone(), + fixture: None, + terminal: Some(terminal.clone()), + fresh_agent: None, + resume_spec: None + }, + limits(), + ) + .await + .is_err()); + assert_eq!(daemon.lock().unwrap().grants, 1); + assert!(!daemon.lock().unwrap().publishers.is_empty()); + let retried = registry.prepare_launch(preparation).await.unwrap(); + assert_eq!(retried.state, LaunchState::Running); + let first = supervisor + .activate_prepared( + retried, + LaunchWorkload { + soul_id: soul.clone(), + fixture: None, + terminal: Some(terminal.clone()), + fresh_agent: None, + resume_spec: None, + }, + limits(), + ) + .await + .unwrap(); + assert_eq!( + daemon.lock().unwrap().grants, + 1, + "healthy initial retry does not dispatch twice" + ); + assert!( + daemon.lock().unwrap().publishers.is_empty(), + "healthy initial retry completes publisher release" + ); + let spec = + freshell_agent_runtime::build_resume_spec(freshell_agent_runtime::ResumeSpecInput { + provider: "opencode", + provider_store_id: "store-exact", + native_session_id: "ses_exact", + terminal: &terminal, + provider_version: Some("1.18.21".into()), + creation_seed_ref: "seed".into(), + checkpoint_revision: 0, + evidence_revision: 0, + never_dispatched: false, + }) + .unwrap(); + registry + .store_verified_resume_spec(soul.clone(), spec) + .await + .unwrap(); + daemon + .lock() + .unwrap() + .running + .remove(&first.view.incarnation_id); + daemon.lock().unwrap().fail = Some(failure); + let request = || RecoverRequest { + soul_id: soul.clone(), + trigger: RecoveryTrigger::HostUnreachable, + expected_intent_revision: None, + expected_control_epoch: None, + }; + let failed = supervisor.recover(request()).await; + assert!( + failed.is_err(), + "injected post-success completion failure must reach actual recovery owner" + ); + let live = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + let view = registry + .inventory() + .await + .unwrap() + .into_iter() + .find(|v| v.incarnation_id == *live.incarnation_id()) + .unwrap(); + assert_eq!(view.launch_state, LaunchState::Running); + assert_eq!(view.recovery_state, RecoveryState::Live); + assert_eq!(view.native_session_id.as_deref(), Some("ses_exact")); + assert_eq!(daemon.lock().unwrap().generations.len(), 2); + assert_eq!(daemon.lock().unwrap().grants, 2); + drop(supervisor); + drop(registry); + let registry = Registry::open(&root, None).unwrap(); + let cold = Supervisor::new( + registry.clone(), + Arc::new(Backend(daemon.clone(), root.clone())), + config, + ) + .unwrap(); + let startup = cold.reconcile_startup().await.unwrap(); + assert!(startup.results.iter().all(|v| v.error.is_none())); + let after = registry.active_handle_for_soul(soul.clone()).await.unwrap(); + assert_eq!(after.incarnation_id(), live.incarnation_id()); + assert_eq!( + daemon.lock().unwrap().grants, + 2, + "accepted conversation must not be redispatched" + ); + assert_eq!( + daemon.lock().unwrap().generations, + BTreeSet::from(["grant-2".into()]), + "healthy fresh startup must retire only prior A" + ); + assert!( + daemon.lock().unwrap().publishers.is_empty(), + "accepted B publisher must be released" + ); + let events_before = daemon.lock().unwrap().events.len(); + assert_eq!( + cold.recover(request()).await.unwrap().outcome, + RecoveryOutcome::Reattached + ); + assert_eq!( + daemon.lock().unwrap().events.len(), + events_before, + "completed preparation is not retried on every healthy poll" + ); + daemon + .lock() + .unwrap() + .running + .remove(after.incarnation_id()); + let next = cold.recover(request()).await.unwrap(); + assert_eq!( + next.outcome, + RecoveryOutcome::Replaced, + "next valid C must fit the unchanged active/pending bound" + ); + assert_eq!( + daemon.lock().unwrap().generations, + BTreeSet::from(["grant-3".into()]) + ); + assert_eq!(daemon.lock().unwrap().grants, 3); + drop(cold); + drop(registry); + callback_task.abort(); + let _ = callback_task.await; + let tasks = std::mem::take(&mut daemon.lock().unwrap().host_tasks); + for task in tasks { + task.abort(); + let _ = task.await; + } + } +} diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index 86cb06511..c561888e1 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -316,6 +316,12 @@ impl Supervisor { .mark_recovery_live(request.soul_id.clone(), handle.incarnation_id.clone()) .await .map_err(map_registry)?; + let owned = self + .registry + .owned_handle(handle.incarnation_id.clone()) + .await + .map_err(map_registry)?; + self.complete_pending_provider_preparation(&owned).await?; Ok(RecoveryResult { outcome: RecoveryOutcome::Reattached, view: self.view_for(&handle.incarnation_id).await?, @@ -392,6 +398,12 @@ impl Supervisor { ) .await .map_err(map_registry)?; + let owned = self + .registry + .owned_handle(handle.incarnation_id.clone()) + .await + .map_err(map_registry)?; + self.complete_pending_provider_preparation(&owned).await?; Ok(RecoveryResult { outcome: RecoveryOutcome::Reattached, view: self.view_for(&handle.incarnation_id).await?, @@ -864,17 +876,13 @@ impl Supervisor { ) .await .map_err(map_registry)?; - self.commit_provider_preparation( - &LaunchWorkload { - soul_id: soul_id.clone(), - fixture: prepared.fixture, - terminal: prepared.terminal.clone(), - fresh_agent: prepared.fresh_agent.clone(), - resume_spec: prepared.resume_spec.clone(), - }, - &prepared.prepared.incarnation_id, - ) - .await?; + let accepted = self + .registry + .owned_handle(prepared.prepared.incarnation_id.clone()) + .await + .map_err(map_registry)?; + self.complete_pending_provider_preparation(&accepted) + .await?; let elapsed_ms = recovery_started .elapsed() .as_millis() @@ -1216,6 +1224,12 @@ impl Supervisor { .mark_recovery_live(soul_id, handle.incarnation_id.clone()) .await .map_err(map_registry)?; + let owned = self + .registry + .owned_handle(handle.incarnation_id.clone()) + .await + .map_err(map_registry)?; + self.complete_pending_provider_preparation(&owned).await?; Ok(RecoveryResult { outcome: RecoveryOutcome::Reattached, view: self.view_for(&handle.incarnation_id).await?, diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 74cb6d760..afdc50bcc 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -17,7 +17,7 @@ use std::{ time::{SystemTime, UNIX_EPOCH}, }; -pub const SCHEMA_VERSION: u32 = 9; +pub const SCHEMA_VERSION: u32 = 10; mod input_reservation; pub(crate) use input_reservation::binary_digest; pub use input_reservation::{BoundInputReservation, OwnedInputReservation}; @@ -661,12 +661,23 @@ impl Registry { } pub async fn mark_running(&self, incarnation_id: IncarnationId) -> Result<(), RegistryError> { + self.mark_running_with_provider_preparation(incarnation_id, false) + .await + } + + /// Initial/pristine acceptance is proved by the owning service. A resumed + /// generation is armed by the exact recovery-success transaction instead. + pub(crate) async fn mark_running_with_provider_preparation( + &self, + incarnation_id: IncarnationId, + preparation_accepted: bool, + ) -> Result<(), RegistryError> { self.run_blocking(move |mut conn| { let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?; let now = now_millis(); let changed = tx.execute( - "UPDATE incarnations SET launch_state='running',updated_at=?1 WHERE incarnation_id=?2 AND launch_state='starting' AND EXISTS (SELECT 1 FROM souls WHERE soul_id=incarnations.soul_id AND desired_state='running')", - params![now, incarnation_id.as_str()], + "UPDATE incarnations SET launch_state='running',provider_preparation_pending=?3,updated_at=?1 WHERE incarnation_id=?2 AND launch_state='starting' AND EXISTS (SELECT 1 FROM souls WHERE soul_id=incarnations.soul_id AND desired_state='running')", + params![now, incarnation_id.as_str(), preparation_accepted], )?; if changed != 1 { return Err(RegistryError::InvalidState(format!("cannot mark RUNNING for {incarnation_id}"))); } tx.execute("UPDATE commands SET state='running',updated_at=?1 WHERE incarnation_id=?2", params![now, incarnation_id.as_str()])?; @@ -675,6 +686,31 @@ impl Registry { }).await } + pub(crate) async fn provider_preparation_is_pending( + &self, + incarnation: IncarnationId, + ) -> Result { + self.run_blocking(move |conn| { + Ok(conn.query_row( + "SELECT EXISTS(SELECT 1 FROM incarnations i JOIN souls s ON s.soul_id=i.soul_id WHERE i.incarnation_id=?1 AND i.provider_preparation_pending=1 AND i.launch_state='running' AND s.desired_state='running' AND s.recovery_state='live')", + params![incarnation.as_str()], |row| row.get(0), + )?) + }).await + } + + pub(crate) async fn mark_provider_preparation_complete( + &self, + incarnation: IncarnationId, + ) -> Result<(), RegistryError> { + self.run_blocking(move |conn| { + conn.execute( + "UPDATE incarnations SET provider_preparation_pending=0 WHERE incarnation_id=?1 AND provider_preparation_pending=1", + params![incarnation.as_str()], + )?; + Ok(()) + }).await + } + pub async fn owned_handle( &self, incarnation_id: IncarnationId, @@ -1680,6 +1716,10 @@ impl Registry { "UPDATE recovery_attempts SET verdict='succeeded',finished_at=?1 WHERE attempt_id=?2", params![now, attempt_id.as_str()], )?; + tx.execute( + "UPDATE incarnations SET provider_preparation_pending=1 WHERE incarnation_id=?1", + params![incarnation_id.as_str()], + )?; tx.commit()?; Ok(()) }) @@ -1973,6 +2013,7 @@ fn create_schema( fresh_agent_spec TEXT, prior_incarnation_id TEXT, recovery_attempt_id TEXT, + provider_preparation_pending INTEGER NOT NULL DEFAULT 0, exit_code INTEGER, oom_killed INTEGER NOT NULL DEFAULT 0, created_at INTEGER NOT NULL, @@ -2426,6 +2467,17 @@ fn migrate_schema(conn: &mut Connection) -> Result<(), RegistryError> { tx.commit()?; schema = 9; } + if schema == 9 { + let add_pending = !column_exists(conn, "incarnations", "provider_preparation_pending")?; + let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?; + if add_pending { + tx.execute_batch("ALTER TABLE incarnations ADD COLUMN provider_preparation_pending INTEGER NOT NULL DEFAULT 0;")?; + } + // Legacy Running does not prove resumed native acceptance. + tx.execute_batch("UPDATE installation SET schema_version=10 WHERE singleton=1;")?; + tx.commit()?; + schema = 10; + } if schema != SCHEMA_VERSION { return Err(RegistryError::Integrity(format!( "unsupported older schema {schema}" @@ -2911,6 +2963,72 @@ mod tests { } } + #[tokio::test] + async fn provider_preparation_migration_preserves_unproved_running_and_accepted_pending_state() + { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().join("registry"); + let registry = Registry::open(&root, None).unwrap(); + let prepared = registry + .prepare_launch(prep(SoulId::new(), RequestId::new(), "pending")) + .await + .unwrap(); + let id = prepared.incarnation_id.clone(); + let conn = Connection::open(®istry.inner.db_path).unwrap(); + conn.execute( + "UPDATE incarnations SET launch_state='running' WHERE incarnation_id=?1", + params![id.as_str()], + ) + .unwrap(); + assert!( + !registry + .provider_preparation_is_pending(id.clone()) + .await + .unwrap(), + "generic Running has no native acceptance proof" + ); + conn.execute_batch("ALTER TABLE incarnations DROP COLUMN provider_preparation_pending; UPDATE installation SET schema_version=9;").unwrap(); + drop(conn); + drop(registry); + let migrated = Registry::open(&root, None).unwrap(); + assert!( + !migrated + .provider_preparation_is_pending(id.clone()) + .await + .unwrap(), + "legacy Running remains uncertain after migration" + ); + let conn = Connection::open(&migrated.inner.db_path).unwrap(); + conn.execute( + "UPDATE incarnations SET launch_state='starting' WHERE incarnation_id=?1", + params![id.as_str()], + ) + .unwrap(); + migrated + .mark_running_with_provider_preparation(id.clone(), true) + .await + .unwrap(); + assert!(migrated + .provider_preparation_is_pending(id.clone()) + .await + .unwrap()); + // Idempotent upgrades preserve a real previously accepted pending proof. + conn.execute_batch("UPDATE installation SET schema_version=9;") + .unwrap(); + drop(conn); + drop(migrated); + let reopened = Registry::open(&root, None).unwrap(); + assert!(reopened + .provider_preparation_is_pending(id.clone()) + .await + .unwrap()); + reopened + .mark_provider_preparation_complete(id.clone()) + .await + .unwrap(); + assert!(!reopened.provider_preparation_is_pending(id).await.unwrap()); + } + #[tokio::test] async fn provider_initial_reservation_migration_retains_legacy_opencode_authority() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 29ac91d46..3a2bf31f6 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -83,6 +83,9 @@ mod claude_source_observation; #[cfg(test)] #[path = "native_auth_lookup_tests.rs"] mod native_auth_lookup_tests; +#[cfg(test)] +#[path = "provider_preparation_tests.rs"] +mod provider_preparation_tests; impl Drop for AuthLookupScope { fn drop(&mut self) { self.authorities @@ -1008,6 +1011,7 @@ impl Supervisor { .map_err(|error| { self.activation_failure(&prepared, state, "read_running_host_status", error) })?; + self.complete_pending_provider_preparation(&handle).await?; return self .launch_result_from_status( prepared.incarnation_id.clone(), @@ -1087,13 +1091,6 @@ impl Supervisor { self.activation_failure(&prepared, state, "deliver_execution_grant", error) })?; crash_if("after_grant_delivery"); - self.registry - .mark_running(handle.incarnation_id().clone()) - .await - .map_err(map_registry) - .map_err(|error| { - self.activation_failure(&prepared, state, "commit_running_state", error) - })?; if let Some(native_session_id) = accepted.native_session_id.as_ref() { self.registry .record_native_session( @@ -1107,8 +1104,18 @@ impl Supervisor { self.activation_failure(&prepared, state, "record_native_identity", error) })?; } + self.registry + .mark_running_with_provider_preparation( + handle.incarnation_id().clone(), + workload.resume_spec.is_none(), + ) + .await + .map_err(map_registry) + .map_err(|error| { + self.activation_failure(&prepared, state, "commit_running_state", error) + })?; if workload.resume_spec.is_none() { - self.commit_provider_preparation(&workload, handle.incarnation_id()) + self.complete_pending_provider_preparation(&handle) .await .map_err(|error| { self.activation_failure(&prepared, state, "commit_provider_inputs", error) @@ -2028,6 +2035,55 @@ impl Supervisor { Ok(()) } + /// Retry only a durable acceptance proof for this immutable incarnation. + /// Running alone never authorizes completion of a resumed generation. + pub(crate) async fn complete_pending_provider_preparation( + &self, + handle: &crate::registry::OwnedRuntimeHandle, + ) -> Result<(), RuntimeError> { + if !self + .registry + .provider_preparation_is_pending(handle.incarnation_id().clone()) + .await + .map_err(map_registry)? + { + return Ok(()); + } + let workload = LaunchWorkload { + soul_id: handle.soul_id().clone(), + fixture: handle.fixture(), + terminal: handle.terminal().cloned(), + fresh_agent: handle.fresh_agent().cloned(), + resume_spec: None, + }; + if let Err(error) = self + .commit_provider_preparation(&workload, handle.incarnation_id()) + .await + { + append_event( + &self.config.lifecycle_log, + "supervisor.provider_preparation.pending", + serde_json::json!({ + "severity":"error", "soulId":handle.soul_id(), "incarnationId":handle.incarnation_id(), + "operation":"complete_accepted_generation", "errorCode":error.code, + }), + ); + return Err(error); + } + self.registry + .mark_provider_preparation_complete(handle.incarnation_id().clone()) + .await + .map_err(map_registry)?; + append_event( + &self.config.lifecycle_log, + "supervisor.provider_preparation.completed", + serde_json::json!({ + "severity":"info", "soulId":handle.soul_id(), "incarnationId":handle.incarnation_id(), + }), + ); + Ok(()) + } + pub(crate) async fn commit_provider_preparation( &self, workload: &LaunchWorkload, @@ -2066,10 +2122,9 @@ impl Supervisor { .commit_prepared_inputs(&handle) .await .map_err(map_backend)?; - // Both callers are after HostGrantAccepted and registry commit. - // Initial/pristine activation calls here only without resume_spec; - // exact resumes reach here from recovery only AFTER native identity - // verification, queued-input replay, and recovery-success commit. + // The pending flag is armed atomically with initial acceptance or + // exact verified recovery success. Healthy retries preserve those + // gates while finishing interrupted release/retirement. if let Some(prior) = self .registry .prior_provider_owner(incarnation.clone()) From 8abf19457a4fca63f1c24f8508dbeb347fa481ac Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:34:21 -0700 Subject: [PATCH 184/493] test(providers): authorize the selected external Amplifier fixture root --- scripts/testing/managed-provider-launch-broker.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 58aabb6f3..384e54fd9 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -30,7 +30,7 @@ const broker=new RestrictedDockerBroker({ : configuration.provider==='codex' ? ['/home/freshell/provider',path.join(root,'source-home/.codex')] : configuration.provider==='amplifier' - ? ['/home/freshell/provider',path.join(root,'source-home/.amplifier')] + ? ['/home/freshell/provider',path.join(root,'source-home/.amplifier'),path.join(root,'scenarios/external-amplifier-bundle')] : ['/home/freshell/provider',path.join(root,'.opencode'),path.join(root,'scenarios/external-native-resources')]), allowedBootstrapFiles:new Set([path.join(root,'grants/environment.env'),path.join(root,'grants/auth.json'),path.join(root,'grants/ca.pem')]), allowedHostGatewayAddresses:gateways, From aabf22920eed624ace002b7b10e7f1b809d2b0b3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:42:45 -0700 Subject: [PATCH 185/493] test(opencode): consume native skill endpoint with the pinned plugin client --- .../src/managed_provider_launch_tests.rs | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 6f36a4cca..4bade59c5 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3492,7 +3492,7 @@ export default async (input,options)=>({config:async(config)=>{ const digest=value=>crypto.createHash('sha256').update(value).digest('hex'); setTimeout(async()=>{ try { - const response=await input.client.app.skills(); + const response=await input.client._client.get({url:'/skill'}); const skill=response.data.find(skill=>skill.name==='closed-external-skill'); if(!skill)throw Error('selected native skill unavailable'); const output=execFileSync(path.join(path.dirname(skill.location),'scripts/run')); @@ -3749,6 +3749,16 @@ export default {{id:'closed-directory-discovery',async server(){{ return value; } } + let failure = path.with_file_name(format!( + "{}.failure", + path.file_name().unwrap().to_string_lossy() + )); + if let Ok(bytes) = std::fs::read(&failure) { + if let Ok(value) = serde_json::from_slice::(&bytes) { + println!("provider_skill_consumer_failure {}", value); + panic!("native skill consumer failed; see classified fixture receipt"); + } + } tokio::time::sleep(Duration::from_millis(50)).await; } }) From cced2347a9ee06870c22239bb4b3d498f84c8632 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:47:54 -0700 Subject: [PATCH 186/493] test(opencode): preserve native skill content newline in consumer proof --- crates/freshell-server/src/managed_provider_launch_tests.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 4bade59c5..726c1c515 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3509,7 +3509,7 @@ export default async (input,options)=>({config:async(config)=>{ original["skills"] = json!({"paths":[selected_skills,"../../scenarios/external-native-resources/selected-skills"]}); std::fs::write(&original_config, serde_json::to_vec(&original).unwrap()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); - let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency"))}); + let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency"))}); let explicit_config = fixture.join("workspace/selected-config.json"); let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; std::fs::write(&explicit_config, explicit_bytes).unwrap(); From 20bb30124ca5c9366475a5b9886d71a82c184d9e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 07:15:43 -0700 Subject: [PATCH 187/493] fix(supervisor): bound accepted completion on ordinary control workers --- .../src/provider_preparation_tests.rs | 54 ++++++++- crates/freshell-supervisor/src/service.rs | 106 ++++++++---------- 2 files changed, 96 insertions(+), 64 deletions(-) diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs index 2f870482c..baa202e70 100644 --- a/crates/freshell-supervisor/src/provider_preparation_tests.rs +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -280,7 +280,41 @@ impl RuntimeBackend for Backend { } } -#[tokio::test] +async fn control_recover( + socket: &std::path::Path, + request: RecoverRequest, +) -> Result { + let mut stream = tokio::net::UnixStream::connect(socket).await.unwrap(); + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::Recover(request), + ); + envelope.auth = Some("synthetic-control-secret".into()); + write_frame(&mut stream, &envelope).await.unwrap(); + let reply: AdminReply = read_frame(&mut stream).await.unwrap(); + match reply.result? { + AdminResult::Recovery(result) => Ok(result), + _ => panic!("unexpected actual control recovery reply"), + } +} + +async fn control_listener(supervisor: &Supervisor) -> tokio::task::JoinHandle<()> { + let owner = supervisor.clone(); + let path = supervisor.config.control_socket_path.clone(); + let socket = path.clone(); + let task = tokio::spawn(async move { serve_control(owner, &socket).await.unwrap() }); + tokio::time::timeout(std::time::Duration::from_secs(5), async { + while !path.exists() { + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + }) + .await + .unwrap(); + task +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn provider_accepted_completion_retries_after_registry_success_on_fresh_healthy_startup() { for failure in ["controller", "publisher", "retire"] { let temp = tempfile::tempdir().unwrap(); @@ -429,7 +463,8 @@ async fn provider_accepted_completion_retries_after_registry_success_on_fresh_he expected_intent_revision: None, expected_control_epoch: None, }; - let failed = supervisor.recover(request()).await; + let control = control_listener(&supervisor).await; + let failed = control_recover(&supervisor.config.control_socket_path, request()).await; assert!( failed.is_err(), "injected post-success completion failure must reach actual recovery owner" @@ -447,6 +482,9 @@ async fn provider_accepted_completion_retries_after_registry_success_on_fresh_he assert_eq!(view.native_session_id.as_deref(), Some("ses_exact")); assert_eq!(daemon.lock().unwrap().generations.len(), 2); assert_eq!(daemon.lock().unwrap().grants, 2); + control.abort(); + let _ = control.await; + std::fs::remove_file(&supervisor.config.control_socket_path).unwrap(); drop(supervisor); drop(registry); let registry = Registry::open(&root, None).unwrap(); @@ -475,8 +513,12 @@ async fn provider_accepted_completion_retries_after_registry_success_on_fresh_he "accepted B publisher must be released" ); let events_before = daemon.lock().unwrap().events.len(); + let control = control_listener(&cold).await; assert_eq!( - cold.recover(request()).await.unwrap().outcome, + control_recover(&cold.config.control_socket_path, request()) + .await + .unwrap() + .outcome, RecoveryOutcome::Reattached ); assert_eq!( @@ -489,7 +531,9 @@ async fn provider_accepted_completion_retries_after_registry_success_on_fresh_he .unwrap() .running .remove(after.incarnation_id()); - let next = cold.recover(request()).await.unwrap(); + let next = control_recover(&cold.config.control_socket_path, request()) + .await + .unwrap(); assert_eq!( next.outcome, RecoveryOutcome::Replaced, @@ -500,6 +544,8 @@ async fn provider_accepted_completion_retries_after_registry_success_on_fresh_he BTreeSet::from(["grant-3".into()]) ); assert_eq!(daemon.lock().unwrap().grants, 3); + control.abort(); + let _ = control.await; drop(cold); drop(registry); callback_task.abort(); diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 3a2bf31f6..22934420f 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -2037,74 +2037,65 @@ impl Supervisor { /// Retry only a durable acceptance proof for this immutable incarnation. /// Running alone never authorizes completion of a resumed generation. - pub(crate) async fn complete_pending_provider_preparation( - &self, - handle: &crate::registry::OwnedRuntimeHandle, - ) -> Result<(), RuntimeError> { - if !self - .registry - .provider_preparation_is_pending(handle.incarnation_id().clone()) - .await - .map_err(map_registry)? - { - return Ok(()); - } - let workload = LaunchWorkload { - soul_id: handle.soul_id().clone(), - fixture: handle.fixture(), - terminal: handle.terminal().cloned(), - fresh_agent: handle.fresh_agent().cloned(), - resume_spec: None, - }; - if let Err(error) = self - .commit_provider_preparation(&workload, handle.incarnation_id()) - .await - { + pub(crate) fn complete_pending_provider_preparation<'a>( + &'a self, + handle: &'a crate::registry::OwnedRuntimeHandle, + ) -> impl std::future::Future> + Send + 'a { + // Recovery embeds this operation on several branches. Keep the large + // provider completion future off the ordinary control-worker stack. + Box::pin(async move { + if !self + .registry + .provider_preparation_is_pending(handle.incarnation_id().clone()) + .await + .map_err(map_registry)? + { + return Ok(()); + } + if let Err(error) = self.commit_provider_preparation(handle).await { + append_event( + &self.config.lifecycle_log, + "supervisor.provider_preparation.pending", + serde_json::json!({ + "severity":"error", "soulId":handle.soul_id(), "incarnationId":handle.incarnation_id(), + "operation":"complete_accepted_generation", "errorCode":error.code, + }), + ); + return Err(error); + } + self.registry + .mark_provider_preparation_complete(handle.incarnation_id().clone()) + .await + .map_err(map_registry)?; append_event( &self.config.lifecycle_log, - "supervisor.provider_preparation.pending", + "supervisor.provider_preparation.completed", serde_json::json!({ - "severity":"error", "soulId":handle.soul_id(), "incarnationId":handle.incarnation_id(), - "operation":"complete_accepted_generation", "errorCode":error.code, + "severity":"info", "soulId":handle.soul_id(), "incarnationId":handle.incarnation_id(), }), ); - return Err(error); - } - self.registry - .mark_provider_preparation_complete(handle.incarnation_id().clone()) - .await - .map_err(map_registry)?; - append_event( - &self.config.lifecycle_log, - "supervisor.provider_preparation.completed", - serde_json::json!({ - "severity":"info", "soulId":handle.soul_id(), "incarnationId":handle.incarnation_id(), - }), - ); - Ok(()) + Ok(()) + }) } - pub(crate) async fn commit_provider_preparation( + async fn commit_provider_preparation( &self, - workload: &LaunchWorkload, - incarnation: &IncarnationId, + handle: &crate::registry::OwnedRuntimeHandle, ) -> Result<(), RuntimeError> { - let context = workload - .terminal - .as_ref() + let context = handle + .terminal() .and_then(|terminal| terminal.provider_launch_context.as_ref()) .or_else(|| { - workload - .fresh_agent - .as_ref() + handle + .fresh_agent() .and_then(|agent| agent.provider_launch_context.as_ref()) }); if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { self.notify_mcp_capability( "commit", &capability.endpoint, - &workload.soul_id, - incarnation, + handle.soul_id(), + handle.incarnation_id(), &capability.grant_id, ) .await?; @@ -2113,13 +2104,8 @@ impl Supervisor { .and_then(|context| context.prepared_inputs.as_ref()) .is_some() { - let handle = self - .registry - .owned_handle(incarnation.clone()) - .await - .map_err(map_registry)?; self.backend - .commit_prepared_inputs(&handle) + .commit_prepared_inputs(handle) .await .map_err(map_backend)?; // The pending flag is armed atomically with initial acceptance or @@ -2127,18 +2113,18 @@ impl Supervisor { // gates while finishing interrupted release/retirement. if let Some(prior) = self .registry - .prior_provider_owner(incarnation.clone()) + .prior_provider_owner(handle.incarnation_id().clone()) .await .map_err(map_registry)? { self.backend - .retire_prior_prepared_inputs(&handle, &prior) + .retire_prior_prepared_inputs(handle, &prior) .await .map_err(map_backend)?; } } if let Some(inputs) = context.and_then(|context| context.prepared_inputs.as_ref()) { - self.commit_provider_path_inputs(&workload.soul_id, &inputs.generation_id); + self.commit_provider_path_inputs(handle.soul_id(), &inputs.generation_id); } Ok(()) } From 295381db02c0ac78e5ba95efb574f61060868296 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 08:27:38 -0700 Subject: [PATCH 188/493] fix: refresh validated file references on fresh actor reopen --- .../freshell-agent-runtime/src/host_actor.rs | 17 +++++++++++++++ .../src/host_actor_tests.rs | 21 +++++++++++++++++-- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/crates/freshell-agent-runtime/src/host_actor.rs b/crates/freshell-agent-runtime/src/host_actor.rs index 5fe187545..28111f58b 100644 --- a/crates/freshell-agent-runtime/src/host_actor.rs +++ b/crates/freshell-agent-runtime/src/host_actor.rs @@ -426,7 +426,23 @@ impl FreshAgentHostActor { // Its grant and resource generation may change while the durable // conversation and every other launch setting remain the same. let current_preparation = expected_profile.provider_launch_context.take(); + let current_references = expected_profile.provider_secret_references.clone(); if current_preparation.is_some() { + // Configuration-file references belong to the host-validated + // preparation. Legacy OneCLI grants remain durable settings. + if !current_references + .iter() + .filter(|reference| !reference.profile.is_configuration_file()) + .eq(loaded + .profile + .provider_secret_references + .iter() + .filter(|reference| !reference.profile.is_configuration_file())) + { + return Err(ActorError::NativeIdentityMismatch); + } + expected_profile.provider_secret_references = + loaded.profile.provider_secret_references.clone(); expected_profile.provider_launch_context = loaded.profile.provider_launch_context.clone(); } @@ -434,6 +450,7 @@ impl FreshAgentHostActor { return Err(ActorError::NativeIdentityMismatch); } loaded.profile.provider_launch_context = current_preparation; + loaded.profile.provider_secret_references = current_references; for command in loaded.commands.values_mut() { if matches!(command.state, CommandState::Dispatching) { command.state = CommandState::Ambiguous; diff --git a/crates/freshell-agent-runtime/src/host_actor_tests.rs b/crates/freshell-agent-runtime/src/host_actor_tests.rs index abc19dc67..a7cd07c7d 100644 --- a/crates/freshell-agent-runtime/src/host_actor_tests.rs +++ b/crates/freshell-agent-runtime/src/host_actor_tests.rs @@ -894,7 +894,8 @@ async fn exact_native_identity_and_profile_are_reused_on_host_actor_recovery() { #[tokio::test] async fn current_provider_preparation_reopens_the_same_fresh_conversation() { use freshell_runtime_protocol::{ - McpCapabilityReference, ProviderLaunchContext, ProviderPreparation, + McpCapabilityReference, ProviderLaunchContext, ProviderPreparation, ProviderSecretProfile, + ProviderSecretReference, }; for provider in [ FreshProvider::Claude, @@ -918,7 +919,7 @@ async fn current_provider_preparation_reopens_the_same_fresh_conversation() { }, }; let dir = tempfile::tempdir().unwrap(); - let mut initial = profile(provider, "same-store", None); + let mut initial = profile(provider.clone(), "same-store", None); initial.provider_launch_context = Some(ProviderLaunchContext { preparation, prepared_inputs: None, @@ -930,6 +931,19 @@ async fn current_provider_preparation_reopens_the_same_fresh_conversation() { host_gateway_address: None, }), }); + initial + .provider_secret_references + .push(ProviderSecretReference { + source_path: "/synthetic/old-config-target".into(), + profile: match provider { + FreshProvider::Claude | FreshProvider::Kilroy => { + ProviderSecretProfile::ClaudeConfigurationFile + } + FreshProvider::Codex => ProviderSecretProfile::CodexConfigurationFile, + FreshProvider::Opencode => ProviderSecretProfile::OpencodeConfigurationFile, + }, + provider_relative_path: Some(".freshell/references/config-0".into()), + }); let actor = FreshAgentHostActor::open(dir.path(), initial, FakeTransport::new("same-native")) .await @@ -944,6 +958,7 @@ async fn current_provider_preparation_reopens_the_same_fresh_conversation() { .as_mut() .unwrap() .grant_id = "grant-replacement".into(); + refreshed.provider_secret_references[0].source_path = "/synthetic/new-config-target".into(); let transport = FakeTransport::new("same-native"); let actor = FreshAgentHostActor::open(dir.path(), refreshed.clone(), transport.clone()) .await @@ -974,6 +989,8 @@ async fn current_provider_preparation_reopens_the_same_fresh_conversation() { .as_mut() .unwrap() .grant_id = "grant-unconfirmed".into(); + next_preparation.provider_secret_references[0].source_path = + "/synthetic/unconfirmed-config-target".into(); let wrong_native = FakeTransport::new("wrong-native-from-provider"); let result = FreshAgentHostActor::open(dir.path(), next_preparation.clone(), wrong_native.clone()) From 4e0837f567a6151acef2680f04ae94b45f64d2ee Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 08:35:36 -0700 Subject: [PATCH 189/493] fix: transport selected OpenCode sound files and MCP directories --- .../managed_mcp_capability/source_stage.rs | 155 +++++++++++++++++ .../src/managed_provider_launch_tests.rs | 32 +++- .../opencode_resources.rs | 160 ++++++++++++++++-- 3 files changed, 325 insertions(+), 22 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index f2152b0ce..92b7ee411 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -118,6 +118,161 @@ mod configured_resource_tests { assert_eq!(plan[0].references.path_variables, vec!["RULE_PATTERN"]); assert_eq!(plan[0].references.unexpanded_source(), raw); } + #[test] + fn managed_provider_sound_and_mcp_cwd_consume_external_closure_after_both_copies() { + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace/nested"); + let external = owned.path().join("external"); + let tool = external.join("tool"); + for path in [&home, &cwd, &tool.join("node_modules/dependency")] { + std::fs::create_dir_all(path).unwrap(); + } + let sound = external.join("selected.wav"); + std::fs::write(&sound, b"RIFF-selected-wave-bytes").unwrap(); + std::fs::write(tool.join("resource"), "mcp-resource").unwrap(); + std::fs::write( + tool.join("node_modules/dependency/index.js"), + "module.exports='mcp-dependency';", + ) + .unwrap(); + std::fs::write(tool.join("server.cjs"), "const fs=require('node:fs');process.stdout.write(fs.readFileSync('resource','utf8')+':'+require('dependency')+':'+fs.readFileSync(process.argv[2],'utf8'));").unwrap(); + let config = external.join("tui.jsonc"); + for (ordinal, spelling) in [ + sound.display().to_string(), + reqwest::Url::from_file_path(&sound).unwrap().to_string(), + "./selected.wav".into(), + ] + .into_iter() + .enumerate() + { + let raw = format!("{{ // selected finite native fields\n\"attention\":{{\"sounds\":{{\"default\":{}}}}},\"mcp\":{{\"probe\":{{\"type\":\"local\",\"command\":[\"node\",\"server.cjs\"],\"cwd\":\"../../external/tool\"}},\"disabled\":{{\"type\":\"local\",\"enabled\":false,\"cwd\":\"/missing-disabled\"}},\"remote\":{{\"type\":\"remote\",\"cwd\":\"/missing-remote\"}}}}}}", serde_json::to_string(&spelling).unwrap()); + std::fs::write(&config, &raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Tui, + role: DiscoveryRole::Workspace, + }], + &cwd, + &home, + &|_, _, _| None, + ) + .unwrap(); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &owned.path().join("workspace"), + &plan, + None, + ) + .unwrap(); + let sound_unit = units + .iter() + .find(|(source, _)| source == &sound) + .expect("selected attention sound must be transported"); + let tool_unit = units + .iter() + .find(|(source, _)| source == &tool) + .expect("local MCP cwd must transport the whole directory"); + let stage = owned.path().join(format!("stage-{ordinal}")); + let host = owned.path().join(format!("host-{ordinal}")); + let selections = units + .iter() + .map(|(source, unit)| CopySelection { + source: source.clone(), + destination_relative: unit.stage_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&selections, &stage, &[], Default::default()).unwrap(); + let selections = units + .iter() + .map(|(_, unit)| CopySelection { + source: stage.join(&unit.stage_relative_path), + destination_relative: unit.provider_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&selections, &host, &[], Default::default()).unwrap(); + let hidden = owned.path().join("hidden-external"); + std::fs::rename(&external, &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg("server.cjs") + .arg(host.join(&sound_unit.1.provider_relative_path)) + .current_dir(host.join(&tool_unit.1.provider_relative_path)) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, &external).unwrap(); + assert!(output.status.success(), "MCP cwd consumer must execute its dependencies/resources and read the sound after both copies: {:?}",output.stderr); + assert_eq!( + output.stdout, + b"mcp-resource:mcp-dependency:RIFF-selected-wave-bytes" + ); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + assert_eq!( + std::fs::metadata(&sound).unwrap().permissions().mode() & 0o777, + 0o644 + ); + } + } + + #[test] + fn managed_provider_sound_and_mcp_cwd_refuse_unavailable_selected_resources() { + use std::os::unix::fs::PermissionsExt; + let owned = tempfile::tempdir().unwrap(); + let config = owned.path().join("tui.jsonc"); + for field in ["sound", "cwd"] { + for case in ["missing", "unreadable", "cyclic", "wrong-type"] { + let target = owned.path().join(format!("{field}-{case}")); + match case { + "cyclic" => std::os::unix::fs::symlink(&target, &target).unwrap(), + "missing" => {} + _ if (field == "cwd") != (case == "wrong-type") => { + std::fs::create_dir(&target).unwrap() + } + _ => std::fs::write(&target, "resource").unwrap(), + } + if case == "unreadable" { + std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0)).unwrap(); + } + let raw=if field=="sound" {serde_json::json!({"attention":{"sounds":{"question":target}}})} else {serde_json::json!({"mcp":{"probe":{"type":"local","command":["node","server.cjs"],"cwd":target}}})}.to_string(); + std::fs::write(&config, &raw).unwrap(); + let result = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Tui, + role: DiscoveryRole::Workspace, + }], + owned.path(), + owned.path(), + &|_, _, _| None, + ); + if case == "unreadable" { + std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o700)) + .unwrap(); + } + let error = result + .err() + .expect("selected unavailable sound/cwd must block preparation"); + assert_eq!( + error.field, + if field == "sound" { + "OpenCode attention.sounds" + } else { + "OpenCode mcp.cwd" + } + ); + assert_eq!(error.declaring_source, config); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + assert!(!owned.path().join("stage").exists()); + } + } + } + #[test] fn managed_provider_configured_skill_paths_execute_whole_directory_after_both_copies() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 726c1c515..b928e8423 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3485,6 +3485,19 @@ export default {{id:'closed-shared-{label}',async tui(api,options){{ std::fs::Permissions::from_mode(0o755), ) .unwrap(); + let mcp_cwd = external.join("mcp-cwd"); + std::fs::create_dir_all(mcp_cwd.join("node_modules/native-mcp-dependency")).unwrap(); + std::fs::write(mcp_cwd.join("resource"), "native-mcp-resource").unwrap(); + std::fs::write( + mcp_cwd.join("node_modules/native-mcp-dependency/index.js"), + "module.exports='native-mcp-dependency';", + ) + .unwrap(); + let mcp_cwd_receipt = fixture.join("workspace/native-mcp-cwd-consumed.txt"); + std::fs::write(mcp_cwd.join("server.cjs"), format!(r#"const fs=require('node:fs'),readline=require('node:readline'); +fs.writeFileSync({},fs.readFileSync('resource','utf8')+':'+require('native-mcp-dependency')); +readline.createInterface({{input:process.stdin}}).on('line',line=>{{const q=JSON.parse(line);if(q.id===undefined)return;const result=q.method==='initialize'?{{protocolVersion:q.params.protocolVersion,capabilities:{{tools:{{}}}},serverInfo:{{name:'closed-cwd-probe',version:'1'}}}}:q.method==='tools/list'?{{tools:[]}}:{{}};process.stdout.write(JSON.stringify({{jsonrpc:'2.0',id:q.id,result}})+'\n');}}); +"#,serde_json::to_string(&mcp_cwd_receipt).unwrap())).unwrap(); let external_receipt = fixture.join("workspace/native-external-consumed.json"); let external_plugin = r#"import fs from 'node:fs';import crypto from 'node:crypto';import path from 'node:path';import {execFileSync} from 'node:child_process'; import adjacent from './adjacent.mjs';import child from 'closed-child';import parent from 'closed-parent'; @@ -3496,7 +3509,9 @@ export default async (input,options)=>({config:async(config)=>{ const skill=response.data.find(skill=>skill.name==='closed-external-skill'); if(!skill)throw Error('selected native skill unavailable'); const output=execFileSync(path.join(path.dirname(skill.location),'scripts/run')); - fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction),nativeSkill:true,skillContent:digest(skill.content),skillExecution:digest(output)})); + await input.client._client.get({url:'/mcp'}); + const mcp=digest(fs.readFileSync(options.mcpReceipt)); + fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction),nativeSkill:true,skillContent:digest(skill.content),skillExecution:digest(output),nativeMcpCwd:mcp})); } catch(error) {fs.writeFileSync(options.receipt+'.failure',JSON.stringify({operation:'native_skill_consumer',type:error instanceof TypeError?'TypeError':error.code==='ENOENT'?'ENOENT':'native_skill_unavailable'}));} },0); }}); @@ -3504,12 +3519,14 @@ export default async (input,options)=>({config:async(config)=>{ std::fs::write(package.join("index.mjs"), external_plugin).unwrap(); let mut original: Value = serde_json::from_slice(&std::fs::read(&original_config).unwrap()).unwrap(); - original["plugin"].as_array_mut().unwrap().push(json!([reqwest::Url::from_file_path(package.join("index.mjs")).unwrap().to_string(), {"receipt":external_receipt,"instruction":external_instruction}])); + original["plugin"].as_array_mut().unwrap().push(json!([reqwest::Url::from_file_path(package.join("index.mjs")).unwrap().to_string(), {"receipt":external_receipt,"instruction":external_instruction,"mcpReceipt":mcp_cwd_receipt}])); + original["mcp"]["closed-cwd-probe"] = + json!({"type":"local","command":["node","server.cjs"],"cwd":mcp_cwd}); original["instructions"] = json!([external_instruction]); original["skills"] = json!({"paths":[selected_skills,"../../scenarios/external-native-resources/selected-skills"]}); std::fs::write(&original_config, serde_json::to_vec(&original).unwrap()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); - let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency"))}); + let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":format!("{:x}",Sha256::digest(b"native-mcp-resource:native-mcp-dependency"))}); let explicit_config = fixture.join("workspace/selected-config.json"); let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; std::fs::write(&explicit_config, explicit_bytes).unwrap(); @@ -3520,14 +3537,17 @@ export default async (input,options)=>({config:async(config)=>{ std::fs::create_dir_all(&tui_plugin).unwrap(); std::fs::write(tui_plugin.join("index.mjs"), r#"import fs from 'node:fs';import crypto from 'node:crypto'; export default {id:'closed-native-tui-fixture',async tui(api,options){ - fs.writeFileSync(options.receipt,JSON.stringify({theme:crypto.createHash('sha256').update(api.tuiConfig.theme).digest('hex')})); + const digest=value=>crypto.createHash('sha256').update(value).digest('hex'); + fs.writeFileSync(options.receipt,JSON.stringify({theme:digest(api.tuiConfig.theme),sounds:['default','question','error'].map(name=>digest(fs.readFileSync(api.tuiConfig.attention.sounds[name])))})); }}; "#).unwrap(); let selected_tui = fixture.join("workspace/selected-tui.jsonc"); std::os::unix::fs::symlink("shared.json", &selected_tui).unwrap(); let project_tui = fixture.join("workspace/tui.jsonc"); let tui_receipt = fixture.join("workspace/native-tui-consumed.json"); - let selected_tui_bytes = format!("{{ // selected origin\n\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&tui_receipt).unwrap()); + let selected_sound = external.join("selected.wav"); + std::fs::write(&selected_sound, b"RIFF-native-selected-wave").unwrap(); + let selected_tui_bytes = format!("{{ // selected origin\n\"attention\":{{\"sounds\":{{\"default\":{},\"question\":{},\"error\":\"../scenarios/external-native-resources/selected.wav\"}}}},\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&selected_sound).unwrap(),serde_json::to_string(&reqwest::Url::from_file_path(&selected_sound).unwrap().to_string()).unwrap(),serde_json::to_string(&tui_receipt).unwrap()); std::fs::write(&project_tui, &selected_tui_bytes).unwrap(); std::fs::write(fixture.join("workspace/selected-theme"), " dark\n").unwrap(); let inline_source = r#"{ // inline instance origin @@ -3896,7 +3916,7 @@ export default {{id:'closed-directory-discovery',async server(){{ read_receipt(&tui_receipt) .await .expect("actual TUI plugin must consume the selected effective TUI configuration"), - json!({"theme":format!("{:x}",Sha256::digest(b"dark"))}) + json!({"theme":format!("{:x}",Sha256::digest(b"dark")),"sounds":vec![format!("{:x}",Sha256::digest(b"RIFF-native-selected-wave"));3]}) ); let ca_digest = format!( "{:x}", diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index e545bbcbd..84692a9a7 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -7,7 +7,7 @@ use std::path::{Path, PathBuf}; enum Token { String(usize, usize), Punctuation(u8), - Other, + Other(usize), } struct Cursor<'a> { raw: &'a str, @@ -56,7 +56,7 @@ impl Cursor<'_> { { self.at += 1; } - Some(Token::Other) + Some(Token::Other(start)) } } } @@ -133,6 +133,121 @@ fn fields(raw: &str) -> Result, &'static str> { Ok(result) } +// Read only member spans: selected nested schemas never deserialize or rewrite +// an entire source document, nor interpret command arrays as file selections. +fn object_members( + raw: &str, + start: usize, +) -> Result, std::ops::Range)>, &'static str> { + let mut cursor = Cursor { raw, at: start }; + if !matches!(cursor.next(), Some(Token::Punctuation(b'{'))) { + return Ok(Vec::new()); + } + let mut result = Vec::new(); + loop { + let key = match cursor.next().ok_or("incomplete selected object")? { + Token::Punctuation(b'}') => return Ok(result), + Token::Punctuation(b',') => continue, + Token::String(start, end) => start..end, + _ => return Err("invalid selected object member"), + }; + if !matches!(cursor.next(), Some(Token::Punctuation(b':'))) { + return Err("invalid selected object member"); + } + let token = cursor.next().ok_or("incomplete selected object value")?; + let start = match token { + Token::String(start, _) | Token::Other(start) => start, + Token::Punctuation(_) => cursor.at - 1, + }; + let mut depth = usize::from(matches!(token, Token::Punctuation(b'{' | b'['))); + while depth > 0 { + match cursor.next().ok_or("incomplete selected object value")? { + Token::Punctuation(b'{' | b'[') => { + depth += 1; + if depth > 64 { + return Err("selected object exceeds shared depth profile"); + } + } + Token::Punctuation(b'}' | b']') => depth -= 1, + _ => {} + } + } + if result.len() >= 100_000 { + return Err("selected object exceeds shared entry profile"); + } + result.push((key, start..cursor.at)); + } +} +fn member<'a>( + raw: &'a str, + members: &[(std::ops::Range, std::ops::Range)], + name: &str, +) -> Option<&'a str> { + members.iter().rev().find_map(|(key, value)| { + if key.len() <= 64 + && serde_json::from_str::(&raw[key.clone()]) + .ok() + .as_deref() + == Some(name) + { + Some(&raw[value.clone()]) + } else { + None + } + }) +} +fn selected_string(raw: &str) -> Result, &'static str> { + if !raw.trim_start().starts_with('"') { + return Ok(None); + } + if raw.len() > 64 * 1024 { + return Err("configured resource selection profile exceeded"); + } + serde_json::from_str(raw) + .map(Some) + .map_err(|_| "invalid selected resource path") +} +fn nested_fields(raw: &str, tui: bool) -> Result, &'static str> { + let top = object_members(raw, 0)?; + let mut result = Vec::new(); + if tui { + if let Some(attention) = member(raw, &top, "attention") { + let entries = object_members(attention, 0)?; + if let Some(sounds) = member(attention, &entries, "sounds") { + let entries = object_members(sounds, 0)?; + for name in ["default", "question", "error", "subagent_done"] { + if let Some(value) = member(sounds, &entries, name) { + if let Some(path) = selected_string(value)?.filter(|path| !path.is_empty()) + { + result.push(("OpenCode attention.sounds", path)); + } + } + } + } + } + } + if let Some(mcp) = member(raw, &top, "mcp") { + for (_, range) in object_members(mcp, 0)? { + let entry = &mcp[range]; + let entries = object_members(entry, 0)?; + if member(entry, &entries, "type") + .and_then(|value| serde_json::from_str::(value).ok()) + .as_deref() + != Some("local") + || member(entry, &entries, "enabled").is_some_and(|value| value.trim() == "false") + { + continue; + } + if let Some(value) = member(entry, &entries, "cwd") { + if let Some(path) = selected_string(value)? { + result.push(("OpenCode mcp.cwd", path)); + } + } + } + } + Ok(result) +} + fn resource_array( cursor: &mut Cursor<'_>, field: &'static str, @@ -230,9 +345,13 @@ pub fn select( Ok(()) }; let mut variables = std::collections::BTreeSet::new(); - for (field, spelling) in fields(source.references.unexpanded_source()) - .map_err(|cause| fail("OpenCode resource", cause))? - { + let raw = source.references.unexpanded_source(); + let mut selections = fields(raw).map_err(|cause| fail("OpenCode resource", cause))?; + selections.extend( + nested_fields(raw, source.source.lane == super::sources::SourceLane::Tui) + .map_err(|cause| fail("OpenCode resource", cause))?, + ); + for (field, spelling) in selections { // Private file-token payloads already use the accepted grant plan. if spelling.contains("{file:") { continue; @@ -268,16 +387,22 @@ pub fn select( if introduced_file.get() { continue; } - if spelling.starts_with("http://") || spelling.starts_with("https://") { + if field != "OpenCode attention.sounds" + && field != "OpenCode mcp.cwd" + && (spelling.starts_with("http://") || spelling.starts_with("https://")) + { continue; } - let mut path = if field == "OpenCode plugin" { + let mut path = if field == "OpenCode plugin" || field == "OpenCode attention.sounds" { if spelling.starts_with("file:") { reqwest::Url::parse(&spelling) .ok() .and_then(|url| url.to_file_path().ok()) .ok_or_else(|| fail(field, "local file URL is invalid"))? - } else if spelling.starts_with(['/', '.']) || spelling.starts_with("~/") { + } else if field == "OpenCode attention.sounds" + || spelling.starts_with(['/', '.']) + || spelling.starts_with("~/") + { home_path(&spelling, home) } else { continue; @@ -324,7 +449,7 @@ pub fn select( selected.map_err(|_| fail(field, "instruction glob traversal failed"))?; let canonical = checked(&selected, field, declaring)?; if !canonical.is_file() { - return Err(fail(field, "selected instruction is not a regular file")); + return Err(fail(field, "selected file resource is not a regular file")); } insert(selected, field)?; matched = true; @@ -343,12 +468,12 @@ pub fn select( continue; } if !path.is_absolute() { - if field == "OpenCode plugin" { + if field == "OpenCode plugin" || field == "OpenCode attention.sounds" { path = declaring .parent() .unwrap_or(&source.instance_cwd) .join(path); - } else if field == "OpenCode skills.paths" { + } else if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { path = source.instance_cwd.join(path); } else { path = source @@ -359,23 +484,26 @@ pub fn select( .unwrap_or_else(|| source.instance_cwd.join(path)); } } - if field == "OpenCode skills.paths" { + if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { // Native instance-cwd resolution normalizes .. before deciding // whether the selected directory is supplied by the workspace. path = freshell_platform::opencode_config::lexical_path(&path); } let canonical = checked(&path, field, declaring)?; variables.extend(names.into_inner()); - if field == "OpenCode instructions" { + if field == "OpenCode instructions" || field == "OpenCode attention.sounds" { if !canonical.is_file() { - return Err(fail(field, "selected instruction is not a regular file")); + return Err(fail(field, "selected file resource is not a regular file")); } insert(path, field)?; continue; } - if field == "OpenCode skills.paths" { + if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { if !canonical.is_dir() { - return Err(fail(field, "selected skill path is not a directory")); + return Err(fail( + field, + "selected directory resource is not a directory", + )); } insert(path, field)?; continue; From b535fa4503e06be7054dc3782803044c3f1dbd43 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 08:42:10 -0700 Subject: [PATCH 190/493] fix: resolve local Amplifier context namespaces at their source bases --- .../amplifier_resources.rs | 287 +++++++++++++++++- .../src/managed_provider_launch_tests.rs | 27 +- 2 files changed, 302 insertions(+), 12 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index dbcd68cec..278b240a5 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -62,6 +62,7 @@ pub(super) fn prepare( if let Some(runtime) = select_local_bundle(source, cwd, excluded, &mut result, &mut state)? { let replacement = serde_json::to_string(&runtime) .map_err(|_| "Amplifier projected source could not be encoded")?; + select_namespace_contexts(excluded, &mut result, &mut state)?; result.rewritten_settings = Some(rewrite_bundle_active( &raw, document, @@ -81,6 +82,66 @@ struct BundleSelection { metadata_bytes: usize, completed: BTreeMap, modules: BTreeMap, + namespaces: BTreeMap, + pending_contexts: Vec<(String, PathBuf)>, + native_includes: bool, +} + +impl BundleSelection { + fn retain_metadata(&mut self, bytes: usize) -> Result<(), String> { + self.metadata_bytes = self + .metadata_bytes + .checked_add(bytes) + .ok_or("Amplifier selected source metadata overflow")?; + if self.metadata_bytes > 8 * 1024 * 1024 { + return Err("Amplifier selected source metadata budget exceeded".into()); + } + Ok(()) + } +} + +fn select_namespace_contexts( + excluded: &[PathBuf], + result: &mut ResourcePreparation, + state: &mut BundleSelection, +) -> Result<(), String> { + for (namespace, path) in &state.pending_contexts { + let Some((source_base, destination_base)) = state.namespaces.get(namespace) else { + if state.native_includes { + continue; + } // Remote/registered bundles retain native resolution. + return Err("Amplifier selected context namespace has no local source base".into()); + }; + let selected = source_base.join(path); + let canonical = std::fs::canonicalize(&selected) + .map_err(|_| "Amplifier selected context is missing, inaccessible or cyclic")?; + if excluded.iter().any(|path| canonical.starts_with(path)) { + return Err("Amplifier selected context aliases credentials or native state; use the approved private grant source".into()); + } + if !canonical.is_file() { + return Err("Amplifier selected context is not a regular file".into()); + } + std::fs::File::open(&canonical).map_err(|_| "Amplifier selected context is unreadable")?; + let destination = destination_base.join(path); + if let Some(existing) = result + .selections + .iter() + .find(|selection| selection.destination_relative == destination) + { + if std::fs::canonicalize(&existing.source).ok().as_ref() != Some(&canonical) { + return Err("Amplifier selected namespace context destination conflicts".into()); + } + continue; + } + if result.selections.len() >= 8192 { + return Err("Amplifier source selection event budget exceeded".into()); + } + result.selections.push(CopySelection { + source: selected, + destination_relative: destination, + }); + } + Ok(()) } fn select_local_bundle( source: &str, @@ -140,13 +201,7 @@ fn select_local_bundle( if state.active.len() >= 64 || state.next_id >= 8192 { return Err("Amplifier local include depth or event budget exceeded".into()); } - state.metadata_bytes = state - .metadata_bytes - .checked_add(canonical.as_os_str().len()) - .ok_or("Amplifier selected source metadata overflow")?; - if state.metadata_bytes > 8 * 1024 * 1024 { - return Err("Amplifier selected source metadata budget exceeded".into()); - } + state.retain_metadata(canonical.as_os_str().len())?; state.active.insert(canonical.clone()); let id = state.next_id; state.next_id += 1; @@ -243,6 +298,49 @@ fn select_local_bundle( return Err("Amplifier selected bundle requires one YAML document".into()); } let mut bundle_replacements = Vec::new(); + if let Some(meta) = native_mapping_value(&bundle_documents[0], "bundle", 0)? { + if let Some(name) = native_mapping_value(meta, "name", 0)? + .and_then(|node| node.data.as_str()) + .filter(|name| !name.is_empty()) + { + let namespace_root = native_mapping_value(meta, "namespace_root", 0)?; + let source_base = if let Some(node) = namespace_root { + let relative = node + .data + .as_str() + .ok_or("Amplifier namespace_root must be a string")?; + let selected = root.join(relative); + let canonical = std::fs::canonicalize(&selected).map_err(|_| { + "Amplifier namespace source base is unavailable or cyclic" + })?; + if !canonical.is_dir() { + return Err("Amplifier namespace source base is not a directory".into()); + } + let start = header_offset + .checked_add(source_offset(yaml, node.span.start)?) + .ok_or("Amplifier namespace source offset overflow")?; + let end = header_offset + .checked_add(source_offset(yaml, node.span.end)?) + .ok_or("Amplifier namespace source offset overflow")?; + // Each selected namespace is relocated into its bundle's + // projected resource root; keep its native include names. + bundle_replacements.push((start, end, "\".\"".into())); + canonical + } else { + root.to_owned() + }; + state.retain_metadata(name.len())?; + state.retain_metadata(source_base.as_os_str().len())?; + state.retain_metadata(relative.as_os_str().len())?; + if state.namespaces.len() >= 8192 { + return Err("Amplifier namespace event budget exceeded".into()); + } + state + .namespaces + .entry(name.into()) + .or_insert((source_base, relative.clone())); + } + } let module_origin = ModuleSourceOrigin { yaml, header_offset, @@ -310,6 +408,8 @@ fn select_local_bundle( .checked_add(source_offset(yaml, node.span.end)?) .ok_or("Amplifier include source offset overflow")?; bundle_replacements.push((start, end, replacement)); + } else { + state.native_includes = true; } } } @@ -337,8 +437,33 @@ fn select_local_bundle( } else { value }; - if include && value.contains(':') { - return Err("Amplifier namespace context requires its retained source-base projection".into()); + if include { + if let Some((namespace, path)) = value.split_once(':') { + state.retain_metadata(value.len())?; + let path = PathBuf::from(path.trim_start_matches('/')); + if namespace.is_empty() + || path.as_os_str().is_empty() + || path.components().any(|component| { + !matches!( + component, + std::path::Component::Normal(_) + | std::path::Component::CurDir + ) + }) + { + return Err( + "Amplifier selected namespace context path is unsupported" + .into(), + ); + } + if state.pending_contexts.len() >= 8192 { + return Err( + "Amplifier namespace context event budget exceeded".into(), + ); + } + state.pending_contexts.push((namespace.into(), path)); + continue; + } } let path = Path::new(value); let selected = if path.is_absolute() { @@ -558,6 +683,150 @@ fn markdown_frontmatter(raw: &str) -> Option<(&str, usize)> { mod module_source_tests { use super::*; + #[test] + fn managed_provider_amplifier_namespace_context_consumes_native_source_bases_after_both_copies() + { + for nested in [false, true] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let bundle = owned.path().join("external-bundle"); + for path in [ + &home.join(".amplifier"), + &cwd, + &bundle.join("behaviors"), + &bundle.join("context"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + let child = bundle.join("behaviors/probe.yaml"); + let raw_child="bundle: {name: probe, namespace_root: ..}\ncontext:\n include: ['probe:context/selected.md']\n"; + let raw = if nested { + format!("bundle: {{name: root}}\nincludes: [{}]\ncontext:\n include: ['probe:context/selected.md']\n",serde_json::to_string(&format!("file://{}",child.display())).unwrap()) + } else { + "bundle: {name: probe}\ncontext:\n include: ['probe:context/selected.md']\n".into() + }; + std::fs::write(bundle.join("bundle.yaml"), &raw).unwrap(); + std::fs::write(&child, raw_child).unwrap(); + std::fs::write( + bundle.join("context/selected.md"), + "selected-namespaced-context", + ) + .unwrap(); + let settings = home.join(".amplifier/settings.yaml"); + let raw_settings = format!( + "bundle: {{active: {}}}\n", + serde_json::to_string(&format!("file://{}", bundle.join("bundle.yaml").display())) + .unwrap() + ); + std::fs::write(&settings, &raw_settings).unwrap(); + let prepared = + prepare(&home, &cwd, &[]).expect("native local namespace context must prepare"); + let stage = owned.path().join("stage"); + let host = owned.path().join("host"); + freshell_platform::provider_environment::copy_environment( + &prepared.selections, + &stage, + &[], + Default::default(), + ) + .unwrap(); + for (relative, raw) in &prepared.rewritten_resources { + std::fs::write(stage.join(relative), raw).unwrap(); + } + let selections = prepared + .selections + .iter() + .map(|selection| CopySelection { + source: stage.join(&selection.destination_relative), + destination_relative: selection.destination_relative.clone(), + }) + .collect::>(); + freshell_platform::provider_environment::copy_environment( + &selections, + &host, + &[], + Default::default(), + ) + .unwrap(); + let child_relative = prepared + .rewritten_resources + .iter() + .find(|(_, raw)| raw.contains("name: probe")) + .unwrap() + .0 + .clone(); + let child_document = MarkedYamlOwned::load_from_str( + &std::fs::read_to_string(host.join(&child_relative)).unwrap(), + ) + .unwrap(); + let namespace_root = child_document[0] + .data + .as_mapping_get("bundle") + .unwrap() + .data + .as_mapping_get("namespace_root") + .and_then(|node| node.data.as_str()) + .unwrap_or("."); + let reference = child_document[0] + .data + .as_mapping_get("context") + .unwrap() + .data + .as_mapping_get("include") + .unwrap() + .data + .as_sequence() + .unwrap()[0] + .data + .as_str() + .unwrap(); + let (namespace, path) = reference.split_once(':').unwrap(); + assert_eq!(namespace, "probe"); + let source_base = host + .join(child_relative.parent().unwrap()) + .join(namespace_root); + let hidden = owned.path().join("hidden-original-bundle"); + std::fs::rename(&bundle, &hidden).unwrap(); + let output=std::process::Command::new("node").args(["-e","process.stdout.write(require('node:fs').readFileSync(require('node:path').join(process.argv[1],process.argv[2])))"]).arg(&source_base).arg(path.trim_start_matches('/')).env_clear().env("PATH",std::env::var_os("PATH").unwrap()).output().unwrap(); + std::fs::rename(&hidden, &bundle).unwrap(); + assert!(output.status.success(),"projected native namespace source base must be readable with original sources hidden: {:?}",output.stderr); + assert_eq!(output.stdout, b"selected-namespaced-context"); + assert_eq!(std::fs::read_to_string(settings).unwrap(), raw_settings); + assert_eq!( + std::fs::read_to_string(bundle.join("bundle.yaml")).unwrap(), + raw + ); + assert_eq!(std::fs::read_to_string(child).unwrap(), raw_child); + } + } + + #[test] + fn managed_provider_amplifier_namespace_context_missing_selected_resource_is_refused() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir_all(home.join(".amplifier")).unwrap(); + let bundle = owned.path().join("bundle.yaml"); + std::fs::write( + &bundle, + "bundle: {name: probe}\ncontext: {include: ['probe:context/missing.md']}\n", + ) + .unwrap(); + std::fs::write( + home.join(".amplifier/settings.yaml"), + format!( + "bundle: {{active: {}}}\n", + serde_json::to_string(&bundle).unwrap() + ), + ) + .unwrap(); + let error = prepare(&home, owned.path(), &[]) + .err() + .expect("missing selected namespace context blocks preparation"); + assert!(error.contains("selected context is missing"), "{error}"); + assert!(!owned.path().join("stage").exists()); + } + #[test] fn managed_provider_amplifier_top_local_forms_consume_modules_context_and_dependencies_after_both_copies( ) { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index b928e8423..e89f44518 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -731,6 +731,8 @@ import hashlib,json,os,resource,secrets,subprocess,urllib.request from pathlib import Path from amplifier_core import HookResult from amplifier_app_cli.session_store import SessionStore +from amplifier_foundation import load_bundle +import yaml _consumer=None async def mount(coordinator,config=None): global _consumer @@ -738,6 +740,10 @@ async def mount(coordinator,config=None): async def consume(event,data): uv=subprocess.check_output(['uv','--version'],text=True).strip() output=subprocess.check_output([str(Path.home()/'.amplifier/skills/probe/run')]) + selected=yaml.safe_load((Path.home()/'.amplifier/settings.yaml').read_text())['bundle']['active'] + loaded=await load_bundle(selected) + loaded.resolve_pending_context() + contexts={name:hashlib.sha256(loaded.context[name].read_bytes()).hexdigest() for name in ['closed-native-bundle:context/self.md','probe:context/selected.md']} request=urllib.request.Request(os.environ['FRESHELL_URL']+'/api/mcp',data=b'{}',headers={'x-auth-token':os.environ['FRESHELL_TOKEN']}) with urllib.request.urlopen(request,timeout=5) as response: authenticated=response.status==200 seed_receipt=Path(config['receipt']).with_suffix('.seed.json') @@ -755,7 +761,7 @@ async def mount(coordinator,config=None): stack=resource.getrlimit(resource.RLIMIT_STACK)[0] profile={'stackAtLeast8MiB':stack==resource.RLIM_INFINITY or stack>=8388608,'aggregateWithin1MiB':len(argv)+len(env)+8*(len(strings)+2)+len(os.readlink('/proc/self/exe').encode())+1<=1048576,'completeStringsWithin32Pages':all(len(value)+1<=131072 for value in strings)} if not all(profile.values()): raise RuntimeError('closed final native execution profile exceeded') - result={'finalExecProfile':profile,'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest()} + result={'finalExecProfile':profile,'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest(),'nativeNamespaceContexts':contexts} Path(config['receipt']).write_text(json.dumps(result)) return HookResult(action='continue') _consumer=consume @@ -780,7 +786,22 @@ async def on_session_ready(coordinator): let native_id = "41a04bb3-86f0-463d-97c5-f085e8a9ea4b"; let hash = |bytes: &[u8]| format!("{:x}", Sha256::digest(bytes)); let conversation_receipt = workspace.join("amplifier-conversation.json"); - let bundle_document=format!("bundle:\n name: closed-native-bundle\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); + std::fs::create_dir_all(bundle.join("behaviors")).unwrap(); + std::fs::create_dir_all(bundle.join("context")).unwrap(); + std::fs::write( + bundle.join("context/self.md"), + "native-self-namespace-context", + ) + .unwrap(); + std::fs::write( + bundle.join("context/selected.md"), + "native-nested-namespace-context", + ) + .unwrap(); + let namespace_child = bundle.join("behaviors/probe.yaml"); + let namespace_child_raw="bundle: {name: probe, namespace_root: ..}\ncontext: {include: ['probe:context/selected.md']}\n"; + std::fs::write(&namespace_child, namespace_child_raw).unwrap(); + let bundle_document=format!("bundle:\n name: closed-native-bundle\nincludes: [{}]\ncontext: {{include: ['closed-native-bundle:context/self.md']}}\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(&format!("file://{}",namespace_child.display())).unwrap(),serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); std::fs::write(bundle.join("bundle.yaml"), &bundle_document).unwrap(); let settings = "bundle:\n active: ../scenarios/external-amplifier-bundle/bundle.yaml\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n".to_owned(); std::fs::write(home.join(".amplifier/settings.yaml"), &settings).unwrap(); @@ -1089,7 +1110,7 @@ async def on_session_ready(coordinator): json!({"provider":"amplifier","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes,"classifiedErrors":classified_errors}) ); } - let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency")}); + let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency"),"nativeNamespaceContexts":{"closed-native-bundle:context/self.md":hash(b"native-self-namespace-context"),"probe:context/selected.md":hash(b"native-nested-namespace-context")}}); assert_eq!(observed.expect("real Amplifier must activate selected local module and executable skill before a model turn"),expected); assert!( !output.exited, From 2cd405178893df99cf212c0ce1eab55bc8678feb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 08:45:42 -0700 Subject: [PATCH 191/493] fix: resolve sound paths before workspace resource classification --- .../src/managed_mcp_capability/source_stage.rs | 6 ++++++ .../managed_provider_references/opencode_resources.rs | 9 ++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 92b7ee411..42708d765 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -143,10 +143,16 @@ mod configured_resource_tests { sound.display().to_string(), reqwest::Url::from_file_path(&sound).unwrap().to_string(), "./selected.wav".into(), + "../external/selected.wav".into(), ] .into_iter() .enumerate() { + let config = if ordinal == 3 { + cwd.parent().unwrap().join("tui.jsonc") + } else { + config.clone() + }; let raw = format!("{{ // selected finite native fields\n\"attention\":{{\"sounds\":{{\"default\":{}}}}},\"mcp\":{{\"probe\":{{\"type\":\"local\",\"command\":[\"node\",\"server.cjs\"],\"cwd\":\"../../external/tool\"}},\"disabled\":{{\"type\":\"local\",\"enabled\":false,\"cwd\":\"/missing-disabled\"}},\"remote\":{{\"type\":\"remote\",\"cwd\":\"/missing-remote\"}}}}}}", serde_json::to_string(&spelling).unwrap()); std::fs::write(&config, &raw).unwrap(); let plan = plan_opencode_sources( diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 84692a9a7..b2ab27ca9 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -484,9 +484,12 @@ pub fn select( .unwrap_or_else(|| source.instance_cwd.join(path)); } } - if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { - // Native instance-cwd resolution normalizes .. before deciding - // whether the selected directory is supplied by the workspace. + if matches!( + field, + "OpenCode skills.paths" | "OpenCode mcp.cwd" | "OpenCode attention.sounds" + ) { + // Native resolution resolution normalizes .. before deciding + // whether the selected resource is supplied by the workspace. path = freshell_platform::opencode_config::lexical_path(&path); } let canonical = checked(&path, field, declaring)?; From 4ff74429cf0fc3dd388e42fc7bc374b6caabb208 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 08:50:33 -0700 Subject: [PATCH 192/493] fix: retain native default for null Amplifier namespace roots --- .../src/managed_mcp_capability/amplifier_resources.rs | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 278b240a5..48addd026 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -303,7 +303,8 @@ fn select_local_bundle( .and_then(|node| node.data.as_str()) .filter(|name| !name.is_empty()) { - let namespace_root = native_mapping_value(meta, "namespace_root", 0)?; + let namespace_root = native_mapping_value(meta, "namespace_root", 0)? + .filter(|node| !node.data.is_null()); let source_base = if let Some(node) = namespace_root { let relative = node .data @@ -686,7 +687,8 @@ mod module_source_tests { #[test] fn managed_provider_amplifier_namespace_context_consumes_native_source_bases_after_both_copies() { - for nested in [false, true] { + for scenario in ["self", "self-null", "nested"] { + let nested = scenario == "nested"; let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let cwd = owned.path().join("workspace"); @@ -704,7 +706,7 @@ mod module_source_tests { let raw = if nested { format!("bundle: {{name: root}}\nincludes: [{}]\ncontext:\n include: ['probe:context/selected.md']\n",serde_json::to_string(&format!("file://{}",child.display())).unwrap()) } else { - "bundle: {name: probe}\ncontext:\n include: ['probe:context/selected.md']\n".into() + format!("bundle: {{name: probe{}}}\ncontext:\n include: ['probe:context/selected.md']\n",if scenario=="self-null" {", namespace_root: null"} else {""}) }; std::fs::write(bundle.join("bundle.yaml"), &raw).unwrap(); std::fs::write(&child, raw_child).unwrap(); From 6c144e1d30033f92b8d3e2a26e8db0c278bf81f2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 10:04:12 -0700 Subject: [PATCH 193/493] fix: preserve native OpenCode TUI and literal tilde resource origins --- .../managed_mcp_capability/source_stage.rs | 138 ++++++++++++++++++ .../src/managed_provider_launch_tests.rs | 25 +++- .../opencode_resources.rs | 20 ++- 3 files changed, 177 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 42708d765..4c30d4110 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -225,6 +225,144 @@ mod configured_resource_tests { } } + #[test] + fn managed_provider_nested_attention_and_literal_tilde_keep_native_origins_after_both_copies() { + for scenario in ["nested", "top-replacement", "tilde"] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let cwd = owned.path().join("external/instance"); + let parent = owned.path().join("external/tui"); + let sound = parent.join(if scenario == "tilde" { + "~/selected.wav" + } else { + "selected.wav" + }); + let tool = cwd.join("~/tool"); + for path in [ + &home, + &cwd, + &parent, + sound.parent().unwrap(), + &tool.join("node_modules/dependency"), + ] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write(&sound, "native-sound").unwrap(); + std::fs::write(home.join("selected.wav"), "wrong-home-sound").unwrap(); + std::fs::create_dir_all(home.join("tool")).unwrap(); + std::fs::write(home.join("tool/resource"), "wrong-home-tool").unwrap(); + std::fs::write(tool.join("resource"), "native-resource").unwrap(); + std::fs::write( + tool.join("node_modules/dependency/index.js"), + "module.exports='native-dependency';", + ) + .unwrap(); + std::fs::write(tool.join("server.cjs"), "const fs=require('node:fs');process.stdout.write(fs.readFileSync('resource','utf8')+':'+require('dependency')+':'+fs.readFileSync(process.argv[2],'utf8'));").unwrap(); + let spelling = if scenario == "tilde" { + "~/selected.wav" + } else { + "./selected.wav" + }; + let attention = serde_json::json!({"sounds":{"default":spelling}}); + let mut document = serde_json::json!({"tui":{"attention":attention},"mcp":{"probe":{"type":"local","command":["node","server.cjs"],"cwd":"~/tool"}}}); + if scenario == "top-replacement" { + document["tui"]["attention"] = + serde_json::json!({"sounds":{"question":"./unselected-missing.wav"}}); + document["attention"] = attention; + } + let raw = document.to_string(); + let config = parent.join("tui.jsonc"); + std::fs::write(&config, &raw).unwrap(); + let make_plan = || { + plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Tui, + role: DiscoveryRole::Workspace, + }], + &cwd, + &home, + &|_, _, _| None, + ) + }; + let plan = + make_plan().expect("effective nested attention and cwd select native origins"); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &owned.path().join("workspace"), + &plan, + None, + ) + .unwrap(); + let sound_unit = units + .iter() + .find(|(source, _)| source == &sound) + .expect("native-origin sound transported"); + let tool_unit = units + .iter() + .find(|(source, _)| source == &tool) + .expect("literal tilde cwd transports complete directory"); + let stage = owned.path().join("stage"); + let host = owned.path().join("host"); + let selections = units + .iter() + .map(|(source, unit)| CopySelection { + source: source.clone(), + destination_relative: unit.stage_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&selections, &stage, &[], Default::default()).unwrap(); + let selections = units + .iter() + .map(|(_, unit)| CopySelection { + source: stage.join(&unit.stage_relative_path), + destination_relative: unit.provider_relative_path.clone().into(), + }) + .collect::>(); + copy_environment(&selections, &host, &[], Default::default()).unwrap(); + let hidden = owned.path().join("hidden-external"); + std::fs::rename(owned.path().join("external"), &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg("server.cjs") + .arg(host.join(&sound_unit.1.provider_relative_path)) + .current_dir(host.join(&tool_unit.1.provider_relative_path)) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, owned.path().join("external")).unwrap(); + assert!( + output.status.success(), + "native-origin two-copy executable consumer failed: {:?}", + output.stderr + ); + assert_eq!( + output.stdout, + b"native-resource:native-dependency:native-sound" + ); + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + std::fs::remove_file(&sound).unwrap(); + assert_eq!( + make_plan() + .err() + .expect("missing native sound must refuse despite HOME decoy") + .field, + "OpenCode attention.sounds" + ); + std::fs::write(&sound, "native-sound").unwrap(); + std::fs::rename(&tool, owned.path().join("hidden-tool")).unwrap(); + assert_eq!( + make_plan() + .err() + .expect("missing native cwd must refuse despite HOME decoy") + .field, + "OpenCode mcp.cwd" + ); + } + } + #[test] fn managed_provider_sound_and_mcp_cwd_refuse_unavailable_selected_resources() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index e89f44518..5d7b6f729 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3506,7 +3506,13 @@ export default {{id:'closed-shared-{label}',async tui(api,options){{ std::fs::Permissions::from_mode(0o755), ) .unwrap(); - let mcp_cwd = external.join("mcp-cwd"); + let mcp_cwd = nested.join("~/mcp-cwd"); + std::fs::create_dir_all(fixture.join("source-home/mcp-cwd")).unwrap(); + std::fs::write( + fixture.join("source-home/mcp-cwd/resource"), + "wrong-home-cwd-decoy", + ) + .unwrap(); std::fs::create_dir_all(mcp_cwd.join("node_modules/native-mcp-dependency")).unwrap(); std::fs::write(mcp_cwd.join("resource"), "native-mcp-resource").unwrap(); std::fs::write( @@ -3542,7 +3548,7 @@ export default async (input,options)=>({config:async(config)=>{ serde_json::from_slice(&std::fs::read(&original_config).unwrap()).unwrap(); original["plugin"].as_array_mut().unwrap().push(json!([reqwest::Url::from_file_path(package.join("index.mjs")).unwrap().to_string(), {"receipt":external_receipt,"instruction":external_instruction,"mcpReceipt":mcp_cwd_receipt}])); original["mcp"]["closed-cwd-probe"] = - json!({"type":"local","command":["node","server.cjs"],"cwd":mcp_cwd}); + json!({"type":"local","command":["node","server.cjs"],"cwd":"~/mcp-cwd"}); original["instructions"] = json!([external_instruction]); original["skills"] = json!({"paths":[selected_skills,"../../scenarios/external-native-resources/selected-skills"]}); std::fs::write(&original_config, serde_json::to_vec(&original).unwrap()).unwrap(); @@ -3567,9 +3573,20 @@ export default {id:'closed-native-tui-fixture',async tui(api,options){ let project_tui = fixture.join("workspace/tui.jsonc"); let tui_receipt = fixture.join("workspace/native-tui-consumed.json"); let selected_sound = external.join("selected.wav"); + let tilde_sound = fixture.join("workspace/~/selected.wav"); + std::fs::create_dir_all(tilde_sound.parent().unwrap()).unwrap(); + std::fs::write(&tilde_sound, b"RIFF-native-literal-tilde-wave").unwrap(); + std::fs::write( + fixture.join("source-home/selected.wav"), + b"wrong-home-sound-decoy", + ) + .unwrap(); std::fs::write(&selected_sound, b"RIFF-native-selected-wave").unwrap(); - let selected_tui_bytes = format!("{{ // selected origin\n\"attention\":{{\"sounds\":{{\"default\":{},\"question\":{},\"error\":\"../scenarios/external-native-resources/selected.wav\"}}}},\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&selected_sound).unwrap(),serde_json::to_string(&reqwest::Url::from_file_path(&selected_sound).unwrap().to_string()).unwrap(),serde_json::to_string(&tui_receipt).unwrap()); + let selected_tui_bytes = format!("{{ // selected origin\n\"tui\":{{\"attention\":{{\"sounds\":{{\"default\":{},\"question\":{},\"error\":\"~/selected.wav\"}}}}}},\"theme\":\"{{file:./selected-theme}}\",\"plugin\":[[\"file:///home/freshell/provider/.config/opencode/plugins/probe-tui/index.mjs\",{{\"receipt\":{}}}]],}}", serde_json::to_string(&selected_sound).unwrap(),serde_json::to_string(&reqwest::Url::from_file_path(&selected_sound).unwrap().to_string()).unwrap(),serde_json::to_string(&tui_receipt).unwrap()); std::fs::write(&project_tui, &selected_tui_bytes).unwrap(); + std::fs::write(fixture.join("source-home/.config/opencode/tui.json"), + serde_json::to_vec(&json!({"tui":{"attention":{"sounds":{"subagent_done":"/missing-replaced-sound"}}},"attention":{"sounds":{"default":selected_sound}}})).unwrap()).unwrap(); + std::fs::write(fixture.join("workspace/selected-theme"), " dark\n").unwrap(); let inline_source = r#"{ // inline instance origin "agent":{"probe":{"description":"{env:INLINE_VALUE}"}}}"#; @@ -3937,7 +3954,7 @@ export default {{id:'closed-directory-discovery',async server(){{ read_receipt(&tui_receipt) .await .expect("actual TUI plugin must consume the selected effective TUI configuration"), - json!({"theme":format!("{:x}",Sha256::digest(b"dark")),"sounds":vec![format!("{:x}",Sha256::digest(b"RIFF-native-selected-wave"));3]}) + json!({"theme":format!("{:x}",Sha256::digest(b"dark")),"sounds":[format!("{:x}",Sha256::digest(b"RIFF-native-selected-wave")),format!("{:x}",Sha256::digest(b"RIFF-native-selected-wave")),format!("{:x}",Sha256::digest(b"RIFF-native-literal-tilde-wave"))]}) ); let ca_digest = format!( "{:x}", diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index b2ab27ca9..921d661e8 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -211,7 +211,16 @@ fn nested_fields(raw: &str, tui: bool) -> Result, &' let top = object_members(raw, 0)?; let mut result = Vec::new(); if tui { - if let Some(attention) = member(raw, &top, "attention") { + // Native TUI normalization is shallow: any present top-level attention + // replaces the entire nested key, including non-object values. + let attention = if let Some(attention) = member(raw, &top, "attention") { + Some(attention) + } else if let Some(tui) = member(raw, &top, "tui") { + member(tui, &object_members(tui, 0)?, "attention") + } else { + None + }; + if let Some(attention) = attention { let entries = object_members(attention, 0)?; if let Some(sounds) = member(attention, &entries, "sounds") { let entries = object_members(sounds, 0)?; @@ -403,10 +412,17 @@ pub fn select( || spelling.starts_with(['/', '.']) || spelling.starts_with("~/") { - home_path(&spelling, home) + if field == "OpenCode attention.sounds" { + PathBuf::from(&spelling) + } else { + home_path(&spelling, home) + } } else { continue; } // Native package identity remains native. + } else if field == "OpenCode mcp.cwd" { + // Native path.resolve(instance cwd, raw): no HOME or URL expansion. + PathBuf::from(&spelling) } else { home_path(&spelling, home) }; From 0731b4538364c629dbfce8eaf201c215f47cd221 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 10:10:24 -0700 Subject: [PATCH 194/493] fix: qualify Amplifier namespace roots by native bundle ancestry --- .../amplifier_resources.rs | 125 +++++++++++++++++- .../src/managed_provider_launch_tests.rs | 8 +- 2 files changed, 127 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 48addd026..8dac32cba 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -83,11 +83,24 @@ struct BundleSelection { completed: BTreeMap, modules: BTreeMap, namespaces: BTreeMap, + bundle_names: BTreeMap, pending_contexts: Vec<(String, PathBuf)>, native_includes: bool, } impl BundleSelection { + fn remember_bundle_name(&mut self, root: &Path, name: &str) -> Result<(), String> { + if !self.bundle_names.contains_key(root) { + if self.bundle_names.len() >= 8192 { + return Err("Amplifier namespace event budget exceeded".into()); + } + self.retain_metadata(root.as_os_str().len())?; + self.retain_metadata(name.len())?; + self.bundle_names.insert(root.to_owned(), name.to_owned()); + } + Ok(()) + } + fn retain_metadata(&mut self, bytes: usize) -> Result<(), String> { self.metadata_bytes = self .metadata_bytes @@ -143,6 +156,83 @@ fn select_namespace_contexts( } Ok(()) } +// FileSourceHandler first finds the nearest native bundle entry starting at +// base_path. An entry in base_path is the source boundary, so the registry's +// ancestor search cannot select its parent. Otherwise the nearest ancestor is +// the finite local ROOT whose different name activates namespace_root. +fn has_different_native_root( + root: &Path, + name: &str, + excluded: &[PathBuf], + state: &mut BundleSelection, +) -> Result { + for (depth, directory) in root.ancestors().enumerate() { + if depth >= 64 { + return Err("Amplifier local ancestor depth profile exceeded".into()); + } + for leaf in ["bundle.md", "bundle.yaml"] { + let path = directory.join(leaf); + match std::fs::metadata(&path) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => return Err("Amplifier ancestor bundle metadata unavailable".into()), + Ok(_) if directory == root => return Ok(false), + Ok(metadata) if !metadata.is_file() => { + return Err("Amplifier ancestor bundle is not a regular file".into()) + } + Ok(_) => {} + } + let canonical = std::fs::canonicalize(&path) + .map_err(|_| "Amplifier ancestor bundle unavailable or cyclic")?; + if let Some(ancestor_name) = state.bundle_names.get(&canonical) { + return Ok(!ancestor_name.is_empty() && ancestor_name != name); + } + if excluded.iter().any(|path| canonical.starts_with(path)) { + return Err("Amplifier ancestor bundle aliases credentials or native state".into()); + } + let limit = (64 * 1024 * 1024usize) + .checked_sub(state.view_bytes) + .ok_or("Amplifier retained source view size overflow")? + .min(16 * 1024 * 1024); + let mut raw = String::new(); + std::fs::File::open(&canonical) + .map_err(|_| "Amplifier ancestor bundle unreadable")? + .take(limit as u64 + 1) + .read_to_string(&mut raw) + .map_err(|_| "Amplifier ancestor bundle unreadable or not UTF-8")?; + if raw.len() > limit { + return Err("Amplifier ancestor bundle exceeds shared source-view profile".into()); + } + state.view_bytes = state + .view_bytes + .checked_add(raw.len()) + .ok_or("Amplifier retained source view size overflow")?; + let yaml = if leaf == "bundle.md" { + markdown_frontmatter(&raw).map(|(yaml, _)| yaml) + } else { + Some(raw.as_str()) + }; + let Some(yaml) = yaml else { + return Ok(false); + }; + bound_yaml_expansion(yaml)?; + let documents = MarkedYamlOwned::load_from_str(yaml) + .map_err(|_| "Amplifier ancestor bundle YAML is invalid")?; + if documents.len() != 1 { + return Err("Amplifier ancestor bundle requires one YAML document".into()); + } + let ancestor_name = native_mapping_value(&documents[0], "bundle", 0)? + .map(|meta| native_mapping_value(meta, "name", 0)) + .transpose()? + .flatten() + .and_then(|node| node.data.as_str()) + .unwrap_or(""); + state.remember_bundle_name(&canonical, ancestor_name)?; + return Ok(!ancestor_name.is_empty() && ancestor_name != name); + } + } + Ok(false) +} + fn select_local_bundle( source: &str, cwd: &Path, @@ -305,6 +395,16 @@ fn select_local_bundle( { let namespace_root = native_mapping_value(meta, "namespace_root", 0)? .filter(|node| !node.data.is_null()); + state.remember_bundle_name(&document, name)?; + // Native standalone self-name resolution uses base_path. + // Only a different named ancestor enables namespace_root. + let namespace_root = if namespace_root.is_some() + && has_different_native_root(root, name, excluded, state)? + { + namespace_root + } else { + None + }; let source_base = if let Some(node) = namespace_root { let relative = node .data @@ -687,7 +787,7 @@ mod module_source_tests { #[test] fn managed_provider_amplifier_namespace_context_consumes_native_source_bases_after_both_copies() { - for scenario in ["self", "self-null", "nested"] { + for scenario in ["self", "self-null", "self-override", "nested"] { let nested = scenario == "nested"; let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); @@ -706,7 +806,7 @@ mod module_source_tests { let raw = if nested { format!("bundle: {{name: root}}\nincludes: [{}]\ncontext:\n include: ['probe:context/selected.md']\n",serde_json::to_string(&format!("file://{}",child.display())).unwrap()) } else { - format!("bundle: {{name: probe{}}}\ncontext:\n include: ['probe:context/selected.md']\n",if scenario=="self-null" {", namespace_root: null"} else {""}) + format!("bundle: {{name: probe{}}}\ncontext:\n include: ['probe:context/selected.md']\n",if scenario=="self-null" {", namespace_root: null"} else if scenario=="self-override" {", namespace_root: .."} else {""}) }; std::fs::write(bundle.join("bundle.yaml"), &raw).unwrap(); std::fs::write(&child, raw_child).unwrap(); @@ -715,6 +815,12 @@ mod module_source_tests { "selected-namespaced-context", ) .unwrap(); + std::fs::create_dir_all(owned.path().join("context")).unwrap(); + std::fs::write( + owned.path().join("context/selected.md"), + "wrong-parent-decoy", + ) + .unwrap(); let settings = home.join(".amplifier/settings.yaml"); let raw_settings = format!( "bundle: {{active: {}}}\n", @@ -785,9 +891,11 @@ mod module_source_tests { .unwrap(); let (namespace, path) = reference.split_once(':').unwrap(); assert_eq!(namespace, "probe"); - let source_base = host - .join(child_relative.parent().unwrap()) - .join(namespace_root); + let source_base = host.join(child_relative.parent().unwrap()).join(if nested { + namespace_root + } else { + "." + }); let hidden = owned.path().join("hidden-original-bundle"); std::fs::rename(&bundle, &hidden).unwrap(); let output=std::process::Command::new("node").args(["-e","process.stdout.write(require('node:fs').readFileSync(require('node:path').join(process.argv[1],process.argv[2])))"]).arg(&source_base).arg(path.trim_start_matches('/')).env_clear().env("PATH",std::env::var_os("PATH").unwrap()).output().unwrap(); @@ -800,6 +908,13 @@ mod module_source_tests { raw ); assert_eq!(std::fs::read_to_string(child).unwrap(), raw_child); + if scenario == "self-override" { + std::fs::remove_file(bundle.join("context/selected.md")).unwrap(); + assert!(prepare(&home, &cwd, &[]) + .err() + .expect("missing standalone context must refuse despite parent decoy") + .contains("selected context is missing")); + } } } diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 5d7b6f729..04c8ae2bd 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -798,10 +798,16 @@ async def on_session_ready(coordinator): "native-nested-namespace-context", ) .unwrap(); + std::fs::create_dir_all(bundle.parent().unwrap().join("context")).unwrap(); + std::fs::write( + bundle.parent().unwrap().join("context/self.md"), + "wrong-standalone-parent-context", + ) + .unwrap(); let namespace_child = bundle.join("behaviors/probe.yaml"); let namespace_child_raw="bundle: {name: probe, namespace_root: ..}\ncontext: {include: ['probe:context/selected.md']}\n"; std::fs::write(&namespace_child, namespace_child_raw).unwrap(); - let bundle_document=format!("bundle:\n name: closed-native-bundle\nincludes: [{}]\ncontext: {{include: ['closed-native-bundle:context/self.md']}}\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(&format!("file://{}",namespace_child.display())).unwrap(),serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); + let bundle_document=format!("bundle:\n name: closed-native-bundle\n namespace_root: ..\nincludes: [{}]\ncontext: {{include: ['closed-native-bundle:context/self.md']}}\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(&format!("file://{}",namespace_child.display())).unwrap(),serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); std::fs::write(bundle.join("bundle.yaml"), &bundle_document).unwrap(); let settings = "bundle:\n active: ../scenarios/external-amplifier-bundle/bundle.yaml\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n".to_owned(); std::fs::write(home.join(".amplifier/settings.yaml"), &settings).unwrap(); From 7eb7f4ce4822124c1c2e11bff66789f520a6f94f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 11:18:44 -0700 Subject: [PATCH 195/493] fix: preserve Amplifier ancestor namespace registration --- .../amplifier_resources.rs | 183 +++++++++++++++--- .../src/managed_provider_launch_tests.rs | 34 +++- 2 files changed, 186 insertions(+), 31 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index 8dac32cba..d322e3f46 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -89,6 +89,26 @@ struct BundleSelection { } impl BundleSelection { + fn remember_namespace( + &mut self, + name: &str, + source: &Path, + destination: &Path, + ) -> Result<(), String> { + if self.namespaces.contains_key(name) { + return Ok(()); + } + if self.namespaces.len() >= 8192 { + return Err("Amplifier namespace event budget exceeded".into()); + } + self.retain_metadata(name.len())?; + self.retain_metadata(source.as_os_str().len())?; + self.retain_metadata(destination.as_os_str().len())?; + self.namespaces + .insert(name.into(), (source.to_owned(), destination.to_owned())); + Ok(()) + } + fn remember_bundle_name(&mut self, root: &Path, name: &str) -> Result<(), String> { if !self.bundle_names.contains_key(root) { if self.bundle_names.len() >= 8192 { @@ -159,13 +179,12 @@ fn select_namespace_contexts( // FileSourceHandler first finds the nearest native bundle entry starting at // base_path. An entry in base_path is the source boundary, so the registry's // ancestor search cannot select its parent. Otherwise the nearest ancestor is -// the finite local ROOT whose different name activates namespace_root. -fn has_different_native_root( +// the finite local ROOT. Its registration and the child override are separate. +fn native_ancestor_root( root: &Path, - name: &str, excluded: &[PathBuf], state: &mut BundleSelection, -) -> Result { +) -> Result, String> { for (depth, directory) in root.ancestors().enumerate() { if depth >= 64 { return Err("Amplifier local ancestor depth profile exceeded".into()); @@ -175,7 +194,7 @@ fn has_different_native_root( match std::fs::metadata(&path) { Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, Err(_) => return Err("Amplifier ancestor bundle metadata unavailable".into()), - Ok(_) if directory == root => return Ok(false), + Ok(_) if directory == root => return Ok(None), Ok(metadata) if !metadata.is_file() => { return Err("Amplifier ancestor bundle is not a regular file".into()) } @@ -184,7 +203,7 @@ fn has_different_native_root( let canonical = std::fs::canonicalize(&path) .map_err(|_| "Amplifier ancestor bundle unavailable or cyclic")?; if let Some(ancestor_name) = state.bundle_names.get(&canonical) { - return Ok(!ancestor_name.is_empty() && ancestor_name != name); + return Ok(Some((directory.to_owned(), ancestor_name.clone()))); } if excluded.iter().any(|path| canonical.starts_with(path)) { return Err("Amplifier ancestor bundle aliases credentials or native state".into()); @@ -212,7 +231,7 @@ fn has_different_native_root( Some(raw.as_str()) }; let Some(yaml) = yaml else { - return Ok(false); + return Ok(None); }; bound_yaml_expansion(yaml)?; let documents = MarkedYamlOwned::load_from_str(yaml) @@ -227,10 +246,10 @@ fn has_different_native_root( .and_then(|node| node.data.as_str()) .unwrap_or(""); state.remember_bundle_name(&canonical, ancestor_name)?; - return Ok(!ancestor_name.is_empty() && ancestor_name != name); + return Ok(Some((directory.to_owned(), ancestor_name.to_owned()))); } } - Ok(false) + Ok(None) } fn select_local_bundle( @@ -396,15 +415,20 @@ fn select_local_bundle( let namespace_root = native_mapping_value(meta, "namespace_root", 0)? .filter(|node| !node.data.is_null()); state.remember_bundle_name(&document, name)?; - // Native standalone self-name resolution uses base_path. - // Only a different named ancestor enables namespace_root. - let namespace_root = if namespace_root.is_some() - && has_different_native_root(root, name, excluded, state)? - { - namespace_root - } else { - None - }; + let ancestor = native_ancestor_root(root, excluded, state)?; + // A named ROOT registers source_root even when its name + // equals the selected child. An unnamed ROOT still permits + // the different-named child's namespace_root override. + if let Some((source_root, root_name)) = &ancestor { + if !root_name.is_empty() { + state.remember_namespace(root_name, source_root, &relative)?; + } + } + let namespace_root = namespace_root.filter(|_| { + ancestor + .as_ref() + .is_some_and(|(_, root_name)| root_name != name) + }); let source_base = if let Some(node) = namespace_root { let relative = node .data @@ -430,16 +454,7 @@ fn select_local_bundle( } else { root.to_owned() }; - state.retain_metadata(name.len())?; - state.retain_metadata(source_base.as_os_str().len())?; - state.retain_metadata(relative.as_os_str().len())?; - if state.namespaces.len() >= 8192 { - return Err("Amplifier namespace event budget exceeded".into()); - } - state - .namespaces - .entry(name.into()) - .or_insert((source_base, relative.clone())); + state.remember_namespace(name, &source_base, &relative)?; } } let module_origin = ModuleSourceOrigin { @@ -918,6 +933,118 @@ mod module_source_tests { } } + #[test] + fn managed_provider_amplifier_ancestor_names_consume_parent_context_after_both_copies() { + for (ancestor_name, namespace_root) in [("probe", true), ("probe", false), ("", true)] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let bundle = owned.path().join("external-bundle"); + let cwd = owned.path().join("workspace"); + for directory in [ + &home.join(".amplifier"), + &cwd, + &bundle.join("context"), + &bundle.join("behaviors/context"), + ] { + std::fs::create_dir_all(directory).unwrap(); + } + let parent_raw = format!("bundle: {{name: {ancestor_name:?}}}\n"); + let child_raw = format!( + "bundle: {{name: probe{}}}\ncontext: {{include: ['probe:context/selected.md']}}\n", + if namespace_root { + ", namespace_root: .." + } else { + "" + } + ); + std::fs::write(bundle.join("bundle.yaml"), &parent_raw).unwrap(); + let child = bundle.join("behaviors/probe.yaml"); + std::fs::write(&child, &child_raw).unwrap(); + std::fs::write( + bundle.join("context/selected.md"), + "correct-ancestor-context", + ) + .unwrap(); + std::fs::write( + bundle.join("behaviors/context/selected.md"), + "wrong-child-decoy", + ) + .unwrap(); + let settings_raw = format!( + "bundle: {{active: {}}}\n", + serde_json::to_string(&format!("file://{}", child.display())).unwrap() + ); + let settings = home.join(".amplifier/settings.yaml"); + std::fs::write(&settings, &settings_raw).unwrap(); + let prepared = prepare(&home, &cwd, &[]).unwrap(); + let stage = owned.path().join("stage"); + let host = owned.path().join("host"); + freshell_platform::provider_environment::copy_environment( + &prepared.selections, + &stage, + &[], + Default::default(), + ) + .unwrap(); + for (relative, raw) in &prepared.rewritten_resources { + std::fs::write(stage.join(relative), raw).unwrap(); + } + let second = prepared + .selections + .iter() + .map(|selection| CopySelection { + source: stage.join(&selection.destination_relative), + destination_relative: selection.destination_relative.clone(), + }) + .collect::>(); + freshell_platform::provider_environment::copy_environment( + &second, + &host, + &[], + Default::default(), + ) + .unwrap(); + let projected_child = &prepared + .rewritten_resources + .iter() + .find(|(_, raw)| raw.contains("name: probe")) + .unwrap() + .0; + let projected_resource = host + .join(projected_child.parent().unwrap()) + .join("context/selected.md"); + let hidden = owned.path().join("hidden-original"); + std::fs::rename(&bundle, &hidden).unwrap(); + let output = std::process::Command::new("node") + .args([ + "-e", + "process.stdout.write(require('node:fs').readFileSync(process.argv[1]))", + ]) + .arg(projected_resource) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, &bundle).unwrap(); + assert!(output.status.success()); + assert_eq!( + output.stdout, b"correct-ancestor-context", + "ancestor={ancestor_name:?}, override={namespace_root}" + ); + assert_eq!(std::fs::read_to_string(&settings).unwrap(), settings_raw); + assert_eq!(std::fs::read_to_string(&child).unwrap(), child_raw); + assert_eq!( + std::fs::read_to_string(bundle.join("bundle.yaml")).unwrap(), + parent_raw + ); + std::fs::remove_file(bundle.join("context/selected.md")).unwrap(); + assert!(prepare(&home, &cwd, &[]) + .err() + .expect("missing ancestor target must refuse despite child decoy") + .contains("selected context is missing")); + } + } + #[test] fn managed_provider_amplifier_namespace_context_missing_selected_resource_is_refused() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 04c8ae2bd..da13a1d8b 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -743,7 +743,7 @@ async def mount(coordinator,config=None): selected=yaml.safe_load((Path.home()/'.amplifier/settings.yaml').read_text())['bundle']['active'] loaded=await load_bundle(selected) loaded.resolve_pending_context() - contexts={name:hashlib.sha256(loaded.context[name].read_bytes()).hexdigest() for name in ['closed-native-bundle:context/self.md','probe:context/selected.md']} + contexts={name:hashlib.sha256(loaded.context[name].read_bytes()).hexdigest() for name in ['closed-native-bundle:context/self.md','probe:context/selected.md','same-root:context/selected.md','same-no-root:context/selected.md','unnamed-root:context/selected.md']} request=urllib.request.Request(os.environ['FRESHELL_URL']+'/api/mcp',data=b'{}',headers={'x-auth-token':os.environ['FRESHELL_TOKEN']}) with urllib.request.urlopen(request,timeout=5) as response: authenticated=response.status==200 seed_receipt=Path(config['receipt']).with_suffix('.seed.json') @@ -807,7 +807,35 @@ async def on_session_ready(coordinator): let namespace_child = bundle.join("behaviors/probe.yaml"); let namespace_child_raw="bundle: {name: probe, namespace_root: ..}\ncontext: {include: ['probe:context/selected.md']}\n"; std::fs::write(&namespace_child, namespace_child_raw).unwrap(); - let bundle_document=format!("bundle:\n name: closed-native-bundle\n namespace_root: ..\nincludes: [{}]\ncontext: {{include: ['closed-native-bundle:context/self.md']}}\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(&format!("file://{}",namespace_child.display())).unwrap(),serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); + let mut namespace_includes = vec![format!("file://{}", namespace_child.display())]; + for (name, ancestor_name, override_root) in [ + ("same-root", "same-root", true), + ("same-no-root", "same-no-root", false), + ("unnamed-root", "", true), + ] { + let root = fixture.join(format!("scenarios/{name}")); + std::fs::create_dir_all(root.join("context")).unwrap(); + std::fs::create_dir_all(root.join("behaviors/context")).unwrap(); + std::fs::write( + root.join("bundle.yaml"), + format!("bundle: {{name: {ancestor_name:?}}}\n"), + ) + .unwrap(); + let child = root.join("behaviors/probe.yaml"); + std::fs::write(&child, format!("bundle: {{name: {name}{}}}\ncontext: {{include: ['{name}:context/selected.md']}}\n", if override_root {", namespace_root: .."} else {""})).unwrap(); + std::fs::write( + root.join("context/selected.md"), + format!("native-{name}-ancestor-context"), + ) + .unwrap(); + std::fs::write( + root.join("behaviors/context/selected.md"), + "wrong-child-context-decoy", + ) + .unwrap(); + namespace_includes.push(format!("file://{}", child.display())); + } + let bundle_document=format!("bundle:\n name: closed-native-bundle\n namespace_root: ..\nincludes: {}\ncontext: {{include: ['closed-native-bundle:context/self.md']}}\nsession:\n orchestrator:\n module: loop-probe\n source: ./modules/loop-probe\n context:\n module: context-probe\n source: ./modules/context-probe\n config:\n conversation_receipt: {}\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\nhooks:\n - module: hooks-probe\n source: ./modules/hooks-probe\n config:\n receipt: {}\n session_id: {}\n key_digest: {}\n",serde_json::to_string(&namespace_includes).unwrap(),serde_json::to_string(conversation_receipt.to_str().unwrap()).unwrap(),serde_json::to_string(receipt.to_str().unwrap()).unwrap(),native_id,hash(b"synthetic-amplifier-onecli-value")); std::fs::write(bundle.join("bundle.yaml"), &bundle_document).unwrap(); let settings = "bundle:\n active: ../scenarios/external-amplifier-bundle/bundle.yaml\nproviders:\n - module: provider-vllm\n config:\n base_url: http://127.0.0.1:9\n default_model: closed-model-free\n".to_owned(); std::fs::write(home.join(".amplifier/settings.yaml"), &settings).unwrap(); @@ -1116,7 +1144,7 @@ async def on_session_ready(coordinator): json!({"provider":"amplifier","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes,"classifiedErrors":classified_errors}) ); } - let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency"),"nativeNamespaceContexts":{"closed-native-bundle:context/self.md":hash(b"native-self-namespace-context"),"probe:context/selected.md":hash(b"native-nested-namespace-context")}}); + let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency"),"nativeNamespaceContexts":{"closed-native-bundle:context/self.md":hash(b"native-self-namespace-context"),"probe:context/selected.md":hash(b"native-nested-namespace-context"),"same-root:context/selected.md":hash(b"native-same-root-ancestor-context"),"same-no-root:context/selected.md":hash(b"native-same-no-root-ancestor-context"),"unnamed-root:context/selected.md":hash(b"native-unnamed-root-ancestor-context")}}); assert_eq!(observed.expect("real Amplifier must activate selected local module and executable skill before a model turn"),expected); assert!( !output.exited, From 46ac23b1da52c788610ca88cceceddd4edbce45c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 11:46:10 -0700 Subject: [PATCH 196/493] fix: replan undispatched initial OpenCode inputs within two retries --- crates/freshell-runtime-protocol/src/lib.rs | 8 + .../freshell-server/src/fresh_agent_proxy.rs | 57 +++- .../src/managed_mcp_capability.rs | 20 ++ .../src/managed_provider_launch_tests.rs | 10 +- .../src/managed_provider_references.rs | 77 +++++ crates/freshell-server/src/managed_runtime.rs | 63 +++- .../managed_runtime_opencode_replan_tests.rs | 144 +++++++++ crates/freshell-supervisor/src/backend.rs | 17 ++ .../claude_source_observation_tests.rs | 4 + .../src/backend/input_publisher.rs | 87 ++++-- .../src/backend/native_auth.rs | 138 +++++++-- .../src/provider_preparation_tests.rs | 284 +++++++++++++++++- .../src/registry/input_reservation.rs | 74 ++++- crates/freshell-supervisor/src/service.rs | 170 ++++++++++- 14 files changed, 1070 insertions(+), 83 deletions(-) create mode 100644 crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 0a669d187..0c5f49b7b 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2864,6 +2864,14 @@ pub struct ProviderInputReservation { #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)] pub enum InitialAuthVolumeObservation { + /// The exact creation-owned volume after a positively rejected initial + /// grant. This is retained ownership evidence, not a claim of absence. + OwnedUndispatched { + #[serde(rename = "daemonId")] + daemon_id: DockerDaemonId, + #[serde(rename = "incarnationId")] + incarnation_id: IncarnationId, + }, Absent { #[serde(rename = "daemonId")] daemon_id: DockerDaemonId, diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 4d19fed92..e86c49a2d 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -645,15 +645,58 @@ impl HostedFreshAgentProxy { }), expected_control_epoch: None, }; - let launched = self - .client - .launch(request_id, request) + let launched = if provider == FreshProvider::Opencode { + let home = std::env::var_os("HOME").map(PathBuf::from).ok_or(())?; + let original = |name: &str| std::env::var(name).ok(); + let references = request + .fresh_agent + .as_ref() + .unwrap() + .provider_secret_references + .iter() + .filter(|r| !r.profile.is_configuration_file()) + .cloned() + .collect(); + crate::managed_provider_references::launch_initial_opencode( + crate::managed_provider_references::OpencodePreparation { + client: &self.client, + soul: &soul, + creation_seed: &message.request_id, + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &original, + references, + auth_authority: None, + selected_sources: None, + accepted_global_root: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: Some(freshell_platform::opencode_config::OwnedConfigEdits { + disable_snapshots: true, + ..Default::default() + }), + }, + request_id, + request, + ) .await .map_err(|error| { - let _ = - crate::managed_mcp_capability::revoke_mcp_capability(&mcp_capability.grant_id); - tracing::warn!(?error, "fresh_agent.create.launch_failed"); - })?; + tracing::warn!(event="fresh_agent.create.initial_opencode_failed",%error); + })? + } else { + self.client + .launch(request_id, request) + .await + .map_err(|error| { + let _ = crate::managed_mcp_capability::revoke_mcp_capability( + &mcp_capability.grant_id, + ); + tracing::warn!(?error, "fresh_agent.create.launch_failed"); + })? + }; let naming_projection = match message.naming_handle.as_deref() { Some(handle) if !handle.trim().is_empty() => { admit_pending_projection( diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 940f7f627..7be3b95d6 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -127,6 +127,26 @@ fn endpoint() -> Result { .ok_or("managed MCP controller is unconfigured".into()) } +#[cfg(test)] +pub(crate) fn accept_initial_replan_fixture_preparation( + soul: &SoulId, + context: &freshell_runtime_protocol::ProviderLaunchContext, +) -> Result<(), String> { + let capability = context + .mcp_capability + .as_ref() + .ok_or("fixture capability missing")?; + let incarnation = IncarnationId::new(); + store().activate( + &capability.grant_id, + soul.clone(), + incarnation.clone(), + &capability_dir()?, + &endpoint()?, + )?; + store().commit_preparation(&capability.grant_id, soul, &incarnation, &capability_dir()?) +} + pub(crate) fn configure( bind_host: &str, port: u16, diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index da13a1d8b..c4b70bb93 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -141,7 +141,9 @@ impl BrokerAuthorizedBackend { } let (daemon_id, reservation_id, retained) = match owner { freshell_supervisor::backend::ClaudeSourceOwner::Initial { observation, .. } => { - let InitialAuthVolumeObservation::Absent { daemon_id } = observation; + let InitialAuthVolumeObservation::Absent { daemon_id } = observation else { + return Err(invalid()); + }; let ClaudeSourceAuthority::InitialReservation { reservation_id } = &request.authority else { @@ -415,6 +417,12 @@ impl RuntimeBackend for BrokerAuthorizedBackend { ) -> Result { self.real().observe_initial_auth_volume(reservation).await } + async fn discard_rejected_provider_inputs( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + self.real().discard_rejected_provider_inputs(handle).await + } async fn commit_prepared_inputs( &self, handle: &OwnedRuntimeHandle, diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index ba0ab8760..c69830f1d 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -1369,3 +1369,80 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( references, }) } + +/// Rebuild only a positively rejected, never-dispatched initial OpenCode input +/// generation. Other failures retain their inputs for ownership reconciliation. +pub(crate) async fn launch_initial_opencode( + preparation: OpencodePreparation<'_>, + mut request_id: freshell_runtime_protocol::RequestId, + mut launch: freshell_runtime_protocol::LaunchRequest, +) -> Result { + use freshell_runtime_protocol::RuntimeErrorCode; + let mut retries = 0; + loop { + match preparation + .client + .launch(request_id.clone(), launch.clone()) + .await + { + Ok(result) => return Ok(result), + Err(error) + if error.runtime_code() == Some(RuntimeErrorCode::ProviderInputsChanged) + && launch.native_session_id.is_none() => + { + let context = launch + .terminal + .as_ref() + .and_then(|s| s.provider_launch_context.as_ref()) + .or_else(|| { + launch + .fresh_agent + .as_ref() + .and_then(|s| s.provider_launch_context.as_ref()) + }); + if let Some(capability) = context.and_then(|c| c.mcp_capability.as_ref()) { + crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id)?; + } + if retries == 2 { + tracing::warn!(event="provider.inputs.initial_replan.exhausted", provider="opencode", soul_id=%launch.soul_id, retries); + return Err("OpenCode configuration reference selection kept changing; two pre-dispatch retries exhausted".into()); + } + retries += 1; + tracing::info!(event="provider.inputs.initial_replan", provider="opencode", soul_id=%launch.soul_id, retry=retries); + let prepared = prepare_managed_opencode(OpencodePreparation { + client: preparation.client, + soul: preparation.soul, + creation_seed: preparation.creation_seed, + home: preparation.home, + cwd: preparation.cwd, + workspace: preparation.workspace, + original: preparation.original, + references: preparation.references.clone(), + auth_authority: None, + selected_sources: None, + accepted_global_root: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: preparation.inline_edits.clone(), + }) + .await?; + if let Some(terminal) = launch.terminal.as_mut() { + terminal.provider_secret_references = prepared.references; + terminal.provider_launch_context = Some(prepared.context); + } else if let Some(fresh) = launch.fresh_agent.as_mut() { + fresh.provider_secret_references = prepared.references; + fresh.provider_launch_context = Some(prepared.context); + } else { + return Err("initial OpenCode workload unavailable".into()); + } + // A new approved generation has a different semantic payload. + // Keep the public creation/soul identity and use a fresh control + // request identity rather than replaying a rejected digest. + request_id = freshell_runtime_protocol::RequestId::new(); + } + Err(error) => return Err(error.to_string()), + } + } +} diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 87060cc6c..b3c52e71e 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -386,15 +386,58 @@ impl ManagedTerminalController for ServerManagedRuntimeController { }), expected_control_epoch: None, }; - let result = match self.client.launch(request_id, launch).await { - Ok(result) => result, - Err(error) => { - if let Some(capability) = &mcp_capability { - let _ = crate::managed_mcp_capability::revoke_mcp_capability( - &capability.grant_id, - ); + let result = if request.mode == "opencode" { + let home = original_provider_home(&request.env)?; + let original = |name: &str| { + request + .env + .get(name) + .cloned() + .or_else(|| std::env::var(name).ok()) + }; + let references = launch + .terminal + .as_ref() + .unwrap() + .provider_secret_references + .iter() + .filter(|r| !r.profile.is_configuration_file()) + .cloned() + .collect(); + crate::managed_provider_references::launch_initial_opencode( + crate::managed_provider_references::OpencodePreparation { + client: &self.client, + soul: &soul_id, + creation_seed: create_key, + home: &home, + cwd: &cwd, + workspace: &workspace, + original: &original, + references, + auth_authority: None, + selected_sources: None, + accepted_global_root: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: None, + }, + request_id, + launch, + ) + .await? + } else { + match self.client.launch(request_id, launch).await { + Ok(result) => result, + Err(error) => { + if let Some(capability) = &mcp_capability { + let _ = crate::managed_mcp_capability::revoke_mcp_capability( + &capability.grant_id, + ); + } + return Err(error.to_string()); } - return Err(error.to_string()); } }; Ok(ManagedTerminalDescriptor { @@ -1266,3 +1309,7 @@ mod tests { #[cfg(all(test, unix))] #[path = "managed_output_rpc_tests.rs"] mod managed_output_rpc_tests; + +#[cfg(all(test, unix))] +#[path = "managed_runtime_opencode_replan_tests.rs"] +mod opencode_replan_tests; diff --git a/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs b/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs new file mode 100644 index 000000000..f738a96a0 --- /dev/null +++ b/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs @@ -0,0 +1,144 @@ +//! Actual initial controller callers with synthetic authenticated control transport. +use super::*; +use freshell_freshagent::hosted_rest::{HostedFreshAgentRestGateway, HostedRestCreate}; +use freshell_runtime_protocol::*; +use std::{collections::BTreeMap, os::unix::fs::PermissionsExt}; +use tokio::net::UnixListener; + +#[test] +fn managed_provider_initial_opencode_callers_replan_changed_selection_with_two_retry_bound() { + const CHILD: &str = "FRESHELL_TEST_INITIAL_OPENCODE_REPLAN_CHILD"; + if std::env::var_os(CHILD).is_none() { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args(["--exact", "managed_runtime::opencode_replan_tests::managed_provider_initial_opencode_callers_replan_changed_selection_with_two_retry_bound", "--nocapture"]) + .env(CHILD, "1").output().unwrap(); + assert!( + output.status.success(), + "stdout={} stderr={}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + print!("{}", String::from_utf8_lossy(&output.stdout)); + return; + } + tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap().block_on(async { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let control = owned.path().join("control"); + for directory in [&home.join(".config/opencode"), &workspace, &control] { std::fs::create_dir_all(directory).unwrap(); } + let config = home.join(".config/opencode/opencode.json"); + let raw = r#"{"agent":{"probe":{"prompt":"{file:{env:RULE_FILE}}"}}}"#; + std::fs::write(&config, raw).unwrap(); + let files = (0..4).map(|index| { + let path = workspace.join(format!("rules-{index}.txt")); + std::fs::write(&path, format!("approved-consumption-{index}")).unwrap(); path + }).collect::>(); + let auth = owned.path().join("auth.json"); + let write_auth = |path: &Path| { + std::fs::write(&auth, serde_json::to_vec(&serde_json::json!({"fixture":{"type":"wellknown","key":"RULE_FILE","token":path}})).unwrap()).unwrap(); + std::fs::set_permissions(&auth, std::fs::Permissions::from_mode(0o600)).unwrap(); + }; + write_auth(&files[0]); + std::env::set_var("HOME", &home); + std::env::remove_var("XDG_CONFIG_HOME"); + std::env::remove_var("OPENCODE_CONFIG"); + std::env::remove_var("OPENCODE_CONFIG_DIR"); + std::env::remove_var("OPENCODE_CONFIG_CONTENT"); + std::env::set_var("FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE", &auth); + std::env::remove_var("FRESHELL_MANAGED_OPENCODE_ONECLI_ENV_FILE"); + std::env::remove_var("FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE"); + std::env::set_var("FRESHELL_MANAGED_FRESH_AGENT_V1", "1"); + let socket = control.join("supervisor.sock"); + let secret = "fixture-opencode-control-secret"; + let secret_path = control.join("secret"); + std::fs::write(&secret_path, secret).unwrap(); + crate::managed_mcp_capability::configure("0.0.0.0", 45678, &socket, &secret_path).unwrap(); + let listener = UnixListener::bind(&socket).unwrap(); + let attempts = Arc::new(Mutex::new(BTreeMap::::new())); + let consumed = Arc::new(Mutex::new(Vec::::new())); + let (shutdown, mut shutdown_rx) = tokio::sync::oneshot::channel(); + let server = tokio::spawn({ + let auth=auth.clone(); let files=files.clone(); let attempts=attempts.clone(); let consumed=consumed.clone(); let control=control.clone(); + async move { loop { + let (mut stream, _) = tokio::select! { accepted=listener.accept() => accepted.unwrap(), _=&mut shutdown_rx => break }; + let envelope: Envelope = read_frame(&mut stream).await.unwrap(); + assert_eq!(envelope.auth.as_deref(), Some(secret)); + let result = match envelope.body { + AdminCommand::Health => Ok(AdminResult::Health {control_epoch:1,installation_id:InstallationId::parse("fixture-installation").unwrap()}), + AdminCommand::Inventory => Ok(AdminResult::Inventory(vec![])), + AdminCommand::ReserveProviderInputs(request) => Ok(AdminResult::ProviderInputReservation(ProviderInputReservation { + reservation_id:uuid::Uuid::new_v4().to_string(),installation_id:InstallationId::parse("fixture-installation").unwrap(),soul_id:request.soul_id,provider:request.provider,creation_seed_ref:request.creation_seed_ref,intent_revision:1,provider_volume_name:"fixture-volume".into(),image_ref:"fixture-image".into(),host_binary_path:"/fixture/host".into(),host_binary_sha256:"a".repeat(64),observation:InitialAuthVolumeObservation::Absent {daemon_id:DockerDaemonId::parse("fixture-daemon").unwrap()}, + })), + AdminCommand::BindProviderPathInputs(_) => Ok(AdminResult::ProviderPathInputsBound), + AdminCommand::Launch(request) => { + let soul=request.creation_seed_ref.clone(); + let mut all=attempts.lock().await; let count=all.entry(soul).or_default(); *count+=1; + let changing=request.creation_seed_ref.contains("changing"); + let uncertain=request.creation_seed_ref.contains("uncertain"); + let references=request.terminal.as_ref().map(|s| &s.provider_secret_references).or_else(|| request.fresh_agent.as_ref().map(|s| &s.provider_secret_references)).unwrap(); + let selected=references.iter().find(|r| r.profile==ProviderSecretProfile::OpencodeConfigurationFile).expect("approved file selection"); + assert_eq!(Path::new(&selected.source_path), files[if changing {*count-1} else {usize::from(*count>1)}]); + let stages=std::fs::read_dir(control.join("mcp-capabilities")).unwrap().filter_map(Result::ok).filter_map(|e| std::fs::read(e.path()).ok()).filter_map(|b|serde_json::from_slice::(&b).ok()).filter(|v|v["soulId"]==request.soul_id.as_str()).count(); + assert!(stages<=2,"initial retry must not accumulate pending stages: {stages}"); + if changing || *count==1 { + let next=if changing {*count} else {1}; + std::fs::write(&auth, serde_json::to_vec(&serde_json::json!({"fixture":{"type":"wellknown","key":"RULE_FILE","token":files[next]}})).unwrap()).unwrap(); + Err(RuntimeError::new(if uncertain {RuntimeErrorCode::HostUnreachable} else {RuntimeErrorCode::ProviderInputsChanged},"configuration reference selection changed before dispatch")) + } else { + consumed.lock().await.push(std::fs::read_to_string(&selected.source_path).unwrap()); + let view:RuntimeView=serde_json::from_value(serde_json::json!({"soulId":request.soul_id,"incarnationId":IncarnationId::new(),"launchState":"running","cleanupState":"none","intentRevision":1,"executionGeneration":1,"desiredState":"running","recoveryState":"live","durabilityState":"unknown","allocationState":"allocated","evidenceRevision":0,"successfulRecoveriesInWindow":0})).unwrap(); + Ok(AdminResult::Launch(LaunchResult {view,worker_pid:123,worker_launch_count:1,host_boot_id:HostBootId::new(),effective_limits:request.limits,fixture_evidence:serde_json::Value::Null})) + } + }, + AdminCommand::FreshAgentReadEvents(_) => Err(RuntimeError::new(RuntimeErrorCode::HostUnreachable,"fixture poll completed")), + _=>panic!("unexpected control fixture operation"), + }; + write_frame(&mut stream,&AdminReply {request_id:envelope.request_id,result}).await.unwrap(); + }} + }); + let client=RuntimeClient::new(&socket, secret); + let controller=ServerManagedRuntimeController {client:client.clone(), recovery:Arc::new(ManagedRecoveryState::default())}; + let (broadcast,_receiver)=tokio::sync::broadcast::channel(32); + let fresh=crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in(Some(client),Arc::new(broadcast)).unwrap().unwrap(); + let routes = if std::env::var("FRESHELL_TEST_INITIAL_OPENCODE_REPLAN_ROUTE").ok().as_deref()==Some("fresh") {vec![true]} else {vec![false,true]}; + for fresh_route in routes { + for outcome in ["stable","changing","uncertain"] { + let changing=outcome=="changing"; + let uncertain=outcome=="uncertain"; + write_auth(&files[0]); + let seed=format!("{}-{}",if fresh_route {"fresh"} else {"terminal"},outcome); + let soul=if fresh_route {SoulId::parse(format!("soul-fresh-opencode-{}",&format!("{:x}",Sha256::digest(seed.as_bytes()))[..32])).unwrap()} else {stable_soul_id(&seed).unwrap()}; + let prior=crate::managed_provider_references::prepare_managed_opencode(crate::managed_provider_references::OpencodePreparation { + client:&controller.client,soul:&soul,creation_seed:&seed,home:&home,cwd:&workspace,workspace:&workspace, + original:&|name|std::env::var(name).ok(),references:crate::managed_provider_bootstrap::named_provider_onecli_references("opencode").unwrap(),auth_authority:None,selected_sources:None,accepted_global_root:None,selected_resource_roots:None,native_source_flags:None,rebind_enabled:None,inherited_inline:None,inline_edits:None, + }).await.unwrap(); + crate::managed_mcp_capability::accept_initial_replan_fixture_preparation(&soul, &prior.context).unwrap(); + let prior_grant=prior.context.mcp_capability.as_ref().unwrap().grant_id.clone(); + let prior_stage=control.join("provider-roots").join(&prior_grant); + let prior_manifest=std::fs::read(prior_stage.join(".freshell/input-manifest.json")).unwrap(); + let result=if fresh_route { + fresh.clone().create_agent(HostedRestCreate {request_id:seed.clone(),provider:"opencode".into(),session_type:"freshopencode".into(),cwd:Some(workspace.to_string_lossy().into_owned()),model:None,effort:None,native_session_id:None,plugins:None,model_selection:None,permission_mode:None,sandbox:None}).await.map(|_|()).map_err(|_|"fresh refusal".to_string()) + } else { + controller.launch(ManagedTerminalLaunch {spec:freshell_platform::SpawnSpec {program:"opencode".into(),args:vec![],env_overrides:BTreeMap::new(),cwd:Some(workspace.to_string_lossy().into_owned()),cols:80,rows:24},env:BTreeMap::new(),terminal_id:seed.clone(),stream_id:seed.clone(),mode:"opencode".into(),resume_session_id:None,provider_model:None,provider_reasoning_effort:None,provider_sandbox:None,provider_permission_mode:None,view_tab_id:None,view_pane_id:None,create_request_id:Some(seed.clone())}).await.map(|_|()) + }; + assert_eq!(result.is_ok(),!changing && !uncertain,"actual initial caller fresh={fresh_route} changing={changing}: {result:?}"); + assert_eq!(attempts.lock().await[&seed],if uncertain {1} else if changing {3} else {2}); + assert_eq!(std::fs::read(prior_stage.join(".freshell/input-manifest.json")).unwrap(),prior_manifest,"prior usable preparation is preserved"); + assert!(control.join("mcp-capabilities").join(format!("{prior_grant}.json")).is_file()); + assert_eq!(std::fs::read_to_string(&prior.references.iter().find(|r|r.profile.is_configuration_file()).unwrap().source_path).unwrap(),"approved-consumption-0"); + if uncertain { // Both current and prior ownership remain available for reconciliation. + let grants=std::fs::read_dir(control.join("mcp-capabilities")).unwrap().filter_map(Result::ok).filter_map(|e|std::fs::read(e.path()).ok()).filter_map(|b|serde_json::from_slice::(&b).ok()).filter(|v|v["soulId"]==soul.as_str()).count(); + assert_eq!(grants,2); + } + println!("initial_opencode_caller {{\"route\":\"{}\",\"outcome\":\"{}\",\"attempts\":{}}}",if fresh_route {"fresh"} else {"terminal"},outcome,attempts.lock().await[&seed]); + assert_eq!(std::fs::read_to_string(&config).unwrap(),raw); + } + } + assert!(consumed.lock().await.iter().all(|v|v=="approved-consumption-1")); + assert_eq!(consumed.lock().await.len(),if std::env::var("FRESHELL_TEST_INITIAL_OPENCODE_REPLAN_ROUTE").ok().as_deref()==Some("fresh") {1} else {2}); + // Polling receives an error then empty inventory and settles on the same runtime. + tokio::task::yield_now().await; + shutdown.send(()).unwrap(); server.await.unwrap(); + }); +} diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index 7bdac280d..fb497dd69 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -154,6 +154,16 @@ pub trait RuntimeBackend: Send + Sync { "initial auth source observation unavailable".into(), )) } + /// Remove only the generation of a positively rejected, stopped initial + /// host. The owning supervisor must prove the grant was never consumed. + async fn discard_rejected_provider_inputs( + &self, + _handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + Err(BackendError::Unavailable( + "rejected provider input cleanup unavailable".into(), + )) + } async fn commit_prepared_inputs( &self, _handle: &OwnedRuntimeHandle, @@ -651,6 +661,13 @@ impl RuntimeBackend for DockerEngineBackend { } Ok(()) } + async fn discard_rejected_provider_inputs( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + self.remove_stopped_input_workload(handle).await?; + self.remove_stopped_indexed_inputs(handle).await + } async fn retire_prior_prepared_inputs( &self, current: &OwnedRuntimeHandle, diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index f205ccb11..60a8e5058 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -1020,6 +1020,10 @@ async fn qualify_candidate_helper_transport( observation: InitialAuthVolumeObservation::Absent { daemon_id }, .. } => daemon_id.clone(), + ClaudeSourceOwner::Initial { + observation: InitialAuthVolumeObservation::OwnedUndispatched { .. }, + .. + } => unreachable!("OpenCode rejected inputs do not authorize Claude source discovery"), ClaudeSourceOwner::Retained { handle, .. } => handle.daemon_id().clone(), }; let retained_name = match owner { diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index e1599d51f..cfb04f68f 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -978,7 +978,7 @@ impl DockerEngineBackend { let Some(inputs) = context(prior).and_then(|v| v.prepared_inputs.as_ref()) else { return Ok(()); }; - let Some(files) = inputs.indexed_files.as_ref() else { + if inputs.indexed_files.is_none() { return Ok(()); }; let next = context(current) @@ -995,6 +995,62 @@ impl DockerEngineBackend { if next.generation_id == inputs.generation_id { return Ok(()); } + self.remove_stopped_indexed_inputs(prior).await + } + + pub(super) async fn remove_stopped_input_workload( + &self, + prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + let state = self.inspect(prior).await?.state; + match state { + BackendRuntimeState::Missing => {} + BackendRuntimeState::Created + | BackendRuntimeState::Exited + | BackendRuntimeState::Dead => { + // Docker refuses removal if this exact proved object becomes + // running again. Retirement never forces a workload stop. + let removed = self + .request_bounded( + "DELETE", + &format!("{DOCKER_API}/containers/{}?force=0", prior.container_id()), + None, + 64 * 1024, + ) + .await?; + if !matches!(removed.status, 204 | 404) { + return Err(fail( + "prior input workload removal uncertain; preserve generation", + )); + } + } + _ => { + return Err(fail( + "prior input workload remains live or uncertain; preserve generation", + )) + } + } + Ok(()) + } + + pub(super) async fn remove_stopped_indexed_inputs( + &self, + prior: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + let context = prior + .terminal() + .and_then(|s| s.provider_launch_context.as_ref()) + .or_else(|| { + prior + .fresh_agent() + .and_then(|s| s.provider_launch_context.as_ref()) + }); + let Some(inputs) = context.and_then(|c| c.prepared_inputs.as_ref()) else { + return Ok(()); + }; + let Some(files) = inputs.indexed_files.as_ref() else { + return Ok(()); + }; if files.volume_name != freshell_runtime_protocol::indexed_provider_volume_name( prior.soul_id(), @@ -1005,34 +1061,7 @@ impl DockerEngineBackend { } let ownership = json!({"com.freshell.managed-inputs":"true","com.freshell.installation-id":prior.installation_id(),"com.freshell.soul-id":prior.soul_id(),"com.freshell.incarnation-id":prior.incarnation_id(),"com.freshell.input-generation":inputs.generation_id}); let operation = async { - let state = self.inspect(prior).await?.state; - match state { - BackendRuntimeState::Missing => {} - BackendRuntimeState::Created - | BackendRuntimeState::Exited - | BackendRuntimeState::Dead => { - // Docker refuses removal if this exact proved object becomes - // running again. Retirement never forces a workload stop. - let removed = self - .request_bounded( - "DELETE", - &format!("{DOCKER_API}/containers/{}?force=0", prior.container_id()), - None, - 64 * 1024, - ) - .await?; - if !matches!(removed.status, 204 | 404) { - return Err(fail( - "prior input workload removal uncertain; preserve generation", - )); - } - } - _ => { - return Err(fail( - "prior input workload remains live or uncertain; preserve generation", - )) - } - } + self.remove_stopped_input_workload(prior).await?; self.release_indexed_publication( prior.soul_id(), prior.incarnation_id(), diff --git a/crates/freshell-supervisor/src/backend/native_auth.rs b/crates/freshell-supervisor/src/backend/native_auth.rs index 01a91076b..a527a8c7a 100644 --- a/crates/freshell-supervisor/src/backend/native_auth.rs +++ b/crates/freshell-supervisor/src/backend/native_auth.rs @@ -105,9 +105,15 @@ impl DockerEngineBackend { return Ok(()); }; let reservation = &bound.reservation; - let freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { - daemon_id: observed, - } = &bound.observation; + let (observed, volume_owner, retained) = match &bound.observation { + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { daemon_id } => { + (daemon_id, &spec.incarnation_id, false) + } + freshell_runtime_protocol::InitialAuthVolumeObservation::OwnedUndispatched { + daemon_id, + incarnation_id, + } => (daemon_id, incarnation_id, true), + }; if !rootless || observed != daemon || reservation.installation_id != spec.installation_id @@ -129,22 +135,18 @@ impl DockerEngineBackend { bound .verify_artifact() .map_err(|_| fail("initial auth reserved runtime artifact changed or unreadable"))?; - let labels = json!({ - "com.freshell.installation-id":spec.installation_id.as_str(), - "com.freshell.soul-id":spec.soul_id.as_str(), - "freshell.input.reservation":reservation.id, - "freshell.input.incarnation":spec.incarnation_id.as_str(), - "freshell.input.intent":"1", - "freshell.input.daemon":daemon.as_str(), - "freshell.input.image":reservation.image_ref, - "freshell.input.binary-sha256":reservation.binary_sha256 - }); + let labels = reserved_volume_labels(reservation, daemon, volume_owner); let path = format!("{DOCKER_API}/volumes/{}", spec.provider_volume_name); let response = self .request_bounded("GET", &path, None, 64 * 1024) .await .map_err(|_| fail("reserved initial volume observation failed"))?; if response.status == 404 { + if retained { + return Err(fail( + "owned undispatched provider volume is missing; preserve inputs", + )); + } if self .daemon_id() .await @@ -228,9 +230,40 @@ impl DockerEngineBackend { ) .await .map_err(|_| fail("initial auth reserved volume observation failed"))?; - if response.status != 404 { - return Err(fail("initial auth absence requires an actual reserved-volume 404; use an owned existing source")); - } + let observation = match &reservation.prior_observation { + Some( + owned + @ freshell_runtime_protocol::InitialAuthVolumeObservation::OwnedUndispatched { + daemon_id: expected, + incarnation_id, + }, + ) => { + let value: Value = serde_json::from_slice(&response.body) + .map_err(|_| fail("owned undispatched volume response invalid"))?; + if reservation.provider != "opencode" + || expected != &daemon_id + || response.status != 200 + || value["Name"] != reservation.provider_volume_name + || value["Driver"] != "local" + || value["Labels"] + != reserved_volume_labels(reservation, &daemon_id, incarnation_id) + || !(value["Options"].is_null() || value["Options"] == json!({})) + { + return Err(fail( + "owned undispatched volume identity changed; preserve inputs", + )); + } + owned.clone() + } + _ => { + if response.status != 404 { + return Err(fail("initial auth absence requires an actual reserved-volume 404; use an owned existing source")); + } + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id: daemon_id.clone(), + } + } + }; if self .daemon_id() .await @@ -239,7 +272,7 @@ impl DockerEngineBackend { { return Err(fail("initial auth source daemon changed")); } - Ok(freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { daemon_id }) + Ok(observation) } pub(super) async fn read_auth_helper( &self, @@ -648,6 +681,56 @@ mod tests { backend.create_stopped(&spec).await.unwrap(); assert_eq!(requests.lock().unwrap().iter().filter(|r|r.starts_with("POST") && r.contains("/volumes/create ")).count(),1, "lost workload-create acknowledgement must reuse the same exact labelled reservation volume"); + let mut retained = binding.clone(); + retained.observation = + freshell_runtime_protocol::InitialAuthVolumeObservation::OwnedUndispatched { + daemon_id: match &binding.observation { + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { + daemon_id, + } => daemon_id.clone(), + _ => unreachable!(), + }, + incarnation_id: binding.incarnation_id.clone(), + }; + retained.reservation.prior_observation = Some(retained.observation.clone()); + assert_eq!( + backend + .observe_initial_volume(&retained.reservation) + .await + .unwrap(), + retained.observation + ); + let mut replanned = spec.clone(); + retained.incarnation_id = IncarnationId::new(); + replanned.incarnation_id = retained.incarnation_id.clone(); + replanned.initial_input_reservation = Some(retained.clone()); + backend + .create_stopped(&replanned) + .await + .expect("fresh approved host reuses exact undispatched creation-owned volume"); + assert_eq!( + requests + .lock() + .unwrap() + .iter() + .filter(|r| r.starts_with("POST") && r.contains("/volumes/create ")) + .count(), + 1 + ); + let owned_value = volume.lock().unwrap().clone().unwrap(); + *volume.lock().unwrap() = None; + assert!( + backend + .observe_initial_volume(&retained.reservation) + .await + .is_err(), + "missing retained volume is not new absence" + ); + assert!( + backend.create_stopped(&replanned).await.is_err(), + "replan cannot recreate a missing owned volume" + ); + *volume.lock().unwrap() = Some(owned_value); volume.lock().unwrap().as_mut().unwrap()["Labels"]["freshell.input.reservation"] = json!("foreign-reservation"); assert!( @@ -661,7 +744,7 @@ mod tests { .iter() .filter(|r| r.contains("/containers/create?")) .count(), - 2 + 3 ); std::fs::write(&spec.host_binary_path, b"changed synthetic artifact").unwrap(); assert!( @@ -675,7 +758,7 @@ mod tests { .iter() .filter(|r| r.contains("/containers/create?")) .count(), - 2 + 3 ); server.abort(); let _ = server.await; @@ -898,3 +981,20 @@ mod tests { } } } + +fn reserved_volume_labels( + reservation: &crate::registry::OwnedInputReservation, + daemon: &DockerDaemonId, + incarnation: &freshell_runtime_protocol::IncarnationId, +) -> Value { + json!({ + "com.freshell.installation-id":reservation.installation_id.as_str(), + "com.freshell.soul-id":reservation.soul_id.as_str(), + "freshell.input.reservation":reservation.id, + "freshell.input.incarnation":incarnation.as_str(), + "freshell.input.intent":"1", + "freshell.input.daemon":daemon.as_str(), + "freshell.input.image":reservation.image_ref, + "freshell.input.binary-sha256":reservation.binary_sha256 + }) +} diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs index baa202e70..d9efb098a 100644 --- a/crates/freshell-supervisor/src/provider_preparation_tests.rs +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -18,6 +18,8 @@ struct Daemon { fail: Option<&'static str>, grants: usize, host_tasks: Vec>, + reject_inputs: bool, + consumed_rejection: bool, } struct Backend(Arc>, std::path::PathBuf); @@ -94,6 +96,17 @@ fn limits() -> RuntimeLimits { #[async_trait] impl RuntimeBackend for Backend { + async fn observe_initial_auth_volume( + &self, + reservation: &crate::registry::OwnedInputReservation, + ) -> Result { + Ok(reservation + .prior_observation + .clone() + .unwrap_or(InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::parse("synthetic-daemon").unwrap(), + })) + } async fn create_stopped( &self, spec: &CreateRuntimeSpec, @@ -140,6 +153,8 @@ impl RuntimeBackend for Backend { loop { let (mut socket, _) = listener.accept().await.unwrap(); let envelope: Envelope = read_frame(&mut socket).await.unwrap(); + let rejected = state.lock().unwrap().reject_inputs; + let consumed_rejection = state.lock().unwrap().consumed_rejection; let result = match envelope.body { HostCommand::Hello { challenge, @@ -151,6 +166,21 @@ impl RuntimeBackend for Backend { effective_limits: limits(), }, HostCommand::GrantExecution { .. } => { + if state.lock().unwrap().reject_inputs { + write_frame( + &mut socket, + &HostReply { + request_id: envelope.request_id, + result: Err(RuntimeError::new( + RuntimeErrorCode::ProviderInputsChanged, + "configuration reference selection changed", + )), + }, + ) + .await + .unwrap(); + continue; + } state.lock().unwrap().grants += 1; HostResult::GrantAccepted { host_boot_id: boot.clone(), @@ -162,13 +192,21 @@ impl RuntimeBackend for Backend { } HostCommand::Status { .. } => HostResult::Status { host_boot_id: boot.clone(), - worker_pid: Some(1234), - worker_launch_count: 1, + worker_pid: if rejected { None } else { Some(1234) }, + worker_launch_count: u64::from(!rejected), max_control_epoch: 100, - max_execution_generation: 100, + max_execution_generation: if rejected && !consumed_rejection { + 0 + } else { + 100 + }, fixture_evidence: serde_json::json!({}), exited: false, - native_session_id: Some("ses_exact".into()), + native_session_id: if rejected { + None + } else { + Some("ses_exact".into()) + }, }, HostCommand::ProbeRecovery { resume_spec: Some(spec), @@ -193,6 +231,17 @@ impl RuntimeBackend for Backend { self.0.lock().unwrap().host_tasks.push(task); Ok(()) } + async fn discard_rejected_provider_inputs( + &self, + handle: &OwnedRuntimeHandle, + ) -> Result<(), BackendError> { + let mut state = self.0.lock().unwrap(); + assert!(state.reject_inputs && !state.consumed_rejection); + assert!(!state.running.contains(handle.incarnation_id())); + state.generations.remove(&generation(handle)); + state.publishers.remove(&generation(handle)); + Ok(()) + } async fn commit_prepared_inputs( &self, handle: &OwnedRuntimeHandle, @@ -253,8 +302,17 @@ impl RuntimeBackend for Backend { configured_limits: limits(), }) } - async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { - panic!("healthy retry must not stop a workload") + async fn request_stop(&self, handle: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + let mut state = self.0.lock().unwrap(); + assert!( + state.reject_inputs && !state.consumed_rejection, + "healthy or consumed retry must not stop a workload" + ); + state.running.remove(handle.incarnation_id()); + state + .events + .push(format!("stop-rejected:{}", generation(handle))); + Ok(()) } async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { panic!("healthy retry must not stop a workload") @@ -557,3 +615,217 @@ async fn provider_accepted_completion_retries_after_registry_success_on_fresh_he } } } + +#[tokio::test] +async fn provider_initial_changed_inputs_requires_unconsumed_host_and_settled_owned_cleanup() { + for consumed in [false, true] { + let temp = tempfile::tempdir().unwrap(); + let root = temp.path().join("registry"); + let registry = Registry::open(&root, None).unwrap(); + let soul = SoulId::new(); + let daemon = Arc::new(Mutex::new(Daemon { + reject_inputs: true, + consumed_rejection: consumed, + ..Default::default() + })); + daemon + .lock() + .unwrap() + .generations + .insert("prior-usable".into()); + let callback = UnixListener::bind(temp.path().join("mcp.sock")).unwrap(); + let callback_task = tokio::spawn(async move { + loop { + let (mut stream, _) = callback.accept().await.unwrap(); + let mut raw = vec![]; + stream.read_to_end(&mut raw).await.unwrap(); + stream.write_all(b"ok").await.unwrap(); + } + }); + let config = SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: temp.path().join("synthetic-host"), + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "provider-initial-replan".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }; + std::fs::write(&config.host_binary_path, "synthetic-host-artifact").unwrap(); + let supervisor = Supervisor::new( + registry.clone(), + Arc::new(Backend(daemon.clone(), root)), + config.clone(), + ) + .unwrap(); + let reservation = registry + .reserve_initial_inputs( + ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "opencode".into(), + creation_seed_ref: "seed".into(), + }, + config.image_ref.clone(), + config.host_binary_path.clone(), + ) + .await + .unwrap(); + let receipt = registry + .record_initial_observation( + reservation, + InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::parse("synthetic-daemon").unwrap(), + }, + ) + .await + .unwrap(); + let mut input = context(&soul, 1); + input + .prepared_inputs + .as_mut() + .unwrap() + .initial_auth_reservation = Some(receipt.reservation_id.clone()); + let terminal:TerminalLaunchSpec=serde_json::from_value(serde_json::json!({"terminalId":"terminal","streamId":"stream","mode":"opencode","program":"opencode","args":[],"env":{},"cwd":temp.path(),"runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"project","workspacePath":temp.path(),"providerLaunchContext":input,"providerSecretReferences":references()})).unwrap(); + let preparation = LaunchPreparation { + soul_id: soul.clone(), + provider: "opencode".into(), + provider_store_id: "store-exact".into(), + native_session_id: None, + creation_seed_ref: "seed".into(), + request_id: RequestId::new(), + payload_digest: "replan-test".into(), + requested_limits: limits(), + profile: RuntimeProfile::Custom, + project_key: "project".into(), + fixture: None, + terminal: Some(terminal.clone()), + fresh_agent: None, + view_intent: None, + admission: crate::admission::AdmissionPolicy::default(), + }; + let prepared = registry.prepare_launch(preparation.clone()).await.unwrap(); + let error = supervisor + .activate_prepared( + prepared.clone(), + LaunchWorkload { + soul_id: soul.clone(), + fixture: None, + terminal: Some(terminal), + fresh_agent: None, + resume_spec: None, + }, + limits(), + ) + .await + .unwrap_err(); + assert_eq!( + error.code, + if consumed { + RuntimeErrorCode::HostUnreachable + } else { + RuntimeErrorCode::ProviderInputsChanged + } + ); + let view = registry.inventory().await.unwrap().pop().unwrap(); + assert_eq!( + view.launch_state, + if consumed { + LaunchState::Starting + } else { + LaunchState::Stopped + } + ); + assert!(daemon.lock().unwrap().generations.contains("prior-usable")); + assert_eq!( + daemon.lock().unwrap().generations.contains("grant-1"), + consumed, + "only positively rejected generation can be retired" + ); + assert_eq!(daemon.lock().unwrap().grants, 0); + if !consumed { + for ordinal in 2..=3 { + let reserved = registry + .reserve_initial_inputs( + ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "opencode".into(), + creation_seed_ref: "seed".into(), + }, + config.image_ref.clone(), + config.host_binary_path.clone(), + ) + .await + .unwrap(); + assert_eq!( + reserved.id, receipt.reservation_id, + "replan reuses exact creation authority" + ); + let retained = reserved.prior_observation.clone().unwrap(); + assert!( + matches!(&retained,InitialAuthVolumeObservation::OwnedUndispatched {incarnation_id,..} if incarnation_id==&prepared.incarnation_id) + ); + registry + .record_initial_observation(reserved, retained) + .await + .unwrap(); + let mut next = preparation.clone(); + next.request_id = RequestId::new(); + next.payload_digest = format!("replan-{ordinal}"); + let mut next_context = context(&soul, ordinal); + next_context + .prepared_inputs + .as_mut() + .unwrap() + .initial_auth_reservation = Some(receipt.reservation_id.clone()); + next.terminal.as_mut().unwrap().provider_launch_context = Some(next_context); + let workload = LaunchWorkload { + soul_id: soul.clone(), + fixture: None, + terminal: next.terminal.clone(), + fresh_agent: None, + resume_spec: None, + }; + let prepared = registry.prepare_launch(next).await.unwrap(); + assert_eq!( + supervisor + .activate_prepared(prepared, workload, limits()) + .await + .unwrap_err() + .code, + RuntimeErrorCode::ProviderInputsChanged + ); + assert_eq!( + daemon.lock().unwrap().generations, + BTreeSet::from(["prior-usable".to_string()]), + "each rejected generation is settled while prior input remains" + ); + } + assert!( + registry + .reserve_initial_inputs( + ReserveProviderInputsRequest { + expected_control_epoch: Some(registry.control_epoch()), + soul_id: soul.clone(), + provider: "opencode".into(), + creation_seed_ref: "seed".into() + }, + config.image_ref.clone(), + config.host_binary_path.clone() + ) + .await + .is_err(), + "two pre-dispatch replans are the complete bound" + ); + } + let tasks = std::mem::take(&mut daemon.lock().unwrap().host_tasks); + for task in tasks { + task.abort(); + assert!(task.await.unwrap_err().is_cancelled()); + } + callback_task.abort(); + assert!(callback_task.await.unwrap_err().is_cancelled()); + } +} diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index 4d160603c..663e69118 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -16,6 +16,7 @@ pub struct OwnedInputReservation { pub(crate) image_ref: String, pub(crate) binary: PathBuf, pub(crate) binary_sha256: String, + pub(crate) prior_observation: Option, } #[derive(Debug, Clone)] pub struct BoundInputReservation { @@ -107,7 +108,51 @@ pub(crate) fn binary_digest(binary: &Path) -> Result { } Ok(format!("{:x}", hash.finalize())) } +// The sole reuse authority is a settled, explicitly rejected initial grant. +// A missing native ID, ordinary stopped host, or uncertain delivery is insufficient. +fn initial_or_rejected_creation(conn: &Connection, soul: &SoulId) -> Result { + let exists: bool = conn.query_row( + "SELECT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)", + [soul.as_str()], + |row| row.get(0), + )?; + if !exists { + return Ok(true); + }; + let eligible:bool=conn.query_row("SELECT EXISTS(SELECT 1 FROM souls s JOIN provider_input_preparations p ON p.soul_id=s.soul_id WHERE s.soul_id=?1 AND s.provider='opencode' AND s.native_session_id IS NULL AND s.desired_state='running' AND s.intent_revision=1 AND s.creation_seed_ref=p.creation_seed_ref AND p.provider=s.provider AND p.consumed=0 AND p.bound_incarnation_id IS NULL AND NOT EXISTS(SELECT 1 FROM incarnations i WHERE i.soul_id=s.soul_id AND i.launch_state!='stopped') AND NOT EXISTS(SELECT 1 FROM commands c WHERE c.soul_id=s.soul_id AND c.state!='provider_inputs_changed') AND (SELECT COUNT(*) FROM commands c WHERE c.soul_id=s.soul_id AND c.state='provider_inputs_changed') BETWEEN 1 AND 2 AND NOT EXISTS(SELECT 1 FROM input_commands c WHERE c.soul_id=s.soul_id))",[soul.as_str()],|row|row.get(0))?; + Ok(eligible) +} + impl Registry { + pub(crate) async fn validate_rejected_initial_inputs( + &self, + incarnation: IncarnationId, + ) -> Result<(), RegistryError> { + self.run_blocking(move |conn| { + rejected_initial_observation(&conn, &incarnation)?; + Ok(()) + }) + .await + } + pub(crate) async fn settle_rejected_initial_inputs( + &self, + incarnation: IncarnationId, + ) -> Result<(), RegistryError> { + self.run_blocking(move|mut conn| { + let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; + let (soul,observation)=rejected_initial_observation(&tx,&incarnation)?; + let retained=match observation { + InitialAuthVolumeObservation::Absent {daemon_id} => InitialAuthVolumeObservation::OwnedUndispatched {daemon_id,incarnation_id:incarnation.clone()}, + owned @ InitialAuthVolumeObservation::OwnedUndispatched {..} => owned, + }; + tx.execute("UPDATE provider_input_preparations SET consumed=0,bound_incarnation_id=NULL,observation=?1 WHERE soul_id=?2 AND bound_incarnation_id=?3 AND consumed=1",params![serde_json::to_string(&retained)?,soul,incarnation.as_str()])?; + tx.execute("UPDATE incarnations SET launch_state='stopped',cleanup_state='verified_empty',updated_at=?1 WHERE incarnation_id=?2",params![now_millis(),incarnation.as_str()])?; + tx.execute("UPDATE commands SET state='provider_inputs_changed',updated_at=?1 WHERE incarnation_id=?2",params![now_millis(),incarnation.as_str()])?; + tx.execute("DELETE FROM admission_reservations WHERE incarnation_id=?1",[incarnation.as_str()])?; + tx.execute("DELETE FROM writer_claims WHERE incarnation_id=?1",[incarnation.as_str()])?; + tx.commit()?;Ok(()) + }).await + } pub(crate) async fn bound_input_reservation( &self, incarnation_id: IncarnationId, @@ -123,7 +168,7 @@ impl Registry { if binary_digest(&binary)?!=sha {return Err(RegistryError::InvalidState("initial auth source artifact changed before launch".into()));} let soul=SoulId::parse(soul).map_err(|_|RegistryError::Integrity("invalid reserved soul".into()))?; Ok(Some(BoundInputReservation { - reservation:OwnedInputReservation{id,provider,provider_volume_name:stable_provider_volume_name(&installation,&soul),installation_id:installation,soul_id:soul,creation_seed_ref:seed,image_ref:image,binary,binary_sha256:sha}, + reservation:OwnedInputReservation{id,provider,provider_volume_name:stable_provider_volume_name(&installation,&soul),installation_id:installation,soul_id:soul,creation_seed_ref:seed,image_ref:image,binary,binary_sha256:sha,prior_observation:Some(serde_json::from_str(&observation)?)}, incarnation_id,observation:serde_json::from_str(&observation)? })) }).await @@ -135,12 +180,13 @@ impl Registry { ) -> Result, RegistryError> { let installation = self.installation_id().clone(); self.run_blocking(move|conn|{ - let row:Option<(String,String,String,String,String,String)>=conn.query_row("SELECT creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,observation,provider FROM provider_input_preparations WHERE soul_id=?1 AND reservation_id=?2 AND consumed=0 AND observation IS NOT NULL AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![soul.as_str(),id],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?))).optional()?; + let row:Option<(String,String,String,String,String,String)>=conn.query_row("SELECT creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,observation,provider FROM provider_input_preparations WHERE soul_id=?1 AND reservation_id=?2 AND consumed=0 AND observation IS NOT NULL",params![soul.as_str(),id],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?))).optional()?; let Some((seed,image,path,digest,observation,provider))=row else{return Ok(None)}; + if !initial_or_rejected_creation(&conn,&soul)? {return Ok(None)}; let binary=PathBuf::from(path); if binary_digest(&binary)?!=digest{return Err(RegistryError::InvalidState("initial auth source artifact changed; reprepare before dispatch".into()));} let volume=stable_provider_volume_name(&installation,&soul); - Ok(Some((OwnedInputReservation{id,provider,installation_id:installation,soul_id:soul,creation_seed_ref:seed,provider_volume_name:volume,image_ref:image,binary,binary_sha256:digest},serde_json::from_str(&observation)?))) + Ok(Some((OwnedInputReservation{id,provider,installation_id:installation,soul_id:soul,creation_seed_ref:seed,provider_volume_name:volume,image_ref:image,binary,binary_sha256:digest,prior_observation:Some(serde_json::from_str(&observation)?)},serde_json::from_str(&observation)?))) }).await } pub(crate) async fn reserve_initial_inputs( @@ -166,7 +212,7 @@ impl Registry { let sha = binary_digest(&binary)?; let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; let exists:bool=tx.query_row("SELECT EXISTS(SELECT 1 FROM souls WHERE soul_id=?1)",params![request.soul_id.as_str()],|row|row.get(0))?; - if exists {return Err(RegistryError::InvalidState("initial source requires a creation reservation; existing souls use their owned recovery source".into()));} + if exists && !initial_or_rejected_creation(&tx,&request.soul_id)? {return Err(RegistryError::InvalidState("initial source requires a creation reservation; existing souls use their owned recovery source".into()));} let stored:Option<(String,String,String,String,String,i64,String)>=tx.query_row("SELECT reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,consumed,provider FROM provider_input_preparations WHERE soul_id=?1",params![request.soul_id.as_str()],|row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?))).optional()?; let id=if let Some((id,seed,image,path,digest,consumed,provider))=stored { if provider!=request.provider || seed!=request.creation_seed_ref || image!=image_ref || Path::new(&path)!=binary || digest!=sha || consumed!=0 {return Err(RegistryError::RequestConflict);} @@ -176,9 +222,11 @@ impl Registry { tx.execute("INSERT INTO provider_input_preparations (soul_id,reservation_id,creation_seed_ref,image_ref,host_binary_path,host_binary_sha256,intent_revision,provider) VALUES (?1,?2,?3,?4,?5,?6,1,?7)",params![request.soul_id.as_str(),id,request.creation_seed_ref,image_ref,binary.to_string_lossy(),sha,request.provider])?; id }; + let prior_observation:Option=tx.query_row("SELECT observation FROM provider_input_preparations WHERE soul_id=?1",[request.soul_id.as_str()],|row|row.get(0))?; + let prior_observation=prior_observation.map(|raw|serde_json::from_str(&raw)).transpose()?; let volume=stable_provider_volume_name(&installation,&request.soul_id); tx.commit()?; - Ok(OwnedInputReservation{id,provider:request.provider,installation_id:installation,soul_id:request.soul_id,creation_seed_ref:request.creation_seed_ref,provider_volume_name:volume,image_ref,binary,binary_sha256:sha}) + Ok(OwnedInputReservation{id,provider:request.provider,installation_id:installation,soul_id:request.soul_id,creation_seed_ref:request.creation_seed_ref,provider_volume_name:volume,image_ref,binary,binary_sha256:sha,prior_observation}) }).await } pub(crate) async fn record_initial_observation( @@ -189,7 +237,8 @@ impl Registry { let stored = serde_json::to_string(&observation)?; self.run_blocking(move|mut conn|{ let tx=conn.transaction_with_behavior(TransactionBehavior::Immediate)?; - let updated=tx.execute("UPDATE provider_input_preparations SET observation=?1 WHERE soul_id=?2 AND reservation_id=?3 AND consumed=0 AND host_binary_sha256=?4 AND provider=?5 AND (observation IS NULL OR observation=?1) AND NOT EXISTS(SELECT 1 FROM souls WHERE soul_id=?2)",params![stored,reservation.soul_id.as_str(),reservation.id,reservation.binary_sha256,reservation.provider])?; + if !initial_or_rejected_creation(&tx,&reservation.soul_id)? {return Err(RegistryError::RequestConflict)}; + let updated=tx.execute("UPDATE provider_input_preparations SET observation=?1 WHERE soul_id=?2 AND reservation_id=?3 AND consumed=0 AND host_binary_sha256=?4 AND provider=?5 AND (observation IS NULL OR observation=?1)",params![stored,reservation.soul_id.as_str(),reservation.id,reservation.binary_sha256,reservation.provider])?; if updated!=1 {return Err(RegistryError::RequestConflict);} tx.commit()?; Ok(ProviderInputReservation{reservation_id:reservation.id,installation_id:reservation.installation_id,soul_id:reservation.soul_id,provider:reservation.provider,creation_seed_ref:reservation.creation_seed_ref,intent_revision:1,provider_volume_name:reservation.provider_volume_name,image_ref:reservation.image_ref,host_binary_path:reservation.binary.to_string_lossy().into_owned(),host_binary_sha256:reservation.binary_sha256,observation}) @@ -256,3 +305,16 @@ pub(super) fn check_bound_reference( } Ok(()) } + +fn rejected_initial_observation( + conn: &Connection, + incarnation: &IncarnationId, +) -> Result<(String, InitialAuthVolumeObservation), RegistryError> { + let row:Option<(String,String)>=conn.query_row("SELECT s.soul_id,p.observation FROM provider_input_preparations p JOIN incarnations i ON i.incarnation_id=p.bound_incarnation_id JOIN souls s ON s.soul_id=p.soul_id WHERE i.incarnation_id=?1 AND i.soul_id=s.soul_id AND p.consumed=1 AND p.provider='opencode' AND s.provider=p.provider AND s.creation_seed_ref=p.creation_seed_ref AND s.native_session_id IS NULL AND s.desired_state='running' AND s.intent_revision=1 AND i.launch_state='starting' AND i.provider_preparation_pending=0 AND NOT EXISTS(SELECT 1 FROM input_commands c WHERE c.soul_id=s.soul_id) AND NOT EXISTS(SELECT 1 FROM commands c WHERE c.soul_id=s.soul_id AND c.incarnation_id!=i.incarnation_id AND c.state!='provider_inputs_changed') AND (SELECT COUNT(*) FROM commands c WHERE c.soul_id=s.soul_id AND c.state='provider_inputs_changed')<3",[incarnation.as_str()],|row|Ok((row.get(0)?,row.get(1)?))).optional()?; + let (soul, raw) = row.ok_or_else(|| { + RegistryError::InvalidState( + "initial provider inputs rejection has no undispatched creation ownership".into(), + ) + })?; + Ok((soul, serde_json::from_str(&raw)?)) +} diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 22934420f..ec37d437d 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -742,9 +742,73 @@ impl Supervisor { { return Err(denied()); } - return Ok(freshell_runtime_protocol::NativeAuthValues( - std::collections::BTreeMap::new(), - )); + let values = match expected { + freshell_runtime_protocol::InitialAuthVolumeObservation::Absent { .. } => { + std::collections::BTreeMap::new() + } + freshell_runtime_protocol::InitialAuthVolumeObservation::OwnedUndispatched { + incarnation_id, + .. + } => { + // Preparation may already have installed a typed auth file + // before refusing the input selection. Read the exact owned + // quiescent store rather than inventing native-auth absence. + let lock = self.lifecycle_lock(&request.soul_id).await; + let _held = lock.lock().await; + if self + .registry + .initial_input_reservation( + request.soul_id.clone(), + request.attempt_authority.clone(), + ) + .await + .map_err(map_registry)? + .is_none() + { + return Err(denied()); + }; + let prior = self + .registry + .owned_handle(incarnation_id) + .await + .map_err(map_registry)?; + let values = tokio::time::timeout( + Duration::from_secs(25), + self.backend.read_native_auth( + &prior, + &request.keys, + &self.config.host_binary_path, + ), + ) + .await + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial replan native auth lookup timed out", + ) + })? + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial replan native auth lookup unavailable", + ) + })?; + if self + .registry + .initial_input_reservation( + request.soul_id.clone(), + request.attempt_authority.clone(), + ) + .await + .map_err(map_registry)? + .is_none() + { + return Err(denied()); + }; + values + } + }; + return Ok(freshell_runtime_protocol::NativeAuthValues(values)); } let authority = { let authorities = self @@ -1084,12 +1148,35 @@ impl Supervisor { .map_err(|error| { self.activation_failure(&prepared, state, "enable_long_lived_runtime", error) })?; - let accepted = self + let accepted = match self .send_grant(&handle, &workload, &authenticated, &grant) .await - .map_err(|error| { - self.activation_failure(&prepared, state, "deliver_execution_grant", error) - })?; + { + Ok(accepted) => accepted, + Err(error) + if error.code == RuntimeErrorCode::ProviderInputsChanged + && workload.resume_spec.is_none() => + { + // The typed refusal alone cannot authorize redispatch: verify + // this exact boot has never crossed its durable grant fence. + self.settle_rejected_initial_inputs(&handle, &authenticated, &grant) + .await?; + return Err(self.activation_failure( + &prepared, + state, + "provider_inputs_changed_before_dispatch", + error, + )); + } + Err(error) => { + return Err(self.activation_failure( + &prepared, + state, + "deliver_execution_grant", + error, + )) + } + }; crash_if("after_grant_delivery"); if let Some(native_session_id) = accepted.native_session_id.as_ref() { self.registry @@ -1141,6 +1228,75 @@ impl Supervisor { }) } + async fn settle_rejected_initial_inputs( + &self, + handle: &crate::registry::OwnedRuntimeHandle, + host: &AuthenticatedHost, + grant: &crate::registry::ExecutionGrantRecord, + ) -> Result<(), RuntimeError> { + let unavailable = || { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initial provider input rejection is unconfirmed; retain ownership and reconcile", + ) + }; + self.registry + .validate_rejected_initial_inputs(handle.incarnation_id().clone()) + .await + .map_err(|_| unavailable())?; + let status = self + .send_authenticated_host_command( + handle.incarnation_id().clone(), + handle.runtime_dir(), + host, + HostCommand::Status { + incarnation_id: handle.incarnation_id().clone(), + }, + ) + .await + .map_err(|_| unavailable())?; + let never_dispatched = matches!( + status, + HostResult::Status { + host_boot_id, + worker_pid: None, + worker_launch_count: 0, + max_execution_generation, + native_session_id: None, + .. + } if host_boot_id == grant.host_boot_id + && max_execution_generation < grant.execution_generation + ); + if !never_dispatched { + return Err(unavailable()); + } + self.backend + .request_stop(handle, 2) + .await + .map_err(|_| unavailable())?; + if !self + .verify_empty_with_budget(handle, Duration::from_secs(3)) + .await + .map_err(|_| unavailable())? + { + return Err(unavailable()); + }; + self.backend + .discard_rejected_provider_inputs(handle) + .await + .map_err(|_| unavailable())?; + self.registry + .settle_rejected_initial_inputs(handle.incarnation_id().clone()) + .await + .map_err(|_| unavailable())?; + append_event( + &self.config.lifecycle_log, + "supervisor.provider_inputs.initial_replan_ready", + serde_json::json!({"severity":"info","soulId":handle.soul_id(),"incarnationId":handle.incarnation_id(),"operation":"settle_unconsumed_generation"}), + ); + Ok(()) + } + fn activation_failure( &self, prepared: &PreparedLaunch, From ca9ecb7b19d101b7a0ec8cb95798ba9c76da39f0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 12:47:41 -0700 Subject: [PATCH 197/493] fix: select native winning OpenCode resource members --- .../managed_mcp_capability/source_stage.rs | 173 ++++++++++++++++++ .../src/managed_provider_launch_tests.rs | 13 +- .../opencode_resources.rs | 83 +++------ 3 files changed, 206 insertions(+), 63 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 4c30d4110..6078eb82c 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -12,6 +12,179 @@ mod configured_resource_tests { use super::*; use crate::managed_provider_references::sources::*; use freshell_platform::provider_environment::{copy_environment, CopySelection}; + #[test] + fn managed_provider_duplicate_resource_members_select_only_native_winners_after_both_copies() { + use std::os::unix::fs::PermissionsExt; + let owned = match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(root) => tempfile::tempdir_in(root).unwrap(), + None => tempfile::tempdir().unwrap(), + }; + let home = owned.path().join("home"); + let cwd = owned.path().join("workspace"); + let selected = owned.path().join("external/selected"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&cwd).unwrap(); + std::fs::create_dir_all(selected.join("node_modules/dependency")).unwrap(); + std::fs::write( + selected.join("node_modules/dependency/index.js"), + "module.exports='winning-dependency';", + ) + .unwrap(); + std::fs::write(selected.join("resource"), "winning-resource").unwrap(); + std::fs::write( + selected.join("SKILL.md"), + "---\nname: winning\ndescription: winning skill\n---\nExecute run.\n", + ) + .unwrap(); + std::fs::write(selected.join("run"), "#!/usr/bin/env node\nconst fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('dependency'));").unwrap(); + std::fs::set_permissions(selected.join("run"), std::fs::Permissions::from_mode(0o755)) + .unwrap(); + let instruction = owned.path().join("external/selected.md"); + std::fs::write(&instruction, "winning-instruction").unwrap(); + let config = home.join("opencode.jsonc"); + let plugin = + serde_json::to_string(&format!("file://{}", selected.join("run").display())).unwrap(); + let instruction_json = serde_json::to_string(&instruction).unwrap(); + let skills_json = serde_json::to_string(&selected).unwrap(); + let obsolete = owned.path().join("obsolete-missing"); + let obsolete_json = serde_json::to_string(&obsolete).unwrap(); + let cases = [ + format!( + r#"{{"plugin":[{obsolete_json}],"plu\u0067in":[[{plugin},{{"instructions":["/unselected-options"]}}]]}}"# + ), + format!( + r#"{{"instructions":[{obsolete_json}],/* final wins */"instructions":[{instruction_json}]}}"# + ), + format!( + r#"{{"skills":{{"paths":[{obsolete_json}]}},"skills":{{"paths":[{skills_json}]}}}}"# + ), + format!(r#"{{"skills":{{"paths":[{obsolete_json}],"pa\u0074hs":[{skills_json}]}}}}"#), + ]; + for (index, raw) in cases.iter().enumerate() { + std::fs::write(&config, raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Home, + }], + &cwd, + &home, + &|_, _, _| None, + ) + .unwrap_or_else(|e| { + panic!( + "native winning member must discard obsolete target: {} {}", + e.field, e.cause + ) + }); + assert_eq!(plan[0].configured_resources.len(), 1); + assert!(!plan[0].configured_resources.contains(&obsolete)); + let units = + resource_units(&home, &home.join(".config/opencode"), &cwd, &plan, None).unwrap(); + let stage = owned.path().join(format!("stage-{index}")); + let host = owned.path().join(format!("host-{index}")); + copy_environment( + &units + .iter() + .map(|(source, mount)| CopySelection { + source: source.clone(), + destination_relative: mount.stage_relative_path.clone().into(), + }) + .collect::>(), + &stage, + &[], + Default::default(), + ) + .unwrap(); + copy_environment( + &units + .iter() + .map(|(_, mount)| CopySelection { + source: stage.join(&mount.stage_relative_path), + destination_relative: mount.provider_relative_path.clone().into(), + }) + .collect::>(), + &host, + &[], + Default::default(), + ) + .unwrap(); + let unit = units + .iter() + .find(|(source, _)| source == if index == 1 { &instruction } else { &selected }) + .unwrap(); + let projected = host.join(&unit.1.provider_relative_path); + let hidden = owned.path().join("hidden-original"); + std::fs::rename(owned.path().join("external"), &hidden).unwrap(); + if index == 1 { + assert_eq!(std::fs::read(&projected).unwrap(), b"winning-instruction"); + } else { + let output = std::process::Command::new(projected.join("run")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + assert!( + output.status.success(), + "winning resource/dependency executable failed: {output:?}" + ); + assert_eq!(output.stdout, b"winning-resource:winning-dependency"); + assert!(std::fs::read_to_string(projected.join("SKILL.md")) + .unwrap() + .contains("name: winning")); + } + std::fs::rename(&hidden, owned.path().join("external")).unwrap(); + assert_eq!(std::fs::read_to_string(&config).unwrap(), *raw); + } + for field in ["plugin", "instructions", "skills-object", "skills-paths"] { + for final_missing in [false, true] { + let final_value = if final_missing { + format!("[{obsolete_json}]") + } else { + "[]".into() + }; + let raw = match field { + "plugin" | "instructions" => { + format!(r#"{{"{field}":[{instruction_json}],"{field}":{final_value}}}"#) + } + "skills-object" => format!( + r#"{{"skills":{{"paths":[{skills_json}]}},"skills":{{"paths":{final_value}}}}}"# + ), + _ => { + format!(r#"{{"skills":{{"paths":[{skills_json}],"paths":{final_value}}}}}"#) + } + }; + std::fs::write(&config, &raw).unwrap(); + let result = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Home, + }], + &cwd, + &home, + &|_, _, _| None, + ); + if final_missing { + let error = result + .err() + .expect("missing final member must prevent publication"); + assert!(error.cause.contains("missing")); + assert_eq!(error.declaring_source, config); + } else { + assert!(result + .unwrap() + .iter() + .all(|source| source.configured_resources.is_empty())); + } + assert_eq!(std::fs::read_to_string(&config).unwrap(), raw); + } + } + } + #[test] fn managed_provider_configured_external_resources_fail_before_publication_when_unavailable() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index c4b70bb93..490a451d0 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3593,7 +3593,18 @@ export default async (input,options)=>({config:async(config)=>{ json!({"type":"local","command":["node","server.cjs"],"cwd":"~/mcp-cwd"}); original["instructions"] = json!([external_instruction]); original["skills"] = json!({"paths":[selected_skills,"../../scenarios/external-native-resources/selected-skills"]}); - std::fs::write(&original_config, serde_json::to_vec(&original).unwrap()).unwrap(); + // Native ConfigParse/jsonc-parser keeps only the final resource member. + // Obsolete targets do not exist; the existing actual plugin/instruction/ + // skill consumers below must still consume each final selected closure. + let ordinary = serde_json::to_string(&original).unwrap().replace( + "\"skills\":{", + "\"skills\":{\"paths\":[\"/missing-obsolete-nested-skills\"],", + ); + let duplicate_resources = format!( + "{{\"plugin\":[\"file:///missing-obsolete-plugin.mjs\"],\"instructions\":[\"/missing-obsolete-instructions.md\"],\"skills\":{{\"paths\":[\"/missing-obsolete-skills\"]}},{}", + &ordinary[1..], + ); + std::fs::write(&original_config, duplicate_resources.as_bytes()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":format!("{:x}",Sha256::digest(b"native-mcp-resource:native-mcp-dependency"))}); let explicit_config = fixture.join("workspace/selected-config.json"); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 921d661e8..1399bca9a 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -64,70 +64,29 @@ impl Cursor<'_> { // A finite field visitor rather than decoded/serialized document rewriting. // Plugin tuple options (and nested unrelated fields) are never path schemas. fn fields(raw: &str) -> Result, &'static str> { - let mut cursor = Cursor { raw, at: 0 }; - let mut depth = 0usize; + // Native JSONC parsing assigns each member in order: only the final + // spelling owns a configured capability, including escaped duplicate keys. + // Reuse raw member spans so plugin tuple options and source origins survive. + let top = object_members(raw, 0)?; let mut result = Vec::new(); - while let Some(token) = cursor.next() { - match token { - Token::Punctuation(b'{' | b'[') => depth += 1, - Token::Punctuation(b'}' | b']') => depth = depth.saturating_sub(1), - Token::String(start, end) if depth == 1 && end - start <= 64 => { - let key: String = serde_json::from_str(&raw[start..end]) - .map_err(|_| "invalid selected resource field")?; - let mut selected = Cursor { raw, at: cursor.at }; - if !matches!(selected.next(), Some(Token::Punctuation(b':'))) { - continue; - } - match key.as_str() { - "plugin" | "instructions" - if matches!(selected.next(), Some(Token::Punctuation(b'['))) => - { - let field = if key == "plugin" { - "OpenCode plugin" - } else { - "OpenCode instructions" - }; - resource_array(&mut selected, field, &mut result)?; - cursor = selected; - } - "skills" if matches!(selected.next(), Some(Token::Punctuation(b'{'))) => { - let mut skills_depth = 1usize; - while skills_depth != 0 { - match selected.next().ok_or("incomplete selected skills field")? { - Token::Punctuation(b'{' | b'[') => skills_depth += 1, - Token::Punctuation(b'}' | b']') => skills_depth -= 1, - Token::String(start, end) - if skills_depth == 1 && end - start <= 64 => - { - let key: String = serde_json::from_str(&raw[start..end]) - .map_err(|_| "invalid selected skills field")?; - if key != "paths" { - continue; - } - let mut paths = Cursor { - raw, - at: selected.at, - }; - if matches!(paths.next(), Some(Token::Punctuation(b':'))) - && matches!(paths.next(), Some(Token::Punctuation(b'['))) - { - resource_array( - &mut paths, - "OpenCode skills.paths", - &mut result, - )?; - selected = paths; - } - } - _ => {} - } - } - cursor = selected; - } - _ => {} - } + for (name, field) in [ + ("plugin", "OpenCode plugin"), + ("instructions", "OpenCode instructions"), + ] { + if let Some(value) = member(raw, &top, name) { + let mut selected = Cursor { raw: value, at: 0 }; + if matches!(selected.next(), Some(Token::Punctuation(b'['))) { + resource_array(&mut selected, field, &mut result)?; + } + } + } + if let Some(skills) = member(raw, &top, "skills") { + let entries = object_members(skills, 0)?; + if let Some(paths) = member(skills, &entries, "paths") { + let mut selected = Cursor { raw: paths, at: 0 }; + if matches!(selected.next(), Some(Token::Punctuation(b'['))) { + resource_array(&mut selected, "OpenCode skills.paths", &mut result)?; } - _ => {} } } Ok(result) From f95241049dc2de04dbe21042ab339ba86804f064 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:08:14 -0700 Subject: [PATCH 198/493] fix: account live managed provider source views per occurrence --- .../managed_mcp_capability/source_stage.rs | 2 +- .../src/managed_provider_launch_tests.rs | 49 ++++- .../opencode_resources.rs | 4 +- .../src/provider_input_preparation.rs | 167 +++++++++++++++++- 4 files changed, 211 insertions(+), 11 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 6078eb82c..c33db1dd1 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -53,7 +53,7 @@ mod configured_resource_tests { r#"{{"plugin":[{obsolete_json}],"plu\u0067in":[[{plugin},{{"instructions":["/unselected-options"]}}]]}}"# ), format!( - r#"{{"instructions":[{obsolete_json}],/* final wins */"instructions":[{instruction_json}]}}"# + r#"{{"instructions":[{obsolete_json}],/* final wins */"\u0069\u006e\u0073\u0074\u0072\u0075\u0063\u0074\u0069\u006f\u006e\u0073":[{instruction_json}]}}"# ), format!( r#"{{"skills":{{"paths":[{obsolete_json}]}},"skills":{{"paths":[{skills_json}]}}}}"# diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 490a451d0..fc8fba0fe 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3580,7 +3580,7 @@ export default async (input,options)=>({config:async(config)=>{ const output=execFileSync(path.join(path.dirname(skill.location),'scripts/run')); await input.client._client.get({url:'/mcp'}); const mcp=digest(fs.readFileSync(options.mcpReceipt)); - fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction),nativeSkill:true,skillContent:digest(skill.content),skillExecution:digest(output),nativeMcpCwd:mcp})); + fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction),nativeLiveViews:config.agent.liveCapacity.description==='native-live-capacity-nonce',nativeSkill:true,skillContent:digest(skill.content),skillExecution:digest(output),nativeMcpCwd:mcp})); } catch(error) {fs.writeFileSync(options.receipt+'.failure',JSON.stringify({operation:'native_skill_consumer',type:error instanceof TypeError?'TypeError':error.code==='ENOENT'?'ENOENT':'native_skill_unavailable'}));} },0); }}); @@ -3600,13 +3600,41 @@ export default async (input,options)=>({config:async(config)=>{ "\"skills\":{", "\"skills\":{\"paths\":[\"/missing-obsolete-nested-skills\"],", ); + let ordinary = ordinary.replace( + "\"instructions\":", + "\"\\u0069\\u006e\\u0073\\u0074\\u0072\\u0075\\u0063\\u0074\\u0069\\u006f\\u006e\\u0073\":", + ); let duplicate_resources = format!( "{{\"plugin\":[\"file:///missing-obsolete-plugin.mjs\"],\"instructions\":[\"/missing-obsolete-instructions.md\"],\"skills\":{{\"paths\":[\"/missing-obsolete-skills\"]}},{}", &ordinary[1..], ); std::fs::write(&original_config, duplicate_resources.as_bytes()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); - let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":format!("{:x}",Sha256::digest(b"native-mcp-resource:native-mcp-dependency"))}); + let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeLiveViews":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":format!("{:x}",Sha256::digest(b"native-mcp-resource:native-mcp-dependency"))}); + // Five logical native aliases share one accepted immutable 16 MiB raw + // source. The actual controller stage and actual host must not turn the + // 80 MiB cumulative read volume into a simultaneous-source limit. + let repeated_static_source = fixture.join("source-home/.config/opencode/static-capacity.jsonc"); + let mut static_capacity = + r#"{"agent":{"liveCapacity":{"description":"native-live-capacity-nonce"}}}"#.to_string(); + // JSONC block padding keeps the same inclusive physical-source size while + // avoiding repeated large strict-JSON renderings in native discovery. + static_capacity.push_str("/*"); + static_capacity.push_str(&"x".repeat(16 * 1024 * 1024 - static_capacity.len() - 2)); + static_capacity.push_str("*/"); + assert_eq!(static_capacity.len(), 16 * 1024 * 1024); + assert_eq!( + freshell_platform::opencode_config::parse_jsonc_object(&static_capacity).unwrap()["agent"] + ["liveCapacity"]["description"], + "native-live-capacity-nonce" + ); + std::fs::write(&repeated_static_source, &static_capacity).unwrap(); + let capacity_aliases = [ + fixture.join("source-home/.config/opencode/config.json"), + fixture.join("source-home/.config/opencode/opencode.jsonc"), + fixture.join("workspace/opencode.jsonc"), + nested.join("opencode.jsonc"), + ]; let explicit_config = fixture.join("workspace/selected-config.json"); let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; std::fs::write(&explicit_config, explicit_bytes).unwrap(); @@ -3847,6 +3875,15 @@ export default {{id:'closed-directory-discovery',async server(){{ ); } } + // Activate capacity aliases only after the unrelated unavailable-resource + // controls; each refusal still tests its selected field without rescanning + // the large static source. Actual launch preparation consumes all aliases. + for alias in capacity_aliases + .into_iter() + .chain([explicit_directory.join("opencode.jsonc")]) + { + std::os::unix::fs::symlink(&repeated_static_source, alias).unwrap(); + } let terminal = controller .launch(request) .await @@ -4042,6 +4079,10 @@ export default {{id:'closed-directory-discovery',async server(){{ ); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); assert_eq!(std::fs::read(&shared_source).unwrap(), shared_bytes); + assert_eq!( + std::fs::read_to_string(&repeated_static_source).unwrap(), + static_capacity + ); assert!( std::fs::symlink_metadata(fixture.join("workspace/opencode.json")) .unwrap() @@ -4167,6 +4208,10 @@ export default {{id:'closed-directory-discovery',async server(){{ ); assert_eq!(std::fs::read(&original_config).unwrap(), original_bytes); assert_eq!(std::fs::read(&shared_source).unwrap(), shared_bytes); + assert_eq!( + std::fs::read_to_string(&repeated_static_source).unwrap(), + static_capacity + ); assert!( std::fs::symlink_metadata(fixture.join("workspace/opencode.json")) .unwrap() diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 1399bca9a..dc040d1d0 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -143,7 +143,9 @@ fn member<'a>( name: &str, ) -> Option<&'a str> { members.iter().rev().find_map(|(key, value)| { - if key.len() <= 64 + // Every ASCII name character can use a six-byte Unicode escape. + // Bound the finite name before decoding, without excluding valid keys. + if key.len() <= 2 + 6 * name.len() && serde_json::from_str::(&raw[key.clone()]) .ok() .as_deref() diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index e15211719..e063f883a 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -370,19 +370,31 @@ pub fn prepare( .map_err(|error| error.message)?; generated.insert(name.into(), flag.runtime_destination.clone()); } - let mut views = 0usize; for source in manifest.sources { - let remaining = (64 * 1024 * 1024usize) - .checked_sub(views) + // Each occurrence owns only this iteration's raw/expanded/output views. + // Preserve native lane evaluation, but release those charges when the + // iteration ends. Only returned child strings and their validation map + // survive; the shared input profile bounds each map to 256 KiB. + let retained_view_bytes = budget + .iter() + .map(|(_, value)| value.len()) + .chain(generated.values().map(String::len)) + .try_fold(0usize, |bytes, length| bytes.checked_add(length)) .ok_or("source view profile overflow")?; + let remaining = + (freshell_platform::provider_input_profiles::MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES + as usize) + .checked_sub(retained_view_bytes) + .ok_or("source view profile overflow")?; + // Raw is at most 16 MiB. The shared transformer checks its two live + // effective views against 32 MiB before appending. Together with the + // bounded retained maps, these fit the 64 MiB simultaneous profile. let raw = read_private( stage, &source.unexpanded_relative_path, - remaining.min(16 * 1024 * 1024), + remaining + .min(freshell_platform::provider_input_profiles::MAX_SOURCE_VIEW_BYTES as usize), )?; - views = views - .checked_add(raw.len()) - .ok_or("source view profile overflow")?; let raw = String::from_utf8(raw).map_err(|_| "prepared source is not UTF-8")?; let recipe = GeneratedFileRecipe { generated_name: source.generated_name.clone().unwrap_or_default(), @@ -524,6 +536,147 @@ mod tests { use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + #[test] + fn provider_secret_repeated_native_source_views_release_live_storage_between_occurrences() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let stage = owned.path().join("stage"); + std::fs::create_dir(&home).unwrap(); + std::fs::create_dir_all(stage.join(".freshell/source-views")).unwrap(); + let limit = 16 * 1024 * 1024; + // Native-valid source at the inclusive per-view limit. Padding remains + // raw JSON whitespace, rather than an invalid synthetic schema field. + let mut raw = + r#"{"agent":{"liveCapacity":{"description":"native-source-nonce"}}}"#.to_string(); + raw.push_str(&" ".repeat(limit - raw.len())); + assert_eq!(raw.len(), limit); + let decoded: serde_json::Value = serde_json::from_str(&raw).unwrap(); + assert_eq!( + decoded["agent"]["liveCapacity"]["description"], + "native-source-nonce" + ); + let relative = ".freshell/source-views/source"; + std::fs::write(stage.join(relative), &raw).unwrap(); + let original_digest = Sha256::digest(raw.as_bytes()); + let sources = (0..8) + .map(|index| SourceView { + logical_path: home.join(format!("alias-{index}/opencode.jsonc")), + discovery_role: DiscoveryRole::Global, + native_destination: None, + lane: if index == 7 { + SourceLane::Tui + } else { + SourceLane::Main + }, + unexpanded_relative_path: relative.into(), + template_relative_path: relative.into(), + template_sha256: Some(format!("{original_digest:x}")), + generated_name: None, + occurrences: vec![], + referenced_variables: vec![], + path_variables: vec![], + original_form: true, + }) + .collect(); + let mut manifest = SourceManifest { + version: 1, + cwd: home.clone(), + original_home: home.clone(), + global_config_root: None, + sources, + files: vec![], + frozen_inputs: vec![], + user_owns_freshell_mcp: false, + native_source_flags: vec![], + resource_roots: vec![], + rebind_enabled: true, + inline_edits: Default::default(), + }; + let bytes = bounded_json_bytes(&manifest, 64 * 1024 * 1024).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + let generation = format!("grant-{}", uuid::Uuid::new_v4()); + let context: ProviderLaunchContext = serde_json::from_value(serde_json::json!({ + "preparation":{"opencode":{"project_config":[],"tui_config":null,"tui_source":null,"inline_config":false}}, + "mcpCapability":{"grantId":generation,"endpoint":"http://host.docker.internal:4000","providerRelativePath":".freshell/mcp-capability.json"}, + "preparedInputs":{"generationId":generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&bytes)),"sourceCount":0,"aliasCount":0,"recipeCount":0}} + })).unwrap(); + let mut secrets = ResolvedProviderSecrets::default(); + let generated = prepare(Some(&context), &mut secrets, &ProviderPathInputs::default(), &stage, &home) + .expect("128 MiB cumulative reads of eight 16 MiB occurrences fit the simultaneous view profile"); + assert!( + generated.is_empty(), + "direct native sources must not become child environment values" + ); + assert!(secrets.opencode_plan.is_some()); + assert_eq!( + Sha256::digest(std::fs::read(stage.join(relative)).unwrap()), + original_digest + ); + assert_eq!( + std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + bytes + ); + // Real over-limit view fails before dispatch, without publishing a plan + // or changing accepted native data and the generation manifest. + std::fs::write(stage.join(relative), format!("{raw} ")).unwrap(); + let mut refused = ResolvedProviderSecrets::default(); + let error = prepare( + Some(&context), + &mut refused, + &ProviderPathInputs::default(), + &stage, + &home, + ) + .unwrap_err(); + assert!(error.contains("byte profile"), "{error}"); + assert!(refused.opencode_plan.is_none()); + assert_eq!( + std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + bytes + ); + // A complete raw view can fit while its actual env expansion cannot. + // Refuse that expansion before a plan becomes a dispatch target. + let prefix = r#"{"agent":{"liveCapacity":{"description":"{env:VALUE}"#; + let suffix = "\"}}}"; + let expansion_raw = format!( + "{prefix}{}{suffix}", + "x".repeat(limit - prefix.len() - suffix.len()) + ); + assert_eq!(expansion_raw.len(), limit); + assert!(serde_json::from_str::(&expansion_raw).is_ok()); + std::fs::write(stage.join(relative), &expansion_raw).unwrap(); + for source in &mut manifest.sources { + source.referenced_variables = vec!["VALUE".into()]; + } + let bytes = bounded_json_bytes(&manifest, 64 * 1024 * 1024).unwrap(); + std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); + let mut context = context; + context.prepared_inputs.as_mut().unwrap().manifest.sha256 = + format!("{:x}", Sha256::digest(&bytes)); + let mut refused = ResolvedProviderSecrets::default(); + refused + .environment + .insert("VALUE".into(), "v".repeat(64 * 1024)); + let error = prepare( + Some(&context), + &mut refused, + &ProviderPathInputs::default(), + &stage, + &home, + ) + .unwrap_err(); + assert!(error.contains("recipe profile limit"), "{error}"); + assert!(refused.opencode_plan.is_none()); + assert_eq!( + std::fs::read_to_string(stage.join(relative)).unwrap(), + expansion_raw + ); + assert_eq!( + std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + bytes + ); + } + #[cfg(target_os = "linux")] #[test] fn provider_secret_actual_merged_profile_reaches_guarded_child() { From bf7a89feea7af62dbcb0049285a3598d91c09586 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:20:11 -0700 Subject: [PATCH 199/493] test: mount shared native capacity fixture backing source --- crates/freshell-server/src/managed_provider_launch_tests.rs | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index fc8fba0fe..fa9ba3cb2 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3614,7 +3614,10 @@ export default async (input,options)=>({config:async(config)=>{ // Five logical native aliases share one accepted immutable 16 MiB raw // source. The actual controller stage and actual host must not turn the // 80 MiB cumulative read volume into a simultaneous-source limit. - let repeated_static_source = fixture.join("source-home/.config/opencode/static-capacity.jsonc"); + // Keep the shared backing file in the workspace mounted for the provider. + // Workspace aliases must remain resolvable during the provider-owned + // refresh, after the controller's original HOME is outside its mounts. + let repeated_static_source = fixture.join("workspace/static-capacity.jsonc"); let mut static_capacity = r#"{"agent":{"liveCapacity":{"description":"native-live-capacity-nonce"}}}"#.to_string(); // JSONC block padding keeps the same inclusive physical-source size while From eaf66d5e3c47d812f83aec4a57966566ae0487c9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:27:27 -0700 Subject: [PATCH 200/493] test: isolate native capacity aliases within source roots --- .../src/managed_provider_launch_tests.rs | 28 +++++++++++++------ 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index fa9ba3cb2..8fdf8ece8 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3611,9 +3611,9 @@ export default async (input,options)=>({config:async(config)=>{ std::fs::write(&original_config, duplicate_resources.as_bytes()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeLiveViews":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":format!("{:x}",Sha256::digest(b"native-mcp-resource:native-mcp-dependency"))}); - // Five logical native aliases share one accepted immutable 16 MiB raw - // source. The actual controller stage and actual host must not turn the - // 80 MiB cumulative read volume into a simultaneous-source limit. + // Five logical native sources share three immutable 16 MiB raw views. + // Their 48 MiB retained controller views fit its simultaneous profile; + // the host must not turn 80 MiB of sequential reads into a live limit. // Keep the shared backing file in the workspace mounted for the provider. // Workspace aliases must remain resolvable during the provider-owned // refresh, after the controller's original HOME is outside its mounts. @@ -3633,11 +3633,10 @@ export default async (input,options)=>({config:async(config)=>{ ); std::fs::write(&repeated_static_source, &static_capacity).unwrap(); let capacity_aliases = [ - fixture.join("source-home/.config/opencode/config.json"), - fixture.join("source-home/.config/opencode/opencode.jsonc"), fixture.join("workspace/opencode.jsonc"), nested.join("opencode.jsonc"), ]; + let global_capacity_source = fixture.join("source-home/.config/opencode/config.json"); let explicit_config = fixture.join("workspace/selected-config.json"); let explicit_bytes = br#"{"agent":{"probe":{"description":"{file:./selected-rule}"}}}"#; std::fs::write(&explicit_config, explicit_bytes).unwrap(); @@ -3881,10 +3880,15 @@ export default {{id:'closed-directory-discovery',async server(){{ // Activate capacity aliases only after the unrelated unavailable-resource // controls; each refusal still tests its selected field without rescanning // the large static source. Actual launch preparation consumes all aliases. - for alias in capacity_aliases - .into_iter() - .chain([explicit_directory.join("opencode.jsonc")]) - { + std::fs::write(&global_capacity_source, &static_capacity).unwrap(); + std::os::unix::fs::symlink( + "config.json", + fixture.join("source-home/.config/opencode/opencode.jsonc"), + ) + .unwrap(); + let directory_capacity_source = explicit_directory.join("opencode.jsonc"); + std::fs::write(&directory_capacity_source, &static_capacity).unwrap(); + for alias in capacity_aliases { std::os::unix::fs::symlink(&repeated_static_source, alias).unwrap(); } let terminal = controller @@ -4086,6 +4090,9 @@ export default {{id:'closed-directory-discovery',async server(){{ std::fs::read_to_string(&repeated_static_source).unwrap(), static_capacity ); + for source in [&global_capacity_source, &directory_capacity_source] { + assert_eq!(std::fs::read_to_string(source).unwrap(), static_capacity); + } assert!( std::fs::symlink_metadata(fixture.join("workspace/opencode.json")) .unwrap() @@ -4215,6 +4222,9 @@ export default {{id:'closed-directory-discovery',async server(){{ std::fs::read_to_string(&repeated_static_source).unwrap(), static_capacity ); + for source in [&global_capacity_source, &directory_capacity_source] { + assert_eq!(std::fs::read_to_string(source).unwrap(), static_capacity); + } assert!( std::fs::symlink_metadata(fixture.join("workspace/opencode.json")) .unwrap() From 6211b481f5c0e0c342ceac0eb49bf749ff160e08 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:30:22 -0700 Subject: [PATCH 201/493] test: allow bounded OpenCode capacity qualification time --- scripts/testing/managed-provider-launch-sandbox.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index d0b69a631..7a4901602 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -219,10 +219,12 @@ def create(name,*arguments): if not gateway: raise RuntimeError("closed controller has no owned bridge gateway") (root/"controller-gateway.json").write_text(json.dumps({"address":gateway})) - # The Claude fixture consumes an ordinary file at the full128MiB - # boundary through two independent controller/native routes. + # Claude consumes an ordinary file at the full128MiB boundary; + # OpenCode qualifies sequential16MiB source views on both routes. # Keep a finite overall wait separate from each helper's deadline. - waited=docker("wait",controller,timeout=360 if args.provider=="claude" else 180) + controller_wait_timeout=360 if args.provider in {"claude","opencode"} else 180 + receipt["controllerWaitTimeoutSeconds"]=controller_wait_timeout + waited=docker("wait",controller,timeout=controller_wait_timeout) exit_code=int(waited.stdout.strip()) outcome=docker("logs",controller,check=False) receipt["exitCode"]=exit_code From 6a30ee8be8dd48bd10f8ab268e5620e66dec7786 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:58:55 -0700 Subject: [PATCH 202/493] test: bound copied resource execution around transient writers --- .../managed_mcp_capability/source_stage.rs | 65 +++++++++++++++++-- 1 file changed, 61 insertions(+), 4 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index c33db1dd1..976ac6d76 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -12,6 +12,28 @@ mod configured_resource_tests { use super::*; use crate::managed_provider_references::sources::*; use freshell_platform::provider_environment::{copy_environment, CopySelection}; + + fn execute_selected_resource(path: &Path) -> std::io::Result { + let mut command = std::process::Command::new(path); + command + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()); + // Match the existing copied-script helper's finite spawn retry, while + // retaining this consumer's isolated environment. ETXTBSY has no child. + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); + loop { + match command.output() { + Err(error) + if error.raw_os_error() == Some(libc::ETXTBSY) + && std::time::Instant::now() < deadline => + { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + result => return result, + } + } + } + #[test] fn managed_provider_duplicate_resource_members_select_only_native_winners_after_both_copies() { use std::os::unix::fs::PermissionsExt; @@ -121,11 +143,19 @@ mod configured_resource_tests { if index == 1 { assert_eq!(std::fs::read(&projected).unwrap(), b"winning-instruction"); } else { - let output = std::process::Command::new(projected.join("run")) - .env_clear() - .env("PATH", std::env::var_os("PATH").unwrap()) - .output() + // A concurrent fork can retain this owned copy's writer until + // exec closes it. Exercise that exact transient spawn error. + let writer = std::fs::OpenOptions::new() + .write(true) + .open(projected.join("run")) .unwrap(); + let release = std::thread::spawn(move || { + std::thread::sleep(std::time::Duration::from_millis(50)); + drop(writer); + }); + let result = execute_selected_resource(&projected.join("run")); + release.join().unwrap(); + let output = result.unwrap(); assert!( output.status.success(), "winning resource/dependency executable failed: {output:?}" @@ -134,6 +164,33 @@ mod configured_resource_tests { assert!(std::fs::read_to_string(projected.join("SKILL.md")) .unwrap() .contains("name: winning")); + if index == 0 { + let writer = std::fs::OpenOptions::new() + .write(true) + .open(projected.join("run")) + .unwrap(); + let started = std::time::Instant::now(); + let blocked = execute_selected_resource(&projected.join("run")); + drop(writer); + assert_eq!(blocked.unwrap_err().raw_os_error(), Some(libc::ETXTBSY)); + assert!(started.elapsed() < std::time::Duration::from_secs(2)); + std::fs::set_permissions( + projected.join("run"), + std::fs::Permissions::from_mode(0o600), + ) + .unwrap(); + assert_eq!( + execute_selected_resource(&projected.join("run")) + .unwrap_err() + .kind(), + std::io::ErrorKind::PermissionDenied + ); + std::fs::set_permissions( + projected.join("run"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + } } std::fs::rename(&hidden, owned.path().join("external")).unwrap(); assert_eq!(std::fs::read_to_string(&config).unwrap(), *raw); From 55b6464245622cf36556f226a906a015dd9d47b6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 15:06:29 -0700 Subject: [PATCH 203/493] fix: select final decoded OpenCode MCP server entries --- .../managed_mcp_capability/source_stage.rs | 22 ++++++++++++++++++- .../src/managed_provider_launch_tests.rs | 4 ++++ .../opencode_resources.rs | 14 +++++++++++- 3 files changed, 38 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 976ac6d76..3cbfb75be 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -81,6 +81,9 @@ mod configured_resource_tests { r#"{{"skills":{{"paths":[{obsolete_json}]}},"skills":{{"paths":[{skills_json}]}}}}"# ), format!(r#"{{"skills":{{"paths":[{obsolete_json}],"pa\u0074hs":[{skills_json}]}}}}"#), + format!( + r#"{{"mcp":{{"same":{{"type":"local","cwd":{obsolete_json}}},"sa\u006de":{{"type":"local","cwd":{skills_json},"command":["node","run"]}}}}}}"# + ), ]; for (index, raw) in cases.iter().enumerate() { std::fs::write(&config, raw).unwrap(); @@ -195,7 +198,13 @@ mod configured_resource_tests { std::fs::rename(&hidden, owned.path().join("external")).unwrap(); assert_eq!(std::fs::read_to_string(&config).unwrap(), *raw); } - for field in ["plugin", "instructions", "skills-object", "skills-paths"] { + for field in [ + "plugin", + "instructions", + "skills-object", + "skills-paths", + "mcp-name", + ] { for final_missing in [false, true] { let final_value = if final_missing { format!("[{obsolete_json}]") @@ -209,6 +218,17 @@ mod configured_resource_tests { "skills-object" => format!( r#"{{"skills":{{"paths":[{skills_json}]}},"skills":{{"paths":{final_value}}}}}"# ), + "mcp-name" => { + let winner = if final_missing { + format!(r#"{{"type":"local","cwd":{obsolete_json}}}"#) + } else { + r#"{"type":"remote","url":"https://example.test/mcp","enabled":false}"# + .into() + }; + format!( + r#"{{"mcp":{{"same":{{"type":"local","cwd":{obsolete_json}}},"\u0073\u0061\u006d\u0065":{winner}}}}}"# + ) + } _ => { format!(r#"{{"skills":{{"paths":[{skills_json}],"paths":{final_value}}}}}"#) } diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 8fdf8ece8..16985bfff 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3604,6 +3604,10 @@ export default async (input,options)=>({config:async(config)=>{ "\"instructions\":", "\"\\u0069\\u006e\\u0073\\u0074\\u0072\\u0075\\u0063\\u0074\\u0069\\u006f\\u006e\\u0073\":", ); + let ordinary = ordinary.replace( + "\"mcp\":{", + "\"mcp\":{\"closed-cwd-probe\":{\"type\":\"local\",\"cwd\":\"/missing-obsolete-mcp\"},\"disabled-duplicate\":{\"type\":\"local\",\"cwd\":\"/missing-obsolete-disabled-mcp\"},\"disabled-duplicate\":{\"type\":\"remote\",\"url\":\"https://example.test/mcp\",\"enabled\":false},", + ).replace("\"closed-cwd-probe\":{\"command\":", "\"closed-cwd-\\u0070robe\":{\"command\":"); let duplicate_resources = format!( "{{\"plugin\":[\"file:///missing-obsolete-plugin.mjs\"],\"instructions\":[\"/missing-obsolete-instructions.md\"],\"skills\":{{\"paths\":[\"/missing-obsolete-skills\"]}},{}", &ordinary[1..], diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index dc040d1d0..c1aa9aa50 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -197,7 +197,19 @@ fn nested_fields(raw: &str, tui: bool) -> Result, &' } } if let Some(mcp) = member(raw, &top, "mcp") { - for (_, range) in object_members(mcp, 0)? { + // Native Object.entries sees only the last value for each decoded + // server name. Discard overwritten entries before selecting their cwd. + let mut names = std::collections::BTreeSet::new(); + let mut winners = Vec::new(); + for (name, range) in object_members(mcp, 0)?.into_iter().rev() { + let name = serde_json::from_str::(&mcp[name]) + .map_err(|_| "invalid selected MCP server name")?; + if names.insert(name) { + winners.push(range); + } + } + drop(names); + for range in winners.into_iter().rev() { let entry = &mcp[range]; let entries = object_members(entry, 0)?; if member(entry, &entries, "type") From 6535514233f4ab7ded0a2f1cacc08018032f30e7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 15:18:53 -0700 Subject: [PATCH 204/493] fix: retain managed preparations after uncertain initial launch --- .../freshell-server/src/fresh_agent_proxy.rs | 12 +- .../src/managed_mcp_capability.rs | 25 + crates/freshell-server/src/managed_runtime.rs | 13 +- .../managed_runtime_opencode_replan_tests.rs | 594 ++++++++++++++++++ 4 files changed, 635 insertions(+), 9 deletions(-) diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index e86c49a2d..38aea5cb4 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -691,10 +691,14 @@ impl HostedFreshAgentProxy { .launch(request_id, request) .await .map_err(|error| { - let _ = crate::managed_mcp_capability::revoke_mcp_capability( - &mcp_capability.grant_id, - ); - tracing::warn!(?error, "fresh_agent.create.launch_failed"); + // A failed reply cannot establish that this exact + // preparation was never accepted. Preserve its journal and + // stage for existing same-conversation reconciliation. + tracing::warn!(event="provider.inputs.initial_launch.unconfirmed", + provider=provider.as_str(),soul_id=%soul, + grant_id=%mcp_capability.grant_id, + stage="controller",operation="launch",%error, + "Retain provider inputs until dispatch outcome is confirmed"); })? }; let naming_projection = match message.naming_handle.as_deref() { diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 7be3b95d6..07acb12d9 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -147,6 +147,31 @@ pub(crate) fn accept_initial_replan_fixture_preparation( store().commit_preparation(&capability.grant_id, soul, &incarnation, &capability_dir()?) } +#[cfg(test)] +pub(crate) async fn apply_initial_caller_fixture_callback(bytes: &[u8]) -> Result, String> { + let settings = SETTINGS.get().ok_or("fixture controller is unconfigured")?; + let home = std::env::var_os("HOME").map(PathBuf::from).unwrap(); + let client = freshell_runtime_client::RuntimeClient::new( + &settings.control_socket, + &settings.control_secret, + ); + apply_prepared_callback( + bytes, + &settings.control_secret, + store(), + &settings.directory, + &settings.endpoint, + &home, + &client, + ) + .await +} + +#[cfg(test)] +pub(crate) fn initial_caller_fixture_authenticates(scope: &str) -> bool { + store().authenticates(scope) +} + pub(crate) fn configure( bind_host: &str, port: u16, diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index b3c52e71e..a48d6f366 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -431,11 +431,14 @@ impl ManagedTerminalController for ServerManagedRuntimeController { match self.client.launch(request_id, launch).await { Ok(result) => result, Err(error) => { - if let Some(capability) = &mcp_capability { - let _ = crate::managed_mcp_capability::revoke_mcp_capability( - &capability.grant_id, - ); - } + // Launch may have bound/dispatched this generation + // before its reply was lost or completion failed. The + // existing ownership journal must survive reconciliation. + tracing::warn!(event="provider.inputs.initial_launch.unconfirmed", + provider=%request.mode,soul_id=%soul_id, + grant_id=mcp_capability.as_ref().map(|c|c.grant_id.as_str()), + stage="controller",operation="launch",%error, + "Retain provider inputs until dispatch outcome is confirmed"); return Err(error.to_string()); } } diff --git a/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs b/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs index f738a96a0..9ca3d57fa 100644 --- a/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs +++ b/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs @@ -5,6 +5,600 @@ use freshell_runtime_protocol::*; use std::{collections::BTreeMap, os::unix::fs::PermissionsExt}; use tokio::net::UnixListener; +// The actual controller callers, supervisor, durable registry and authenticated +// host protocol participate. Only the daemon/worker is synthetic; no provider +// dispatch is repeated by the lost-reply proxy or the reconciliation probe. +mod uncertain_launch { + use super::*; + use async_trait::async_trait; + use freshell_supervisor::{ + backend::*, + registry::{OwnedRuntimeHandle, Registry}, + service::{Supervisor, SupervisorConfig}, + }; + use futures_util::FutureExt; + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + + #[derive(Default)] + struct State { + dispatches: usize, + fail_completion: bool, + accepted: Option, + host_tasks: Vec>, + host_shutdowns: Vec>, + } + struct Backend(Arc>, PathBuf); + #[async_trait] + impl RuntimeBackend for Backend { + async fn observe_initial_auth_volume( + &self, + _: &freshell_supervisor::registry::OwnedInputReservation, + ) -> Result { + Ok(InitialAuthVolumeObservation::Absent { + daemon_id: DockerDaemonId::parse("fixture-daemon").unwrap(), + }) + } + async fn observe_claude_sources( + &self, + _: &ClaudeSourceOwner, + request: &ObserveClaudeSourcesRequest, + _: &str, + ) -> Result + { + let r = &request.observation_request; + let mut observation = freshell_platform::claude_inputs::observation::observe( + &r.cwd, + &self.1, + r.native_source_runtime, + &r.setting_sources, + ) + .unwrap(); + // Synthetic namespace reports the same fixed provider identity as + // the managed host; the native source evidence remains finite. + observation.home = Some(r.home.clone()); + observation.uid = 65534; + observation.euid = 65534; + observation.gid = 0; + observation.egid = 0; + observation.supplementary_groups = vec![0]; + Ok( + freshell_platform::claude_inputs::observation::ObservationReply { + request_sha256: format!("{:x}", Sha256::digest(serde_json::to_vec(r).unwrap())), + observation, + }, + ) + } + async fn create_stopped( + &self, + spec: &CreateRuntimeSpec, + ) -> Result { + Ok(BackendCreated { + daemon_id: DockerDaemonId::parse("fixture-daemon").unwrap(), + container_id: format!("{:x}", Sha256::digest(spec.incarnation_id.as_str())), + immutable_config_digest: "a".repeat(64), + }) + } + async fn start_host(&self, handle: &OwnedRuntimeHandle) -> Result<(), BackendError> { + let listener = UnixListener::bind(handle.runtime_dir().join("host.sock")).unwrap(); + let secret = std::fs::read(handle.runtime_dir().join("secret")).unwrap(); + let limits = handle.requested_limits(); + let state = self.0.clone(); + let (shutdown, mut stopped) = tokio::sync::oneshot::channel(); + let task = tokio::spawn(async move { + let boot = HostBootId::new(); + loop { + let (mut stream, _) = + tokio::select! {v=listener.accept()=>v.unwrap(), _=&mut stopped=>break}; + let envelope: Envelope = read_frame(&mut stream).await.unwrap(); + let result = match envelope.body { + HostCommand::Hello { + challenge, + incarnation_id, + } => HostResult::Hello { + host_boot_id: boot.clone(), + proof: host_proof(&secret, &challenge, &boot, &incarnation_id), + capabilities: vec![], + effective_limits: limits, + }, + HostCommand::GrantExecution { .. } => { + state.lock().unwrap().dispatches += 1; + HostResult::GrantAccepted { + host_boot_id: boot.clone(), + worker_pid: 1234, + worker_launch_count: 1, + fixture_evidence: serde_json::Value::Null, + native_session_id: Some("native-exact-caller".into()), + } + } + HostCommand::Status { .. } => HostResult::Status { + host_boot_id: boot.clone(), + worker_pid: Some(1234), + worker_launch_count: 1, + max_control_epoch: 100, + max_execution_generation: 100, + fixture_evidence: serde_json::Value::Null, + exited: false, + native_session_id: Some("native-exact-caller".into()), + }, + HostCommand::ProbeRecovery { + resume_spec: Some(spec), + .. + } => HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { + evidence_revision: spec.evidence_revision, + resume_spec: spec, + }), + other => panic!("unexpected fixture host command {other:?}"), + }; + write_frame( + &mut stream, + &HostReply { + request_id: envelope.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + }); + let mut s = self.0.lock().unwrap(); + s.host_tasks.push(task); + s.host_shutdowns.push(shutdown); + Ok(()) + } + async fn commit_prepared_inputs(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + let mut s = self.0.lock().unwrap(); + if std::mem::take(&mut s.fail_completion) { + return Err(BackendError::Unavailable( + "synthetic postaccept completion failure".into(), + )); + } + Ok(()) + } + async fn inspect(&self, h: &OwnedRuntimeHandle) -> Result { + Ok(BackendInspection { + state: BackendRuntimeState::Running, + configured_limits: h.requested_limits(), + }) + } + async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + panic!("accepted initial worker must remain running") + } + async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + panic!("accepted initial worker must remain running") + } + async fn verify_empty(&self, _: &OwnedRuntimeHandle) -> Result { + Ok(false) + } + async fn list_known( + &self, + handles: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + let mut result = vec![]; + for h in handles { + result.push((h.incarnation_id().clone(), self.inspect(h).await?)); + } + Ok(result) + } + async fn read_limits(&self, h: &OwnedRuntimeHandle) -> Result { + Ok(h.requested_limits()) + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + Ok(()) + } + } + + #[test] + fn managed_provider_initial_callers_retain_accepted_inputs_after_uncertain_launch() { + const CASE: &str = "FRESHELL_TEST_UNCERTAIN_INITIAL_CALLER"; + if std::env::var_os(CASE).is_none() { + let mut failures = vec![]; + let routes = [ + ("terminal", "codex"), + ("terminal", "claude"), + ("terminal", "amplifier"), + ("fresh", "codex"), + ("fresh", "claude"), + ("fresh", "kilroy"), + ]; + for (route, provider) in routes { + if route == "terminal" && !cfg!(feature = "managed-provider-qualification") { + continue; + } + if std::env::var("FRESHELL_TEST_UNCERTAIN_CALLER_ROUTE") + .is_ok_and(|selected| selected != route) + { + continue; + } + for failure in ["lost-reply", "completion"] { + let case = format!("{route}:{provider}:{failure}"); + let output=std::process::Command::new(std::env::current_exe().unwrap()).args(["--exact","managed_runtime::opencode_replan_tests::uncertain_launch::managed_provider_initial_callers_retain_accepted_inputs_after_uncertain_launch","--nocapture"]).env(CASE,&case).output().unwrap(); + print!("{}", String::from_utf8_lossy(&output.stdout)); + if !output.status.success() { + failures.push(format!( + "{case}: {}", + String::from_utf8_lossy(&output.stderr) + )); + } + } + } + assert!(failures.is_empty(), "actual caller failures: {failures:?}"); + return; + } + let case = std::env::var(CASE).unwrap(); + let parts = case.split(':').collect::>(); + let (route, provider, failure) = (parts[0], parts[1], parts[2]); + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + runtime.block_on(Box::pin(exercise_case(&case, route, provider, failure))); + } + + async fn exercise_case(case: &str, route: &str, provider: &str, failure: &str) { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let control = owned.path().join("control"); + for p in [ + &home, + &workspace, + &control, + &home.join(".codex"), + &home.join(".claude"), + &home.join(".amplifier"), + ] { + std::fs::create_dir_all(p).unwrap(); + } + std::fs::write(home.join(".codex/config.toml"), "model='fixture'\n").unwrap(); + std::fs::write( + home.join(".claude/settings.json"), + "{\"permissions\":{\"allow\":[\"Read\"]}}", + ) + .unwrap(); + std::fs::write( + home.join(".amplifier/settings.yaml"), + "fixture: ordinary-source\n", + ) + .unwrap(); + std::env::set_var("HOME", &home); + std::env::set_var("FRESHELL_MANAGED_FRESH_AGENT_V1", "1"); + std::env::set_var( + "FRESHELL_MANAGED_PROVIDER_QUALIFICATION", + "claude,codex,amplifier", + ); + for name in [ + "CODEX_HOME", + "CLAUDE_CONFIG_DIR", + "FRESHELL_RUNTIME_CAPABILITY_DIR", + "FRESHELL_MANAGED_CLAUDE_ONECLI_ENV_FILE", + "FRESHELL_MANAGED_CODEX_ONECLI_ENV_FILE", + "FRESHELL_MANAGED_AMPLIFIER_ONECLI_ENV_FILE", + ] { + std::env::remove_var(name); + } + let socket = control.join("supervisor.sock"); + let secret = "fixture-accepted-caller-control"; + let secret_path = control.join("secret"); + std::fs::write(&secret_path, secret).unwrap(); + crate::managed_mcp_capability::configure("0.0.0.0", 45678, &socket, &secret_path).unwrap(); + let registry_root = owned.path().join("registry"); + let registry = Registry::open(®istry_root, None).unwrap(); + let state = Arc::new(std::sync::Mutex::new(State { + fail_completion: failure == "completion", + ..Default::default() + })); + let config = SupervisorConfig { + runtime_root: owned.path().join("runtime"), + control_socket_path: socket.clone(), + host_binary_path: owned.path().join("synthetic-host"), + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "caller-uncertainty".into(), + control_secret: secret.into(), + lifecycle_log: owned.path().join("lifecycle.jsonl"), + admission: Default::default(), + }; + std::fs::write(&config.host_binary_path, "fixture-host-artifact").unwrap(); + let backend = Arc::new(Backend(state.clone(), home.clone())); + let supervisor = + Supervisor::new(registry.clone(), backend.clone(), config.clone()).unwrap(); + let callback = UnixListener::bind(control.join("mcp.sock")).unwrap(); + let (callback_shutdown, mut callback_stop) = tokio::sync::oneshot::channel(); + let callback_task = tokio::spawn(async move { + loop { + let (mut stream, _) = + tokio::select! {v=callback.accept()=>v.unwrap(),_=&mut callback_stop=>break}; + let mut bytes = vec![]; + stream.read_to_end(&mut bytes).await.unwrap(); + let reply = + crate::managed_mcp_capability::apply_initial_caller_fixture_callback(&bytes) + .await; + stream + .write_all(&reply.unwrap_or_else(|_| b"error".to_vec())) + .await + .unwrap(); + } + }); + let listener = UnixListener::bind(&socket).unwrap(); + let control_state = state.clone(); + let owner = supervisor.clone(); + let lose_reply = failure == "lost-reply"; + let (control_shutdown, mut control_stop) = tokio::sync::oneshot::channel(); + let control_task = tokio::spawn(async move { + loop { + let (mut stream, _) = + tokio::select! {v=listener.accept()=>v.unwrap(),_=&mut control_stop=>break}; + let envelope: Envelope = read_frame(&mut stream).await.unwrap(); + let launch = if let AdminCommand::Launch(request) = &envelope.body { + Some(request.clone()) + } else { + None + }; + let reply = Box::pin(owner.dispatch(envelope)).await; + if let Some(request) = launch { + control_state.lock().unwrap().accepted = Some(*request); + if lose_reply { + assert!( + reply.result.is_ok(), + "lost reply must follow actual successful launch: {:?}", + reply.result + ); + continue; + } + } + write_frame(&mut stream, &reply).await.unwrap(); + } + }); + let client = RuntimeClient::new(&socket, secret); + let controller = ServerManagedRuntimeController { + client: client.clone(), + recovery: Arc::new(ManagedRecoveryState::default()), + }; + let seed = format!("caller-{route}-{provider}-{failure}"); + let result = if route == "fresh" { + let (broadcast, _receiver) = tokio::sync::broadcast::channel(32); + let fresh = crate::fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( + Some(client.clone()), + Arc::new(broadcast), + ) + .unwrap() + .unwrap(); + fresh + .create_agent(HostedRestCreate { + request_id: seed.clone(), + provider: if provider == "kilroy" { + "claude" + } else { + provider + } + .into(), + session_type: if provider == "kilroy" { + "kilroy".into() + } else { + format!("fresh{provider}") + }, + cwd: Some(workspace.to_string_lossy().into_owned()), + model: None, + effort: None, + native_session_id: None, + plugins: None, + model_selection: None, + permission_mode: None, + sandbox: None, + }) + .await + .map(|_| ()) + .map_err(|_| "fresh refusal".to_string()) + } else { + controller + .launch(ManagedTerminalLaunch { + spec: freshell_platform::SpawnSpec { + program: provider.into(), + args: vec![], + env_overrides: BTreeMap::new(), + cwd: Some(workspace.to_string_lossy().into_owned()), + cols: 80, + rows: 24, + }, + env: BTreeMap::new(), + terminal_id: seed.clone(), + stream_id: seed.clone(), + mode: provider.into(), + resume_session_id: None, + provider_model: None, + provider_reasoning_effort: None, + provider_sandbox: None, + provider_permission_mode: None, + view_tab_id: None, + view_pane_id: None, + create_request_id: Some(seed.clone()), + }) + .await + .map(|_| ()) + }; + assert!( + result.is_err(), + "fixture must reach uncertain caller outcome" + ); + let launch = state + .lock() + .unwrap() + .accepted + .clone() + .expect("real launch reached supervisor"); + let context = launch + .terminal + .as_ref() + .and_then(|s| s.provider_launch_context.as_ref()) + .or_else(|| { + launch + .fresh_agent + .as_ref() + .and_then(|s| s.provider_launch_context.as_ref()) + }) + .unwrap(); + let grant = &context.mcp_capability.as_ref().unwrap().grant_id; + let journal = control + .join("mcp-capabilities") + .join(format!("{grant}.json")); + let stage = control.join("provider-roots").join(grant); + println!("initial_caller_accepted {{\"case\":\"{case}\",\"dispatches\":{},\"journalRetained\":{},\"stageRetained\":{}}}",state.lock().unwrap().dispatches,journal.is_file(),stage.is_dir()); + // Preserve every assertion failure until all exact owned tasks join. + let retained = journal.is_file() && stage.is_dir(); + let before = registry.inventory().await.unwrap(); + // Registry ownership is exclusive. Join and drop the old control + // owner before opening SQLite through a fresh supervisor. + control_shutdown.send(()).unwrap(); + control_task.await.unwrap(); + drop(supervisor); + drop(registry); + let mut cold_control = None; + let mut cold_shutdown = None; + let verification = std::panic::AssertUnwindSafe(async { + let mut proof = None; + if retained { + let manifest = std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(); + let config_binds = context + .prepared_inputs + .as_ref() + .unwrap() + .config_binds + .iter() + .map(|bind| { + let path = stage.join(&bind.source_relative_path); + let bytes = std::fs::read(&path).unwrap(); + (path, bytes) + }) + .collect::>(); + assert!( + !config_binds.is_empty(), + "fixture must protect real config-bind sources" + ); + let stored: serde_json::Value = + serde_json::from_slice(&std::fs::read(&journal).unwrap()).unwrap(); + assert!( + crate::managed_mcp_capability::initial_caller_fixture_authenticates( + stored["scope"].as_str().unwrap() + ) + ); + assert_eq!(before.len(), 1); + assert_eq!( + before[0].native_session_id.as_deref(), + Some("native-exact-caller") + ); + // Cold supervisor+SQLite reopen retries accepted completion and + // adopts the current host through Recover, never Launch again. + let cold = Supervisor::new( + Registry::open(®istry_root, None).unwrap(), + backend, + config, + ) + .unwrap(); + std::fs::remove_file(&socket).unwrap(); + let listener = UnixListener::bind(&socket).unwrap(); + let owner = cold.clone(); + let (stop, mut stopped) = tokio::sync::oneshot::channel(); + cold_shutdown = Some(stop); + cold_control = Some(tokio::spawn(async move { + loop { + let (mut stream, _) = + tokio::select! {v=listener.accept()=>v.unwrap(),_=&mut stopped=>break}; + let envelope: Envelope = + read_frame(&mut stream).await.unwrap(); + write_frame(&mut stream, &Box::pin(owner.dispatch(envelope)).await) + .await + .unwrap(); + } + })); + let mut envelope = Envelope::new( + RequestId::new(), + ControlRole::Web, + AdminCommand::Recover(RecoverRequest { + soul_id: launch.soul_id.clone(), + trigger: RecoveryTrigger::StartupReconcile, + expected_intent_revision: None, + expected_control_epoch: None, + }), + ); + envelope.auth = Some(secret.into()); + let reply = Box::pin(cold.dispatch(envelope)).await; + assert!( + matches!( + reply.result, + Ok(AdminResult::Recovery(RecoveryResult { + outcome: RecoveryOutcome::AlreadyLive | RecoveryOutcome::Reattached, + .. + })) + ), + "current healthy native reconciliation: {:?}", + reply.result + ); + crate::managed_mcp_capability::rehydrate_running_capabilities(&client) + .await + .unwrap(); + let after = cold.registry().inventory().await.unwrap(); + assert_eq!(after[0].incarnation_id, before[0].incarnation_id); + assert_eq!(after[0].native_session_id, before[0].native_session_id); + assert_eq!( + std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + manifest + ); + assert!( + crate::managed_mcp_capability::initial_caller_fixture_authenticates( + stored["scope"].as_str().unwrap() + ) + ); + for (path, bytes) in config_binds { + assert_eq!( + std::fs::read(path).unwrap(), + bytes, + "reconciliation preserves immutable config-bind sources" + ); + } + proof = Some(after[0].incarnation_id.clone()); + } + proof + }) + .catch_unwind() + .await; + if let Some(shutdown) = cold_shutdown { + shutdown.send(()).unwrap(); + } + if let Some(task) = cold_control { + task.await.unwrap(); + } + callback_shutdown.send(()).unwrap(); + callback_task.await.unwrap(); + let (tasks, shutdowns) = { + let mut s = state.lock().unwrap(); + ( + std::mem::take(&mut s.host_tasks), + std::mem::take(&mut s.host_shutdowns), + ) + }; + for shutdown in shutdowns { + let _ = shutdown.send(()); + } + let mut task_failures = vec![]; + for task in tasks { + if let Err(error) = task.await { + task_failures.push(error); + } + } + let proof = verification.unwrap_or_else(|panic| std::panic::resume_unwind(panic)); + assert!( + task_failures.is_empty(), + "owned host tasks failed after join: {task_failures:?}" + ); + assert!( + retained, + "actual {case} caller deleted accepted preparation journal/stage" + ); + assert!(proof.is_some()); + assert_eq!(state.lock().unwrap().dispatches, 1); + println!("initial_caller_reconciled {{\"case\":\"{case}\",\"sameNativeIdentity\":true,\"dispatches\":1,\"allFixtureTasksJoined\":true}}"); + } +} + #[test] fn managed_provider_initial_opencode_callers_replan_changed_selection_with_two_retry_bound() { const CHILD: &str = "FRESHELL_TEST_INITIAL_OPENCODE_REPLAN_CHILD"; From 805bc212aec0fb22c8279032ef2648fc4b97971d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 16:38:56 -0700 Subject: [PATCH 205/493] test: route Rust workspace tests through strict sandbox --- docs/development/test-sandbox.md | 22 ++++ scripts/sandbox-test.sh | 121 +++++++++++++++++++- scripts/testing/run-rust-tests.ts | 71 +++++++++--- test/unit/tooling/run-rust-tests.test.ts | 134 +++++++++++++++++++++++ 4 files changed, 333 insertions(+), 15 deletions(-) create mode 100644 test/unit/tooling/run-rust-tests.test.ts diff --git a/docs/development/test-sandbox.md b/docs/development/test-sandbox.md index 6e2170ece..e17b39644 100644 --- a/docs/development/test-sandbox.md +++ b/docs/development/test-sandbox.md @@ -42,6 +42,28 @@ to the unprivileged `sandbox` UID before test code and the self-test verifies th effective capabilities. The mode never mounts Docker/admin sockets or real provider homes. The runtime gate's separate trusted broker is the only component allowed to hold the real Docker socket; this wrapper never grants it to tests. +For the Rust phase of the existing full suite, explicitly select this mode: + +```bash +FRESHELL_RUST_TEST_SANDBOX=1 pnpm run test +# Or run the existing standalone Rust launcher with Cargo test arguments: +FRESHELL_RUST_TEST_SANDBOX=1 pnpm run test:rust -- --exact +``` + +Only the exact value `1` enables routing; other test backend settings do not. +The Rust launcher runs its unchanged locked server build and workspace tests +inside the strict sandbox, selects the container's server binary, and gives +both temporary-directory variables a newly owned executable directory under +the sandbox Cargo target volume. Cargo uses three parallel build jobs within +the existing container resource limits. The launcher removes its temporary +directory on exit. Sandbox, +dependency, and inner-runner failures return failure without host Cargo fallback. +The other full-suite phases keep their configured backends and coordinator. +The wrapper records its owned container and settles it before returning a +signal failure; stopping the attached Docker client alone is insufficient. +Warm dependency caches through the documented ordinary sandbox preparation +before a strict run if needed; strict mode keeps its read-only home and no network. + ## Safety guarantees | What | Guarantee | diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index a709a98a0..217c512b7 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -130,9 +130,123 @@ else NETWORK_ARGS=(--network "${NETWORK_NAME}") fi +OWNED_CONTROL_DIR="$(mktemp -d "${TMPDIR:-/tmp}/freshell-sandbox-XXXXXXXX")" +OWNED_CONTAINER_NAME="$(basename "${OWNED_CONTROL_DIR}")" +OWNED_CID_FILE="${OWNED_CONTROL_DIR}/container.id" +DOCKER_CLIENT_PID="" +REQUESTED_SIGNAL="" + +sandbox_log() { + # All interpolated values are fixed events, numeric statuses or the owned + # mktemp-generated identity, never caller command text or credentials. + printf '{"severity":"%s","event":"%s","container":"%s","code":%s}\n' \ + "$1" "$2" "${OWNED_CONTAINER_NAME}" "$3" >&2 +} + +# shellcheck disable=SC2317 # Reached by the installed EXIT/signal trap handlers. +owned_docker_client_running() { + [ -n "${DOCKER_CLIENT_PID}" ] \ + && ps -p "${DOCKER_CLIENT_PID}" -o args= 2>/dev/null \ + | grep -F -- "--name ${OWNED_CONTAINER_NAME}" \ + | grep -F -- "${REPO_ROOT}:/workspace" >/dev/null +} + +# shellcheck disable=SC2317 # Reached by the installed EXIT trap handler. +cleanup_owned_container() { + local cleanup_status=0 + if [ -s "${OWNED_CID_FILE}" ]; then + local owned_cid + owned_cid="$(cat "${OWNED_CID_FILE}")" + if ! [[ "${owned_cid}" =~ ^[a-f0-9]{64}$ ]]; then + sandbox_log error sandbox_invalid_owned_identity 1 + return 1 + fi + if docker container inspect "${owned_cid}" >/dev/null 2>&1; then + if [ -n "${REQUESTED_SIGNAL}" ]; then + timeout 15s docker kill --signal "${REQUESTED_SIGNAL}" "${owned_cid}" >/dev/null || { + docker container inspect "${owned_cid}" >/dev/null 2>&1 && cleanup_status=1 + } + # Give the inner runner time to forward the same signal and clean its + # own executable temp directory before stopping any remaining work. + timeout 10s docker wait "${owned_cid}" >/dev/null 2>&1 || true + fi + if docker container inspect "${owned_cid}" >/dev/null 2>&1; then + timeout 15s docker stop --time 5 "${owned_cid}" >/dev/null 2>&1 || { + docker container inspect "${owned_cid}" >/dev/null 2>&1 && cleanup_status=1 + } + fi + # --rm may remove it before wait; only a successful absence check + # counts as settlement when that races with the explicit wait. + timeout 15s docker wait "${owned_cid}" >/dev/null 2>&1 || { + docker container inspect "${owned_cid}" >/dev/null 2>&1 && cleanup_status=1 + } + timeout 15s docker rm --force "${owned_cid}" >/dev/null 2>&1 || { + docker container inspect "${owned_cid}" >/dev/null 2>&1 && cleanup_status=1 + } + fi + # A daemon error is not proof of absence. List succeeds first, then check + # the exact ID rather than treating any inspect failure as cleanup. + local remaining + remaining="$(docker ps -aq --no-trunc --filter "id=${owned_cid}")" || cleanup_status=1 + [ -z "${remaining}" ] || cleanup_status=1 + fi + return "${cleanup_status}" +} + +# shellcheck disable=SC2317 # Invoked by the installed EXIT trap. +finish_owned_sandbox() { + local result=$? + trap - EXIT + trap '' INT TERM HUP + cleanup_owned_container || result=1 + if [ -n "${DOCKER_CLIENT_PID}" ]; then + # The container is already settled; terminate a lingering attached client + # rather than leaving this wrapper waiting on a transport after cleanup. + if owned_docker_client_running; then + kill -TERM "${DOCKER_CLIENT_PID}" 2>/dev/null || true + fi + wait "${DOCKER_CLIENT_PID}" 2>/dev/null || true + fi + rm -rf -- "${OWNED_CONTROL_DIR}" || result=1 + if [ "${result}" -eq 0 ]; then + sandbox_log info sandbox_workload_finished "${result}" + else + sandbox_log error sandbox_workload_failed "${result}" + fi + exit "${result}" +} + +# shellcheck disable=SC2317 # Invoked by the installed signal traps. +interrupt_owned_sandbox() { + local result="$1" + REQUESTED_SIGNAL="$2" + trap '' INT TERM HUP + sandbox_log info sandbox_workload_signal "${result}" + # Docker publishes the cid before starting the workload. If interrupted + # during creation, let this owned client publish its identity before cleanup. + if [ -n "${DOCKER_CLIENT_PID}" ]; then + for _attempt in $(seq 1 300); do + [ -s "${OWNED_CID_FILE}" ] && break + owned_docker_client_running || break + sleep 0.1 + done + if [ ! -s "${OWNED_CID_FILE}" ] && owned_docker_client_running; then + sandbox_log error sandbox_identity_not_settled 1 + kill -TERM "${DOCKER_CLIENT_PID}" 2>/dev/null || true + result=1 + fi + fi + exit "${result}" +} +trap 'finish_owned_sandbox' EXIT +trap 'interrupt_owned_sandbox 130 SIGINT' INT +trap 'interrupt_owned_sandbox 143 SIGTERM' TERM +trap 'interrupt_owned_sandbox 129 SIGHUP' HUP + DOCKER_ARGS=( # Reap orphaned test children so zombies cannot exhaust the PID limit. run --rm --init + --name "${OWNED_CONTAINER_NAME}" --cidfile "${OWNED_CID_FILE}" "${NETWORK_ARGS[@]}" --pids-limit 512 --memory 8g @@ -151,6 +265,7 @@ DOCKER_ARGS=( ) if [ "${RUNTIME_SUITE}" -eq 1 ]; then + # shellcheck disable=SC2054 # Commas belong to each single tmpfs option string. DOCKER_ARGS+=( --read-only --tmpfs /tmp:rw,nosuid,nodev,size=512m @@ -202,7 +317,11 @@ done # which on Debian unconditionally overwrites PATH and would clobber the # cargo/rustup PATH entry set via the image's Dockerfile ENV. DOCKER_STATUS=0 -docker "${DOCKER_ARGS[@]}" "${IMAGE_TAG}" bash -c "${CMD}" || DOCKER_STATUS=$? +sandbox_log info sandbox_workload_started 0 +docker "${DOCKER_ARGS[@]}" "${IMAGE_TAG}" bash -c "${CMD}" & +DOCKER_CLIENT_PID=$! +wait "${DOCKER_CLIENT_PID}" || DOCKER_STATUS=$? +DOCKER_CLIENT_PID="" # Guard-rail: verify the pre-create step above actually did its job, rather # than assuming it. If dockerd (or a future volume/mount this script doesn't diff --git a/scripts/testing/run-rust-tests.ts b/scripts/testing/run-rust-tests.ts index 668ec1b8a..4148c4de3 100644 --- a/scripts/testing/run-rust-tests.ts +++ b/scripts/testing/run-rust-tests.ts @@ -13,9 +13,10 @@ function log(severity: 'info' | 'error', event: string, fields: Record { +function runChild(command: string, args: string[], env: NodeJS.ProcessEnv, sandbox = false): Promise { return new Promise((resolve) => { - log('info', 'rust_test_phase_started', { command, args }) + const metadata = sandbox ? { command, mode: 'runtime-suite', argumentCount: args.length } : { command, args } + log('info', 'rust_test_phase_started', metadata) let settled = false let child: ChildProcess try { @@ -27,8 +28,7 @@ function runChild(command: string, args: string[], env: NodeJS.ProcessEnv): Prom }) } catch (error) { log('error', 'rust_test_phase_spawn_failed', { - command, - args, + ...metadata, error: error instanceof Error ? error.message : String(error), }) resolve(1) @@ -36,39 +36,82 @@ function runChild(command: string, args: string[], env: NodeJS.ProcessEnv): Prom } const signals = ['SIGINT', 'SIGTERM', 'SIGHUP'] as const - const forwardSignal = (signal: typeof signals[number]) => { - if (child.exitCode === null && !child.killed) child.kill(signal) - } - for (const signal of signals) process.once(signal, forwardSignal) + // Node signal listeners receive no argument. Capture each signal so the + // sandbox wrapper receives the actual request and can settle its workload. + const handlers = signals.map(signal => { + const handler = () => { if (child.exitCode === null) child.kill(signal) } + process.on(signal, handler) + return { signal, handler } + }) const finish = (code: number, signal: NodeJS.Signals | null) => { if (settled) return settled = true - for (const forwardedSignal of signals) process.off(forwardedSignal, forwardSignal) + for (const { signal: forwardedSignal, handler } of handlers) process.off(forwardedSignal, handler) if (code !== 0 || signal) { - log('error', 'rust_test_phase_failed', { command, args, code, signal }) + log('error', 'rust_test_phase_failed', { ...metadata, code, signal }) } else { - log('info', 'rust_test_phase_finished', { command, args, code }) + log('info', 'rust_test_phase_finished', { ...metadata, code }) } resolve(code) } child.once('error', (error) => { log('error', 'rust_test_phase_spawn_failed', { - command, - args, + ...metadata, error: error.message, }) finish(1, null) }) - child.once('exit', (code, signal) => { + child.once('close', (code, signal) => { const exitCode = code ?? (signal ? 128 + (osConstants.signals[signal] ?? 1) : 1) finish(exitCode, signal) }) }) } +function shellQuote(value: string): string { + return `'${value.replaceAll("'", "'\\''")}'` +} + +function sandboxCommand(argv: string[]): string { + // Fixed wrapper text owns only its newly allocated target subdirectory. + // Arguments remain separately quoted words; none becomes shell code. + return `set -e +owned_tmp="$(mktemp -d /workspace/target/.freshell-rust-tests-XXXXXXXX)" +runner_pid= +cleanup() { + status=$? + trap - EXIT + rm -rf -- "$owned_tmp" || { [ "$status" -ne 0 ] || status=1; } + exit "$status" +} +interrupt() { + trap '' INT TERM HUP + if [ -n "$runner_pid" ]; then + kill -s "$1" "$runner_pid" 2>/dev/null || true + wait "$runner_pid" || true + fi + exit "$2" +} +trap cleanup EXIT +trap 'interrupt INT 130' INT +trap 'interrupt TERM 143' TERM +trap 'interrupt HUP 129' HUP +export TMPDIR="$owned_tmp" FRESHELL_TEST_EXECUTABLE_TMP_DIR="$owned_tmp" FRESHELL_RUST_TEST_SANDBOX=0 CARGO_BUILD_JOBS=3 +node --import tsx /workspace/scripts/testing/run-rust-tests.ts ${argv.map(shellQuote).join(' ')} & +runner_pid=$! +wait "$runner_pid"` +} + export async function main(argv: string[] = process.argv.slice(2)): Promise { + if (process.env.FRESHELL_RUST_TEST_SANDBOX === '1') { + log('info', 'rust_test_sandbox_route_selected', { mode: 'runtime-suite', cargoBuildJobs: 3, forwardedArgumentCount: argv.length }) + // Do not inherit a host binary selection. The unchanged inner runner + // selects its own freshly built /workspace/target/debug server. + const { FRESHELL_SERVER_BIN: _hostBinary, ...env } = process.env + return runChild(path.join(PROJECT_ROOT, 'scripts/sandbox-test.sh'), ['--runtime-suite', sandboxCommand(argv)], env, true) + } const binary = path.join(PROJECT_ROOT, 'target', 'debug', `freshell-server${process.platform === 'win32' ? '.exe' : ''}`) const cargo = process.platform === 'win32' ? 'cargo.exe' : 'cargo' const buildExitCode = await runChild(cargo, ['build', '-p', 'freshell-server', '--locked'], process.env) diff --git a/test/unit/tooling/run-rust-tests.test.ts b/test/unit/tooling/run-rust-tests.test.ts new file mode 100644 index 000000000..5650c55f1 --- /dev/null +++ b/test/unit/tooling/run-rust-tests.test.ts @@ -0,0 +1,134 @@ +// @vitest-environment node +import { spawnSync } from 'node:child_process' +import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import { createRequire } from 'node:module' +import { describe, expect, it } from 'vitest' + +const require = createRequire(import.meta.url) +const tsx = require.resolve('tsx') +const launcher = path.resolve('scripts/testing/run-rust-tests.ts') + +function fixture() { + const root = mkdtempSync(path.join(os.tmpdir(), 'rust-launcher-')) + mkdirSync(path.join(root, 'scripts/testing'), { recursive: true }) + mkdirSync(path.join(root, 'target')) + mkdirSync(path.join(root, 'bin')) + copyFileSync(launcher, path.join(root, 'scripts/testing/run-rust-tests.ts')) + writeFileSync(path.join(root, 'package.json'), '{"type":"module"}') + symlinkSync(path.resolve('node_modules'), path.join(root, 'node_modules'), 'dir') + const receipt = path.join(root, 'receipt.jsonl') + const executable = (file: string, body: string) => { + writeFileSync(`${file}.mjs`, body) + const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` + writeFileSync(file, `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(`${file}.mjs`)} "$@"\n`) + chmodSync(file, 0o755) + } + executable(path.join(root, 'bin/cargo'), ` + import fs from 'node:fs'; + const args = process.argv.slice(2); + fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'cargo', args, + binary:process.env.FRESHELL_SERVER_BIN, route:process.env.FRESHELL_RUST_TEST_SANDBOX, jobs:process.env.CARGO_BUILD_JOBS, + tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); + process.exit(Number(args[0]==='build' ? process.env.BUILD_EXIT || 0 : process.env.TEST_EXIT || 0)); + `) + executable(path.join(root, 'scripts/sandbox-test.sh'), ` + import fs from 'node:fs'; + import cp from 'node:child_process'; + const args = process.argv.slice(2); + fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'sandbox', args})+'\\n'); + if (process.env.SANDBOX_EXIT) process.exit(Number(process.env.SANDBOX_EXIT)); + const command = args[1] + .replace('mktemp -d /workspace/target/', 'mktemp -d ' + ${JSON.stringify(root)} + '/target/') + .replace('node --import tsx /workspace/scripts/testing/run-rust-tests.ts', 'node --import tsx ' + ${JSON.stringify(root)} + '/scripts/testing/run-rust-tests.ts'); + const result = cp.spawnSync('bash', ['-c', command], {env:process.env, stdio:'inherit'}); + process.exit(result.status ?? 1); + `) + return { + root, + run(env: NodeJS.ProcessEnv = {}, args: string[] = []) { + const result = spawnSync(process.execPath, ['--import', tsx, path.join(root, 'scripts/testing/run-rust-tests.ts'), ...args], { + env: { ...process.env, PATH: `${root}/bin:${process.env.PATH}`, RECEIPT: receipt, + FRESHELL_RUST_TEST_SANDBOX: '0', FRESHELL_SERVER_BIN: '/host/stale-server', ...env }, + encoding: 'utf8', timeout: 15_000, + }) + const events = existsSync(receipt) ? readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) : [] + return { ...result, events } + }, + cleanup() { rmSync(root, { recursive: true, force: true }) }, + } +} + +describe('Rust test launcher children', () => { + it('builds before testing and selects the current checkout binary with disabled routing', () => { + const f = fixture() + try { + const result = f.run({ CARGO_BUILD_JOBS: '7' }) + expect(result.status, result.stderr).toBe(0) + expect(result.events.map(e => e.args)).toEqual([ + ['build', '-p', 'freshell-server', '--locked'], ['test', '--workspace', '--locked'], + ]) + expect(result.events[1].binary).toBe(path.join(f.root, 'target/debug/freshell-server')) + expect(result.events[1].jobs).toBe('7') + } finally { f.cleanup() } + }) + + it('preserves build failure and does not launch tests', () => { + const f = fixture() + try { + const result = f.run({ BUILD_EXIT: '23' }) + expect(result.status).toBe(23) + expect(result.events.map(e => e.args[0])).toEqual(['build']) + } finally { f.cleanup() } + }) + + it('requires exact opt-in rather than cloud settings or truthy values', () => { + const f = fixture() + try { + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: 'true', FRESHELL_VITEST_BACKEND: 'cloud' }) + expect(result.status).toBe(0) + expect(result.events.map(e => e.kind)).toEqual(['cargo', 'cargo']) + } finally { f.cleanup() } + }) + + it('routes first to strict sandbox, disables recursion, owns executable temp and preserves arguments', () => { + const f = fixture() + try { + const args = ['--', 'filter with space', "quote'word", '$(touch SHOULD_NOT_EXIST)', 'a;echo nope', '', '/workspace/target/argument', 'line\nbreak', 'unicode-🚀', '--nocapture'] + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1' }, args) + expect(result.status, result.stderr).toBe(0) + expect(result.events.map(e => e.kind)).toEqual(['sandbox', 'cargo', 'cargo']) + expect(result.events[0].args[0]).toBe('--runtime-suite') + expect(result.events[1].route).toBe('0') + expect(result.events[1].jobs).toBe('3') + expect(result.events[2].args).toEqual(['test', '--workspace', '--locked', ...args.slice(1)]) + expect(result.events[2].binary).toBe(path.join(f.root, 'target/debug/freshell-server')) + expect(result.events[2].tmp).toBe(result.events[2].executableTmp) + expect(result.events[2].tmp).toMatch(new RegExp(`^${f.root}/target/\\.freshell-rust-tests-[^/]+$`)) + expect(existsSync(result.events[2].tmp)).toBe(false) + } finally { f.cleanup() } + }) + + it.each([17, 125, 143])('fails closed on sandbox failure %s without host Cargo', code => { + const f = fixture() + try { + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1', SANDBOX_EXIT: String(code) }) + expect(result.status).toBe(code) + expect(result.events.map(e => e.kind)).toEqual(['sandbox']) + } finally { f.cleanup() } + }) + + it('cleans only its temporary directory while preserving inner failure', () => { + const f = fixture() + try { + const sentinel = path.join(f.root, 'target/preexisting') + writeFileSync(sentinel, 'keep') + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1', TEST_EXIT: '31' }) + expect(result.status).toBe(31) + expect(readFileSync(sentinel, 'utf8')).toBe('keep') + expect(result.events[2].tmp).toBeTruthy() + expect(existsSync(result.events[2].tmp)).toBe(false) + } finally { f.cleanup() } + }) +}) From e4107acd1d81935bfb4de315500ffb9657dd6fbb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:04:47 -0700 Subject: [PATCH 206/493] fix: settle sandbox launches without published container IDs --- scripts/sandbox-test.sh | 44 ++++++- test/unit/tooling/sandbox-test.config.ts | 12 ++ test/unit/tooling/sandbox-test.sandbox.ts | 140 ++++++++++++++++++++++ 3 files changed, 191 insertions(+), 5 deletions(-) create mode 100644 test/unit/tooling/sandbox-test.config.ts create mode 100644 test/unit/tooling/sandbox-test.sandbox.ts diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 217c512b7..34726b054 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -133,6 +133,7 @@ fi OWNED_CONTROL_DIR="$(mktemp -d "${TMPDIR:-/tmp}/freshell-sandbox-XXXXXXXX")" OWNED_CONTAINER_NAME="$(basename "${OWNED_CONTROL_DIR}")" OWNED_CID_FILE="${OWNED_CONTROL_DIR}/container.id" +printf '%s\n' "${OWNED_CONTAINER_NAME}" > "${OWNED_CONTROL_DIR}/container.name" DOCKER_CLIENT_PID="" REQUESTED_SIGNAL="" @@ -154,9 +155,16 @@ owned_docker_client_running() { # shellcheck disable=SC2317 # Reached by the installed EXIT trap handler. cleanup_owned_container() { local cleanup_status=0 + local owned_cid="" if [ -s "${OWNED_CID_FILE}" ]; then - local owned_cid owned_cid="$(cat "${OWNED_CID_FILE}")" + else + # Creation may reach the daemon before the attached client publishes its + # CID. The preallocated exact name is still ours; missing publication is + # unknown state, not evidence that no container was created. + owned_cid="$(docker ps -aq --no-trunc --filter "name=^/${OWNED_CONTAINER_NAME}$")" || return 1 + fi + if [ -n "${owned_cid}" ]; then if ! [[ "${owned_cid}" =~ ^[a-f0-9]{64}$ ]]; then sandbox_log error sandbox_invalid_owned_identity 1 return 1 @@ -190,24 +198,50 @@ cleanup_owned_container() { remaining="$(docker ps -aq --no-trunc --filter "id=${owned_cid}")" || cleanup_status=1 [ -z "${remaining}" ] || cleanup_status=1 fi + # Always query the exact name, including failures before CID publication. + # Only a successful daemon response establishes absence. + local remaining_by_name + remaining_by_name="$(docker ps -aq --no-trunc --filter "name=^/${OWNED_CONTAINER_NAME}$")" || cleanup_status=1 + [ -z "${remaining_by_name}" ] || cleanup_status=1 return "${cleanup_status}" } # shellcheck disable=SC2317 # Invoked by the installed EXIT trap. finish_owned_sandbox() { local result=$? + local container_settled=0 trap - EXIT trap '' INT TERM HUP - cleanup_owned_container || result=1 + if [ ! -s "${OWNED_CID_FILE}" ] && [ -n "${DOCKER_CLIENT_PID}" ]; then + # Join creation before looking up the name: querying first could observe + # absence while this client is still creating the owned object. + if owned_docker_client_running; then + kill -TERM "${DOCKER_CLIENT_PID}" 2>/dev/null || true + fi + wait "${DOCKER_CLIENT_PID}" 2>/dev/null || true + DOCKER_CLIENT_PID="" + fi + if cleanup_owned_container; then + container_settled=1 + else + result=1 + fi if [ -n "${DOCKER_CLIENT_PID}" ]; then - # The container is already settled; terminate a lingering attached client - # rather than leaving this wrapper waiting on a transport after cleanup. + # Cleanup has been attempted; join a lingering attached client instead of + # leaving this wrapper waiting on its transport. Unknown daemon state + # retains the control directory below. if owned_docker_client_running; then kill -TERM "${DOCKER_CLIENT_PID}" 2>/dev/null || true fi wait "${DOCKER_CLIENT_PID}" 2>/dev/null || true fi - rm -rf -- "${OWNED_CONTROL_DIR}" || result=1 + if [ "${container_settled}" -eq 1 ]; then + rm -rf -- "${OWNED_CONTROL_DIR}" || result=1 + else + sandbox_log error sandbox_owned_container_unsettled 1 + printf '[sandbox] owned container %s could not be settled; control evidence retained at %q\n' \ + "${OWNED_CONTAINER_NAME}" "${OWNED_CONTROL_DIR}" >&2 + fi if [ "${result}" -eq 0 ]; then sandbox_log info sandbox_workload_finished "${result}" else diff --git a/test/unit/tooling/sandbox-test.config.ts b/test/unit/tooling/sandbox-test.config.ts new file mode 100644 index 000000000..886410881 --- /dev/null +++ b/test/unit/tooling/sandbox-test.config.ts @@ -0,0 +1,12 @@ +import { defineConfig } from 'vitest/config' +import baseConfig from '../../../config/vitest/vitest.config.js' + +// Signals and cleanup run only through the disposable sandbox. This explicit +// entry avoids running these fixtures in the ordinary cloud/local unit lane. +export default defineConfig({ + ...baseConfig, + test: { + ...baseConfig.test, + include: ['test/unit/tooling/sandbox-test.sandbox.ts', 'test/unit/tooling/run-rust-tests.test.ts'], + }, +}) diff --git a/test/unit/tooling/sandbox-test.sandbox.ts b/test/unit/tooling/sandbox-test.sandbox.ts new file mode 100644 index 000000000..e7daa29b8 --- /dev/null +++ b/test/unit/tooling/sandbox-test.sandbox.ts @@ -0,0 +1,140 @@ +// @vitest-environment node +import { spawnSync } from 'node:child_process' +import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import { describe, expect, it } from 'vitest' + +// These fixtures exercise creation-time signals and Docker cleanup. Run them +// through scripts/sandbox-test.sh --runtime-suite, never directly on the host. +if (process.env.FRESHELL_SANDBOX_MODE !== 'runtime-suite') { + throw new Error('Sandbox wrapper fixtures require scripts/sandbox-test.sh --runtime-suite and sandbox-test.config.ts') +} + +function fixture(mode: string) { + const root = mkdtempSync(path.join(os.tmpdir(), 'sandbox-wrapper-')) + for (const subdir of ['scripts', 'bin', 'control']) mkdirSync(path.join(root, subdir)) + copyFileSync(path.resolve('scripts/sandbox-test.sh'), path.join(root, 'scripts/sandbox-test.sh')) + const receipt = path.join(root, 'receipt.jsonl') + const state = path.join(root, 'owned-object.json') + const foreign = path.join(root, 'foreign-object') + writeFileSync(foreign, 'untouched') + writeFileSync(path.join(root, 'bin/docker.mjs'), String.raw` + import fs from 'node:fs'; + const args = process.argv.slice(2), mode = process.env.FIXTURE_MODE; + const state = process.env.FIXTURE_STATE; + const cid = 'a'.repeat(64); + const record = event => fs.appendFileSync(process.env.FIXTURE_RECEIPT, JSON.stringify(event)+'\n'); + const object = () => fs.existsSync(state) ? JSON.parse(fs.readFileSync(state, 'utf8')) : null; + record({kind:'docker', args}); + if (args[0] === 'image') process.exit(0); + if (args[0] === 'run') { + const name = args[args.indexOf('--name')+1], cidfile = args[args.indexOf('--cidfile')+1]; + record({kind:'run', name, cidfile}); + if (mode !== 'no-object' && mode !== 'late-creation') fs.writeFileSync(state, JSON.stringify({cid, name})); + if (mode === 'published' || mode === 'removed-race') fs.writeFileSync(cidfile, cid); + if (mode === 'removed-race') fs.unlinkSync(state); + if (mode === 'late-creation') { + process.kill(process.ppid, 'SIGTERM'); + setTimeout(() => { + fs.writeFileSync(state, JSON.stringify({cid, name})); + record({kind:'creation-client-settled'}); + process.exit(0); + }, 150); + } else process.exit(mode === 'published' ? 37 : 42); + } else if (args[0] === 'ps') { + if (mode === 'query-failed') process.exit(71); + const current = object(), filter = args[args.indexOf('--filter')+1]; + if (current && (filter === 'id='+cid || filter === 'name=^/'+current.name+'$')) console.log(cid); + } else if (args[0] === 'container' && args[1] === 'inspect') { + process.exit(object() && args[2] === cid ? 0 : 1); + } else if (['kill', 'stop', 'wait', 'rm'].includes(args[0])) { + if (args.at(-1) !== cid) process.exit(72); + if (args[0] === 'rm') { + if (mode === 'remove-failed') process.exit(73); + if (object()) fs.unlinkSync(state); + } + } else process.exit(74); + `) + const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` + writeFileSync(path.join(root, 'bin/docker'), `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(path.join(root, 'bin/docker.mjs'))} "$@"\n`) + chmodSync(path.join(root, 'bin/docker'), 0o755) + return { + root, state, foreign, + run() { + const result = spawnSync('bash', [path.join(root, 'scripts/sandbox-test.sh'), '--runtime-suite', 'true'], { + env: { ...process.env, PATH: `${root}/bin:${process.env.PATH}`, TMPDIR: path.join(root, 'control'), + FIXTURE_MODE: mode, FIXTURE_STATE: state, FIXTURE_RECEIPT: receipt }, + encoding: 'utf8', timeout: 40_000, + }) + const events = readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) + const run = events.find(event => event.kind === 'run') + return { ...result, events, control: path.dirname(run.cidfile), name: run.name } + }, + cleanup() { rmSync(root, { recursive: true, force: true }) }, + } +} + +describe('sandbox wrapper owned launch settlement', () => { + it('removes an owned daemon object after failed launch without CID publication', () => { + const f = fixture('failed-launch') + try { + const result = f.run() + expect(result.status, result.stderr).toBe(42) + expect(existsSync(f.state)).toBe(false) + expect(existsSync(result.control)).toBe(false) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + expect(result.events.some(event => event.args?.includes(`name=^/${result.name}$`))).toBe(true) + } finally { f.cleanup() } + }) + + it('joins a creation-time interrupted client before resolving its unpublished object', () => { + const f = fixture('late-creation') + try { + const result = f.run() + expect(result.status, result.stderr).toBe(143) + expect(existsSync(f.state)).toBe(false) + expect(existsSync(result.control)).toBe(false) + const joined = result.events.findIndex(event => event.kind === 'creation-client-settled') + const query = result.events.findIndex(event => event.args?.[0] === 'ps') + expect(joined).toBeGreaterThan(-1) + expect(query).toBeGreaterThan(joined) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + + it.each(['query-failed', 'remove-failed'])('retains actionable control evidence when %s leaves settlement unknown', mode => { + const f = fixture(mode) + try { + const result = f.run() + expect(result.status, result.stderr).toBe(1) + expect(existsSync(f.state)).toBe(true) + expect(existsSync(result.control)).toBe(true) + expect(readFileSync(path.join(result.control, 'container.name'), 'utf8').trim()).toBe(result.name) + expect(result.stderr).toContain('sandbox_owned_container_unsettled') + expect(result.stderr).toContain(result.control) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + + it('preserves failed launch status after a successful exact-name absence query', () => { + const f = fixture('no-object') + try { + const result = f.run() + expect(result.status, result.stderr).toBe(42) + expect(result.events.some(event => event.args?.includes(`name=^/${result.name}$`))).toBe(true) + expect(existsSync(result.control)).toBe(false) + } finally { f.cleanup() } + }) + + it.each(['published', 'removed-race'])('preserves ordinary nonzero status and settles %s CID paths', mode => { + const f = fixture(mode) + try { + const result = f.run() + expect(result.status, result.stderr).toBe(mode === 'published' ? 37 : 42) + expect(existsSync(f.state)).toBe(false) + expect(existsSync(result.control)).toBe(false) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) +}) From c6b9c019d4e82b14fc1454408884960a21ffa742 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:55:01 -0700 Subject: [PATCH 207/493] fix(startup): avoid nested package-manager bootstrap --- package.json | 2 +- .../tooling/source-runtime-rust.test.ts | 51 +++++++++++++++++-- 2 files changed, 47 insertions(+), 6 deletions(-) diff --git a/package.json b/package.json index 59e26ec96..d60e4de76 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,7 @@ "prepare:claude-sidecar": "tsx scripts/ensure-claude-sidecar.ts", "prepare:electron-runtime": "tsx scripts/prepare-electron-runtime.ts", "verify:electron-artifact": "tsx scripts/verify-electron-artifact.ts", - "prestart": "pnpm run prepare:rust-runtime", + "prestart": "tsx scripts/prepare-rust-runtime.ts", "start": "cross-env NODE_ENV=production tsx scripts/start-rust-server.ts target/release/freshell-server", "serve:precheck": "cross-env FRESHELL_PRECHECK_INTENT=serve tsx scripts/precheck.ts", "serve": "pnpm run serve:precheck && pnpm run build && pnpm run start", diff --git a/test/integration/tooling/source-runtime-rust.test.ts b/test/integration/tooling/source-runtime-rust.test.ts index a05946cb3..1d39429a8 100644 --- a/test/integration/tooling/source-runtime-rust.test.ts +++ b/test/integration/tooling/source-runtime-rust.test.ts @@ -2,7 +2,7 @@ import { spawn, type ChildProcess } from 'node:child_process' import { once } from 'node:events' -import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' import net from 'node:net' import os from 'node:os' import path from 'node:path' @@ -25,6 +25,10 @@ let child: ChildProcess | undefined let homeDir: string | undefined let ownedRustPid: number | undefined +function recordFixture(event: string, fields: Record): void { + process.stdout.write(`${JSON.stringify({ severity: 'info', event, ...fields })}\n`) +} + async function findFreePort(): Promise { return new Promise((resolve, reject) => { const server = net.createServer() @@ -102,7 +106,10 @@ afterEach(async () => { } child = undefined ownedRustPid = undefined - if (homeDir) await rm(homeDir, { recursive: true, force: true }) + if (homeDir) { + await rm(homeDir, { recursive: true, force: true }) + recordFixture('source_runtime_fixture_home_removed', { homeDir }) + } homeDir = undefined }) @@ -110,11 +117,13 @@ describe('source runtime', () => { it('recognizes an owned child that exited by signal before the wait began', async () => { const signaledChild = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { stdio: 'ignore' }) await once(signaledChild, 'spawn') + recordFixture('source_runtime_signal_fixture_started', { pid: signaledChild.pid }) signaledChild.kill('SIGTERM') await once(signaledChild, 'exit') expect(signaledChild.signalCode).toBe('SIGTERM') await waitForExit(signaledChild, 100) + recordFixture('source_runtime_signal_fixture_joined', { pid: signaledChild.pid, signal: signaledChild.signalCode }) }) it.each([ @@ -142,6 +151,7 @@ describe('source runtime', () => { ensureAuthTokenFile({ env, envPath, generateToken: () => token }) child = spawn(binary, [], { cwd: homeDir, env, stdio: 'ignore' }) ownedRustPid = child.pid + recordFixture('source_runtime_auth_fixture_started', { pid: child.pid, port, homeDir, launch }) await waitForHealth(`http://127.0.0.1:${port}`, 5_000) const response = await fetch(`http://127.0.0.1:${port}/api/server-info`, { @@ -152,17 +162,32 @@ describe('source runtime', () => { expect(child.kill('SIGTERM')).toBe(true) await waitForExit(child) expect(child.exitCode).toBe(0) + recordFixture('source_runtime_auth_fixture_joined', { pid: child.pid, exit: child.exitCode, launch }) child = undefined ownedRustPid = undefined } }) it('starts the release Rust binary through the detected package manager start script and serves the built SPA', async () => { - homeDir = await mkdtemp(path.join(os.tmpdir(), 'freshell-source-runtime-')) + homeDir = await mkdtemp(path.join( + process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR ?? os.tmpdir(), 'freshell-source-runtime-', + )) const port = await findFreePort() const token = `source-runtime-${process.pid}` const manager = detectProjectManager(PROJECT_ROOT).manager const startCommand = resolveManagerCommand({ manager, args: ['start'], env: process.env }) + // The resolved manager is already available. An isolated runtime HOME + // must not require another manager bootstrap through a PATH shim (for + // example Corepack, whose cache belongs to the original HOME). + const shimDir = path.join(homeDir, 'bin') + await mkdir(shimDir) + const shimPath = path.join(shimDir, process.platform === 'win32' ? 'pnpm.cmd' : 'pnpm') + const shimSource = + process.platform === 'win32' + ? '@echo off\r\necho Unexpected nested package-manager bootstrap 1>&2\r\nexit /b 1\r\n' + : '#!/bin/sh\nprintf "%s\\n" "Unexpected nested package-manager bootstrap" >&2\nexit 1\n' + await writeFile(shimPath, shimSource, { mode: 0o755 }) + let startupOutput = '' child = spawn(startCommand.command, startCommand.args, { cwd: PROJECT_ROOT, env: { @@ -175,14 +200,29 @@ describe('source runtime', () => { FRESHELL_HOME: homeDir, FRESHELL_BIND_HOST: '127.0.0.1', FRESHELL_CLIENT_DIR: path.join(PROJECT_ROOT, 'dist', 'client'), + PATH: `${shimDir}${path.delimiter}${process.env.PATH ?? ''}`, }, - stdio: 'ignore', + stdio: ['ignore', 'pipe', 'pipe'], shell: startCommand.viaShell ?? false, }) + recordFixture('source_runtime_start_fixture_started', { pid: child.pid, port, homeDir }) + for (const stream of [child.stdout, child.stderr]) { + stream?.on('data', (data: Buffer) => { + // Keep synthetic startup diagnostics bounded while still draining + // both pipes so diagnostics cannot block the owned server. + startupOutput += data.toString().slice(0, Math.max(0, 64 * 1024 - startupOutput.length)) + }) + } - await waitForHealth(`http://127.0.0.1:${port}`) + try { + await waitForHealth(`http://127.0.0.1:${port}`) + } catch (error) { + throw new Error(`${error instanceof Error ? error.message : String(error)}; ` + + `owned child pid=${child.pid} exit=${child.exitCode} signal=${child.signalCode}; ${startupOutput}`) + } const rustPid = await waitForRustChild(child.pid!) ownedRustPid = rustPid + recordFixture('source_runtime_release_child_identified', { pid: rustPid, parentPid: child.pid }) const response = await fetch(`http://127.0.0.1:${port}/api/server-info`, { headers: { 'x-auth-token': token }, }) @@ -197,6 +237,7 @@ describe('source runtime', () => { process.kill(rustPid, 'SIGTERM') await waitForExit(child) + recordFixture('source_runtime_start_fixture_joined', { pid: child.pid, rustPid, exit: child.exitCode, signal: child.signalCode }) ownedRustPid = undefined child = undefined }) From 2102dfebd47fd4e470c839e9aa87727a769467d1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 18:21:47 -0700 Subject: [PATCH 208/493] fix(testing): isolate source runtime process suite --- docs/development/test-sandbox.md | 14 +- scripts/testing/run-source-runtime-tests.ts | 113 ++++++++++---- .../tooling/runtime-manager-launch.test.ts | 138 +++++++++++++++++- 3 files changed, 235 insertions(+), 30 deletions(-) diff --git a/docs/development/test-sandbox.md b/docs/development/test-sandbox.md index e17b39644..87323355d 100644 --- a/docs/development/test-sandbox.md +++ b/docs/development/test-sandbox.md @@ -58,7 +58,19 @@ the sandbox Cargo target volume. Cargo uses three parallel build jobs within the existing container resource limits. The launcher removes its temporary directory on exit. Sandbox, dependency, and inner-runner failures return failure without host Cargo fallback. -The other full-suite phases keep their configured backends and coordinator. +The same exact opt-in routes the source-runtime process suite through the strict +sandbox. Its prebuild guard, client build, tools build, and locked release Rust +build still run natively in that order. The sandbox prepares its separate locked +release binary, then runs the complete runtime suite against the native client +and tools artifacts without rebuilding those bind-mounted files. It owns an +executable temporary directory, transports the checkout commit identity, and +returns failures without host runtime fallback. A direct focused invocation is: + +```bash +FRESHELL_RUST_TEST_SANDBOX=1 pnpm run test:source-runtime +``` + +The client and Electron phases keep their configured backends and coordinator. The wrapper records its owned container and settles it before returning a signal failure; stopping the attached Docker client alone is insufficient. Warm dependency caches through the documented ordinary sandbox preparation diff --git a/scripts/testing/run-source-runtime-tests.ts b/scripts/testing/run-source-runtime-tests.ts index a871d918e..488696172 100644 --- a/scripts/testing/run-source-runtime-tests.ts +++ b/scripts/testing/run-source-runtime-tests.ts @@ -1,6 +1,7 @@ #!/usr/bin/env node -import { spawn, type ChildProcess } from 'node:child_process' +import { execFileSync, spawn, type ChildProcess } from 'node:child_process' +import { constants as osConstants } from 'node:os' import path from 'node:path' import { fileURLToPath, pathToFileURL } from 'node:url' import { @@ -58,9 +59,13 @@ export function buildSourceRuntimePhases(managerCommand?: string): SourceRuntime ] } -function runChild(command: string, args: string[], env: NodeJS.ProcessEnv, viaShell = false): Promise { +function runChild(command: string, args: string[], env: NodeJS.ProcessEnv, viaShell = false, phase = 'runtime-suite'): Promise { return new Promise((resolve) => { - log('info', 'source_runtime_phase_started', { command, args }) + // Selectors may contain caller text. Log counts and identities, never + // the shell payload or forwarded arguments. + const metadata = { phase, argumentCount: args.length } + log('info', 'source_runtime_phase_started', metadata) + let settled = false let child: ChildProcess try { child = spawn(command, args, { @@ -70,65 +75,123 @@ function runChild(command: string, args: string[], env: NodeJS.ProcessEnv, viaSh windowsHide: true, shell: viaShell, }) + log('info', 'source_runtime_child_started', { ...metadata, pid: child.pid }) } catch (error) { log('error', 'source_runtime_phase_spawn_failed', { - command, - args, + ...metadata, error: error instanceof Error ? error.message : String(error), }) resolve(1) return } - const forwardSignal = (signal: NodeJS.Signals) => { - if (child.exitCode === null && !child.killed) child.kill(signal) - } - process.once('SIGINT', forwardSignal) - process.once('SIGTERM', forwardSignal) - process.once('SIGHUP', forwardSignal) + const handlers = (['SIGINT', 'SIGTERM', 'SIGHUP'] as const).map(signal => { + // Node signal listeners have no argument; capture the actual signal. + const handler = () => { if (child.exitCode === null) child.kill(signal) } + process.on(signal, handler) + return { signal, handler } + }) const finish = (code: number, signal: NodeJS.Signals | null) => { - process.off('SIGINT', forwardSignal) - process.off('SIGTERM', forwardSignal) - process.off('SIGHUP', forwardSignal) + if (settled) return + settled = true + for (const { signal: forwarded, handler } of handlers) process.off(forwarded, handler) if (code !== 0 || signal) { - log('error', 'source_runtime_phase_failed', { command, args, code, signal }) + log('error', 'source_runtime_phase_failed', { ...metadata, pid: child.pid, code, signal }) } else { - log('info', 'source_runtime_phase_finished', { command, args, code }) + log('info', 'source_runtime_phase_finished', { ...metadata, pid: child.pid, code }) } resolve(code) } child.once('error', (error) => { log('error', 'source_runtime_phase_child_error', { - command, - args, + ...metadata, pid: child.pid, error: error.message, }) finish(1, null) }) - child.once('exit', (code, signal) => finish(code ?? 1, signal)) + // Join closed stdio as well as process exit before starting another phase. + child.once('close', (code, signal) => finish(code ?? (signal ? 128 + (osConstants.signals[signal] ?? 1) : 1), signal)) }) } +export async function runSourceRuntimeSuite(argv: string[]): Promise { + const vitest = resolveVitestCommand(PROJECT_ROOT) + const config = path.join(PROJECT_ROOT, 'config', 'vitest', 'vitest.runtime.config.ts') + return runChild(vitest.command, [...vitest.args, 'run', '--config', config, ...argv], process.env) +} + +function shellQuote(value: string): string { + return `'${value.replaceAll("'", "'\\''")}'` +} + +function sandboxCommand(argv: string[], commit: string): string { + return `set -e +owned_tmp="$(mktemp -d /freshell-test-artifacts/.freshell-source-runtime-XXXXXXXX)" +runner_pid= +cleanup() { + status=$? + trap - EXIT + if rm -rf -- "$owned_tmp" && [ ! -e "$owned_tmp" ]; then + printf '{"severity":"info","event":"source_runtime_temp_settled","directory":"%s","code":%s}\\n' "$owned_tmp" "$status" + else + status=1 + printf '{"severity":"error","event":"source_runtime_temp_cleanup_failed","directory":"%s","code":1}\\n' "$owned_tmp" + fi + exit "$status" +} +interrupt() { + trap '' INT TERM HUP + if [ -n "$runner_pid" ]; then + kill -s "$1" "$runner_pid" 2>/dev/null || true + wait "$runner_pid" || true + fi + exit "$2" +} +trap cleanup EXIT +trap 'interrupt INT 130' INT +trap 'interrupt TERM 143' TERM +trap 'interrupt HUP 129' HUP +export TMPDIR="$owned_tmp" FRESHELL_TEST_EXECUTABLE_TMP_DIR="$owned_tmp" FRESHELL_RUST_TEST_SANDBOX=0 CARGO_BUILD_JOBS=3 FRESHELL_BUILD_COMMIT=${shellQuote(commit)} +printf '{"severity":"info","event":"source_runtime_temp_created","directory":"%s"}\\n' "$owned_tmp" +# Native client/tools preparation has already completed. Only the container +# target is hidden by the sandbox volume, so prepare its current server there. +cargo build --release -p freshell-server --locked & +runner_pid=$! +wait "$runner_pid" +node --import tsx --input-type=module -e 'import { runSourceRuntimeSuite } from "/workspace/scripts/testing/run-source-runtime-tests.ts"; process.exitCode = await runSourceRuntimeSuite(process.argv.slice(1))' -- ${argv.map(shellQuote).join(' ')} & +runner_pid=$! +wait "$runner_pid"` +} + export async function main(argv: string[] = process.argv.slice(2)): Promise { + const forwarded = argv[0] === '--' ? argv.slice(1) : argv const noTestsOverride = '--passWithNoTests' - if (argv.some((arg) => arg === noTestsOverride || arg.startsWith(`${noTestsOverride}=`))) { + if (forwarded.some((arg) => arg === noTestsOverride || arg.startsWith(`${noTestsOverride}=`))) { log('error', 'source_runtime_vacuous_flag_rejected', { message: `The source-runtime lane must fail when its selector is empty; ${noTestsOverride} is not allowed.`, }) return 2 } - for (const phase of buildSourceRuntimePhases()) { - const exitCode = await runChild(phase.command, phase.args, process.env, phase.viaShell ?? false) + const phaseNames = ['prebuild', 'build-client', 'build-tools', 'build-release'] + for (const [index, phase] of buildSourceRuntimePhases().entries()) { + const exitCode = await runChild(phase.command, phase.args, process.env, phase.viaShell ?? false, phaseNames[index]) if (exitCode !== 0) return exitCode } - const vitest = resolveVitestCommand(PROJECT_ROOT) - const config = path.join(PROJECT_ROOT, 'config', 'vitest', 'vitest.runtime.config.ts') - return runChild(vitest.command, [...vitest.args, 'run', '--config', config, ...argv], process.env) + if (process.env.FRESHELL_RUST_TEST_SANDBOX === '1') { + // Linked-worktree Git metadata is outside the bind mount. Transport only + // the validated commit identity, never the host Git directory or binary. + const commit = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: PROJECT_ROOT, encoding: 'utf8' }).trim() + if (!/^[a-f0-9]{40}$/.test(commit)) throw new Error('Cannot identify the source-runtime build commit.') + log('info', 'source_runtime_sandbox_route_selected', { mode: 'runtime-suite', forwardedArgumentCount: forwarded.length, commit }) + const { FRESHELL_SERVER_BIN: _hostBinary, ...env } = process.env + return runChild(path.join(PROJECT_ROOT, 'scripts/sandbox-test.sh'), ['--runtime-suite', sandboxCommand(forwarded, commit)], env, false, 'sandbox') + } + return runSourceRuntimeSuite(forwarded) } if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) { diff --git a/test/unit/tooling/runtime-manager-launch.test.ts b/test/unit/tooling/runtime-manager-launch.test.ts index 02f92cfa5..57124a1ee 100644 --- a/test/unit/tooling/runtime-manager-launch.test.ts +++ b/test/unit/tooling/runtime-manager-launch.test.ts @@ -1,5 +1,6 @@ // @vitest-environment node import { EventEmitter } from 'node:events' +import { spawnSync } from 'node:child_process' import fs from 'node:fs' import { createRequire } from 'node:module' import os from 'node:os' @@ -34,7 +35,7 @@ function createLaunchFixture(): LaunchFixture { const base = fs.mkdtempSync(path.join(os.tmpdir(), 'freshell-runtime-manager-')) // Spaces in the fixture root prove manager argv and entrypoint paths // survive child-process spawning without shell re-parsing. - const root = `${base} dir with spaces` + const root = path.join(base, 'dir with spaces') fs.mkdirSync(root) const pnpmCjs = path.join(root, 'pnpm.cjs') @@ -59,7 +60,7 @@ afterEach(() => { vi.restoreAllMocks() vi.unstubAllEnvs() spawn.mockReset() - fs.rmSync(fixture.root, { recursive: true, force: true }) + fs.rmSync(path.dirname(fixture.root), { recursive: true, force: true }) }) describe('runtime test script manager launches', () => { @@ -68,7 +69,10 @@ describe('runtime test script manager launches', () => { const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) queueMicrotask(() => { if (command.endsWith('.cmd')) child.emit('error', new Error('spawn EINVAL')) - else child.emit('exit', 0, null) + else { + child.emit('exit', 0, null) + child.emit('close', 0, null) + } }) return child }) @@ -77,7 +81,10 @@ describe('runtime test script manager launches', () => { function mockSpawnSucceedingThroughShell(): void { spawn.mockImplementation(() => { const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) - queueMicrotask(() => child.emit('exit', 0, null)) + queueMicrotask(() => { + child.emit('exit', 0, null) + child.emit('close', 0, null) + }) return child }) } @@ -91,6 +98,129 @@ describe('runtime test script manager launches', () => { })) } + it('keeps native preparation but routes the process suite through the strict sandbox on exact opt-in', async () => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + mockSpawnResolvingChildren() + + expect(await runSourceRuntimeTests(['--', 'filter with space', '', "quote'word", '$(command)', 'unicode-🚀'])).toBe(0) + const specs = launchedSpecs() + expect(specs.slice(0, 4).map(spec => spec.args)).toEqual([ + [fixture.pnpmCjs, 'run', 'prebuild'], + [fixture.pnpmCjs, 'run', 'build:client'], + [fixture.pnpmCjs, 'run', 'build:tools'], + ['build', '--release', '-p', 'freshell-server', '--locked'], + ]) + expect(specs).toHaveLength(5) + expect(specs[4].command).toBe(path.resolve('scripts/sandbox-test.sh')) + expect((specs[4].args as string[])[0]).toBe('--runtime-suite') + }) + + it.each(['0', 'true', ''])('keeps direct runtime dispatch for non-opt-in %j even with cloud configured', async value => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', value) + vi.stubEnv('FRESHELL_VITEST_BACKEND', 'cloud') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + mockSpawnResolvingChildren() + expect(await runSourceRuntimeTests([])).toBe(0) + expect(launchedSpecs().at(-1)).toMatchObject({ command: process.execPath, args: [expectedVitestEntrypoint, 'run', '--config', expect.any(String)] }) + }) + + it.each([17, 125, 143])('returns sandbox failure %s without a host runtime fallback', async code => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + spawn.mockImplementation((command: string) => { + const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) + queueMicrotask(() => { + const result = command.endsWith('sandbox-test.sh') ? code : 0 + child.emit('exit', result, null) + child.emit('close', result, null) + }) + return child + }) + expect(await runSourceRuntimeTests([])).toBe(code) + expect(spawn).toHaveBeenCalledTimes(5) + }) + + it.each([0, 31])('executes the sandbox shell boundary with exact argv and cleanup on suite exit %s', async code => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + const receipt = path.join(fixture.root, 'receipt.jsonl') + const helper = path.join(fixture.root, 'record.mjs') + fs.writeFileSync(helper, `import fs from 'node:fs'; + fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:process.argv[2], args:process.argv.slice(3), + route:process.env.FRESHELL_RUST_TEST_SANDBOX, jobs:process.env.CARGO_BUILD_JOBS, + tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); + process.exit(process.argv[2]==='node' ? Number(process.env.SUITE_EXIT) : 0);`) + const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` + for (const name of ['cargo', 'node']) { + fs.writeFileSync(path.join(fixture.binDir, name), `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(helper)} ${name} "$@"\n`, { mode: 0o755 }) + } + let shellOutput = '' + spawn.mockImplementation((command: string, args: string[]) => { + const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) + queueMicrotask(() => { + let status = 0 + if (command.endsWith('sandbox-test.sh')) { + const payload = args[1].replace('/freshell-test-artifacts/', `${quote(fixture.root)}/`) + const result = spawnSync('bash', ['-c', payload], { + cwd: fixture.root, + env: { ...process.env, PATH: `${fixture.binDir}:${process.env.PATH}`, RECEIPT: receipt, SUITE_EXIT: String(code) }, + encoding: 'utf8', timeout: 10_000, + }) + shellOutput = result.stdout + result.stderr + status = result.status ?? 1 + } + child.emit('exit', status, null) + child.emit('close', status, null) + }) + return child + }) + const args = ['--', 'filter with space', '', "quote'word", '$(touch SHOULD_NOT_EXIST)', 'a;echo nope', 'line\nbreak', 'unicode-🚀', '/workspace/target/argument'] + expect(await runSourceRuntimeTests(args), shellOutput).toBe(code) + const events = fs.readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) + expect(events.map(event => event.kind)).toEqual(['cargo', 'node']) + expect(events[0].args).toEqual(['build', '--release', '-p', 'freshell-server', '--locked']) + // Execute the real shell, then inspect the literal node argv that it delivered. + expect(events[1].args.slice(0, 4)).toEqual(['--import', 'tsx', '--input-type=module', '-e']) + expect(events[1].args.slice(5)).toEqual(['--', ...args.slice(1)]) + expect(events[1].route).toBe('0') + expect(events[1].jobs).toBe('3') + expect(events[1].tmp).toBe(events[1].executableTmp) + expect(fs.existsSync(events[1].tmp)).toBe(false) + expect(fs.existsSync(path.join(fixture.root, 'SHOULD_NOT_EXIST'))).toBe(false) + expect(shellOutput).not.toContain('quote') + expect(shellOutput).toContain('source_runtime_temp_settled') + }) + + it('forwards SIGTERM from a simulated listener and removes its handlers after child close', async () => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + const before = process.listenerCount('SIGTERM') + const kill = vi.fn() + spawn.mockImplementation((command: string) => { + const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false, kill }) + queueMicrotask(() => { + if (command.endsWith('sandbox-test.sh')) { + process.emit('SIGTERM') + child.emit('exit', null, 'SIGTERM') + child.emit('close', null, 'SIGTERM') + } else { + child.emit('exit', 0, null) + child.emit('close', 0, null) + } + }) + return child + }) + expect(await runSourceRuntimeTests([])).toBe(143) + expect(kill).toHaveBeenCalledExactlyOnceWith('SIGTERM') + expect(process.listenerCount('SIGTERM')).toBe(before) + }) + + it.each(['--passWithNoTests', '--passWithNoTests=true'])('rejects vacuous coverage flag %s before preparation', async arg => { + expect(await runSourceRuntimeTests(['--', arg])).toBe(2) + expect(spawn).not.toHaveBeenCalled() + }) + it('launches the source-runtime build phases through node and the detected pnpm entrypoint on native Windows', async () => { vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') // POSIX-style path so path.basename recognizes the pnpm entrypoint on every From f9c9824280abf9043f6620958b97e2f47ef798bb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 18:42:17 -0700 Subject: [PATCH 209/493] test(testing): isolate runtime launcher commit fixtures --- .../tooling/runtime-manager-launch.test.ts | 35 +++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/test/unit/tooling/runtime-manager-launch.test.ts b/test/unit/tooling/runtime-manager-launch.test.ts index 57124a1ee..545d84f17 100644 --- a/test/unit/tooling/runtime-manager-launch.test.ts +++ b/test/unit/tooling/runtime-manager-launch.test.ts @@ -7,10 +7,11 @@ import os from 'node:os' import path from 'node:path' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -const { spawn } = vi.hoisted(() => ({ spawn: vi.fn() })) +const { spawn, execFileSync } = vi.hoisted(() => ({ spawn: vi.fn(), execFileSync: vi.fn() })) vi.mock('node:child_process', async (original) => ({ ...await original(), spawn, + execFileSync, })) import { main as runSourceRuntimeTests } from '../../../scripts/testing/run-source-runtime-tests.js' @@ -21,6 +22,7 @@ const require = createRequire(import.meta.url) const vitestPackagePath = require.resolve('vitest/package.json') const vitestManifest = JSON.parse(fs.readFileSync(vitestPackagePath, 'utf8')) as { bin: { vitest: string } } const expectedVitestEntrypoint = path.resolve(path.dirname(vitestPackagePath), vitestManifest.bin.vitest) +const fixtureCommit = 'a'.repeat(40) interface LaunchFixture { root: string @@ -32,7 +34,8 @@ interface LaunchFixture { let fixture: LaunchFixture function createLaunchFixture(): LaunchFixture { - const base = fs.mkdtempSync(path.join(os.tmpdir(), 'freshell-runtime-manager-')) + // The strict sandbox supplies executable storage separately from its /tmp. + const base = fs.mkdtempSync(path.join(process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR ?? os.tmpdir(), 'freshell-runtime-manager-')) // Spaces in the fixture root prove manager argv and entrypoint paths // survive child-process spawning without shell re-parsing. const root = path.join(base, 'dir with spaces') @@ -54,12 +57,15 @@ function createLaunchFixture(): LaunchFixture { beforeEach(() => { fixture = createLaunchFixture() + // Cloud unit tests have no checkout metadata; commit lookup is a fixture. + execFileSync.mockReturnValue(`${fixtureCommit}\n`) }) afterEach(() => { vi.restoreAllMocks() vi.unstubAllEnvs() spawn.mockReset() + execFileSync.mockReset() fs.rmSync(path.dirname(fixture.root), { recursive: true, force: true }) }) @@ -114,6 +120,29 @@ describe('runtime test script manager launches', () => { expect(specs).toHaveLength(5) expect(specs[4].command).toBe(path.resolve('scripts/sandbox-test.sh')) expect((specs[4].args as string[])[0]).toBe('--runtime-suite') + expect(execFileSync).toHaveBeenCalledExactlyOnceWith('git', ['rev-parse', 'HEAD'], { + cwd: path.resolve('.'), encoding: 'utf8', + }) + }) + + it('rejects failed commit lookup before sandbox dispatch', async () => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + mockSpawnResolvingChildren() + execFileSync.mockImplementationOnce(() => { throw new Error('Git metadata unavailable') }) + + await expect(runSourceRuntimeTests([])).rejects.toThrow('Git metadata unavailable') + expect(spawn).toHaveBeenCalledTimes(4) + }) + + it('rejects an invalid commit identity before sandbox dispatch', async () => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + mockSpawnResolvingChildren() + execFileSync.mockReturnValueOnce('unknown\n') + + await expect(runSourceRuntimeTests([])).rejects.toThrow('Cannot identify the source-runtime build commit.') + expect(spawn).toHaveBeenCalledTimes(4) }) it.each(['0', 'true', ''])('keeps direct runtime dispatch for non-opt-in %j even with cloud configured', async value => { @@ -149,6 +178,7 @@ describe('runtime test script manager launches', () => { fs.writeFileSync(helper, `import fs from 'node:fs'; fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:process.argv[2], args:process.argv.slice(3), route:process.env.FRESHELL_RUST_TEST_SANDBOX, jobs:process.env.CARGO_BUILD_JOBS, + commit:process.env.FRESHELL_BUILD_COMMIT, tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); process.exit(process.argv[2]==='node' ? Number(process.env.SUITE_EXIT) : 0);`) const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` @@ -185,6 +215,7 @@ describe('runtime test script manager launches', () => { expect(events[1].args.slice(5)).toEqual(['--', ...args.slice(1)]) expect(events[1].route).toBe('0') expect(events[1].jobs).toBe('3') + expect(events[1].commit).toBe(fixtureCommit) expect(events[1].tmp).toBe(events[1].executableTmp) expect(fs.existsSync(events[1].tmp)).toBe(false) expect(fs.existsSync(path.join(fixture.root, 'SHOULD_NOT_EXIST'))).toBe(false) From 85b4a20e909c2055ab494db258dbac14a34e22c0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 21:20:38 -0700 Subject: [PATCH 210/493] fix(testing): bound Rust sandbox workers and isolate executable fixtures --- scripts/testing/run-rust-tests.ts | 15 +++- test/unit/tooling/run-rust-tests-fixture.ts | 75 +++++++++++++++++++ test/unit/tooling/run-rust-tests.test.ts | 71 +++--------------- .../rust-launcher-resources.sandbox.ts | 61 +++++++++++++++ test/unit/tooling/sandbox-test.config.ts | 2 +- test/unit/tooling/sandbox-test.sandbox.ts | 2 +- 6 files changed, 162 insertions(+), 64 deletions(-) create mode 100644 test/unit/tooling/run-rust-tests-fixture.ts create mode 100644 test/unit/tooling/rust-launcher-resources.sandbox.ts diff --git a/scripts/testing/run-rust-tests.ts b/scripts/testing/run-rust-tests.ts index 4148c4de3..12342a8f9 100644 --- a/scripts/testing/run-rust-tests.ts +++ b/scripts/testing/run-rust-tests.ts @@ -7,6 +7,9 @@ import { constants as osConstants } from 'node:os' const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url)) const PROJECT_ROOT = path.resolve(SCRIPT_DIR, '../..') +const SANDBOX_CARGO_BUILD_JOBS = 3 +const SANDBOX_RUST_TEST_THREADS = 4 +const SANDBOX_TOKIO_WORKER_THREADS = 2 function log(severity: 'info' | 'error', event: string, fields: Record = {}): void { const stream = severity === 'error' ? process.stderr : process.stdout @@ -78,7 +81,7 @@ function sandboxCommand(argv: string[]): string { // Fixed wrapper text owns only its newly allocated target subdirectory. // Arguments remain separately quoted words; none becomes shell code. return `set -e -owned_tmp="$(mktemp -d /workspace/target/.freshell-rust-tests-XXXXXXXX)" +owned_tmp="$(mktemp -d /freshell-test-artifacts/.freshell-rust-tests-XXXXXXXX)" runner_pid= cleanup() { status=$? @@ -98,7 +101,9 @@ trap cleanup EXIT trap 'interrupt INT 130' INT trap 'interrupt TERM 143' TERM trap 'interrupt HUP 129' HUP -export TMPDIR="$owned_tmp" FRESHELL_TEST_EXECUTABLE_TMP_DIR="$owned_tmp" FRESHELL_RUST_TEST_SANDBOX=0 CARGO_BUILD_JOBS=3 +# Preserve every test while bounding independent libtest/Tokio runtimes within +# the sandbox's existing PID budget. Runtime construction precedes fixture locks. +export TMPDIR="$owned_tmp" FRESHELL_TEST_EXECUTABLE_TMP_DIR="$owned_tmp" FRESHELL_RUST_TEST_SANDBOX=0 CARGO_BUILD_JOBS=${SANDBOX_CARGO_BUILD_JOBS} RUST_TEST_THREADS=${SANDBOX_RUST_TEST_THREADS} TOKIO_WORKER_THREADS=${SANDBOX_TOKIO_WORKER_THREADS} node --import tsx /workspace/scripts/testing/run-rust-tests.ts ${argv.map(shellQuote).join(' ')} & runner_pid=$! wait "$runner_pid"` @@ -106,7 +111,11 @@ wait "$runner_pid"` export async function main(argv: string[] = process.argv.slice(2)): Promise { if (process.env.FRESHELL_RUST_TEST_SANDBOX === '1') { - log('info', 'rust_test_sandbox_route_selected', { mode: 'runtime-suite', cargoBuildJobs: 3, forwardedArgumentCount: argv.length }) + log('info', 'rust_test_sandbox_route_selected', { + mode: 'runtime-suite', cargoBuildJobs: SANDBOX_CARGO_BUILD_JOBS, + rustTestThreads: SANDBOX_RUST_TEST_THREADS, tokioWorkerThreads: SANDBOX_TOKIO_WORKER_THREADS, + forwardedArgumentCount: argv.length, + }) // Do not inherit a host binary selection. The unchanged inner runner // selects its own freshly built /workspace/target/debug server. const { FRESHELL_SERVER_BIN: _hostBinary, ...env } = process.env diff --git a/test/unit/tooling/run-rust-tests-fixture.ts b/test/unit/tooling/run-rust-tests-fixture.ts new file mode 100644 index 000000000..5869ca433 --- /dev/null +++ b/test/unit/tooling/run-rust-tests-fixture.ts @@ -0,0 +1,75 @@ +import { spawnSync } from 'node:child_process' +import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import { createRequire } from 'node:module' + +const require = createRequire(import.meta.url) +const tsx = require.resolve('tsx') +const launcher = path.resolve('scripts/testing/run-rust-tests.ts') + +export function fixture() { + const temporaryRoot = process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR ?? os.tmpdir() + const root = mkdtempSync(path.join(temporaryRoot, 'rust-launcher-')) + mkdirSync(path.join(root, 'scripts/testing'), { recursive: true }) + mkdirSync(path.join(root, 'target')) + mkdirSync(path.join(root, 'bin')) + const artifacts = mkdtempSync(path.join(temporaryRoot, 'rust-artifacts-')) + copyFileSync(launcher, path.join(root, 'scripts/testing/run-rust-tests.ts')) + writeFileSync(path.join(root, 'package.json'), '{"type":"module"}') + symlinkSync(path.resolve('node_modules'), path.join(root, 'node_modules'), 'dir') + const receipt = path.join(root, 'receipt.jsonl') + const executable = (file: string, body: string) => { + writeFileSync(`${file}.mjs`, body) + const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` + writeFileSync(file, `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(`${file}.mjs`)} "$@"\n`) + chmodSync(file, 0o755) + } + executable(path.join(root, 'bin/cargo'), ` + import fs from 'node:fs'; + const args = process.argv.slice(2); + if (args[0] === 'test' && process.env.RUNTIME_PROBE) { + const cp = await import('node:child_process'); + const result = cp.spawnSync(process.env.RUNTIME_PROBE, {env:process.env, stdio:'inherit'}); + process.exit(result.status ?? 1); + } + if (args[0] === 'test' && process.env.COMMONJS_PROBE) { + const cp = await import('node:child_process'); + const file = process.env.TMPDIR+'/fixture.js'; + fs.writeFileSync(file, "const fs = require('node:fs'); fs.writeFileSync(process.env.COMMONJS_PROBE, 'executed');"); + const result = cp.spawnSync(process.execPath, [file], {env:process.env, stdio:'inherit'}); + process.exit(result.status ?? 1); + } + fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'cargo', args, + binary:process.env.FRESHELL_SERVER_BIN, route:process.env.FRESHELL_RUST_TEST_SANDBOX, jobs:process.env.CARGO_BUILD_JOBS, + tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); + process.exit(Number(args[0]==='build' ? process.env.BUILD_EXIT || 0 : process.env.TEST_EXIT || 0)); + `) + executable(path.join(root, 'scripts/sandbox-test.sh'), ` + import fs from 'node:fs'; + import cp from 'node:child_process'; + const args = process.argv.slice(2); + fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'sandbox', args})+'\\n'); + if (process.env.SANDBOX_EXIT) process.exit(Number(process.env.SANDBOX_EXIT)); + const command = args[1] + .replaceAll('/freshell-test-artifacts', ${JSON.stringify(artifacts)}) + .replace('mktemp -d /workspace/target/', 'mktemp -d ' + ${JSON.stringify(root)} + '/target/') + .replace('node --import tsx /workspace/scripts/testing/run-rust-tests.ts', 'node --import tsx ' + ${JSON.stringify(root)} + '/scripts/testing/run-rust-tests.ts'); + const result = cp.spawnSync('bash', ['-c', command], {env:process.env, stdio:'inherit'}); + process.exit(result.status ?? 1); + `) + return { + root, artifacts, + run(env: NodeJS.ProcessEnv = {}, args: string[] = []) { + const result = spawnSync(process.execPath, ['--import', tsx, path.join(root, 'scripts/testing/run-rust-tests.ts'), ...args], { + env: { ...process.env, PATH: `${root}/bin:${process.env.PATH}`, RECEIPT: receipt, + FRESHELL_RUST_TEST_SANDBOX: '0', FRESHELL_SERVER_BIN: '/host/stale-server', ...env }, + encoding: 'utf8', timeout: 15_000, + }) + const events = existsSync(receipt) ? readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) : [] + return { ...result, events } + }, + cleanup() { rmSync(root, { recursive: true, force: true }); rmSync(artifacts, { recursive: true, force: true }) }, + } +} + diff --git a/test/unit/tooling/run-rust-tests.test.ts b/test/unit/tooling/run-rust-tests.test.ts index 5650c55f1..bfa8cada1 100644 --- a/test/unit/tooling/run-rust-tests.test.ts +++ b/test/unit/tooling/run-rust-tests.test.ts @@ -1,66 +1,19 @@ // @vitest-environment node -import { spawnSync } from 'node:child_process' -import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' -import os from 'node:os' +import { existsSync, readFileSync, writeFileSync } from 'node:fs' import path from 'node:path' -import { createRequire } from 'node:module' import { describe, expect, it } from 'vitest' - -const require = createRequire(import.meta.url) -const tsx = require.resolve('tsx') -const launcher = path.resolve('scripts/testing/run-rust-tests.ts') - -function fixture() { - const root = mkdtempSync(path.join(os.tmpdir(), 'rust-launcher-')) - mkdirSync(path.join(root, 'scripts/testing'), { recursive: true }) - mkdirSync(path.join(root, 'target')) - mkdirSync(path.join(root, 'bin')) - copyFileSync(launcher, path.join(root, 'scripts/testing/run-rust-tests.ts')) - writeFileSync(path.join(root, 'package.json'), '{"type":"module"}') - symlinkSync(path.resolve('node_modules'), path.join(root, 'node_modules'), 'dir') - const receipt = path.join(root, 'receipt.jsonl') - const executable = (file: string, body: string) => { - writeFileSync(`${file}.mjs`, body) - const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` - writeFileSync(file, `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(`${file}.mjs`)} "$@"\n`) - chmodSync(file, 0o755) - } - executable(path.join(root, 'bin/cargo'), ` - import fs from 'node:fs'; - const args = process.argv.slice(2); - fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'cargo', args, - binary:process.env.FRESHELL_SERVER_BIN, route:process.env.FRESHELL_RUST_TEST_SANDBOX, jobs:process.env.CARGO_BUILD_JOBS, - tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); - process.exit(Number(args[0]==='build' ? process.env.BUILD_EXIT || 0 : process.env.TEST_EXIT || 0)); - `) - executable(path.join(root, 'scripts/sandbox-test.sh'), ` - import fs from 'node:fs'; - import cp from 'node:child_process'; - const args = process.argv.slice(2); - fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'sandbox', args})+'\\n'); - if (process.env.SANDBOX_EXIT) process.exit(Number(process.env.SANDBOX_EXIT)); - const command = args[1] - .replace('mktemp -d /workspace/target/', 'mktemp -d ' + ${JSON.stringify(root)} + '/target/') - .replace('node --import tsx /workspace/scripts/testing/run-rust-tests.ts', 'node --import tsx ' + ${JSON.stringify(root)} + '/scripts/testing/run-rust-tests.ts'); - const result = cp.spawnSync('bash', ['-c', command], {env:process.env, stdio:'inherit'}); - process.exit(result.status ?? 1); - `) - return { - root, - run(env: NodeJS.ProcessEnv = {}, args: string[] = []) { - const result = spawnSync(process.execPath, ['--import', tsx, path.join(root, 'scripts/testing/run-rust-tests.ts'), ...args], { - env: { ...process.env, PATH: `${root}/bin:${process.env.PATH}`, RECEIPT: receipt, - FRESHELL_RUST_TEST_SANDBOX: '0', FRESHELL_SERVER_BIN: '/host/stale-server', ...env }, - encoding: 'utf8', timeout: 15_000, - }) - const events = existsSync(receipt) ? readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) : [] - return { ...result, events } - }, - cleanup() { rmSync(root, { recursive: true, force: true }) }, - } -} +import { fixture } from './run-rust-tests-fixture.js' describe('Rust test launcher children', () => { + it('executes CommonJS fixtures without inheriting the checkout module format', () => { + const f = fixture() + try { + const marker = path.join(f.root, 'executed') + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1', COMMONJS_PROBE: marker }) + expect(result.status, result.stderr).toBe(0) + expect(readFileSync(marker, 'utf8')).toBe('executed') + } finally { f.cleanup() } + }) it('builds before testing and selects the current checkout binary with disabled routing', () => { const f = fixture() try { @@ -105,7 +58,7 @@ describe('Rust test launcher children', () => { expect(result.events[2].args).toEqual(['test', '--workspace', '--locked', ...args.slice(1)]) expect(result.events[2].binary).toBe(path.join(f.root, 'target/debug/freshell-server')) expect(result.events[2].tmp).toBe(result.events[2].executableTmp) - expect(result.events[2].tmp).toMatch(new RegExp(`^${f.root}/target/\\.freshell-rust-tests-[^/]+$`)) + expect(result.events[2].tmp).toMatch(new RegExp(`^${f.artifacts}/\\.freshell-rust-tests-[^/]+$`)) expect(existsSync(result.events[2].tmp)).toBe(false) } finally { f.cleanup() } }) diff --git a/test/unit/tooling/rust-launcher-resources.sandbox.ts b/test/unit/tooling/rust-launcher-resources.sandbox.ts new file mode 100644 index 000000000..de8e471b9 --- /dev/null +++ b/test/unit/tooling/rust-launcher-resources.sandbox.ts @@ -0,0 +1,61 @@ +// @vitest-environment node +import { spawnSync } from 'node:child_process' +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import path from 'node:path' +import { expect, it } from 'vitest' +import { fixture } from './run-rust-tests-fixture.js' + +if (process.env.FRESHELL_SANDBOX_MODE !== 'runtime-suite') { + throw new Error('Rust resource regression requires the disposable runtime sandbox') +} + +it('starts concurrent default Tokio runtimes within the unchanged sandbox PID budget', () => { + const probe = mkdtempSync('/freshell-test-artifacts/.rust-runtime-probe-') + const f = fixture() + try { + mkdirSync(path.join(probe, 'src')) + writeFileSync(path.join(probe, 'Cargo.toml'), `[package] +name = "rust-runtime-probe" +version = "0.0.0" +edition = "2021" +[workspace] +[dependencies] +tokio = { version = "=1.52.3", features = ["rt-multi-thread"] } +`) + writeFileSync(path.join(probe, 'src/main.rs'), `fn main() { + // Like #[tokio::test(flavor = "multi_thread")], every runtime resolves + // its worker count before a test can acquire a shared fixture lock. + let runtimes: Vec<_> = (0..8).map(|_| tokio::runtime::Builder::new_multi_thread().build().unwrap()).collect(); + for runtime in &runtimes { assert_eq!(runtime.block_on(async { 42 }), 42); } +} +`) + const build = spawnSync('cargo', ['build', '--offline', '--manifest-path', path.join(probe, 'Cargo.toml')], { + env: { ...process.env, CARGO_BUILD_JOBS: '3', CARGO_TARGET_DIR: path.join(probe, 'target') }, + encoding: 'utf8', timeout: 120_000, + }) + expect(build.status, build.stderr).toBe(0) + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1', RUNTIME_PROBE: path.join(probe, 'target/debug/rust-runtime-probe') }) + expect(result.status, result.stderr).toBe(0) + } finally { + f.cleanup() + rmSync(probe, { recursive: true, force: true }) + } +}, 150_000) + +it('runs every process-intensive libtest case within the unchanged PID budget', () => { + const f = fixture() + try { + const source = path.join(f.artifacts, 'cases.rs') + const binary = path.join(f.artifacts, 'cases') + writeFileSync(source, Array.from({ length: 12 }, (_, index) => `#[test] +fn case_${index}() { + let children: Vec<_> = (0..64).map(|_| std::thread::spawn(|| std::thread::sleep(std::time::Duration::from_millis(300)))).collect(); + for child in children { child.join().unwrap(); } +}`).join('\n')) + const build = spawnSync('rustc', ['--test', source, '-o', binary], { encoding: 'utf8', timeout: 30_000 }) + expect(build.status, build.stderr).toBe(0) + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1', RUNTIME_PROBE: binary }) + expect(result.status, result.stderr + result.stdout).toBe(0) + expect(result.stdout).toContain('12 passed; 0 failed; 0 ignored') + } finally { f.cleanup() } +}) diff --git a/test/unit/tooling/sandbox-test.config.ts b/test/unit/tooling/sandbox-test.config.ts index 886410881..8bd95e548 100644 --- a/test/unit/tooling/sandbox-test.config.ts +++ b/test/unit/tooling/sandbox-test.config.ts @@ -7,6 +7,6 @@ export default defineConfig({ ...baseConfig, test: { ...baseConfig.test, - include: ['test/unit/tooling/sandbox-test.sandbox.ts', 'test/unit/tooling/run-rust-tests.test.ts'], + include: ['test/unit/tooling/sandbox-test.sandbox.ts', 'test/unit/tooling/run-rust-tests.test.ts', 'test/unit/tooling/rust-launcher-resources.sandbox.ts'], }, }) diff --git a/test/unit/tooling/sandbox-test.sandbox.ts b/test/unit/tooling/sandbox-test.sandbox.ts index e7daa29b8..98c920cb6 100644 --- a/test/unit/tooling/sandbox-test.sandbox.ts +++ b/test/unit/tooling/sandbox-test.sandbox.ts @@ -12,7 +12,7 @@ if (process.env.FRESHELL_SANDBOX_MODE !== 'runtime-suite') { } function fixture(mode: string) { - const root = mkdtempSync(path.join(os.tmpdir(), 'sandbox-wrapper-')) + const root = mkdtempSync(path.join(process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR ?? os.tmpdir(), 'sandbox-wrapper-')) for (const subdir of ['scripts', 'bin', 'control']) mkdirSync(path.join(root, subdir)) copyFileSync(path.resolve('scripts/sandbox-test.sh'), path.join(root, 'scripts/sandbox-test.sh')) const receipt = path.join(root, 'receipt.jsonl') From 5ac077e2741beb70fe2e070722a6d484ec3016a7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 22:10:57 -0700 Subject: [PATCH 211/493] fix(testing): settle strict Rust sandboxes after watchdog timeouts --- docs/development/test-sandbox.md | 16 +- scripts/sandbox-test.sh | 137 +++++++++++------- scripts/testing/run-rust-tests.ts | 8 +- scripts/testing/sandbox-owner-guard.py | 100 +++++++++++++ test/unit/tooling/run-rust-tests-fixture.ts | 6 + .../rust-launcher-resources.sandbox.ts | 41 +++++- test/unit/tooling/sandbox-test.sandbox.ts | 116 ++++++++++++++- 7 files changed, 361 insertions(+), 63 deletions(-) create mode 100644 scripts/testing/sandbox-owner-guard.py diff --git a/docs/development/test-sandbox.md b/docs/development/test-sandbox.md index 87323355d..8d0b00ea2 100644 --- a/docs/development/test-sandbox.md +++ b/docs/development/test-sandbox.md @@ -54,9 +54,13 @@ Only the exact value `1` enables routing; other test backend settings do not. The Rust launcher runs its unchanged locked server build and workspace tests inside the strict sandbox, selects the container's server binary, and gives both temporary-directory variables a newly owned executable directory under -the sandbox Cargo target volume. Cargo uses three parallel build jobs within -the existing container resource limits. The launcher removes its temporary -directory on exit. Sandbox, +the sandbox Cargo target volume's `/freshell-test-artifacts` alias, outside +the checkout's Node package lookup scope. Cargo uses three parallel build jobs; +libtest runs four cases at once and default Tokio runtimes use two workers +within the existing container resource limits. Explicit test worker counts +and the complete test workload remain unchanged. The launcher logs its owned +temporary directory and removes it on exit or before awaiting an interrupted +child that might ignore termination. Sandbox, dependency, and inner-runner failures return failure without host Cargo fallback. The same exact opt-in routes the source-runtime process suite through the strict sandbox. Its prebuild guard, client build, tools build, and locked release Rust @@ -73,6 +77,12 @@ FRESHELL_RUST_TEST_SANDBOX=1 pnpm run test:source-runtime The client and Electron phases keep their configured backends and coordinator. The wrapper records its owned container and settles it before returning a signal failure; stopping the attached Docker client alone is insufficient. +The strict Rust route also starts an independent guard for its exact owned +container. If the phase watchdog kills the launcher, the guard reuses the +wrapper's CID/name settlement checks; unknown daemon state retains the +control directory and an error receipt. Ordinary completion joins the guard +before returning. The coordinator's watchdog limits and kill behavior remain +unchanged. Warm dependency caches through the documented ordinary sandbox preparation before a strict run if needed; strict mode keeps its read-only home and no network. diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 34726b054..9cffdc0e7 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -78,62 +78,75 @@ if [ "${1:-}" = "--native-tools" ]; then --output "${NATIVE_RECEIPT_DIR%/}/native-session-names.json" fi -MOUNT_CORPUS=0 -RUNTIME_SUITE=0 -while [ "$#" -gt 0 ]; do - case "${1}" in - --corpus) - MOUNT_CORPUS=1 - shift - ;; - --runtime-suite) - RUNTIME_SUITE=1 - shift - ;; - --) - shift - break - ;; - *) - break - ;; - esac -done +SETTLE_ONLY=0 +if [ "${1:-}" = "--settle-owned" ]; then + SETTLE_ONLY=1 + OWNED_CONTROL_DIR="${2:?owned control directory is required}" + OWNED_CONTAINER_NAME="$(basename "${OWNED_CONTROL_DIR}")" + if ! [[ "${OWNED_CONTAINER_NAME}" =~ ^freshell-sandbox-[[:alnum:]]{8}$ ]] \ + || [ "$(cat "${OWNED_CONTROL_DIR}/container.name")" != "${OWNED_CONTAINER_NAME}" ]; then + echo '[sandbox] invalid owned settlement control directory' >&2 + exit 1 + fi +else + MOUNT_CORPUS=0 + RUNTIME_SUITE=0 + while [ "$#" -gt 0 ]; do + case "${1}" in + --corpus) + MOUNT_CORPUS=1 + shift + ;; + --runtime-suite) + RUNTIME_SUITE=1 + shift + ;; + --) + shift + break + ;; + *) + break + ;; + esac + done -if [ "${MOUNT_CORPUS}" -eq 1 ] && [ "${RUNTIME_SUITE}" -eq 1 ]; then - echo "[sandbox] --runtime-suite may not mount the real --corpus provider data" >&2 - exit 2 -fi + if [ "${MOUNT_CORPUS}" -eq 1 ] && [ "${RUNTIME_SUITE}" -eq 1 ]; then + echo "[sandbox] --runtime-suite may not mount the real --corpus provider data" >&2 + exit 2 + fi -if [ "$#" -lt 1 ]; then - echo "usage: $0 [--corpus | --runtime-suite] \"\"" >&2 - exit 2 -fi -CMD="$1" + if [ "$#" -lt 1 ]; then + echo "usage: $0 [--corpus | --runtime-suite] \"\"" >&2 + exit 2 + fi + CMD="$1" -if ! docker image inspect "${IMAGE_TAG}" >/dev/null 2>&1; then - echo "[sandbox] image ${IMAGE_TAG} not found, building it first..." >&2 - "${REPO_ROOT}/scripts/sandbox-build.sh" -fi + if ! docker image inspect "${IMAGE_TAG}" >/dev/null 2>&1; then + echo "[sandbox] image ${IMAGE_TAG} not found, building it first..." >&2 + "${REPO_ROOT}/scripts/sandbox-build.sh" + fi -# Ordinary destructive tests keep the established dedicated bridge network. -# Managed-runtime lifecycle tests are stricter: no network at all. Neither mode -# ever uses the host network. -NETWORK_ARGS=() -if [ "${RUNTIME_SUITE}" -eq 1 ]; then - NETWORK_ARGS=(--network none) -else - NETWORK_NAME="freshell-sandbox" - if ! docker network inspect "${NETWORK_NAME}" >/dev/null 2>&1; then - docker network create --driver bridge "${NETWORK_NAME}" >/dev/null + # Ordinary destructive tests keep the established dedicated bridge network. + # Managed-runtime lifecycle tests are stricter: no network at all. Neither mode + # ever uses the host network. + NETWORK_ARGS=() + if [ "${RUNTIME_SUITE}" -eq 1 ]; then + NETWORK_ARGS=(--network none) + else + NETWORK_NAME="freshell-sandbox" + if ! docker network inspect "${NETWORK_NAME}" >/dev/null 2>&1; then + docker network create --driver bridge "${NETWORK_NAME}" >/dev/null + fi + NETWORK_ARGS=(--network "${NETWORK_NAME}") fi - NETWORK_ARGS=(--network "${NETWORK_NAME}") -fi -OWNED_CONTROL_DIR="$(mktemp -d "${TMPDIR:-/tmp}/freshell-sandbox-XXXXXXXX")" -OWNED_CONTAINER_NAME="$(basename "${OWNED_CONTROL_DIR}")" + OWNED_CONTROL_DIR="$(mktemp -d "${TMPDIR:-/tmp}/freshell-sandbox-XXXXXXXX")" + OWNED_CONTAINER_NAME="$(basename "${OWNED_CONTROL_DIR}")" + printf '%s\n' "${OWNED_CONTAINER_NAME}" > "${OWNED_CONTROL_DIR}/container.name" +fi OWNED_CID_FILE="${OWNED_CONTROL_DIR}/container.id" -printf '%s\n' "${OWNED_CONTAINER_NAME}" > "${OWNED_CONTROL_DIR}/container.name" +OWNER_GUARD_PID="" DOCKER_CLIENT_PID="" REQUESTED_SIGNAL="" @@ -206,6 +219,14 @@ cleanup_owned_container() { return "${cleanup_status}" } +# The independent owner guard uses this same exact-ID/name settlement path. +# It runs after a SIGKILL, when the ordinary wrapper cannot execute its traps. +if [ "${SETTLE_ONLY}" -eq 1 ]; then + REQUESTED_SIGNAL="SIGTERM" + cleanup_owned_container + exit $? +fi + # shellcheck disable=SC2317 # Invoked by the installed EXIT trap. finish_owned_sandbox() { local result=$? @@ -235,6 +256,16 @@ finish_owned_sandbox() { fi wait "${DOCKER_CLIENT_PID}" 2>/dev/null || true fi + # The owner completed its cleanup attempt even when daemon state is + # unknown. Join the guard, preserve that failure and retain its evidence. + if [ -n "${OWNER_GUARD_PID}" ]; then + touch "${OWNED_CONTROL_DIR}/owner.released" + if ! python3 "${REPO_ROOT}/scripts/testing/sandbox-owner-guard.py" --join "${OWNED_CONTROL_DIR}"; then + sandbox_log error sandbox_owner_guard_unsettled 1 + result=1 + container_settled=0 + fi + fi if [ "${container_settled}" -eq 1 ]; then rm -rf -- "${OWNED_CONTROL_DIR}" || result=1 else @@ -277,6 +308,14 @@ trap 'interrupt_owned_sandbox 130 SIGINT' INT trap 'interrupt_owned_sandbox 143 SIGTERM' TERM trap 'interrupt_owned_sandbox 129 SIGHUP' HUP +# Only the strict Rust route opts in. A detached owner guard survives the +# unchanged coordinator's SIGKILL tree and settles this preallocated object. +if [ "${RUNTIME_SUITE}" -eq 1 ] && [ "${FRESHELL_SANDBOX_OWNER_GUARD:-}" = "1" ]; then + OWNER_GUARD_PID="$(python3 "${REPO_ROOT}/scripts/testing/sandbox-owner-guard.py" "${OWNED_CONTROL_DIR}" "$$" "${BASH_SOURCE[0]}")" + printf '{"severity":"info","event":"sandbox_owner_guard_started","container":"%s","pid":%s}\n' \ + "${OWNED_CONTAINER_NAME}" "${OWNER_GUARD_PID}" >&2 +fi + DOCKER_ARGS=( # Reap orphaned test children so zombies cannot exhaust the PID limit. run --rm --init diff --git a/scripts/testing/run-rust-tests.ts b/scripts/testing/run-rust-tests.ts index 12342a8f9..67f4239d1 100644 --- a/scripts/testing/run-rust-tests.ts +++ b/scripts/testing/run-rust-tests.ts @@ -77,11 +77,12 @@ function shellQuote(value: string): string { return `'${value.replaceAll("'", "'\\''")}'` } -function sandboxCommand(argv: string[]): string { +export function sandboxCommand(argv: string[]): string { // Fixed wrapper text owns only its newly allocated target subdirectory. // Arguments remain separately quoted words; none becomes shell code. return `set -e owned_tmp="$(mktemp -d /freshell-test-artifacts/.freshell-rust-tests-XXXXXXXX)" +printf '{"severity":"info","event":"rust_test_temp_created","directory":"%s"}\\n' "$owned_tmp" runner_pid= cleanup() { status=$? @@ -93,6 +94,9 @@ interrupt() { trap '' INT TERM HUP if [ -n "$runner_pid" ]; then kill -s "$1" "$runner_pid" 2>/dev/null || true + # A child may ignore termination. Remove our artifacts before waiting; + # the sandbox owner separately settles all remaining container work. + rm -rf -- "$owned_tmp" wait "$runner_pid" || true fi exit "$2" @@ -119,7 +123,7 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise= deadline: + return 1 + time.sleep(0.025) + try: + done = json.loads((control / "owner-guard.done").read_text()) + except (OSError, ValueError): + return 1 + return 0 if done.get("code") == 0 else 1 + + +def watch(control, owner, owner_identity, wrapper): + # Reparent before reporting readiness, so a phase-tree snapshot cannot + # include the guard as a descendant of the attached wrapper. + os.setsid() + if os.fork(): + os._exit(0) + with open(os.devnull, "rb+") as null: + for descriptor in (0, 1, 2): + os.dup2(null.fileno(), descriptor) + record(control, "owner-guard.json", {"pid": os.getpid(), "identity": identity(os.getpid())}) + while identity(owner) == owner_identity: + if (control / "owner.released").exists(): + record(control, "owner-guard.done", {"severity": "info", "event": "sandbox_owner_guard_finished", "code": 0}) + return + time.sleep(0.1) + # No caller command or provider stream enters this cleanup invocation. + # Reuse the wrapper's exact CID/name checks, including unpublished IDs. + with (control / "owner-guard.log").open("wb") as log: + child = subprocess.Popen(["bash", wrapper, "--settle-owned", str(control)], stdout=log, stderr=log, start_new_session=True) + try: + code = child.wait(timeout=120) + except subprocess.TimeoutExpired: + os.killpg(child.pid, signal.SIGKILL) + child.wait() + code = 1 + record(control, "owner-guard.done", {"severity": "info" if code == 0 else "error", "event": "sandbox_owner_guard_settled" if code == 0 else "sandbox_owned_container_unsettled", "code": code}) + if code == 0: + shutil.rmtree(control) + + +def main(): + if sys.argv[1] == "--join": + return join(Path(sys.argv[2])) + control, owner, wrapper = Path(sys.argv[1]), int(sys.argv[2]), os.path.abspath(sys.argv[3]) + owner_identity = identity(owner) + if owner_identity is None: + return 1 + child = os.fork() + if child == 0: + try: + watch(control, owner, owner_identity, wrapper) + except Exception as cause: + if control.exists(): + record(control, "owner-guard.done", {"severity": "error", "event": "sandbox_owner_guard_failed", "code": 1, "cause": type(cause).__name__}) + os._exit(1) + os._exit(0) + _, status = os.waitpid(child, 0) + if status != 0: + return 1 + deadline = time.monotonic() + 5 + while not (control / "owner-guard.json").exists(): + if time.monotonic() >= deadline: + return 1 + time.sleep(0.01) + print(json.loads((control / "owner-guard.json").read_text())["pid"]) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/test/unit/tooling/run-rust-tests-fixture.ts b/test/unit/tooling/run-rust-tests-fixture.ts index 5869ca433..6dc0adf56 100644 --- a/test/unit/tooling/run-rust-tests-fixture.ts +++ b/test/unit/tooling/run-rust-tests-fixture.ts @@ -43,6 +43,12 @@ export function fixture() { fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'cargo', args, binary:process.env.FRESHELL_SERVER_BIN, route:process.env.FRESHELL_RUST_TEST_SANDBOX, jobs:process.env.CARGO_BUILD_JOBS, tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); + if (args[0] === 'test' && process.env.HOLD_TEST) { + process.on('SIGTERM', () => {}); + fs.writeFileSync(process.env.HOLD_TEST, JSON.stringify({pid:process.pid, tmp:process.env.TMPDIR})); + setInterval(() => {}, 10000); + await new Promise(() => {}); + } process.exit(Number(args[0]==='build' ? process.env.BUILD_EXIT || 0 : process.env.TEST_EXIT || 0)); `) executable(path.join(root, 'scripts/sandbox-test.sh'), ` diff --git a/test/unit/tooling/rust-launcher-resources.sandbox.ts b/test/unit/tooling/rust-launcher-resources.sandbox.ts index de8e471b9..47276a654 100644 --- a/test/unit/tooling/rust-launcher-resources.sandbox.ts +++ b/test/unit/tooling/rust-launcher-resources.sandbox.ts @@ -1,14 +1,51 @@ // @vitest-environment node -import { spawnSync } from 'node:child_process' -import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { spawn, spawnSync } from 'node:child_process' +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import path from 'node:path' import { expect, it } from 'vitest' import { fixture } from './run-rust-tests-fixture.js' +import { sandboxCommand } from '../../../scripts/testing/run-rust-tests.js' +import { descendantPids, readProcessSnapshot } from '../../../scripts/testing/process-tree.js' if (process.env.FRESHELL_SANDBOX_MODE !== 'runtime-suite') { throw new Error('Rust resource regression requires the disposable runtime sandbox') } +it('removes its temporary artifacts when an interrupted child ignores termination', async () => { + const f = fixture() + const marker = path.join(f.root, 'held') + const sentinel = path.join(f.artifacts, 'foreign') + writeFileSync(sentinel, 'keep') + const child = spawn('bash', ['-c', sandboxCommand([])], { + env: { ...process.env, PATH: `${f.root}/bin:${process.env.PATH}`, HOLD_TEST: marker, RECEIPT: path.join(f.root, 'receipt.jsonl') }, + stdio: 'ignore', + }) + const joined = new Promise(resolve => child.once('close', resolve)) + let ownedTmp: string | undefined + try { + await expect.poll(() => existsSync(marker), { timeout: 5_000 }).toBe(true) + const running = JSON.parse(readFileSync(marker, 'utf8')) + ownedTmp = running.tmp + console.info(JSON.stringify({ severity: 'info', event: 'rust_interruption_fixture_started', pid: child.pid, childPid: running.pid, temporaryDirectory: ownedTmp })) + expect(existsSync(running.tmp)).toBe(true) + child.kill('SIGTERM') + await expect.poll(() => existsSync(running.tmp), { timeout: 2_000 }).toBe(false) + expect(readFileSync(sentinel, 'utf8')).toBe('keep') + } finally { + // This regression lives only in the disposable PID namespace. It owns + // the exact shell tree and joins it even when the expected RED fails. + if (child.pid !== undefined && child.exitCode === null) { + const snapshot = readProcessSnapshot(process.platform) + for (const pid of [child.pid, ...descendantPids(child.pid, snapshot)]) { + try { process.kill(pid, 'SIGKILL') } catch { /* already settled */ } + } + } + await joined + if (ownedTmp) rmSync(ownedTmp, { recursive: true, force: true }) + f.cleanup() + } +}) + it('starts concurrent default Tokio runtimes within the unchanged sandbox PID budget', () => { const probe = mkdtempSync('/freshell-test-artifacts/.rust-runtime-probe-') const f = fixture() diff --git a/test/unit/tooling/sandbox-test.sandbox.ts b/test/unit/tooling/sandbox-test.sandbox.ts index 98c920cb6..93669329f 100644 --- a/test/unit/tooling/sandbox-test.sandbox.ts +++ b/test/unit/tooling/sandbox-test.sandbox.ts @@ -3,7 +3,8 @@ import { spawnSync } from 'node:child_process' import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import os from 'node:os' import path from 'node:path' -import { describe, expect, it } from 'vitest' +import { describe, expect, it, vi } from 'vitest' +import { spawnAndWait } from '../../../scripts/testing/coordinator-upstream.js' // These fixtures exercise creation-time signals and Docker cleanup. Run them // through scripts/sandbox-test.sh --runtime-suite, never directly on the host. @@ -15,6 +16,11 @@ function fixture(mode: string) { const root = mkdtempSync(path.join(process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR ?? os.tmpdir(), 'sandbox-wrapper-')) for (const subdir of ['scripts', 'bin', 'control']) mkdirSync(path.join(root, subdir)) copyFileSync(path.resolve('scripts/sandbox-test.sh'), path.join(root, 'scripts/sandbox-test.sh')) + const guardSource = process.env.FIXTURE_GUARD_SOURCE ?? 'scripts/testing/sandbox-owner-guard.py' + if (existsSync(guardSource)) { + mkdirSync(path.join(root, 'scripts/testing')) + copyFileSync(guardSource, path.join(root, 'scripts/testing/sandbox-owner-guard.py')) + } const receipt = path.join(root, 'receipt.jsonl') const state = path.join(root, 'owned-object.json') const foreign = path.join(root, 'foreign-object') @@ -32,18 +38,20 @@ function fixture(mode: string) { const name = args[args.indexOf('--name')+1], cidfile = args[args.indexOf('--cidfile')+1]; record({kind:'run', name, cidfile}); if (mode !== 'no-object' && mode !== 'late-creation') fs.writeFileSync(state, JSON.stringify({cid, name})); - if (mode === 'published' || mode === 'removed-race') fs.writeFileSync(cidfile, cid); + if (mode === 'published' || mode === 'published-success' || mode === 'removed-race' || mode === 'watchdog-published') fs.writeFileSync(cidfile, cid); if (mode === 'removed-race') fs.unlinkSync(state); - if (mode === 'late-creation') { + if (mode.startsWith('watchdog-')) { + setInterval(() => {}, 10000); + } else if (mode === 'late-creation') { process.kill(process.ppid, 'SIGTERM'); setTimeout(() => { fs.writeFileSync(state, JSON.stringify({cid, name})); record({kind:'creation-client-settled'}); process.exit(0); }, 150); - } else process.exit(mode === 'published' ? 37 : 42); + } else process.exit(mode === 'published-success' ? 0 : mode === 'published' ? 37 : 42); } else if (args[0] === 'ps') { - if (mode === 'query-failed') process.exit(71); + if (mode === 'query-failed' || mode === 'watchdog-query-failed') process.exit(71); const current = object(), filter = args[args.indexOf('--filter')+1]; if (current && (filter === 'id='+cid || filter === 'name=^/'+current.name+'$')) console.log(cid); } else if (args[0] === 'container' && args[1] === 'inspect') { @@ -61,10 +69,28 @@ function fixture(mode: string) { chmodSync(path.join(root, 'bin/docker'), 0o755) return { root, state, foreign, - run() { + async runTimeout() { + const watchdogLog = vi.spyOn(console, 'error').mockImplementation(() => {}) + let status: number + let watchdogEvents: unknown[][] + try { + status = await spawnAndWait('bash', [path.join(root, 'scripts/sandbox-test.sh'), '--runtime-suite', 'true'], { + ...process.env, PATH: `${root}/bin:${process.env.PATH}`, TMPDIR: path.join(root, 'control'), + FIXTURE_MODE: mode, FIXTURE_STATE: state, FIXTURE_RECEIPT: receipt, + FRESHELL_SANDBOX_OWNER_GUARD: '1', + FRESHELL_TEST_COORDINATOR_PHASE_TIMEOUT_MS: '600', FRESHELL_TEST_COORDINATOR_PHASE_WATCH_POLL_MS: '25', + }, false) + watchdogEvents = [...watchdogLog.mock.calls] + } finally { watchdogLog.mockRestore() } + const events = readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) + const run = events.find(event => event.kind === 'run') + return { status, events, watchdogEvents, control: path.dirname(run.cidfile), name: run.name } + }, + run(guard = false) { const result = spawnSync('bash', [path.join(root, 'scripts/sandbox-test.sh'), '--runtime-suite', 'true'], { env: { ...process.env, PATH: `${root}/bin:${process.env.PATH}`, TMPDIR: path.join(root, 'control'), - FIXTURE_MODE: mode, FIXTURE_STATE: state, FIXTURE_RECEIPT: receipt }, + FIXTURE_MODE: mode, FIXTURE_STATE: state, FIXTURE_RECEIPT: receipt, + FRESHELL_SANDBOX_OWNER_GUARD: guard ? '1' : '0' }, encoding: 'utf8', timeout: 40_000, }) const events = readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) @@ -76,6 +102,82 @@ function fixture(mode: string) { } describe('sandbox wrapper owned launch settlement', () => { + it.each(['published', 'published-success'])('fails when the independent guard reports an error before %s completion', mode => { + const f = fixture(mode) + try { + const guardPath = path.join(f.root, 'scripts/testing/sandbox-owner-guard.py') + writeFileSync(guardPath, readFileSync(guardPath, 'utf8').replace( + ' while identity(owner) == owner_identity:', + ' raise RuntimeError("fixture guard failure")\n while identity(owner) == owner_identity:', + )) + const result = f.run(true) + expect(result.status, result.stderr).toBe(1) + expect(result.stderr).toContain('sandbox_owner_guard_unsettled') + expect(existsSync(f.state)).toBe(false) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + it('preserves an ordinary successful workload after joining its successful guard', () => { + const f = fixture('published-success') + try { + const result = f.run(true) + expect(result.status, result.stderr).toBe(0) + expect(existsSync(f.state)).toBe(false) + expect(existsSync(result.control)).toBe(false) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + it('settles its guard before returning an ordinary failure with unknown daemon state', () => { + const f = fixture('query-failed') + try { + const result = f.run(true) + expect(result.status, result.stderr).toBe(1) + expect(existsSync(f.state)).toBe(true) + expect(existsSync(path.join(result.control, 'owner-guard.done'))).toBe(true) + expect(result.stderr).toContain('sandbox_owned_container_unsettled') + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + + it('preserves an ordinary failed workload status after joining its guard', () => { + const f = fixture('published') + try { + const result = f.run(true) + expect(result.status, result.stderr).toBe(37) + expect(existsSync(f.state)).toBe(false) + expect(existsSync(result.control)).toBe(false) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + + it('retains unknown daemon evidence after the phase watchdog and settles the independent guard', async () => { + const f = fixture('watchdog-query-failed') + try { + const result = await f.runTimeout() + expect(result.status).toBe(125) + const done = path.join(result.control, 'owner-guard.done') + await expect.poll(() => existsSync(done), { timeout: 8_000 }).toBe(true) + expect(JSON.parse(readFileSync(done, 'utf8'))).toMatchObject({ event: 'sandbox_owned_container_unsettled', code: 1 }) + const guard = JSON.parse(readFileSync(path.join(result.control, 'owner-guard.json'), 'utf8')) + await expect.poll(() => existsSync(`/proc/${guard.pid}`), { timeout: 5_000 }).toBe(false) + expect(existsSync(f.state)).toBe(true) + expect(readFileSync(path.join(result.control, 'container.name'), 'utf8').trim()).toBe(result.name) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) + it.each(['watchdog-published', 'watchdog-unpublished'])('settles only its owned object after the actual phase watchdog kills %s', async mode => { + const f = fixture(mode) + try { + const result = await f.runTimeout() + expect(result.status).toBe(125) + expect(JSON.parse(String(result.watchdogEvents[0][0]))).toMatchObject({ event: 'phase_watchdog_settled', reason: 'phase_timeout', timeoutMs: 600 }) + const deadline = Date.now() + 8_000 + while (existsSync(result.control) && Date.now() < deadline) await new Promise(resolve => setTimeout(resolve, 25)) + expect(existsSync(f.state)).toBe(false) + expect(existsSync(result.control)).toBe(false) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { f.cleanup() } + }) it('removes an owned daemon object after failed launch without CID publication', () => { const f = fixture('failed-launch') try { From 6a082d1c9c4398db489cf0a93ff4d938c472e8ab Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 9 Oct 2026 22:11:08 -0700 Subject: [PATCH 212/493] test(freshagent): isolate OpenCode homes and relocated sidecar dependencies --- crates/freshell-freshagent/src/lib.rs | 2 + .../src/model_capabilities_tests.rs | 8 ++ .../src/session_handoff/tests.rs | 35 +++++++ .../freshell-freshagent/src/terminal_tabs.rs | 15 +++ .../src/test_environment.rs | 98 +++++++++++++++++++ 5 files changed, 158 insertions(+) create mode 100644 crates/freshell-freshagent/src/test_environment.rs diff --git a/crates/freshell-freshagent/src/lib.rs b/crates/freshell-freshagent/src/lib.rs index 13d6aaa9f..56df42889 100644 --- a/crates/freshell-freshagent/src/lib.rs +++ b/crates/freshell-freshagent/src/lib.rs @@ -59,6 +59,8 @@ pub mod spawn_gate; pub(crate) mod summary; pub mod target_resolver; pub mod terminal_tabs; +#[cfg(test)] +mod test_environment; pub use claude::FreshClaudeState; // Kata 09v1 + SYNC-06: the TWO claude_snapshot items visible outside this diff --git a/crates/freshell-freshagent/src/model_capabilities_tests.rs b/crates/freshell-freshagent/src/model_capabilities_tests.rs index ea0054f38..97b772249 100644 --- a/crates/freshell-freshagent/src/model_capabilities_tests.rs +++ b/crates/freshell-freshagent/src/model_capabilities_tests.rs @@ -435,6 +435,14 @@ async fn claude_catalog_probe_uses_the_configured_sidecar_directory() { &catalog_entry, ) .expect("copy model catalog helper"); + std::fs::copy( + PathBuf::from(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../freshell-claude-sidecar/exec-profile.mjs" + )), + directory.path().join("exec-profile.mjs"), + ) + .expect("copy model catalog exec dependency"); std::fs::write( &sdk_entry, r#" diff --git a/crates/freshell-freshagent/src/session_handoff/tests.rs b/crates/freshell-freshagent/src/session_handoff/tests.rs index 84c90d2c8..06667206b 100644 --- a/crates/freshell-freshagent/src/session_handoff/tests.rs +++ b/crates/freshell-freshagent/src/session_handoff/tests.rs @@ -7212,6 +7212,11 @@ async fn a_fresh_created_kilroy_session_adopts_the_canonical_key() { /// id is unchanged in every coordinator record and response. #[tokio::test] async fn opencode_handoff_keeps_shared_serve_alive_and_session_id_stable() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::opencode_handoff_keeps_shared_serve_alive_and_session_id_stable", + ) { + return; + } let _guard = ENV_LOCK.lock().await; // The claude-lane env surface this file's tests mutate // (FRESHELL_CLAUDE_SIDECAR/NODE, FAKE_SIDECAR_REQUEST_LOG, @@ -7341,6 +7346,11 @@ async fn opencode_handoff_keeps_shared_serve_alive_and_session_id_stable() { /// failed. #[tokio::test] async fn an_opencode_placeholder_handoff_resolves_the_durable_owner_and_completes() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::an_opencode_placeholder_handoff_resolves_the_durable_owner_and_completes", + ) { + return; + } let _guard = ENV_LOCK.lock().await; let _claude_env = crate::claude::tests::CLAUDE_ENV_LOCK.lock().await; let env = FakeOpencodeServeEnv::install(); @@ -7452,6 +7462,11 @@ async fn an_opencode_placeholder_handoff_resolves_the_durable_owner_and_complete /// killed (OpenCode invariant). #[tokio::test] async fn opencode_handoff_stop_aborts_the_active_turn_through_the_manager_before_reaping() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::opencode_handoff_stop_aborts_the_active_turn_through_the_manager_before_reaping", + ) { + return; + } let _guard = ENV_LOCK.lock().await; // The claude-lane env surface this file's tests mutate // (FRESHELL_CLAUDE_SIDECAR/NODE, FAKE_SIDECAR_REQUEST_LOG, @@ -7589,6 +7604,11 @@ async fn opencode_handoff_stop_aborts_the_active_turn_through_the_manager_before /// daemon-side turn. #[tokio::test] async fn opencode_handoff_stop_aborts_a_daemon_turn_left_running_by_a_finished_local_task() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::opencode_handoff_stop_aborts_a_daemon_turn_left_running_by_a_finished_local_task", + ) { + return; + } let _guard = ENV_LOCK.lock().await; // The claude-lane env surface this file's tests mutate // (FRESHELL_CLAUDE_SIDECAR/NODE, FAKE_SIDECAR_REQUEST_LOG, @@ -7726,6 +7746,11 @@ async fn opencode_handoff_stop_aborts_a_daemon_turn_left_running_by_a_finished_l /// parked) while the abort answers normally. #[tokio::test] async fn opencode_handoff_during_compaction_aborts_the_daemon_side_summarize() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::opencode_handoff_during_compaction_aborts_the_daemon_side_summarize", + ) { + return; + } let _guard = ENV_LOCK.lock().await; // The claude-lane env surface this file's tests mutate // (FRESHELL_CLAUDE_SIDECAR/NODE, FAKE_SIDECAR_REQUEST_LOG, @@ -7844,6 +7869,11 @@ async fn opencode_handoff_during_compaction_aborts_the_daemon_side_summarize() { /// handoff must not complete while the daemon-side abort is unanswered. #[tokio::test] async fn opencode_handoff_stop_aborts_an_in_flight_rest_driven_turn_before_reaping() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::opencode_handoff_stop_aborts_an_in_flight_rest_driven_turn_before_reaping", + ) { + return; + } let _guard = ENV_LOCK.lock().await; let _claude_env = crate::claude::tests::CLAUDE_ENV_LOCK.lock().await; let env = FakeOpencodeServeEnv::install(); @@ -7975,6 +8005,11 @@ async fn opencode_handoff_stop_aborts_an_in_flight_rest_driven_turn_before_reapi /// confirms the daemon-side turn's death BEFORE the reap. #[tokio::test] async fn opencode_handoff_stop_waits_for_a_mid_dispatch_rest_drive_under_the_gate() { + if crate::test_environment::run_with_opencode_home( + "session_handoff::tests::opencode_handoff_stop_waits_for_a_mid_dispatch_rest_drive_under_the_gate", + ) { + return; + } let _guard = ENV_LOCK.lock().await; let _claude_env = crate::claude::tests::CLAUDE_ENV_LOCK.lock().await; let env = FakeOpencodeServeEnv::install(); diff --git a/crates/freshell-freshagent/src/terminal_tabs.rs b/crates/freshell-freshagent/src/terminal_tabs.rs index 6d426131e..0172fba9b 100644 --- a/crates/freshell-freshagent/src/terminal_tabs.rs +++ b/crates/freshell-freshagent/src/terminal_tabs.rs @@ -5308,6 +5308,11 @@ if (args.includes('app-server')) {{ /// record. #[tokio::test] async fn rest_cli_terminal_create_seeds_the_requested_name_into_the_pending_record() { + if crate::test_environment::run_with_opencode_home( + "terminal_tabs::tests::rest_cli_terminal_create_seeds_the_requested_name_into_the_pending_record", + ) { + return; + } use crate::naming::SessionNaming as _; use freshell_protocol::session_names::{NameIntent, SessionNameRef}; @@ -5607,6 +5612,11 @@ if (args.includes('app-server')) {{ #[tokio::test] async fn create_opencode_tab_fresh_spawns_with_hostname_port_args_and_arms_locator() { + if crate::test_environment::run_with_opencode_home( + "terminal_tabs::tests::create_opencode_tab_fresh_spawns_with_hostname_port_args_and_arms_locator", + ) { + return; + } let home = unique_temp_home("opencode-fresh"); let argv_file = unique_argv_file("opencode-fresh"); let state = @@ -8392,6 +8402,11 @@ if (args.includes('app-server')) {{ #[tokio::test] async fn create_opencode_tab_with_session_ref_flows_into_pane_content() { + if crate::test_environment::run_with_opencode_home( + "terminal_tabs::tests::create_opencode_tab_with_session_ref_flows_into_pane_content", + ) { + return; + } let argv_file = unique_argv_file("opencode-synth"); let state = state_with_registry().with_cli_commands(std::sync::Arc::new(vec![recording_cli_spec( diff --git a/crates/freshell-freshagent/src/test_environment.rs b/crates/freshell-freshagent/src/test_environment.rs new file mode 100644 index 000000000..f042ab52e --- /dev/null +++ b/crates/freshell-freshagent/src/test_environment.rs @@ -0,0 +1,98 @@ +//! Process-local homes for tests that exercise the ordinary OpenCode TUI. + +use std::io::Write; +use std::process::{Command, Stdio}; + +const HOME_CHILD: &str = "FRESHELL_OPENCODE_HOME_TEST_CHILD"; + +fn record(value: serde_json::Value) -> std::io::Result<()> { + eprintln!("{value}"); + if let Some(path) = std::env::var_os("FRESHELL_TEST_RESOURCE_RECEIPT") { + let mut file = std::fs::OpenOptions::new() + .create(true) + .append(true) + .open(path)?; + file.write_all(format!("{value}\n").as_bytes())?; + } + Ok(()) +} + +/// Re-execute the exact existing contract with a writable synthetic home. +/// The parent keeps its actual home and holds the TempDir until the child joins. +pub(crate) fn run_with_opencode_home(test: &str) -> bool { + if let Ok(child_test) = std::env::var(HOME_CHILD) { + assert_eq!(child_test, test, "isolated child ran the wrong contract"); + return false; + } + let home = tempfile::tempdir().expect("create isolated OpenCode test home"); + let original_home = std::env::var_os("HOME"); + let original_profile = std::env::var_os("USERPROFILE"); + let child = Command::new(std::env::current_exe().expect("resolve freshagent test binary")) + .args(["--exact", test, "--nocapture"]) + .env(HOME_CHILD, test) + .env("HOME", home.path()) + .env("USERPROFILE", home.path()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("spawn exact isolated OpenCode contract"); + let pid = child.id(); + // A diagnostic write failure must still join the already spawned child. + let started = record( + serde_json::json!({"severity":"info", "event":"opencode_home_test_started", "test":test, "pid":pid, "home":home.path()}), + ); + let output = child + .wait_with_output() + .expect("join isolated OpenCode contract"); + let stdout = String::from_utf8_lossy(&output.stdout); + let exact_completed = stdout + .lines() + .any(|line| line == format!("test {test} ... ok")) + && stdout.lines().any(|line| line == "running 1 test") + && stdout.contains("test result: ok. 1 passed; 0 failed; 0 ignored;"); + let joined = record( + serde_json::json!({"severity":if output.status.success() && exact_completed {"info"} else {"error"}, "event":"opencode_home_test_joined", "test":test, "pid":pid, "code":output.status.code(), "exactCompleted":exact_completed}), + ); + let plugin_installed = + freshell_platform::opencode_plugin::rebind_plugin_path(home.path()).is_file(); + let selector_installed = + freshell_platform::opencode_plugin::tui_config_path(home.path()).is_file(); + let home_path = home.path().to_owned(); + let removed = home.close(); + let settled = record( + serde_json::json!({"severity":if removed.is_ok() {"info"} else {"error"}, "event":"opencode_home_test_settled", "test":test, "pid":pid, "home":home_path}), + ); + // Propagate every failure after the child's join and owned-home cleanup. + started.expect("record started test resource"); + joined.expect("record joined test resource"); + removed.expect("remove joined child's owned home"); + settled.expect("record settled test resource"); + assert!(!home_path.exists(), "joined child's home must be removed"); + // Prefix child output on failure so its harness lines cannot masquerade as + // completions of the outer 1296-test workload. + assert!( + output.status.success() && exact_completed, + "isolated contract failed: {test}\n{}", + stdout + .lines() + .map(|line| format!("[home-child stdout] {line}")) + .chain( + String::from_utf8_lossy(&output.stderr) + .lines() + .map(|line| format!("[home-child stderr] {line}")) + ) + .collect::>() + .join("\n"), + ); + assert!( + plugin_installed, + "the contract must install the real TUI plugin" + ); + assert!( + selector_installed, + "the contract must materialize its real TUI selector" + ); + assert_eq!(std::env::var_os("HOME"), original_home); + assert_eq!(std::env::var_os("USERPROFILE"), original_profile); + true +} From 076d88e903e1a88ad59404c2df0384ff5a1274f2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 00:13:38 -0700 Subject: [PATCH 213/493] fix(testing): retain evidence for unfinished sandbox creation Keep abrupt owner settlement unresolved until a published or discovered full container ID confirms creation. Cover publication after cleanup queries with the actual guest watchdog, and sequence sandbox fixture files within the shared PID budget. --- scripts/sandbox-test.sh | 7 ++ test/unit/tooling/sandbox-test.config.ts | 3 + test/unit/tooling/sandbox-test.sandbox.ts | 118 +++++++++++++++++++++- 3 files changed, 124 insertions(+), 4 deletions(-) diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 9cffdc0e7..f293ea6e7 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -216,6 +216,13 @@ cleanup_owned_container() { local remaining_by_name remaining_by_name="$(docker ps -aq --no-trunc --filter "name=^/${OWNED_CONTAINER_NAME}$")" || cleanup_status=1 [ -z "${remaining_by_name}" ] || cleanup_status=1 + if [ "${SETTLE_ONLY}" -eq 1 ] && [ -z "${owned_cid}" ]; then + # An abruptly killed client cannot join an already accepted daemon + # creation. Empty lookups may precede publication: retain our evidence + # unless a published or discovered full ID proves creation completed. + sandbox_log error sandbox_creation_unconfirmed 1 + cleanup_status=1 + fi return "${cleanup_status}" } diff --git a/test/unit/tooling/sandbox-test.config.ts b/test/unit/tooling/sandbox-test.config.ts index 8bd95e548..eab761465 100644 --- a/test/unit/tooling/sandbox-test.config.ts +++ b/test/unit/tooling/sandbox-test.config.ts @@ -8,5 +8,8 @@ export default defineConfig({ test: { ...baseConfig.test, include: ['test/unit/tooling/sandbox-test.sandbox.ts', 'test/unit/tooling/run-rust-tests.test.ts', 'test/unit/tooling/rust-launcher-resources.sandbox.ts'], + // Process-budget probes must not compete with other files for the same + // container's PID allowance. Preserve concurrency inside each probe. + fileParallelism: false, }, }) diff --git a/test/unit/tooling/sandbox-test.sandbox.ts b/test/unit/tooling/sandbox-test.sandbox.ts index 93669329f..81d07d1b1 100644 --- a/test/unit/tooling/sandbox-test.sandbox.ts +++ b/test/unit/tooling/sandbox-test.sandbox.ts @@ -1,6 +1,6 @@ // @vitest-environment node import { spawnSync } from 'node:child_process' -import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs' import os from 'node:os' import path from 'node:path' import { describe, expect, it, vi } from 'vitest' @@ -24,9 +24,52 @@ function fixture(mode: string) { const receipt = path.join(root, 'receipt.jsonl') const state = path.join(root, 'owned-object.json') const foreign = path.join(root, 'foreign-object') + const ownedProcesses: Array<{ role: string; pid: number; identity: string }> = [] + const joinedProcesses = new Set() + const capturePendingProcesses = () => { + if (mode !== 'watchdog-pending-creation') return + const files = [{ role: 'creator', file: path.join(root, 'creator.json') }, ...readdirSync(path.join(root, 'control')) + .map(name => ({ role: 'guard', file: path.join(root, 'control', name, 'owner-guard.json') }))] + for (const { role, file } of files) { + if (!existsSync(file) || ownedProcesses.some(owned => owned.role === role)) continue + let owned: { role: string; pid: number; identity: string } + try { owned = { role, ...JSON.parse(readFileSync(file, 'utf8')) } } catch (cause) { + if ((cause as NodeJS.ErrnoException).code === 'ENOENT') continue + throw cause + } + ownedProcesses.push(owned) + console.info(JSON.stringify({ severity: 'info', event: 'sandbox_pending_fixture_process_captured', root, ...owned })) + } + } writeFileSync(foreign, 'untouched') + // A daemon request can outlive the attached Docker client. This creator + // reparents inside the guest, then publishes only after both absence queries. + writeFileSync(path.join(root, 'pending-creator.py'), String.raw` +import json, os, sys, time +from pathlib import Path +root, name = Path(sys.argv[1]), sys.argv[2] +if os.fork(): os._exit(0) +os.setsid() +if os.fork(): os._exit(0) +with open(os.devnull, 'rb+') as null: + for fd in (0, 1, 2): os.dup2(null.fileno(), fd) +fields = Path(f'/proc/{os.getpid()}/stat').read_text().rpartition(') ')[2].split() +(root / 'creator.tmp').write_text(json.dumps({'pid': os.getpid(), 'identity': fields[19]})) +(root / 'creator.tmp').replace(root / 'creator.json') +deadline = time.monotonic() + 8 +while not (root / 'absence-queries-complete').exists(): + if time.monotonic() >= deadline: os._exit(1) + time.sleep(.025) +time.sleep(.1) +(root / 'owned-object.json').write_text(json.dumps({'cid': 'a' * 64, 'name': name})) +with (root / 'receipt.jsonl').open('a') as log: + log.write(json.dumps({'kind': 'pending-daemon-created'}) + '\n') +(root / 'creator.done').write_text('0') +`) writeFileSync(path.join(root, 'bin/docker.mjs'), String.raw` import fs from 'node:fs'; + import { spawnSync } from 'node:child_process'; + import path from 'node:path'; const args = process.argv.slice(2), mode = process.env.FIXTURE_MODE; const state = process.env.FIXTURE_STATE; const cid = 'a'.repeat(64); @@ -37,7 +80,11 @@ function fixture(mode: string) { if (args[0] === 'run') { const name = args[args.indexOf('--name')+1], cidfile = args[args.indexOf('--cidfile')+1]; record({kind:'run', name, cidfile}); - if (mode !== 'no-object' && mode !== 'late-creation') fs.writeFileSync(state, JSON.stringify({cid, name})); + if (mode === 'watchdog-pending-creation') { + const root = path.dirname(state); + const starter = spawnSync('python3', [root+'/pending-creator.py', root, name], {stdio:'ignore', timeout:5000}); + if (starter.status !== 0) process.exit(75); + } else if (mode !== 'no-object' && mode !== 'late-creation') fs.writeFileSync(state, JSON.stringify({cid, name})); if (mode === 'published' || mode === 'published-success' || mode === 'removed-race' || mode === 'watchdog-published') fs.writeFileSync(cidfile, cid); if (mode === 'removed-race') fs.unlinkSync(state); if (mode.startsWith('watchdog-')) { @@ -54,6 +101,11 @@ function fixture(mode: string) { if (mode === 'query-failed' || mode === 'watchdog-query-failed') process.exit(71); const current = object(), filter = args[args.indexOf('--filter')+1]; if (current && (filter === 'id='+cid || filter === 'name=^/'+current.name+'$')) console.log(cid); + if (mode === 'watchdog-pending-creation' && !current && filter.startsWith('name=')) { + const queries = fs.readFileSync(process.env.FIXTURE_RECEIPT, 'utf8').trim().split('\n').map(line => JSON.parse(line)) + .filter(event => event.kind === 'docker' && event.args[0] === 'ps'); + if (queries.length >= 2) fs.writeFileSync(path.dirname(state)+'/absence-queries-complete', '0'); + } } else if (args[0] === 'container' && args[1] === 'inspect') { process.exit(object() && args[2] === cid ? 0 : 1); } else if (['kill', 'stop', 'wait', 'rm'].includes(args[0])) { @@ -70,6 +122,7 @@ function fixture(mode: string) { return { root, state, foreign, async runTimeout() { + const monitor = setInterval(capturePendingProcesses, 10) const watchdogLog = vi.spyOn(console, 'error').mockImplementation(() => {}) let status: number let watchdogEvents: unknown[][] @@ -81,7 +134,11 @@ function fixture(mode: string) { FRESHELL_TEST_COORDINATOR_PHASE_TIMEOUT_MS: '600', FRESHELL_TEST_COORDINATOR_PHASE_WATCH_POLL_MS: '25', }, false) watchdogEvents = [...watchdogLog.mock.calls] - } finally { watchdogLog.mockRestore() } + } finally { + clearInterval(monitor) + capturePendingProcesses() + watchdogLog.mockRestore() + } const events = readFileSync(receipt, 'utf8').trim().split('\n').map(line => JSON.parse(line)) const run = events.find(event => event.kind === 'run') return { status, events, watchdogEvents, control: path.dirname(run.cidfile), name: run.name } @@ -97,7 +154,33 @@ function fixture(mode: string) { const run = events.find(event => event.kind === 'run') return { ...result, events, control: path.dirname(run.cidfile), name: run.name } }, - cleanup() { rmSync(root, { recursive: true, force: true }) }, + cleanup() { + rmSync(root, { recursive: true, force: true }) + if (mode === 'watchdog-pending-creation') { + console.info(JSON.stringify({ severity: 'info', event: 'sandbox_pending_fixture_root_removed', root, present: existsSync(root) })) + } + }, + async joinPendingCreators() { + // Join by captured birth identity before inspecting or removing fixture + // state, including RED. No guest PID is ever used as host authority. + capturePendingProcesses() + for (const owned of ownedProcesses) { + if (joinedProcesses.has(owned.pid)) continue + const identity = () => { + try { + const fields = readFileSync(`/proc/${owned.pid}/stat`, 'utf8').split(') ').at(-1)!.split(' ') + return fields[0] === 'Z' ? undefined : fields[19] + } catch (cause) { + if ((cause as NodeJS.ErrnoException).code === 'ENOENT') return undefined + throw cause + } + } + await expect.poll(identity, { timeout: 10_000 }).not.toBe(owned.identity) + joinedProcesses.add(owned.pid) + console.info(JSON.stringify({ severity: 'info', event: 'sandbox_pending_fixture_process_joined', root, ...owned, identityNow: identity() ?? null })) + } + expect(ownedProcesses.map(owned => owned.role).sort()).toEqual(['creator', 'guard']) + }, } } @@ -178,6 +261,33 @@ describe('sandbox wrapper owned launch settlement', () => { expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') } finally { f.cleanup() } }) + it('retains unknown creation evidence when a pending daemon publishes after watchdog cleanup queries', async () => { + const f = fixture('watchdog-pending-creation') + try { + const result = await f.runTimeout() + expect(result.status).toBe(125) + expect(JSON.parse(String(result.watchdogEvents[0][0]))).toMatchObject({ event: 'phase_watchdog_settled', reason: 'phase_timeout', timeoutMs: 600 }) + await expect.poll(() => existsSync(path.join(f.root, 'creator.done')), { timeout: 10_000 }).toBe(true) + await f.joinPendingCreators() + const events = readFileSync(path.join(f.root, 'receipt.jsonl'), 'utf8').trim().split('\n').map(line => JSON.parse(line)) + const published = events.findIndex(event => event.kind === 'pending-daemon-created') + const queries = events.flatMap((event, index) => event.args?.[0] === 'ps' ? [index] : []) + console.info(JSON.stringify({ severity: 'info', event: 'sandbox_pending_creation_proof', root: f.root, + watchdog: JSON.parse(String(result.watchdogEvents[0][0])), queryIndices: queries, publicationIndex: published, + objectPresent: existsSync(f.state), controlPresent: existsSync(result.control) })) + expect(queries.length).toBe(2) + expect(queries.every(index => index < published)).toBe(true) + expect(existsSync(f.state)).toBe(true) + expect(existsSync(result.control)).toBe(true) + expect(JSON.parse(readFileSync(path.join(result.control, 'owner-guard.done'), 'utf8'))).toMatchObject({ event: 'sandbox_owned_container_unsettled', code: 1 }) + expect(readFileSync(path.join(result.control, 'owner-guard.log'), 'utf8')).toContain('sandbox_creation_unconfirmed') + expect(readFileSync(path.join(result.control, 'container.name'), 'utf8').trim()).toBe(result.name) + expect(readFileSync(f.foreign, 'utf8')).toBe('untouched') + } finally { + await f.joinPendingCreators() + f.cleanup() + } + }) it('removes an owned daemon object after failed launch without CID publication', () => { const f = fixture('failed-launch') try { From 01da331eb140c65a967a3e4b38ee4959fc17b7b6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 01:57:09 -0700 Subject: [PATCH 214/493] fix(test): prepare frozen sidecar before strict runtime suite --- scripts/testing/run-source-runtime-tests.ts | 11 +++- .../tooling/runtime-manager-launch.test.ts | 65 ++++++++++++++++--- 2 files changed, 64 insertions(+), 12 deletions(-) diff --git a/scripts/testing/run-source-runtime-tests.ts b/scripts/testing/run-source-runtime-tests.ts index 488696172..001c7b13d 100644 --- a/scripts/testing/run-source-runtime-tests.ts +++ b/scripts/testing/run-source-runtime-tests.ts @@ -187,9 +187,16 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise { [fixture.pnpmCjs, 'run', 'build:tools'], ['build', '--release', '-p', 'freshell-server', '--locked'], ]) - expect(specs).toHaveLength(5) - expect(specs[4].command).toBe(path.resolve('scripts/sandbox-test.sh')) - expect((specs[4].args as string[])[0]).toBe('--runtime-suite') + expect(specs).toHaveLength(6) + expect(specs[5].command).toBe(path.resolve('scripts/sandbox-test.sh')) + expect((specs[5].args as string[])[0]).toBe('--runtime-suite') expect(execFileSync).toHaveBeenCalledExactlyOnceWith('git', ['rev-parse', 'HEAD'], { cwd: path.resolve('.'), encoding: 'utf8', }) @@ -157,17 +161,17 @@ describe('runtime test script manager launches', () => { it.each([17, 125, 143])('returns sandbox failure %s without a host runtime fallback', async code => { vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') vi.stubEnv('npm_execpath', fixture.pnpmCjs) - spawn.mockImplementation((command: string) => { + spawn.mockImplementation((command: string, args: string[]) => { const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) queueMicrotask(() => { - const result = command.endsWith('sandbox-test.sh') ? code : 0 + const result = command.endsWith('sandbox-test.sh') && args[0] === '--runtime-suite' ? code : 0 child.emit('exit', result, null) child.emit('close', result, null) }) return child }) expect(await runSourceRuntimeTests([])).toBe(code) - expect(spawn).toHaveBeenCalledTimes(5) + expect(spawn).toHaveBeenCalledTimes(6) }) it.each([0, 31])('executes the sandbox shell boundary with exact argv and cleanup on suite exit %s', async code => { @@ -181,17 +185,16 @@ describe('runtime test script manager launches', () => { commit:process.env.FRESHELL_BUILD_COMMIT, tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); process.exit(process.argv[2]==='node' ? Number(process.env.SUITE_EXIT) : 0);`) - const quote = (word: string) => `'${word.replaceAll("'", "'\\''")}'` for (const name of ['cargo', 'node']) { - fs.writeFileSync(path.join(fixture.binDir, name), `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(helper)} ${name} "$@"\n`, { mode: 0o755 }) + fs.writeFileSync(path.join(fixture.binDir, name), `#!/bin/sh\nexec ${shellQuote(process.execPath)} ${shellQuote(helper)} ${name} "$@"\n`, { mode: 0o755 }) } let shellOutput = '' spawn.mockImplementation((command: string, args: string[]) => { const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) queueMicrotask(() => { let status = 0 - if (command.endsWith('sandbox-test.sh')) { - const payload = args[1].replace('/freshell-test-artifacts/', `${quote(fixture.root)}/`) + if (command.endsWith('sandbox-test.sh') && args[0] === '--runtime-suite') { + const payload = args[1].replace('/freshell-test-artifacts/', `${shellQuote(fixture.root)}/`) const result = spawnSync('bash', ['-c', payload], { cwd: fixture.root, env: { ...process.env, PATH: `${fixture.binDir}:${process.env.PATH}`, RECEIPT: receipt, SUITE_EXIT: String(code) }, @@ -223,6 +226,48 @@ describe('runtime test script manager launches', () => { expect(shellOutput).toContain('source_runtime_temp_settled') }) + it.each([0, 33])('prepares sandbox prerequisites before the strict suite and propagates preparation exit %s', async preparationExit => { + vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') + vi.stubEnv('npm_execpath', fixture.pnpmCjs) + const readyPath = path.join(fixture.root, 'prerequisites-ready') + const receiptPath = path.join(fixture.root, 'prerequisite-events.jsonl') + const helperPath = path.join(fixture.root, 'prerequisite-child.mjs') + fs.writeFileSync(helperPath, `import fs from 'node:fs'; + const preparing = process.argv.includes('scripts/ensure-claude-sidecar.ts'); + fs.appendFileSync(process.env.RECEIPT, JSON.stringify({preparing, ready:fs.existsSync(process.env.READY)})+'\\n'); + if (preparing) { + if (Number(process.env.PREPARATION_EXIT)) process.exit(Number(process.env.PREPARATION_EXIT)); + fs.writeFileSync(process.env.READY, 'prepared'); + } else if (!fs.existsSync(process.env.READY)) process.exit(41);`) + fs.writeFileSync(path.join(fixture.binDir, 'cargo'), '#!/bin/sh\nexit 0\n', { mode: 0o755 }) + fs.writeFileSync(path.join(fixture.binDir, 'node'), `#!/bin/sh\nexec ${shellQuote(process.execPath)} ${shellQuote(helperPath)} "$@"\n`, { mode: 0o755 }) + let shellOutput = '' + spawn.mockImplementation((command: string, args: string[]) => { + const child = Object.assign(new EventEmitter(), { exitCode: null, killed: false }) + queueMicrotask(() => { + let status = 0 + if (command.endsWith('sandbox-test.sh')) { + const payload = args.at(-1)!.replace('/freshell-test-artifacts/', `${shellQuote(fixture.root)}/`) + const result = spawnSync('bash', ['-c', payload], { + cwd: fixture.root, + env: { ...process.env, PATH: `${fixture.binDir}:${process.env.PATH}`, RECEIPT: receiptPath, READY: readyPath, PREPARATION_EXIT: String(preparationExit) }, + encoding: 'utf8', timeout: 10_000, + }) + shellOutput += result.stdout + result.stderr + status = result.status ?? 1 + } + child.emit('close', status, null) + }) + return child + }) + + expect(await runSourceRuntimeTests([]), shellOutput).toBe(preparationExit) + const events = fs.readFileSync(receiptPath, 'utf8').trim().split('\n').map(line => JSON.parse(line)) + expect(events).toEqual(preparationExit === 0 + ? [{ preparing: true, ready: false }, { preparing: false, ready: true }] + : [{ preparing: true, ready: false }]) + }) + it('forwards SIGTERM from a simulated listener and removes its handlers after child close', async () => { vi.stubEnv('FRESHELL_RUST_TEST_SANDBOX', '1') vi.stubEnv('npm_execpath', fixture.pnpmCjs) From 85729d54270154f3cf9ff7b1970a1e64370b0696 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 03:10:16 -0700 Subject: [PATCH 215/493] test: stage complete Claude sidecar interrupt fixture --- .../claude_sidecar_interrupt_dispatch.rs | 54 +++++++++++++++---- 1 file changed, 43 insertions(+), 11 deletions(-) diff --git a/crates/freshell-freshagent/tests/claude_sidecar_interrupt_dispatch.rs b/crates/freshell-freshagent/tests/claude_sidecar_interrupt_dispatch.rs index 8dd081354..087501244 100644 --- a/crates/freshell-freshagent/tests/claude_sidecar_interrupt_dispatch.rs +++ b/crates/freshell-freshagent/tests/claude_sidecar_interrupt_dispatch.rs @@ -26,8 +26,9 @@ //! fake-sidecar tests already impose. use std::io::{BufRead, BufReader, Write}; -use std::process::{Command, Stdio}; +use std::process::{Child, Command, Stdio}; use std::sync::mpsc; +use std::thread::JoinHandle; use std::time::Duration; /// Stub SDK entry: supplies `query` for the sidecar's SDK import @@ -53,11 +54,39 @@ fn real_sidecar_source(module: &str) -> String { .unwrap_or_else(|e| panic!("read the real crates/freshell-claude-sidecar/{module}: {e}")) } +/// Keep the child and its pipe readers owned until every assertion has settled, +/// including an early startup failure. Rust's Child does not reap itself on drop. +struct SidecarProcess { + child: Child, + readers: Vec>, +} + +impl Drop for SidecarProcess { + fn drop(&mut self) { + // Killing an already-exited child can fail; wait still reaps that child. + let _ = self.child.kill(); + let waited = self.child.wait(); + let readers = std::mem::take(&mut self.readers); + let joined = readers + .into_iter() + .map(JoinHandle::join) + .collect::>(); + if !std::thread::panicking() { + waited.expect("reap owned sidecar"); + for result in joined { + result.expect("join owned sidecar pipe reader"); + } + } + } +} + #[test] fn real_sidecar_dispatches_interrupt_frames_to_handle_interrupt() { let dir = tempfile::tempdir().expect("create temp dir"); for module in [ "index.mjs", + "exec-profile.mjs", + "hosted-mcp.mjs", "monotonic-clock.mjs", "permission-channel.mjs", "session-settings.mjs", @@ -73,35 +102,39 @@ fn real_sidecar_dispatches_interrupt_frames_to_handle_interrupt() { std::fs::write(sdk_dir.join("package.json"), STUB_SDK_PACKAGE_JSON).expect("stub package"); std::fs::write(sdk_dir.join("index.mjs"), STUB_SDK_INDEX).expect("stub entry"); - let mut child = Command::new("node") + let child = Command::new("node") .arg(dir.path().join("index.mjs")) .stdin(Stdio::piped()) .stdout(Stdio::piped()) .stderr(Stdio::piped()) .spawn() .expect("spawn node with the real sidecar source (node is required by this crate's tests)"); + let mut sidecar = SidecarProcess { + child, + readers: Vec::new(), + }; // Reader threads: forward each line with a tag so the main thread can poll // with timeouts (the pre-fix failure mode is "no stdout frame ever arrives", // which must fail the test, not hang it). let (tx, rx) = mpsc::channel::<(&'static str, String)>(); - let stdout = child.stdout.take().expect("piped stdout"); - let stderr = child.stderr.take().expect("piped stderr"); + let stdout = sidecar.child.stdout.take().expect("piped stdout"); + let stderr = sidecar.child.stderr.take().expect("piped stderr"); let tx_out = tx.clone(); - std::thread::spawn(move || { + sidecar.readers.push(std::thread::spawn(move || { for line in BufReader::new(stdout).lines().map_while(Result::ok) { if tx_out.send(("stdout", line)).is_err() { break; } } - }); - std::thread::spawn(move || { + })); + sidecar.readers.push(std::thread::spawn(move || { for line in BufReader::new(stderr).lines().map_while(Result::ok) { if tx.send(("stderr", line)).is_err() { break; } } - }); + })); // Gate on the sidecar's startup log so the interrupt frame is never written // before the dispatch loop is armed. @@ -124,7 +157,7 @@ fn real_sidecar_dispatches_interrupt_frames_to_handle_interrupt() { "sidecar never logged ready; stderr so far: {stderr_lines:?}" ); - let mut stdin = child.stdin.take().expect("piped stdin"); + let mut stdin = sidecar.child.stdin.take().expect("piped stdin"); stdin .write_all(b"{\"type\":\"interrupt\",\"sessionId\":\"nope\"}\n") .expect("write interrupt frame"); @@ -149,8 +182,7 @@ fn real_sidecar_dispatches_interrupt_frames_to_handle_interrupt() { } } - child.kill().ok(); - child.wait().ok(); + drop(sidecar); assert!( !stderr_lines From ac90b09d4ccece86e00586ce0f1b9d7fba795fdb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 04:20:32 -0700 Subject: [PATCH 216/493] test(session-host): observe private payload reads deterministically --- crates/freshell-session-host/src/main.rs | 37 +++--- .../src/provider_input_preparation.rs | 110 +++++++++++++++--- 2 files changed, 116 insertions(+), 31 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index c644a5c2b..71550e3c6 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -4278,7 +4278,9 @@ mod tests { #[cfg(unix)] #[test] fn provider_config_projection_metadata_rejects_oversize_before_read_and_publication() { - use std::io::Read; + use crate::provider_input_preparation::{ + private_read_observation::Observation, read_private, + }; let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let stage = owned.path().join("stage"); @@ -4309,28 +4311,33 @@ mod tests { ) .unwrap(); for (path, original) in &fixtures { - let mut file = std::fs::OpenOptions::new() + let reads = Observation::start(path); + assert_eq!( + read_private( + path.parent().unwrap(), + path.file_name().unwrap().to_str().unwrap(), + MAX_PROJECTION_METADATA_BYTES, + ) + .unwrap(), + *original + ); + assert!( + reads.calls() > 0, + "fixture must observe actual payload reads" + ); + drop(reads); + let file = std::fs::OpenOptions::new() .read(true) .write(true) .open(path) .unwrap(); file.set_len(64 * 1024 * 1024 + 1).unwrap(); - let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); - file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) - .unwrap(); - file.read_exact(&mut [0]).unwrap(); - assert_ne!( - file.metadata().unwrap().accessed().unwrap(), - old_access, - "fixture filesystem must observe payload reads" - ); - file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) - .unwrap(); drop(file); + let reads = Observation::start(path); assert!(apply_staged_provider_config(&stage, &home).is_err()); assert_eq!( - std::fs::metadata(path).unwrap().accessed().unwrap(), - old_access, + reads.calls(), + 0, "oversized ordinary metadata payload was read: {}", path.display() ); diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index e063f883a..12b02d0b2 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -30,9 +30,13 @@ pub(crate) fn read_private(root: &Path, relative: &str, limit: usize) -> Result< use std::os::unix::fs::OpenOptionsExt; options.custom_flags(libc::O_NOFOLLOW); } - let mut file = options + let file = options .open(&path) .map_err(|_| "prepared private source unreadable")?; + #[cfg(not(test))] + let mut file = file; + #[cfg(test)] + let mut file = private_read_observation::ObservedFile::new(file, &path); let opened = file .metadata() .map_err(|_| "prepared private source unavailable")?; @@ -104,6 +108,87 @@ pub(crate) fn read_private(root: &Path, relative: &str, limit: usize) -> Result< Ok(bytes) } +// Observe the actual Read boundary in tests, independently of filesystem +// access-time policy. Production continues to read the same physical File. +#[cfg(test)] +pub(crate) mod private_read_observation { + use std::{ + cell::{Cell, RefCell}, + fs::File, + io::Read, + path::{Path, PathBuf}, + rc::Rc, + }; + + struct Reads { + path: PathBuf, + calls: Cell, + } + + thread_local! { + static ACTIVE: RefCell>> = const { RefCell::new(None) }; + } + + pub(crate) struct Observation { + reads: Rc, + previous: Option>, + } + + impl Observation { + pub(crate) fn start(path: &Path) -> Self { + let reads = Rc::new(Reads { + path: path.to_owned(), + calls: Cell::new(0), + }); + let previous = ACTIVE.with(|active| active.replace(Some(reads.clone()))); + Self { reads, previous } + } + + pub(crate) fn calls(&self) -> usize { + self.reads.calls.get() + } + } + + impl Drop for Observation { + fn drop(&mut self) { + ACTIVE.with(|active| active.replace(self.previous.take())); + } + } + + pub(super) struct ObservedFile { + file: File, + path: PathBuf, + } + + impl ObservedFile { + pub(super) fn new(file: File, path: &Path) -> Self { + Self { + file, + path: path.to_owned(), + } + } + + pub(super) fn metadata(&self) -> std::io::Result { + self.file.metadata() + } + } + + impl Read for ObservedFile { + fn read(&mut self, buffer: &mut [u8]) -> std::io::Result { + ACTIVE.with(|active| { + if let Some(reads) = active + .borrow() + .as_ref() + .filter(|reads| reads.path == self.path) + { + reads.calls.set(reads.calls.get() + 1); + } + }); + self.file.read(buffer) + } + } +} + pub(crate) fn final_claude_source_request( context: Option<&ProviderLaunchContext>, stage: &Path, @@ -483,18 +568,17 @@ mod tests { #[cfg(target_os = "linux")] #[test] fn provider_secret_private_reader_rejects_oversize_before_payload_read() { - use std::io::Read; + use private_read_observation::Observation; let owned = tempfile::tempdir().unwrap(); let private = owned.path().join("metadata"); let native = owned.path().join("native-history.jsonl"); std::fs::write(&native, b"synthetic-native-store-preserved").unwrap(); - let old_access = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1); for limit in [ freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, freshell_platform::claude_inputs::MAX_NATIVE_SETTINGS_BYTES, 64 * 1024 * 1024, ] { - let mut file = std::fs::OpenOptions::new() + let file = std::fs::OpenOptions::new() .read(true) .write(true) .create(true) @@ -502,28 +586,22 @@ mod tests { .open(&private) .unwrap(); file.set_len(limit as u64).unwrap(); + let reads = Observation::start(&private); let accepted = read_private(owned.path(), "metadata", limit).unwrap(); assert_eq!( accepted.len(), limit, "inclusive physical file limit must remain supported" ); + assert!(reads.calls() > 0, "accepted payload must actually be read"); drop(accepted); + drop(reads); file.set_len(limit as u64 + 1).unwrap(); - file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) - .unwrap(); - file.read_exact(&mut [0u8; 1]).unwrap(); - assert_ne!( - std::fs::metadata(&private).unwrap().accessed().unwrap(), - old_access, - "fixture must report actual reads" - ); - file.set_times(std::fs::FileTimes::new().set_accessed(old_access)) - .unwrap(); + let reads = Observation::start(&private); assert!(read_private(owned.path(), "metadata", limit).is_err()); assert_eq!( - std::fs::metadata(&private).unwrap().accessed().unwrap(), - old_access, + reads.calls(), + 0, "host must reject oversized private metadata before reading its payload" ); assert_eq!( From 05fc57b8b7f5b0fdda22dfc63945e71d239a0513 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 06:50:00 -0700 Subject: [PATCH 217/493] fix sandbox Rust temporary path allocation --- .../claude_source_observation_tests.rs | 2 +- docs/development/test-sandbox.md | 10 +-- scripts/testing/run-rust-tests.ts | 19 ++++-- test/unit/tooling/run-rust-tests-fixture.ts | 11 +++- test/unit/tooling/run-rust-tests.test.ts | 4 +- .../rust-launcher-resources.sandbox.ts | 63 ++++++++++++++++++- 6 files changed, 93 insertions(+), 16 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 60a8e5058..3d5ffc42a 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -879,7 +879,7 @@ async fn qualify_transport(mode: &'static str) { assert_eq!( result.is_ok(), mode == "success" || matches!(mode, "retained-success" | "retained-terminal-success"), - "{mode}: unavailable, contradictory or unjoined evidence cannot be accepted" + "{mode}: unavailable, contradictory or unjoined evidence cannot be accepted: {result:?}" ); if let Ok(reply) = result { let raw = freshell_runtime_protocol::bounded_json_bytes( diff --git a/docs/development/test-sandbox.md b/docs/development/test-sandbox.md index 8d0b00ea2..b451b008b 100644 --- a/docs/development/test-sandbox.md +++ b/docs/development/test-sandbox.md @@ -53,13 +53,15 @@ FRESHELL_RUST_TEST_SANDBOX=1 pnpm run test:rust -- --exact Only the exact value `1` enables routing; other test backend settings do not. The Rust launcher runs its unchanged locked server build and workspace tests inside the strict sandbox, selects the container's server binary, and gives -both temporary-directory variables a newly owned executable directory under -the sandbox Cargo target volume's `/freshell-test-artifacts` alias, outside -the checkout's Node package lookup scope. Cargo uses three parallel build jobs; +ordinary temporary files a short owned directory under `/freshell-test-artifacts` +so nested Unix socket paths fit the operating system's limit and temporary +executables can run. Explicit executable fixtures use a separate +owned directory under the sandbox Cargo target volume's `/freshell-test-artifacts` +alias, outside the checkout's Node package lookup scope. Cargo uses three parallel build jobs; libtest runs four cases at once and default Tokio runtimes use two workers within the existing container resource limits. Explicit test worker counts and the complete test workload remain unchanged. The launcher logs its owned -temporary directory and removes it on exit or before awaiting an interrupted +temporary directories and removes both on exit or before awaiting an interrupted child that might ignore termination. Sandbox, dependency, and inner-runner failures return failure without host Cargo fallback. The same exact opt-in routes the source-runtime process suite through the strict diff --git a/scripts/testing/run-rust-tests.ts b/scripts/testing/run-rust-tests.ts index 67f4239d1..3064fd983 100644 --- a/scripts/testing/run-rust-tests.ts +++ b/scripts/testing/run-rust-tests.ts @@ -78,16 +78,21 @@ function shellQuote(value: string): string { } export function sandboxCommand(argv: string[]): string { - // Fixed wrapper text owns only its newly allocated target subdirectory. + // Fixed wrapper text owns only its two newly allocated temporary directories. // Arguments remain separately quoted words; none becomes shell code. return `set -e owned_tmp="$(mktemp -d /freshell-test-artifacts/.freshell-rust-tests-XXXXXXXX)" -printf '{"severity":"info","event":"rust_test_temp_created","directory":"%s"}\\n' "$owned_tmp" +# Ordinary Rust fixtures append UUID-bearing Unix socket paths. Keep their +# actual root short; executable fixtures retain the separate artifact boundary. +owned_runtime_tmp="$owned_tmp" runner_pid= +remove_owned_temps() { + rm -rf -- "$owned_tmp" "$owned_runtime_tmp" +} cleanup() { status=$? trap - EXIT - rm -rf -- "$owned_tmp" || { [ "$status" -ne 0 ] || status=1; } + remove_owned_temps || { [ "$status" -ne 0 ] || status=1; } exit "$status" } interrupt() { @@ -96,7 +101,7 @@ interrupt() { kill -s "$1" "$runner_pid" 2>/dev/null || true # A child may ignore termination. Remove our artifacts before waiting; # the sandbox owner separately settles all remaining container work. - rm -rf -- "$owned_tmp" + remove_owned_temps wait "$runner_pid" || true fi exit "$2" @@ -105,9 +110,13 @@ trap cleanup EXIT trap 'interrupt INT 130' INT trap 'interrupt TERM 143' TERM trap 'interrupt HUP 129' HUP +# Three random characters leave room for tempfile/runtime/incarnation UUIDs +# under Linux sun_path, while this existing volume permits fixture execution. +owned_runtime_tmp="$(mktemp -d /freshell-test-artifacts/XXX)" +printf '{"severity":"info","event":"rust_test_temp_created","directory":"%s","runtimeDirectory":"%s"}\\n' "$owned_tmp" "$owned_runtime_tmp" # Preserve every test while bounding independent libtest/Tokio runtimes within # the sandbox's existing PID budget. Runtime construction precedes fixture locks. -export TMPDIR="$owned_tmp" FRESHELL_TEST_EXECUTABLE_TMP_DIR="$owned_tmp" FRESHELL_RUST_TEST_SANDBOX=0 CARGO_BUILD_JOBS=${SANDBOX_CARGO_BUILD_JOBS} RUST_TEST_THREADS=${SANDBOX_RUST_TEST_THREADS} TOKIO_WORKER_THREADS=${SANDBOX_TOKIO_WORKER_THREADS} +export TMPDIR="$owned_runtime_tmp" FRESHELL_TEST_EXECUTABLE_TMP_DIR="$owned_tmp" FRESHELL_RUST_TEST_SANDBOX=0 CARGO_BUILD_JOBS=${SANDBOX_CARGO_BUILD_JOBS} RUST_TEST_THREADS=${SANDBOX_RUST_TEST_THREADS} TOKIO_WORKER_THREADS=${SANDBOX_TOKIO_WORKER_THREADS} node --import tsx /workspace/scripts/testing/run-rust-tests.ts ${argv.map(shellQuote).join(' ')} & runner_pid=$! wait "$runner_pid"` diff --git a/test/unit/tooling/run-rust-tests-fixture.ts b/test/unit/tooling/run-rust-tests-fixture.ts index 6dc0adf56..bbf726fe8 100644 --- a/test/unit/tooling/run-rust-tests-fixture.ts +++ b/test/unit/tooling/run-rust-tests-fixture.ts @@ -15,6 +15,9 @@ export function fixture() { mkdirSync(path.join(root, 'target')) mkdirSync(path.join(root, 'bin')) const artifacts = mkdtempSync(path.join(temporaryRoot, 'rust-artifacts-')) + const runtimeTemplate = process.env.FRESHELL_SANDBOX_MODE === 'runtime-suite' + ? '/freshell-test-artifacts/XXX' + : path.join(os.tmpdir(), 'XXX') copyFileSync(launcher, path.join(root, 'scripts/testing/run-rust-tests.ts')) writeFileSync(path.join(root, 'package.json'), '{"type":"module"}') symlinkSync(path.resolve('node_modules'), path.join(root, 'node_modules'), 'dir') @@ -35,7 +38,7 @@ export function fixture() { } if (args[0] === 'test' && process.env.COMMONJS_PROBE) { const cp = await import('node:child_process'); - const file = process.env.TMPDIR+'/fixture.js'; + const file = process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR+'/fixture.js'; fs.writeFileSync(file, "const fs = require('node:fs'); fs.writeFileSync(process.env.COMMONJS_PROBE, 'executed');"); const result = cp.spawnSync(process.execPath, [file], {env:process.env, stdio:'inherit'}); process.exit(result.status ?? 1); @@ -45,7 +48,8 @@ export function fixture() { tmp:process.env.TMPDIR, executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})+'\\n'); if (args[0] === 'test' && process.env.HOLD_TEST) { process.on('SIGTERM', () => {}); - fs.writeFileSync(process.env.HOLD_TEST, JSON.stringify({pid:process.pid, tmp:process.env.TMPDIR})); + fs.writeFileSync(process.env.HOLD_TEST, JSON.stringify({pid:process.pid, tmp:process.env.TMPDIR, + executableTmp:process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR})); setInterval(() => {}, 10000); await new Promise(() => {}); } @@ -58,7 +62,8 @@ export function fixture() { fs.appendFileSync(process.env.RECEIPT, JSON.stringify({kind:'sandbox', args})+'\\n'); if (process.env.SANDBOX_EXIT) process.exit(Number(process.env.SANDBOX_EXIT)); const command = args[1] - .replaceAll('/freshell-test-artifacts', ${JSON.stringify(artifacts)}) + .replace('mktemp -d /freshell-test-artifacts/.freshell-rust-tests-', 'mktemp -d ' + ${JSON.stringify(artifacts)} + '/.freshell-rust-tests-') + .replace('mktemp -d /freshell-test-artifacts/XXX', 'mktemp -d ' + ${JSON.stringify(runtimeTemplate)}) .replace('mktemp -d /workspace/target/', 'mktemp -d ' + ${JSON.stringify(root)} + '/target/') .replace('node --import tsx /workspace/scripts/testing/run-rust-tests.ts', 'node --import tsx ' + ${JSON.stringify(root)} + '/scripts/testing/run-rust-tests.ts'); const result = cp.spawnSync('bash', ['-c', command], {env:process.env, stdio:'inherit'}); diff --git a/test/unit/tooling/run-rust-tests.test.ts b/test/unit/tooling/run-rust-tests.test.ts index bfa8cada1..3e97580f0 100644 --- a/test/unit/tooling/run-rust-tests.test.ts +++ b/test/unit/tooling/run-rust-tests.test.ts @@ -57,9 +57,8 @@ describe('Rust test launcher children', () => { expect(result.events[1].jobs).toBe('3') expect(result.events[2].args).toEqual(['test', '--workspace', '--locked', ...args.slice(1)]) expect(result.events[2].binary).toBe(path.join(f.root, 'target/debug/freshell-server')) - expect(result.events[2].tmp).toBe(result.events[2].executableTmp) - expect(result.events[2].tmp).toMatch(new RegExp(`^${f.artifacts}/\\.freshell-rust-tests-[^/]+$`)) expect(existsSync(result.events[2].tmp)).toBe(false) + expect(existsSync(result.events[2].executableTmp)).toBe(false) } finally { f.cleanup() } }) @@ -82,6 +81,7 @@ describe('Rust test launcher children', () => { expect(readFileSync(sentinel, 'utf8')).toBe('keep') expect(result.events[2].tmp).toBeTruthy() expect(existsSync(result.events[2].tmp)).toBe(false) + expect(existsSync(result.events[2].executableTmp)).toBe(false) } finally { f.cleanup() } }) }) diff --git a/test/unit/tooling/rust-launcher-resources.sandbox.ts b/test/unit/tooling/rust-launcher-resources.sandbox.ts index 47276a654..829d81ef3 100644 --- a/test/unit/tooling/rust-launcher-resources.sandbox.ts +++ b/test/unit/tooling/rust-launcher-resources.sandbox.ts @@ -22,14 +22,18 @@ it('removes its temporary artifacts when an interrupted child ignores terminatio }) const joined = new Promise(resolve => child.once('close', resolve)) let ownedTmp: string | undefined + let ownedExecutableTmp: string | undefined try { await expect.poll(() => existsSync(marker), { timeout: 5_000 }).toBe(true) const running = JSON.parse(readFileSync(marker, 'utf8')) ownedTmp = running.tmp - console.info(JSON.stringify({ severity: 'info', event: 'rust_interruption_fixture_started', pid: child.pid, childPid: running.pid, temporaryDirectory: ownedTmp })) + ownedExecutableTmp = running.executableTmp + console.info(JSON.stringify({ severity: 'info', event: 'rust_interruption_fixture_started', pid: child.pid, childPid: running.pid, temporaryDirectory: ownedTmp, executableTemporaryDirectory: ownedExecutableTmp })) expect(existsSync(running.tmp)).toBe(true) + expect(existsSync(running.executableTmp)).toBe(true) child.kill('SIGTERM') await expect.poll(() => existsSync(running.tmp), { timeout: 2_000 }).toBe(false) + await expect.poll(() => existsSync(running.executableTmp), { timeout: 2_000 }).toBe(false) expect(readFileSync(sentinel, 'utf8')).toBe('keep') } finally { // This regression lives only in the disposable PID namespace. It owns @@ -42,6 +46,7 @@ it('removes its temporary artifacts when an interrupted child ignores terminatio } await joined if (ownedTmp) rmSync(ownedTmp, { recursive: true, force: true }) + if (ownedExecutableTmp) rmSync(ownedExecutableTmp, { recursive: true, force: true }) f.cleanup() } }) @@ -96,3 +101,59 @@ fn case_${index}() { expect(result.stdout).toContain('12 passed; 0 failed; 0 ignored') } finally { f.cleanup() } }) + +function assertRustProbe(sourceText: string, env: NodeJS.ProcessEnv = {}): void { + const f = fixture() + try { + const source = path.join(f.artifacts, 'probe.rs') + const binary = path.join(f.artifacts, 'probe') + writeFileSync(source, sourceText) + const build = spawnSync('rustc', [source, '-o', binary], { encoding: 'utf8', timeout: 30_000 }) + expect(build.status, build.stderr).toBe(0) + const result = f.run({ FRESHELL_RUST_TEST_SANDBOX: '1', RUNTIME_PROBE: binary, ...env }) + expect(result.status, result.stderr + result.stdout).toBe(0) + } finally { f.cleanup() } +} + +it('exchanges native-auth and incarnation socket messages', () => { + assertRustProbe(`use std::{io::{Read, Write}, os::unix::net::{UnixListener, UnixStream}, path::Path}; +fn exchange(path: &Path) { + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + let listener = UnixListener::bind(path).unwrap(); + let mut client = UnixStream::connect(path).unwrap(); + client.write_all(b"private-ipc").unwrap(); + let (mut server, _) = listener.accept().unwrap(); + let mut message = [0; 11]; + server.read_exact(&mut message).unwrap(); + assert_eq!(&message, b"private-ipc"); + std::fs::remove_file(path).unwrap(); +} +fn main() { + let root = std::env::temp_dir(); + let uuid = "00000000-0000-0000-0000-000000000000"; + exchange(&root.join(format!("freshell-inputs-{uuid}/reply.sock"))); + exchange(&root.join(format!(".tmpXXXXXX/runtime/incarnation-{uuid}/host.sock"))); +} +`) +}) + +it.each(['native', 'script'])('executes ordinary temporary %s fixtures', kind => { + assertRustProbe(`use std::{os::unix::fs::PermissionsExt, process::Command}; +fn main() { + if std::env::args().nth(1).as_deref() == Some("copied") { return; } + let root = std::env::temp_dir(); + if std::env::var("EXECUTION_KIND").unwrap() == "native" { + let executable = root.join("copied-native"); + std::fs::copy(std::env::current_exe().unwrap(), &executable).unwrap(); + assert!(Command::new(executable).arg("copied").status().unwrap().success()); + } else { + let script = root.join("fixture.sh"); + std::fs::write(&script, "#!/bin/sh\\nprintf ordinary-script").unwrap(); + std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o700)).unwrap(); + let result = Command::new(script).output().unwrap(); + assert!(result.status.success()); + assert_eq!(result.stdout, b"ordinary-script"); + } +} +`, { EXECUTION_KIND: kind }) +}) From 1d9f7c30a99f7bbef5aa4fa736820c3259678746 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 06:50:07 -0700 Subject: [PATCH 218/493] isolate OpenCode integration test homes --- crates/freshell-ws/tests/common/mod.rs | 13 +++ .../freshell-ws/tests/common/opencode_home.rs | 98 +++++++++++++++++++ .../tests/opencode_switch_rebind.rs | 13 +++ .../tests/rest_locator_identity.rs | 3 + .../tests/restore_stale_cwd_rehome.rs | 5 + 5 files changed, 132 insertions(+) create mode 100644 crates/freshell-ws/tests/common/opencode_home.rs diff --git a/crates/freshell-ws/tests/common/mod.rs b/crates/freshell-ws/tests/common/mod.rs index 6355be381..d6bd65790 100644 --- a/crates/freshell-ws/tests/common/mod.rs +++ b/crates/freshell-ws/tests/common/mod.rs @@ -17,6 +17,9 @@ use tokio_tungstenite::tungstenite::Message as WsMessage; use freshell_ws::WsState; +mod opencode_home; +pub use opencode_home::run_with_opencode_home; + pub const AUTH_TOKEN: &str = "s3cr3t-token-abcdef"; /// Frame-receive / poll budget for the WS e2e suites. DEFLAKE (the-usual @@ -1755,6 +1758,16 @@ pub async fn next_frame_of_type(ws: &mut TestWs, wanted: &str) -> serde_json::Va .expect("no ws error"); if let WsMessage::Text(text) = &msg { let value: serde_json::Value = serde_json::from_str(text).expect("json frame"); + if wanted == "terminal.created" && value["type"] == "error" { + eprintln!( + "{}", + serde_json::json!({ + "severity": "error", "event": "test_terminal_create_error", + "requestId": value["requestId"], "code": value["code"], + "message": value["message"], + }) + ); + } if value["type"] == serde_json::json!(wanted) { return value; } diff --git a/crates/freshell-ws/tests/common/opencode_home.rs b/crates/freshell-ws/tests/common/opencode_home.rs new file mode 100644 index 000000000..64f6c5b92 --- /dev/null +++ b/crates/freshell-ws/tests/common/opencode_home.rs @@ -0,0 +1,98 @@ +//! Process-local homes for tests that exercise the ordinary OpenCode TUI. + +use std::io::Write; +use std::process::{Command, Stdio}; + +const HOME_CHILD: &str = "FRESHELL_WS_OPENCODE_HOME_TEST_CHILD"; + +fn record(value: serde_json::Value) -> std::io::Result<()> { + eprintln!("{value}"); + if let Some(path) = std::env::var_os("FRESHELL_TEST_RESOURCE_RECEIPT") { + let mut file = std::fs::OpenOptions::new() + .create(true) + .append(true) + .open(path)?; + file.write_all(format!("{value}\n").as_bytes())?; + } + Ok(()) +} + +/// Re-execute the exact existing contract with a writable synthetic home. +/// The parent keeps its actual home and holds the TempDir until the child joins. +pub fn run_with_opencode_home(test: &str) -> bool { + if let Ok(child_test) = std::env::var(HOME_CHILD) { + assert_eq!(child_test, test, "isolated child ran the wrong contract"); + return false; + } + let home = tempfile::tempdir().expect("create isolated OpenCode test home"); + let original_home = std::env::var_os("HOME"); + let original_profile = std::env::var_os("USERPROFILE"); + let child = Command::new(std::env::current_exe().expect("resolve WS integration test binary")) + .args(["--exact", test, "--nocapture"]) + .env(HOME_CHILD, test) + .env("HOME", home.path()) + .env("USERPROFILE", home.path()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("spawn exact isolated OpenCode contract"); + let pid = child.id(); + // A diagnostic write failure must still join the already spawned child. + let started = record( + serde_json::json!({"severity":"info", "event":"opencode_home_test_started", "test":test, "pid":pid, "home":home.path()}), + ); + let output = child + .wait_with_output() + .expect("join isolated OpenCode contract"); + let stdout = String::from_utf8_lossy(&output.stdout); + let exact_completed = stdout + .lines() + .any(|line| line == format!("test {test} ... ok")) + && stdout.lines().any(|line| line == "running 1 test") + && stdout.contains("test result: ok. 1 passed; 0 failed; 0 ignored;"); + let plugin_installed = + freshell_platform::opencode_plugin::rebind_plugin_path(home.path()).is_file(); + let selector_installed = + freshell_platform::opencode_plugin::tui_config_path(home.path()).is_file(); + let joined = record( + serde_json::json!({"severity":if output.status.success() && exact_completed {"info"} else {"error"}, "event":"opencode_home_test_joined", "test":test, "pid":pid, "code":output.status.code(), "exactCompleted":exact_completed, "pluginInstalled":plugin_installed, "selectorInstalled":selector_installed}), + ); + let home_path = home.path().to_owned(); + let removed = home.close(); + let settled = record( + serde_json::json!({"severity":if removed.is_ok() {"info"} else {"error"}, "event":"opencode_home_test_settled", "test":test, "pid":pid, "home":home_path}), + ); + // Propagate every failure after the child's join and owned-home cleanup. + started.expect("record started test resource"); + joined.expect("record joined test resource"); + removed.expect("remove joined child's owned home"); + settled.expect("record settled test resource"); + assert!(!home_path.exists(), "joined child's home must be removed"); + // Prefix child output on failure so its harness lines cannot masquerade as + // completions of the outer integration workload. + assert!( + output.status.success() && exact_completed, + "isolated contract failed: {test}\n{}", + stdout + .lines() + .map(|line| format!("[home-child stdout] {line}")) + .chain( + String::from_utf8_lossy(&output.stderr) + .lines() + .map(|line| format!("[home-child stderr] {line}")) + ) + .collect::>() + .join("\n"), + ); + assert!( + plugin_installed, + "the contract must install the real TUI plugin" + ); + assert!( + selector_installed, + "the contract must materialize its real TUI selector" + ); + assert_eq!(std::env::var_os("HOME"), original_home); + assert_eq!(std::env::var_os("USERPROFILE"), original_profile); + true +} diff --git a/crates/freshell-ws/tests/opencode_switch_rebind.rs b/crates/freshell-ws/tests/opencode_switch_rebind.rs index be1b0e6ad..e1a4947dd 100644 --- a/crates/freshell-ws/tests/opencode_switch_rebind.rs +++ b/crates/freshell-ws/tests/opencode_switch_rebind.rs @@ -554,6 +554,11 @@ async fn wait_for_is_subagent( #[cfg(unix)] #[tokio::test(flavor = "multi_thread")] async fn tui_switch_signal_reclassifies_is_subagent_in_both_directions() { + if common::run_with_opencode_home( + "tui_switch_signal_reclassifies_is_subagent_in_both_directions", + ) { + return; + } const ROOT: &str = "ses_switchroot"; const CHILD: &str = "ses_switchchild"; @@ -663,6 +668,11 @@ fn opencode_session_ref(session_id: &str) -> SessionNameRef { #[cfg(unix)] #[tokio::test(flavor = "multi_thread")] async fn cli_db_association_transfers_pending_and_switch_surfaces_own_record() { + if common::run_with_opencode_home( + "cli_db_association_transfers_pending_and_switch_surfaces_own_record", + ) { + return; + } const ASSOC: &str = "ses_assoc000000000000000000000a"; const SWITCH: &str = "ses_switch00000000000000000000b"; @@ -860,6 +870,9 @@ async fn cli_db_association_transfers_pending_and_switch_surfaces_own_record() { #[cfg(unix)] #[tokio::test(flavor = "multi_thread")] async fn tui_switch_signal_rebinds_and_restart_resumes_the_new_id() { + if common::run_with_opencode_home("tui_switch_signal_rebinds_and_restart_resumes_the_new_id") { + return; + } // ---- env setup (single test fn: this binary owns process env) ---- let capture_for = |phase: &str| { std::env::temp_dir().join(format!( diff --git a/crates/freshell-ws/tests/rest_locator_identity.rs b/crates/freshell-ws/tests/rest_locator_identity.rs index ff71bcc4a..554c0b5fc 100644 --- a/crates/freshell-ws/tests/rest_locator_identity.rs +++ b/crates/freshell-ws/tests/rest_locator_identity.rs @@ -342,6 +342,9 @@ fn cleanup(h: &Harness) { /// exactly as the plugin would, via the pub drain (no sweep-timer race). #[tokio::test(flavor = "multi_thread")] async fn rest_created_opencode_pane_binds_identity_row_and_ledger() { + if common::run_with_opencode_home("rest_created_opencode_pane_binds_identity_row_and_ledger") { + return; + } let h = spawn_merged_server(vec![common::sleeper_cli_spec("opencode")], None).await; let (tid, _pane_id) = rest_create(&h, "opencode").await; diff --git a/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs b/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs index 552d5697b..694f9ed39 100644 --- a/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs +++ b/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs @@ -154,6 +154,11 @@ async fn next_created_or_error(ws: &mut common::TestWs, request_id: &str) -> Val #[tokio::test(flavor = "multi_thread")] async fn restore_create_with_deleted_cwd_rehomes_instead_of_failing_spawn() { + if common::run_with_opencode_home( + "restore_create_with_deleted_cwd_rehomes_instead_of_failing_spawn", + ) { + return; + } // Fallback target: a validated defaultCwd in a live temp dir. let default_dir = live_temp_dir("default"); let (ws_url, registry) = spawn_server_with_default_cwd(&default_dir).await; From ce7d522992028a02c39e0b8a2d95558cf7f85479 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 07:23:27 -0700 Subject: [PATCH 219/493] fix: clear Boolean and shared test import compiler warnings --- crates/freshell-platform/src/claude_inputs/observation.rs | 2 +- crates/freshell-ws/tests/common/mod.rs | 3 +-- crates/freshell-ws/tests/opencode_switch_rebind.rs | 8 +++++--- crates/freshell-ws/tests/rest_locator_identity.rs | 4 +++- crates/freshell-ws/tests/restore_stale_cwd_rehome.rs | 2 +- 5 files changed, 11 insertions(+), 8 deletions(-) diff --git a/crates/freshell-platform/src/claude_inputs/observation.rs b/crates/freshell-platform/src/claude_inputs/observation.rs index d2a977569..7c4a10be7 100644 --- a/crates/freshell-platform/src/claude_inputs/observation.rs +++ b/crates/freshell-platform/src/claude_inputs/observation.rs @@ -323,7 +323,7 @@ impl NativeObservation { .any(|pair| pair[0] >= pair[1]) || self.local_settings.len() > 2 || local_enabled != (self.local_decision != LocalDecision::Disabled) - || local_enabled != !self.local_settings.is_empty() + || local_enabled == self.local_settings.is_empty() || (!self.local_settings.is_empty() && self.local_settings[0] != self.cwd.join(".claude/settings.local.json")) || (!local_enabled diff --git a/crates/freshell-ws/tests/common/mod.rs b/crates/freshell-ws/tests/common/mod.rs index d6bd65790..5e7441a19 100644 --- a/crates/freshell-ws/tests/common/mod.rs +++ b/crates/freshell-ws/tests/common/mod.rs @@ -17,8 +17,7 @@ use tokio_tungstenite::tungstenite::Message as WsMessage; use freshell_ws::WsState; -mod opencode_home; -pub use opencode_home::run_with_opencode_home; +pub mod opencode_home; pub const AUTH_TOKEN: &str = "s3cr3t-token-abcdef"; diff --git a/crates/freshell-ws/tests/opencode_switch_rebind.rs b/crates/freshell-ws/tests/opencode_switch_rebind.rs index e1a4947dd..85fc91ece 100644 --- a/crates/freshell-ws/tests/opencode_switch_rebind.rs +++ b/crates/freshell-ws/tests/opencode_switch_rebind.rs @@ -554,7 +554,7 @@ async fn wait_for_is_subagent( #[cfg(unix)] #[tokio::test(flavor = "multi_thread")] async fn tui_switch_signal_reclassifies_is_subagent_in_both_directions() { - if common::run_with_opencode_home( + if common::opencode_home::run_with_opencode_home( "tui_switch_signal_reclassifies_is_subagent_in_both_directions", ) { return; @@ -668,7 +668,7 @@ fn opencode_session_ref(session_id: &str) -> SessionNameRef { #[cfg(unix)] #[tokio::test(flavor = "multi_thread")] async fn cli_db_association_transfers_pending_and_switch_surfaces_own_record() { - if common::run_with_opencode_home( + if common::opencode_home::run_with_opencode_home( "cli_db_association_transfers_pending_and_switch_surfaces_own_record", ) { return; @@ -870,7 +870,9 @@ async fn cli_db_association_transfers_pending_and_switch_surfaces_own_record() { #[cfg(unix)] #[tokio::test(flavor = "multi_thread")] async fn tui_switch_signal_rebinds_and_restart_resumes_the_new_id() { - if common::run_with_opencode_home("tui_switch_signal_rebinds_and_restart_resumes_the_new_id") { + if common::opencode_home::run_with_opencode_home( + "tui_switch_signal_rebinds_and_restart_resumes_the_new_id", + ) { return; } // ---- env setup (single test fn: this binary owns process env) ---- diff --git a/crates/freshell-ws/tests/rest_locator_identity.rs b/crates/freshell-ws/tests/rest_locator_identity.rs index 554c0b5fc..7821884da 100644 --- a/crates/freshell-ws/tests/rest_locator_identity.rs +++ b/crates/freshell-ws/tests/rest_locator_identity.rs @@ -342,7 +342,9 @@ fn cleanup(h: &Harness) { /// exactly as the plugin would, via the pub drain (no sweep-timer race). #[tokio::test(flavor = "multi_thread")] async fn rest_created_opencode_pane_binds_identity_row_and_ledger() { - if common::run_with_opencode_home("rest_created_opencode_pane_binds_identity_row_and_ledger") { + if common::opencode_home::run_with_opencode_home( + "rest_created_opencode_pane_binds_identity_row_and_ledger", + ) { return; } let h = spawn_merged_server(vec![common::sleeper_cli_spec("opencode")], None).await; diff --git a/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs b/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs index 694f9ed39..25f51a3ae 100644 --- a/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs +++ b/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs @@ -154,7 +154,7 @@ async fn next_created_or_error(ws: &mut common::TestWs, request_id: &str) -> Val #[tokio::test(flavor = "multi_thread")] async fn restore_create_with_deleted_cwd_rehomes_instead_of_failing_spawn() { - if common::run_with_opencode_home( + if common::opencode_home::run_with_opencode_home( "restore_create_with_deleted_cwd_rehomes_instead_of_failing_spawn", ) { return; From e239491618aefc24cbcfc9e0452839c530023710 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 07:29:04 -0700 Subject: [PATCH 220/493] refactor: satisfy required runtime compiler checks --- crates/freshell-runtime-client/src/lib.rs | 2 +- crates/freshell-runtime-protocol/src/lib.rs | 6 ++-- .../src/claude_source_observation.rs | 4 +-- crates/freshell-supervisor/src/backend.rs | 2 +- .../claude_source_observation_tests.rs | 8 ++--- .../freshell-supervisor/src/backend/docker.rs | 31 ++++++++++++----- .../src/backend/native_auth.rs | 34 +++++++++---------- .../src/claude_source_observation.rs | 4 +-- crates/freshell-supervisor/src/recovery.rs | 19 ++--------- .../src/registry/input_reservation.rs | 3 +- crates/freshell-supervisor/src/service.rs | 2 +- 11 files changed, 58 insertions(+), 57 deletions(-) diff --git a/crates/freshell-runtime-client/src/lib.rs b/crates/freshell-runtime-client/src/lib.rs index 72cb2fe99..faa4c850f 100644 --- a/crates/freshell-runtime-client/src/lib.rs +++ b/crates/freshell-runtime-client/src/lib.rs @@ -125,7 +125,7 @@ impl RuntimeClient { match self .request( RequestId::new(), - AdminCommand::ObserveClaudeSources(request), + AdminCommand::ObserveClaudeSources(Box::new(request)), ) .await? { diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 0c5f49b7b..834275ae2 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -1580,8 +1580,8 @@ mod provider_launch_context_tests { profile: ProviderSecretProfile::ClaudeConfigurationFile, provider_relative_path: Some(".freshell/references/one".into()), }; - assert!(validate_provider_secrets("claude", &[reference.clone()]).is_ok()); - assert!(validate_provider_secrets("codex", &[reference.clone()]).is_err()); + assert!(validate_provider_secrets("claude", std::slice::from_ref(&reference)).is_ok()); + assert!(validate_provider_secrets("codex", std::slice::from_ref(&reference)).is_err()); assert!( validate_provider_secrets("claude", &[reference.clone(), reference.clone()]).is_err() ); @@ -2799,7 +2799,7 @@ pub enum AdminCommand { Health, ReserveProviderInputs(ReserveProviderInputsRequest), NativeAuthLookup(NativeAuthLookupRequest), - ObserveClaudeSources(ObserveClaudeSourcesRequest), + ObserveClaudeSources(Box), BindProviderPathInputs(BindProviderPathInputsRequest), Launch(Box), Stop(StopRequest), diff --git a/crates/freshell-session-host/src/claude_source_observation.rs b/crates/freshell-session-host/src/claude_source_observation.rs index 84822fb82..311ef635b 100644 --- a/crates/freshell-session-host/src/claude_source_observation.rs +++ b/crates/freshell-session-host/src/claude_source_observation.rs @@ -864,7 +864,7 @@ fn write_reply(reply: &ObservationReply, output: &mut impl Write) -> Result<(), #[cfg(test)] mod tests { use super::*; - use std::io::{Cursor, Error, ErrorKind}; + use std::io::{Cursor, Error}; #[test] fn claude_retained_candidate_seed_preserves_metadata_and_executes_only_frozen_ordinary_inputs() @@ -1616,7 +1616,7 @@ process.stdin.on('data',()=>{}).on('end',()=>{ fn read(&mut self, bytes: &mut [u8]) -> std::io::Result { if self.header.position() == 4 { self.body_reads += 1; - return Err(Error::new(ErrorKind::Other, "body must remain unread")); + return Err(Error::other("body must remain unread")); } self.header.read(bytes) } diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index fb497dd69..b27840922 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -129,7 +129,7 @@ pub enum ClaudeSourceOwner { observation: freshell_runtime_protocol::InitialAuthVolumeObservation, }, Retained { - handle: OwnedRuntimeHandle, + handle: Box, recovery_source: Option, }, } diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 3d5ffc42a..105dd847e 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -453,7 +453,7 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() (None,Some(serde_json::from_value(json!({"sessionId":"retained-session","provider":flavor,"sessionType":if flavor=="kilroy" {"kilroy"} else {"freshclaude"},"runtimeVariant":"claude","providerStoreId":"retained-store","cwd":prior_workspace,"workspacePath":prior_workspace,"runAsUid":65534,"runAsGid":0,"providerLaunchContext":prior_context})).unwrap())) }; ClaudeSourceOwner::Retained { - handle: OwnedRuntimeHandle::from_registry( + handle: Box::new(OwnedRuntimeHandle::from_registry( InstallationId::new(), reservation.soul_id, IncarnationId::new(), @@ -474,7 +474,7 @@ async fn provider_claude_physical_candidate_publication_gates_frozen_consumer() terminal, agent, "retained-owned-home".into(), - ), + )), recovery_source: Some(freshell_runtime_protocol::ClaudeRecoverySourceBinding { provider: flavor.into(), git_common_dir: None, @@ -679,7 +679,7 @@ async fn qualify_transport(mode: &'static str) { })).unwrap())) }; ClaudeSourceOwner::Retained { - handle: OwnedRuntimeHandle::from_registry( + handle: Box::new(OwnedRuntimeHandle::from_registry( InstallationId::new(), reservation.soul_id.clone(), IncarnationId::new(), @@ -700,7 +700,7 @@ async fn qualify_transport(mode: &'static str) { terminal, agent, "owned-retained-home".into(), - ), + )), recovery_source: None, } } else { diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index e2cf3d40d..19b585b0e 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -58,6 +58,12 @@ enum MountPurpose { Adopt, } +struct WorkloadPaths<'a> { + workspace_path: &'a str, + cwd_path: &'a str, + git_common_path: Option<&'a str>, +} + pub fn terminal_mounts(spec: &TerminalLaunchSpec) -> Result { terminal_mounts_for(spec, MountPurpose::Create) } @@ -73,9 +79,11 @@ fn terminal_mounts_for( purpose: MountPurpose, ) -> Result { workload_mounts( - &spec.workspace_path, - &spec.cwd, - spec.git_common_dir.as_deref(), + WorkloadPaths { + workspace_path: &spec.workspace_path, + cwd_path: &spec.cwd, + git_common_path: spec.git_common_dir.as_deref(), + }, &spec.provider_bootstrap_files, &spec.provider_secret_references, spec.provider_launch_context.as_ref(), @@ -99,9 +107,11 @@ fn fresh_agent_mounts_for( purpose: MountPurpose, ) -> Result { workload_mounts( - &spec.workspace_path, - &spec.cwd, - spec.git_common_dir.as_deref(), + WorkloadPaths { + workspace_path: &spec.workspace_path, + cwd_path: &spec.cwd, + git_common_path: spec.git_common_dir.as_deref(), + }, &spec.provider_bootstrap_files, &spec.provider_secret_references, spec.provider_launch_context.as_ref(), @@ -111,15 +121,18 @@ fn fresh_agent_mounts_for( } fn workload_mounts( - workspace_path: &str, - cwd_path: &str, - git_common_path: Option<&str>, + paths: WorkloadPaths<'_>, bootstrap_files: &[ProviderBootstrapFile], secret_references: &[ProviderSecretReference], context: Option<&ProviderLaunchContext>, provider: &str, purpose: MountPurpose, ) -> Result { + let WorkloadPaths { + workspace_path, + cwd_path, + git_common_path, + } = paths; if let Some(context) = context { context.validate(provider).map_err(|error| error.message)?; } diff --git a/crates/freshell-supervisor/src/backend/native_auth.rs b/crates/freshell-supervisor/src/backend/native_auth.rs index a527a8c7a..5fd681528 100644 --- a/crates/freshell-supervisor/src/backend/native_auth.rs +++ b/crates/freshell-supervisor/src/backend/native_auth.rs @@ -454,6 +454,23 @@ impl DockerEngineBackend { } } +fn reserved_volume_labels( + reservation: &crate::registry::OwnedInputReservation, + daemon: &DockerDaemonId, + incarnation: &freshell_runtime_protocol::IncarnationId, +) -> Value { + json!({ + "com.freshell.installation-id":reservation.installation_id.as_str(), + "com.freshell.soul-id":reservation.soul_id.as_str(), + "freshell.input.reservation":reservation.id, + "freshell.input.incarnation":incarnation.as_str(), + "freshell.input.intent":"1", + "freshell.input.daemon":daemon.as_str(), + "freshell.input.image":reservation.image_ref, + "freshell.input.binary-sha256":reservation.binary_sha256 + }) +} + #[cfg(test)] mod tests { use super::*; @@ -981,20 +998,3 @@ mod tests { } } } - -fn reserved_volume_labels( - reservation: &crate::registry::OwnedInputReservation, - daemon: &DockerDaemonId, - incarnation: &freshell_runtime_protocol::IncarnationId, -) -> Value { - json!({ - "com.freshell.installation-id":reservation.installation_id.as_str(), - "com.freshell.soul-id":reservation.soul_id.as_str(), - "freshell.input.reservation":reservation.id, - "freshell.input.incarnation":incarnation.as_str(), - "freshell.input.intent":"1", - "freshell.input.daemon":daemon.as_str(), - "freshell.input.image":reservation.image_ref, - "freshell.input.binary-sha256":reservation.binary_sha256 - }) -} diff --git a/crates/freshell-supervisor/src/claude_source_observation.rs b/crates/freshell-supervisor/src/claude_source_observation.rs index 30df4749f..0bdacfa4c 100644 --- a/crates/freshell-supervisor/src/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/claude_source_observation.rs @@ -96,7 +96,7 @@ impl Supervisor { }; recovery = Some(authority.start); ClaudeSourceOwner::Retained { - handle, + handle: Box::new(handle), recovery_source, } } @@ -453,7 +453,7 @@ mod tests { let mut envelope = Envelope::new( RequestId::new(), ControlRole::Web, - AdminCommand::ObserveClaudeSources(request), + AdminCommand::ObserveClaudeSources(Box::new(request)), ); envelope.auth = Some("synthetic-control-secret".into()); let result = tokio::time::timeout( diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index c561888e1..e3b6f251e 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -2092,22 +2092,7 @@ fn terminal_without_first_boot_state( }) } -fn refresh_terminal_provider_config( - terminal: &mut TerminalLaunchSpec, - user_root: &std::path::Path, -) { - let Some(context) = terminal.provider_launch_context.as_mut() else { - return; - }; - refresh_provider_context_from_user_root( - context, - &terminal.mode, - std::path::Path::new(&terminal.workspace_path), - std::path::Path::new(&terminal.cwd), - user_root, - ); -} - +#[cfg(test)] fn replace_terminal_mcp_reference( terminal: &mut TerminalLaunchSpec, reference: freshell_runtime_protocol::McpCapabilityReference, @@ -2143,6 +2128,7 @@ fn install_prepared_provider_context( Ok(()) } +#[cfg(test)] fn refresh_provider_context_from_user_root( context: &mut freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -2255,6 +2241,7 @@ fn refresh_provider_context_from_user_root( } } +#[cfg(test)] fn safe_provider_config_source( root: &std::path::Path, candidate: &std::path::Path, diff --git a/crates/freshell-supervisor/src/registry/input_reservation.rs b/crates/freshell-supervisor/src/registry/input_reservation.rs index 663e69118..2084076ef 100644 --- a/crates/freshell-supervisor/src/registry/input_reservation.rs +++ b/crates/freshell-supervisor/src/registry/input_reservation.rs @@ -159,7 +159,8 @@ impl Registry { ) -> Result, RegistryError> { let installation = self.installation_id().clone(); self.run_blocking(move |conn| { - let row: Option<(String,String,String,String,String,String,String,String)> = conn.query_row( + type BoundInputReservationRow = (String, String, String, String, String, String, String, String); + let row: Option = conn.query_row( "SELECT p.soul_id,p.reservation_id,p.creation_seed_ref,p.image_ref,p.host_binary_path,p.host_binary_sha256,p.observation,p.provider FROM provider_input_preparations p JOIN incarnations i ON i.incarnation_id=p.bound_incarnation_id JOIN souls s ON s.soul_id=p.soul_id WHERE p.bound_incarnation_id=?1 AND p.consumed=1 AND i.launch_state='prepared' AND i.soul_id=p.soul_id AND s.desired_state='running' AND s.intent_revision=p.intent_revision AND s.provider=p.provider AND s.creation_seed_ref=p.creation_seed_ref", params![incarnation_id.as_str()], |row|Ok((row.get(0)?,row.get(1)?,row.get(2)?,row.get(3)?,row.get(4)?,row.get(5)?,row.get(6)?,row.get(7)?)) ).optional()?; diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index ec37d437d..a5a14902f 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -300,7 +300,7 @@ impl Supervisor { self.registry .assert_epoch(request.expected_control_epoch) .map_err(map_registry)?; - self.observe_claude_sources(request) + self.observe_claude_sources(*request) .await .map(AdminResult::ClaudeSourceObservation) } From 517075bbcabdf5331a317987587b56b00441f96c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 08:30:29 -0700 Subject: [PATCH 221/493] docs: record completed managed configuration reference task --- .../2026-10-05-managed-provider-environment.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index a591557a0..26f967f03 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -234,20 +234,20 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, `GrantError` and `ReferenceError` retain classified safe operation/profile/field/source-ordinal data and never include raw grant lines, expanded values or referenced bytes. `ResolvedFileReferencePlan` contains canonical sources, declaring source locations/ordinals, runtime destinations, referenced variable names, generated-placeholder recipes and unexpanded templates only; it has no transient expanded-text/map field and no derived Debug exposing private input. Controller and host translate existing finite protocol profiles to `EnvironmentProfile` in their wrappers. -- [ ] **Step 1: Add failing consumers.** Table-drive absolute, relative, `~/`, spaces, in-root symlinks and external references for each selected field/origin row in the inline finite provider matrix. OpenCode cases cover global JSON/JSONC, original-workspace project config at nested cwd and ancestor, TUI, inline instance-cwd origin, nested `{file:{env:NODE_EXTRA_CA_CERTS}}`, one-pass env substitutions that introduce complete file tokens plus secret prefix/suffix text, original ambient/request path variables, missing/empty-variable semantics, comment-prefixed tokens, runtime CA destination→source mapping, loader-source-specific authEnv precedence, local path/file-URL plugins and instruction resources. More than four valid file references must work for terminal and fresh launch specs; duplicate destination, excessive per-profile/aggregate counts and cross-provider reference must fail before mounts. Change a synthetic grant's path between planning and spawn: dispatch is prevented with changed-input retry, previous usable stage survives, no different file is silently consumed. +- [x] **Step 1: Add failing consumers.** Table-drive absolute, relative, `~/`, spaces, in-root symlinks and external references for each selected field/origin row in the inline finite provider matrix. OpenCode cases cover global JSON/JSONC, original-workspace project config at nested cwd and ancestor, TUI, inline instance-cwd origin, nested `{file:{env:NODE_EXTRA_CA_CERTS}}`, one-pass env substitutions that introduce complete file tokens plus secret prefix/suffix text, original ambient/request path variables, missing/empty-variable semantics, comment-prefixed tokens, runtime CA destination→source mapping, loader-source-specific authEnv precedence, local path/file-URL plugins and instruction resources. More than four valid file references must work for terminal and fresh launch specs; duplicate destination, excessive per-profile/aggregate counts and cross-provider reference must fail before mounts. Change a synthetic grant's path between planning and spawn: dispatch is prevented with changed-input retry, previous usable stage survives, no different file is silently consumed. The child executes the real supported configuration resolver where available and otherwise the pinned resolver-equivalent synthetic provider fixture; report the actual file-consumption nonce/digest and native trim/escape behavior, never the referenced bytes. Distinguish same relative spelling in two declaring directories. Copy original config bytes beforehand and assert they remain unchanged afterward. Missing/unreadable/dangling/cyclic referenced targets must prevent launch with an actionable named field/path, no capability omission. Sentinel credential and generated-placeholder cases assert absence from staged templates, durable registry/JSON, grant metadata, Docker inspect and logs while child consumption succeeds. Test initial launch, resume/reissue and both terminal/fresh routes using synthetic grants only. Cover exact native auth fallback and typed-grant precedence, new absent versus unexpected/missing retained volumes, callback nested lookup without deadlock, duplicate auth-key order, same-path changed reference selection, timeout/lost helper acknowledgement and exact cleanup. Capture sentinel-bearing Docker logs/create bodies, IPC prepared metadata, registry/caches/receipts and staged ordinary roots: auth values and unvalidated candidate expressions must be absent. Pure decoder tests must preserve current interpolation, provider allowlists, proxy validation and unknown/legacy-profile rejection, and detect actual 128 KiB+1 read growth. -- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_references`; existing stage/host consumer tests must fail because original absolute/relative references are unresolved in runtime. A compile-only missing module is setup evidence, not behavioral red. -- [ ] **Step 3: Apply precise implementation patches.** Extract `provider_secret_resolution.rs:88-432`'s existing pure keys/env parsing, interpolation and provider/proxy validation into `provider_environment_grants.rs`; do not create a second permissive parser. Keep auth/CA-file profiles out of environment decoding and map protocol profiles in wrappers. `read_environment_grant` verifies the canonical regular private grant and reads at most 128 KiB+1 through a bounded reader before decoding, rejecting source growth; mounted host bytes use the same pure decoder. Add platform URL dependency only if the existing validator requires it. Do not instrument raw maps/values. +- [x] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider_references`; existing stage/host consumer tests must fail because original absolute/relative references are unresolved in runtime. A compile-only missing module is setup evidence, not behavioral red. +- [x] **Step 3: Apply precise implementation patches.** Extract `provider_secret_resolution.rs:88-432`'s existing pure keys/env parsing, interpolation and provider/proxy validation into `provider_environment_grants.rs`; do not create a second permissive parser. Keep auth/CA-file profiles out of environment decoding and map protocol profiles in wrappers. `read_environment_grant` verifies the canonical regular private grant and reads at most 128 KiB+1 through a bounded reader before decoding, rejecting source growth; mounted host bytes use the same pure decoder. Add platform URL dependency only if the existing validator requires it. Do not instrument raw maps/values. Immediately before capability stage/publication and final mount construction, controller builds a transient original host/provider env lookup with request overrides for terminal launches, current server launch env for fresh launches, then approved OneCLI child env overlay matching direct child precedence. Use original host HOME/cwd/config origins. Select only file-expression variable needs for reference planning; never expand the durable environment allowlist in `managed_runtime.rs:714`. Apply well-known authEnv only to pinned local loader sources; inline/TUI receive no overlay. Prefer the selected typed OpencodeOnecliAuthFile, read through a bounded 128 KiB private reader and shared order-preserving auth decoder; do not substitute host user's auth or add generic OPENCODE_AUTH_CONTENT forwarding. Without a grant, perform a finite selected-key lookup from the exact registry-owned quiescent soul volume before mount planning. First-launch empty auth needs explicit installation/soul intent authority plus confirmed expected-volume absence, checked again at launch; missing native ID, unavailable daemon or unexpected volume is not new-absence proof. Use one fixed provider-UID cold helper and existing owned create/start/wait/delete pattern, with only the exact read-only soul volume and a single-use private exchange-directory socket. Auth reply is bounded memory-only, has no value-printing Debug/cache/durable/log path, and Docker logging is disabled; fixed sanitized failures only. Accept/framed exchange and helper supervision run concurrently within one aligned operation/callback deadline. Keep socket path within sun_path bounds and mapped-group directory 0710/socket 0660 access. For replacement callback, invoke an attempt-bound lookup under the recovery ownership already held, never reacquire the same soul lock or await IPC with capability-store/registry locks held. No live worker is stopped for a lookup. Host recomputes validated reference/mount metadata from actual selected auth/grant bytes before dispatch; same grant/path with changed reference target triggers bounded replan/failure, while unrelated auth changes need not. Retain native auth unless matching typed grant replaces it. Drop decoded maps after obtaining canonical source metadata and unexpanded templates; controller reads only approved payloads through bounded transient managed-secret publication, never ordinary-stage expansion. Add a small shared token/string transformer to existing `opencode_config.rs` with the pinned single-env-pass→file-scan order. Recognize an outer file expression containing complete nested env tokens so the inner brace cannot terminate the outer match. Expand only its path transiently; missing/empty env is empty text as upstream, then resolve absolute/relative/`~/` against original origin. Canonicalize/check source metadata and replace the whole outer expression with `{file:/home/freshell/provider/.freshell/references/}` in staged copies; preserve unrelated env tokens. Do not try to canonicalize the runtime HOME on the controller: map a known runtime auth/CA file destination back to its existing typed source reference and reuse its grant/destination. Source errors identify original field/ordinal and variable names, not raw expanded credential values. Keep exact accepted original sources and ordered logical occurrence metadata. Produce each lane's current unexpanded normalization/owned edits before making recipes: raw main documents retain comments/escapes, inherited inline normalizes after MCP edits, selected rebind TUI normalizes after its existing plugin edit, ordinary TUI retains raw form. Plan one transient document-level env pass followed by the pinned snapshotted file-match pass with full line-prefix context, including relevant key lexemes and partial inserted tokens; decoded strings and original offsets across later serialization are insufficient. Source-view generation/offsets belong to the effective unexpanded document. For env-introduced file tokens or secret prefix/suffix, use a bounded private document-context/source-event recipe beside that source; a contained raw lexeme placeholder is allowed only when the document-level match proves containment. Preserve the single pass, raw escapes, comment-prefix rule and introduced env tokens remaining literal. Replace the affected staged source string with `{env:FRESHELL_CONFIG_REFERENCE_}`; recipe records only unexpanded effective source/span with lane/view identity and full-context event metadata, variable provenance, approved selections and runtime destinations; never expanded values. Cross-lexeme/outside-lexeme accepted occurrences require a bounded whole-effective-document child-only recipe: validate/perform owned edits on retained accepted source first, then emit a native env wrapper. Its expanded document exists only in the child secret value and native parser input; wrapper consumption is a mandatory actual-provider fixture, not assumed proof. Original-form alias families may instead preserve native substitution, per the inline Selected C05 projection table. Session-host preparation uses the same transformer plus freshly resolved approved child-secret env to evaluate that source string and replace file paths with approved runtime destinations. Put resulting generated strings only in the child secret environment, never terminal.env, launch serialization or durable records. Preserve JSONC raw string lexemes/escapes and comment/source spans; ordinary parsed strings are not equivalent to raw provider substitution. Reserved generated env names must be unique, bounded to 128 and collision-checked against user/request/secret and source-specific auth lookup before spawn; auth must not shadow a generated name. Arbitrary unrelated ambient secret text outside a file-path expression does not gain a new forwarding framework; generated secret-bearing prefix/suffix values use existing approved env/auth child transport. Ordinary ambient/request values used only to select a referenced path require no child value after direct path rewrite. Classify all `{file:...}` target payloads as sensitive transport regardless of filename. Use existing indexed secret mounts; allow ordinary read-only referenced resources without forcing 0600 source chmod, while existing OneCLI environment/auth profiles retain private-mode validation. Replace both current `provider_secret_references.len() > 4` ceilings in `TerminalLaunchSpec` and `FreshAgentLaunchSpec` with the same bounded rule: at most four legacy references, at most 128 unique configuration-file destinations, and at most 132 total; legacy profiles stay unique, new finite configuration-file profiles may repeat only for distinct safe destinations. Central `validate_provider_secrets` enforces these cardinalities/profile/destination rules for both entrypoints. Bounded config-file grants allow up to 16 MiB each/32 MiB combined (capacity validation may revise this explicit profile before implementation) and remain temporary process-only payloads; actual read bytes enforce limits. Add canonical sources to ordinary-copy exclusions. For the exact outside/shared-origin aliases identified in the C05 table, publish indexed files into one attempt-owned local-driver tmpfs volume using fixed nonsecret options type=tmpfs/device=tmpfs/o=size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec. The provider-UID publisher creates normal files/files-ordinal components with 0600, receives at most 64 KiB raw chunks with one outstanding acknowledgement and encoded frames below 1 MiB, and logs no payload. Its exact exchange-directory socket is private and supervised concurrently; Docker logging is none. Typed VolumeOptions.Subpath regular-file mounts with NoCopy=true/ReadOnly=true expose indexed sources at approved exact aliases or private indexed destinations. Keep publisher and workload active mount references overlapping: population acknowledgement returns before workload creation; supervisor retains publisher until successful mount/startup GrantAccepted and then releases it. A stopped volume object retains no tmpfs content; cold last-unmount requires authorized repopulation. Allow at most one active and one pending 64 MiB volume per soul; abort/uncertain-delivery/startup cleanup uses exact preparation ownership and cannot accumulate generations. Legacy small auth publication remains separate. Use existing child-secret map for finite selected original-form file-path inputs only, never ambient env forwarding. Validate every logical/effective destination, uniqueness/content identity, profile and reserved mount collision before spawn. Workspace file/root mountpoints and their parents must already exist; Docker must not create host-workspace entries. Outside-root mountpoints may be constructed in the container image; provider-owned volume placeholders require explicit precreation. For ordinary configured resource/plugin paths, copy complete package/resource closure via Task 1 and rewrite only recognized field nodes to the corresponding projected paths, preserving adjacent imports. Include complete selected implicit external config/skill/plugin roots, not just config leaves. Precreate descriptor-owned physical .freshell/provider-roots/ subdirectories/placeholders in the exact soul volume before main mounting, under initial ownership or retained provider UID as appropriate. Mount only selected subpaths at required discovery paths with NoCopy=true and writable provider behavior. Populate via canonical HOME paths with the common provider-owned transaction; keep mounted root inodes stable and replace only owned leaves. Private reference placeholders are explicit exclusions, never copied bytes. Adopt compares HostConfig.Mounts subpath/NoCopy plus actual mount type/name/source/destination/RW and exact owned volume options; flattened Mounts listing has no subpath field. Keep ordered main/TUI logical source occurrences separately from canonical read identities. Ordinary independent source destinations can use exact run-owned config binds, including discovered outside-workspace ancestors at their original logical filenames. Shared workspace-symlink families retain origin-dependent relative references and expose each selected target through exact private aliases; only source-independent absolute/HOME-sensitive collisions get identical selective rewrites to distinct indexed destinations. A shared child-only document recipe requires compatible lane evaluation; differing source-specific auth is not implicitly flattened. Keep physical HOME for all supported workspace topologies; the considered logical-HOME simplification is not selected because default-HOME workspaces can be masked or mutated. Preserve native original-form evaluation where exact target aliases are injective; generated shared recipes may not flatten differing auth lanes. Validate representability/collisions before dispatch, retain original data and return classified preparation conflict rather than wrong payload or disabled capability. Do not add parser/document machinery solely to certify arbitrary env-injected JSON programs outside the requested ordinary field/reference matrix; actual required matrix/consumer fixtures remain acceptance gates. Never deduplicate config occurrences by inode or attempt independent binds through symlinks that collapse to one destination. The same authoritative ordered source plan drives the preparer and native discovery. Inline origin is instance cwd and TUI/local origins follow their declaring sources, not private stage paths. Resolve/bound all files before publishing the new capability root; append final canonical references to the immutable launch spec before Docker constructs mounts. Recompute planning on replacement/reissue and refresh unexpanded templates through the provider-owned transaction on exact resume. Extend the existing authenticated private capability callback with one complete prepared metadata context for both terminal and fresh launches: selected roots, references, unexpanded recipes/origins, exact config mounts, new grant and prior/pending ownership IDs. No auth lookup values enter this reply. Source/recipe descriptors reference immutable private staged files, view generation and offsets; no raw source/template snippets or 100,000-entry manifests enter callback/launch/registry metadata. The shared 1 MiB control-frame ceiling is checked by bounded serialization before publication; oversized metadata is an explicit profile-limit error. Complete manifests stay on their stage/provider filesystem under the shared metadata budget. Consume it before supervisor registry preparation and immutable Docker mount publication; stop cloning stale references or separately rescanning roots. Replace immediate old-grant revocation with idempotent commit/abort. Retire prior inputs only after Host GrantAccepted, relevant registry commit and required exact-native verification; MCP activation is too early. Startup orphan cleanup protects pending/prior generations and reconciles uncertain delivery/dispatch. A lost response cannot authorize revoking both generations or redispatching a conversation. Partial new-stage/capability failures remove only this attempt's owned new inputs. Reference identity is the approved preparation snapshot: ordered source occurrence/lane/logical origin, controller-validated selection and canonical read/bind source, finite validated logical/runtime aliases, typed profile/destination and immutable candidate generation. At host preparation, freshly available auth/grant-derived candidates map only through that occurrence-specific snapshot to an authorized mounted source. Host cannot detect an unmounted original symlink retarget or canonicalize the controller tree. Controller-only ambient path inputs already consumed at planning are marked frozen selections, not fictitiously re-evaluated. Unknown/new/ambiguous aliases cause bounded controller replan before dispatch; stable and newly validated legitimate aliases must work. Use at most two pre-dispatch replan retries, preserving old inputs and recording classified failure on exhaustion. Selection identity is not inode/latest-content identity: actual bounded read consumes the mounted generation. Reuse that same read for comparison, auth publication and generated child values; no reread may silently select another plan. Next preparation resolves later original path changes. Original host paths need not exist in container HOME: do not attempt to canonicalize an unavailable host tree inside the session host. Known relocated auth/CA paths map back to their approved source metadata before comparison. A changed grant/source choice aborts before dispatch with classified changed-input preparation retry that rebuilds stage/mounts; it never reads an unapproved file, persists a grant snapshot or replaces an already dispatched conversation. Failed resolution revokes only this attempt's new grant/stage and leaves previous projection usable. Both terminal and fresh paths consume the one computed context and child-only generated env, installed before provider validation/spawn. -- [ ] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. -- [ ] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. -- [ ] **Step 7: Focused compiling checkpoint commits.** Record Task 2's start commit. Commit coherent, compiling changes with their behavioral tests at major checkpoints, for example shared bounded decoder/planner, source-correct grant/mount preparation, then generation commit/abort and host refresh integration. Each commit must include the necessary protocol/callsite changes for its logical unit and pass its affected checks; do not force this entire task into one commit or commit broken intermediate wiring. Stage only that checkpoint's exact files and use a descriptive message such as `fix: resolve managed provider file references at their source origins`. Record the final Task 2 commit; the fresh task specification-plus-quality review covers the complete start..end range, including every intermediate checkpoint. +- [x] **Step 4: Green.** Run planner, protocol, capability, secret and mount validation tests; expected actual consumer succeeds, original config unchanged and sentinel absent from all non-child surfaces. +- [x] **Step 5: Refactor.** Keep provider syntax/origin knowledge in the narrow planner, generic copying in platform and referenced payload bytes in existing child secret resolution. Use one pure bounded grant decoder and one pinned string transformer at controller/host; delete the old host-only decoder implementation after its tests move. Remove duplicate scans from terminal/fresh-agent/reissue paths and avoid introducing a general provider configuration or ambient-env forwarding framework. +- [x] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-platform --locked provider_environment_grants`; `mise exec rust@1.96 -- cargo test -p freshell-platform --locked opencode_config`; `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_provider`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_mcp`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked provider_secret`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`. Expected all named tests execute/pass. Task 3 provides complete launched child proof, including the env-dependent/reference-drift cases, not just planner tests. +- [x] **Step 7: Focused compiling checkpoint commits.** Record Task 2's start commit. Commit coherent, compiling changes with their behavioral tests at major checkpoints, for example shared bounded decoder/planner, source-correct grant/mount preparation, then generation commit/abort and host refresh integration. Each commit must include the necessary protocol/callsite changes for its logical unit and pass its affected checks; do not force this entire task into one commit or commit broken intermediate wiring. Stage only that checkpoint's exact files and use a descriptive message such as `fix: resolve managed provider file references at their source origins`. Record the final Task 2 commit; the fresh task specification-plus-quality review covers the complete start..end range, including every intermediate checkpoint. ### Task 3: Transactional provider projection and complete environment smoke From f6c335e54564b550adc2816916bc0ee8c440daf9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 09:08:39 -0700 Subject: [PATCH 222/493] fix: publish managed provider refreshes transactionally --- .../src/managed_mcp_capability.rs | 93 +++ crates/freshell-session-host/src/main.rs | 691 +++++++++++++++--- .../tests/provider_config_refresh.rs | 286 +++++++- crates/freshell-supervisor/src/backend.rs | 20 +- scripts/testing/runtime-test-broker.ts | 2 +- .../testing/runtime-test-broker.test.ts | 29 +- 6 files changed, 1000 insertions(+), 121 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 07acb12d9..b9a5dcb09 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -11413,6 +11413,99 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ } } + #[test] + fn managed_mcp_capability_publication_failure_cleans_new_stage_and_keeps_prior_grant() { + use std::{fs, os::unix::fs::PermissionsExt}; + let root = tempfile::tempdir().unwrap(); + let directory = root.path().join("control/mcp-capabilities"); + let home = root.path().join("home"); + let workspace = root.path().join("workspace"); + fs::create_dir_all(&directory).unwrap(); + fs::create_dir_all(home.join(".config/opencode/plugins")).unwrap(); + fs::create_dir(&workspace).unwrap(); + fs::write( + home.join(".config/opencode/plugins/command"), + "#!/bin/sh\nprintf prior-usable", + ) + .unwrap(); + fs::set_permissions( + home.join(".config/opencode/plugins/command"), + fs::Permissions::from_mode(0o755), + ) + .unwrap(); + let references = provider_source_references("opencode", &home).unwrap(); + let store = CapabilityStore::default(); + let soul = SoulId::new(); + let first = store + .issue( + soul.clone(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &references)), + Default::default(), + None, + ) + .unwrap(); + let first_root = staged_provider_root_path(&directory, &first.grant_id).unwrap(); + let first_file = grant_path(&directory, &first.grant_id).unwrap(); + store + .activate( + &first.grant_id, + soul.clone(), + IncarnationId::new(), + &directory, + &first.endpoint, + ) + .unwrap(); + let before = fs::read(&first_file).unwrap(); + let token = store + .grants + .lock() + .unwrap() + .get(&first.grant_id) + .unwrap() + .token + .clone(); + assert!(store.authenticates(&token)); + // Provider roots and ephemeral input staging remain writable; only the + // final capability publication is denied after staging has completed. + fs::set_permissions(&directory, fs::Permissions::from_mode(0o500)).unwrap(); + let inline = "{\"theme\":\"synthetic-new-ephemeral\"}"; + let failed = store.issue_replacement_with_inputs( + soul, + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &references)), + OpencodeEphemeralInput { + inline_config: Some(inline), + workspace: Some(&workspace), + cwd: Some(&workspace), + ..Default::default() + }, + ); + fs::set_permissions(&directory, fs::Permissions::from_mode(0o700)).unwrap(); + assert!(failed.unwrap_err().contains("journal publication failed")); + assert!( + fs::read(&first_file).unwrap() == before, + "prior grant changed" + ); + assert!(store.authenticates(&token)); + assert_eq!(store.grants.lock().unwrap().len(), 1); + assert_eq!(fs::read_dir(&directory).unwrap().count(), 1); + let roots = root.path().join("control/provider-roots"); + assert_eq!( + fs::read_dir(&roots).unwrap().count(), + 1, + "failed capability publication left a staged root or temporary generation" + ); + assert!(first_root.join("plugins/command").is_file()); + assert!(!first_root.join("ephemeral/inline-config.json").exists()); + assert_eq!( + fs::read(first_root.join("plugins/command")).unwrap(), + b"#!/bin/sh\nprintf prior-usable" + ); + } + #[test] fn opencode_provider_home_tui_source_is_approved_and_reread_safely() { let control = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 71550e3c6..011d0c125 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1010,14 +1010,40 @@ async fn grant_execution( .map_err(|error| error.runtime_error())?, None => terminal, }; + let resolved_secrets = prepared_secrets.take().ok_or_else(|| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "prepared terminal credentials unavailable", + ) + })?; + let prepare_files = |terminal: &TerminalLaunchSpec| { + prepare_provider_files( + &terminal.mode, + &resolved_secrets.provider_files, + &resolved_secrets.configuration_files, + terminal.run_as_uid, + terminal.run_as_gid, + terminal + .provider_launch_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + .is_some_and(|inputs| inputs.indexed_files.is_some()), + ) + }; // First boot transfers the durable provider-home inode to the // unprivileged provider. An exact-resume host lacks FOWNER and // DAC_OVERRIDE, so it stages current approved config and asks a // provider-owned helper to refresh only that projection. Docker's // ownership proof pins the same soul-scoped volume on replacement. if !exact_resume { - prepare_provider_state_for_fresh_launch(&mut terminal)?; + prepare_provider_state_for_fresh_launch(&mut terminal, prepare_files)?; } else { + prepare_files(&terminal).map_err(|error| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + format!("prepare OneCLI provider files: {error}"), + ) + })?; refresh_provider_context( terminal.provider_launch_context.as_ref(), &terminal.mode, @@ -1028,30 +1054,6 @@ async fn grant_execution( ) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; } - let resolved_secrets = prepared_secrets.take().ok_or_else(|| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - "prepared terminal credentials unavailable", - ) - })?; - prepare_provider_files( - &terminal.mode, - &resolved_secrets.provider_files, - &resolved_secrets.configuration_files, - terminal.run_as_uid, - terminal.run_as_gid, - terminal - .provider_launch_context - .as_ref() - .and_then(|context| context.prepared_inputs.as_ref()) - .is_some_and(|inputs| inputs.indexed_files.is_some()), - ) - .map_err(|error| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - format!("prepare OneCLI provider files: {error}"), - ) - })?; let prepared = providers::prepare_terminal( terminal, &resolved_secrets.environment, @@ -1133,6 +1135,33 @@ async fn grant_execution( launch.permission_mode = resume.permission_mode.clone(); launch.cwd = resume.cwd.clone(); } + let mut resolved_secrets = prepared_secrets.take().ok_or_else(|| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "prepared fresh-agent credentials unavailable", + ) + })?; + let prepare_files = || { + prepare_provider_files( + launch.provider.native_provider(), + &resolved_secrets.provider_files, + &resolved_secrets.configuration_files, + launch.run_as_uid, + launch.run_as_gid, + launch + .provider_launch_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + .is_some_and(|inputs| inputs.indexed_files.is_some()), + ) + .map_err(|error| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + format!("prepare hosted OneCLI provider files: {error}"), + ) + })?; + Ok::<(), RuntimeError>(()) + }; if !exact_resume { prepare_provider_bootstrap_files( launch.provider.native_provider(), @@ -1150,6 +1179,7 @@ async fn grant_execution( format!("prepare hosted fresh-agent provider state failed: {error}"), ) })?; + prepare_files()?; prepare_provider_context( launch.provider_launch_context.as_ref(), launch.provider.native_provider(), @@ -1164,6 +1194,7 @@ async fn grant_execution( ) })?; } else { + prepare_files()?; refresh_provider_context( launch.provider_launch_context.as_ref(), launch.provider.native_provider(), @@ -1174,30 +1205,6 @@ async fn grant_execution( ) .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; } - let mut resolved_secrets = prepared_secrets.take().ok_or_else(|| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - "prepared fresh-agent credentials unavailable", - ) - })?; - prepare_provider_files( - launch.provider.native_provider(), - &resolved_secrets.provider_files, - &resolved_secrets.configuration_files, - launch.run_as_uid, - launch.run_as_gid, - launch - .provider_launch_context - .as_ref() - .and_then(|context| context.prepared_inputs.as_ref()) - .is_some_and(|inputs| inputs.indexed_files.is_some()), - ) - .map_err(|error| { - RuntimeError::new( - RuntimeErrorCode::HostUnreachable, - format!("prepare hosted OneCLI provider files: {error}"), - ) - })?; providers::prepare_fresh_agent_child_environment( &launch, &mut resolved_secrets.environment, @@ -1266,6 +1273,7 @@ async fn grant_execution( fn prepare_provider_state_for_fresh_launch( terminal: &mut TerminalLaunchSpec, + prepare_files: impl Fn(&TerminalLaunchSpec) -> Result<(), String>, ) -> Result<(), RuntimeError> { if terminal.mode == "amplifier" && terminal.provider_bootstrap_files.is_empty() @@ -1282,6 +1290,7 @@ fn prepare_provider_state_for_fresh_launch( terminal.run_as_uid, terminal.run_as_gid, ) + .and_then(|_| prepare_files(terminal)) .and_then(|_| { prepare_provider_context_for_fresh_launch(terminal, &ProviderConfigPaths::default()) }) @@ -1336,6 +1345,7 @@ fn prepare_provider_state_for_fresh_launch( ) }, |terminal| { + prepare_files(terminal)?; prepare_provider_context_for_fresh_launch(terminal, &ProviderConfigPaths::default()) }, ) @@ -2476,38 +2486,359 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() }) { return Err("resource mountpoint metadata has an invalid owned root".into()); } - copy_environment_with_events_for_mounts( - "provider_publication", - &provider, - &selections, - provider_home, - &[], - &mounted, - ) - .map_err(|error| error.to_string())?; - let mut stale = previous.difference(¤t).collect::>(); - stale.sort_by_key(|path| std::cmp::Reverse(Path::new(path).components().count())); - for relative in stale { - if protected.contains(relative) { - continue; + publish_provider_projection(ProviderProjection { + provider: &provider, + selections: &selections, + home: provider_home, + previous: &previous, + current: ¤t, + protected: &protected, + mounted: &mounted, + }) +} + +struct ProviderProjection<'a> { + provider: &'a str, + selections: &'a [freshell_platform::provider_environment::CopySelection], + home: &'a Path, + previous: &'a std::collections::BTreeSet, + current: &'a std::collections::BTreeSet, + protected: &'a std::collections::BTreeSet, + mounted: &'a [PathBuf], +} + +// One provider-owned transaction covers replacement, stale removal and manifest +// publication. Its private stage and backups live on HOME's filesystem; only +// exact manifest leaves move, so shared directories keep native children. +fn publish_provider_projection(projection: ProviderProjection<'_>) -> Result<(), String> { + publish_provider_projection_with_manifest(projection, |source, destination| { + std::fs::rename(source, destination) + }) +} + +fn publish_provider_projection_with_manifest( + projection: ProviderProjection<'_>, + publish_manifest: impl FnOnce(&Path, &Path) -> std::io::Result<()>, +) -> Result<(), String> { + let ProviderProjection { + provider, + selections, + home, + previous, + current, + protected, + mounted, + } = projection; + use std::{collections::BTreeMap, fs, os::unix::fs::PermissionsExt}; + emit_environment_event( + serde_json::json!({"severity":"info","event":"provider.projection.transaction.started","provider":provider,"stage":"stage","previous_entries":previous.len(),"desired_entries":current.len()}), + ); + let transaction = tempfile::Builder::new() + .prefix(".freshell-provider-transaction-") + .tempdir_in(home) + .map_err(|error| format!("provider transaction stage: {error}"))?; + let desired = transaction.path().join("desired"); + let backup = transaction.path().join("backup"); + fs::create_dir(&backup).map_err(|error| error.to_string())?; + let staged = (|| { + let report = copy_environment_with_events( + "provider_publication", + provider, + selections, + &desired, + &[], + ) + .map_err(|error| error.to_string())?; + let actual = report + .projected_paths + .iter() + .map(|path| path.to_string_lossy().into_owned()) + .collect::>(); + if &actual != current { + return Err( + "provider projection manifest does not describe the complete selected environment" + .into(), + ); + } + // Serialize, write and sync the complete manifest before changing HOME. + let manifest = write_projection_manifest(transaction.path(), current)?; + // Links crossing mounted resource boundaries must still resolve in the + // final HOME graph, not relative to the independent alias mount. + for relative in current { + let source = desired.join(relative); + if fs::symlink_metadata(&source) + .map_err(|error| error.to_string())? + .file_type() + .is_symlink() + { + let target = fs::read_link(&source).map_err(|error| error.to_string())?; + let physical = fs::canonicalize(&source).map_err(|error| error.to_string())?; + let target_relative = physical + .strip_prefix(&desired) + .map_err(|_| "staged resource link escapes its complete projection")?; + if mounted.iter().any(|root| { + Path::new(relative).starts_with(root) && !target_relative.starts_with(root) + }) { + fs::remove_file(&source).map_err(|error| error.to_string())?; + std::os::unix::fs::symlink(home.join(target_relative), &source) + .map_err(|error| error.to_string())?; + } else if target.is_absolute() { + return Err( + "staged resource link must resolve within the selected projection".into(), + ); + } + } + } + Ok(manifest) + })(); + let manifest = match staged { + Ok(manifest) => manifest, + Err(cause) => { + emit_environment_event( + serde_json::json!({"severity":"error","event":"provider.projection.transaction.failed","provider":provider,"stage":"stage","cause":cause}), + ); + return Err(cause); + } + }; + let mut old = BTreeMap::new(); + let mut incoming = BTreeMap::new(); + let prepared: Result<(), String> = (|| { + for relative in previous.union(current) { + if protected.contains(relative) { + continue; + } + match fs::symlink_metadata(home.join(relative)) { + Ok(metadata) => { + old.insert(relative.clone(), metadata); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => { + return Err(format!("provider transaction inspect {relative}: {error}")) + } + } + if current.contains(relative) { + let metadata = fs::symlink_metadata(desired.join(relative)) + .map_err(|error| error.to_string())?; + if let Some(existing) = old.get(relative) { + let shared_directory = existing.is_dir() && metadata.is_dir(); + if !shared_directory + && !previous.contains(relative) + && !mounted.contains(&PathBuf::from(relative)) + { + return Err(format!("provider transaction destination {relative} is provider-created; retain it and repair the selected path")); + } + if existing.is_dir() && !metadata.is_dir() { + validate_owned_projection_descendants(home, Path::new(relative), previous)?; + } + } + incoming.insert(relative.clone(), metadata); + } + } + // Mounted files keep their inode. Back up their complete prior contents + // before any mutation, with the same shared copy byte profile. + let limits = freshell_platform::provider_environment::CopyLimits::default(); + let mut bytes = 0u64; + for (relative, metadata) in &old { + if mounted.contains(&PathBuf::from(relative)) && metadata.is_file() { + bytes = bytes + .checked_add(metadata.len()) + .ok_or("provider mounted backup byte accounting overflow")?; + if metadata.len() > limits.max_file_bytes || bytes > limits.max_total_bytes { + return Err(format!( + "provider mounted backup {relative} exceeds shared resource byte limits" + )); + } + let saved = backup.join(relative); + fs::create_dir_all(saved.parent().ok_or("backup parent unavailable")?) + .map_err(|error| error.to_string())?; + fs::copy(home.join(relative), &saved) + .map_err(|error| format!("provider transaction backup {relative}: {error}"))?; + } + } + Ok(()) + })(); + if let Err(cause) = prepared { + emit_environment_event( + serde_json::json!({"severity":"error","event":"provider.projection.transaction.failed","provider":provider,"stage":"validate","cause":cause}), + ); + return Err(cause); + } + let mut moved = Vec::::new(); + let mut installed = Vec::::new(); + let mut created = Vec::::new(); + let mut removed = Vec::<(String, u32)>::new(); + let mut touched_mounts = Vec::::new(); + let publication: Result<(), String> = (|| { + for (relative, metadata) in &old { + if metadata.is_dir() || !previous.contains(relative) || protected.contains(relative) { + continue; + } + if mounted.contains(&PathBuf::from(relative)) { + continue; + } + let saved = backup.join(relative); + fs::create_dir_all(saved.parent().ok_or("backup parent unavailable")?) + .map_err(|error| error.to_string())?; + fs::rename(home.join(relative), saved) + .map_err(|error| format!("provider transaction backup {relative}: {error}"))?; + moved.push(relative.clone()); } - let destination = provider_home.join(relative); - if let Ok(metadata) = std::fs::symlink_metadata(&destination) { + let mut directories = old + .iter() + .filter(|(_, metadata)| metadata.is_dir()) + .collect::>(); + directories + .sort_by_key(|(path, _)| std::cmp::Reverse(Path::new(path).components().count())); + for (relative, metadata) in directories { + if !previous.contains(relative) + || protected.contains(relative) + || incoming + .get(relative) + .is_some_and(|metadata| metadata.is_dir()) + || mounted.contains(&PathBuf::from(relative)) + { + continue; + } + match fs::remove_dir(home.join(relative)) { + Ok(()) => removed.push((relative.clone(), metadata.permissions().mode())), + Err(error) + if error.kind() == std::io::ErrorKind::DirectoryNotEmpty + && !incoming.contains_key(relative) => {} + Err(error) => { + return Err(format!( + "provider transaction remove directory {relative}: {error}" + )) + } + } + } + let mut nodes = incoming.iter().collect::>(); + nodes.sort_by_key(|(path, _)| Path::new(path).components().count()); + for (relative, metadata) in nodes { + let destination = home.join(relative); if metadata.is_dir() { - match std::fs::remove_dir(&destination) { - Ok(()) => {} - Err(error) if error.kind() == std::io::ErrorKind::DirectoryNotEmpty => {} - Err(error) => return Err(error.to_string()), + if !destination.exists() { + fs::create_dir(&destination).map_err(|error| { + format!("provider transaction create {relative}: {error}") + })?; + created.push(relative.clone()); + fs::set_permissions(&destination, fs::Permissions::from_mode(0o700)) + .map_err(|error| error.to_string())?; } + } else if mounted.contains(&PathBuf::from(relative)) { + if !metadata.is_file() + || !old.get(relative).is_some_and(|metadata| metadata.is_file()) + { + return Err(format!( + "provider transaction mounted file {relative} must retain a regular inode" + )); + } + touched_mounts.push(relative.clone()); + replace_mounted_projection_file(&desired.join(relative), &destination)?; } else { - std::fs::remove_file(&destination).map_err(|error| error.to_string())?; + fs::rename(desired.join(relative), &destination) + .map_err(|error| format!("provider transaction install {relative}: {error}"))?; + installed.push(relative.clone()); } } + publish_manifest(&manifest, &home.join(PROVIDER_PROJECTION_MANIFEST)) + .map_err(|error| format!("provider transaction publish manifest: {error}"))?; + Ok(()) + })(); + if let Err(cause) = publication { + let rollback: Result<(), String> = (|| { + for relative in installed.iter().rev() { + fs::remove_file(home.join(relative)).map_err(|error| error.to_string())?; + } + for relative in created.iter().rev() { + fs::remove_dir(home.join(relative)).map_err(|error| error.to_string())?; + } + for (relative, mode) in removed.iter().rev() { + fs::create_dir(home.join(relative)).map_err(|error| error.to_string())?; + fs::set_permissions(home.join(relative), fs::Permissions::from_mode(*mode)) + .map_err(|error| error.to_string())?; + } + for relative in moved.iter().rev() { + fs::rename(backup.join(relative), home.join(relative)) + .map_err(|error| error.to_string())?; + } + for relative in touched_mounts.iter().rev() { + replace_mounted_projection_file(&backup.join(relative), &home.join(relative))?; + } + Ok(()) + })(); + emit_environment_event( + serde_json::json!({"severity":if rollback.is_ok(){"info"}else{"error"},"event":"provider.projection.transaction.rollback","provider":provider,"stage":"rollback","restored":moved.len(),"removed":installed.len(),"successful":rollback.is_ok()}), + ); + emit_environment_event( + serde_json::json!({"severity":"error","event":"provider.projection.transaction.failed","provider":provider,"stage":"publish","cause":cause}), + ); + if let Err(rollback_cause) = rollback { + let retained = transaction.keep(); + return Err(format!( + "{cause}; rollback failed: {rollback_cause}; retain backup {} and repair", + retained.display() + )); + } + return Err(cause); + } + emit_environment_event( + serde_json::json!({"severity":"info","event":"provider.projection.transaction.committed","provider":provider,"stage":"commit","installed":installed.len(),"retired":moved.len()}), + ); + if let Err(error) = transaction.close() { + emit_environment_event( + serde_json::json!({"severity":"error","event":"provider.projection.transaction.cleanup_failed","provider":provider,"stage":"cleanup","cause":error.to_string()}), + ); + // Publication has committed. Cleanup failure is observable but cannot + // retroactively turn the complete environment into a failed refresh. } - write_projection_manifest(provider_home, ¤t)?; Ok(()) } +fn validate_owned_projection_descendants( + home: &Path, + relative: &Path, + previous: &std::collections::BTreeSet, +) -> Result<(), String> { + for entry in std::fs::read_dir(home.join(relative)).map_err(|error| error.to_string())? { + let entry = entry.map_err(|error| error.to_string())?; + let child = relative.join(entry.file_name()); + if !previous.contains( + child + .to_str() + .ok_or("provider projection path is not UTF-8")?, + ) { + return Err(format!("provider transaction incompatible replacement {} contains provider-created descendants", relative.display())); + } + if std::fs::symlink_metadata(entry.path()) + .map_err(|error| error.to_string())? + .is_dir() + { + validate_owned_projection_descendants(home, &child, previous)?; + } + } + Ok(()) +} + +fn replace_mounted_projection_file(source: &Path, destination: &Path) -> Result<(), String> { + use std::{fs, io::Write}; + let mut input = fs::File::open(source).map_err(|error| error.to_string())?; + let mut output = fs::OpenOptions::new() + .write(true) + .truncate(true) + .open(destination) + .map_err(|error| error.to_string())?; + std::io::copy(&mut input, &mut output).map_err(|error| error.to_string())?; + output.flush().map_err(|error| error.to_string())?; + output.sync_all().map_err(|error| error.to_string())?; + fs::set_permissions( + destination, + fs::metadata(source) + .map_err(|error| error.to_string())? + .permissions(), + ) + .map_err(|error| error.to_string()) +} + fn prepare_provider_context( context: Option<&freshell_runtime_protocol::ProviderLaunchContext>, provider: &str, @@ -2639,44 +2970,26 @@ fn prepare_provider_files_at( // Retain native auth until a matching typed grant has been written and // owned. The final destination replacement occurs only for that grant. for (index, (relative, contents)) in provider_files.iter().enumerate() { + use std::io::Write; let destination = home.join(relative); - let target = auth_dir.join(format!("provider-{index}")); - let temporary = auth_dir.join(format!("provider-{index}-tmp-{}", std::process::id())); - let mut options = std::fs::OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - { - use std::os::unix::fs::OpenOptionsExt; - options.mode(0o600); - } - use std::io::Write as _; - let mut file = options - .open(&temporary) + let mut target = tempfile::Builder::new() + .prefix(&format!("provider-{index}-")) + .tempfile_in(auth_dir) .map_err(|error| error.to_string())?; - file.write_all(contents) + target + .write_all(contents) .map_err(|error| error.to_string())?; - file.sync_all().map_err(|error| error.to_string())?; - std::fs::rename(&temporary, &target).map_err(|error| error.to_string())?; - set_mode(&target, 0o600)?; - set_owner(&target, run_as_uid, run_as_gid)?; - let parent = destination.parent().ok_or("auth file has no parent")?; - run_provider_path_command( - run_as_uid, - run_as_gid, - "/usr/bin/mkdir", - &["-p", "--", parent.to_str().ok_or("auth path is not UTF-8")?], - )?; - run_provider_path_command( - run_as_uid, - run_as_gid, - "/usr/bin/ln", - &[ - "-sfnT", - "--", - target.to_str().ok_or("auth path is not UTF-8")?, - destination.to_str().ok_or("auth path is not UTF-8")?, - ], - )?; + target + .as_file() + .sync_all() + .map_err(|error| error.to_string())?; + set_mode(target.path(), 0o600)?; + set_owner(target.path(), run_as_uid, run_as_gid)?; + publish_configuration_link(&destination, target.path(), run_as_uid, run_as_gid) + .map_err(str::to_owned)?; + // The accepted link now owns this target for this host lifetime. A + // failed publication instead drops the unselected temporary target. + target.keep().map_err(|error| error.error.to_string())?; } let mut aggregate = 0u64; for (index, (relative, source)) in configuration_files.iter().enumerate() { @@ -2819,22 +3132,42 @@ fn publish_configuration_link( ], ) .map_err(|_| "configuration-file parent creation failed")?; + let temporary = parent.join(format!(".freshell-file-link-{}", uuid::Uuid::new_v4())); + let temporary_name = temporary + .to_str() + .ok_or("configuration-file temporary path is not UTF-8")?; run_provider_path_command( uid, gid, "/usr/bin/ln", &[ - "-sfnT", + "-sT", "--", target .to_str() .ok_or("configuration-file target is not UTF-8")?, + temporary_name, + ], + ) + .map_err(|_| "configuration-file link staging failed")?; + let published = run_provider_path_command( + uid, + gid, + "/usr/bin/mv", + &[ + "-fT", + "--", + temporary_name, destination .to_str() .ok_or("configuration-file destination is not UTF-8")?, ], - ) - .map_err(|_| "configuration-file link publication failed") + ); + if published.is_err() { + run_provider_path_command(uid, gid, "/usr/bin/rm", &["-f", "--", temporary_name]) + .map_err(|_| "configuration-file failed link cleanup failed")?; + } + published.map_err(|_| "configuration-file link publication failed") } fn run_provider_path_command( @@ -3638,6 +3971,146 @@ mod tests { .file_type() .is_symlink()); } + + #[test] + fn provider_config_manifest_rename_failure_restores_every_published_leaf() { + use std::{collections::BTreeSet, fs, os::unix::fs::PermissionsExt}; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let selected = root.path().join("selected"); + fs::create_dir_all(home.join("skills")).unwrap(); + fs::create_dir(&selected).unwrap(); + fs::write( + home.join("skills/command"), + "#!/bin/sh\nprintf old-resource", + ) + .unwrap(); + fs::set_permissions( + home.join("skills/command"), + fs::Permissions::from_mode(0o751), + ) + .unwrap(); + fs::write(home.join("skills/stale"), "prior-owned").unwrap(); + fs::write(home.join("skills/provider-created"), "native-state").unwrap(); + fs::write(selected.join("command"), "#!/bin/sh\nprintf new-resource").unwrap(); + fs::set_permissions(selected.join("command"), fs::Permissions::from_mode(0o751)).unwrap(); + fs::write(selected.join("new"), "new-owned").unwrap(); + let previous = BTreeSet::from([ + "skills".into(), + "skills/command".into(), + "skills/stale".into(), + ]); + let current = BTreeSet::from([ + "skills".into(), + "skills/command".into(), + "skills/new".into(), + ]); + write_projection_manifest(&home, &previous).unwrap(); + let before = fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(); + let occupied = root.path().join("occupied"); + fs::create_dir(&occupied).unwrap(); + let selections = [freshell_platform::provider_environment::CopySelection { + source: selected, + destination_relative: "skills".into(), + }]; + let result = publish_provider_projection_with_manifest( + ProviderProjection { + provider: "codex", + selections: &selections, + home: &home, + previous: &previous, + current: ¤t, + protected: &BTreeSet::new(), + mounted: &[], + }, + |source, _destination| { + // This callback occupies only the final publication boundary. The + // preceding production filesystem operations all execute normally. + assert_eq!( + fs::read(home.join("skills/command")).unwrap(), + b"#!/bin/sh\nprintf new-resource" + ); + assert!(!home.join("skills/stale").exists()); + assert!(home.join("skills/new").exists()); + fs::rename(source, &occupied) + }, + ); + assert!(result.unwrap_err().contains("publish manifest")); + assert_eq!( + fs::read(home.join("skills/command")).unwrap(), + b"#!/bin/sh\nprintf old-resource" + ); + assert_eq!( + fs::metadata(home.join("skills/command")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o751 + ); + assert_eq!(fs::read(home.join("skills/stale")).unwrap(), b"prior-owned"); + assert_eq!( + fs::read(home.join("skills/provider-created")).unwrap(), + b"native-state" + ); + assert!(!home.join("skills/new").exists()); + assert_eq!( + fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(), + before + ); + assert!(!fs::read_dir(&home).unwrap().any(|entry| entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with(".freshell-provider-transaction-"))); + } + + #[test] + fn provider_native_auth_failed_replacement_retains_prior_usable_destination() { + use std::{fs, os::unix::fs::PermissionsExt}; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let auth_dir = root.path().join("auth"); + let relative = ".local/share/opencode/auth.json"; + let auth = home.join(relative); + fs::create_dir_all(auth.parent().unwrap()).unwrap(); + let uid = unsafe { libc::geteuid() }; + let gid = unsafe { libc::getegid() }; + prepare_provider_files_at( + &home, + &auth_dir, + &[(relative, b"prior-typed-auth".to_vec())], + &[], + uid, + gid, + false, + ) + .unwrap(); + let prior_target = fs::read_link(&auth).unwrap(); + fs::set_permissions(auth.parent().unwrap(), fs::Permissions::from_mode(0o500)).unwrap(); + let failed = prepare_provider_files_at( + &home, + &auth_dir, + &[(relative, b"replacement-auth".to_vec())], + &[], + uid, + gid, + false, + ); + fs::set_permissions(auth.parent().unwrap(), fs::Permissions::from_mode(0o700)).unwrap(); + assert!(failed.is_err()); + assert_eq!( + fs::read(&auth).unwrap(), + b"prior-typed-auth", + "failed selected auth publication changed the old destination's contents" + ); + assert_eq!(fs::read_link(&auth).unwrap(), prior_target); + assert_eq!( + fs::read_dir(&auth_dir).unwrap().count(), + 1, + "failed auth publication must remove its new resources" + ); + } use super::*; #[cfg(unix)] diff --git a/crates/freshell-session-host/tests/provider_config_refresh.rs b/crates/freshell-session-host/tests/provider_config_refresh.rs index 99b01e2e7..315969797 100644 --- a/crates/freshell-session-host/tests/provider_config_refresh.rs +++ b/crates/freshell-session-host/tests/provider_config_refresh.rs @@ -1,9 +1,44 @@ +#[cfg(unix)] +fn executable_fixture_root() -> tempfile::TempDir { + match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { + Some(directory) => tempfile::tempdir_in(directory), + None => tempfile::tempdir(), + } + .unwrap() +} + +#[cfg(unix)] +fn write_runnable_plugin(base: &std::path::Path, relative: &str, nonce: &str) -> Vec { + use std::{fs, os::unix::fs::PermissionsExt}; + let plugin = base.join(relative); + fs::create_dir_all(plugin.join("node_modules/selected")).unwrap(); + fs::write(plugin.join("asset"), format!("{nonce}-resource")).unwrap(); + fs::write( + plugin.join("node_modules/selected/index.js"), + format!("module.exports='{}-dependency'", nonce), + ) + .unwrap(); + fs::write(plugin.join("command"), "#!/usr/bin/env node\nprocess.stdout.write(require('selected')+':'+require('node:fs').readFileSync(__dirname+'/asset','utf8'));\n").unwrap(); + fs::set_permissions(plugin.join("command"), fs::Permissions::from_mode(0o751)).unwrap(); + [ + "", + "/command", + "/asset", + "/node_modules", + "/node_modules/selected", + "/node_modules/selected/index.js", + ] + .into_iter() + .map(|suffix| format!("{relative}{suffix}")) + .collect() +} + #[cfg(unix)] #[test] fn provider_uid_refresh_helper_updates_projection_without_erasing_provider_state() { use std::{fs, os::unix::fs::PermissionsExt, process::Command}; - let root = tempfile::tempdir().unwrap(); + let root = executable_fixture_root(); fs::set_permissions(root.path(), fs::Permissions::from_mode(0o755)).unwrap(); let home = root.path().join("provider-home"); let stage = root.path().join("stage"); @@ -14,16 +49,37 @@ fn provider_uid_refresh_helper_updates_projection_without_erasing_provider_state fs::write(plugins.join("old.js"), b"old").unwrap(); fs::write(plugins.join("provider-created.js"), b"durable").unwrap(); fs::write(staged_plugins.join("new.js"), b"new").unwrap(); + let mut previous = write_runnable_plugin(&home, ".claude/plugins/runtime", "old"); + previous.extend([ + ".claude".into(), + ".claude/plugins".into(), + ".claude/plugins/old.js".into(), + ]); + let mut current = + write_runnable_plugin(&stage.join("projection"), ".claude/plugins/runtime", "new"); + current.extend([ + ".claude".into(), + ".claude/plugins".into(), + ".claude/plugins/new.js".into(), + ]); fs::write( home.join(".freshell-config-projection.json"), - br#"[".claude",".claude/plugins",".claude/plugins/old.js"]"#, + serde_json::to_vec(&previous).unwrap(), ) .unwrap(); fs::write( stage.join("projection/.freshell-config-projection.json"), - br#"[".claude",".claude/plugins",".claude/plugins/new.js"]"#, + serde_json::to_vec(¤t).unwrap(), ) .unwrap(); + let execute = || { + let output = Command::new(plugins.join("runtime/command")) + .output() + .unwrap(); + assert!(output.status.success()); + output.stdout + }; + assert_eq!(execute(), b"old-dependency:old-resource"); fs::write(stage.join("manifest.json"), br#"[".claude/plugins"]"#).unwrap(); let uid = unsafe { libc::geteuid() }; @@ -48,6 +104,7 @@ fn provider_uid_refresh_helper_updates_projection_without_erasing_provider_state status.success(), "provider UID/GID refresh helper failed: {status}" ); + assert_eq!(execute(), b"new-dependency:new-resource"); assert!(!plugins.join("old.js").exists()); assert_eq!(fs::read(plugins.join("new.js")).unwrap(), b"new"); assert_eq!( @@ -100,7 +157,11 @@ fn provider_config_spawned_helper_emits_safe_jsonl_for_success_and_typed_failure .collect() }; let events = parse(&success.stderr); - assert_eq!(events.len(), 2, "copy logs must be per-operation"); + assert_eq!( + events.len(), + 4, + "copy and transaction logs must be per-operation" + ); for event in &events { eprintln!("{event}"); } @@ -129,7 +190,11 @@ fn provider_config_spawned_helper_emits_safe_jsonl_for_success_and_typed_failure assert!(!failed.status.success()); assert!(failed.stdout.is_empty()); let events = parse(&failed.stderr); - assert_eq!(events.len(), 3, "start, typed failure and final fatal only"); + assert_eq!( + events.len(), + 5, + "bounded transaction/copy failure and final fatal only" + ); for event in &events { eprintln!("{event}"); } @@ -306,3 +371,214 @@ fn provider_config_preparation_worker_consumes_bounded_private_control_and_clean accepted ); } +#[cfg(unix)] +#[test] +fn provider_config_manifest_failure_preserves_runnable_previous_projection() { + use std::{ + fs, + os::unix::{fs::PermissionsExt, process::CommandExt}, + process::Command, + }; + let root = executable_fixture_root(); + let home = root.path().join("home"); + let stage = root.path().join("stage"); + write_runnable_plugin(&home, "plugin", "old"); + write_runnable_plugin(&stage.join("projection"), "plugin", "new"); + fs::write(home.join("plugin/stale"), "old-owned").unwrap(); + fs::write(home.join("plugin/provider-created"), "native-state").unwrap(); + let previous = [ + "plugin", + "plugin/command", + "plugin/asset", + "plugin/stale", + "plugin/node_modules", + "plugin/node_modules/selected", + "plugin/node_modules/selected/index.js", + ]; + let manifest = serde_json::to_vec(&previous).unwrap(); + fs::write(home.join(".freshell-config-projection.json"), &manifest).unwrap(); + let mut current = previous + .iter() + .filter(|path| **path != "plugin/stale") + .map(|path| path.to_string()) + .collect::>(); + for index in 0..40 { + let relative = format!("plugin/additional-resource-{index:03}"); + fs::write(stage.join("projection").join(&relative), "new").unwrap(); + current.push(relative); + } + fs::write( + stage.join("projection/.freshell-config-projection.json"), + serde_json::to_vec(¤t).unwrap(), + ) + .unwrap(); + fs::write(stage.join("manifest.json"), br#"["plugin"]"#).unwrap(); + let mut command = Command::new(env!("CARGO_BIN_EXE_freshell-session-host")); + command + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home); + // A real filesystem write fault, scoped to the disposable child. Every + // resource fits; the complete new manifest does not. No production hook. + unsafe { + command.pre_exec(|| { + libc::signal(libc::SIGXFSZ, libc::SIG_IGN); + let limit = libc::rlimit { + rlim_cur: 512, + rlim_max: 512, + }; + if libc::setrlimit(libc::RLIMIT_FSIZE, &limit) != 0 { + return Err(std::io::Error::last_os_error()); + } + Ok(()) + }); + } + let output = command.output().unwrap(); + assert!( + !output.status.success(), + "manifest fault must reject refresh" + ); + let executed = Command::new(home.join("plugin/command")).output().unwrap(); + assert!(executed.status.success()); + assert_eq!( + executed.stdout, b"old-dependency:old-resource", + "failed refresh replaced the usable environment" + ); + assert_eq!( + fs::read(home.join(".freshell-config-projection.json")).unwrap(), + manifest + ); + assert_eq!(fs::read(home.join("plugin/stale")).unwrap(), b"old-owned"); + assert_eq!( + fs::read(home.join("plugin/provider-created")).unwrap(), + b"native-state" + ); + assert!(!home.join("plugin/additional-resource-000").exists()); + // Fail after earlier manifest-owned leaves have moved into backup. The + // inaccessible stale leaf is ordered last, so this exercises rollback. + fs::create_dir(home.join("zzblocked")).unwrap(); + fs::write(home.join("zzblocked/stale"), "retained-stale").unwrap(); + let mut extended = previous + .iter() + .map(|path| path.to_string()) + .collect::>(); + extended.extend(["zzblocked".into(), "zzblocked/stale".into()]); + let extended_manifest = serde_json::to_vec(&extended).unwrap(); + fs::write( + home.join(".freshell-config-projection.json"), + &extended_manifest, + ) + .unwrap(); + fs::set_permissions(home.join("zzblocked"), fs::Permissions::from_mode(0o500)).unwrap(); + let denied = Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home) + .output() + .unwrap(); + fs::set_permissions(home.join("zzblocked"), fs::Permissions::from_mode(0o700)).unwrap(); + assert!( + !denied.status.success(), + "stale leaf permission failure must block refresh" + ); + assert_eq!( + Command::new(home.join("plugin/command")) + .output() + .unwrap() + .stdout, + b"old-dependency:old-resource" + ); + assert_eq!( + fs::read(home.join(".freshell-config-projection.json")).unwrap(), + extended_manifest + ); + assert_eq!( + fs::read(home.join("zzblocked/stale")).unwrap(), + b"retained-stale" + ); + let events = String::from_utf8(denied.stderr).unwrap(); + assert!(events.contains("provider.projection.transaction.rollback")); + assert!(events.contains("\"successful\":true")); + // An unreadable selected file must fail during staging, with no mutation. + fs::set_permissions( + stage.join("projection/plugin/asset"), + fs::Permissions::from_mode(0o000), + ) + .unwrap(); + let unreadable = Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home) + .output() + .unwrap(); + fs::set_permissions( + stage.join("projection/plugin/asset"), + fs::Permissions::from_mode(0o600), + ) + .unwrap(); + assert!(!unreadable.status.success()); + assert_eq!( + Command::new(home.join("plugin/command")) + .output() + .unwrap() + .stdout, + b"old-dependency:old-resource" + ); + assert_eq!( + fs::read(home.join(".freshell-config-projection.json")).unwrap(), + extended_manifest + ); + let accepted = Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home) + .output() + .unwrap(); + assert!(accepted.status.success(), "{accepted:?}"); + assert_eq!( + Command::new(home.join("plugin/command")) + .output() + .unwrap() + .stdout, + b"new-dependency:new-resource" + ); + assert!(!home.join("plugin/stale").exists()); + assert_eq!( + fs::read(home.join("plugin/provider-created")).unwrap(), + b"native-state" + ); +} + +#[cfg(unix)] +#[test] +fn provider_config_selected_collision_cannot_replace_provider_created_leaf() { + use std::{fs, process::Command}; + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let stage = root.path().join("stage"); + fs::create_dir_all(home.join("skills")).unwrap(); + fs::create_dir_all(stage.join("projection/skills")).unwrap(); + fs::write(home.join("skills/provider-created"), "native-state").unwrap(); + fs::write(stage.join("projection/skills/provider-created"), "incoming").unwrap(); + fs::write(stage.join("manifest.json"), br#"["skills"]"#).unwrap(); + fs::write( + stage.join("projection/.freshell-config-projection.json"), + br#"["skills","skills/provider-created"]"#, + ) + .unwrap(); + let result = Command::new(env!("CARGO_BIN_EXE_freshell-session-host")) + .arg("refresh-provider-config") + .arg(&stage) + .arg(&home) + .output() + .unwrap(); + assert!( + !result.status.success(), + "unowned leaf collision must block before publication" + ); + assert_eq!( + fs::read(home.join("skills/provider-created")).unwrap(), + b"native-state" + ); + assert!(!home.join(".freshell-config-projection.json").exists()); +} diff --git a/crates/freshell-supervisor/src/backend.rs b/crates/freshell-supervisor/src/backend.rs index b27840922..3b3d6b1fd 100644 --- a/crates/freshell-supervisor/src/backend.rs +++ b/crates/freshell-supervisor/src/backend.rs @@ -1161,7 +1161,7 @@ fn runtime_tmpfs(mode: Option<&str>) -> std::collections::BTreeMap { }) describe('managed provider tmpfs', () => { + it('dispatches exact 0710 private parents and refuses mode or authority changes before Docker', async () => { + await withHttpBroker(async ({ request, forwarded }) => { + const base = { '/tmp': 'rw,noexec,nosuid,nodev,size=128m' } + const privateParent = 'rw,noexec,nosuid,nodev,size=16m,mode=0710' + for (const option of [ + privateParent.replace('0710', '0700'), + privateParent.replace('0710', '0750'), + `${privateParent},uid=65534`, + privateParent.replace('noexec', 'exec'), + ]) { + const before = forwarded.length + expect((await request('POST', '/v1.47/containers/create?name=owned-runtime', createBody( + [...ordinaryBinds, actorBind], { ...base, '/run/freshell-private': option }, + ))).status).toBe(403) + expect(forwarded).toHaveLength(before) + } + expect((await request('POST', '/v1.47/containers/create?name=owned-runtime', createBody( + [...ordinaryBinds, actorBind], { ...base, '/run/freshell-private': privateParent }, + ))).status).toBe(201) + }) + }) it('accepts the bounded private provider mount and the OpenCode exec mount', () => { const validate = (body: Buffer): { ok: boolean } => ( broker() as unknown as { validateCreate(body: Buffer): { ok: boolean } } ).validateCreate(body) const base = { '/tmp': 'rw,noexec,nosuid,nodev,size=128m' } - const privateTmp = { '/run/freshell-private': 'rw,noexec,nosuid,nodev,size=16m,mode=0700' } + const privateTmp = { '/run/freshell-private': 'rw,noexec,nosuid,nodev,size=16m,mode=0710' } const opencodeTmp = { '/run/opencode-tmp': 'rw,exec,nosuid,nodev,size=64m,mode=1777' } const binds = [...ordinaryBinds, actorBind] expect(validate(createBody(binds, { ...base, ...privateTmp })).ok).toBe(true) + for (const mode of ['0700', '0750', '0711', '0777']) { + expect(validate(createBody(binds, { ...base, '/run/freshell-private': `rw,noexec,nosuid,nodev,size=16m,mode=${mode}` })).ok).toBe(false) + } + expect(validate(createBody(binds, { ...base, '/run/freshell-private': 'rw,noexec,nosuid,nodev,size=16m,mode=0710,uid=65534' })).ok).toBe(false) expect(validate(createBody(binds, { ...base, ...privateTmp, ...opencodeTmp })).ok).toBe(true) - expect(validate(createBody(binds, { ...base, '/run/freshell-private': 'rw,exec,nosuid,nodev,size=16m,mode=0700' })).ok).toBe(false) + expect(validate(createBody(binds, { ...base, '/run/freshell-private': 'rw,exec,nosuid,nodev,size=16m,mode=0710' })).ok).toBe(false) expect(validate(createBody(binds, { ...base, ...privateTmp, '/run/unapproved': 'rw' })).ok).toBe(false) }) }) From 0822c40be058f24a5d368aeac5b28c13abb0d2fe Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:05:50 -0700 Subject: [PATCH 223/493] fix: refresh provider configuration through physical resource roots --- crates/freshell-session-host/src/main.rs | 232 +++++++++++++++++++++-- 1 file changed, 221 insertions(+), 11 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 011d0c125..8a70003e6 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1784,10 +1784,14 @@ fn copy_provider_config_references_with_ownership( }; CopySelection { source: root.join(&reference.relative_path), - destination_relative: reference.provider_relative_path.clone().into(), + destination_relative: physical_provider_destination( + context, + &reference.provider_relative_path, + ), } }) .collect::>(); + let config_selection_count = selections.len(); selections.extend( context .prepared_inputs @@ -1800,6 +1804,37 @@ fn copy_provider_config_references_with_ownership( ); let protected = private_bound_destinations(context); let mut excluded = provider_credential_sources(user_provider_root); + // A prepared config leaf is the selected view of the raw file still held + // by its complete resource directory. Copy that view at its typed physical + // destination, retaining every other resource and the shared conflict policy. + for config in &selections[..config_selection_count] { + if !std::fs::metadata(&config.source).is_ok_and(|metadata| metadata.is_file()) { + continue; + } + let Ok(selected_source) = std::fs::canonicalize(&config.source) else { + continue; + }; + for resource in &selections[config_selection_count..] { + let Ok(suffix) = config + .destination_relative + .strip_prefix(&resource.destination_relative) + else { + continue; + }; + if suffix.as_os_str().is_empty() { + continue; + } + let original = resource.source.join(suffix); + if !std::fs::metadata(&original).is_ok_and(|metadata| metadata.is_file()) { + continue; + } + if let Ok(original) = std::fs::canonicalize(original) { + if original != selected_source { + excluded.push(original); + } + } + } + } for selection in &selections { for relative in &protected { if let Ok(suffix) = Path::new(relative).strip_prefix(&selection.destination_relative) { @@ -1853,6 +1888,69 @@ fn resource_mount_roots( .collect() } +// A resource's native HOME alias is a separate volume-subpath mount. Publish +// through its declared physical HOME path, where the transaction and journal +// share a filesystem. Every selected source still goes through the copier. +fn physical_provider_destination( + context: &freshell_runtime_protocol::ProviderLaunchContext, + relative: &str, +) -> PathBuf { + use freshell_runtime_protocol::ProviderResourceKind; + let destination = Path::new(relative); + context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .filter_map(|unit| { + let alias = Path::new(&unit.effective_destination) + .strip_prefix("/home/freshell/provider") + .ok()?; + if alias.as_os_str().is_empty() { + return None; + } + let suffix = destination.strip_prefix(alias).ok()?; + if unit.kind == ProviderResourceKind::File && !suffix.as_os_str().is_empty() { + return None; + } + Some(( + alias.components().count(), + if suffix.as_os_str().is_empty() { + PathBuf::from(&unit.provider_relative_path) + } else { + Path::new(&unit.provider_relative_path).join(suffix) + }, + )) + }) + .max_by_key(|(depth, _)| *depth) + .map_or_else(|| destination.to_path_buf(), |(_, physical)| physical) +} + +fn protected_resource_alias_entries( + context: &freshell_runtime_protocol::ProviderLaunchContext, + entries: &std::collections::BTreeSet, +) -> Vec { + let aliases = context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .filter_map(|unit| { + Path::new(&unit.effective_destination) + .strip_prefix("/home/freshell/provider") + .ok() + .filter(|alias| !alias.as_os_str().is_empty()) + }) + .collect::>(); + entries + .iter() + .filter(move |relative| { + aliases + .iter() + .any(|alias| Path::new(relative).starts_with(alias)) + }) + .cloned() + .collect() +} + fn provider_credential_sources(root: &Path) -> Vec { [ ".credentials.json", @@ -2095,7 +2193,11 @@ fn refresh_provider_config_in_stage( )?; let mut destinations = provider_config_references(context) .iter() - .map(|reference| reference.provider_relative_path.clone()) + .map(|reference| { + physical_provider_destination(context, &reference.provider_relative_path) + .to_string_lossy() + .into_owned() + }) .collect::>(); destinations.extend( context @@ -2116,13 +2218,18 @@ fn refresh_provider_config_in_stage( .map_err(|_| "provider projection identity exceeds 64 MiB")?, ) .map_err(|error| error.to_string())?; + // Legacy manifests may name native aliases. Those entries confer no stale + // removal authority across a mount boundary; retain their files and inodes. + // Store only exact protected entries in the existing bounded metadata. + let mut protected = private_bound_destinations(context); + let previous = read_projection_manifest(provider_home)?; + let current = read_projection_manifest(&projection)?; + protected.extend(protected_resource_alias_entries(context, &previous)); + protected.extend(protected_resource_alias_entries(context, ¤t)); std::fs::write( stage.path().join("private-mountpoints.json"), - freshell_runtime_protocol::bounded_json_bytes( - &private_bound_destinations(context), - MAX_PROJECTION_METADATA_BYTES, - ) - .map_err(|_| "private mountpoint metadata exceeds 64 MiB")?, + freshell_runtime_protocol::bounded_json_bytes(&protected, MAX_PROJECTION_METADATA_BYTES) + .map_err(|_| "private mountpoint metadata exceeds 64 MiB")?, ) .map_err(|_| "private mountpoint metadata write failed")?; let mounted = freshell_runtime_protocol::bounded_json_bytes( @@ -2429,7 +2536,9 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() // selected skills alias. The common copier resolves those routes from // the staged graph, then validates every physical publication parent. // Manifest entries instead name physical inodes used for stale removal. - if previous.contains(relative) || current.contains(relative) { + if !protected.contains(relative) + && (previous.contains(relative) || current.contains(relative)) + { let mut parent = provider_home.to_path_buf(); for part in path.parent().into_iter().flat_map(Path::components) { parent.push(part.as_os_str()); @@ -4300,8 +4409,27 @@ mod tests { "dependency-initial", ) .unwrap(); + // The selected template is the prepared view; its containing resource + // unit still supplies the original, unprepared config leaf. + std::fs::write(unit.join("opencode.jsonc"), r#"{"rules":"raw-resource"}"#).unwrap(); + let selected_config = source.join(".freshell/selected-config.json"); + std::fs::write(&selected_config, r#"{"rules":"prepared-initial"}"#).unwrap(); std::fs::write(destination.join("native-owned-sibling"), "preserved").unwrap(); let inode = std::fs::metadata(&destination).unwrap().ino(); + let alias = home.join(".config/opencode"); + std::fs::create_dir_all(&alias).unwrap(); + std::fs::write(alias.join("provider-state"), "native-alias-state").unwrap(); + let alias_inode = std::fs::metadata(&alias).unwrap().ino(); + // The native alias is a separate volume-subpath mount. A prior logical + // manifest must not authorize publication or stale removal through it. + write_projection_manifest( + &home, + &std::collections::BTreeSet::from([ + ".config/opencode".into(), + ".config/opencode/provider-state".into(), + ]), + ) + .unwrap(); let capability = McpCapabilityReference { grant_id: format!("grant-{}", uuid::Uuid::new_v4()), endpoint: "http://host.docker.internal:1234".into(), @@ -4323,14 +4451,27 @@ mod tests { resource_mounts: vec![ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-0".into(), provider_relative_path: ".freshell/provider-roots/root-0".into(), - logical_destination: "/synthetic/outside/.opencode".into(), - effective_destination: "/synthetic/outside/.opencode".into(), + logical_destination: "/home/freshell/provider/.config/opencode".into(), + effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, }], indexed_files: None, }), mcp_capability: Some(capability), - config: vec![], + config: vec![ + ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".config/opencode".into(), + format: "directory".into(), + }, + ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: ".freshell/selected-config.json".into(), + provider_relative_path: ".config/opencode/opencode.jsonc".into(), + format: "jsonc".into(), + }, + ], preparation: ProviderPreparation::Opencode { project_config: vec![], tui_config: None, @@ -4340,6 +4481,31 @@ mod tests { }; let uid = unsafe { libc::geteuid() }; let gid = unsafe { libc::getegid() }; + copy_provider_config_references( + &context, + "opencode", + owned.path(), + &source, + &home, + uid, + gid, + ) + .unwrap(); + let config = destination.join("opencode.jsonc"); + assert_eq!( + serde_json::from_slice::(&std::fs::read(&config).unwrap()).unwrap() + ["rules"], + "prepared-initial" + ); + // Initial publication writes its current manifest. Restore the legacy + // alias entries so refresh still exercises their lack of authority over + // the separately mounted native alias and its provider-created state. + let mut legacy = read_projection_manifest(&home).unwrap(); + legacy.extend([ + ".config/opencode".to_string(), + ".config/opencode/provider-state".to_string(), + ]); + write_projection_manifest(&home, &legacy).unwrap(); refresh_provider_config_references( &context, "opencode", @@ -4359,6 +4525,7 @@ mod tests { "dependency-refreshed", ) .unwrap(); + std::fs::write(&selected_config, r#"{"rules":"prepared-refreshed"}"#).unwrap(); refresh_provider_config_references( &context, "opencode", @@ -4373,11 +4540,54 @@ mod tests { std::fs::read_to_string(destination.join("node_modules/dependency/index.js")).unwrap(), "dependency-refreshed" ); + assert_eq!( + serde_json::from_slice::(&std::fs::read(&config).unwrap()).unwrap() + ["rules"], + "prepared-refreshed" + ); assert_eq!(std::fs::metadata(&destination).unwrap().ino(), inode); + assert_eq!(std::fs::metadata(&alias).unwrap().ino(), alias_inode); + assert_eq!( + std::fs::read_to_string(alias.join("provider-state")).unwrap(), + "native-alias-state" + ); + assert!(!read_projection_manifest(&home) + .unwrap() + .iter() + .any(|entry| Path::new(entry).starts_with(".config/opencode"))); assert_eq!( std::fs::read_to_string(destination.join("native-owned-sibling")).unwrap(), "preserved" ); + // A valid resource unit cannot mask a missing independently selected + // configuration reference, even when both map to the same physical root. + let manifest = std::fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(); + let mut missing = context.clone(); + missing.config[1].relative_path = ".freshell/missing-selected-config".into(); + assert!(refresh_provider_config_references( + &missing, + "opencode", + owned.path(), + &source, + &home, + uid, + gid, + ) + .is_err()); + assert_eq!( + std::fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(), + manifest + ); + assert_eq!( + std::fs::read_to_string(destination.join("node_modules/dependency/index.js")).unwrap(), + "dependency-refreshed" + ); + assert_eq!(std::fs::metadata(&destination).unwrap().ino(), inode); + assert_eq!(std::fs::metadata(&alias).unwrap().ino(), alias_inode); + assert_eq!( + std::fs::read_to_string(alias.join("provider-state")).unwrap(), + "native-alias-state" + ); } #[test] From 04d6213cb176afe73c8e0414d9f96d08512ac2bc Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:05:55 -0700 Subject: [PATCH 224/493] fix: bind selected global configuration at its resource destination --- .../src/managed_mcp_capability.rs | 152 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 21 ++- 2 files changed, 172 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index b9a5dcb09..babcb2de1 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -10219,6 +10219,158 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .all(|(_, mount)| mount.provider_relative_path == units[0].1.provider_relative_path)); } + #[test] + fn managed_provider_global_resource_config_references_resolve_at_native_path() { + use crate::managed_provider_references::sources::*; + use freshell_platform::provider_environment::{ + copy_environment, CopyLimits, CopySelection, + }; + use freshell_runtime_protocol::*; + for config_parent in [".config", "custom-xdg"] { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let global = home.join(config_parent).join("opencode"); + std::fs::create_dir_all(&global).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let relative_policy = global.join("relative-policy.txt"); + let absolute_policy = home.join("absolute-policy.txt"); + let plugin = global.join("selected-plugin.js"); + std::fs::write(&plugin, "export default {};\n").unwrap(); + std::fs::write(&relative_policy, "relative-policy-nonce").unwrap(); + std::fs::write(&absolute_policy, "absolute-policy-nonce").unwrap(); + let raw = serde_json::json!({"plugin":[format!("file://{}", plugin.display())], "agent":{"probe":{ + "prompt":"{file:./relative-policy.txt}", + "description":format!("{{file:{}}}", absolute_policy.display()) + }}}) + .to_string(); + let configuration = global.join("opencode.jsonc"); + std::fs::write(&configuration, &raw).unwrap(); + let discovery = OpencodeDiscovery { + home: &home, + cwd: &workspace, + worktree: &workspace, + config_root: &global, + env: &|_| None, + }; + let roots = select_opencode_resource_roots(&discovery).unwrap(); + let plan = plan_opencode_sources( + discover_opencode(discovery).unwrap(), + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let references = [ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "opencode.jsonc".into(), + provider_relative_path: ".config/opencode/opencode.jsonc".into(), + format: "jsonc".into(), + }]; + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let grant = CapabilityStore::default() + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &references)), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + config_root: Some(&global), + source_plan: Some(&plan), + resource_roots: Some(&roots), + secret_references: Some(&[]), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &grant.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let guest = owned.path().join("guest-home"); + let runtime_home = Path::new("/home/freshell/provider"); + let selections = inputs + .resource_mounts + .iter() + .filter_map(|unit| { + let destination = Path::new(&unit.effective_destination) + .strip_prefix(runtime_home) + .ok()?; + Some(CopySelection { + source: stage.join(&unit.stage_relative_path), + destination_relative: destination.into(), + }) + }) + .collect::>(); + copy_environment(&selections, &guest, &[], CopyLimits::default()).unwrap(); + // Model the exact declared file mounts in an owned temporary HOME. + // A config bind at a different filename cannot change the native + // global file that the consumer subsequently loads. + for bind in &inputs.config_binds { + let destination = guest.join( + Path::new(&bind.effective_destination) + .strip_prefix(runtime_home) + .unwrap(), + ); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + std::fs::copy(stage.join(&bind.source_relative_path), destination).unwrap(); + } + let manifest: freshell_platform::opencode_recipe::SourceManifest = + serde_json::from_slice( + &std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + for file in &manifest.files { + let destination = guest.join(format!(".freshell/references/{}", file.source_id)); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + std::fs::copy(&file.canonical_source, destination).unwrap(); + } + let native = guest.join(".config/opencode/opencode.jsonc"); + let selected: serde_json::Value = + serde_json::from_slice(&std::fs::read(&native).unwrap()).unwrap(); + let consumed = ["prompt", "description"].map(|field| { + let token = selected["agent"]["probe"][field].as_str().unwrap(); + let path = Path::new( + token + .strip_prefix("{file:") + .unwrap() + .strip_suffix('}') + .unwrap(), + ); + let source = if path.is_absolute() { + guest.join(path.strip_prefix(runtime_home).unwrap()) + } else { + native.parent().unwrap().join(path) + }; + std::fs::read_to_string(source) + .expect("native global config must resolve every selected reference") + }); + assert_eq!(consumed, ["relative-policy-nonce", "absolute-policy-nonce"]); + assert_eq!(std::fs::read_to_string(&configuration).unwrap(), raw); + std::fs::remove_file(relative_policy).unwrap(); + assert!( + plan_opencode_sources( + vec![SourceOccurrence { + logical_path: configuration.clone(), + canonical_source: configuration, + lane: SourceLane::Main, + role: DiscoveryRole::Global + }], + &workspace, + &home, + &|_, _, _| None, + ) + .is_err(), + "a missing selected reference remains an error" + ); + } + } + #[test] fn managed_provider_home_workspace_discovery_keeps_project_origins() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 3cbfb75be..d0e35fe09 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1247,7 +1247,26 @@ pub(super) fn describe( let relative = physical .strip_prefix(root) .map_err(|_| "selected global config escaped its owned stage")?; - if relative.starts_with(".freshell/resources") { + if let Some((unit, suffix)) = resource_mounts.iter().find_map(|unit| { + if !destination.starts_with(&unit.logical_destination) { + return None; + } + let suffix = relative.strip_prefix(&unit.stage_relative_path).ok()?; + if unit.kind == ProviderResourceKind::File && !suffix.as_os_str().is_empty() { + return None; + } + Some((unit, suffix)) + }) { + // The shared copier can place a global config in its + // already selected resource unit. Its private template + // must cover that unit's native alias, not a new nested + // filename containing the controller's stage path. + if suffix.as_os_str().is_empty() { + Path::new(&unit.effective_destination).to_owned() + } else { + Path::new(&unit.effective_destination).join(suffix) + } + } else if relative.starts_with(".freshell/resources") { runtime_home.join(relative) } else { runtime_home.join(".config/opencode").join(relative) From a46da8f51a6ebc43fd425faaae355e1a5766da5c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:06:07 -0700 Subject: [PATCH 225/493] test: exercise complete provider environments through the restricted runtime --- .../src/managed_provider_launch_tests.rs | 139 ++++- scripts/testing/runtime-sandbox.ts | 75 ++- scripts/testing/runtime-test-broker.ts | 346 ++++++++++-- test/e2e-browser/helpers/managed-runtime.ts | 4 +- test/fixtures/providers/parity-provider.mjs | 287 ++++++---- .../server/managed-provider-parity.test.ts | 210 +++++-- .../server/provider-parity-broker.ts | 80 +++ .../server/provider-parity-environment.ts | 103 ++++ .../tooling/testing/runtime-sandbox.test.ts | 290 ++++++++++ .../testing/runtime-test-broker.test.ts | 533 +++++++++++++++++- 10 files changed, 1858 insertions(+), 209 deletions(-) create mode 100644 test/integration/server/provider-parity-broker.ts create mode 100644 test/integration/server/provider-parity-environment.ts create mode 100644 test/unit/tooling/testing/runtime-sandbox.test.ts diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 16985bfff..230727841 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -16,6 +16,85 @@ use std::{ time::Duration, }; +#[test] +fn provider_parity_explicit_opencode_roots() { + use crate::managed_provider_references::sources::{ + discover_opencode, plan_opencode_sources, select_opencode_resource_roots, OpencodeDiscovery, + }; + let owned = tempfile::tempdir().unwrap(); + let workspace = owned.path().join("workspace"); + let project = workspace.join(".opencode"); + let ancestor = owned.path().join(".opencode"); + let home = owned.path().join("home"); + let global = home.join(".config/opencode"); + for directory in [&project, &ancestor, &global] { + std::fs::create_dir_all(directory).unwrap(); + } + let plugin = workspace.join("selected-plugin.js"); + std::fs::write(&plugin, "export default {};\n").unwrap(); + let configuration = serde_json::to_vec(&json!({"plugin":[plugin]})).unwrap(); + std::fs::write(workspace.join("opencode.jsonc"), &configuration).unwrap(); + let project_config = project.join("opencode.json"); + std::fs::write(&project_config, &configuration).unwrap(); + let ancestor_marker = ancestor.join("unrelated-resource"); + std::fs::write(&ancestor_marker, "retain unrelated ancestor bytes").unwrap(); + + // Exercise the actual TypeScript fixture environment through a finite child, + // then pass its values to the real native source producer. + let repo = Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap(); + let helper = std::process::Command::new("pnpm") + .args(["exec", "tsx", "--eval", "import(process.argv[1]).then(module => process.stdout.write(JSON.stringify(module.providerParityCommands(process.argv[2]))))"]) + .arg(repo.join("test/integration/server/provider-parity-environment.ts")) + .arg(&workspace) + .current_dir(repo) + .output().unwrap(); + assert!( + helper.status.success(), + "finite fixture environment producer failed" + ); + let environment: BTreeMap = serde_json::from_slice(&helper.stdout).unwrap(); + let lookup = |key: &str| environment.get(key).cloned(); + let request = || OpencodeDiscovery { + home: &home, + cwd: &workspace, + worktree: &workspace, + config_root: &global, + env: &lookup, + }; + let sources = discover_opencode(request()).unwrap(); + for original in [workspace.join("opencode.jsonc"), project_config.clone()] { + assert!(sources.iter().any(|source| source.logical_path == original)); + } + let plan = + plan_opencode_sources(sources, &workspace, &home, &|_, _, name| lookup(name)).unwrap(); + assert!(plan + .iter() + .any(|source| source.configured_resources.contains(&workspace))); + let roots = select_opencode_resource_roots(&request()).unwrap(); + assert!(roots.iter().any(|root| root.logical_path == project)); + assert_eq!( + std::fs::read_to_string(&ancestor_marker).unwrap(), + "retain unrelated ancestor bytes" + ); + assert!( + !roots.iter().any(|root| root.logical_path == ancestor), + "fixture selected an unrelated ancestor outside its owned project" + ); + + // A broken selected source remains an error; isolating discovery does not + // make missing selected inputs acceptable. + std::fs::remove_file(&project_config).unwrap(); + #[cfg(unix)] + { + std::os::unix::fs::symlink(project.join("missing-selected"), &project_config).unwrap(); + assert!(discover_opencode(request()).is_err()); + } +} + // Read only the fixture's actual native process (or its bounded hook ancestry). // argv/environment bytes stay transient; only profile compliance is reported. const NATIVE_EXEC_PROFILE: &str = r#" @@ -711,6 +790,20 @@ async fn managed_provider_amplifier_terminal_native_startup_consumes_prepared_in .unwrap(); std::fs::write(skill.join("run"),"#!/usr/bin/env node\nconst fs=require('node:fs'),path=require('node:path');process.stdout.write(fs.readFileSync(path.join(__dirname,'resource'),'utf8')+':'+require('probe-dependency'));").unwrap(); std::fs::set_permissions(skill.join("run"), std::fs::Permissions::from_mode(0o755)).unwrap(); + let large_resources = skill.join("node_modules/probe-dependency/selected-resources"); + std::fs::create_dir(&large_resources).unwrap(); + for index in 0..10_005 { + std::fs::write( + large_resources.join(format!("selected-{index:05}")), + b"complete-selected-resource", + ) + .unwrap(); + } + let source_owner_before = std::fs::symlink_metadata(skill.join("run")).unwrap(); + let outside_target = workspace.join("task3-unrelated-target"); + std::fs::write(&outside_target, b"unrelated-target-nonce").unwrap(); + std::fs::set_permissions(&outside_target, std::fs::Permissions::from_mode(0o444)).unwrap(); + let outside_before = std::fs::symlink_metadata(&outside_target).unwrap(); let context_module = r#"__amplifier_module_type__='context' import hashlib,json from pathlib import Path @@ -748,6 +841,28 @@ async def mount(coordinator,config=None): async def consume(event,data): uv=subprocess.check_output(['uv','--version'],text=True).strip() output=subprocess.check_output([str(Path.home()/'.amplifier/skills/probe/run')]) + resources=Path.home()/'.amplifier/skills/probe/node_modules/probe-dependency/selected-resources' + eligible=list(resources.iterdir()) + if len(eligible)!=10005 or any(p.read_bytes()!=b'complete-selected-resource' or p.lstat().st_uid!=65534 or p.lstat().st_gid!=0 for p in eligible): raise RuntimeError('complete large selected projection unavailable') + sibling=Path.home()/'.amplifier/skills/provider-created-sibling' + link=Path.home()/'.amplifier/skills/provider-created-link' + target=Path(config['receipt']).parent/'task3-unrelated-target' + ownership_receipt=Path(config['receipt']).with_suffix('.ownership.json') + def physical(p): + stat=p.lstat() + return [stat.st_ino,stat.st_uid,stat.st_gid,stat.st_mode & 0o777] + if not ownership_receipt.exists(): + sibling.write_bytes(b'provider-created-sibling-nonce') + sibling.chmod(0o640) + link.symlink_to(target) + ownership_receipt.write_text(json.dumps({'sibling':physical(sibling),'link':physical(link),'target':physical(target)})) + preserved=json.loads(ownership_receipt.read_text()) + if sibling.read_bytes()!=b'provider-created-sibling-nonce' or link.read_bytes()!=b'unrelated-target-nonce' or preserved!={'sibling':physical(sibling),'link':physical(link),'target':physical(target)}: raise RuntimeError('adopted provider state or unrelated target ownership changed') + stub=Path.home()/'.amplifier/projects'/str(Path.cwd().resolve()).replace('/','-').replace('\\','-').replace(':','')/'sessions'/coordinator.session_id + if not stub.is_dir() or any(p.lstat().st_uid!=65534 for p in [stub,stub/'metadata.json',stub/'transcript.jsonl',stub/'events.jsonl']): raise RuntimeError('exact Amplifier stub not provider owned') + private=Path('/run/freshell-private').lstat() + if (os.getuid(),os.getgid(),private.st_uid,private.st_gid,private.st_mode & 0o777)!=(65534,0,0,0,0o710): raise RuntimeError('exact provider private-parent topology unavailable') + selected=yaml.safe_load((Path.home()/'.amplifier/settings.yaml').read_text())['bundle']['active'] loaded=await load_bundle(selected) loaded.resolve_pending_context() @@ -769,7 +884,7 @@ async def mount(coordinator,config=None): stack=resource.getrlimit(resource.RLIMIT_STACK)[0] profile={'stackAtLeast8MiB':stack==resource.RLIM_INFINITY or stack>=8388608,'aggregateWithin1MiB':len(argv)+len(env)+8*(len(strings)+2)+len(os.readlink('/proc/self/exe').encode())+1<=1048576,'completeStringsWithin32Pages':all(len(value)+1<=131072 for value in strings)} if not all(profile.values()): raise RuntimeError('closed final native execution profile exceeded') - result={'finalExecProfile':profile,'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest(),'nativeNamespaceContexts':contexts} + result={'finalExecProfile':profile,'nativeCallback':event=='on_session_ready','exactIdentity':data.get('session_id')==config['session_id'],'physicalHome':str(Path.home()),'authenticated':authenticated,'scopedAlias':os.environ.get('FRESHELL')=='1','onecli':hashlib.sha256(os.environ.get('OPENAI_API_KEY','').encode()).hexdigest()==config['key_digest'],'nativeModuleInstaller':uv.startswith('uv 0.8.22'),'resourceAndDependency':hashlib.sha256(output).hexdigest(),'nativeNamespaceContexts':contexts,'completeOwnershipProjection':True} Path(config['receipt']).write_text(json.dumps(result)) return HookResult(action='continue') _consumer=consume @@ -1152,7 +1267,7 @@ async def on_session_ready(coordinator): json!({"provider":"amplifier","route":"terminal","exited":output.exited,"exitCode":output.exit_code,"outputBytes":text.len(),"classes":classes,"classifiedErrors":classified_errors}) ); } - let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency"),"nativeNamespaceContexts":{"closed-native-bundle:context/self.md":hash(b"native-self-namespace-context"),"probe:context/selected.md":hash(b"native-nested-namespace-context"),"same-root:context/selected.md":hash(b"native-same-root-ancestor-context"),"same-no-root:context/selected.md":hash(b"native-same-no-root-ancestor-context"),"unnamed-root:context/selected.md":hash(b"native-unnamed-root-ancestor-context")}}); + let expected = json!({"finalExecProfile":native_exec_profile(),"nativeCallback":true,"exactIdentity":true,"physicalHome":"/home/freshell/provider","authenticated":true,"scopedAlias":true,"onecli":true,"nativeModuleInstaller":true,"resourceAndDependency":hash(b"closed-amplifier-resource:closed-amplifier-dependency"),"completeOwnershipProjection":true,"nativeNamespaceContexts":{"closed-native-bundle:context/self.md":hash(b"native-self-namespace-context"),"probe:context/selected.md":hash(b"native-nested-namespace-context"),"same-root:context/selected.md":hash(b"native-same-root-ancestor-context"),"same-no-root:context/selected.md":hash(b"native-same-no-root-ancestor-context"),"unnamed-root:context/selected.md":hash(b"native-unnamed-root-ancestor-context")}}); assert_eq!(observed.expect("real Amplifier must activate selected local module and executable skill before a model turn"),expected); assert!( !output.exited, @@ -1170,6 +1285,26 @@ async def on_session_ready(coordinator): std::fs::read_to_string(bundle.join("bundle.yaml")).unwrap(), bundle_document ); + use std::os::unix::fs::MetadataExt; + for (path, before) in [ + (skill.join("run"), &source_owner_before), + (outside_target.clone(), &outside_before), + ] { + let after = std::fs::symlink_metadata(path).unwrap(); + assert_eq!( + (after.ino(), after.uid(), after.gid(), after.mode()), + (before.ino(), before.uid(), before.gid(), before.mode()), + "provider preparation changed an original source or unrelated target inode" + ); + } + assert_eq!( + std::fs::read(&outside_target).unwrap(), + b"unrelated-target-nonce" + ); + println!( + "provider_launch_ownership {}", + json!({"provider":"amplifier","resources":10005,"providerUid":65534,"providerGid":0,"exactPrivateParent0710":true,"originalSourcesPreserved":true,"newStubProviderOwned":true}) + ); let soul = SoulId::parse(terminal.soul_id.clone()).unwrap(); let prior = registry.active_handle_for_soul(soul.clone()).await.unwrap(); let prior_generation = prior diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index 36157448e..172d7d8c1 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -1,6 +1,7 @@ import { createHash, randomUUID } from 'node:crypto' import { execFileSync, spawnSync } from 'node:child_process' import fs from 'node:fs' +import http from 'node:http' import net from 'node:net' import os from 'node:os' import path from 'node:path' @@ -142,7 +143,8 @@ export class RuntimeHarness { private readonly assertions: AssertionRecord[] = [] private brokerStarted = false - constructor(repoRoot: string, runId = randomUUID(), phase: 1 | 2 | 3 | 4 | 5 = 1) { + constructor(repoRoot: string, runId = randomUUID(), phase: 1 | 2 | 3 | 4 | 5 = 1, + private readonly providerFixtureHome?: string, private readonly providerFixtureXdgConfigHome?: string) { this.repoRoot = fs.realpathSync(repoRoot) this.phase = phase this.runId = runId @@ -250,6 +252,26 @@ export class RuntimeHarness { for (const volume of providerVolumes) { try { docker(['volume', 'rm', volume]) } catch (error) { errors.push(`provider volume ${volume}: ${String(error)}`) } } + // Publishers may succeed before a runtime receives a receipt. Keep their + // exact typed ownership checks in the still-live broker during cleanup. + const indexedVolumes = new Set((this.broker?.eventsSnapshot?.() ?? []).flatMap(event => { + const volume = event.providerVolumeName + return event.decision === 'forward' && event.method === 'POST' + && event.url.startsWith('/v1.47/containers/create?') + && /^[a-f0-9]{64}$/.test(event.containerId ?? '') + && /^freshell-inputs-[a-f0-9]{24}$/.test(volume ?? '') + && event.helperName === `${volume}-publisher` ? [volume!] : [] + })) + for (const volume of indexedVolumes) { + try { + const url = `/v1.47/volumes/${volume}` + const deleted = await brokerRequestStatus(this.broker.proxySocketPath, 'DELETE', url) + if (deleted !== 204 && deleted !== 404) throw new Error('owned indexed volume deletion unavailable') + if (await brokerRequestStatus(this.broker.proxySocketPath, 'GET', url) !== 404) { + throw new Error('owned indexed volume absence unconfirmed') + } + } catch (error) { errors.push(`indexed volume ${volume}: ${String(error)}`) } + } for (const volume of [...this.trackedVolumes]) { try { docker(['volume', 'rm', volume]) } catch (error) { errors.push(`volume ${volume}: ${String(error)}`) } } @@ -270,7 +292,7 @@ export class RuntimeHarness { ...this.trackedContainers, ...this.broker?.receiptIds?.() ?? [], ])], - volumes: [...this.trackedVolumes, ...providerVolumes], + volumes: [...new Set([...this.trackedVolumes, ...providerVolumes, ...indexedVolumes])], unsafeBrokerAttempts: this.broker?.unsafeAttempts?.() ?? [], completedAt: new Date().toISOString(), } @@ -420,6 +442,9 @@ export class RuntimeHarness { '--network', 'none', '--read-only', '--cap-drop', 'ALL', '--cap-add', 'DAC_OVERRIDE', '--cap-add', 'CHOWN', '--security-opt', 'no-new-privileges', '--tmpfs', '/tmp:rw,noexec,nosuid,nodev,size=64m', '-v', `${this.testRoot}:${this.testRoot}:rw`, + // Helpers are created by the host Docker daemon; their exchange sources + // must live in the existing same-path shared root, outside private /tmp. + '-e', `TMPDIR=${this.testRoot}`, '-v', `${this.buildDir}:${this.buildDir}:ro`, '-v', `${volumeName}:/var/lib/freshell-supervisor:rw`, ] @@ -1058,12 +1083,38 @@ export class RuntimeHarness { } private phase2ProviderUserRoots(): string[] { + const home = this.providerFixtureHome ?? os.homedir() + const validateFixtureDirectory = (selected: string) => { + const metadata = fs.lstatSync(selected) + if (!path.isAbsolute(selected) || path.normalize(selected) !== selected || fs.realpathSync(selected) !== selected + || !metadata.isDirectory() || metadata.isSymbolicLink() + || (process.getuid && metadata.uid !== process.getuid())) { + throw new Error('provider fixture root must retain its selected owned physical directory') + } + } + if (this.providerFixtureHome !== undefined) validateFixtureDirectory(home) + let opencodeRoot = path.join(home, '.config/opencode') + if (this.providerFixtureHome !== undefined && this.providerFixtureXdgConfigHome !== undefined) { + const selected = this.providerFixtureXdgConfigHome + const relative = path.relative(home, selected) + const nativeRoots = ['.claude/projects', '.codex/sessions', '.amplifier/projects', '.local/share/opencode'] + if (!relative || path.isAbsolute(relative) || relative === '..' || relative.startsWith(`..${path.sep}`) + || nativeRoots.some(root => relative === root || relative.startsWith(`${root}${path.sep}`))) { + throw new Error('provider fixture XDG root must remain outside native stores within its owned HOME') + } + validateFixtureDirectory(selected) + opencodeRoot = path.join(selected, 'opencode') + } const roots: string[] = [] - for (const relative of ['.claude', '.codex', '.config/opencode', '.amplifier']) { + for (const selected of [path.join(home, '.claude'), path.join(home, '.codex'), opencodeRoot, path.join(home, '.amplifier')]) { try { - const root = fs.realpathSync(path.join(os.homedir(), relative)) - if (fs.statSync(root).isDirectory()) roots.push(root) - } catch {} + const root = fs.realpathSync(selected) + const metadata = fs.lstatSync(root) + if (this.providerFixtureHome !== undefined) validateFixtureDirectory(selected) + if (metadata.isDirectory()) roots.push(root) + } catch (error) { + if (this.providerFixtureHome !== undefined && (error as NodeJS.ErrnoException).code !== 'ENOENT') throw error + } } return roots } @@ -1138,6 +1189,18 @@ export class RuntimeHarness { } } +function brokerRequestStatus(socketPath: string, method: 'GET' | 'DELETE', url: string): Promise { + return new Promise((resolve, reject) => { + const request = http.request({ socketPath, path: url, method }, response => { + response.resume() + response.once('error', reject) + response.once('end', () => resolve(response.statusCode ?? 0)) + }) + request.once('error', reject) + request.end() + }) +} + export function defaultLimits(): RuntimeLimits { return { cpuMilli: 500, memoryBytes: 128 * 1024 * 1024, swapBytes: 0, pidsMax: 64 } } diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 8ac0922ac..09ff17052 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -47,6 +47,14 @@ function privateHelperEnvironment(value: unknown, home: boolean): boolean { return !home || names.has('HOME') } +function mappedProviderDestination(value: string): boolean { + const home = '/home/freshell/provider' + if (!isStrictDescendant(value, home)) return false + // Ordinary selected inputs cannot alias the provider's native stores. + return !['.claude/projects', '.codex/sessions', '.amplifier/projects', '.local/share/opencode'] + .some(relative => value === `${home}/${relative}` || isStrictDescendant(value, `${home}/${relative}`)) +} + function preparedMountsMatch(actual: unknown[], input: PreparedInputAuthorization): boolean { const expected: Record[] = input.configurationBinds.map((bind) => ({ Type: 'bind', Source: path.join(input.stageRoot, bind.sourceRelativePath), Target: bind.effectiveDestination, ReadOnly: true, @@ -187,6 +195,7 @@ export class RestrictedDockerBroker { private readonly claudeCandidates = new Map() private readonly candidateVolumes = new Set() private readonly initialProviderVolumes = new Map() + private readonly initialReservations = new Map() private readonly preparedInputs = new Map() private readonly inputVolumes = new Set() private readonly inputPublishers = new Map= 6) throw new Error('closed Claude fixture exceeds its two routes and two replans') + this.claudeCandidates.set(key, structuredClone(input)) + } + + private validateClaudeCandidateAuthorization(input: ClaudeCandidateAuthorization): void { const initial = this.initialProviderVolumes.get(input.providerVolumeName) const uuid = (value: string) => /^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(value) const fixture = path.dirname(this.policy.runtimeRootPrefix) @@ -213,6 +230,7 @@ export class RestrictedDockerBroker { const destination = (value: string) => typeof value === 'string' && path.isAbsolute(value) && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) && (value === input.workspacePath || isStrictDescendant(value, input.workspacePath) + || mappedProviderDestination(value) || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) const unit = (value: string, prefix: string, file = false) => typeof value === 'string' && new RegExp(`^${prefix}/[a-zA-Z0-9_-]{1,128}${file ? '/source' : ''}$`).test(value) @@ -220,7 +238,7 @@ export class RestrictedDockerBroker { || !uuid(input.requestId) || !uuid(input.reservationId) || !input.generationId.startsWith('grant-') || !uuid(input.generationId.slice(6)) || relative !== `scenarios/${relative.split('/')[1]}/control/provider-roots/${input.generationId}` || !/^[a-zA-Z0-9_-]{1,128}$/.test(relative.split('/')[1] ?? '') - || !this.policy.allowedWorkspaceRoots?.has(input.workspacePath) || !destination(input.cwd) + || !this.isAllowedWorkspacePath(input.workspacePath) || !destination(input.cwd) || !(input.cwd === input.workspacePath || isStrictDescendant(input.cwd, input.workspacePath)) || (input.gitCommonDir !== undefined && (!path.isAbsolute(input.gitCommonDir) || path.normalize(input.gitCommonDir) !== input.gitCommonDir || /[\x00-\x1f\x7f]/.test(input.gitCommonDir) || !(input.gitCommonDir === input.workspacePath || isStrictDescendant(input.gitCommonDir, input.workspacePath) || isStrictDescendant(input.gitCommonDir, fixture)))) @@ -251,12 +269,10 @@ export class RestrictedDockerBroker { claim(mount.effectiveDestination) } } - const key = candidateVolumeName(input), previous = this.claudeCandidates.get(key) - if (previous && JSON.stringify(previous) !== JSON.stringify(input)) throw new Error('Claude candidate request changed') - if (!previous && this.claudeCandidates.size >= 6) throw new Error('closed Claude fixture exceeds its two routes and two replans') - this.claudeCandidates.set(key, structuredClone(input)) } + + authorizeInitialProviderVolume(input:InitialProviderVolumeAuthorization):void { const stable=`freshell-provider-${createHash('sha256').update(`${input.installationId}\0${input.soulId}`).digest('hex').slice(0,24)}` if (input.providerVolumeName!==stable || !/^[a-zA-Z0-9_-]{1,128}$/.test(input.installationId) @@ -287,10 +303,12 @@ export class RestrictedDockerBroker { const destination = (value: string) => path.isAbsolute(value) && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) && (this.isAllowedWorkspacePath(value) + || mappedProviderDestination(value) || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) const targets = new Set() const claim = (value: string) => { - if (!destination(value) || targets.has(value)) throw new Error('prepared input target is outside selected fixture roots or duplicated') + if (!destination(value)) throw new Error('prepared input target is outside selected fixture roots') + if (targets.has(value)) throw new Error('prepared input target is duplicated') targets.add(value) } for (const bind of input.configurationBinds) { @@ -388,6 +406,11 @@ export class RestrictedDockerBroker { const target = parseContainerTarget(url) const destructive = isDestructiveDockerRequest(method, target?.action) + if (method === 'POST' && (url === `${DOCKER_API_PREFIX}/volumes/create` + || url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`))) { + this.registerPublishedFixtureInputs(body, url) + } + if (method === 'GET' && url === '/_ping') { await this.forwardAndReply(request, response, body, { destructive: false }) return @@ -447,6 +470,24 @@ export class RestrictedDockerBroker { await this.forwardAndReply(request, response, body, { destructive: false }) return } + if (method === 'GET' && candidateVolume && !candidate + && /^freshell-claude-candidate-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(candidateVolume)) { + const rejection = this.initialClaudePublicationRejection(candidateVolume.slice('freshell-claude-candidate-'.length)) + if (rejection) { + this.block(response, method, url, 403, `candidate absence qualification unavailable at ${rejection}`, false) + return + } + const inspected = await this.forward(request, body) + if (inspected.statusCode !== 404) { + this.block(response, method, url, 403, 'unregistered candidate probe may expose only actual absence', false) + return + } + // Absence is read-only evidence. Register no candidate or mutation owner; + // the later exact POST still requires the complete publication handoff. + this.record({ method, url, decision: 'forward', destructive: false, unsafeAttempt: false }) + replyDocker(response, inspected) + return + } if (method === 'DELETE' && candidate && this.candidateVolumes.has(candidateVolume!)) { const inspected = await this.forward({ method: 'GET', url, headers: {} } as IncomingMessage, Buffer.alloc(0)) let actual: Record = {} @@ -507,6 +548,9 @@ export class RestrictedDockerBroker { this.block(response,method,url,403,'initial volume must match the exact reserved soul source',true) return } + this.initialReservations.set(initial.providerVolumeName, { + reservationId: labels['freshell.input.reservation'], incarnationId: labels['freshell.input.incarnation'], + }) await this.forwardAndReply(request,response,body,{destructive:false}) return } @@ -644,7 +688,7 @@ export class RestrictedDockerBroker { if (typeof parsed.Id !== 'string' || !/^[a-f0-9]{64}$/.test(parsed.Id) || this.receiptIds().has(parsed.Id)) throw new Error('invalid Claude source helper identity') helper.containerId = parsed.Id } - this.record({ method, url, decision: 'forward', destructive: false, unsafeAttempt: false, helperName: name, containerId: helper.containerId, requestDigest: sha256(body) }) + this.record({ method, url, decision: 'forward', destructive: false, unsafeAttempt: false, helperName: name, containerId: helper.containerId, requestDigest: sha256(body), reason: this.helperRejectionReason(forwarded, helper.expected) }) replyDocker(response, forwarded) } catch { this.record({ method, url, decision: 'forward', reason: 'Claude source helper acknowledgement unavailable', destructive: false, unsafeAttempt: false, helperName: name, requestDigest: sha256(body) }) @@ -654,7 +698,8 @@ export class RestrictedDockerBroker { return } if (/^freshell-inputs-[a-f0-9]{24}-publisher$/.test(name) || mountpointHelper) { - const input=mountpointHelper ? this.validateMountpointHelper(body) : this.validateInputPublisher(name,body) + const mountpoint = mountpointHelper ? this.validateMountpointHelper(body) : undefined + const input=mountpointHelper ? mountpoint?.input : this.validateInputPublisher(name,body) let previousAbsent=true if (input && this.inputPublishers.has(name)) { const previous=await this.forward({method:'GET',url:`${DOCKER_API_PREFIX}/containers/${name}/json`,headers:{}} as IncomingMessage,Buffer.alloc(0)) @@ -662,7 +707,7 @@ export class RestrictedDockerBroker { this.record({method:'GET',url:`${DOCKER_API_PREFIX}/containers/${name}/json`,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name}) } if (!input || !previousAbsent) { - this.block(response,method,url,403,'indexed publisher must match an owned fixed preparation',false) + this.block(response,method,url,403,mountpoint?.reason ?? 'indexed publisher must match an owned fixed preparation',false) return } const publisher:{input:PreparedInputAuthorization;expected:Record;containerId?:string}={input,expected:JSON.parse(body.toString())} @@ -674,7 +719,7 @@ export class RestrictedDockerBroker { if (typeof parsed.Id!=='string' || !/^[a-f0-9]{64}$/.test(parsed.Id) || this.receiptIds().has(parsed.Id)) throw new Error('invalid indexed publisher identity') publisher.containerId=parsed.Id } - this.record({method,url,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name,containerId:publisher.containerId,providerVolumeName:mountpointHelper ? input.providerVolumeName : input.indexedFiles!.volumeName}) + this.record({method,url,decision:'forward',destructive:false,unsafeAttempt:false,helperName:name,containerId:publisher.containerId,providerVolumeName:mountpointHelper ? input.providerVolumeName : input.indexedFiles!.volumeName,reason:this.helperRejectionReason(forwarded,publisher.expected)}) replyDocker(response,forwarded) } catch { response.statusCode=503 @@ -947,8 +992,7 @@ export class RestrictedDockerBroker { if (!Array.isArray(mounts)) return false const exchangeTarget = reader ? '/run/freshell-source-exchange' : '/run/freshell-input-exchange' const exchange = mounts.find((mount) => mount?.Type === 'bind' && mount.Target === exchangeTarget && mount.ReadOnly === true - && typeof mount.Source === 'string' && path.dirname(mount.Source) === os.tmpdir() - && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + && this.isAllowedExchangeSource(mount.Source, 'freshell-inputs')) if (!exchange) return false const expected: Record[] = [ { Type: 'bind', Source: input.hostBinaryPath, Target: '/runtime/freshell-session-host', ReadOnly: true }, @@ -1017,10 +1061,9 @@ export class RestrictedDockerBroker { if (!Array.isArray(mounts) || mounts.some((mount) => !mount || Object.keys(mount).some((key) => !['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key)))) return false const binary = mounts.find((mount) => mount.Type === 'bind' && mount.Source === owner.hostBinaryPath && mount.Target === '/runtime/freshell-session-host' && mount.ReadOnly === true) - const workspace = mounts.find((mount) => mount.Type === 'bind' && mount.Source === mount.Target && this.policy.allowedWorkspaceRoots?.has(mount.Source) && mount.ReadOnly === true) + const workspace = mounts.find((mount) => mount.Type === 'bind' && mount.Source === mount.Target && this.isAllowedWorkspacePath(mount.Source) && mount.ReadOnly === true) const exchange = mounts.find((mount) => mount.Type === 'bind' && mount.Target === '/run/freshell-source-exchange' && mount.ReadOnly === true - && typeof mount.Source === 'string' && path.dirname(mount.Source) === os.tmpdir() - && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + && this.isAllowedExchangeSource(mount.Source, 'freshell-inputs')) if (!binary || !workspace || !exchange || [binary,workspace,exchange].some(mount => mount.VolumeOptions !== undefined)) return false const home = mounts.find(mount => mount.Target === '/home/freshell/provider') if (home) { @@ -1098,26 +1141,33 @@ export class RestrictedDockerBroker { const binary=mounts.find((mount)=>mount.Type==='bind' && mount.Source===input.hostBinaryPath && mount.Target==='/runtime/freshell-session-host' && mount.ReadOnly===true) const volume=mounts.find((mount)=>mount.Type==='volume' && mount.Source===input.indexedFiles!.volumeName && mount.Target==='/run/freshell-input-files' && mount.ReadOnly===false && equalRecord(mount.VolumeOptions,{NoCopy:true})) const exchange=mounts.find((mount)=>mount.Type==='bind' && mount.Target==='/run/freshell-input-exchange' && mount.ReadOnly===true - && typeof mount.Source==='string' && path.dirname(mount.Source)===os.tmpdir() - && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + && this.isAllowedExchangeSource(mount.Source, 'freshell-inputs')) if (!binary || !volume || !exchange || [binary,volume,exchange].some((mount)=>Object.keys(mount).some((key)=>!['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key))) || binary.VolumeOptions!==undefined || exchange.VolumeOptions!==undefined) return return input } - private validateMountpointHelper(body:Buffer):PreparedInputAuthorization | undefined { + private validateMountpointHelper(body:Buffer): { input?: PreparedInputAuthorization; reason?: string } { + const reject = (phase: 'json' | 'owner' | 'image' | 'labels' | 'identity_argv' | 'environment' | 'shape' + | 'limits_authority' | 'mount_count' | 'exchange' | 'mounts') => ({ reason: `mountpoint helper unavailable at ${phase}` }) let parsed:Record - try {parsed=JSON.parse(body.toString())} catch {return} - const input=[...this.preparedInputs.values()].find((input)=>parsed.Image===input.imageRef - && equalRecord(parsed.Labels,{'com.freshell.installation-id':input.installationId, - 'com.freshell.soul-id':input.soulId,'com.freshell.incarnation-id':input.incarnationId})) - if (!input || parsed.User!=='0:0' || parsed.Tty!==false + try {parsed=JSON.parse(body.toString())} catch {return reject('json')} + const owners=[...this.preparedInputs.values()].filter((input)=> + parsed.Labels?.['com.freshell.installation-id']===input.installationId + && parsed.Labels?.['com.freshell.soul-id']===input.soulId + && parsed.Labels?.['com.freshell.incarnation-id']===input.incarnationId) + if (!owners.length) return reject('owner') + const input=owners.find((input)=>parsed.Image===input.imageRef) + if (!input) return reject('image') + if (!equalRecord(parsed.Labels,{'com.freshell.installation-id':input.installationId, + 'com.freshell.soul-id':input.soulId,'com.freshell.incarnation-id':input.incarnationId})) return reject('labels') + if (parsed.User!=='0:0' || parsed.Tty!==false || JSON.stringify(parsed.Entrypoint)!==JSON.stringify(['/runtime/freshell-session-host']) - || JSON.stringify(parsed.Cmd)!==JSON.stringify(['prepare-provider-mountpoints']) - || !privateHelperEnvironment(parsed.Env,false)) return + || JSON.stringify(parsed.Cmd)!==JSON.stringify(['prepare-provider-mountpoints'])) return reject('identity_argv') + if (!privateHelperEnvironment(parsed.Env,false)) return reject('environment') const host=parsed.HostConfig ?? {} if (Object.keys(parsed).some((key)=>!['Image','User','Tty','Labels','Entrypoint','Cmd','Env','HostConfig'].includes(key)) - || Object.keys(host).some((key)=>!['NetworkMode','ReadonlyRootfs','CapDrop','CapAdd','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','LogConfig','Mounts'].includes(key))) return + || Object.keys(host).some((key)=>!['NetworkMode','ReadonlyRootfs','CapDrop','CapAdd','SecurityOpt','Memory','MemorySwap','NanoCpus','PidsLimit','LogConfig','Mounts'].includes(key))) return reject('shape') if (host.NetworkMode!=='none' || host.ReadonlyRootfs!==true || JSON.stringify(host.CapDrop)!==JSON.stringify(['ALL']) || JSON.stringify(host.CapAdd)!==JSON.stringify(['CHOWN','SETGID','SETUID']) @@ -1125,19 +1175,213 @@ export class RestrictedDockerBroker { || host.Memory!==128*1024*1024 || host.MemorySwap!==128*1024*1024 || host.NanoCpus!==500_000_000 || host.PidsLimit!==32 || !equalRecord(host.LogConfig,{Type:'none',Config:host.LogConfig?.Config}) - || Object.keys(host.LogConfig?.Config ?? {}).length) return + || Object.keys(host.LogConfig?.Config ?? {}).length) return reject('limits_authority') const mounts=host.Mounts - if (!Array.isArray(mounts) || mounts.length!==3) return + if (!Array.isArray(mounts) || mounts.length!==3) return reject('mount_count') const binary=mounts.find((mount)=>mount.Type==='bind' && mount.Source===input.hostBinaryPath && mount.Target==='/runtime/freshell-session-host' && mount.ReadOnly===true) const volume=mounts.find((mount)=>mount.Type==='volume' && mount.Source===input.providerVolumeName && mount.Target==='/home/freshell/provider' && mount.ReadOnly===false && equalRecord(mount.VolumeOptions,{NoCopy:true})) - const exchange=mounts.find((mount)=>mount.Type==='bind' && mount.Target==='/run/freshell-input-exchange' && mount.ReadOnly===true - && typeof mount.Source==='string' && path.dirname(mount.Source)===os.tmpdir() - && /^freshell-inputs-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(path.basename(mount.Source))) + const exchange=mounts.find((mount)=>mount.Type==='bind' && mount.Target==='/run/freshell-input-exchange' && mount.ReadOnly===true) + if (!exchange || !this.isAllowedExchangeSource(exchange.Source, 'freshell-inputs')) return reject('exchange') if (!binary || !volume || !exchange || [binary,volume,exchange].some((mount)=>Object.keys(mount).some((key)=>!['Type','Source','Target','ReadOnly','VolumeOptions'].includes(key))) - || binary.VolumeOptions!==undefined || exchange.VolumeOptions!==undefined) return - return input + || binary.VolumeOptions!==undefined || exchange.VolumeOptions!==undefined) return reject('mounts') + return { input } + } + + // Browser fixtures publish through the actual controller, whose generated + // stages exist before Docker receives their first request. Register that + // immutable handoff here; the ordinary Docker validators still decide every + // request. Explicit native fixture handoffs continue to use the same APIs. + private initialClaudePublicationRejection(requestId: string): string | undefined { + let phase = 'physical_stage' + try { + const root = path.dirname(this.policy.runtimeRootPrefix) + for (const scenario of this.fixtureDirectories(path.join(root, 'scenarios'), 32)) { + const control = path.join(root, 'scenarios', scenario, 'control') + for (const grant of this.fixtureDirectories(path.join(control, 'provider-roots'), 128)) { + if (!/^grant-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(grant)) continue + const stageRoot = path.join(control, 'provider-roots', grant) + const bindingFile = path.join(stageRoot, '.freshell/claude-candidate-binding.json') + if (!fs.existsSync(bindingFile)) continue + phase = 'owner_capability' + const binding = this.fixtureMetadata(bindingFile).value + const capability = this.fixtureMetadata(path.join(control, 'mcp-capabilities', `${grant}.json`), 128 * 1024).value + const owner = [...this.initialProviderVolumes.values()].find(owner => owner.soulId === binding.soulId && owner.soulId === capability.soulId) + if (!owner || capability.incarnationId != null || binding.version !== 1 || binding.provider !== 'claude' + || binding.generationId !== grant || binding.intentRevision !== 1 || binding.registryAttemptId != null + || !/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(binding.initialReservationId ?? '')) continue + phase = 'prepared_metadata' + const context = this.fixtureMetadata(path.join(stageRoot, '.freshell/prepared-context.json')) + const result = this.fixtureMetadata(path.join(stageRoot, '.freshell/prepared-result.json')) + const inputs = context.value.preparedInputs + if (!context.value.preparation?.claude || !inputs || inputs.generationId !== grant + || context.value.mcpCapability?.grantId !== grant || inputs.initialAuthReservation !== binding.initialReservationId + || JSON.stringify(result.value.providerLaunchContext) !== JSON.stringify(context.value) + || !Array.isArray(result.value.providerSecretReferences) || inputs.manifest?.relativePath !== '.freshell/input-manifest.json' + || sha256Hex(context.bytes) !== binding.preparedContextSha256 || sha256Hex(result.bytes) !== binding.preparedResultSha256) continue + phase = 'authority_binding' + if (sha256Hex(Buffer.from(JSON.stringify({ kind: 'initial_reservation', reservationId: binding.initialReservationId }))) !== binding.authoritySha256) continue + phase = 'manifest_artifact' + const manifest = this.fixtureMetadata(path.join(stageRoot, inputs.manifest.relativePath), 64 * 1024 * 1024) + const binary = fs.lstatSync(owner.hostBinaryPath) + if (sha256Hex(manifest.bytes) !== binding.manifestSha256 || inputs.manifest.sha256 !== binding.manifestSha256 + || !binary.isFile() || binary.isSymbolicLink() || binary.size > 128 * 1024 * 1024 + || sha256Hex(fs.readFileSync(owner.hostBinaryPath)) !== owner.binarySha256) continue + phase = 'selected_target' + this.validateClaudeCandidateAuthorization({ ...owner, generationId: grant, stageRoot, + requestId, reservationId: binding.initialReservationId, workspacePath: binding.workspacePath, cwd: binding.cwd, + gitCommonDir: binding.gitCommonDir === null ? undefined : binding.gitCommonDir, manifestRelativePath: inputs.manifest.relativePath, + configurationBinds: inputs.configBinds ?? [], resourceMounts: inputs.resourceMounts ?? [] }) + return undefined + } + } + } catch { /* Unqualified publication exposes no daemon metadata. */ } + return phase + } + + private registerPublishedFixtureInputs(body: Buffer, url: string): void { + let phase = 'owner' + try { + const request = JSON.parse(body.toString()) + const labels = request.Labels ?? {} + const owner = [...this.initialProviderVolumes.values()].find(source => + source.installationId === labels['com.freshell.installation-id'] + && source.soulId === labels['com.freshell.soul-id']) + if (!owner || (request.Image !== undefined && request.Image !== owner.imageRef)) return + const candidate = labels['com.freshell.claude-candidate'] === 'ordinary' + const generation = labels['com.freshell.input-generation'] + const incarnationId = labels['com.freshell.incarnation-id'] + if (candidate && this.claudeCandidates.has(`freshell-claude-candidate-${labels['com.freshell.source-request-id']}`)) return + if (!candidate && [...this.preparedInputs.values()].some(input => input.installationId === owner.installationId + && input.soulId === owner.soulId && input.incarnationId === incarnationId + && (generation === undefined || generation === input.generationId))) return + const reservation = this.initialReservations.get(owner.providerVolumeName) + if (!candidate && (!reservation || reservation.incarnationId !== incarnationId)) return + const root = path.dirname(this.policy.runtimeRootPrefix) + phase = 'physical_directories' + const scenarios = this.fixtureDirectories(path.join(root, 'scenarios'), 32) + for (const scenario of scenarios) { + const control = path.join(root, 'scenarios', scenario, 'control') + for (const grant of this.fixtureDirectories(path.join(control, 'provider-roots'), 128)) { + if (!/^grant-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(grant) + || (generation !== undefined && generation !== grant)) continue + const stageRoot = path.join(control, 'provider-roots', grant) + phase = 'capability_metadata' + const capability = this.fixtureMetadata(path.join(control, 'mcp-capabilities', `${grant}.json`), 128 * 1024).value + if (capability.soulId !== owner.soulId || (capability.incarnationId != null + && capability.incarnationId !== incarnationId)) continue + const contextDocument = this.fixtureMetadata(path.join(stageRoot, '.freshell/prepared-context.json')) + phase = 'prepared_metadata' + const resultDocument = this.fixtureMetadata(path.join(stageRoot, '.freshell/prepared-result.json')) + const context = contextDocument.value + const result = resultDocument.value + const inputs = context.preparedInputs + if (!inputs || inputs.generationId !== grant || context.mcpCapability?.grantId !== grant + || JSON.stringify(result.providerLaunchContext) !== JSON.stringify(context) + || !Array.isArray(result.providerSecretReferences) + || inputs.manifest?.relativePath !== '.freshell/input-manifest.json') continue + const manifest = this.fixtureMetadata(path.join(stageRoot, inputs.manifest.relativePath), 64 * 1024 * 1024) + phase = 'manifest_hash' + if (sha256Hex(manifest.bytes) !== inputs.manifest.sha256) continue + phase = 'host_artifact' + const binary = fs.lstatSync(owner.hostBinaryPath) + if (!binary.isFile() || binary.isSymbolicLink() || binary.size > 128 * 1024 * 1024 + || sha256Hex(fs.readFileSync(owner.hostBinaryPath)) !== owner.binarySha256) continue + if (candidate) { + phase = 'candidate_binding' + const binding = this.fixtureMetadata(path.join(stageRoot, '.freshell/claude-candidate-binding.json')).value + if (binding.version !== 1 || binding.provider !== 'claude' || binding.soulId !== owner.soulId + || binding.generationId !== grant || binding.intentRevision !== 1 || binding.registryAttemptId != null + || binding.initialReservationId !== inputs.initialAuthReservation + || binding.initialReservationId !== labels['com.freshell.source-reservation-id'] + || binding.manifestSha256 !== inputs.manifest.sha256 + || binding.preparedContextSha256 !== sha256Hex(contextDocument.bytes) + || binding.preparedResultSha256 !== sha256Hex(resultDocument.bytes)) continue + phase = 'candidate_authorization' + this.authorizeClaudeCandidate({ ...owner, generationId: grant, stageRoot, + reservationId: binding.initialReservationId, requestId: labels['com.freshell.source-request-id'], + workspacePath: binding.workspacePath, cwd: binding.cwd, gitCommonDir: binding.gitCommonDir === null ? undefined : binding.gitCommonDir, + manifestRelativePath: inputs.manifest.relativePath, configurationBinds: inputs.configBinds ?? [], + resourceMounts: inputs.resourceMounts ?? [], + }) + } else { + if (inputs.initialAuthReservation !== reservation!.reservationId) continue + const indexedSecretSources: { grantIndex: number; sourceId: number }[] = [] + result.providerSecretReferences.forEach((reference: Record, grantIndex: number) => { + if (!['claude_configuration_file', 'codex_configuration_file', 'opencode_configuration_file', 'amplifier_configuration_file'].includes(reference.profile)) return + const source = reference.providerRelativePath?.match(/^\.freshell\/references\/(\d+)$/)?.[1] + if (source === undefined) throw new Error('fixture configuration source index invalid') + indexedSecretSources.push({ grantIndex, sourceId: Number(source) }) + }) + if (indexedSecretSources.length !== inputs.manifest.sourceCount) continue + phase = 'prepared_authorization' + this.authorizePreparedInputs({ ...owner, incarnationId, generationId: grant, stageRoot, + configurationBinds: (inputs.configBinds ?? []).map((bind: Record) => ({ + sourceRelativePath: bind.sourceRelativePath, effectiveDestination: bind.effectiveDestination, + })), resourceMounts: (inputs.resourceMounts ?? []).map((mount: Record) => ({ + providerRelativePath: mount.providerRelativePath, effectiveDestination: mount.effectiveDestination, kind: mount.kind, + })), indexedFiles: inputs.indexedFiles, indexedSecretSources, + }) + } + return + } + } + this.record({ method: 'POST', url, decision: 'block', reason: `fixture publication unavailable at ${phase}`, destructive: false, unsafeAttempt: false }) + } catch (error) { + // Only fixed validator categories are observable. Discard all other + // exception messages, which may include filesystem or input details. + const categories: Record = { + 'prepared input owner/stage is outside this fixture run': 'owner_or_stage', + 'prepared input metadata exceeds control profile': 'metadata_limit', + 'prepared input target is outside selected fixture roots': 'target_root', + 'prepared input target is duplicated': 'duplicate_target', + 'prepared template source is invalid': 'invalid_template_source', + 'prepared resource source is invalid': 'invalid_resource_source', + 'prepared indexed volume identity/profile is invalid': 'indexed_identity_or_profile', + 'prepared indexed source id is invalid': 'indexed_source_id', + 'prepared private source has no indexed volume': 'missing_indexed_volume', + 'prepared private input index is invalid': 'private_input_index', + 'prepared input generation changed; reconcile before replacement': 'changed_generation', + } + const category = error instanceof Error ? categories[error.message] ?? 'unclassified' : 'unclassified' + this.record({ method: 'POST', url, decision: 'block', reason: `fixture publication unavailable at ${phase}: ${category}`, destructive: false, unsafeAttempt: false }) + } + } + + private fixtureDirectories(directory: string, limit: number): string[] { + if (!fs.existsSync(directory)) return [] + this.physicalFixturePath(directory) + const entries = fs.readdirSync(directory, { withFileTypes: true }) + if (entries.length > limit || entries.some(entry => entry.isSymbolicLink())) throw new Error('fixture directory profile invalid') + return entries.filter(entry => entry.isDirectory()).map(entry => entry.name) + } + + private physicalFixturePath(file: string): void { + const root = path.dirname(this.policy.runtimeRootPrefix) + const relative = path.relative(root, file) + if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) throw new Error('fixture metadata outside owned root') + let parent = root + const owner = fs.lstatSync(root).uid + for (const part of relative.split(path.sep)) { + parent = path.join(parent, part) + const stat = fs.lstatSync(parent) + if (stat.isSymbolicLink() || stat.uid !== owner) throw new Error('fixture metadata physical owner changed') + } + } + + private fixtureMetadata(file: string, limit = 1024 * 1024): { value: Record; bytes: Buffer } { + this.physicalFixturePath(file) + const descriptor = fs.openSync(file, fs.constants.O_RDONLY | fs.constants.O_NOFOLLOW) + try { + const stat = fs.fstatSync(descriptor) + if (!stat.isFile() || stat.size > limit) throw new Error('fixture metadata profile exceeded') + const bytes = Buffer.alloc(stat.size) + if (fs.readSync(descriptor, bytes) !== stat.size || fs.readSync(descriptor, Buffer.alloc(1)) !== 0) throw new Error('fixture metadata changed') + const value = JSON.parse(bytes.toString('utf8')) + if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error('fixture metadata invalid') + return { value, bytes } + } finally { fs.closeSync(descriptor) } } private validateHistoryHelper(name: string, body: Buffer): BrokerReceipt | undefined { @@ -1192,7 +1436,7 @@ export class RestrictedDockerBroker { const volume = mounts.find((mount) => mount.Type === 'volume' && mount.Target === '/home/freshell/provider' && mount.ReadOnly === true && same(mount.VolumeOptions, { NoCopy: true })) const exchange = mounts.find((mount) => mount.Type === 'bind' && mount.Target === '/run/freshell-auth-exchange' && mount.ReadOnly === true) if (!binary || !volume || !exchange || typeof exchange.Source !== 'string' - || !new RegExp(`^freshell-auth-${uuid}$`).test(path.basename(exchange.Source)) || path.dirname(exchange.Source) !== os.tmpdir() + || !this.isAllowedExchangeSource(exchange.Source, 'freshell-auth') || !this.policy.allowedHostBinaryPaths.has(binary.Source) || !this.policy.allowedImageRefs.has(parsed.Image)) return return this.receipts().find((receipt) => receipt.providerVolumeName === volume.Source && receipt.imageRef === parsed.Image && receipt.hostBinaryPath === binary.Source) @@ -1206,6 +1450,34 @@ export class RestrictedDockerBroker { return false } + private isAllowedExchangeSource(source: unknown, prefix: 'freshell-inputs' | 'freshell-auth'): source is string { + if (typeof source !== 'string' || path.normalize(source) !== source + || !new RegExp(`^${prefix}-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$`).test(path.basename(source))) return false + if (path.dirname(source) === os.tmpdir()) return true + if (path.dirname(source) !== path.dirname(this.policy.runtimeRootPrefix)) return false + try { + this.physicalFixturePath(source) + return fs.lstatSync(source).isDirectory() + } catch { return false } + } + + /** Called only for the already validated fixed helpers; never persist daemon payloads. */ + private helperRejectionReason(response: DockerResponse, expected: Record): string | undefined { + if (response.statusCode === 201) return + let missingBindSource = false + if (response.body.length <= 64 * 1024) { + try { + const message = JSON.parse(response.body.toString()).message + missingBindSource = typeof message === 'string' + && message.startsWith('invalid mount config for type "bind": bind source path does not exist:') + } catch {} + } + const exchange = expected.HostConfig.Mounts.find((mount: Record) => + mount.Type === 'bind' && ['/run/freshell-source-exchange', '/run/freshell-input-exchange'].includes(mount.Target)) + const hostExists = exchange ? fs.existsSync(exchange.Source) : false + return `upstream helper rejected: status=${response.statusCode}; cause=${missingBindSource ? 'missing_bind_source' : 'unclassified'}; ownedExchangeHostExists=${hostExists}` + } + private async forwardAndReply(request: IncomingMessage, response: ServerResponse, body: Buffer, event: { destructive: boolean; containerId?: string; helperName?:string }): Promise { const forwarded = await this.forward(request, body) this.record({ method: request.method ?? 'GET', url: request.url ?? '/', decision: 'forward', destructive: event.destructive, unsafeAttempt: false, containerId: event.containerId,helperName:event.helperName }) @@ -1295,5 +1567,9 @@ function isStrictDescendant(candidate: string, root: string): boolean { } function sha256(bytes: Buffer): string { - return `sha256:${createHash('sha256').update(bytes).digest('hex')}` + return `sha256:${sha256Hex(bytes)}` +} + +function sha256Hex(bytes: Buffer): string { + return createHash('sha256').update(bytes).digest('hex') } diff --git a/test/e2e-browser/helpers/managed-runtime.ts b/test/e2e-browser/helpers/managed-runtime.ts index 7807cc94a..23c539ff7 100644 --- a/test/e2e-browser/helpers/managed-runtime.ts +++ b/test/e2e-browser/helpers/managed-runtime.ts @@ -12,7 +12,7 @@ import { } from '../../../scripts/testing/provider-qualification-receipt.js' import { RuntimeHarness, type SupervisorInstance } from '../../../scripts/testing/runtime-sandbox.js' import { RustServer } from './rust-server.js' -import type { TestServerInfo } from './test-server.js' +import type { E2eServerInfo as TestServerInfo } from './server-fixture-support.js' import type { QualifiableTerminalProvider } from '../../../scripts/testing/provider-qualification-selection.js' import { buildFreshAgentQualificationReceipt, @@ -100,7 +100,7 @@ export class ManagedRuntimeBrowserRig { webHomeDir?: string, ) { this.repoRoot = fs.realpathSync(repoRoot) - this.runtime = new RuntimeHarness(this.repoRoot, undefined, phase) + this.runtime = new RuntimeHarness(this.repoRoot, undefined, phase, webHomeDir, serverEnv.XDG_CONFIG_HOME) this.serverEnv = { ...serverEnv } this.webHomeDir = webHomeDir this.supervisorEnv = { ...supervisorEnv } diff --git a/test/fixtures/providers/parity-provider.mjs b/test/fixtures/providers/parity-provider.mjs index 52a737d18..cf5979561 100644 --- a/test/fixtures/providers/parity-provider.mjs +++ b/test/fixtures/providers/parity-provider.mjs @@ -2,10 +2,11 @@ // Records what a provider child can actually see on either terminal route. // The wrapper basename supplies the provider, so no test-only environment has // to cross the managed launch allowlist. -import { spawn } from 'node:child_process' +import { spawn, spawnSync } from 'node:child_process' import { createHash, randomUUID } from 'node:crypto' import fs from 'node:fs' import path from 'node:path' +import { fileURLToPath, pathToFileURL } from 'node:url' const provider = path.basename(process.argv[1]).replace(/^parity-/, '') const argv = process.argv.slice(2) @@ -122,118 +123,184 @@ function rpcProbe(recipe) { })() } -const home = process.env.HOME ?? '' -const providerHome = { - claude: process.env.CLAUDE_CONFIG_DIR ?? path.join(home, '.claude'), - codex: process.env.CODEX_HOME ?? path.join(home, '.codex'), - opencode: path.join(process.env.XDG_CONFIG_HOME ?? path.join(home, '.config'), 'opencode'), - amplifier: process.env.AMPLIFIER_HOME ?? path.join(home, '.amplifier'), -}[provider] -const pluginRelativePath = { - claude: 'plugins/parity-plugin.txt', - codex: 'skills/parity/SKILL.md', - opencode: 'plugins/parity-plugin.js', - amplifier: 'bundles/parity-bundle.yaml', -}[provider] -const pluginPath = providerHome && pluginRelativePath ? path.join(providerHome, pluginRelativePath) : null -const nativeIdArgIndex = provider === 'claude' - ? Math.max(argv.indexOf('--session-id'), argv.indexOf('--resume')) - : provider === 'amplifier' && argv.includes('resume') ? argv.length - 2 : -1 -const nativeId = nativeIdArgIndex >= 0 ? argv[nativeIdArgIndex + 1] : null -const onecliKey = { - claude: 'ANTHROPIC_API_KEY', codex: 'OPENAI_API_KEY', - opencode: 'OPENROUTER_API_KEY', amplifier: 'OPENAI_API_KEY', -}[provider] -append({ - kind: 'launch', - launchId, - onecliChildDigest: process.env[onecliKey] - ? createHash('sha256').update(process.env[onecliKey]).digest('hex') : null, - onecliControlPresent: Boolean(process.env.ONECLI_URL), - argv, - cwd, - env: { - FRESHELL: process.env.FRESHELL, - FRESHELL_URL: process.env.FRESHELL_URL, - FRESHELL_TOKEN_PRESENT: Boolean(process.env.FRESHELL_TOKEN), - FRESHELL_TERMINAL_ID: process.env.FRESHELL_TERMINAL_ID, - FRESHELL_TAB_ID: process.env.FRESHELL_TAB_ID, - FRESHELL_PANE_ID: process.env.FRESHELL_PANE_ID, - OPENCODE_TUI_CONFIG: process.env.OPENCODE_TUI_CONFIG, - }, - tuiConfig: provider === 'opencode' && process.env.OPENCODE_TUI_CONFIG - ? fs.readFileSync(process.env.OPENCODE_TUI_CONFIG, 'utf8') : null, - inlineConfig: provider === 'opencode' +export async function observeOpenCodeParitySelection(providerHome, workspace, tuiPath, bridgeRecipe) { + const projected = path.join(providerHome, "project/.opencode/opencode.json"); + const hasProjection = fs.existsSync(projected); + const project = readJson(hasProjection ? projected : path.join(workspace, '.opencode/opencode.json')); + // The generated Freshell recipe is exercised by the actual MCP probe. Keep + // every user-selected project setting in the comparison on both routes. + if (hasProjection && project?.mcp?.freshell?.type === 'local' && bridgeRecipe + && JSON.stringify(project.mcp.freshell.command) === JSON.stringify([bridgeRecipe.command, ...bridgeRecipe.args])) { + delete project.mcp.freshell; + if (Object.keys(project.mcp).length === 0) delete project.mcp; + } + const tuiConfig = tuiPath ? fs.readFileSync(tuiPath, 'utf8') : null; + const tui = tuiConfig ? JSON.parse(tuiConfig) : null; + const spec = tui?.plugin?.find(value => typeof value === 'string' + && (value.endsWith('/freshell-rebind-plugin.ts') || value.endsWith('/.freshell/opencode'))); + let rebindPlugin = null; + if (spec) { + let entry = fileURLToPath(spec); + if (fs.statSync(entry).isDirectory()) { + const target = readJson(path.join(entry, 'package.json'))?.exports?.['./tui']; + if (target !== './freshell-rebind-plugin.ts') throw new Error('selected TUI package entrypoint unavailable'); + entry = path.join(entry, target); + } + const plugin = await import(pathToFileURL(entry).href); + const emitted = []; + const deps = { env: { HOME: providerHome, FRESHELL_TERMINAL_ID: 'parity-tui-probe' }, + writeFile: (_directory, _name, body) => emitted.push(JSON.parse(body)), now: () => 1 }; + plugin.emitHello(deps); + plugin.createEmitter(deps)({ sessionID: 'ses_parity' }); + rebindPlugin = { spec, entrypoint: path.basename(entry), + hello: emitted.some(value => value.hello === true && value.source === 'opencode-tui-plugin'), + sessionId: emitted.find(value => value.source === 'opencode-tui-plugin' && value.session_id)?.session_id ?? null }; + } + return { + projectedProjectConfig: project ? JSON.stringify(project) : null, + tuiConfig, + rebindPlugin, + }; +} + +async function runProvider() { + const home = process.env.HOME ?? '' + const providerHome = { + claude: process.env.CLAUDE_CONFIG_DIR ?? path.join(home, '.claude'), + codex: process.env.CODEX_HOME ?? path.join(home, '.codex'), + opencode: path.join(process.env.XDG_CONFIG_HOME ?? path.join(home, '.config'), 'opencode'), + amplifier: process.env.AMPLIFIER_HOME ?? path.join(home, '.amplifier'), + }[provider] + const pluginRelativePath = { + claude: 'plugins/parity-plugin.txt', + codex: 'skills/parity/SKILL.md', + opencode: 'plugins/parity-plugin.js', + amplifier: 'bundles/parity-bundle.yaml', + }[provider] + const pluginPath = providerHome && pluginRelativePath ? path.join(providerHome, pluginRelativePath) : null + const environmentRelative = { + claude: 'plugins/parity-environment', codex: 'skills/parity-environment', + opencode: 'plugins/parity-environment', amplifier: 'bundles/parity-environment', + }[provider] + const execution = spawnSync(path.join(providerHome, environmentRelative, 'command'), [], { encoding: 'utf8' }) + if (execution.error || execution.status !== 0) throw new Error('selected executable/dependency/resource consumer failed') + const selectedEnvironment = JSON.parse(execution.stdout) + const selectedConfigReferences = provider === 'opencode' ? (() => { - try { - const vendor = JSON.parse(process.env.OPENCODE_CONFIG_CONTENT ?? '')?.mcp?.vendor - return vendor ? createHash('sha256').update(JSON.stringify(vendor)).digest('hex') : null - } catch { return null } + const selected = readJson(path.join(providerHome, 'opencode.jsonc'))?.agent?.parity + return [selected?.prompt, selected?.description].map(value => { + if (typeof value !== 'string') throw new Error('configured file reference missing') + return value.replace(/\{file:([^}]+)\}/g, (_match, file) => fs.readFileSync(path.resolve(providerHome, file), 'utf8')) + }) })() - : null, - providerConfig: providerHome ? Object.fromEntries( - ['settings.json', 'config.toml', 'opencode.json', 'opencode.jsonc', 'config.yaml', 'new-provider.jsonc'] - .map(name => [name, readJson(path.join(providerHome, name)) ?? (fs.existsSync(path.join(providerHome, name)) ? fs.readFileSync(path.join(providerHome, name), 'utf8') : null)]) - .filter(([, value]) => value !== null), - ) : {}, - providerPlugin: pluginPath && fs.existsSync(pluginPath) ? fs.readFileSync(pluginPath, 'utf8') : null, - providerOwned: providerHome && fs.existsSync(path.join(providerHome, 'plugins/provider-owned.txt')) - ? fs.readFileSync(path.join(providerHome, 'plugins/provider-owned.txt'), 'utf8') : null, - projectPlugin: existing('parity-plugin.js'), - projectedProjectConfig: provider === 'opencode' && providerHome - ? (fs.existsSync(path.join(providerHome, 'project/.opencode/opencode.json')) - ? fs.readFileSync(path.join(providerHome, 'project/.opencode/opencode.json'), 'utf8') : null) - : null, - nativeSession: nativeIdArgIndex >= 0 && argv[nativeIdArgIndex + 1] - ? { source: 'argv', value: argv[nativeIdArgIndex + 1] } - : { source: 'provider-store', value: null }, - projectConfig: { - opencodeJson: existing('opencode.json'), - opencodeJsonc: existing('opencode.jsonc'), - dotOpencodeJson: existing('.opencode/opencode.json'), - dotOpencodeJsonc: existing('.opencode/opencode.jsonc'), - }, - mcpRecipePresent: Boolean(mcpRecipe()), -}) -rpcProbe(mcpRecipe()) -if (provider === 'codex' || provider === 'opencode') { - // These CLIs materialize native identity on the first submitted prompt. - // Drive the existing behavioral terminal fakes after recording the launch. - const fake = path.join(import.meta.dirname, - provider === 'codex' ? 'fake-codex-terminal.mjs' : 'fake-opencode-terminal.mjs') - const terminal = spawn(process.execPath, [fake, ...argv], { stdio: ['inherit', 'pipe', 'inherit'] }) - let output = '' - terminal.stdout.on('data', chunk => { - output += String(chunk) - process.stdout.write(chunk) - const match = output.match(provider === 'codex' - ? /codex: session ([^\s]+) started/ - : /opencode: session ([^\s]+) started/) - if (match) { - append({ kind: 'identity', terminalId: process.env.FRESHELL_TERMINAL_ID, - nativeSession: { source: 'provider-store', value: match[1] } }) - output = '' - } + : [] + const privateParent = '/run/freshell-private' + const privateTopology = fs.existsSync(privateParent) + ? (() => { const stat = fs.statSync(privateParent); return { uid: stat.uid, gid: stat.gid, mode: stat.mode & 0o777 } })() + : null + const opencodeSelection = provider === "opencode" + ? await observeOpenCodeParitySelection(providerHome, cwd, process.env.OPENCODE_TUI_CONFIG, mcpRecipe()) : {}; + const nativeIdArgIndex = provider === 'claude' + ? Math.max(argv.indexOf('--session-id'), argv.indexOf('--resume')) + : provider === 'amplifier' && argv.includes('resume') ? argv.length - 2 : -1 + const nativeId = nativeIdArgIndex >= 0 ? argv[nativeIdArgIndex + 1] : null + const onecliKey = { + claude: 'ANTHROPIC_API_KEY', codex: 'OPENAI_API_KEY', + opencode: 'OPENROUTER_API_KEY', amplifier: 'OPENAI_API_KEY', + }[provider] + append({ + kind: 'launch', + selectedEnvironment, + selectedConfigReferences, + childIdentity: { uid: process.getuid(), gid: process.getgid() }, + privateTopology, + launchId, + onecliChildDigest: process.env[onecliKey] + ? createHash('sha256').update(process.env[onecliKey]).digest('hex') : null, + onecliControlPresent: Boolean(process.env.ONECLI_URL), + argv, + cwd, + env: { + FRESHELL: process.env.FRESHELL, + FRESHELL_URL: process.env.FRESHELL_URL, + FRESHELL_TOKEN_PRESENT: Boolean(process.env.FRESHELL_TOKEN), + FRESHELL_TERMINAL_ID: process.env.FRESHELL_TERMINAL_ID, + FRESHELL_TAB_ID: process.env.FRESHELL_TAB_ID, + FRESHELL_PANE_ID: process.env.FRESHELL_PANE_ID, + OPENCODE_TUI_CONFIG: process.env.OPENCODE_TUI_CONFIG, + }, + ...opencodeSelection, + inlineConfig: provider === 'opencode' + ? (() => { + try { + const vendor = JSON.parse(process.env.OPENCODE_CONFIG_CONTENT ?? '')?.mcp?.vendor + return vendor ? createHash('sha256').update(JSON.stringify(vendor)).digest('hex') : null + } catch { return null } + })() + : null, + providerConfig: providerHome ? Object.fromEntries( + ['settings.json', 'config.toml', 'opencode.json', 'opencode.jsonc', 'config.yaml', 'new-provider.jsonc'] + .map(name => [name, readJson(path.join(providerHome, name)) ?? (fs.existsSync(path.join(providerHome, name)) ? fs.readFileSync(path.join(providerHome, name), 'utf8') : null)]) + .filter(([, value]) => value !== null), + ) : {}, + providerPlugin: pluginPath && fs.existsSync(pluginPath) ? fs.readFileSync(pluginPath, 'utf8') : null, + providerOwned: providerHome && fs.existsSync(path.join(providerHome, 'plugins/provider-owned.txt')) + ? fs.readFileSync(path.join(providerHome, 'plugins/provider-owned.txt'), 'utf8') : null, + projectPlugin: existing('parity-plugin.js'), + nativeSession: nativeIdArgIndex >= 0 && argv[nativeIdArgIndex + 1] + ? { source: 'argv', value: argv[nativeIdArgIndex + 1] } + : { source: 'provider-store', value: null }, + projectConfig: { + opencodeJson: existing('opencode.json'), + opencodeJsonc: existing('opencode.jsonc'), + dotOpencodeJson: existing('.opencode/opencode.json'), + dotOpencodeJsonc: existing('.opencode/opencode.jsonc'), + }, + mcpRecipePresent: Boolean(mcpRecipe()), }) - terminal.on('exit', code => { process.exitCode = code ?? 1 }) -} else { - process.stdin.on('data', () => { - if (!nativeId) return - if (provider === 'claude') { - const sessionDir = path.join(providerHome, 'projects', 'provider-parity') - fs.mkdirSync(sessionDir, { recursive: true }) - fs.appendFileSync(path.join(sessionDir, `${nativeId}.jsonl`), - `${JSON.stringify({ type: 'user', sessionId: nativeId, cwd, message: { role: 'user', content: 'fixture prompt' } })}\n`) - } else if (provider === 'amplifier') { - const projects = path.join(providerHome, 'projects') - for (const project of fs.readdirSync(projects)) { - const events = path.join(projects, project, 'sessions', nativeId, 'events.jsonl') - if (fs.existsSync(events)) { - fs.appendFileSync(events, `${JSON.stringify({ event: 'prompt:submit', text: 'fixture prompt' })}\n`) - break + rpcProbe(mcpRecipe()) + if (provider === 'codex' || provider === 'opencode') { + // These CLIs materialize native identity on the first submitted prompt. + // Drive the existing behavioral terminal fakes after recording the launch. + const fake = path.join(import.meta.dirname, + provider === 'codex' ? 'fake-codex-terminal.mjs' : 'fake-opencode-terminal.mjs') + const terminal = spawn(process.execPath, [fake, ...argv], { stdio: ['inherit', 'pipe', 'inherit'] }) + let output = '' + terminal.stdout.on('data', chunk => { + output += String(chunk) + process.stdout.write(chunk) + const match = output.match(provider === 'codex' + ? /codex: session ([^\s]+) started/ + : /opencode: session ([^\s]+) started/) + if (match) { + append({ kind: 'identity', terminalId: process.env.FRESHELL_TERMINAL_ID, + nativeSession: { source: 'provider-store', value: match[1] } }) + output = '' + } + }) + terminal.on('exit', code => { process.exitCode = code ?? 1 }) + } else { + process.stdin.on('data', () => { + if (!nativeId) return + if (provider === 'claude') { + const sessionDir = path.join(providerHome, 'projects', 'provider-parity') + fs.mkdirSync(sessionDir, { recursive: true }) + fs.appendFileSync(path.join(sessionDir, `${nativeId}.jsonl`), + `${JSON.stringify({ type: 'user', sessionId: nativeId, cwd, message: { role: 'user', content: 'fixture prompt' } })}\n`) + } else if (provider === 'amplifier') { + const projects = path.join(providerHome, 'projects') + for (const project of fs.readdirSync(projects)) { + const events = path.join(projects, project, 'sessions', nativeId, 'events.jsonl') + if (fs.existsSync(events)) { + fs.appendFileSync(events, `${JSON.stringify({ event: 'prompt:submit', text: 'fixture prompt' })}\n`) + break + } } } - } - }) - process.stdin.resume() + }) + process.stdin.resume() + } + } + +if (["claude", "codex", "opencode", "amplifier"].includes(provider)) await runProvider(); diff --git a/test/integration/server/managed-provider-parity.test.ts b/test/integration/server/managed-provider-parity.test.ts index 7169e08ea..f8556fdce 100644 --- a/test/integration/server/managed-provider-parity.test.ts +++ b/test/integration/server/managed-provider-parity.test.ts @@ -1,13 +1,18 @@ import fs from 'node:fs' import path from 'node:path' -import os from 'node:os' -import { randomUUID } from 'node:crypto' +import { createHash, randomUUID } from 'node:crypto' +import http from 'node:http' +import { execFileSync } from 'node:child_process' +import { pathToFileURL } from 'node:url' import WebSocket from 'ws' import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest' import { WS_PROTOCOL_VERSION } from '../../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig } from '../../e2e-browser/helpers/managed-runtime.js' import { exposeGrantsToHarness, fakeOnecliGrants, withMissingGrant } from './provider-parity-onecli.js' +import { makeProviderParityResourcesReadable, prepareProviderParityUserRoots, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from './provider-parity-environment.js' +import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from './provider-parity-broker.js' +import type { InitialProviderVolumeAuthorization } from '../../../scripts/testing/runtime-test-broker.js' type Provider = 'claude' | 'codex' | 'opencode' | 'amplifier' type RecordRow = { provider: Provider; kind: 'launch' | 'mcp' | 'identity'; [key: string]: any } @@ -79,8 +84,9 @@ class TerminalWire { }) } - async create(provider: Provider, cwd: string): Promise<{ requestId: string; terminalId: string }> { + async create(provider: Provider, cwd: string, authorize?: (requestId: string) => void): Promise<{ requestId: string; terminalId: string }> { const requestId = `provider-parity-${provider}-${randomUUID()}` + authorize?.(requestId) this.send({ type: 'terminal.create', requestId, mode: provider, shell: 'system', cwd, tabId: `tab-${provider}`, paneId: `pane-${provider}` }) const frame = await this.wait(message => ( message.requestId === requestId && (message.type === 'terminal.created' || message.type === 'error') @@ -146,13 +152,14 @@ function ordinaryProviderArgs(argv: string[], provider: Provider): string[] { return result } -function normalized(record: RecordRow): unknown { +function normalized(record: RecordRow) { const env = record.env as Record const tuiConfig = record.provider === 'opencode' && typeof record.tuiConfig === 'string' ? JSON.parse(record.tuiConfig) as { plugin?: string[]; [key: string]: unknown } : record.tuiConfig if (tuiConfig && typeof tuiConfig === 'object' && Array.isArray(tuiConfig.plugin)) { - tuiConfig.plugin = tuiConfig.plugin.map(plugin => plugin.endsWith('/freshell-rebind-plugin.ts') + tuiConfig.plugin = tuiConfig.plugin.map((plugin: string) => plugin === record.rebindPlugin?.spec + && record.rebindPlugin?.hello === true && record.rebindPlugin?.sessionId === 'ses_parity' ? '' : plugin) } return { @@ -170,7 +177,14 @@ function normalized(record: RecordRow): unknown { ? { OPENCODE_TUI_CONFIG_PRESENT: Boolean(env.OPENCODE_TUI_CONFIG) } : {}), }, - providerConfig: record.providerConfig, + providerConfig: Object.fromEntries(Object.entries(record.providerConfig ?? {}).map(([name, value]) => { + if (record.provider !== 'opencode' || name !== 'opencode.jsonc') return [name, value] + const config = typeof value === 'string' ? JSON.parse(value) : structuredClone(value) + // The actual child separately consumes both references. Their transported + // paths legitimately differ between ordinary and managed HOME. + if (config.agent?.parity) config.agent.parity = { prompt: '', description: '' } + return [name, config] + })), providerPlugin: record.providerPlugin, projectedProjectConfig: record.projectedProjectConfig, tuiConfig, @@ -208,45 +222,52 @@ describe('managed provider parity argv comparison', () => { }) }) -function makeReadableTree(directory: string): void { - fs.chmodSync(directory, 0o755) - for (const entry of fs.readdirSync(directory, { withFileTypes: true })) { - const file = path.join(directory, entry.name) - if (entry.isDirectory()) { - fs.chmodSync(file, 0o755) - makeReadableTree(file) - } else { - fs.chmodSync(file, 0o644) - } - } -} - describe('ordinary and managed terminal provider parity', () => { let rig: ManagedRuntimeBrowserRig + let initialProviderSource: Omit let workspace: string let webHome: string let xdgConfigHome: string let inheritedFreshellUrl: string | undefined + let inheritedProviderEnvironment: Record = {} let onecli: ReturnType let restoreOnecliEnvironment: (() => void) | undefined const sockets: TerminalWire[] = [] + const registeredVolumes: (InitialProviderVolumeAuthorization & { requestId: string })[] = [] + const userOpencodeConfig = () => JSON.stringify({ + plugin: [pathToFileURL(path.join(xdgConfigHome, 'opencode/plugins/parity-plugin.js')).href], + agent: { parity: { + prompt: '{file:./parity-policy.txt}', + description: `{file:${path.join(webHome, 'absolute-policy.txt')}}`, + } }, + }) + const homeTuiConfig = () => JSON.stringify({ + plugin: [pathToFileURL(path.join(xdgConfigHome, 'opencode/home-selected-tui.js')).href], + }) + const projectOpencodeConfig = () => JSON.stringify({ + plugin: [pathToFileURL(path.join(workspace, 'parity-plugin.js')).href], + }) beforeAll(async () => { inheritedFreshellUrl = process.env.FRESHELL_URL delete process.env.FRESHELL_URL workspace = fs.mkdtempSync(path.join(root, '.provider-parity-')) - webHome = fs.mkdtempSync(path.join(os.tmpdir(), 'freshell-parity-home-')) + // Select this synthetic project, without inheriting repository-local skills. + execFileSync('git', ['init', '--quiet', workspace]) + webHome = fs.mkdtempSync(path.join(root, '.provider-parity-home-')) onecli = fakeOnecliGrants(providers) restoreOnecliEnvironment = exposeGrantsToHarness(onecli.serverEnv) xdgConfigHome = path.join(webHome, 'custom-xdg') - fs.mkdirSync(path.join(xdgConfigHome, 'opencode'), { recursive: true }) - fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), '{ // home-selected\n "plugin": ["file://home-selected-tui.js"]\n}\n') + prepareProviderParityUserRoots(webHome, xdgConfigHome) + fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), homeTuiConfig()) + fs.writeFileSync(path.join(xdgConfigHome, 'opencode/home-selected-tui.js'), 'export default { name: "home-selected-tui" }\n') fs.chmodSync(workspace, 0o777) fs.mkdirSync(path.join(workspace, '.opencode'), { mode: 0o777 }) - fs.writeFileSync(path.join(workspace, 'opencode.jsonc'), '{ // provider parity\n "plugin": ["file://parity-plugin.js"]\n}\n') - fs.writeFileSync(path.join(workspace, '.opencode/opencode.json'), JSON.stringify({ plugin: ['file://parity-plugin.js'] })) + fs.writeFileSync(path.join(workspace, 'opencode.jsonc'), projectOpencodeConfig()) + fs.writeFileSync(path.join(workspace, '.opencode/opencode.json'), projectOpencodeConfig()) fs.writeFileSync(path.join(workspace, 'parity-plugin.js'), 'export default {}\n') - fs.writeFileSync(path.join(workspace, 'user-tui.jsonc'), '{ // user-selected\n "plugin": ["file://user-selected-tui.js"]\n}\n') + fs.writeFileSync(path.join(workspace, 'user-tui.jsonc'), JSON.stringify({ plugin: [pathToFileURL(path.join(workspace, 'user-selected-tui.js')).href] })) + fs.writeFileSync(path.join(workspace, 'user-selected-tui.js'), 'export default { name: "user-selected-tui" }\n') fs.chmodSync(path.join(workspace, '.opencode'), 0o777) for (const [source, target] of [ [fixture, 'parity-provider.mjs'], @@ -257,12 +278,12 @@ describe('ordinary and managed terminal provider parity', () => { ] as const) fs.copyFileSync(source, path.join(workspace, target)) fs.mkdirSync(path.join(workspace, 'node_modules'), { mode: 0o755 }) fs.cpSync(fs.realpathSync(path.join(root, 'node_modules/ws')), path.join(workspace, 'node_modules/ws'), { recursive: true }) - makeReadableTree(path.join(workspace, 'node_modules/ws')) + makeProviderParityResourcesReadable(path.join(workspace, 'node_modules')) for (const provider of providers) { const wrapper = path.join(workspace, `parity-${provider}`) const script = provider === 'codex' || provider === 'opencode' - ? `#!/usr/bin/env node\nconst target = process.argv.includes(${JSON.stringify(provider === 'codex' ? 'app-server' : 'serve')}) ? ${JSON.stringify(path.join(workspace, provider === 'codex' ? 'fake-app-server.mjs' : 'fake-opencode.cjs'))} : ${JSON.stringify(path.join(workspace, 'parity-provider.mjs'))}\nvoid import(target).catch(error => { console.error(error); process.exitCode = 1 })\n` - : `#!/usr/bin/env node\nimport ${JSON.stringify(path.join(workspace, 'parity-provider.mjs'))}\n` + ? `#!/usr/bin/env node\nconst target = process.argv.includes(${JSON.stringify(provider === 'codex' ? 'app-server' : 'serve')}) ? ${JSON.stringify(provider === 'codex' ? './fake-app-server.mjs' : './fake-opencode.cjs')} : './parity-provider.mjs'\nvoid import(new URL(target, import.meta.url)).catch(error => { console.error(error); process.exitCode = 1 })\n` + : `#!/usr/bin/env node\nimport './parity-provider.mjs'\n` fs.writeFileSync(wrapper, script, { mode: 0o755 }) fs.chmodSync(wrapper, 0o755) } @@ -276,9 +297,6 @@ describe('ordinary and managed terminal provider parity', () => { endpoint: `https://${nestedSecretMarker}.example`, } } }), ...onecli.serverEnv, - ...Object.fromEntries( - providers.map(provider => [`${provider.toUpperCase()}_CMD`, path.join(workspace, `parity-${provider}`)]), - ), }, { NODE_OPTIONS: '', XDG_CONFIG_HOME: xdgConfigHome }, 'test', { enabledProviders: providers, providerSettings: { @@ -288,11 +306,57 @@ describe('ordinary and managed terminal provider parity', () => { amplifier: { model: 'glm-5.3', effort: 'provider-default' }, }, }, webHome) + // The controller validates executables before setting a child's cwd. + // The worker environment selects installed commands and owned OpenCode roots. + const providerEnvironment = providerParityCommands(path.join(rig.runtime.testRoot, 'workspace')) + inheritedProviderEnvironment = Object.fromEntries(Object.keys(providerEnvironment).map(key => [key, process.env[key]])) + Object.assign(process.env, providerEnvironment) await rig.start() + // prepare() recreates this run's root. Populate the project afterwards so + // provider configuration discovery cannot inherit the repository's inputs. + const preparedWorkspace = workspace + workspace = path.join(rig.runtime.testRoot, 'workspace') + fs.cpSync(preparedWorkspace, workspace, { recursive: true }) + // cpSync creates a private destination directory. Restore only the owned + // project leaf's shared mode so the provider user can traverse its mount. + fs.chmodSync(workspace, 0o777) + makeProviderParityResourcesReadable(path.join(workspace, 'node_modules')) + fs.rmSync(preparedWorkspace, { recursive: true }) + fs.writeFileSync(path.join(workspace, 'opencode.jsonc'), projectOpencodeConfig()) + fs.writeFileSync(path.join(workspace, '.opencode/opencode.json'), projectOpencodeConfig()) + fs.writeFileSync(path.join(workspace, 'user-tui.jsonc'), JSON.stringify({ plugin: [pathToFileURL(path.join(workspace, 'user-selected-tui.js')).href] })) + const health = await rig.runtime.adminOk(rig.supervisor, { method: 'health' }) + expect(health.kind).toBe('health') + const daemon = await new Promise<{ ID: string }>((resolve, reject) => { + const request = http.get({ socketPath: rig.runtime.broker.proxySocketPath, path: '/v1.47/info' }, response => { + const chunks: Buffer[] = [] + let size = 0 + response.on('data', chunk => { + size += chunk.length + if (size > 64 * 1024) request.destroy(new Error('owned daemon metadata exceeds fixture bound')) + else chunks.push(Buffer.from(chunk)) + }) + response.once('error', reject) + response.once('end', () => { + try { + if (response.statusCode !== 200) throw new Error('owned daemon metadata unavailable') + resolve(JSON.parse(Buffer.concat(chunks).toString('utf8'))) + } catch (error) { reject(error) } + }) + }) + request.once('error', reject) + }) + initialProviderSource = { + installationId: health.data.installation_id, + imageRef: rig.runtime.imageRef, + hostBinaryPath: rig.runtime.testHostBinary, + binarySha256: createHash('sha256').update(fs.readFileSync(rig.runtime.testHostBinary)).digest('hex'), + daemonId: daemon.ID, + } for (const [directory, filename, content] of [ ['.claude', 'settings.json', JSON.stringify({ paritySetting: true })], ['.codex', 'config.toml', 'parity_setting = true\n'], - ['custom-xdg/opencode', 'opencode.jsonc', '{ "plugin": ["file://user-parity-plugin.js"] }\n'], + ['custom-xdg/opencode', 'opencode.jsonc', userOpencodeConfig()], ['.amplifier', 'config.yaml', 'parity_setting: true\n'], ]) { const location = path.join(rig.info.homeDir, directory) @@ -309,15 +373,24 @@ describe('ordinary and managed terminal provider parity', () => { fs.mkdirSync(path.dirname(target), { recursive: true }) fs.writeFileSync(target, content) } + for (const [directory, relative] of [ + ['.claude', 'plugins/parity-environment'], ['.codex', 'skills/parity-environment'], + ['custom-xdg/opencode', 'plugins/parity-environment'], ['.amplifier', 'bundles/parity-environment'], + ]) writeProviderParityEnvironment(path.join(rig.info.homeDir, directory), relative) + fs.writeFileSync(path.join(xdgConfigHome, 'opencode/parity-policy.txt'), 'relative-config-reference-nonce') + fs.writeFileSync(path.join(webHome, 'absolute-policy.txt'), 'absolute-config-reference-nonce') }, 900_000) beforeEach(() => { // Vitest shuffles cases. Each case sees the same user-owned inputs even // when a prior recovery case edited them to exercise refresh semantics. - fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), '{ // home-selected\n "plugin": ["file://home-selected-tui.js"]\n}\n') - fs.writeFileSync(path.join(workspace, '.opencode/opencode.json'), JSON.stringify({ plugin: ['file://parity-plugin.js'] })) + fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), homeTuiConfig()) + fs.writeFileSync(path.join(workspace, '.opencode/opencode.json'), projectOpencodeConfig()) + // Every provider observes this shared project file. Prepare the selected + // owned subtree for the provider user independently of shuffled case order. + makeProviderParityResourcesReadable(path.join(workspace, '.opencode')) const userConfig = path.join(xdgConfigHome, 'opencode') - fs.writeFileSync(path.join(userConfig, 'opencode.jsonc'), '{ "plugin": ["file://user-parity-plugin.js"] }\n') + fs.writeFileSync(path.join(userConfig, 'opencode.jsonc'), userOpencodeConfig()) fs.rmSync(path.join(userConfig, 'new-provider.jsonc'), { force: true }) fs.writeFileSync(path.join(userConfig, 'plugins/parity-plugin.js'), 'export default { name: "parity" }\n') }) @@ -326,14 +399,28 @@ describe('ordinary and managed terminal provider parity', () => { try { await Promise.allSettled(sockets.map(socket => socket.close())) if (rig) { + let volumes: ReturnType = [] + let captureError: unknown + try { + volumes = captureProviderParityVolumes(registeredVolumes) + rig.runtime.writeIncident('provider-parity-registered-volumes', volumes) + } catch (error) { captureError = error } const cleanup = await rig.stop() expect(cleanup.ok, cleanup.errors.join('\n')).toBe(true) + if (captureError) throw captureError + rig.runtime.writeIncident('provider-parity-registered-volume-cleanup', { + removed: cleanupProviderParityVolumes(volumes), + }) } } finally { if (workspace) fs.rmSync(workspace, { recursive: true, force: true }) if (webHome) fs.rmSync(webHome, { recursive: true, force: true }) if (onecli) fs.rmSync(onecli.directory, { recursive: true, force: true }) restoreOnecliEnvironment?.() + for (const [key, value] of Object.entries(inheritedProviderEnvironment)) { + if (value === undefined) delete process.env[key] + else process.env[key] = value + } if (inheritedFreshellUrl === undefined) delete process.env.FRESHELL_URL else process.env.FRESHELL_URL = inheritedFreshellUrl } @@ -346,13 +433,41 @@ describe('ordinary and managed terminal provider parity', () => { let managedSoulId: string | undefined let managedTerminalId: string | undefined for (const managed of [false, true]) { - if (provider === 'opencode') makeReadableTree(path.join(workspace, '.opencode')) const wire = await TerminalWire.connect(rig.info, managed) sockets.push(wire) let terminalId: string + let managedRequest: InitialProviderVolumeAuthorization | undefined try { - ;({ terminalId } = await wire.create(provider, workspace)) + ;({ terminalId } = await wire.create(provider, workspace, managed + ? requestId => { + managedRequest = authorizeProviderParityRequest(rig.runtime.broker, initialProviderSource, requestId) + registeredVolumes.push({ ...managedRequest, requestId }) + rig.runtime.writeIncident('provider-parity-registered-requests', registeredVolumes) + } + : undefined)) } catch (error) { + if (managedRequest) { + const receipt = rig.runtime.broker.receipts().findLast(value => value.soulId === managedRequest!.soulId && value.workspacePath === workspace) + if (receipt) { + const diagnostic: Record = { containerId: receipt.containerId, soulId: receipt.soulId } + const logs = rig.runtime.containerLogs(receipt.containerId) + Object.assign(diagnostic, providerParityFailureDiagnostics(logs)) + try { + diagnostic.guest = JSON.parse(rig.runtime.execOwnedContainerAsExact(receipt.containerId, '65534:0', [ + '/usr/local/bin/node', '-e', ` +const fs=require('node:fs'),path=require('node:path'); +const workspace=process.argv[1],command=process.argv[2]; +function metadata(file){try{const s=fs.lstatSync(file);let executable=false,readable=false;try{fs.accessSync(file,fs.constants.X_OK);executable=true}catch{}try{fs.accessSync(file,fs.constants.R_OK);readable=true}catch{}return{mode:s.mode&511,uid:s.uid,gid:s.gid,executable,readable}}catch{return{available:false}}} +const ancestors=[];for(let current=path.dirname(command),depth=0;depth<64;depth++){ancestors.push(metadata(current));const parent=path.dirname(current);if(parent===current)break;current=parent} +const mounts=fs.readFileSync('/proc/self/mountinfo','utf8').trim().split('\\n').map(line=>line.split(' ')).filter(fields=>workspace===fields[4]||workspace.startsWith(fields[4]+'/')).sort((a,b)=>b[4].length-a[4].length); +const mount=mounts[0],separator=mount?.indexOf('-'); +process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),command:metadata(command),dependency:{parent:metadata(path.join(workspace,'node_modules')),package:metadata(path.join(workspace,'node_modules/ws')),entry:metadata(path.join(workspace,'node_modules/ws/index.js'))},ancestors,mount:mount?{options:mount[5],filesystem:mount[separator+1],superOptions:mount[separator+3]}:null})); +`, workspace, path.join(workspace, `parity-${provider}`), + ])) + } catch { diagnostic.guest = { available: false } } + rig.runtime.writeIncident(`provider-parity-${provider}-create-failure`, diagnostic) + } + } throw new Error(`${managed ? 'managed' : 'ordinary'} ${provider} create: ${String(error)}`) } const launch = await waitFor(`${provider} ${managed ? 'managed' : 'ordinary'} launch`, () => ( @@ -361,7 +476,15 @@ describe('ordinary and managed terminal provider parity', () => { throw new Error(`${String(error)}; records: ${JSON.stringify(wire.recordRows())}; output: ${wire.terminalOutput()}`) }) observed.push(launch) + expect(launch.selectedEnvironment).toEqual({ + dependency: 'installed-dependency-nonce', resource: 'selected-resource-nonce', entries: 4201, + }) + expect(launch.selectedConfigReferences).toEqual(provider === 'opencode' + ? ['relative-config-reference-nonce', 'absolute-config-reference-nonce'] : []) + for (const sentinel of [nestedSecretMarker, 'fixture-secret-byte']) expect(JSON.stringify(launch)).not.toContain(sentinel) if (managed) { + expect(launch.childIdentity).toEqual({ uid: 65534, gid: 0 }) + expect(launch.privateTopology).toEqual({ uid: 0, gid: 0, mode: 0o710 }) managedTerminalId = terminalId expect(launch.onecliChildDigest).toBe(onecli.childDigests[provider]) } else { @@ -385,6 +508,7 @@ describe('ordinary and managed terminal provider parity', () => { wire.recordRows().find(row => row.provider === provider && row.kind === 'mcp') )) mcpResults.push(mcp) + for (const sentinel of [nestedSecretMarker, 'fixture-secret-byte']) expect(JSON.stringify(mcp)).not.toContain(sentinel) if (provider !== 'amplifier') { expect(mcp.error, `${provider} ${managed ? 'managed' : 'ordinary'} MCP probe`).toBeUndefined() expect(mcp.tools).toContain('freshell') @@ -402,8 +526,12 @@ describe('ordinary and managed terminal provider parity', () => { expect(normalized(observed[1])).toEqual(normalized(observed[0])) if (provider === 'opencode') { const selected = JSON.parse(observed[1].tuiConfig) - expect(selected.plugin, `managed TUI: ${observed[1].tuiConfig}; direct TUI: ${observed[0].tuiConfig}`).toContain('file://home-selected-tui.js') - expect(selected.plugin.some((plugin: string) => plugin.endsWith('/freshell-rebind-plugin.ts'))).toBe(true) + expect(selected.plugin, `managed TUI: ${observed[1].tuiConfig}; direct TUI: ${observed[0].tuiConfig}`) + .toContain(pathToFileURL(path.join(xdgConfigHome, 'opencode/home-selected-tui.js')).href) + expect(selected.plugin).toContain(observed[1].rebindPlugin.spec) + for (const launch of observed) { + expect(launch.rebindPlugin).toMatchObject({ entrypoint: 'freshell-rebind-plugin.ts', hello: true, sessionId: 'ses_parity' }) + } expect(observed[1].inlineConfig).toEqual(observed[0].inlineConfig) expect(observed[1].inlineConfig).toMatch(/^[a-f0-9]{64}$/) } @@ -609,7 +737,7 @@ describe('ordinary and managed terminal provider parity', () => { ]) await wire.wait(frame => frame.type === 'terminal.session.associated' && frame.terminalId === terminalId && frame.sessionRef?.sessionId === switchedId, 30_000).catch(async error => { - throw new Error(`${String(error)}; inventory: ${JSON.stringify(await rig.inventory())}; web: ${rig.web.capturedOutput().slice(-3000)}`) + throw new Error(`${String(error)}; inventory: ${JSON.stringify(await rig.inventory())}; web: ${JSON.stringify(providerParityFailureDiagnostics(Object.values(rig.web.capturedOutput()).join('\n'))).slice(-3000)}`) }) await wire.wait(frame => frame.type === 'terminal.meta.updated' && frame.upsert?.some((row: any) => row.terminalId === terminalId && row.sessionId === switchedId), 30_000) @@ -624,7 +752,7 @@ describe('ordinary and managed terminal provider parity', () => { const view = await rig.runningViewForTerminal(terminalId) return view?.containerId && view.incarnationId !== before.incarnationId ? view : undefined }, 120_000).catch(async error => { - throw new Error(`${String(error)}; inventory: ${JSON.stringify(await rig.inventory())}; supervisor: ${rig.runtime.containerLogs(rig.supervisor.containerId).slice(-4000)}; web: ${rig.web.capturedOutput().slice(-4000)}`) + throw new Error(`${String(error)}; inventory: ${JSON.stringify(await rig.inventory())}; supervisor: ${rig.runtime.containerLogs(rig.supervisor.containerId).slice(-4000)}; web: ${JSON.stringify(providerParityFailureDiagnostics(Object.values(rig.web.capturedOutput()).join('\n'))).slice(-4000)}`) }) expect(after.soulId).toBe(before.soulId) expect((await rig.inventory()).find((row: any) => row.terminalId === terminalId)?.nativeSessionId).toBe(switchedId) diff --git a/test/integration/server/provider-parity-broker.ts b/test/integration/server/provider-parity-broker.ts new file mode 100644 index 000000000..d75903256 --- /dev/null +++ b/test/integration/server/provider-parity-broker.ts @@ -0,0 +1,80 @@ +import { createHash } from 'node:crypto' +import { execFileSync } from 'node:child_process' +import fs from 'node:fs' + +import type { InitialProviderVolumeAuthorization, RestrictedDockerBroker } from '../../../scripts/testing/runtime-test-broker.js' + +export function authorizeProviderParityRequest( + broker: RestrictedDockerBroker, + source: Omit, + requestId: string, +): InitialProviderVolumeAuthorization { + const soulId = `soul-terminal-${createHash('sha256').update(requestId).digest('hex').slice(0, 32)}` + const providerVolumeName = `freshell-provider-${createHash('sha256').update(`${source.installationId}\0${soulId}`).digest('hex').slice(0, 24)}` + const authorization = { ...source, soulId, providerVolumeName } + broker.authorizeInitialProviderVolume(authorization) + return authorization +} + +type DockerCommand = (args: string[]) => string +type CapturedProviderVolume = { owner: InitialProviderVolumeAuthorization; labels: Record } +const docker: DockerCommand = args => execFileSync('docker', args, { + encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], maxBuffer: 1024 * 1024, timeout: 120_000, +}) + +function volumeExists(name: string, run: DockerCommand): boolean { + return run(['volume', 'ls', '--filter', `name=^${name}$`, '--format', '{{.Name}}']).trim() === name +} + +function ownedVolume(owner: InitialProviderVolumeAuthorization, run: DockerCommand): Record { + const value = JSON.parse(run(['volume', 'inspect', owner.providerVolumeName, '--format', '{{json .}}'])) + const labels = value.Labels ?? {} + const expected = { + 'com.freshell.installation-id': owner.installationId, 'com.freshell.soul-id': owner.soulId, + 'freshell.input.daemon': owner.daemonId, 'freshell.input.image': owner.imageRef, + 'freshell.input.binary-sha256': owner.binarySha256, 'freshell.input.intent': '1', + 'freshell.input.reservation': labels['freshell.input.reservation'], + 'freshell.input.incarnation': labels['freshell.input.incarnation'], + } + if (value.Name !== owner.providerVolumeName || value.Driver !== 'local' + || (value.Options && Object.keys(value.Options).length) + || Object.keys(labels).length !== Object.keys(expected).length + || !Object.entries(expected).every(([key, value]) => labels[key] === value) + || !/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(labels['freshell.input.reservation']) + || !/^[a-zA-Z0-9_-]{1,128}$/.test(labels['freshell.input.incarnation'])) { + throw new Error('registered provider volume ownership changed; retain it') + } + return labels +} + +/** Capture before rig.stop removes the run-owned binary copies. */ +export function captureProviderParityVolumes(owners: InitialProviderVolumeAuthorization[], run = docker): CapturedProviderVolume[] { + return owners.flatMap(owner => { + if (!volumeExists(owner.providerVolumeName, run)) return [] + const binary = fs.lstatSync(owner.hostBinaryPath) + if (!binary.isFile() || binary.isSymbolicLink() || binary.size > 128 * 1024 * 1024 + || createHash('sha256').update(fs.readFileSync(owner.hostBinaryPath)).digest('hex') !== owner.binarySha256 + || run(['info', '--format', '{{.ID}}']).trim() !== owner.daemonId) { + throw new Error('registered provider volume artifact or daemon changed; retain it') + } + return [{ owner, labels: ownedVolume(owner, run) }] + }) +} + +/** Called only after rig.stop joins its workloads; never force or prune. */ +export function cleanupProviderParityVolumes(volumes: CapturedProviderVolume[], run = docker): string[] { + const removed: string[] = [] + for (const { owner, labels } of volumes) { + if (!volumeExists(owner.providerVolumeName, run)) continue + const current = ownedVolume(owner, run) + if (!Object.entries(labels).every(([key, value]) => current[key] === value) + || run(['info', '--format', '{{.ID}}']).trim() !== owner.daemonId + || run(['container', 'ls', '-a', '--filter', `volume=${owner.providerVolumeName}`, '--format', '{{.ID}}']).trim()) { + throw new Error('registered provider volume remains attached or changed; retain it') + } + run(['volume', 'rm', owner.providerVolumeName]) + if (volumeExists(owner.providerVolumeName, run)) throw new Error('registered provider volume cleanup not confirmed') + removed.push(owner.providerVolumeName) + } + return removed +} diff --git a/test/integration/server/provider-parity-environment.ts b/test/integration/server/provider-parity-environment.ts new file mode 100644 index 000000000..1ef3c2ec6 --- /dev/null +++ b/test/integration/server/provider-parity-environment.ts @@ -0,0 +1,103 @@ +import fs from 'node:fs' +import path from 'node:path' + +/** Establish selected fixture roots before the harness captures its policy. */ +export function prepareProviderParityUserRoots(home: string, xdgConfigHome: string): void { + for (const root of ['.claude', '.codex', '.amplifier']) { + fs.mkdirSync(path.join(home, root), { recursive: true }) + } + fs.mkdirSync(path.join(xdgConfigHome, 'opencode'), { recursive: true }) +} + +/** Retain only fixed classifications from the owned host's existing logs. */ +export function providerParityFailureDiagnostics(logs: string) { + const events = logs.split('\n').flatMap(line => { + try { + const value = JSON.parse(line) + return value !== null && typeof value === 'object' && !Array.isArray(value) ? [value] : [] + } catch { return [] } + }) + const classify = (cause: unknown) => { + const message = typeof cause === 'string' ? cause : '' + const operations = [ + ['provider transaction backup ', 'backup'], ['provider transaction install ', 'install'], + ['provider transaction create ', 'create'], ['provider transaction remove directory ', 'remove_directory'], + ['provider transaction publish manifest:', 'manifest'], ['provider transaction mounted file ', 'mounted_file'], + ['provider refresh control ', 'control'], ['provider refresh transaction creation ', 'create_stage'], + ] as const + const errors = [ + ['(os error 18)', 'cross_device'], ['(os error 13)', 'permission_denied'], + ['(os error 2)', 'missing'], ['(os error 30)', 'read_only'], + ['(os error 16)', 'busy'], ['(os error 1)', 'operation_not_permitted'], + ] as const + return { + operation: operations.find(([prefix]) => message.startsWith(prefix))?.[1] ?? 'unknown', + errno: errors.find(([marker]) => message.includes(marker))?.[1] ?? 'unknown', + } + } + return { + transactionEvents: events.flatMap(value => { + return ['provider.projection.transaction.started', 'provider.projection.transaction.failed', + 'provider.projection.transaction.rollback', 'provider.projection.transaction.committed'].includes(value.event) + && ['stage', 'validate', 'publish', 'rollback', 'commit'].includes(value.stage) + ? [{ event: value.event, stage: value.stage }] : [] + }).slice(-16), + transactionFailures: events.flatMap(value => value.event === 'provider.projection.transaction.failed' + && ['stage', 'validate', 'publish'].includes(value.stage) + ? [{ stage: value.stage, ...classify(value.cause) }] : []).slice(-16), + workerFailures: events.filter(value => value.event === 'session_host.fatal') + .slice(-16).map(value => classify(value.error)), + filesystemCause: { + permissionDenied: logs.includes('Permission denied'), missingPath: logs.includes('No such file or directory'), + readOnly: logs.includes('Read-only file system'), unownedDestination: logs.includes('provider-created'), + }, + } +} + +/** Make only the owned fixture's selected resources readable to its child. */ +export function makeProviderParityResourcesReadable(directory: string): void { + fs.chmodSync(directory, 0o755) + for (const entry of fs.readdirSync(directory, { withFileTypes: true })) { + const file = path.join(directory, entry.name) + if (entry.isDirectory()) makeProviderParityResourcesReadable(file) + else { + const source = fs.lstatSync(file) + if (!source.isFile()) throw new Error('selected fixture resource is not a physical regular file') + fs.chmodSync(file, 0o644 | (source.mode & 0o111)) + } + } +} + +/** Build installed, offline resources; the child must execute and import them. */ +export function writeProviderParityEnvironment(providerRoot: string, relative: string): void { + const selected = path.join(providerRoot, relative) + const dependency = path.join(selected, 'node_modules/parity-installed') + fs.mkdirSync(path.join(dependency, 'entries'), { recursive: true }) + // The executable keeps its module format when selected from an ESM project. + fs.writeFileSync(path.join(selected, 'package.json'), JSON.stringify({ type: 'commonjs' })) + fs.writeFileSync(path.join(dependency, 'package.json'), JSON.stringify({ name: 'parity-installed', main: 'index.cjs' })) + fs.writeFileSync(path.join(dependency, 'index.cjs'), "module.exports=require('./entries/entry-4200.cjs')\n") + for (let index = 0; index <= 4200; index++) { + fs.writeFileSync(path.join(dependency, `entries/entry-${index}.cjs`), "module.exports='installed-dependency-nonce'\n") + } + fs.writeFileSync(path.join(selected, 'asset'), 'selected-resource-nonce') + fs.writeFileSync(path.join(selected, 'references.json'), JSON.stringify({ resource: './asset' })) + fs.writeFileSync(path.join(selected, 'command'), `#!/usr/bin/env node +const fs=require('node:fs'),path=require('node:path'); +const config=JSON.parse(fs.readFileSync(path.join(__dirname,'references.json'),'utf8')); +process.stdout.write(JSON.stringify({dependency:require('parity-installed'),resource:fs.readFileSync(path.resolve(__dirname,config.resource),'utf8'),entries:fs.readdirSync(path.join(__dirname,'node_modules/parity-installed/entries')).length})); +`, { mode: 0o751 }) +} + +/** Provider commands and OpenCode selection for the owned fixture workspace. */ +export function providerParityCommands(workspace: string): Record { + const commands = Object.fromEntries(['claude', 'codex', 'opencode', 'amplifier'].map(provider => [ + `${provider.toUpperCase()}_CMD`, path.join(workspace, `parity-${provider}`), + ])) + return { + ...commands, + OPENCODE_DISABLE_PROJECT_CONFIG: '1', + OPENCODE_CONFIG_DIR: path.join(workspace, '.opencode'), + OPENCODE_CONFIG: path.join(workspace, 'opencode.jsonc'), + } +} diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts new file mode 100644 index 000000000..257a55c9c --- /dev/null +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -0,0 +1,290 @@ +import { createHash } from 'node:crypto' +import fs from 'node:fs/promises' +import http from 'node:http' +import os from 'node:os' +import path from 'node:path' +import { describe, expect, it, vi } from 'vitest' + +const fakeDocker = vi.hoisted(() => ({ settled: false, commands: [] as string[][] })) +vi.mock('node:child_process', () => { + const api = { + spawn: () => { throw new Error('unexpected spawned process in isolated harness fixture') }, + execFileSync: (program: string, args: string[]) => { + if (program === 'git' && args.join(' ') === 'rev-parse HEAD') return 'cleanup-fixture-commit\n' + throw new Error('unexpected executable in isolated cleanup fixture') + }, + spawnSync: (program: string, args: string[]) => { + if (program !== 'docker' || !['inspect', 'rm'].includes(args[0]) || !args.at(-1)?.match(/^[a-f0-9]{64}$/)) { + throw new Error('unexpected child process in isolated cleanup fixture') + } + fakeDocker.commands.push(args) + if (args[0] === 'rm') fakeDocker.settled = true + return { status: 0, stdout: '', stderr: '' } + }, + } + return { ...api, default: api } +}) + +import { RuntimeHarness } from '../../../../scripts/testing/runtime-sandbox.js' +import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' +import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' +import { prepareProviderParityUserRoots } from '../../../integration/server/provider-parity-environment.js' + +function useOwnedHarnessArtifacts(harness: RuntimeHarness, host: string): void { + const artifact = harness as unknown as { ensureRuntimeImage(): void; buildBinaries(): void; collectCapabilities(): unknown; collectBuildInfo(): unknown } + vi.spyOn(artifact, 'ensureRuntimeImage').mockImplementation(() => { harness.imageRef = `sha256:${'a'.repeat(64)}` }) + vi.spyOn(artifact, 'buildBinaries').mockImplementation(() => { harness.testHostBinary = host; harness.releaseHostBinary = host }) + vi.spyOn(artifact, 'collectCapabilities').mockReturnValue({ fixture: true }) + vi.spyOn(artifact, 'collectBuildInfo').mockReturnValue({ fixture: true }) +} + +it.each(['claude', 'opencode'] as const)('qualifies original isolated HOME aliases through the actual browser harness policy: %s', async provider => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-home-')) + const repository = path.join(directory, 'repository') + const home = path.join(directory, 'home') + const workspace = path.join(repository, 'workspace') + const xdgConfigHome = path.join(home, 'custom-xdg') + const socket = path.join(directory, 'daemon.sock') + const forwarded: string[] = [] + const daemon = http.createServer((request, response) => { + forwarded.push(request.url!) + response.statusCode = request.method === 'POST' ? 201 : 404 + response.end('{}') + }) + const inheritedDockerHost = process.env.DOCKER_HOST + let harness: RuntimeHarness | undefined + try { + await fs.mkdir(home, { recursive: true }) + await fs.mkdir(path.join(xdgConfigHome, 'opencode'), { recursive: true }) + await fs.mkdir(workspace, { recursive: true }) + await fs.mkdir(path.join(repository, 'test/runtime'), { recursive: true }) + await fs.writeFile(path.join(repository, 'test/runtime/gate-manifest.json'), '{}') + await new Promise(resolve => daemon.listen(socket, resolve)) + process.env.DOCKER_HOST = `unix://${socket}` + const rig = new ManagedRuntimeBrowserRig(repository, 2, { XDG_CONFIG_HOME: xdgConfigHome }, {}, 'test', { enabledProviders: [provider] }, home) + harness = rig.runtime + const host = path.join(directory, 'host') + await fs.writeFile(host, 'owned fixture host artifact') + useOwnedHarnessArtifacts(harness, host) + const outsideRoot = path.join(directory, 'outside-provider') + await fs.mkdir(outsideRoot) + await fs.symlink(outsideRoot, path.join(home, '.amplifier')) + await expect(harness.prepare()).rejects.toThrow('selected owned physical directory') + expect(forwarded).toEqual([]) + await fs.unlink(path.join(home, '.amplifier')) + prepareProviderParityUserRoots(home, xdgConfigHome) + if (provider === 'opencode') { + const symlink = path.join(home, 'linked-xdg') + await fs.symlink(xdgConfigHome, symlink) + for (const selected of [home, outsideRoot, path.join(home, '../sibling'), symlink, + `${xdgConfigHome}/../custom-xdg`, path.join(home, '.codex/sessions')]) { + const invalid = new ManagedRuntimeBrowserRig(repository, 2, { XDG_CONFIG_HOME: selected }, {}, 'test', { enabledProviders: [provider] }, home).runtime + useOwnedHarnessArtifacts(invalid, host) + try { + await expect(invalid.prepare(), selected).rejects.toThrow(/provider fixture (root|XDG root)/) + expect(forwarded).toEqual([]) + } finally { + await invalid.broker?.close() + await fs.rm(invalid.testRoot, { recursive: true, force: true }) + } + } + } + await harness.prepare() + const generationId = 'grant-11111111-1111-4111-8111-111111111111' + const reservationId = '22222222-2222-4222-8222-222222222222' + const requestId = '33333333-3333-4333-8333-333333333333' + const installationId = 'installation-home', soulId = 'soul-home' + const owner = { installationId, soulId, imageRef: harness.imageRef, hostBinaryPath: host, + binarySha256: createHash('sha256').update(await fs.readFile(host)).digest('hex'), daemonId: 'daemon-home', + providerVolumeName: `freshell-provider-${createHash('sha256').update(`${installationId}\0${soulId}`).digest('hex').slice(0, 24)}` } + harness.broker.authorizeInitialProviderVolume(owner) + const control = path.join(harness.testRoot, 'scenarios/scenario/control') + const metadata = path.join(control, 'provider-roots', generationId, '.freshell') + await fs.mkdir(metadata, { recursive: true }) + await fs.mkdir(path.join(metadata, 'templates')) + await fs.writeFile(path.join(metadata, 'templates/source-0'), '{"fixtureSetting":true}') + await fs.mkdir(path.join(control, 'mcp-capabilities'), { recursive: true }) + const manifest = Buffer.from(JSON.stringify({ version: 1, sources: [], aliases: [], recipes: [] })) + const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') + await fs.writeFile(path.join(metadata, 'input-manifest.json'), manifest) + const indexedVolume = `freshell-inputs-${createHash('sha256').update(`${soulId}\0${generationId}`).digest('hex').slice(0, 24)}` + const context = { preparation: provider === 'claude' ? { claude: { mcp_args: [] } } + : { opencode: { project_config: [], tui_config: null, inline_config: false } }, config: [], + mcpCapability: { grantId: generationId, endpoint: 'http://127.0.0.1:1/api/managed-mcp', providerRelativePath: '.freshell/mcp-capability.json' }, + preparedInputs: { generationId, initialAuthReservation: reservationId, + manifest: { relativePath: '.freshell/input-manifest.json', sha256: digest(manifest), sourceCount: 0, aliasCount: 0, recipeCount: 0 }, + // ordinary_stage supplies both mapped and original selected-HOME aliases. + configBinds: (provider === 'claude' + ? ['/home/freshell/provider/.claude/settings.json', path.join(home, '.claude/settings.json')] + : ['/home/freshell/provider/.config/opencode/tui.jsonc', path.join(xdgConfigHome, 'opencode/tui.jsonc')]) + .map(effectiveDestination => ({ sourceRelativePath: '.freshell/templates/source-0', logicalDestination: effectiveDestination, effectiveDestination })), + resourceMounts: [], ...(provider === 'opencode' ? { indexedFiles: { volumeName: indexedVolume, aliases: [] } } : {}) } } + const contextBytes = Buffer.from(JSON.stringify(context)) + const resultBytes = Buffer.from(JSON.stringify({ providerLaunchContext: context, providerSecretReferences: [] })) + await fs.writeFile(path.join(metadata, 'prepared-context.json'), contextBytes) + await fs.writeFile(path.join(metadata, 'prepared-result.json'), resultBytes) + await fs.writeFile(path.join(control, 'mcp-capabilities', `${generationId}.json`), JSON.stringify({ soulId, incarnationId: null })) + const binding = { version: 1, provider: 'claude', soulId, generationId, initialReservationId: reservationId, + intentRevision: 1, registryAttemptId: null, workspacePath: workspace, cwd: workspace, gitCommonDir: null, + authoritySha256: digest(JSON.stringify({ kind: 'initial_reservation', reservationId })), manifestSha256: digest(manifest), + preparedContextSha256: digest(contextBytes), preparedResultSha256: digest(resultBytes) } + const publication = path.join(metadata, 'claude-candidate-binding.json') + const url = provider === 'claude' ? `/v1.47/volumes/freshell-claude-candidate-${requestId}` : '/v1.47/volumes/create' + const volume = { Name: indexedVolume, Driver: 'local', + DriverOpts: { type: 'tmpfs', device: 'tmpfs', o: 'size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec' }, + Labels: { 'com.freshell.managed-inputs': 'true', 'com.freshell.installation-id': installationId, + 'com.freshell.soul-id': soulId, 'com.freshell.incarnation-id': 'incarnation-home', 'com.freshell.input-generation': generationId } } + const request = (method: string, target = url, body?: unknown) => new Promise((resolve, reject) => { + const outgoing = http.request({ socketPath: harness!.broker.proxySocketPath, path: target, method }, response => { + response.resume(); response.once('end', () => resolve(response.statusCode!)) + }) + outgoing.once('error', reject); outgoing.end(body === undefined ? undefined : JSON.stringify(body)) + }) + const selectedRequest = () => provider === 'claude' ? request('GET') : request('POST', url, volume) + if (provider === 'opencode') { + expect(await request('POST', '/v1.47/volumes/create', { Name: owner.providerVolumeName, Driver: 'local', Labels: { + 'com.freshell.installation-id': installationId, 'com.freshell.soul-id': soulId, + 'freshell.input.reservation': reservationId, 'freshell.input.incarnation': 'incarnation-home', 'freshell.input.intent': '1', + 'freshell.input.daemon': owner.daemonId, 'freshell.input.image': owner.imageRef, 'freshell.input.binary-sha256': owner.binarySha256, + } })).toBe(201) + forwarded.length = 0 + await fs.writeFile(path.join(control, 'mcp-capabilities', `${generationId}.json`), JSON.stringify({ soulId: 'foreign', incarnationId: null })) + } + await fs.writeFile(publication, JSON.stringify({ ...binding, soulId: 'foreign' })) + expect(await selectedRequest()).toBe(403) + expect(forwarded).toEqual([]) + await fs.writeFile(path.join(control, 'mcp-capabilities', `${generationId}.json`), JSON.stringify({ soulId, incarnationId: null })) + for (const outside of [path.join(home, 'outside/settings.json'), path.join(directory, 'sibling/.claude/settings.json'), + '/home/freshell/provider/.local/share/opencode/session.json']) { + const changed = structuredClone(context) + changed.preparedInputs.configBinds[1].effectiveDestination = outside + changed.preparedInputs.configBinds[1].logicalDestination = outside + const changedContext = Buffer.from(JSON.stringify(changed)) + const changedResult = Buffer.from(JSON.stringify({ providerLaunchContext: changed, providerSecretReferences: [] })) + await fs.writeFile(path.join(metadata, 'prepared-context.json'), changedContext) + await fs.writeFile(path.join(metadata, 'prepared-result.json'), changedResult) + await fs.writeFile(publication, JSON.stringify({ ...binding, + preparedContextSha256: digest(changedContext), preparedResultSha256: digest(changedResult) })) + expect(await selectedRequest()).toBe(403) + expect(forwarded).toEqual([]) + } + await fs.writeFile(path.join(metadata, 'prepared-context.json'), contextBytes) + await fs.writeFile(path.join(metadata, 'prepared-result.json'), resultBytes) + await fs.writeFile(publication, JSON.stringify(binding)) + expect(await selectedRequest(), JSON.stringify(harness.broker.eventsSnapshot().slice(-4))).toBe(provider === 'claude' ? 404 : 201) + expect(forwarded).toEqual([url]) + if (provider === 'claude') { + expect(await request('DELETE')).toBe(403) + expect(await request('POST')).toBe(403) + } else { + expect(await request('POST', url, { ...volume, Labels: { ...volume.Labels, 'com.freshell.soul-id': 'foreign' } })).toBe(403) + expect(await request('DELETE', `/v1.47/volumes/${indexedVolume}?force=1`)).toBe(403) + } + expect(forwarded).toEqual([url]) + } finally { + await harness?.broker?.close() + await new Promise(resolve => daemon.close(() => resolve())) + if (inheritedDockerHost === undefined) delete process.env.DOCKER_HOST + else process.env.DOCKER_HOST = inheritedDockerHost + if (harness) await fs.rm(harness.testRoot, { recursive: true, force: true }) + await fs.rm(directory, { recursive: true, force: true }) + vi.restoreAllMocks() + } +}) + +describe('runtime harness indexed input cleanup', () => { + it.each(['removed', 'foreign-label', 'attached', 'retained-after-delete'] as const)( + 'settles the actual publisher before exact nonforce cleanup: %s', async outcome => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-cleanup-')) + const calls: { method: string; url: string }[] = [] + let volume: any, exists = false + fakeDocker.settled = false + fakeDocker.commands = [] + const daemon = http.createServer(async (request, response) => { + const chunks: Buffer[] = [] + for await (const chunk of request) chunks.push(Buffer.from(chunk)) + const body = chunks.length ? JSON.parse(Buffer.concat(chunks).toString()) : undefined + calls.push({ method: request.method!, url: request.url! }) + if (request.url === '/v1.47/volumes/create') { + volume = { Name: body.Name, Driver: body.Driver, Options: body.DriverOpts, Labels: body.Labels } + exists = true + response.statusCode = 201 + response.end(JSON.stringify(volume)) + } else if (request.url?.startsWith('/v1.47/containers/create?')) { + response.statusCode = 201 + response.end(JSON.stringify({ Id: 'a'.repeat(64) })) + } else if (request.method === 'GET') { + response.statusCode = exists ? 200 : 404 + response.end(JSON.stringify(exists ? volume : {})) + } else if (request.method === 'DELETE') { + if (!fakeDocker.settled || outcome === 'attached') response.statusCode = 409 + else { + response.statusCode = 204 + if (outcome !== 'retained-after-delete') exists = false + } + response.end() + } else { response.statusCode = 500; response.end() } + }) + const harness = new RuntimeHarness(directory) + const broker = new RestrictedDockerBroker({ realSocketPath: path.join(directory, 'daemon.sock'), + proxySocketPath: path.join(directory, 'broker.sock'), runtimeRootPrefix: path.join(directory, 'r'), + allowedHostBinaryPaths: new Set([path.join(directory, 'host')]), allowedImageRefs: new Set(['sha256:fixture']), + allowedWorkspaceRoots: new Set(['/workspace']), testRunId: harness.runId, logPath: path.join(directory, 'broker.jsonl') }) + harness.broker = broker + const request = (method: string, url: string, body?: unknown) => new Promise((resolve, reject) => { + const outgoing = http.request({ socketPath: broker.proxySocketPath, path: url, method }, response => { + response.resume() + response.once('end', () => resolve(response.statusCode!)) + }) + outgoing.once('error', reject) + outgoing.end(body === undefined ? undefined : JSON.stringify(body)) + }) + try { + await new Promise(resolve => daemon.listen(path.join(directory, 'daemon.sock'), resolve)) + await broker.start() + // Exercise cleanup's actual live-broker lifecycle without image/build preparation. + ;(harness as unknown as { brokerStarted: boolean }).brokerStarted = true + const generationId = 'grant-cleanup', soulId = 'soul-cleanup' + const input: PreparedInputAuthorization = { installationId: 'installation-cleanup', soulId, + incarnationId: 'incarnation-cleanup', generationId, imageRef: 'sha256:fixture', hostBinaryPath: path.join(directory, 'host'), + providerVolumeName: `freshell-provider-${'b'.repeat(24)}`, + stageRoot: path.join(directory, 'scenarios/scenario/control/provider-roots', generationId), + configurationBinds: [], resourceMounts: [], indexedSecretSources: [], + indexedFiles: { volumeName: `freshell-inputs-${createHash('sha256').update(`${soulId}\0${generationId}`).digest('hex').slice(0, 24)}`, aliases: [] } } + broker.authorizePreparedInputs(input) + const labels = { 'com.freshell.managed-inputs': 'true', 'com.freshell.installation-id': input.installationId, + 'com.freshell.soul-id': soulId, 'com.freshell.incarnation-id': input.incarnationId, 'com.freshell.input-generation': generationId } + expect(await request('POST', '/v1.47/volumes/create', { Name: input.indexedFiles!.volumeName, Driver: 'local', + DriverOpts: { type: 'tmpfs', device: 'tmpfs', o: 'size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec' }, Labels: labels })).toBe(201) + const exchange = path.join(directory, 'freshell-inputs-11111111-1111-4111-8111-111111111111') + await fs.mkdir(exchange) + const publisher = { Image: input.imageRef, User: '65534:0', Tty: false, Labels: labels, + Entrypoint: ['/runtime/freshell-session-host'], Cmd: ['publish-provider-inputs'], Env: ['PATH='], + HostConfig: { NetworkMode: 'none', ReadonlyRootfs: true, CapDrop: ['ALL'], SecurityOpt: ['no-new-privileges'], + Memory: 128 * 1024 * 1024, MemorySwap: 128 * 1024 * 1024, NanoCpus: 500_000_000, PidsLimit: 32, LogConfig: { Type: 'none', Config: {} }, + Mounts: [{ Type: 'bind', Source: input.hostBinaryPath, Target: '/runtime/freshell-session-host', ReadOnly: true }, + { Type: 'volume', Source: input.indexedFiles!.volumeName, Target: '/run/freshell-input-files', ReadOnly: false, VolumeOptions: { NoCopy: true } }, + { Type: 'bind', Source: exchange, Target: '/run/freshell-input-exchange', ReadOnly: true }] } } + // A rejected publisher must not become cleanup authority. + expect(await request('POST', `/v1.47/containers/create?name=freshell-inputs-${'c'.repeat(24)}-publisher`, publisher)).toBe(403) + expect(await request('POST', `/v1.47/containers/create?name=${input.indexedFiles!.volumeName}-publisher`, publisher)).toBe(201) + if (outcome === 'foreign-label') volume.Labels['com.freshell.soul-id'] = 'foreign' + const result = await harness.cleanup() + expect(fakeDocker.settled).toBe(true) + expect(fakeDocker.commands).toEqual([['inspect', 'a'.repeat(64)], ['rm', '-f', 'a'.repeat(64)]]) + expect(result.ok).toBe(outcome === 'removed') + expect(exists).toBe(outcome !== 'removed') + const deletes = calls.filter(call => call.method === 'DELETE') + expect(deletes).toEqual(outcome === 'foreign-label' ? [] : [{ method: 'DELETE', url: `/v1.47/volumes/${input.indexedFiles!.volumeName}` }]) + const cleanup = JSON.parse(await fs.readFile(path.join(harness.evidenceDir, 'cleanup.json'), 'utf8')) + expect(cleanup.volumes).toContain(input.indexedFiles!.volumeName) + expect(cleanup.volumes).not.toContain(`freshell-inputs-${'c'.repeat(24)}`) + if (outcome === 'removed') expect(calls.at(-1)).toEqual({ method: 'GET', url: `/v1.47/volumes/${input.indexedFiles!.volumeName}` }) + } finally { + await broker.close() + await new Promise(resolve => daemon.close(() => resolve())) + await fs.rm(directory, { recursive: true, force: true }) + } + }, + ) +}) diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 600a21e85..684b33cda 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -1,11 +1,125 @@ import { createHash } from 'node:crypto' +import { spawnSync } from 'node:child_process' import os from 'node:os' import fs from 'node:fs/promises' import http from 'node:http' import path from 'node:path' +import { pathToFileURL } from 'node:url' import { describe, expect, it, vi } from 'vitest' import { RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' +import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../../integration/server/provider-parity-broker.js' +import { makeProviderParityResourcesReadable, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from '../../../integration/server/provider-parity-environment.js' + +it('executes file and package TUI entrypoints while observing the selected project settings', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-parity-selection-')) + try { + const workspace = path.join(directory, 'workspace') + const home = path.join(directory, 'home') + const packageRoot = path.join(home, '.freshell/opencode') + const entry = path.join(packageRoot, 'freshell-rebind-plugin.ts') + await fs.mkdir(path.join(workspace, '.opencode'), { recursive: true }) + await fs.mkdir(path.join(home, 'project/.opencode'), { recursive: true }) + await fs.mkdir(packageRoot, { recursive: true }) + await fs.copyFile('extensions/opencode/freshell-rebind-plugin.ts', entry) + await fs.writeFile(path.join(packageRoot, 'package.json'), JSON.stringify({ exports: { './tui': './freshell-rebind-plugin.ts' } })) + const userConfig = { plugin: ['file:///selected/user-plugin.js'], custom: { retained: true }, + mcp: { vendor: { type: 'local', command: ['retained-user-tool'] } } } + await fs.writeFile(path.join(workspace, '.opencode/opencode.json'), JSON.stringify(userConfig)) + const projected = path.join(home, 'project/.opencode/opencode.json') + const selectedTui = path.join(directory, 'tui.json') + const consumer = `import { observeOpenCodeParitySelection } from ${JSON.stringify(pathToFileURL(path.resolve('test/fixtures/providers/parity-provider.mjs')).href)}; +process.stdout.write(JSON.stringify(await observeOpenCodeParitySelection(...process.argv.slice(1), { command: 'actual-mcp', args: ['argument'] })));` + const inspect = () => { + const child = spawnSync(process.execPath, ['--input-type=module', '-e', consumer, home, workspace, selectedTui], { encoding: 'utf8' }) + expect(child.error).toBeUndefined() + expect(child.status, child.stderr).toBe(0) + return JSON.parse(child.stdout) + } + for (const managed of [false, true]) { + if (managed) await fs.writeFile(projected, JSON.stringify({ ...userConfig, mcp: { ...userConfig.mcp, + freshell: { type: 'local', command: ['actual-mcp', 'argument'] } } })) + else await fs.unlink(projected).catch(() => {}) + const spec = pathToFileURL(managed ? packageRoot : entry).href + await fs.writeFile(selectedTui, JSON.stringify({ theme: 'selected', plugin: ['file:///selected/user-plugin.js', spec] })) + const observed = inspect() + expect(JSON.parse(observed.projectedProjectConfig)).toEqual(userConfig) + expect(observed.rebindPlugin).toMatchObject({ spec, hello: true, sessionId: 'ses_parity' }) + expect(JSON.parse(observed.tuiConfig)).toEqual({ theme: 'selected', plugin: ['file:///selected/user-plugin.js', spec] }) + } + const configuredBridge = { ...userConfig, mcp: { ...userConfig.mcp, + freshell: { type: 'local', command: ['selected-user-bridge'] } } } + await fs.writeFile(projected, JSON.stringify(configuredBridge)) + expect(JSON.parse(inspect().projectedProjectConfig)).toEqual(configuredBridge) + await fs.unlink(entry) + const missing = spawnSync(process.execPath, ['--input-type=module', '-e', consumer, home, workspace, selectedTui], { encoding: 'utf8' }) + expect(missing.status).not.toBe(0) + } finally { await fs.rm(directory, { recursive: true, force: true }) } +}) + +it('classifies owned provider publication failures without retaining error bodies', () => { + const secret = 'synthetic-private-error-body' + const logs = [ + 'null', 'false', '[]', 'not-json', + JSON.stringify({ event: 'provider.projection.transaction.started', stage: 'stage' }), + JSON.stringify({ event: 'provider.projection.transaction.failed', stage: 'publish', cause: `provider transaction install .config/${secret}: Invalid cross-device link (os error 18)` }), + JSON.stringify({ event: 'session_host.fatal', error: `provider refresh control ${secret}` }), + JSON.stringify({ event: 'session_host.fatal', error: secret }), + ].join('\n') + const diagnostic = providerParityFailureDiagnostics(logs) + expect(diagnostic).toMatchObject({ + transactionFailures: [{ stage: 'publish', operation: 'install', errno: 'cross_device' }], + workerFailures: [{ operation: 'control', errno: 'unknown' }, { operation: 'unknown', errno: 'unknown' }], + }) + expect(diagnostic.transactionEvents).toEqual([ + { event: 'provider.projection.transaction.started', stage: 'stage' }, + { event: 'provider.projection.transaction.failed', stage: 'publish' }, + ]) + expect(JSON.stringify(diagnostic)).not.toContain(secret) +}) + +it('executes selected OpenCode resources after fixture readability preparation', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-parity-readable-')) + try { + writeProviderParityEnvironment(directory, '.opencode/plugins/parity') + makeProviderParityResourcesReadable(path.join(directory, '.opencode')) + const execution = spawnSync(path.join(directory, '.opencode/plugins/parity/command'), [], { encoding: 'utf8' }) + expect(execution.error).toBeUndefined() + expect(execution.status, execution.stderr).toBe(0) + expect(JSON.parse(execution.stdout)).toEqual({ dependency: 'installed-dependency-nonce', resource: 'selected-resource-nonce', entries: 4201 }) + } finally { await fs.rm(directory, { recursive: true, force: true }) } +}) + +it('executes complete selected parity resources inside an ESM workspace', async () => { + const workspace = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-parity-module-')) + try { + await fs.writeFile(path.join(workspace, 'package.json'), JSON.stringify({ type: 'module' })) + const home = path.join(workspace, '.codex') + const selected = 'skills/parity-environment' + writeProviderParityEnvironment(home, selected) + const execution = spawnSync(path.join(home, selected, 'command'), [], { encoding: 'utf8' }) + expect(execution.error).toBeUndefined() + expect(execution.status, execution.stderr).toBe(0) + expect(JSON.parse(execution.stdout)).toEqual({ + dependency: 'installed-dependency-nonce', resource: 'selected-resource-nonce', entries: 4201, + }) + } finally { + await fs.rm(workspace, { recursive: true, force: true }) + } +}) + +it('executes the isolated parity command from a different controller directory', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-parity-command-')) + try { + const workspace = path.join(directory, 'workspace') + writeProviderParityEnvironment(workspace, '.') + await fs.rename(path.join(workspace, 'command'), path.join(workspace, 'parity-codex')) + const execution = spawnSync(providerParityCommands(workspace).CODEX_CMD, [], { cwd: directory, encoding: 'utf8' }) + expect(execution.error).toBeUndefined() + expect(execution.status, execution.stderr).toBe(0) + expect(JSON.parse(execution.stdout)).toEqual({ dependency: 'installed-dependency-nonce', resource: 'selected-resource-nonce', entries: 4201 }) + } finally { await fs.rm(directory, { recursive: true, force: true }) } +}) const runtimeDir = '/tmp/owned/r/scenario/x/incarnation-one' const actorKey = createHash('sha256').update('installation-one\0soul-one').digest('hex') @@ -566,6 +680,76 @@ describe('fixed initial Claude candidate broker', () => { }) describe('scoped Claude source observer broker', () => { + it.each(['source', 'mountpoints'] as const)('allows the producer %s exchange only directly under the owned shared root', async role => { + await withHttpBroker(async ({ directory, request, forwarded, broker: instance }) => { + const owner = { ...sourceAuthorization(), hostBinaryPath: path.join(directory, 'freshell-session-host') } + instance.authorizeInitialProviderVolume(owner) + const input = publisherAuthorization() + input.hostBinaryPath = owner.hostBinaryPath + input.stageRoot = path.join(directory, 'scenarios/scenario/control/provider-roots', input.generationId) + instance.authorizePreparedInputs(input) + const body = role === 'source' ? sourceHelperBody() : publisherBody(input) + body.HostConfig.Mounts[0].Source = owner.hostBinaryPath + if (role === 'mountpoints') { + body.User = '0:0'; body.Cmd = ['prepare-provider-mountpoints'] + body.Labels = { 'com.freshell.installation-id': input.installationId, 'com.freshell.soul-id': input.soulId, 'com.freshell.incarnation-id': input.incarnationId } + body.HostConfig.CapAdd = ['CHOWN', 'SETGID', 'SETUID'] + body.HostConfig.Mounts[1] = { Type: 'volume', Source: input.providerVolumeName, Target: '/home/freshell/provider', ReadOnly: false, VolumeOptions: { NoCopy: true } } + } + const basename = 'freshell-inputs-33333333-3333-4333-8333-333333333333' + const exchange = path.join(directory, basename) + await fs.mkdir(exchange, { mode: 0o700 }) + const linked = path.join(directory, 'freshell-inputs-44444444-4444-4444-8444-444444444444') + await fs.symlink(exchange, linked) + body.HostConfig.Mounts[2].Source = exchange + const name = role === 'source' ? sourceName : 'freshell-mountpoints-11111111-1111-4111-8111-111111111111' + for (const rejected of [directory + '-sibling/' + basename, path.join(directory, 'nested', basename), path.join(directory, 'native'), linked]) { + const changed = structuredClone(body) + changed.HostConfig.Mounts[2].Source = rejected + expect((await request('POST', `/v1.47/containers/create?name=${name}`, changed)).status).toBe(403) + expect(forwarded).toHaveLength(0) + } + const writable = structuredClone(body) + writable.HostConfig.Mounts[2].ReadOnly = false + expect((await request('POST', `/v1.47/containers/create?name=${name}`, writable)).status).toBe(403) + expect(forwarded).toHaveLength(0) + expect((await request('POST', `/v1.47/containers/create?name=${name}`, body)).status).toBe(201) + }, undefined, true) + }) + it('records bounded upstream helper rejection and host exchange absence without daemon payloads', async () => { + await withHttpBroker(async ({ request, broker: instance, rejectHelperCreate }) => { + instance.authorizeInitialProviderVolume(sourceAuthorization()) + rejectHelperCreate() + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, sourceHelperBody())).status).toBe(400) + expect(instance.eventsSnapshot().at(-1)).toMatchObject({ + decision: 'forward', + reason: 'upstream helper rejected: status=400; cause=missing_bind_source; ownedExchangeHostExists=false', + }) + expect(JSON.stringify(instance.eventsSnapshot())).not.toContain('PRIVATE_DAEMON_SENTINEL') + }) + }) + it('allows the reserved fixed reader for an owned nested project and denies outside or writable mounts', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance }) => { + instance.authorizeInitialProviderVolume(sourceAuthorization()) + const body = sourceHelperBody() + body.WorkingDir = '/workspace/project/nested' + body.HostConfig.Mounts[1] = { Type: 'bind', Source: '/workspace/project', Target: '/workspace/project', ReadOnly: true } + for (const outside of ['/workspace-sibling/project', '/outside/project']) { + const changed = structuredClone(body) + changed.WorkingDir = outside + changed.HostConfig.Mounts[1].Source = outside + changed.HostConfig.Mounts[1].Target = outside + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, changed)).status).toBe(403) + expect(forwarded).toHaveLength(0) + } + const writable = structuredClone(body) + writable.HostConfig.Mounts[1].ReadOnly = false + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, writable)).status).toBe(403) + expect(forwarded).toHaveLength(0) + expect((await request('POST', `/v1.47/containers/create?name=${sourceName}`, body)).status).toBe(201) + expect(forwarded).toHaveLength(1) + }) + }) it('allows the reserved fixed reader and exact private lifecycle without creating initial HOME or exposing logs', async () => { await withHttpBroker(async ({ request, forwarded, broker: instance }) => { const body = sourceHelperBody() @@ -654,6 +838,31 @@ describe('scoped Claude source observer broker', () => { }) describe('receipt-owned auth exchange HTTP routes', () => { + it('allows the producer auth exchange only directly under the owned shared root', async () => { + await withHttpBroker(async ({ directory, request, forwarded }) => { + const binds = [...ordinaryBinds, actorBind].map(bind => bind.replace('/tmp/owned', directory)) + expect((await request('POST', '/v1.47/containers/create?name=owned-runtime', createBody(binds))).status).toBe(201) + const body = authHelperBody() + body.HostConfig.Mounts[0].Source = path.join(directory, 'freshell-session-host') + const exchange = path.join(directory, authName) + await fs.mkdir(exchange, { mode: 0o710 }) + const linked = path.join(directory, authName.replace('111111111111', '222222222222')) + await fs.symlink(exchange, linked) + body.HostConfig.Mounts[2].Source = exchange + const before = forwarded.length + for (const rejected of [directory + '-sibling/' + authName, path.join(directory, 'nested', authName), path.join(directory, 'native'), linked]) { + const changed = structuredClone(body) + changed.HostConfig.Mounts[2].Source = rejected + expect((await request('POST', `/v1.47/containers/create?name=${authName}`, changed)).status).toBe(403) + expect(forwarded).toHaveLength(before) + } + const writable = structuredClone(body) + writable.HostConfig.Mounts[2].ReadOnly = false + expect((await request('POST', `/v1.47/containers/create?name=${authName}`, writable)).status).toBe(403) + expect(forwarded).toHaveLength(before) + expect((await request('POST', `/v1.47/containers/create?name=${authName}`, body)).status).toBe(201) + }, undefined, true) + }) it('allows only the fixed private helper, lifecycle and ownership inspection with Docker logs disabled', async () => { await withHttpBroker(async ({ request, forwarded }) => { await request('POST', '/v1.47/containers/create?name=owned-runtime', createBody([...ordinaryBinds, actorBind])) @@ -690,15 +899,18 @@ describe('receipt-owned auth exchange HTTP routes', () => { type ForwardedRequest = { method: string; url: string; body: any } type BrokerHttpFixture = { + directory: string setRuntimeRunning:(running:boolean)=>void request(method: string, url: string, body?: unknown): Promise<{ status: number; body: string }> forwarded: ForwardedRequest[] broker: RestrictedDockerBroker loseHelperAck(): void + rejectHelperCreate(): void returnHelperId(id: string): void replaceHelperInspection():void loseCandidateVolumeAck(): void replaceCandidateVolumeInspection(): void + clearCandidateVolumeInspection(): void } function publisherAuthorization():PreparedInputAuthorization { @@ -723,7 +935,286 @@ function publisherBody(input:PreparedInputAuthorization):any { {Type:'bind',Source:path.join(os.tmpdir(),'freshell-inputs-11111111-1111-4111-8111-111111111111'),Target:'/run/freshell-input-exchange',ReadOnly:true}]}}; } +async function publishControllerPreparation(directory: string, input: Pick, + reservationId: string, provider: 'claude' | 'codex') { + const metadata = path.join(input.stageRoot, '.freshell') + await fs.mkdir(metadata, { recursive: true }) + const manifest = Buffer.from(JSON.stringify({ version: 1, sources: [], aliases: [], recipes: [] })) + await fs.writeFile(path.join(metadata, 'input-manifest.json'), manifest) + const context = { + preparation: provider === 'claude' ? { claude: { mcp_args: [] } } : { codex: { tui_args: [], sidecar_args: [] } }, + config: [], mcpCapability: { grantId: input.generationId, endpoint: 'http://127.0.0.1:1/api/managed-mcp', providerRelativePath: '.freshell/mcp-capability.json' }, + preparedInputs: { generationId: input.generationId, initialAuthReservation: reservationId, + manifest: { relativePath: '.freshell/input-manifest.json', sha256: createHash('sha256').update(manifest).digest('hex'), sourceCount: 0, aliasCount: 0, recipeCount: 0 }, + configBinds: input.configurationBinds.map(bind => ({ ...bind, logicalDestination: bind.effectiveDestination })), + resourceMounts: input.resourceMounts.map(mount => ({ ...mount, logicalDestination: mount.effectiveDestination })), + ...(input.indexedFiles ? { indexedFiles: input.indexedFiles } : {}), + }, + } + const contextBytes = Buffer.from(JSON.stringify(context)) + const resultBytes = Buffer.from(JSON.stringify({ providerLaunchContext: context, providerSecretReferences: [] })) + await fs.writeFile(path.join(metadata, 'prepared-context.json'), contextBytes) + await fs.writeFile(path.join(metadata, 'prepared-result.json'), resultBytes) + const capabilities = path.join(directory, 'scenarios/scenario/control/mcp-capabilities') + await fs.mkdir(capabilities, { recursive: true }) + await fs.writeFile(path.join(capabilities, `${input.generationId}.json`), JSON.stringify({ + endpoint: context.mcpCapability.endpoint, scope: 'synthetic-controller-scope', soulId: input.soulId, + incarnationId: null, expiresAt: Date.now() + 60_000, preparation: { phase: 'pending', priorGrantIds: [] }, + })) + return { metadata, capabilities, context, contextBytes, resultBytes } +} + describe('fixed indexed publisher broker',()=>{ + it('probes actual candidate absence before creation only with a qualified owned publication', async () => { + await withHttpBroker(async ({ directory, request, forwarded, broker: instance, replaceCandidateVolumeInspection, clearCandidateVolumeInspection }) => { + const input = candidateAuthorization() + const binary = Buffer.from('candidate absence host artifact') + input.hostBinaryPath = path.join(directory, 'freshell-session-host') + await fs.writeFile(input.hostBinaryPath, binary) + input.binarySha256 = createHash('sha256').update(binary).digest('hex') + input.stageRoot = path.join(directory, 'scenarios/scenario/control/provider-roots', input.generationId) + const { metadata, context, contextBytes, resultBytes } = await publishControllerPreparation(directory, input, input.reservationId, 'claude') + const binding = { version: 1, provider: 'claude', soulId: input.soulId, generationId: input.generationId, + initialReservationId: input.reservationId, intentRevision: 1, registryAttemptId: null, + authoritySha256: createHash('sha256').update(JSON.stringify({ kind: 'initial_reservation', reservationId: input.reservationId })).digest('hex'), workspacePath: input.workspacePath, cwd: input.cwd, + gitCommonDir: null, + manifestSha256: context.preparedInputs.manifest.sha256, + preparedContextSha256: createHash('sha256').update(contextBytes).digest('hex'), + preparedResultSha256: createHash('sha256').update(resultBytes).digest('hex') } + const publication = path.join(metadata, 'claude-candidate-binding.json') + await fs.writeFile(publication, JSON.stringify(binding)) + const url = `/v1.47/volumes/freshell-claude-candidate-${input.requestId}` + expect((await request('GET', url)).status).toBe(403) + expect(instance.eventsSnapshot().at(-1)).toMatchObject({ + reason: 'candidate absence qualification unavailable at owner_capability', + }) + instance.authorizeInitialProviderVolume({ ...sourceAuthorization(), hostBinaryPath: input.hostBinaryPath, binarySha256: input.binarySha256 }) + await fs.writeFile(publication, JSON.stringify({ ...binding, preparedResultSha256: 'f'.repeat(64) })) + expect((await request('GET', url)).status).toBe(403) + expect(instance.eventsSnapshot().at(-1)).toMatchObject({ + reason: 'candidate absence qualification unavailable at prepared_metadata', + }) + expect(forwarded).toHaveLength(0) + await fs.writeFile(publication, JSON.stringify(binding)) + await fs.writeFile(publication, JSON.stringify({ ...binding, authoritySha256: 'e'.repeat(64) })) + expect((await request('GET', url)).status).toBe(403) + expect(instance.eventsSnapshot().at(-1)).toMatchObject({ + reason: 'candidate absence qualification unavailable at authority_binding', + }) + await fs.writeFile(publication, JSON.stringify({ ...binding, gitCommonDir: '../foreign' })) + expect((await request('GET', url)).status).toBe(403) + expect(instance.eventsSnapshot().at(-1)).toMatchObject({ + reason: 'candidate absence qualification unavailable at selected_target', + }) + expect(forwarded).toHaveLength(0) + await fs.writeFile(publication, JSON.stringify(binding)) + replaceCandidateVolumeInspection() + const foreign = await request('GET', url) + expect(foreign.status).toBe(403) + expect(foreign.body).not.toContain('PRIVATE_FOREIGN_CANDIDATE') + clearCandidateVolumeInspection() + expect((await request('GET', url)).status).toBe(404) + expect((await request('DELETE', url)).status).toBe(403) + const malformed = url.replace(input.requestId, '11111111-1111-1111-1111-111111111111') + expect((await request('GET', malformed)).status).toBe(403) + expect(forwarded.every(operation => operation.method === 'GET')).toBe(true) + const changed = { Name: url.split('/').at(-1), Driver: 'local', DriverOpts: {}, Labels: { foreign: 'owner' } } + expect((await request('POST', '/v1.47/volumes/create', changed)).status).toBe(403) + expect(forwarded.every(operation => operation.method === 'GET')).toBe(true) + }, undefined, true) + }) + it('cleans the exact registered failed-launch volume and retains changed or attached ownership', async () => { + await withHttpBroker(async ({ directory }) => { + const binary = Buffer.from('registered failed-launch artifact') + const owner = { ...sourceAuthorization(), hostBinaryPath: path.join(directory, 'freshell-session-host'), + binarySha256: createHash('sha256').update(binary).digest('hex') } + await fs.writeFile(owner.hostBinaryPath, binary) + const labels = { + 'com.freshell.installation-id': owner.installationId, 'com.freshell.soul-id': owner.soulId, + 'freshell.input.daemon': owner.daemonId, 'freshell.input.image': owner.imageRef, + 'freshell.input.binary-sha256': owner.binarySha256, 'freshell.input.intent': '1', + 'freshell.input.reservation': '11111111-1111-4111-8111-111111111111', + 'freshell.input.incarnation': 'incarnation-one', + } + let currentLabels = { ...labels }, daemon = owner.daemonId, attached = false, exists = true + const removals: string[][] = [] + const run = (args: string[]): string => { + if (args[0] === 'info') return daemon + if (args[0] === 'container') return attached ? 'e'.repeat(64) : '' + if (args[1] === 'ls') return exists ? owner.providerVolumeName : '' + if (args[1] === 'inspect') return JSON.stringify({ Name: owner.providerVolumeName, Driver: 'local', Options: null, Labels: currentLabels }) + if (args[1] === 'rm') { removals.push(args); exists = false; return owner.providerVolumeName } + throw new Error('unexpected cleanup command') + } + const captured = captureProviderParityVolumes([owner], run) + currentLabels['freshell.input.incarnation'] = 'foreign' + expect(() => cleanupProviderParityVolumes(captured, run)).toThrow() + currentLabels = { ...labels } + attached = true + expect(() => cleanupProviderParityVolumes(captured, run)).toThrow() + attached = false + daemon = 'foreign' + expect(() => cleanupProviderParityVolumes(captured, run)).toThrow() + daemon = owner.daemonId + expect(removals).toEqual([]) + expect(cleanupProviderParityVolumes(captured, run)).toEqual([owner.providerVolumeName]) + expect(removals).toEqual([['volume', 'rm', owner.providerVolumeName]]) + expect(exists).toBe(false) + }, undefined, true) + }) + it('registers the exact owned published Claude candidate before its ordinary volume request', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance, directory }) => { + const input = candidateAuthorization() + input.workspacePath = '/workspace/project' + input.cwd = '/workspace/project/nested' + input.configurationBinds[0].effectiveDestination = '/home/freshell/provider/.claude/settings.json' + input.resourceMounts[0].effectiveDestination = '/home/freshell/provider/.claude/skills' + input.hostBinaryPath = path.join(directory, 'freshell-session-host') + const binary = Buffer.from('owned candidate host artifact') + await fs.writeFile(input.hostBinaryPath, binary) + input.binarySha256 = createHash('sha256').update(binary).digest('hex') + input.stageRoot = path.join(directory, 'scenarios/scenario/control/provider-roots', input.generationId) + instance.authorizeInitialProviderVolume({ ...sourceAuthorization(), hostBinaryPath: input.hostBinaryPath, binarySha256: input.binarySha256 }) + const { metadata, context, contextBytes, resultBytes } = await publishControllerPreparation(directory, input, input.reservationId, 'claude') + await fs.writeFile(path.join(metadata, 'claude-candidate-binding.json'), JSON.stringify({ + version: 1, provider: 'claude', soulId: input.soulId, generationId: input.generationId, + initialReservationId: input.reservationId, intentRevision: 1, + workspacePath: input.workspacePath, cwd: input.cwd, gitCommonDir: null, + manifestSha256: context.preparedInputs.manifest.sha256, + preparedContextSha256: createHash('sha256').update(contextBytes).digest('hex'), + preparedResultSha256: createHash('sha256').update(resultBytes).digest('hex'), + })) + const volume = { Name: `freshell-claude-candidate-${input.requestId}`, Driver: 'local', DriverOpts: {}, Labels: candidateLabels(input) } + const bindingPath = path.join(metadata, 'claude-candidate-binding.json') + const originalBinding = await fs.readFile(bindingPath) + for (const field of ['soulId', 'generationId', 'initialReservationId', 'gitCommonDir', 'preparedContextSha256', 'preparedResultSha256', 'manifestSha256']) { + const changed = JSON.parse(originalBinding.toString()) + changed[field] = 'foreign' + await fs.writeFile(bindingPath, JSON.stringify(changed)) + expect((await request('POST', '/v1.47/volumes/create', volume)).status, field).toBe(403) + expect(forwarded).toHaveLength(0) + } + await fs.writeFile(bindingPath, originalBinding) + for (const destination of ['/home/freshell/provider-sibling/.claude/skills', '/outside/skills', '/home/freshell/provider/.claude/projects']) { + const changed = structuredClone(input) + changed.resourceMounts[0].effectiveDestination = destination + expect(() => instance.authorizeClaudeCandidate(changed), destination).toThrow() + } + expect((await request('POST', '/v1.47/volumes/create', volume)).status, JSON.stringify(instance.eventsSnapshot())).toBe(201) + const before = forwarded.length + const changed = structuredClone(volume) + changed.Labels['com.freshell.soul-id'] = 'foreign' + expect((await request('POST', '/v1.47/volumes/create', changed)).status).toBe(403) + expect(forwarded).toHaveLength(before) + }, undefined, true) + }) + it('registers the exact owned published preparation before its indexed volume request', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance, directory }) => { + const source = sourceAuthorization() + const hostBinaryPath = path.join(directory, 'freshell-session-host') + const binary = Buffer.from('owned fixture host artifact') + await fs.writeFile(hostBinaryPath, binary) + const owner = { ...source, hostBinaryPath, binarySha256: createHash('sha256').update(binary).digest('hex') } + instance.authorizeInitialProviderVolume(owner) + const generationId = 'grant-44444444-4444-4444-8444-444444444444' + const input = { ...publisherAuthorization(), soulId: owner.soulId, generationId, hostBinaryPath, providerVolumeName: owner.providerVolumeName, + stageRoot: path.join(directory, 'scenarios/scenario/control/provider-roots', generationId), + configurationBinds: [{ sourceRelativePath: '.freshell/templates/codex-0', effectiveDestination: '/home/freshell/provider/.codex/config.toml' }], + resourceMounts: [{ providerRelativePath: '.freshell/provider-roots/root-0', effectiveDestination: '/home/freshell/provider/.codex/skills', kind: 'directory' as const }], + indexedFiles: { volumeName: `freshell-inputs-${createHash('sha256').update(`${owner.soulId}\0${generationId}`).digest('hex').slice(0, 24)}`, aliases: [] }, + indexedSecretSources: [] } + const { metadata, capabilities, context } = await publishControllerPreparation(directory, input, '11111111-1111-4111-8111-111111111111', 'codex') + const initial = { Name: owner.providerVolumeName, Driver: 'local', Labels: { + 'com.freshell.installation-id': owner.installationId, 'com.freshell.soul-id': owner.soulId, + 'freshell.input.reservation': context.preparedInputs.initialAuthReservation, + 'freshell.input.incarnation': input.incarnationId, 'freshell.input.intent': '1', + 'freshell.input.daemon': owner.daemonId, 'freshell.input.image': owner.imageRef, + 'freshell.input.binary-sha256': owner.binarySha256, + } } + expect((await request('POST', '/v1.47/volumes/create', initial)).status).toBe(201) + const volume = { Name: input.indexedFiles.volumeName, Driver: 'local', DriverOpts: indexedOptions, Labels: publisherLabels(input) } + const capabilityPath = path.join(capabilities, `${generationId}.json`) + const capabilityBytes = await fs.readFile(capabilityPath) + for (const field of ['soulId', 'incarnationId']) { + const changed = JSON.parse(capabilityBytes.toString()) + changed[field] = 'foreign' + await fs.writeFile(capabilityPath, JSON.stringify(changed)) + expect((await request('POST', '/v1.47/volumes/create', volume)).status, field).toBe(403) + expect(forwarded).toHaveLength(1) + } + await fs.writeFile(capabilityPath, capabilityBytes) + const resultPath = path.join(metadata, 'prepared-result.json') + const resultBytes = await fs.readFile(resultPath) + for (const field of ['generationId', 'initialAuthReservation', 'manifest']) { + const changed = JSON.parse(resultBytes.toString()) + changed.providerLaunchContext.preparedInputs[field] = 'foreign' + await fs.writeFile(resultPath, JSON.stringify(changed)) + expect((await request('POST', '/v1.47/volumes/create', volume)).status, field).toBe(403) + expect(forwarded).toHaveLength(1) + } + await fs.writeFile(resultPath, Buffer.alloc(1024 * 1024 + 1, 32)) + expect((await request('POST', '/v1.47/volumes/create', volume)).status).toBe(403) + await fs.rm(resultPath) + await fs.symlink(path.join(metadata, 'prepared-context.json'), resultPath) + expect((await request('POST', '/v1.47/volumes/create', volume)).status).toBe(403) + await fs.rm(resultPath) + await fs.writeFile(resultPath, resultBytes) + await fs.writeFile(hostBinaryPath, 'changed owned artifact') + expect((await request('POST', '/v1.47/volumes/create', volume)).status).toBe(403) + await fs.writeFile(hostBinaryPath, binary) + expect(forwarded).toHaveLength(1) + const invalidContext = structuredClone(context) + invalidContext.preparedInputs.configBinds[0].sourceRelativePath = '.freshell/PRIVATE_TEMPLATE_SENTINEL' + await fs.writeFile(path.join(metadata, 'prepared-context.json'), JSON.stringify(invalidContext)) + await fs.writeFile(resultPath, JSON.stringify({ providerLaunchContext: invalidContext, providerSecretReferences: [] })) + expect((await request('POST', '/v1.47/volumes/create', volume)).status).toBe(403) + expect(instance.eventsSnapshot().slice(-2)[0]).toMatchObject({ + reason: 'fixture publication unavailable at prepared_authorization: invalid_template_source', + }) + expect(JSON.stringify(instance.eventsSnapshot())).not.toContain('PRIVATE_TEMPLATE_SENTINEL') + await fs.writeFile(path.join(metadata, 'prepared-context.json'), JSON.stringify(context)) + await fs.writeFile(resultPath, resultBytes) + for (const destination of ['/home/freshell/provider-sibling/.codex/skills', '/outside/skills', '/home/freshell/provider/.codex/sessions']) { + const changed = structuredClone(input) + changed.resourceMounts[0].effectiveDestination = destination + expect(() => instance.authorizePreparedInputs(changed), destination).toThrow() + } + expect((await request('POST', '/v1.47/volumes/create', volume)).status, JSON.stringify(instance.eventsSnapshot().slice(-4))).toBe(201) + const before = forwarded.length + for (const mutation of ['owner', 'generation', 'permissions', 'volume']) { + const changed = structuredClone(volume) + if (mutation === 'owner') changed.Labels['com.freshell.soul-id'] = 'foreign' + if (mutation === 'generation') changed.Labels['com.freshell.input-generation'] = 'grant-foreign' + if (mutation === 'permissions') changed.DriverOpts.o = 'size=64m,mode=0755' + if (mutation === 'volume') changed.Name = `freshell-inputs-${'f'.repeat(24)}` + expect((await request('POST', '/v1.47/volumes/create', changed)).status, mutation).toBe(403) + expect(forwarded).toHaveLength(before) + } + }, undefined, true) + }) + it('registers each parity request before its first volume observation while retaining foreign denials', async () => { + await withHttpBroker(async ({ request, forwarded, broker: instance }) => { + const { soulId: _soulId, providerVolumeName: _volume, ...source } = sourceAuthorization() + const owner = authorizeProviderParityRequest(instance, source, 'provider-parity-codex-exact-request') + expect(instance.receipts()).toEqual([]) + expect((await request('GET', `/v1.47/volumes/${owner.providerVolumeName}`)).status).toBe(200) + expect(forwarded).toHaveLength(1) + const denied = [ + ['GET', `/v1.47/volumes/freshell-provider-${'f'.repeat(24)}`], + ['GET', `/v1.47/volumes/${owner.providerVolumeName}?foreign=1`], + ['DELETE', `/v1.47/volumes/${owner.providerVolumeName}`], + ['POST', '/v1.47/volumes/create'], + ] + for (const [method, url] of denied) { + expect((await request(method, url)).status).toBe(403) + expect(forwarded).toHaveLength(1) + } + expect(() => instance.authorizeInitialProviderVolume({ ...owner, soulId: 'foreign' })).toThrow() + expect(() => instance.authorizeInitialProviderVolume({ ...owner, daemonId: 'foreign' })).toThrow() + expect(() => instance.authorizeInitialProviderVolume({ ...owner, binarySha256: 'b'.repeat(64) })).toThrow() + }) + }) it('observes and creates only the explicitly reserved initial soul volume before any workload receipt',async()=>{ await withHttpBroker(async ({request,forwarded,broker:instance})=>{ const installationId='installation-one',soulId='soul-initial'; @@ -762,20 +1253,30 @@ describe('fixed indexed publisher broker',()=>{ body.Labels={'com.freshell.installation-id':input.installationId,'com.freshell.soul-id':input.soulId,'com.freshell.incarnation-id':input.incarnationId}; body.HostConfig.CapAdd=['CHOWN','SETGID','SETUID']; body.HostConfig.Mounts[1]={Type:'volume',Source:input.providerVolumeName,Target:'/home/freshell/provider',ReadOnly:false,VolumeOptions:{NoCopy:true}}; + const wrongEnvironment=structuredClone(body); + wrongEnvironment.Env=['PATH=PRIVATE_MOUNTPOINT_SENTINEL']; + expect((await request('POST',`/v1.47/containers/create?name=${name}`,wrongEnvironment)).status).toBe(403); + expect(instance.eventsSnapshot().at(-1)?.reason).toBe('mountpoint helper unavailable at environment'); + expect(JSON.stringify(instance.eventsSnapshot())).not.toContain('PRIVATE_MOUNTPOINT_SENTINEL'); + expect(forwarded).toHaveLength(0); expect((await request('POST',`/v1.47/containers/create?name=${name}`,body)).status).toBe(201); const before=forwarded.length; - for (const mutate of [ - (value:any)=>value.HostConfig.CapAdd.push('DAC_OVERRIDE'), - (value:any)=>value.Cmd.push('/outside'), - (value:any)=>value.HostConfig.Mounts[1].Source='freshell-provider-ffffffffffffffffffffffff', - (value:any)=>value.HostConfig.Mounts[1].VolumeOptions.NoCopy=false, - (value:any)=>value.HostConfig.Mounts.push({Type:'bind',Source:'/operator-home',Target:'/outside',ReadOnly:false}), - (value:any)=>value.Labels['com.freshell.soul-id']='foreign', - ]) { + for (const [phase, mutate] of [ + ['limits_authority', (value:any)=>value.HostConfig.CapAdd.push('DAC_OVERRIDE')], + ['identity_argv', (value:any)=>value.Cmd.push('/outside')], + ['mounts', (value:any)=>value.HostConfig.Mounts[1].Source='freshell-provider-ffffffffffffffffffffffff'], + ['mounts', (value:any)=>value.HostConfig.Mounts[1].VolumeOptions.NoCopy=false], + ['mount_count', (value:any)=>value.HostConfig.Mounts.push({Type:'bind',Source:'/operator-home',Target:'/outside',ReadOnly:false})], + ['owner', (value:any)=>value.Labels['com.freshell.soul-id']='foreign'], + ['image', (value:any)=>value.Image='foreign-image'], + ['labels', (value:any)=>value.Labels['extra']='PRIVATE_LABEL_SENTINEL'], + ] as const) { const changed=structuredClone(body);mutate(changed); expect((await request('POST',`/v1.47/containers/create?name=${name.replace('111111111111','222222222222')}`,changed)).status).toBe(403); + expect(instance.eventsSnapshot().at(-1)?.reason).toBe(`mountpoint helper unavailable at ${phase}`); expect(forwarded).toHaveLength(before); } + expect(JSON.stringify(instance.eventsSnapshot())).not.toContain('PRIVATE_LABEL_SENTINEL'); expect((await request('POST',`/v1.47/containers/${helperId}/start`)).status).toBe(204); expect((await request('POST',`/v1.47/containers/${helperId}/wait?condition=not-running`)).status).toBe(200); expect((await request('DELETE',`/v1.47/containers/${helperId}?force=1`)).status).toBe(204); @@ -880,12 +1381,13 @@ describe('fixed indexed publisher broker',()=>{ }); }); -async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise,gateways?:Set) { +async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise,gateways?:Set, physicalStage = false) { const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-broker-history-')) const socket = path.join(dir, 'docker.sock') const proxy = path.join(dir, 'broker.sock') const forwarded: ForwardedRequest[] = [] let loseAck = false + let rejectHelper = false let runtimeRunning=false let helperAckId = helperId let indexedVolumeMetadata:unknown @@ -941,6 +1443,11 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise res.statusCode=200;res.end(JSON.stringify({Id:helperId,Config:foreignHelper?{...inspected,User:'0:0'}:inspected,HostConfig:inspected.HostConfig}));return; } if (req.url?.startsWith('/v1.47/containers/create?')) { + if (body.Entrypoint && rejectHelper) { + res.statusCode = 400 + res.end(JSON.stringify({ message: 'invalid mount config for type "bind": bind source path does not exist: PRIVATE_DAEMON_SENTINEL' })) + return + } if(body.Cmd?.[0]==='publish-provider-inputs') sourceHelperDeleted=false if (body.Entrypoint?.[0] === '/runtime/freshell-session-host' && loseAck) { req.socket.destroy() @@ -965,8 +1472,8 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise allowedHostGatewayAddresses:gateways, realSocketPath: socket, proxySocketPath: proxy, - runtimeRootPrefix: '/tmp/owned/r', - allowedHostBinaryPaths: new Set(['/tmp/owned/freshell-session-host', '/tmp/owned/other-approved-host']), + runtimeRootPrefix: physicalStage ? path.join(dir, 'r') : '/tmp/owned/r', + allowedHostBinaryPaths: new Set(physicalStage ? [path.join(dir, 'freshell-session-host')] : ['/tmp/owned/freshell-session-host', '/tmp/owned/other-approved-host']), allowedImageRefs: new Set(['sha256:fixture-image', 'sha256:other-approved-image']), allowTerminalWorkloads: true, allowedWorkspaceRoots: new Set(['/workspace']), @@ -985,8 +1492,8 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise try { await new Promise((resolve) => upstream.listen(socket, resolve)) await instance.start() - await run({ request, forwarded, broker: instance, setRuntimeRunning:(running)=>{runtimeRunning=running}, loseHelperAck: () => { loseAck = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true}, - loseCandidateVolumeAck:()=>{candidateVolumeAckUnavailable=true},replaceCandidateVolumeInspection:()=>{candidateVolumeMetadata.Labels['com.freshell.soul-id']='foreign'} }) + await run({ directory: dir, request, forwarded, broker: instance, setRuntimeRunning:(running)=>{runtimeRunning=running}, loseHelperAck: () => { loseAck = true }, rejectHelperCreate: () => { rejectHelper = true }, returnHelperId: (id) => { helperAckId = id },replaceHelperInspection:()=>{foreignHelper=true}, + loseCandidateVolumeAck:()=>{candidateVolumeAckUnavailable=true},replaceCandidateVolumeInspection:()=>{candidateVolumeMetadata ??= { Name: 'PRIVATE_FOREIGN_CANDIDATE', Driver: 'local', Labels: {} }; candidateVolumeMetadata.Labels['com.freshell.soul-id']='foreign'}, clearCandidateVolumeInspection:()=>{candidateVolumeMetadata=undefined} }) } finally { await instance.close() await new Promise((resolve) => upstream.close(() => resolve())) From 338490467aaf1414987a94b4a5d8864fdf0df8b1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:21:38 -0700 Subject: [PATCH 226/493] docs: mark transactional provider projection complete --- .../2026-10-05-managed-provider-environment.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 26f967f03..ecac9c298 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -257,14 +257,14 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Interfaces:** Consumes `copy_environment`, `CopyReport`, `ReferencePlan` and existing `.freshell-config-projection.json`. Produces an all-or-error installed projection and unchanged previously working environment on failed refresh. Only prior manifest-owned paths may be replaced/removed. -- [ ] **Step 1: Add failing process tests.** Add backend `provider_private_parent_mount` production/adoption cases (so Task 3's `provider` selector actually runs them) and broker dispatch tests that accept the exact 0710 private-parent topology and reject wrong modes/extra authority; test through validator/dispatch behavior, not a source-text assertion. Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Include Task 1's >10,000-entry initial preparation fixture, new/adopted Amplifier stubs, provider-created sibling and unrelated symlink target in the actual production initial-preparation/ownership/child smoke; assert complete selected resources run as the provider UID and only creator-recorded inodes changed ownership, with adopted sibling/target owner/content unchanged. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. -- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked --test provider_config_refresh`. Expected current stale-delete-before-copy path loses prior usable projection on injected failure, or resets executable mode. -- [ ] **Step 3: Apply precise transaction patch.** The provider UID 65534/GID 0 reads the exact approved read-only controller stage and creates private projection/backup inodes on its own HOME volume; it never renames or chmods transport inodes and never stages the full ordinary tree in tmpfs. Stage the complete desired projection and manifest on that same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. For terminal and fresh initial launch, finish all host-required stub/bootstrap and Amplifier work first, transfer HOME ownership last, then install selected auth links and run provider-owned preparation. No later trusted-host HOME traversal. Exact resume skips host bootstrap/ownership reclamation. Change the private-parent Docker tmpfs producer, host initial/resume reset and disposable broker validator together to 0710. Use physical lchown ownership walks and preserve ordinary execute bits. Preserve existing setpriv boundary; do not add DAC_OVERRIDE/FOWNER. +- [x] **Step 1: Add failing process tests.** Add backend `provider_private_parent_mount` production/adoption cases (so Task 3's `provider` selector actually runs them) and broker dispatch tests that accept the exact 0710 private-parent topology and reject wrong modes/extra authority; test through validator/dispatch behavior, not a source-text assertion. Extend actual `refresh-provider-config`/setpriv integration to execute the projected skill and import a projected package as provider uid before/after refresh. Change resource and plugin dependency nonce between refreshes. Include Task 1's >10,000-entry initial preparation fixture, new/adopted Amplifier stubs, provider-created sibling and unrelated symlink target in the actual production initial-preparation/ownership/child smoke; assert complete selected resources run as the provider UID and only creator-recorded inodes changed ownership, with adopted sibling/target owner/content unchanged. Add precommit read/limit/permission faults and publish failure: previous executable/dependency remain runnable, old manifest retained, stale old entries still present on failed transaction, provider-created sibling/native state unchanged. On success stale manifest-owned entries disappear and provider-created sibling remains. Controller capability-file write failure leaves no new grant/temp root and retains old usable grant. Expand four-provider managed parity to execute scripts, read assets, load dependencies, resolve config references, verify child identity/MCP and sentinel-free observations. Synthetic tree includes more than 4096 legitimate dependency entries; no online install may mask omission. +- [x] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked --test provider_config_refresh`. Expected current stale-delete-before-copy path loses prior usable projection on injected failure, or resets executable mode. +- [x] **Step 3: Apply precise transaction patch.** The provider UID 65534/GID 0 reads the exact approved read-only controller stage and creates private projection/backup inodes on its own HOME volume; it never renames or chmods transport inodes and never stages the full ordinary tree in tmpfs. Stage the complete desired projection and manifest on that same filesystem before changing HOME. The provider-owned helper validates all operations and builds a journal of exact manifest-owned paths. Move replaced/stale owned leaf files and links into a transaction backup, install staged leaves into retained directories, and atomically publish the manifest last. Never move/replace a containing directory with unowned children; incompatible file/directory replacement with unowned descendants is a precommit error. On any prepublication copy/ownership/rename failure, restore each moved leaf, remove only newly installed transaction-owned leaves and empty created parents, retain old manifest, and return a structured preparation error. Do not remove unowned children of projected directories; remove a stale directory only when empty after its owned leaves are handled. Keep backup until manifest commit succeeds, then cleanup with structured errors if cleanup fails. For terminal and fresh initial launch, finish all host-required stub/bootstrap and Amplifier work first, transfer HOME ownership last, then install selected auth links and run provider-owned preparation. No later trusted-host HOME traversal. Exact resume skips host bootstrap/ownership reclamation. Change the private-parent Docker tmpfs producer, host initial/resume reset and disposable broker validator together to 0710. Use physical lchown ownership walks and preserve ordinary execute bits. Preserve existing setpriv boundary; do not add DAC_OVERRIDE/FOWNER. Native auth destinations survive when no corresponding typed auth-file grant is selected; CA/dynamic-file grants do not authorize auth deletion. Publish selected auth replacements before configuration/provider consumers, preserve old native destination on failure, and exclude native auth from ordinary projection manifest cleanup. Controller keeps temp-root→rename→capability publication ordering and revokes only new capability/resources on failure. Run-owned ephemeral references are cleaned with their grant; old grant/root/volume remain on failed replacement. Preparation error becomes explicit pre-spawn failure, never synthetic absence of identity. Log transaction stage/start/commit/rollback/failure once with safe counters and paths. -- [ ] **Step 4: Green.** Repeat refresh and controller fault cases; expected previous environment survives every precommit failure and complete new environment runs after success. -- [ ] **Step 5: Refactor.** Consolidate initial and exact-resume projection into one transaction helper; manifest authority remains limited to copied files. Share fixture setup for executable/resource/dependency cases without duplicating policy implementations in JS. -- [ ] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`; `pnpm run test:vitest run test/unit/tooling/testing/runtime-test-broker.test.ts --config config/vitest/vitest.config.ts`. Require positive counts and exact producer/broker/host 0710 agreement in the actual child smoke. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. Node fixtures alone do not certify OpenCode's Bun-compiled plugin loader or fresh Claude's bundled 2.1.237 versus terminal 2.1.263; Task 6 actual pinned managed provider proof and dedicated fresh-path smoke must load the selected resources without reinstall masking omitted dependencies. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. -- [ ] **Step 7: Commit.** Stage listed files and `git commit -m "fix: publish managed provider refreshes transactionally"`. +- [x] **Step 4: Green.** Repeat refresh and controller fault cases; expected previous environment survives every precommit failure and complete new environment runs after success. +- [x] **Step 5: Refactor.** Consolidate initial and exact-resume projection into one transaction helper; manifest authority remains limited to copied files. Share fixture setup for executable/resource/dependency cases without duplicating policy implementations in JS. +- [x] **Step 6: Impact verification.** Run host refresh tests above and existing host `provider_config` tests; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked provider`; `pnpm run test:vitest run test/unit/tooling/testing/runtime-test-broker.test.ts --config config/vitest/vitest.config.ts`. Require positive counts and exact producer/broker/host 0710 agreement in the actual child smoke. Run `pnpm run test:vitest run test/integration/server/managed-provider-parity.test.ts --testNamePattern "preserves provider-visible launch and MCP behavior" --config config/vitest/vitest.config.ts` with `FRESHELL_PROVIDER_PARITY_ROWS_DIR` unset. Expected all four actual controller→host→child rows pass. Node fixtures alone do not certify OpenCode's Bun-compiled plugin loader or fresh Claude's bundled 2.1.237 versus terminal 2.1.263; Task 6 actual pinned managed provider proof and dedicated fresh-path smoke must load the selected resources without reinstall masking omitted dependencies. This positive owned-Docker smoke has no kill/restart branch. Destructive parity/recovery branches require the separately validated documented disposable broker/sandbox harness; do not put the rig inside a socketless sandbox or mount the unrestricted Docker socket. +- [x] **Step 7: Commit.** Stage listed files and `git commit -m "fix: publish managed provider refreshes transactionally"`. ### Task 4: Inspected negative recovery evidence and conservative startup failures From 7d4f9c28c99af18439ac25ae66372e2a8ee577a7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:38:07 -0700 Subject: [PATCH 227/493] refactor: satisfy managed provider compiler checks --- .../src/managed_mcp_capability.rs | 40 +- .../amplifier_resources.rs | 2 +- .../managed_mcp_capability/claude_memory.rs | 11 +- .../managed_mcp_capability/source_stage.rs | 27 +- .../src/managed_provider_references.rs | 1975 +++++++++-------- .../opencode_resources.rs | 7 +- .../managed_provider_references/sources.rs | 2 + 7 files changed, 1046 insertions(+), 1018 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index babcb2de1..6f966d77a 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -505,7 +505,7 @@ async fn apply_prepared_callback( read_prior_manifest_bytes(provider, store, directory, &callback.soul_id, &context)?; enum AcceptedOrigins { Codex(freshell_platform::codex_inputs::CodexInputManifest), - Claude(freshell_platform::claude_inputs::ClaudeInputManifest), + Claude(Box), Amplifier(freshell_platform::amplifier_inputs::AmplifierInputManifest), } let origins = match provider { @@ -524,7 +524,7 @@ async fn apply_prepared_callback( manifest.accepted_native_plan = Some(plan.clone()); } manifest.validate().map_err(str::to_string)?; - AcceptedOrigins::Claude(manifest) + AcceptedOrigins::Claude(Box::new(manifest)) } "amplifier" => { let manifest: freshell_platform::amplifier_inputs::AmplifierInputManifest = @@ -894,12 +894,12 @@ fn read_prior_manifest_bytes( .mcp_capability .as_ref() .ok_or("prior source context lacks generation authority")?; - if !store + if store .grants .lock() .unwrap() .get(&capability.grant_id) - .is_some_and(|grant| grant.soul_id == *soul) + .is_none_or(|grant| grant.soul_id != *soul) { return Err("prior provider source context belongs to another owner".into()); } @@ -2374,17 +2374,19 @@ impl CapabilityStore { .ok_or("prepared source requires original provider HOME")? .1; let prepared = source_stage::describe( - root, - home, + source_stage::DescriptionRoots { + root, + home, + config_root: opencode_input.config_root, + workspace: opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + }, plan, &soul_id, &reference, opencode_input.initial_auth_reservation, - opencode_input.config_root, - opencode_input - .workspace - .or(opencode_input.cwd) - .unwrap_or(home), opencode_input.secret_references.unwrap_or(&[]), ) .and_then(|inputs| { @@ -2597,12 +2599,12 @@ impl CapabilityStore { let retained = recoverable_input_path(directory, &soul_id)?; let prior_stage = if let Some(recovery) = input.claude_recovery_source { recovery.validate().map_err(|error| error.message)?; - if !self + if self .grants .lock() .unwrap() .get(&recovery.prior_generation_id) - .is_some_and(|grant| grant.soul_id == soul_id) + .is_none_or(|grant| grant.soul_id != soul_id) { return Err("Claude exact prior generation ownership unavailable; retain inputs and reprepare".into()); } @@ -2922,7 +2924,7 @@ fn read_prepared_context( capability: &McpCapabilityReference, ) -> Result { let result = (|| { - let stage = staged_provider_root_path(&directory, &capability.grant_id)?; + let stage = staged_provider_root_path(directory, &capability.grant_id)?; let bytes = read_prepared_metadata(&stage, ".freshell/prepared-context.json", freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES) .map_err(|_| "complete prepared provider context unavailable or exceeds metadata profile; retain inputs and reprepare")?; let context: freshell_runtime_protocol::ProviderLaunchContext = @@ -3043,12 +3045,12 @@ fn bind_claude_candidate_request_and_plan( let generation = grant.grant_id.clone(); let inputs = context.prepared_inputs.as_ref().ok_or_else(blocked)?; if inputs.generation_id != generation - || !store + || store .grants .lock() .unwrap() .get(&generation) - .is_some_and(|grant| grant.soul_id == request.soul_id) + .is_none_or(|grant| grant.soul_id != request.soul_id) { return Err(blocked()); } @@ -7983,8 +7985,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ &client, ) .await - .err() - .expect( + .expect_err( "legacy metadata cannot grant current HOME authority for an unrecorded original source", ); assert!(error.contains("original Codex source origins are unrecorded")); @@ -8872,8 +8873,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ &client, ) .await - .err() - .expect("lost selected frozen ordinary source must block another fresh generation"); + .expect_err("lost selected frozen ordinary source must block another fresh generation"); assert!( error.contains("frozen selected source unavailable"), "{error}" diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs index d322e3f46..b846cbc5d 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_resources.rs @@ -1236,7 +1236,7 @@ mod module_source_tests { for unreadable in [false, true] { if unreadable { std::fs::write(&selected, "bundle: {name: selected}\n").unwrap(); - std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0)) + std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0o0)) .unwrap(); } let raw = format!( diff --git a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs index e1811fe21..62e4d71ba 100644 --- a/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs +++ b/crates/freshell-server/src/managed_mcp_capability/claude_memory.rs @@ -10,6 +10,7 @@ use std::{ sync::Arc, }; +#[cfg(test)] pub(super) fn imports( home: &Path, roots: &[PathBuf], @@ -20,6 +21,8 @@ pub(super) fn imports( .map(|(fresh, frozen)| fresh.into_iter().chain(frozen).collect()) } +type FrozenSource<'a> = dyn Fn(&Path) -> Result + 'a; + pub(super) fn imports_with_frozen( home: &Path, roots: &[PathBuf], @@ -27,7 +30,7 @@ pub(super) fn imports_with_frozen( frozen_roots: &[PathBuf], frozen_rules: &[PathBuf], excluded: &[PathBuf], - frozen_source: Option<&dyn Fn(&Path) -> Result>, + frozen_source: Option<&FrozenSource<'_>>, ) -> Result<(Vec, Vec), String> { let excluded = excluded .iter() @@ -83,7 +86,7 @@ struct ImportGraph<'a> { rule_directories: BTreeSet, rule_entries: usize, rule_path_bytes: usize, - frozen_source: Option<&'a dyn Fn(&Path) -> Result>, + frozen_source: Option<&'a FrozenSource<'a>>, } impl ImportGraph<'_> { fn source(&self, logical: &Path, frozen: bool) -> Result { @@ -443,10 +446,10 @@ mod tests { let root = owned.path().join("CLAUDE.md"); let nested = owned.path().join("nested.md"); std::fs::write(&root, "@nested.md\n").unwrap(); - let missing = imports(owned.path(), &[root.clone()], &[], &[]).unwrap_err(); + let missing = imports(owned.path(), std::slice::from_ref(&root), &[], &[]).unwrap_err(); assert!(missing.contains("nested.md") && missing.contains("unavailable")); std::fs::write(&nested, "@CLAUDE.md\n").unwrap(); - let cycle = imports(owned.path(), &[root.clone()], &[], &[]).unwrap_err(); + let cycle = imports(owned.path(), std::slice::from_ref(&root), &[], &[]).unwrap_err(); assert!(cycle.contains("cyclic")); assert_eq!(std::fs::read_to_string(root).unwrap(), "@nested.md\n"); assert_eq!(std::fs::read_to_string(nested).unwrap(), "@CLAUDE.md\n"); diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index d0e35fe09..8b3464874 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -272,7 +272,7 @@ mod configured_resource_tests { for unreadable in [false, true] { if unreadable { std::fs::write(&resource, "resource").unwrap(); - std::fs::set_permissions(&resource, std::fs::Permissions::from_mode(0)) + std::fs::set_permissions(&resource, std::fs::Permissions::from_mode(0o0)) .unwrap(); } let raw = serde_json::to_string(&serde_json::json!({field:[resource]})).unwrap(); @@ -630,7 +630,8 @@ mod configured_resource_tests { _ => std::fs::write(&target, "resource").unwrap(), } if case == "unreadable" { - std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0)).unwrap(); + std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o0)) + .unwrap(); } let raw=if field=="sound" {serde_json::json!({"attention":{"sounds":{"question":target}}})} else {serde_json::json!({"mcp":{"probe":{"type":"local","command":["node","server.cjs"],"cwd":target}}})}.to_string(); std::fs::write(&config, &raw).unwrap(); @@ -779,7 +780,7 @@ mod configured_resource_tests { let cyclic = owned.path().join("cyclic"); std::fs::write(&file, "not a directory").unwrap(); std::fs::create_dir(&unreadable).unwrap(); - std::fs::set_permissions(&unreadable, std::fs::Permissions::from_mode(0)).unwrap(); + std::fs::set_permissions(&unreadable, std::fs::Permissions::from_mode(0o0)).unwrap(); std::os::unix::fs::symlink(&cyclic, &cyclic).unwrap(); for selected in [&missing, &file, &unreadable, &cyclic] { let raw = serde_json::json!({"skills":{"paths":[selected],"urls":["https://example.invalid/skills"]}}).to_string(); @@ -968,7 +969,7 @@ pub(super) fn resource_units( { continue; } - let canonical = std::fs::canonicalize(&logical) + let canonical = std::fs::canonicalize(logical) .map_err(|_| "selected OpenCode configured resource is unavailable")?; roots.insert((logical.clone(), role), canonical); } @@ -1185,20 +1186,30 @@ pub(super) fn stage( write_private_bytes(&root.join(".freshell/input-manifest.json"), &metadata) } +pub(super) struct DescriptionRoots<'a> { + pub root: &'a Path, + pub home: &'a Path, + pub config_root: Option<&'a Path>, + pub workspace: &'a Path, +} + pub(super) fn describe( - root: &Path, - home: &Path, + roots: DescriptionRoots<'_>, plan: &[PlannedSourceOccurrence], soul: &freshell_runtime_protocol::SoulId, capability: &freshell_runtime_protocol::McpCapabilityReference, initial_auth_reservation: Option<&str>, - config_root: Option<&Path>, - workspace: &Path, secret_references: &[freshell_runtime_protocol::ProviderSecretReference], ) -> Result { use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; use std::io::Read; + let DescriptionRoots { + root, + home, + config_root, + workspace, + } = roots; let runtime_home = Path::new("/home/freshell/provider"); let default_global = home.join(".config/opencode"); let global = config_root.unwrap_or(&default_global); diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index c69830f1d..266cbff7b 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -15,7 +15,6 @@ pub struct FileSourceOrigin<'a> { pub original_home: &'a Path, } -#[derive(Debug)] pub struct ReferenceError { pub operation: &'static str, pub field: &'static str, @@ -24,6 +23,18 @@ pub struct ReferenceError { pub variables: Vec, pub cause: &'static str, } +impl std::fmt::Debug for ReferenceError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("ReferenceError") + .field("operation", &self.operation) + .field("field", &self.field) + .field("source_ordinal", &self.source_ordinal) + .field("declaring_source", &self.declaring_source) + .field("variables", &self.variables) + .field("cause", &self.cause) + .finish() + } +} impl std::fmt::Display for ReferenceError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { write!( @@ -42,7 +53,6 @@ impl std::error::Error for ReferenceError {} pub struct FileReferenceSelection { pub canonical_source: PathBuf, pub runtime_destination: String, - pub source_ordinal: usize, } pub use freshell_platform::opencode_recipe::{ApprovedFileOccurrence, GeneratedFileRecipe}; @@ -79,6 +89,7 @@ impl ResolvedFileReferencePlan { } } +#[cfg(test)] pub fn resolve_file_references( raw: &str, origin: FileSourceOrigin<'_>, @@ -92,6 +103,7 @@ pub fn resolve_file_references( }) } +#[cfg(test)] pub(super) fn resolve_file_references_allocated( raw: &str, origin: FileSourceOrigin<'_>, @@ -102,6 +114,7 @@ pub(super) fn resolve_file_references_allocated( resolve_file_references_from_view(raw, None, origin, env, recipe_ordinal, allocate) } +#[cfg(test)] pub(super) fn resolve_file_references_from_view( raw: &str, source_view: Option>, @@ -253,7 +266,6 @@ pub(super) fn resolve_file_references_from_view_relocated( selections.push(FileReferenceSelection { canonical_source: canonical, runtime_destination: destination.clone(), - source_ordinal: token.ordinal, }); destination }; @@ -406,6 +418,7 @@ pub(super) fn resolve_file_references_from_view_relocated( }) } +#[cfg(test)] pub fn evaluate_generated_recipe( recipe: &GeneratedFileRecipe, env: &dyn Fn(&str) -> Option, @@ -420,1029 +433,1029 @@ pub fn evaluate_generated_recipe( }) } -#[cfg(test)] -mod tests { - use super::*; - use sha2::{Digest, Sha256}; +pub(crate) struct PreparedManagedOpencode { + pub context: freshell_runtime_protocol::ProviderLaunchContext, + pub references: Vec, +} - // Pinned resolver-equivalent consumer for OpenCode 1.18.21 variable.ts. - // Receipt is a digest of values consumed by its native JSONC/file substitution. - const CONSUMER: &str = r#" -const fs=require('fs'),path=require('path'),crypto=require('crypto'); -const source=process.argv[1],home=process.argv[2],env=JSON.parse(process.argv[3]); -let text=fs.readFileSync(source,'utf8').replace(/\{env:([^}]+)\}/g,(_,n)=>env[n]||''); -let out='',cursor=0; -for(const m of Array.from(text.matchAll(/\{file:[^}]+\}/g))) { - out+=text.slice(cursor,m.index); - if(text.slice(text.lastIndexOf('\n',m.index-1)+1,m.index).trimStart().startsWith('//')) out+=m[0]; - else { - const target=m[0].slice(6,-1),prefix='/home/freshell/provider/'; - if(!target.startsWith(prefix)) process.exit(71); - const bytes=fs.readFileSync(path.join(home,target.slice(prefix.length)),'utf8'); - out+=JSON.stringify(bytes.trim()).slice(1,-1); - } - cursor=m.index+m[0].length; +/// Terminal, fresh and scoped replacement callers share this one preparation. +/// The original lookup and decoded grants never enter the returned metadata. +pub(crate) struct OpencodePreparation<'a> { + pub client: &'a freshell_runtime_client::RuntimeClient, + pub soul: &'a freshell_runtime_protocol::SoulId, + pub creation_seed: &'a str, + pub home: &'a Path, + pub cwd: &'a Path, + pub workspace: &'a Path, + pub original: &'a (dyn Fn(&str) -> Option + Sync), + pub references: Vec, + pub auth_authority: Option<&'a str>, + pub selected_sources: Option>, + pub accepted_global_root: Option, + pub selected_resource_roots: Option>, + pub native_source_flags: Option>, + pub rebind_enabled: Option, + pub inherited_inline: Option, + pub inline_edits: Option, } -out+=text.slice(cursor); -const parsed=JSON.parse(out),receipts=Object.values(parsed).map(v=>crypto.createHash('sha256').update(v).digest('hex')); -process.stdout.write(JSON.stringify(receipts)); -"#; - fn digest(value: &str) -> String { - format!("{:x}", Sha256::digest(value.as_bytes())) - } - fn consume(plan: &ResolvedFileReferencePlan, runtime: &Path) -> Vec { - consume_with_env(plan, runtime, &|_| None) - } - fn consume_with_env( - plan: &ResolvedFileReferencePlan, - runtime: &Path, - env: &dyn Fn(&str) -> Option, - ) -> Vec { - std::fs::create_dir_all(runtime.join(".freshell/references")).unwrap(); - for selection in &plan.selections { - let relative = selection - .runtime_destination - .strip_prefix("/home/freshell/provider/") - .unwrap(); - std::fs::copy(&selection.canonical_source, runtime.join(relative)).unwrap(); - } - let source = runtime.join("source.json"); - std::fs::write(&source, plan.template.as_bytes()).unwrap(); - let mut child = std::collections::BTreeMap::new(); - for recipe in &plan.recipes { - child.insert( - recipe.generated_name.clone(), - evaluate_generated_recipe(recipe, env).unwrap(), - ); - } - consume_published(runtime, &child) - } +pub(crate) async fn prepare_managed_opencode( + request: OpencodePreparation<'_>, +) -> Result { + prepare_managed_opencode_with_publisher( + request, + crate::managed_mcp_capability::issue_prepared_opencode_context, + ) + .await +} - fn consume_published( - runtime: &Path, - child: &std::collections::BTreeMap, - ) -> Vec { - let result = std::process::Command::new("node") - .arg("-e") - .arg(CONSUMER) - .arg(runtime.join("source.json")) - .arg(runtime) - .arg(serde_json::to_string(&child).unwrap()) - .output() - .unwrap(); - assert!( - result.status.success(), - "pinned consumer failed with {:?}", - result.status.code() - ); - serde_json::from_slice(&result.stdout).unwrap() +pub(crate) async fn prepare_managed_opencode_with_publisher( + request: OpencodePreparation<'_>, + publish: impl FnOnce( + &freshell_runtime_protocol::SoulId, + &Path, + &freshell_runtime_protocol::ProviderLaunchContext, + crate::managed_mcp_capability::OpencodeEphemeralInput<'_>, + ) -> Result, +) -> Result { + let OpencodePreparation { + client, + soul, + creation_seed, + home, + cwd, + workspace, + original, + mut references, + auth_authority, + selected_sources, + accepted_global_root, + selected_resource_roots, + native_source_flags, + rebind_enabled, + inherited_inline, + inline_edits, + } = request; + let rebind_enabled = rebind_enabled.unwrap_or_else(|| { + !matches!( + original("FRESHELL_OPENCODE_REBIND").as_deref(), + Some("0" | "false") + ) + }); + use sources::*; + let inherited_inline = inherited_inline + .or_else(|| original("OPENCODE_CONFIG_CONTENT")) + .filter(|raw| !raw.is_empty()); + let mut inline_edits = inline_edits.unwrap_or_default(); + inline_edits.freshell_command = Some(vec!["node".into(), "/opt/freshell-mcp/server.js".into()]); + if inline_edits.bash_permission.is_none() { + inline_edits.bash_permission = original("FRESHELL_MANAGED_OPENCODE_BASH_PERMISSION") + .filter(|value| matches!(value.as_str(), "allow" | "ask" | "deny")); } - - #[cfg(unix)] - #[test] - fn managed_provider_references_actual_maximum_sources_aliases_reach_child_and_overflow_preserves_projection( - ) { - let owned = tempfile::tempdir().unwrap(); - let source = owned.path().join("opencode.json"); - let runtime = owned.path().join("runtime"); - let mut fields = serde_json::Map::new(); - let mut expected = Vec::new(); - for ordinal in 0..128 { - let name = format!("payload-{ordinal:03}"); - let payload = format!(" selected-reference-nonce-{ordinal:03}\nquote\"slash\\ "); - std::fs::write(owned.path().join(&name), &payload).unwrap(); - for suffix in ["a", "b"] { - let alias = format!("alias-{ordinal:03}-{suffix}"); - std::os::unix::fs::symlink(&name, owned.path().join(&alias)).unwrap(); - fields.insert( - format!("field-{ordinal:03}-{suffix}"), - serde_json::Value::String(format!("{{file:{alias}}}")), - ); - expected.push(digest(payload.trim())); - } + let config_root = if let Some(root) = accepted_global_root { + if !root.is_absolute() || root.file_name() != Some(std::ffi::OsStr::new("opencode")) { + return Err( + "accepted global configuration root is invalid; retain prior inputs and reprepare" + .into(), + ); } - let raw = serde_json::Value::Object(fields.clone()).to_string(); - std::fs::write(&source, &raw).unwrap(); - let resolve = |raw: &str| { - resolve_file_references( - raw, - FileSourceOrigin { - declaring_config: &source, - cwd: owned.path(), - original_home: owned.path(), - }, - &|_| None, - ) + root + } else { + let xdg = original("XDG_CONFIG_HOME") + .filter(|value| !value.is_empty()) + .map(PathBuf::from) + .unwrap_or_else(|| home.join(".config")); + let xdg = if xdg.is_absolute() { + xdg + } else { + cwd.join(xdg) }; - let plan = resolve(&raw).unwrap(); - assert_eq!(plan.selections.len(), 128); - assert_eq!(plan.occurrences.len(), 256); - assert_eq!( - plan.occurrences - .iter() - .map(|occurrence| &occurrence.approved_alias) - .collect::>() - .len(), - 256 - ); - assert_eq!(consume(&plan, &runtime), expected); - let prior_source = std::fs::read(runtime.join("source.json")).unwrap(); - std::os::unix::fs::symlink("payload-000", owned.path().join("alias-overflow")).unwrap(); - let mut alias_overflow = fields.clone(); - alias_overflow.insert( - "overflow".into(), - serde_json::Value::String("{file:alias-overflow}".into()), - ); - let error = resolve(&serde_json::Value::Object(alias_overflow).to_string()) - .err() - .unwrap(); - assert_eq!(error.cause, "configuration alias count exceeds 256"); - std::fs::write(owned.path().join("source-overflow"), b"PRIVATE_OVERFLOW").unwrap(); - fields.insert( - "overflow".into(), - serde_json::Value::String("{file:source-overflow}".into()), - ); - let error = resolve(&serde_json::Value::Object(fields).to_string()) - .err() - .unwrap(); - assert_eq!(error.cause, "configuration reference count exceeds 128"); - assert!(!error.to_string().contains("PRIVATE_OVERFLOW")); - assert_eq!(std::fs::read(&source).unwrap(), raw.as_bytes()); - assert_eq!( - std::fs::read(runtime.join("source.json")).unwrap(), - prior_source - ); - assert_eq!(consume_published(&runtime, &Default::default()), expected); - assert_eq!( - std::fs::read_dir(runtime.join(".freshell/references")) - .unwrap() - .count(), - 128 - ); - } - - #[test] - fn managed_provider_references_actual_maximum_file_events_reach_child_and_overflow_preserves_projection( - ) { - let owned = tempfile::tempdir().unwrap(); - let source = owned.path().join("opencode.json"); - let runtime = owned.path().join("runtime"); - std::fs::write(owned.path().join("rules"), b" event-capacity-nonce ").unwrap(); - let raw = serde_json::Value::Object( - (0..8192) - .map(|ordinal| { - ( - format!("field-{ordinal:04}"), - serde_json::Value::String("{file:rules}".into()), - ) - }) - .collect(), + xdg.join("opencode") + }; + let xdg = config_root + .parent() + .ok_or("global configuration root parent unavailable")?; + let resource_roots = if let Some(roots) = selected_resource_roots { + roots + } else { + sources::select_opencode_resource_roots(&OpencodeDiscovery { + home, + cwd, + worktree: workspace, + config_root: &config_root, + env: original, + }) + .map_err(str::to_string)? + }; + let base = crate::managed_provider_bootstrap::provider_launch_context_from_home_at_with_xdg( + "opencode", + workspace, + cwd, + home, + Some(xdg), + None, + )? + .ok_or("OpenCode provider preparation unavailable")?; + let selected = if let Some(sources) = selected_sources { + sources + } else { + discover_opencode(OpencodeDiscovery { + home, + cwd, + worktree: workspace, + config_root: &config_root, + env: original, + }) + .map_err(str::to_string)? + }; + let mut loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; + if let Some(raw) = inherited_inline { + if raw.len() > 16 * 1024 * 1024 { + return Err("OpenCode inherited inline source exceeds 16 MiB".into()); + } + let retained = sources::retained_source_view_bytes(&loaded) + .map_err(str::to_string)? + .checked_add(raw.len()) + .ok_or("source-view size overflow")?; + let remaining = (64 * 1024 * 1024usize) + .checked_sub(retained) + .ok_or("simultaneous source views exceed 64 MiB")?; + let user_owns_freshell_mcp = loaded + .iter() + .filter(|source| source.source.lane == SourceLane::Main) + .try_fold(false, |owned, source| { + freshell_platform::opencode_config::source_owns_freshell_mcp( + &source.view, + &source.source.logical_path, + cwd, + ) + .map(|user_owns| owned || user_owns) + }) + .map_err(|error| error.to_string())?; + let view = freshell_platform::opencode_config::merge_owned_entries_bounded( + Some(&raw), + &freshell_platform::opencode_config::OpencodeConfigPlan { + sources: vec![], + user_owns_freshell_mcp, + inline_prepared: false, + }, + &inline_edits, + remaining.min(64 * 1024), ) - .to_string(); - std::fs::write(&source, &raw).unwrap(); - let resolve = |raw: &str| { - resolve_file_references( - raw, - FileSourceOrigin { - declaring_config: &source, - cwd: owned.path(), - original_home: owned.path(), - }, - &|_| None, - ) + .map_err(|error| error.to_string())?; + let inline = LoadedSourceOccurrence { + requires_config_bind: true, + source: SourceOccurrence { + logical_path: "OPENCODE_CONFIG_CONTENT".into(), + canonical_source: "OPENCODE_CONFIG_CONTENT".into(), + lane: SourceLane::Inline, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, + }, + view: std::sync::Arc::new(view), }; - let plan = resolve(&raw).unwrap(); - assert_eq!(plan.substitution_events, 8192); - assert_eq!(plan.occurrences.len(), 8192); - assert_eq!(plan.selections.len(), 1); - let expected = vec![digest("event-capacity-nonce"); 8192]; - assert_eq!(consume(&plan, &runtime), expected); - let overflow = format!( - "{},\"overflow\":\"{{file:rules}}\"}}", - &raw[..raw.len() - 1] - ); - let error = resolve(&overflow).err().unwrap(); - assert_eq!( - error.cause, - "configuration source, expansion or event profile limit exceeded" - ); - assert_eq!(std::fs::read(&source).unwrap(), raw.as_bytes()); - assert_eq!( - std::fs::read(runtime.join("source.json")).unwrap(), - plan.template.as_bytes() - ); - assert_eq!(consume_published(&runtime, &Default::default()), expected); - assert_eq!( - std::fs::read_dir(runtime.join(".freshell/references")) - .unwrap() - .count(), - 1 - ); + let position = loaded + .iter() + .position(|source| source.source.lane == SourceLane::Tui) + .unwrap_or(loaded.len()); + loaded.insert(position, inline); } - - #[test] - fn managed_provider_references_child_consumes_original_absolute_relative_home_spaces_and_symlink( - ) { - let temp = tempfile::tempdir().unwrap(); - let home = temp.path().join("original-home"); - let declaring = temp.path().join("project/nested"); - std::fs::create_dir_all(&home).unwrap(); - std::fs::create_dir_all(&declaring).unwrap(); - let payload = " synthetic-nonce-quote\"slash\\\ntrim "; - for path in [ - home.join("home-ref"), - declaring.join("relative-ref"), - declaring.join("with spaces"), - temp.path().join("external-ref"), + let native_source_flags = if let Some(flags) = native_source_flags { + flags + } else { + let mut flags = Vec::new(); + for (name, kind) in [ + ( + "OPENCODE_CONFIG", + freshell_platform::opencode_recipe::NativeSourceFlagKind::Main, + ), + ( + "OPENCODE_TUI_CONFIG", + freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui, + ), ] { - std::fs::write(path, payload).unwrap(); + if let Some(path) = original(name).filter(|path| !path.is_empty()) { + let path = if Path::new(&path).is_absolute() { + PathBuf::from(path) + } else { + cwd.join(path) + }; + let source_ordinal = loaded + .iter() + .position(|source| source.source.logical_path == path && source.source.role == DiscoveryRole::Workspace && match kind { + freshell_platform::opencode_recipe::NativeSourceFlagKind::Main => source.source.lane == SourceLane::Main, + freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => source.source.lane == SourceLane::Tui, + freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory => false, + }).ok_or_else(|| format!("selected {name} source is missing or unavailable; repair it before dispatch"))?; + let destination = sources::runtime_config_destination( + home, + &config_root, + &path, + loaded[source_ordinal].source.role, + ); + flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { + kind, + source_ordinal: Some( + u16::try_from(source_ordinal) + .map_err(|_| "native source ordinal overflow")?, + ), + runtime_destination: destination + .to_str() + .ok_or("selected native source destination is not UTF-8")? + .into(), + }); + } } - #[cfg(unix)] - std::os::unix::fs::symlink("relative-ref", declaring.join("linked-ref")).unwrap(); - let targets = vec![ - temp.path().join("external-ref").display().to_string(), - "relative-ref".into(), - "~/home-ref".into(), - "with spaces".into(), - "linked-ref".into(), - ]; - for target in targets { - let raw = serde_json::json!({"instructions":format!("{{file:{target}}}")}).to_string(); - let original = declaring.join("opencode.json"); - std::fs::write(&original, &raw).unwrap(); - let plan = resolve_file_references( - &raw, - FileSourceOrigin { - declaring_config: &original, - cwd: &declaring, - original_home: &home, - }, - &|_| None, - ) - .unwrap(); - assert_eq!( - consume(&plan, &temp.path().join("runtime")), - vec![digest(payload.trim())] - ); - assert_eq!(std::fs::read_to_string(&original).unwrap(), raw); - assert!(!plan.template.contains("synthetic-nonce")); + if let Some(path) = original("OPENCODE_CONFIG_DIR").filter(|path| !path.is_empty()) { + let path = if Path::new(&path).is_absolute() { + PathBuf::from(path) + } else { + cwd.join(path) + }; + if !resource_roots.iter().any(|root| root.logical_path == path) { + return Err( + "selected OPENCODE_CONFIG_DIR root is unavailable before dispatch".into(), + ); + } + flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { + kind: freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory, + source_ordinal: None, + runtime_destination: sources::runtime_directory_destination( + home, + &config_root, + &path, + DiscoveryRole::Workspace, + ) + .to_str() + .ok_or("selected directory is not UTF-8")? + .into(), + }); } - } - #[test] - fn managed_provider_references_same_relative_spelling_keeps_each_declaring_origin() { - let temp = tempfile::tempdir().unwrap(); - for (index, nonce) in ["first-origin-nonce", "second-origin-nonce"] - .iter() - .enumerate() + flags + }; + let mut retained_views = + sources::retained_source_view_bytes(&loaded).map_err(str::to_string)?; + let mut edited = std::collections::BTreeSet::new(); + for ordinal in 0..loaded.len() { + let source = &loaded[ordinal]; + if rebind_enabled + && source.source.lane == SourceLane::Tui + && native_source_flags.iter().any(|flag| { + flag.kind == freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui + && Path::new(&flag.runtime_destination) + == sources::runtime_config_destination( + home, + &config_root, + &source.source.logical_path, + source.source.role, + ) + }) { - let origin = temp.path().join(format!("origin-{index}")); - std::fs::create_dir_all(&origin).unwrap(); - std::fs::write(origin.join("rules"), nonce).unwrap(); - let config = origin.join("opencode.json"); - let raw = r#"{"instructions":"{file:rules}"}"#; - let plan = resolve_file_references( - raw, - FileSourceOrigin { - declaring_config: &config, - cwd: temp.path(), - original_home: temp.path(), - }, - &|_| None, + let canonical = source.source.canonical_source.clone(); + if !edited.insert(canonical.clone()) { + continue; + } + let shared = loaded + .iter() + .filter(|source| source.source.canonical_source == canonical) + .count() + > 1; + let remaining = (64 * 1024 * 1024usize) + .checked_sub(retained_views) + .ok_or("simultaneous source views exceed 64 MiB")?; + let runtime_home = Path::new("/home/freshell/provider"); + let plugin = if shared { + freshell_platform::opencode_plugin::rebind_plugin_package_path(runtime_home) + } else { + freshell_platform::opencode_plugin::rebind_plugin_path(runtime_home) + }; + let content = freshell_platform::opencode_plugin::selected_tui_content_bounded( + &source.view, + &plugin, + remaining, ) - .unwrap(); - assert_eq!( - consume(&plan, &temp.path().join(format!("runtime-{index}"))), - vec![digest(nonce)] - ); + .map_err(|_| { + format!( + "selected OpenCode TUI source {} has invalid JSONC/plugin list or exceeds the source-view profile", + source.source.logical_path.display() + ) + })?; + if content.len() > 16 * 1024 * 1024 { + return Err("selected OpenCode effective TUI source exceeds 16 MiB".into()); + } + retained_views = retained_views + .checked_add(content.len()) + .ok_or("source-view byte count overflow")?; + let view = std::sync::Arc::new(content); + for (index, source) in loaded.iter_mut().enumerate() { + if index == ordinal || shared && source.source.canonical_source == canonical { + source.view = view.clone(); + source.requires_config_bind = true; + } + } } } - #[test] - fn managed_provider_references_nested_path_env_is_consumed_without_forwarding_value() { - let temp = tempfile::tempdir().unwrap(); - let target = temp.path().join("private-path-ref"); - std::fs::write(&target, "nested-path-consumption-nonce").unwrap(); - let source = temp.path().join("opencode.json"); - let raw = r#"{"instructions":"{file:{env:NODE_EXTRA_CA_CERTS}}"}"#; - let plan = resolve_file_references( - raw, - FileSourceOrigin { - declaring_config: &source, - cwd: temp.path(), - original_home: temp.path(), - }, - &|name| (name == "NODE_EXTRA_CA_CERTS").then(|| target.display().to_string()), + let names = source_environment_names(&loaded).map_err(|error| error.to_string())?; + let mut environment = environment::SourceEnvironment::read(names, &references, original)?; + // Even a typed auth grant needs explicit ownership of a newly absent soul + // volume before its descriptor-owned mountpoints can be constructed. + let reservation = if auth_authority.is_none() { + Some( + client + .reserve_provider_inputs(soul.clone(), "opencode".into(), creation_seed.into()) + .await + .map_err(|error| error.to_string())? + .reservation_id, ) - .unwrap(); - assert_eq!( - consume(&plan, &temp.path().join("runtime")), - vec![digest("nested-path-consumption-nonce")] - ); - assert_eq!(plan.referenced_variables, vec!["NODE_EXTRA_CA_CERTS"]); - assert!(!plan.template.contains("private-path-ref")); + } else { + None + }; + if environment.needs_native_auth() + && loaded + .iter() + .any(|source| source.source.lane == SourceLane::Main) + { + let authority = if let Some(authority) = auth_authority { + authority.to_owned() + } else { + reservation + .clone() + .ok_or("initial provider input reservation unavailable")? + }; + let values = client + .native_auth_lookup(soul.clone(), authority, environment.names().to_vec()) + .await + .map_err(|error| error.to_string())?; + environment.set_native_auth(values.0)?; } - #[test] - fn managed_provider_references_missing_and_cyclic_targets_fail_without_echoing_candidate_values( - ) { - let temp = tempfile::tempdir().unwrap(); - let source = temp.path().join("opencode.json"); - #[cfg(unix)] - { - std::os::unix::fs::symlink("cycle-b", temp.path().join("cycle-a")).unwrap(); - std::os::unix::fs::symlink("cycle-a", temp.path().join("cycle-b")).unwrap(); - } - for target in ["private-missing-candidate", "cycle-a"] { - let result = resolve_file_references( - r#"{"x":"{file:{env:PATH_INPUT}}"}"#, - FileSourceOrigin { - declaring_config: &source, - cwd: temp.path(), - original_home: temp.path(), - }, - &|_| Some(temp.path().join(target).display().to_string()), - ); - let err = result - .err() - .expect("missing or cyclic reference cannot launch"); - assert_eq!(err.source_ordinal, 0); - assert_eq!(err.variables, vec!["PATH_INPUT"]); - assert!(!format!("{err:?} {err}").contains(target)); + let plan = environment + .plan_sources(loaded, cwd, home) + .map_err(|error| error.to_string())?; + let frozen = environment.frozen_inputs(&plan)?; + let frozen_names = frozen + .iter() + .map(|(name, _)| name.to_owned()) + .collect::>(); + let mut files = std::collections::BTreeMap::new(); + for source in &plan { + for selection in &source.references.selections { + let id = selection + .runtime_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .and_then(|value| value.parse::().ok()) + .ok_or("invalid prepared source index")?; + if files + .insert(id, selection.canonical_source.clone()) + .is_some_and(|prior| prior != selection.canonical_source) + { + return Err("prepared source changes one selected index".into()); + } } } - #[test] - fn managed_provider_references_env_introduced_file_tokens_and_private_prefix_use_child_recipe() - { - let temp = tempfile::tempdir().unwrap(); - let target = temp.path().join("reference"); - std::fs::write(&target, " payload-nonce ").unwrap(); - let source = temp.path().join("opencode.json"); - let raw = r#"{"instructions":"{env:OPENAI_API_KEY}"}"#; - let lookup = |name: &str| { - (name == "OPENAI_API_KEY").then(|| { - format!( - "sentinel-private-prefix{{file:{}}}sentinel-private-suffix", - target.display() - ) - }) - }; - let plan = resolve_file_references( - raw, - FileSourceOrigin { - declaring_config: &source, - cwd: temp.path(), - original_home: temp.path(), - }, - &lookup, - ) - .unwrap(); - assert!(!plan.template.contains("sentinel-private")); - assert_eq!(plan.recipes.len(), 1); - assert!(!plan.recipes[0] - .unexpanded_source - .contains("sentinel-private")); - assert_eq!( - consume_with_env(&plan, &temp.path().join("runtime"), &lookup), - vec![digest( - "sentinel-private-prefixpayload-noncesentinel-private-suffix" - )] - ); + for (expected, (id, path)) in files.into_iter().enumerate() { + if id as usize != expected { + return Err("prepared source indexes are not contiguous".into()); + } + references.push(freshell_runtime_protocol::ProviderSecretReference { + source_path: path.to_str().ok_or("selected source is not UTF-8")?.into(), + profile: freshell_runtime_protocol::ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{id}")), + }); } - - #[test] - fn managed_provider_references_changed_grant_selection_blocks_before_child_and_keeps_prior_projection( - ) { - let temp = tempfile::tempdir().unwrap(); - let a = temp.path().join("a"); - let b = temp.path().join("b"); - std::fs::write(&a, "prior-usable-nonce").unwrap(); - std::fs::write(&b, "unapproved-new-nonce").unwrap(); - let source = temp.path().join("opencode.json"); - let initial = |name: &str| { - (name == "OPENAI_API_KEY").then(|| format!("prefix{{file:{}}}", a.display())) - }; - let plan = resolve_file_references( - r#"{"x":"{env:OPENAI_API_KEY}"}"#, - FileSourceOrigin { - declaring_config: &source, - cwd: temp.path(), - original_home: temp.path(), - }, - &initial, - ) - .unwrap(); - let previous = temp.path().join("runtime"); - assert_eq!( - consume_with_env(&plan, &previous, &initial), - vec![digest("prefixprior-usable-nonce")] - ); - let changed = |name: &str| { - (name == "OPENAI_API_KEY") - .then(|| format!("sentinel-private-changed{{file:{}}}", b.display())) - }; - let err = evaluate_generated_recipe(&plan.recipes[0], &changed) - .err() - .unwrap(); - assert!(err.cause.contains("selection changed")); - assert!(!format!("{err:?} {err}").contains("sentinel-private")); - assert_eq!( - std::fs::read_to_string(previous.join(".freshell/references/0")).unwrap(), - "prior-usable-nonce" - ); - let same_selection = |name: &str| { - (name == "OPENAI_API_KEY").then(|| format!("different-prefix{{file:{}}}", a.display())) - }; - assert!(evaluate_generated_recipe(&plan.recipes[0], &same_selection) - .unwrap() - .contains("different-prefix")); - } - #[test] - fn managed_provider_references_whole_document_native_wrapper_handles_outside_lexeme_expansion() - { - let temp = tempfile::tempdir().unwrap(); - let target = temp.path().join("reference"); - std::fs::write(&target, "document-consumption-nonce").unwrap(); - let source = temp.path().join("opencode.json"); - let lookup = |name: &str| { - (name == "OPENAI_API_KEY").then(|| { - serde_json::json!({"instructions":format!("{{file:{}}}",target.display())}) - .to_string() - }) - }; - let plan = resolve_file_references( - "{env:OPENAI_API_KEY}", - FileSourceOrigin { - declaring_config: &source, - cwd: temp.path(), - original_home: temp.path(), - }, - &lookup, - ) - .unwrap(); - assert_eq!( - consume_with_env(&plan, &temp.path().join("runtime"), &lookup), - vec![digest("document-consumption-nonce")] - ); - assert_eq!(plan.recipes[0].unexpanded_source, "{env:OPENAI_API_KEY}"); - } - #[test] - fn managed_provider_references_virtual_inline_uses_instance_cwd() { - let temp = tempfile::tempdir().unwrap(); - std::fs::write(temp.path().join("rules"), "inline-instance-nonce").unwrap(); - let plan = resolve_file_references( - r#"{"x":"{file:rules}"}"#, - FileSourceOrigin { - declaring_config: Path::new("OPENCODE_CONFIG_CONTENT"), - cwd: temp.path(), - original_home: temp.path(), - }, - &|_| None, - ) - .unwrap(); - assert_eq!( - consume(&plan, &temp.path().join("runtime")), - vec![digest("inline-instance-nonce")] - ); - } - #[test] - fn managed_provider_references_relative_parent_is_lexical_before_following_directory_symlink() { - let temp = tempfile::tempdir().unwrap(); - let logical = temp.path().join("logical"); - let actual = temp.path().join("actual/config"); - std::fs::create_dir_all(&logical).unwrap(); - std::fs::create_dir_all(&actual).unwrap(); - std::fs::write(logical.join("rules"), "native-lexical-origin-nonce").unwrap(); - std::fs::write( - temp.path().join("actual/rules"), - "wrong-physical-origin-nonce", - ) - .unwrap(); - #[cfg(unix)] - std::os::unix::fs::symlink(&actual, logical.join("linked")).unwrap(); - let source = logical.join("linked/opencode.json"); - let plan = resolve_file_references( - r#"{"x":"{file:../rules}"}"#, - FileSourceOrigin { - declaring_config: &source, - cwd: temp.path(), - original_home: temp.path(), - }, - &|_| None, - ) - .unwrap(); - assert_eq!( - consume(&plan, &temp.path().join("runtime")), - vec![digest("native-lexical-origin-nonce")] - ); + freshell_runtime_protocol::validate_provider_secrets("opencode", &references) + .map_err(|error| error.message)?; + let context = publish( + soul, + home, + &base, + crate::managed_mcp_capability::OpencodeEphemeralInput { + cwd: Some(cwd), + workspace: Some(workspace), + config_root: Some(&config_root), + source_plan: Some(&plan), + resource_roots: Some(&resource_roots), + initial_auth_reservation: reservation.as_deref(), + secret_references: Some(&references), + frozen_inputs: Some(&frozen_names), + native_source_flags: Some(&native_source_flags), + rebind_enabled: Some(rebind_enabled), + inline_edits: Some(&inline_edits), + ..Default::default() + }, + )?; + let capability = context + .mcp_capability + .as_ref() + .ok_or("prepared source capability unavailable")?; + if !frozen.is_empty() { + if let Err(error) = client + .bind_provider_path_inputs(soul.clone(), capability.grant_id.clone(), frozen) + .await + { + let _ = crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id); + return Err(error.to_string()); + } } + context + .prepared_inputs + .as_ref() + .ok_or("prepared source descriptors unavailable")? + .validate_references(&references) + .map_err(|error| error.message)?; + Ok(PreparedManagedOpencode { + context, + references, + }) } -pub(crate) struct PreparedManagedOpencode { - pub context: freshell_runtime_protocol::ProviderLaunchContext, - pub references: Vec, -} - -/// Terminal, fresh and scoped replacement callers share this one preparation. -/// The original lookup and decoded grants never enter the returned metadata. -pub(crate) struct OpencodePreparation<'a> { - pub client: &'a freshell_runtime_client::RuntimeClient, - pub soul: &'a freshell_runtime_protocol::SoulId, - pub creation_seed: &'a str, - pub home: &'a Path, - pub cwd: &'a Path, - pub workspace: &'a Path, - pub original: &'a (dyn Fn(&str) -> Option + Sync), - pub references: Vec, - pub auth_authority: Option<&'a str>, - pub selected_sources: Option>, - pub accepted_global_root: Option, - pub selected_resource_roots: Option>, - pub native_source_flags: Option>, - pub rebind_enabled: Option, - pub inherited_inline: Option, - pub inline_edits: Option, +/// Rebuild only a positively rejected, never-dispatched initial OpenCode input +/// generation. Other failures retain their inputs for ownership reconciliation. +pub(crate) async fn launch_initial_opencode( + preparation: OpencodePreparation<'_>, + mut request_id: freshell_runtime_protocol::RequestId, + mut launch: freshell_runtime_protocol::LaunchRequest, +) -> Result { + use freshell_runtime_protocol::RuntimeErrorCode; + let mut retries = 0; + loop { + match preparation + .client + .launch(request_id.clone(), launch.clone()) + .await + { + Ok(result) => return Ok(result), + Err(error) + if error.runtime_code() == Some(RuntimeErrorCode::ProviderInputsChanged) + && launch.native_session_id.is_none() => + { + let context = launch + .terminal + .as_ref() + .and_then(|s| s.provider_launch_context.as_ref()) + .or_else(|| { + launch + .fresh_agent + .as_ref() + .and_then(|s| s.provider_launch_context.as_ref()) + }); + if let Some(capability) = context.and_then(|c| c.mcp_capability.as_ref()) { + crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id)?; + } + if retries == 2 { + tracing::warn!(event="provider.inputs.initial_replan.exhausted", provider="opencode", soul_id=%launch.soul_id, retries); + return Err("OpenCode configuration reference selection kept changing; two pre-dispatch retries exhausted".into()); + } + retries += 1; + tracing::info!(event="provider.inputs.initial_replan", provider="opencode", soul_id=%launch.soul_id, retry=retries); + let prepared = prepare_managed_opencode(OpencodePreparation { + client: preparation.client, + soul: preparation.soul, + creation_seed: preparation.creation_seed, + home: preparation.home, + cwd: preparation.cwd, + workspace: preparation.workspace, + original: preparation.original, + references: preparation.references.clone(), + auth_authority: None, + selected_sources: None, + accepted_global_root: None, + selected_resource_roots: None, + native_source_flags: None, + rebind_enabled: None, + inherited_inline: None, + inline_edits: preparation.inline_edits.clone(), + }) + .await?; + if let Some(terminal) = launch.terminal.as_mut() { + terminal.provider_secret_references = prepared.references; + terminal.provider_launch_context = Some(prepared.context); + } else if let Some(fresh) = launch.fresh_agent.as_mut() { + fresh.provider_secret_references = prepared.references; + fresh.provider_launch_context = Some(prepared.context); + } else { + return Err("initial OpenCode workload unavailable".into()); + } + // A new approved generation has a different semantic payload. + // Keep the public creation/soul identity and use a fresh control + // request identity rather than replaying a rejected digest. + request_id = freshell_runtime_protocol::RequestId::new(); + } + Err(error) => return Err(error.to_string()), + } + } } -pub(crate) async fn prepare_managed_opencode( - request: OpencodePreparation<'_>, -) -> Result { - prepare_managed_opencode_with_publisher( - request, - crate::managed_mcp_capability::issue_prepared_opencode_context, - ) - .await -} +#[cfg(test)] +mod tests { + use super::*; + use sha2::{Digest, Sha256}; -pub(crate) async fn prepare_managed_opencode_with_publisher( - request: OpencodePreparation<'_>, - publish: impl FnOnce( - &freshell_runtime_protocol::SoulId, - &Path, - &freshell_runtime_protocol::ProviderLaunchContext, - crate::managed_mcp_capability::OpencodeEphemeralInput<'_>, - ) -> Result, -) -> Result { - let OpencodePreparation { - client, - soul, - creation_seed, - home, - cwd, - workspace, - original, - mut references, - auth_authority, - selected_sources, - accepted_global_root, - selected_resource_roots, - native_source_flags, - rebind_enabled, - inherited_inline, - inline_edits, - } = request; - let rebind_enabled = rebind_enabled.unwrap_or_else(|| { - !matches!( - original("FRESHELL_OPENCODE_REBIND").as_deref(), - Some("0" | "false") - ) - }); - use sources::*; - let inherited_inline = inherited_inline - .or_else(|| original("OPENCODE_CONFIG_CONTENT")) - .filter(|raw| !raw.is_empty()); - let mut inline_edits = inline_edits.unwrap_or_default(); - inline_edits.freshell_command = Some(vec!["node".into(), "/opt/freshell-mcp/server.js".into()]); - if inline_edits.bash_permission.is_none() { - inline_edits.bash_permission = original("FRESHELL_MANAGED_OPENCODE_BASH_PERMISSION") - .filter(|value| matches!(value.as_str(), "allow" | "ask" | "deny")); + // Pinned resolver-equivalent consumer for OpenCode 1.18.21 variable.ts. + // Receipt is a digest of values consumed by its native JSONC/file substitution. + const CONSUMER: &str = r#" +const fs=require('fs'),path=require('path'),crypto=require('crypto'); +const source=process.argv[1],home=process.argv[2],env=JSON.parse(process.argv[3]); +let text=fs.readFileSync(source,'utf8').replace(/\{env:([^}]+)\}/g,(_,n)=>env[n]||''); +let out='',cursor=0; +for(const m of Array.from(text.matchAll(/\{file:[^}]+\}/g))) { + out+=text.slice(cursor,m.index); + if(text.slice(text.lastIndexOf('\n',m.index-1)+1,m.index).trimStart().startsWith('//')) out+=m[0]; + else { + const target=m[0].slice(6,-1),prefix='/home/freshell/provider/'; + if(!target.startsWith(prefix)) process.exit(71); + const bytes=fs.readFileSync(path.join(home,target.slice(prefix.length)),'utf8'); + out+=JSON.stringify(bytes.trim()).slice(1,-1); + } + cursor=m.index+m[0].length; +} +out+=text.slice(cursor); +const parsed=JSON.parse(out),receipts=Object.values(parsed).map(v=>crypto.createHash('sha256').update(v).digest('hex')); +process.stdout.write(JSON.stringify(receipts)); +"#; + + fn digest(value: &str) -> String { + format!("{:x}", Sha256::digest(value.as_bytes())) } - let config_root = if let Some(root) = accepted_global_root { - if !root.is_absolute() || root.file_name() != Some(std::ffi::OsStr::new("opencode")) { - return Err( - "accepted global configuration root is invalid; retain prior inputs and reprepare" - .into(), + fn consume(plan: &ResolvedFileReferencePlan, runtime: &Path) -> Vec { + consume_with_env(plan, runtime, &|_| None) + } + fn consume_with_env( + plan: &ResolvedFileReferencePlan, + runtime: &Path, + env: &dyn Fn(&str) -> Option, + ) -> Vec { + std::fs::create_dir_all(runtime.join(".freshell/references")).unwrap(); + for selection in &plan.selections { + let relative = selection + .runtime_destination + .strip_prefix("/home/freshell/provider/") + .unwrap(); + std::fs::copy(&selection.canonical_source, runtime.join(relative)).unwrap(); + } + let source = runtime.join("source.json"); + std::fs::write(&source, plan.template.as_bytes()).unwrap(); + let mut child = std::collections::BTreeMap::new(); + for recipe in &plan.recipes { + child.insert( + recipe.generated_name.clone(), + evaluate_generated_recipe(recipe, env).unwrap(), ); } - root - } else { - let xdg = original("XDG_CONFIG_HOME") - .filter(|value| !value.is_empty()) - .map(PathBuf::from) - .unwrap_or_else(|| home.join(".config")); - let xdg = if xdg.is_absolute() { - xdg - } else { - cwd.join(xdg) - }; - xdg.join("opencode") - }; - let xdg = config_root - .parent() - .ok_or("global configuration root parent unavailable")?; - let resource_roots = if let Some(roots) = selected_resource_roots { - roots - } else { - sources::select_opencode_resource_roots(&OpencodeDiscovery { - home, - cwd, - worktree: workspace, - config_root: &config_root, - env: original, - }) - .map_err(str::to_string)? - }; - let base = crate::managed_provider_bootstrap::provider_launch_context_from_home_at_with_xdg( - "opencode", - workspace, - cwd, - home, - Some(xdg), - None, - )? - .ok_or("OpenCode provider preparation unavailable")?; - let selected = if let Some(sources) = selected_sources { - sources - } else { - discover_opencode(OpencodeDiscovery { - home, - cwd, - worktree: workspace, - config_root: &config_root, - env: original, - }) - .map_err(str::to_string)? - }; - let mut loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; - if let Some(raw) = inherited_inline { - if raw.len() > 16 * 1024 * 1024 { - return Err("OpenCode inherited inline source exceeds 16 MiB".into()); + consume_published(runtime, &child) + } + + fn consume_published( + runtime: &Path, + child: &std::collections::BTreeMap, + ) -> Vec { + let result = std::process::Command::new("node") + .arg("-e") + .arg(CONSUMER) + .arg(runtime.join("source.json")) + .arg(runtime) + .arg(serde_json::to_string(&child).unwrap()) + .output() + .unwrap(); + assert!( + result.status.success(), + "pinned consumer failed with {:?}", + result.status.code() + ); + serde_json::from_slice(&result.stdout).unwrap() + } + + #[cfg(unix)] + #[test] + fn managed_provider_references_actual_maximum_sources_aliases_reach_child_and_overflow_preserves_projection( + ) { + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("opencode.json"); + let runtime = owned.path().join("runtime"); + let mut fields = serde_json::Map::new(); + let mut expected = Vec::new(); + for ordinal in 0..128 { + let name = format!("payload-{ordinal:03}"); + let payload = format!(" selected-reference-nonce-{ordinal:03}\nquote\"slash\\ "); + std::fs::write(owned.path().join(&name), &payload).unwrap(); + for suffix in ["a", "b"] { + let alias = format!("alias-{ordinal:03}-{suffix}"); + std::os::unix::fs::symlink(&name, owned.path().join(&alias)).unwrap(); + fields.insert( + format!("field-{ordinal:03}-{suffix}"), + serde_json::Value::String(format!("{{file:{alias}}}")), + ); + expected.push(digest(payload.trim())); + } } - let retained = sources::retained_source_view_bytes(&loaded) - .map_err(str::to_string)? - .checked_add(raw.len()) - .ok_or("source-view size overflow")?; - let remaining = (64 * 1024 * 1024usize) - .checked_sub(retained) - .ok_or("simultaneous source views exceed 64 MiB")?; - let user_owns_freshell_mcp = loaded - .iter() - .filter(|source| source.source.lane == SourceLane::Main) - .try_fold(false, |owned, source| { - freshell_platform::opencode_config::source_owns_freshell_mcp( - &source.view, - &source.source.logical_path, - cwd, - ) - .map(|user_owns| owned || user_owns) - }) - .map_err(|error| error.to_string())?; - let view = freshell_platform::opencode_config::merge_owned_entries_bounded( - Some(&raw), - &freshell_platform::opencode_config::OpencodeConfigPlan { - sources: vec![], - user_owns_freshell_mcp, - inline_prepared: false, - }, - &inline_edits, - remaining.min(64 * 1024), - ) - .map_err(|error| error.to_string())?; - let inline = LoadedSourceOccurrence { - requires_config_bind: true, - source: SourceOccurrence { - logical_path: "OPENCODE_CONFIG_CONTENT".into(), - canonical_source: "OPENCODE_CONFIG_CONTENT".into(), - lane: SourceLane::Inline, - role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, - }, - view: std::sync::Arc::new(view), + let raw = serde_json::Value::Object(fields.clone()).to_string(); + std::fs::write(&source, &raw).unwrap(); + let resolve = |raw: &str| { + resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: owned.path(), + original_home: owned.path(), + }, + &|_| None, + ) }; - let position = loaded - .iter() - .position(|source| source.source.lane == SourceLane::Tui) - .unwrap_or(loaded.len()); - loaded.insert(position, inline); - } - let native_source_flags = if let Some(flags) = native_source_flags { - flags - } else { - let mut flags = Vec::new(); - for (name, kind) in [ - ( - "OPENCODE_CONFIG", - freshell_platform::opencode_recipe::NativeSourceFlagKind::Main, - ), - ( - "OPENCODE_TUI_CONFIG", - freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui, - ), - ] { - if let Some(path) = original(name).filter(|path| !path.is_empty()) { - let path = if Path::new(&path).is_absolute() { - PathBuf::from(path) - } else { - cwd.join(path) - }; - let source_ordinal = loaded - .iter() - .position(|source| source.source.logical_path == path && source.source.role == DiscoveryRole::Workspace && match kind { - freshell_platform::opencode_recipe::NativeSourceFlagKind::Main => source.source.lane == SourceLane::Main, - freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui => source.source.lane == SourceLane::Tui, - freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory => false, - }).ok_or_else(|| format!("selected {name} source is missing or unavailable; repair it before dispatch"))?; - let destination = sources::runtime_config_destination( - home, - &config_root, - &path, - loaded[source_ordinal].source.role, - ); - flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { - kind, - source_ordinal: Some( - u16::try_from(source_ordinal) - .map_err(|_| "native source ordinal overflow")?, - ), - runtime_destination: destination - .to_str() - .ok_or("selected native source destination is not UTF-8")? - .into(), - }); - } - } - if let Some(path) = original("OPENCODE_CONFIG_DIR").filter(|path| !path.is_empty()) { - let path = if Path::new(&path).is_absolute() { - PathBuf::from(path) - } else { - cwd.join(path) - }; - if !resource_roots.iter().any(|root| root.logical_path == path) { - return Err( - "selected OPENCODE_CONFIG_DIR root is unavailable before dispatch".into(), - ); - } - flags.push(freshell_platform::opencode_recipe::NativeSourceFlag { - kind: freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory, - source_ordinal: None, - runtime_destination: sources::runtime_directory_destination( - home, - &config_root, - &path, - DiscoveryRole::Workspace, - ) - .to_str() - .ok_or("selected directory is not UTF-8")? - .into(), - }); - } - flags - }; - let mut retained_views = - sources::retained_source_view_bytes(&loaded).map_err(str::to_string)?; - let mut edited = std::collections::BTreeSet::new(); - for ordinal in 0..loaded.len() { - let source = &loaded[ordinal]; - if rebind_enabled - && source.source.lane == SourceLane::Tui - && native_source_flags.iter().any(|flag| { - flag.kind == freshell_platform::opencode_recipe::NativeSourceFlagKind::Tui - && Path::new(&flag.runtime_destination) - == sources::runtime_config_destination( - home, - &config_root, - &source.source.logical_path, - source.source.role, - ) - }) - { - let canonical = source.source.canonical_source.clone(); - if !edited.insert(canonical.clone()) { - continue; - } - let shared = loaded + let plan = resolve(&raw).unwrap(); + assert_eq!(plan.selections.len(), 128); + assert_eq!(plan.occurrences.len(), 256); + assert_eq!( + plan.occurrences .iter() - .filter(|source| source.source.canonical_source == canonical) - .count() - > 1; - let remaining = (64 * 1024 * 1024usize) - .checked_sub(retained_views) - .ok_or("simultaneous source views exceed 64 MiB")?; - let runtime_home = Path::new("/home/freshell/provider"); - let plugin = if shared { - freshell_platform::opencode_plugin::rebind_plugin_package_path(runtime_home) - } else { - freshell_platform::opencode_plugin::rebind_plugin_path(runtime_home) - }; - let content = freshell_platform::opencode_plugin::selected_tui_content_bounded( - &source.view, - &plugin, - remaining, - ) - .map_err(|_| { - format!( - "selected OpenCode TUI source {} has invalid JSONC/plugin list or exceeds the source-view profile", - source.source.logical_path.display() - ) - })?; - if content.len() > 16 * 1024 * 1024 { - return Err("selected OpenCode effective TUI source exceeds 16 MiB".into()); - } - retained_views = retained_views - .checked_add(content.len()) - .ok_or("source-view byte count overflow")?; - let view = std::sync::Arc::new(content); - for (index, source) in loaded.iter_mut().enumerate() { - if index == ordinal || shared && source.source.canonical_source == canonical { - source.view = view.clone(); - source.requires_config_bind = true; - } - } - } + .map(|occurrence| &occurrence.approved_alias) + .collect::>() + .len(), + 256 + ); + assert_eq!(consume(&plan, &runtime), expected); + let prior_source = std::fs::read(runtime.join("source.json")).unwrap(); + std::os::unix::fs::symlink("payload-000", owned.path().join("alias-overflow")).unwrap(); + let mut alias_overflow = fields.clone(); + alias_overflow.insert( + "overflow".into(), + serde_json::Value::String("{file:alias-overflow}".into()), + ); + let error = resolve(&serde_json::Value::Object(alias_overflow).to_string()) + .err() + .unwrap(); + assert_eq!(error.cause, "configuration alias count exceeds 256"); + std::fs::write(owned.path().join("source-overflow"), b"PRIVATE_OVERFLOW").unwrap(); + fields.insert( + "overflow".into(), + serde_json::Value::String("{file:source-overflow}".into()), + ); + let error = resolve(&serde_json::Value::Object(fields).to_string()) + .err() + .unwrap(); + assert_eq!(error.cause, "configuration reference count exceeds 128"); + assert!(!error.to_string().contains("PRIVATE_OVERFLOW")); + assert_eq!(std::fs::read(&source).unwrap(), raw.as_bytes()); + assert_eq!( + std::fs::read(runtime.join("source.json")).unwrap(), + prior_source + ); + assert_eq!(consume_published(&runtime, &Default::default()), expected); + assert_eq!( + std::fs::read_dir(runtime.join(".freshell/references")) + .unwrap() + .count(), + 128 + ); } - let names = source_environment_names(&loaded).map_err(|error| error.to_string())?; - let mut environment = environment::SourceEnvironment::read(names, &references, original)?; - // Even a typed auth grant needs explicit ownership of a newly absent soul - // volume before its descriptor-owned mountpoints can be constructed. - let reservation = if auth_authority.is_none() { - Some( - client - .reserve_provider_inputs(soul.clone(), "opencode".into(), creation_seed.into()) - .await - .map_err(|error| error.to_string())? - .reservation_id, + + #[test] + fn managed_provider_references_actual_maximum_file_events_reach_child_and_overflow_preserves_projection( + ) { + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("opencode.json"); + let runtime = owned.path().join("runtime"); + std::fs::write(owned.path().join("rules"), b" event-capacity-nonce ").unwrap(); + let raw = serde_json::Value::Object( + (0..8192) + .map(|ordinal| { + ( + format!("field-{ordinal:04}"), + serde_json::Value::String("{file:rules}".into()), + ) + }) + .collect(), ) - } else { - None - }; - if environment.needs_native_auth() - && loaded - .iter() - .any(|source| source.source.lane == SourceLane::Main) - { - let authority = if let Some(authority) = auth_authority { - authority.to_owned() - } else { - reservation - .clone() - .ok_or("initial provider input reservation unavailable")? + .to_string(); + std::fs::write(&source, &raw).unwrap(); + let resolve = |raw: &str| { + resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: owned.path(), + original_home: owned.path(), + }, + &|_| None, + ) }; - let values = client - .native_auth_lookup(soul.clone(), authority, environment.names().to_vec()) - .await - .map_err(|error| error.to_string())?; - environment.set_native_auth(values.0)?; + let plan = resolve(&raw).unwrap(); + assert_eq!(plan.substitution_events, 8192); + assert_eq!(plan.occurrences.len(), 8192); + assert_eq!(plan.selections.len(), 1); + let expected = vec![digest("event-capacity-nonce"); 8192]; + assert_eq!(consume(&plan, &runtime), expected); + let overflow = format!( + "{},\"overflow\":\"{{file:rules}}\"}}", + &raw[..raw.len() - 1] + ); + let error = resolve(&overflow).err().unwrap(); + assert_eq!( + error.cause, + "configuration source, expansion or event profile limit exceeded" + ); + assert_eq!(std::fs::read(&source).unwrap(), raw.as_bytes()); + assert_eq!( + std::fs::read(runtime.join("source.json")).unwrap(), + plan.template.as_bytes() + ); + assert_eq!(consume_published(&runtime, &Default::default()), expected); + assert_eq!( + std::fs::read_dir(runtime.join(".freshell/references")) + .unwrap() + .count(), + 1 + ); } - let plan = environment - .plan_sources(loaded, cwd, home) - .map_err(|error| error.to_string())?; - let frozen = environment.frozen_inputs(&plan)?; - let frozen_names = frozen - .iter() - .map(|(name, _)| name.to_owned()) - .collect::>(); - let mut files = std::collections::BTreeMap::new(); - for source in &plan { - for selection in &source.references.selections { - let id = selection - .runtime_destination - .strip_prefix("/home/freshell/provider/.freshell/references/") - .and_then(|value| value.parse::().ok()) - .ok_or("invalid prepared source index")?; - if files - .insert(id, selection.canonical_source.clone()) - .is_some_and(|prior| prior != selection.canonical_source) - { - return Err("prepared source changes one selected index".into()); - } + + #[test] + fn managed_provider_references_child_consumes_original_absolute_relative_home_spaces_and_symlink( + ) { + let temp = tempfile::tempdir().unwrap(); + let home = temp.path().join("original-home"); + let declaring = temp.path().join("project/nested"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&declaring).unwrap(); + let payload = " synthetic-nonce-quote\"slash\\\ntrim "; + for path in [ + home.join("home-ref"), + declaring.join("relative-ref"), + declaring.join("with spaces"), + temp.path().join("external-ref"), + ] { + std::fs::write(path, payload).unwrap(); } - } - for (expected, (id, path)) in files.into_iter().enumerate() { - if id as usize != expected { - return Err("prepared source indexes are not contiguous".into()); + #[cfg(unix)] + std::os::unix::fs::symlink("relative-ref", declaring.join("linked-ref")).unwrap(); + let targets = vec![ + temp.path().join("external-ref").display().to_string(), + "relative-ref".into(), + "~/home-ref".into(), + "with spaces".into(), + "linked-ref".into(), + ]; + for target in targets { + let raw = serde_json::json!({"instructions":format!("{{file:{target}}}")}).to_string(); + let original = declaring.join("opencode.json"); + std::fs::write(&original, &raw).unwrap(); + let plan = resolve_file_references( + &raw, + FileSourceOrigin { + declaring_config: &original, + cwd: &declaring, + original_home: &home, + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest(payload.trim())] + ); + assert_eq!(std::fs::read_to_string(&original).unwrap(), raw); + assert!(!plan.template.contains("synthetic-nonce")); } - references.push(freshell_runtime_protocol::ProviderSecretReference { - source_path: path.to_str().ok_or("selected source is not UTF-8")?.into(), - profile: freshell_runtime_protocol::ProviderSecretProfile::OpencodeConfigurationFile, - provider_relative_path: Some(format!(".freshell/references/{id}")), - }); } - freshell_runtime_protocol::validate_provider_secrets("opencode", &references) - .map_err(|error| error.message)?; - let context = publish( - soul, - home, - &base, - crate::managed_mcp_capability::OpencodeEphemeralInput { - cwd: Some(cwd), - workspace: Some(workspace), - config_root: Some(&config_root), - source_plan: Some(&plan), - resource_roots: Some(&resource_roots), - initial_auth_reservation: reservation.as_deref(), - secret_references: Some(&references), - frozen_inputs: Some(&frozen_names), - native_source_flags: Some(&native_source_flags), - rebind_enabled: Some(rebind_enabled), - inline_edits: Some(&inline_edits), - ..Default::default() - }, - )?; - let capability = context - .mcp_capability - .as_ref() - .ok_or("prepared source capability unavailable")?; - if !frozen.is_empty() { - if let Err(error) = client - .bind_provider_path_inputs(soul.clone(), capability.grant_id.clone(), frozen) - .await + #[test] + fn managed_provider_references_same_relative_spelling_keeps_each_declaring_origin() { + let temp = tempfile::tempdir().unwrap(); + for (index, nonce) in ["first-origin-nonce", "second-origin-nonce"] + .iter() + .enumerate() { - let _ = crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id); - return Err(error.to_string()); + let origin = temp.path().join(format!("origin-{index}")); + std::fs::create_dir_all(&origin).unwrap(); + std::fs::write(origin.join("rules"), nonce).unwrap(); + let config = origin.join("opencode.json"); + let raw = r#"{"instructions":"{file:rules}"}"#; + let plan = resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &config, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join(format!("runtime-{index}"))), + vec![digest(nonce)] + ); } } - context - .prepared_inputs - .as_ref() - .ok_or("prepared source descriptors unavailable")? - .validate_references(&references) - .map_err(|error| error.message)?; - Ok(PreparedManagedOpencode { - context, - references, - }) -} - -/// Rebuild only a positively rejected, never-dispatched initial OpenCode input -/// generation. Other failures retain their inputs for ownership reconciliation. -pub(crate) async fn launch_initial_opencode( - preparation: OpencodePreparation<'_>, - mut request_id: freshell_runtime_protocol::RequestId, - mut launch: freshell_runtime_protocol::LaunchRequest, -) -> Result { - use freshell_runtime_protocol::RuntimeErrorCode; - let mut retries = 0; - loop { - match preparation - .client - .launch(request_id.clone(), launch.clone()) - .await + #[test] + fn managed_provider_references_nested_path_env_is_consumed_without_forwarding_value() { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("private-path-ref"); + std::fs::write(&target, "nested-path-consumption-nonce").unwrap(); + let source = temp.path().join("opencode.json"); + let raw = r#"{"instructions":"{file:{env:NODE_EXTRA_CA_CERTS}}"}"#; + let plan = resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|name| (name == "NODE_EXTRA_CA_CERTS").then(|| target.display().to_string()), + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest("nested-path-consumption-nonce")] + ); + assert_eq!(plan.referenced_variables, vec!["NODE_EXTRA_CA_CERTS"]); + assert!(!plan.template.contains("private-path-ref")); + } + #[test] + fn managed_provider_references_missing_and_cyclic_targets_fail_without_echoing_candidate_values( + ) { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("opencode.json"); + #[cfg(unix)] { - Ok(result) => return Ok(result), - Err(error) - if error.runtime_code() == Some(RuntimeErrorCode::ProviderInputsChanged) - && launch.native_session_id.is_none() => - { - let context = launch - .terminal - .as_ref() - .and_then(|s| s.provider_launch_context.as_ref()) - .or_else(|| { - launch - .fresh_agent - .as_ref() - .and_then(|s| s.provider_launch_context.as_ref()) - }); - if let Some(capability) = context.and_then(|c| c.mcp_capability.as_ref()) { - crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id)?; - } - if retries == 2 { - tracing::warn!(event="provider.inputs.initial_replan.exhausted", provider="opencode", soul_id=%launch.soul_id, retries); - return Err("OpenCode configuration reference selection kept changing; two pre-dispatch retries exhausted".into()); - } - retries += 1; - tracing::info!(event="provider.inputs.initial_replan", provider="opencode", soul_id=%launch.soul_id, retry=retries); - let prepared = prepare_managed_opencode(OpencodePreparation { - client: preparation.client, - soul: preparation.soul, - creation_seed: preparation.creation_seed, - home: preparation.home, - cwd: preparation.cwd, - workspace: preparation.workspace, - original: preparation.original, - references: preparation.references.clone(), - auth_authority: None, - selected_sources: None, - accepted_global_root: None, - selected_resource_roots: None, - native_source_flags: None, - rebind_enabled: None, - inherited_inline: None, - inline_edits: preparation.inline_edits.clone(), - }) - .await?; - if let Some(terminal) = launch.terminal.as_mut() { - terminal.provider_secret_references = prepared.references; - terminal.provider_launch_context = Some(prepared.context); - } else if let Some(fresh) = launch.fresh_agent.as_mut() { - fresh.provider_secret_references = prepared.references; - fresh.provider_launch_context = Some(prepared.context); - } else { - return Err("initial OpenCode workload unavailable".into()); - } - // A new approved generation has a different semantic payload. - // Keep the public creation/soul identity and use a fresh control - // request identity rather than replaying a rejected digest. - request_id = freshell_runtime_protocol::RequestId::new(); - } - Err(error) => return Err(error.to_string()), + std::os::unix::fs::symlink("cycle-b", temp.path().join("cycle-a")).unwrap(); + std::os::unix::fs::symlink("cycle-a", temp.path().join("cycle-b")).unwrap(); } + for target in ["private-missing-candidate", "cycle-a"] { + let result = resolve_file_references( + r#"{"x":"{file:{env:PATH_INPUT}}"}"#, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| Some(temp.path().join(target).display().to_string()), + ); + let err = result + .err() + .expect("missing or cyclic reference cannot launch"); + assert_eq!(err.source_ordinal, 0); + assert_eq!(err.variables, vec!["PATH_INPUT"]); + assert!(!format!("{err:?} {err}").contains(target)); + } + } + #[test] + fn managed_provider_references_env_introduced_file_tokens_and_private_prefix_use_child_recipe() + { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("reference"); + std::fs::write(&target, " payload-nonce ").unwrap(); + let source = temp.path().join("opencode.json"); + let raw = r#"{"instructions":"{env:OPENAI_API_KEY}"}"#; + let lookup = |name: &str| { + (name == "OPENAI_API_KEY").then(|| { + format!( + "sentinel-private-prefix{{file:{}}}sentinel-private-suffix", + target.display() + ) + }) + }; + let plan = resolve_file_references( + raw, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &lookup, + ) + .unwrap(); + assert!(!plan.template.contains("sentinel-private")); + assert_eq!(plan.recipes.len(), 1); + assert!(!plan.recipes[0] + .unexpanded_source + .contains("sentinel-private")); + assert_eq!( + consume_with_env(&plan, &temp.path().join("runtime"), &lookup), + vec![digest( + "sentinel-private-prefixpayload-noncesentinel-private-suffix" + )] + ); + } + + #[test] + fn managed_provider_references_changed_grant_selection_blocks_before_child_and_keeps_prior_projection( + ) { + let temp = tempfile::tempdir().unwrap(); + let a = temp.path().join("a"); + let b = temp.path().join("b"); + std::fs::write(&a, "prior-usable-nonce").unwrap(); + std::fs::write(&b, "unapproved-new-nonce").unwrap(); + let source = temp.path().join("opencode.json"); + let initial = |name: &str| { + (name == "OPENAI_API_KEY").then(|| format!("prefix{{file:{}}}", a.display())) + }; + let plan = resolve_file_references( + r#"{"x":"{env:OPENAI_API_KEY}"}"#, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &initial, + ) + .unwrap(); + let previous = temp.path().join("runtime"); + assert_eq!( + consume_with_env(&plan, &previous, &initial), + vec![digest("prefixprior-usable-nonce")] + ); + let changed = |name: &str| { + (name == "OPENAI_API_KEY") + .then(|| format!("sentinel-private-changed{{file:{}}}", b.display())) + }; + let err = evaluate_generated_recipe(&plan.recipes[0], &changed) + .err() + .unwrap(); + assert!(err.cause.contains("selection changed")); + assert!(!format!("{err:?} {err}").contains("sentinel-private")); + assert_eq!( + std::fs::read_to_string(previous.join(".freshell/references/0")).unwrap(), + "prior-usable-nonce" + ); + let same_selection = |name: &str| { + (name == "OPENAI_API_KEY").then(|| format!("different-prefix{{file:{}}}", a.display())) + }; + assert!(evaluate_generated_recipe(&plan.recipes[0], &same_selection) + .unwrap() + .contains("different-prefix")); + } + #[test] + fn managed_provider_references_whole_document_native_wrapper_handles_outside_lexeme_expansion() + { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("reference"); + std::fs::write(&target, "document-consumption-nonce").unwrap(); + let source = temp.path().join("opencode.json"); + let lookup = |name: &str| { + (name == "OPENAI_API_KEY").then(|| { + serde_json::json!({"instructions":format!("{{file:{}}}",target.display())}) + .to_string() + }) + }; + let plan = resolve_file_references( + "{env:OPENAI_API_KEY}", + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &lookup, + ) + .unwrap(); + assert_eq!( + consume_with_env(&plan, &temp.path().join("runtime"), &lookup), + vec![digest("document-consumption-nonce")] + ); + assert_eq!(plan.recipes[0].unexpanded_source, "{env:OPENAI_API_KEY}"); + } + #[test] + fn managed_provider_references_virtual_inline_uses_instance_cwd() { + let temp = tempfile::tempdir().unwrap(); + std::fs::write(temp.path().join("rules"), "inline-instance-nonce").unwrap(); + let plan = resolve_file_references( + r#"{"x":"{file:rules}"}"#, + FileSourceOrigin { + declaring_config: Path::new("OPENCODE_CONFIG_CONTENT"), + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest("inline-instance-nonce")] + ); + } + #[test] + fn managed_provider_references_relative_parent_is_lexical_before_following_directory_symlink() { + let temp = tempfile::tempdir().unwrap(); + let logical = temp.path().join("logical"); + let actual = temp.path().join("actual/config"); + std::fs::create_dir_all(&logical).unwrap(); + std::fs::create_dir_all(&actual).unwrap(); + std::fs::write(logical.join("rules"), "native-lexical-origin-nonce").unwrap(); + std::fs::write( + temp.path().join("actual/rules"), + "wrong-physical-origin-nonce", + ) + .unwrap(); + #[cfg(unix)] + std::os::unix::fs::symlink(&actual, logical.join("linked")).unwrap(); + let source = logical.join("linked/opencode.json"); + let plan = resolve_file_references( + r#"{"x":"{file:../rules}"}"#, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .unwrap(); + assert_eq!( + consume(&plan, &temp.path().join("runtime")), + vec![digest("native-lexical-origin-nonce")] + ); } } diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index c1aa9aa50..cc5788b73 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -94,10 +94,9 @@ fn fields(raw: &str) -> Result, &'static str> { // Read only member spans: selected nested schemas never deserialize or rewrite // an entire source document, nor interpret command arrays as file selections. -fn object_members( - raw: &str, - start: usize, -) -> Result, std::ops::Range)>, &'static str> { +type ObjectMember = (std::ops::Range, std::ops::Range); + +fn object_members(raw: &str, start: usize) -> Result, &'static str> { let mut cursor = Cursor { raw, at: start }; if !matches!(cursor.next(), Some(Token::Punctuation(b'{'))) { return Ok(Vec::new()); diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 6b00fe501..ed9054b14 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -457,6 +457,7 @@ pub fn source_environment_names( Ok(names.into_iter().collect()) } +#[cfg(test)] pub fn plan_opencode_sources( sources: Vec, cwd: &Path, @@ -466,6 +467,7 @@ pub fn plan_opencode_sources( plan_loaded_opencode_sources(read_opencode_sources(sources)?, cwd, home, lookup) } +#[cfg(test)] pub fn plan_loaded_opencode_sources( sources: Vec, cwd: &Path, From 123d9dd636a214e593fe8e28a6518a6af3a6fe6c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:38:20 -0700 Subject: [PATCH 228/493] fix: require inspected evidence before declaring recovery unavailable --- .../freshell-agent-runtime/src/host_actor.rs | 108 ++- .../src/host_actor_tests.rs | 72 ++ crates/freshell-agent-runtime/src/lib.rs | 770 +++++++++++++++--- crates/freshell-runtime-protocol/src/lib.rs | 32 + crates/freshell-server/src/managed_runtime.rs | 4 + .../tests/managed_recovery_budget_tests.rs | 296 +++++++ crates/freshell-session-host/src/main.rs | 119 ++- .../providers/deterministic_fresh_agent.rs | 2 + .../src/providers/mod.rs | 2 + .../src/tests/probe_recovery_tests.rs | 219 +++++ .../src/checkpoint_tests.rs | 300 +++++++ crates/freshell-supervisor/src/checkpoints.rs | 96 ++- crates/freshell-supervisor/src/recovery.rs | 244 +++--- crates/freshell-supervisor/src/registry.rs | 160 +++- .../freshell-supervisor/src/resume_catalog.rs | 1 + .../src/shell_recovery_tests.rs | 461 +++++++++++ 16 files changed, 2523 insertions(+), 363 deletions(-) create mode 100644 crates/freshell-server/src/tests/managed_recovery_budget_tests.rs create mode 100644 crates/freshell-session-host/src/tests/probe_recovery_tests.rs create mode 100644 crates/freshell-supervisor/src/checkpoint_tests.rs create mode 100644 crates/freshell-supervisor/src/shell_recovery_tests.rs diff --git a/crates/freshell-agent-runtime/src/host_actor.rs b/crates/freshell-agent-runtime/src/host_actor.rs index 28111f58b..03a52ef8e 100644 --- a/crates/freshell-agent-runtime/src/host_actor.rs +++ b/crates/freshell-agent-runtime/src/host_actor.rs @@ -312,6 +312,8 @@ struct PersistedDecision { struct PersistedActorState { schema_version: u32, profile: FreshAgentProfile, + #[serde(default)] + initialization_state: Option, commands: BTreeMap, #[serde(default)] completed_forks: BTreeMap, @@ -327,6 +329,7 @@ impl PersistedActorState { Self { schema_version: 1, profile, + initialization_state: None, commands: BTreeMap::new(), completed_forks: BTreeMap::new(), decisions: BTreeMap::new(), @@ -414,58 +417,74 @@ impl FreshAgentHostActor { fs::create_dir_all(&state_dir).map_err(persist)?; let writer_lock = WriterLock::acquire(&state_dir.join("fresh-agent-writer.lock"))?; let state_path = state_dir.join("fresh-agent-state.json"); - let mut state = if state_path.exists() { - let bytes = fs::read(&state_path).map_err(persist)?; - let mut loaded: PersistedActorState = - serde_json::from_slice(&bytes).map_err(persist)?; - let mut expected_profile = profile.clone(); - if expected_profile.native_session_id.is_none() { - expected_profile.native_session_id = loaded.profile.native_session_id.clone(); - } - // The host validates a fresh preparation for each incarnation. - // Its grant and resource generation may change while the durable - // conversation and every other launch setting remain the same. - let current_preparation = expected_profile.provider_launch_context.take(); - let current_references = expected_profile.provider_secret_references.clone(); - if current_preparation.is_some() { - // Configuration-file references belong to the host-validated - // preparation. Legacy OneCLI grants remain durable settings. - if !current_references - .iter() - .filter(|reference| !reference.profile.is_configuration_file()) - .eq(loaded - .profile - .provider_secret_references + let (mut state, newly_initialized) = match fs::read(&state_path) { + Ok(bytes) => { + let mut loaded: PersistedActorState = + serde_json::from_slice(&bytes).map_err(persist)?; + // A successful transport start may still have returned no + // identity. Reopening that durable actor must not create a + // second conversation from missing metadata alone. + if loaded.profile.native_session_id.is_none() { + return Err(ActorError::AmbiguousDispatch); + } + let mut expected_profile = profile.clone(); + if expected_profile.native_session_id.is_none() { + expected_profile.native_session_id = loaded.profile.native_session_id.clone(); + } + // The host validates a fresh preparation for each incarnation. + // Its grant and resource generation may change while the durable + // conversation and every other launch setting remain the same. + let current_preparation = expected_profile.provider_launch_context.take(); + let current_references = expected_profile.provider_secret_references.clone(); + if current_preparation.is_some() { + // Configuration-file references belong to the host-validated + // preparation. Legacy OneCLI grants remain durable settings. + if !current_references .iter() - .filter(|reference| !reference.profile.is_configuration_file())) - { + .filter(|reference| !reference.profile.is_configuration_file()) + .eq(loaded + .profile + .provider_secret_references + .iter() + .filter(|reference| !reference.profile.is_configuration_file())) + { + return Err(ActorError::NativeIdentityMismatch); + } + expected_profile.provider_secret_references = + loaded.profile.provider_secret_references.clone(); + expected_profile.provider_launch_context = + loaded.profile.provider_launch_context.clone(); + } + if loaded.profile != expected_profile { return Err(ActorError::NativeIdentityMismatch); } - expected_profile.provider_secret_references = - loaded.profile.provider_secret_references.clone(); - expected_profile.provider_launch_context = - loaded.profile.provider_launch_context.clone(); - } - if loaded.profile != expected_profile { - return Err(ActorError::NativeIdentityMismatch); - } - loaded.profile.provider_launch_context = current_preparation; - loaded.profile.provider_secret_references = current_references; - for command in loaded.commands.values_mut() { - if matches!(command.state, CommandState::Dispatching) { - command.state = CommandState::Ambiguous; + loaded.profile.provider_launch_context = current_preparation; + loaded.profile.provider_secret_references = current_references; + for command in loaded.commands.values_mut() { + if matches!(command.state, CommandState::Dispatching) { + command.state = CommandState::Ambiguous; + } } - } - for decision in loaded.decisions.values_mut() { - if matches!(decision.state, DecisionState::Dispatching) { - decision.state = DecisionState::Ambiguous; + for decision in loaded.decisions.values_mut() { + if matches!(decision.state, DecisionState::Dispatching) { + decision.state = DecisionState::Ambiguous; + } } + (loaded, false) } - loaded - } else { - PersistedActorState::new(profile) + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + (PersistedActorState::new(profile), true) + } + Err(error) => return Err(persist(error)), }; + if newly_initialized { + // Publish intent before the first provider creation. A resumed + // actor keeps its accepted preparation until exact identity is + // verified below. + state.initialization_state = Some(CommandState::Dispatching); + write_state(&state_dir, &state)?; + } let started = transport .start(&state.profile) .await @@ -483,6 +502,7 @@ impl FreshAgentHostActor { event_count_cap, )?; } + state.initialization_state = Some(CommandState::Completed); write_state(&state_dir, &state)?; let actor = Arc::new(Self { state_dir, diff --git a/crates/freshell-agent-runtime/src/host_actor_tests.rs b/crates/freshell-agent-runtime/src/host_actor_tests.rs index a7cd07c7d..14069eae4 100644 --- a/crates/freshell-agent-runtime/src/host_actor_tests.rs +++ b/crates/freshell-agent-runtime/src/host_actor_tests.rs @@ -1,6 +1,78 @@ use super::*; use std::sync::atomic::{AtomicUsize, Ordering}; +struct StartupAcknowledgementGap { + directory: PathBuf, + starts: AtomicUsize, + intent_seen: std::sync::atomic::AtomicBool, +} +#[async_trait] +impl FreshAgentTransport for StartupAcknowledgementGap { + async fn start(&self, _: &FreshAgentProfile) -> Result { + self.starts.fetch_add(1, Ordering::SeqCst); + let intent = self.directory.join("fresh-agent-state.json"); + self.intent_seen.store(intent.is_file(), Ordering::SeqCst); + fs::write( + self.directory.join("provider-conversation"), + b"native-created-before-ack", + ) + .unwrap(); + Err("native identity acknowledgement lost".into()) + } + async fn dispatch( + &self, + _: &RequestId, + _: &str, + _: &FreshAgentProfile, + ) -> Result { + panic!("no prompt dispatch") + } + async fn resolve_permission(&self, _: &str, _: Value) -> Result<(), DispatchFailure> { + panic!("no decision") + } + async fn interrupt(&self) -> Result<(), String> { + Ok(()) + } + async fn stop(self: Arc) -> Result<(), String> { + Ok(()) + } + fn take_event_stream(&self) -> Option> { + None + } +} +#[tokio::test] +async fn recovery_initialization_identity_ack_loss_never_creates_a_second_conversation() { + let dir = tempfile::tempdir().unwrap(); + let transport = Arc::new(StartupAcknowledgementGap { + directory: dir.path().into(), + starts: AtomicUsize::new(0), + intent_seen: std::sync::atomic::AtomicBool::new(false), + }); + assert!(FreshAgentHostActor::open( + dir.path(), + profile(FreshProvider::Opencode, "init-gap", None), + transport.clone() + ) + .await + .is_err()); + assert!( + transport.intent_seen.load(Ordering::SeqCst), + "initialization intent must be durable before provider creation" + ); + let result = FreshAgentHostActor::open( + dir.path(), + profile(FreshProvider::Opencode, "init-gap", None), + transport.clone(), + ) + .await; + assert!(matches!(result, Err(ActorError::AmbiguousDispatch))); + assert_eq!(transport.starts.load(Ordering::SeqCst), 1); + assert_eq!( + fs::read(dir.path().join("provider-conversation")).unwrap(), + b"native-created-before-ack" + ); +} + struct SnapshotTransport { value: Value, live: std::sync::atomic::AtomicBool, diff --git a/crates/freshell-agent-runtime/src/lib.rs b/crates/freshell-agent-runtime/src/lib.rs index 7633e4743..cd527d51f 100644 --- a/crates/freshell-agent-runtime/src/lib.rs +++ b/crates/freshell-agent-runtime/src/lib.rs @@ -7,8 +7,8 @@ use freshell_runtime_protocol::{ AllocationState, DurablePosition, EvidenceStoreState, IdentityProvenance, ProviderSessionRef, - RecoveryBlockReason, RecoveryPath, ResumeSpec, RetryHint, RuntimeError, RuntimeErrorCode, - TerminalLaunchSpec, RESUME_SPEC_SCHEMA_VERSION, + RecoveryBlockReason, RecoveryEvidenceVerdict, RecoveryPath, RecoveryPathEvidence, ResumeSpec, + RetryHint, RuntimeError, RuntimeErrorCode, TerminalLaunchSpec, RESUME_SPEC_SCHEMA_VERSION, }; use rusqlite::{Connection, OpenFlags}; use serde::{Deserialize, Serialize}; @@ -261,6 +261,8 @@ pub enum ProviderStoreProbe { reason: String, evidence: Vec, store_state: EvidenceStoreState, + #[serde(default)] + negative_paths: Vec, }, Blocked { reason: RecoveryBlockReason, @@ -598,31 +600,58 @@ fn strip_value_flags(args: &[String], flags: &[&str]) -> Vec { } pub fn probe_provider_store(provider_home: &Path, spec: &ResumeSpec) -> ProviderStoreProbe { - if !provider_home.is_dir() { + match fs::metadata(provider_home) { + Ok(metadata) if metadata.is_dir() => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return blocked( + RecoveryBlockReason::StoreMissing, + "soul provider-state mount is unavailable", + vec![provider_home.display().to_string()], + Some("restore the soul provider-state volume"), + ) + } + _ => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "soul provider-state mount cannot be inspected", + vec![provider_home.display().to_string()], + Some("repair the soul provider-state volume"), + ) + } + } + let expected = spec.provider_session.native_session_id.as_str(); + if expected.is_empty() { return blocked( - RecoveryBlockReason::StoreMissing, - "soul provider-state mount is unavailable", + RecoveryBlockReason::AmbiguousIdentity, + "native identity is unrecorded", vec![provider_home.display().to_string()], - Some("restore the soul provider-state volume"), + None, ); } - let expected = spec.provider_session.native_session_id.as_str(); match spec.provider_session.provider.as_str() { "phase1-fixture" | "native-session-fixture" => { probe_native_fixture(provider_home, expected) } "claude" => probe_bounded_tree( - &provider_home.join(".claude/projects"), + &spec + .environment + .get("CLAUDE_CONFIG_DIR") + .map(std::path::PathBuf::from) + .unwrap_or_else(|| provider_home.join(".claude")) + .join("projects"), expected, - ArtifactMatch::FileNameOrJsonText, ), "codex" => probe_bounded_tree( - &provider_home.join(".codex/sessions"), + &spec + .environment + .get("CODEX_HOME") + .map(std::path::PathBuf::from) + .unwrap_or_else(|| provider_home.join(".codex")) + .join("sessions"), expected, - ArtifactMatch::FileNameOrJsonText, ), "amplifier" => probe_amplifier(provider_home, expected), - "opencode" => probe_opencode(provider_home, expected), + "opencode" => probe_opencode(provider_home, expected, spec), other => blocked( RecoveryBlockReason::ImplementationUnavailable, &format!("no managed recovery probe for provider {other}"), @@ -634,6 +663,23 @@ pub fn probe_provider_store(provider_home: &Path, spec: &ResumeSpec) -> Provider fn probe_native_fixture(provider_home: &Path, expected: &str) -> ProviderStoreProbe { let path = provider_home.join("native-session-state.json"); + match fs::symlink_metadata(&path) { + Ok(metadata) + if metadata.is_file() + && !metadata.file_type().is_symlink() + && metadata.len() <= 1024 * 1024 => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return probe_native_fixture_checkpoint(provider_home, expected, &path) + } + _ => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "native fixture state cannot be completely inspected", + vec![path.display().to_string()], + None, + ) + } + } let bytes = match fs::read(&path) { Ok(bytes) => bytes, Err(error) if error.kind() == std::io::ErrorKind::NotFound => { @@ -693,18 +739,25 @@ fn probe_native_fixture_checkpoint( primary_path: &Path, ) -> ProviderStoreProbe { let directory = provider_home.join(".freshell/checkpoints/native-session"); - if !directory.exists() { - return ProviderStoreProbe::DefinitivelyUnavailable { - reason: "native fixture state and checkpoints are absent".into(), - evidence: vec![ - primary_path.display().to_string(), - directory.display().to_string(), - ], - store_state: EvidenceStoreState::Missing, - }; + match fs::symlink_metadata(&directory) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return native_fixture_absence(primary_path, &directory, EvidenceStoreState::Missing) + } + _ => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "native fixture checkpoint directory cannot be inspected", + vec![directory.display().to_string()], + None, + ) + } } let entries = match fs::read_dir(&directory) { Ok(entries) => entries, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return native_fixture_absence(primary_path, &directory, EvidenceStoreState::Missing); + } Err(error) => { return blocked( RecoveryBlockReason::StoreUnreadable, @@ -715,7 +768,15 @@ fn probe_native_fixture_checkpoint( } }; let mut candidates = Vec::new(); - for entry in entries { + for (index, entry) in entries.enumerate() { + if index >= 50_000 { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "native fixture checkpoint directory exceeds the inspection bound", + vec![directory.display().to_string()], + None, + ); + } let entry = match entry { Ok(entry) => entry, Err(error) => { @@ -747,27 +808,32 @@ fn probe_native_fixture_checkpoint( Some("replace checkpoint symlinks with private regular files"), ); } - if !metadata.is_file() || metadata.len() > 1024 * 1024 { - continue; - } let revision = path .file_stem() .and_then(|value| value.to_str()) .and_then(|value| value.parse::().ok()); + if revision.is_some() && (!metadata.is_file() || metadata.len() > 1024 * 1024) { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "native fixture checkpoint cannot be completely inspected", + vec![path.display().to_string()], + None, + ); + } + if !metadata.is_file() { + continue; + } if let Some(revision) = revision { candidates.push((revision, path)); } } candidates.sort_by_key(|(revision, _)| *revision); let Some((revision, path)) = candidates.pop() else { - return ProviderStoreProbe::DefinitivelyUnavailable { - reason: "native fixture state is absent and no numeric checkpoint exists".into(), - evidence: vec![ - primary_path.display().to_string(), - directory.display().to_string(), - ], - store_state: EvidenceStoreState::Missing, - }; + return native_fixture_absence( + primary_path, + &directory, + EvidenceStoreState::PresentReadable, + ); }; let bytes = match fs::read(&path) { Ok(bytes) => bytes, @@ -821,14 +887,133 @@ fn probe_native_fixture_checkpoint( } } -fn probe_opencode(provider_home: &Path, expected: &str) -> ProviderStoreProbe { - let path = provider_home.join(".local/share/opencode/opencode.db"); - if !path.exists() { - return ProviderStoreProbe::DefinitivelyUnavailable { - reason: "OpenCode session database is absent".into(), - evidence: vec![path.display().to_string()], - store_state: EvidenceStoreState::Missing, - }; +fn inspected_absence( + reason: String, + evidence: Vec, + store_state: EvidenceStoreState, + paths: &[RecoveryPath], +) -> ProviderStoreProbe { + let negative_paths = paths + .iter() + .map(|path| RecoveryPathEvidence { + path: *path, + verdict: RecoveryEvidenceVerdict::DefinitiveNegative, + reason_code: "INSPECTED_PROVIDER_STORE_ABSENCE".into(), + evidence_refs: evidence.clone(), + store_state, + }) + .collect(); + ProviderStoreProbe::DefinitivelyUnavailable { + reason, + evidence, + store_state, + negative_paths, + } +} + +fn native_fixture_absence( + primary: &Path, + checkpoints: &Path, + checkpoint_state: EvidenceStoreState, +) -> ProviderStoreProbe { + let mut result = inspected_absence( + "native fixture state and numeric checkpoints are absent".into(), + vec![ + primary.display().to_string(), + checkpoints.display().to_string(), + ], + EvidenceStoreState::Missing, + &[RecoveryPath::NativeResume, RecoveryPath::CheckpointRestore], + ); + if let ProviderStoreProbe::DefinitivelyUnavailable { negative_paths, .. } = &mut result { + negative_paths[0].evidence_refs = vec![primary.display().to_string()]; + negative_paths[1].evidence_refs = vec![checkpoints.display().to_string()]; + negative_paths[1].store_state = checkpoint_state; + } + result +} + +fn probe_opencode(provider_home: &Path, expected: &str, spec: &ResumeSpec) -> ProviderStoreProbe { + // OpenCode v1.18's relative OPENCODE_DB is relative to its XDG data + // directory. Channel is compiled into the executable, not an environment + // override; an explicit DB or disabled channel DB establishes its authority. + let data = spec + .environment + .get("XDG_DATA_HOME") + .filter(|value| !value.is_empty()) + .map(std::path::PathBuf::from) + .unwrap_or_else(|| provider_home.join(".local/share")) + .join("opencode"); + let explicit_db = spec + .environment + .get("OPENCODE_DB") + .filter(|value| !value.is_empty()); + let channel_disabled = spec + .environment + .get("OPENCODE_DISABLE_CHANNEL_DB") + .is_some_and(|value| value == "1" || value == "true"); + if explicit_db.is_some_and(|value| value == ":memory:") { + return blocked( + RecoveryBlockReason::AmbiguousIdentity, + "OpenCode in-memory database is unavailable for durable inspection", + vec!["database=:memory:".into()], + None, + ); + } + let path = explicit_db + .map(|value| { + let path = std::path::PathBuf::from(value); + if path.is_absolute() { + path + } else { + data.join(path) + } + }) + .unwrap_or_else(|| data.join("opencode.db")); + let path = if path.is_absolute() { + path + } else if Path::new(&spec.cwd).is_absolute() { + Path::new(&spec.cwd).join(path) + } else { + return blocked( + RecoveryBlockReason::AmbiguousIdentity, + "OpenCode relative database root requires a recorded absolute provider cwd", + vec!["providerCwd=unestablished".into()], + None, + ); + }; + let authoritative = explicit_db.is_some() || channel_disabled; + if !authoritative + && spec + .provider_version + .as_deref() + .is_some_and(|version| version.contains('-')) + { + return blocked( + RecoveryBlockReason::AmbiguousIdentity, + "OpenCode channel database is unrecorded", + vec![data.display().to_string()], + None, + ); + } + match fs::metadata(&path) { + Ok(metadata) if metadata.is_file() => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound && authoritative => { + return inspected_absence( + "OpenCode effective session database is absent".into(), + vec![path.display().to_string()], + EvidenceStoreState::Missing, + &[RecoveryPath::NativeResume, RecoveryPath::CheckpointRestore], + ); + } + _ => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "OpenCode effective database cannot be established and inspected", + vec![path.display().to_string()], + None, + ) + } } let connection = match Connection::open_with_flags( &path, @@ -855,11 +1040,21 @@ fn probe_opencode(provider_home: &Path, expected: &str) -> ProviderStoreProbe { Ok(true) => ProviderStoreProbe::Ready { evidence: vec![path.display().to_string(), format!("session.id={expected}")], }, - Ok(false) => ProviderStoreProbe::DefinitivelyUnavailable { - reason: format!("OpenCode database has no exact session row {expected}"), - evidence: vec![path.display().to_string()], - store_state: EvidenceStoreState::PresentReadable, - }, + Ok(false) if authoritative => inspected_absence( + format!("OpenCode effective database has no exact session row {expected}"), + vec![ + path.display().to_string(), + format!("exactSessionQuery={expected}"), + ], + EvidenceStoreState::PresentReadable, + &[RecoveryPath::NativeResume, RecoveryPath::CheckpointRestore], + ), + Ok(false) => blocked( + RecoveryBlockReason::AmbiguousIdentity, + "OpenCode effective channel database is unrecorded; default no-match is not absence", + vec![path.display().to_string()], + None, + ), Err(error) => blocked( RecoveryBlockReason::UnsupportedProtocol, &format!("query OpenCode session database: {error}"), @@ -871,13 +1066,9 @@ fn probe_opencode(provider_home: &Path, expected: &str) -> ProviderStoreProbe { fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { let root = provider_home.join(".amplifier/projects"); - if !root.exists() { - return ProviderStoreProbe::DefinitivelyUnavailable { - reason: "Amplifier project store is absent".into(), - evidence: vec![root.display().to_string()], - store_state: EvidenceStoreState::Missing, - }; - } + // This tree alone does not establish all effective Amplifier stores. + // Exact readable positives are usable; an unrecorded store is unknown. + let mut incomplete = false; let mut queue = VecDeque::from([(root.clone(), 0_u8)]); let mut matches = Vec::new(); let mut visited = 0_usize; @@ -902,7 +1093,11 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { ) } }; - if metadata.file_type().is_symlink() || !metadata.is_dir() { + if metadata.file_type().is_symlink() { + incomplete = true; + continue; + } + if !metadata.is_dir() { continue; } if path.file_name().is_some_and(|name| name == expected) { @@ -910,6 +1105,7 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { continue; } if depth >= 8 { + incomplete = true; continue; } let entries = match fs::read_dir(&path) { @@ -938,23 +1134,146 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { } } match matches.as_slice() { - [] => ProviderStoreProbe::DefinitivelyUnavailable { - reason: format!("Amplifier store has no exact session {expected}"), - evidence: vec![root.display().to_string()], - store_state: EvidenceStoreState::PresentReadable, - }, - [session_dir] if freshell_sessions::amplifier_stub::stub_is_unused(session_dir) => { - ProviderStoreProbe::DefinitivelyUnavailable { - reason: format!( - "Amplifier session {expected} is a pristine launcher stub, not a durable conversation" - ), - evidence: vec![session_dir.display().to_string(), "turnCount=0".into()], - store_state: EvidenceStoreState::PresentReadable, + [] => blocked( + RecoveryBlockReason::AmbiguousIdentity, + "Amplifier native/checkpoint store authority is incomplete", + vec![root.display().to_string()], + None, + ), + [session_dir] => { + let entries = match fs::read_dir(session_dir) { + Ok(entries) => entries, + Err(error) => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + &format!("inspect Amplifier session artifacts: {error}"), + vec![session_dir.display().to_string()], + None, + ) + } + }; + let mut materialized_backup = false; + let mut identity_recorded = false; + for entry in entries { + let entry = match entry { + Ok(entry) => entry, + Err(error) => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + &format!("inspect Amplifier session entry: {error}"), + vec![session_dir.display().to_string()], + None, + ) + } + }; + let path = entry.path(); + let name = entry.file_name(); + let name = name.to_string_lossy(); + if !(name.starts_with("transcript.jsonl") + || name.starts_with("metadata.json") + || name == "events.jsonl") + { + continue; + } + let metadata = match fs::symlink_metadata(&path) { + Ok(metadata) + if metadata.is_file() + && !metadata.file_type().is_symlink() + && metadata.len() <= 1024 * 1024 => + { + metadata + } + _ => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "Amplifier session artifact cannot be completely inspected", + vec![path.display().to_string()], + None, + ) + } + }; + let bytes = match fs::read(&path) { + Ok(bytes) => bytes, + Err(error) => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + &format!("read Amplifier session artifact: {error}"), + vec![path.display().to_string()], + None, + ) + } + }; + if name != "transcript.jsonl" + && name.starts_with("transcript.jsonl") + && metadata.len() > 0 + { + materialized_backup = true; + } + if name.starts_with("metadata.json") { + match serde_json::from_slice::(&bytes) { + Ok(value) => { + if let Some(actual) = + value.get("session_id").and_then(serde_json::Value::as_str) + { + if actual != expected { + return blocked( + RecoveryBlockReason::WrongNativeIdentity, + "Amplifier metadata belongs to another native identity", + vec![path.display().to_string()], + None, + ); + } + identity_recorded = true; + } + if name != "metadata.json" && value.get("turn_count").is_some() { + materialized_backup = true; + } + } + Err(_) => { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "Amplifier session metadata is malformed", + vec![path.display().to_string()], + None, + ) + } + } + } + } + if !identity_recorded { + return blocked( + RecoveryBlockReason::AmbiguousIdentity, + "Amplifier native identity is unrecorded in readable metadata", + vec![session_dir.display().to_string()], + None, + ); + } + if !materialized_backup + && freshell_sessions::amplifier_stub::stub_is_unused(session_dir) + { + if incomplete { + return blocked( + RecoveryBlockReason::AmbiguousIdentity, + "Amplifier store scan is incomplete", + vec![root.display().to_string()], + None, + ); + } + inspected_absence( + format!("Amplifier session {expected} is an inspected pristine launcher stub"), + vec![session_dir.display().to_string(), "turnCount=0".into()], + EvidenceStoreState::PresentReadable, + &[RecoveryPath::NativeResume], + ) + } else { + ProviderStoreProbe::Ready { + evidence: vec![ + session_dir.display().to_string(), + format!("sessionId={expected}"), + ], + } } } - [session_dir] => ProviderStoreProbe::Ready { - evidence: vec![session_dir.display().to_string(), format!("sessionId={expected}")], - }, many => blocked( RecoveryBlockReason::AmbiguousIdentity, &format!( @@ -967,19 +1286,7 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { } } -#[derive(Debug, Clone, Copy)] -enum ArtifactMatch { - FileNameOrJsonText, -} - -fn probe_bounded_tree(root: &Path, expected: &str, mode: ArtifactMatch) -> ProviderStoreProbe { - if !root.exists() { - return ProviderStoreProbe::DefinitivelyUnavailable { - reason: "provider session root is absent".into(), - evidence: vec![root.display().to_string()], - store_state: EvidenceStoreState::Missing, - }; - } +fn probe_bounded_tree(root: &Path, expected: &str) -> ProviderStoreProbe { let mut queue = VecDeque::from([(root.to_path_buf(), 0_u8)]); let mut visited = 0_usize; while let Some((path, depth)) = queue.pop_front() { @@ -994,14 +1301,9 @@ fn probe_bounded_tree(root: &Path, expected: &str, mode: ArtifactMatch) -> Provi } let metadata = match fs::symlink_metadata(&path) { Ok(metadata) => metadata, - Err(error) => { - return blocked( - RecoveryBlockReason::StoreUnreadable, - &format!("inspect provider artifact {}: {error}", path.display()), - vec![root.display().to_string()], - Some("repair provider-state volume permissions"), - ) - } + // Keep searching for an exact readable positive. No negative is + // emitted from this partial tree, including skipped candidates. + Err(_) => continue, }; if metadata.file_type().is_symlink() { continue; @@ -1012,56 +1314,38 @@ fn probe_bounded_tree(root: &Path, expected: &str, mode: ArtifactMatch) -> Provi } let entries = match fs::read_dir(&path) { Ok(entries) => entries, - Err(error) => { - return blocked( - RecoveryBlockReason::StoreUnreadable, - &format!("read provider directory {}: {error}", path.display()), - vec![root.display().to_string()], - Some("repair provider-state volume permissions"), - ) - } + Err(_) => continue, }; for entry in entries { match entry { Ok(entry) => queue.push_back((entry.path(), depth + 1)), - Err(error) => { - return blocked( - RecoveryBlockReason::StoreUnreadable, - &format!("read provider directory entry: {error}"), - vec![root.display().to_string()], - Some("repair provider-state volume permissions"), - ) - } + Err(_) => continue, } } continue; } - if matches!(mode, ArtifactMatch::FileNameOrJsonText) { - let filename_match = path - .file_name() - .and_then(|name| name.to_str()) - .is_some_and(|name| name.contains(expected)); - let content_match = if filename_match || metadata.len() > 1024 * 1024 { - false - } else { - fs::read(&path).ok().is_some_and(|bytes| { - bytes - .windows(expected.len()) - .any(|w| w == expected.as_bytes()) - }) + if !metadata.is_file() || metadata.len() > 1024 * 1024 { + continue; + } + let Ok(bytes) = fs::read(&path) else { + continue; + }; + let filename_match = path + .file_name() + .and_then(|name| name.to_str()) + .is_some_and(|name| name.contains(expected)); + let content_match = bytes + .windows(expected.len()) + .any(|w| w == expected.as_bytes()); + if filename_match || content_match { + return ProviderStoreProbe::Ready { + evidence: vec![path.display().to_string()], }; - if filename_match || content_match { - return ProviderStoreProbe::Ready { - evidence: vec![path.display().to_string()], - }; - } } } - ProviderStoreProbe::DefinitivelyUnavailable { - reason: format!("no exact provider artifact for {expected}"), - evidence: vec![root.display().to_string()], - store_state: EvidenceStoreState::PresentReadable, - } + blocked(RecoveryBlockReason::AmbiguousIdentity, + "complete effective provider native/checkpoint stores are unrecorded; tree no-match is not absence", + vec![root.display().to_string()], None) } fn blocked( @@ -1314,6 +1598,234 @@ mod tests { )); } + #[test] + fn native_fixture_probe_oversized_or_unreadable_checkpoint_cannot_certify_absence() { + let home = tempfile::tempdir().unwrap(); + let dir = home.path().join(".freshell/checkpoints/native-session"); + fs::create_dir_all(&dir).unwrap(); + let file = fs::File::create(dir.join("7.json")).unwrap(); + file.set_len(1024 * 1024 + 1).unwrap(); + assert!(matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test("native-session-fixture", "exact", home.path()) + ), + ProviderStoreProbe::Blocked { .. } + )); + } + + #[test] + fn recovery_store_opencode_relative_data_root_uses_recorded_provider_cwd() { + assert_opencode_data_root("relative-data", true); + } + + #[test] + fn recovery_store_opencode_empty_data_root_uses_provider_home() { + assert_opencode_data_root("", false); + } + + fn assert_opencode_data_root(data_root: &str, relative: bool) { + let home = tempfile::tempdir().unwrap(); + let cwd = home.path().join("provider-workspace"); + fs::create_dir(&cwd).unwrap(); + let mut spec = resume_spec_for_test("opencode", "exact", home.path()); + spec.cwd = cwd.display().to_string(); + spec.environment + .insert("XDG_DATA_HOME".into(), data_root.into()); + spec.environment + .insert("OPENCODE_DB".into(), "selected.db".into()); + let data = if relative { + cwd.join(data_root) + } else { + home.path().join(".local/share") + }; + let path = data.join("opencode/selected.db"); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + let db = Connection::open(&path).unwrap(); + db.execute("CREATE TABLE session(id TEXT PRIMARY KEY)", []) + .unwrap(); + db.execute("INSERT INTO session(id) VALUES ('exact')", []) + .unwrap(); + assert!( + matches!( + probe_provider_store(home.path(), &spec), + ProviderStoreProbe::Ready { .. } + ), + "data root {data_root:?}" + ); + db.execute("DELETE FROM session WHERE id='exact'", []) + .unwrap(); + let result = probe_provider_store(home.path(), &spec); + let ProviderStoreProbe::DefinitivelyUnavailable { + negative_paths, + store_state, + .. + } = result + else { + panic!("{result:?}"); + }; + assert_eq!(store_state, EvidenceStoreState::PresentReadable); + assert_eq!(negative_paths.len(), 2); + assert!(negative_paths.iter().all(|row| row + .evidence_refs + .iter() + .any(|reference| reference == &path.display().to_string()))); + } + + #[test] + fn recovery_store_opencode_unresolved_relative_data_root_blocks() { + let home = tempfile::tempdir().unwrap(); + let mut spec = resume_spec_for_test("opencode", "exact", home.path()); + spec.cwd = "unrecorded-relative-cwd".into(); + spec.environment + .insert("XDG_DATA_HOME".into(), "unrecorded-relative-data".into()); + spec.environment + .insert("OPENCODE_DB".into(), "selected.db".into()); + assert!( + matches!( + probe_provider_store(home.path(), &spec), + ProviderStoreProbe::Blocked { .. } + ), + "unresolved provider cwd must not authorize probe-cwd absence" + ); + let path = home.path().join("absolute.db"); + let db = Connection::open(&path).unwrap(); + db.execute("CREATE TABLE session(id TEXT PRIMARY KEY)", []) + .unwrap(); + db.execute("INSERT INTO session(id) VALUES ('exact')", []) + .unwrap(); + spec.environment + .insert("OPENCODE_DB".into(), path.display().to_string()); + assert!( + matches!( + probe_provider_store(home.path(), &spec), + ProviderStoreProbe::Ready { .. } + ), + "explicit absolute database remains authoritative" + ); + } + + #[test] + fn recovery_store_opencode_resolves_data_and_database_overrides_before_inspection() { + let home = tempfile::tempdir().unwrap(); + for route in ["xdg", "absolute", "relative", "channel-disabled"] { + let mut spec = resume_spec_for_test("opencode", "exact", home.path()); + let path = match route { + "xdg" => { + spec.environment.insert( + "XDG_DATA_HOME".into(), + home.path().join("selected-data").display().to_string(), + ); + home.path().join("selected-data/opencode/opencode.db") + } + "absolute" => { + let path = home.path().join("selected.db"); + spec.environment + .insert("OPENCODE_DB".into(), path.display().to_string()); + path + } + "relative" => { + spec.environment + .insert("OPENCODE_DB".into(), "relative.db".into()); + home.path().join(".local/share/opencode/relative.db") + } + "channel-disabled" => { + spec.environment + .insert("OPENCODE_DISABLE_CHANNEL_DB".into(), "1".into()); + home.path().join(".local/share/opencode/opencode.db") + } + _ => unreachable!(), + }; + fs::create_dir_all(path.parent().unwrap()).unwrap(); + let db = Connection::open(path).unwrap(); + db.execute("CREATE TABLE session(id TEXT PRIMARY KEY)", []) + .unwrap(); + db.execute("INSERT INTO session(id) VALUES ('exact')", []) + .unwrap(); + assert!( + matches!( + probe_provider_store(home.path(), &spec), + ProviderStoreProbe::Ready { .. } + ), + "{route}" + ); + } + let mut memory = resume_spec_for_test("opencode", "exact", home.path()); + memory + .environment + .insert("OPENCODE_DB".into(), ":memory:".into()); + assert!(matches!( + probe_provider_store(home.path(), &memory), + ProviderStoreProbe::Blocked { .. } + )); + let mut unknown_channel = resume_spec_for_test("opencode", "absent", home.path()); + unknown_channel.provider_version = Some("0.0.0-preview".into()); + assert!( + matches!( + probe_provider_store(home.path(), &unknown_channel), + ProviderStoreProbe::Blocked { .. } + ), + "unrecorded installed channel cannot certify a default-database negative" + ); + } + + #[test] + fn recovery_store_incomplete_scan_blocks_but_exact_positive_survives() { + let home = tempfile::tempdir().unwrap(); + for provider in ["claude", "codex"] { + let root = home.path().join(if provider == "claude" { + ".claude/projects" + } else { + ".codex/sessions" + }); + fs::create_dir_all(&root).unwrap(); + let large = fs::File::create(root.join("oversized.jsonl")).unwrap(); + large.set_len(1024 * 1024 + 1).unwrap(); + let spec = resume_spec_for_test(provider, "exact", home.path()); + assert!( + matches!( + probe_provider_store(home.path(), &spec), + ProviderStoreProbe::Blocked { .. } + ), + "{provider}" + ); + fs::write(root.join("exact.jsonl"), br#"{"sessionId":"exact"}"#).unwrap(); + assert!( + matches!( + probe_provider_store(home.path(), &spec), + ProviderStoreProbe::Ready { .. } + ), + "{provider}" + ); + } + } + + #[test] + fn recovery_store_amplifier_backup_transcript_preserves_materialized_session() { + let home = tempfile::tempdir().unwrap(); + let cwd = home.path().join("workspace"); + fs::create_dir(&cwd).unwrap(); + let session = freshell_sessions::amplifier_stub::ensure_session( + &home.path().join(".amplifier"), + "exact", + cwd.to_str().unwrap(), + "terminal", + ) + .unwrap(); + fs::write( + session.session_dir.join("transcript.jsonl.backup"), + b"synthetic retained turn", + ) + .unwrap(); + assert!(matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test("amplifier", "exact", home.path()) + ), + ProviderStoreProbe::Ready { .. } + )); + } + #[test] fn native_fixture_probe_uses_exact_verified_checkpoint_when_primary_is_missing() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index 834275ae2..d10cdaf7c 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -340,6 +340,10 @@ pub enum RecoveryProbe { evidence: Vec, #[serde(default)] store_state: EvidenceStoreState, + /// Path-specific negative inspection results. Empty legacy rows are + /// intentionally insufficient to authorize loss of a coding session. + #[serde(default)] + negative_paths: Vec, }, Blocked { path: RecoveryPath, @@ -4063,6 +4067,34 @@ mod tests { )); } + #[test] + fn recovery_negative_wire_preserves_inspected_rows_and_defaults_legacy_unknown() { + let legacy = serde_json::json!({"kind":"definitively_unavailable","data":{ + "path":"native_resume","reason":"old absence","evidence":["store"],"store_state":"missing" + }}); + let mut probe: RecoveryProbe = serde_json::from_value(legacy).unwrap(); + let RecoveryProbe::DefinitivelyUnavailable { negative_paths, .. } = &mut probe else { + panic!("wrong probe") + }; + assert!( + negative_paths.is_empty(), + "legacy wire must not invent inspection" + ); + negative_paths.push(RecoveryPathEvidence { + path: RecoveryPath::CheckpointRestore, + verdict: RecoveryEvidenceVerdict::DefinitiveNegative, + reason_code: "EXACT_SQLITE_NO_MATCH".into(), + evidence_refs: vec!["provider://effective.db/exact-session".into()], + store_state: EvidenceStoreState::PresentReadable, + }); + let decoded: RecoveryProbe = + serde_json::from_slice(&serde_json::to_vec(&probe).unwrap()).unwrap(); + assert_eq!( + decoded, probe, + "typed inspection must survive the host IPC wire" + ); + } + #[test] fn legacy_amplifier_secret_profile_deserializes_but_is_not_reinterpreted() { let reference: ProviderSecretReference = serde_json::from_value(serde_json::json!({ diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index a48d6f366..6f1be82e6 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -1021,6 +1021,10 @@ fn git_stdout(cwd: &Path, args: &[&str]) -> Option { #[cfg(test)] mod tests { use super::*; + #[cfg(unix)] + mod recovery_budget { + include!("tests/managed_recovery_budget_tests.rs"); + } #[test] fn managed_env_drops_server_and_provider_secrets() { diff --git a/crates/freshell-server/src/tests/managed_recovery_budget_tests.rs b/crates/freshell-server/src/tests/managed_recovery_budget_tests.rs new file mode 100644 index 000000000..303cc6918 --- /dev/null +++ b/crates/freshell-server/src/tests/managed_recovery_budget_tests.rs @@ -0,0 +1,296 @@ +use super::*; +use freshell_runtime_protocol::*; +use freshell_supervisor::{backend::*, registry::*, service::*}; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; + +struct StartupBackend { + repaired: AtomicBool, + inspections: AtomicUsize, +} +#[async_trait::async_trait] +impl RuntimeBackend for StartupBackend { + async fn create_stopped(&self, _: &CreateRuntimeSpec) -> Result { + panic!("dispatched conversation must not be replaced") + } + async fn start_host(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + panic!("no replacement host") + } + async fn inspect(&self, _: &OwnedRuntimeHandle) -> Result { + self.inspections.fetch_add(1, Ordering::SeqCst); + if !self.repaired.load(Ordering::SeqCst) { + return Err(BackendError::Unavailable( + "synthetic startup provider unavailable".into(), + )); + } + Ok(BackendInspection { + state: BackendRuntimeState::Running, + configured_limits: limits(), + }) + } + async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + panic!("must retain dispatched data") + } + async fn force_stop(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + panic!("must retain dispatched data") + } + async fn verify_empty(&self, _: &OwnedRuntimeHandle) -> Result { + Ok(false) + } + async fn list_known( + &self, + _: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + Ok(vec![]) + } + async fn read_limits(&self, _: &OwnedRuntimeHandle) -> Result { + Ok(limits()) + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + Ok(()) + } +} +fn limits() -> RuntimeLimits { + RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + } +} + +async fn control( + supervisor: Supervisor, + path: &Path, + seen: Arc>>, +) -> ( + tokio::sync::oneshot::Sender<()>, + tokio::task::JoinHandle<()>, +) { + let listener = tokio::net::UnixListener::bind(path).unwrap(); + let (tx, mut rx) = tokio::sync::oneshot::channel(); + let task = tokio::spawn(async move { + loop { + let mut socket = + tokio::select! {_=&mut rx=>break,accepted=listener.accept()=>accepted.unwrap().0}; + let envelope: Envelope = read_frame(&mut socket).await.unwrap(); + if let AdminCommand::Recover(request) = &envelope.body { + seen.lock().unwrap().push(request.trigger); + } + let reply = supervisor.dispatch(envelope).await; + write_frame(&mut socket, &reply).await.unwrap(); + } + }); + (tx, task) +} + +#[tokio::test] +async fn startup_recovery_naturally_exhausts_and_cold_hydration_requires_manual_retry() { + let root = tempfile::tempdir().unwrap(); + let registry_root = root.path().join("registry"); + let registry = Registry::open(®istry_root, None).unwrap(); + let runtime = root.path().join("runtime"); + std::fs::create_dir(&runtime).unwrap(); + let binary = runtime.join("synthetic-host"); + std::fs::write(&binary, b"fixture").unwrap(); + let config = SupervisorConfig { + runtime_root: runtime.clone(), + control_socket_path: root.path().join("control.sock"), + host_binary_path: binary.clone(), + image_ref: format!("sha256:{}", "b".repeat(64)), + test_run_id: "natural-startup-budget".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: root.path().join("lifecycle.jsonl"), + admission: Default::default(), + }; + let soul = SoulId::new(); + let prepared = registry + .prepare_launch(LaunchPreparation { + soul_id: soul.clone(), + provider: "opencode".into(), + provider_store_id: "exact-store".into(), + native_session_id: None, + creation_seed_ref: "original-seed".into(), + request_id: RequestId::new(), + payload_digest: "fixture".into(), + requested_limits: limits(), + profile: RuntimeProfile::Custom, + project_key: "fixture".into(), + fixture: None, + terminal: None, + fresh_agent: None, + view_intent: None, + admission: Default::default(), + }) + .await + .unwrap(); + registry + .commit_created( + prepared.incarnation_id.clone(), + BackendCreatedRecord { + daemon_id: DockerDaemonId::new(), + container_id: "a".repeat(64), + image_ref: config.image_ref.clone(), + runtime_dir: runtime.clone(), + host_binary_path: binary, + immutable_config_digest: format!("sha256:{}", "c".repeat(64)), + }, + ) + .await + .unwrap(); + registry + .commit_execution_grant(prepared.incarnation_id.clone(), HostBootId::new(), limits()) + .await + .unwrap(); + registry + .mark_running(prepared.incarnation_id.clone()) + .await + .unwrap(); + let input = RequestId::new(); + registry + .begin_input(soul.clone(), input.clone(), b"accepted-turn", None) + .await + .unwrap(); + registry + .mark_input_dispatching(soul.clone(), input) + .await + .unwrap(); + let output = runtime.join("retained-output"); + std::fs::write(&output, b"startup diagnostic").unwrap(); + let backend = Arc::new(StartupBackend { + repaired: AtomicBool::new(false), + inspections: AtomicUsize::new(0), + }); + let supervisor = Supervisor::new(registry.clone(), backend.clone(), config.clone()).unwrap(); + let seen = Arc::new(std::sync::Mutex::new(vec![])); + let (shutdown, task) = control( + supervisor.clone(), + &config.control_socket_path, + seen.clone(), + ) + .await; + let client = RuntimeClient::new(&config.control_socket_path, &config.control_secret); + let started = tokio::time::Instant::now(); + let exhausted = recover_with_policy(&client, soul.clone(), RecoveryTrigger::ProviderExit) + .await + .unwrap(); + assert!(started.elapsed() >= std::time::Duration::from_secs(12)); + assert_eq!(backend.inspections.load(Ordering::SeqCst), 3); + assert_eq!( + *seen.lock().unwrap(), + vec![ + RecoveryTrigger::ProviderExit, + RecoveryTrigger::ProviderExit, + RecoveryTrigger::ProviderExit, + RecoveryTrigger::RetryExhausted + ] + ); + assert!(matches!( + exhausted.probe, + Some(RecoveryProbe::Blocked { + reason: RecoveryBlockReason::RetryBudget, + .. + }) + )); + shutdown.send(()).unwrap(); + task.await.unwrap(); + drop(supervisor); + drop(registry); + drop(client); + std::fs::remove_file(&config.control_socket_path).unwrap(); + let registry = Registry::open(®istry_root, None).unwrap(); + let supervisor = Supervisor::new(registry.clone(), backend.clone(), config.clone()).unwrap(); + let (shutdown, task) = control(supervisor, &config.control_socket_path, seen).await; + let client = RuntimeClient::new(&config.control_socket_path, &config.control_secret); + backend.repaired.store(true, Ordering::SeqCst); + let secret = b"synthetic-host-secret"; + std::fs::write(runtime.join("secret"), secret).unwrap(); + let listener = tokio::net::UnixListener::bind(runtime.join("host.sock")).unwrap(); + let (stop_host, mut stop_rx) = tokio::sync::oneshot::channel(); + let host = tokio::spawn(async move { + let boot = HostBootId::new(); + loop { + let mut socket = + tokio::select! {_=&mut stop_rx=>break,a=listener.accept()=>a.unwrap().0}; + let envelope: Envelope = read_frame(&mut socket).await.unwrap(); + let result = match envelope.body { + HostCommand::Hello { + challenge, + incarnation_id, + } => HostResult::Hello { + host_boot_id: boot.clone(), + proof: host_proof(secret, &challenge, &boot, &incarnation_id), + capabilities: vec![], + effective_limits: limits(), + }, + HostCommand::Status { .. } => HostResult::Status { + host_boot_id: boot.clone(), + worker_pid: Some(123), + worker_launch_count: 1, + max_control_epoch: 1, + max_execution_generation: 1, + fixture_evidence: serde_json::json!({}), + exited: false, + native_session_id: None, + }, + HostCommand::ProbeRecovery { .. } => { + HostResult::RecoveryProbe(RecoveryProbe::Blocked { + path: RecoveryPath::NativeResume, + reason: RecoveryBlockReason::AmbiguousIdentity, + retry_hint: RetryHint { + automatic_after_ms: None, + manual_retry: true, + repair: Some("capture identity".into()), + }, + evidence: vec![], + }) + } + other => panic!("unexpected repaired-host command {other:?}"), + }; + write_frame( + &mut socket, + &HostReply { + request_id: envelope.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + }); + let automatic = client + .recover(soul.clone(), RecoveryTrigger::StartupReconcile) + .await + .unwrap(); + let manual = client + .recover(soul.clone(), RecoveryTrigger::ManualRetry) + .await + .unwrap(); + shutdown.send(()).unwrap(); + task.await.unwrap(); + stop_host.send(()).unwrap(); + host.await.unwrap(); + assert_eq!( + automatic.outcome, + RecoveryOutcome::Blocked, + "cold automatic request must remain blocked after repair" + ); + assert!(matches!( + automatic.probe, + Some(RecoveryProbe::Blocked { + reason: RecoveryBlockReason::RetryBudget, + .. + }) + )); + assert_eq!( + manual.outcome, + RecoveryOutcome::Reattached, + "only explicit manual retry rearms recovery" + ); + assert_eq!(manual.view.incarnation_id, prepared.incarnation_id); + assert_eq!(std::fs::read(output).unwrap(), b"startup diagnostic"); + assert_eq!( + registry.inventory().await.unwrap().len(), + 1, + "no replacement conversation" + ); +} diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 8a70003e6..19611d6f1 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -36,6 +36,8 @@ struct PersistedHostState { max_control_epoch: u64, max_execution_generation: u64, grant_id: Option, + #[serde(default)] + provider: Option, worker_pid: Option, worker_launch_count: u64, fixture_evidence: serde_json::Value, @@ -225,10 +227,13 @@ async fn serve(args: &[String]) -> Result<(), String> { return Err("incarnation secret is too short".into()); } let host_boot_id = HostBootId::new(); - let persisted = PersistedHostState { - host_boot_id: Some(host_boot_id.clone()), - ..Default::default() + let mut persisted = match std::fs::read(state_dir.join("host-state.json")) { + Ok(bytes) => serde_json::from_slice::(&bytes) + .map_err(|error| format!("read durable host state: {error}"))?, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => PersistedHostState::default(), + Err(error) => return Err(format!("read durable host state: {error}")), }; + persisted.host_boot_id = Some(host_boot_id.clone()); write_state(&state_dir, &persisted)?; append_event_with_secret( &state_dir, @@ -471,6 +476,17 @@ async fn dispatch( .map(str::to_string) }); } + } else if persisted.grant_id.is_some() + || persisted.worker_pid.is_some() + || persisted.worker_launch_count > 0 + { + // After a host boot, persisted initialization/PID + // metadata cannot establish ownership or liveness of + // an in-memory worker. It also cannot prove its end. + return Err(RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "initialization was recorded but worker liveness is unconfirmed; preserve the incarnation and inspect it before retry", + )); } Ok(HostResult::Status { host_boot_id: state.host_boot_id.clone(), @@ -743,19 +759,10 @@ async fn dispatch( ensure_incarnation(&incarnation_id, state)?; let probe = match resume_spec { Some(spec) => providers::probe_resume(*spec, run_as_uid, run_as_gid).await, - None if never_dispatched => RecoveryProbe::PristineSeedReady { - seed: creation_seed_ref, - never_dispatched_proof: format!( - "{}:command-journal-empty", - state.incarnation_id - ), - }, - None => RecoveryProbe::DefinitivelyUnavailable { - path: RecoveryPath::NativeResume, - reason: "no durable native identity or checkpoint is recorded".into(), - evidence: vec![state.state_dir.display().to_string()], - store_state: freshell_runtime_protocol::EvidenceStoreState::Missing, - }, + None => { + probe_without_identity(state, creation_seed_ref, never_dispatched) + .await? + } }; Ok(HostResult::RecoveryProbe(probe)) } @@ -765,6 +772,59 @@ async fn dispatch( } } +async fn probe_without_identity( + state: &HostState, + seed: String, + never_dispatched: bool, +) -> Result { + use freshell_runtime_protocol::{EvidenceStoreState, RecoveryBlockReason, RetryHint}; + let persisted = state.persisted.lock().await.clone(); + let provider = persisted.provider.as_deref().or_else(|| { + persisted + .fixture_evidence + .get("mode") + .and_then(serde_json::Value::as_str) + }); + let shell_policy = provider.is_some_and(|provider| { + freshell_agent_runtime::recovery_paths(provider) + .is_some_and(|paths| !paths.contains(&RecoveryPath::NativeResume)) + }); + let initialization_undispatched = persisted.grant_id.is_none() + && persisted.worker_launch_count == 0 + && persisted.worker_pid.is_none(); + if never_dispatched && (initialization_undispatched || shell_policy) { + return Ok(RecoveryProbe::PristineSeedReady { + seed, + never_dispatched_proof: format!( + "{}:input-and-initialization-undispatched", + state.incarnation_id + ), + }); + } + if shell_policy + && state + .pty + .lock() + .await + .as_ref() + .is_some_and(HostedPty::exited) + { + return Ok(RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::Reattach, + reason: "shell provider process ended after dispatch".into(), + evidence: vec![format!("incarnationId={}", state.incarnation_id)], + store_state: EvidenceStoreState::NotApplicable, + negative_paths: Vec::new(), + }); + } + Ok(RecoveryProbe::Blocked { + path:if shell_policy {RecoveryPath::Reattach} else {RecoveryPath::NativeResume}, + reason:RecoveryBlockReason::AmbiguousIdentity, + retry_hint:RetryHint {automatic_after_ms:None,manual_retry:true,repair:Some(if shell_policy {"verify the exact shell process has ended, then retry"} else {"capture the exact native identity and inspect its provider stores, then retry"}.into())}, + evidence:vec![if shell_policy {"shell process termination is unconfirmed"} else {"native identity is unrecorded; required provider-store absence has not been established"}.into()], + }) +} + fn unsupported_fresh_agent() -> RuntimeError { RuntimeError::new( RuntimeErrorCode::UnsupportedWorkload, @@ -831,6 +891,24 @@ async fn grant_execution( fresh_agent, resume_spec, } = request; + let provider = terminal + .as_ref() + .map(|terminal| terminal.mode.clone()) + .or_else(|| { + fresh_agent + .as_ref() + .map(|agent| agent.provider.native_provider().to_string()) + }) + .or_else(|| { + fixture.map(|fixture| { + if fixture == FixtureKind::NativeSession { + "native-session-fixture" + } else { + "phase1-fixture" + } + .to_string() + }) + }); let mut persisted = state.persisted.lock().await; if control_epoch < persisted.max_control_epoch || execution_generation < persisted.max_execution_generation @@ -945,6 +1023,7 @@ async fn grant_execution( persisted.max_control_epoch = control_epoch; persisted.max_execution_generation = execution_generation; persisted.grant_id = Some(grant_id.clone()); + persisted.provider = provider.clone(); persisted.worker_pid = None; write_state(&state.state_dir, persisted).map_err(registry_like_error) }; @@ -3980,6 +4059,8 @@ fn now_millis() -> i64 { #[cfg(test)] mod tests { + #[path = "probe_recovery_tests.rs"] + mod probe_recovery; fn copy_provider_config_references( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -6539,15 +6620,11 @@ mod tests { } fn tempfile_like_dir() -> PathBuf { - let dir = std::env::temp_dir().join(format!("freshell-host-test-{}", uuid_like())); + let dir = std::env::temp_dir().join(format!("freshell-host-test-{}", uuid::Uuid::new_v4())); std::fs::create_dir_all(&dir).unwrap(); dir } - fn uuid_like() -> String { - format!("{}-{}", std::process::id(), now_millis()) - } - pub(crate) mod created_path_ownership { use super::*; use std::{ diff --git a/crates/freshell-session-host/src/providers/deterministic_fresh_agent.rs b/crates/freshell-session-host/src/providers/deterministic_fresh_agent.rs index 5ae495246..baeb3172d 100644 --- a/crates/freshell-session-host/src/providers/deterministic_fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/deterministic_fresh_agent.rs @@ -386,6 +386,7 @@ pub(crate) async fn probe_resume( reason: "deterministic provider state is absent".into(), evidence: vec!["fixtureState=missing".into()], store_state: EvidenceStoreState::Missing, + negative_paths: Vec::new(), }); }; if state.schema_version != 1 { @@ -404,6 +405,7 @@ pub(crate) async fn probe_resume( reason: "deterministic provider state belongs to another native identity".into(), evidence: vec!["fixtureIdentity=wrong".into()], store_state: EvidenceStoreState::PresentReadable, + negative_paths: Vec::new(), }); } Ok(ProviderStoreProbe::Ready { diff --git a/crates/freshell-session-host/src/providers/mod.rs b/crates/freshell-session-host/src/providers/mod.rs index 9c9b05976..7b06be80c 100644 --- a/crates/freshell-session-host/src/providers/mod.rs +++ b/crates/freshell-session-host/src/providers/mod.rs @@ -659,11 +659,13 @@ pub async fn probe_resume( reason, evidence, store_state, + negative_paths, }) => RecoveryProbe::DefinitivelyUnavailable { path: RecoveryPath::NativeResume, reason, evidence, store_state, + negative_paths, }, Ok(ProviderStoreProbe::Blocked { reason, diff --git a/crates/freshell-session-host/src/tests/probe_recovery_tests.rs b/crates/freshell-session-host/src/tests/probe_recovery_tests.rs new file mode 100644 index 000000000..8deb3a047 --- /dev/null +++ b/crates/freshell-session-host/src/tests/probe_recovery_tests.rs @@ -0,0 +1,219 @@ +use super::*; + +#[test] +fn parallel_host_fixture_state_directories_are_independent() { + let barrier = Arc::new(std::sync::Barrier::new(32)); + let workers: Vec<_> = (0..32) + .map(|_| { + let barrier = Arc::clone(&barrier); + std::thread::spawn(move || { + barrier.wait(); + let state = test_host_state(); + let marker = state.incarnation_id.to_string(); + std::fs::write(state.state_dir.join("fixture-owner"), &marker).unwrap(); + (state, marker) + }) + }) + .collect(); + let fixtures: Vec<_> = workers + .into_iter() + .map(|worker| worker.join().unwrap()) + .collect(); + for (state, marker) in fixtures { + assert_eq!( + std::fs::read_to_string(state.state_dir.join("fixture-owner")).unwrap(), + marker, + "parallel fixtures must retain their own physical state" + ); + } +} + +async fn probe(state: &Arc, never_dispatched: bool) -> RecoveryProbe { + let result = dispatch( + authenticated_host_envelope( + state, + HostCommand::ProbeRecovery { + incarnation_id: state.incarnation_id.clone(), + resume_spec: None, + creation_seed_ref: "original-seed".into(), + never_dispatched, + run_as_uid: unsafe { libc::geteuid() }, + run_as_gid: unsafe { libc::getegid() }, + }, + ), + state, + ) + .await + .unwrap(); + let HostResult::RecoveryProbe(probe) = result else { + panic!("wrong result") + }; + probe +} + +#[tokio::test] +async fn dispatched_without_identity_blocks_and_preserves_data() { + let state = test_host_state(); + std::fs::write(state.state_dir.join("conversation"), b"accepted-turn").unwrap(); + state.persisted.lock().await.fixture_evidence = serde_json::json!({"mode":"opencode"}); + assert!(matches!( + probe(&state, false).await, + RecoveryProbe::Blocked { .. } + )); + assert_eq!( + std::fs::read(state.state_dir.join("conversation")).unwrap(), + b"accepted-turn" + ); +} + +#[tokio::test] +async fn startup_before_identity_retains_diagnostic() { + let state = test_host_state(); + let terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"early-exit", "streamId":"early-exit", "mode":"shell", "program":"/bin/sh", + "args":["-c","echo synthetic-startup-failure >&2; exit 23"], "env":{}, "cwd":state.state_dir, + "runAsUid":unsafe {libc::geteuid()}, "runAsGid":unsafe {libc::getegid()}, "cols":80,"rows":24, + "projectKey":"fixture", "workspacePath":state.state_dir + })).unwrap(); + let pty = HostedPty::spawn( + &state.state_dir, + state.incarnation_id.clone(), + &terminal, + std::collections::BTreeMap::new(), + None, + None, + ) + .await + .unwrap(); + *state.pty.lock().await = Some(pty); + state.persisted.lock().await.fixture_evidence = serde_json::json!({"mode":"opencode"}); + let deadline = tokio::time::Instant::now() + Duration::from_secs(5); + loop { + let mut guard = state.pty.lock().await; + let pty = guard.as_mut().unwrap(); + let output = pty.read_output(state.incarnation_id.clone(), 0, 64 * 1024, None); + if serde_json::to_string(&output) + .unwrap() + .contains("synthetic-startup-failure") + { + break; + } + assert!( + tokio::time::Instant::now() < deadline, + "startup diagnostic missing" + ); + drop(guard); + sleep(Duration::from_millis(10)).await; + } + let result = probe(&state, false).await; + let mut guard = state.pty.lock().await; + let pty = guard.as_mut().unwrap(); + let output = pty.read_output(state.incarnation_id.clone(), 0, 64 * 1024, None); + pty.stop().await; + assert!(matches!(result, RecoveryProbe::Blocked { .. })); + assert!(serde_json::to_string(&output) + .unwrap() + .contains("synthetic-startup-failure")); + assert_eq!( + state.persisted.lock().await.worker_launch_count, + 0, + "no replacement" + ); +} + +#[tokio::test] +async fn pristine_seed_still_recovers() { + let state = test_host_state(); + assert!(matches!(probe(&state, true).await, + RecoveryProbe::PristineSeedReady { seed, .. } if seed == "original-seed")); +} + +#[tokio::test] +async fn cold_initialization_without_worker_does_not_claim_live() { + for (grant_recorded, worker_pid, worker_launch_count) in [ + (true, Some(123), 1), + (false, Some(123), 1), + (false, None, 1), + ] { + let state = test_host_state(); + { + let mut persisted = state.persisted.lock().await; + persisted.provider = Some("opencode".into()); + persisted.grant_id = grant_recorded.then(GrantId::new); + persisted.worker_pid = worker_pid; + persisted.worker_launch_count = worker_launch_count; + write_state(&state.state_dir, &persisted).unwrap(); + } + std::fs::write( + state.state_dir.join("retained-output"), + b"startup diagnostic", + ) + .unwrap(); + let status = dispatch( + authenticated_host_envelope( + &state, + HostCommand::Status { + incarnation_id: state.incarnation_id.clone(), + }, + ), + &state, + ) + .await; + assert!( + status.is_err(), + "durable grant and stale PID cannot prove a live worker: {status:?}" + ); + assert!(matches!( + probe(&state, false).await, + RecoveryProbe::Blocked { + reason: freshell_runtime_protocol::RecoveryBlockReason::AmbiguousIdentity, + .. + } + )); + assert_eq!( + std::fs::read(state.state_dir.join("retained-output")).unwrap(), + b"startup diagnostic" + ); + } +} + +#[tokio::test] +async fn shell_without_native_paths_uses_terminal_end_policy() { + let state = test_host_state(); + state.persisted.lock().await.fixture_evidence = serde_json::json!({"mode":"shell"}); + let terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"ended-shell", "streamId":"ended-shell", "mode":"shell", "program":"/bin/sh", + "args":["-c","echo synthetic-shell-end; exit 0"], "env":{}, "cwd":state.state_dir, + "runAsUid":unsafe {libc::geteuid()}, "runAsGid":unsafe {libc::getegid()}, "cols":80,"rows":24, + "projectKey":"fixture", "workspacePath":state.state_dir + })).unwrap(); + *state.pty.lock().await = Some( + HostedPty::spawn( + &state.state_dir, + state.incarnation_id.clone(), + &terminal, + std::collections::BTreeMap::new(), + None, + None, + ) + .await + .unwrap(), + ); + let deadline = tokio::time::Instant::now() + Duration::from_secs(5); + while !state.pty.lock().await.as_ref().unwrap().exited() { + assert!( + tokio::time::Instant::now() < deadline, + "shell end was not observed" + ); + sleep(Duration::from_millis(10)).await; + } + let result = probe(&state, false).await; + state.pty.lock().await.as_mut().unwrap().stop().await; + assert!(matches!( + result, + RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::Reattach, + .. + } + )); +} diff --git a/crates/freshell-supervisor/src/checkpoint_tests.rs b/crates/freshell-supervisor/src/checkpoint_tests.rs new file mode 100644 index 000000000..00fcbac9f --- /dev/null +++ b/crates/freshell-supervisor/src/checkpoint_tests.rs @@ -0,0 +1,300 @@ +use super::*; +use crate::{ + admission::AdmissionPolicy, + registry::{BackendCreatedRecord, LaunchPreparation, Registry}, +}; +use freshell_runtime_protocol::*; + +pub(crate) fn limits() -> RuntimeLimits { + RuntimeLimits { + cpu_milli: 500, + memory_bytes: 64 * 1024 * 1024, + swap_bytes: 0, + pids_max: 32, + } +} + +pub(crate) async fn context( + root: &std::path::Path, + provider: &str, + dispatched: bool, +) -> (Registry, RecoveryContext) { + let registry = Registry::open(root.join("registry"), None).unwrap(); + let soul = SoulId::new(); + let prepared = registry + .prepare_launch(LaunchPreparation { + soul_id: soul.clone(), + provider: provider.into(), + provider_store_id: "store".into(), + native_session_id: if provider == "shell" { + None + } else { + Some("exact-session".into()) + }, + creation_seed_ref: "original-seed".into(), + request_id: RequestId::new(), + payload_digest: "fixture".into(), + requested_limits: limits(), + profile: RuntimeProfile::Custom, + project_key: "fixture".into(), + fixture: None, + terminal: None, + fresh_agent: None, + view_intent: None, + admission: AdmissionPolicy::default(), + }) + .await + .unwrap(); + let runtime = root.join("runtime"); + std::fs::create_dir_all(&runtime).unwrap(); + std::fs::write(runtime.join("host"), b"synthetic-host").unwrap(); + registry + .commit_created( + prepared.incarnation_id.clone(), + BackendCreatedRecord { + daemon_id: DockerDaemonId::new(), + container_id: "a".repeat(64), + image_ref: format!("sha256:{}", "b".repeat(64)), + runtime_dir: runtime.clone(), + host_binary_path: runtime.join("host"), + immutable_config_digest: format!("sha256:{}", "c".repeat(64)), + }, + ) + .await + .unwrap(); + if dispatched { + registry + .commit_execution_grant(prepared.incarnation_id.clone(), HostBootId::new(), limits()) + .await + .unwrap(); + registry + .mark_running(prepared.incarnation_id) + .await + .unwrap(); + let request = RequestId::new(); + registry + .begin_input(soul.clone(), request.clone(), b"synthetic-input", None) + .await + .unwrap(); + registry + .mark_input_dispatching(soul.clone(), request) + .await + .unwrap(); + } + let context = registry.recovery_context(soul).await.unwrap(); + (registry, context) +} + +pub(crate) fn spec(provider: &str) -> ResumeSpec { + serde_json::from_value(serde_json::json!({ + "providerSession":{"provider":provider,"providerStoreId":"store","nativeSessionId":"exact-session"}, + "mode":provider,"program":provider,"providerHome":"/home/freshell/provider","cwd":"/workspace", + "workspacePath":"/workspace","creationSeedRef":"original-seed","checkpointRevision":0, + "allocationState":"verified_durable","evidenceRevision":1,"neverDispatched":false + })).unwrap() +} + +fn negative(reason: &str, evidence: Vec, store_state: EvidenceStoreState) -> RecoveryProbe { + RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::NativeResume, + reason: reason.into(), + evidence, + store_state, + negative_paths: vec![], + } +} + +#[tokio::test] +async fn zero_revision_blocks() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "opencode", true).await; + assert!(matches!(probe(&ctx), RecoveryProbe::Blocked { .. })); +} +#[tokio::test] +async fn reason_text_is_not_inspection() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "opencode", true).await; + let p = negative( + "checkpoint missing", + vec!["/fixture/.freshell/checkpoints/looks-inspected".into()], + EvidenceStoreState::Missing, + ); + assert!(matches!( + probe_after_provider(&ctx, &p), + RecoveryProbe::Blocked { .. } + )); +} +#[tokio::test] +async fn legacy_negative_blocks() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "opencode", true).await; + assert!(matches!( + probe_after_provider( + &ctx, + &negative("missing", vec![], EvidenceStoreState::Missing) + ), + RecoveryProbe::Blocked { .. } + )); +} + +#[tokio::test] +async fn unreadable_unknown_or_empty_inspected_rows_block() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "opencode", true).await; + for (store_state, refs) in [ + (EvidenceStoreState::Unknown, vec!["store".into()]), + (EvidenceStoreState::PresentUnreadable, vec!["store".into()]), + (EvidenceStoreState::Missing, vec![]), + (EvidenceStoreState::Missing, vec![String::new()]), + ] { + let mut p = negative( + "missing", + vec!["aggregate".into()], + EvidenceStoreState::Missing, + ); + if let RecoveryProbe::DefinitivelyUnavailable { negative_paths, .. } = &mut p { + negative_paths.push(RecoveryPathEvidence { + path: RecoveryPath::CheckpointRestore, + verdict: RecoveryEvidenceVerdict::DefinitiveNegative, + reason_code: "INSPECTED_ABSENCE".into(), + evidence_refs: refs, + store_state, + }); + } + assert!( + matches!( + probe_after_provider(&ctx, &p), + RecoveryProbe::Blocked { .. } + ), + "{p:?}" + ); + } +} +#[tokio::test] +async fn conflicting_rows_block() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "opencode", true).await; + let mut p = negative( + "checkpoint missing", + vec!["checkpoint=missing".into(), "checkpoint=present".into()], + EvidenceStoreState::Missing, + ); + if let RecoveryProbe::DefinitivelyUnavailable { negative_paths, .. } = &mut p { + negative_paths.extend( + [ + EvidenceStoreState::Missing, + EvidenceStoreState::PresentReadable, + ] + .map(|store_state| RecoveryPathEvidence { + path: RecoveryPath::CheckpointRestore, + verdict: RecoveryEvidenceVerdict::DefinitiveNegative, + reason_code: "INSPECTED_ABSENCE".into(), + evidence_refs: vec!["synthetic store inspection".into()], + store_state, + }), + ); + } + assert!(matches!( + probe_after_provider(&ctx, &p), + RecoveryProbe::Blocked { .. } + )); +} +#[tokio::test] +async fn unreadable_store_blocks() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "native-session-fixture", true).await; + std::fs::write(temp.path().join("native-session-state.json"), b"malformed").unwrap(); + let p = + freshell_agent_runtime::probe_provider_store(temp.path(), &spec("native-session-fixture")); + assert!(matches!( + p, + freshell_agent_runtime::ProviderStoreProbe::Blocked { + reason: RecoveryBlockReason::StoreUnreadable, + .. + } + )); + assert!(matches!(probe(&ctx), RecoveryProbe::Blocked { .. })); +} +#[tokio::test] +async fn independent_present_checkpoint_restores() { + let temp = tempfile::tempdir().unwrap(); + let (_, mut ctx) = context(temp.path(), "native-session-fixture", true).await; + let dir = temp.path().join(".freshell/checkpoints/native-session"); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write( + dir.join("7.json"), + br#"{"sessionId":"exact-session","checkpointRevision":7}"#, + ) + .unwrap(); + let p = + freshell_agent_runtime::probe_provider_store(temp.path(), &spec("native-session-fixture")); + let freshell_agent_runtime::ProviderStoreProbe::Ready { evidence } = p else { + panic!("{p:?}") + }; + assert!(evidence.iter().any(|e| e == "checkpointRevision=7")); + let mut spec = spec("native-session-fixture"); + spec.checkpoint_revision = 7; + spec.checkpoint_references = vec![CheckpointReference { + kind: "fixture".into(), + reference: dir.join("7.json").display().to_string(), + revision: 7, + verified: true, + }]; + ctx.checkpoint_revision = 7; + ctx.resume_spec = Some(spec); + assert!(matches!( + probe_after_provider( + &ctx, + &RecoveryProbe::Blocked { + path: RecoveryPath::NativeResume, + reason: RecoveryBlockReason::StoreMissing, + retry_hint: RetryHint { + automatic_after_ms: None, + manual_retry: true, + repair: None + }, + evidence: vec![] + } + ), + RecoveryProbe::ResumeReady { .. } + )); +} +#[tokio::test] +async fn inspected_shared_store_absence_remains_definitive() { + let temp = tempfile::tempdir().unwrap(); + let (_, ctx) = context(temp.path(), "opencode", true).await; + let db = temp.path().join(".local/share/opencode/opencode.db"); + std::fs::create_dir_all(db.parent().unwrap()).unwrap(); + let connection = rusqlite::Connection::open(&db).unwrap(); + connection + .execute("CREATE TABLE session(id TEXT PRIMARY KEY)", []) + .unwrap(); + let mut effective_spec = spec("opencode"); + effective_spec + .environment + .insert("OPENCODE_DB".into(), db.display().to_string()); + let p = freshell_agent_runtime::probe_provider_store(temp.path(), &effective_spec); + let freshell_agent_runtime::ProviderStoreProbe::DefinitivelyUnavailable { + reason, + evidence, + store_state, + negative_paths, + } = p + else { + panic!("{p:?}") + }; + let probe = RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::NativeResume, + reason, + evidence, + store_state, + negative_paths, + }; + assert!(matches!( + probe_after_provider(&ctx, &probe), + RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::CheckpointRestore, + .. + } + )); +} diff --git a/crates/freshell-supervisor/src/checkpoints.rs b/crates/freshell-supervisor/src/checkpoints.rs index e3aea58aa..cbf0e3c95 100644 --- a/crates/freshell-supervisor/src/checkpoints.rs +++ b/crates/freshell-supervisor/src/checkpoints.rs @@ -2,23 +2,18 @@ //! //! A nonzero revision is not sufficient. Recovery is ready only when the //! durable resume specification carries a matching, explicitly verified -//! checkpoint reference. Missing metadata is a definitive negative for this -//! path; mismatched or unverified metadata remains blocked and can never -//! participate in a loss certificate. +//! checkpoint reference. Missing metadata is unknown. Loss additionally +//! requires a fresh, explicit inspection of the applicable provider stores. use crate::registry::RecoveryContext; use freshell_runtime_protocol::{ - EvidenceStoreState, RecoveryBlockReason, RecoveryPath, RecoveryProbe, RetryHint, + EvidenceStoreState, RecoveryBlockReason, RecoveryPath, RecoveryPathEvidence, RecoveryProbe, + RetryHint, }; pub fn probe(context: &RecoveryContext) -> RecoveryProbe { if context.checkpoint_revision == 0 { - return RecoveryProbe::DefinitivelyUnavailable { - path: RecoveryPath::CheckpointRestore, - reason: "no checkpoint revision is recorded for this soul".into(), - evidence: vec!["checkpointRevision=0".into()], - store_state: EvidenceStoreState::Missing, - }; + return blocked(context, "checkpoint revision is unrecorded; provider checkpoint-store absence has not been established"); } let Some(spec) = context.resume_spec.clone() else { @@ -58,30 +53,67 @@ pub fn probe_after_provider( context: &RecoveryContext, provider_probe: &RecoveryProbe, ) -> RecoveryProbe { - if let RecoveryProbe::DefinitivelyUnavailable { - reason, - evidence, - store_state: EvidenceStoreState::Missing, - .. - } = provider_probe - { - let inspected_checkpoint_store = reason.to_ascii_lowercase().contains("checkpoint") - || evidence.iter().any(|item| { - item.contains("/.freshell/checkpoints/") - || item.contains(r"\.freshell\checkpoints\") - }); - if inspected_checkpoint_store { - return RecoveryProbe::DefinitivelyUnavailable { - path: RecoveryPath::CheckpointRestore, - reason: "provider-side checkpoint store was inspected and is absent".into(), - evidence: evidence.clone(), - store_state: EvidenceStoreState::Missing, - }; - } + if matches!( + provider_probe, + RecoveryProbe::DefinitivelyUnavailable { .. } + ) { + return match inspected_negative(provider_probe, RecoveryPath::CheckpointRestore) { + Ok(row) => RecoveryProbe::DefinitivelyUnavailable { + path: row.path, + reason: row.reason_code.clone(), + evidence: row.evidence_refs.clone(), + store_state: row.store_state, + negative_paths: vec![row], + }, + Err(probe) => probe, + }; } probe(context) } +/// Select exactly one explicit inspected row before any normalization. Both +/// live and offline loss collection use this same boundary; prose, locators +/// and aggregate store status cannot create a path-specific verdict. +pub(crate) fn inspected_negative( + probe: &RecoveryProbe, + path: RecoveryPath, +) -> Result { + let RecoveryProbe::DefinitivelyUnavailable { negative_paths, .. } = probe else { + return Err(probe.clone()); + }; + let mut selected = negative_paths.iter().filter(|row| row.path == path); + let row = selected.next(); + if let Some(row) = row.filter(|row| { + selected.next().is_none() + && !row.reason_code.trim().is_empty() + && !row.evidence_refs.is_empty() + && row + .evidence_refs + .iter() + .all(|reference| !reference.trim().is_empty()) + && matches!( + row.store_state, + EvidenceStoreState::Missing | EvidenceStoreState::PresentReadable + ) + }) { + return Ok(row.clone()); + } + Err(RecoveryProbe::Blocked { + path, + reason: RecoveryBlockReason::StoreUnreadable, + retry_hint: RetryHint { + automatic_after_ms: None, + manual_retry: true, + repair: Some( + "inspect the exact provider store for this recovery path, then retry".into(), + ), + }, + evidence: vec![ + "required recovery path has missing, conflicting or uninspected store evidence".into(), + ], + }) +} + fn blocked(context: &RecoveryContext, message: &str) -> RecoveryProbe { RecoveryProbe::Blocked { path: RecoveryPath::CheckpointRestore, @@ -97,3 +129,7 @@ fn blocked(context: &RecoveryContext, message: &str) -> RecoveryProbe { ], } } + +#[cfg(test)] +#[path = "checkpoint_tests.rs"] +pub(crate) mod tests; diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index e3b6f251e..4d30cd973 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -109,6 +109,12 @@ impl Supervisor { .await .map_err(map_registry)?; } + if !status.exited { + return Ok(live_reattach_probe(&context, &status)); + } + if context.provider == "shell" { + return Ok(shell_terminal_end_probe(&context)); + } let candidate = match resume_catalog::exact_resume_candidate( &context, status.native_session_id.as_deref(), @@ -141,16 +147,6 @@ impl Supervisor { other => other, }; - if !status.exited { - return Ok(RecoveryProbe::ReattachReady { - handle: ReattachHandle { - incarnation_id: context.prior_handle.incarnation_id().clone(), - host_boot_id: status.host_boot_id, - container_id: context.prior_handle.container_id().to_string(), - }, - protocol_version: CONTROL_PROTOCOL_VERSION, - }); - } return Ok(after_native_probe(&context, captured)); } Err(_) if context.resume_spec.is_some() => return offline_probe(&context), @@ -258,6 +254,22 @@ impl Supervisor { view, }); } + if view.desired_state == DesiredState::Running + && view.recovery_state == freshell_runtime_protocol::RecoveryState::Blocked + && matches!( + view.recovery_reason.as_deref(), + Some("RETRY_BUDGET" | "BLOCKED_RETRY_BUDGET") + ) + && request.trigger != RecoveryTrigger::ManualRetry + { + return Ok(RecoveryResult { + outcome: RecoveryOutcome::Blocked, + probe: Some(blocked(RecoveryPath::NativeResume, RecoveryBlockReason::RetryBudget, + "automatic recovery budget is exhausted; repair the dependency and explicitly retry", None)), + view, prior_incarnation_id: None, attempt_id: None, + expected_native_session_id: None, observed_native_session_id: None, incident_id: None, + }); + } } if self .registry @@ -1305,50 +1317,8 @@ impl Supervisor { for path in provider_paths { let row = match path { RecoveryPath::Reattach => self.prove_reattach_unavailable(context).await?, - RecoveryPath::NativeResume => match native_probe { - RecoveryProbe::DefinitivelyUnavailable { - reason, - evidence, - store_state, - .. - } => { - if matches!( - store_state, - EvidenceStoreState::Unknown | EvidenceStoreState::PresentUnreadable - ) { - return Err(blocked( - RecoveryPath::NativeResume, - RecoveryBlockReason::StoreUnreadable, - "native provider state is not definitively readable or missing", - None, - )); - } - RecoveryPathEvidence { - path: RecoveryPath::NativeResume, - verdict: RecoveryEvidenceVerdict::DefinitiveNegative, - reason_code: reason.clone(), - evidence_refs: evidence.clone(), - store_state: *store_state, - } - } - other => return Err(other.clone()), - }, - RecoveryPath::CheckpointRestore => { - match checkpoints::probe_after_provider(context, native_probe) { - RecoveryProbe::DefinitivelyUnavailable { - reason, - evidence, - store_state, - .. - } => RecoveryPathEvidence { - path: RecoveryPath::CheckpointRestore, - verdict: RecoveryEvidenceVerdict::DefinitiveNegative, - reason_code: reason, - evidence_refs: evidence, - store_state, - }, - other => return Err(other), - } + RecoveryPath::NativeResume | RecoveryPath::CheckpointRestore => { + checkpoints::inspected_negative(native_probe, *path)? } RecoveryPath::PristineSeed => { if context.never_dispatched { @@ -1360,6 +1330,10 @@ impl Supervisor { ), }); } + if context.dispatched_command_count == 0 { + return Err(blocked(RecoveryPath::PristineSeed, RecoveryBlockReason::AmbiguousIdentity, + "initialization/native identity is uncertain; no durable input dispatch establishes pristine-seed unavailability", None)); + } RecoveryPathEvidence { path: RecoveryPath::PristineSeed, verdict: RecoveryEvidenceVerdict::DefinitiveNegative, @@ -1367,6 +1341,10 @@ impl Supervisor { evidence_refs: vec![ format!("acceptedCommandCount={}", context.accepted_command_count), format!("completedCommandCount={}", context.completed_command_count), + format!( + "dispatchedCommandCount={}", + context.dispatched_command_count + ), ], store_state: EvidenceStoreState::NotApplicable, } @@ -1458,14 +1436,7 @@ impl Supervisor { ) })?; if !status.exited { - return Err(RecoveryProbe::ReattachReady { - handle: ReattachHandle { - incarnation_id: context.prior_handle.incarnation_id().clone(), - host_boot_id: status.host_boot_id, - container_id: context.prior_handle.container_id().to_string(), - }, - protocol_version: CONTROL_PROTOCOL_VERSION, - }); + return Err(live_reattach_probe(context, &status)); } Ok(RecoveryPathEvidence { path: RecoveryPath::Reattach, @@ -1905,95 +1876,94 @@ fn fresh_agent_replay_settings( } fn after_native_probe(context: &RecoveryContext, probe: RecoveryProbe) -> RecoveryProbe { - match probe { - ready @ RecoveryProbe::ResumeReady { .. } - | ready @ RecoveryProbe::Blocked { .. } - | ready @ RecoveryProbe::PristineSeedReady { .. } - | ready @ RecoveryProbe::ReattachReady { .. } => ready, - RecoveryProbe::DefinitivelyUnavailable { - reason, - mut evidence, - store_state, - .. - } => { - if context.never_dispatched { - return RecoveryProbe::PristineSeedReady { - seed: context.creation_seed_ref.clone(), - never_dispatched_proof: format!( - "{}:supervisor-and-host-command-journals-empty", - context.soul_id - ), - }; - } - match checkpoints::probe_after_provider( - context, - &RecoveryProbe::DefinitivelyUnavailable { - path: RecoveryPath::NativeResume, - reason: reason.clone(), - evidence: evidence.clone(), - store_state, - }, - ) { - blocked @ RecoveryProbe::Blocked { .. } => blocked, - RecoveryProbe::DefinitivelyUnavailable { - reason: checkpoint_reason, - evidence: checkpoint_evidence, - .. - } => { - evidence.extend(checkpoint_evidence); - RecoveryProbe::DefinitivelyUnavailable { - path: RecoveryPath::NativeResume, - reason: format!( - "native resume unavailable ({reason}); checkpoint unavailable ({checkpoint_reason}); pristine seed forbidden after dispatch" - ), - evidence, - store_state, - } - } - other => other, - } - } + if !matches!(probe, RecoveryProbe::DefinitivelyUnavailable { .. }) { + return probe; + } + if context.never_dispatched { + return RecoveryProbe::PristineSeedReady { + seed: context.creation_seed_ref.clone(), + never_dispatched_proof: format!( + "{}:supervisor-and-host-command-journals-empty", + context.soul_id + ), + }; + } + if let Err(blocked) = checkpoints::inspected_negative(&probe, RecoveryPath::NativeResume) { + return blocked; + } + match checkpoints::probe_after_provider(context, &probe) { + RecoveryProbe::DefinitivelyUnavailable { .. } => probe, + other => other, + } +} + +fn live_reattach_probe( + context: &RecoveryContext, + status: &crate::service::HostStatus, +) -> RecoveryProbe { + if status.worker_pid.is_none() { + return blocked( + RecoveryPath::Reattach, + RecoveryBlockReason::ProviderUnavailable, + "host has no confirmed provider worker; process termination remains unknown", + Some(2_000), + ); + } + RecoveryProbe::ReattachReady { + handle: ReattachHandle { + incarnation_id: context.prior_handle.incarnation_id().clone(), + host_boot_id: status.host_boot_id.clone(), + container_id: context.prior_handle.container_id().to_string(), + }, + protocol_version: CONTROL_PROTOCOL_VERSION, + } +} + +/// Called only after authenticated worker exit or an owned ended backend +/// inspection. Final loss certification independently rechecks that proof. +fn shell_terminal_end_probe(context: &RecoveryContext) -> RecoveryProbe { + if context.never_dispatched { + return RecoveryProbe::PristineSeedReady { + seed: context.creation_seed_ref.clone(), + never_dispatched_proof: format!( + "{}:supervisor-input-journal-undispatched", + context.soul_id + ), + }; + } + RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::Reattach, + reason: "non-resumable shell ended after input dispatch".into(), + evidence: vec![format!( + "incarnationId={}", + context.prior_handle.incarnation_id() + )], + store_state: EvidenceStoreState::NotApplicable, + negative_paths: Vec::new(), } } fn offline_probe(context: &RecoveryContext) -> Result { + if context.provider == "shell" { + return Ok(shell_terminal_end_probe(context)); + } if let Some(spec) = context.resume_spec.clone() { return Ok(RecoveryProbe::ResumeReady { evidence_revision: spec.evidence_revision, resume_spec: Box::new(spec), }); } - if context.native_session_id.is_some() { - return Ok(blocked( - RecoveryPath::NativeResume, - RecoveryBlockReason::StoreUnreadable, - "a native identity was allocated, but no verified durable resume evidence was captured", - None, - )); - } if context.never_dispatched { return Ok(RecoveryProbe::PristineSeedReady { seed: context.creation_seed_ref.clone(), - never_dispatched_proof: format!("{}:supervisor-input-journal-empty", context.soul_id), + never_dispatched_proof: format!( + "{}:supervisor-input-journal-and-initialization-undispatched", + context.soul_id + ), }); } - match checkpoints::probe(context) { - blocked_probe @ RecoveryProbe::Blocked { .. } => Ok(blocked_probe), - RecoveryProbe::DefinitivelyUnavailable { - mut evidence, - reason, - .. - } => { - evidence.push("nativeResume=definitively_unavailable".into()); - Ok(RecoveryProbe::DefinitivelyUnavailable { - path: RecoveryPath::NativeResume, - reason: format!("native resume unavailable; {reason}"), - evidence, - store_state: freshell_runtime_protocol::EvidenceStoreState::Missing, - }) - } - other => Ok(other), - } + Ok(blocked(RecoveryPath::NativeResume, RecoveryBlockReason::AmbiguousIdentity, + "native identity/resume metadata is unrecorded or unverified; provider store absence has not been established", None)) } fn downtime_bucket(elapsed_ms: u64) -> &'static str { @@ -2293,6 +2263,7 @@ fn map_registry(error: RegistryError) -> RuntimeError { #[cfg(test)] mod tests { + include!("shell_recovery_tests.rs"); use super::*; use freshell_runtime_protocol::{FreshAgentLaunchSpec, FreshProvider, IncarnationId}; @@ -2396,6 +2367,7 @@ mod tests { &RecoveryProbe::DefinitivelyUnavailable { path: RecoveryPath::NativeResume, reason: "structured-negative".into(), + negative_paths: Vec::new(), evidence: vec!["digest-only".into()], store_state: EvidenceStoreState::Missing, } diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index afdc50bcc..705ac8431 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -273,6 +273,7 @@ pub struct RecoveryContext { pub requested_limits: RuntimeLimits, pub accepted_command_count: u64, pub completed_command_count: u64, + pub dispatched_command_count: u64, pub never_dispatched: bool, pub recovery_window_started_at: Option, pub successful_recoveries_in_window: u64, @@ -1475,12 +1476,14 @@ impl Registry { return Err(RegistryError::RecoveryInProgress(soul_id)); } let now = now_millis(); + let persisted_reason: Option = tx.query_row("SELECT recovery_reason FROM souls WHERE soul_id=?1", [soul_id.as_str()], |row| row.get(0))?; + let sticky_budget = matches!(persisted_reason.as_deref(), Some("RETRY_BUDGET" | "BLOCKED_RETRY_BUDGET")); let budget_available = freshell_agent_runtime::successful_recovery_budget_available( context.recovery_window_started_at, context.successful_recoveries_in_window, now, ); - if !budget_available && trigger != RecoveryTrigger::ManualRetry { + if (!budget_available || sticky_budget) && trigger != RecoveryTrigger::ManualRetry { let reason = "BLOCKED_RETRY_BUDGET"; tx.execute( "UPDATE souls SET recovery_state='blocked',recovery_reason=?1,updated_at=?2 WHERE soul_id=?3", @@ -1492,7 +1495,7 @@ impl Registry { reason: reason.into(), }); } - if !budget_available && trigger == RecoveryTrigger::ManualRetry { + if (!budget_available || sticky_budget) && trigger == RecoveryTrigger::ManualRetry { // A deliberate user retry is the explicit rearm operation. It // clears the persisted flap window but keeps identity/history // intact and still runs through the same fenced transaction. @@ -2671,7 +2674,17 @@ fn load_recovery_context( .11 .map(|value| serde_json::from_str(&value)) .transpose()?; - let never_dispatched = dispatched_count == 0 && row.2.is_none(); + // Coding providers may create a conversation while handling the initial + // execution grant, before an input row or native-ID acknowledgement exists. + // Only the existing settled rejection transaction proves an issued grant + // was unconsumed. Shell has no implicit provider initialization turn. + let requires_native = freshell_agent_runtime::recovery_paths(&row.0) + .is_some_and(|paths| paths.contains(&RecoveryPath::NativeResume)); + let initialization_uncertain: bool = requires_native && conn.query_row( + "SELECT EXISTS(SELECT 1 FROM incarnations i WHERE i.soul_id=?1 AND i.grant_id IS NOT NULL AND NOT EXISTS(SELECT 1 FROM commands c WHERE c.incarnation_id=i.incarnation_id AND c.state='provider_inputs_changed'))", + [soul_id.as_str()], |row| row.get(0), + )?; + let never_dispatched = dispatched_count == 0 && row.2.is_none() && !initialization_uncertain; Ok(RecoveryContext { soul_id: soul_id.clone(), provider: row.0, @@ -2698,6 +2711,7 @@ fn load_recovery_context( .unwrap_or_else(|| prior_handle.requested_limits()), accepted_command_count, completed_command_count, + dispatched_command_count: dispatched_count, never_dispatched, recovery_window_started_at: row.14, successful_recoveries_in_window: row.15, @@ -2932,6 +2946,146 @@ pub(crate) fn failpoint(_name: &'static str) -> Result<(), RegistryError> { #[cfg(test)] mod tests { + #[tokio::test] + async fn recovery_initialization_grant_before_identity_is_not_pristine() { + let dir = tempfile::tempdir().unwrap(); + let registry = Registry::open(dir.path(), None).unwrap(); + let soul = SoulId::new(); + let mut launch = prep(soul.clone(), RequestId::new(), "startup"); + launch.provider = "opencode".into(); + let prepared = registry.prepare_launch(launch).await.unwrap(); + registry + .commit_created( + prepared.incarnation_id.clone(), + BackendCreatedRecord { + daemon_id: DockerDaemonId::new(), + container_id: "a".repeat(64), + image_ref: format!("sha256:{}", "b".repeat(64)), + runtime_dir: dir.path().join("runtime"), + host_binary_path: dir.path().join("host"), + immutable_config_digest: format!("sha256:{}", "c".repeat(64)), + }, + ) + .await + .unwrap(); + assert!( + registry + .recovery_context(soul.clone()) + .await + .unwrap() + .never_dispatched + ); + registry + .commit_execution_grant(prepared.incarnation_id, HostBootId::new(), limits()) + .await + .unwrap(); + let ctx = registry.recovery_context(soul).await.unwrap(); + assert!(ctx.native_session_id.is_none()); + assert_eq!(ctx.accepted_command_count, 0); + assert!(!ctx.never_dispatched, "an issued initialization grant has uncertain provider execution even before input/native-ID capture"); + } + + #[tokio::test] + async fn recovery_five_natural_successes_exhaust_durable_budget_until_manual_retry() { + let dir = tempfile::tempdir().unwrap(); + let registry = Registry::open(dir.path(), None).unwrap(); + let soul = SoulId::new(); + materialize_test_runtime(®istry, soul.clone()).await; + for expected in 1..=5 { + let start = registry + .begin_recovery( + soul.clone(), + RecoveryTrigger::ProviderExit, + RecoveryPath::PristineSeed, + ) + .await + .unwrap(); + registry + .mark_stop_outcome( + start.context.prior_handle.incarnation_id().clone(), + StopOutcome::VerifiedEmpty, + ) + .await + .unwrap(); + let attempt = start.attempt_id.clone(); + let replacement = registry + .prepare_replacement(start, None, None, None, RecoveryPath::PristineSeed) + .await + .unwrap(); + let id = replacement.prepared.incarnation_id.clone(); + registry + .commit_created( + id.clone(), + BackendCreatedRecord { + daemon_id: DockerDaemonId::new(), + container_id: format!("{expected:064x}"), + image_ref: format!("sha256:{}", "b".repeat(64)), + runtime_dir: dir.path().join(format!("runtime-{expected}")), + host_binary_path: dir.path().join("host"), + immutable_config_digest: format!("sha256:{}", "c".repeat(64)), + }, + ) + .await + .unwrap(); + registry + .commit_execution_grant(id.clone(), HostBootId::new(), limits()) + .await + .unwrap(); + registry.mark_running(id.clone()).await.unwrap(); + registry + .mark_recovery_success(soul.clone(), id, attempt) + .await + .unwrap(); + assert_eq!( + registry + .recovery_context(soul.clone()) + .await + .unwrap() + .successful_recoveries_in_window, + expected + ); + } + assert!(matches!( + registry + .begin_recovery( + soul.clone(), + RecoveryTrigger::ProviderExit, + RecoveryPath::PristineSeed + ) + .await, + Err(RegistryError::RecoveryBlocked { .. }) + )); + drop(registry); + let cold = Registry::open(dir.path(), None).unwrap(); + assert!(matches!( + cold.begin_recovery( + soul.clone(), + RecoveryTrigger::StartupReconcile, + RecoveryPath::PristineSeed + ) + .await, + Err(RegistryError::RecoveryBlocked { .. }) + )); + let manual = cold + .begin_recovery( + soul, + RecoveryTrigger::ManualRetry, + RecoveryPath::PristineSeed, + ) + .await + .unwrap(); + assert_eq!( + manual.context.successful_recoveries_in_window, 5, + "pre-rearm context preserves history" + ); + assert_eq!( + cold.recovery_context(manual.context.soul_id) + .await + .unwrap() + .successful_recoveries_in_window, + 0 + ); + } use super::*; fn limits() -> RuntimeLimits { diff --git a/crates/freshell-supervisor/src/resume_catalog.rs b/crates/freshell-supervisor/src/resume_catalog.rs index 82ceb7866..d714eb8f0 100644 --- a/crates/freshell-supervisor/src/resume_catalog.rs +++ b/crates/freshell-supervisor/src/resume_catalog.rs @@ -342,6 +342,7 @@ mod tests { }, accepted_command_count: 2, completed_command_count: 1, + dispatched_command_count: 1, never_dispatched: false, recovery_window_started_at: None, successful_recoveries_in_window: 0, diff --git a/crates/freshell-supervisor/src/shell_recovery_tests.rs b/crates/freshell-supervisor/src/shell_recovery_tests.rs new file mode 100644 index 000000000..2ac082acc --- /dev/null +++ b/crates/freshell-supervisor/src/shell_recovery_tests.rs @@ -0,0 +1,461 @@ +struct ShellBackend(std::sync::Mutex); +#[async_trait::async_trait] +impl crate::backend::RuntimeBackend for ShellBackend { + async fn create_stopped( + &self, + _: &crate::backend::CreateRuntimeSpec, + ) -> Result { + panic!("no shell replacement permitted after dispatch") + } + async fn start_host(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + panic!("no new host") + } + async fn inspect( + &self, + _: &OwnedRuntimeHandle, + ) -> Result { + Ok(crate::backend::BackendInspection { + state: *self.0.lock().unwrap(), + configured_limits: checkpoints::tests::limits(), + }) + } + async fn request_stop(&self, _: &OwnedRuntimeHandle, _: u64) -> Result<(), BackendError> { + *self.0.lock().unwrap() = BackendRuntimeState::Exited; + Ok(()) + } + async fn force_stop(&self, h: &OwnedRuntimeHandle) -> Result<(), BackendError> { + self.request_stop(h, 0).await + } + async fn verify_empty(&self, _: &OwnedRuntimeHandle) -> Result { + Ok(matches!( + *self.0.lock().unwrap(), + BackendRuntimeState::Exited | BackendRuntimeState::Missing + )) + } + async fn list_known( + &self, + h: &[OwnedRuntimeHandle], + ) -> Result, BackendError> { + let mut v = vec![]; + for h in h { + v.push((h.incarnation_id().clone(), self.inspect(h).await?)); + } + Ok(v) + } + async fn read_limits( + &self, + _: &OwnedRuntimeHandle, + ) -> Result { + Ok(checkpoints::tests::limits()) + } + async fn enable_long_lived(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { + Ok(()) + } +} + +async fn shell_supervisor( + root: &std::path::Path, + state: BackendRuntimeState, + dispatched: bool, +) -> (Supervisor, RecoveryContext) { + let (registry, ctx) = checkpoints::tests::context(root, "shell", dispatched).await; + registry + .plan_or_apply_migration( + freshell_runtime_protocol::MigrationPlanRequest { + requested_mode: ManagedRolloutMode::ManagedOptIn, + apply: true, + backup_path: None, + legacy_metadata_path: None, + expected_control_epoch: None, + }, + true, + true, + ) + .await + .unwrap(); + let config = crate::service::SupervisorConfig { + runtime_root: root.join("runtime"), + control_socket_path: root.join("control.sock"), + host_binary_path: root.join("runtime/host"), + image_ref: format!("sha256:{}", "b".repeat(64)), + test_run_id: "shell-recovery".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: root.join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }; + ( + Supervisor::new( + registry, + std::sync::Arc::new(ShellBackend(std::sync::Mutex::new(state))), + config, + ) + .unwrap(), + ctx, + ) +} + +async fn shell_host( + ctx: &RecoveryContext, +) -> ( + tokio::sync::oneshot::Sender<()>, + tokio::task::JoinHandle, +) { + recovery_host( + ctx, + RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::NativeResume, + reason: "no identity".into(), + evidence: vec!["synthetic".into()], + store_state: EvidenceStoreState::Missing, + negative_paths: vec![], + }, + ) + .await +} + +async fn recovery_host( + ctx: &RecoveryContext, + probe: RecoveryProbe, +) -> ( + tokio::sync::oneshot::Sender<()>, + tokio::task::JoinHandle, +) { + recovery_host_status(ctx, probe, true, Some(123)).await +} + +async fn recovery_host_status( + ctx: &RecoveryContext, + probe: RecoveryProbe, + exited: bool, + worker_pid: Option, +) -> ( + tokio::sync::oneshot::Sender<()>, + tokio::task::JoinHandle, +) { + let runtime = ctx.prior_handle.runtime_dir(); + let secret = b"synthetic-host-secret"; + std::fs::write(runtime.join("secret"), secret).unwrap(); + let listener = tokio::net::UnixListener::bind(runtime.join("host.sock")).unwrap(); + let boot = freshell_runtime_protocol::HostBootId::new(); + let native_session_id = ctx.native_session_id.clone(); + let (tx, mut rx) = tokio::sync::oneshot::channel(); + let task = tokio::spawn(async move { + let mut native_probes = 0; + loop { + let socket = tokio::select! { _=&mut rx => break, v=listener.accept()=>v.unwrap().0 }; + let mut socket = socket; + let envelope: freshell_runtime_protocol::Envelope = + freshell_runtime_protocol::read_frame(&mut socket) + .await + .unwrap(); + let result = match envelope.body { + HostCommand::Hello { + incarnation_id, + challenge, + } => HostResult::Hello { + host_boot_id: boot.clone(), + proof: freshell_runtime_protocol::host_proof( + secret, + &challenge, + &boot, + &incarnation_id, + ), + capabilities: vec![], + effective_limits: checkpoints::tests::limits(), + }, + HostCommand::Status { .. } => HostResult::Status { + host_boot_id: boot.clone(), + worker_pid, + worker_launch_count: 1, + max_control_epoch: 1, + max_execution_generation: 1, + fixture_evidence: serde_json::json!({}), + exited, + native_session_id: native_session_id.clone(), + }, + HostCommand::ProbeRecovery { .. } => { + native_probes += 1; + HostResult::RecoveryProbe(probe.clone()) + } + HostCommand::Stop { .. } => HostResult::Stopped, + other => panic!("unexpected shell command {other:?}"), + }; + freshell_runtime_protocol::write_frame( + &mut socket, + &freshell_runtime_protocol::HostReply { + request_id: envelope.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + native_probes + }); + (tx, task) +} + +#[tokio::test] +async fn recovery_online_without_worker_does_not_claim_live() { + for worker_pid in [Some(123), None] { + let temp = tempfile::tempdir().unwrap(); + let (supervisor, ctx) = + shell_supervisor(temp.path(), BackendRuntimeState::Running, true).await; + let output = ctx.prior_handle.runtime_dir().join("retained-output"); + std::fs::write(&output, b"retained startup diagnostic").unwrap(); + let probe = RecoveryProbe::Blocked { + path: RecoveryPath::NativeResume, + reason: RecoveryBlockReason::AmbiguousIdentity, + retry_hint: RetryHint { + automatic_after_ms: None, + manual_retry: true, + repair: None, + }, + evidence: vec!["unrecorded worker".into()], + }; + let (shutdown, task) = recovery_host_status(&ctx, probe, false, worker_pid).await; + let result = supervisor + .recover(RecoverRequest { + soul_id: ctx.soul_id.clone(), + trigger: RecoveryTrigger::ProviderExit, + expected_intent_revision: None, + expected_control_epoch: None, + }) + .await; + shutdown.send(()).unwrap(); + assert_eq!(task.await.unwrap(), 0); + let result = result.unwrap(); + assert_eq!( + result.outcome, + if worker_pid.is_some() { + RecoveryOutcome::Reattached + } else { + RecoveryOutcome::Blocked + }, + "a live host without a worker cannot prove reattachment" + ); + assert_eq!( + result.view.incarnation_id, + *ctx.prior_handle.incarnation_id() + ); + assert!(result.incident_id.is_none()); + assert_eq!( + result.view.cleanup_state, + freshell_runtime_protocol::CleanupState::None + ); + assert_eq!( + std::fs::read(output).unwrap(), + b"retained startup diagnostic" + ); + assert_eq!( + supervisor + .backend + .inspect(&ctx.prior_handle) + .await + .unwrap() + .state, + BackendRuntimeState::Running + ); + assert!(!supervisor + .backend + .verify_empty(&ctx.prior_handle) + .await + .unwrap()); + assert_eq!(supervisor.registry.inventory().await.unwrap().len(), 1); + } +} + +async fn shell_ended(online: bool) { + let temp = tempfile::tempdir().unwrap(); + let (supervisor, ctx) = shell_supervisor( + temp.path(), + if online { + BackendRuntimeState::Running + } else { + BackendRuntimeState::Exited + }, + true, + ) + .await; + let output = ctx.prior_handle.runtime_dir().join("retained-output"); + std::fs::write(&output, b"last-shell-diagnostic").unwrap(); + let host = if online { + Some(shell_host(&ctx).await) + } else { + None + }; + let result = supervisor + .recover(RecoverRequest { + soul_id: ctx.soul_id.clone(), + trigger: RecoveryTrigger::ProviderExit, + expected_intent_revision: None, + expected_control_epoch: None, + }) + .await; + if let Some((shutdown, task)) = host { + shutdown.send(()).unwrap(); + assert_eq!( + task.await.unwrap(), + 0, + "shell has no native/checkpoint recovery paths" + ); + } + let result = result.unwrap(); + assert_eq!(result.outcome, RecoveryOutcome::Lost); + let certificate = supervisor + .registry + .loss_certificate(result.incident_id.unwrap()) + .await + .unwrap(); + assert_eq!( + certificate.decision.state, + freshell_runtime_protocol::LossDecisionState::NonResumableTerminalEnded + ); + assert_eq!(certificate.recovery_paths.len(), 2); + assert!(certificate + .recovery_paths + .iter() + .all(|p| matches!(p.path, RecoveryPath::Reattach | RecoveryPath::PristineSeed))); + assert_eq!(std::fs::read(output).unwrap(), b"last-shell-diagnostic"); + assert_eq!(supervisor.registry.inventory().await.unwrap().len(), 1); +} + +#[tokio::test] +async fn shell_dispatched_verified_end_online() { + shell_ended(true).await; +} +#[tokio::test] +async fn shell_dispatched_verified_end_offline() { + shell_ended(false).await; +} +#[tokio::test] +async fn shell_uncertain_end_blocks() { + let temp = tempfile::tempdir().unwrap(); + let (s, ctx) = shell_supervisor(temp.path(), BackendRuntimeState::Restarting, true).await; + let result = s + .recover(RecoverRequest { + soul_id: ctx.soul_id, + trigger: RecoveryTrigger::ProviderExit, + expected_intent_revision: None, + expected_control_epoch: None, + }) + .await + .unwrap(); + assert_eq!(result.outcome, RecoveryOutcome::Blocked); + assert!(result.incident_id.is_none()); + assert_eq!( + result.view.cleanup_state, + freshell_runtime_protocol::CleanupState::None + ); +} +#[tokio::test] +async fn shell_never_dispatched_preserves_pristine_seed() { + let temp = tempfile::tempdir().unwrap(); + let (s, ctx) = shell_supervisor(temp.path(), BackendRuntimeState::Exited, false).await; + assert!( + matches!(s.probe_recovery(ctx.soul_id).await.unwrap(),RecoveryProbe::PristineSeedReady {seed,..} if seed=="original-seed") + ); +} + +#[tokio::test] +async fn recovery_inspected_sqlite_absence_certifies_loss_and_preserves_output() { + let temp = tempfile::tempdir().unwrap(); + let (registry, ctx) = checkpoints::tests::context(temp.path(), "opencode", true).await; + let db = temp.path().join("effective.db"); + let connection = rusqlite::Connection::open(&db).unwrap(); + connection + .execute("CREATE TABLE session(id TEXT PRIMARY KEY)", []) + .unwrap(); + drop(connection); + let mut spec = checkpoints::tests::spec("opencode"); + spec.environment + .insert("OPENCODE_DB".into(), db.display().to_string()); + registry + .store_verified_resume_spec(ctx.soul_id.clone(), spec.clone()) + .await + .unwrap(); + let inspected = freshell_agent_runtime::probe_provider_store(temp.path(), &spec); + let freshell_agent_runtime::ProviderStoreProbe::DefinitivelyUnavailable { + reason, + evidence, + store_state, + negative_paths, + } = inspected + else { + panic!("{inspected:?}") + }; + let probe = RecoveryProbe::DefinitivelyUnavailable { + path: RecoveryPath::NativeResume, + reason, + evidence, + store_state, + negative_paths, + }; + registry + .plan_or_apply_migration( + freshell_runtime_protocol::MigrationPlanRequest { + requested_mode: ManagedRolloutMode::ManagedOptIn, + apply: true, + backup_path: None, + legacy_metadata_path: None, + expected_control_epoch: None, + }, + true, + true, + ) + .await + .unwrap(); + let config = crate::service::SupervisorConfig { + runtime_root: temp.path().join("runtime"), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: ctx.prior_handle.host_binary_path().to_path_buf(), + image_ref: ctx.prior_handle.image_ref().into(), + test_run_id: "inspected-loss".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: Default::default(), + }; + let supervisor = Supervisor::new( + registry, + std::sync::Arc::new(ShellBackend(std::sync::Mutex::new( + BackendRuntimeState::Running, + ))), + config, + ) + .unwrap(); + let output = ctx.prior_handle.runtime_dir().join("retained-output"); + std::fs::write(&output, b"last-provider-diagnostic").unwrap(); + let (shutdown, task) = recovery_host(&ctx, probe).await; + let result = supervisor + .recover(RecoverRequest { + soul_id: ctx.soul_id, + trigger: RecoveryTrigger::ProviderExit, + expected_intent_revision: None, + expected_control_epoch: None, + }) + .await; + shutdown.send(()).unwrap(); + assert!(task.await.unwrap() > 0); + let result = result.unwrap(); + assert_eq!(result.outcome, RecoveryOutcome::Lost); + let certificate = supervisor + .registry + .loss_certificate(result.incident_id.unwrap()) + .await + .unwrap(); + assert_eq!( + certificate.decision.state, + freshell_runtime_protocol::LossDecisionState::Lost + ); + assert_eq!(certificate.recovery_paths.len(), 4); + assert!(certificate + .recovery_paths + .iter() + .filter(|row| matches!( + row.path, + RecoveryPath::NativeResume | RecoveryPath::CheckpointRestore + )) + .all(|row| row.store_state == EvidenceStoreState::PresentReadable)); + assert_eq!(std::fs::read(output).unwrap(), b"last-provider-diagnostic"); + assert_eq!(supervisor.registry.inventory().await.unwrap().len(), 1); +} From d004655a3a3c4bbd5ccf07f0e36034e9cad9a6d4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:03:32 -0700 Subject: [PATCH 229/493] fix: preserve readable large provider transcript recovery --- crates/freshell-agent-runtime/src/lib.rs | 166 ++++++++++++++++++++++- 1 file changed, 161 insertions(+), 5 deletions(-) diff --git a/crates/freshell-agent-runtime/src/lib.rs b/crates/freshell-agent-runtime/src/lib.rs index cd527d51f..a82abaacb 100644 --- a/crates/freshell-agent-runtime/src/lib.rs +++ b/crates/freshell-agent-runtime/src/lib.rs @@ -15,6 +15,7 @@ use serde::{Deserialize, Serialize}; use std::{ collections::{BTreeMap, VecDeque}, fs, + io::Read, path::Path, }; @@ -1324,20 +1325,36 @@ fn probe_bounded_tree(root: &Path, expected: &str) -> ProviderStoreProbe { } continue; } - if !metadata.is_file() || metadata.len() > 1024 * 1024 { + if !metadata.is_file() { continue; } - let Ok(bytes) = fs::read(&path) else { - continue; - }; let filename_match = path .file_name() .and_then(|name| name.to_str()) .is_some_and(|name| name.contains(expected)); + if filename_match { + // Exact filename recovery needs readability, not a complete + // transcript scan. Keep this check bounded as transcripts grow. + if fs::File::open(&path) + .and_then(|mut file| file.read(&mut [0_u8; 1])) + .is_ok() + { + return ProviderStoreProbe::Ready { + evidence: vec![path.display().to_string()], + }; + } + continue; + } + if metadata.len() > 1024 * 1024 { + continue; + } + let Ok(bytes) = fs::read(&path) else { + continue; + }; let content_match = bytes .windows(expected.len()) .any(|w| w == expected.as_bytes()); - if filename_match || content_match { + if content_match { return ProviderStoreProbe::Ready { evidence: vec![path.display().to_string()], }; @@ -1769,6 +1786,145 @@ mod tests { ); } + #[test] + fn recovery_store_large_exact_claude_transcript_remains_ready() { + assert_large_exact_transcript_ready("claude"); + } + + #[test] + fn recovery_store_large_exact_codex_transcript_remains_ready() { + assert_large_exact_transcript_ready("codex"); + } + + fn assert_large_exact_transcript_ready(provider: &str) { + let home = tempfile::tempdir().unwrap(); + let root = home.path().join(if provider == "claude" { + ".claude/projects" + } else { + ".codex/sessions" + }); + fs::create_dir_all(&root).unwrap(); + let path = root.join("exact.jsonl"); + fs::write(&path, br#"{"sessionId":"exact"}"#).unwrap(); + fs::OpenOptions::new() + .write(true) + .open(&path) + .unwrap() + .set_len(1024 * 1024 + 1) + .unwrap(); + let result = probe_provider_store( + home.path(), + &resume_spec_for_test(provider, "exact", home.path()), + ); + match result { + ProviderStoreProbe::Ready { evidence } => { + assert_eq!(evidence, vec![path.display().to_string()]); + } + other => panic!("readable large {provider} exact transcript must recover: {other:?}"), + } + } + + #[cfg(unix)] + #[test] + fn recovery_store_exact_symlink_and_directory_candidates_stay_unknown() { + use std::os::unix::fs::symlink; + + let home = tempfile::tempdir().unwrap(); + let outside = home.path().join("outside.jsonl"); + fs::write(&outside, br#"{"sessionId":"exact"}"#).unwrap(); + for provider in ["claude", "codex"] { + let root = home.path().join(if provider == "claude" { + ".claude/projects" + } else { + ".codex/sessions" + }); + fs::create_dir_all(root.join("exact-directory")).unwrap(); + symlink(&outside, root.join("exact.jsonl")).unwrap(); + assert!(matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test(provider, "exact", home.path()) + ), + ProviderStoreProbe::Blocked { + reason: RecoveryBlockReason::AmbiguousIdentity, + .. + } + )); + } + } + + #[test] + fn recovery_store_large_nonmatching_transcript_content_stays_unknown() { + let home = tempfile::tempdir().unwrap(); + for provider in ["claude", "codex"] { + let root = home.path().join(if provider == "claude" { + ".claude/projects" + } else { + ".codex/sessions" + }); + fs::create_dir_all(&root).unwrap(); + let path = root.join("different.jsonl"); + fs::write(&path, br#"{"sessionId":"exact"}"#).unwrap(); + fs::OpenOptions::new() + .write(true) + .open(&path) + .unwrap() + .set_len(1024 * 1024 + 1) + .unwrap(); + assert!(matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test(provider, "exact", home.path()) + ), + ProviderStoreProbe::Blocked { + reason: RecoveryBlockReason::AmbiguousIdentity, + .. + } + )); + } + } + + #[cfg(unix)] + #[test] + fn recovery_store_unreadable_exact_transcripts_block_without_certifying_absence() { + use std::os::unix::fs::PermissionsExt; + + let home = tempfile::tempdir().unwrap(); + for provider in ["claude", "codex"] { + let root = home.path().join(if provider == "claude" { + ".claude/projects" + } else { + ".codex/sessions" + }); + fs::create_dir_all(&root).unwrap(); + let path = root.join("exact.jsonl"); + fs::write(&path, br#"{"sessionId":"exact"}"#).unwrap(); + let file = fs::OpenOptions::new().write(true).open(&path).unwrap(); + fs::set_permissions(&path, fs::Permissions::from_mode(0o000)).unwrap(); + for size in [24, 1024 * 1024 + 1] { + file.set_len(size).unwrap(); + assert_eq!( + fs::File::open(&path).unwrap_err().kind(), + std::io::ErrorKind::PermissionDenied, + "synthetic unreadable fixture requires unprivileged execution" + ); + assert!( + matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test(provider, "exact", home.path()) + ), + ProviderStoreProbe::Blocked { + reason: RecoveryBlockReason::AmbiguousIdentity, + .. + } + ), + "{provider} unreadable exact transcript of {size} bytes" + ); + } + } + } + #[test] fn recovery_store_incomplete_scan_blocks_but_exact_positive_survives() { let home = tempfile::tempdir().unwrap(); From ce9c9470a76e42695dbc228f7138a57f292ddf7f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:26:58 -0700 Subject: [PATCH 230/493] fix: preserve readable large Amplifier recovery artifacts --- crates/freshell-agent-runtime/src/lib.rs | 250 +++++++++++++++++++++-- 1 file changed, 233 insertions(+), 17 deletions(-) diff --git a/crates/freshell-agent-runtime/src/lib.rs b/crates/freshell-agent-runtime/src/lib.rs index a82abaacb..62f124311 100644 --- a/crates/freshell-agent-runtime/src/lib.rs +++ b/crates/freshell-agent-runtime/src/lib.rs @@ -1066,6 +1066,8 @@ fn probe_opencode(provider_home: &Path, expected: &str, spec: &ResumeSpec) -> Pr } fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { + const MAX_INSPECTED_METADATA_BYTES: u64 = 1024 * 1024; + let root = provider_home.join(".amplifier/projects"); // This tree alone does not establish all effective Amplifier stores. // Exact readable positives are usable; an unrecorded store is unknown. @@ -1153,7 +1155,8 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { ) } }; - let mut materialized_backup = false; + let mut materialized = false; + let mut events_scan_incomplete = false; let mut identity_recorded = false; for entry in entries { let entry = match entry { @@ -1177,11 +1180,7 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { continue; } let metadata = match fs::symlink_metadata(&path) { - Ok(metadata) - if metadata.is_file() - && !metadata.file_type().is_symlink() - && metadata.len() <= 1024 * 1024 => - { + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => { metadata } _ => { @@ -1193,7 +1192,30 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { ) } }; - let bytes = match fs::read(&path) { + // Exact materialized history needs readability, not a whole-file + // parse. Only metadata carries the bounded identity document. + let inspection = if name.starts_with("metadata.json") { + if metadata.len() > MAX_INSPECTED_METADATA_BYTES { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "Amplifier session metadata exceeds the bounded identity probe", + vec![path.display().to_string()], + None, + ); + } + let mut bytes = Vec::new(); + fs::File::open(&path) + .and_then(|file| { + file.take(MAX_INSPECTED_METADATA_BYTES + 1) + .read_to_end(&mut bytes) + }) + .map(|_| bytes) + } else { + fs::File::open(&path) + .and_then(|mut file| file.read(&mut [0_u8; 1])) + .map(|_| Vec::new()) + }; + let bytes = match inspection { Ok(bytes) => bytes, Err(error) => { return blocked( @@ -1204,13 +1226,21 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { ) } }; - if name != "transcript.jsonl" - && name.starts_with("transcript.jsonl") - && metadata.len() > 0 - { - materialized_backup = true; + if name.starts_with("transcript.jsonl") && metadata.len() > 0 { + materialized = true; + } + if name == "events.jsonl" && metadata.len() > MAX_INSPECTED_METADATA_BYTES { + events_scan_incomplete = true; } if name.starts_with("metadata.json") { + if bytes.len() as u64 > MAX_INSPECTED_METADATA_BYTES { + return blocked( + RecoveryBlockReason::StoreUnreadable, + "Amplifier session metadata exceeds the bounded identity probe", + vec![path.display().to_string()], + None, + ); + } match serde_json::from_slice::(&bytes) { Ok(value) => { if let Some(actual) = @@ -1226,8 +1256,8 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { } identity_recorded = true; } - if name != "metadata.json" && value.get("turn_count").is_some() { - materialized_backup = true; + if value.get("turn_count").is_some() { + materialized = true; } } Err(_) => { @@ -1249,9 +1279,18 @@ fn probe_amplifier(provider_home: &Path, expected: &str) -> ProviderStoreProbe { None, ); } - if !materialized_backup - && freshell_sessions::amplifier_stub::stub_is_unused(session_dir) - { + // The stub predicate scans events for prompt:submit only when no + // recorded turn or transcript has already established materialization. + // An oversized unscanned event log cannot prove a pristine stub. + if !materialized && events_scan_incomplete { + return blocked( + RecoveryBlockReason::AmbiguousIdentity, + "Amplifier event history exceeds the bounded pristine-stub inspection", + vec![session_dir.join("events.jsonl").display().to_string()], + None, + ); + } + if !materialized && freshell_sessions::amplifier_stub::stub_is_unused(session_dir) { if incomplete { return blocked( RecoveryBlockReason::AmbiguousIdentity, @@ -1956,6 +1995,183 @@ mod tests { } } + #[test] + fn recovery_store_amplifier_large_primary_transcript_remains_ready() { + for recorded_turn in [false, true] { + assert_large_amplifier_artifact_ready("transcript.jsonl", recorded_turn); + } + } + + #[test] + fn recovery_store_amplifier_large_events_with_recorded_turn_remain_ready() { + assert_large_amplifier_artifact_ready("events.jsonl", true); + } + + #[test] + fn recovery_store_amplifier_large_backup_transcripts_remain_ready() { + for name in ["transcript.jsonl.backup", "transcript.jsonl.bak"] { + assert_large_amplifier_artifact_ready(name, false); + } + } + + fn amplifier_session_fixture(home: &Path, recorded_turn: bool) -> std::path::PathBuf { + let session = home.join(".amplifier/projects/project/sessions/exact"); + fs::create_dir_all(&session).unwrap(); + let metadata = if recorded_turn { + br#"{"session_id":"exact","turn_count":1}"#.as_slice() + } else { + br#"{"session_id":"exact"}"#.as_slice() + }; + fs::write(session.join("metadata.json"), metadata).unwrap(); + fs::write(session.join("transcript.jsonl"), b"").unwrap(); + fs::write(session.join("events.jsonl"), b"").unwrap(); + session + } + + fn assert_large_amplifier_artifact_ready(name: &str, recorded_turn: bool) { + let home = tempfile::tempdir().unwrap(); + let session = amplifier_session_fixture(home.path(), recorded_turn); + let path = session.join(name); + let file = fs::File::create(&path).unwrap(); + file.set_len(1024 * 1024 + 1).unwrap(); + let result = probe_provider_store( + home.path(), + &resume_spec_for_test("amplifier", "exact", home.path()), + ); + match result { + ProviderStoreProbe::Ready { evidence } => { + assert_eq!( + evidence, + vec![session.display().to_string(), "sessionId=exact".into()] + ); + } + other => panic!("readable large Amplifier {name} must recover: {other:?}"), + } + } + + #[cfg(unix)] + #[test] + fn recovery_store_amplifier_unreadable_artifacts_block_without_absence() { + use std::os::unix::fs::PermissionsExt; + + for name in [ + "transcript.jsonl", + "events.jsonl", + "transcript.jsonl.backup", + ] { + let home = tempfile::tempdir().unwrap(); + let session = amplifier_session_fixture(home.path(), true); + let path = session.join(name); + let file = fs::File::create(&path).unwrap(); + fs::set_permissions(&path, fs::Permissions::from_mode(0o000)).unwrap(); + for size in [24, 1024 * 1024 + 1] { + file.set_len(size).unwrap(); + assert_eq!( + fs::File::open(&path).unwrap_err().kind(), + std::io::ErrorKind::PermissionDenied, + "synthetic unreadable fixture requires unprivileged execution" + ); + assert!( + matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test("amplifier", "exact", home.path()) + ), + ProviderStoreProbe::Blocked { + reason: RecoveryBlockReason::StoreUnreadable, + .. + } + ), + "unreadable {name} of {size} bytes cannot certify readiness or absence" + ); + } + } + } + + #[test] + fn recovery_store_amplifier_incomplete_identity_and_events_stay_unknown() { + for artifact in ["metadata.json", "metadata.json.backup", "events.jsonl"] { + let home = tempfile::tempdir().unwrap(); + let session = amplifier_session_fixture(home.path(), artifact != "events.jsonl"); + let path = session.join(artifact); + fs::File::create(&path) + .unwrap() + .set_len(1024 * 1024 + 1) + .unwrap(); + assert!( + matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test("amplifier", "exact", home.path()) + ), + ProviderStoreProbe::Blocked { .. } + ), + "incomplete {artifact} inspection cannot certify readiness or absence" + ); + } + for metadata in [ + br#"{"session_id":"wrong","turn_count":1}"#.as_slice(), + b"{}", + b"invalid", + ] { + let home = tempfile::tempdir().unwrap(); + let session = amplifier_session_fixture(home.path(), true); + fs::write(session.join("metadata.json"), metadata).unwrap(); + fs::File::create(session.join("transcript.jsonl")) + .unwrap() + .set_len(1024 * 1024 + 1) + .unwrap(); + assert!( + matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test("amplifier", "exact", home.path()) + ), + ProviderStoreProbe::Blocked { .. } + ), + "large readable history does not establish metadata identity" + ); + } + } + + #[cfg(unix)] + #[test] + fn recovery_store_amplifier_symlink_and_nonregular_artifacts_stay_unknown() { + use std::os::unix::fs::symlink; + + for name in [ + "events.jsonl", + "transcript.jsonl.backup", + "metadata.json.backup", + ] { + for linked in [false, true] { + let home = tempfile::tempdir().unwrap(); + let session = amplifier_session_fixture(home.path(), true); + let path = session.join(name); + if name == "events.jsonl" { + fs::remove_file(&path).unwrap(); + } + if linked { + let outside = home.path().join("outside"); + fs::write(&outside, br#"{"session_id":"exact","turn_count":1}"#).unwrap(); + symlink(&outside, &path).unwrap(); + } else { + fs::create_dir(&path).unwrap(); + } + assert!(matches!( + probe_provider_store( + home.path(), + &resume_spec_for_test("amplifier", "exact", home.path()) + ), + ProviderStoreProbe::Blocked { + reason: RecoveryBlockReason::StoreUnreadable, + .. + } + )); + } + } + } + #[test] fn recovery_store_amplifier_backup_transcript_preserves_materialized_session() { let home = tempfile::tempdir().unwrap(); From 1d97b13c766713589e0df258abf40d30fcedd6b6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:01:28 -0700 Subject: [PATCH 231/493] test: cover offline recovery without verified provider metadata --- .../src/checkpoint_tests.rs | 21 ++- .../src/shell_recovery_tests.rs | 164 +++++++++++++++++- 2 files changed, 174 insertions(+), 11 deletions(-) diff --git a/crates/freshell-supervisor/src/checkpoint_tests.rs b/crates/freshell-supervisor/src/checkpoint_tests.rs index 00fcbac9f..531b15416 100644 --- a/crates/freshell-supervisor/src/checkpoint_tests.rs +++ b/crates/freshell-supervisor/src/checkpoint_tests.rs @@ -18,6 +18,21 @@ pub(crate) async fn context( root: &std::path::Path, provider: &str, dispatched: bool, +) -> (Registry, RecoveryContext) { + context_with_native_identity( + root, + provider, + dispatched, + (provider != "shell").then_some("exact-session"), + ) + .await +} + +pub(crate) async fn context_with_native_identity( + root: &std::path::Path, + provider: &str, + dispatched: bool, + native_session_id: Option<&str>, ) -> (Registry, RecoveryContext) { let registry = Registry::open(root.join("registry"), None).unwrap(); let soul = SoulId::new(); @@ -26,11 +41,7 @@ pub(crate) async fn context( soul_id: soul.clone(), provider: provider.into(), provider_store_id: "store".into(), - native_session_id: if provider == "shell" { - None - } else { - Some("exact-session".into()) - }, + native_session_id: native_session_id.map(str::to_owned), creation_seed_ref: "original-seed".into(), request_id: RequestId::new(), payload_digest: "fixture".into(), diff --git a/crates/freshell-supervisor/src/shell_recovery_tests.rs b/crates/freshell-supervisor/src/shell_recovery_tests.rs index 2ac082acc..161d38b72 100644 --- a/crates/freshell-supervisor/src/shell_recovery_tests.rs +++ b/crates/freshell-supervisor/src/shell_recovery_tests.rs @@ -1,11 +1,11 @@ -struct ShellBackend(std::sync::Mutex); +struct RecoveryBackend(std::sync::Mutex); #[async_trait::async_trait] -impl crate::backend::RuntimeBackend for ShellBackend { +impl crate::backend::RuntimeBackend for RecoveryBackend { async fn create_stopped( &self, _: &crate::backend::CreateRuntimeSpec, ) -> Result { - panic!("no shell replacement permitted after dispatch") + panic!("no replacement permitted after dispatch") } async fn start_host(&self, _: &OwnedRuntimeHandle) -> Result<(), BackendError> { panic!("no new host") @@ -59,6 +59,15 @@ async fn shell_supervisor( dispatched: bool, ) -> (Supervisor, RecoveryContext) { let (registry, ctx) = checkpoints::tests::context(root, "shell", dispatched).await; + recovery_supervisor(root, registry, ctx, state).await +} + +async fn recovery_supervisor( + root: &std::path::Path, + registry: crate::registry::Registry, + ctx: RecoveryContext, + state: BackendRuntimeState, +) -> (Supervisor, RecoveryContext) { registry .plan_or_apply_migration( freshell_runtime_protocol::MigrationPlanRequest { @@ -78,7 +87,7 @@ async fn shell_supervisor( control_socket_path: root.join("control.sock"), host_binary_path: root.join("runtime/host"), image_ref: format!("sha256:{}", "b".repeat(64)), - test_run_id: "shell-recovery".into(), + test_run_id: "recovery-fixture".into(), control_secret: "synthetic-control-secret".into(), lifecycle_log: root.join("lifecycle.jsonl"), admission: crate::admission::AdmissionPolicy::default(), @@ -86,7 +95,7 @@ async fn shell_supervisor( ( Supervisor::new( registry, - std::sync::Arc::new(ShellBackend(std::sync::Mutex::new(state))), + std::sync::Arc::new(RecoveryBackend(std::sync::Mutex::new(state))), config, ) .unwrap(), @@ -94,6 +103,149 @@ async fn shell_supervisor( ) } +async fn offline_coding_provider_without_verified_resume(native_session_id: Option<&str>) { + let temp = tempfile::tempdir().unwrap(); + let (registry, ctx) = checkpoints::tests::context_with_native_identity( + temp.path(), + "opencode", + true, + native_session_id, + ) + .await; + assert_eq!(ctx.native_session_id.as_deref(), native_session_id); + assert!(ctx.resume_spec.is_none()); + assert_eq!(ctx.checkpoint_revision, 0); + assert_eq!(ctx.dispatched_command_count, 1); + assert!(!ctx.never_dispatched); + let (supervisor, ctx) = + recovery_supervisor(temp.path(), registry, ctx, BackendRuntimeState::Exited).await; + let output = ctx.prior_handle.runtime_dir().join("retained-output"); + std::fs::write(&output, b"provider exited before verified identity capture").unwrap(); + let before = supervisor.registry.inventory().await.unwrap(); + let connection = rusqlite::Connection::open(supervisor.registry.database_path()).unwrap(); + let retained_input = || { + connection + .query_row( + "SELECT protected_payload,state,incarnation_id FROM input_commands WHERE soul_id=?1", + [ctx.soul_id.as_str()], + |row| { + Ok(( + row.get::<_, Vec>(0)?, + row.get::<_, String>(1)?, + row.get::<_, String>(2)?, + )) + }, + ) + .unwrap() + }; + let input_before = retained_input(); + assert_eq!(input_before.0, b"synthetic-input"); + assert_eq!(input_before.1, "dispatching"); + let result = supervisor + .recover(RecoverRequest { + soul_id: ctx.soul_id.clone(), + trigger: RecoveryTrigger::ProviderExit, + expected_intent_revision: None, + expected_control_epoch: None, + }) + .await + .unwrap(); + + assert_eq!(result.outcome, RecoveryOutcome::Blocked); + let Some(RecoveryProbe::Blocked { + path, + reason, + retry_hint, + evidence, + }) = result.probe + else { + panic!( + "offline missing metadata must remain unknown: {:?}", + result.probe + ); + }; + assert_eq!(path, RecoveryPath::NativeResume); + assert_eq!(reason, RecoveryBlockReason::AmbiguousIdentity); + assert!(retry_hint.manual_retry); + assert!(retry_hint.automatic_after_ms.is_none()); + assert!(retry_hint.repair.is_some_and(|repair| !repair.is_empty())); + assert!(!evidence.is_empty()); + assert_eq!(result.view.soul_id, ctx.soul_id); + assert_eq!( + result.view.incarnation_id, + *ctx.prior_handle.incarnation_id() + ); + assert_eq!(result.view.container_id, before[0].container_id); + assert_eq!( + result.view.execution_generation, + before[0].execution_generation + ); + assert_eq!(result.view.native_session_id.as_deref(), native_session_id); + assert_eq!( + result.view.recovery_state, + freshell_runtime_protocol::RecoveryState::Blocked + ); + assert_eq!( + result.view.cleanup_state, + freshell_runtime_protocol::CleanupState::None + ); + assert!(result.incident_id.is_none()); + assert!(result.view.incident_id.is_none()); + assert!(result.attempt_id.is_none()); + assert!(result.prior_incarnation_id.is_none()); + assert!(supervisor + .registry + .unresolved_loss_incident_ids() + .await + .unwrap() + .is_empty()); + assert_eq!( + connection + .query_row("SELECT COUNT(*) FROM loss_incidents", [], |row| row + .get::<_, u64>(0)) + .unwrap(), + 0 + ); + let after = supervisor.registry.inventory().await.unwrap(); + assert_eq!(after.len(), 1, "recovery must not create a replacement"); + assert_eq!(after[0], result.view); + let retained = supervisor + .registry + .recovery_context(ctx.soul_id.clone()) + .await + .unwrap(); + assert_eq!( + retained.prior_handle.incarnation_id(), + ctx.prior_handle.incarnation_id() + ); + assert_eq!( + retained.prior_handle.container_id(), + ctx.prior_handle.container_id() + ); + assert_eq!( + retained.prior_handle.runtime_dir(), + ctx.prior_handle.runtime_dir() + ); + assert_eq!(retained.dispatched_command_count, 1); + assert_eq!(retained.native_session_id.as_deref(), native_session_id); + assert!(retained.resume_spec.is_none()); + assert_eq!(retained_input(), input_before); + assert_eq!( + std::fs::read(output).unwrap(), + b"provider exited before verified identity capture" + ); +} + +#[tokio::test] +async fn recovery_offline_dispatched_without_identity_blocks_and_preserves_data() { + offline_coding_provider_without_verified_resume(None).await; +} + +#[tokio::test] +async fn recovery_offline_identity_without_verified_resume_blocks_and_preserves_data() { + offline_coding_provider_without_verified_resume(Some("exact-session")).await; +} + async fn shell_host( ctx: &RecoveryContext, ) -> ( @@ -417,7 +569,7 @@ async fn recovery_inspected_sqlite_absence_certifies_loss_and_preserves_output() }; let supervisor = Supervisor::new( registry, - std::sync::Arc::new(ShellBackend(std::sync::Mutex::new( + std::sync::Arc::new(RecoveryBackend(std::sync::Mutex::new( BackendRuntimeState::Running, ))), config, From 8f69ef5bc3e895f8c5fa465bf21150bb255cd3a6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:15:11 -0700 Subject: [PATCH 232/493] docs: record inspected recovery task completion --- .../2026-10-05-managed-provider-environment.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index ecac9c298..4443fe8d1 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -274,15 +274,15 @@ pub fn resolve_file_references(raw: &str, origin: FileSourceOrigin<'_>, **Interfaces:** Add defaulted `negative_paths: Vec` to definitive-negative variants of both `ProviderStoreProbe` and `RecoveryProbe`. Reuse existing `RecoveryPathEvidence`, `RecoveryPath`, `EvidenceStoreState`; no string-based evidence inference. Metadata-only after-dispatch failures return existing Blocked/AmbiguousIdentity with manual-retry repair, not Missing. -- [ ] **Step 1: Add failing decisions.** For coding providers with required NativeResume/CheckpointRestore paths, host dispatched-journal/no resume identity returns Blocked and preserves data; supervisor zero checkpoint revision is unknown/blocked. Managed shell has only Reattach and PristineSeed: after verified actual process end and dispatched input, online/offline recovery must reach `NonResumableTerminalEnded`, retain output and never fabricate native/checkpoint evidence. Uncertain process termination remains blocked/unsettled; proven never-dispatched shell still uses its existing pristine-seed-only policy. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. +- [x] **Step 1: Add failing decisions.** For coding providers with required NativeResume/CheckpointRestore paths, host dispatched-journal/no resume identity returns Blocked and preserves data; supervisor zero checkpoint revision is unknown/blocked. Managed shell has only Reattach and PristineSeed: after verified actual process end and dispatched input, online/offline recovery must reach `NonResumableTerminalEnded`, retain output and never fabricate native/checkpoint evidence. Uncertain process termination remains blocked/unsettled; proven never-dispatched shell still uses its existing pristine-seed-only policy. Reason text containing `checkpoint` or a checkpoint-looking evidence pathname does not authorize loss. Legacy empty negative rows block. Test live and offline missing metadata, unreadable/malformed store, duplicate/conflicting rows, independently present fixture checkpoint, and genuinely absent native/checkpoint store. Real SQLite exact-row absence with adapter-declared shared native/checkpoint storage still permits the existing loss decision. Startup provider fixture exits before identity and must retain its diagnostic, not replace a dispatched conversation. Preserve proven never-dispatched pristine-seed success separately. Exercise initialization dispatch before ID capture, actor-ID acknowledgement loss and natural retry exhaustion followed by cold hydration; manually forced markers do not prove budget behavior. Name host cases `probe_recovery::{dispatched_without_identity_blocks_and_preserves_data, startup_before_identity_retains_diagnostic, pristine_seed_still_recovers, shell_without_native_paths_uses_terminal_end_policy}` and execute actual ProbeRecovery decision/fixture behavior. Name supervisor cases `checkpoints::tests::{zero_revision_blocks, reason_text_is_not_inspection, legacy_negative_blocks, conflicting_rows_block, unreadable_store_blocks, independent_present_checkpoint_restores, inspected_shared_store_absence_remains_definitive}`. The last cases call real probe/refinement helpers with inspected fixture results; destructive process/corruption setup remains in sandbox. Add supervisor production recovery/loss cases `recovery::tests::{shell_dispatched_verified_end_online, shell_dispatched_verified_end_offline, shell_uncertain_end_blocks, shell_never_dispatched_preserves_pristine_seed}`; exact positive end cases assert the actual `NonResumableTerminalEnded` certificate, no replacement and retained output. Before production changes, add these tests using the current interfaces, record behavioral RED, then extend them for additive evidence. Missing-module compilation and zero matches are setup failures, never RED evidence. -- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. -- [ ] **Step 3: Apply exact decision replacements.** Apply unknown conversions only to declared coding-provider recovery paths. Host `ProbeRecovery` with None resume plus false `never_dispatched` for a coding provider returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero for required CheckpointRestore is Blocked/unknown. Resolve provider/capability from the existing authoritative incarnation/context, not absence of a resume spec. For shell, bypass nonexistent NativeResume/CheckpointRestore probes in both online and offline recovery; certify unavailable Reattach only after verified process end/VerifiedEmpty and unavailable PristineSeed only from durable dispatched-input evidence, then use existing strict loss certification to produce `NonResumableTerminalEnded`. Keep exact inventory/cleanup validation; no fake Missing store row, weakened proof or identity requirement for a path shell does not declare. A live shell reattaches; uncertain process/backend termination remains Blocked with writer completion unresolved. Never-dispatched shell retains actual pristine proof. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. +- [x] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`. Expected current metadata-only negative satisfies definitive loss incorrectly. Process kill/corruption variants run only with `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"` and the supervisor recovery command below. +- [x] **Step 3: Apply exact decision replacements.** Apply unknown conversions only to declared coding-provider recovery paths. Host `ProbeRecovery` with None resume plus false `never_dispatched` for a coding provider returns Blocked NativeResume/AmbiguousIdentity, reason that identity is unrecorded and store absence unestablished, manual retry/repair true. `checkpoints::probe` revision zero for required CheckpointRestore is Blocked/unknown. Resolve provider/capability from the existing authoritative incarnation/context, not absence of a resume spec. For shell, bypass nonexistent NativeResume/CheckpointRestore probes in both online and offline recovery; certify unavailable Reattach only after verified process end/VerifiedEmpty and unavailable PristineSeed only from durable dispatched-input evidence, then use existing strict loss certification to produce `NonResumableTerminalEnded`. Keep exact inventory/cleanup validation; no fake Missing store row, weakened proof or identity requirement for a path shell does not declare. A live shell reattaches; uncertain process/backend termination remains Blocked with writer completion unresolved. Never-dispatched shell retains actual pristine proof. `probe_after_provider` removes prose/path substring checks and accepts exactly one explicit inspected CheckpointRestore row with nonempty evidence refs and store state Missing or PresentReadable; missing/duplicate/unreadable/unknown rows block. Provider adapters emit NativeResume negatives only after complete inspection for effective identity, roots and accepted artifacts; a declaration or same executable invocation is insufficient. OpenCode exact SQLite no-match can cover both paths only when effective DB is established, including accepted data-root/DB/channel overrides. Claude/Codex scans that skip unreadable, deep, symlinked, oversized or database-selected candidates cannot certify absence: preserve exact positive recovery and block unknown negatives until complete authority exists. Amplifier inspection includes accepted backup transcript/metadata. The NativeSession fixture requires separate complete checkpoint inspection; oversize skipped candidate is Unknown. CheckpointRestore negatives require demonstrably complete shared or independent inspection. Replace absence `Path::exists()` in changed paths with error-aware metadata/query handling: only NotFound/readable no-match is absence; permission/I/O errors block. Preserve rows intact through host and `after_native_probe`. `collect_loss_path_evidence` selects exact path rows and rejects missing/conflicting/uninspected rows before normalization. Offline coding-provider missing spec/identity after dispatch uses the same blocked classification; shell follows the capability-specific verified terminal-end policy above. Existing LostDecision inventory validation stays strict; no weakened cleanup authority. -- [ ] **Step 4: Green.** All metadata-only paths block and retain data, confirmed inspected absence still authorizes the correct final result, independently present fixture checkpoint restores. -- [ ] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification, pre-IPC input journals, queued-only replay and actor-native identity adoption. Input-row/native-ID absence alone is not pristine proof: consult durable execution-grant/actor evidence and record initialization intent before provider creation. Ambiguous startup before ID capture blocks; seed replacement needs actual no-dispatch/zero-turn proof. Preserve existing five-successful-replacements-per-hour budget and make final retry-budget Blocked sticky across automatic requests/cold restart until ManualRetry. -- [ ] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Record actual executed/passed counts: host `probe_recovery` must include at least the four named tests, supervisor `checkpoint` at least the seven named `checkpoints::tests` tests, and sandbox supervisor `recovery` must execute all four named shell cases; every other selector needs a positive count; zero matches or skipped required cases fail the gate. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. -- [ ] **Step 7: Commit.** Stage this task's exact files and `git commit -m "fix: require inspected evidence before declaring recovery unavailable"`. +- [x] **Step 4: Green.** All metadata-only paths block and retain data, confirmed inspected absence still authorizes the correct final result, independently present fixture checkpoint restores. +- [x] **Step 5: Refactor.** Share the exact inspected-row selection helper between live/offline collection, remove all checkpoint prose inference, retain exact identity verification, pre-IPC input journals, queued-only replay and actor-native identity adoption. Input-row/native-ID absence alone is not pristine proof: consult durable execution-grant/actor evidence and record initialization intent before provider creation. Ambiguous startup before ID capture blocks; seed replacement needs actual no-dispatch/zero-turn proof. Preserve existing five-successful-replacements-per-hour budget and make final retry-budget Blocked sticky across automatic requests/cold restart until ManualRetry. +- [x] **Step 6: Impact verification.** Run `mise exec rust@1.96 -- cargo test -p freshell-runtime-protocol --locked recovery`; `mise exec rust@1.96 -- cargo test -p freshell-session-host --locked probe_recovery`; `mise exec rust@1.96 -- cargo test -p freshell-supervisor --locked checkpoint`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-agent-runtime --locked native_fixture_probe"`; `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Record actual executed/passed counts: host `probe_recovery` must include at least the four named tests, supervisor `checkpoint` at least the seven named `checkpoints::tests` tests, and sandbox supervisor `recovery` must execute all four named shell cases; every other selector needs a positive count; zero matches or skipped required cases fail the gate. Expected tests run/pass, no unintended new conversation or data cleanup. Verify repeated startup failure naturally exhausts the existing durable budget, stays blocked across restart and resumes only after ManualRetry. +- [x] **Step 7: Commit.** Stage this task's exact files and `git commit -m "fix: require inspected evidence before declaring recovery unavailable"`. ### Task 5: Typed final recovery status, retained spool and bounded transitions From 7a6b8188cb7345952cd995d8b7633881431fd59e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:25:58 -0700 Subject: [PATCH 233/493] fix: settle managed recovery and retain terminal spool output --- .../freshell-protocol/src/server_messages.rs | 31 + crates/freshell-protocol/tests/roundtrip.rs | 1 + crates/freshell-runtime-protocol/src/lib.rs | 6 + .../src/terminal_spool.rs | 298 +++++++ crates/freshell-server/src/main.rs | 9 +- .../src/managed_output_rpc_tests.rs | 300 ++++++- .../src/managed_output_validation.rs | 55 ++ crates/freshell-server/src/managed_runtime.rs | 748 ++++++++++++++---- .../src/managed_runtime_api.rs | 34 +- crates/freshell-server/src/terminals.rs | 24 +- .../managed_recovery_transition_tests.rs | 74 ++ .../tests/managed_runtime_lifecycle_tests.rs | 388 +++++++++ .../src/output_journal.rs | 281 ++++--- crates/freshell-session-host/src/pty.rs | 133 +++- crates/freshell-supervisor/src/lib.rs | 1 + crates/freshell-supervisor/src/recovery.rs | 1 + crates/freshell-supervisor/src/registry.rs | 27 + .../src/retained_output.rs | 155 ++++ .../src/retained_output_tests.rs | 244 ++++++ crates/freshell-supervisor/src/service.rs | 59 +- .../src/managed_output_tests.rs | 103 +++ crates/freshell-terminal/src/pty.rs | 16 +- crates/freshell-terminal/src/registry.rs | 209 ++++- crates/freshell-ws/src/auto_resume.rs | 2 + .../src/connection_writer_tests.rs | 6 + .../src/managed_retained_socket_tests.rs | 183 +++++ crates/freshell-ws/src/terminal.rs | 334 +++++++- 27 files changed, 3395 insertions(+), 327 deletions(-) create mode 100644 crates/freshell-runtime-protocol/src/terminal_spool.rs create mode 100644 crates/freshell-server/src/tests/managed_recovery_transition_tests.rs create mode 100644 crates/freshell-server/src/tests/managed_runtime_lifecycle_tests.rs create mode 100644 crates/freshell-supervisor/src/retained_output.rs create mode 100644 crates/freshell-supervisor/src/retained_output_tests.rs create mode 100644 crates/freshell-ws/src/managed_retained_socket_tests.rs diff --git a/crates/freshell-protocol/src/server_messages.rs b/crates/freshell-protocol/src/server_messages.rs index e86ee83bc..1c8d87f7c 100644 --- a/crates/freshell-protocol/src/server_messages.rs +++ b/crates/freshell-protocol/src/server_messages.rs @@ -408,6 +408,7 @@ pub enum SessionRepairEvent { pub enum RuntimeStatus { Running, Recovering, + Blocked, /// The auto-resume SETTLE frame (kata znhn item 3): broadcast with the /// OLD terminal id whenever a planned auto-resume settles without a /// replacement (guard-abort, retries exhausted, flap circuit breaker, @@ -1508,6 +1509,28 @@ pub struct TerminalCreated { pub struct TerminalExit { pub exit_code: i64, pub terminal_id: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub managed_retained: Option, + /// False explicitly denotes an unavailable exit result; exitCode is -1 + /// for compatibility with clients that require the existing numeric field. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub exit_code_available: Option, +} + +/// Existing managed projection shape, without a runtime-control dependency. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ManagedRecoverySummary { + pub desired_state: String, + pub recovery_state: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub reason: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub attempt_id: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub incident_id: Option, + pub durability_state: String, + pub allocation_state: String, } #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] @@ -1534,6 +1557,12 @@ pub struct InventoryTerminal { pub description: Option, #[serde(skip_serializing_if = "Option::is_none")] pub runtime_status: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub soul_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub incarnation_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub recovery_summary: Option, #[serde(skip_serializing_if = "Option::is_none")] pub session_ref: Option, /// Unified agent names (Task 1 wire / Task 2 Rust side): canonical @@ -1674,6 +1703,8 @@ pub struct TerminalSessionAssociated { pub struct TerminalStatus { pub status: RuntimeStatus, pub terminal_id: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub recovery_summary: Option, #[serde(skip_serializing_if = "Option::is_none")] pub attempt: Option, /// Auto-resume `recovering` frames only: the bounded retry budget. The diff --git a/crates/freshell-protocol/tests/roundtrip.rs b/crates/freshell-protocol/tests/roundtrip.rs index 06f0d708c..0577a1381 100644 --- a/crates/freshell-protocol/tests/roundtrip.rs +++ b/crates/freshell-protocol/tests/roundtrip.rs @@ -637,6 +637,7 @@ fn terminal_status_exited_settle_frame_roundtrips() { // existing terminal.status message, with the typed resumeCycles field // (flap-circuit-breaker settles only). let msg = ServerMessage::TerminalStatus(TerminalStatus { + recovery_summary: None, status: RuntimeStatus::Exited, terminal_id: "t1".into(), attempt: None, diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index d10cdaf7c..ee9a4d0e7 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -20,6 +20,10 @@ pub mod fresh_agent_ops; pub use fresh_agent_ops::*; pub mod provider_inputs; pub use provider_inputs::*; +pub mod terminal_spool; +pub use terminal_spool::{ + TerminalExitObservation, TerminalSpoolCompletion, TerminalSpoolDescriptor, +}; #[cfg(test)] #[path = "fresh_agent_ops_tests.rs"] @@ -2417,6 +2421,8 @@ pub struct RuntimeOutputBatch { pub truncated: bool, #[serde(default)] pub exited: bool, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub completion: Option, #[serde(skip_serializing_if = "Option::is_none")] pub exit_code: Option, /// Provider-native durable conversation id discovered by the session host. diff --git a/crates/freshell-runtime-protocol/src/terminal_spool.rs b/crates/freshell-runtime-protocol/src/terminal_spool.rs new file mode 100644 index 000000000..730512a11 --- /dev/null +++ b/crates/freshell-runtime-protocol/src/terminal_spool.rs @@ -0,0 +1,298 @@ +//! Bounded retained-output codec. Filesystem ownership and writer settlement +//! belong to the session host and supervisor, never to this codec. +use crate::{IncarnationId, RuntimeOutputBatch, RuntimeOutputFrame}; +use serde::{Deserialize, Serialize}; +use std::io::{BufRead, Read}; + +pub const MAX_OUTPUT_FRAME_BYTES: usize = 64 * 1024; +pub const MAX_SPOOL_SEGMENT_BYTES: u64 = 32 * 1024 * 1024; +pub const MAX_SPOOL_LINE_BYTES: usize = 6 * MAX_OUTPUT_FRAME_BYTES + 4096; +pub const SPOOL_DESCRIPTOR_FILE: &str = "terminal-spool-descriptor.json"; +pub const SPOOL_COMPLETION_FILE: &str = "terminal-spool-completion.json"; +pub const SPOOL_CURRENT_FILE: &str = "terminal-spool-current.jsonl"; +pub const SPOOL_PREVIOUS_FILE: &str = "terminal-spool-previous.jsonl"; + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TerminalSpoolDescriptor { + pub schema_version: u32, + pub incarnation_id: IncarnationId, + pub terminal_id: String, + pub stream_epoch: String, + pub segment_limit_bytes: u64, +} +impl TerminalSpoolDescriptor { + pub fn validate(&self) -> Result<(), String> { + if self.schema_version != 1 + || self.terminal_id.is_empty() + || self.terminal_id.len() > 1024 + || self.stream_epoch.is_empty() + || self.stream_epoch.len() > 1024 + || self.segment_limit_bytes == 0 + || self.segment_limit_bytes > MAX_SPOOL_SEGMENT_BYTES + { + return Err("invalid retained output descriptor".into()); + } + Ok(()) + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "availability", rename_all = "snake_case")] +pub enum TerminalExitObservation { + Known { code: i64 }, + Unavailable, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TerminalSpoolCompletion { + pub incarnation_id: IncarnationId, + pub terminal_id: String, + pub stream_epoch: String, + pub retained_from_seq: u64, + pub head_seq: u64, + pub exit: TerminalExitObservation, +} +impl TerminalSpoolCompletion { + pub fn validate(&self, source: &TerminalSpoolDescriptor, head: u64) -> Result<(), String> { + if self.incarnation_id != source.incarnation_id + || self.terminal_id != source.terminal_id + || self.stream_epoch != source.stream_epoch + || self.head_seq != head + || self.retained_from_seq == 0 + || self.retained_from_seq > head.saturating_add(1) + { + return Err("retained completion does not match settled source".into()); + } + Ok(()) + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TerminalSpoolFrame { + pub frame: RuntimeOutputFrame, +} + +/// Keeps one bounded JSON line in memory. A non-newline final record is an +/// uncommitted torn tail, including valid JSON without its commit newline. +pub fn scan_spool( + mut reader: R, + source: &TerminalSpoolDescriptor, + mut accept: impl FnMut(RuntimeOutputFrame) -> Result<(), String>, +) -> Result { + source.validate()?; + let mut bytes = Vec::new(); + loop { + bytes.clear(); + let length = Read::by_ref(&mut reader) + .take((MAX_SPOOL_LINE_BYTES + 1) as u64) + .read_until(b'\n', &mut bytes) + .map_err(|_| "retained output read failed")?; + if length == 0 { + return Ok(false); + } + if length > MAX_SPOOL_LINE_BYTES { + return Err("retained output frame exceeds codec bound".into()); + } + if bytes.last() != Some(&b'\n') { + return Ok(true); + } + let row: TerminalSpoolFrame = serde_json::from_slice(&bytes) + .map_err(|_| "invalid committed retained output frame")?; + let frame = row.frame; + // A restarted host can leave its prior epoch in the bounded segments. + if frame.stream_epoch != source.stream_epoch { + continue; + } + if frame.terminal_id != source.terminal_id + || frame.seq_start == 0 + || frame.seq_start != frame.seq_end + || frame.seq_end >= i64::MAX as u64 + || frame.data.len() > MAX_OUTPUT_FRAME_BYTES + { + return Err("retained output frame ownership or bounds mismatch".into()); + } + accept(frame)?; + } +} + +/// Streaming page accumulator: no full-spool collection and no partial frame +/// with an advanced cursor. A too-small caller budget is an explicit error. +pub struct RetainedOutputPage { + source: TerminalSpoolDescriptor, + after: u64, + budget: usize, + first: Option, + head: u64, + used: usize, + more: bool, + frames: Vec, +} +impl RetainedOutputPage { + pub fn new(source: TerminalSpoolDescriptor, after: u64, budget: u64) -> Result { + source.validate()?; + Ok(Self { + source, + after, + budget: budget.clamp(1, MAX_OUTPUT_FRAME_BYTES as u64) as usize, + first: None, + head: 0, + used: 0, + more: false, + frames: vec![], + }) + } + pub fn accept(&mut self, frame: RuntimeOutputFrame) -> Result<(), String> { + if self.first.is_some() && frame.seq_start != self.head.saturating_add(1) { + return Err("retained output sequences are not contiguous".into()); + } + self.first.get_or_insert(frame.seq_start); + self.head = frame.seq_end; + if frame.seq_end <= self.after { + return Ok(()); + } + let cost = frame.data.len().max(1); + if self.frames.is_empty() && cost > self.budget { + return Err("output budget cannot hold a complete retained frame".into()); + } + if self.more || self.used.saturating_add(cost) > self.budget { + self.more = true; + } else { + self.used += cost; + self.frames.push(frame); + } + Ok(()) + } + pub fn finish( + self, + torn: bool, + completion: Option, + ) -> Result { + if let Some(c) = &completion { + c.validate(&self.source, self.head)?; + } + let retained = self.first.unwrap_or(self.head.saturating_add(1)); + let reset = self.after > self.head || self.after.saturating_add(1) < retained; + Ok(RuntimeOutputBatch { + incarnation_id: self.source.incarnation_id, + terminal_id: self.source.terminal_id, + stream_epoch: self.source.stream_epoch, + retained_from_seq: retained, + head_seq: self.head, + reset_required: reset, + truncated: reset || torn || self.more, + exited: completion.is_some(), + exit_code: completion.as_ref().and_then(|c| match c.exit { + TerminalExitObservation::Known { code } => Some(code), + TerminalExitObservation::Unavailable => None, + }), + completion, + native_session_id: None, + frames: self.frames, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + fn source() -> TerminalSpoolDescriptor { + TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id: IncarnationId::new(), + terminal_id: "owned".into(), + stream_epoch: "launch-hostboot".into(), + segment_limit_bytes: MAX_SPOOL_SEGMENT_BYTES, + } + } + fn frame(source: &TerminalSpoolDescriptor, seq: u64, data: &str) -> RuntimeOutputFrame { + RuntimeOutputFrame { + terminal_id: source.terminal_id.clone(), + stream_epoch: source.stream_epoch.clone(), + seq_start: seq, + seq_end: seq, + data: data.into(), + } + } + fn line(frame: RuntimeOutputFrame) -> Vec { + let mut bytes = serde_json::to_vec(&TerminalSpoolFrame { frame }).unwrap(); + bytes.push(b'\n'); + bytes + } + #[test] + fn retained_output_codec_preserves_utf8_full_pages_for_two_readers() { + let source = source(); + let data = "🦀".repeat(16384); + let bytes = [ + line(frame(&source, 1, &data)), + line(frame(&source, 2, "final diagnostic")), + ] + .concat(); + for _ in 0..2 { + let mut cursor = 0; + let mut output = String::new(); + while cursor < 2 { + let mut page = RetainedOutputPage::new(source.clone(), cursor, 65536).unwrap(); + let torn = scan_spool(std::io::Cursor::new(&bytes), &source, |frame| { + page.accept(frame) + }) + .unwrap(); + let batch = page.finish(torn, None).unwrap(); + assert!(!batch.frames.is_empty()); + for frame in batch.frames { + cursor = frame.seq_end; + output += &frame.data; + } + } + assert_eq!(output, data.clone() + "final diagnostic"); + } + } + #[test] + fn retained_output_codec_rejects_oversized_committed_frames_without_allocating_spool() { + let source = source(); + let bytes = line(frame(&source, 1, &"x".repeat(MAX_SPOOL_LINE_BYTES + 1))); + let mut seen = false; + assert!(scan_spool(std::io::Cursor::new(bytes), &source, |_| { + seen = true; + Ok(()) + }) + .is_err()); + assert!(!seen); + } + #[test] + fn retained_output_codec_rejects_foreign_and_repeated_sequences() { + let source = source(); + let mut foreign = frame(&source, 1, "other"); + foreign.terminal_id = "other".into(); + assert!(scan_spool(std::io::Cursor::new(line(foreign)), &source, |_| Ok(())).is_err()); + let mut page = RetainedOutputPage::new(source.clone(), 0, 65536).unwrap(); + page.accept(frame(&source, 1, "first")).unwrap(); + assert!(page.accept(frame(&source, 1, "duplicate")).is_err()); + } + #[test] + fn retained_output_codec_never_advances_a_cursor_over_partial_frame() { + let source = source(); + let mut page = RetainedOutputPage::new(source.clone(), 0, 2).unwrap(); + assert!(page.accept(frame(&source, 1, "🦀")).is_err()); + } + #[test] + fn retained_output_completion_rejects_stale_head_epoch_and_foreign_incarnation() { + let source = source(); + let mut completion = TerminalSpoolCompletion { + incarnation_id: source.incarnation_id.clone(), + terminal_id: source.terminal_id.clone(), + stream_epoch: source.stream_epoch.clone(), + retained_from_seq: 1, + head_seq: 2, + exit: TerminalExitObservation::Unavailable, + }; + assert!(completion.validate(&source, 1).is_err()); + completion.head_seq = 1; + completion.stream_epoch = "foreign".into(); + assert!(completion.validate(&source, 1).is_err()); + completion.stream_epoch = source.stream_epoch.clone(); + completion.incarnation_id = IncarnationId::new(); + assert!(completion.validate(&source, 1).is_err()); + } +} diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index fba8dd876..9b2700d38 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -1128,7 +1128,7 @@ async fn main() -> ExitCode { let registry = freshell_terminal::TerminalRegistry::new() .with_ownership(std::sync::Arc::clone(&ownership)); #[cfg(feature = "managed-runtime-v1")] - let (managed_runtime_available, managed_runtime_client) = { + let (managed_runtime_available, managed_runtime_client, managed_runtime_controller) = { let controller = managed_runtime::ServerManagedRuntimeController::from_env() .await .map_err(|error| { @@ -1152,10 +1152,10 @@ async fn main() -> ExitCode { }); managed_mcp_capability::spawn_renewal_loop(client.clone()); } - registry.set_managed_controller(controller.map(|controller| { + registry.set_managed_controller(controller.clone().map(|controller| { controller as Arc })); - (available, client) + (available, client, controller) }; #[cfg(not(feature = "managed-runtime-v1"))] let managed_runtime_available = false; @@ -3056,7 +3056,8 @@ async fn main() -> ExitCode { .unwrap_or_else(|error| { eprintln!("managed runtime API initialization failed: {error}"); std::process::exit(1); - }); + }) + .with_terminal_controller(managed_runtime_controller.clone()); state.spawn_projector(); managed_runtime_api::router(state) }; diff --git a/crates/freshell-server/src/managed_output_rpc_tests.rs b/crates/freshell-server/src/managed_output_rpc_tests.rs index ad4f319cf..19629f901 100644 --- a/crates/freshell-server/src/managed_output_rpc_tests.rs +++ b/crates/freshell-server/src/managed_output_rpc_tests.rs @@ -10,6 +10,15 @@ async fn read_fixture( source_epoch: &str, expected_epoch: &str, foreign_terminal: bool, +) -> Result { + read_fixture_cached(source_epoch, expected_epoch, foreign_terminal, false).await +} + +async fn read_fixture_cached( + source_epoch: &str, + expected_epoch: &str, + foreign_terminal: bool, + cached_blocked: bool, ) -> Result { let dir = tempfile::tempdir().unwrap(); let socket = dir.path().join("control.sock"); @@ -53,6 +62,7 @@ async fn read_fixture( reset_required: changed, truncated: changed, exited: false, + completion: None, exit_code: None, native_session_id: Some("native-owned".into()), frames: vec![RuntimeOutputFrame { @@ -85,10 +95,31 @@ async fn read_fixture( client: RuntimeClient::new(&socket, "synthetic-epoch-test-secret"), recovery: Arc::new(ManagedRecoveryState::default()), }; + let soul = SoulId::new(); + if cached_blocked { + controller.recovery.slots.lock().await.insert( + soul.to_string(), + SoulRecoverySlot { + sequence: 1, + task_in_flight: false, + incarnation_id: None, + terminal_id: None, + intent_revision: None, + evidence_revision: None, + publication: RecoveryPublication::Failed { + code: RuntimeErrorCode::InvalidRequest, + previous: None, + }, + observed_epoch: None, + observed_exit_code: None, + observed_completion: None, + }, + ); + } let result = controller .read_output( ManagedTerminalDescriptor { - soul_id: SoulId::new().to_string(), + soul_id: soul.to_string(), incarnation_id: IncarnationId::new().to_string(), terminal_id: "terminal-owned".into(), stream_id: expected_epoch, @@ -101,13 +132,36 @@ async fn read_fixture( 65536, ) .await; - tokio::time::timeout(std::time::Duration::from_secs(5), server) - .await - .unwrap() - .unwrap(); + if cached_blocked && result.is_err() { + // The old cached-error path never reaches the socket. Join that idle + // fixture before the behavioral assertion rather than leaking a task. + server.abort(); + assert!(server.await.unwrap_err().is_cancelled()); + } else { + tokio::time::timeout(std::time::Duration::from_secs(5), server) + .await + .unwrap() + .unwrap(); + } result } +#[tokio::test] +async fn managed_runtime_cached_blocked_output_remains_readable() { + let output = read_fixture_cached("same-epoch", "same-epoch", false, true) + .await + .expect("final blocked verdict cannot suppress retained output access"); + assert_eq!(output.chunks[0].data, "cursor-relative output"); + let lifecycle = output + .lifecycle + .expect("classified failed recovery remains a typed blocked observation"); + assert_eq!(lifecycle.state, ManagedLifecycleState::Blocked); + assert_eq!( + lifecycle.recovery_summary.unwrap().reason.as_deref(), + Some("invalid_request") + ); +} + #[tokio::test] async fn changed_host_epoch_replays_from_zero_in_one_supervisor_round_trip() { let output = read_fixture("new-epoch", "old-epoch", false).await.unwrap(); @@ -137,3 +191,239 @@ async fn authenticated_source_cannot_substitute_another_terminal_for_this_facade "authenticated transport is not proof of matching terminal ownership" ); } + +#[tokio::test] +async fn managed_runtime_failed_retained_read_publishes_blocked_without_fabricating_output() { + let root = tempfile::tempdir().unwrap(); + let soul = SoulId::new(); + let incarnation = IncarnationId::new(); + let controller = Arc::new(ServerManagedRuntimeController { + client: RuntimeClient::new( + root.path().join("missing-control.sock"), + "synthetic-failed-read", + ), + recovery: Arc::new(ManagedRecoveryState::default()), + }); + controller.recovery.slots.lock().await.insert( + soul.to_string(), + SoulRecoverySlot { + sequence: 1, + task_in_flight: false, + incarnation_id: Some(incarnation.to_string()), + terminal_id: Some("failed-retained-read".into()), + intent_revision: Some(1), + evidence_revision: Some(1), + publication: RecoveryPublication::Failed { + code: RuntimeErrorCode::HostUnreachable, + previous: None, + }, + observed_epoch: None, + observed_exit_code: None, + observed_completion: None, + }, + ); + let registry = freshell_terminal::TerminalRegistry::new(); + let descriptor = ManagedTerminalDescriptor { + soul_id: soul.to_string(), + incarnation_id: incarnation.to_string(), + terminal_id: "failed-retained-read".into(), + stream_id: "failed-read-epoch".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: Some("native-retained-failure".into()), + create_request_id: Some("failed-read-create".into()), + }; + registry.register_managed(descriptor.clone()); + registry.set_managed_controller(Some(controller.clone())); + let result = registry + .refresh_managed_output(&descriptor.terminal_id, 65536) + .await; + assert!( + result.is_err(), + "unreadable retained output must remain an actual error" + ); + let inventory = registry.inventory(); + assert_eq!(inventory.len(), 1); + assert_eq!( + inventory[0].runtime_status, + Some(freshell_protocol::RuntimeStatus::Blocked) + ); + assert_eq!( + inventory[0] + .recovery_summary + .as_ref() + .unwrap() + .reason + .as_deref(), + Some("host_unreachable") + ); + assert_eq!( + registry.managed_output_cursor(&descriptor.terminal_id), + Some(0) + ); + assert!(registry + .managed_descriptor(&descriptor.terminal_id) + .is_some()); + assert!(controller + .input(descriptor, "must not dispatch".into()) + .await + .is_err()); +} + +async fn first_eof_evidence() -> ( + ManagedOutputRead, + (Option, Option, Option), + Vec, + String, +) { + use tracing::instrument::WithSubscriber; + use tracing_subscriber::prelude::*; + let events = Arc::new(std::sync::Mutex::new(vec![])); + let subscriber = + tracing_subscriber::registry().with(super::tests::transitions::CaptureLogs(events.clone())); + let root = tempfile::tempdir().unwrap(); + let socket_path = root.path().join("first-eof-control.sock"); + let listener = UnixListener::bind(&socket_path).unwrap(); + let soul = SoulId::new(); + let incarnation = IncarnationId::new(); + let controller = Arc::new(ServerManagedRuntimeController { + client: RuntimeClient::new(&socket_path, "synthetic-first-eof"), + recovery: Arc::default(), + }); + let server_controller = controller.clone(); + let server_soul = soul.clone(); + let server_incarnation = incarnation.clone(); + let server = tokio::spawn(async move { + let mut observation = None; + for _ in 0..3 { + let (mut peer, _) = listener.accept().await.unwrap(); + let request: Envelope = read_frame(&mut peer).await.unwrap(); + let result = match request.body { + AdminCommand::Health => AdminResult::Health { + control_epoch: 1, + installation_id: InstallationId::new(), + }, + AdminCommand::TerminalReadOutput(_) => { + AdminResult::TerminalOutput(RuntimeOutputBatch { + incarnation_id: server_incarnation.clone(), + terminal_id: "first-eof-terminal".into(), + stream_epoch: "actual-first-host-epoch".into(), + retained_from_seq: 1, + head_seq: 1, + reset_required: false, + truncated: false, + exited: true, + exit_code: Some(19), + native_session_id: None, + completion: None, + frames: vec![RuntimeOutputFrame { + terminal_id: "first-eof-terminal".into(), + stream_epoch: "actual-first-host-epoch".into(), + seq_start: 1, + seq_end: 1, + data: "final tail".into(), + }], + }) + } + AdminCommand::Recover(_) => { + let slots = server_controller.recovery.slots.lock().await; + let slot = slots + .get(server_soul.as_str()) + .expect("recovery owns a slot before dispatch"); + observation = Some(( + slot.observed_epoch.clone(), + slot.observed_exit_code, + slot.observed_completion.clone(), + )); + drop(slots); + let view: RuntimeView = serde_json::from_value(serde_json::json!({ + "soulId":server_soul,"incarnationId":server_incarnation,"launchState":"stopped", + "cleanupState":"verified_empty","desiredState":"stopped","recoveryState":"stopped", + "intentRevision":1,"executionGeneration":1,"terminalId":"first-eof-terminal", + "terminalStreamId":"actual-first-host-epoch","terminalMode":"shell","terminalCwd":"/workspace" + })).unwrap(); + AdminResult::Recovery(freshell_runtime_protocol::RecoveryResult { + outcome: RecoveryOutcome::Stopped, + view, + probe: None, + prior_incarnation_id: None, + attempt_id: None, + expected_native_session_id: None, + observed_native_session_id: None, + incident_id: None, + }) + } + other => panic!("unexpected first EOF command: {other:?}"), + }; + write_frame( + &mut peer, + &AdminReply { + request_id: request.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + observation.unwrap() + }); + let read = controller + .read_output( + ManagedTerminalDescriptor { + soul_id: soul.to_string(), + incarnation_id: incarnation.to_string(), + terminal_id: "first-eof-terminal".into(), + stream_id: "actual-first-host-epoch".into(), + mode: "shell".into(), + cwd: "/workspace".into(), + resume_session_id: None, + create_request_id: None, + }, + 0, + 65536, + ) + .with_subscriber(subscriber) + .await; + let observed = tokio::time::timeout(std::time::Duration::from_secs(15), server) + .await + .unwrap() + .unwrap(); + // The fixture and recovery publication settle before any behavioral assertion. + tokio::time::timeout(std::time::Duration::from_secs(5), async { + while controller.recovery_in_flight(soul.as_str()).await { + tokio::task::yield_now().await; + } + }) + .await + .unwrap(); + let captured = events.lock().unwrap().clone(); + (read.unwrap(), observed, captured, incarnation.to_string()) +} + +#[tokio::test] +async fn managed_runtime_first_eof_observation_precedes_recovery_dispatch() { + let (read, observed, _, incarnation) = first_eof_evidence().await; + assert_eq!(read.chunks[0].data, "final tail"); + assert_eq!(observed.0.as_deref(), Some("actual-first-host-epoch")); + assert_eq!(observed.1, Some(19)); + let completion = observed + .2 + .expect("first EOF retains its coherent completion before dispatch"); + assert_eq!(completion.head_seq, 1); + assert_eq!(completion.exit_code, Some(19)); + assert_eq!(completion.incarnation_id, incarnation); +} + +#[tokio::test] +async fn managed_runtime_first_eof_transition_identifies_the_existing_handle() { + let (_, _, events, incarnation) = first_eof_evidence().await; + let event = events + .iter() + .find(|event| event["fields"]["event"] == "managed_runtime.recovery_transition") + .expect("first EOF publishes a recovery transition"); + assert_eq!(event["fields"]["terminal_id"], "first-eof-terminal"); + assert_eq!(event["fields"]["incarnation_id"], incarnation); + for field in ["soul_id", "from", "to", "cause"] { + assert!(!event["fields"][field].is_null()); + } +} diff --git a/crates/freshell-server/src/managed_output_validation.rs b/crates/freshell-server/src/managed_output_validation.rs index 365c2c877..7ba56bbed 100644 --- a/crates/freshell-server/src/managed_output_validation.rs +++ b/crates/freshell-server/src/managed_output_validation.rs @@ -11,6 +11,26 @@ pub(super) fn validate(output: &RuntimeOutputBatch, terminal_id: &str) -> Result { return Err("managed output has invalid retained sequence bounds".into()); } + if let Some(completion) = &output.completion { + let source = freshell_runtime_protocol::TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id: output.incarnation_id.clone(), + terminal_id: output.terminal_id.clone(), + stream_epoch: output.stream_epoch.clone(), + segment_limit_bytes: freshell_runtime_protocol::terminal_spool::MAX_SPOOL_SEGMENT_BYTES, + }; + completion.validate(&source, output.head_seq)?; + let expected_code = match completion.exit { + freshell_runtime_protocol::TerminalExitObservation::Known { code } => Some(code), + freshell_runtime_protocol::TerminalExitObservation::Unavailable => None, + }; + if !output.exited + || completion.retained_from_seq != output.retained_from_seq + || output.exit_code != expected_code + { + return Err("managed output completion disagrees with sampled lifecycle".into()); + } + } let mut previous = None; for frame in &output.frames { if frame.terminal_id != terminal_id || frame.stream_epoch != output.stream_epoch { @@ -43,6 +63,7 @@ mod tests { reset_required: false, truncated: false, exited: false, + completion: None, exit_code: None, native_session_id: Some("native-owned".into()), frames: vec![RuntimeOutputFrame { @@ -79,6 +100,40 @@ mod tests { assert!(validate(&malformed, "owned").is_err()); } + #[test] + fn rejects_completion_from_a_different_source_or_sampled_final_head() { + use freshell_runtime_protocol::{TerminalExitObservation, TerminalSpoolCompletion}; + let mut complete = batch(); + complete.exited = true; + complete.exit_code = Some(7); + complete.completion = Some(TerminalSpoolCompletion { + incarnation_id: complete.incarnation_id.clone(), + terminal_id: complete.terminal_id.clone(), + stream_epoch: complete.stream_epoch.clone(), + retained_from_seq: 1, + head_seq: 2, + exit: TerminalExitObservation::Known { code: 7 }, + }); + assert!(validate(&complete, "owned").is_ok()); + for mutation in 0..7 { + let mut malformed = complete.clone(); + let completion = malformed.completion.as_mut().unwrap(); + match mutation { + 0 => completion.terminal_id = "foreign".into(), + 1 => completion.incarnation_id = IncarnationId::new(), + 2 => completion.stream_epoch = "foreign-epoch".into(), + 3 => completion.head_seq = 1, + 4 => completion.retained_from_seq = 3, + 5 => malformed.exited = false, + _ => malformed.exit_code = None, + } + assert!( + validate(&malformed, "owned").is_err(), + "completion mismatch {mutation} must not complete a facade" + ); + } + } + #[test] fn rejects_overflow_duplicate_and_out_of_bounds_sequences() { let mut malformed = batch(); diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 6f1be82e6..a3539a0d6 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -2,18 +2,19 @@ use freshell_runtime_client::{ClientError, RuntimeClient}; use freshell_runtime_protocol::{ - DesiredState, LaunchRequest, LaunchState, RecoveryBlockReason, RecoveryOutcome, RecoveryProbe, - RecoveryResult, RecoveryTrigger, RequestId, RuntimeErrorCode, RuntimeLimits, RuntimeProfile, - RuntimeView, SoulId, StopOutcome, TerminalLaunchSpec, ViewIntentKind, ViewIntentRequest, - ViewVisibilityIntent, + CleanupState, DesiredState, LaunchRequest, LaunchState, RecoveryBlockReason, RecoveryOutcome, + RecoveryProbe, RecoveryResult, RecoveryState, RecoveryTrigger, RequestId, RuntimeErrorCode, + RuntimeLimits, RuntimeProfile, RuntimeView, SoulId, StopOutcome, TerminalLaunchSpec, + ViewIntentKind, ViewIntentRequest, ViewVisibilityIntent, }; use freshell_terminal::registry::{ - ManagedOutputChunk, ManagedOutputRead, ManagedTerminalController, ManagedTerminalDescriptor, - ManagedTerminalFuture, ManagedTerminalLaunch, + ManagedLifecycle, ManagedLifecycleState, ManagedOutputChunk, ManagedOutputCompletion, + ManagedOutputRead, ManagedTerminalController, ManagedTerminalDescriptor, ManagedTerminalFuture, + ManagedTerminalLaunch, }; use sha2::{Digest, Sha256}; use std::{ - collections::{HashMap, HashSet}, + collections::HashMap, path::{Path, PathBuf}, process::Command, sync::Arc, @@ -23,12 +24,165 @@ use tokio::sync::Mutex; #[path = "managed_output_validation.rs"] mod managed_output_validation; +#[derive(Debug, Clone)] +enum RecoveryPublication { + Recovering { + previous: Option>, + }, + Finished(Box), + Failed { + code: RuntimeErrorCode, + previous: Option>, + }, +} +impl RecoveryPublication { + fn result(&self) -> Option<&RecoveryResult> { + match self { + Self::Finished(result) => Some(result), + Self::Recovering { previous } | Self::Failed { previous, .. } => previous.as_deref(), + } + } +} +#[derive(Debug, Clone)] +struct SoulRecoverySlot { + sequence: u64, + task_in_flight: bool, + // Durable supervisor generation, independent of the local task sequence. + incarnation_id: Option, + terminal_id: Option, + intent_revision: Option, + evidence_revision: Option, + publication: RecoveryPublication, + observed_epoch: Option, + observed_exit_code: Option, + observed_completion: Option, +} +struct RecoveryObservation { + terminal_id: String, + incarnation_id: String, + epoch: String, + exit_code: Option, + completion: Option, +} +impl SoulRecoverySlot { + fn observe(&mut self, observation: RecoveryObservation) { + self.terminal_id = Some(observation.terminal_id); + self.incarnation_id = Some(observation.incarnation_id); + self.observed_epoch = Some(observation.epoch); + self.observed_exit_code = observation.exit_code; + self.observed_completion = observation.completion; + } +} #[derive(Default)] struct ManagedRecoveryState { - in_flight: Mutex>, - blocked: Mutex>, + slots: Mutex>, } +fn wire_label(value: &impl serde::Serialize) -> String { + serde_json::to_value(value) + .ok() + .and_then(|value| value.as_str().map(str::to_owned)) + .unwrap_or_default() +} +fn recovery_summary(view: &RuntimeView) -> freshell_protocol::ManagedRecoverySummary { + freshell_protocol::ManagedRecoverySummary { + desired_state: wire_label(&view.desired_state), + recovery_state: wire_label(&view.recovery_state), + reason: view.recovery_reason.clone(), + attempt_id: view.recovery_attempt_id.as_ref().map(ToString::to_string), + incident_id: view.incident_id.as_ref().map(ToString::to_string), + durability_state: wire_label(&view.durability_state), + allocation_state: wire_label(&view.allocation_state), + } +} +fn lifecycle(publication: &RecoveryPublication) -> ManagedLifecycle { + let mut summary = publication + .result() + .map(|result| recovery_summary(&result.view)); + if let RecoveryPublication::Failed { code, .. } = publication { + let summary = summary.get_or_insert_with(|| freshell_protocol::ManagedRecoverySummary { + desired_state: wire_label(&DesiredState::Running), + recovery_state: wire_label(&RecoveryState::Blocked), + reason: None, + attempt_id: None, + incident_id: None, + durability_state: wire_label(&freshell_runtime_protocol::DurabilityState::Unknown), + allocation_state: wire_label(&freshell_runtime_protocol::AllocationState::Allocated), + }); + summary.recovery_state = wire_label(&RecoveryState::Blocked); + summary.reason = Some(wire_label(code).to_ascii_lowercase()); + } + let state = match publication { + RecoveryPublication::Recovering { .. } => ManagedLifecycleState::Recovering, + RecoveryPublication::Failed { .. } => ManagedLifecycleState::Blocked, + RecoveryPublication::Finished(result) => match result.outcome { + RecoveryOutcome::AlreadyLive + | RecoveryOutcome::Reattached + | RecoveryOutcome::Replaced => ManagedLifecycleState::Running, + RecoveryOutcome::Lost | RecoveryOutcome::Stopped + if result.view.cleanup_state == CleanupState::VerifiedEmpty => + { + ManagedLifecycleState::Ended + } + _ => ManagedLifecycleState::Blocked, + }, + }; + ManagedLifecycle { + state, + recovery_summary: summary, + } +} +fn classified_cause(publication: &RecoveryPublication) -> String { + match publication { + RecoveryPublication::Failed { code, .. } => wire_label(code), + RecoveryPublication::Recovering { .. } => "recovering".into(), + RecoveryPublication::Finished(result) => result + .probe + .as_ref() + .map(|probe| match probe { + RecoveryProbe::Blocked { reason, .. } => wire_label(reason), + RecoveryProbe::DefinitivelyUnavailable { .. } => "verified_unavailable".into(), + _ => "continuity_verified".into(), + }) + .unwrap_or_else(|| { + result.view.recovery_reason.clone().unwrap_or_else(|| { + match result.outcome { + RecoveryOutcome::AlreadyLive + | RecoveryOutcome::Reattached + | RecoveryOutcome::Replaced => "continuity_verified", + RecoveryOutcome::Lost => "verified_unavailable", + RecoveryOutcome::Stopped => "stopped", + RecoveryOutcome::Blocked => "blocked", + } + .into() + }) + }), + } +} +fn transition( + soul: &str, + from: Option<&RecoveryPublication>, + to: &RecoveryPublication, + terminal: Option<&str>, + incarnation: Option<&str>, +) { + let old = from.map(lifecycle); + let new = lifecycle(to); + let cause = classified_cause(to); + if old.as_ref().map(|l| &l.state) == Some(&new.state) + && from.map(classified_cause).as_deref() == Some(cause.as_str()) + { + return; + } + let result = to.result(); + tracing::info!(event="managed_runtime.recovery_transition",soul_id=soul, + terminal_id=result.and_then(|r|r.view.terminal_id.as_deref()).or(terminal), + incarnation_id=result.map(|r|r.view.incarnation_id.as_str()).or(incarnation), + attempt_id=result.and_then(|r|r.attempt_id.as_ref()).map(|id|id.as_str()), + incident_id=result.and_then(|r|r.incident_id.as_ref()).map(|id|id.as_str()), + from=old.map(|l|format!("{:?}",l.state)).unwrap_or_else(||"unobserved".into()), + to=?new.state,outcome=result.map(|r|wire_label(&r.outcome)),cause,"Managed recovery state changed"); +} #[derive(Clone)] pub struct ServerManagedRuntimeController { client: RuntimeClient, @@ -67,60 +221,259 @@ impl ServerManagedRuntimeController { } async fn recovery_in_flight(&self, soul_id: &str) -> bool { - self.recovery.in_flight.lock().await.contains(soul_id) + self.recovery + .slots + .lock() + .await + .get(soul_id) + .is_some_and(|slot| slot.task_in_flight) } - async fn blocked_recovery(&self, soul_id: &str) -> Option { - self.recovery.blocked.lock().await.get(soul_id).cloned() + async fn hydrate(&self, view: RuntimeView) { + let mut slots = self.recovery.slots.lock().await; + if slots.get(view.soul_id.as_str()).is_some_and(|slot| { + slot.task_in_flight + || slot + .intent_revision + .is_some_and(|revision| revision > view.intent_revision) + || slot + .evidence_revision + .is_some_and(|revision| revision > view.evidence_revision) + }) { + return; + } + let result = RecoveryResult { + outcome: match view.recovery_state { + RecoveryState::Live => RecoveryOutcome::AlreadyLive, + RecoveryState::Recovering => RecoveryOutcome::Blocked, + RecoveryState::Blocked => RecoveryOutcome::Blocked, + RecoveryState::Lost => RecoveryOutcome::Lost, + RecoveryState::Stopped => RecoveryOutcome::Stopped, + }, + attempt_id: view.recovery_attempt_id.clone(), + incident_id: view.incident_id.clone(), + prior_incarnation_id: view.prior_incarnation_id.clone(), + expected_native_session_id: view.native_session_id.clone(), + observed_native_session_id: view.native_session_id.clone(), + probe: None, + view: view.clone(), + }; + let publication = if view.recovery_state == RecoveryState::Recovering { + RecoveryPublication::Recovering { + previous: Some(Box::new(result)), + } + } else { + RecoveryPublication::Finished(Box::new(result)) + }; + if let Some(slot) = slots.get_mut(view.soul_id.as_str()) { + transition( + view.soul_id.as_str(), + Some(&slot.publication), + &publication, + view.terminal_id.as_deref(), + Some(view.incarnation_id.as_str()), + ); + slot.incarnation_id = Some(view.incarnation_id.to_string()); + slot.terminal_id = view.terminal_id.clone(); + slot.intent_revision = Some(view.intent_revision); + slot.evidence_revision = Some(view.evidence_revision); + slot.publication = publication; + } else { + transition( + view.soul_id.as_str(), + None, + &publication, + view.terminal_id.as_deref(), + Some(view.incarnation_id.as_str()), + ); + slots.insert( + view.soul_id.to_string(), + SoulRecoverySlot { + sequence: 0, + task_in_flight: false, + incarnation_id: Some(view.incarnation_id.to_string()), + terminal_id: view.terminal_id.clone(), + intent_revision: Some(view.intent_revision), + evidence_revision: Some(view.evidence_revision), + publication, + observed_epoch: None, + observed_exit_code: None, + observed_completion: None, + }, + ); + } } - async fn schedule_recovery(&self, soul: SoulId, trigger: RecoveryTrigger) { - let key = soul.to_string(); - { - let mut in_flight = self.recovery.in_flight.lock().await; - if !in_flight.insert(key.clone()) { - return; + async fn begin_recovery(&self, soul: &SoulId, manual: bool) -> Option { + self.begin_recovery_observed(soul, manual, None).await + } + + async fn begin_recovery_observed( + &self, + soul: &SoulId, + manual: bool, + observation: Option, + ) -> Option { + let mut slots = self.recovery.slots.lock().await; + if let Some(slot) = slots.get_mut(soul.as_str()) { + if let Some(observation) = observation { + slot.observe(observation); } + if slot.task_in_flight { + return None; + } + if !manual && lifecycle(&slot.publication).state != ManagedLifecycleState::Running { + return None; + } + let sequence = slot.sequence.checked_add(1)?; + let publication = RecoveryPublication::Recovering { + previous: slot.publication.result().cloned().map(Box::new), + }; + transition( + soul.as_str(), + Some(&slot.publication), + &publication, + slot.terminal_id.as_deref(), + slot.incarnation_id.as_deref(), + ); + slot.sequence = sequence; + slot.task_in_flight = true; + slot.publication = publication; + Some(sequence) + } else { + let publication = RecoveryPublication::Recovering { previous: None }; + let mut slot = SoulRecoverySlot { + sequence: 1, + task_in_flight: true, + incarnation_id: None, + terminal_id: None, + intent_revision: None, + evidence_revision: None, + publication, + observed_epoch: None, + observed_exit_code: None, + observed_completion: None, + }; + if let Some(observation) = observation { + slot.observe(observation); + } + transition( + soul.as_str(), + None, + &slot.publication, + slot.terminal_id.as_deref(), + slot.incarnation_id.as_deref(), + ); + slots.insert(soul.to_string(), slot); + Some(1) } - self.recovery.blocked.lock().await.remove(&key); - let client = self.client.clone(); - let state = Arc::clone(&self.recovery); + } + + async fn publish_recovery( + &self, + soul: &SoulId, + sequence: u64, + result: &Result, + ) { + let mut slots = self.recovery.slots.lock().await; + let Some(slot) = slots.get_mut(soul.as_str()) else { + return; + }; + if slot.sequence != sequence + || !slot.task_in_flight + || !matches!(slot.publication, RecoveryPublication::Recovering { .. }) + { + return; + } + let publication = match result { + Ok(result) => RecoveryPublication::Finished(Box::new(result.clone())), + Err(error) => RecoveryPublication::Failed { + code: error + .runtime_code() + .unwrap_or(RuntimeErrorCode::HostUnreachable), + previous: slot.publication.result().cloned().map(Box::new), + }, + }; + if let Ok(result) = result { + slot.incarnation_id = Some(result.view.incarnation_id.to_string()); + slot.terminal_id = result.view.terminal_id.clone(); + slot.intent_revision = Some(result.view.intent_revision); + slot.evidence_revision = Some(result.view.evidence_revision); + } + transition( + soul.as_str(), + Some(&slot.publication), + &publication, + slot.terminal_id.as_deref(), + slot.incarnation_id.as_deref(), + ); + slot.publication = publication; + slot.task_in_flight = false; + } + + async fn schedule_recovery( + &self, + soul: SoulId, + trigger: RecoveryTrigger, + observation: Option, + ) { + let Some(sequence) = self + .begin_recovery_observed(&soul, false, observation) + .await + else { + return; + }; + let controller = self.clone(); tokio::spawn(async move { if let Some(delay) = managed_recovery_test_delay() { tokio::time::sleep(delay).await; } - let result = recover_with_policy(&client, soul, trigger).await; - match result { - Ok(result) - if matches!( - result.outcome, - RecoveryOutcome::Replaced - | RecoveryOutcome::Reattached - | RecoveryOutcome::AlreadyLive - ) => - { - state.blocked.lock().await.remove(&key); - } - Ok(result) => { - state.blocked.lock().await.insert( - key.clone(), - format!( - "managed soul recovery ended in {:?}: {}", - result.outcome, - recovery_probe_summary(result.probe.as_ref()) - ), - ); - } - Err(error) => { - state.blocked.lock().await.insert(key.clone(), error); - } - } - state.in_flight.lock().await.remove(&key); + let result = recover_with_policy(&controller.client, soul.clone(), trigger).await; + controller.publish_recovery(&soul, sequence, &result).await; }); } + + pub async fn retry_with_request_id( + &self, + request: RequestId, + soul: SoulId, + revision: u64, + ) -> Result { + let sequence = self.begin_recovery(&soul, true).await.ok_or_else(|| { + ClientError::Runtime( + RuntimeErrorCode::InvalidRequest, + "recovery is already in progress".into(), + ) + })?; + let result = self + .client + .recover_expected_with_request_id( + request, + soul.clone(), + RecoveryTrigger::ManualRetry, + Some(revision), + ) + .await; + self.publish_recovery(&soul, sequence, &result).await; + result + } } impl ManagedTerminalController for ServerManagedRuntimeController { + fn lifecycle_observation<'a>( + &'a self, + terminal: &'a ManagedTerminalDescriptor, + ) -> ManagedTerminalFuture<'a, Option> { + Box::pin(async move { + self.recovery + .slots + .lock() + .await + .get(&terminal.soul_id) + .map(|slot| lifecycle(&slot.publication)) + }) + } + fn lookup_terminal<'a>( &'a self, terminal_id: &'a str, @@ -132,16 +485,14 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .inventory() .await .map_err(|error| error.to_string())?; - let Some(view) = views.iter().rev().find(|view| { - view.terminal_id.as_deref() == Some(terminal_id) - && view.desired_state == DesiredState::Running - && !matches!( - view.launch_state, - LaunchState::Stopped | LaunchState::Failed - ) - }) else { + let Some(view) = views + .iter() + .rev() + .find(|view| view.terminal_id.as_deref() == Some(terminal_id)) + else { return Ok(None); }; + self.hydrate(view.clone()).await; managed_descriptor_from_view(view, create_request_id).map(Some) }) } @@ -443,6 +794,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { } } }; + self.hydrate(result.view.clone()).await; Ok(ManagedTerminalDescriptor { soul_id: result.view.soul_id.to_string(), incarnation_id: result.view.incarnation_id.to_string(), @@ -464,41 +816,21 @@ impl ManagedTerminalController for ServerManagedRuntimeController { Box::pin(async move { let soul_key = terminal.soul_id; let soul = SoulId::parse(&soul_key).map_err(|e| e.to_string())?; - if self.recovery_in_flight(&soul_key).await { + if self + .recovery + .slots + .lock() + .await + .get(&soul_key) + .is_some_and(|slot| { + lifecycle(&slot.publication).state != ManagedLifecycleState::Running + }) + { return Err( - "managed soul recovery is in progress; input was not dispatched".into(), + "managed runtime is not running; input was not dispatched; use explicit retry" + .into(), ); } - if self.blocked_recovery(&soul_key).await.is_some() { - // New user input is an explicit retry/repair signal. Re-arm the - // persisted flap ceiling and prove continuity before accepting - // any bytes; never substitute a fresh provider session. - let recovered = self - .client - .recover(soul.clone(), RecoveryTrigger::ManualRetry) - .await - .map_err(|error| error.to_string())?; - match recovered.outcome { - RecoveryOutcome::Reattached - | RecoveryOutcome::Replaced - | RecoveryOutcome::AlreadyLive => { - self.recovery.blocked.lock().await.remove(&soul_key); - } - _ => { - let message = format!( - "managed soul recovery ended in {:?}: {}", - recovered.outcome, - recovery_probe_summary(recovered.probe.as_ref()) - ); - self.recovery - .blocked - .lock() - .await - .insert(soul_key, message.clone()); - return Err(message); - } - } - } self.client .input(RequestId::new(), soul, data) .await @@ -537,8 +869,18 @@ impl ManagedTerminalController for ServerManagedRuntimeController { { StopOutcome::VerifiedEmpty => { crate::managed_mcp_capability::revoke_soul_capabilities(&soul)?; - self.recovery.in_flight.lock().await.remove(&soul_key); - self.recovery.blocked.lock().await.remove(&soul_key); + let views = self + .client + .inventory() + .await + .map_err(|error| error.to_string())?; + if let Some(view) = views + .into_iter() + .rev() + .find(|view| view.soul_id.as_str() == soul_key) + { + self.hydrate(view).await; + } Ok(()) } other => Err(format!( @@ -557,22 +899,6 @@ impl ManagedTerminalController for ServerManagedRuntimeController { Box::pin(async move { let soul_key = terminal.soul_id.clone(); let soul = SoulId::parse(&soul_key).map_err(|e| e.to_string())?; - if let Some(error) = self.blocked_recovery(&soul_key).await { - return Err(error); - } - if self.recovery_in_flight(&soul_key).await { - return Ok(ManagedOutputRead { - stream_epoch: None, - incarnation_id: None, - reset_required: false, - truncated: false, - retained_from_seq: after_seq, - head_seq: after_seq, - exit_code: None, - native_session_id: terminal.resume_session_id, - chunks: Vec::new(), - }); - } let output = match self .client .read_output_for_epoch( @@ -585,31 +911,69 @@ impl ManagedTerminalController for ServerManagedRuntimeController { { Ok(output) => output, Err(error) if recoverable_host_error(&error) => { - self.schedule_recovery(soul, RecoveryTrigger::HostUnreachable) + self.schedule_recovery(soul, RecoveryTrigger::HostUnreachable, None) .await; - return Ok(ManagedOutputRead { - stream_epoch: None, - incarnation_id: None, - reset_required: false, - truncated: false, - retained_from_seq: after_seq, - head_seq: after_seq, - exit_code: None, - native_session_id: terminal.resume_session_id, - chunks: Vec::new(), - }); + return Err("managed retained output unavailable".into()); } Err(error) => return Err(error.to_string()), }; managed_output_validation::validate(&output, &terminal.terminal_id)?; - if output.exited { - // Never hold the WS poll's 500ms timeout across stop/start or - // retry backoff. A single per-soul background task owns the - // 2s/10s retry policy and leaves the pane attached meanwhile. - self.schedule_recovery(soul, RecoveryTrigger::ProviderExit) - .await; + let mut observation = self + .recovery + .slots + .lock() + .await + .get(&soul_key) + .map(|slot| lifecycle(&slot.publication)); + // Keep the actual EOF/epoch before scheduling recovery. Neither a + // recovery verdict nor a later flag may change this sampled head. + let completion = output.exited.then(|| ManagedOutputCompletion { + stream_epoch: output.stream_epoch.clone(), + incarnation_id: output.incarnation_id.to_string(), + head_seq: output.head_seq as i64, + exit_code: output.exit_code, + }); + { + let mut slots = self.recovery.slots.lock().await; + if let Some(slot) = slots.get_mut(&soul_key) { + slot.observed_epoch = Some(output.stream_epoch.clone()); + slot.observed_exit_code = output.exit_code; + slot.observed_completion = completion.clone(); + } } + if output.exited + && observation + .as_ref() + .is_none_or(|l| l.state == ManagedLifecycleState::Running) + { + self.schedule_recovery( + soul.clone(), + RecoveryTrigger::ProviderExit, + Some(RecoveryObservation { + terminal_id: terminal.terminal_id.clone(), + incarnation_id: output.incarnation_id.to_string(), + epoch: output.stream_epoch.clone(), + exit_code: output.exit_code, + completion: completion.clone(), + }), + ) + .await; + observation = self + .recovery + .slots + .lock() + .await + .get(&soul_key) + .map(|slot| lifecycle(&slot.publication)); + } + let observation = observation.unwrap_or(ManagedLifecycle { + state: ManagedLifecycleState::Running, + recovery_summary: None, + }); + let ended = observation.state == ManagedLifecycleState::Ended; Ok(ManagedOutputRead { + lifecycle: Some(observation), + completion: ended.then_some(completion).flatten(), stream_epoch: Some(output.stream_epoch.clone()), incarnation_id: Some(output.incarnation_id.to_string()), // Replacement starts a new durable output epoch. Force the @@ -619,12 +983,10 @@ impl ManagedTerminalController for ServerManagedRuntimeController { truncated: output.truncated, retained_from_seq: output.retained_from_seq.min(i64::MAX as u64) as i64, head_seq: output.head_seq.min(i64::MAX as u64) as i64, - // Provider exit is observed state, not user stop. Keep the pane - // alive while the recovery task runs or reports BLOCKED. - exit_code: if output.exited { - None - } else { + exit_code: if ended && output.exited { output.exit_code + } else { + None }, native_session_id: output.native_session_id, chunks: output @@ -672,7 +1034,7 @@ async fn recover_with_policy( client: &RuntimeClient, soul: SoulId, trigger: RecoveryTrigger, -) -> Result { +) -> Result { let mut last_result = None; let mut last_error = None; for attempt in 0..=AUTOMATIC_RECOVERY_RETRY_DELAYS.len() { @@ -686,16 +1048,13 @@ async fn recover_with_policy( } last_result = Some(result); } - Err(error) => last_error = Some(error.to_string()), + Err(error) => last_error = Some(error), } } if last_result.is_some() { - return client - .recover(soul, RecoveryTrigger::RetryExhausted) - .await - .map_err(|error| error.to_string()); + return client.recover(soul, RecoveryTrigger::RetryExhausted).await; } - Err(last_error.unwrap_or_else(|| "managed recovery exhausted without a verdict".into())) + Err(last_error.unwrap_or(ClientError::UnexpectedResult)) } fn automatic_retryable(result: &RecoveryResult) -> bool { @@ -717,22 +1076,6 @@ fn automatic_retryable(result: &RecoveryResult) -> bool { ) } -fn recovery_probe_summary(probe: Option<&RecoveryProbe>) -> String { - match probe { - Some(RecoveryProbe::Blocked { - reason, retry_hint, .. - }) => retry_hint - .repair - .clone() - .unwrap_or_else(|| format!("{reason:?}")), - Some(RecoveryProbe::DefinitivelyUnavailable { reason, .. }) => reason.clone(), - Some(RecoveryProbe::PristineSeedReady { .. }) => "pristine seed ready".into(), - Some(RecoveryProbe::ResumeReady { .. }) => "native resume ready".into(), - Some(RecoveryProbe::ReattachReady { .. }) => "reattach ready".into(), - None => "no recovery probe was returned".into(), - } -} - fn managed_descriptor_from_view( view: &RuntimeView, reconcile_create_request_id: Option, @@ -1022,6 +1365,107 @@ fn git_stdout(cwd: &Path, args: &[&str]) -> Option { mod tests { use super::*; #[cfg(unix)] + mod lifecycle_directory { + include!("tests/managed_runtime_lifecycle_tests.rs"); + } + pub(super) mod transitions { + include!("tests/managed_recovery_transition_tests.rs"); + } + + #[cfg(unix)] + #[tokio::test] + async fn shell_terminal_ended_facade_retains_output() { + use freshell_runtime_protocol::*; + let root = tempfile::tempdir().unwrap(); + let socket = root.path().join("retained.sock"); + let listener = tokio::net::UnixListener::bind(&socket).unwrap(); + let soul = SoulId::new(); + let incarnation = IncarnationId::new(); + let view: RuntimeView = serde_json::from_value(serde_json::json!({ + "soulId":soul,"incarnationId":incarnation,"launchState":"stopped", + "cleanupState":"verified_empty","intentRevision":1,"executionGeneration":1, + "terminalId":"ended-shell","terminalStreamId":"actual-host-epoch", + "terminalMode":"shell","terminalCwd":"/workspace", + "desiredState":"running","recoveryState":"lost", + "recoveryReason":"non_resumable_terminal_ended", + "nativeSessionId":"identity-retained" + })) + .unwrap(); + let (shutdown, mut stop) = tokio::sync::oneshot::channel(); + let task = tokio::spawn(async move { + loop { + let mut peer = tokio::select! { + _ = &mut stop => break, + accepted = listener.accept() => accepted.unwrap().0, + }; + let request: Envelope = read_frame(&mut peer).await.unwrap(); + let result = match request.body { + AdminCommand::Health => AdminResult::Health { + control_epoch: 1, + installation_id: InstallationId::new(), + }, + AdminCommand::Inventory => AdminResult::Inventory(vec![view.clone()]), + AdminCommand::TerminalReadOutput(_) => { + AdminResult::TerminalOutput(RuntimeOutputBatch { + incarnation_id: incarnation.clone(), + terminal_id: "ended-shell".into(), + stream_epoch: "actual-host-epoch".into(), + retained_from_seq: 1, + head_seq: 1, + reset_required: false, + truncated: false, + exited: true, + completion: None, + exit_code: Some(17), + native_session_id: Some("identity-retained".into()), + frames: vec![RuntimeOutputFrame { + terminal_id: "ended-shell".into(), + stream_epoch: "actual-host-epoch".into(), + seq_start: 1, + seq_end: 1, + data: "final shell diagnostic".into(), + }], + }) + } + other => panic!("read-only reconnect must not recover or launch: {other:?}"), + }; + write_frame( + &mut peer, + &AdminReply { + request_id: request.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + }); + let controller = ServerManagedRuntimeController { + client: RuntimeClient::new(&socket, "synthetic-retained"), + recovery: Arc::new(ManagedRecoveryState::default()), + }; + let descriptor = controller + .lookup_terminal("ended-shell", None) + .await + .unwrap(); + let read = if let Some(descriptor) = &descriptor { + Some(controller.read_output(descriptor.clone(), 0, 65536).await) + } else { + None + }; + shutdown.send(()).unwrap(); + task.await.unwrap(); + let descriptor = + descriptor.expect("verified ended shell keeps its exact facade on cold reconnect"); + assert_eq!( + descriptor.resume_session_id.as_deref(), + Some("identity-retained") + ); + let read = read.unwrap().expect("ended shell output remains readable"); + assert_eq!(read.chunks[0].data, "final shell diagnostic"); + assert_eq!(read.exit_code, Some(17)); + } + #[cfg(unix)] mod recovery_budget { include!("tests/managed_recovery_budget_tests.rs"); } diff --git a/crates/freshell-server/src/managed_runtime_api.rs b/crates/freshell-server/src/managed_runtime_api.rs index 882528e24..9da8c0d0f 100644 --- a/crates/freshell-server/src/managed_runtime_api.rs +++ b/crates/freshell-server/src/managed_runtime_api.rs @@ -159,6 +159,7 @@ impl ManagedViewProjectionStore { pub struct ManagedRuntimeApiState { pub auth_token: Arc, pub client: Option, + terminal_controller: Option>, projection_store: ManagedViewProjectionStore, pane_ledger: Arc, broadcast_tx: Arc>, @@ -175,6 +176,7 @@ impl ManagedRuntimeApiState { let state = Self { auth_token, client, + terminal_controller: None, projection_store: ManagedViewProjectionStore::open(projection_path)?, pane_ledger, broadcast_tx, @@ -190,6 +192,14 @@ impl ManagedRuntimeApiState { Ok(state) } + pub fn with_terminal_controller( + mut self, + controller: Option>, + ) -> Self { + self.terminal_controller = controller; + self + } + pub fn spawn_projector(&self) { if self.client.is_none() { return; @@ -496,15 +506,21 @@ async fn retry_soul( Ok(client) => client, Err(rejection) => return rejection.into_response(), }; - match client - .recover_expected_with_request_id( - request_id, - soul, - RecoveryTrigger::ManualRetry, - Some(revision), - ) - .await - { + let result = if let Some(controller) = &state.terminal_controller { + controller + .retry_with_request_id(request_id, soul, revision) + .await + } else { + client + .recover_expected_with_request_id( + request_id, + soul, + RecoveryTrigger::ManualRetry, + Some(revision), + ) + .await + }; + match result { Ok(result) => Json(result).into_response(), Err(error) => client_error(error), } diff --git a/crates/freshell-server/src/terminals.rs b/crates/freshell-server/src/terminals.rs index 6780e0486..ecc6e16f7 100644 --- a/crates/freshell-server/src/terminals.rs +++ b/crates/freshell-server/src/terminals.rs @@ -728,13 +728,31 @@ async fn directory_items(state: &TerminalsState) -> Vec { obj.insert( "status".into(), Value::String( - match e.status { - TerminalRunStatus::Running => "running", - TerminalRunStatus::Exited => "exited", + match e.runtime_status { + Some(freshell_protocol::RuntimeStatus::Blocked) => "blocked", + Some(freshell_protocol::RuntimeStatus::Recovering) => "recovering", + Some(freshell_protocol::RuntimeStatus::Running) => "running", + Some(freshell_protocol::RuntimeStatus::Exited) => "exited", + None => match e.status { + TerminalRunStatus::Running => "running", + TerminalRunStatus::Exited => "exited", + }, } .to_string(), ), ); + if let Some(status) = e.runtime_status { + obj.insert("runtimeStatus".into(), json!(status)); + } + if let Some(summary) = e.recovery_summary { + obj.insert("recoverySummary".into(), json!(summary)); + } + if let Some(soul) = e.soul_id { + obj.insert("soulId".into(), json!(soul)); + } + if let Some(incarnation) = e.incarnation_id { + obj.insert("incarnationId".into(), json!(incarnation)); + } obj.insert("hasClients".into(), Value::Bool(e.has_clients)); if resume_target_is_subagent { obj.insert("resumeTargetIsSubagent".into(), Value::Bool(true)); diff --git a/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs b/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs new file mode 100644 index 000000000..aeb4ff0b8 --- /dev/null +++ b/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs @@ -0,0 +1,74 @@ +use super::*; +#[derive(Clone)] +pub(crate) struct CaptureLogs(pub(crate) Arc>>); +#[derive(Default)] +struct Fields(serde_json::Map); +impl tracing::field::Visit for Fields { + fn record_str(&mut self, field: &tracing::field::Field, value: &str) { + self.0.insert(field.name().into(), value.into()); + } + fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) { + self.0 + .insert(field.name().into(), format!("{value:?}").into()); + } +} +impl tracing_subscriber::Layer for CaptureLogs { + fn on_event(&self, event: &tracing::Event<'_>, _: tracing_subscriber::layer::Context<'_, S>) { + let mut fields = Fields::default(); + event.record(&mut fields); + self.0.lock().unwrap().push( + serde_json::json!({"severity":event.metadata().level().as_str(),"fields":fields.0}), + ); + } +} +#[tokio::test] +async fn managed_runtime_recovery_transition_changes_cause_only_and_redacts_failed_payload() { + use tracing::instrument::WithSubscriber; + use tracing_subscriber::prelude::*; + let bytes = Arc::new(std::sync::Mutex::new(vec![])); + let capture = bytes.clone(); + let subscriber = tracing_subscriber::registry().with(CaptureLogs(capture)); + let (before_retry,soul)=async { + let soul=SoulId::new();let incarnation=freshell_runtime_protocol::IncarnationId::new(); + let controller=ServerManagedRuntimeController {client:RuntimeClient::new("/unused-transition-fixture","synthetic"),recovery:Arc::default()}; + let mut view:RuntimeView=serde_json::from_value(serde_json::json!({"soulId":soul,"incarnationId":incarnation, + "launchState":"running","cleanupState":"none","desiredState":"running","recoveryState":"blocked", + "intentRevision":1,"executionGeneration":1,"terminalId":"transition-terminal","terminalStreamId":"actual-epoch", + "terminalMode":"opencode","terminalCwd":"/workspace","recoveryReason":"ambiguous_identity", + "incidentId":"incident-first","recoveryAttemptId":"recovery-first"})).unwrap(); + for _ in 0..40 {controller.hydrate(view.clone()).await;} + view.incident_id=Some(freshell_runtime_protocol::IncidentId::parse("incident-second").unwrap()); + for _ in 0..40 {controller.hydrate(view.clone()).await;} + let before_retry=bytes.lock().unwrap().len(); + let sequence=controller.begin_recovery(&soul,true).await.unwrap(); + controller.publish_recovery(&soul,sequence,&Err(ClientError::Runtime(RuntimeErrorCode::HostUnreachable,"raw secret=synthetic-sensitive-error".into()))).await; + for _ in 0..40 {controller.publish_recovery(&soul,sequence,&Err(ClientError::Runtime(RuntimeErrorCode::HostUnreachable,"raw secret=synthetic-sensitive-error".into()))).await;} + let sequence=controller.begin_recovery(&soul,true).await.unwrap(); + controller.publish_recovery(&soul,sequence,&Err(ClientError::Runtime(RuntimeErrorCode::InvalidRequest,"raw secret=synthetic-sensitive-error".into()))).await; + let sequence=controller.begin_recovery(&soul,true).await.unwrap(); + view.recovery_state=RecoveryState::Live;view.recovery_reason=None; + controller.publish_recovery(&soul,sequence,&Ok(RecoveryResult {outcome:RecoveryOutcome::AlreadyLive,view,probe:None, + prior_incarnation_id:None,attempt_id:None,expected_native_session_id:None,observed_native_session_id:None,incident_id:None})).await; + (before_retry,soul) + }.with_subscriber(subscriber).await; + assert_eq!( + before_retry, 1, + "unchanged lifecycle and classified cause do not log metadata-only updates" + ); + let events = bytes.lock().unwrap().clone(); + assert!(!serde_json::to_string(&events) + .unwrap() + .contains("synthetic-sensitive-error")); + assert_eq!(events.len(), 7); + for event in events { + assert_eq!( + event["fields"]["event"], + "managed_runtime.recovery_transition" + ); + assert_eq!(event["fields"]["soul_id"], soul.as_str()); + assert_eq!(event["fields"]["terminal_id"], "transition-terminal"); + for field in ["incarnation_id", "from", "to", "cause"] { + assert!(!event["fields"][field].is_null()); + } + } +} diff --git a/crates/freshell-server/src/tests/managed_runtime_lifecycle_tests.rs b/crates/freshell-server/src/tests/managed_runtime_lifecycle_tests.rs new file mode 100644 index 000000000..4419794f3 --- /dev/null +++ b/crates/freshell-server/src/tests/managed_runtime_lifecycle_tests.rs @@ -0,0 +1,388 @@ +use super::*; +use freshell_runtime_protocol::*; +use freshell_terminal::TerminalRegistry; +use serde_json::{json, Value}; + +async fn tool_listing(url: &str, mcp: bool) -> Value { + let mut command = tokio::process::Command::new("node"); + command.args(["--import", "tsx"]); + if mcp { + command.args(["--input-type=module","-e", + "import {executeAction} from './tools/freshell-mcp/freshell-tool.ts'; executeAction('list-terminals').then(v=>{console.log(JSON.stringify(v));process.exit(0)}).catch(e=>{console.error(e);process.exit(1)});"]); + } else { + command.args(["tools/freshell-cli/index.ts", "list-terminals"]); + } + command + .current_dir( + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap(), + ) + .env("FRESHELL_URL", url) + .env("FRESHELL_TOKEN", "synthetic-directory-token") + .kill_on_drop(true); + let result = tokio::time::timeout(std::time::Duration::from_secs(20), command.output()) + .await + .unwrap() + .unwrap(); + assert!( + result.status.success(), + "actual tool failed: {}", + String::from_utf8_lossy(&result.stderr) + ); + serde_json::from_slice(&result.stdout).expect("actual tool JSON") +} + +async fn directory_fixture( + recovery_state: &str, +) -> ( + Vec, + Vec, + bool, + String, +) { + directory_fixture_with_retry(recovery_state, false).await +} +async fn directory_fixture_with_retry( + recovery_state: &str, + manual_retry: bool, +) -> ( + Vec, + Vec, + bool, + String, +) { + let root = tempfile::tempdir().unwrap(); + let socket = root.path().join("directory-control.sock"); + let listener = tokio::net::UnixListener::bind(&socket).unwrap(); + let soul = SoulId::new(); + let incarnation = IncarnationId::new(); + let ended = matches!(recovery_state, "lost" | "stopped"); + let outcome = if recovery_state == "lost" { + RecoveryOutcome::Lost + } else if recovery_state == "stopped" { + RecoveryOutcome::Stopped + } else { + RecoveryOutcome::Blocked + }; + let view:RuntimeView=serde_json::from_value(json!({"soulId":soul,"incarnationId":incarnation, + "launchState":if ended {"stopped"} else {"running"},"cleanupState":if ended {"verified_empty"} else {"none"}, + "intentRevision":1,"executionGeneration":1,"terminalId":"directory-retained","terminalStreamId":"directory-epoch", + "terminalMode":if ended {"shell"} else {"opencode"},"terminalCwd":"/workspace", + "desiredState":if recovery_state=="stopped" {"stopped"} else {"running"},"recoveryState":recovery_state, + "recoveryReason":if ended {"non_resumable_terminal_ended"} else {"ambiguous_identity"}, + "nativeSessionId":"native-directory-retained","incidentId":"incident-directory-retained", + "recoveryAttemptId":"recovery-directory-retained" + })).unwrap(); + let mut control_view = view.clone(); + let (stop_control, mut control_stop) = tokio::sync::oneshot::channel(); + let control = tokio::spawn(async move { + loop { + let mut peer = tokio::select! {_=&mut control_stop=>break,accepted=listener.accept()=>accepted.unwrap().0}; + let request: Envelope = read_frame(&mut peer).await.unwrap(); + let result = match request.body { + AdminCommand::Health => AdminResult::Health { + control_epoch: 1, + installation_id: InstallationId::new(), + }, + AdminCommand::Inventory => AdminResult::Inventory(vec![control_view.clone()]), + AdminCommand::InventorySnapshot => { + AdminResult::InventorySnapshot(RuntimeInventorySnapshot { + revision: 1, + readiness: serde_json::from_value( + json!({"inventoryRevision":1,"initialScanState":"complete"}), + ) + .unwrap(), + souls: vec![control_view.clone()], + view_intents: vec![], + pending_projection_count: 0, + }) + } + AdminCommand::TerminalReadOutput(read) => { + AdminResult::TerminalOutput(RuntimeOutputBatch { + incarnation_id: incarnation.clone(), + terminal_id: "directory-retained".into(), + stream_epoch: "directory-epoch".into(), + retained_from_seq: 1, + head_seq: 1, + reset_required: false, + truncated: false, + exited: ended, + completion: None, + exit_code: ended.then_some(23), + native_session_id: Some("native-directory-retained".into()), + frames: if read.after_seq == 0 { + vec![RuntimeOutputFrame { + terminal_id: "directory-retained".into(), + stream_epoch: "directory-epoch".into(), + seq_start: 1, + seq_end: 1, + data: "retained directory diagnostic".into(), + }] + } else { + vec![] + }, + }) + } + AdminCommand::Recover(request) => { + if request.trigger == RecoveryTrigger::ManualRetry { + control_view.recovery_state = RecoveryState::Live; + control_view.launch_state = LaunchState::Running; + control_view.cleanup_state = CleanupState::None; + control_view.recovery_reason = None; + } + AdminResult::Recovery(RecoveryResult { + outcome: if request.trigger == RecoveryTrigger::ManualRetry { + RecoveryOutcome::AlreadyLive + } else { + outcome + }, + view: control_view.clone(), + probe: None, + prior_incarnation_id: None, + attempt_id: control_view.recovery_attempt_id.clone(), + expected_native_session_id: None, + observed_native_session_id: None, + incident_id: control_view.incident_id.clone(), + }) + } + AdminCommand::TerminalInput(_) => AdminResult::TerminalInput { + state: CommandState::Completed, + }, + other => panic!("unexpected managed directory command {other:?}"), + }; + write_frame( + &mut peer, + &AdminReply { + request_id: request.request_id, + result: Ok(result), + }, + ) + .await + .unwrap(); + } + }); + let controller = Arc::new(ServerManagedRuntimeController { + client: RuntimeClient::new(&socket, "synthetic-control"), + recovery: Arc::new(ManagedRecoveryState::default()), + }); + let registry = TerminalRegistry::new(); + registry.set_managed_controller(Some(controller.clone())); + // Register the same exact identity even on the old cold-lookup failure so + // every plain/paged/tool surface actually executes before RED assertions. + let descriptor = controller + .lookup_terminal("directory-retained", None) + .await + .unwrap() + .unwrap_or_else(|| managed_descriptor_from_view(&view, None).unwrap()); + registry.register_managed(descriptor); + let read = registry + .refresh_managed_output("directory-retained", 65536) + .await + .unwrap(); + registry.publish_managed_lifecycle("directory-retained", read.lifecycle.clone()); + if let Some(code) = read.exit_code { + if registry.managed_output_cursor("directory-retained") == Some(read.head_seq) { + registry.finish_managed_exit("directory-retained", code); + } + } + for _ in 0..100 { + if !controller.recovery_in_flight(soul.as_str()).await { + break; + } + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + let (broadcast, _) = tokio::sync::broadcast::channel(16); + let state = crate::terminals::TerminalsState { + auth_token: Arc::new("synthetic-directory-token".into()), + settings: crate::settings_store::SettingsStore::load(Some(root.path()), vec![]), + registry: registry.clone(), + broadcast_tx: Arc::new(broadcast), + terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), + identity: freshell_ws::identity::TerminalIdentityRegistry::new(), + }; + let api_state = crate::managed_runtime_api::ManagedRuntimeApiState::new( + Arc::new("synthetic-directory-token".into()), + Some(controller.runtime_client()), + None, + Arc::new(freshell_ws::pane_ledger::PaneLedger::new(Some( + root.path().join("ledger"), + ))), + state.broadcast_tx.clone(), + ) + .await + .unwrap() + .with_terminal_controller(Some(controller.clone())); + let app = crate::terminals::router(state).merge(crate::managed_runtime_api::router(api_state)); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + let (stop_http, http_stop) = tokio::sync::oneshot::channel(); + let http = tokio::spawn(async move { + axum::serve(listener, app) + .with_graceful_shutdown(async { + http_stop.await.ok(); + }) + .await + .unwrap(); + }); + let client = reqwest::Client::builder().no_proxy().build().unwrap(); + if manual_retry { + let initial = registry + .managed_input("directory-retained", "blocked bytes".into()) + .await; + // Retain the result, and assert after both fixtures have actually joined. + let response = client + .post(format!("{url}/api/runtime/souls/{soul}/retry")) + .header("x-auth-token", "synthetic-directory-token") + .json(&json!({"requestId":RequestId::new(),"expectedIntentRevision":1})) + .send() + .await + .unwrap(); + let retry: Value = response.json().await.unwrap(); + let read = registry + .refresh_managed_output("directory-retained", 65536) + .await + .unwrap(); + registry.publish_managed_lifecycle("directory-retained", read.lifecycle); + let mut listings = vec![]; + for suffix in ["/api/terminals", "/api/terminals?priority=visible"] { + let listing: Value = client + .get(format!("{url}{suffix}")) + .header("x-auth-token", "synthetic-directory-token") + .send() + .await + .unwrap() + .json() + .await + .unwrap(); + listings.push(if listing.is_array() { + listing + } else { + listing["items"].clone() + }); + } + listings.push(tool_listing(&url, false).await); + listings.push(tool_listing(&url, true).await); + let input_blocked = registry + .managed_input("directory-retained", "retry bytes".into()) + .await + .is_err(); + let inventory = registry.inventory(); + let snapshot = registry.directory()[0].snapshot.clone(); + stop_http.send(()).unwrap(); + http.await.unwrap(); + stop_control.send(()).unwrap(); + control.await.unwrap(); + assert!( + initial.is_err(), + "cold retained blocked/recovering facade refuses input before manual retry" + ); + assert_eq!( + retry["outcome"], "already_live", + "actual HTTP retry publishes a full successful outcome" + ); + return (listings, inventory, input_blocked, snapshot); + } + let mut listings = vec![]; + for suffix in ["/api/terminals", "/api/terminals?priority=visible"] { + let response: Value = client + .get(format!("{url}{suffix}")) + .header("x-auth-token", "synthetic-directory-token") + .send() + .await + .unwrap() + .json() + .await + .unwrap(); + listings.push(if response.is_array() { + response + } else { + response["items"].clone() + }); + } + listings.push(tool_listing(&url, false).await); + listings.push(tool_listing(&url, true).await); + let input_blocked = registry + .managed_input("directory-retained", "not dispatched".into()) + .await + .is_err(); + let inventory = registry.inventory(); + let snapshot = registry.directory()[0].snapshot.clone(); + stop_http.send(()).unwrap(); + http.await.unwrap(); + stop_control.send(()).unwrap(); + control.await.unwrap(); + (listings, inventory, input_blocked, snapshot) +} + +#[tokio::test] +async fn managed_runtime_actual_plain_paged_mcp_cli_keep_blocked_retained_facade() { + let (listings, inventory, input_blocked, snapshot) = directory_fixture("blocked").await; + for listing in listings { + assert_eq!(listing[0]["terminalId"], "directory-retained"); + assert_eq!( + listing[0]["status"], "blocked", + "actual Rust facade and all pass-through listings report blocked" + ); + assert_eq!( + listing[0]["sessionRef"]["sessionId"], + "native-directory-retained" + ); + assert_eq!( + listing[0]["recoverySummary"]["incidentId"], + "incident-directory-retained" + ); + } + assert_eq!(inventory[0].terminal_id, "directory-retained"); + assert!( + input_blocked, + "blocked retained handle refuses input without automatic retry" + ); + assert_eq!(snapshot, "retained directory diagnostic"); +} + +#[tokio::test] +async fn managed_runtime_actual_plain_paged_mcp_cli_keep_verified_final_facades() { + for state in ["lost", "stopped"] { + let (listings, inventory, input_blocked, snapshot) = directory_fixture(state).await; + for listing in listings { + assert_eq!(listing[0]["status"], "exited"); + assert_eq!(listing[0]["terminalId"], "directory-retained"); + } + assert_eq!( + inventory[0].status, + freshell_protocol::TerminalRunStatus::Exited + ); + assert!(input_blocked); + assert_eq!(snapshot, "retained directory diagnostic"); + } +} + +#[tokio::test] +async fn managed_runtime_actual_http_retry_from_blocked_and_cold_recovering_restores_all_listings_and_input( +) { + for state in ["blocked", "recovering"] { + let (listings, inventory, input_blocked, snapshot) = + directory_fixture_with_retry(state, true).await; + for listing in listings { + assert_eq!(listing[0]["status"], "running"); + assert_eq!(listing[0]["terminalId"], "directory-retained"); + assert_eq!( + listing[0]["sessionRef"]["sessionId"], + "native-directory-retained" + ); + assert_eq!(listing[0]["recoverySummary"]["recoveryState"], "live"); + } + assert_eq!( + inventory[0].runtime_status, + Some(freshell_protocol::RuntimeStatus::Running) + ); + assert!( + !input_blocked, + "successful explicit retry unblocks actual input" + ); + assert_eq!(snapshot, "retained directory diagnostic"); + } +} diff --git a/crates/freshell-session-host/src/output_journal.rs b/crates/freshell-session-host/src/output_journal.rs index 69a4c59a0..d47d5a39d 100644 --- a/crates/freshell-session-host/src/output_journal.rs +++ b/crates/freshell-session-host/src/output_journal.rs @@ -1,9 +1,15 @@ -use freshell_runtime_protocol::{IncarnationId, RuntimeOutputBatch, RuntimeOutputFrame}; -use serde::{Deserialize, Serialize}; +use freshell_runtime_protocol::{ + terminal_spool::{ + scan_spool, RetainedOutputPage, TerminalSpoolFrame as SpoolFrame, SPOOL_COMPLETION_FILE, + SPOOL_DESCRIPTOR_FILE, + }, + IncarnationId, RuntimeOutputBatch, RuntimeOutputFrame, TerminalExitObservation, + TerminalSpoolCompletion, TerminalSpoolDescriptor, +}; use std::{ - collections::{BTreeMap, VecDeque}, + collections::VecDeque, fs::OpenOptions, - io::{BufRead, BufReader, Write}, + io::{BufReader, Write}, path::{Path, PathBuf}, }; @@ -11,12 +17,6 @@ pub const DEFAULT_RING_BYTES: usize = 1024 * 1024; pub const DEFAULT_SPOOL_BYTES: u64 = 64 * 1024 * 1024; pub const MAX_TRANSPORT_CHUNK_BYTES: usize = 64 * 1024; -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -struct SpoolFrame { - frame: RuntimeOutputFrame, -} - pub struct OutputJournal { incarnation_id: IncarnationId, terminal_id: String, @@ -27,6 +27,8 @@ pub struct OutputJournal { ring_limit: usize, spool_limit: u64, next_seq: u64, + completion: Option, + failure: Option, } impl OutputJournal { @@ -73,9 +75,23 @@ impl OutputJournal { ring: VecDeque::new(), ring_bytes: 0, ring_limit: ring_limit.max(MAX_TRANSPORT_CHUNK_BYTES), - spool_limit: spool_limit.max((MAX_TRANSPORT_CHUNK_BYTES * 4) as u64), + spool_limit: spool_limit + .clamp((MAX_TRANSPORT_CHUNK_BYTES * 4) as u64, DEFAULT_SPOOL_BYTES), next_seq: 1, + completion: None, + failure: None, }; + atomic_json( + &journal.state_dir.join(SPOOL_DESCRIPTOR_FILE), + &journal.descriptor(), + )?; + // Create a committed empty source before the PTY output writer exists. + OpenOptions::new() + .create(true) + .append(true) + .open(journal.current_path()) + .and_then(|file| file.sync_all()) + .map_err(|_| "initialize retained spool failed")?; journal.seed_sequence_from_spool()?; Ok(journal) } @@ -84,7 +100,6 @@ impl OutputJournal { let mut written = Vec::new(); for chunk in split_utf8(data, MAX_TRANSPORT_CHUNK_BYTES) { let seq = self.next_seq; - self.next_seq = self.next_seq.saturating_add(1); let frame = RuntimeOutputFrame { terminal_id: self.terminal_id.clone(), stream_epoch: self.stream_epoch.clone(), @@ -93,6 +108,10 @@ impl OutputJournal { data: chunk.to_string(), }; self.append_spool(&frame)?; + self.next_seq = self + .next_seq + .checked_add(1) + .ok_or("output sequence exhausted")?; self.ring_bytes = self.ring_bytes.saturating_add(frame.data.len()); self.ring.push_back(frame.clone()); while self.ring_bytes > self.ring_limit { @@ -106,74 +125,88 @@ impl OutputJournal { Ok(written) } - pub fn read(&self, after_seq: u64, max_bytes: u64) -> Result { - let budget = max_bytes.clamp(1, MAX_TRANSPORT_CHUNK_BYTES as u64) as usize; - let mut by_seq = BTreeMap::::new(); - for path in [self.previous_path(), self.current_path()] { - if !path.exists() { - continue; - } - let file = std::fs::File::open(path).map_err(|e| e.to_string())?; - for line in BufReader::new(file).lines() { - let line = line.map_err(|e| e.to_string())?; - if line.trim().is_empty() { - continue; - } - let row: SpoolFrame = serde_json::from_str(&line).map_err(|e| e.to_string())?; - if row.frame.stream_epoch == self.stream_epoch { - by_seq.insert(row.frame.seq_start, row.frame); - } + fn descriptor(&self) -> TerminalSpoolDescriptor { + TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id: self.incarnation_id.clone(), + terminal_id: self.terminal_id.clone(), + stream_epoch: self.stream_epoch.clone(), + segment_limit_bytes: self.spool_limit / 2, + } + } + + pub fn record_failure(&mut self) { + self.failure = Some("retained output writer failed".into()); + } + + pub fn complete(&mut self, code: i64) -> Result<(), String> { + if let Some(error) = &self.failure { + return Err(error.clone()); + } + let batch = self.read(0, MAX_TRANSPORT_CHUNK_BYTES as u64)?; + let completion = TerminalSpoolCompletion { + incarnation_id: self.incarnation_id.clone(), + terminal_id: self.terminal_id.clone(), + stream_epoch: self.stream_epoch.clone(), + retained_from_seq: batch.retained_from_seq, + head_seq: batch.head_seq, + exit: TerminalExitObservation::Known { code }, + }; + atomic_json(&self.state_dir.join(SPOOL_COMPLETION_FILE), &completion)?; + self.completion = Some(completion); + Ok(()) + } + + fn scan( + &self, + mut accept: impl FnMut(RuntimeOutputFrame) -> Result<(), String>, + ) -> Result { + let source = self.descriptor(); + let mut torn = false; + for (path, required) in [(self.previous_path(), false), (self.current_path(), true)] { + let file = match std::fs::File::open(path) { + Ok(file) => file, + Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => return Err("retained output spool unavailable".into()), + }; + if file + .metadata() + .map_err(|_| "retained spool metadata unavailable")? + .len() + > source.segment_limit_bytes + { + return Err("retained spool exceeds segment bound".into()); } + torn |= scan_spool(BufReader::new(file), &source, &mut accept)?; } - for frame in &self.ring { - by_seq.insert(frame.seq_start, frame.clone()); + Ok(torn) + } + + pub fn read(&self, after_seq: u64, max_bytes: u64) -> Result { + if let Some(error) = &self.failure { + return Err(error.clone()); } - let retained_from_seq = by_seq.keys().next().copied().unwrap_or(self.next_seq); - let head_seq = self.next_seq.saturating_sub(1); - let reset_required = - after_seq > head_seq || after_seq.saturating_add(1) < retained_from_seq; - let effective_after = if reset_required { - retained_from_seq.saturating_sub(1) + // First scan establishes bounds without retaining the spool in memory. + let mut first = None; + let mut head = 0; + self.scan(|frame| { + first.get_or_insert(frame.seq_start); + head = frame.seq_end; + Ok(()) + })?; + let retained = first.unwrap_or(head.saturating_add(1)); + let reset = after_seq > head || after_seq.saturating_add(1) < retained; + let effective = if reset { + retained.saturating_sub(1) } else { after_seq }; - let mut frames = Vec::new(); - let mut used = 0usize; - let mut more = false; - for (_, frame) in by_seq.range((effective_after.saturating_add(1))..) { - let cost = frame.data.len().max(1); - if !frames.is_empty() && used.saturating_add(cost) > budget { - more = true; - break; - } - if cost > budget && frames.is_empty() { - // append() already chunks at <=64KiB, but a smaller caller budget - // should still make forward progress without emitting an oversized frame. - let Some(first) = split_utf8(&frame.data, budget).first().copied() else { - continue; - }; - let mut partial = frame.clone(); - partial.data = first.to_string(); - frames.push(partial); - more = true; - break; - } - used = used.saturating_add(cost); - frames.push(frame.clone()); - } - Ok(RuntimeOutputBatch { - incarnation_id: self.incarnation_id.clone(), - terminal_id: self.terminal_id.clone(), - stream_epoch: self.stream_epoch.clone(), - retained_from_seq, - head_seq, - reset_required, - truncated: reset_required || more, - exited: false, - exit_code: None, - native_session_id: None, - frames, - }) + let mut page = RetainedOutputPage::new(self.descriptor(), effective, max_bytes)?; + let torn = self.scan(|frame| page.accept(frame))?; + let mut batch = page.finish(torn, self.completion.clone())?; + batch.reset_required |= reset; + batch.truncated |= reset; + Ok(batch) } pub fn head_seq(&self) -> u64 { @@ -195,7 +228,7 @@ impl OutputJournal { frame: frame.clone(), }) .map_err(|e| e.to_string())?; - let segment_limit = (self.spool_limit / 2).max((MAX_TRANSPORT_CHUNK_BYTES * 2) as u64); + let segment_limit = self.spool_limit / 2; let current = self.current_path(); let current_len = current.metadata().map(|m| m.len()).unwrap_or(0); if current_len > 0 && current_len.saturating_add(line.len() as u64 + 1) > segment_limit { @@ -217,22 +250,12 @@ impl OutputJournal { } fn seed_sequence_from_spool(&mut self) -> Result<(), String> { - let mut max = 0; - for path in [self.previous_path(), self.current_path()] { - if !path.exists() { - continue; - } - let file = std::fs::File::open(path).map_err(|e| e.to_string())?; - for line in BufReader::new(file).lines() { - let line = line.map_err(|e| e.to_string())?; - if let Ok(row) = serde_json::from_str::(&line) { - if row.frame.stream_epoch == self.stream_epoch { - max = max.max(row.frame.seq_end); - } - } - } - } - self.next_seq = max.saturating_add(1).max(1); + let mut head = 0; + self.scan(|frame| { + head = frame.seq_end; + Ok(()) + })?; + self.next_seq = head.checked_add(1).ok_or("output sequence exhausted")?; Ok(()) } @@ -244,6 +267,22 @@ impl OutputJournal { } } +pub fn atomic_json(path: &Path, value: &impl serde::Serialize) -> Result<(), String> { + let temporary = path.with_extension("tmp"); + let bytes = serde_json::to_vec(value).map_err(|_| "retained metadata encoding failed")?; + let mut file = OpenOptions::new() + .create(true) + .truncate(true) + .write(true) + .open(&temporary) + .map_err(|_| "retained metadata unavailable")?; + file.write_all(&bytes) + .and_then(|_| file.sync_all()) + .map_err(|_| "retained metadata write failed")?; + std::fs::rename(temporary, path).map_err(|_| "retained metadata commit failed")?; + sync_parent(path) +} + fn configured_bytes(key: &str, default: u64) -> u64 { std::env::var(key) .ok() @@ -289,6 +328,66 @@ fn sync_parent(path: &Path) -> Result<(), String> { mod tests { use super::*; + #[test] + fn retained_output_requires_existing_spool_instead_of_empty_success() { + let dir = tempfile::tempdir().unwrap(); + let journal = + OutputJournal::new(dir.path(), IncarnationId::new(), "t".into(), "e".into()).unwrap(); + // No append has happened: even an empty stream needs a durable source + // association and an existing empty spool before its writer can start. + assert!(dir.path().join("terminal-spool-current.jsonl").is_file()); + std::fs::remove_file(dir.path().join("terminal-spool-current.jsonl")).unwrap(); + assert!( + journal.read(0, 65536).is_err(), + "missing spool cannot masquerade as retained empty output" + ); + } + + #[test] + fn retained_output_ignores_only_a_torn_final_frame_and_keeps_complete_prefix() { + let dir = tempfile::tempdir().unwrap(); + let mut journal = + OutputJournal::new(dir.path(), IncarnationId::new(), "t".into(), "e".into()).unwrap(); + journal.append("complete diagnostic").unwrap(); + let mut file = OpenOptions::new() + .append(true) + .open(journal.current_path()) + .unwrap(); + file.write_all(b"{\"frame\":{\"data\":\"partial").unwrap(); + file.sync_data().unwrap(); + let read = journal + .read(0, 65536) + .expect("a torn tail preserves complete committed frames"); + assert_eq!(read.frames.len(), 1); + assert_eq!(read.frames[0].data, "complete diagnostic"); + assert!(read.truncated, "torn tail must be visible"); + } + + #[test] + fn retained_output_rejects_foreign_terminal_frames_in_same_epoch() { + let dir = tempfile::tempdir().unwrap(); + let mut journal = + OutputJournal::new(dir.path(), IncarnationId::new(), "t".into(), "e".into()).unwrap(); + journal.append("owned diagnostic").unwrap(); + let mut file = OpenOptions::new() + .append(true) + .open(journal.current_path()) + .unwrap(); + let foreign = RuntimeOutputFrame { + terminal_id: "foreign".into(), + stream_epoch: "e".into(), + seq_start: 2, + seq_end: 2, + data: "wrong soul".into(), + }; + serde_json::to_writer(&mut file, &SpoolFrame { frame: foreign }).unwrap(); + file.write_all(b"\n").unwrap(); + assert!( + journal.read(0, 65536).is_err(), + "matching epoch alone does not prove owned terminal" + ); + } + #[test] fn bounds_ring_and_returns_explicit_reset_for_expired_cursor() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-session-host/src/pty.rs b/crates/freshell-session-host/src/pty.rs index b9555baeb..6e78f094b 100644 --- a/crates/freshell-session-host/src/pty.rs +++ b/crates/freshell-session-host/src/pty.rs @@ -67,7 +67,14 @@ impl HostedPty { let sink = Box::new(move |message: ServerMessage| { if let ServerMessage::TerminalOutput(frame) = message { if let Ok(mut journal) = sink_output.lock() { - let _ = journal.append(&frame.data); + if journal.append(&frame.data).is_err() { + journal.record_failure(); + tracing::error!( + event = "managed_runtime.output_writer_failed", + cause = "spool_write", + "Retained output writer failed" + ); + } } } }); @@ -79,7 +86,21 @@ impl HostedPty { .as_ref() .map(|prepared| Arc::clone(&prepared.sidecar)); let runtime_handle = tokio::runtime::Handle::try_current().ok(); + let settled_output = Arc::clone(&output); let on_exit = Box::new(move |code: i64| { + // The PTY reader calls this only after its final framer flush. + let settled = settled_output + .lock() + .map(|mut journal| journal.complete(code).is_ok()) + .unwrap_or(false); + if !settled { + tracing::error!( + event = "managed_runtime.output_completion_failed", + cause = "spool_settlement", + "Output completion could not be committed" + ); + return; + } exit_value.store(code, Ordering::SeqCst); exit_flag.store(true, Ordering::SeqCst); if let (Some(handle), Some(sidecar)) = @@ -263,8 +284,6 @@ impl HostedPty { batch.reset_required = true; batch.truncated = true; } - batch.exited = self.exited(); - batch.exit_code = self.exited().then(|| self.exit_code()); batch.native_session_id = self .native_session_id .lock() @@ -274,7 +293,7 @@ impl HostedPty { } pub async fn stop(&mut self) { - self.pty.kill(); + self.pty.stop_and_join_output(); if let Some(codex) = self.codex.take() { codex.event_task.abort(); let _ = codex.sidecar.shutdown().await; @@ -450,6 +469,112 @@ fn output_read_position( mod output_epoch_tests { use super::output_read_position; + async fn retained_pty( + root: &std::path::Path, + script: &str, + ) -> (super::HostedPty, freshell_runtime_protocol::IncarnationId) { + let launch:super::TerminalLaunchSpec=serde_json::from_value(serde_json::json!({ + "terminalId":"retained-pty","streamId":"launch-prefix","mode":"shell","program":"/bin/sh", + "args":["-c",script],"env":{"PATH":"/usr/bin:/bin"},"cwd":root, + "runAsUid":unsafe {libc::geteuid()},"runAsGid":unsafe {libc::getegid()}, + "cols":120,"rows":30,"projectKey":"retained-fixture","workspacePath":root + })).unwrap(); + let incarnation = freshell_runtime_protocol::IncarnationId::new(); + ( + super::HostedPty::spawn( + root, + incarnation.clone(), + &launch, + Default::default(), + None, + None, + ) + .await + .unwrap(), + incarnation, + ) + } + + #[tokio::test] + async fn retained_output_pty_eof_flushes_final_head_before_completion() { + let root = tempfile::tempdir().unwrap(); + let (hosted, incarnation) = retained_pty( + root.path(), + "head -c 150000 /dev/zero | tr '\\000' 'x'; printf 'final-tail'; exit 17", + ) + .await; + tokio::time::timeout(std::time::Duration::from_secs(5), async { + while !hosted.exited() { + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + }) + .await + .unwrap(); + let completion = std::fs::read(root.path().join("terminal-spool-completion.json")); + let mut windows = vec![]; + let head = hosted.head_seq(); + for _reader in 0..2 { + let mut cursor = 0; + let mut window = String::new(); + while cursor < head { + let batch = hosted + .read_output( + incarnation.clone(), + cursor, + 65536, + Some(hosted.stream_epoch()), + ) + .unwrap(); + assert!(batch.exited); + assert_eq!(batch.head_seq, head); + assert_eq!(batch.exit_code, Some(17)); + for frame in batch.frames { + cursor = frame.seq_end; + window.push_str(&frame.data); + } + } + windows.push(window); + } + drop(hosted); // actual PTY waiter and reader join before assertions + assert_eq!(windows[0], "x".repeat(150000) + "final-tail"); + assert_eq!(windows[0], windows[1]); + let completion: serde_json::Value = serde_json::from_slice( + &completion.expect("settled EOF publishes coherent durable completion"), + ) + .unwrap(); + assert_eq!(completion["headSeq"], head); + assert!(completion["streamEpoch"] + .as_str() + .unwrap() + .starts_with("launch-prefix-hostboot-")); + assert_eq!(completion["exit"]["code"], 17); + } + + #[tokio::test] + async fn retained_output_successful_stop_joins_reader_before_reply() { + let root = tempfile::tempdir().unwrap(); + let (mut hosted, _) = retained_pty(root.path(), "printf 'stop-tail'; sleep 300").await; + tokio::time::timeout(std::time::Duration::from_secs(5), async { + while hosted.head_seq() == 0 { + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + }) + .await + .unwrap(); + hosted.stop().await; + let exited = hosted.exited(); + let completion = std::fs::read(root.path().join("terminal-spool-completion.json")); + drop(hosted); + assert!( + exited, + "successful Stop settles the output reader before returning" + ); + assert!( + completion.is_ok(), + "successful Stop freezes a retained completion before reply" + ); + } + #[test] fn obsolete_source_epoch_forces_one_call_full_replay() { assert_eq!( diff --git a/crates/freshell-supervisor/src/lib.rs b/crates/freshell-supervisor/src/lib.rs index b81d03fde..7dd4b0340 100644 --- a/crates/freshell-supervisor/src/lib.rs +++ b/crates/freshell-supervisor/src/lib.rs @@ -11,5 +11,6 @@ pub mod recovery; pub mod registry; pub mod repair; pub mod resume_catalog; +mod retained_output; pub mod service; pub mod view_intents; diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index 4d30cd973..8882bd7d3 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -2264,6 +2264,7 @@ fn map_registry(error: RegistryError) -> RuntimeError { #[cfg(test)] mod tests { include!("shell_recovery_tests.rs"); + include!("retained_output_tests.rs"); use super::*; use freshell_runtime_protocol::{FreshAgentLaunchSpec, FreshProvider, IncarnationId}; diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 705ac8431..10312cbda 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -164,6 +164,15 @@ pub struct OwnedRuntimeHandle { provider_volume_name: String, } +/// Read-only association to the last committed terminal incarnation. This is +/// deliberately not an OwnedRuntimeHandle and cannot authorize input or stop. +pub(crate) struct RetainedOutputSource { + pub incarnation_id: IncarnationId, + pub runtime_dir: PathBuf, + pub terminal: TerminalLaunchSpec, + pub native_session_id: Option, +} + impl OwnedRuntimeHandle { #[allow(clippy::too_many_arguments)] pub(crate) fn from_registry( @@ -765,6 +774,24 @@ impl Registry { .await } + pub(crate) async fn retained_output_source( + &self, + soul_id: SoulId, + ) -> Result { + // The existing context resolver proves installation ownership and + // selects the current committed incarnation (or its committed prior). + let context = self.recovery_context(soul_id).await?; + let handle = context.prior_handle; + Ok(RetainedOutputSource { + incarnation_id: handle.incarnation_id().clone(), + runtime_dir: handle.runtime_dir().to_path_buf(), + terminal: handle.terminal().cloned().ok_or_else(|| { + RegistryError::InvalidState("soul has no retained terminal".into()) + })?, + native_session_id: context.native_session_id, + }) + } + pub async fn begin_stop(&self, soul_id: SoulId) -> Result { self.begin_stop_expected(soul_id, None).await } diff --git a/crates/freshell-supervisor/src/retained_output.rs b/crates/freshell-supervisor/src/retained_output.rs new file mode 100644 index 000000000..dd45d0922 --- /dev/null +++ b/crates/freshell-supervisor/src/retained_output.rs @@ -0,0 +1,155 @@ +use crate::registry::RetainedOutputSource; +use freshell_runtime_protocol::{ + terminal_spool::{ + scan_spool, RetainedOutputPage, SPOOL_COMPLETION_FILE, SPOOL_CURRENT_FILE, + SPOOL_DESCRIPTOR_FILE, SPOOL_PREVIOUS_FILE, + }, + RuntimeOutputBatch, TerminalExitObservation, TerminalSpoolCompletion, TerminalSpoolDescriptor, +}; +use std::{ + fs::{File, OpenOptions}, + io::{BufReader, Read, Write}, + path::Path, +}; + +pub(crate) fn validate_source( + source: &RetainedOutputSource, + runtime_root: &Path, +) -> Result<(), String> { + let directory = std::fs::canonicalize(&source.runtime_dir) + .map_err(|_| "retained output directory unavailable")?; + if directory != runtime_root.join(source.incarnation_id.as_str()) { + return Err("retained output directory does not belong to the owned incarnation".into()); + } + for name in [ + SPOOL_DESCRIPTOR_FILE, + SPOOL_COMPLETION_FILE, + SPOOL_CURRENT_FILE, + SPOOL_PREVIOUS_FILE, + ] { + match std::fs::symlink_metadata(directory.join(name)) { + Ok(metadata) if !metadata.file_type().is_file() => { + return Err("retained output source must be an owned regular file".into()); + } + Ok(_) => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("retained output source metadata unavailable".into()), + } + } + Ok(()) +} + +fn metadata(path: &Path) -> Result { + let file = File::open(path).map_err(|_| "retained output metadata unavailable")?; + let mut bytes = Vec::new(); + file.take(16385) + .read_to_end(&mut bytes) + .map_err(|_| "retained metadata read failed")?; + if bytes.len() > 16384 { + return Err("retained output metadata exceeds bound".into()); + } + serde_json::from_slice(&bytes).map_err(|_| "invalid retained output metadata".into()) +} +fn scan( + source: &RetainedOutputSource, + descriptor: &TerminalSpoolDescriptor, + mut accept: impl FnMut(freshell_runtime_protocol::RuntimeOutputFrame) -> Result<(), String>, +) -> Result { + let mut torn = false; + for (name, required) in [(SPOOL_PREVIOUS_FILE, false), (SPOOL_CURRENT_FILE, true)] { + let file = match File::open(source.runtime_dir.join(name)) { + Ok(file) => file, + Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => return Err("retained output spool unavailable".into()), + }; + if file + .metadata() + .map_err(|_| "retained spool metadata unavailable")? + .len() + > descriptor.segment_limit_bytes + { + return Err("retained spool exceeds segment bound".into()); + } + torn |= scan_spool(BufReader::new(file), descriptor, &mut accept)?; + } + Ok(torn) +} + +pub(crate) fn read( + source: RetainedOutputSource, + after: u64, + budget: u64, + expected_epoch: Option, + verified_empty: bool, +) -> Result { + let descriptor: TerminalSpoolDescriptor = + metadata(&source.runtime_dir.join(SPOOL_DESCRIPTOR_FILE))?; + descriptor.validate()?; + if descriptor.incarnation_id != source.incarnation_id + || descriptor.terminal_id != source.terminal.terminal_id + || !descriptor + .stream_epoch + .starts_with(&format!("{}-", source.terminal.stream_id)) + { + return Err("retained output source association mismatch".into()); + } + let mut first = None; + let mut head = 0; + scan(&source, &descriptor, |frame| { + first.get_or_insert(frame.seq_start); + head = frame.seq_end; + Ok(()) + })?; + let retained = first.unwrap_or(head.saturating_add(1)); + let changed = expected_epoch + .as_deref() + .is_some_and(|epoch| epoch != descriptor.stream_epoch); + let reset = changed || after > head || after.saturating_add(1) < retained; + let effective = if reset { + retained.saturating_sub(1) + } else { + after + }; + let completion_path = source.runtime_dir.join(SPOOL_COMPLETION_FILE); + let completion = match metadata::(&completion_path) { + Ok(completion) => Some(completion), + Err(_) if !completion_path.exists() && !verified_empty => None, + Err(_) if !completion_path.exists() && verified_empty => { + // An exact backend empty proof settles a writer whose host could + // not commit EOF. Freeze its actual retained head; code is unknown. + let completion = TerminalSpoolCompletion { + incarnation_id: descriptor.incarnation_id.clone(), + terminal_id: descriptor.terminal_id.clone(), + stream_epoch: descriptor.stream_epoch.clone(), + retained_from_seq: retained, + head_seq: head, + exit: TerminalExitObservation::Unavailable, + }; + let temporary = completion_path.with_extension("tmp"); + let mut file = OpenOptions::new() + .create(true) + .truncate(true) + .write(true) + .open(&temporary) + .map_err(|_| "completion unavailable")?; + let bytes = + serde_json::to_vec(&completion).map_err(|_| "completion encoding failed")?; + file.write_all(&bytes) + .and_then(|_| file.sync_all()) + .map_err(|_| "completion write failed")?; + std::fs::rename(temporary, &completion_path).map_err(|_| "completion commit failed")?; + File::open(&source.runtime_dir) + .and_then(|dir| dir.sync_all()) + .map_err(|_| "completion parent sync failed")?; + Some(completion) + } + Err(error) => return Err(error), + }; + let mut page = RetainedOutputPage::new(descriptor.clone(), effective, budget)?; + let torn = scan(&source, &descriptor, |frame| page.accept(frame))?; + let mut batch = page.finish(torn, completion)?; + batch.reset_required |= reset; + batch.truncated |= reset; + batch.native_session_id = source.native_session_id; + Ok(batch) +} diff --git a/crates/freshell-supervisor/src/retained_output_tests.rs b/crates/freshell-supervisor/src/retained_output_tests.rs new file mode 100644 index 000000000..c03bd6a1a --- /dev/null +++ b/crates/freshell-supervisor/src/retained_output_tests.rs @@ -0,0 +1,244 @@ +// Retained reads use actual supervisor/SQLite ownership and real spool files. +async fn retained_output_fixture( + root: &std::path::Path, + state: BackendRuntimeState, +) -> (Supervisor, RecoveryContext) { + let (registry, context) = checkpoints::tests::context(root, "shell", true).await; + // Match the producer's runtime_root/incarnation layout rather than the + // checkpoint helper's deliberately minimal shared-root fixture. + let runtime_dir = root + .join("runtime") + .join(context.prior_handle.incarnation_id().as_str()); + std::fs::create_dir(&runtime_dir).unwrap(); + let terminal: freshell_runtime_protocol::TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"retained-terminal","streamId":"launch-prefix","mode":"shell","program":"/bin/sh", + "cwd":"/workspace","runAsUid":65534,"runAsGid":0,"cols":120,"rows":30, + "projectKey":"fixture","workspacePath":"/workspace" + })).unwrap(); + let conn = rusqlite::Connection::open(registry.database_path()).unwrap(); + conn.execute( + "UPDATE incarnations SET terminal_spec=?1,terminal_id=?2,runtime_dir=?4 WHERE incarnation_id=?3", + rusqlite::params![ + serde_json::to_string(&terminal).unwrap(), + terminal.terminal_id, + context.prior_handle.incarnation_id().as_str(), + runtime_dir.to_str().unwrap() + ], + ) + .unwrap(); + conn.execute("UPDATE souls SET recovery_state='blocked',recovery_reason='ambiguous_identity' WHERE soul_id=?1",[context.soul_id.as_str()]).unwrap(); + drop(conn); + let context = registry.recovery_context(context.soul_id).await.unwrap(); + let incarnation = context.prior_handle.incarnation_id(); + let directory = context.prior_handle.runtime_dir(); + std::fs::write( + directory.join("terminal-spool-descriptor.json"), + serde_json::to_vec(&serde_json::json!({ + "schemaVersion":1,"incarnationId":incarnation,"terminalId":"retained-terminal", + "streamEpoch":"launch-prefix-hostboot-exact","segmentLimitBytes":33554432 + })) + .unwrap(), + ) + .unwrap(); + let frames = (1..=3).map(|seq|serde_json::to_string(&serde_json::json!({"frame":{ + "terminalId":"retained-terminal","streamEpoch":"launch-prefix-hostboot-exact", + "seqStart":seq,"seqEnd":seq,"data":if seq==3 {"final diagnostic".into()} else {"x".repeat(65536)} + }})).unwrap()+"\n").collect::(); + std::fs::write(directory.join("terminal-spool-current.jsonl"), frames).unwrap(); + recovery_supervisor(root, registry, context, state).await +} + +async fn retained_output_read( + supervisor: &Supervisor, + soul: SoulId, + after: u64, +) -> Result { + let mut envelope = freshell_runtime_protocol::Envelope::new( + freshell_runtime_protocol::RequestId::new(), + freshell_runtime_protocol::ControlRole::Web, + freshell_runtime_protocol::AdminCommand::TerminalReadOutput( + freshell_runtime_protocol::TerminalReadOutputRequest { + soul_id: soul, + after_seq: after, + max_bytes: 65536, + expected_stream_epoch: Some("launch-prefix-hostboot-exact".into()), + expected_control_epoch: None, + }, + ), + ); + envelope.auth = Some("synthetic-control-secret".into()); + match supervisor.dispatch(envelope).await.result? { + freshell_runtime_protocol::AdminResult::TerminalOutput(output) => Ok(output), + other => panic!("unexpected retained read {other:?}"), + } +} + +#[tokio::test] +async fn retained_output_blocked_handle_replays_without_input_authority() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Running).await; + let output = retained_output_read(&supervisor, context.soul_id.clone(), 0).await; + assert!( + supervisor + .registry + .active_handle_for_soul(context.soul_id) + .await + .is_err(), + "retained resolver must not relax input authority" + ); + let output = + output.expect("blocked exact owned spool remains readable without active input handle"); + assert_eq!(output.frames[0].data, "x".repeat(65536)); + assert_eq!(output.head_seq, 3); + assert!( + !output.exited, + "unsettled writer is independent of recovery outcome" + ); +} + +#[tokio::test] +async fn retained_output_verified_end_survives_cleanup_and_cold_supervisor() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + let result = supervisor + .recover(RecoverRequest { + soul_id: context.soul_id.clone(), + trigger: RecoveryTrigger::ProviderExit, + expected_intent_revision: None, + expected_control_epoch: None, + }) + .await + .unwrap(); + assert_eq!(result.outcome, RecoveryOutcome::Lost); + let config = (*supervisor.config).clone(); + let database_root = supervisor + .registry + .database_path() + .parent() + .unwrap() + .to_path_buf(); + drop(supervisor); + let registry = crate::registry::Registry::open(&database_root, None).unwrap(); + let supervisor = Supervisor::new( + registry, + std::sync::Arc::new(RecoveryBackend(std::sync::Mutex::new( + BackendRuntimeState::Exited, + ))), + config, + ) + .unwrap(); + let mut windows = vec![]; + for _reader in 0..2 { + let mut cursor = 0; + let mut data = String::new(); + loop { + let batch = retained_output_read(&supervisor, context.soul_id.clone(), cursor) + .await + .expect("cleaned up exact stream survives supervisor restart"); + assert!( + batch.exited, + "verified empty backend settles writer before frozen scan" + ); + assert_eq!(batch.stream_epoch, "launch-prefix-hostboot-exact"); + for frame in batch.frames { + cursor = frame.seq_end; + data.push_str(&frame.data); + } + if cursor == batch.head_seq { + break; + } + } + windows.push(data); + } + assert_eq!(windows[0], windows[1]); + assert_eq!(windows[0], "x".repeat(131072) + "final diagnostic"); +} + +#[tokio::test] +async fn retained_output_missing_spool_reports_real_failure() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + std::fs::remove_file( + context + .prior_handle + .runtime_dir() + .join("terminal-spool-current.jsonl"), + ) + .unwrap(); + let error = retained_output_read(&supervisor, context.soul_id, 0) + .await + .unwrap_err(); + assert_eq!( + error.code, + RuntimeErrorCode::HostUnreachable, + "owned but unavailable output is not an unknown soul" + ); +} + +#[tokio::test] +async fn retained_output_loss_with_unconfirmed_cleanup_does_not_invent_completion() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Running).await; + let conn = rusqlite::Connection::open(supervisor.registry.database_path()).unwrap(); + conn.execute( + "UPDATE souls SET recovery_state='lost' WHERE soul_id=?1", + [context.soul_id.as_str()], + ) + .unwrap(); + conn.execute( + "UPDATE incarnations SET cleanup_state='termination_unconfirmed' WHERE incarnation_id=?1", + [context.prior_handle.incarnation_id().as_str()], + ) + .unwrap(); + drop(conn); + let read = retained_output_read(&supervisor, context.soul_id.clone(), 0) + .await + .unwrap(); + assert!(!read.exited); + assert!(read.completion.is_none()); + assert!(read.exit_code.is_none()); + assert_eq!(read.head_seq, 3); + assert_eq!(read.frames[0].data, "x".repeat(65536)); + assert!(!context + .prior_handle + .runtime_dir() + .join("terminal-spool-completion.json") + .exists()); +} + +#[tokio::test] +async fn retained_output_rejects_registry_path_outside_owned_runtime_root() { + let root = tempfile::tempdir().unwrap(); + let outside = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + for name in [ + "terminal-spool-descriptor.json", + "terminal-spool-current.jsonl", + ] { + std::fs::copy( + context.prior_handle.runtime_dir().join(name), + outside.path().join(name), + ) + .unwrap(); + } + let conn = rusqlite::Connection::open(supervisor.registry.database_path()).unwrap(); + conn.execute( + "UPDATE incarnations SET runtime_dir=?1 WHERE incarnation_id=?2", + rusqlite::params![ + outside.path().to_str().unwrap(), + context.prior_handle.incarnation_id().as_str() + ], + ) + .unwrap(); + drop(conn); + let result = retained_output_read(&supervisor, context.soul_id, 0).await; + assert!( + result.is_err(), + "stored directory association cannot read another runtime root" + ); +} diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index a5a14902f..0ca498701 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -2026,28 +2026,63 @@ impl Supervisor { max_bytes: u64, expected_stream_epoch: Option, ) -> Result { + let source = self + .registry + .retained_output_source(soul_id.clone()) + .await + .map_err(map_registry)?; + crate::retained_output::validate_source(&source, &self.config.runtime_root) + .map_err(|error| RuntimeError::new(RuntimeErrorCode::OwnershipMismatch, error))?; + // This independent capability is used only for exact read-only backend + // emptiness verification, never input, kill, or replacement dispatch. let handle = self .registry - .active_handle_for_soul(soul_id.clone()) + .owned_handle(source.incarnation_id.clone()) .await .map_err(map_registry)?; - let host = self - .authenticate_host(handle.incarnation_id(), handle.runtime_dir()) - .await?; - match self - .send_authenticated_host_command( - handle.incarnation_id().clone(), - handle.runtime_dir(), + let live = async { + let host = self + .authenticate_host(&source.incarnation_id, &source.runtime_dir) + .await?; + self.send_authenticated_host_command( + source.incarnation_id.clone(), + &source.runtime_dir, &host, HostCommand::TerminalReadOutput { - incarnation_id: handle.incarnation_id().clone(), + incarnation_id: source.incarnation_id.clone(), after_seq, max_bytes, - expected_stream_epoch, + expected_stream_epoch: expected_stream_epoch.clone(), }, ) - .await? - { + .await + } + .await; + let reply = match live { + Ok(reply) => reply, + Err(_) => { + let verified_empty = self.backend.verify_empty(&handle).await.unwrap_or(false); + let output = tokio::task::spawn_blocking(move || { + crate::retained_output::read( + source, + after_seq, + max_bytes, + expected_stream_epoch, + verified_empty, + ) + }) + .await + .map_err(|_| { + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "retained output reader failed", + ) + })? + .map_err(|error| RuntimeError::new(RuntimeErrorCode::HostUnreachable, error))?; + return Ok(output); + } + }; + match reply { HostResult::TerminalOutput(output) => { if let Some(native_session_id) = output.native_session_id.as_ref() { match self diff --git a/crates/freshell-terminal/src/managed_output_tests.rs b/crates/freshell-terminal/src/managed_output_tests.rs index f011fc227..a65c195e7 100644 --- a/crates/freshell-terminal/src/managed_output_tests.rs +++ b/crates/freshell-terminal/src/managed_output_tests.rs @@ -94,6 +94,8 @@ fn fixture() -> (TerminalRegistry, Arc) { calls: AtomicUsize::new(0), observed: Mutex::new(Vec::new()), response: Mutex::new(ManagedOutputRead { + lifecycle: None, + completion: None, stream_epoch: Some("epoch-new".into()), incarnation_id: Some("incarnation-new".into()), reset_required: true, @@ -260,3 +262,104 @@ fn managed_opencode_same_runtime_accepts_a_signaled_session_switch() { ); assert_eq!(registry.managed_output_cursor("terminal-stable"), Some(206)); } + +#[test] +fn managed_output_shell_terminal_ended_final_tail() { + let (registry, controller) = fixture(); + { + let mut inner = registry.inner.lock().unwrap(); + let handle = inner.terminals.get_mut("terminal-stable").unwrap(); + handle.managed.as_mut().unwrap().mode = "shell".into(); + handle.shared.lock().unwrap().mode = "shell".into(); + } + { + let mut response = controller.response.lock().unwrap(); + response.stream_epoch = Some("epoch-old".into()); + response.incarnation_id = Some("incarnation-old".into()); + response.reset_required = false; + response.retained_from_seq = 206; + response.head_seq = 208; + response.exit_code = Some(17); + response.chunks = vec![ManagedOutputChunk { + seq_start: 207, + seq_end: 207, + data: "x".repeat(65536), + }]; + } + immediate(registry.refresh_managed_output("terminal-stable", 65536)).unwrap(); + assert_eq!(registry.managed_output_cursor("terminal-stable"), Some(207)); + assert_eq!(registry.inventory()[0].status, TerminalRunStatus::Running); + controller.response.lock().unwrap().chunks = vec![ManagedOutputChunk { + seq_start: 208, + seq_end: 208, + data: "final tail".into(), + }]; + immediate(registry.refresh_managed_output("terminal-stable", 65536)).unwrap(); + registry.finish_managed_exit("terminal-stable", 17); + assert_eq!(registry.inventory()[0].status, TerminalRunStatus::Exited); + assert!(registry.exists("terminal-stable")); + assert_eq!(registry.managed_output_cursor("terminal-stable"), Some(208)); + assert!( + registry + .live_managed_owner_for_session("shell", "native-original") + .is_some(), + "retained managed membership continues to dominate ordinary recreation after exit" + ); + assert!(registry.directory()[0].snapshot.ends_with("final tail")); +} + +#[test] +fn managed_output_paced_exit_preserves_explicit_code_availability() { + for code in [None, Some(-1)] { + let (registry, _) = fixture(); + immediate(registry.refresh_managed_output("terminal-stable", 65536)).unwrap(); + registry.set_paced_page_max_bytes(0); + let captured = Arc::new(Mutex::new(Vec::new())); + let sink = captured.clone(); + let start = registry + .attach( + "terminal-stable", + 77, + Arc::new(move |frame| sink.lock().unwrap().push(frame)), + Some("explicit-availability".into()), + 0, + false, + true, + None, + None, + None, + PacedAttachOptions::default(), + ) + .paced + .expect("managed retained tail uses paced replay"); + assert!(registry.finish_managed_completion("terminal-stable", code)); + let mut cursor = start.session.page_end; + loop { + match registry.complete_paced_tail( + "terminal-stable", + 77, + "explicit-availability", + cursor, + 2, + 0, + ) { + PacedTailCompletion::Handoff { end_seq, .. } => cursor = end_seq, + PacedTailCompletion::Completed { .. } => break, + other => panic!("unexpected completion: {other:?}"), + } + } + let frames = captured.lock().unwrap(); + let exits: Vec<_> = frames + .iter() + .filter_map(|frame| match frame { + ServerMessage::TerminalExit(exit) => Some(exit), + _ => None, + }) + .collect(); + assert_eq!(exits.len(), 1); + assert_eq!(exits[0].managed_retained, Some(true)); + assert_eq!(exits[0].exit_code_available, Some(code.is_some())); + assert_eq!(exits[0].exit_code, code.unwrap_or(-1)); + assert_eq!(registry.managed_attached_to(77), vec!["terminal-stable"]); + } +} diff --git a/crates/freshell-terminal/src/pty.rs b/crates/freshell-terminal/src/pty.rs index c799a758e..60ea94ca4 100644 --- a/crates/freshell-terminal/src/pty.rs +++ b/crates/freshell-terminal/src/pty.rs @@ -448,12 +448,10 @@ pub(crate) fn take_group_kill_log() -> Vec { GROUP_KILL_LOG.with(|log| std::mem::take(&mut *log.borrow_mut())) } -impl Drop for PtyTerminal { - fn drop(&mut self) { - // Kill the child, then join the waiter (returns once the child is reaped; - // on Windows it also closes the master) and the reader (it exits once the - // stream EOFs \u2014 unix: slave EIO; Windows: master closed by the waiter). - // Both joins are bounded: kill() guarantees the child is exiting. +impl PtyTerminal { + /// Stop the owned child and settle both output and exit observation before + /// the caller publishes a successful stop reply. + pub fn stop_and_join_output(&mut self) { self.kill(); if let Some(handle) = self.waiter_thread.take() { let _ = handle.join(); @@ -464,6 +462,12 @@ impl Drop for PtyTerminal { } } +impl Drop for PtyTerminal { + fn drop(&mut self) { + self.stop_and_join_output(); + } +} + /// The `onData` loop: read raw bytes -> decode UTF-8 (holding partial scalars) -> /// frame into `terminal.output` messages -> forward each message to the live /// streaming `sink` when one is wired, OTHERWISE append to the in-memory capture diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 20c40946e..94aa2492e 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -580,6 +580,10 @@ struct TerminalShared { /// Highest `seqEnd` produced (drives `attach.ready.headSeq`). head_seq: i64, status: TerminalRunStatus, + managed_identity: Option<(String, String)>, + managed_lifecycle: Option, + managed_exit_code_available: Option, + managed_output_failure: Option, /// The exit code from the last `terminal.exit` fan-out (kill or natural exit). /// `None` while `status == Running`. Kept so a client that attaches AFTER the /// terminal already exited (the create-then-instant-exit race) can still be @@ -741,6 +745,12 @@ impl TerminalShared { /// (else `clearDeadTerminals` recreates it, losing scrollback). fn inventory(&self) -> InventoryTerminal { InventoryTerminal { + soul_id: self.managed_identity.as_ref().map(|i| i.0.clone()), + incarnation_id: self.managed_identity.as_ref().map(|i| i.1.clone()), + recovery_summary: self + .managed_lifecycle + .as_ref() + .and_then(|l| l.recovery_summary.clone()), created_at: self.created_at, last_activity_at: self.last_activity_at, mode: self.mode.clone(), @@ -750,7 +760,10 @@ impl TerminalShared { codex_durability: None, cwd: self.cwd.clone(), description: self.description.clone(), - runtime_status: None, + runtime_status: self + .managed_lifecycle + .as_ref() + .map(|l| l.state.runtime_status()), session_ref: None, session_name: self.session_name.clone(), name_ref: self.name_ref.clone(), @@ -785,6 +798,10 @@ pub struct IdentityProbeRow { /// merge, `sessionRef` derivation, and `lastLine` extraction happen in the router). #[derive(Debug, Clone)] pub struct DirectoryEntry { + pub runtime_status: Option, + pub recovery_summary: Option, + pub soul_id: Option, + pub incarnation_id: Option, pub terminal_id: String, pub title: String, pub description: Option, @@ -856,8 +873,40 @@ pub struct ManagedOutputChunk { pub data: String, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ManagedLifecycleState { + Running, + Recovering, + Blocked, + Ended, +} +impl ManagedLifecycleState { + fn runtime_status(self) -> freshell_protocol::RuntimeStatus { + use freshell_protocol::RuntimeStatus; + match self { + Self::Running => RuntimeStatus::Running, + Self::Recovering => RuntimeStatus::Recovering, + Self::Blocked => RuntimeStatus::Blocked, + Self::Ended => RuntimeStatus::Exited, + } + } +} +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ManagedLifecycle { + pub state: ManagedLifecycleState, + pub recovery_summary: Option, +} +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ManagedOutputCompletion { + pub stream_epoch: String, + pub incarnation_id: String, + pub head_seq: i64, + pub exit_code: Option, +} #[derive(Debug, Clone, PartialEq, Eq)] pub struct ManagedOutputRead { + pub lifecycle: Option, + pub completion: Option, /// The session host's durable output epoch, not the web's launch-time ID. /// None denotes a coalesced/in-flight read with no source observation. pub stream_epoch: Option, @@ -876,6 +925,14 @@ pub struct ManagedOutputRead { /// remain the browser-facing replay/fan-out owner while the web server has no PTY, /// PID, Docker socket, or direct kill authority for managed rows. pub trait ManagedTerminalController: Send + Sync { + /// A cached controller verdict remains observable even if replay is unreadable. + /// This never supplies output or authority to launch, replace, or stop. + fn lifecycle_observation<'a>( + &'a self, + _terminal: &'a ManagedTerminalDescriptor, + ) -> ManagedTerminalFuture<'a, Option> { + Box::pin(std::future::ready(None)) + } fn lookup_terminal<'a>( &'a self, terminal_id: &'a str, @@ -2361,6 +2418,10 @@ impl TerminalRegistry { let now = now_ms(); let shared = Arc::new(Mutex::new(TerminalShared { + managed_identity: None, + managed_lifecycle: None, + managed_exit_code_available: None, + managed_output_failure: None, terminal_id: terminal_id.clone(), stream_id: stream_id.clone(), replay: VecDeque::new(), @@ -2833,11 +2894,15 @@ impl TerminalRegistry { // like a client that was already attached when the process died. if s.status == TerminalRunStatus::Exited { let exit = ServerMessage::TerminalExit(TerminalExit { + managed_retained: s.managed_identity.as_ref().map(|_| true), + exit_code_available: s.managed_exit_code_available, exit_code: s.exit_code.unwrap_or(0), terminal_id: terminal_id.to_string(), }); sink(exit); - s.subscribers.remove(&conn_id); + if s.managed_identity.is_none() { + s.subscribers.remove(&conn_id); + } } AttachOutcome { @@ -3926,6 +3991,8 @@ impl TerminalRegistry { s.status = TerminalRunStatus::Exited; s.exit_code = Some(0); let exit = ServerMessage::TerminalExit(TerminalExit { + managed_retained: None, + exit_code_available: None, exit_code: 0, terminal_id: terminal_id.to_string(), }); @@ -4073,6 +4140,8 @@ impl TerminalRegistry { let respawn_key = s.create_request_id.clone(); let lifetime_ms = now.saturating_sub(s.created_at); let exit = ServerMessage::TerminalExit(TerminalExit { + managed_retained: None, + exit_code_available: None, exit_code, terminal_id: terminal_id.to_string(), }); @@ -4421,6 +4490,16 @@ impl TerminalRegistry { .map(|shared| { let s = shared.lock().expect("terminal lock"); DirectoryEntry { + runtime_status: s + .managed_lifecycle + .as_ref() + .map(|l| l.state.runtime_status()), + recovery_summary: s + .managed_lifecycle + .as_ref() + .and_then(|l| l.recovery_summary.clone()), + soul_id: s.managed_identity.as_ref().map(|i| i.0.clone()), + incarnation_id: s.managed_identity.as_ref().map(|i| i.1.clone()), terminal_id: s.terminal_id.clone(), title: s.title.clone(), description: s.description.clone(), @@ -4545,12 +4624,7 @@ impl TerminalRegistry { if descriptor.resume_session_id.as_deref() != Some(session_id) { return None; } - let running = handle - .shared - .lock() - .ok() - .is_some_and(|shared| shared.status == TerminalRunStatus::Running); - running.then(|| descriptor.clone()) + Some(descriptor.clone()) }) } @@ -4560,6 +4634,13 @@ impl TerminalRegistry { pub fn register_managed(&self, descriptor: ManagedTerminalDescriptor) { let created_at = now_ms(); let shared = Arc::new(Mutex::new(TerminalShared { + managed_identity: Some(( + descriptor.soul_id.clone(), + descriptor.incarnation_id.clone(), + )), + managed_lifecycle: None, + managed_exit_code_available: None, + managed_output_failure: None, terminal_id: descriptor.terminal_id.clone(), stream_id: descriptor.stream_id.clone(), replay: VecDeque::new(), @@ -4751,6 +4832,8 @@ impl TerminalRegistry { }; let Some(claim) = ManagedReadClaim::acquire(flag) else { return Ok(ManagedOutputRead { + lifecycle: None, + completion: None, stream_epoch: None, incarnation_id: None, reset_required: false, @@ -4774,9 +4857,19 @@ impl TerminalRegistry { .expect("managed controller lock") .clone() .ok_or_else(|| "managed runtime controller unavailable".to_string())?; - let mut read = controller - .read_output(terminal, after_seq, max_bytes) - .await?; + let mut read = match controller + .read_output(terminal.clone(), after_seq, max_bytes) + .await + { + Ok(read) => read, + Err(error) => { + self.publish_managed_lifecycle( + terminal_id, + controller.lifecycle_observation(&terminal).await, + ); + return Err(error); + } + }; let mut announcements = Vec::new(); let mut epoch_changed = false; { @@ -4829,6 +4922,10 @@ impl TerminalRegistry { .incarnation_id .clone_from(read.incarnation_id.as_ref().expect("validated incarnation")); state.stream_id.clone_from(epoch); + state.managed_identity = Some(( + descriptor.soul_id.clone(), + descriptor.incarnation_id.clone(), + )); state.geometry_epoch = state.geometry_epoch.saturating_add(1); for subscriber in state.subscribers.values() { announcements.push(( @@ -4875,7 +4972,76 @@ impl TerminalRegistry { /// Managed equivalent of `finish_pty_exit`: the OS process is already /// gone and the supervisor-owned enclosure has been stopped; retain the /// facade/replay tail but mark it exited and tell attached viewers once. + /// Lifecycle publication follows full-page ingest and native association. + /// None is no observation and must not reset an existing final verdict. + pub fn publish_managed_lifecycle( + &self, + terminal_id: &str, + lifecycle: Option, + ) { + let Some(lifecycle) = lifecycle else { + return; + }; + let Some(shared) = self.shared_for(terminal_id) else { + return; + }; + let mut state = shared.lock().expect("terminal lock"); + if state.managed_identity.is_none() || state.managed_lifecycle.as_ref() == Some(&lifecycle) + { + return; + } + if lifecycle.state == ManagedLifecycleState::Running { + state.status = TerminalRunStatus::Running; + state.exit_code = None; + state.managed_exit_code_available = None; + } + let summary = lifecycle.recovery_summary.clone(); + let message = ServerMessage::TerminalStatus(freshell_protocol::TerminalStatus { + terminal_id: terminal_id.into(), + status: lifecycle.state.runtime_status(), + reason: summary.as_ref().and_then(|s| s.reason.clone()), + recovery_summary: summary, + attempt: None, + max_attempts: None, + exit_code: None, + resume_cycles: None, + }); + state.managed_lifecycle = Some(lifecycle); + for subscriber in state.subscribers.values() { + (subscriber.sink)(message.clone()); + } + } + + /// Error cause is classified before storage/logging; shared by every socket. + pub fn observe_managed_output_failure(&self, terminal_id: &str, cause: Option<&str>) { + let Some(shared) = self.shared_for(terminal_id) else { + return; + }; + let mut state = shared.lock().expect("terminal lock"); + if state.managed_output_failure.as_deref() == cause { + return; + } + let previous = state.managed_output_failure.take(); + state.managed_output_failure = cause.map(str::to_owned); + let Some((soul, incarnation)) = state.managed_identity.as_ref() else { + return; + }; + match cause { + Some(cause) => { + tracing::warn!(event="managed_runtime.output_refresh_failed",terminal_id,soul_id=%soul,incarnation_id=%incarnation,cause,"Managed retained output read failed") + } + None if previous.is_some() => { + tracing::info!(event="managed_runtime.output_refresh_recovered",terminal_id,soul_id=%soul,incarnation_id=%incarnation,cause="read_succeeded","Managed retained output read resumed") + } + None => {} + } + } + pub fn finish_managed_exit(&self, terminal_id: &str, exit_code: i64) -> bool { + self.finish_managed_completion(terminal_id, Some(exit_code)) + } + pub fn finish_managed_completion(&self, terminal_id: &str, code: Option) -> bool { + let exit_code = code.unwrap_or(-1); let shared = { let inner = self.inner.lock().expect("registry lock"); let Some(handle) = inner.terminals.get(terminal_id) else { @@ -4892,12 +5058,15 @@ impl TerminalRegistry { } state.status = TerminalRunStatus::Exited; state.exit_code = Some(exit_code); + state.managed_exit_code_available = Some(code.is_some()); let now = now_ms(); state.last_activity_at = now; state.last_output_activity_at = now; state.last_meaningful_activity_at = now; state.last_meaningful_output_at = now; let exit = ServerMessage::TerminalExit(TerminalExit { + managed_retained: Some(true), + exit_code_available: Some(code.is_some()), exit_code, terminal_id: terminal_id.to_string(), }); @@ -4906,7 +5075,6 @@ impl TerminalRegistry { // its exit until the replay drain completes, exactly as the local // PTY natural-exit path does. let mut staged_notify: Vec = Vec::new(); - let mut retire_now: Vec = Vec::new(); for (conn_id, subscriber) in state.subscribers.iter_mut() { if subscriber.paced_deferred { subscriber.paced_exit_pending = Some(exit_code); @@ -4915,17 +5083,12 @@ impl TerminalRegistry { } } else { (subscriber.sink)(exit.clone()); - retire_now.push(*conn_id); } } - for conn_id in retire_now { - state.subscribers.remove(&conn_id); - } drop(state); for notify in staged_notify { notify(terminal_id, exit_code); } - self.release_session_ref_ownership(terminal_id, "registry/managed-exit"); self.notify_activity(ActivityEvent::Exit { terminal_id: terminal_id.to_string(), at: now_ms(), @@ -4968,6 +5131,10 @@ impl TerminalRegistry { }; let create_request_id = opts.create_request_id.clone(); let shared = Arc::new(Mutex::new(TerminalShared { + managed_identity: None, + managed_lifecycle: None, + managed_exit_code_available: None, + managed_output_failure: None, terminal_id: opts.terminal_id.clone(), stream_id: opts.stream_id, replay: VecDeque::new(), @@ -6352,10 +6519,16 @@ fn deliver_staged_paced_exit(s: &mut TerminalShared, conn_id: u64) -> Option, ) { let msg = freshell_protocol::ServerMessage::TerminalStatus(freshell_protocol::TerminalStatus { + recovery_summary: None, status: freshell_protocol::RuntimeStatus::Exited, terminal_id: terminal_id.to_string(), attempt: None, diff --git a/crates/freshell-ws/src/connection_writer_tests.rs b/crates/freshell-ws/src/connection_writer_tests.rs index 3f71a4f9e..ce5d1aeb2 100644 --- a/crates/freshell-ws/src/connection_writer_tests.rs +++ b/crates/freshell-ws/src/connection_writer_tests.rs @@ -631,6 +631,8 @@ async fn preludes_always_precede_replay_and_exit_follows_output() { sender.push_server(output(2)); sender.push_server(ServerMessage::TerminalExit( freshell_protocol::TerminalExit { + managed_retained: None, + exit_code_available: None, terminal_id: "term".into(), exit_code: 0, }, @@ -887,6 +889,8 @@ async fn superseding_attach_discards_old_queued_output_and_old_exit() { sender.push_server(output(1)); sender.push_server(ServerMessage::TerminalExit( freshell_protocol::TerminalExit { + managed_retained: None, + exit_code_available: None, terminal_id: "term".into(), exit_code: 0, }, @@ -904,6 +908,8 @@ async fn superseding_attach_discards_old_queued_output_and_old_exit() { sender.push_server(replay); sender.push_server(ServerMessage::TerminalExit( freshell_protocol::TerminalExit { + managed_retained: None, + exit_code_available: None, terminal_id: "term".into(), exit_code: 0, }, diff --git a/crates/freshell-ws/src/managed_retained_socket_tests.rs b/crates/freshell-ws/src/managed_retained_socket_tests.rs new file mode 100644 index 000000000..c377578f1 --- /dev/null +++ b/crates/freshell-ws/src/managed_retained_socket_tests.rs @@ -0,0 +1,183 @@ +async fn managed_socket_windows( + final_head: i64, + rounds: usize, +) -> Result, String> { + use tokio_tungstenite::tungstenite::Message as ClientMessage; + let state = super::pane_reconcile_gate_tests::state(); + let terminal = "T-managed-short-lived"; + state.registry.register_managed(ManagedTerminalDescriptor { + soul_id: "soul-short-lived".into(), + incarnation_id: "incarnation-short-lived".into(), + terminal_id: terminal.into(), + stream_id: "S-managed-short-lived".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: None, + create_request_id: None, + }); + state + .registry + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { final_head }))); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("ws://{}/ws", listener.local_addr().unwrap()); + let token = state.auth_token.to_string(); + let (stop, stopped) = tokio::sync::oneshot::channel(); + let server = tokio::spawn(async move { + axum::serve(listener, crate::router(state)) + .with_graceful_shutdown(async { + stopped.await.ok(); + }) + .await + }); + let mut clients = vec![]; + for _ in 0..2 { + clients.push(tokio_tungstenite::connect_async(&url).await.unwrap().0); + } + let observed = async { + let mut windows = vec![]; + for round in 0..rounds { + if round > 0 { + for client in &mut clients { + client.close(None).await.ok(); + } + clients.clear(); + for _ in 0..2 { + clients.push( + tokio_tungstenite::connect_async(&url) + .await + .map_err(|e| e.to_string())? + .0, + ); + } + } + for (index, client) in clients.iter_mut().enumerate() { + client + .send(ClientMessage::Text( + serde_json::json!({"type":"hello","token":token, + "protocolVersion":freshell_protocol::WS_PROTOCOL_VERSION}) + .to_string(), + )) + .await + .map_err(|e| e.to_string())?; + for _ in 0..4 { + tokio::time::timeout(std::time::Duration::from_secs(5), client.next()) + .await + .map_err(|_| "hello timeout")? + .ok_or("hello closed")? + .map_err(|e| e.to_string())?; + } + let attach = serde_json::json!({"type":"terminal.attach","terminalId":terminal, + "intent":"viewport_hydrate", "cols":80,"rows":24,"sinceSeq":0, + "attachRequestId":format!("reader-{index}")}); + serde_json::from_value::(attach.clone()) + .map_err(|error| format!("invalid attach fixture: {error}"))?; + client + .send(ClientMessage::Text(attach.to_string())) + .await + .map_err(|e| e.to_string())?; + } + for (reader, client) in clients.iter_mut().enumerate() { + let mut data = String::new(); + let mut exits = 0; + let mut received = vec![]; + loop { + let message = + tokio::time::timeout(std::time::Duration::from_secs(5), client.next()) + .await + .map_err(|_| { + format!("output timeout for reader {reader}; received {received:?}") + })? + .ok_or("output closed")? + .map_err(|e| e.to_string())?; + let ClientMessage::Text(text) = message else { + continue; + }; + let frame: serde_json::Value = + serde_json::from_str(&text).map_err(|e| e.to_string())?; + received.push( + serde_json::json!({"type":frame["type"],"terminalId":frame["terminalId"], + "code":frame["code"],"seqStart":frame["seqStart"],"seqEnd":frame["seqEnd"], + "bytes":frame["data"].as_str().map(str::len)}), + ); + if frame["type"] == "error" { + return Err(format!("unexpected server error: {received:?}")); + } + if frame["terminalId"] != terminal { + continue; + } + match frame["type"].as_str() { + Some("terminal.output") | Some("terminal.snapshot") => { + data += frame["data"].as_str().unwrap_or_default() + } + Some("terminal.exit") => { + exits += 1; + if frame["managedRetained"] != true { + return Err("managed exit retired client identity".into()); + } + break; + } + Some("error") => return Err(format!("unexpected {}", frame["code"])), + _ => {} + } + } + // Ping orders behind any duplicate exit already queued. + client + .send(ClientMessage::Text("{\"type\":\"ping\"}".into())) + .await + .map_err(|e| e.to_string())?; + loop { + let message = + tokio::time::timeout(std::time::Duration::from_secs(5), client.next()) + .await + .map_err(|_| "pong timeout")? + .ok_or("pong closed")? + .map_err(|e| e.to_string())?; + let ClientMessage::Text(text) = message else { + continue; + }; + let frame: serde_json::Value = + serde_json::from_str(&text).map_err(|e| e.to_string())?; + if frame["type"] == "terminal.exit" { + exits += 1; + } + if frame["type"] == "pong" { + break; + } + } + windows.push((data, exits)); + } + } + Ok::<_, String>(windows) + } + .await; + for client in &mut clients { + client.close(None).await.ok(); + } + drop(clients); + stop.send(()).unwrap(); + tokio::time::timeout(std::time::Duration::from_secs(10), server) + .await + .unwrap() + .unwrap() + .unwrap(); + observed +} + +#[tokio::test] +async fn managed_retained_final_tail_two_real_sockets_reconnect_with_one_exit_each() { + let windows = managed_socket_windows(3, 1).await.unwrap(); + assert_eq!(windows[0].0, "x".repeat(3 * 65536)); + assert_eq!(windows[0].0, windows[1].0); + assert_eq!(windows[0].1, 1); + assert_eq!(windows[1].1, 1); +} + +#[tokio::test] +async fn managed_retained_reconnect_after_final_preserves_two_reader_handles() { + let windows = managed_socket_windows(3, 2).await.unwrap(); + assert_eq!(windows.len(), 4); + for (data, exits) in windows { + assert_eq!(data, "x".repeat(3 * 65536)); + assert_eq!(exits, 1); + } +} diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index 961c295c7..15182ee5e 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -576,11 +576,8 @@ async fn run_loop( // The capability gates creation; output follows the row owner. _ = managed_output_ticker.tick(), if state.registry.has_managed_controller() => { for terminal_id in state.registry.managed_attached_to(conn_id) { - match refresh_managed_output_and_associate(state, &terminal_id, 64 * 1024).await { - Ok(()) => {} - Err(error) => tracing::warn!(terminal_id = %terminal_id, error = %error, - "managed_runtime.output_refresh_failed"), - } + // The shared registry logs changed classified failures once. + let _ = refresh_managed_output_and_associate(state, &terminal_id, 64 * 1024).await; } } inbound = ws_rx.next() => { @@ -7351,6 +7348,7 @@ pub async fn respawn_agent_terminal( // registry API surface for no user-visible gain. let msg = freshell_protocol::ServerMessage::TerminalStatus(freshell_protocol::TerminalStatus { + recovery_summary: None, status: freshell_protocol::RuntimeStatus::Recovering, terminal_id: terminal_id.clone(), attempt: None, @@ -8495,25 +8493,77 @@ async fn refresh_managed_output_and_associate( // still binding the native identity. let gate = managed_output_gate(terminal_id); let _guard = gate.lock().await; - if !state.registry.is_live(terminal_id) { + if state.registry.managed_descriptor(terminal_id).is_none() { return Ok(()); } - // Bound the remote read, but never cancel the subsequent durable identity - // transaction mid-write merely because it took longer than the read SLA. - let read = tokio::time::timeout( + let result = tokio::time::timeout( std::time::Duration::from_millis(500), state .registry .refresh_managed_output(terminal_id, max_bytes), ) .await - .map_err(|_| "managed output read timed out".to_string())??; + .map_err(|_| "managed output read timed out".to_string()) + .and_then(|read| read); + let read = match result { + Ok(read) => { + state + .registry + .observe_managed_output_failure(terminal_id, None); + read + } + Err(error) => { + let cause = if error.contains("timed out") || error.contains("timeout") { + "timeout" + } else if error.contains("identity") || error.contains("ownership") { + "identity" + } else if error.contains("frame") + || error.contains("sequence") + || error.contains("bounds") + { + "invalid_output" + } else { + "transport_unavailable" + }; + state + .registry + .observe_managed_output_failure(terminal_id, Some(cause)); + return Err(error); + } + }; if let Some(session_id) = read.native_session_id.as_deref() { crate::opencode_association::associate_managed_session(state, terminal_id, session_id) .await; } - if let Some(exit_code) = read.exit_code { - state.registry.finish_managed_exit(terminal_id, exit_code); + state + .registry + .publish_managed_lifecycle(terminal_id, read.lifecycle); + let delivered = state + .registry + .managed_output_cursor(terminal_id) + .unwrap_or(0); + if let Some(completion) = read.completion { + let descriptor = state + .registry + .managed_descriptor(terminal_id) + .ok_or("managed terminal disappeared")?; + if completion.stream_epoch != descriptor.stream_id + || completion.incarnation_id != descriptor.incarnation_id + || completion.head_seq != read.head_seq + { + return Err("managed completion source mismatch".into()); + } + if delivered >= completion.head_seq { + state + .registry + .finish_managed_completion(terminal_id, completion.exit_code); + } + } else if let Some(exit_code) = read.exit_code { + // Transport-independent controller fixtures and older compatible + // controllers carry a known settled code with their sampled head. + if delivered >= read.head_seq { + state.registry.finish_managed_exit(terminal_id, exit_code); + } } Ok(()) } @@ -12649,7 +12699,9 @@ mod managed_output_order_tests { ManagedTerminalDescriptor, ManagedTerminalFuture, }; - struct ExitedOpenCodeRead; + struct ExitedOpenCodeRead { + final_head: i64, + } impl ManagedTerminalController for ExitedOpenCodeRead { fn lookup_terminal<'a>( @@ -12681,7 +12733,7 @@ mod managed_output_order_tests { _: u16, _: u16, ) -> ManagedTerminalFuture<'a, Result<(), String>> { - Box::pin(async { Err("unused resize".into()) }) + Box::pin(async { Ok(()) }) } fn stop<'a>( @@ -12694,24 +12746,34 @@ mod managed_output_order_tests { fn read_output<'a>( &'a self, _: ManagedTerminalDescriptor, - _: i64, + after: i64, _: u64, ) -> ManagedTerminalFuture<'a, Result> { - Box::pin(async { + Box::pin(async move { Ok(ManagedOutputRead { + lifecycle: None, + completion: None, stream_epoch: Some("S-managed-short-lived".into()), incarnation_id: Some("incarnation-short-lived".into()), reset_required: false, truncated: false, retained_from_seq: 1, - head_seq: 1, + head_seq: self.final_head, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), - chunks: vec![ManagedOutputChunk { - seq_start: 1, - seq_end: 1, - data: "last output\n".into(), - }], + chunks: if after < self.final_head { + vec![ManagedOutputChunk { + seq_start: after + 1, + seq_end: after + 1, + data: if self.final_head == 1 { + "last output\n".into() + } else { + "x".repeat(65536) + }, + }] + } else { + vec![] + }, }) }) } @@ -12733,7 +12795,7 @@ mod managed_output_order_tests { }); state .registry - .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead))); + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { final_head: 1 }))); refresh_managed_output_and_associate(&state, terminal_id, 1024) .await @@ -12757,6 +12819,232 @@ mod managed_output_order_tests { assert_eq!(row.resume_session_id.as_deref(), Some("ses-short-lived")); assert_eq!(row.snapshot, "last output\n"); } + + #[tokio::test] + async fn managed_final_exit_waits_for_all_retained_pages() { + let state = super::pane_reconcile_gate_tests::state(); + let terminal = "T-managed-short-lived"; + state.registry.register_managed(ManagedTerminalDescriptor { + soul_id: "soul-short-lived".into(), + incarnation_id: "incarnation-short-lived".into(), + terminal_id: terminal.into(), + stream_id: "S-managed-short-lived".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: None, + create_request_id: Some("req-short-lived".into()), + }); + state + .registry + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { final_head: 3 }))); + for delivered in 1..=3 { + refresh_managed_output_and_associate(&state, terminal, 65536) + .await + .unwrap(); + assert_eq!( + state.registry.managed_output_cursor(terminal), + Some(delivered) + ); + assert_eq!( + state.registry.directory()[0].status, + if delivered == 3 { + freshell_protocol::TerminalRunStatus::Exited + } else { + freshell_protocol::TerminalRunStatus::Running + }, + "exit cannot strand undelivered final pages" + ); + } + assert_eq!(state.registry.directory()[0].snapshot.len(), 3 * 65536); + } + + include!("managed_retained_socket_tests.rs"); + + struct FailedRead(StdMutex>); + impl ManagedTerminalController for FailedRead { + fn lookup_terminal<'a>( + &'a self, + _: &'a str, + _: Option, + ) -> ManagedTerminalFuture<'a, Result, String>> { + Box::pin(async { Err("unused".into()) }) + } + fn launch<'a>( + &'a self, + _: ManagedTerminalLaunch, + ) -> ManagedTerminalFuture<'a, Result> { + Box::pin(async { Err("unused".into()) }) + } + fn input<'a>( + &'a self, + _: ManagedTerminalDescriptor, + _: String, + ) -> ManagedTerminalFuture<'a, Result<(), String>> { + Box::pin(async { Err("unused".into()) }) + } + fn resize<'a>( + &'a self, + _: ManagedTerminalDescriptor, + _: u16, + _: u16, + ) -> ManagedTerminalFuture<'a, Result<(), String>> { + Box::pin(async { Err("unused".into()) }) + } + fn stop<'a>( + &'a self, + _: ManagedTerminalDescriptor, + ) -> ManagedTerminalFuture<'a, Result<(), String>> { + Box::pin(async { Err("unused".into()) }) + } + fn read_output<'a>( + &'a self, + _: ManagedTerminalDescriptor, + after: i64, + _: u64, + ) -> ManagedTerminalFuture<'a, Result> { + Box::pin(async move { + if let Some(error) = self.0.lock().unwrap().clone() { + return Err(error); + } + Ok(ManagedOutputRead { + lifecycle: None, + completion: None, + stream_epoch: None, + incarnation_id: None, + reset_required: false, + truncated: false, + retained_from_seq: after, + head_seq: after, + exit_code: None, + native_session_id: None, + chunks: vec![], + }) + }) + } + } + + #[derive(Clone)] + struct CapturedEvents(Arc>>); + impl tracing_subscriber::Layer for CapturedEvents { + fn on_event( + &self, + event: &tracing::Event<'_>, + _: tracing_subscriber::layer::Context<'_, S>, + ) { + struct Fields(String); + impl tracing::field::Visit for Fields { + fn record_debug( + &mut self, + field: &tracing::field::Field, + value: &dyn std::fmt::Debug, + ) { + self.0.push_str(&format!("{}={value:?};", field.name())); + } + } + let mut fields = Fields(String::new()); + event.record(&mut fields); + self.0.lock().unwrap().push(fields.0); + } + } + + #[tokio::test] + async fn managed_output_failure_transitions_are_shared_by_two_readers_and_redacted() { + use tracing::instrument::WithSubscriber; + use tracing_subscriber::prelude::*; + let events = Arc::new(StdMutex::new(vec![])); + let subscriber = tracing_subscriber::registry().with(CapturedEvents(events.clone())); + async { + let state = super::pane_reconcile_gate_tests::state(); + let terminal = "T-managed-failures"; + state.registry.register_managed(ManagedTerminalDescriptor { + soul_id: "soul-failure".into(), + incarnation_id: "incarnation-failure".into(), + terminal_id: terminal.into(), + stream_id: "epoch-failure".into(), + mode: "shell".into(), + cwd: "/workspace".into(), + resume_session_id: None, + create_request_id: None, + }); + let controller = Arc::new(FailedRead(StdMutex::new(Some( + "transport unavailable secret=synthetic-sensitive".into(), + )))); + state + .registry + .set_managed_controller(Some(controller.clone())); + for conn in 1..=2 { + assert!( + state + .registry + .attach( + terminal, + conn, + Arc::new(|_| {}), + None, + 0, + false, + false, + None, + None, + None, + Default::default() + ) + .found + ); + } + for _ in 0..20 { + for conn in 1..=2 { + assert_eq!(state.registry.managed_attached_to(conn), vec![terminal]); + assert!( + refresh_managed_output_and_associate(&state, terminal, 65536) + .await + .is_err() + ); + } + } + let first = events + .lock() + .unwrap() + .iter() + .filter(|e| e.contains("managed_runtime.output_refresh_failed")) + .count(); + assert_eq!( + first, 1, + "one shared warning for forty unchanged socket polls" + ); + *controller.0.lock().unwrap() = Some("managed output read timed out".into()); + assert!( + refresh_managed_output_and_associate(&state, terminal, 65536) + .await + .is_err() + ); + *controller.0.lock().unwrap() = None; + refresh_managed_output_and_associate(&state, terminal, 65536) + .await + .unwrap(); + let captured = events.lock().unwrap(); + assert_eq!( + captured + .iter() + .filter(|e| e.contains("managed_runtime.output_refresh_failed")) + .count(), + 2 + ); + assert_eq!( + captured + .iter() + .filter(|e| e.contains("managed_runtime.output_refresh_recovered")) + .count(), + 1 + ); + assert!( + captured.iter().all(|e| !e.contains("synthetic-sensitive")), + "raw remote error contents must not enter logs" + ); + } + .with_subscriber(subscriber) + .await; + } } #[cfg(test)] mod host_stats_dispatch_tests { From 08b4dcd91047bfd85bbf32dc39050487f010ba27 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:26:06 -0700 Subject: [PATCH 234/493] fix: retain managed terminal output and publish final recovery outcomes --- shared/managed-runtime.ts | 1 + shared/ws-protocol.ts | 12 +++- src/App.tsx | 19 +++++- src/components/BackgroundSessions.tsx | 11 ++-- src/components/OverviewView.tsx | 59 +++++++++---------- src/components/Sidebar.tsx | 11 +++- src/components/TerminalView.tsx | 53 +++++++++++------ src/lib/managed-terminal.ts | 30 ++++++++++ src/lib/session-type-utils.ts | 7 ++- src/store/panesSlice.ts | 1 + src/store/selectors/sidebarSelectors.ts | 28 ++++++--- src/store/tabsSlice.ts | 10 +++- src/store/types.ts | 5 +- .../managed-recovery-contextual-ui.spec.ts | 26 ++++++++ .../components/App.ws-bootstrap.test.tsx | 15 +++++ .../components/BackgroundSessions.test.tsx | 17 ++++++ .../client/components/OverviewView.test.tsx | 13 ++++ test/unit/client/components/Sidebar.test.tsx | 18 ++++++ .../TerminalView.exitBanner.test.tsx | 39 +++++++++++- .../sidebarSelectors.runningTerminal.test.ts | 12 ++++ 20 files changed, 310 insertions(+), 77 deletions(-) create mode 100644 src/lib/managed-terminal.ts diff --git a/shared/managed-runtime.ts b/shared/managed-runtime.ts index d542274c6..f09366d69 100644 --- a/shared/managed-runtime.ts +++ b/shared/managed-runtime.ts @@ -180,6 +180,7 @@ export const ManagedRuntimeProjectionFieldsSchema = z.object({ soulId: NonEmptyIdSchema.optional(), incarnationId: NonEmptyIdSchema.optional(), runtimeState: ManagedRuntimeLaunchStateSchema.optional(), + runtimeStatus: z.enum(['running', 'recovering', 'blocked', 'exited']).optional(), viewIntentId: NonEmptyIdSchema.optional(), viewIntentRevision: NonNegativeIntSchema.optional(), soulIntentRevision: NonNegativeIntSchema.optional(), diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index dce69c025..1578dff78 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -14,7 +14,7 @@ import { LiveTerminalHandleSchema, SessionRefSchema, type RestoreError } from '. import { CodexDurabilityRefSchema, type CodexDurabilityRef } from './codex-durability.js' import type { SessionNameRecord, SessionNameRef, SessionNameUpdate } from './session-names.js' import { TabNameSourceSchema } from './session-names.js' -import type { ManagedRuntimeInventoryChangedMessage, ManagedRuntimeViewChangedMessage } from './managed-runtime.js' +import type { ManagedRuntimeInventoryChangedMessage, ManagedRuntimeViewChangedMessage, ManagedRuntimeRecoverySummary } from './managed-runtime.js' // ────────────────────────────────────────────────────────────── // Shared enums and helpers @@ -1509,12 +1509,15 @@ export type TerminalExitMessage = { type: 'terminal.exit' terminalId: string exitCode: number + managedRetained?: boolean + exitCodeAvailable?: boolean } export type TerminalStatusMessage = { type: 'terminal.status' terminalId: string - status: 'running' | 'recovering' | 'exited' + status: 'running' | 'recovering' | 'blocked' | 'exited' + recoverySummary?: ManagedRuntimeRecoverySummary reason?: string attempt?: number /** Auto-resume 'recovering' frames only: the bounded retry budget. The @@ -1933,7 +1936,10 @@ export type TerminalInventoryMessage = { createdAt: number lastActivityAt: number status: 'running' | 'exited' - runtimeStatus?: 'running' | 'recovering' + runtimeStatus?: 'running' | 'recovering' | 'blocked' | 'exited' + soulId?: string + incarnationId?: string + recoverySummary?: ManagedRuntimeRecoverySummary cwd?: string codexDurability?: CodexDurabilityRef /** Server→client only, additive + optional: the terminal's resume target is an opencode subagent (child) session. */ diff --git a/src/App.tsx b/src/App.tsx index 867d069c7..4f310d225 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -108,7 +108,9 @@ import { X, Copy, Check, PanelLeft, AlertTriangle } from 'lucide-react' import { updateSettingsLocal } from '@/store/settingsSlice' import { setTerminalMetaSnapshot, upsertTerminalMeta, removeTerminalMeta } from '@/store/terminalMetaSlice' -import { clearAllReconcilePendingPanes, clearDeadSessionAdjudication, clearDeadTerminals, clearReconcileWarming, clearTerminalLiveHandles, setReconcilePendingPanes } from '@/store/panesSlice' +import { clearAllReconcilePendingPanes, clearDeadSessionAdjudication, clearDeadTerminals, clearReconcileWarming, clearTerminalLiveHandles, setReconcilePendingPanes, updatePaneContent } from '@/store/panesSlice' +import { collectPaneEntries } from '@/lib/pane-utils' +import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal.js' import { addTerminalFreshRecoveryRequestId, addTerminalRestoreRequestId, setPaneReconcileActive } from '@/lib/terminal-restore' import { reconcileTerminalSessionAssociation } from '@/lib/terminal-session-association' import { foldTerminalInventoryTitles } from '@/lib/terminal-inventory-titles' @@ -1642,9 +1644,22 @@ export default function App() { sessionRef: terminal.sessionRef, }) } + if (isRetainedManagedTerminal(terminal)) { + for (const [tabId, layout] of Object.entries(appStore.getState().panes.layouts)) { + for (const { paneId, content } of collectPaneEntries(layout)) { + if (content.kind === 'terminal' && content.terminalId === terminal.terminalId) { + dispatch(updatePaneContent({ tabId, paneId, content: { + ...content, + ...managedTerminalProjection(terminal), + ...(terminal.status === 'exited' ? { status: 'exited' } : {}), + } })) + } + } + } + } } const liveIds = terminals - .filter((t: any) => t.status === 'running') + .filter((t: any) => t.status === 'running' || isRetainedManagedTerminal(t)) .map((t: any) => t.terminalId as string) dispatch(setLiveTerminalIds(liveIds)) dispatch(setServerRestarted(false)) diff --git a/src/components/BackgroundSessions.tsx b/src/components/BackgroundSessions.tsx index db871fd51..4e7a1abfc 100644 --- a/src/components/BackgroundSessions.tsx +++ b/src/components/BackgroundSessions.tsx @@ -10,6 +10,7 @@ import { addTab } from '@/store/tabsSlice' import { initLayout } from '@/store/panesSlice' import { fetchTerminalDirectoryWindow } from '@/store/terminalDirectoryThunks' import type { BackgroundTerminal } from '@/store/types' +import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal' const EMPTY_TERMINALS: BackgroundTerminal[] = [] @@ -47,7 +48,7 @@ export default function BackgroundSessions() { } }, [refresh]) - const detachedRunning = terminals.filter((t) => t.status === 'running' && !t.hasClients) + const detachedRunning = terminals.filter((t) => (t.status === 'running' || isRetainedManagedTerminal(t)) && !t.hasClients) // b8ke fence-heal (fix b): detached terminals have NO mounted // TerminalView to consume their typed kill refusal — fold it HERE so @@ -97,7 +98,7 @@ export default function BackgroundSessions() { {detachedRunning.length === 0 ? (
- No detached running terminals. Close a tab to detach it, or kill it with Shift+Close. + No detached terminals. Close a tab to detach it, or kill it with Shift+Close.
) : (
@@ -128,7 +129,8 @@ export default function BackgroundSessions() { dispatch(addTab({ id: tabId, title: t.title, - status: 'running', + status: t.status === 'exited' ? 'exited' : 'running', + ...managedTerminalProjection(t), mode: mode as any, sessionRef: t.sessionRef, ...(initialCwd ? { initialCwd } : {}), @@ -139,7 +141,8 @@ export default function BackgroundSessions() { kind: 'terminal', mode: mode as any, terminalId: t.terminalId, - status: 'running', + status: t.status === 'exited' ? 'exited' : 'running', + ...managedTerminalProjection(t), sessionRef: t.sessionRef, ...(initialCwd ? { initialCwd } : {}), }, diff --git a/src/components/OverviewView.tsx b/src/components/OverviewView.tsx index 5d69c8e69..19e97210d 100644 --- a/src/components/OverviewView.tsx +++ b/src/components/OverviewView.tsx @@ -14,17 +14,17 @@ import { collectTerminalIds } from '@/lib/pane-utils' import { cn } from '@/lib/utils' import { RefreshCw, Circle, Play, Pencil, Trash2, Sparkles, ExternalLink } from 'lucide-react' import { ContextIds } from '@/components/context-menu/context-menu-constants' +import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal.js' +import type { BackgroundTerminal } from '@/store/types' -type TerminalOverview = { +type TerminalOverview = BackgroundTerminal & { terminalId: string title: string description?: string createdAt: number lastActivityAt: number - status: 'running' | 'exited' hasClients: boolean cwd?: string - mode?: string /** Unified agent names (Task 1/2 projection): the terminal's naming * identity and its last-known canonical record. */ nameRef?: SessionNameRef @@ -147,8 +147,29 @@ export default function OverviewView({ onOpenTab }: { onOpenTab?: () => void }) }, [ws]) const runningTerminals = items.filter((t) => t.status === 'running') + const attachableTerminals = items.filter((t) => t.status !== 'exited' && (t.status === 'running' || isRetainedManagedTerminal(t))) const exitedTerminals = items.filter((t) => t.status === 'exited') + const openTerminal = (terminal: TerminalOverview) => { + const existing = findTabByTerminalId(terminal.terminalId) + if (existing) { + dispatch(setActiveTab(existing.id)) + } else { + const tabId = nanoid() + const status = terminal.status === 'exited' ? 'exited' : 'running' + const mode = isRetainedManagedTerminal(terminal) ? terminal.mode : 'shell' + dispatch(addTab({ id: tabId, title: terminal.title, status, mode })) + dispatch(initLayout({ tabId, content: { + kind: 'terminal', mode, terminalId: terminal.terminalId, status, + ...(isRetainedManagedTerminal(terminal) ? { + initialCwd: terminal.cwd, sessionRef: terminal.sessionRef, + ...managedTerminalProjection(terminal), + } : {}), + } })) + } + onOpenTab?.() + } + return (
{/* Header */} @@ -193,32 +214,21 @@ export default function OverviewView({ onOpenTab }: { onOpenTab?: () => void }) )} {/* Running terminals */} - {runningTerminals.length > 0 && ( + {attachableTerminals.length > 0 && (

- Running + {attachableTerminals.length === runningTerminals.length ? 'Running' : 'Terminals'}

- {runningTerminals + {attachableTerminals .sort((a, b) => b.lastActivityAt - a.lastActivityAt) .map((t) => ( { - const existing = findTabByTerminalId(t.terminalId) - if (existing) { - dispatch(setActiveTab(existing.id)) - onOpenTab?.() - return - } - const tabId = nanoid() - dispatch(addTab({ id: tabId, title: t.title, status: 'running', mode: 'shell' })) - dispatch(initLayout({ tabId, content: { kind: 'terminal', mode: 'shell', terminalId: t.terminalId, status: 'running' } })) - onOpenTab?.() - }} + onOpen={() => openTerminal(t)} onRename={async (title, description) => { const scoped = terminalCardScopedRename(t) await renameOverviewTerminal({ @@ -270,18 +280,7 @@ export default function OverviewView({ onOpenTab }: { onOpenTab?: () => void }) key={t.terminalId} terminal={t} isOpen={!!findTabByTerminalId(t.terminalId)} - onOpen={() => { - const existing = findTabByTerminalId(t.terminalId) - if (existing) { - dispatch(setActiveTab(existing.id)) - onOpenTab?.() - return - } - const tabId = nanoid() - dispatch(addTab({ id: tabId, title: t.title, status: 'exited', mode: 'shell' })) - dispatch(initLayout({ tabId, content: { kind: 'terminal', mode: 'shell', terminalId: t.terminalId, status: 'exited' } })) - onOpenTab?.() - }} + onOpen={() => openTerminal(t)} onRename={async (title, description) => { const scoped = terminalCardScopedRename(t) await renameOverviewTerminal({ diff --git a/src/components/Sidebar.tsx b/src/components/Sidebar.tsx index b7801564e..dc5d3cc3d 100644 --- a/src/components/Sidebar.tsx +++ b/src/components/Sidebar.tsx @@ -13,6 +13,7 @@ import { resolveSessionTypeConfig, buildResumeContent } from '@/lib/session-type import { getFreshAgentProviderConfig } from '@/lib/fresh-agent-provider-utils' import { resolveFreshAgentType } from '@/lib/fresh-agent-registry' import type { BackgroundTerminal, CodingCliProviderName } from '@/store/types' +import { managedTerminalProjection, sameManagedTerminalProjection } from '@/lib/managed-terminal' import { ALL_AGENTS, ALL_REPOS, @@ -76,6 +77,7 @@ export function areTerminalsEqual(a: BackgroundTerminal[], b: BackgroundTerminal ai.createdAt !== bi.createdAt || ai.cwd !== bi.cwd || ai.status !== bi.status || + !sameManagedTerminalProjection(ai, bi) || ai.hasClients !== bi.hasClients || ai.mode !== bi.mode || !sameSessionRef(ai.sessionRef, bi.sessionRef) @@ -102,6 +104,7 @@ export function areSessionItemsEqual(a: SessionItem[], b: SessionItem[]): boolea ai.subtitle !== bi.subtitle || ai.hasTab !== bi.hasTab || ai.isRunning !== bi.isRunning || + !sameManagedTerminalProjection(ai, bi) || ai.runningTerminalId !== bi.runningTerminalId || !areTerminalIdsEqual(ai.runningTerminalIds, bi.runningTerminalIds) || ai.archived !== bi.archived || @@ -154,6 +157,7 @@ function isSessionItemEqual(a: SessionItem, b: SessionItem): boolean { a.timestamp === b.timestamp && a.hasTab === b.hasTab && a.isRunning === b.isRunning && + sameManagedTerminalProjection(a, b) && a.runningTerminalId === b.runningTerminalId && areTerminalIdsEqual(a.runningTerminalIds, b.runningTerminalIds) && a.archived === b.archived && @@ -474,7 +478,7 @@ export default function Sidebar({ const provider = item.provider as CodingCliProviderName const state = store.getState() const currentActiveTabId = state.tabs.activeTabId - const runningTerminalId = item.isRunning ? item.runningTerminalId : undefined + const runningTerminalId = item.isRunning || item.soulId ? item.runningTerminalId : undefined const localServerInstanceId = state.connection.serverInstanceId // 1. Dedup: if session is already open in a pane, focus it @@ -523,6 +527,7 @@ export default function Sidebar({ provider, sessionType, terminalId: runningTerminalId, + managedRuntime: managedTerminalProjection(item), firstUserMessage: item.firstUserMessage, isSubagent: item.isSubagent, isNonInteractive: item.isNonInteractive, @@ -543,6 +548,7 @@ export default function Sidebar({ provider, sessionType, terminalId: runningTerminalId, + managedRuntime: managedTerminalProjection(item), firstUserMessage: item.firstUserMessage, isSubagent: item.isSubagent, isNonInteractive: item.isNonInteractive, @@ -561,7 +567,7 @@ export default function Sidebar({ cwd: item.cwd, freshAgentProviderSettings, liveTerminal: runningTerminalId && localServerInstanceId - ? { terminalId: runningTerminalId, serverInstanceId: localServerInstanceId } + ? { terminalId: runningTerminalId, serverInstanceId: localServerInstanceId, ...managedTerminalProjection(item) } : undefined, }), })) @@ -1105,6 +1111,7 @@ function areSidebarItemPropsEqual(prev: SidebarItemProps, next: SidebarItemProps a.timestamp === b.timestamp && a.hasTab === b.hasTab && a.isRunning === b.isRunning && + sameManagedTerminalProjection(a, b) && a.runningTerminalId === b.runningTerminalId && areTerminalIdsEqual(a.runningTerminalIds, b.runningTerminalIds) && a.archived === b.archived && diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index ca97abe69..4292dcd61 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -37,6 +37,7 @@ import { retryManagedConversation } from '@/lib/managed-runtime-retry' import { confirmManagedRuntimeStopped } from '@/lib/managed-runtime-stop' import { isManagedRuntimeRecoveryDecision, ManagedRuntimeRecoveryCard } from '@/components/ManagedRuntimeRecoveryCard' import { isManagedRuntimeRecoveryPending } from '@/lib/managed-runtime-recovery-message' +import { canReplayManagedTerminal } from '@/lib/managed-terminal.js' import { buildReconcileRequestForPanes, foldVerdicts } from '@/lib/pane-reconcile' import type { PaneReconcileRequest, SessionRuntimeOwnerMessage } from '@shared/ws-protocol' import { @@ -902,6 +903,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te // Extract terminal-specific fields (safe because we check kind later) const isTerminal = paneContent.kind === 'terminal' const terminalContent = isTerminal ? paneContent : null + const managedReplayAvailable = canReplayManagedTerminal(terminalContent) // kata b8ke (round-1 review: convergence is bidirectional): a TERMINAL // pane observing a FRESH-AGENT owner for its canonical session — the @@ -3464,7 +3466,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te opts?: AttachTerminalOptions, ) => { if (suppressNetworkEffects) return - if (isManagedRuntimeRecoveryDecision(contentRef.current?.recoverySummary)) { + if (isManagedRuntimeRecoveryDecision(contentRef.current?.recoverySummary) && !canReplayManagedTerminal(contentRef.current)) { log.debug('attach gate declined: managed runtime recovery requires an explicit decision', { terminalId: tid, paneId: paneIdRef.current, @@ -4082,7 +4084,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te useEffect(() => { if (suppressNetworkEffects) return if (!isTerminal || !terminalContent) return - if (isManagedRuntimeRecoveryDecision(terminalContent.recoverySummary)) return + if (isManagedRuntimeRecoveryDecision(terminalContent.recoverySummary) && !managedReplayAvailable) return if (shouldWaitForProviderBehavior) return const termCandidate = termRef.current if (!termCandidate) return @@ -4647,7 +4649,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te } unsub = ws.onMessage((msg) => { - if (isManagedRuntimeRecoveryDecision(contentRef.current?.recoverySummary)) return + if (isManagedRuntimeRecoveryDecision(contentRef.current?.recoverySummary) && !canReplayManagedTerminal(contentRef.current)) return const tid = terminalIdRef.current const reqId = requestIdRef.current @@ -5959,6 +5961,9 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te ) } if (msg.terminalId === tid) { + if (contentRef.current?.soulId && msg.recoverySummary) { + updateContent({ runtimeStatus: msg.status, recoverySummary: msg.recoverySummary }) + } if ( msg.status === 'running' || msg.status === 'recovering' @@ -6016,18 +6021,29 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te } if (msg.type === 'terminal.exit' && msg.terminalId === tid) { - // Record on EVERY exit path (including the early-return branches - // below): this is the only moment both paneId and the dying - // terminalId are simultaneously known — the tail clears - // terminalIdRef/terminalId, and subsequent auto-resume frames - // (terminal.status 'recovering' / terminal.replaced) carry the old - // id and can only be matched via the recorded lastTerminalId. - dispatch(recordTerminalExit({ - paneId: paneIdRef.current, - terminalId: msg.terminalId, - exitCode: msg.exitCode, - at: Date.now(), - })) + // An unavailable managed code clears stale exit evidence. Known + // codes retain the ordinary lifecycle record used by auto-resume. + if (msg.managedRetained === true && msg.exitCodeAvailable === false) { + dispatch(clearTerminalLifecycle({ paneId: paneIdRef.current })) + } else { + dispatch(recordTerminalExit({ + paneId: paneIdRef.current, + terminalId: msg.terminalId, + exitCode: msg.exitCode, + at: Date.now(), + })) + } + + if (msg.managedRetained === true) { + launchAttemptRef.current = null + clearQuarantineRepair() + dispatch(clearPaneRuntimeActivity({ paneId: paneIdRef.current })) + discardPendingInput('terminal_gone') + updateContent({ status: 'exited', runtimeStatus: 'exited' }) + const exitTab = tabRef.current + if (exitTab) dispatch(updateTab({ id: exitTab.id, updates: { status: 'exited' } })) + return + } const launchAttempt = launchAttemptRef.current const exitedDuringLaunch = launchAttempt?.terminalId === tid && !launchAttempt.attachReady @@ -7027,6 +7043,9 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te terminalContent?.reconcileEpoch, terminalContent?.recoverySummary?.desiredState, terminalContent?.recoverySummary?.recoveryState, + // A durable inventory can establish a retained handle after the decision + // card mounted. Replay starts then without waiting for an identity change. + managedReplayAvailable, // reconcilePendingSince: re-run when the pane's pre-verdict wait state // changes -- the verdict fold (or the bounded timeout) clears the entry // and the deferred mount-create must then proceed (Task 8). @@ -7206,9 +7225,9 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const hasFatalConnectionError = isFatalConnectionErrorCode(connectionErrorCode) const managedRecoveryDecision = isManagedRuntimeRecoveryDecision(terminalContent.recoverySummary) const managedTerminal = Boolean(terminalContent.soulId || terminalContent.recoverySummary) - const managedAutomaticRecovery = (managedTerminal && !terminalContent.recoverySummary) + const managedAutomaticRecovery = terminalContent.runtimeStatus !== 'exited' && ((managedTerminal && !terminalContent.recoverySummary) || terminalContent.recoverySummary?.recoveryState === 'live' - || terminalContent.recoverySummary?.recoveryState === 'recovering' + || terminalContent.recoverySummary?.recoveryState === 'recovering') // Keep recovery diagnostics in state, but show only actionable failures. const visibleNotice = managedTerminal ? null : activeNotice const visibleCrashTrace = managedTerminal ? null : terminalContent.crashTrace diff --git a/src/lib/managed-terminal.ts b/src/lib/managed-terminal.ts new file mode 100644 index 000000000..25bbb0729 --- /dev/null +++ b/src/lib/managed-terminal.ts @@ -0,0 +1,30 @@ +import type { ManagedRuntimeProjectionFields } from '../../shared/managed-runtime.js' + +/** A retained facade remains attachable independently of its process state. */ +export function isRetainedManagedTerminal(terminal: { terminalId?: string; soulId?: string }): boolean { + return Boolean(terminal.terminalId && terminal.soulId) +} + +export function canReplayManagedTerminal(terminal?: (ManagedRuntimeProjectionFields & { terminalId?: string }) | null): boolean { + return Boolean(terminal?.terminalId && terminal.soulId && terminal.runtimeStatus) +} + +export function managedTerminalProjection(terminal?: ManagedRuntimeProjectionFields): ManagedRuntimeProjectionFields { + if (!terminal?.soulId) return {} + return { + soulId: terminal.soulId, + incarnationId: terminal.incarnationId, + runtimeStatus: terminal.runtimeStatus, + recoverySummary: terminal.recoverySummary, + } +} + +export function sameManagedTerminalProjection(a: ManagedRuntimeProjectionFields, b: ManagedRuntimeProjectionFields): boolean { + if (a.soulId !== b.soulId || a.incarnationId !== b.incarnationId || a.runtimeStatus !== b.runtimeStatus) return false + const first = a.recoverySummary, second = b.recoverySummary + if (first === second) return true + if (!first || !second) return false + return first.desiredState === second.desiredState && first.recoveryState === second.recoveryState + && first.reason === second.reason && first.attemptId === second.attemptId && first.incidentId === second.incidentId + && first.durabilityState === second.durabilityState && first.allocationState === second.allocationState +} diff --git a/src/lib/session-type-utils.ts b/src/lib/session-type-utils.ts index d774e9247..88affe790 100644 --- a/src/lib/session-type-utils.ts +++ b/src/lib/session-type-utils.ts @@ -1,3 +1,5 @@ +import type { ManagedRuntimeProjectionFields } from '../../shared/managed-runtime.js' +import { managedTerminalProjection } from './managed-terminal.js' import type { ComponentType } from 'react' import { PROVIDER_ICONS, DefaultProviderIcon } from '@/components/icons/provider-icons' import { isNonShellMode, getProviderLabel } from '@/lib/coding-cli-utils' @@ -192,7 +194,7 @@ export function buildResumeContent(opts: { liveTerminal?: { terminalId: string serverInstanceId: string - } + } & ManagedRuntimeProjectionFields }): TerminalPaneInput | FreshAgentPaneInput { const freshAgentType = resolveFreshAgentType(opts.sessionType) if (freshAgentType) { @@ -247,7 +249,8 @@ export function buildResumeContent(opts: { ? { terminalId: opts.liveTerminal.terminalId, serverInstanceId: opts.liveTerminal.serverInstanceId, - status: 'running' as const, + status: opts.liveTerminal.runtimeStatus === 'exited' ? 'exited' as const : 'running' as const, + ...managedTerminalProjection(opts.liveTerminal), } : {}), sessionRef: { diff --git a/src/store/panesSlice.ts b/src/store/panesSlice.ts index d2cf99776..c0d86c320 100644 --- a/src/store/panesSlice.ts +++ b/src/store/panesSlice.ts @@ -57,6 +57,7 @@ function normalizeManagedRuntimeProjection( soulId: input.soulId, incarnationId: input.incarnationId, runtimeState: input.runtimeState, + runtimeStatus: input.runtimeStatus, viewIntentId: input.viewIntentId, viewIntentRevision: input.viewIntentRevision, soulIntentRevision: input.soulIntentRevision, diff --git a/src/store/selectors/sidebarSelectors.ts b/src/store/selectors/sidebarSelectors.ts index 159744f7c..40075f548 100644 --- a/src/store/selectors/sidebarSelectors.ts +++ b/src/store/selectors/sidebarSelectors.ts @@ -1,4 +1,6 @@ import { createSelector } from '@reduxjs/toolkit' +import type { ManagedRuntimeProjectionFields } from '../../../shared/managed-runtime.js' +import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal' import type { RootState } from '../store' import type { BackgroundTerminal, CodingCliProviderName, WorktreeGrouping, ProjectGroup } from '../types' import { collectSessionRefsFromTabs } from '@/lib/session-utils' @@ -21,7 +23,7 @@ import { isScopedSessionRow, selectSessionRowRecord } from '@/store/selectors/se */ const selectSessionStatusTiers = makeSelectSessionStatusTiers() -export interface SidebarSessionItem { +export interface SidebarSessionItem extends ManagedRuntimeProjectionFields { id: string sessionId: string provider: CodingCliProviderName @@ -137,6 +139,8 @@ function getCodexDurabilityReason(durability?: CodexDurabilityRef): string | und } type RunningSessionInfo = { + terminal: BackgroundTerminal + isRunning: boolean terminalId: string createdAt: number allTerminalIds: string[] @@ -201,7 +205,7 @@ export function buildSessionItems( } for (const terminal of terminals || []) { - if (terminal.status === 'running') { + if (terminal.status === 'running' || isRetainedManagedTerminal(terminal)) { const codexDurabilitySessionId = terminal.mode === 'codex' ? getCodexDurabilitySessionId(terminal.codexDurability) : undefined @@ -235,12 +239,17 @@ export function buildSessionItems( existing.codexDurabilityState = codexDurabilityState } existing.codexDurabilityReason = existing.codexDurabilityReason ?? codexDurabilityReason - if (terminal.createdAt < existing.createdAt) { + if ((terminal.soulId && !existing.terminal.soulId) + || (Boolean(terminal.soulId) === Boolean(existing.terminal.soulId) && terminal.createdAt < existing.createdAt)) { + existing.terminal = terminal + existing.isRunning = terminal.status === 'running' existing.terminalId = terminal.terminalId existing.createdAt = terminal.createdAt } } else { runningSessionMap.set(sessionKey, { + terminal, + isRunning: terminal.status === 'running', terminalId: terminal.terminalId, createdAt: terminal.createdAt, allTerminalIds: [terminal.terminalId], @@ -320,7 +329,8 @@ export function buildSessionItems( cwd: session.cwd, hasTab: tabInfo?.hasTab ?? false, ratchetedActivity, - isRunning: !!runningTerminalId, + isRunning: runningTerminal ? runningTerminal.isRunning : !!runningTerminalId, + ...managedTerminalProjection(runningTerminal?.terminal), runningTerminalId, runningTerminalIds, isSubagent: session.isSubagent, @@ -417,7 +427,8 @@ export function buildSessionItems( cwd: input.cwd, hasTab, ratchetedActivity: sessionActivity[key], - isRunning: !!runningTerminalId, + isRunning: runningTerminal ? runningTerminal.isRunning : !!runningTerminalId, + ...managedTerminalProjection(runningTerminal?.terminal), runningTerminalId, runningTerminalIds, isSubagent: input.metadata?.isSubagent @@ -531,7 +542,7 @@ export function buildSessionItems( } for (const terminal of terminals || []) { - if (terminal.status !== 'running') continue + if (terminal.status !== 'running' && !isRetainedManagedTerminal(terminal)) continue if (terminal.sessionRef) continue if (!terminal.mode || terminal.mode === 'shell' || !isNonShellMode(terminal.mode)) continue @@ -574,7 +585,8 @@ export function buildSessionItems( cwd: terminal.cwd, hasTab: paneInfo?.hasTab ?? false, ratchetedActivity: sessionActivity[key], - isRunning: true, + isRunning: terminal.status === 'running', + ...managedTerminalProjection(terminal), runningTerminalId: terminal.terminalId, runningTerminalIds: [terminal.terminalId], isFallback: true, @@ -722,7 +734,7 @@ export function filterSessionItemsByVisibility( if (!settings.showSubagents && item.isSubagent) return false if (settings.ignoreCodexSubagents && item.isSubagent && item.provider === 'codex') return false if (shouldHideAsNonInteractive(item, settings.showNoninteractiveSessions)) return false - if (settings.hideEmptySessions && !item.hasTitle && !item.hasTab && !item.isRunning) return false + if (settings.hideEmptySessions && !item.hasTitle && !item.hasTab && !item.isRunning && !item.soulId) return false if (isExcludedByFirstUserMessage(item.firstUserMessage, exclusions, settings.excludeFirstChatMustStart)) return false return true }) diff --git a/src/store/tabsSlice.ts b/src/store/tabsSlice.ts index 7ff5fdaaf..7a090b264 100644 --- a/src/store/tabsSlice.ts +++ b/src/store/tabsSlice.ts @@ -10,6 +10,7 @@ import { findTabIdForSession, collectSessionRefsFromTabs, liveTerminalRowIdentit import { getProviderLabel } from '@/lib/coding-cli-utils' import { basenameSegment } from '@shared/path-basename' import { buildResumeContent } from '@/lib/session-type-utils' +import { managedTerminalProjection } from '@/lib/managed-terminal' import { getFreshAgentProviderConfig, getFreshAgentProviderLabel } from '@/lib/fresh-agent-provider-utils' import { resolveFreshAgentType } from '@/lib/fresh-agent-registry' import { recordClosedTabSnapshot, pushReopenEntry, popReopenEntry } from './tabRegistrySlice' @@ -1694,13 +1695,14 @@ export const reopenClosedTab = createAsyncThunk( export const openSessionTab = createAsyncThunk( 'tabs/openSessionTab', async ( - { sessionId, title, cwd, provider, sessionType, terminalId, forceNew, firstUserMessage, isSubagent, isNonInteractive, hasTitle, liveTerminalOnly }: { + { sessionId, title, cwd, provider, sessionType, terminalId, managedRuntime, forceNew, firstUserMessage, isSubagent, isNonInteractive, hasTitle, liveTerminalOnly }: { sessionId: string title?: string cwd?: string provider?: CodingCliProviderName sessionType?: string terminalId?: string + managedRuntime?: ManagedRuntimeProjectionFields forceNew?: boolean firstUserMessage?: string isSubagent?: boolean @@ -1821,7 +1823,8 @@ export const openSessionTab = createAsyncThunk( id: tabId, // Coding agents name by working directory; provider label is the fallback. title: title || (cwd ? basenameSegment(cwd) : null) || getProviderLabel(resolvedProvider, extensions), - status: 'running', + status: managedRuntime?.runtimeStatus === 'exited' ? 'exited' : 'running', + ...managedTerminalProjection(managedRuntime), mode: resolvedProvider, codingCliProvider: resolvedProvider, initialCwd: cwd, @@ -1837,7 +1840,8 @@ export const openSessionTab = createAsyncThunk( serverInstanceId: localServerInstanceId, sessionRef: liveTerminalOnly ? undefined : desiredResumeContent.sessionRef, initialCwd: cwd, - status: 'running', + status: managedRuntime?.runtimeStatus === 'exited' ? 'exited' : 'running', + ...managedTerminalProjection(managedRuntime), }, })) return diff --git a/src/store/types.ts b/src/store/types.ts index 54af6709c..5e298b414 100644 --- a/src/store/types.ts +++ b/src/store/types.ts @@ -80,14 +80,13 @@ export interface Tab extends ManagedRuntimeProjectionFields { nameSource?: TabNameSource } -export interface BackgroundTerminal { +export interface BackgroundTerminal extends ManagedRuntimeProjectionFields { terminalId: string title: string createdAt: number lastActivityAt: number cwd?: string - status: 'running' | 'exited' - runtimeStatus?: 'running' | 'recovering' + status: 'running' | 'recovering' | 'blocked' | 'exited' hasClients: boolean mode?: TabMode sessionRef?: SessionLocator diff --git a/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts b/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts index d6f11bcb9..e8b23952e 100644 --- a/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts +++ b/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts @@ -984,3 +984,29 @@ test('routine notices stay silent while cleanup failure is a yellow actionable p expect(receipts.some((receipt) => receipt.noticeId === 'cleanup-failed' && receipt.state === 'dismissed')).toBe(true) expect(receipts.every((receipt) => receipt.profileId.startsWith('profile:'))).toBe(true) }) + +for (const runtimeStatus of ['blocked', 'exited'] as const) { + test(`retained managed ${runtimeStatus} terminal attaches for replay without recreating its provider`, async ({ freshellPage, page, terminal, harness }) => { + await terminal.waitForTerminal() + await installPane(page, 'terminal', runtimeStatus === 'blocked' ? 'blocked' : 'lost') + const before = await paneContent(page) + if (before.kind !== 'terminal' || !before.terminalId) throw new Error('Expected the existing terminal handle') + await harness.clearSentWsMessages() + await page.evaluate((runtimeStatus) => { + const harness = window.__FRESHELL_TEST_HARNESS__! + const state = harness.getState() + const tabId = state.tabs.activeTabId! + const root = state.panes.layouts[tabId] + if (root?.type !== 'leaf') throw new Error('Expected one pane') + harness.dispatch({ type: 'panes/updatePaneContent', payload: { tabId, paneId: root.id, + content: { ...root.content, runtimeStatus, status: runtimeStatus === 'exited' ? 'exited' : 'running' } } }) + }, runtimeStatus) + await expect.poll(async () => (await harness.getSentWsMessages() as Array<{ type?: string; terminalId?: string }>). + some((frame) => frame.type === 'terminal.attach' && frame.terminalId === before.terminalId)).toBe(true) + const messages = await harness.getSentWsMessages() as Array<{ type?: string }> + expect(messages.filter((frame) => ['terminal.create', 'terminal.input'].includes(frame.type ?? ''))).toEqual([]) + await expect.poll(() => paneContent(page)).toMatchObject({ terminalId: before.terminalId, + createRequestId: before.createRequestId, soulId: SOUL_ID, runtimeStatus }) + await expect(page.getByTestId('managed-runtime-recovery-card')).toBeVisible() + }) +} diff --git a/test/unit/client/components/App.ws-bootstrap.test.tsx b/test/unit/client/components/App.ws-bootstrap.test.tsx index 40b0268ce..29e204002 100644 --- a/test/unit/client/components/App.ws-bootstrap.test.tsx +++ b/test/unit/client/components/App.ws-bootstrap.test.tsx @@ -1046,6 +1046,21 @@ describe('App WS bootstrap recovery', () => { }) }) + it('keeps a verified final managed handle during the inventory census', async () => { + const store=createStore({tabs:[{id:'retained-tab',mode:'shell',status:'exited'}],panes:{layouts:{ + 'retained-tab':{type:'leaf',id:'retained-pane',content:{kind:'terminal',mode:'shell',status:'exited', + terminalId:'retained-managed',streamId:'retained-epoch',createRequestId:'retained-create',soulId:'soul-retained'}}}, + activePane:{'retained-tab':'retained-pane'}}}) + render() + await waitFor(()=>expect(messageHandler).toBeTypeOf('function')) + act(()=>messageHandler?.({type:'terminal.inventory',terminals:[{terminalId:'retained-managed',mode:'shell',status:'exited', + runtimeStatus:'exited',soulId:'soul-retained',incarnationId:'inc-retained',title:'Shell',createdAt:1,lastActivityAt:2}]})) + const layout=store.getState().panes.layouts['retained-tab'] + expect(layout.type).toBe('leaf') + if(layout.type!=='leaf') throw new Error('expected leaf') + expect(layout.content).toMatchObject({terminalId:'retained-managed',streamId:'retained-epoch',createRequestId:'retained-create',status:'exited'}) + }) + it('recovers an OpenCode sessionRef from inventory before clearing a stale live handle', async () => { const store = createStore({ tabs: [{ diff --git a/test/unit/client/components/BackgroundSessions.test.tsx b/test/unit/client/components/BackgroundSessions.test.tsx index 35ad2f2b2..de028312e 100644 --- a/test/unit/client/components/BackgroundSessions.test.tsx +++ b/test/unit/client/components/BackgroundSessions.test.tsx @@ -79,6 +79,23 @@ describe('BackgroundSessions', () => { cleanup() }) + it.each(['blocked', 'exited'] as const)('attaches a retained managed %s row with its existing identity', async (status) => { + mockGetTerminalDirectoryPage.mockResolvedValue({items:[{terminalId:'retained-managed',title:'Retained managed', + mode:'codex',status,runtimeStatus:status,soulId:'soul-retained',hasClients:false,createdAt:1,lastActivityAt:2, + sessionRef:{provider:'codex',sessionId:'native-retained'}}],nextCursor:null,revision:1}) + const store=makeStore() + const user=userEvent.setup() + render() + await user.click(await screen.findByRole('button',{name:'Attach'})) + const [tab]=store.getState().tabs.tabs + const layout=store.getState().panes.layouts[tab.id] + expect(layout.type).toBe('leaf') + if(layout.type!=='leaf') throw new Error('expected leaf') + expect(layout.content).toMatchObject({terminalId:'retained-managed',mode:'codex',soulId:'soul-retained',runtimeStatus:status, + sessionRef:{provider:'codex',sessionId:'native-retained'}}) + expect(layout.content.status).toBe(status==='exited'?'exited':'running') + }) + it('attaches with the real terminal mode, not hardcoded shell', async () => { const store = makeStore() const user = userEvent.setup() diff --git a/test/unit/client/components/OverviewView.test.tsx b/test/unit/client/components/OverviewView.test.tsx index 37752130a..21a283ea6 100644 --- a/test/unit/client/components/OverviewView.test.tsx +++ b/test/unit/client/components/OverviewView.test.tsx @@ -109,6 +109,19 @@ describe('OverviewView terminal cards — unified agent names', () => { afterEach(() => cleanup()) + it.each(['blocked', 'exited'] as const)('opens the retained managed %s card without creating a shell replacement', async (status) => { + mockApiGet.mockResolvedValue([scopedRow({status,runtimeStatus:status,soulId:'soul-retained'})]) + const store=buildStore() + renderOverview(store) + fireEvent.click(await screen.findByRole('button',{name:'Open terminal Row last-known name'})) + const [tab]=store.getState().tabs.tabs + const layout=store.getState().panes.layouts[tab.id] + expect(layout.type).toBe('leaf') + if(layout.type!=='leaf') throw new Error('expected leaf') + expect(layout.content).toMatchObject({terminalId:'term-1',mode:'claude',soulId:'soul-retained',runtimeStatus:status}) + expect(mockApiPost).not.toHaveBeenCalled() + }) + it('resolves a scoped card display through the live canonical cache without a directory refetch', async () => { mockApiGet.mockResolvedValue([scopedRow()]) const store = buildStore() diff --git a/test/unit/client/components/Sidebar.test.tsx b/test/unit/client/components/Sidebar.test.tsx index 19ba2159d..a35d6abef 100644 --- a/test/unit/client/components/Sidebar.test.tsx +++ b/test/unit/client/components/Sidebar.test.tsx @@ -825,6 +825,24 @@ describe('Sidebar Component - Session-Centric Display', () => { }) describe('running session decoration', () => { + it.each(['blocked', 'exited'] as const)('clicks the retained managed %s terminal without resuming a new process', async (status) => { + const id=sessionId('managed-retained') + const projects:ProjectGroup[]=[{projectPath:'/repo',sessions:[{sessionId:id,projectPath:'/repo',lastActivityAt:1, + title:'Retained managed session',cwd:'/repo'}]}] + const terminal={terminalId:'retained-managed',title:'Retained',mode:'claude',status,runtimeStatus:status, + soulId:'soul-retained',hasClients:false,createdAt:1,lastActivityAt:2,sessionRef:{provider:'claude',sessionId:id},cwd:'/repo'} as unknown as BackgroundTerminal + const store=createTestStore({projects,serverInstanceId:'srv-local'}) + renderSidebar(store,[terminal]) + await act(async()=>{vi.advanceTimersByTime(100)}) + fireEvent.click(screen.getByText('Retained managed session').closest('button')!) + const [tab]=store.getState().tabs.tabs + const layout=store.getState().panes.layouts[tab.id] + expect(layout.type).toBe('leaf') + if(layout.type!=='leaf') throw new Error('expected leaf') + expect(layout.content).toMatchObject({terminalId:'retained-managed',soulId:'soul-retained',runtimeStatus:status, + sessionRef:{provider:'claude',sessionId:id}}) + }) + it('marks session as running when matching terminal exists', async () => { const projects: ProjectGroup[] = [ { diff --git a/test/unit/client/components/TerminalView.exitBanner.test.tsx b/test/unit/client/components/TerminalView.exitBanner.test.tsx index 4dd242151..c882a098e 100644 --- a/test/unit/client/components/TerminalView.exitBanner.test.tsx +++ b/test/unit/client/components/TerminalView.exitBanner.test.tsx @@ -1,7 +1,7 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import { act, render, screen, fireEvent, cleanup } from '@testing-library/react' import { configureStore } from '@reduxjs/toolkit' -import { Provider } from 'react-redux' +import { Provider, useSelector } from 'react-redux' import tabsReducer from '@/store/tabsSlice' import panesReducer from '@/store/panesSlice' import settingsReducer, { defaultSettings } from '@/store/settingsSlice' @@ -178,10 +178,21 @@ function paneState(store: ReturnType['store']) { return layout.content } -async function renderPane(store: any, paneContent: TerminalPaneContent) { +function StoredTerminalPane() { + const content = useSelector((state: ReturnType['store']['getState']>) => { + const root = state.panes.layouts[TAB] + if (root.type !== 'leaf' || root.content.kind !== 'terminal') { + throw new Error('Expected the fixture terminal pane') + } + return root.content + }) + return +} + +async function renderPane(store: any, _paneContent: TerminalPaneContent) { render( - + ) await act(async () => { @@ -335,6 +346,28 @@ describe('TerminalView exited-pane error banner', () => { expect(screen.queryByRole('alert')).toBeNull() }) + it('retains managed terminal, stream and restore identity after final exit', async () => { + const {store,paneContent}=makeStore({mode:'shell',status:'running',managed:true,withSessionRef:false}) + act(()=>store.dispatch(updatePaneContent({tabId:TAB,paneId:PANE,content:{...paneContent, + terminalId:'retained-managed',streamId:'retained-epoch'}}))) + await renderPane(store,paneState(store)) + act(()=>messageHandler!({type:'terminal.exit',terminalId:'retained-managed',exitCode:17,managedRetained:true,exitCodeAvailable:true})) + expect(paneState(store)).toMatchObject({terminalId:'retained-managed',streamId:'retained-epoch',createRequestId:REQ,status:'exited',soulId:'managed-soul'}) + }) + + it('does not display a compatibility exit code when managed completion reports it unavailable', async () => { + const {store,paneContent}=makeStore({mode:'codex',status:'running',managed:true, + lifecycle:{lastTerminalId:'retained-managed',exit:{exitCode:9,at:Date.now()}}}) + act(()=>store.dispatch(updatePaneContent({tabId:TAB,paneId:PANE,content:{...paneContent, + terminalId:'retained-managed',streamId:'retained-epoch'}}))) + await renderPane(store,paneState(store)) + act(()=>messageHandler!({type:'terminal.exit',terminalId:'retained-managed',exitCode:-1, + managedRetained:true,exitCodeAvailable:false})) + expect(selectExitRecordFrom(store.getState().terminalLifecycle,PANE)).toBeUndefined() + expect(screen.getByRole('alert')).not.toHaveTextContent('code -1') + expect(paneState(store)).toMatchObject({terminalId:'retained-managed',streamId:'retained-epoch',status:'exited'}) + }) + it('shows a codeless alert with Relaunch for an exited agent pane with NO exit record (post-reload)', async () => { const { store, paneContent } = makeStore({ mode: 'claude', diff --git a/test/unit/client/store/selectors/sidebarSelectors.runningTerminal.test.ts b/test/unit/client/store/selectors/sidebarSelectors.runningTerminal.test.ts index df7ed69ed..c0a504b16 100644 --- a/test/unit/client/store/selectors/sidebarSelectors.runningTerminal.test.ts +++ b/test/unit/client/store/selectors/sidebarSelectors.runningTerminal.test.ts @@ -80,6 +80,18 @@ describe('sidebarSelectors running session mapping', () => { paneTitles: {}, } as any + it.each(['blocked', 'exited'] as const)('retains exact managed %s membership without a running indicator', (status) => { + const state = createState() + const terminal = {terminalId:'retained-managed',title:'Retained',createdAt:10,lastActivityAt:20, + status, runtimeStatus:status,soulId:'soul-retained',hasClients:false,mode:'codex', + sessionRef:{provider:'codex',sessionId:'session-1'}} as unknown as BackgroundTerminal + const [item] = makeSelectSortedSessionItems()(state,[terminal],'') + expect(item.runningTerminalId).toBe('retained-managed') + expect(item.isRunning).toBe(false) + const synthetic = buildSessionItems([],[],emptyPanes,[{...terminal,sessionRef:undefined}],{},'repo') + expect(synthetic[0]).toMatchObject({sessionId:'terminal:retained-managed',runningTerminalId:'retained-managed',isRunning:false}) + }) + it('pins session runningTerminalId to the oldest running terminal when duplicate mappings exist', () => { const selector = makeSelectSortedSessionItems() const state = createState() From b6b65599bc0400ae2c2c74f341ee0e9e0eb48421 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:17:25 -0700 Subject: [PATCH 235/493] fix: retain managed terminal output and publish final recovery outcomes --- crates/freshell-supervisor/src/registry.rs | 116 ++++++- .../src/retained_output_tests.rs | 286 ++++++++++++++++++ crates/freshell-terminal/src/registry.rs | 5 +- crates/freshell-ws/src/terminal.rs | 173 ++++++++++- src/components/TerminalView.tsx | 2 +- .../managed-recovery-contextual-ui.spec.ts | 20 ++ .../TerminalView.lifecycle.test.tsx | 41 +++ 7 files changed, 624 insertions(+), 19 deletions(-) diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 10312cbda..4b15cc38e 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -778,18 +778,34 @@ impl Registry { &self, soul_id: SoulId, ) -> Result { - // The existing context resolver proves installation ownership and - // selects the current committed incarnation (or its committed prior). - let context = self.recovery_context(soul_id).await?; - let handle = context.prior_handle; - Ok(RetainedOutputSource { - incarnation_id: handle.incarnation_id().clone(), - runtime_dir: handle.runtime_dir().to_path_buf(), - terminal: handle.terminal().cloned().ok_or_else(|| { - RegistryError::InvalidState("soul has no retained terminal".into()) - })?, - native_session_id: context.native_session_id, + let installation = self.installation_id().clone(); + self.run_blocking(move |mut conn| { + // Read one registry snapshot. Recovery's current control handle + // can be an unaccepted replacement; retained reads instead follow + // the last successful acceptance, independently of launch state. + let tx = conn.transaction()?; + let native_session_id: Option = tx + .query_row( + "SELECT native_session_id FROM souls WHERE soul_id=?1", + [soul_id.as_str()], + |row| row.get(0), + ) + .optional()? + .ok_or_else(|| RegistryError::UnknownSoul(soul_id.clone()))?; + let incarnation_id = load_retained_incarnation(&tx, &soul_id)?; + let handle = load_owned_handle(&tx, installation, &incarnation_id)?; + let source = RetainedOutputSource { + incarnation_id, + runtime_dir: handle.runtime_dir().to_path_buf(), + terminal: handle.terminal().cloned().ok_or_else(|| { + RegistryError::InvalidState("soul has no retained terminal".into()) + })?, + native_session_id, + }; + tx.commit()?; + Ok(source) }) + .await } pub async fn begin_stop(&self, soul_id: SoulId) -> Result { @@ -1743,8 +1759,8 @@ impl Registry { params![next_window, next_successes, now, soul_id.as_str()], )?; tx.execute( - "UPDATE recovery_attempts SET verdict='succeeded',finished_at=?1 WHERE attempt_id=?2", - params![now, attempt_id.as_str()], + "UPDATE recovery_attempts SET verdict='succeeded',finished_at=?1,evidence_ref=?3 WHERE attempt_id=?2", + params![now, attempt_id.as_str(), serde_json::to_string(&vec![format!("{ACCEPTED_INCARNATION_REF}{}", incarnation_id.as_str())])?], )?; tx.execute( "UPDATE incarnations SET provider_preparation_pending=1 WHERE incarnation_id=?1", @@ -2562,6 +2578,80 @@ fn load_prepared_row( }) } +const ACCEPTED_INCARNATION_REF: &str = "accepted-incarnation://"; + +fn load_retained_incarnation( + conn: &Connection, + soul_id: &SoulId, +) -> Result { + type AcceptedAttempt = (String, String, Option, i64, i64); + let accepted: Option = conn.query_row( + "SELECT attempt_id,prior_incarnation_id,evidence_ref,started_at,finished_at FROM recovery_attempts WHERE soul_id=?1 AND verdict='succeeded' AND finished_at IS NOT NULL ORDER BY rowid DESC LIMIT 1", + [soul_id.as_str()], + |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?, row.get(4)?)), + ).optional()?; + let candidates = if let Some((attempt, prior, evidence, started, finished)) = accepted { + let exact = evidence + .map(|value| serde_json::from_str::>(&value)) + .transpose()? + .unwrap_or_default() + .into_iter() + .find_map(|value| { + value + .strip_prefix(ACCEPTED_INCARNATION_REF) + .map(str::to_owned) + }); + if let Some(exact) = exact { + let belongs: bool = conn.query_row( + "SELECT EXISTS(SELECT 1 FROM incarnations WHERE incarnation_id=?1 AND soul_id=?2 AND prior_incarnation_id=?3)", + params![exact, soul_id.as_str(), prior], |row| row.get(0), + )?; + if !belongs { + return Err(RegistryError::Integrity( + "accepted retained incarnation relationship is missing".into(), + )); + } + vec![exact] + } else { + // Older success rows lack the acceptance reference. Their original + // attempt link is sufficient when preserved. begin_recovery can + // overwrite that link on the accepted prior, so otherwise require + // one child created within this successful attempt's interval. + // Spool existence and current launch state are never acceptance. + let mut stmt = conn.prepare("SELECT incarnation_id FROM incarnations WHERE soul_id=?1 AND prior_incarnation_id=?2 AND recovery_attempt_id=?3")?; + let linked = stmt + .query_map(params![soul_id.as_str(), prior, attempt], |row| { + row.get::<_, String>(0) + })? + .collect::, _>>()?; + if linked.is_empty() { + let mut stmt = conn.prepare("SELECT incarnation_id FROM incarnations WHERE soul_id=?1 AND prior_incarnation_id=?2 AND created_at BETWEEN ?3 AND ?4")?; + let candidates = stmt + .query_map(params![soul_id.as_str(), prior, started, finished], |row| { + row.get::<_, String>(0) + })? + .collect::, _>>()?; + candidates + } else { + linked + } + } + } else { + let mut stmt = conn.prepare("SELECT incarnation_id FROM incarnations WHERE soul_id=?1 AND prior_incarnation_id IS NULL")?; + let candidates = stmt + .query_map([soul_id.as_str()], |row| row.get::<_, String>(0))? + .collect::, _>>()?; + candidates + }; + let [raw] = candidates.as_slice() else { + return Err(RegistryError::InvalidState( + "retained output acceptance is missing or ambiguous; preserve runtime data and inspect the successful recovery attempt".into(), + )); + }; + IncarnationId::parse(raw.clone()) + .map_err(|_| RegistryError::Integrity("invalid retained incarnation id".into())) +} + pub(crate) fn load_owned_handle( conn: &Connection, installation: InstallationId, diff --git a/crates/freshell-supervisor/src/retained_output_tests.rs b/crates/freshell-supervisor/src/retained_output_tests.rs index c03bd6a1a..c47e682ac 100644 --- a/crates/freshell-supervisor/src/retained_output_tests.rs +++ b/crates/freshell-supervisor/src/retained_output_tests.rs @@ -73,6 +73,292 @@ async fn retained_output_read( } } +async fn retained_output_replacement( + supervisor: &Supervisor, + soul: SoulId, + with_spool: bool, +) -> crate::registry::ReplacementPreparation { + let registry = &supervisor.registry; + let start = registry + .begin_recovery( + soul.clone(), + RecoveryTrigger::ManualRetry, + RecoveryPath::CheckpointRestore, + ) + .await + .unwrap(); + registry + .mark_stop_outcome( + start.context.prior_handle.incarnation_id().clone(), + StopOutcome::VerifiedEmpty, + ) + .await + .unwrap(); + let replacement = registry + .prepare_replacement( + start.clone(), + start.context.prior_handle.terminal().cloned(), + None, + None, + RecoveryPath::CheckpointRestore, + ) + .await + .unwrap(); + let runtime_dir = supervisor + .config + .runtime_root + .join(replacement.prepared.incarnation_id.as_str()); + std::fs::create_dir(&runtime_dir).unwrap(); + registry + .commit_created( + replacement.prepared.incarnation_id.clone(), + crate::registry::BackendCreatedRecord { + daemon_id: freshell_runtime_protocol::DockerDaemonId::new(), + container_id: replacement + .prepared + .incarnation_id + .as_str() + .replace('-', "") + .repeat(2), + image_ref: format!("sha256:{}", "b".repeat(64)), + host_binary_path: runtime_dir.join("host"), + runtime_dir: runtime_dir.clone(), + immutable_config_digest: format!("sha256:{}", "c".repeat(64)), + }, + ) + .await + .unwrap(); + if with_spool { + registry + .commit_execution_grant( + replacement.prepared.incarnation_id.clone(), + freshell_runtime_protocol::HostBootId::new(), + checkpoints::tests::limits(), + ) + .await + .unwrap(); + registry + .mark_running(replacement.prepared.incarnation_id.clone()) + .await + .unwrap(); + std::fs::write(runtime_dir.join("terminal-spool-descriptor.json"), serde_json::to_vec(&serde_json::json!({ + "schemaVersion":1,"incarnationId":replacement.prepared.incarnation_id,"terminalId":"retained-terminal", + "streamEpoch":format!("launch-prefix-replacement-hostboot-{}", replacement.prepared.incarnation_id),"segmentLimitBytes":33554432 + })).unwrap()).unwrap(); + std::fs::write(runtime_dir.join("terminal-spool-current.jsonl"), serde_json::to_string(&serde_json::json!({"frame":{ + "terminalId":"retained-terminal","streamEpoch":format!("launch-prefix-replacement-hostboot-{}", replacement.prepared.incarnation_id),"seqStart":1,"seqEnd":1,"data":format!("replacement output {}", replacement.prepared.incarnation_id) + }})).unwrap()+"\n").unwrap(); + } + replacement +} + +async fn retained_output_assert_warm_and_cold( + supervisor: Supervisor, + soul: SoulId, + accepted: Option, +) { + let config = (*supervisor.config).clone(); + let database_root = supervisor + .registry + .database_path() + .parent() + .unwrap() + .to_path_buf(); + let assert_read = |output: freshell_runtime_protocol::RuntimeOutputBatch| { + assert_eq!( + output.stream_epoch, + accepted + .as_ref() + .map(|id| format!("launch-prefix-replacement-hostboot-{id}")) + .unwrap_or_else(|| "launch-prefix-hostboot-exact".into()) + ); + assert_eq!( + output.frames[0].data, + accepted + .as_ref() + .map(|id| format!("replacement output {id}")) + .unwrap_or_else(|| "x".repeat(65536)) + ); + assert_eq!(output.reset_required, accepted.is_some()); + }; + assert_read( + retained_output_read(&supervisor, soul.clone(), 0) + .await + .expect("warm read resolves accepted output"), + ); + drop(supervisor); + let cold = Supervisor::new( + crate::registry::Registry::open(database_root, None).unwrap(), + std::sync::Arc::new(RecoveryBackend(std::sync::Mutex::new( + BackendRuntimeState::Exited, + ))), + config, + ) + .unwrap(); + assert_read( + retained_output_read(&cold, soul, 0) + .await + .expect("cold read resolves accepted output"), + ); +} + +#[tokio::test] +async fn retained_output_owned_preparing_replacement_keeps_committed_prior() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + retained_output_replacement(&supervisor, context.soul_id.clone(), false).await; + retained_output_assert_warm_and_cold(supervisor, context.soul_id, None).await; +} + +#[tokio::test] +async fn retained_output_failed_activation_without_spool_keeps_committed_prior() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + let replacement = + retained_output_replacement(&supervisor, context.soul_id.clone(), false).await; + supervisor + .registry + .mark_stop_outcome( + replacement.prepared.incarnation_id, + StopOutcome::VerifiedEmpty, + ) + .await + .unwrap(); + supervisor + .registry + .mark_recovery_blocked( + context.soul_id.clone(), + Some(replacement.attempt_id), + RecoveryBlockReason::ProviderUnavailable, + vec!["synthetic activation failure".into()], + ) + .await + .unwrap(); + retained_output_assert_warm_and_cold(supervisor, context.soul_id, None).await; +} + +#[tokio::test] +async fn retained_output_wrong_identity_spool_cannot_displace_committed_prior() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + let replacement = retained_output_replacement(&supervisor, context.soul_id.clone(), true).await; + supervisor + .registry + .mark_stop_outcome( + replacement.prepared.incarnation_id, + StopOutcome::VerifiedEmpty, + ) + .await + .unwrap(); + supervisor + .registry + .mark_recovery_blocked( + context.soul_id.clone(), + Some(replacement.attempt_id), + RecoveryBlockReason::WrongNativeIdentity, + vec!["synthetic wrong identity".into()], + ) + .await + .unwrap(); + retained_output_assert_warm_and_cold(supervisor, context.soul_id, None).await; +} + +#[tokio::test] +async fn retained_output_accepted_replacement_becomes_source_and_survives_later_failure() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + let replacement = retained_output_replacement(&supervisor, context.soul_id.clone(), true).await; + supervisor + .registry + .mark_recovery_success( + context.soul_id.clone(), + replacement.prepared.incarnation_id.clone(), + replacement.attempt_id, + ) + .await + .unwrap(); + let output = retained_output_read(&supervisor, context.soul_id.clone(), 0) + .await + .unwrap(); + assert_eq!( + output.frames[0].data, + format!("replacement output {}", replacement.prepared.incarnation_id) + ); + let failed = retained_output_replacement(&supervisor, context.soul_id.clone(), true).await; + supervisor + .registry + .mark_stop_outcome(failed.prepared.incarnation_id, StopOutcome::VerifiedEmpty) + .await + .unwrap(); + supervisor + .registry + .mark_recovery_blocked( + context.soul_id.clone(), + Some(failed.attempt_id), + RecoveryBlockReason::WrongNativeIdentity, + vec![], + ) + .await + .unwrap(); + retained_output_assert_warm_and_cold( + supervisor, + context.soul_id, + Some(replacement.prepared.incarnation_id), + ) + .await; +} + +#[tokio::test] +async fn retained_output_legacy_success_with_overwritten_attempt_keeps_accepted_stream() { + let root = tempfile::tempdir().unwrap(); + let (supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + let replacement = retained_output_replacement(&supervisor, context.soul_id.clone(), true).await; + supervisor + .registry + .mark_recovery_success( + context.soul_id.clone(), + replacement.prepared.incarnation_id.clone(), + replacement.attempt_id.clone(), + ) + .await + .unwrap(); + // Old successful rows did not carry an exact accepted-incarnation ref. + let conn = rusqlite::Connection::open(supervisor.registry.database_path()).unwrap(); + conn.execute( + "UPDATE recovery_attempts SET evidence_ref=NULL WHERE attempt_id=?1", + [replacement.attempt_id.as_str()], + ) + .unwrap(); + drop(conn); + let failed = retained_output_replacement(&supervisor, context.soul_id.clone(), false).await; + supervisor + .registry + .mark_stop_outcome(failed.prepared.incarnation_id, StopOutcome::VerifiedEmpty) + .await + .unwrap(); + supervisor + .registry + .mark_recovery_blocked( + context.soul_id.clone(), + Some(failed.attempt_id), + RecoveryBlockReason::ProviderUnavailable, + vec![], + ) + .await + .unwrap(); + retained_output_assert_warm_and_cold( + supervisor, + context.soul_id, + Some(replacement.prepared.incarnation_id), + ) + .await; +} + #[tokio::test] async fn retained_output_blocked_handle_replays_without_input_authority() { let root = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 94aa2492e..51279a51e 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -4765,6 +4765,9 @@ impl TerminalRegistry { let terminal = self .managed_descriptor(terminal_id) .ok_or_else(|| "managed terminal not found".to_string())?; + if !self.is_pty_running(terminal_id) { + return Err("recovery is blocked: retained managed terminal has ended".to_string()); + } let controller = self .managed_controller .read() @@ -5075,7 +5078,7 @@ impl TerminalRegistry { // its exit until the replay drain completes, exactly as the local // PTY natural-exit path does. let mut staged_notify: Vec = Vec::new(); - for (conn_id, subscriber) in state.subscribers.iter_mut() { + for subscriber in state.subscribers.values_mut() { if subscriber.paced_deferred { subscriber.paced_exit_pending = Some(exit_code); if let Some(notify) = subscriber.paced_exit_notify.clone() { diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index 15182ee5e..a2ee472c6 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -3411,9 +3411,12 @@ fn settle_managed_compat_adoption( .create_request_id .as_deref() .is_some_and(|id| id != create.request_id) - || !registry.is_pty_running(&descriptor.terminal_id) + || registry + .managed_descriptor(&descriptor.terminal_id) + .as_ref() + != Some(descriptor) { - return Err("managed adoption identity or liveness changed".to_string()); + return Err("managed adoption identity or membership changed".to_string()); } let locator = descriptor .resume_session_id @@ -3556,6 +3559,7 @@ mod managed_runtime_id_tests { struct LookupOnlyController { descriptor: ManagedTerminalDescriptor, + final_tail: bool, } impl ManagedTerminalController for LookupOnlyController { @@ -3581,7 +3585,13 @@ mod managed_runtime_id_tests { _terminal: ManagedTerminalDescriptor, _data: String, ) -> ManagedTerminalFuture<'a, Result<(), String>> { - Box::pin(async { Ok(()) }) + Box::pin(async move { + assert!( + !self.final_tail, + "ended compatibility facade must not dispatch controller input" + ); + Ok(()) + }) } fn resize<'a>( @@ -3606,7 +3616,29 @@ mod managed_runtime_id_tests { _after_seq: i64, _max_bytes: u64, ) -> ManagedTerminalFuture<'a, Result> { - Box::pin(async { Err("not needed".into()) }) + Box::pin(async move { + if !self.final_tail { + return Err("not needed".into()); + } + Ok(ManagedOutputRead { + lifecycle: None, + completion: Some(freshell_terminal::registry::ManagedOutputCompletion { + stream_epoch: self.descriptor.stream_id.clone(), + incarnation_id: self.descriptor.incarnation_id.clone(), + head_seq: 1, + exit_code: Some(17), + }), + stream_epoch: Some(self.descriptor.stream_id.clone()), + incarnation_id: Some(self.descriptor.incarnation_id.clone()), + reset_required: false, + truncated: false, + retained_from_seq: 1, + head_seq: 1, + exit_code: Some(17), + native_session_id: self.descriptor.resume_session_id.clone(), + chunks: vec![], + }) + }) } } @@ -3669,6 +3701,7 @@ mod managed_runtime_id_tests { }; registry.set_managed_controller(Some(Arc::new(LookupOnlyController { descriptor: descriptor.clone(), + final_tail: false, }))); let adopted = @@ -3717,6 +3750,7 @@ mod managed_runtime_id_tests { .registry .set_managed_controller(Some(Arc::new(LookupOnlyController { descriptor: descriptor.clone(), + final_tail: false, }))); if already_registered { state.registry.register_managed(descriptor); @@ -3832,6 +3866,11 @@ mod managed_runtime_id_tests { ) }; assert!(check(&descriptor).is_ok()); + registry.finish_managed_exit(&descriptor.terminal_id, 17); + assert!( + check(&descriptor).is_ok(), + "exact ended managed membership remains adoptable" + ); descriptor.mode = "claude".into(); assert!(check(&descriptor).is_err()); descriptor.mode = "opencode".into(); @@ -3840,6 +3879,132 @@ mod managed_runtime_id_tests { descriptor.create_request_id = Some("create-match".into()); descriptor.resume_session_id = Some("ses_other".into()); assert!(check(&descriptor).is_err()); + descriptor.resume_session_id = Some("ses_match".into()); + descriptor.incarnation_id = "another-incarnation".into(); + assert!( + check(&descriptor).is_err(), + "stale incarnation cannot adopt retained membership" + ); + } + + #[tokio::test] + async fn managed_legacy_create_reuses_ended_facade_and_replays_final_tail() { + let mut state = super::pane_reconcile_gate_tests::state(); + state.cli_commands = Arc::new(vec![freshell_platform::CliCommandSpec { + name: "opencode".into(), + label: "OpenCode".into(), + default_cmd: "opencode".into(), + ..Default::default() + }]); + let request = "legacy-ended-managed-create"; + let terminal = stable_managed_uuid(request, b"terminal") + .simple() + .to_string(); + let descriptor = ManagedTerminalDescriptor { + soul_id: "retained-ended-soul".into(), + incarnation_id: "retained-ended-incarnation".into(), + terminal_id: terminal.clone(), + stream_id: "retained-ended-stream".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: Some("ses_retained_ended".into()), + create_request_id: Some(request.into()), + }; + state.registry.register_managed(descriptor.clone()); + state + .registry + .ingest_managed_output(&terminal, 1, 1, "final retained tail\n".into()); + assert!(state.registry.finish_managed_exit(&terminal, 17)); + assert!(!state.registry.is_pty_running(&terminal)); + // The lookup controller makes launch fail; a successful create must + // have retained this exact already-ended facade without launching. + state + .registry + .set_managed_controller(Some(Arc::new(LookupOnlyController { + descriptor: descriptor.clone(), + final_tail: true, + }))); + let frames = Arc::new(std::sync::Mutex::new(Vec::::new())); + let collector = Arc::clone(&frames); + let sink: freshell_terminal::FrameSink = + Arc::new(move |frame| collector.lock().unwrap().push(frame)); + let create = serde_json::from_value(serde_json::json!({ "requestId":request, "mode":"opencode", "shell":"system", "restore":true, + "sessionRef":{"provider":"opencode","sessionId":"ses_retained_ended"} })).unwrap(); + assert!( + super::handle_create( + create, + None, + &mut crate::create_gate::CreateOutput::Channel(&sink), + &state, + 17, + true, + &mut crate::create_limit::CreateRateLimiter::new(100, 1000), + &super::ConnectionIdentity::default(), + super::now_ms() + ) + .await + ); + assert!(frames.lock().unwrap().iter().any(|frame| matches!(frame, ServerMessage::TerminalCreated(created) if created.terminal_id == terminal)), + "legacy create must acknowledge the exact retained ended handle"); + assert_eq!( + state.registry.managed_descriptor(&terminal), + Some(descriptor) + ); + // Attach rereads the completed source without launching a replacement, + // preserving the original managed identity and final replay tail. + let attach = super::TerminalAttach { + terminal_id: terminal.clone(), + intent: freshell_protocol::TerminalAttachIntent::ViewportHydrate, + cols: 120, + rows: 30, + attach_request_id: None, + surface_reset: None, + expected_session_ref: None, + max_replay_bytes: None, + replay_page_bytes: None, + priority: None, + since_seq: Some(0), + create_request_id: None, + tab_id: None, + observed_epoch: None, + observed_generation: None, + }; + assert!(matches!( + super::handle_attach(attach, &state, 17, &sink, false, false, None).await, + super::AttachReply::Legacy + )); + let frames = frames.lock().unwrap(); + assert!(frames.iter().any(|frame| matches!(frame, ServerMessage::TerminalOutput(output) if output.terminal_id == terminal && output.data.contains("final retained tail")))); + assert!(frames.iter().any(|frame| matches!(frame, ServerMessage::TerminalExit(exit) if exit.terminal_id == terminal && exit.exit_code == 17))); + drop(frames); + } + + #[tokio::test] + async fn managed_ended_compat_facade_refuses_input_without_controller_dispatch() { + let registry = freshell_terminal::TerminalRegistry::new(); + let descriptor = ManagedTerminalDescriptor { + soul_id: "ended-input-soul".into(), + incarnation_id: "ended-input-incarnation".into(), + terminal_id: "ended-input-terminal".into(), + stream_id: "ended-input-stream".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: Some("ses_ended_input".into()), + create_request_id: Some("ended-input-create".into()), + }; + registry.register_managed(descriptor.clone()); + registry.finish_managed_exit(&descriptor.terminal_id, 17); + registry.set_managed_controller(Some(Arc::new(LookupOnlyController { + descriptor: descriptor.clone(), + final_tail: true, + }))); + assert!( + registry + .managed_input(&descriptor.terminal_id, "must not dispatch".into()) + .await + .is_err(), + "ended facade must refuse input before controller dispatch" + ); } } diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index 4292dcd61..f3c2b1873 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -3925,7 +3925,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const currentContent = contentRef.current if (!tid || !currentContent) return false if (!paneRefreshTargetMatchesContent(request.target, currentContent)) return false - if (isManagedRuntimeRecoveryDecision(currentContent.recoverySummary)) return false + if (isManagedRuntimeRecoveryDecision(currentContent.recoverySummary) && !canReplayManagedTerminal(currentContent)) return false handledRefreshRequestIdRef.current = request.requestId // An explicit pane refresh is user intent, not automatic recovery cycling: diff --git a/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts b/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts index e8b23952e..ee05f778f 100644 --- a/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts +++ b/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts @@ -1008,5 +1008,25 @@ for (const runtimeStatus of ['blocked', 'exited'] as const) { await expect.poll(() => paneContent(page)).toMatchObject({ terminalId: before.terminalId, createRequestId: before.createRequestId, soulId: SOUL_ID, runtimeStatus }) await expect(page.getByTestId('managed-runtime-recovery-card')).toBeVisible() + if (runtimeStatus === 'blocked') { + await harness.clearSentWsMessages() + // Invoke the existing user-visible Refresh command, through its menu. + await page.locator('[data-context="tab"]').first().click({ button: 'right' }) + await page.getByRole('menuitem', { name: 'Refresh tab', exact: true }).click() + await expect.poll(async () => { + const frames = await harness.getSentWsMessages() as Array<{ type?: string; terminalId?: string; intent?: string }> + const detach = frames.findIndex(frame => frame.type === 'terminal.detach' && frame.terminalId === before.terminalId) + const attach = frames.findIndex(frame => frame.type === 'terminal.attach' && frame.terminalId === before.terminalId && frame.intent === 'viewport_hydrate') + return detach >= 0 && attach > detach + }).toBe(true) + const refreshed = await harness.getSentWsMessages() as Array<{ type?: string }> + expect(refreshed.filter(frame => ['terminal.create', 'terminal.input', 'pane.reconcile.request'].includes(frame.type ?? ''))).toEqual([]) + await expect.poll(() => paneContent(page)).toMatchObject({ terminalId: before.terminalId, + createRequestId: before.createRequestId, soulId: SOUL_ID, runtimeStatus }) + await expect.poll(async () => { + const state = await harness.getState() + return state.panes.refreshRequestsByPane?.[state.tabs.activeTabId!] + }).toBeUndefined() + } }) } diff --git a/test/unit/client/components/TerminalView.lifecycle.test.tsx b/test/unit/client/components/TerminalView.lifecycle.test.tsx index 1aaf5ab8c..e8a47fb23 100644 --- a/test/unit/client/components/TerminalView.lifecycle.test.tsx +++ b/test/unit/client/components/TerminalView.lifecycle.test.tsx @@ -6058,6 +6058,7 @@ describe('TerminalView lifecycle updates', () => { mode?: TerminalPaneContent['mode'] recoverySummary?: TerminalPaneContent['recoverySummary'] soulId?: string + runtimeStatus?: TerminalPaneContent['runtimeStatus'] hidden?: boolean clearSends?: boolean requestId?: string @@ -6098,6 +6099,7 @@ describe('TerminalView lifecycle updates', () => { ...(opts?.streamId ? { streamId: opts.streamId } : {}), ...(opts?.recoverySummary ? { recoverySummary: opts.recoverySummary } : {}), ...(opts?.soulId ? { soulId: opts.soulId } : {}), + ...(opts?.runtimeStatus ? { runtimeStatus: opts.runtimeStatus } : {}), ...(opts?.contentServerInstanceId ? { serverInstanceId: opts.contentServerInstanceId } : {}), } @@ -6650,6 +6652,45 @@ describe('TerminalView lifecycle updates', () => { expect(store.getState().panes.refreshRequestsByPane[tabId]).toBeUndefined() }) + it('refreshes the exact retained blocked handle without recreating or sending input', async () => { + const { store, tabId, paneId, terminalId } = await renderTerminalHarness({ + terminalId: 'term-retained-blocked-refresh', soulId: 'retained-soul', runtimeStatus: 'blocked', + streamId: 'retained-stream', fromStore: true, + recoverySummary: { desiredState: 'running', recoveryState: 'blocked', reason: 'wrong_native_identity', + durabilityState: 'resume_captured', allocationState: 'verified_durable' }, + }) + const before = (store.getState().panes.layouts[tabId] as Extract).content + wsMocks.send.mockClear() + act(() => { store.dispatch(requestPaneRefresh({ tabId, paneId })) }) + await waitFor(() => { + const sends = wsMocks.send.mock.calls.map(([msg]) => msg) + const detach = sends.findIndex(msg => msg.type === 'terminal.detach' && msg.terminalId === terminalId) + const attach = sends.findIndex(msg => msg.type === 'terminal.attach' && msg.terminalId === terminalId) + expect(detach).toBeGreaterThanOrEqual(0) + expect(attach).toBeGreaterThan(detach) + expect(sends[attach]).toMatchObject({ terminalId, sinceSeq: 0, intent: 'viewport_hydrate' }) + expect(store.getState().panes.refreshRequestsByPane[tabId]).toBeUndefined() + }) + expect((store.getState().panes.layouts[tabId] as Extract).content).toMatchObject(before) + expect(wsMocks.send.mock.calls.map(([msg]) => msg).filter(msg => + ['terminal.create', 'terminal.input', 'pane.reconcile.request'].includes(msg.type))).toEqual([]) + }) + + it('refuses blocked refresh when retained runtime membership is unknown', async () => { + const { store, tabId, paneId } = await renderTerminalHarness({ + terminalId: 'term-unknown-blocked-refresh', soulId: 'unconfirmed-soul', fromStore: true, + ackInitialAttach: false, waitForMessageHandler: false, waitForTerminalInstance: false, + recoverySummary: { desiredState: 'running', recoveryState: 'blocked', reason: 'unknown_handle', + durabilityState: 'unknown', allocationState: 'allocated' }, + }) + wsMocks.send.mockClear() + act(() => { store.dispatch(requestPaneRefresh({ tabId, paneId })) }) + await act(async () => { await new Promise(resolve => setTimeout(resolve, 0)) }) + expect(wsMocks.send.mock.calls.map(([msg]) => msg).filter(msg => + ['terminal.detach', 'terminal.attach', 'terminal.create', 'terminal.input'].includes(msg.type))).toEqual([]) + expect(store.getState().panes.refreshRequestsByPane[tabId]?.[paneId]).toBeDefined() + }) + it('sends sinceSeq=0 when attaching without previously rendered output', async () => { const { terminalId } = await renderTerminalHarness({ status: 'running', terminalId: 'term-v2-attach' }) reconnectHandler?.() From 8e984bf8709188cc67a3b421642852fac3c9b577 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:33:56 -0700 Subject: [PATCH 236/493] docs: record approved managed recovery output task --- .../2026-10-05-managed-provider-environment.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 4443fe8d1..6400993e9 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -297,16 +297,16 @@ Managed output observations distinguish Running, active Recovering, Blocked with `None` means no lifecycle observation, not Running. Registry stores the current managed lifecycle independently of attachable facade identity. One controller mutex owns each soul's Recovering, Finished(RecoveryResult) or classified Failed slot; retain full typed result, exit/epoch and durable generation. Background and HTTP manual recovery publish through one state transition; successful retry resumes actual output/input. Supervisor retained-output resolver is read-only, exact owned incarnation, never authority for input/replacement/cleanup. -- [ ] **Step 1: Add failing output/status behaviors.** Starting from running and recovering, drive final Lost, Stopped, Blocked and shell `NonResumableTerminalEnded` outcomes; inventory distinguishes blocked and exited, retains terminal ID/native identity/reason/incident and output. Invoke actual `/api/terminals` plain and paged listings plus MCP/CLI `list-terminals` against the managed facade: for the Blocked cases, returned user-facing process status must be blocked, never running, while attachment/replay still succeeds and input is refused; verified ended shell/Lost/Stopped cases report exited after settled completion. Drive a successful retry and assert those same listings return running again. Deliver >64 KiB final tail in multiple pages, recover/stop between pages, reconnect two readers and verify identical retained window with exactly one terminal.exit after final head. Add named `managed_runtime::tests::shell_terminal_ended_facade_retains_output` and terminal `managed_output_shell_terminal_ended_final_tail`: exercise actual ended-shell facade/inventory, drain retained output once to final head, refuse input, preserve identity and show exited status through plain/paged/MCP/CLI listings. Task 5 managed-runtime/managed-output selectors must execute these tests positively. Final output survives supervisor cleanup and web/controller reattachment using retained spool. Drive 20 identical failed polls across two attached sockets; expect one structured transition and no repeated output-refresh warning, then changed failure cause gives one new event and later successful retry gives a recovered event. Success Reattached/Replaced/AlreadyLive restores Running/input and exact identity with existing incarnation reset behavior. -- [ ] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime`; `mise exec rust@1.96 -- cargo test -p freshell-terminal --locked managed_output`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked managed`. Expected cached final errors suppress retained output, leave Running, and repeat warning events. Destructive spool cleanup/stop cases run in the documented sandbox. -- [ ] **Step 3: Apply precise lifecycle/read patch.** Replace separate in-flight/blocked locks with atomic per-soul typed slots and one state-publication function for background/manual recovery. Preserve observed exit code/epoch before scheduling. Settled Lost/Stopped and shell `NonResumableTerminalEnded` reads return successful `Ended` observations; settled Blocked or classified preparation/identity failure returns `Blocked` plus retained output, never a cached Err before output access. Success clears failure and reports Running. Keep one recovery task per soul and existing timeout/backoff; read path never sleeps or starts/stops synchronously. +- [x] **Step 1: Add failing output/status behaviors.** Starting from running and recovering, drive final Lost, Stopped, Blocked and shell `NonResumableTerminalEnded` outcomes; inventory distinguishes blocked and exited, retains terminal ID/native identity/reason/incident and output. Invoke actual `/api/terminals` plain and paged listings plus MCP/CLI `list-terminals` against the managed facade: for the Blocked cases, returned user-facing process status must be blocked, never running, while attachment/replay still succeeds and input is refused; verified ended shell/Lost/Stopped cases report exited after settled completion. Drive a successful retry and assert those same listings return running again. Deliver >64 KiB final tail in multiple pages, recover/stop between pages, reconnect two readers and verify identical retained window with exactly one terminal.exit after final head. Add named `managed_runtime::tests::shell_terminal_ended_facade_retains_output` and terminal `managed_output_shell_terminal_ended_final_tail`: exercise actual ended-shell facade/inventory, drain retained output once to final head, refuse input, preserve identity and show exited status through plain/paged/MCP/CLI listings. Task 5 managed-runtime/managed-output selectors must execute these tests positively. Final output survives supervisor cleanup and web/controller reattachment using retained spool. Drive 20 identical failed polls across two attached sockets; expect one structured transition and no repeated output-refresh warning, then changed failure cause gives one new event and later successful retry gives a recovered event. Success Reattached/Replaced/AlreadyLive restores Running/input and exact identity with existing incarnation reset behavior. +- [x] **Step 2: Red.** Run `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime`; `mise exec rust@1.96 -- cargo test -p freshell-terminal --locked managed_output`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked managed`. Expected cached final errors suppress retained output, leave Running, and repeat warning events. Destructive spool cleanup/stop cases run in the documented sandbox. +- [x] **Step 3: Apply precise lifecycle/read patch.** Replace separate in-flight/blocked locks with atomic per-soul typed slots and one state-publication function for background/manual recovery. Preserve observed exit code/epoch before scheduling. Settled Lost/Stopped and shell `NonResumableTerminalEnded` reads return successful `Ended` observations; settled Blocked or classified preparation/identity failure returns `Blocked` plus retained output, never a cached Err before output access. Success clears failure and reports Running. Keep one recovery task per soul and existing timeout/backoff; read path never sleeps or starts/stops synchronously. Extend supervisor terminal-output resolution from active-only handle to exact retained owned handle when state is Recovering/Blocked/Lost/Stopped or verified shell terminal-ended. Persist actual output epoch (including host boot identity), terminal ID and incarnation before the output writer starts, preserving that association across host restart. Distinguish prior committed and failed replacement streams; source prefix is not actual epoch. Reuse existing two 32 MiB spool segments and bounded torn/truncated-frame handling. Query host if still accessible; otherwise read existing current/previous incarnation spool files, validating ownership/path/epoch/incarnation and parsing within two existing rotation limits. Preserve 64 KiB transport pages, UTF-8 framing, retained start/head, reset/truncation flags and real errors. Persist coherent completion keyed by terminal/incarnation/actual epoch after writer settlement. Normal EOF flush and existing successful host Stop reader join settle output; after abrupt removal, exact backend VerifiedEmpty settles the writer before spool scanning/freezing. Include retained bounds and known exit code or explicit unavailable status. Never pair stale sampled head with later exited flag. Lost with CleanupTerminationUnconfirmed or BackendUnavailable is a recovery outcome with unresolved writer completion, never fabricated stopped/empty output. Hydrate controller outcome/lifecycle from authoritative supervisor inventory when reconnecting after a web/controller restart, rather than relying on the in-memory finished map. Do not replace missing/unreadable spool with empty success. WS ordering is: ingest complete page into replay; associate native identity as applicable; publish runtimeStatus/recovery lifecycle; call `finish_managed_exit` last only when delivered sequence reaches final head. Final outcome is visible while tail drains; paced replay defers terminal.exit as today. Never clear replay/terminal identity for blocked/final outcomes, and permit refresh/manual retry on blocked facades while refusing provider input. Lost/Stopped/shell terminal-ended use known exit code or explicit unavailable completion semantics; unknown cleanup does not complete a stream. Managed terminal.exit retains client handle, stream and restore keys; ordinary exits retain existing behavior. Emit `managed_runtime.recovery_transition` only on changed state/classified cause with terminal/soul/incarnation/attempt/incident/from/to/outcome/cause fields. A controller/registry-shared transport-error fingerprint suppresses repeated unchanged output warnings across sockets and emits one recovered/change event. Do not log full probe/raw credential-bearing error strings. Facade inventory exposes additive RuntimeStatus::Blocked and existing recovery summary; final Exited follows current lifecycle handling. Server `terminals.rs::directory_items` must project managed process state from runtime status/recovery outcome before its internal facade `TerminalRunStatus` fallback. A blocked facade's user-facing `status` is blocked in plain and paged directory output, so unchanged MCP/CLI list pass-throughs also report blocked. Extend the actual directory item schema/types to permit that status; preserve ordinary terminal statuses and internal attachment/lifetime checks. If consumer rendering transforms statuses, audit and correct that transformation too. -- [ ] **Step 4: Green.** Repeat focused tests; expected full final tail, accurate inventory/status, one exit and bounded transitions; successful retry behavior remains green. -- [ ] **Step 5: Refactor.** Consolidate final/manual/background result publication, keep shared spool descriptor/frame codec in runtime protocol and local readers in their owning crates, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. -- [ ] **Step 6: Impact verification.** Run the three focused Cargo commands from Step 2 plus `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked retained_output"` after new tests use that filter, and `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected all run/pass. Include existing `managed_runtime_api::stop_contract_tests` with `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime_api::stop_contract_tests` and record a positive executed test count; a zero-test filter fails the explicit-stop verification. -- [ ] **Step 7: Commit.** Stage listed lifecycle/output files and `git commit -m "fix: retain managed terminal output and publish final recovery outcomes"`. +- [x] **Step 4: Green.** Repeat focused tests; expected full final tail, accurate inventory/status, one exit and bounded transitions; successful retry behavior remains green. +- [x] **Step 5: Refactor.** Consolidate final/manual/background result publication, keep shared spool descriptor/frame codec in runtime protocol and local readers in their owning crates, and keep terminal transport independent of supervisor/Tokio types. Audit sweeps/discovery/attach/claims/clear-dead consumers so blocked identity remains owned and replayable. +- [x] **Step 6: Impact verification.** Run the three focused Cargo commands from Step 2 plus `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked retained_output"` after new tests use that filter, and `scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-supervisor --locked recovery"`. Expected all run/pass. Include existing `managed_runtime_api::stop_contract_tests` with `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_runtime_api::stop_contract_tests` and record a positive executed test count; a zero-test filter fails the explicit-stop verification. +- [x] **Step 7: Commit.** Stage listed lifecycle/output files and `git commit -m "fix: retain managed terminal output and publish final recovery outcomes"`. ### Task 6: Browser status/reattachment and complete configured-cloud managed environment proof From 02a0782ed497f0e8c43691e7b1d7bb3c22e04bc8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 04:20:52 -0700 Subject: [PATCH 237/493] test: share provider parity helpers across test suites --- .../{integration/server => helpers}/provider-parity-broker.ts | 2 +- .../server => helpers}/provider-parity-environment.ts | 0 test/integration/server/managed-provider-parity.test.ts | 4 ++-- test/unit/tooling/testing/runtime-sandbox.test.ts | 2 +- test/unit/tooling/testing/runtime-test-broker.test.ts | 4 ++-- 5 files changed, 6 insertions(+), 6 deletions(-) rename test/{integration/server => helpers}/provider-parity-broker.ts (98%) rename test/{integration/server => helpers}/provider-parity-environment.ts (100%) diff --git a/test/integration/server/provider-parity-broker.ts b/test/helpers/provider-parity-broker.ts similarity index 98% rename from test/integration/server/provider-parity-broker.ts rename to test/helpers/provider-parity-broker.ts index d75903256..53f12419f 100644 --- a/test/integration/server/provider-parity-broker.ts +++ b/test/helpers/provider-parity-broker.ts @@ -2,7 +2,7 @@ import { createHash } from 'node:crypto' import { execFileSync } from 'node:child_process' import fs from 'node:fs' -import type { InitialProviderVolumeAuthorization, RestrictedDockerBroker } from '../../../scripts/testing/runtime-test-broker.js' +import type { InitialProviderVolumeAuthorization, RestrictedDockerBroker } from '../../scripts/testing/runtime-test-broker.js' export function authorizeProviderParityRequest( broker: RestrictedDockerBroker, diff --git a/test/integration/server/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts similarity index 100% rename from test/integration/server/provider-parity-environment.ts rename to test/helpers/provider-parity-environment.ts diff --git a/test/integration/server/managed-provider-parity.test.ts b/test/integration/server/managed-provider-parity.test.ts index f8556fdce..1a8ab3c13 100644 --- a/test/integration/server/managed-provider-parity.test.ts +++ b/test/integration/server/managed-provider-parity.test.ts @@ -10,8 +10,8 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest' import { WS_PROTOCOL_VERSION } from '../../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig } from '../../e2e-browser/helpers/managed-runtime.js' import { exposeGrantsToHarness, fakeOnecliGrants, withMissingGrant } from './provider-parity-onecli.js' -import { makeProviderParityResourcesReadable, prepareProviderParityUserRoots, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from './provider-parity-environment.js' -import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from './provider-parity-broker.js' +import { makeProviderParityResourcesReadable, prepareProviderParityUserRoots, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from '../../helpers/provider-parity-environment.js' +import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../helpers/provider-parity-broker.js' import type { InitialProviderVolumeAuthorization } from '../../../scripts/testing/runtime-test-broker.js' type Provider = 'claude' | 'codex' | 'opencode' | 'amplifier' diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 257a55c9c..ac5272b07 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -28,7 +28,7 @@ vi.mock('node:child_process', () => { import { RuntimeHarness } from '../../../../scripts/testing/runtime-sandbox.js' import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' -import { prepareProviderParityUserRoots } from '../../../integration/server/provider-parity-environment.js' +import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' function useOwnedHarnessArtifacts(harness: RuntimeHarness, host: string): void { const artifact = harness as unknown as { ensureRuntimeImage(): void; buildBinaries(): void; collectCapabilities(): unknown; collectBuildInfo(): unknown } diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 684b33cda..96cb4069f 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -8,8 +8,8 @@ import { pathToFileURL } from 'node:url' import { describe, expect, it, vi } from 'vitest' import { RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' -import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../../integration/server/provider-parity-broker.js' -import { makeProviderParityResourcesReadable, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from '../../../integration/server/provider-parity-environment.js' +import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../../helpers/provider-parity-broker.js' +import { makeProviderParityResourcesReadable, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from '../../../helpers/provider-parity-environment.js' it('executes file and package TUI entrypoints while observing the selected project settings', async () => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-parity-selection-')) From e08fd4a69c00490f03f2f799968a223fcba27d79 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 04:20:58 -0700 Subject: [PATCH 238/493] test: cover empty background refresh and retained attachment --- .../components/component-edge-cases.test.tsx | 54 +++++++++++++++---- 1 file changed, 44 insertions(+), 10 deletions(-) diff --git a/test/unit/client/components/component-edge-cases.test.tsx b/test/unit/client/components/component-edge-cases.test.tsx index 223c7c9c5..c409df748 100644 --- a/test/unit/client/components/component-edge-cases.test.tsx +++ b/test/unit/client/components/component-edge-cases.test.tsx @@ -73,6 +73,7 @@ vi.mock('@/lib/api', () => ({ hasMore: false, }), searchSessions: vi.fn().mockResolvedValue({ results: [] }), + getTerminalDirectoryPage: vi.fn().mockResolvedValue({ items: [], nextCursor: null, revision: 1 }), })) // Mock lucide-react icons @@ -166,10 +167,11 @@ import { } from '@shared/settings' // Import the mocked api to get access to the mocks -import { api, fetchSidebarSessionsSnapshot, searchSessions } from '@/lib/api' +import { api, fetchSidebarSessionsSnapshot, getTerminalDirectoryPage, searchSessions } from '@/lib/api' const mockApiTyped = vi.mocked(api) const mockFetchSidebarSessionsSnapshot = vi.mocked(fetchSidebarSessionsSnapshot) const mockSearchSessions = vi.mocked(searchSessions) +const mockGetTerminalDirectoryPage = vi.mocked(getTerminalDirectoryPage) // ============================================================================ // Test Utilities @@ -234,10 +236,6 @@ function createTestStore(state: TestStoreState = {}) { expandedProjects: new Set(), ...state.sessions, }, - panes: { - layouts: {}, - activePane: {}, - }, terminalDirectory: { windows: {}, searches: {}, @@ -294,6 +292,7 @@ describe('Component Edge Cases', () => { hasMore: false, }) mockSearchSessions.mockResolvedValue({ results: [] }) + mockGetTerminalDirectoryPage.mockResolvedValue({ items: [], nextCursor: null, revision: 1 }) }) afterEach(() => { @@ -668,13 +667,48 @@ describe('Component Edge Cases', () => { }) describe('BackgroundSessions', () => { - it('shows empty state message when no detached terminals', () => { - const store = createTestStore() + it.each(['blocked', 'exited'] as const)( + 'refreshes an empty directory and attaches a retained %s detached terminal', async (runtimeStatus) => { + const store = createTestStore() + await act(async () => { + renderWithStore(, store) + }) - renderWithStore(, store) + expect(screen.getByRole('button', { name: 'Refresh' })).toBeEnabled() + expect(screen.queryByRole('button', { name: 'Attach' })).not.toBeInTheDocument() + expect(screen.queryByRole('button', { name: 'Kill' })).not.toBeInTheDocument() + + mockGetTerminalDirectoryPage.mockResolvedValueOnce({ + items: [{ + terminalId: 'retained-terminal', + soulId: 'retained-soul', + title: 'Retained conversation', + mode: 'codex', + createdAt: Date.now(), + lastActivityAt: Date.now(), + status: 'exited', + runtimeStatus, + hasClients: false, + }], + nextCursor: null, + revision: 2, + }) + await act(async () => { + fireEvent.click(screen.getByRole('button', { name: 'Refresh' })) + }) - expect(screen.getByText(/No detached running terminals/)).toBeInTheDocument() - }) + expect(screen.getByRole('button', { name: 'Attach' })).toBeEnabled() + expect(screen.getByRole('button', { name: 'Kill' })).toBeEnabled() + fireEvent.click(screen.getByRole('button', { name: 'Attach' })) + const [tab] = store.getState().tabs.tabs + expect(tab).toMatchObject({ soulId: 'retained-soul', mode: 'codex', status: 'exited' }) + expect(store.getState().tabs.activeTabId).toBe(tab.id) + expect(store.getState().panes.layouts[tab.id]).toMatchObject({ + type: 'leaf', + content: { terminalId: 'retained-terminal', soulId: 'retained-soul', status: 'exited', runtimeStatus }, + }) + }, + ) }) }) From 38bf75d20bb7c207efb4d77c6a72c47f310eaded Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 05:56:53 -0700 Subject: [PATCH 239/493] test: cover retained managed terminal status and event ordering --- .../managed-recovery-contextual-ui.spec.ts | 48 ++++++++++++++- .../components/BackgroundSessions.test.tsx | 61 +++++++++++++++---- .../client/components/OverviewView.test.tsx | 50 ++++++++++++--- .../TerminalView.exitBanner.test.tsx | 52 ++++++++++++++++ .../unit/lib/managed-runtime-recovery.test.ts | 47 +++++++++++++- 5 files changed, 234 insertions(+), 24 deletions(-) diff --git a/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts b/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts index ee05f778f..bade88d9a 100644 --- a/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts +++ b/test/e2e-browser/specs/managed-recovery-contextual-ui.spec.ts @@ -310,7 +310,7 @@ test('fresh-agent: cold lost pane without a soul retains saved history and a clo expect(response.request().method()).toBe('GET') expect(response.status()).toBe(200) const snapshot = await response.json() - await expect(page.getByText('Sanitized prompt', { exact: true })).toBeVisible() + await expect(page.getByTestId('terminal-work-area').getByText('Sanitized prompt', { exact: true })).toBeVisible() await expect(page.getByText(SAVED_HISTORY_TEXT, { exact: true })).toBeVisible() await expect(page.getByText(arrayAnswer, { exact: false })).toBeVisible() expect(JSON.stringify(snapshot.turns)).toContain('Latest saved array tool output') @@ -988,6 +988,8 @@ test('routine notices stay silent while cleanup failure is a yellow actionable p for (const runtimeStatus of ['blocked', 'exited'] as const) { test(`retained managed ${runtimeStatus} terminal attaches for replay without recreating its provider`, async ({ freshellPage, page, terminal, harness }) => { await terminal.waitForTerminal() + await terminal.executeCommandInserted("printf '%s%s\\n' 'retained-' 'contextual-output'") + await terminal.waitForOutput('retained-contextual-output') await installPane(page, 'terminal', runtimeStatus === 'blocked' ? 'blocked' : 'lost') const before = await paneContent(page) if (before.kind !== 'terminal' || !before.terminalId) throw new Error('Expected the existing terminal handle') @@ -999,7 +1001,7 @@ for (const runtimeStatus of ['blocked', 'exited'] as const) { const root = state.panes.layouts[tabId] if (root?.type !== 'leaf') throw new Error('Expected one pane') harness.dispatch({ type: 'panes/updatePaneContent', payload: { tabId, paneId: root.id, - content: { ...root.content, runtimeStatus, status: runtimeStatus === 'exited' ? 'exited' : 'running' } } }) + content: { ...root.content, runtimeStatus, status: runtimeStatus === 'exited' ? 'exited' : 'error' } } }) }, runtimeStatus) await expect.poll(async () => (await harness.getSentWsMessages() as Array<{ type?: string; terminalId?: string }>). some((frame) => frame.type === 'terminal.attach' && frame.terminalId === before.terminalId)).toBe(true) @@ -1008,6 +1010,7 @@ for (const runtimeStatus of ['blocked', 'exited'] as const) { await expect.poll(() => paneContent(page)).toMatchObject({ terminalId: before.terminalId, createRequestId: before.createRequestId, soulId: SOUL_ID, runtimeStatus }) await expect(page.getByTestId('managed-runtime-recovery-card')).toBeVisible() + await terminal.waitForOutput('retained-contextual-output') if (runtimeStatus === 'blocked') { await harness.clearSentWsMessages() // Invoke the existing user-visible Refresh command, through its menu. @@ -1030,3 +1033,44 @@ for (const runtimeStatus of ['blocked', 'exited'] as const) { } }) } + +for (const recoveryState of ['blocked', 'lost', 'stopped'] as const) { + test(`Overview retains ${recoveryState} diagnostics and opens the same managed replay`, async ({ freshellPage, page, terminal, harness }) => { + await terminal.waitForTerminal() + await terminal.executeCommandInserted("printf '%s%s\\n' 'overview-' 'retained-output'") + await terminal.waitForOutput('overview-retained-output') + await installPane(page, 'terminal', recoveryState) + const before = await paneContent(page) + if (before.kind !== 'terminal' || !before.terminalId) throw new Error('Expected the existing terminal') + const runtimeStatus = recoveryState === 'blocked' ? 'blocked' : 'exited' + const row = { terminalId: before.terminalId, title: 'Retained contextual agent', mode: 'codex', + status: runtimeStatus, runtimeStatus, soulId: SOUL_ID, incarnationId: 'contextual-incarnation', + sessionRef: before.sessionRef, recoverySummary: recoverySummary(recoveryState), + hasClients: false, createdAt: 1, lastActivityAt: 2 } + await page.route(/\/api\/terminals(?:\?.*)?$/, route => route.fulfill({ json: [row] })) + await page.evaluate(({ runtimeStatus, recoveryState }) => { + const harness = window.__FRESHELL_TEST_HARNESS__! + const state = harness.getState() + const tabId = state.tabs.activeTabId! + const root = state.panes.layouts[tabId] + if (root.type !== 'leaf') throw new Error('Expected one pane') + harness.dispatch({ type: 'panes/updatePaneContent', payload: { tabId, paneId: root.id, + content: { ...root.content, runtimeStatus, status: recoveryState === 'stopped' ? 'exited' : 'error' } } }) + }, { runtimeStatus, recoveryState }) + await harness.clearSentWsMessages() + await page.getByTitle('Panes (Ctrl+B O)').click() + const card = page.locator(`[data-context="overview-terminal"][data-terminal-id="${before.terminalId}"]`) + await expect(card).toBeVisible() + await expect(card).toContainText(recoveryState === 'blocked' ? 'Blocked' : recoveryState === 'lost' ? 'Lost' : 'Stopped') + await expect(card).toContainText('provider_unavailable') + await expect(page.getByText(`0 running, ${runtimeStatus === 'exited' ? 1 : 0} exited`, { exact: true })).toBeVisible() + await card.click() + await terminal.waitForOutput('overview-retained-output') + expect(await paneContent(page)).toMatchObject({ terminalId: before.terminalId, soulId: SOUL_ID, + createRequestId: before.createRequestId, sessionRef: before.sessionRef, + status: recoveryState === 'stopped' ? 'exited' : 'error', + recoverySummary: { recoveryState, reason: 'provider_unavailable' } }) + const frames = await harness.getSentWsMessages() as Array<{ type?: string }> + expect(frames.filter(frame => ['terminal.create', 'terminal.input'].includes(frame.type ?? ''))).toEqual([]) + }) +} diff --git a/test/unit/client/components/BackgroundSessions.test.tsx b/test/unit/client/components/BackgroundSessions.test.tsx index de028312e..310aac364 100644 --- a/test/unit/client/components/BackgroundSessions.test.tsx +++ b/test/unit/client/components/BackgroundSessions.test.tsx @@ -49,6 +49,29 @@ function makeStore() { }) } +const retainedCases = [ + { recoveryState: 'blocked', runtimeStatus: 'blocked', paneStatus: 'error', label: 'Blocked' }, + { recoveryState: 'lost', runtimeStatus: 'exited', paneStatus: 'error', label: 'Lost' }, + { recoveryState: 'stopped', runtimeStatus: 'exited', paneStatus: 'exited', label: 'Stopped' }, +] as const + +function retainedDirectoryPage({ recoveryState, runtimeStatus }: typeof retainedCases[number]) { + return { + items: [{ + terminalId: 'retained-managed', title: 'Retained managed', mode: 'codex', + status: runtimeStatus, runtimeStatus, soulId: 'soul-retained', hasClients: false, + createdAt: 1, lastActivityAt: 2, + recoverySummary: { + desiredState: 'running', recoveryState, reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained', + }, + sessionRef: { provider: 'codex', sessionId: 'native-retained' }, + }], + nextCursor: null, + revision: 1, + } +} + describe('BackgroundSessions', () => { beforeEach(() => { sentMessages.length = 0 @@ -79,21 +102,33 @@ describe('BackgroundSessions', () => { cleanup() }) - it.each(['blocked', 'exited'] as const)('attaches a retained managed %s row with its existing identity', async (status) => { - mockGetTerminalDirectoryPage.mockResolvedValue({items:[{terminalId:'retained-managed',title:'Retained managed', - mode:'codex',status,runtimeStatus:status,soulId:'soul-retained',hasClients:false,createdAt:1,lastActivityAt:2, - sessionRef:{provider:'codex',sessionId:'native-retained'}}],nextCursor:null,revision:1}) - const store=makeStore() - const user=userEvent.setup() + it.each(retainedCases)('shows the retained $recoveryState row reason and final status', async (row) => { + mockGetTerminalDirectoryPage.mockResolvedValue(retainedDirectoryPage(row)) + render() + await screen.findByRole('button', { name: 'Attach' }) + expect(screen.getByText('STORE_UNREADABLE')).toBeVisible() + expect(screen.getByText(row.label)).toBeVisible() + }) + + it.each(retainedCases)('attaches the retained $recoveryState row with its exact status and existing identity', async (row) => { + mockGetTerminalDirectoryPage.mockResolvedValue(retainedDirectoryPage(row)) + const store = makeStore() + const user = userEvent.setup() render() - await user.click(await screen.findByRole('button',{name:'Attach'})) - const [tab]=store.getState().tabs.tabs - const layout=store.getState().panes.layouts[tab.id] + const attach = await screen.findByRole('button', { name: 'Attach' }) + await user.click(attach) + const [tab] = store.getState().tabs.tabs + const layout = store.getState().panes.layouts[tab.id] expect(layout.type).toBe('leaf') - if(layout.type!=='leaf') throw new Error('expected leaf') - expect(layout.content).toMatchObject({terminalId:'retained-managed',mode:'codex',soulId:'soul-retained',runtimeStatus:status, - sessionRef:{provider:'codex',sessionId:'native-retained'}}) - expect(layout.content.status).toBe(status==='exited'?'exited':'running') + if (layout.type !== 'leaf') throw new Error('expected leaf') + expect(layout.content).toMatchObject({ + terminalId: 'retained-managed', mode: 'codex', soulId: 'soul-retained', runtimeStatus: row.runtimeStatus, + sessionRef: { provider: 'codex', sessionId: 'native-retained' }, + }) + expect(layout.content.status).toBe(row.paneStatus) + expect(tab.status).toBe(layout.content.status) + expect(layout.content).toMatchObject({ recoverySummary: { reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained' } }) }) it('attaches with the real terminal mode, not hardcoded shell', async () => { diff --git a/test/unit/client/components/OverviewView.test.tsx b/test/unit/client/components/OverviewView.test.tsx index 21a283ea6..8b02a9f86 100644 --- a/test/unit/client/components/OverviewView.test.tsx +++ b/test/unit/client/components/OverviewView.test.tsx @@ -98,6 +98,24 @@ function renderOverview(store: ReturnType) { ) } +const retainedCases = [ + { recoveryState: 'blocked', runtimeStatus: 'blocked', paneStatus: 'error', label: 'Blocked' }, + { recoveryState: 'lost', runtimeStatus: 'exited', paneStatus: 'error', label: 'Lost' }, + { recoveryState: 'stopped', runtimeStatus: 'exited', paneStatus: 'exited', label: 'Stopped' }, +] as const + +function retainedRow({ recoveryState, runtimeStatus }: typeof retainedCases[number]) { + return scopedRow({ + status: runtimeStatus, + runtimeStatus, + soulId: 'soul-retained', + recoverySummary: { + desiredState: 'running', recoveryState, reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained', + }, + }) +} + describe('OverviewView terminal cards — unified agent names', () => { beforeEach(() => { mockApiGet.mockReset() @@ -109,16 +127,32 @@ describe('OverviewView terminal cards — unified agent names', () => { afterEach(() => cleanup()) - it.each(['blocked', 'exited'] as const)('opens the retained managed %s card without creating a shell replacement', async (status) => { - mockApiGet.mockResolvedValue([scopedRow({status,runtimeStatus:status,soulId:'soul-retained'})]) - const store=buildStore() + it.each(retainedCases)('shows the retained $recoveryState reason and status without counting it as running', async (row) => { + mockApiGet.mockResolvedValue([retainedRow(row)]) + renderOverview(buildStore()) + const card = await screen.findByRole('button', { name: 'Open terminal Row last-known name' }) + expect(within(card).getByText('STORE_UNREADABLE')).toBeVisible() + expect(within(card).getByText(row.label)).toBeVisible() + expect(screen.getByText(`0 running, ${row.runtimeStatus === 'exited' ? 1 : 0} exited`)).toBeVisible() + }) + + it.each(retainedCases)('opens the retained $recoveryState card with its exact status and existing identity', async (row) => { + mockApiGet.mockResolvedValue([retainedRow(row)]) + const store = buildStore() renderOverview(store) - fireEvent.click(await screen.findByRole('button',{name:'Open terminal Row last-known name'})) - const [tab]=store.getState().tabs.tabs - const layout=store.getState().panes.layouts[tab.id] + const card = await screen.findByRole('button', { name: 'Open terminal Row last-known name' }) + fireEvent.click(card) + const [tab] = store.getState().tabs.tabs + const layout = store.getState().panes.layouts[tab.id] expect(layout.type).toBe('leaf') - if(layout.type!=='leaf') throw new Error('expected leaf') - expect(layout.content).toMatchObject({terminalId:'term-1',mode:'claude',soulId:'soul-retained',runtimeStatus:status}) + if (layout.type !== 'leaf') throw new Error('expected leaf') + expect(layout.content).toMatchObject({ + terminalId: 'term-1', mode: 'claude', soulId: 'soul-retained', runtimeStatus: row.runtimeStatus, + }) + expect(layout.content.status).toBe(row.paneStatus) + expect(tab.status).toBe(layout.content.status) + expect(layout.content).toMatchObject({ recoverySummary: { reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained' } }) expect(mockApiPost).not.toHaveBeenCalled() }) diff --git a/test/unit/client/components/TerminalView.exitBanner.test.tsx b/test/unit/client/components/TerminalView.exitBanner.test.tsx index c882a098e..b578bac2e 100644 --- a/test/unit/client/components/TerminalView.exitBanner.test.tsx +++ b/test/unit/client/components/TerminalView.exitBanner.test.tsx @@ -368,6 +368,58 @@ describe('TerminalView exited-pane error banner', () => { expect(paneState(store)).toMatchObject({terminalId:'retained-managed',streamId:'retained-epoch',status:'exited'}) }) + it.each([ + ['blocked', 'blocked', 'error'], ['lost', 'exited', 'error'], ['stopped', 'exited', 'exited'], + ] as const)('projects an authoritative %s terminal status without replacing its retained identity', async (recoveryState, runtimeStatus, status) => { + const { store, paneContent } = makeStore({ mode: 'codex', status: 'running', recoveryState: 'live' }) + act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { + ...paneContent, terminalId: 'retained-managed', streamId: 'retained-epoch', runtimeStatus: 'running', + } }))) + await renderPane(store, paneState(store)) + const summary = { ...paneContent.recoverySummary!, recoveryState, + reason: 'STORE_UNREADABLE', attemptId: 'attempt-retained', incidentId: 'incident-retained' } + act(() => messageHandler!({ type: 'terminal.status', terminalId: 'retained-managed', + status: runtimeStatus, recoverySummary: summary })) + expect(paneState(store)).toMatchObject({ status, runtimeStatus, recoverySummary: summary, + terminalId: 'retained-managed', streamId: 'retained-epoch', createRequestId: REQ, + sessionRef: { provider: 'codex', sessionId: SESSION_ID } }) + expect(store.getState().tabs.tabs[0].status).toBe(status) + }) + + it.each([ + ['blocked', 'error'], ['lost', 'error'], ['stopped', 'exited'], + ] as const)('keeps an authoritative %s final status when an unqualified running frame arrives later', async (recoveryState, status) => { + const { store, paneContent } = makeStore({ mode: 'codex', status, recoveryState }) + act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { + ...paneContent, terminalId: 'retained-managed', streamId: 'retained-epoch', + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', + } }))) + await renderPane(store, paneState(store)) + act(() => messageHandler!({ type: 'terminal.status', terminalId: 'retained-managed', status: 'running' })) + expect(paneState(store)).toMatchObject({ status, recoverySummary: { recoveryState }, + terminalId: 'retained-managed', streamId: 'retained-epoch', createRequestId: REQ }) + expect(store.getState().tabs.tabs[0].status).toBe(status) + }) + + it.each([ + ['blocked', 'error'], ['lost', 'error'], ['stopped', 'exited'], + ] as const)('retains the %s final status when replay completes with unavailable exit evidence', async (recoveryState, status) => { + const { store, paneContent } = makeStore({ mode: 'codex', status, recoveryState, + lifecycle: { lastTerminalId: 'retained-managed', exit: { exitCode: 9, at: Date.now() } } }) + act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { + ...paneContent, terminalId: 'retained-managed', streamId: 'retained-epoch', + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', + } }))) + await renderPane(store, paneState(store)) + act(() => messageHandler!({ type: 'terminal.exit', terminalId: 'retained-managed', + exitCode: -1, managedRetained: true, exitCodeAvailable: false })) + expect(paneState(store)).toMatchObject({ status, recoverySummary: { recoveryState }, + terminalId: 'retained-managed', streamId: 'retained-epoch', createRequestId: REQ }) + expect(store.getState().tabs.tabs[0].status).toBe(status) + expect(selectExitRecordFrom(store.getState().terminalLifecycle, PANE)).toBeUndefined() + expect(screen.queryByText(/code -1/)).toBeNull() + }) + it('shows a codeless alert with Relaunch for an exited agent pane with NO exit record (post-reload)', async () => { const { store, paneContent } = makeStore({ mode: 'claude', diff --git a/test/unit/lib/managed-runtime-recovery.test.ts b/test/unit/lib/managed-runtime-recovery.test.ts index 3c62c2e23..61bb2456f 100644 --- a/test/unit/lib/managed-runtime-recovery.test.ts +++ b/test/unit/lib/managed-runtime-recovery.test.ts @@ -615,7 +615,7 @@ describe('managed runtime recovery merge', () => { }) const plan = buildManagedRuntimeMergePlan(snapshot([blocked]), baseState()) expect(plan.creates[0].content).toMatchObject({ - status: 'recovering', + status: 'error', soulId: 'soul-one', sessionRef: { provider: 'opencode', sessionId: 'ses_one' }, recoverySummary: { @@ -659,10 +659,55 @@ describe('managed runtime recovery merge', () => { kind: 'terminal', soulId: 'soul-one', viewIntentId: 'view-one', + status: 'error', recoverySummary: { desiredState: 'stopped', recoveryState: 'lost' }, }) }) + it.each([ + ['live', 'running', 'running'], + ['recovering', 'stopped', 'recovering'], + ['blocked', 'stopped', 'error'], + ['lost', 'stopped', 'error'], + ['stopped', 'stopped', 'exited'], + ] as const)('projects %s recovery with launch state %s while retaining the same conversation and diagnostics', (recoveryState, launchState, status) => { + const state = baseState() + state.panes.layouts['user-tab'].content = { + kind: 'terminal', createRequestId: 'create-one', terminalId: 'terminal-one', + streamId: 'transport-stream', mode: 'opencode', status: 'running', soulId: 'soul-one', + viewIntentId: 'view-one', sessionRef: { provider: 'opencode', sessionId: 'ses_one' }, + } + const retained = soul({ recoveryState, launchState, + desiredState: recoveryState === 'lost' || recoveryState === 'stopped' ? 'stopped' : 'running', + recoveryReason: 'STORE_UNREADABLE', recoveryAttemptId: 'attempt-retained', incidentId: 'incident-retained' }) + const plan = buildManagedRuntimeMergePlan(snapshot([retained]), state) + expect(plan.creates).toHaveLength(0) + expect(plan.updates).toHaveLength(1) + expect(plan.updates[0].content).toMatchObject({ + terminalId: 'terminal-one', streamId: 'transport-stream', status, + soulId: 'soul-one', incarnationId: 'incarnation-one', + sessionRef: { provider: 'opencode', sessionId: 'ses_one' }, + recoverySummary: { recoveryState, reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained' }, + }) + expect(plan.updates[0].tabUpdates.status).toBe(status) + }) + + it.each([ + ['live', 'running'], ['recovering', 'recovering'], ['blocked', 'blocked'], + ['lost', 'exited'], ['stopped', 'exited'], + ] as const)('projects %s inventory into the authoritative retained transport status %s', (recoveryState, runtimeStatus) => { + const retained = soul({ + recoveryState, launchState: recoveryState === 'live' ? 'running' : 'stopped', + desiredState: recoveryState === 'lost' || recoveryState === 'stopped' ? 'stopped' : 'running', + }) + const plan = buildManagedRuntimeMergePlan(snapshot([retained]), baseState()) + expect(plan.creates[0].content).toMatchObject({ + terminalId: 'terminal-one', soulId: 'soul-one', runtimeStatus, + }) + expect(plan.creates[0].tab).toMatchObject({ soulId: 'soul-one', runtimeStatus }) + }) + it('updates an already represented stopped/lost Fresh Agent in place', () => { const state = baseState() state.tabs.tabs[0].mode = 'freshopencode' From dea51b5327e05fe47addd111fa2d64134779be19 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 05:57:02 -0700 Subject: [PATCH 240/493] test: define exact external cloud fixture selector exception --- .../tooling/testing/coordinator-command-matrix.test.ts | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/test/unit/tooling/testing/coordinator-command-matrix.test.ts b/test/unit/tooling/testing/coordinator-command-matrix.test.ts index 7353ed5fd..9ab07a991 100644 --- a/test/unit/tooling/testing/coordinator-command-matrix.test.ts +++ b/test/unit/tooling/testing/coordinator-command-matrix.test.ts @@ -88,6 +88,10 @@ describe('coordinator command matrix', () => { 'config/vitest/vitest.server.config.ts', 'test/server/ws-protocol.test.ts', './test/server/ws-protocol.test.ts', + 'test/unit/server/other.test.ts', + 'test/unit/server/e2e-cloud-external-target.test.ts.bak', + 'test/unit/server/e2e-cloud-external-target.test.ts/child', + 'config\\vitest\\server.config.ts', ])('rejects retired server selectors (%s) with a Rust-lane hint', (selector) => { const disposition = classifyCommand({ commandKey: 'test:vitest', @@ -103,7 +107,10 @@ describe('coordinator command matrix', () => { 'test/integration/server/managed-provider-parity.test.ts', 'test/integration/server/fresh-agent-parity.test.ts', 'test/integration/server/provider-parity-contract.test.ts', - ])('allows the Rust-server parity Vitest spec by its exact path only (%s)', spec => { + 'test/unit/server/e2e-cloud-external-target.test.ts', + './test/unit/server/e2e-cloud-external-target.test.ts', + 'test\\unit\\server\\e2e-cloud-external-target.test.ts', + ])('allows the retained tooling or Rust fixture spec by its exact normalized path only (%s)', spec => { const args = ['run', spec, '--config', 'config/vitest/vitest.config.ts'] expectPhase(classifyCommand({ commandKey: 'test:vitest', forwardedArgs: args }), { runner: 'vitest', config: 'direct', args, From 2dc1846b5ece2e7daa55de73e4f74c8fc20c598f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 06:26:56 -0700 Subject: [PATCH 241/493] test: use valid retained recovery summaries for attachment --- .../client/components/BackgroundSessions.test.tsx | 12 ++++++++---- test/unit/client/components/OverviewView.test.tsx | 12 ++++++++---- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/test/unit/client/components/BackgroundSessions.test.tsx b/test/unit/client/components/BackgroundSessions.test.tsx index 310aac364..398543b5f 100644 --- a/test/unit/client/components/BackgroundSessions.test.tsx +++ b/test/unit/client/components/BackgroundSessions.test.tsx @@ -9,6 +9,7 @@ import panesReducer from '../../../../src/store/panesSlice' import settingsReducer from '../../../../src/store/settingsSlice' import terminalDirectoryReducer from '../../../../src/store/terminalDirectorySlice' import freshAgentReducer, { applyRuntimeOwner } from '../../../../src/store/freshAgentSlice' +import type { ManagedRuntimeRecoverySummary } from '@shared/managed-runtime' const sentMessages: any[] = [] const mockGetTerminalDirectoryPage = vi.fn() @@ -55,16 +56,19 @@ const retainedCases = [ { recoveryState: 'stopped', runtimeStatus: 'exited', paneStatus: 'exited', label: 'Stopped' }, ] as const +const retainedRecoverySummary: ManagedRuntimeRecoverySummary = { + desiredState: 'running', recoveryState: 'blocked', reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained', + durabilityState: 'resume_captured', allocationState: 'verified_durable', +} + function retainedDirectoryPage({ recoveryState, runtimeStatus }: typeof retainedCases[number]) { return { items: [{ terminalId: 'retained-managed', title: 'Retained managed', mode: 'codex', status: runtimeStatus, runtimeStatus, soulId: 'soul-retained', hasClients: false, createdAt: 1, lastActivityAt: 2, - recoverySummary: { - desiredState: 'running', recoveryState, reason: 'STORE_UNREADABLE', - attemptId: 'attempt-retained', incidentId: 'incident-retained', - }, + recoverySummary: { ...retainedRecoverySummary, recoveryState }, sessionRef: { provider: 'codex', sessionId: 'native-retained' }, }], nextCursor: null, diff --git a/test/unit/client/components/OverviewView.test.tsx b/test/unit/client/components/OverviewView.test.tsx index 8b02a9f86..a96a6b8b3 100644 --- a/test/unit/client/components/OverviewView.test.tsx +++ b/test/unit/client/components/OverviewView.test.tsx @@ -7,6 +7,7 @@ import sessionNamesReducer, { receiveSessionNames } from '@/store/sessionNamesSl import tabsReducer from '@/store/tabsSlice' import panesReducer from '@/store/panesSlice' import type { SessionNameRecord, SessionNameRef, SessionNameUpdate } from '@shared/session-names' +import type { ManagedRuntimeRecoverySummary } from '@shared/managed-runtime' const { mockApiGet, mockApiPatch, mockApiDelete, mockApiPost, wsMessageHandlers } = vi.hoisted(() => ({ mockApiGet: vi.fn(), @@ -104,15 +105,18 @@ const retainedCases = [ { recoveryState: 'stopped', runtimeStatus: 'exited', paneStatus: 'exited', label: 'Stopped' }, ] as const +const retainedRecoverySummary: ManagedRuntimeRecoverySummary = { + desiredState: 'running', recoveryState: 'blocked', reason: 'STORE_UNREADABLE', + attemptId: 'attempt-retained', incidentId: 'incident-retained', + durabilityState: 'resume_captured', allocationState: 'verified_durable', +} + function retainedRow({ recoveryState, runtimeStatus }: typeof retainedCases[number]) { return scopedRow({ status: runtimeStatus, runtimeStatus, soulId: 'soul-retained', - recoverySummary: { - desiredState: 'running', recoveryState, reason: 'STORE_UNREADABLE', - attemptId: 'attempt-retained', incidentId: 'incident-retained', - }, + recoverySummary: { ...retainedRecoverySummary, recoveryState }, }) } From beb0e26988bde6213f70cf7176edc11f35c6094b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 06:32:53 -0700 Subject: [PATCH 242/493] fix: preserve retained managed terminal status in browser views --- src/components/BackgroundSessions.tsx | 11 ++++-- .../ManagedTerminalRecoveryDetails.tsx | 18 +++++++++ src/components/OverviewView.tsx | 14 +++++-- src/components/TerminalView.tsx | 36 +++++++++--------- src/lib/managed-terminal.ts | 20 ++++++++++ src/lib/recovery/managed-runtime-recovery.ts | 37 ++++++++++++------- 6 files changed, 98 insertions(+), 38 deletions(-) create mode 100644 src/components/ManagedTerminalRecoveryDetails.tsx diff --git a/src/components/BackgroundSessions.tsx b/src/components/BackgroundSessions.tsx index 4e7a1abfc..67132a74b 100644 --- a/src/components/BackgroundSessions.tsx +++ b/src/components/BackgroundSessions.tsx @@ -10,7 +10,8 @@ import { addTab } from '@/store/tabsSlice' import { initLayout } from '@/store/panesSlice' import { fetchTerminalDirectoryWindow } from '@/store/terminalDirectoryThunks' import type { BackgroundTerminal } from '@/store/types' -import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal' +import { isRetainedManagedTerminal, managedTerminalPaneStatus, managedTerminalProjection } from '@/lib/managed-terminal.js' +import ManagedTerminalRecoveryDetails from './ManagedTerminalRecoveryDetails.js' const EMPTY_TERMINALS: BackgroundTerminal[] = [] @@ -115,6 +116,7 @@ export default function BackgroundSessions() { >
{t.title}
+
{t.cwd ? t.cwd + ' • ' : ''}idle {formatAge(idleMs)}
@@ -126,10 +128,11 @@ export default function BackgroundSessions() { const tabId = nanoid() const mode = ((t.mode as any) || 'shell') as string const initialCwd = typeof t.cwd === 'string' && t.cwd.trim() ? t.cwd : undefined + const status = managedTerminalPaneStatus(t, t.status === 'exited' ? 'exited' : 'running') dispatch(addTab({ id: tabId, title: t.title, - status: t.status === 'exited' ? 'exited' : 'running', + status, ...managedTerminalProjection(t), mode: mode as any, sessionRef: t.sessionRef, @@ -141,8 +144,8 @@ export default function BackgroundSessions() { kind: 'terminal', mode: mode as any, terminalId: t.terminalId, - status: t.status === 'exited' ? 'exited' : 'running', - ...managedTerminalProjection(t), + status, + ...managedTerminalProjection(t), sessionRef: t.sessionRef, ...(initialCwd ? { initialCwd } : {}), }, diff --git a/src/components/ManagedTerminalRecoveryDetails.tsx b/src/components/ManagedTerminalRecoveryDetails.tsx new file mode 100644 index 000000000..fccd87ca7 --- /dev/null +++ b/src/components/ManagedTerminalRecoveryDetails.tsx @@ -0,0 +1,18 @@ +import type { ManagedRuntimeProjectionFields } from '../../shared/managed-runtime.js' +import { managedTerminalRecoveryLabel } from '../lib/managed-terminal.js' + +export default function ManagedTerminalRecoveryDetails({ terminal }: { terminal: ManagedRuntimeProjectionFields }) { + const label = managedTerminalRecoveryLabel(terminal) + const summary = terminal.recoverySummary + if (!label || !summary) return null + return ( +
+
+ {label} + {summary.reason && {summary.reason}} +
+ {summary.attemptId &&
Attempt {summary.attemptId}
} + {summary.incidentId &&
Incident {summary.incidentId}
} +
+ ) +} diff --git a/src/components/OverviewView.tsx b/src/components/OverviewView.tsx index 19e97210d..05a5768e4 100644 --- a/src/components/OverviewView.tsx +++ b/src/components/OverviewView.tsx @@ -14,7 +14,8 @@ import { collectTerminalIds } from '@/lib/pane-utils' import { cn } from '@/lib/utils' import { RefreshCw, Circle, Play, Pencil, Trash2, Sparkles, ExternalLink } from 'lucide-react' import { ContextIds } from '@/components/context-menu/context-menu-constants' -import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal.js' +import { isRetainedManagedTerminal, managedTerminalPaneStatus, managedTerminalProjection } from '@/lib/managed-terminal.js' +import ManagedTerminalRecoveryDetails from './ManagedTerminalRecoveryDetails.js' import type { BackgroundTerminal } from '@/store/types' type TerminalOverview = BackgroundTerminal & { @@ -156,9 +157,14 @@ export default function OverviewView({ onOpenTab }: { onOpenTab?: () => void }) dispatch(setActiveTab(existing.id)) } else { const tabId = nanoid() - const status = terminal.status === 'exited' ? 'exited' : 'running' + const status = managedTerminalPaneStatus(terminal, terminal.status === 'exited' ? 'exited' : 'running') const mode = isRetainedManagedTerminal(terminal) ? terminal.mode : 'shell' - dispatch(addTab({ id: tabId, title: terminal.title, status, mode })) + dispatch(addTab({ id: tabId, title: terminal.title, status, mode, + ...(isRetainedManagedTerminal(terminal) ? { + initialCwd: terminal.cwd, sessionRef: terminal.sessionRef, + ...managedTerminalProjection(terminal), + } : {}), + })) dispatch(initLayout({ tabId, content: { kind: 'terminal', mode, terminalId: terminal.terminalId, status, ...(isRetainedManagedTerminal(terminal) ? { @@ -499,6 +505,8 @@ function TerminalCard({ )}
+ + {nativeSyncStatus && nativeSyncStatus.status !== 'synced' ? (
Date: Sun, 11 Oct 2026 06:32:54 -0700 Subject: [PATCH 243/493] fix: route the external Cloud tooling fixture by exact path --- scripts/testing/coordinator-command-matrix.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/scripts/testing/coordinator-command-matrix.ts b/scripts/testing/coordinator-command-matrix.ts index c8da4a752..b8c9dbd20 100644 --- a/scripts/testing/coordinator-command-matrix.ts +++ b/scripts/testing/coordinator-command-matrix.ts @@ -109,12 +109,13 @@ export function classifyCommand(input: CoordinatorInput): CommandDisposition { function isRetiredServerConfigSelector(arg: string): boolean { const normalized = arg.replaceAll('\\', '/') - // These integration specs exercise the Rust server through owned - // Docker/WebSocket fixtures, despite living in the retired Node-server tree. + // These specs exercise owned tooling or Rust Docker/WebSocket fixtures, + // despite living in the retired Node-server test trees. if ([ 'test/integration/server/managed-provider-parity.test.ts', 'test/integration/server/fresh-agent-parity.test.ts', 'test/integration/server/provider-parity-contract.test.ts', + 'test/unit/server/e2e-cloud-external-target.test.ts', ].includes(normalized.replace(/^\.\//, ''))) { return false } From aa0792c71ae3884ad765de02894d3b601c7327b8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 06:38:08 -0700 Subject: [PATCH 244/493] test: verify stopped inventory on its existing retained view --- .../unit/lib/managed-runtime-recovery.test.ts | 29 +++++++++++++++---- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/test/unit/lib/managed-runtime-recovery.test.ts b/test/unit/lib/managed-runtime-recovery.test.ts index 61bb2456f..8b5c0d66c 100644 --- a/test/unit/lib/managed-runtime-recovery.test.ts +++ b/test/unit/lib/managed-runtime-recovery.test.ts @@ -701,11 +701,30 @@ describe('managed runtime recovery merge', () => { recoveryState, launchState: recoveryState === 'live' ? 'running' : 'stopped', desiredState: recoveryState === 'lost' || recoveryState === 'stopped' ? 'stopped' : 'running', }) - const plan = buildManagedRuntimeMergePlan(snapshot([retained]), baseState()) - expect(plan.creates[0].content).toMatchObject({ - terminalId: 'terminal-one', soulId: 'soul-one', runtimeStatus, - }) - expect(plan.creates[0].tab).toMatchObject({ soulId: 'soul-one', runtimeStatus }) + const state = baseState() + if (retained.desiredState === 'stopped') { + state.panes.layouts['user-tab'].content = { + kind: 'terminal', createRequestId: 'create-one', terminalId: 'terminal-one', + streamId: 'transport-stream', mode: 'opencode', status: 'running', soulId: 'soul-one', + viewIntentId: 'view-one', sessionRef: { provider: 'opencode', sessionId: 'ses_one' }, + } + } + const plan = buildManagedRuntimeMergePlan(snapshot([retained]), state) + if (retained.desiredState === 'stopped') { + expect(plan.creates).toHaveLength(0) + expect(plan.updates).toHaveLength(1) + expect(plan.updates[0].content).toMatchObject({ + terminalId: 'terminal-one', streamId: 'transport-stream', soulId: 'soul-one', runtimeStatus, + sessionRef: { provider: 'opencode', sessionId: 'ses_one' }, + }) + expect(plan.updates[0].tabUpdates).toMatchObject({ soulId: 'soul-one', runtimeStatus }) + } else { + expect(plan.creates).toHaveLength(1) + expect(plan.creates[0].content).toMatchObject({ + terminalId: 'terminal-one', soulId: 'soul-one', runtimeStatus, + }) + expect(plan.creates[0].tab).toMatchObject({ soulId: 'soul-one', runtimeStatus }) + } }) it('updates an already represented stopped/lost Fresh Agent in place', () => { From a59eedddf831c262688ee8bc9759bbf6efc1c265 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:15:02 -0700 Subject: [PATCH 245/493] test: qualify replacement input publication against retained owners --- .../testing/runtime-test-broker.test.ts | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 96cb4069f..febe71f2f 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -966,6 +966,60 @@ async function publishControllerPreparation(directory: string, input: Pick{ + it.each(['exited', 'running', 'foreign-owner', 'changed-image', 'changed-host', 'foreign-capability'] as const)( + 'qualifies a replacement publication against its retained owner (%s)', async (state) => { + await withHttpBroker(async ({ directory, request, forwarded, broker: instance, setRuntimeRunning }) => { + const artifact = Buffer.from('replacement owned host artifact') + const owner = { ...sourceAuthorization(), hostBinaryPath: path.join(directory, 'freshell-session-host'), + binarySha256: createHash('sha256').update(artifact).digest('hex') } + await fs.writeFile(owner.hostBinaryPath, artifact) + instance.authorizeInitialProviderVolume(owner) + const priorRuntime = path.join(directory, 'r/scenario/incarnation-one') + const actor = path.join(path.dirname(priorRuntime), 'souls', + createHash('sha256').update(`${owner.installationId}\0${owner.soulId}`).digest('hex'), 'actor') + const prior = JSON.parse(createBody([ + `${owner.hostBinaryPath}:/runtime/freshell-session-host:ro`, + `${priorRuntime}:/run/freshell:rw`, `${owner.providerVolumeName}:/home/freshell/provider:rw`, + '/workspace:/workspace:rw', `${actor}:/run/freshell-host-actor:rw`, + ]).toString()) + prior.Labels['com.freshell.soul-id'] = owner.soulId + expect((await request('POST', '/v1.47/containers/create?name=retained-runtime', prior)).status).toBe(201) + expect(instance.receipts()).toMatchObject([{ containerId: ownedId, soulId: owner.soulId, + incarnationId: 'incarnation-one', providerVolumeName: owner.providerVolumeName }]) + + const generationId = 'grant-55555555-5555-4555-8555-555555555555' + const input = { ...publisherAuthorization(), soulId: owner.soulId, incarnationId: 'incarnation-two', + generationId, hostBinaryPath: owner.hostBinaryPath, providerVolumeName: owner.providerVolumeName, + stageRoot: path.join(directory, 'scenarios/scenario/control/provider-roots', generationId), + configurationBinds: [], resourceMounts: [], indexedSecretSources: [], + indexedFiles: { volumeName: `freshell-inputs-${createHash('sha256').update(`${owner.soulId}\0${generationId}`).digest('hex').slice(0, 24)}`, aliases: [] } } + const publication = await publishControllerPreparation(directory, input, '', 'codex') + // Replacement producer publications have no initial reservation; their + // authority is the exact quiescent retained runtime, not the old tuple. + const context = publication.context as Record + delete context.preparedInputs.initialAuthReservation + await fs.writeFile(path.join(publication.metadata, 'prepared-context.json'), JSON.stringify(context)) + await fs.writeFile(path.join(publication.metadata, 'prepared-result.json'), JSON.stringify({ + providerLaunchContext: context, providerSecretReferences: [], + })) + const capabilityPath = path.join(publication.capabilities, `${generationId}.json`) + const capability = JSON.parse(await fs.readFile(capabilityPath, 'utf8')) + capability.incarnationId = state === 'foreign-capability' ? 'foreign-incarnation' : input.incarnationId + await fs.writeFile(capabilityPath, JSON.stringify(capability)) + setRuntimeRunning(state === 'running') + const upstreamPrior = forwarded.find(operation => operation.url.includes('name=retained-runtime'))!.body + if (state === 'foreign-owner') upstreamPrior.Labels['com.freshell.soul-id'] = 'foreign' + if (state === 'changed-image') upstreamPrior.Image = 'sha256:other-approved-image' + if (state === 'changed-host') await fs.writeFile(owner.hostBinaryPath, 'changed artifact') + const result = await request('POST', '/v1.47/volumes/create', { + Name: input.indexedFiles.volumeName, Driver: 'local', DriverOpts: indexedOptions, Labels: publisherLabels(input), + }) + expect(result.status, JSON.stringify(instance.eventsSnapshot().slice(-3))).toBe(state === 'exited' ? 201 : 403) + const published = forwarded.filter(operation => operation.url === '/v1.47/volumes/create') + expect(published).toHaveLength(state === 'exited' ? 1 : 0) + }, undefined, true) + }, + ) it('probes actual candidate absence before creation only with a qualified owned publication', async () => { await withHttpBroker(async ({ directory, request, forwarded, broker: instance, replaceCandidateVolumeInspection, clearCandidateVolumeInspection }) => { const input = candidateAuthorization() From 5e141f0923e4c8617f45fb0f5443c89a936917c5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:15:03 -0700 Subject: [PATCH 246/493] test: exercise closed Cloud external target handoff --- .../server/e2e-cloud-external-target.test.ts | 102 ++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 test/unit/server/e2e-cloud-external-target.test.ts diff --git a/test/unit/server/e2e-cloud-external-target.test.ts b/test/unit/server/e2e-cloud-external-target.test.ts new file mode 100644 index 000000000..5aa6cd77e --- /dev/null +++ b/test/unit/server/e2e-cloud-external-target.test.ts @@ -0,0 +1,102 @@ +import { spawnSync } from 'node:child_process' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { parse } from 'yaml' +import { describe, expect, it } from 'vitest' + +const root = path.resolve(import.meta.dirname, '../../..') +const targetUrl = 'https://synthetic-owned.trycloudflare.com' +const targetToken = 'synthetic token: quote " newline\nvalue' +const managedFixture = { candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', + remoteWorkspacePath: '/tmp/synthetic/workspace', expectedNonce: 'synthetic-environment-nonce' } +const checkpointFixture = { candidateSha: 'a'.repeat(40), runId: managedFixture.runId, + syntheticWorkspacePath: '/tmp/synthetic/workspace', syntheticHomePath: '/tmp/synthetic/home', + nonGitWorkspacePath: '/tmp/synthetic/non-git', restrictedUid: 65534, expectedNonce: 'synthetic-checkpoint-nonce' } + +async function dispatch(overrides: Record) { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-external-dispatch-')) + const callsFile = path.join(directory, 'calls.jsonl') + const receivedFile = path.join(directory, 'received.yaml') + const gcloud = path.join(directory, 'gcloud') + await fs.writeFile(gcloud, `#!${process.execPath} +const fs = require('node:fs'); +const args = process.argv.slice(2); +fs.appendFileSync(process.env.FRESHELL_TEST_CALLS, JSON.stringify(args) + '\\n'); +if (args.slice(0, 3).join(' ') === 'run jobs create') { + const file = args.find(arg => arg.startsWith('--env-vars-file='))?.slice('--env-vars-file='.length); + if (!file) process.exit(91); + fs.copyFileSync(file, process.env.FRESHELL_TEST_RECEIVED); + process.exit(19); // Stop before execution; every GCP command is this fake. +} +process.exit(0); +`) + await fs.chmod(gcloud, 0o755) + const env = { ...process.env, PATH: `${directory}${path.delimiter}${process.env.PATH ?? ''}`, + FRESHELL_E2E_BACKEND: 'cloud', FRESHELL_TEST_CALLS: callsFile, FRESHELL_TEST_RECEIVED: receivedFile, + FRESHELL_E2E_TARGET_URL: targetUrl, FRESHELL_E2E_TARGET_TOKEN: targetToken, + FRESHELL_E2E_TARGET_WS_URL: targetUrl.replace('https:', 'wss:') + '/ws', + FRESHELL_E2E_TARGET_TIMEOUT_MS: '45000', FRESHELL_E2E_TARGET_HOME: '/PRIVATE_AMBIENT_HOME', + UNRELATED_AMBIENT_SECRET: 'PRIVATE_AMBIENT_VALUE', FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', + FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: JSON.stringify(managedFixture) } as NodeJS.ProcessEnv + delete env.FRESHELL_CODE_CHECKPOINT_FIXTURE + for (const [key, value] of Object.entries(overrides)) { + if (value === undefined) delete env[key] + else env[key] = value + } + try { + const result = spawnSync('bash', [path.join(root, 'scripts/e2e-cloud.sh'), 'run', '--shards=1', + '--account=synthetic-robot@example.invalid', '--project=chromium-external-target', + env.FRESHELL_E2E_EXTERNAL_FIXTURE_KIND === 'code-checkpoint' + ? 'test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts' + : 'test/e2e-browser/specs/managed-provider-environment-external.spec.ts'], + { cwd: root, env, encoding: 'utf8', timeout: 30_000 }) + const calls = (await fs.readFile(callsFile, 'utf8').catch(() => '')).split('\n').filter(Boolean).map(line => JSON.parse(line) as string[]) + const received = await fs.readFile(receivedFile, 'utf8').catch(() => undefined) + return { result, calls, received: received === undefined ? undefined : parse(received) as Record } + } finally { await fs.rm(directory, { recursive: true, force: true }) } +} + +describe('closed Cloud external target handoff', () => { + it.each([ + { FRESHELL_E2E_TARGET_TOKEN: undefined }, + { FRESHELL_E2E_TARGET_URL: undefined }, + { FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: undefined }, + { FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'unrecognized' }, + { FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: undefined }, + { FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: '{bad-json' }, + { FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: JSON.stringify({ ...managedFixture, candidateSha: 'unknown' }) }, + { FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: JSON.stringify({ ...managedFixture, remoteWorkspacePath: '../outside' }) }, + { FRESHELL_CODE_CHECKPOINT_FIXTURE: JSON.stringify(checkpointFixture) }, + ])('refuses an incomplete or mismatched external selection before identity and dispatch (%j)', async (overrides) => { + const { result, calls, received } = await dispatch(overrides) + expect(result.error).toBeUndefined() + expect(result.status).not.toBe(0) + expect(calls.filter(args => args[0] !== 'info')).toEqual([]) + expect(received).toBeUndefined() + expect(result.stdout + result.stderr).not.toContain(targetToken) + }) + + it.each(['managed-environment', 'code-checkpoint'] as const)( + 'dispatches only the paired %s fixture through the actual private env file', async (kind) => { + const overrides = kind === 'managed-environment' ? {} : { + FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: kind, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: undefined, + FRESHELL_CODE_CHECKPOINT_FIXTURE: JSON.stringify(checkpointFixture), + } + const { result, calls, received } = await dispatch(overrides) + expect(result.error).toBeUndefined() + expect(result.status, result.stderr).toBe(19) + expect(calls.filter(args => args.slice(0, 3).join(' ') === 'run jobs create')).toHaveLength(1) + expect(received).toMatchObject({ FRESHELL_E2E_TARGET_URL: targetUrl, + FRESHELL_E2E_TARGET_TOKEN: targetToken, FRESHELL_E2E_TARGET_WS_URL: targetUrl.replace('https:', 'wss:') + '/ws', + FRESHELL_E2E_TARGET_TIMEOUT_MS: '45000', FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: kind }) + const selected = kind === 'managed-environment' ? 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE' : 'FRESHELL_CODE_CHECKPOINT_FIXTURE' + expect(JSON.parse(received![selected])).toEqual(kind === 'managed-environment' ? managedFixture : checkpointFixture) + expect(received).not.toHaveProperty(kind === 'managed-environment' ? 'FRESHELL_CODE_CHECKPOINT_FIXTURE' : 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE') + expect(received).not.toHaveProperty('FRESHELL_E2E_TARGET_HOME') + expect(received).not.toHaveProperty('UNRELATED_AMBIENT_SECRET') + expect(result.stdout + result.stderr).not.toContain(targetToken) + expect(result.stdout + result.stderr).not.toContain('PRIVATE_AMBIENT_VALUE') + }, + ) +}) From 93fb507211ea461d0cb59956f5de6edbd109cf8a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:27:20 -0700 Subject: [PATCH 247/493] fix: qualify replacement fixture inputs from the current retained owner --- scripts/testing/runtime-test-broker.ts | 38 +++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 09ff17052..010c7f472 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -408,7 +408,7 @@ export class RestrictedDockerBroker { if (method === 'POST' && (url === `${DOCKER_API_PREFIX}/volumes/create` || url.startsWith(`${DOCKER_API_PREFIX}/containers/create?`))) { - this.registerPublishedFixtureInputs(body, url) + await this.registerPublishedFixtureInputs(body, url) } if (method === 'GET' && url === '/_ping') { @@ -1240,7 +1240,7 @@ export class RestrictedDockerBroker { return phase } - private registerPublishedFixtureInputs(body: Buffer, url: string): void { + private async registerPublishedFixtureInputs(body: Buffer, url: string): Promise { let phase = 'owner' try { const request = JSON.parse(body.toString()) @@ -1257,7 +1257,36 @@ export class RestrictedDockerBroker { && input.soulId === owner.soulId && input.incarnationId === incarnationId && (generation === undefined || generation === input.generationId))) return const reservation = this.initialReservations.get(owner.providerVolumeName) - if (!candidate && (!reservation || reservation.incarnationId !== incarnationId)) return + const initial = reservation?.incarnationId === incarnationId + if (!candidate && !initial) { + // A replacement has no initial reservation. Qualify the latest owned + // runtime for this soul, never an earlier quiescent incarnation while + // its successor is live or has changed ownership. + const prior = this.receipts().filter(receipt => receipt.installationId === owner.installationId + && receipt.soulId === owner.soulId).at(-1) + if (!prior || !incarnationId || prior.incarnationId === incarnationId + || prior.providerVolumeName !== owner.providerVolumeName || prior.imageRef !== owner.imageRef + || prior.hostBinaryPath !== owner.hostBinaryPath) return + phase = 'retained_owner' + const inspected = await this.forward({ method: 'GET', + url: `${DOCKER_API_PREFIX}/containers/${prior.containerId}/json`, headers: {} } as IncomingMessage, Buffer.alloc(0)) + if (this.receipts().filter(receipt => receipt.installationId === owner.installationId + && receipt.soulId === owner.soulId).at(-1) !== prior) return + let actual: Record = {} + try { actual = JSON.parse(inspected.body.toString()) } catch {} + const expectedLabels = { project: 'freshell', 'com.freshell.managed': 'true', + 'com.freshell.installation-id': prior.installationId, 'com.freshell.soul-id': prior.soulId, + 'com.freshell.incarnation-id': prior.incarnationId, 'com.freshell.runtime-test-run-id': this.policy.testRunId } + const binaryBind = `${owner.hostBinaryPath}:/runtime/freshell-session-host:ro` + const sameBinary = actual.HostConfig?.Binds?.includes(binaryBind) + || actual.HostConfig?.Mounts?.some((mount: Record) => mount.Type === 'bind' + && mount.Source === owner.hostBinaryPath && mount.Target === '/runtime/freshell-session-host' + && mount.ReadOnly === true) + if (inspected.statusCode !== 404 && (inspected.statusCode !== 200 || actual.Id !== prior.containerId + || actual.Config?.Image !== prior.imageRef || !ownedHelperLabels(actual.Config?.Labels, expectedLabels) + || !sameBinary || actual.State?.Running !== false || actual.State?.Paused === true + || actual.State?.Restarting === true || !['created', 'exited', 'dead'].includes(actual.State?.Status))) return + } const root = path.dirname(this.policy.runtimeRootPrefix) phase = 'physical_directories' const scenarios = this.fixtureDirectories(path.join(root, 'scenarios'), 32) @@ -1306,7 +1335,8 @@ export class RestrictedDockerBroker { resourceMounts: inputs.resourceMounts ?? [], }) } else { - if (inputs.initialAuthReservation !== reservation!.reservationId) continue + if (initial ? inputs.initialAuthReservation !== reservation!.reservationId + : inputs.initialAuthReservation != null) continue const indexedSecretSources: { grantIndex: number; sourceId: number }[] = [] result.providerSecretReferences.forEach((reference: Record, grantIndex: number) => { if (!['claude_configuration_file', 'codex_configuration_file', 'opencode_configuration_file', 'amplifier_configuration_file'].includes(reference.profile)) return From 17f78e253cc6ff6fe9293e12b4333cd4dc3a9f2c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:27:21 -0700 Subject: [PATCH 248/493] fix: validate and forward paired synthetic Cloud browser targets --- scripts/e2e-cloud.sh | 19 ++++ scripts/testing/e2e-external-target.ts | 122 +++++++++++++++++++++++++ test/e2e-browser/playwright.config.ts | 20 +++- 3 files changed, 159 insertions(+), 2 deletions(-) create mode 100644 scripts/testing/e2e-external-target.ts diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index 2f7999abb..aac6b5e09 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -469,6 +469,22 @@ cmd_run() { done pw_args=("${normalized[@]}") + # External browser targets are a closed fixture/project pair. Validate before + # identity, image lookup or paid dispatch; ordinary lanes remain unchanged. + local external_handoff=false + if [ -n "${FRESHELL_E2E_EXTERNAL_FIXTURE_KIND:-}${FRESHELL_MANAGED_ENVIRONMENT_FIXTURE:-}${FRESHELL_CODE_CHECKPOINT_FIXTURE:-}" ]; then + external_handoff=true + fi + for arg in "${pw_args[@]}"; do + case "$arg" in + --project=chromium-external-target|*managed-provider-environment-external.spec.ts|*code-checkpoint-workspace-external.spec.ts) + external_handoff=true ;; + esac + done + if $external_handoff; then + (cd "$ROOT" && pnpm exec tsx scripts/testing/e2e-external-target.ts validate "${pw_args[@]}") + fi + # The container entrypoint treats --dry-run as a diagnostic-only request: it # calculates and prints shard assignments, then exits before Playwright runs. # Such a task has no completion/retry receipt by design, so keep Cloud Run's @@ -647,6 +663,9 @@ cmd_run() { trap cleanup_run_job EXIT trap 'exit 130' INT trap 'exit 143' TERM + if $external_handoff; then + (cd "$ROOT" && pnpm exec tsx scripts/testing/e2e-external-target.ts append-env "$RUN_ENV_FILE" "${pw_args[@]}") + fi # 4 CPU / 4Gi (the coordinator's pre-authorized escalation for the # unified-agent-names acceptance lane): the 2-CPU instances' cold-attempt diff --git a/scripts/testing/e2e-external-target.ts b/scripts/testing/e2e-external-target.ts new file mode 100644 index 000000000..b41d41049 --- /dev/null +++ b/scripts/testing/e2e-external-target.ts @@ -0,0 +1,122 @@ +import fs from 'node:fs' +import path from 'node:path' +import { pathToFileURL } from 'node:url' + +export const EXTERNAL_TARGET_SPECS = { + 'managed-environment': 'managed-provider-environment-external.spec.ts', + 'code-checkpoint': 'code-checkpoint-workspace-external.spec.ts', +} as const + +export type ManagedEnvironmentFixture = { + candidateSha: string + runId: string + remoteWorkspacePath: string + expectedNonce: string +} +export type CodeCheckpointFixture = { + candidateSha: string + runId: string + syntheticWorkspacePath: string + syntheticHomePath: string + nonGitWorkspacePath: string + restrictedUid: number + expectedNonce: string +} +export type PairedExternalFixture = { + kind: keyof typeof EXTERNAL_TARGET_SPECS + spec: string + targetUrl: string + targetToken: string +} & ({ kind: 'managed-environment'; fixture: ManagedEnvironmentFixture } + | { kind: 'code-checkpoint'; fixture: CodeCheckpointFixture }) + +function fail(field: string): never { + // Fixed field names only: token and fixture contents never enter errors. + throw new Error(`External browser fixture is incomplete or invalid at ${field}`) +} + +export function externalFixtureRequested(args: string[]): boolean { + return args.some((arg, index) => arg === '--project=chromium-external-target' + || (arg === '--project' && args[index + 1] === 'chromium-external-target') + || Object.values(EXTERNAL_TARGET_SPECS).some(spec => path.basename(arg) === spec)) +} + +export function resolvePairedExternalFixture( + env: NodeJS.ProcessEnv = process.env, + requested = false, +): PairedExternalFixture | undefined { + const kind = env.FRESHELL_E2E_EXTERNAL_FIXTURE_KIND + if (!requested && !kind && !env.FRESHELL_MANAGED_ENVIRONMENT_FIXTURE && !env.FRESHELL_CODE_CHECKPOINT_FIXTURE) return + if (kind !== 'managed-environment' && kind !== 'code-checkpoint') fail('fixture kind') + const matching = kind === 'managed-environment' ? 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE' : 'FRESHELL_CODE_CHECKPOINT_FIXTURE' + const other = kind === 'managed-environment' ? 'FRESHELL_CODE_CHECKPOINT_FIXTURE' : 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE' + if (!env[matching] || env[other]) fail('matching fixture pair') + let value: Record + try { value = JSON.parse(env[matching]!) } catch { fail('fixture JSON') } + if (!value || typeof value !== 'object' || Array.isArray(value)) fail('fixture JSON') + const paths = kind === 'managed-environment' ? ['remoteWorkspacePath'] + : ['syntheticWorkspacePath', 'syntheticHomePath', 'nonGitWorkspacePath'] + const fields = ['candidateSha', 'runId', 'expectedNonce', ...paths, ...(kind === 'code-checkpoint' ? ['restrictedUid'] : [])] + if (Object.keys(value).length !== fields.length || Object.keys(value).some(key => !fields.includes(key))) fail('fixture fields') + if (typeof value.candidateSha !== 'string' || !/^[a-f0-9]{40}$/.test(value.candidateSha)) fail('candidate SHA') + if (typeof value.runId !== 'string' || !/^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(value.runId)) fail('run ID') + if (typeof value.expectedNonce !== 'string' || !value.expectedNonce.trim()) fail('expected nonce') + for (const field of paths) { + const selected = value[field] + if (typeof selected !== 'string' || !path.posix.isAbsolute(selected) || selected.includes('\0') + || selected.split('/').some(part => part === '.' || part === '..')) fail(field) + } + if (kind === 'code-checkpoint' && (!Number.isSafeInteger(value.restrictedUid) || Number(value.restrictedUid) <= 0)) fail('restricted UID') + const targetUrl = env.FRESHELL_E2E_TARGET_URL + const targetToken = env.FRESHELL_E2E_TARGET_TOKEN + if (!targetUrl || !targetToken?.trim()) fail('target URL/token pair') + let url: URL + try { url = new URL(targetUrl) } catch { fail('target URL') } + if (!['https:', 'http:'].includes(url.protocol) || url.username || url.password || url.search || url.hash + || (url.pathname !== '/' && url.pathname !== '')) fail('target URL') + if (env.FRESHELL_E2E_TARGET_WS_URL) { + let ws: URL + try { ws = new URL(env.FRESHELL_E2E_TARGET_WS_URL) } catch { fail('target WebSocket URL') } + if (ws.host !== url.host || ws.protocol !== (url.protocol === 'https:' ? 'wss:' : 'ws:') + || ws.username || ws.password || ws.search || ws.hash || ws.pathname !== '/ws') fail('target WebSocket URL') + } + if (env.FRESHELL_E2E_TARGET_TIMEOUT_MS && (!/^\d+$/.test(env.FRESHELL_E2E_TARGET_TIMEOUT_MS) + || Number(env.FRESHELL_E2E_TARGET_TIMEOUT_MS) <= 0)) fail('target timeout') + return { kind, spec: EXTERNAL_TARGET_SPECS[kind], targetUrl, targetToken, fixture: value } as PairedExternalFixture +} + +export function validateExternalSelection(args: string[], env: NodeJS.ProcessEnv = process.env) { + const paired = resolvePairedExternalFixture(env, externalFixtureRequested(args)) + if (!paired) return + const projects = args.flatMap((arg, index) => arg.startsWith('--project=') ? [arg.slice('--project='.length)] + : arg === '--project' ? [args[index + 1]] : []) + if (projects.length !== 1 || projects[0] !== 'chromium-external-target') fail('dedicated project selection') + const externalSpecs = args.filter(arg => Object.values(EXTERNAL_TARGET_SPECS).some(spec => path.basename(arg) === spec)) + if (externalSpecs.some(arg => path.basename(arg) !== paired.spec)) fail('matching spec selection') + return paired +} + +export function appendExternalTargetEnvironment(file: string, args: string[], env: NodeJS.ProcessEnv = process.env): void { + if (!validateExternalSelection(args, env)) return + const names = ['FRESHELL_E2E_TARGET_URL', 'FRESHELL_E2E_TARGET_WS_URL', 'FRESHELL_E2E_TARGET_TOKEN', + 'FRESHELL_E2E_TARGET_TIMEOUT_MS', 'FRESHELL_E2E_EXTERNAL_FIXTURE_KIND', + 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE', 'FRESHELL_CODE_CHECKPOINT_FIXTURE'] + // JSON quoted strings are YAML scalars, preserving newlines/metacharacters. + fs.appendFileSync(file, names.filter(name => env[name] !== undefined) + .map(name => `${name}: ${JSON.stringify(env[name])}\n`).join('')) +} + +if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) { + try { + const [operation, ...args] = process.argv.slice(2) + if (operation === 'validate') validateExternalSelection(args) + else if (operation === 'append-env') { + const [file, ...selectors] = args + if (!file) fail('private env file') + appendExternalTargetEnvironment(file, selectors) + } else fail('operation') + } catch (error) { + console.error(error instanceof Error ? error.message : 'External browser fixture validation failed') + process.exitCode = 1 + } +} diff --git a/test/e2e-browser/playwright.config.ts b/test/e2e-browser/playwright.config.ts index 20b8412d7..6e40003ae 100644 --- a/test/e2e-browser/playwright.config.ts +++ b/test/e2e-browser/playwright.config.ts @@ -1,9 +1,20 @@ import { defineConfig, devices } from '@playwright/test' import { DEFAULT_TEST_TIMEOUT_MS } from './helpers/test-harness.js' +import { + EXTERNAL_TARGET_SPECS, + externalFixtureRequested, + resolvePairedExternalFixture, + validateExternalSelection, +} from '../../scripts/testing/e2e-external-target.js' -/** Match-all browser projects exclude only the separately selected live-CLI smoke. */ +/** Ordinary browser discovery excludes separately selected smoke/owned targets. */ export const CONTINUITY_SMOKE_SPEC = /continuity-smoke\.spec\.ts$/ -export const MATCH_ALL_TEST_IGNORE = [CONTINUITY_SMOKE_SPEC] +export const MATCH_ALL_TEST_IGNORE = [CONTINUITY_SMOKE_SPEC, + ...Object.values(EXTERNAL_TARGET_SPECS).map(spec => `**/${spec}`)] +const externalFixture = resolvePairedExternalFixture(process.env, externalFixtureRequested(process.argv)) +if (externalFixture && process.argv.some(arg => arg === 'test' || arg === '--project' || arg.startsWith('--project='))) { + validateExternalSelection(process.argv) +} /** * These specs require local provider binaries or a host Docker supervisor. @@ -87,6 +98,11 @@ export default defineConfig({ use: devices['Desktop Chrome'], testIgnore: MATCH_ALL_TEST_IGNORE, }, + ...(externalFixture ? [{ + name: 'chromium-external-target', + use: devices['Desktop Chrome'], + testMatch: [`**/${externalFixture.spec}`], + }] : []), ...(continuityRequested ? [{ name: 'continuity-smoke', use: devices['Desktop Chrome'], From 24c8836da979e99c839a1bb4e031feec42826599 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:44:24 -0700 Subject: [PATCH 249/493] test: exercise managed fixture host PTY refusal through real endpoints --- crates/freshell-server/src/terminals.rs | 57 ++++ crates/freshell-terminal/src/registry.rs | 79 ++++++ .../src/managed_fixture_denies_host_pty.rs | 244 ++++++++++++++++++ crates/freshell-ws/src/terminal.rs | 4 + 4 files changed, 384 insertions(+) create mode 100644 crates/freshell-ws/src/managed_fixture_denies_host_pty.rs diff --git a/crates/freshell-server/src/terminals.rs b/crates/freshell-server/src/terminals.rs index ecc6e16f7..a033ccd9d 100644 --- a/crates/freshell-server/src/terminals.rs +++ b/crates/freshell-server/src/terminals.rs @@ -1473,6 +1473,63 @@ mod title_scope_tests { } } +#[cfg(all(test, feature = "managed-runtime-v1"))] +mod managed_fixture_denies_host_pty { + use super::*; + use axum::{body::Body, http::Request}; + use tower::util::ServiceExt; + + #[tokio::test] + async fn http_cli_fallback() { + let registry = TerminalRegistry::new().with_managed_environment_fixture_only(true); + let (tx, _rx) = tokio::sync::broadcast::channel::(16); + let state = freshell_freshagent::FreshAgentState::new( + Arc::new("fixture-token".into()), + Arc::new(tx), + ) + .with_terminal_registry(registry.clone()); + // CLI and MCP create-tab requests use this production authenticated route. + let response = freshell_freshagent::router(state) + .oneshot( + Request::builder() + .method("POST") + .uri("/api/tabs") + .header("x-auth-token", "fixture-token") + .header("content-type", "application/json") + .body(Body::from(r#"{"mode":"shell","cwd":"/tmp"}"#)) + .unwrap(), + ) + .await + .unwrap(); + let status = response.status(); + let bytes = axum::body::to_bytes(response.into_body(), usize::MAX) + .await + .unwrap(); + let body: Value = serde_json::from_slice(&bytes).unwrap(); + let rows = registry.directory(); + let pids: Vec<_> = rows + .iter() + .filter_map(|row| registry.pid_of(&row.terminal_id)) + .collect(); + for row in &rows { + registry.kill(&row.terminal_id); + } + assert!(rows.is_empty(), "ordinary rows must be absent: {rows:?}"); + assert!( + pids.is_empty(), + "ordinary child PIDs must be absent: {pids:?}" + ); + assert_eq!(status, axum::http::StatusCode::BAD_REQUEST, "{body}"); + assert!( + body["message"] + .as_str() + .unwrap_or_default() + .contains("managed environment fixture denies host PTY"), + "{body}" + ); + } +} + #[cfg(test)] mod sidebar_projection_tests { use super::*; diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 51279a51e..81ef571e6 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -1629,6 +1629,12 @@ pub trait PaneIdentityBinder: Send + Sync + std::fmt::Debug { } impl TerminalRegistry { + /// Construction seam for the closed managed-environment acceptance target. + /// The refusal behavior is added after its endpoint regression is observed. + pub fn with_managed_environment_fixture_only(self, _fixture_only: bool) -> Self { + self + } + pub fn new() -> Self { Self { inner: Arc::new(Mutex::new(RegistryInner { @@ -15550,3 +15556,76 @@ mod tests { #[cfg(test)] #[path = "managed_output_tests.rs"] mod managed_output_tests; + +#[cfg(test)] +mod fixture_denies_host_pty { + use super::*; + + fn missing_program() -> SpawnSpec { + SpawnSpec { + program: "/freshell-fixture-missing-program".into(), + args: vec![], + env_overrides: BTreeMap::new(), + cwd: Some("/tmp".into()), + cols: 80, + rows: 24, + } + } + + fn create(registry: &TerminalRegistry) -> io::Result<()> { + registry.create( + &missing_program(), + &BTreeMap::new(), + "fixture-host".into(), + "fixture-stream".into(), + "shell", + None, + None, + None, + None, + ) + } + + #[test] + fn refuses_before_opening_a_host_pty() { + let registry = TerminalRegistry::new().with_managed_environment_fixture_only(true); + let error = create(®istry).expect_err("host creation must be refused"); + assert!(registry.directory().is_empty()); + assert_eq!(registry.pid_of("fixture-host"), None); + assert_eq!(error.kind(), io::ErrorKind::PermissionDenied, "{error}"); + assert!(error + .to_string() + .contains("managed environment fixture denies host PTY")); + } + + #[test] + fn ordinary_construction_keeps_existing_spawn_errors() { + for registry in [ + TerminalRegistry::new(), + TerminalRegistry::new().with_managed_environment_fixture_only(false), + ] { + let error = create(®istry).expect_err("the selected program does not exist"); + assert_ne!(error.kind(), io::ErrorKind::PermissionDenied, "{error}"); + assert!(registry.directory().is_empty()); + } + } + + #[test] + fn managed_facade_remains_available_without_a_host_pid() { + let registry = TerminalRegistry::new().with_managed_environment_fixture_only(true); + registry.register_managed(ManagedTerminalDescriptor { + soul_id: "fixture-soul".into(), + incarnation_id: "fixture-incarnation".into(), + terminal_id: "fixture-managed".into(), + stream_id: "fixture-stream".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: None, + create_request_id: Some("fixture-create".into()), + }); + assert!(registry.is_managed("fixture-managed")); + assert_eq!(registry.pid_of("fixture-managed"), None); + assert!(registry.ingest_managed_output("fixture-managed", 1, 1, "fixture output".into())); + assert_eq!(registry.directory()[0].snapshot, "fixture output"); + } +} diff --git a/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs b/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs new file mode 100644 index 000000000..52ec94fa2 --- /dev/null +++ b/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs @@ -0,0 +1,244 @@ +//! Real authenticated endpoint coverage. The finite controller below is a +//! routing fixture; genuine provider/Bun preparation is qualified by the owned +//! managed-environment browser target separately. +use super::*; +use freshell_terminal::registry::{ + ManagedOutputRead, ManagedTerminalController, ManagedTerminalDescriptor, ManagedTerminalFuture, +}; +use serde_json::{json, Value}; +use tokio_tungstenite::tungstenite::Message as ClientMessage; + +struct SyntheticConfiguration { + configuration: std::path::PathBuf, + output: StdMutex>, +} + +impl ManagedTerminalController for SyntheticConfiguration { + fn lookup_terminal<'a>( + &'a self, + _: &'a str, + _: Option, + ) -> ManagedTerminalFuture<'a, Result, String>> { + Box::pin(async { Ok(None) }) + } + fn launch<'a>( + &'a self, + request: ManagedTerminalLaunch, + ) -> ManagedTerminalFuture<'a, Result> { + Box::pin(async move { + if request.mode != "opencode" { + return Err("unexpected provider".into()); + } + let consumed = tokio::process::Command::new("/bin/sh") + .arg(&self.configuration) + .env_clear() + .env("PATH", "/usr/bin:/bin") + .output() + .await + .map_err(|error| error.to_string())?; + if !consumed.status.success() { + return Err("synthetic configuration failed".into()); + } + *self.output.lock().unwrap() = + Some(String::from_utf8(consumed.stdout).map_err(|error| error.to_string())?); + Ok(ManagedTerminalDescriptor { + soul_id: "fixture-soul".into(), + incarnation_id: "fixture-incarnation".into(), + terminal_id: request.terminal_id, + stream_id: request.stream_id, + mode: request.mode, + cwd: request.spec.cwd.unwrap_or_default(), + resume_session_id: request.resume_session_id, + create_request_id: request.create_request_id, + }) + }) + } + fn input<'a>( + &'a self, + _: ManagedTerminalDescriptor, + _: String, + ) -> ManagedTerminalFuture<'a, Result<(), String>> { + Box::pin(async { Ok(()) }) + } + fn resize<'a>( + &'a self, + _: ManagedTerminalDescriptor, + _: u16, + _: u16, + ) -> ManagedTerminalFuture<'a, Result<(), String>> { + Box::pin(async { Ok(()) }) + } + fn stop<'a>( + &'a self, + _: ManagedTerminalDescriptor, + ) -> ManagedTerminalFuture<'a, Result<(), String>> { + Box::pin(async { Ok(()) }) + } + fn read_output<'a>( + &'a self, + _: ManagedTerminalDescriptor, + _: i64, + _: u64, + ) -> ManagedTerminalFuture<'a, Result> { + Box::pin(async { Err("this routing fixture does not serve replay".into()) }) + } +} + +async fn create_over_socket( + capabilities: Option, + managed: bool, +) -> (Value, Vec, Option) { + let root = tempfile::tempdir().unwrap(); + let configuration = root.path().join("configuration.sh"); + std::fs::write( + &configuration, + "printf 'synthetic-configuration-consumed:%s' 'endpoint-nonce'\n", + ) + .unwrap(); + let controller = Arc::new(SyntheticConfiguration { + configuration, + output: StdMutex::new(None), + }); + let mut state = super::pane_reconcile_gate_tests::state(); + state.registry = + freshell_terminal::TerminalRegistry::new().with_managed_environment_fixture_only(true); + state + .registry + .set_managed_controller(Some(controller.clone())); + state.cli_commands = Arc::new(vec![freshell_platform::CliCommandSpec { + name: "opencode".into(), + label: "OpenCode".into(), + default_cmd: "opencode".into(), + ..Default::default() + }]); + let registry = state.registry.clone(); + let token = state.auth_token.to_string(); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("ws://{}/ws", listener.local_addr().unwrap()); + let (stop, stopped) = tokio::sync::oneshot::channel(); + let server = tokio::spawn(async move { + axum::serve(listener, crate::router(state)) + .with_graceful_shutdown(async { + stopped.await.ok(); + }) + .await + }); + let mut client = tokio_tungstenite::connect_async(url).await.unwrap().0; + let observed = async { + let mut hello = json!({"type":"hello","token":token, + "protocolVersion":freshell_protocol::WS_PROTOCOL_VERSION}); + if let Some(capabilities) = capabilities { + hello["capabilities"] = capabilities; + } + client + .send(ClientMessage::Text(hello.to_string())) + .await + .map_err(|e| e.to_string())?; + let mut handshake = vec![]; + for _ in 0..4 { + let message = tokio::time::timeout(std::time::Duration::from_secs(5), client.next()) + .await + .map_err(|_| "hello timeout")? + .ok_or("hello closed")? + .map_err(|e| e.to_string())?; + if let ClientMessage::Text(text) = message { + handshake.push(serde_json::from_str::(&text).map_err(|e| e.to_string())?); + } + } + client + .send(ClientMessage::Text( + json!({"type":"terminal.create", + "requestId":"fixture-endpoint", "mode":if managed {"opencode"} else {"shell"}, + "shell":"system","cwd":root.path()}) + .to_string(), + )) + .await + .map_err(|e| e.to_string())?; + loop { + let message = tokio::time::timeout(std::time::Duration::from_secs(10), client.next()) + .await + .map_err(|_| "create timeout")? + .ok_or("create closed")? + .map_err(|e| e.to_string())?; + if let ClientMessage::Text(text) = message { + let frame: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?; + if frame["type"] == "error" || frame["type"] == "terminal.created" { + return Ok::<_, String>((frame, handshake)); + } + } + } + } + .await; + client.close(None).await.ok(); + drop(client); + let rows = registry.directory(); + let row_observations: Vec = rows + .iter() + .map(|row| { + json!({"terminalId":row.terminal_id, + "managed":registry.is_managed(&row.terminal_id),"pid":registry.pid_of(&row.terminal_id)}) + }) + .collect(); + // RED can create an ordinary child. Reap only handles from this test's registry, + // before any assertion, including the error/timeout path. + for row in &rows { + registry.kill(&row.terminal_id); + } + stop.send(()).unwrap(); + tokio::time::timeout(std::time::Duration::from_secs(10), server) + .await + .unwrap() + .unwrap() + .unwrap(); + let (frame, handshake) = observed.expect("authenticated endpoint completes"); + assert_eq!(handshake[0]["type"], "ready"); + let output = controller.output.lock().unwrap().clone(); + (frame, row_observations, output) +} + +fn assert_denial(frame: Value, rows: Vec, output: Option) { + assert!( + rows.is_empty(), + "ordinary registry rows/PIDs must be absent: {rows:?}" + ); + assert_eq!(frame["type"], "error", "{frame}"); + assert!( + frame["message"] + .as_str() + .unwrap_or_default() + .contains("managed environment fixture denies host PTY"), + "{frame}" + ); + assert!( + output.is_none(), + "unnegotiated requests must not launch managed either" + ); +} + +#[tokio::test] +async fn legacy_ws() { + let (frame, rows, output) = create_over_socket(None, false).await; + assert_denial(frame, rows, output); +} + +#[tokio::test] +async fn non_negotiated_ws() { + let (frame, rows, output) = + create_over_socket(Some(json!({"paneReconcileV1":true})), false).await; + assert_denial(frame, rows, output); +} + +#[tokio::test] +async fn managed_coding_still_works() { + let (frame, rows, output) = + create_over_socket(Some(json!({"managedRuntimeV1":true})), true).await; + assert_eq!(frame["type"], "terminal.created", "{frame}"); + assert_eq!(rows.len(), 1); + assert_eq!(rows[0]["terminalId"], frame["terminalId"]); + assert_eq!(rows[0]["managed"], true); + assert!(rows[0]["pid"].is_null()); + assert_eq!( + output.as_deref(), + Some("synthetic-configuration-consumed:endpoint-nonce") + ); +} diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index a2ee472c6..802773425 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -81,6 +81,10 @@ mod terminal_create_ordering_tests; #[path = "terminal_launch_prep_tests.rs"] mod terminal_launch_prep_tests; +#[cfg(test)] +#[path = "managed_fixture_denies_host_pty.rs"] +mod managed_fixture_denies_host_pty; + #[path = "connection_writer.rs"] mod connection_writer; #[path = "interactive_creates.rs"] From d08faff27d84b42eec377c49aa7ed671c667ef70 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:56:59 -0700 Subject: [PATCH 250/493] fix: refuse ordinary host terminals in managed acceptance fixtures --- crates/freshell-server/src/main.rs | 10 +++++++++- crates/freshell-terminal/src/registry.rs | 22 +++++++++++++++++++--- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index 9b2700d38..5487ae679 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -1125,8 +1125,11 @@ async fn main() -> ExitCode { // the registry's kill/exit/confirmed-kill paths release coordinator // ownership fenced (release-only integration; the WS/REST/auto-resume // lanes claim). + let managed_environment_fixture_only = + std::env::var("FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY").as_deref() == Ok("1"); let registry = freshell_terminal::TerminalRegistry::new() - .with_ownership(std::sync::Arc::clone(&ownership)); + .with_ownership(std::sync::Arc::clone(&ownership)) + .with_managed_environment_fixture_only(managed_environment_fixture_only); #[cfg(feature = "managed-runtime-v1")] let (managed_runtime_available, managed_runtime_client, managed_runtime_controller) = { let controller = managed_runtime::ServerManagedRuntimeController::from_env() @@ -1159,6 +1162,11 @@ async fn main() -> ExitCode { }; #[cfg(not(feature = "managed-runtime-v1"))] let managed_runtime_available = false; + if managed_environment_fixture_only && !managed_runtime_available { + tracing::error!("managed_environment_fixture.controller_unavailable"); + eprintln!("managed environment fixture requires an initialized managed runtime controller"); + std::process::exit(1); + } #[cfg(feature = "managed-runtime-v1")] let hosted_fresh_gateway = fresh_agent_proxy::HostedFreshAgentProxy::from_opt_in( managed_runtime_client.clone(), diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 81ef571e6..7b8fa1e8b 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -1239,6 +1239,9 @@ pub type IdentityReadoptPauseHook = std::sync::Arc< #[derive(Clone)] pub struct TerminalRegistry { inner: Arc>, + // Closed scratch acceptance target only. Ordinary construction retains + // host terminal creation; managed facade registration never spawns here. + managed_environment_fixture_only: bool, conn_seq: Arc, /// DIAG-05: a LIVE count of currently-open `/ws` connections (distinct /// from `conn_seq`, which is a monotonic minting counter that never goes @@ -1629,14 +1632,16 @@ pub trait PaneIdentityBinder: Send + Sync + std::fmt::Debug { } impl TerminalRegistry { - /// Construction seam for the closed managed-environment acceptance target. - /// The refusal behavior is added after its endpoint regression is observed. - pub fn with_managed_environment_fixture_only(self, _fixture_only: bool) -> Self { + /// Disable ordinary host PTYs for the closed managed-environment fixture. + /// This is an immutable construction input, not a product permission policy. + pub fn with_managed_environment_fixture_only(mut self, fixture_only: bool) -> Self { + self.managed_environment_fixture_only = fixture_only; self } pub fn new() -> Self { Self { + managed_environment_fixture_only: false, inner: Arc::new(Mutex::new(RegistryInner { terminals: HashMap::new(), revision: 0, @@ -2385,6 +2390,17 @@ impl TerminalRegistry { ring_max_bytes: Option, on_exit: Option, ) -> io::Result<()> { + if self.managed_environment_fixture_only { + tracing::warn!( + terminal_id = %terminal_id, + mode, + "terminal.create.fixture_host_pty_refused" + ); + return Err(io::Error::new( + io::ErrorKind::PermissionDenied, + "managed environment fixture denies host PTY creation", + )); + } // Duplicate-live-resume enforcement (amplifier identity plan, // validated fix F5/V7): the callers' `has_live_resume` pre-check is // check-then-act and can race across WS/REST tasks — this registry's From 6a14e38baba1b65079a52ccffc3a8ad6c7a414ea Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 08:14:32 -0700 Subject: [PATCH 251/493] test(runtime): exercise closed managed fixture input routing --- scripts/testing/runtime-sandbox.ts | 8 +- .../tooling/testing/runtime-sandbox.test.ts | 83 +++++++++++++++++++ 2 files changed, 90 insertions(+), 1 deletion(-) diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index 172d7d8c1..bd5dc1f0a 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -96,6 +96,11 @@ export function phase2BootstrapFiles( return [...files] } +export type RuntimeFixtureInputs = { + workspaceRoots: string[] + bootstrapFiles: string[] +} + export class RuntimeGateAssertionError extends Error { constructor( readonly caseId: string, @@ -144,7 +149,8 @@ export class RuntimeHarness { private brokerStarted = false constructor(repoRoot: string, runId = randomUUID(), phase: 1 | 2 | 3 | 4 | 5 = 1, - private readonly providerFixtureHome?: string, private readonly providerFixtureXdgConfigHome?: string) { + private readonly providerFixtureHome?: string, private readonly providerFixtureXdgConfigHome?: string, + _fixtureInputs?: RuntimeFixtureInputs) { this.repoRoot = fs.realpathSync(repoRoot) this.phase = phase this.runId = runId diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index ac5272b07..de45e1ce7 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -192,6 +192,89 @@ it.each(['claude', 'opencode'] as const)('qualifies original isolated HOME alias } }) +describe('closed managed environment fixture inputs', () => { + it.each(['selected-workspace', 'selected-bootstrap', 'ambient-bootstrap'] as const)( + 'routes only the explicitly selected fixture inputs: %s', async selection => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-input-')) + const repository = path.join(directory, 'repository') + const home = path.join(directory, 'home') + const workspace = selection === 'selected-workspace' + ? path.join(directory, 'selected-workspace') : path.join(repository, 'selected-workspace') + const grant = path.join(directory, 'selected.env') + const socket = path.join(directory, 'daemon.sock') + const forwarded: string[] = [] + const daemon = http.createServer((request, response) => { + request.resume() + forwarded.push(request.url!) + response.writeHead(201).end(JSON.stringify({ Id: 'd'.repeat(64) })) + }) + const previousDockerHost = process.env.DOCKER_HOST + const bootstrapKey = 'FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE' + const previousBootstrap = process.env[bootstrapKey] + let harness: RuntimeHarness | undefined + try { + await fs.mkdir(path.join(repository, 'test/runtime'), { recursive: true }) + await fs.mkdir(home) + await fs.mkdir(workspace) + await fs.writeFile(path.join(repository, 'test/runtime/gate-manifest.json'), '{}') + await fs.writeFile(grant, 'synthetic-selected-grant', { mode: 0o600 }) + const host = path.join(directory, 'host') + await fs.writeFile(host, 'owned fixture host artifact') + await new Promise(resolve => daemon.listen(socket, resolve)) + process.env.DOCKER_HOST = `unix://${socket}` + if (selection === 'ambient-bootstrap') process.env[bootstrapKey] = grant + else delete process.env[bootstrapKey] + harness = new RuntimeHarness(repository, undefined, 2, home, undefined, { + workspaceRoots: [workspace], bootstrapFiles: selection === 'selected-bootstrap' ? [grant] : [], + }) + useOwnedHarnessArtifacts(harness, host) + await harness.prepare() + const create = (selectedWorkspace: string, bootstrap?: string) => new Promise((resolve, reject) => { + const request = http.request({ socketPath: harness!.broker.proxySocketPath, + path: '/v1.47/containers/create?name=closed-fixture', method: 'POST' }, response => { + response.resume(); response.once('end', () => resolve(response.statusCode!)) + }) + request.once('error', reject) + request.end(JSON.stringify({ Image: harness!.imageRef, Env: [], Labels: { + project: 'freshell', 'com.freshell.managed': 'true', + 'com.freshell.installation-id': 'installation-fixture', + 'com.freshell.soul-id': 'soul-fixture', 'com.freshell.incarnation-id': 'incarnation-fixture', + 'com.freshell.runtime-test-run-id': harness!.runId, + }, HostConfig: { NetworkMode: 'bridge', ExtraHosts: [], PidMode: '', ReadonlyRootfs: true, + Privileged: false, CapDrop: ['ALL'], CapAdd: ['CHOWN', 'SETGID', 'SETUID'], + SecurityOpt: ['no-new-privileges:true'], RestartPolicy: { Name: 'no' }, + NanoCpus: 500_000_000, Memory: 128 * 1024 * 1024, PidsLimit: 64, + Tmpfs: { '/tmp': 'rw,noexec,nosuid,nodev,size=128m' }, Binds: [ + `${host}:/runtime/freshell-session-host:ro`, + `${path.join(harness!.testRoot, 'r/scenario/x/incarnation-fixture')}:/run/freshell:rw`, + `freshell-provider-${'a'.repeat(24)}:/home/freshell/provider:rw`, + `${selectedWorkspace}:${selectedWorkspace}:rw`, + ...(bootstrap ? [`${bootstrap}:/run/freshell-bootstrap/provider-0:ro`] : []), + ] } })) + }) + const expected = selection === 'ambient-bootstrap' ? 403 : 201 + expect(await create(workspace, selection === 'selected-workspace' ? undefined : grant), + JSON.stringify(harness.broker.eventsSnapshot())).toBe(expected) + expect(forwarded).toEqual(expected === 201 ? ['/v1.47/containers/create?name=closed-fixture'] : []) + // Closing the selected roots must also remove the repository's ambient grant. + forwarded.length = 0 + expect(await create(repository)).toBe(403) + expect(forwarded).toEqual([]) + } finally { + await harness?.broker?.close() + await new Promise(resolve => daemon.close(() => resolve())) + if (previousDockerHost === undefined) delete process.env.DOCKER_HOST + else process.env.DOCKER_HOST = previousDockerHost + if (previousBootstrap === undefined) delete process.env[bootstrapKey] + else process.env[bootstrapKey] = previousBootstrap + if (harness) await fs.rm(harness.testRoot, { recursive: true, force: true }) + await fs.rm(directory, { recursive: true, force: true }) + vi.restoreAllMocks() + } + }, + ) +}) + describe('runtime harness indexed input cleanup', () => { it.each(['removed', 'foreign-label', 'attached', 'retained-after-delete'] as const)( 'settles the actual publisher before exact nonforce cleanup: %s', async outcome => { From 39f202924ef0664f0ccae226b4b5a7f27f526e29 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 08:19:48 -0700 Subject: [PATCH 252/493] fix(runtime): select only closed managed fixture inputs --- scripts/testing/runtime-sandbox.ts | 31 +++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index bd5dc1f0a..212d26eae 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -101,6 +101,16 @@ export type RuntimeFixtureInputs = { bootstrapFiles: string[] } +function validateOwnedFixturePath(selected: string, kind: 'directory' | 'file', label: string): string { + const metadata = fs.lstatSync(selected) + if (!path.isAbsolute(selected) || path.normalize(selected) !== selected || fs.realpathSync(selected) !== selected + || !(kind === 'directory' ? metadata.isDirectory() : metadata.isFile()) || metadata.isSymbolicLink() + || (process.getuid && metadata.uid !== process.getuid())) { + throw new Error(`${label} must retain its selected owned physical ${kind}`) + } + return selected +} + export class RuntimeGateAssertionError extends Error { constructor( readonly caseId: string, @@ -147,11 +157,15 @@ export class RuntimeHarness { private readonly trackedVolumes = new Set() private readonly assertions: AssertionRecord[] = [] private brokerStarted = false + private readonly fixtureInputs: RuntimeFixtureInputs | undefined constructor(repoRoot: string, runId = randomUUID(), phase: 1 | 2 | 3 | 4 | 5 = 1, private readonly providerFixtureHome?: string, private readonly providerFixtureXdgConfigHome?: string, - _fixtureInputs?: RuntimeFixtureInputs) { + fixtureInputs?: RuntimeFixtureInputs) { this.repoRoot = fs.realpathSync(repoRoot) + this.fixtureInputs = fixtureInputs && { + workspaceRoots: [...fixtureInputs.workspaceRoots], bootstrapFiles: [...fixtureInputs.bootstrapFiles], + } this.phase = phase this.runId = runId this.candidateSha = git(this.repoRoot, ['rev-parse', 'HEAD']).trim() @@ -1076,10 +1090,16 @@ export class RuntimeHarness { } private phase2BootstrapFiles(): string[] { + if (this.fixtureInputs) { + return this.fixtureInputs.bootstrapFiles.map(selected => validateOwnedFixturePath(selected, 'file', 'bootstrap fixture input')) + } return phase2BootstrapFiles() } private phase2WorkspaceRoots(): string[] { + if (this.fixtureInputs) { + return this.fixtureInputs.workspaceRoots.map(selected => validateOwnedFixturePath(selected, 'directory', 'workspace fixture root')) + } const roots = new Set([this.repoRoot, this.testRoot]) try { const common = git(this.repoRoot, ['rev-parse', '--git-common-dir']).trim() @@ -1090,14 +1110,7 @@ export class RuntimeHarness { private phase2ProviderUserRoots(): string[] { const home = this.providerFixtureHome ?? os.homedir() - const validateFixtureDirectory = (selected: string) => { - const metadata = fs.lstatSync(selected) - if (!path.isAbsolute(selected) || path.normalize(selected) !== selected || fs.realpathSync(selected) !== selected - || !metadata.isDirectory() || metadata.isSymbolicLink() - || (process.getuid && metadata.uid !== process.getuid())) { - throw new Error('provider fixture root must retain its selected owned physical directory') - } - } + const validateFixtureDirectory = (selected: string) => validateOwnedFixturePath(selected, 'directory', 'provider fixture root') if (this.providerFixtureHome !== undefined) validateFixtureDirectory(home) let opencodeRoot = path.join(home, '.config/opencode') if (this.providerFixtureHome !== undefined && this.providerFixtureXdgConfigHome !== undefined) { From 12b8564ebdcdb35b3d79a01a88be408408a18d3d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 08:26:51 -0700 Subject: [PATCH 253/493] test(browser): execute closed external project discovery --- .../helpers/selection-nonvacuity.test.ts | 55 +++++++++++-------- .../tooling/testing/browser-selection.test.ts | 2 + 2 files changed, 35 insertions(+), 22 deletions(-) create mode 100644 test/unit/tooling/testing/browser-selection.test.ts diff --git a/test/e2e-browser/helpers/selection-nonvacuity.test.ts b/test/e2e-browser/helpers/selection-nonvacuity.test.ts index d6454f28c..b0cd958b0 100644 --- a/test/e2e-browser/helpers/selection-nonvacuity.test.ts +++ b/test/e2e-browser/helpers/selection-nonvacuity.test.ts @@ -14,7 +14,6 @@ const playwrightCli = require.resolve('@playwright/test/cli') const configLoader = require.resolve('playwright/lib/common/configLoader') const playwrightConfig = path.join(browserRoot, 'playwright.config.ts') const cloudConfig = path.join(browserRoot, 'playwright.cloud.config.ts') -const continuityPattern = { kind: 'regexp', source: 'continuity-smoke\\.spec\\.ts$', flags: '' } interface ResolvedProject { name: string @@ -26,6 +25,9 @@ function cleanEnvironment(overrides: Record = {}): NodeJS.Proces const env = { ...process.env } delete env.CI delete env.FRESHELL_SMOKE + for (const key of ['FRESHELL_E2E_EXTERNAL_FIXTURE_KIND', 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE', + 'FRESHELL_CODE_CHECKPOINT_FIXTURE', 'FRESHELL_E2E_TARGET_URL', 'FRESHELL_E2E_TARGET_WS_URL', + 'FRESHELL_E2E_TARGET_TOKEN', 'FRESHELL_E2E_TARGET_TIMEOUT_MS']) delete env[key] return { ...env, ...overrides } } @@ -94,23 +96,36 @@ function listedSpecFiles(output: string): Set { } describe('browser selection non-vacuity', () => { - it('resolves only Rust application projects with the exact continuity exclusion', () => { - const defaultProjects = resolvedConfig(playwrightConfig, cleanEnvironment()) - expect(defaultProjects.map((project) => project.name)).toEqual(['chromium']) - expect(defaultProjects[0].testIgnore).toEqual([continuityPattern]) - - const ciProjects = resolvedConfig(playwrightConfig, cleanEnvironment({ CI: '1' })) - expect(ciProjects.map((project) => project.name)).toEqual(['chromium', 'firefox', 'webkit']) - for (const project of ciProjects) expect(project.testIgnore).toEqual([continuityPattern]) - - const continuityProjects = resolvedConfig(playwrightConfig, cleanEnvironment({ FRESHELL_SMOKE: '1' })) - expect(continuityProjects.map((project) => project.name)).toEqual(['chromium', 'continuity-smoke']) - expect(continuityProjects[0].testIgnore).toEqual([continuityPattern]) - expect(continuityProjects[1]).toMatchObject({ - name: 'continuity-smoke', - testIgnore: [], - testMatch: [continuityPattern], - }) + it('discovers the managed external cases only through their paired dedicated Cloud project', () => { + const spec = 'managed-provider-environment-external.spec.ts' + const ordinary = listedProjects(cleanEnvironment(), cloudConfig) + expect(listedSpecFiles(ordinary.output).has(spec)).toBe(false) + expect(listedSpecFiles(ordinary.output).has('code-checkpoint-workspace-external.spec.ts')).toBe(false) + const fixture = JSON.stringify({ candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', + remoteWorkspacePath: '/tmp/selected-managed-fixture/workspace', expectedNonce: 'selection-fixture-nonce' }) + const paired = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', + FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: fixture, FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', + FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token' }) + const selected = listedProjects(paired, cloudConfig, ['--project=chromium-external-target', spec]) + expect(selected.labels).toEqual(['chromium-external-target']) + expect(selected.tests).toBe(3) + expect(selected.files).toBe(1) + expect([...listedSpecFiles(selected.output)]).toEqual([spec]) + // The source exists now: ordinary projects still cannot execute it. + const ordinaryExplicit = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, + '--list', '--project=chromium', spec], { cwd: projectRoot, env: paired, encoding: 'utf8' }) + expect(ordinaryExplicit.status).toBe(1) + }, 90_000) + + it.each(['missing', 'mismatched'] as const)('refuses external discovery with a %s fixture pair', pairing => { + const env = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', + FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token', + ...(pairing === 'mismatched' ? { FRESHELL_CODE_CHECKPOINT_FIXTURE: '{}' } : {}) }) + const listed = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, '--list', + '--project=chromium-external-target', 'managed-provider-environment-external.spec.ts'], + { cwd: projectRoot, env, encoding: 'utf8' }) + expect(listed.status).toBe(1) + expect(`${listed.stdout}\n${listed.stderr}`).toContain('matching fixture pair') }) it('selects a non-vacuous Chromium lane and all CI application projects', () => { @@ -172,10 +187,6 @@ describe('browser selection non-vacuity', () => { const cloudProjects = resolvedConfig(cloudConfig, cleanEnvironment()) expect(cloudProjects.map((project) => project.name)).toEqual(['chromium']) - expect(cloudProjects[0].testIgnore).toEqual([ - continuityPattern, - ...CLOUD_SKIP_SPECS.map((spec) => ({ kind: 'string' as const, source: `**/${spec}` })), - ]) expect(new Set(CLOUD_SKIP_SPECS).size).toBe(CLOUD_SKIP_SPECS.length) for (const localOnly of LOCAL_ONLY_SPECS) expect(CLOUD_SKIP_SPECS).toContain(localOnly.spec) diff --git a/test/unit/tooling/testing/browser-selection.test.ts b/test/unit/tooling/testing/browser-selection.test.ts new file mode 100644 index 000000000..60665fb65 --- /dev/null +++ b/test/unit/tooling/testing/browser-selection.test.ts @@ -0,0 +1,2 @@ +// The default unit lane excludes browser helpers; execute their real CLI discovery checks here. +import '../../../e2e-browser/helpers/selection-nonvacuity.test.js' From 6ad6f84da472ec84b9fc96d518af53c782ab35dd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 08:52:14 -0700 Subject: [PATCH 254/493] test: assemble isolated managed environment acceptance fixture --- .../managed-provider-environment-target.ts | 320 ++++++++++++++++++ test/e2e-browser/helpers/managed-runtime.ts | 14 +- .../helpers/selection-nonvacuity.test.ts | 255 +------------- ...aged-provider-environment-external.spec.ts | 148 ++++++++ .../tooling/testing/browser-selection.test.ts | 255 +++++++++++++- .../tooling/testing/runtime-sandbox.test.ts | 55 ++- 6 files changed, 788 insertions(+), 259 deletions(-) create mode 100644 scripts/testing/managed-provider-environment-target.ts create mode 100644 test/e2e-browser/specs/managed-provider-environment-external.spec.ts diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts new file mode 100644 index 000000000..f4b48f849 --- /dev/null +++ b/scripts/testing/managed-provider-environment-target.ts @@ -0,0 +1,320 @@ +import assert from 'node:assert/strict' +import { spawn, type ChildProcess } from 'node:child_process' +import { createHash, randomUUID } from 'node:crypto' +import fs from 'node:fs/promises' +import http from 'node:http' +import os from 'node:os' +import path from 'node:path' +import { fileURLToPath } from 'node:url' +import WebSocket from 'ws' +import { ManagedRuntimeBrowserRig, type ManagedRuntimeView } from '../../test/e2e-browser/helpers/managed-runtime.js' +import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../test/helpers/provider-parity-broker.js' +import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' +import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' +import { resolvePairedExternalFixture } from './e2e-external-target.js' + +const cloudflaredVersion = '2026.10.0' +const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' +const scenarios = ['healthy', 'missing-reference', 'startup-before-identity'] as const +const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) +const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') + +async function poll(description: string, operation: () => Promise, timeout = 180_000): Promise { + const end = Date.now() + timeout + while (Date.now() < end) { + const value = await operation() + if (value !== undefined) return value + await delay(250) + } + throw new Error(`Managed environment timed out: ${description}`) +} + +async function writeJson(selected: string, value: unknown) { + await fs.writeFile(selected, JSON.stringify(value, null, 2), { mode: 0o600 }) +} + +async function readJson(selected: string): Promise { + try { return JSON.parse(await fs.readFile(selected, 'utf8')) } + catch (error) { if ((error as NodeJS.ErrnoException).code === 'ENOENT') return; throw error } +} + +async function processIdentity(child: ChildProcess) { + assert(child.pid) + const raw = await fs.readFile(`/proc/${child.pid}/stat`, 'utf8') + const fields = raw.slice(raw.lastIndexOf(')') + 2).trim().split(/\s+/) + return { pid: child.pid, parentPid: Number(fields[1]), birthTicks: fields[19], cwd: await fs.readlink(`/proc/${child.pid}/cwd`) } +} + +function joined(child: ChildProcess): Promise { + return new Promise((resolve, reject) => { + child.once('error', reject) + child.once('exit', (code, signal) => code === null ? reject(new Error(`Owned child exited with ${signal}`)) : resolve(code)) + }) +} + +async function stopExact(child: ChildProcess | undefined, identity: Awaited> | undefined, + completion: Promise | undefined) { + if (!child || !identity || !completion) return + if (child.exitCode === null && child.signalCode === null) { + try { + const current = await processIdentity(child) + assert.deepEqual(current, identity, 'Owned child identity changed; retain it') + child.kill('SIGTERM') + } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + } + await completion.catch(error => { if (!child.signalCode) throw error }) +} + +async function daemonId(socketPath: string): Promise { + return new Promise((resolve, reject) => { + const request = http.get({ socketPath, path: '/v1.47/info', timeout: 30_000 }, response => { + const chunks: Buffer[] = [] + let bytes = 0 + response.on('data', chunk => { bytes += chunk.length; if (bytes > 65536) request.destroy(new Error('Daemon metadata exceeded its bound')); else chunks.push(chunk) }) + response.once('end', () => { + try { assert.equal(response.statusCode, 200); const value = JSON.parse(Buffer.concat(chunks).toString()); assert.equal(typeof value.ID, 'string'); resolve(value.ID) } + catch (error) { reject(error) } + }) + }) + request.once('timeout', () => request.destroy(new Error('Daemon metadata timed out'))) + request.once('error', reject) + }) +} + +async function configure(root: string, nonce: string, candidateSha: string, runId: string) { + const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') + const resources = path.join(workspace, 'selected-resources'), plugin = path.join(resources, 'fixture-plugin') + await fs.mkdir(path.join(plugin, 'node_modules/managed-fixture-dependency'), { recursive: true, mode: 0o755 }) + await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) + await fs.chmod(home, 0o700) + await fs.chmod(root, 0o751) + await fs.writeFile(path.join(plugin, 'node_modules/managed-fixture-dependency/package.json'), '{"type":"module","exports":"./index.mjs"}') + await fs.writeFile(path.join(plugin, 'node_modules/managed-fixture-dependency/index.mjs'), `export default ${JSON.stringify(nonce)};`) + await fs.writeFile(path.join(plugin, 'selected-resource'), nonce, { mode: 0o600 }) + const command = path.join(plugin, 'command') + await fs.writeFile(command, `#!/bin/sh\nprintf '%s\\n' '${JSON.stringify({ resource: nonce })}'\n`, { mode: 0o751 }) + await fs.chmod(command, 0o751) + await fs.writeFile(path.join(plugin, 'package.json'), '{"type":"module","exports":{"./server":"./server.mjs","./tui":"./tui.mjs"}}') + await fs.writeFile(path.join(plugin, 'server.mjs'), `export default {id:'managed-environment-server',async server(){return {async config(){if(process.cwd().endsWith('/startup-before-identity'))throw new Error('startup-fixture failed before native identity')}}}};`) + // Exact pinned API: OpenCode v1.18.21 packages/plugin/src/tui.ts and specs/tui-plugins.md. + // The SDK creates only an empty local session; no prompt or model call is made. + await fs.writeFile(path.join(plugin, 'tui.mjs'), `import fs from 'node:fs';import path from 'node:path';import dependency from 'managed-fixture-dependency'; +export default {id:'managed-environment-tui',async tui(api,options){ + const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); + const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); + const cwd=api.state.path.directory;const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); + const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,dependency,resource,absoluteReference:api.state.config.agent.fixtureAbsolute.prompt,relativeReference:api.state.config.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; + fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); +}};`) + const absoluteRule = path.join(resources, 'absolute-rule.txt') + await fs.writeFile(absoluteRule, nonce, { mode: 0o600 }) + const options = { expectedNonce: nonce, candidateSha, runId } + await writeJson(path.join(xdg, 'opencode/opencode.json'), { plugin: [[plugin, options]], agent: { fixtureAbsolute: { prompt: `{file:${absoluteRule}}` } } }) + await writeJson(path.join(xdg, 'opencode/tui.json'), { plugin: [[plugin, options]] }) + for (const scenario of [...scenarios, 'preflight']) { + const cwd = path.join(workspace, scenario) + await fs.mkdir(path.join(cwd, '.managed-fixture'), { recursive: true, mode: 0o777 }) + await fs.chmod(cwd, 0o777) + await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) + await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) + await writeJson(path.join(cwd, 'opencode.json'), { agent: { fixtureRelative: { + prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } } }) + } + return { home, xdg, workspace, command } +} + +async function socketCreate(rig: ManagedRuntimeBrowserRig, requestId: string, mode: 'shell' | 'opencode', cwd: string, + capabilities?: Record) { + const ws = new WebSocket(rig.info.wsUrl) + const messages: any[] = [] + const closed = new Promise(resolve => ws.once('close', () => resolve())) + ws.on('message', bytes => messages.push(JSON.parse(bytes.toString()))) + try { + await new Promise((resolve, reject) => { ws.once('open', resolve); ws.once('error', reject) }) + ws.send(JSON.stringify({ type: 'hello', token: rig.info.token, ...(capabilities ? { capabilities } : {}) })) + await poll('authenticated ready', async () => messages.find(message => message.type === 'ready'), 30_000) + ws.send(JSON.stringify({ type: 'terminal.create', requestId, mode, cwd, cols: 100, rows: 30 })) + return await poll('terminal create response', async () => messages.find(message => + message.type === 'terminal.created' || message.type === 'terminal.error'), 120_000) + } finally { ws.close(); await closed } +} + +async function stageProof(rig: ManagedRuntimeBrowserRig, view: ManagedRuntimeView, nonce: string) { + const base = path.join(rig.runtime.scenarioPath(rig.supervisor.scenarioId), 'control') + for (const generation of await fs.readdir(path.join(base, 'provider-roots'))) { + const capability = await readJson(path.join(base, 'mcp-capabilities', `${generation}.json`)) + if (capability?.soulId !== view.soulId || (capability.incarnationId && capability.incarnationId !== view.incarnationId)) continue + const stageRoot = path.join(base, 'provider-roots', generation) + const context = await readJson(path.join(stageRoot, '.freshell/prepared-context.json')) + if (context?.preparedInputs?.generationId !== generation) continue + const manifest = await fs.readFile(path.join(stageRoot, '.freshell/input-manifest.json')) + assert.equal(digest(manifest), context.preparedInputs.manifest.sha256) + const selected: string[] = [] + async function walk(directory: string) { + for (const entry of await fs.readdir(directory, { withFileTypes: true })) { + const file = path.join(directory, entry.name) + if (entry.isDirectory()) await walk(file) + else if (entry.isFile() && entry.name === 'command' && (await fs.readFile(file, 'utf8')).includes(nonce)) selected.push(file) + } + } + await walk(stageRoot) + if (!selected.length) continue + for (const file of selected) assert.equal(requireMode(await fs.stat(file)), '0751') + return { observed: true, generation, manifestSha256: digest(manifest), executableMode: '0751', terminalId: view.terminalId, + soulId: view.soulId, incarnationId: view.incarnationId } + } + throw new Error('No current owned controller stage contains the selected executable') +} + +const requireMode = (metadata: { mode: number }) => (metadata.mode & 0o777).toString(8).padStart(4, '0') + +export async function runManagedEnvironmentTarget() { + assert.equal(process.env.FRESHELL_E2E_BACKEND, 'cloud', 'Managed acceptance requires configured Cloud') + assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Managed acceptance requires the robot identity') + const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') + const root = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-managed-environment-')) + const grants = fakeOnecliGrants(['opencode']) + const nonce = randomUUID() + let rig: ManagedRuntimeBrowserRig | undefined + let tunnel: ChildProcess | undefined, tunnelIdentity: Awaited> | undefined, tunnelJoin: Promise | undefined + let browser: ChildProcess | undefined, browserIdentity: Awaited> | undefined, browserJoin: Promise | undefined + const owners: InitialProviderVolumeAuthorization[] = [] + let capturedVolumes: ReturnType = [] + const proof: Record = { expectedNonce: nonce } + try { + const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') + const environment = { ...grants.serverEnv, HOME: home, XDG_CONFIG_HOME: xdg } + rig = new ManagedRuntimeBrowserRig(repoRoot, 2, { ...environment, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY: '1' }, + environment, 'release', { enabledProviders: ['opencode'] }, home, + { workspaceRoots: [workspace], bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) + const selected = await configure(root, nonce, rig.runtime.candidateSha, rig.runtime.runId) + await rig.start() + const health = await rig.runtime.adminOk(rig.supervisor, { method: 'health' }) + const dockerHost = process.env.DOCKER_HOST + assert(!dockerHost || dockerHost.startsWith('unix://'), 'Managed target requires the existing local Unix Docker endpoint') + const source = { installationId: health.data.installation_id, imageRef: rig.runtime.imageRef, + hostBinaryPath: rig.runtime.releaseHostBinary, binarySha256: digest(await fs.readFile(rig.runtime.releaseHostBinary)), + daemonId: await daemonId(dockerHost?.slice(7) || '/var/run/docker.sock') } + const authorize = (requestId: string) => owners.push(authorizeProviderParityRequest(rig!.runtime.broker, source, requestId)) + const before = await rig.inventory() + for (const capabilities of [undefined, { paneReconcileV1: true }] as (Record | undefined)[]) { + const refused = await socketCreate(rig, randomUUID(), 'shell', selected.workspace, capabilities) + assert.equal(refused.type, 'terminal.error') + assert(JSON.stringify(refused).includes('managed environment fixture denies host PTY')) + } + const fallback = await fetch(`${rig.info.baseUrl}/api/tabs`, { method: 'POST', headers: { 'x-auth-token': rig.info.token, + 'content-type': 'application/json' }, body: JSON.stringify({ mode: 'shell', cwd: selected.workspace }), signal: AbortSignal.timeout(30_000) }) + assert.equal(fallback.status, 400) + assert((await fallback.text()).includes('managed environment fixture denies host PTY')) + assert.deepEqual(await rig.inventory(), before) + proof.hostPtyDenials = { legacy: true, nonNegotiated: true, httpCliMcpFallback: true, ordinaryRows: 0 } + const preflightRequest = `managed-preflight-${randomUUID()}` + authorize(preflightRequest) + const created = await socketCreate(rig, preflightRequest, 'opencode', path.join(selected.workspace, 'preflight'), { managedRuntimeV1: true }) + assert.equal(created.type, 'terminal.created') + assert.equal(created.managed, true) + assert.equal(created.pid, null) + await poll('genuine managed preflight configuration', async () => readJson(path.join(selected.workspace, 'preflight/.managed-fixture/child-receipt.json'))) + const toolHome = path.join(root, 'tools') + await fs.mkdir(toolHome, { mode: 0o700 }) + const tool = path.join(toolHome, 'cloudflared') + const download = await fetch(`https://github.com/cloudflare/cloudflared/releases/download/${cloudflaredVersion}/cloudflared-linux-amd64`, { signal: AbortSignal.timeout(120_000) }) + assert(download.ok) + const bytes = Buffer.from(await download.arrayBuffer()) + assert.equal(digest(bytes), cloudflaredSha, 'Pinned cloudflared checksum mismatch') + await fs.writeFile(tool, bytes, { mode: 0o700 }) + const version = spawn(tool, ['--version'], { cwd: root, env: { ...process.env, HOME: toolHome } }) + const versionJoin = joined(version), versionIdentity = await processIdentity(version) + let versionText = ''; version.stdout.on('data', bytes => { versionText += bytes }) + assert.equal(await versionJoin, 0) + assert.match(versionText, /^cloudflared version 2026\.10\.0(?:\s|$)/) + proof.cloudflared = { version: cloudflaredVersion, sha256: cloudflaredSha, versionIdentity } + tunnel = spawn(tool, ['tunnel', '--url', `http://127.0.0.1:${rig.info.port}`], + { cwd: root, env: { ...process.env, HOME: toolHome, XDG_CONFIG_HOME: toolHome }, stdio: ['ignore', 'pipe', 'pipe'] }) + tunnelJoin = joined(tunnel); tunnelJoin.catch(() => {}) + tunnelIdentity = await processIdentity(tunnel) + let tunnelLog = ''; const tunnelOutput = (bytes: Buffer) => { tunnelLog += bytes.toString(); if (tunnelLog.length > 65536) tunnelLog = tunnelLog.slice(-65536) } + tunnel.stdout!.on('data', tunnelOutput); tunnel.stderr!.on('data', tunnelOutput) + const origin = await poll('pinned Quick Tunnel hostname', async () => { + assert.equal(tunnel!.exitCode, null, 'Owned tunnel exited before publication') + return tunnelLog.match(/https:\/\/[a-z0-9-]+\.trycloudflare\.com/)?.[0] + }, 120_000) + await poll('tunnel health', async () => { + const response = await fetch(`${origin}/api/health`, { signal: AbortSignal.timeout(15_000) }).catch(() => undefined) + return response?.ok ? true : undefined + }, 120_000) + const fixture = { candidateSha: rig.runtime.candidateSha, runId: rig.runtime.runId, + remoteWorkspacePath: selected.workspace, expectedNonce: nonce } + const cloudEnvironment: NodeJS.ProcessEnv = { ...process.env, FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', + FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: JSON.stringify(fixture), FRESHELL_E2E_TARGET_URL: origin, + FRESHELL_E2E_TARGET_WS_URL: origin.replace('https:', 'wss:') + '/ws', FRESHELL_E2E_TARGET_TOKEN: rig.info.token } + delete cloudEnvironment.FRESHELL_CODE_CHECKPOINT_FIXTURE + delete cloudEnvironment.FRESHELL_E2E_TARGET_HOME + resolvePairedExternalFixture(cloudEnvironment, true) + await writeJson(path.join(root, 'handoff.json'), { fixture, origin, token: rig.info.token, tunnelIdentity }) + browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--project=chromium-external-target', + 'test/e2e-browser/specs/managed-provider-environment-external.spec.ts'], { cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'inherit', 'inherit'] }) + browserJoin = joined(browser); browserJoin.catch(() => {}) + browserIdentity = await processIdentity(browser) + for (const scenario of scenarios) { + const cwd = path.join(selected.workspace, scenario) + const request = await poll(`Cloud ${scenario} picker request`, async () => { + assert.equal(browser!.exitCode, null, 'Cloud browser exited before completing its three cases') + return readJson(path.join(cwd, '.managed-fixture/request.json')) + }, 600_000) + for (const key of ['candidateSha', 'runId', 'remoteWorkspacePath', 'expectedNonce'] as const) assert.equal(request[key], fixture[key]) + assert.equal(request.scenario, scenario) + assert.equal(typeof request.createRequestId, 'string') + assert(request.createRequestId.length > 0) + authorize(request.createRequestId) + await writeJson(path.join(cwd, '.managed-fixture/authorized.json'), request) + if (scenario === 'healthy') { + const child = await poll('actual selected Bun child', async () => readJson(path.join(cwd, '.managed-fixture/child-receipt.json'))) + assert.equal(child.expectedNonce, nonce); assert.equal(child.candidateSha, fixture.candidateSha); assert.equal(child.runId, fixture.runId) + assert.equal(child.bun, true); assert.equal(child.opencodeVersion, '1.18.21'); assert.match(child.bunExecutable, /opencode/) + assert.equal(child.executableMode, '0711') + const view = await poll('created managed native identity', async () => (await rig!.inventory()).find(row => row.terminalId === child.terminalId && row.nativeSessionId === child.nativeSessionId && row.launchState === 'running')) + const stage = await stageProof(rig, view, nonce) + const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, + nativeSessionId: view.nativeSessionId, stage, child } + await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) + proof.healthy = receipt + } + } + assert.equal(await browserJoin, 0, 'Configured Cloud browser acceptance failed') + proof.cloudBrowser = { exit: 0, identity: browserIdentity, expectedCases: 3 } + const healthy = proof.healthy as any + const original = (await rig.inventory()).find(row => row.soulId === healthy.soulId) + assert(original?.containerId) + await fs.rename(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'), path.join(root, 'initial-child-receipt.json')) + rig.runtime.stopOwnedRuntimeExact(original.containerId) + const replacement = await poll('current-owner replacement incarnation', async () => (await rig!.inventory()).find(row => + row.soulId === original.soulId && row.incarnationId !== original.incarnationId && row.launchState === 'running'), 300_000) + const replacementChild = await poll('replacement Bun child', async () => readJson(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'))) + const replacementStage = await stageProof(rig, replacement, nonce) + assert.equal(replacement.nativeSessionId, original.nativeSessionId) + assert.equal(replacementChild.bun, true) + proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, + soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, child: replacementChild } + await writeJson(path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json'), proof) + console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.acceptance_passed', candidateSha: fixture.candidateSha, + runId: fixture.runId, browserCases: 3, receipt: path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json') })) + } finally { + await stopExact(browser, browserIdentity, browserJoin) + await stopExact(tunnel, tunnelIdentity, tunnelJoin) + if (rig) { + capturedVolumes = captureProviderParityVolumes(owners) + await rig.stop() + cleanupProviderParityVolumes(capturedVolumes) + } + await fs.rm(grants.directory, { recursive: true, force: true }) + await fs.rm(root, { recursive: true, force: true }) + } +} + +if (process.argv[1] && import.meta.url === new URL(`file://${path.resolve(process.argv[1])}`).href) { + runManagedEnvironmentTarget().catch(error => { + console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.acceptance_failed', error: error instanceof Error ? error.message : 'Unknown fixture failure' })) + process.exitCode = 1 + }) +} diff --git a/test/e2e-browser/helpers/managed-runtime.ts b/test/e2e-browser/helpers/managed-runtime.ts index 23c539ff7..9e21f9974 100644 --- a/test/e2e-browser/helpers/managed-runtime.ts +++ b/test/e2e-browser/helpers/managed-runtime.ts @@ -10,7 +10,7 @@ import { type ProviderQualificationReceiptV2, type ProviderQualificationRow, } from '../../../scripts/testing/provider-qualification-receipt.js' -import { RuntimeHarness, type SupervisorInstance } from '../../../scripts/testing/runtime-sandbox.js' +import { RuntimeHarness, type RuntimeFixtureInputs, type SupervisorInstance } from '../../../scripts/testing/runtime-sandbox.js' import { RustServer } from './rust-server.js' import type { E2eServerInfo as TestServerInfo } from './server-fixture-support.js' import type { QualifiableTerminalProvider } from '../../../scripts/testing/provider-qualification-selection.js' @@ -23,6 +23,8 @@ import { export const P2_OPENCODE_VERSION = '1.18.21' export const P2_OPENCODE_MODEL = 'openai/gpt-5.6-luna' +export type ManagedEnvironmentInputs = RuntimeFixtureInputs & { defaultCwd: string } + export type ManagedRuntimeView = { soulId: string incarnationId: string @@ -83,6 +85,7 @@ export class ManagedRuntimeBrowserRig { private readonly providerSettings: Record> private readonly freshAgentModes: string[] private readonly fixtureFreshAgentModes: string[] + private readonly defaultCwd: string private qualificationSupervisorBin: string | undefined constructor( @@ -98,9 +101,14 @@ export class ManagedRuntimeBrowserRig { fixtureFreshAgentModes?: string[] } = { enabledProviders: ['opencode'] }, webHomeDir?: string, + fixtureInputs?: ManagedEnvironmentInputs, ) { this.repoRoot = fs.realpathSync(repoRoot) - this.runtime = new RuntimeHarness(this.repoRoot, undefined, phase, webHomeDir, serverEnv.XDG_CONFIG_HOME) + this.runtime = new RuntimeHarness(this.repoRoot, undefined, phase, webHomeDir, serverEnv.XDG_CONFIG_HOME, fixtureInputs) + this.defaultCwd = fixtureInputs?.defaultCwd ?? this.repoRoot + if (fixtureInputs && !fixtureInputs.workspaceRoots.includes(this.defaultCwd)) { + throw new Error('managed environment default cwd must be an explicitly selected workspace') + } this.serverEnv = { ...serverEnv } this.webHomeDir = webHomeDir this.supervisorEnv = { ...supervisorEnv } @@ -181,7 +189,7 @@ export class ManagedRuntimeBrowserRig { await fsp.writeFile(path.join(freshell, 'config.json'), JSON.stringify({ version: 1, settings: { - defaultCwd: this.repoRoot, + defaultCwd: this.defaultCwd, codingCli: { enabledProviders: this.enabledProviders, providers: { diff --git a/test/e2e-browser/helpers/selection-nonvacuity.test.ts b/test/e2e-browser/helpers/selection-nonvacuity.test.ts index b0cd958b0..81dc5bf1f 100644 --- a/test/e2e-browser/helpers/selection-nonvacuity.test.ts +++ b/test/e2e-browser/helpers/selection-nonvacuity.test.ts @@ -1,253 +1,2 @@ -import { execFileSync, spawnSync } from 'node:child_process' -import { createRequire } from 'node:module' -import path from 'node:path' -import { describe, expect, it } from 'vitest' -import { LOCAL_ONLY_SPECS } from '../playwright.config.js' -import { CLOUD_SKIP_SPECS, CLOUD_SKIP_TITLES } from '../playwright.cloud.config.js' -import { createE2eServerHandle } from './external-target.js' -import { assertRustServerInfo, RustServer } from './rust-server.js' - -const require = createRequire(import.meta.url) -const projectRoot = path.resolve(import.meta.dirname, '../../..') -const browserRoot = path.resolve(import.meta.dirname, '..') -const playwrightCli = require.resolve('@playwright/test/cli') -const configLoader = require.resolve('playwright/lib/common/configLoader') -const playwrightConfig = path.join(browserRoot, 'playwright.config.ts') -const cloudConfig = path.join(browserRoot, 'playwright.cloud.config.ts') - -interface ResolvedProject { - name: string - testIgnore: Array<{ kind: 'regexp' | 'string'; source: string; flags?: string }> - testMatch: Array<{ kind: 'regexp' | 'string'; source: string; flags?: string }> -} - -function cleanEnvironment(overrides: Record = {}): NodeJS.ProcessEnv { - const env = { ...process.env } - delete env.CI - delete env.FRESHELL_SMOKE - for (const key of ['FRESHELL_E2E_EXTERNAL_FIXTURE_KIND', 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE', - 'FRESHELL_CODE_CHECKPOINT_FIXTURE', 'FRESHELL_E2E_TARGET_URL', 'FRESHELL_E2E_TARGET_WS_URL', - 'FRESHELL_E2E_TARGET_TOKEN', 'FRESHELL_E2E_TARGET_TIMEOUT_MS']) delete env[key] - return { ...env, ...overrides } -} - -function resolvedConfig(configPath: string, env: NodeJS.ProcessEnv): ResolvedProject[] { - const script = String.raw` -const { loadConfigFromFile } = require(process.argv[1]) -const normalize = (value) => (Array.isArray(value) ? value : [value]).map((pattern) => - pattern instanceof RegExp - ? { kind: 'regexp', source: pattern.source, flags: pattern.flags } - : { kind: 'string', source: String(pattern) }, -) -loadConfigFromFile(process.argv[2]).then((config) => { - process.stdout.write(JSON.stringify(config.projects.map(({ project }) => ({ - name: project.name, - testIgnore: normalize(project.testIgnore), - testMatch: normalize(project.testMatch), - })))) -}).catch((error) => { - console.error(error) - process.exitCode = 1 -}) -` - const output = execFileSync(process.execPath, ['-e', script, configLoader, configPath], { - cwd: projectRoot, - env, - encoding: 'utf8', - }) - return JSON.parse(output) as ResolvedProject[] -} - -function listedProjects( - env: NodeJS.ProcessEnv, - configPath = playwrightConfig, - selectors: string[] = [], -): { output: string; labels: string[]; tests: number; files: number } { - const output = execFileSync(process.execPath, [ - playwrightCli, - 'test', - '--config', configPath, - '--list', - ...selectors, - ], { - cwd: projectRoot, - env, - encoding: 'utf8', - }) - const total = output.match(/Total: (\d+) tests? in (\d+) files?/) - if (!total) throw new Error(`Playwright list output did not include a total:\n${output}`) - return { - output, - labels: [...new Set([...output.matchAll(/^\s*\[([^\]]+)] ›/gm)].map((match) => match[1]))], - tests: Number(total[1]), - files: Number(total[2]), - } -} - -// Playwright --list roster lines have the exact shape -// " [project] › :: › ". Extracting the spec -// token makes presence an exact-filename membership check — substring -// containment passes vacuously for colliding names -// (opencode-restart-recovery.spec.ts is contained inside -// freshopencode-restart-recovery.spec.ts, so a deleted shorter spec would -// still satisfy a raw toContain; delta-review round-1 F3). -function listedSpecFiles(output: string): Set<string> { - return new Set([...output.matchAll(/› (\S+\.spec\.ts):\d+:\d+/g)].map((match) => match[1])) -} - -describe('browser selection non-vacuity', () => { - it('discovers the managed external cases only through their paired dedicated Cloud project', () => { - const spec = 'managed-provider-environment-external.spec.ts' - const ordinary = listedProjects(cleanEnvironment(), cloudConfig) - expect(listedSpecFiles(ordinary.output).has(spec)).toBe(false) - expect(listedSpecFiles(ordinary.output).has('code-checkpoint-workspace-external.spec.ts')).toBe(false) - const fixture = JSON.stringify({ candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', - remoteWorkspacePath: '/tmp/selected-managed-fixture/workspace', expectedNonce: 'selection-fixture-nonce' }) - const paired = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', - FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: fixture, FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', - FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token' }) - const selected = listedProjects(paired, cloudConfig, ['--project=chromium-external-target', spec]) - expect(selected.labels).toEqual(['chromium-external-target']) - expect(selected.tests).toBe(3) - expect(selected.files).toBe(1) - expect([...listedSpecFiles(selected.output)]).toEqual([spec]) - // The source exists now: ordinary projects still cannot execute it. - const ordinaryExplicit = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, - '--list', '--project=chromium', spec], { cwd: projectRoot, env: paired, encoding: 'utf8' }) - expect(ordinaryExplicit.status).toBe(1) - }, 90_000) - - it.each(['missing', 'mismatched'] as const)('refuses external discovery with a %s fixture pair', pairing => { - const env = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', - FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token', - ...(pairing === 'mismatched' ? { FRESHELL_CODE_CHECKPOINT_FIXTURE: '{}' } : {}) }) - const listed = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, '--list', - '--project=chromium-external-target', 'managed-provider-environment-external.spec.ts'], - { cwd: projectRoot, env, encoding: 'utf8' }) - expect(listed.status).toBe(1) - expect(`${listed.stdout}\n${listed.stderr}`).toContain('matching fixture pair') - }) - - it('selects a non-vacuous Chromium lane and all CI application projects', () => { - const chromium = listedProjects(cleanEnvironment()) - expect(chromium.labels).toEqual(['chromium']) - expect(chromium.output).toContain('[chromium]') - expect(chromium.tests).toBeGreaterThanOrEqual(308) - expect(chromium.files).toBeGreaterThanOrEqual(86) - // Local coverage pin (kata 67jt): every CLOUD_SKIP_SPECS entry must - // remain listed on the base/local lane — cloud-skip never means "not - // covered"; it means "covered locally". A renamed or deleted spec that - // forgets the list fails here. Presence is exact-filename roster - // membership (listedSpecFiles), never substring containment: the - // opencode-/freshopencode-restart-recovery pair collides as substrings. - const baseSpecTokens = listedSpecFiles(chromium.output) - for (const spec of CLOUD_SKIP_SPECS) { - expect(baseSpecTokens, `base lane must still list ${spec}`).toContain(spec) - } - // Self-proving matcher (cheap synthetic negative): a roster holding only - // the longer colliding name must not count as the shorter one, and the - // real base lane lists both as distinct roster items. - expect(baseSpecTokens.has('freshopencode-restart-recovery.spec.ts')).toBe(true) - expect(baseSpecTokens.has('opencode-restart-recovery.spec.ts')).toBe(true) - const collisionOnlyRoster = listedSpecFiles( - ' [chromium] › freshopencode-restart-recovery.spec.ts:360:3 › Freshopencode restart recovery › resumes a session', - ) - expect(collisionOnlyRoster.has('freshopencode-restart-recovery.spec.ts')).toBe(true) - expect(collisionOnlyRoster.has('opencode-restart-recovery.spec.ts')).toBe(false) - // CLOUD_SKIP_TITLES non-vacuity: each grepInvert title must actually - // select a test in the base lane — else the cloud exclusion would be - // vacuously green after an edit (today's known hit: editor-pane.spec.ts - // "loads the editor lazily and requests a new JS asset after the click"). - expect(CLOUD_SKIP_TITLES.length).toBeGreaterThan(0) - for (const title of CLOUD_SKIP_TITLES) { - expect(chromium.output, `grepInvert source must exist in the base lane: ${title.source}`).toMatch(title) - } - - const ci = listedProjects(cleanEnvironment({ CI: '1' })) - expect(ci.labels).toEqual(['chromium', 'firefox', 'webkit']) - const retiredProjectNames = [`legacy${'-chromium'}`, `rust${'-chromium'}`] - expect(ci.output).not.toMatch(new RegExp(`${retiredProjectNames.join('|')}|Total:\\s*0 tests in`, 'i')) - }) - - it('creates Rust fixtures and selects the supported cloud projects', async () => { - const server = await createE2eServerHandle({}) - expect(server).toBeInstanceOf(RustServer) - expect(typeof server.start).toBe('function') - expect(typeof server.stop).toBe('function') - expect(() => server.info).toThrow('RustServer not started') - - expect(LOCAL_ONLY_SPECS).toContainEqual({ - spec: 'mcp-qa-smoke-rust.spec.ts', - classification: 'local-only-provider-binary', - selector: '--project=chromium test/e2e-browser/specs/mcp-qa-smoke-rust.spec.ts', - }) - expect(CLOUD_SKIP_SPECS).toContain('mcp-qa-smoke-rust.spec.ts') - expect(CLOUD_SKIP_SPECS).not.toContain('server-build-mismatch-rust.spec.ts') - expect(CLOUD_SKIP_SPECS).not.toContain('tabs-client-retire.spec.ts') - - const cloudProjects = resolvedConfig(cloudConfig, cleanEnvironment()) - expect(cloudProjects.map((project) => project.name)).toEqual(['chromium']) - expect(new Set(CLOUD_SKIP_SPECS).size).toBe(CLOUD_SKIP_SPECS.length) - for (const localOnly of LOCAL_ONLY_SPECS) expect(CLOUD_SKIP_SPECS).toContain(localOnly.spec) - - const cloud = listedProjects(cleanEnvironment(), cloudConfig) - expect(cloud.labels).toEqual(['chromium']) - expect(cloud.output).toContain('[chromium]') - expect(cloud.tests).toBeGreaterThan(0) - expect(cloud.files).toBeGreaterThan(0) - // Cloud exclusion pin (kata 67jt): no skip-listed spec may appear in - // the cloud selection at all — a pattern typo, a minimatch-semantics - // change, or a config refactor that drops the testIgnore mapping all - // fail here. - for (const spec of CLOUD_SKIP_SPECS) { - expect(cloud.output, `cloud lane must not list ${spec}`).not.toContain(spec) - } - // grepInvert honored at selection level: excluded titles absent from - // the cloud list while their spec files still run there. - for (const title of CLOUD_SKIP_TITLES) { - expect(cloud.output, `cloud lane must not list excluded title: ${title.source}`).not.toMatch(title) - } - - // Explicit positional file args are an INTERSECTION with testIgnore, - // never a bypass (Playwright 1.58.2 cliFileMatcher semantics): a mixed - // focused invocation must keep its cloud-legal member and silently drop - // the skip-listed one — so even a contaminated manifest or filter could - // not execute skip-listed specs under the cloud config. - const mixed = listedProjects(cleanEnvironment(), cloudConfig, [ - 'test/e2e-browser/specs/truly-idle-alerting.spec.ts', - 'test/e2e-browser/specs/host-stats-pane.spec.ts', - ]) - expect(mixed.output).toContain('host-stats-pane.spec.ts') - expect(mixed.output).not.toContain('truly-idle-alerting') - expect(mixed.files).toBe(1) - - // An ALL-skip focused invocation fails LOUDLY (exit 1, "No tests - // found") instead of silently reading as a green zero-test run — a - // vacuous focused cloud proof can never masquerade as coverage. - const allSkip = spawnSync(process.execPath, [ - playwrightCli, - 'test', - '--config', cloudConfig, - '--list', - 'test/e2e-browser/specs/truly-idle-alerting.spec.ts', - ], { cwd: projectRoot, env: cleanEnvironment(), encoding: 'utf8' }) - expect(allSkip.status).toBe(1) - expect(`${allSkip.stdout}\n${allSkip.stderr}`).toContain('No tests found') - - const migrated = listedProjects(cleanEnvironment(), cloudConfig, [ - 'test/e2e-browser/specs/server-build-mismatch-rust.spec.ts', - 'test/e2e-browser/specs/tabs-client-retire.spec.ts', - ]) - expect(migrated.tests).toBe(4) - expect(migrated.files).toBe(2) - expect(migrated.output).toContain('mismatched ready buildId reloads exactly once and converges') - expect(migrated.output).toContain('sentinel persists across a real navigation') - expect(migrated.output).toContain('a seeded sentinel suppresses a repeat mismatch (no reload)') - expect(migrated.output).toContain('closed browser client is removed from the Tabs UI through the unload retire API') - }) - - it('rejects a healthy response that does not identify Rust provenance', () => { - expect(() => assertRustServerInfo({ runtime: 'node', commit: 'abcdef0' })).toThrow(/runtime must be "rust"/) - expect(() => assertRustServerInfo({ runtime: 'rust' })).toThrow(/provenance/) - }) -}) +// Keep the former helper entrypoint linked to the default unit lane's actual discovery suite. +import '../../unit/tooling/testing/browser-selection.test.js' diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts new file mode 100644 index 000000000..f0a6be257 --- /dev/null +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -0,0 +1,148 @@ +import type { Page } from '@playwright/test' +import { resolvePairedExternalFixture } from '../../../scripts/testing/e2e-external-target.js' +import { test, expect } from '../helpers/fixtures.js' +import { openPanePicker } from '../helpers/pane-picker.js' +import { TestHarness } from '../helpers/test-harness.js' + +const paired = resolvePairedExternalFixture(process.env, true) +if (paired?.kind !== 'managed-environment') throw new Error('Managed environment requires its paired owned target') +const fixture = paired.fixture +const headers = { 'x-auth-token': paired.targetToken } + +async function json(page: Page, route: string) { + const response = await page.request.get(`${paired!.targetUrl}${route}`, { headers, timeout: 30_000 }) + expect(response.ok()).toBe(true) + return response.json() +} + +async function readReceipt(page: Page, selected: string): Promise<any | null> { + const response = await page.request.get(`${paired!.targetUrl}/api/files/read?path=${encodeURIComponent(selected)}`, + { headers, timeout: 30_000 }) + if (response.status() === 404) return null + expect(response.ok()).toBe(true) + return JSON.parse((await response.json()).content) +} + +async function activePane(page: Page) { + return page.evaluate(() => { + const state = window.__FRESHELL_TEST_HARNESS__!.getState() + const root = state.panes.layouts[state.tabs.activeTabId!] + if (root?.type !== 'leaf') throw new Error('Expected the real picker to have one leaf pane') + return { tabId: state.tabs.activeTabId!, paneId: root.id, content: root.content } + }) +} + +async function launch(page: Page, scenario: string) { + await page.goto(`${paired!.targetUrl}/?token=${encodeURIComponent(paired!.targetToken)}`) + await page.waitForFunction(() => Boolean(window.__FRESHELL_TEST_HARNESS__), undefined, { timeout: 60_000 }) + await new TestHarness(page).waitForConnection(120_000) + const info = await json(page, '/api/server-info') + expect(info.runtime).toBe('rust') + expect(info.commit).toBe(fixture.candidateSha) + expect(info.buildDirty).toBe(false) + const readiness = await json(page, '/api/runtime/readiness') + expect(readiness.readiness.initialScanState).toBe('complete') + expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getReceivedWsMessages?.() + .some((message: any) => message.type === 'ready' && message.managedRuntimeV1 === true))).toBe(true) + + const add = page.locator('[data-context="tab-add"]') + if (await add.isVisible()) await add.click() + const picker = await openPanePicker(page) + const before = await activePane(page) + // Authorize the actual UI-generated request before its normal create effect runs. + // The existing test harness suppresses effects only for this picker leaf. + await page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.setTerminalNetworkEffectsSuppressed(id, true), before.paneId) + await picker.getByRole('button', { name: /^OpenCode$/ }).click() + const cwd = `${fixture.remoteWorkspacePath}/${scenario}` + const directory = page.getByRole('combobox', { name: /Starting directory for OpenCode/i }) + await directory.fill(cwd) + await directory.press('Enter') + await expect.poll(async () => (await activePane(page)).content.kind).toBe('terminal') + const selected = await activePane(page) + if (selected.content.kind !== 'terminal') throw new Error('The OpenCode picker did not create terminal content') + expect(selected.paneId).toBe(before.paneId) + expect(selected.content.mode).toBe('opencode') + expect(selected.content.createRequestId).toBeTruthy() + const request = { ...fixture, scenario, createRequestId: selected.content.createRequestId, paneId: selected.paneId } + const written = await page.request.post(`${paired!.targetUrl}/api/files/write`, { headers, + data: { path: `${cwd}/.managed-fixture/request.json`, content: JSON.stringify(request) } }) + expect(written.ok()).toBe(true) + await expect.poll(async () => readReceipt(page, `${cwd}/.managed-fixture/authorized.json`), + { timeout: 120_000 }).toMatchObject({ candidateSha: fixture.candidateSha, runId: fixture.runId, + expectedNonce: fixture.expectedNonce, createRequestId: selected.content.createRequestId }) + await page.evaluate(id => { + const harness = window.__FRESHELL_TEST_HARNESS__! + harness.setTerminalNetworkEffectsSuppressed(id, false) + const state = harness.getState() + const tabId = state.tabs.activeTabId! + const root = state.panes.layouts[tabId] + if (root?.type !== 'leaf' || root.id !== id) throw new Error('The selected picker leaf changed') + harness.dispatch({ type: 'panes/updatePaneContent', payload: { tabId, paneId: id, content: { ...root.content } } }) + }, selected.paneId) + await expect.poll(async () => { + const pane = await activePane(page) + return pane.content.kind === 'terminal' ? pane.content.terminalId : null + }, { timeout: 120_000 }).toBeTruthy() + return { ...(await activePane(page)), cwd, createRequestId: selected.content.createRequestId } +} + +test('executes selected resources with Bun in both managed stages and retains identity on reconnect', async ({ page }) => { + test.setTimeout(360_000) + const pane = await launch(page, 'healthy') + if (pane.content.kind !== 'terminal') throw new Error('Expected the real OpenCode terminal') + const terminalId = pane.content.terminalId! + const marker = `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` + await expect.poll(() => page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.getTerminalBuffer(id), terminalId), + { timeout: 120_000 }).toContain(marker) + let receipt: any + await expect.poll(async () => { + receipt = await readReceipt(page, `${pane.cwd}/.managed-fixture/target-receipt.json`) + return receipt + }, { timeout: 120_000 }).toMatchObject({ candidateSha: fixture.candidateSha, runId: fixture.runId, + expectedNonce: fixture.expectedNonce, terminalId, stage: { observed: true }, child: { + terminalId, bun: true, dependency: fixture.expectedNonce, resource: fixture.expectedNonce, + absoluteReference: fixture.expectedNonce, relativeReference: fixture.expectedNonce, executableMode: '0711', + } }) + expect(receipt.soulId).toBeTruthy() + expect(receipt.incarnationId).toBeTruthy() + expect(receipt.nativeSessionId).toBeTruthy() + const beforeCreates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length) + await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.forceDisconnect()) + await new TestHarness(page).waitForConnection(120_000) + await expect.poll(async () => (await activePane(page)).content).toMatchObject({ terminalId, + sessionRef: { provider: 'opencode', sessionId: receipt.nativeSessionId } }) + await expect.poll(() => page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.getTerminalBuffer(id), terminalId)).toContain(marker) + expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length)).toBe(beforeCreates) + const rows = await json(page, '/api/terminals') + expect(rows.find((row: any) => row.terminalId === terminalId)).toMatchObject({ terminalId, + soulId: receipt.soulId, incarnationId: receipt.incarnationId, runtimeStatus: 'running' }) +}) + +for (const scenario of ['missing-reference', 'startup-before-identity']) { + test(`${scenario} retains an actionable blocked result without a replacement conversation`, async ({ page }) => { + test.setTimeout(300_000) + const pane = await launch(page, scenario) + if (pane.content.kind !== 'terminal') throw new Error('Expected the retained managed terminal') + const terminalId = pane.content.terminalId! + let row: any + await expect.poll(async () => { + row = (await json(page, '/api/terminals')).find((item: any) => item.terminalId === terminalId) + return row + }, { timeout: 120_000 }).toMatchObject({ terminalId, runtimeStatus: 'blocked', recoverySummary: { recoveryState: 'blocked' } }) + expect(row.recoverySummary.reason).toBeTruthy() + expect(row.sessionRef).toBeUndefined() + const diagnostic = page.locator('[data-testid="terminal-work-area"]') + await expect(diagnostic).toContainText(/blocked|unable|failed/i) + await expect(diagnostic).toContainText(scenario === 'missing-reference' ? /missing-selected/i : /startup-fixture/i) + const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length) + await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.forceDisconnect()) + await new TestHarness(page).waitForConnection(120_000) + expect((await activePane(page)).content).toMatchObject({ terminalId, status: 'error' }) + expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) + expect((await json(page, '/api/terminals')).filter((item: any) => item.cwd === pane.cwd)).toHaveLength(1) + }) +} diff --git a/test/unit/tooling/testing/browser-selection.test.ts b/test/unit/tooling/testing/browser-selection.test.ts index 60665fb65..e18135be9 100644 --- a/test/unit/tooling/testing/browser-selection.test.ts +++ b/test/unit/tooling/testing/browser-selection.test.ts @@ -1,2 +1,253 @@ -// The default unit lane excludes browser helpers; execute their real CLI discovery checks here. -import '../../../e2e-browser/helpers/selection-nonvacuity.test.js' +import { execFileSync, spawnSync } from 'node:child_process' +import { createRequire } from 'node:module' +import path from 'node:path' +import { describe, expect, it } from 'vitest' +import { LOCAL_ONLY_SPECS } from '../../../e2e-browser/playwright.config.js' +import { CLOUD_SKIP_SPECS, CLOUD_SKIP_TITLES } from '../../../e2e-browser/playwright.cloud.config.js' +import { createE2eServerHandle } from '../../../e2e-browser/helpers/external-target.js' +import { assertRustServerInfo, RustServer } from '../../../e2e-browser/helpers/rust-server.js' + +const require = createRequire(import.meta.url) +const projectRoot = path.resolve(import.meta.dirname, '../../../..') +const browserRoot = path.join(projectRoot, 'test/e2e-browser') +const playwrightCli = require.resolve('@playwright/test/cli') +const configLoader = require.resolve('playwright/lib/common/configLoader') +const playwrightConfig = path.join(browserRoot, 'playwright.config.ts') +const cloudConfig = path.join(browserRoot, 'playwright.cloud.config.ts') + +interface ResolvedProject { + name: string + testIgnore: Array<{ kind: 'regexp' | 'string'; source: string; flags?: string }> + testMatch: Array<{ kind: 'regexp' | 'string'; source: string; flags?: string }> +} + +function cleanEnvironment(overrides: Record<string, string> = {}): NodeJS.ProcessEnv { + const env = { ...process.env } + delete env.CI + delete env.FRESHELL_SMOKE + for (const key of ['FRESHELL_E2E_EXTERNAL_FIXTURE_KIND', 'FRESHELL_MANAGED_ENVIRONMENT_FIXTURE', + 'FRESHELL_CODE_CHECKPOINT_FIXTURE', 'FRESHELL_E2E_TARGET_URL', 'FRESHELL_E2E_TARGET_WS_URL', + 'FRESHELL_E2E_TARGET_TOKEN', 'FRESHELL_E2E_TARGET_TIMEOUT_MS']) delete env[key] + return { ...env, ...overrides } +} + +function resolvedConfig(configPath: string, env: NodeJS.ProcessEnv): ResolvedProject[] { + const script = String.raw` +const { loadConfigFromFile } = require(process.argv[1]) +const normalize = (value) => (Array.isArray(value) ? value : [value]).map((pattern) => + pattern instanceof RegExp + ? { kind: 'regexp', source: pattern.source, flags: pattern.flags } + : { kind: 'string', source: String(pattern) }, +) +loadConfigFromFile(process.argv[2]).then((config) => { + process.stdout.write(JSON.stringify(config.projects.map(({ project }) => ({ + name: project.name, + testIgnore: normalize(project.testIgnore), + testMatch: normalize(project.testMatch), + })))) +}).catch((error) => { + console.error(error) + process.exitCode = 1 +}) +` + const output = execFileSync(process.execPath, ['-e', script, configLoader, configPath], { + cwd: projectRoot, + env, + encoding: 'utf8', + }) + return JSON.parse(output) as ResolvedProject[] +} + +function listedProjects( + env: NodeJS.ProcessEnv, + configPath = playwrightConfig, + selectors: string[] = [], +): { output: string; labels: string[]; tests: number; files: number } { + const output = execFileSync(process.execPath, [ + playwrightCli, + 'test', + '--config', configPath, + '--list', + ...selectors, + ], { + cwd: projectRoot, + env, + encoding: 'utf8', + }) + const total = output.match(/Total: (\d+) tests? in (\d+) files?/) + if (!total) throw new Error(`Playwright list output did not include a total:\n${output}`) + return { + output, + labels: [...new Set([...output.matchAll(/^\s*\[([^\]]+)] ›/gm)].map((match) => match[1]))], + tests: Number(total[1]), + files: Number(total[2]), + } +} + +// Playwright --list roster lines have the exact shape +// " [project] › <specFile>:<line>:<col> › <title>". Extracting the spec +// token makes presence an exact-filename membership check — substring +// containment passes vacuously for colliding names +// (opencode-restart-recovery.spec.ts is contained inside +// freshopencode-restart-recovery.spec.ts, so a deleted shorter spec would +// still satisfy a raw toContain; delta-review round-1 F3). +function listedSpecFiles(output: string): Set<string> { + return new Set([...output.matchAll(/› (\S+\.spec\.ts):\d+:\d+/g)].map((match) => match[1])) +} + +describe('browser selection non-vacuity', () => { + it('discovers the managed external cases only through their paired dedicated Cloud project', () => { + const spec = 'managed-provider-environment-external.spec.ts' + const ordinary = listedProjects(cleanEnvironment(), cloudConfig) + expect(listedSpecFiles(ordinary.output).has(spec)).toBe(false) + expect(listedSpecFiles(ordinary.output).has('code-checkpoint-workspace-external.spec.ts')).toBe(false) + const fixture = JSON.stringify({ candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', + remoteWorkspacePath: '/tmp/selected-managed-fixture/workspace', expectedNonce: 'selection-fixture-nonce' }) + const paired = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', + FRESHELL_MANAGED_ENVIRONMENT_FIXTURE: fixture, FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', + FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token' }) + const selected = listedProjects(paired, cloudConfig, ['--project=chromium-external-target', spec]) + expect(selected.labels).toEqual(['chromium-external-target']) + expect(selected.tests).toBe(3) + expect(selected.files).toBe(1) + expect([...listedSpecFiles(selected.output)]).toEqual([spec]) + // The source exists now: ordinary projects still cannot execute it. + const ordinaryExplicit = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, + '--list', '--project=chromium', spec], { cwd: projectRoot, env: paired, encoding: 'utf8' }) + expect(ordinaryExplicit.status).toBe(1) + }, 90_000) + + it.each(['missing', 'mismatched'] as const)('refuses external discovery with a %s fixture pair', pairing => { + const env = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', + FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token', + ...(pairing === 'mismatched' ? { FRESHELL_CODE_CHECKPOINT_FIXTURE: '{}' } : {}) }) + const listed = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, '--list', + '--project=chromium-external-target', 'managed-provider-environment-external.spec.ts'], + { cwd: projectRoot, env, encoding: 'utf8' }) + expect(listed.status).toBe(1) + expect(`${listed.stdout}\n${listed.stderr}`).toContain('matching fixture pair') + }) + + it('selects a non-vacuous Chromium lane and all CI application projects', () => { + const chromium = listedProjects(cleanEnvironment()) + expect(chromium.labels).toEqual(['chromium']) + expect(chromium.output).toContain('[chromium]') + expect(chromium.tests).toBeGreaterThanOrEqual(308) + expect(chromium.files).toBeGreaterThanOrEqual(86) + // Local coverage pin (kata 67jt): every CLOUD_SKIP_SPECS entry must + // remain listed on the base/local lane — cloud-skip never means "not + // covered"; it means "covered locally". A renamed or deleted spec that + // forgets the list fails here. Presence is exact-filename roster + // membership (listedSpecFiles), never substring containment: the + // opencode-/freshopencode-restart-recovery pair collides as substrings. + const baseSpecTokens = listedSpecFiles(chromium.output) + for (const spec of CLOUD_SKIP_SPECS) { + expect(baseSpecTokens, `base lane must still list ${spec}`).toContain(spec) + } + // Self-proving matcher (cheap synthetic negative): a roster holding only + // the longer colliding name must not count as the shorter one, and the + // real base lane lists both as distinct roster items. + expect(baseSpecTokens.has('freshopencode-restart-recovery.spec.ts')).toBe(true) + expect(baseSpecTokens.has('opencode-restart-recovery.spec.ts')).toBe(true) + const collisionOnlyRoster = listedSpecFiles( + ' [chromium] › freshopencode-restart-recovery.spec.ts:360:3 › Freshopencode restart recovery › resumes a session', + ) + expect(collisionOnlyRoster.has('freshopencode-restart-recovery.spec.ts')).toBe(true) + expect(collisionOnlyRoster.has('opencode-restart-recovery.spec.ts')).toBe(false) + // CLOUD_SKIP_TITLES non-vacuity: each grepInvert title must actually + // select a test in the base lane — else the cloud exclusion would be + // vacuously green after an edit (today's known hit: editor-pane.spec.ts + // "loads the editor lazily and requests a new JS asset after the click"). + expect(CLOUD_SKIP_TITLES.length).toBeGreaterThan(0) + for (const title of CLOUD_SKIP_TITLES) { + expect(chromium.output, `grepInvert source must exist in the base lane: ${title.source}`).toMatch(title) + } + + const ci = listedProjects(cleanEnvironment({ CI: '1' })) + expect(ci.labels).toEqual(['chromium', 'firefox', 'webkit']) + const retiredProjectNames = [`legacy${'-chromium'}`, `rust${'-chromium'}`] + expect(ci.output).not.toMatch(new RegExp(`${retiredProjectNames.join('|')}|Total:\\s*0 tests in`, 'i')) + }) + + it('creates Rust fixtures and selects the supported cloud projects', async () => { + const server = await createE2eServerHandle({}) + expect(server).toBeInstanceOf(RustServer) + expect(typeof server.start).toBe('function') + expect(typeof server.stop).toBe('function') + expect(() => server.info).toThrow('RustServer not started') + + expect(LOCAL_ONLY_SPECS).toContainEqual({ + spec: 'mcp-qa-smoke-rust.spec.ts', + classification: 'local-only-provider-binary', + selector: '--project=chromium test/e2e-browser/specs/mcp-qa-smoke-rust.spec.ts', + }) + expect(CLOUD_SKIP_SPECS).toContain('mcp-qa-smoke-rust.spec.ts') + expect(CLOUD_SKIP_SPECS).not.toContain('server-build-mismatch-rust.spec.ts') + expect(CLOUD_SKIP_SPECS).not.toContain('tabs-client-retire.spec.ts') + + const cloudProjects = resolvedConfig(cloudConfig, cleanEnvironment()) + expect(cloudProjects.map((project) => project.name)).toEqual(['chromium']) + expect(new Set(CLOUD_SKIP_SPECS).size).toBe(CLOUD_SKIP_SPECS.length) + for (const localOnly of LOCAL_ONLY_SPECS) expect(CLOUD_SKIP_SPECS).toContain(localOnly.spec) + + const cloud = listedProjects(cleanEnvironment(), cloudConfig) + expect(cloud.labels).toEqual(['chromium']) + expect(cloud.output).toContain('[chromium]') + expect(cloud.tests).toBeGreaterThan(0) + expect(cloud.files).toBeGreaterThan(0) + // Cloud exclusion pin (kata 67jt): no skip-listed spec may appear in + // the cloud selection at all — a pattern typo, a minimatch-semantics + // change, or a config refactor that drops the testIgnore mapping all + // fail here. + for (const spec of CLOUD_SKIP_SPECS) { + expect(cloud.output, `cloud lane must not list ${spec}`).not.toContain(spec) + } + // grepInvert honored at selection level: excluded titles absent from + // the cloud list while their spec files still run there. + for (const title of CLOUD_SKIP_TITLES) { + expect(cloud.output, `cloud lane must not list excluded title: ${title.source}`).not.toMatch(title) + } + + // Explicit positional file args are an INTERSECTION with testIgnore, + // never a bypass (Playwright 1.58.2 cliFileMatcher semantics): a mixed + // focused invocation must keep its cloud-legal member and silently drop + // the skip-listed one — so even a contaminated manifest or filter could + // not execute skip-listed specs under the cloud config. + const mixed = listedProjects(cleanEnvironment(), cloudConfig, [ + 'test/e2e-browser/specs/truly-idle-alerting.spec.ts', + 'test/e2e-browser/specs/host-stats-pane.spec.ts', + ]) + expect(mixed.output).toContain('host-stats-pane.spec.ts') + expect(mixed.output).not.toContain('truly-idle-alerting') + expect(mixed.files).toBe(1) + + // An ALL-skip focused invocation fails LOUDLY (exit 1, "No tests + // found") instead of silently reading as a green zero-test run — a + // vacuous focused cloud proof can never masquerade as coverage. + const allSkip = spawnSync(process.execPath, [ + playwrightCli, + 'test', + '--config', cloudConfig, + '--list', + 'test/e2e-browser/specs/truly-idle-alerting.spec.ts', + ], { cwd: projectRoot, env: cleanEnvironment(), encoding: 'utf8' }) + expect(allSkip.status).toBe(1) + expect(`${allSkip.stdout}\n${allSkip.stderr}`).toContain('No tests found') + + const migrated = listedProjects(cleanEnvironment(), cloudConfig, [ + 'test/e2e-browser/specs/server-build-mismatch-rust.spec.ts', + 'test/e2e-browser/specs/tabs-client-retire.spec.ts', + ]) + expect(migrated.tests).toBe(4) + expect(migrated.files).toBe(2) + expect(migrated.output).toContain('mismatched ready buildId reloads exactly once and converges') + expect(migrated.output).toContain('sentinel persists across a real navigation') + expect(migrated.output).toContain('a seeded sentinel suppresses a repeat mismatch (no reload)') + expect(migrated.output).toContain('closed browser client is removed from the Tabs UI through the unload retire API') + }) + + it('rejects a healthy response that does not identify Rust provenance', () => { + expect(() => assertRustServerInfo({ runtime: 'node', commit: 'abcdef0' })).toThrow(/runtime must be "rust"/) + expect(() => assertRustServerInfo({ runtime: 'rust' })).toThrow(/provenance/) + }) +}) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index de45e1ce7..3c075a610 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -5,12 +5,17 @@ import os from 'node:os' import path from 'node:path' import { describe, expect, it, vi } from 'vitest' -const fakeDocker = vi.hoisted(() => ({ settled: false, commands: [] as string[][] })) +const fakeDocker = vi.hoisted(() => ({ settled: false, commands: [] as string[][], supervisorCommands: [] as string[][] })) vi.mock('node:child_process', () => { const api = { spawn: () => { throw new Error('unexpected spawned process in isolated harness fixture') }, execFileSync: (program: string, args: string[]) => { if (program === 'git' && args.join(' ') === 'rev-parse HEAD') return 'cleanup-fixture-commit\n' + if (program === 'docker' && args[0] === 'volume' && args[1] === 'create') return `${args[2]}\n` + if (program === 'docker' && args[0] === 'run' && args.includes('/var/lib/freshell-supervisor')) { + fakeDocker.supervisorCommands.push(args) + return `${'f'.repeat(64)}\n` + } throw new Error('unexpected executable in isolated cleanup fixture') }, spawnSync: (program: string, args: string[]) => { @@ -193,6 +198,54 @@ it.each(['claude', 'opencode'] as const)('qualifies original isolated HOME alias }) describe('closed managed environment fixture inputs', () => { + it('dispatches the selected private HOME with workspace and bootstrap into its supervisor', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-supervisor-input-')) + const repository = path.join(directory, 'repository'), home = path.join(directory, 'home') + const xdg = path.join(home, 'fixture-xdg'), workspace = path.join(directory, 'workspace') + const grant = path.join(directory, 'grant.env'), socket = path.join(directory, 'daemon.sock') + const previousDockerHost = process.env.DOCKER_HOST + const daemon = http.createServer((_request, response) => response.writeHead(500).end()) + let harness: RuntimeHarness | undefined + try { + await fs.mkdir(path.join(repository, 'test/runtime'), { recursive: true }) + await fs.writeFile(path.join(repository, 'test/runtime/gate-manifest.json'), '{}') + await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true }) + await fs.chmod(home, 0o700) + await fs.mkdir(workspace) + await fs.writeFile(grant, 'synthetic-closed-grant', { mode: 0o600 }) + const host = path.join(directory, 'host') + await fs.writeFile(host, 'owned fixture host artifact') + await new Promise<void>(resolve => daemon.listen(socket, resolve)) + process.env.DOCKER_HOST = `unix://${socket}` + const rig = new ManagedRuntimeBrowserRig(repository, 2, { HOME: home, XDG_CONFIG_HOME: xdg }, + { HOME: home, XDG_CONFIG_HOME: xdg }, 'test', { enabledProviders: ['opencode'] }, home, + { workspaceRoots: [workspace], bootstrapFiles: [grant], defaultCwd: workspace }) + harness = rig.runtime + useOwnedHarnessArtifacts(harness, host) + await harness.prepare() + fakeDocker.supervisorCommands = [] + await harness.startSupervisor({ scenarioId: 'closed-home', waitForHealth: false, + env: { HOME: home, XDG_CONFIG_HOME: xdg } }) + expect(fakeDocker.supervisorCommands).toHaveLength(1) + const dispatched = fakeDocker.supervisorCommands[0] + expect(dispatched).toContain(`${workspace}:${workspace}:ro`) + expect(dispatched).toContain(`${grant}:${grant}:ro`) + expect(dispatched).toContain(`HOME=${home}`) + expect(dispatched).toContain(`XDG_CONFIG_HOME=${xdg}`) + expect(dispatched).toContain(`${home}:${home}:ro`) + expect(dispatched).not.toContain(`${repository}:${repository}:ro`) + expect((await fs.stat(home)).mode & 0o777).toBe(0o700) + } finally { + await harness?.broker?.close() + await new Promise<void>(resolve => daemon.close(() => resolve())) + if (previousDockerHost === undefined) delete process.env.DOCKER_HOST + else process.env.DOCKER_HOST = previousDockerHost + if (harness) await fs.rm(harness.testRoot, { recursive: true, force: true }) + await fs.rm(directory, { recursive: true, force: true }) + vi.restoreAllMocks() + } + }) + it.each(['selected-workspace', 'selected-bootstrap', 'ambient-bootstrap'] as const)( 'routes only the explicitly selected fixture inputs: %s', async selection => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-input-')) From 67afe9a0c1845793ba243c480f7e1f9f2d54acd9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 09:00:22 -0700 Subject: [PATCH 255/493] fix: mount selected synthetic HOME for managed supervisor --- scripts/testing/runtime-sandbox.ts | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index 212d26eae..9f77506ff 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -469,6 +469,10 @@ export class RuntimeHarness { '-v', `${volumeName}:/var/lib/freshell-supervisor:rw`, ] if (this.phase >= 2) { + if (this.fixtureInputs && this.providerFixtureHome !== undefined) { + const home = validateOwnedFixturePath(this.providerFixtureHome, 'directory', 'provider fixture root') + args.push('-v', `${home}:${home}:ro`) + } for (const root of this.phase2WorkspaceRoots()) { if (root === this.testRoot || root === this.buildDir) continue args.push('-v', `${root}:${root}:ro`) From b2a7b69da0e1fee5c5437ee07a42a35a63ea47f6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 09:22:11 -0700 Subject: [PATCH 256/493] test: bind managed acceptance to protocol and owned cleanup --- .../managed-provider-environment-target.ts | 209 ++++++++++++++---- ...aged-provider-environment-external.spec.ts | 2 +- 2 files changed, 169 insertions(+), 42 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index f4b48f849..3dbee8040 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -5,7 +5,7 @@ import fs from 'node:fs/promises' import http from 'node:http' import os from 'node:os' import path from 'node:path' -import { fileURLToPath } from 'node:url' +import { fileURLToPath, pathToFileURL } from 'node:url' import WebSocket from 'ws' import { ManagedRuntimeBrowserRig, type ManagedRuntimeView } from '../../test/e2e-browser/helpers/managed-runtime.js' import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../test/helpers/provider-parity-broker.js' @@ -38,31 +38,71 @@ async function readJson(selected: string): Promise<any | undefined> { catch (error) { if ((error as NodeJS.ErrnoException).code === 'ENOENT') return; throw error } } -async function processIdentity(child: ChildProcess) { +async function processIdentity(child: { pid?: number }) { assert(child.pid) const raw = await fs.readFile(`/proc/${child.pid}/stat`, 'utf8') const fields = raw.slice(raw.lastIndexOf(')') + 2).trim().split(/\s+/) return { pid: child.pid, parentPid: Number(fields[1]), birthTicks: fields[19], cwd: await fs.readlink(`/proc/${child.pid}/cwd`) } } -function joined(child: ChildProcess): Promise<number> { +type NativeExit = { code: number | null; signal: NodeJS.Signals | null } + +function joined(child: ChildProcess): Promise<NativeExit> { return new Promise((resolve, reject) => { child.once('error', reject) - child.once('exit', (code, signal) => code === null ? reject(new Error(`Owned child exited with ${signal}`)) : resolve(code)) + child.once('exit', (code, signal) => resolve({ code, signal })) }) } +async function bounded<T>(completion: Promise<T>, milliseconds: number, description: string): Promise<T> { + let timer: ReturnType<typeof setTimeout> | undefined + try { + return await Promise.race([completion, new Promise<never>((_resolve, reject) => { + timer = setTimeout(() => reject(new Error(`Managed environment timed out: ${description}`)), milliseconds) + })]) + } finally { clearTimeout(timer) } +} + async function stopExact(child: ChildProcess | undefined, identity: Awaited<ReturnType<typeof processIdentity>> | undefined, - completion: Promise<number> | undefined) { - if (!child || !identity || !completion) return + completion: Promise<NativeExit> | undefined) { + if (!child || !completion) return if (child.exitCode === null && child.signalCode === null) { try { + assert(identity, 'Live owned child has no captured identity; retain its fixture') const current = await processIdentity(child) assert.deepEqual(current, identity, 'Owned child identity changed; retain it') child.kill('SIGTERM') } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + try { return await bounded(completion, 15_000, 'owned child graceful join') } + catch (error) { + if (child.exitCode !== null || child.signalCode !== null) throw error + assert.deepEqual(await processIdentity(child), identity, 'Owned child identity changed before escalation; retain it') + child.kill('SIGKILL') + } + } + return bounded(completion, 15_000, 'owned child final join') +} + +async function terminalDirectory(rig: ManagedRuntimeBrowserRig): Promise<any[]> { + const response = await fetch(`${rig.info.baseUrl}/api/terminals`, { + headers: { 'x-auth-token': rig.info.token }, signal: AbortSignal.timeout(30_000) }) + assert(response.ok, 'Owned terminal directory request failed') + const rows = await response.json() + assert(Array.isArray(rows), 'Owned terminal directory must return its real rows') + return rows +} + +async function webChildren(rig: ManagedRuntimeBrowserRig): Promise<number[]> { + const { pid } = rig.web.processEvidence() + const children = new Set<number>() + for (const thread of await fs.readdir(`/proc/${pid}/task`)) { + try { + const selected = await fs.readFile(`/proc/${pid}/task/${thread}/children`, 'utf8') + for (const child of selected.trim().split(/\s+/).filter(Boolean)) children.add(Number(child)) + } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } } - await completion.catch(error => { if (!child.signalCode) throw error }) + assert.equal(rig.web.processEvidence().pid, pid, 'Owned scratch web server changed during child observation') + return [...children].sort((left, right) => left - right) } async function daemonId(socketPath: string): Promise<string> { @@ -102,8 +142,8 @@ async function configure(root: string, nonce: string, candidateSha: string, runI export default {id:'managed-environment-tui',async tui(api,options){ const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); - const cwd=api.state.path.directory;const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); - const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,dependency,resource,absoluteReference:api.state.config.agent.fixtureAbsolute.prompt,relativeReference:api.state.config.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; + const cwd=process.cwd();const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); + const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) const absoluteRule = path.join(resources, 'absolute-rule.txt') @@ -130,13 +170,19 @@ async function socketCreate(rig: ManagedRuntimeBrowserRig, requestId: string, mo const closed = new Promise<void>(resolve => ws.once('close', () => resolve())) ws.on('message', bytes => messages.push(JSON.parse(bytes.toString()))) try { - await new Promise<void>((resolve, reject) => { ws.once('open', resolve); ws.once('error', reject) }) + await bounded(new Promise<void>((resolve, reject) => { ws.once('open', resolve); ws.once('error', reject) }), + 30_000, 'owned WebSocket open') ws.send(JSON.stringify({ type: 'hello', token: rig.info.token, ...(capabilities ? { capabilities } : {}) })) - await poll('authenticated ready', async () => messages.find(message => message.type === 'ready'), 30_000) + const ready = await poll('authenticated ready', async () => messages.find(message => message.type === 'ready'), 30_000) + if (capabilities?.managedRuntimeV1) assert.equal(ready.capabilities?.managedRuntimeV1, true) ws.send(JSON.stringify({ type: 'terminal.create', requestId, mode, cwd, cols: 100, rows: 30 })) return await poll('terminal create response', async () => messages.find(message => - message.type === 'terminal.created' || message.type === 'terminal.error'), 120_000) - } finally { ws.close(); await closed } + message.requestId === requestId && (message.type === 'terminal.created' || message.type === 'error')), 120_000) + } finally { + if (ws.readyState !== WebSocket.CLOSED) ws.close() + try { await bounded(closed, 5_000, 'owned WebSocket close') } + catch { ws.terminate(); await bounded(closed, 5_000, 'owned WebSocket terminate join') } + } } async function stageProof(rig: ManagedRuntimeBrowserRig, view: ManagedRuntimeView, nonce: string) { @@ -150,11 +196,17 @@ async function stageProof(rig: ManagedRuntimeBrowserRig, view: ManagedRuntimeVie const manifest = await fs.readFile(path.join(stageRoot, '.freshell/input-manifest.json')) assert.equal(digest(manifest), context.preparedInputs.manifest.sha256) const selected: string[] = [] - async function walk(directory: string) { + let entries = 0 + async function walk(directory: string, depth = 0) { + assert(depth <= 64, 'Owned controller-stage observation exceeded its depth bound') for (const entry of await fs.readdir(directory, { withFileTypes: true })) { + assert(++entries <= 100_000, 'Owned controller-stage observation exceeded its entry bound') const file = path.join(directory, entry.name) - if (entry.isDirectory()) await walk(file) - else if (entry.isFile() && entry.name === 'command' && (await fs.readFile(file, 'utf8')).includes(nonce)) selected.push(file) + if (entry.isDirectory()) await walk(file, depth + 1) + else if (entry.isFile() && entry.name === 'command') { + assert((await fs.stat(file)).size <= 4096, 'Synthetic selected command changed beyond its finite fixture size') + if ((await fs.readFile(file, 'utf8')).includes(nonce)) selected.push(file) + } } } await walk(stageRoot) @@ -171,13 +223,17 @@ const requireMode = (metadata: { mode: number }) => (metadata.mode & 0o777).toSt export async function runManagedEnvironmentTarget() { assert.equal(process.env.FRESHELL_E2E_BACKEND, 'cloud', 'Managed acceptance requires configured Cloud') assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Managed acceptance requires the robot identity') + assert.equal(process.env.GCLOUD_ROBOT_ACCOUNT, 'gcloud-robot@misc-puttering-project.iam.gserviceaccount.com') const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') const root = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-managed-environment-')) const grants = fakeOnecliGrants(['opencode']) const nonce = randomUUID() let rig: ManagedRuntimeBrowserRig | undefined - let tunnel: ChildProcess | undefined, tunnelIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, tunnelJoin: Promise<number> | undefined - let browser: ChildProcess | undefined, browserIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, browserJoin: Promise<number> | undefined + let webIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined + let version: ChildProcess | undefined, versionIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, + versionJoin: Promise<NativeExit> | undefined + let tunnel: ChildProcess | undefined, tunnelIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, tunnelJoin: Promise<NativeExit> | undefined + let browser: ChildProcess | undefined, browserIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, browserJoin: Promise<NativeExit> | undefined const owners: InitialProviderVolumeAuthorization[] = [] let capturedVolumes: ReturnType<typeof captureProviderParityVolumes> = [] const proof: Record<string, unknown> = { expectedNonce: nonce } @@ -189,6 +245,8 @@ export async function runManagedEnvironmentTarget() { { workspaceRoots: [workspace], bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) const selected = await configure(root, nonce, rig.runtime.candidateSha, rig.runtime.runId) await rig.start() + webIdentity = await processIdentity(rig.web.processEvidence()) + proof.scratchWeb = webIdentity const health = await rig.runtime.adminOk(rig.supervisor, { method: 'health' }) const dockerHost = process.env.DOCKER_HOST assert(!dockerHost || dockerHost.startsWith('unix://'), 'Managed target requires the existing local Unix Docker endpoint') @@ -197,9 +255,11 @@ export async function runManagedEnvironmentTarget() { daemonId: await daemonId(dockerHost?.slice(7) || '/var/run/docker.sock') } const authorize = (requestId: string) => owners.push(authorizeProviderParityRequest(rig!.runtime.broker, source, requestId)) const before = await rig.inventory() + assert.deepEqual(before, []) + assert.deepEqual(await terminalDirectory(rig), []) for (const capabilities of [undefined, { paneReconcileV1: true }] as (Record<string, true> | undefined)[]) { const refused = await socketCreate(rig, randomUUID(), 'shell', selected.workspace, capabilities) - assert.equal(refused.type, 'terminal.error') + assert.equal(refused.type, 'error') assert(JSON.stringify(refused).includes('managed environment fixture denies host PTY')) } const fallback = await fetch(`${rig.info.baseUrl}/api/tabs`, { method: 'POST', headers: { 'x-auth-token': rig.info.token, @@ -207,14 +267,31 @@ export async function runManagedEnvironmentTarget() { assert.equal(fallback.status, 400) assert((await fallback.text()).includes('managed environment fixture denies host PTY')) assert.deepEqual(await rig.inventory(), before) - proof.hostPtyDenials = { legacy: true, nonNegotiated: true, httpCliMcpFallback: true, ordinaryRows: 0 } + const ordinaryRows = await terminalDirectory(rig) + assert.deepEqual(ordinaryRows, []) + const childPids = await poll('absence of owned ordinary PTY child PIDs', async () => { + const children = await webChildren(rig!) + return children.length === 0 ? children : undefined + }, 15_000) + proof.hostPtyDenials = { legacy: true, nonNegotiated: true, httpCliMcpFallback: true, + observedRegistryRows: ordinaryRows, observedChildPids: childPids, scratchWeb: rig.web.processEvidence() } const preflightRequest = `managed-preflight-${randomUUID()}` authorize(preflightRequest) - const created = await socketCreate(rig, preflightRequest, 'opencode', path.join(selected.workspace, 'preflight'), { managedRuntimeV1: true }) + const created = await socketCreate(rig, preflightRequest, 'opencode', path.join(selected.workspace, 'preflight'), + { paneReconcileV1: true, managedRuntimeV1: true }) assert.equal(created.type, 'terminal.created') - assert.equal(created.managed, true) - assert.equal(created.pid, null) - await poll('genuine managed preflight configuration', async () => readJson(path.join(selected.workspace, 'preflight/.managed-fixture/child-receipt.json'))) + const preflightChild = await poll('genuine managed preflight configuration', async () => readJson(path.join(selected.workspace, 'preflight/.managed-fixture/child-receipt.json'))) + assert.equal(preflightChild.terminalId, created.terminalId) + assert.equal(preflightChild.candidateSha, rig.runtime.candidateSha) + assert.equal(preflightChild.runId, rig.runtime.runId) + assert.equal(preflightChild.expectedNonce, nonce) + assert.equal(preflightChild.bun, true) + const preflightView = await poll('genuine managed preflight native identity', async () => (await rig!.inventory()) + .find(row => row.terminalId === created.terminalId && row.nativeSessionId === preflightChild.nativeSessionId && row.launchState === 'running')) + assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) + assert.deepEqual(await webChildren(rig), [], 'Managed child must belong to the disposable session host, not the host web process') + proof.preflight = { terminalId: created.terminalId, soulId: preflightView.soulId, incarnationId: preflightView.incarnationId, + stage: await stageProof(rig, preflightView, nonce), child: preflightChild } const toolHome = path.join(root, 'tools') await fs.mkdir(toolHome, { mode: 0o700 }) const tool = path.join(toolHome, 'cloudflared') @@ -223,12 +300,17 @@ export async function runManagedEnvironmentTarget() { const bytes = Buffer.from(await download.arrayBuffer()) assert.equal(digest(bytes), cloudflaredSha, 'Pinned cloudflared checksum mismatch') await fs.writeFile(tool, bytes, { mode: 0o700 }) - const version = spawn(tool, ['--version'], { cwd: root, env: { ...process.env, HOME: toolHome } }) - const versionJoin = joined(version), versionIdentity = await processIdentity(version) - let versionText = ''; version.stdout.on('data', bytes => { versionText += bytes }) - assert.equal(await versionJoin, 0) + version = spawn(tool, ['--version'], { cwd: root, env: { ...process.env, HOME: toolHome } }) + versionJoin = joined(version); versionJoin.catch(() => {}) + versionIdentity = await processIdentity(version).catch(error => { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error + return undefined + }) + let versionText = ''; version.stdout!.on('data', bytes => { versionText += bytes }) + const versionExit = await bounded(versionJoin, 30_000, 'pinned cloudflared version join') + assert.equal(versionExit.code, 0) assert.match(versionText, /^cloudflared version 2026\.10\.0(?:\s|$)/) - proof.cloudflared = { version: cloudflaredVersion, sha256: cloudflaredSha, versionIdentity } + proof.cloudflared = { version: cloudflaredVersion, sha256: cloudflaredSha, versionIdentity: versionIdentity ?? 'UNKNOWN', versionExit } tunnel = spawn(tool, ['tunnel', '--url', `http://127.0.0.1:${rig.info.port}`], { cwd: root, env: { ...process.env, HOME: toolHome, XDG_CONFIG_HOME: toolHome }, stdio: ['ignore', 'pipe', 'pipe'] }) tunnelJoin = joined(tunnel); tunnelJoin.catch(() => {}) @@ -259,8 +341,11 @@ export async function runManagedEnvironmentTarget() { for (const scenario of scenarios) { const cwd = path.join(selected.workspace, scenario) const request = await poll(`Cloud ${scenario} picker request`, async () => { + const request = await readJson(path.join(cwd, '.managed-fixture/request.json')) + if (request) return request assert.equal(browser!.exitCode, null, 'Cloud browser exited before completing its three cases') - return readJson(path.join(cwd, '.managed-fixture/request.json')) + assert.equal(browser!.signalCode, null, 'Cloud browser was signaled before completing its three cases') + return undefined }, 600_000) for (const key of ['candidateSha', 'runId', 'remoteWorkspacePath', 'expectedNonce'] as const) assert.equal(request[key], fixture[key]) assert.equal(request.scenario, scenario) @@ -281,8 +366,9 @@ export async function runManagedEnvironmentTarget() { proof.healthy = receipt } } - assert.equal(await browserJoin, 0, 'Configured Cloud browser acceptance failed') - proof.cloudBrowser = { exit: 0, identity: browserIdentity, expectedCases: 3 } + const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud browser final join') + assert.equal(browserExit.code, 0, 'Configured Cloud browser acceptance failed') + proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, expectedCases: 3, printedFooter: 'UNKNOWN' } const healthy = proof.healthy as any const original = (await rig.inventory()).find(row => row.soulId === healthy.soulId) assert(original?.containerId) @@ -294,25 +380,66 @@ export async function runManagedEnvironmentTarget() { const replacementStage = await stageProof(rig, replacement, nonce) assert.equal(replacement.nativeSessionId, original.nativeSessionId) assert.equal(replacementChild.bun, true) + assert.equal(replacementChild.terminalId, replacement.terminalId) + assert.equal(replacementChild.nativeSessionId, replacement.nativeSessionId) + assert.equal(replacementChild.candidateSha, rig.runtime.candidateSha) + assert.equal(replacementChild.runId, rig.runtime.runId) + assert.equal(replacementChild.expectedNonce, nonce) + assert.equal(replacementChild.executableMode, '0711') proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, child: replacementChild } await writeJson(path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json'), proof) console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.acceptance_passed', candidateSha: fixture.candidateSha, - runId: fixture.runId, browserCases: 3, receipt: path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json') })) + runId: fixture.runId, expectedBrowserCases: 3, receipt: path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json') })) } finally { - await stopExact(browser, browserIdentity, browserJoin) - await stopExact(tunnel, tunnelIdentity, tunnelJoin) + const errors: Error[] = [] + const cleanup: Record<string, unknown> = {} + const attempt = async (operation: string, action: () => unknown | Promise<unknown>) => { + try { cleanup[operation] = await action() } + catch (error) { errors.push(new Error(`Owned fixture cleanup failed: ${operation}`, { cause: error })); cleanup[operation] = 'FAILED; retain owned fixture' } + } + await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin)) + await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin)) + await attempt('versionJoin', () => stopExact(version, versionIdentity, versionJoin)) if (rig) { - capturedVolumes = captureProviderParityVolumes(owners) - await rig.stop() - cleanupProviderParityVolumes(capturedVolumes) + if (!webIdentity && rig.web) await attempt('scratchWebIdentity', async () => { + try { webIdentity = await processIdentity(rig!.web.processEvidence()) } + catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT' + && (!(error instanceof Error) || error.message !== 'RustServer has no live owned process evidence')) throw error + return 'Already absent according to the owning helper' + } + }) + for (const owner of owners) await attempt(`capture:${owner.providerVolumeName}`, () => { + capturedVolumes.push(...captureProviderParityVolumes([owner])) + }) + let rigSettled = false + await attempt('rigStop', async () => { + const result = await rig!.stop() + assert(result.ok, 'Owned rig cleanup did not settle all workloads') + if (webIdentity) { + try { + const current = await processIdentity({ pid: webIdentity.pid }) + assert.notEqual(current.birthTicks, webIdentity.birthTicks, 'Owned scratch web process remains live') + } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + } + rigSettled = true + return result + }) + if (rigSettled) for (const volume of capturedVolumes) await attempt(`remove:${volume.owner.providerVolumeName}`, + () => cleanupProviderParityVolumes([volume])) + } + if (errors.length === 0) { + await attempt('grantDirectory', () => fs.rm(grants.directory, { recursive: true, force: true })) + await attempt('fixtureDirectory', () => fs.rm(root, { recursive: true, force: true })) } - await fs.rm(grants.directory, { recursive: true, force: true }) - await fs.rm(root, { recursive: true, force: true }) + if (rig) await attempt('cleanupReceipt', () => writeJson(path.join(rig!.runtime.evidenceDir, 'managed-environment-cleanup.json'), + { cleanup, knownOwnedSettled: errors.length === 0, retainedFixture: errors.length ? root : undefined })) + if (errors.length) throw new AggregateError(errors, 'Managed environment owned cleanup remains unsettled') } } -if (process.argv[1] && import.meta.url === new URL(`file://${path.resolve(process.argv[1])}`).href) { +if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) { runManagedEnvironmentTarget().catch(error => { console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.acceptance_failed', error: error instanceof Error ? error.message : 'Unknown fixture failure' })) process.exitCode = 1 diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index f0a6be257..13c1ec888 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -43,7 +43,7 @@ async function launch(page: Page, scenario: string) { const readiness = await json(page, '/api/runtime/readiness') expect(readiness.readiness.initialScanState).toBe('complete') expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getReceivedWsMessages?.() - .some((message: any) => message.type === 'ready' && message.managedRuntimeV1 === true))).toBe(true) + .some((message: any) => message.type === 'ready' && message.capabilities?.managedRuntimeV1 === true))).toBe(true) const add = page.locator('[data-context="tab-add"]') if (await add.isVisible()) await add.click() From 1d44ac17982c6424650bebff33b1142b18658449 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 10:09:47 -0700 Subject: [PATCH 257/493] fix(testing): negotiate managed fixture websocket protocol --- scripts/testing/managed-provider-environment-target.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 3dbee8040..8902deb27 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -7,6 +7,7 @@ import os from 'node:os' import path from 'node:path' import { fileURLToPath, pathToFileURL } from 'node:url' import WebSocket from 'ws' +import { WS_PROTOCOL_VERSION, type ClientMessage } from '../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig, type ManagedRuntimeView } from '../../test/e2e-browser/helpers/managed-runtime.js' import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../test/helpers/provider-parity-broker.js' import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' @@ -172,7 +173,9 @@ async function socketCreate(rig: ManagedRuntimeBrowserRig, requestId: string, mo try { await bounded(new Promise<void>((resolve, reject) => { ws.once('open', resolve); ws.once('error', reject) }), 30_000, 'owned WebSocket open') - ws.send(JSON.stringify({ type: 'hello', token: rig.info.token, ...(capabilities ? { capabilities } : {}) })) + const hello = { type: 'hello', token: rig.info.token, protocolVersion: WS_PROTOCOL_VERSION, + ...(capabilities ? { capabilities } : {}) } satisfies ClientMessage + ws.send(JSON.stringify(hello)) const ready = await poll('authenticated ready', async () => messages.find(message => message.type === 'ready'), 30_000) if (capabilities?.managedRuntimeV1) assert.equal(ready.capabilities?.managedRuntimeV1, true) ws.send(JSON.stringify({ type: 'terminal.create', requestId, mode, cwd, cols: 100, rows: 30 })) From 580549e83f6196f2c57dd7638f29beb222227fe4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 10:27:42 -0700 Subject: [PATCH 258/493] fix(testing): use typed managed create and reconnect helpers --- scripts/testing/managed-provider-environment-target.ts | 3 ++- .../specs/managed-provider-environment-external.spec.ts | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 8902deb27..1fd352a61 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -178,7 +178,8 @@ async function socketCreate(rig: ManagedRuntimeBrowserRig, requestId: string, mo ws.send(JSON.stringify(hello)) const ready = await poll('authenticated ready', async () => messages.find(message => message.type === 'ready'), 30_000) if (capabilities?.managedRuntimeV1) assert.equal(ready.capabilities?.managedRuntimeV1, true) - ws.send(JSON.stringify({ type: 'terminal.create', requestId, mode, cwd, cols: 100, rows: 30 })) + const create = { type: 'terminal.create', requestId, mode, shell: 'system', cwd } satisfies ClientMessage + ws.send(JSON.stringify(create)) return await poll('terminal create response', async () => messages.find(message => message.requestId === requestId && (message.type === 'terminal.created' || message.type === 'error')), 120_000) } finally { diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 13c1ec888..aa6368988 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -108,7 +108,7 @@ test('executes selected resources with Bun in both managed stages and retains id expect(receipt.nativeSessionId).toBeTruthy() const beforeCreates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length) - await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.forceDisconnect()) + await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) await expect.poll(async () => (await activePane(page)).content).toMatchObject({ terminalId, sessionRef: { provider: 'opencode', sessionId: receipt.nativeSessionId } }) @@ -138,7 +138,7 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { await expect(diagnostic).toContainText(scenario === 'missing-reference' ? /missing-selected/i : /startup-fixture/i) const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length) - await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.forceDisconnect()) + await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) expect((await activePane(page)).content).toMatchObject({ terminalId, status: 'error' }) expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() From 32b8add10cf762392523fa36b6dea3896f999606 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 10:46:58 -0700 Subject: [PATCH 259/493] test: exercise ordinary terminal refusal in managed fixture --- .../testing/managed-provider-environment-target.ts | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 1fd352a61..3cf678b83 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -17,6 +17,7 @@ import { resolvePairedExternalFixture } from './e2e-external-target.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' const scenarios = ['healthy', 'missing-reference', 'startup-before-identity'] as const +const ordinaryFixtureMode = 'managed-fixture-ordinary' const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') @@ -129,6 +130,15 @@ async function configure(root: string, nonce: string, candidateSha: string, runI await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) await fs.chmod(home, 0o700) await fs.chmod(root, 0o751) + // Register a known CLI without a managed adapter through the normal HOME + // extension scan. Shell uses the managed facade when a controller is present; + // this mode reaches the ordinary registry.create refusal instead. + const ordinaryExtension = path.join(home, '.freshell/extensions', ordinaryFixtureMode) + await fs.mkdir(ordinaryExtension, { recursive: true, mode: 0o700 }) + await writeJson(path.join(ordinaryExtension, 'freshell.json'), { + name: ordinaryFixtureMode, version: '1.0.0', label: 'Ordinary terminal fixture', + description: 'Owned ordinary terminal refusal fixture', category: 'cli', cli: { command: '/bin/false' }, + }) await fs.writeFile(path.join(plugin, 'node_modules/managed-fixture-dependency/package.json'), '{"type":"module","exports":"./index.mjs"}') await fs.writeFile(path.join(plugin, 'node_modules/managed-fixture-dependency/index.mjs'), `export default ${JSON.stringify(nonce)};`) await fs.writeFile(path.join(plugin, 'selected-resource'), nonce, { mode: 0o600 }) @@ -267,7 +277,7 @@ export async function runManagedEnvironmentTarget() { assert(JSON.stringify(refused).includes('managed environment fixture denies host PTY')) } const fallback = await fetch(`${rig.info.baseUrl}/api/tabs`, { method: 'POST', headers: { 'x-auth-token': rig.info.token, - 'content-type': 'application/json' }, body: JSON.stringify({ mode: 'shell', cwd: selected.workspace }), signal: AbortSignal.timeout(30_000) }) + 'content-type': 'application/json' }, body: JSON.stringify({ mode: ordinaryFixtureMode, cwd: selected.workspace }), signal: AbortSignal.timeout(30_000) }) assert.equal(fallback.status, 400) assert((await fallback.text()).includes('managed environment fixture denies host PTY')) assert.deepEqual(await rig.inventory(), before) @@ -277,7 +287,7 @@ export async function runManagedEnvironmentTarget() { const children = await webChildren(rig!) return children.length === 0 ? children : undefined }, 15_000) - proof.hostPtyDenials = { legacy: true, nonNegotiated: true, httpCliMcpFallback: true, + proof.hostPtyDenials = { legacy: true, nonNegotiated: true, httpCliMcpFallback: true, httpMode: ordinaryFixtureMode, observedRegistryRows: ordinaryRows, observedChildPids: childPids, scratchWeb: rig.web.processEvidence() } const preflightRequest = `managed-preflight-${randomUUID()}` authorize(preflightRequest) From 047cbbbc1e101b81081ca37f90fbbcf80f134fb4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 11:04:21 -0700 Subject: [PATCH 260/493] test: protect managed create failure evidence before cleanup --- scripts/testing/managed-environment-create.ts | 13 +++++ .../managed-provider-environment-target.ts | 3 +- .../tooling/testing/runtime-sandbox.test.ts | 49 +++++++++++++++++++ 3 files changed, 64 insertions(+), 1 deletion(-) create mode 100644 scripts/testing/managed-environment-create.ts diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts new file mode 100644 index 000000000..330ebadc3 --- /dev/null +++ b/scripts/testing/managed-environment-create.ts @@ -0,0 +1,13 @@ +import assert from 'node:assert/strict' +import type { ErrorMessage } from '../../shared/ws-protocol.js' + +type ManagedCreateOwner = { + web: { capturedOutput(): { stdout: string; stderr: string } } + runtime: { containerLogs(id: string): string; writeIncident(name: string, value: unknown): void } + supervisor: { containerId: string } +} + +export function assertManagedTerminalCreated(_owner: ManagedCreateOwner, + response: ErrorMessage | { type: 'terminal.created' }) { + assert.equal(response.type, 'terminal.created') +} diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 3cf678b83..b6673a809 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -13,6 +13,7 @@ import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupPr import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' +import { assertManagedTerminalCreated } from './managed-environment-create.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -293,7 +294,7 @@ export async function runManagedEnvironmentTarget() { authorize(preflightRequest) const created = await socketCreate(rig, preflightRequest, 'opencode', path.join(selected.workspace, 'preflight'), { paneReconcileV1: true, managedRuntimeV1: true }) - assert.equal(created.type, 'terminal.created') + assertManagedTerminalCreated(rig, created) const preflightChild = await poll('genuine managed preflight configuration', async () => readJson(path.join(selected.workspace, 'preflight/.managed-fixture/child-receipt.json'))) assert.equal(preflightChild.terminalId, created.terminalId) assert.equal(preflightChild.candidateSha, rig.runtime.candidateSha) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 3c075a610..cd78d9de5 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -34,6 +34,55 @@ import { RuntimeHarness } from '../../../../scripts/testing/runtime-sandbox.js' import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' +import { assertManagedTerminalCreated } from '../../../../scripts/testing/managed-environment-create.js' +import type { ErrorMessage } from '../../../../shared/ws-protocol.js' + +describe('managed environment create failure retention', () => { + it.each(['known', 'unknown'] as const)('retains safe %s failure evidence before normal owned cleanup', async kind => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-create-failure-')) + const home = path.join(directory, 'home') + await fs.mkdir(home) + const rig = new ManagedRuntimeBrowserRig(directory, 2, {}, {}, 'release', { enabledProviders: [] }, home) + const secretMarkers = ['AUTH_SECRET_MARKER', 'CONFIG_SECRET_MARKER', 'CONVERSATION_SECRET_MARKER'] + const reason = 'selected workspace input mountpoint is unavailable' + const response: ErrorMessage = { type: 'error', code: 'PTY_SPAWN_FAILED', timestamp: new Date().toISOString(), + message: kind === 'known' ? `runtime backend configuration rejected: ${reason}; ${secretMarkers.join(' ')}` + : secretMarkers.join(' ') } + const failureEvent = { event: 'supervisor.activation_failed', data: { stage: 'create_stopped_runtime', + errorCode: 'BACKEND_UNAVAILABLE', errorClass: 'runtime_activation_failed', payload: secretMarkers } } + const logs = vi.spyOn(rig.runtime, 'containerLogs').mockReturnValue(JSON.stringify(failureEvent)) + const web = { capturedOutput: vi.fn(() => ({ stdout: JSON.stringify(failureEvent), stderr: secretMarkers.join(' ') })), + stop: async () => fs.rm(home, { recursive: true }) } + rig.web = web as unknown as typeof rig.web + rig.supervisor = { containerId: 'd'.repeat(64) } as typeof rig.supervisor + try { + expect(() => assertManagedTerminalCreated(rig, response)).toThrow() + expect(await rig.stop()).toEqual({ ok: true, errors: [] }) + await expect(fs.stat(home)).rejects.toMatchObject({ code: 'ENOENT' }) + const stored = await fs.readFile(path.join(rig.runtime.evidenceDir, 'incidents/managed-environment-create-failure.json'), 'utf8') + const incident = JSON.parse(stored) + expect(incident).toMatchObject({ severity: 'error', event: 'managed_environment.create_failed', + code: 'PTY_SPAWN_FAILED', reason: kind === 'known' ? reason : 'UNKNOWN', + supervisor: { event: 'supervisor.activation_failed', stage: 'create_stopped_runtime', errorCode: 'BACKEND_UNAVAILABLE' } }) + for (const marker of secretMarkers) expect(stored).not.toContain(marker) + expect(logs).toHaveBeenCalledWith(rig.supervisor.containerId) + expect(web.capturedOutput).toHaveBeenCalledOnce() + } finally { + await fs.rm(rig.runtime.testRoot, { recursive: true, force: true }) + await fs.rm(directory, { recursive: true, force: true }) + vi.restoreAllMocks() + } + }) + + it('accepts a real created response without reading failure logs or writing an incident', () => { + const owner = { web: { capturedOutput: vi.fn() }, runtime: { containerLogs: vi.fn(), writeIncident: vi.fn() }, + supervisor: { containerId: 'd'.repeat(64) } } + expect(() => assertManagedTerminalCreated(owner, { type: 'terminal.created' })).not.toThrow() + expect(owner.web.capturedOutput).not.toHaveBeenCalled() + expect(owner.runtime.containerLogs).not.toHaveBeenCalled() + expect(owner.runtime.writeIncident).not.toHaveBeenCalled() + }) +}) function useOwnedHarnessArtifacts(harness: RuntimeHarness, host: string): void { const artifact = harness as unknown as { ensureRuntimeImage(): void; buildBinaries(): void; collectCapabilities(): unknown; collectBuildInfo(): unknown } From 97ae489cb638a1dedcb69e99cfa9571f8ff6e717 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 11:09:56 -0700 Subject: [PATCH 261/493] fix: retain safe managed create failure evidence --- scripts/testing/managed-environment-create.ts | 61 ++++++++++++++++++- 1 file changed, 59 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 330ebadc3..565fae449 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -1,5 +1,5 @@ import assert from 'node:assert/strict' -import type { ErrorMessage } from '../../shared/ws-protocol.js' +import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' type ManagedCreateOwner = { web: { capturedOutput(): { stdout: string; stderr: string } } @@ -7,7 +7,64 @@ type ManagedCreateOwner = { supervisor: { containerId: string } } -export function assertManagedTerminalCreated(_owner: ManagedCreateOwner, +const failureReasons = [ + 'initial provider source lost its current prepared reservation', + 'runtime image must be pinned by sha256 id/digest', + 'provider volume name is not installation-scoped', + 'initial auth reservation does not match the prepared runtime', + 'initial auth reserved runtime artifact changed or unreadable', + 'workspace path must already be canonical', + 'cwd path must already be canonical', + 'terminal cwd must be inside the approved workspace', + 'provider bootstrap file must be a canonical regular file', + 'provider secret reference must be a canonical regular file', + 'provider OneCLI grant must be private', + 'referenced provider user root is unavailable', + 'prepared provider stage is unavailable', + 'managed MCP grant must be a canonical regular file', + 'managed MCP grant must be private', + 'prepared input file is unavailable', + 'prepared input file must be physical and stage-owned', + 'prepared input manifest changed; reprepare before mount construction', + 'selected prepared resource is unavailable', + 'prepared resource is not a physical staged unit', + 'selected workspace input mountpoint is unavailable', + 'selected workspace input topology changed; reprepare before launch', +] as const + +const activationStages = ['capability_pending', 'ensure_runtime_directory', 'ensure_host_actor_directory', + 'revalidate_initial_provider_source', 'create_stopped_runtime', 'commit_backend_identity', + 'load_owned_handle', 'authenticate_running_host', 'read_running_host_status', 'adopt_running_worker', + 'start_session_host', 'authenticate_started_host', 'commit_execution_grant', 'enable_long_lived_runtime', + 'record_native_identity', 'commit_running_state', 'commit_provider_inputs', 'load_running_view'] + +export function assertManagedTerminalCreated(owner: ManagedCreateOwner, response: ErrorMessage | { type: 'terminal.created' }) { + if (response.type === 'error') { + // Persist only fixed classifications. Provider config, credentials and + // conversation text can occur in error/log payloads and stay out of it. + const reason = failureReasons.find(value => response.message.includes(value)) ?? 'UNKNOWN' + const code = ErrorCode.safeParse(response.code) + const web = owner.web.capturedOutput() + let supervisor: Record<string, string> | 'UNKNOWN' = 'UNKNOWN' + try { + for (const line of owner.runtime.containerLogs(owner.supervisor.containerId).split('\n')) { + try { + const entry = JSON.parse(line) + if (entry.event !== 'supervisor.activation_failed') continue + const data = entry.data + supervisor = { event: 'supervisor.activation_failed', + stage: activationStages.includes(data?.stage) ? data.stage : 'UNKNOWN', + errorCode: ['BACKEND_UNAVAILABLE', 'OWNERSHIP_MISMATCH', 'INVALID_REQUEST', 'REGISTRY_FAILURE', + 'HOST_UNREACHABLE'].includes(data?.errorCode) ? data.errorCode : 'UNKNOWN' } + } catch { /* Non-JSON owning-process output is not retained. */ } + } + } catch { /* An absent owned supervisor leaves its observation unknown. */ } + owner.runtime.writeIncident('managed-environment-create-failure', { + severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, + server: { source: 'owned-server-typed-reply', + capturedOutputAvailable: Boolean(web.stdout || web.stderr) }, supervisor, + }) + } assert.equal(response.type, 'terminal.created') } From b7a66627f9bb5fd876b40f342f9a7efae4b5794c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 11:28:08 -0700 Subject: [PATCH 262/493] test: cover actual managed failure diagnostic producers --- .../tooling/testing/runtime-sandbox.test.ts | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index cd78d9de5..46f3325fc 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -38,23 +38,38 @@ import { assertManagedTerminalCreated } from '../../../../scripts/testing/manage import type { ErrorMessage } from '../../../../shared/ws-protocol.js' describe('managed environment create failure retention', () => { - it.each(['known', 'unknown'] as const)('retains safe %s failure evidence before normal owned cleanup', async kind => { + it.each(['known', 'unknown', 'file-os', 'file-index', 'file-auth', 'file-unknown'] as const)('retains safe %s failure evidence before normal owned cleanup', async kind => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-create-failure-')) const home = path.join(directory, 'home') await fs.mkdir(home) const rig = new ManagedRuntimeBrowserRig(directory, 2, {}, {}, 'release', { enabledProviders: [] }, home) const secretMarkers = ['AUTH_SECRET_MARKER', 'CONFIG_SECRET_MARKER', 'CONVERSATION_SECRET_MARKER'] - const reason = 'selected workspace input mountpoint is unavailable' + const reasons = { + known: 'selected workspace input mountpoint is unavailable', unknown: 'UNKNOWN', + 'file-os': 'workspace filesystem error', 'file-index': 'prepared provider inputs configuration source', + 'file-auth': 'initial auth reserved runtime artifact changed or unreadable', 'file-unknown': 'UNKNOWN', + } + const reason = reasons[kind] + const fileEvidence = kind.startsWith('file-') + const producerReason = kind === 'file-os' ? 'workspace: Permission denied (os error 13)' + : kind === 'file-index' ? 'prepared provider inputs configuration source: invalid, conflicting or over-bound metadata; reprepare before launch' + : reason const response: ErrorMessage = { type: 'error', code: 'PTY_SPAWN_FAILED', timestamp: new Date().toISOString(), - message: kind === 'known' ? `runtime backend configuration rejected: ${reason}; ${secretMarkers.join(' ')}` + message: reason !== 'UNKNOWN' ? `managed runtime launch failed: runtime supervisor rejected request: BackendUnavailable: runtime backend configuration rejected: ${producerReason}; ${secretMarkers.join(' ')}` : secretMarkers.join(' ') } const failureEvent = { event: 'supervisor.activation_failed', data: { stage: 'create_stopped_runtime', errorCode: 'BACKEND_UNAVAILABLE', errorClass: 'runtime_activation_failed', payload: secretMarkers } } - const logs = vi.spyOn(rig.runtime, 'containerLogs').mockReturnValue(JSON.stringify(failureEvent)) + const runtimeRoot = path.join(rig.runtime.testRoot, 'r', 'browse-owned', 'x') + const lifecycle = path.join(path.dirname(runtimeRoot), 'evidence', 'lifecycle.jsonl') + if (fileEvidence) { + await fs.mkdir(path.dirname(lifecycle), { recursive: true }) + await fs.writeFile(lifecycle, `${JSON.stringify(failureEvent)}\n`) + } + const logs = vi.spyOn(rig.runtime, 'containerLogs').mockReturnValue(fileEvidence ? secretMarkers.join(' ') : JSON.stringify(failureEvent)) const web = { capturedOutput: vi.fn(() => ({ stdout: JSON.stringify(failureEvent), stderr: secretMarkers.join(' ') })), stop: async () => fs.rm(home, { recursive: true }) } rig.web = web as unknown as typeof rig.web - rig.supervisor = { containerId: 'd'.repeat(64) } as typeof rig.supervisor + rig.supervisor = { containerId: 'd'.repeat(64), runtimeRoot } as typeof rig.supervisor try { expect(() => assertManagedTerminalCreated(rig, response)).toThrow() expect(await rig.stop()).toEqual({ ok: true, errors: [] }) @@ -62,8 +77,9 @@ describe('managed environment create failure retention', () => { const stored = await fs.readFile(path.join(rig.runtime.evidenceDir, 'incidents/managed-environment-create-failure.json'), 'utf8') const incident = JSON.parse(stored) expect(incident).toMatchObject({ severity: 'error', event: 'managed_environment.create_failed', - code: 'PTY_SPAWN_FAILED', reason: kind === 'known' ? reason : 'UNKNOWN', + code: 'PTY_SPAWN_FAILED', reason, supervisor: { event: 'supervisor.activation_failed', stage: 'create_stopped_runtime', errorCode: 'BACKEND_UNAVAILABLE' } }) + if (kind === 'file-os') expect(incident.osError).toBe(13) for (const marker of secretMarkers) expect(stored).not.toContain(marker) expect(logs).toHaveBeenCalledWith(rig.supervisor.containerId) expect(web.capturedOutput).toHaveBeenCalledOnce() From f07cd4f050ab425cc0b25b8633220ed0ad30ac78 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 11:31:55 -0700 Subject: [PATCH 263/493] fix: retain owned lifecycle and safe managed launch causes --- scripts/testing/managed-environment-create.ts | 47 +++++++++++++++++-- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 565fae449..1890366a2 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -1,10 +1,12 @@ import assert from 'node:assert/strict' +import fs from 'node:fs' +import path from 'node:path' import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' type ManagedCreateOwner = { web: { capturedOutput(): { stdout: string; stderr: string } } runtime: { containerLogs(id: string): string; writeIncident(name: string, value: unknown): void } - supervisor: { containerId: string } + supervisor: { containerId: string; runtimeRoot?: string } } const failureReasons = [ @@ -30,6 +32,22 @@ const failureReasons = [ 'prepared resource is not a physical staged unit', 'selected workspace input mountpoint is unavailable', 'selected workspace input topology changed; reprepare before launch', + 'managed provider launch context is unsafe or mismatched', + 'managed provider grant profile, destination or capacity is invalid', + 'prepared provider file indexes do not match approved configuration grants; reprepare before launch', + 'prepared config template is unavailable', + 'prepared config template read failed', + 'prepared input manifest is unavailable', + 'prepared input manifest read failed', + 'selected prepared resource metadata failed', + 'prepared input file metadata failed', + 'selected workspace input mountpoint metadata failed', + 'git common dir must already be canonical', + 'provider user root escaped its approved source root', + 'managed provider root must be canonical', + 'invalid managed MCP grant id', + 'host binary is not a regular file', + 'runtime dir is not a directory', ] as const const activationStages = ['capability_pending', 'ensure_runtime_directory', 'ensure_host_actor_directory', @@ -43,12 +61,33 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, if (response.type === 'error') { // Persist only fixed classifications. Provider config, credentials and // conversation text can occur in error/log payloads and stay out of it. - const reason = failureReasons.find(value => response.message.includes(value)) ?? 'UNKNOWN' + const inputFields = ['control byte limit', 'initial auth reservation', 'generation', 'manifest', 'resource entries', + 'mount destination', 'logical destination', 'effective destination', 'configuration source', 'resource source', + 'indexed file publication', 'indexed source ID', 'ancestor destination'] + const inputReason = inputFields.map(field => `prepared provider inputs ${field}`) + .find(value => response.message.includes(`${value}: invalid, conflicting or over-bound metadata`)) + const filesystemLabels = ['host binary', 'runtime dir', 'workspace', 'cwd', 'git common dir', + 'provider bootstrap file', 'provider secret reference'] + const filesystemLabel = filesystemLabels.find(value => response.message.includes(`${value}:`)) + const errno = response.message.match(/\(os error (\d{1,4})\)/) + const osError = errno ? Number(errno[1]) : undefined + const reason = failureReasons.find(value => response.message.includes(value)) ?? inputReason + ?? (filesystemLabel && osError !== undefined ? `${filesystemLabel} filesystem error` : 'UNKNOWN') const code = ErrorCode.safeParse(response.code) const web = owner.web.capturedOutput() let supervisor: Record<string, string> | 'UNKNOWN' = 'UNKNOWN' try { - for (const line of owner.runtime.containerLogs(owner.supervisor.containerId).split('\n')) { + let lifecycle = '' + if (owner.supervisor.runtimeRoot) { + // This is the supervisor's exact owned lifecycle path, also collected + // by RuntimeHarness cleanup. It is not written to Docker stdout. + const file = path.join(path.dirname(owner.supervisor.runtimeRoot), 'evidence', 'lifecycle.jsonl') + try { + if (fs.statSync(file).size <= 2 * 1024 * 1024) lifecycle = fs.readFileSync(file, 'utf8').slice(-20_000) + } catch { /* Missing owning-file evidence remains unknown. */ } + } + const logs = `${owner.runtime.containerLogs(owner.supervisor.containerId)}\n${lifecycle}` + for (const line of logs.split('\n')) { try { const entry = JSON.parse(line) if (entry.event !== 'supervisor.activation_failed') continue @@ -61,7 +100,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, } } catch { /* An absent owned supervisor leaves its observation unknown. */ } owner.runtime.writeIncident('managed-environment-create-failure', { - severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, + severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), server: { source: 'owned-server-typed-reply', capturedOutputAvailable: Boolean(web.stdout || web.stderr) }, supervisor, }) From 5a8dc4c2b4a2f291dfb91bdd88214b22dfddaebf Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 11:50:20 -0700 Subject: [PATCH 264/493] fix: close optional OneCLI inputs in managed fixture --- scripts/testing/managed-provider-environment-target.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index b6673a809..313d98207 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -254,7 +254,10 @@ export async function runManagedEnvironmentTarget() { const proof: Record<string, unknown> = { expectedNonce: nonce } try { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') - const environment = { ...grants.serverEnv, HOME: home, XDG_CONFIG_HOME: xdg } + // The synthetic grant is the only selected OneCLI input. RustServer merges + // ambient env, so explicitly close optional host auth/CA references too. + const environment = { ...grants.serverEnv, HOME: home, XDG_CONFIG_HOME: xdg, + FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE: '', FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE: '' } rig = new ManagedRuntimeBrowserRig(repoRoot, 2, { ...environment, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY: '1' }, environment, 'release', { enabledProviders: ['opencode'] }, home, { workspaceRoots: [workspace], bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) From de3aae015c3d1b4aa4274abbf3338cb90fc31690 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 12:41:44 -0700 Subject: [PATCH 265/493] test(managed-runtime): cover driver daemon socket selection --- scripts/testing/managed-environment-create.ts | 20 +++++++ .../managed-provider-environment-target.ts | 23 +------- .../tooling/testing/runtime-sandbox.test.ts | 55 ++++++++++++++++++- 3 files changed, 75 insertions(+), 23 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 1890366a2..1f7d262ad 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -1,8 +1,28 @@ import assert from 'node:assert/strict' import fs from 'node:fs' +import http from 'node:http' import path from 'node:path' import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' +export async function readManagedEnvironmentDaemonId(): Promise<string> { + const dockerHost = process.env.DOCKER_HOST + assert(!dockerHost || dockerHost.startsWith('unix://'), 'Managed target requires the existing local Unix Docker endpoint') + const socketPath = dockerHost?.slice(7) || '/var/run/docker.sock' + return new Promise((resolve, reject) => { + const request = http.get({ socketPath, path: '/v1.47/info', timeout: 30_000 }, response => { + const chunks: Buffer[] = [] + let bytes = 0 + response.on('data', chunk => { bytes += chunk.length; if (bytes > 65536) request.destroy(new Error('Daemon metadata exceeded its bound')); else chunks.push(chunk) }) + response.once('end', () => { + try { assert.equal(response.statusCode, 200); const value = JSON.parse(Buffer.concat(chunks).toString()); assert.equal(typeof value.ID, 'string'); resolve(value.ID) } + catch (error) { reject(error) } + }) + }) + request.once('timeout', () => request.destroy(new Error('Daemon metadata timed out'))) + request.once('error', reject) + }) +} + type ManagedCreateOwner = { web: { capturedOutput(): { stdout: string; stderr: string } } runtime: { containerLogs(id: string): string; writeIncident(name: string, value: unknown): void } diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 313d98207..54a4ef105 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -2,7 +2,6 @@ import assert from 'node:assert/strict' import { spawn, type ChildProcess } from 'node:child_process' import { createHash, randomUUID } from 'node:crypto' import fs from 'node:fs/promises' -import http from 'node:http' import os from 'node:os' import path from 'node:path' import { fileURLToPath, pathToFileURL } from 'node:url' @@ -13,7 +12,7 @@ import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupPr import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' -import { assertManagedTerminalCreated } from './managed-environment-create.js' +import { assertManagedTerminalCreated, readManagedEnvironmentDaemonId } from './managed-environment-create.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -108,22 +107,6 @@ async function webChildren(rig: ManagedRuntimeBrowserRig): Promise<number[]> { return [...children].sort((left, right) => left - right) } -async function daemonId(socketPath: string): Promise<string> { - return new Promise((resolve, reject) => { - const request = http.get({ socketPath, path: '/v1.47/info', timeout: 30_000 }, response => { - const chunks: Buffer[] = [] - let bytes = 0 - response.on('data', chunk => { bytes += chunk.length; if (bytes > 65536) request.destroy(new Error('Daemon metadata exceeded its bound')); else chunks.push(chunk) }) - response.once('end', () => { - try { assert.equal(response.statusCode, 200); const value = JSON.parse(Buffer.concat(chunks).toString()); assert.equal(typeof value.ID, 'string'); resolve(value.ID) } - catch (error) { reject(error) } - }) - }) - request.once('timeout', () => request.destroy(new Error('Daemon metadata timed out'))) - request.once('error', reject) - }) -} - async function configure(root: string, nonce: string, candidateSha: string, runId: string) { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') const resources = path.join(workspace, 'selected-resources'), plugin = path.join(resources, 'fixture-plugin') @@ -266,11 +249,9 @@ export async function runManagedEnvironmentTarget() { webIdentity = await processIdentity(rig.web.processEvidence()) proof.scratchWeb = webIdentity const health = await rig.runtime.adminOk(rig.supervisor, { method: 'health' }) - const dockerHost = process.env.DOCKER_HOST - assert(!dockerHost || dockerHost.startsWith('unix://'), 'Managed target requires the existing local Unix Docker endpoint') const source = { installationId: health.data.installation_id, imageRef: rig.runtime.imageRef, hostBinaryPath: rig.runtime.releaseHostBinary, binarySha256: digest(await fs.readFile(rig.runtime.releaseHostBinary)), - daemonId: await daemonId(dockerHost?.slice(7) || '/var/run/docker.sock') } + daemonId: await readManagedEnvironmentDaemonId() } const authorize = (requestId: string) => owners.push(authorizeProviderParityRequest(rig!.runtime.broker, source, requestId)) const before = await rig.inventory() assert.deepEqual(before, []) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 46f3325fc..12ea4495a 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -1,4 +1,5 @@ import { createHash } from 'node:crypto' +import { EventEmitter } from 'node:events' import fs from 'node:fs/promises' import http from 'node:http' import os from 'node:os' @@ -30,13 +31,63 @@ vi.mock('node:child_process', () => { return { ...api, default: api } }) -import { RuntimeHarness } from '../../../../scripts/testing/runtime-sandbox.js' +import { RuntimeHarness, dockerSocketPath } from '../../../../scripts/testing/runtime-sandbox.js' import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' -import { assertManagedTerminalCreated } from '../../../../scripts/testing/managed-environment-create.js' +import { assertManagedTerminalCreated, readManagedEnvironmentDaemonId } from '../../../../scripts/testing/managed-environment-create.js' import type { ErrorMessage } from '../../../../shared/ws-protocol.js' +describe('managed environment daemon identity', () => { + it('reads the exact configured Unix daemon before registering a source', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-daemon-')) + const socket = path.join(directory, 'daemon.sock') + const requests: string[] = [] + const server = http.createServer((request, response) => { + requests.push(request.url!) + response.writeHead(200, { 'content-type': 'application/json' }) + response.end(JSON.stringify({ ID: 'owned-fixture-daemon' })) + }) + const previous = process.env.DOCKER_HOST + try { + await new Promise<void>(resolve => server.listen(socket, resolve)) + process.env.DOCKER_HOST = `unix://${socket}` + expect(await readManagedEnvironmentDaemonId()).toBe('owned-fixture-daemon') + expect(requests).toEqual(['/v1.47/info']) + } finally { + if (previous === undefined) delete process.env.DOCKER_HOST + else process.env.DOCKER_HOST = previous + await new Promise<void>((resolve, reject) => server.close(error => error ? reject(error) : resolve())) + await fs.rm(directory, { recursive: true }) + } + }) + + it('dispatches the default identity lookup to the same rootless socket as the owned runtime', async () => { + const previous = process.env.DOCKER_HOST + delete process.env.DOCKER_HOST + const dispatched: http.RequestOptions[] = [] + const get = vi.spyOn(http, 'get').mockImplementation(((options: http.RequestOptions, + callback: (response: http.IncomingMessage) => void) => { + dispatched.push(options) + const request = new EventEmitter() + queueMicrotask(() => { + const response = Object.assign(new EventEmitter(), { statusCode: 200 }) + callback(response as http.IncomingMessage) + response.emit('data', Buffer.from('{"ID":"owned-rootless-daemon"}')) + response.emit('end') + }) + return request as http.ClientRequest + }) as typeof http.get) + try { + expect(await readManagedEnvironmentDaemonId()).toBe('owned-rootless-daemon') + expect(dispatched).toEqual([{ socketPath: dockerSocketPath(), path: '/v1.47/info', timeout: 30_000 }]) + } finally { + get.mockRestore() + if (previous !== undefined) process.env.DOCKER_HOST = previous + } + }) +}) + describe('managed environment create failure retention', () => { it.each(['known', 'unknown', 'file-os', 'file-index', 'file-auth', 'file-unknown'] as const)('retains safe %s failure evidence before normal owned cleanup', async kind => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-create-failure-')) From 1210bc9acd62a34d8ef7c2520b919cbc50ed4866 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 12:46:14 -0700 Subject: [PATCH 266/493] fix(managed-runtime): use fixture Docker socket for source identity --- scripts/testing/managed-environment-create.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 1f7d262ad..643dd2e2e 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -3,11 +3,12 @@ import fs from 'node:fs' import http from 'node:http' import path from 'node:path' import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' +import { dockerSocketPath } from './runtime-sandbox.js' export async function readManagedEnvironmentDaemonId(): Promise<string> { const dockerHost = process.env.DOCKER_HOST assert(!dockerHost || dockerHost.startsWith('unix://'), 'Managed target requires the existing local Unix Docker endpoint') - const socketPath = dockerHost?.slice(7) || '/var/run/docker.sock' + const socketPath = dockerSocketPath() return new Promise((resolve, reject) => { const request = http.get({ socketPath, path: '/v1.47/info', timeout: 30_000 }, response => { const chunks: Buffer[] = [] From d609e2c4f15985aa730d0de1972e26d6de99af9c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 13:09:31 -0700 Subject: [PATCH 267/493] test: preserve rejected managed preparation destinations in diagnostics --- .../testing/runtime-test-broker.test.ts | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index febe71f2f..cde503baf 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -966,6 +966,52 @@ async function publishControllerPreparation(directory: string, input: Pick<Prepa } describe('fixed indexed publisher broker',()=>{ + it.each(['/outside/selected-plugin', '/home/freshell/provider-sibling/config', '/home/freshell/provider/.codex/sessions'])( + 'retains the actual rejected preparation destination without widening authority (%s)', async destination => { + await withHttpBroker(async ({ request, forwarded, broker: instance, directory }) => { + const binary = Buffer.from('owned publication artifact') + const owner = { ...sourceAuthorization(), hostBinaryPath: path.join(directory, 'freshell-session-host'), + binarySha256: createHash('sha256').update(binary).digest('hex') } + await fs.writeFile(owner.hostBinaryPath, binary) + instance.authorizeInitialProviderVolume(owner) + const generationId = 'grant-66666666-6666-4666-8666-666666666666' + const input = { ...publisherAuthorization(), soulId: owner.soulId, generationId, + hostBinaryPath: owner.hostBinaryPath, providerVolumeName: owner.providerVolumeName, + stageRoot: path.join(directory, 'scenarios/scenario/control/provider-roots', generationId), + configurationBinds: [{ sourceRelativePath: '.freshell/templates/codex-0', effectiveDestination: destination }], + resourceMounts: [], indexedSecretSources: [], + indexedFiles: { volumeName: `freshell-inputs-${createHash('sha256').update(`${owner.soulId}\0${generationId}`).digest('hex').slice(0, 24)}`, aliases: [] } } + const publication = await publishControllerPreparation(directory, input, '11111111-1111-4111-8111-111111111111', 'codex') + const capabilityPath = path.join(publication.capabilities, `${generationId}.json`) + const capability = JSON.parse(await fs.readFile(capabilityPath, 'utf8')) + capability.scope = 'PRIVATE_SCOPE_CONTENT_SENTINEL' + await fs.writeFile(capabilityPath, JSON.stringify(capability)) + await fs.mkdir(path.join(publication.metadata, 'templates')) + await fs.writeFile(path.join(publication.metadata, 'templates/codex-0'), 'PRIVATE_CONFIG_CONTENT_SENTINEL') + const initial = { Name: owner.providerVolumeName, Driver: 'local', Labels: { + 'com.freshell.installation-id': owner.installationId, 'com.freshell.soul-id': owner.soulId, + 'freshell.input.reservation': publication.context.preparedInputs.initialAuthReservation, + 'freshell.input.incarnation': input.incarnationId, 'freshell.input.intent': '1', + 'freshell.input.daemon': owner.daemonId, 'freshell.input.image': owner.imageRef, + 'freshell.input.binary-sha256': owner.binarySha256, + } } + expect((await request('POST', '/v1.47/volumes/create', initial)).status).toBe(201) + const before = forwarded.length + const denied = await request('POST', '/v1.47/volumes/create', { + Name: input.indexedFiles.volumeName, Driver: 'local', DriverOpts: indexedOptions, Labels: publisherLabels(input), + }) + expect(denied.status).toBe(403) + expect(forwarded).toHaveLength(before) + const diagnostic = instance.eventsSnapshot().find(event => event.reason === 'fixture publication unavailable at prepared_authorization: target_root') + expect(diagnostic).toMatchObject({ targetPath: destination, preparationStageRoot: input.stageRoot, + selectedWorkspaceRoots: ['/workspace'] }) + const events = JSON.stringify(instance.eventsSnapshot()) + expect(events).not.toContain('PRIVATE_SCOPE_CONTENT_SENTINEL') + expect(events).not.toContain('PRIVATE_CONFIG_CONTENT_SENTINEL') + expect(instance.receipts()).toEqual([]) + }, undefined, true) + }, + ) it.each(['exited', 'running', 'foreign-owner', 'changed-image', 'changed-host', 'foreign-capability'] as const)( 'qualifies a replacement publication against its retained owner (%s)', async (state) => { await withHttpBroker(async ({ directory, request, forwarded, broker: instance, setRuntimeRunning }) => { From e64a017829dcf011b54896f6f5d3cee18d732d01 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 13:12:54 -0700 Subject: [PATCH 268/493] fix: retain rejected preparation path in safe broker diagnostics --- scripts/testing/runtime-test-broker.ts | 29 ++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 010c7f472..661a5e266 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -8,6 +8,17 @@ const DOCKER_API_PREFIX = '/v1.47' const MAX_REQUEST_BYTES = 2 * 1024 * 1024 const INDEXED_TMPFS_OPTIONS = 'size=64m,mode=0700,uid=65534,gid=0,nr_inodes=1024,nosuid,nodev,noexec' +class PreparedInputTargetError extends Error { + constructor(readonly targetPath: string, readonly preparationStageRoot: string) { + super('prepared input target is outside selected fixture roots') + } +} + +function diagnosticPath(value: string): string | undefined { + return value.length <= 4096 && path.isAbsolute(value) && path.normalize(value) === value + && !/[\x00-\x1f\x7f]/.test(value) ? value : undefined +} + function indexedLabels(input: PreparedInputAuthorization): Record<string, string> { return { 'com.freshell.managed-inputs': 'true', @@ -116,6 +127,9 @@ export type BrokerEvent = { helperName?: string requestDigest?: string providerVolumeName?: string + targetPath?: string + preparationStageRoot?: string + selectedWorkspaceRoots?: string[] destructive: boolean unsafeAttempt: boolean } @@ -307,7 +321,7 @@ export class RestrictedDockerBroker { || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) const targets = new Set<string>() const claim = (value: string) => { - if (!destination(value)) throw new Error('prepared input target is outside selected fixture roots') + if (!destination(value)) throw new PreparedInputTargetError(value, input.stageRoot) if (targets.has(value)) throw new Error('prepared input target is duplicated') targets.add(value) } @@ -1359,8 +1373,8 @@ export class RestrictedDockerBroker { } this.record({ method: 'POST', url, decision: 'block', reason: `fixture publication unavailable at ${phase}`, destructive: false, unsafeAttempt: false }) } catch (error) { - // Only fixed validator categories are observable. Discard all other - // exception messages, which may include filesystem or input details. + // Keep fixed categories and the rejected pathname, never input contents + // or arbitrary exception messages. const categories: Record<string, string> = { 'prepared input owner/stage is outside this fixture run': 'owner_or_stage', 'prepared input metadata exceeds control profile': 'metadata_limit', @@ -1375,7 +1389,14 @@ export class RestrictedDockerBroker { 'prepared input generation changed; reconcile before replacement': 'changed_generation', } const category = error instanceof Error ? categories[error.message] ?? 'unclassified' : 'unclassified' - this.record({ method: 'POST', url, decision: 'block', reason: `fixture publication unavailable at ${phase}: ${category}`, destructive: false, unsafeAttempt: false }) + const target = error instanceof PreparedInputTargetError ? { + targetPath: diagnosticPath(error.targetPath), + preparationStageRoot: diagnosticPath(error.preparationStageRoot), + selectedWorkspaceRoots: [...this.policy.allowedWorkspaceRoots ?? []] + .filter(root => diagnosticPath(root) !== undefined), + } : {} + this.record({ method: 'POST', url, decision: 'block', reason: `fixture publication unavailable at ${phase}: ${category}`, + ...target, destructive: false, unsafeAttempt: false }) } } From 58becfc5365e418dd0254baa8c429d0d57592d18 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 13:43:28 -0700 Subject: [PATCH 269/493] fix: keep recovery observation in test builds --- crates/freshell-server/src/managed_runtime.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index a3539a0d6..7a5aca45f 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -220,6 +220,7 @@ impl ServerManagedRuntimeController { self.client.clone() } + #[cfg(test)] async fn recovery_in_flight(&self, soul_id: &str) -> bool { self.recovery .slots From 67ee32aabd6cedb6a99c5750b88254db61598c48 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 13:43:28 -0700 Subject: [PATCH 270/493] test: exercise managed OpenCode fixture selection through native discovery --- .../src/managed_provider_launch_tests.rs | 80 ++++++++++++++++++- .../managed-provider-environment-target.ts | 3 +- test/helpers/provider-parity-environment.ts | 5 ++ 3 files changed, 86 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 230727841..935b45f33 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -48,7 +48,7 @@ fn provider_parity_explicit_opencode_roots() { .unwrap(); let helper = std::process::Command::new("pnpm") .args(["exec", "tsx", "--eval", "import(process.argv[1]).then(module => process.stdout.write(JSON.stringify(module.providerParityCommands(process.argv[2]))))"]) - .arg(repo.join("test/integration/server/provider-parity-environment.ts")) + .arg(repo.join("test/helpers/provider-parity-environment.ts")) .arg(&workspace) .current_dir(repo) .output().unwrap(); @@ -95,6 +95,84 @@ fn provider_parity_explicit_opencode_roots() { } } +#[test] +fn managed_provider_environment_selects_only_its_cwd_configuration() { + use crate::managed_provider_references::sources::{ + discover_opencode, select_opencode_resource_roots, OpencodeDiscovery, + }; + let owned = tempfile::tempdir().unwrap(); + let workspace = owned.path().join("workspace"); + let home = owned.path().join("home"); + let global = home.join(".config/opencode"); + let unrelated = owned.path().join(".opencode"); + for directory in [&workspace, &global, &unrelated] { + std::fs::create_dir_all(directory).unwrap(); + } + let marker = unrelated.join("unrelated-resource"); + std::fs::write(&marker, "unrelated owned ancestor").unwrap(); + let global_config = global.join("opencode.json"); + std::fs::write(&global_config, "{}").unwrap(); + let repo = Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap(); + // This is the same selection function the full managed driver dispatches. + let helper = std::process::Command::new("pnpm") + .args(["exec", "tsx", "--eval", "import(process.argv[1]).then(module => process.stdout.write(JSON.stringify(module.managedEnvironmentOpencodeSelection())))"]) + .arg(repo.join("test/helpers/provider-parity-environment.ts")) + .current_dir(repo) + .output().unwrap(); + assert!(helper.status.success(), "owned fixture selector failed"); + let environment: BTreeMap<String, String> = serde_json::from_slice(&helper.stdout).unwrap(); + let lookup = |key: &str| environment.get(key).cloned(); + for scenario in [ + "healthy", + "missing-reference", + "startup-before-identity", + "preflight", + ] { + let cwd = workspace.join(scenario); + std::fs::create_dir_all(&cwd).unwrap(); + let selected = cwd.join("opencode.json"); + std::fs::write(&selected, "{}").unwrap(); + let request = || OpencodeDiscovery { + home: &home, + cwd: &cwd, + worktree: &workspace, + config_root: &global, + env: &lookup, + }; + let roots = select_opencode_resource_roots(&request()).unwrap(); + assert!( + !roots.iter().any(|root| root.logical_path == unrelated), + "managed fixture selected an unrelated ancestor outside its owned workspace" + ); + let sources = discover_opencode(request()).unwrap(); + assert!( + sources.iter().any(|source| source.logical_path == selected), + "explicit relative selection lost the scenario's cwd configuration" + ); + assert!( + sources + .iter() + .any(|source| source.logical_path == global_config), + "global plugin and absolute-reference configuration was lost" + ); + // Closing ancestor discovery must still reject a broken selected input. + std::fs::remove_file(&selected).unwrap(); + #[cfg(unix)] + { + std::os::unix::fs::symlink(cwd.join("missing-selected"), &selected).unwrap(); + assert!(discover_opencode(request()).is_err()); + } + } + assert_eq!( + std::fs::read_to_string(marker).unwrap(), + "unrelated owned ancestor" + ); +} + // Read only the fixture's actual native process (or its bounded hook ancestry). // argv/environment bytes stay transient; only profile compliance is reported. const NATIVE_EXEC_PROFILE: &str = r#" diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 54a4ef105..0cbfa82c4 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -9,6 +9,7 @@ import WebSocket from 'ws' import { WS_PROTOCOL_VERSION, type ClientMessage } from '../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig, type ManagedRuntimeView } from '../../test/e2e-browser/helpers/managed-runtime.js' import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../test/helpers/provider-parity-broker.js' +import { managedEnvironmentOpencodeSelection } from '../../test/helpers/provider-parity-environment.js' import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' @@ -239,7 +240,7 @@ export async function runManagedEnvironmentTarget() { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') // The synthetic grant is the only selected OneCLI input. RustServer merges // ambient env, so explicitly close optional host auth/CA references too. - const environment = { ...grants.serverEnv, HOME: home, XDG_CONFIG_HOME: xdg, + const environment = { ...grants.serverEnv, ...managedEnvironmentOpencodeSelection(), HOME: home, XDG_CONFIG_HOME: xdg, FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE: '', FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE: '' } rig = new ManagedRuntimeBrowserRig(repoRoot, 2, { ...environment, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY: '1' }, environment, 'release', { enabledProviders: ['opencode'] }, home, diff --git a/test/helpers/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts index 1ef3c2ec6..852bd9d48 100644 --- a/test/helpers/provider-parity-environment.ts +++ b/test/helpers/provider-parity-environment.ts @@ -1,6 +1,11 @@ import fs from 'node:fs' import path from 'node:path' +/** Select each managed environment scenario's own relative configuration. */ +export function managedEnvironmentOpencodeSelection(): Record<string, string> { + return {} +} + /** Establish selected fixture roots before the harness captures its policy. */ export function prepareProviderParityUserRoots(home: string, xdgConfigHome: string): void { for (const root of ['.claude', '.codex', '.amplifier']) { From 794ad50709fddbfcca61422708fd95b5b5bacd60 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:01:27 -0700 Subject: [PATCH 271/493] refactor: keep provider qualification assertions idiomatic --- crates/freshell-server/src/managed_mcp_capability.rs | 2 +- .../freshell-server/src/managed_provider_launch_tests.rs | 9 ++++----- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 6f966d77a..cedbf9425 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -6576,7 +6576,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .map(|(path, _)| (path.clone(), std::fs::read(path).unwrap())) .collect::<Vec<_>>(); if case == "unreadable-marker" { - std::fs::set_permissions(checkout.join(".git"), std::fs::Permissions::from_mode(0)) + std::fs::set_permissions(checkout.join(".git"), std::fs::Permissions::from_mode(0o0)) .unwrap(); } let sources = match case { diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 935b45f33..220250aaf 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -2215,7 +2215,7 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ let roots = observed.roots.unwrap(); assert_eq!(roots.checkout, workspace); assert_eq!(roots.canonical, workspace); - assert_eq!(observed.local_settings, [local.clone()]); + assert_eq!(observed.local_settings, std::slice::from_ref(&local)); assert_eq!(std::fs::read(&local).unwrap(), b"{}"); println!( "provider_source_observer {}", @@ -4062,7 +4062,7 @@ export default {{id:'closed-directory-discovery',async server(){{ for unreadable in [false, true] { let hidden = selected.with_extension("closed-unavailable"); if unreadable { - std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0)).unwrap(); + std::fs::set_permissions(&selected, std::fs::Permissions::from_mode(0o0)).unwrap(); } else { std::fs::rename(&selected, &hidden).unwrap(); } @@ -4076,9 +4076,8 @@ export default {{id:'closed-directory-discovery',async server(){{ } else { std::fs::rename(&hidden, &selected).unwrap(); } - let error = failed - .err() - .expect("selected unavailable resource must block the actual launch"); + let error = + failed.expect_err("selected unavailable resource must block the actual launch"); assert!( error.contains(field) && error.contains(if unreadable { "unreadable" } else { "missing" }) From 0497ddd0b3008952cb850504dea71609aeb4c516 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:01:27 -0700 Subject: [PATCH 272/493] fix: select each managed OpenCode fixture configuration --- test/helpers/provider-parity-environment.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/test/helpers/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts index 852bd9d48..425513f64 100644 --- a/test/helpers/provider-parity-environment.ts +++ b/test/helpers/provider-parity-environment.ts @@ -3,7 +3,10 @@ import path from 'node:path' /** Select each managed environment scenario's own relative configuration. */ export function managedEnvironmentOpencodeSelection(): Record<string, string> { - return {} + return { + OPENCODE_DISABLE_PROJECT_CONFIG: '1', + OPENCODE_CONFIG: './opencode.json', + } } /** Establish selected fixture roots before the harness captures its policy. */ From acabcef73f0ba4afd9c00a9b6adf61317ba8c92b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:22:40 -0700 Subject: [PATCH 273/493] test: preserve managed grant delivery failure evidence --- .../tooling/testing/runtime-sandbox.test.ts | 63 +++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 12ea4495a..06bd58fa1 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -149,6 +149,69 @@ describe('managed environment create failure retention', () => { expect(owner.runtime.containerLogs).not.toHaveBeenCalled() expect(owner.runtime.writeIncident).not.toHaveBeenCalled() }) + + it.each(['owned-credential', 'foreign-preparation', 'owned-unknown'] as const)('retains safe grant delivery evidence for %s before owned cleanup', async kind => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-grant-failure-')) + const home = path.join(directory, 'home') + await fs.mkdir(home) + const rig = new ManagedRuntimeBrowserRig(directory, 2, {}, {}, 'release', { enabledProviders: [] }, home) + const markers = ['AUTH_SECRET_MARKER', 'CONFIG_SECRET_MARKER', 'CONVERSATION_SECRET_MARKER'] + const incarnationId = '11111111-1111-4111-8111-111111111111' + const hostId = 'a'.repeat(64) + const runtimeRoot = path.join(rig.runtime.testRoot, 'r', 'grant-owned', 'x') + const lifecycle = path.join(path.dirname(runtimeRoot), 'evidence', 'lifecycle.jsonl') + const events = [ + { event: 'supervisor.host_started', data: { incarnationId, containerId: hostId } }, + { event: 'supervisor.activation_failed', data: { incarnationId, stage: 'deliver_execution_grant', + errorCode: 'HOST_UNREACHABLE', launchState: 'starting', errorClass: 'runtime_activation_failed' } }, + ] + await fs.mkdir(path.dirname(lifecycle), { recursive: true }) + await fs.writeFile(lifecycle, events.map(event => JSON.stringify(event)).join('\n') + '\n') + const reason = kind === 'owned-credential' ? 'OneCLI grant is unavailable' + : kind === 'foreign-preparation' ? 'prepared private source unreadable' : 'UNKNOWN' + const message = kind === 'owned-credential' + ? `resolve managed provider credentials: ${reason}: Permission denied (os error 13)` + : kind === 'foreign-preparation' ? reason : markers.join(' ') + const response: ErrorMessage = { type: 'error', code: 'PTY_SPAWN_FAILED', timestamp: new Date().toISOString(), + message: `managed runtime launch failed: runtime supervisor rejected request: HostUnreachable: ${message}; ${markers.join(' ')}` } + const childLogs = kind === 'owned-credential' + ? JSON.stringify({ event: 'provider.projection.transaction.failed', stage: 'validate', + cause: `provider transaction mounted file /synthetic/owned/path: Permission denied (os error 13); ${markers.join(' ')}` }) + : JSON.stringify({ event: 'session_host.fatal', error: markers.join(' ') }) + const logs = vi.spyOn(rig.runtime, 'containerLogs').mockImplementation(id => id === hostId ? childLogs : '') + const receipts = vi.spyOn(rig.runtime.broker, 'receiptIds').mockReturnValue(new Set(kind === 'foreign-preparation' ? [] : [hostId])) + rig.web = { capturedOutput: () => ({ stdout: '', stderr: markers.join(' ') }), + stop: async () => fs.rm(home, { recursive: true }) } as unknown as typeof rig.web + rig.supervisor = { containerId: 'd'.repeat(64), runtimeRoot } as typeof rig.supervisor + try { + expect(() => assertManagedTerminalCreated(rig, response)).toThrow() + receipts.mockRestore() + expect(await rig.stop()).toEqual({ ok: true, errors: [] }) + await expect(fs.stat(home)).rejects.toMatchObject({ code: 'ENOENT' }) + const stored = await fs.readFile(path.join(rig.runtime.evidenceDir, 'incidents/managed-environment-create-failure.json'), 'utf8') + const incident = JSON.parse(stored) + expect.soft(incident.reason).toBe(reason) + expect.soft(incident.supervisor).toMatchObject({ stage: 'deliver_execution_grant', errorCode: 'HOST_UNREACHABLE' }) + if (kind === 'foreign-preparation') { + expect.soft(incident.sessionHost).toBe('UNKNOWN') + expect.soft(logs.mock.calls.some(([id]) => id === hostId)).toBe(false) + } else if (kind === 'owned-credential') { + expect.soft(incident.sessionHost).toMatchObject({ containerId: hostId, + transactionFailures: [{ stage: 'validate', operation: 'mounted_file', errno: 'permission_denied' }] }) + expect.soft(incident.osError).toBe(13) + expect.soft(logs.mock.calls.some(([id]) => id === hostId)).toBe(true) + } else { + expect.soft(incident.sessionHost).toMatchObject({ containerId: hostId, + workerFailures: [{ operation: 'unknown', errno: 'unknown' }] }) + expect.soft(logs.mock.calls.some(([id]) => id === hostId)).toBe(true) + } + for (const marker of markers) expect.soft(stored).not.toContain(marker) + } finally { + await fs.rm(rig.runtime.testRoot, { recursive: true, force: true }) + await fs.rm(directory, { recursive: true, force: true }) + vi.restoreAllMocks() + } + }) }) function useOwnedHarnessArtifacts(harness: RuntimeHarness, host: string): void { From 50547458fc07bc2d05b366e18e7c0d9c7cce1ce6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:42:07 -0700 Subject: [PATCH 274/493] test: prepare owned broker before grant diagnostics --- .../tooling/testing/runtime-sandbox.test.ts | 79 +++++++++++-------- 1 file changed, 47 insertions(+), 32 deletions(-) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 06bd58fa1..64116b463 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -153,38 +153,49 @@ describe('managed environment create failure retention', () => { it.each(['owned-credential', 'foreign-preparation', 'owned-unknown'] as const)('retains safe grant delivery evidence for %s before owned cleanup', async kind => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-grant-failure-')) const home = path.join(directory, 'home') - await fs.mkdir(home) - const rig = new ManagedRuntimeBrowserRig(directory, 2, {}, {}, 'release', { enabledProviders: [] }, home) - const markers = ['AUTH_SECRET_MARKER', 'CONFIG_SECRET_MARKER', 'CONVERSATION_SECRET_MARKER'] - const incarnationId = '11111111-1111-4111-8111-111111111111' - const hostId = 'a'.repeat(64) - const runtimeRoot = path.join(rig.runtime.testRoot, 'r', 'grant-owned', 'x') - const lifecycle = path.join(path.dirname(runtimeRoot), 'evidence', 'lifecycle.jsonl') - const events = [ - { event: 'supervisor.host_started', data: { incarnationId, containerId: hostId } }, - { event: 'supervisor.activation_failed', data: { incarnationId, stage: 'deliver_execution_grant', - errorCode: 'HOST_UNREACHABLE', launchState: 'starting', errorClass: 'runtime_activation_failed' } }, - ] - await fs.mkdir(path.dirname(lifecycle), { recursive: true }) - await fs.writeFile(lifecycle, events.map(event => JSON.stringify(event)).join('\n') + '\n') - const reason = kind === 'owned-credential' ? 'OneCLI grant is unavailable' - : kind === 'foreign-preparation' ? 'prepared private source unreadable' : 'UNKNOWN' - const message = kind === 'owned-credential' - ? `resolve managed provider credentials: ${reason}: Permission denied (os error 13)` - : kind === 'foreign-preparation' ? reason : markers.join(' ') - const response: ErrorMessage = { type: 'error', code: 'PTY_SPAWN_FAILED', timestamp: new Date().toISOString(), - message: `managed runtime launch failed: runtime supervisor rejected request: HostUnreachable: ${message}; ${markers.join(' ')}` } - const childLogs = kind === 'owned-credential' - ? JSON.stringify({ event: 'provider.projection.transaction.failed', stage: 'validate', - cause: `provider transaction mounted file /synthetic/owned/path: Permission denied (os error 13); ${markers.join(' ')}` }) - : JSON.stringify({ event: 'session_host.fatal', error: markers.join(' ') }) - const logs = vi.spyOn(rig.runtime, 'containerLogs').mockImplementation(id => id === hostId ? childLogs : '') - const receipts = vi.spyOn(rig.runtime.broker, 'receiptIds').mockReturnValue(new Set(kind === 'foreign-preparation' ? [] : [hostId])) - rig.web = { capturedOutput: () => ({ stdout: '', stderr: markers.join(' ') }), - stop: async () => fs.rm(home, { recursive: true }) } as unknown as typeof rig.web - rig.supervisor = { containerId: 'd'.repeat(64), runtimeRoot } as typeof rig.supervisor + let rig: ManagedRuntimeBrowserRig | undefined + let restoreReceipts: (() => void) | undefined try { - expect(() => assertManagedTerminalCreated(rig, response)).toThrow() + await fs.mkdir(home, { mode: 0o700 }) + await fs.mkdir(path.join(directory, 'test/runtime'), { recursive: true }) + await fs.writeFile(path.join(directory, 'test/runtime/gate-manifest.json'), '{}') + const host = path.join(directory, 'host') + await fs.writeFile(host, 'owned fixture host artifact') + rig = new ManagedRuntimeBrowserRig(directory, 2, {}, {}, 'release', { enabledProviders: [] }, home, + { workspaceRoots: [directory], bootstrapFiles: [], defaultCwd: directory }) + useOwnedHarnessArtifacts(rig.runtime, host) + await rig.runtime.prepare() + const markers = ['AUTH_SECRET_MARKER', 'CONFIG_SECRET_MARKER', 'CONVERSATION_SECRET_MARKER'] + const incarnationId = '11111111-1111-4111-8111-111111111111' + const hostId = 'a'.repeat(64) + const runtimeRoot = path.join(rig.runtime.testRoot, 'r', 'grant-owned', 'x') + const lifecycle = path.join(path.dirname(runtimeRoot), 'evidence', 'lifecycle.jsonl') + const events = [ + { event: 'supervisor.host_started', data: { incarnationId, containerId: hostId } }, + { event: 'supervisor.activation_failed', data: { incarnationId, stage: 'deliver_execution_grant', + errorCode: 'HOST_UNREACHABLE', launchState: 'starting', errorClass: 'runtime_activation_failed' } }, + ] + await fs.mkdir(path.dirname(lifecycle), { recursive: true }) + await fs.writeFile(lifecycle, events.map(event => JSON.stringify(event)).join('\n') + '\n') + const reason = kind === 'owned-credential' ? 'OneCLI grant is unavailable' + : kind === 'foreign-preparation' ? 'prepared private source unreadable' : 'UNKNOWN' + const message = kind === 'owned-credential' + ? `resolve managed provider credentials: ${reason}: Permission denied (os error 13)` + : kind === 'foreign-preparation' ? reason : markers.join(' ') + const response: ErrorMessage = { type: 'error', code: 'PTY_SPAWN_FAILED', timestamp: new Date().toISOString(), + message: `managed runtime launch failed: runtime supervisor rejected request: HostUnreachable: ${message}; ${markers.join(' ')}` } + const childLogs = kind === 'owned-credential' + ? JSON.stringify({ event: 'provider.projection.transaction.failed', stage: 'validate', + cause: `provider transaction mounted file /synthetic/owned/path: Permission denied (os error 13); ${markers.join(' ')}` }) + : JSON.stringify({ event: 'session_host.fatal', error: markers.join(' ') }) + const logs = vi.spyOn(rig.runtime, 'containerLogs').mockImplementation(id => id === hostId ? childLogs : '') + const receipts = vi.spyOn(rig.runtime.broker, 'receiptIds').mockReturnValue(new Set(kind === 'foreign-preparation' ? [] : [hostId])) + restoreReceipts = () => receipts.mockRestore() + rig.web = { capturedOutput: () => ({ stdout: '', stderr: markers.join(' ') }), + stop: async () => fs.rm(home, { recursive: true }) } as unknown as typeof rig.web + rig.supervisor = { containerId: 'd'.repeat(64), runtimeRoot } as typeof rig.supervisor + const owner = rig + expect(() => assertManagedTerminalCreated(owner, response)).toThrow() receipts.mockRestore() expect(await rig.stop()).toEqual({ ok: true, errors: [] }) await expect(fs.stat(home)).rejects.toMatchObject({ code: 'ENOENT' }) @@ -207,7 +218,11 @@ describe('managed environment create failure retention', () => { } for (const marker of markers) expect.soft(stored).not.toContain(marker) } finally { - await fs.rm(rig.runtime.testRoot, { recursive: true, force: true }) + restoreReceipts?.() + if (rig) { + await rig.stop() + await fs.rm(rig.runtime.testRoot, { recursive: true, force: true }) + } await fs.rm(directory, { recursive: true, force: true }) vi.restoreAllMocks() } From 99c17a2a6d595b51e3eff8b83d9e972dc7ef6c67 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:47:22 -0700 Subject: [PATCH 275/493] fix: retain managed grant delivery diagnostics --- scripts/testing/managed-environment-create.ts | 32 +++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 643dd2e2e..b91818b32 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -4,6 +4,7 @@ import http from 'node:http' import path from 'node:path' import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' import { dockerSocketPath } from './runtime-sandbox.js' +import { providerParityFailureDiagnostics } from '../../test/helpers/provider-parity-environment.js' export async function readManagedEnvironmentDaemonId(): Promise<string> { const dockerHost = process.env.DOCKER_HOST @@ -26,11 +27,14 @@ export async function readManagedEnvironmentDaemonId(): Promise<string> { type ManagedCreateOwner = { web: { capturedOutput(): { stdout: string; stderr: string } } - runtime: { containerLogs(id: string): string; writeIncident(name: string, value: unknown): void } + runtime: { containerLogs(id: string): string; writeIncident(name: string, value: unknown): void; + broker?: { receiptIds(): Set<string> } } supervisor: { containerId: string; runtimeRoot?: string } } const failureReasons = [ + 'OneCLI grant is unavailable', + 'prepared private source unreadable', 'initial provider source lost its current prepared reservation', 'runtime image must be pinned by sha256 id/digest', 'provider volume name is not installation-scoped', @@ -75,6 +79,7 @@ const activationStages = ['capability_pending', 'ensure_runtime_directory', 'ens 'revalidate_initial_provider_source', 'create_stopped_runtime', 'commit_backend_identity', 'load_owned_handle', 'authenticate_running_host', 'read_running_host_status', 'adopt_running_worker', 'start_session_host', 'authenticate_started_host', 'commit_execution_grant', 'enable_long_lived_runtime', + 'deliver_execution_grant', 'record_native_identity', 'commit_running_state', 'commit_provider_inputs', 'load_running_view'] export function assertManagedTerminalCreated(owner: ManagedCreateOwner, @@ -97,6 +102,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, const code = ErrorCode.safeParse(response.code) const web = owner.web.capturedOutput() let supervisor: Record<string, string> | 'UNKNOWN' = 'UNKNOWN' + let sessionHost: Record<string, unknown> | 'UNKNOWN' = 'UNKNOWN' try { let lifecycle = '' if (owner.supervisor.runtimeRoot) { @@ -107,6 +113,23 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, if (fs.statSync(file).size <= 2 * 1024 * 1024) lifecycle = fs.readFileSync(file, 'utf8').slice(-20_000) } catch { /* Missing owning-file evidence remains unknown. */ } } + // Host identities come from this supervisor's owned lifecycle file. + // A matching broker receipt is required before reading that host's logs. + const startedHosts = new Map<string, string>() + let failedIncarnation: string | undefined + for (const line of lifecycle.split('\n')) { + try { + const entry = JSON.parse(line) + const data = entry.data + if (typeof data?.incarnationId !== 'string') continue + if (entry.event === 'supervisor.host_started' + && typeof data.containerId === 'string' && /^[a-f0-9]{64}$/.test(data.containerId)) { + startedHosts.set(data.incarnationId, data.containerId) + } else if (entry.event === 'supervisor.activation_failed') { + failedIncarnation = data.incarnationId + } + } catch { /* Invalid owning-file evidence cannot select a host. */ } + } const logs = `${owner.runtime.containerLogs(owner.supervisor.containerId)}\n${lifecycle}` for (const line of logs.split('\n')) { try { @@ -119,11 +142,16 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, 'HOST_UNREACHABLE'].includes(data?.errorCode) ? data.errorCode : 'UNKNOWN' } } catch { /* Non-JSON owning-process output is not retained. */ } } + const hostId = failedIncarnation && startedHosts.get(failedIncarnation) + if (hostId && owner.runtime.broker?.receiptIds().has(hostId)) { + sessionHost = { containerId: hostId, + ...providerParityFailureDiagnostics(owner.runtime.containerLogs(hostId)) } + } } catch { /* An absent owned supervisor leaves its observation unknown. */ } owner.runtime.writeIncident('managed-environment-create-failure', { severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), server: { source: 'owned-server-typed-reply', - capturedOutputAvailable: Boolean(web.stdout || web.stderr) }, supervisor, + capturedOutputAvailable: Boolean(web.stdout || web.stderr) }, supervisor, sessionHost, }) } assert.equal(response.type, 'terminal.created') From 4e4da568d00b464c791bfb95f85105a2b12124f9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 15:20:55 -0700 Subject: [PATCH 276/493] test: reproduce native auth preparation under private provider identity --- .../src/provider_input_preparation.rs | 223 ++++++++++++++++++ 1 file changed, 223 insertions(+) diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 12b02d0b2..bd280c483 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -204,6 +204,7 @@ pub(crate) fn final_claude_source_request( > { use freshell_platform::claude_inputs::{observation::ObservationRequest, ClaudeInputManifest}; use sha2::{Digest, Sha256}; + let Some(context) = context.filter(|context| context.preparation.provider() == "claude") else { return Ok(None); }; @@ -614,6 +615,228 @@ mod tests { use freshell_runtime_protocol::*; use sha2::{Digest, Sha256}; + #[cfg(target_os = "linux")] + fn private_native_auth_case(case: &str, test: &str) { + use std::os::unix::fs::{MetadataExt, PermissionsExt}; + const CHILD: &str = "FRESHELL_TEST_PRIVATE_NATIVE_AUTH_ROOT"; + if let Some(root) = std::env::var_os(CHILD) { + assert_eq!(unsafe { libc::geteuid() }, 0); + let status = std::fs::read_to_string("/proc/self/status").unwrap(); + let capabilities = status + .lines() + .find_map(|line| line.strip_prefix("CapEff:\t")) + .unwrap(); + assert_eq!( + u64::from_str_radix(capabilities, 16).unwrap(), + (1 << 0) | (1 << 6) | (1 << 7) + ); + let root = std::path::PathBuf::from(root); + let home = root.join("home"); + let stage = root.join("stage"); + let bytes = std::fs::read(stage.join(".freshell/input-manifest.json")).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:4000".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + config: vec![], + mcp_capability: Some(capability.clone()), + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id, + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(&bytes)), + source_count: 1, + alias_count: 0, + recipe_count: 1, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }), + }; + // The deployed controller cannot traverse this private provider HOME. + assert!( + freshell_platform::native_auth_exchange::read_selected_native_auth(&home, &[]) + .is_err() + ); + let mut secrets = ResolvedProviderSecrets::default(); + let result = prepare( + Some(&context), + &mut secrets, + &ProviderPathInputs::default(), + &stage, + &home, + ); + match case { + "selected" => { + let generated = result + .expect("private native auth must be read under the provider identity"); + assert!( + generated["FRESHELL_CONFIG_REFERENCE_0"].contains("selected-native-nonce") + ); + assert!(!generated + .values() + .any(|value| value.contains("unrequested-native-marker"))); + } + "absent" => { + result.expect("a provider-readable absent auth store must remain optional"); + } + "foreign" => assert!(result + .expect_err("foreign native auth must be refused") + .contains("escaped owned HOME")), + "reserved" => assert!(result + .expect_err( + "reserved native keys must remain invalid even with no selected keys" + ) + .contains("native auth file cannot be decoded")), + _ => unreachable!(), + } + return; + } + assert_eq!( + unsafe { libc::geteuid() }, + 0, + "this process/identity test requires the documented disposable sandbox" + ); + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let stage = owned.path().join("stage"); + std::fs::set_permissions(owned.path(), std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::create_dir_all(stage.join(".freshell/source-views")).unwrap(); + std::fs::create_dir_all(home.join(".local/share/opencode")).unwrap(); + let auth = home.join(".local/share/opencode/auth.json"); + match case { + "selected" => std::fs::write(&auth, r#"{"selected":{"type":"wellknown","key":"AUTH_NONCE","token":"selected-native-nonce"},"other":{"type":"wellknown","key":"OTHER_NONCE","token":"unrequested-native-marker"}}"#).unwrap(), + "reserved" => std::fs::write(&auth, r#"{"shadow":{"type":"wellknown","key":"FRESHELL_CONFIG_REFERENCE_0","token":"reserved-native-marker"}}"#).unwrap(), + "foreign" => { + let outside = owned.path().join("outside-auth"); + std::fs::write(&outside, "outside-native-marker").unwrap(); + std::os::unix::fs::symlink(outside, &auth).unwrap(); + } + "absent" => {}, _ => unreachable!(), + } + for directory in [ + &home, + &home.join(".local"), + &home.join(".local/share"), + &home.join(".local/share/opencode"), + ] { + crate::set_owner(directory, 65534, 0).unwrap(); + std::fs::set_permissions(directory, std::fs::Permissions::from_mode(0o700)).unwrap(); + } + if matches!(case, "selected" | "reserved") { + crate::set_owner(&auth, 65534, 0).unwrap(); + std::fs::set_permissions(&auth, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + let source = if case == "selected" { + r#"{"fixture":"{env:AUTH_NONCE}"}"# + } else { + "{}" + }; + std::fs::write(stage.join(".freshell/source-views/source-0"), source).unwrap(); + let manifest = SourceManifest { + version: 1, + cwd: owned.path().to_owned(), + original_home: home.clone(), + global_config_root: None, + sources: vec![SourceView { + discovery_role: DiscoveryRole::Workspace, + logical_path: owned.path().join("opencode.json"), + native_destination: None, + lane: SourceLane::Main, + unexpanded_relative_path: ".freshell/source-views/source-0".into(), + template_relative_path: ".freshell/templates/source-0".into(), + template_sha256: None, + generated_name: Some("FRESHELL_CONFIG_REFERENCE_0".into()), + occurrences: vec![], + referenced_variables: if case == "selected" { + vec!["AUTH_NONCE".into()] + } else { + vec![] + }, + path_variables: vec![], + original_form: false, + }], + files: vec![], + frozen_inputs: vec![], + user_owns_freshell_mcp: true, + inline_edits: Default::default(), + rebind_enabled: true, + native_source_flags: vec![], + resource_roots: vec![], + }; + std::fs::write( + stage.join(".freshell/input-manifest.json"), + serde_json::to_vec(&manifest).unwrap(), + ) + .unwrap(); + let before = std::fs::metadata(&home).unwrap(); + let joined = std::process::Command::new("/usr/bin/setpriv") + .args([ + "--bounding-set=-all,+chown,+setuid,+setgid", + "--inh-caps=-all", + "--ambient-caps=-all", + ]) + .arg(std::env::current_exe().unwrap()) + .args(["--exact", test, "--nocapture", "--test-threads=1"]) + .env(CHILD, owned.path()) + .status() + .unwrap(); + let after = std::fs::metadata(&home).unwrap(); + assert_eq!( + (before.uid(), before.gid(), before.mode()), + (after.uid(), after.gid(), after.mode()) + ); + assert_eq!(after.mode() & 0o777, 0o700); + assert!( + joined.success(), + "the joined controller-capability child must preserve the private native auth contract" + ); + } + + #[cfg(target_os = "linux")] + #[test] + fn provider_secret_native_auth_private_home_selected() { + private_native_auth_case( + "selected", + "provider_input_preparation::tests::provider_secret_native_auth_private_home_selected", + ); + } + #[cfg(target_os = "linux")] + #[test] + fn provider_secret_native_auth_private_home_absent() { + private_native_auth_case( + "absent", + "provider_input_preparation::tests::provider_secret_native_auth_private_home_absent", + ); + } + #[cfg(target_os = "linux")] + #[test] + fn provider_secret_native_auth_private_home_foreign() { + private_native_auth_case( + "foreign", + "provider_input_preparation::tests::provider_secret_native_auth_private_home_foreign", + ); + } + #[cfg(target_os = "linux")] + #[test] + fn provider_secret_native_auth_private_home_reserved() { + private_native_auth_case( + "reserved", + "provider_input_preparation::tests::provider_secret_native_auth_private_home_reserved", + ); + } + #[test] fn provider_secret_repeated_native_source_views_release_live_storage_between_occurrences() { let owned = tempfile::tempdir().unwrap(); From bece73cfb3e782f68eab83619cb64fa8000a58db Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 15:35:00 -0700 Subject: [PATCH 277/493] test(managed): run native auth cases under controller identity --- .../src/provider_input_preparation.rs | 72 +++++++++++-------- .../managed-provider-source-sandbox.py | 25 ++++--- 2 files changed, 60 insertions(+), 37 deletions(-) diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index bd280c483..10ade272d 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -619,7 +619,8 @@ mod tests { fn private_native_auth_case(case: &str, test: &str) { use std::os::unix::fs::{MetadataExt, PermissionsExt}; const CHILD: &str = "FRESHELL_TEST_PRIVATE_NATIVE_AUTH_ROOT"; - if let Some(root) = std::env::var_os(CHILD) { + let restricted = std::env::var_os("FRESHELL_TEST_RESTRICTED_READER").is_some(); + if restricted { assert_eq!(unsafe { libc::geteuid() }, 0); let status = std::fs::read_to_string("/proc/self/status").unwrap(); let capabilities = status @@ -630,6 +631,8 @@ mod tests { u64::from_str_radix(capabilities, 16).unwrap(), (1 << 0) | (1 << 6) | (1 << 7) ); + } + if let Some(root) = std::env::var_os(CHILD) { let root = std::path::PathBuf::from(root); let home = root.join("home"); let stage = root.join("stage"); @@ -664,11 +667,13 @@ mod tests { indexed_files: None, }), }; - // The deployed controller cannot traverse this private provider HOME. - assert!( - freshell_platform::native_auth_exchange::read_selected_native_auth(&home, &[]) - .is_err() - ); + if restricted { + // The deployed controller cannot traverse this private provider HOME. + assert!( + freshell_platform::native_auth_exchange::read_selected_native_auth(&home, &[]) + .is_err() + ); + } let mut secrets = ResolvedProviderSecrets::default(); let result = prepare( Some(&context), @@ -703,11 +708,11 @@ mod tests { } return; } - assert_eq!( - unsafe { libc::geteuid() }, - 0, - "this process/identity test requires the documented disposable sandbox" - ); + let identity = if restricted { + (65534, 0) + } else { + (unsafe { libc::geteuid() }, unsafe { libc::getegid() }) + }; let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let stage = owned.path().join("stage"); @@ -725,18 +730,24 @@ mod tests { } "absent" => {}, _ => unreachable!(), } - for directory in [ - &home, - &home.join(".local"), - &home.join(".local/share"), - &home.join(".local/share/opencode"), - ] { - crate::set_owner(directory, 65534, 0).unwrap(); - std::fs::set_permissions(directory, std::fs::Permissions::from_mode(0o700)).unwrap(); - } if matches!(case, "selected" | "reserved") { - crate::set_owner(&auth, 65534, 0).unwrap(); std::fs::set_permissions(&auth, std::fs::Permissions::from_mode(0o600)).unwrap(); + if restricted { + crate::set_owner(&auth, identity.0, identity.1).unwrap(); + } + } + // Set permissions while still the owner, then transfer deepest paths + // first. The controller has neither FOWNER nor DAC traversal rights. + for directory in [ + home.join(".local/share/opencode"), + home.join(".local/share"), + home.join(".local"), + home.clone(), + ] { + std::fs::set_permissions(&directory, std::fs::Permissions::from_mode(0o700)).unwrap(); + if restricted { + crate::set_owner(&directory, identity.0, identity.1).unwrap(); + } } let source = if case == "selected" { r#"{"fixture":"{env:AUTH_NONCE}"}"# @@ -781,18 +792,23 @@ mod tests { ) .unwrap(); let before = std::fs::metadata(&home).unwrap(); - let joined = std::process::Command::new("/usr/bin/setpriv") - .args([ - "--bounding-set=-all,+chown,+setuid,+setgid", - "--inh-caps=-all", - "--ambient-caps=-all", - ]) - .arg(std::env::current_exe().unwrap()) + // Inherit the wrapper's exact capabilities; dropping the bounding set + // here would itself require SETPCAP, which the controller never has. + let joined = std::process::Command::new(std::env::current_exe().unwrap()) .args(["--exact", test, "--nocapture", "--test-threads=1"]) .env(CHILD, owned.path()) .status() .unwrap(); let after = std::fs::metadata(&home).unwrap(); + // Join provider-owned cleanup before assertions can panic, preserving + // the private HOME mode even when the behavioral child fails. + crate::run_provider_refresh_cleanup( + identity.0, + identity.1, + "/usr/bin/find", + &[home.to_str().unwrap(), "-mindepth", "1", "-delete"], + ) + .unwrap(); assert_eq!( (before.uid(), before.gid(), before.mode()), (after.uid(), after.gid(), after.mode()) diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 080a76d65..055f7b350 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -20,7 +20,7 @@ def docker(*args, check=True): def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--provider",choices=["amplifier","codex","claude","claude-reader"],default="amplifier") + parser.add_argument("--provider",choices=["amplifier","codex","claude","claude-reader","native-auth-reader"],default="amplifier") parser.add_argument("--test-binary", type=Path, required=True) parser.add_argument("--receipt-dir", type=Path, required=True) args = parser.parse_args() @@ -29,18 +29,23 @@ def main(): "codex": "managed_mcp_capability::tests::managed_provider_codex_native_typed_inputs_survive_transport", "claude": "managed_provider_sdk_tests::managed_provider_claude_default_sdk_initializes_fixed_mcp_without_a_prompt", "claude-reader": "claude_source_observation::tests::claude_final_reader_real_pipe_bounds_binding_failures_and_joins", + "native-auth-reader": "provider_input_preparation::tests::provider_secret_native_auth_private_home_", }[args.provider] + native_auth_reader = args.provider == "native-auth-reader" + selection = [selected_test] if native_auth_reader else ["--exact", selected_test] + expected_tests = [selected_test + case for case in ("absent", "foreign", "reserved", "selected")] if native_auth_reader else [selected_test] binary = args.test_binary.resolve(strict=True) - artifact_prefix = "freshell_session_host" if args.provider == "claude-reader" else "freshell_server" + artifact_prefix = "freshell_session_host" if args.provider in ("claude-reader", "native-auth-reader") else "freshell_server" if not binary.is_file() or not re.fullmatch(artifact_prefix + r"-[0-9a-f]{16}", binary.name): raise RuntimeError("source fixture requires its compiled server test artifact") expected_directory = Path(__file__).resolve().parents[2] / "target/debug/deps" if binary.parent != expected_directory.resolve(): raise RuntimeError("source fixture test artifact must belong to this worktree") - listing = subprocess.run([str(binary), "--exact", selected_test, "--list"], + listing = subprocess.run([str(binary), *selection, "--list"], env={"PATH": "/usr/bin:/bin"}, text=True, capture_output=True, timeout=20, check=True) - if f"{selected_test}: test" not in listing.stdout.splitlines(): + listed_tests = [line.removesuffix(": test") for line in listing.stdout.splitlines() if line.endswith(": test")] + if sorted(listed_tests) != sorted(expected_tests): raise RuntimeError("source fixture compiled artifact contains no exact selected test") if not args.receipt_dir.is_dir(): raise RuntimeError("source fixture receipt directory must already exist") @@ -54,11 +59,12 @@ def main(): created = False receipt = {"image": IMAGE, "testArtifactSha256": digest, "test": selected_test, "runId": name, "rootless": True} - reader = args.provider == "claude-reader" + reader = args.provider in ("claude-reader", "native-auth-reader") identity_args = ["--cap-add", "CHOWN", "--cap-add", "SETGID", "--cap-add", "SETUID", "--env", "FRESHELL_TEST_RESTRICTED_READER=1"] if reader else [] + test_env = ["--env", "RUST_TEST_THREADS=4", "--env", "TOKIO_WORKER_THREADS=2"] if native_auth_reader else [] try: docker("create", "--name", name, "--label", f"{LABEL}={name}", - "--network", "none", "--read-only", "--user", "0:0" if reader else "65534:0", "--cap-drop", "ALL", *identity_args, + "--network", "none", "--read-only", "--user", "0:0" if reader else "65534:0", "--cap-drop", "ALL", *identity_args, *test_env, "--security-opt", "no-new-privileges:true", "--pids-limit", "128", "--memory", "2g", "--mount", f"type=bind,src={binary},dst=/runtime/source-consumer-tests,readonly", "--tmpfs", "/fixture:rw,exec,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", @@ -68,7 +74,7 @@ def main(): "--env", "AMPLIFIER_HOME=/home/freshell/provider/.amplifier", "--env", "FRESHELL_TEST_EXECUTABLE_TMP_DIR=/fixture", "--entrypoint", "/runtime/source-consumer-tests", IMAGE, - "--exact", selected_test, "--nocapture", "--test-threads", "1") + *selection, "--nocapture", "--test-threads", "4" if native_auth_reader else "1") created = True result = docker("start", "--attach", name, check=False) receipt["exitCode"] = result.returncode @@ -79,7 +85,8 @@ def main(): value = json.loads(line.split(marker, 1)[1]) cases.append({"provider": value["provider"], "case": value["case"], "digest": value["digest"]}) receipt["consumedCases"] = cases - if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":5,"claude":1,"claude-reader":0}[args.provider] or "test result: ok. 1 passed;" not in result.stdout): + expected_count = len(expected_tests) + if result.returncode == 0 and (len(cases) != {"amplifier":18,"codex":5,"claude":1,"claude-reader":0,"native-auth-reader":0}[args.provider] or f"test result: ok. {expected_count} passed;" not in result.stdout): receipt["exitCode"] = 2 receipt["failureClass"] = "native_consumer_count_mismatch" # Only the Rust fixture's classified assertion/digest receipt is emitted. @@ -94,7 +101,7 @@ def main(): raise RuntimeError("source fixture cleanup ownership changed") docker("rm", "--force", value["Id"]) receipt["ownedCleanup"] = docker("container", "inspect", name, check=False).returncode != 0 - receipt_name = {"amplifier":"task002-native-source-consumers.json", "codex":"task002-native-codex-source-consumers.json", "claude":"task002-native-claude-sdk-consumers.json", "claude-reader":"task002-repair13-native-claude-reader.json"}[args.provider] + receipt_name = {"amplifier":"task002-native-source-consumers.json", "codex":"task002-native-codex-source-consumers.json", "claude":"task002-native-claude-sdk-consumers.json", "claude-reader":"task002-repair13-native-claude-reader.json", "native-auth-reader":"task006-native-auth-reader.json"}[args.provider] (args.receipt_dir / receipt_name).write_text(json.dumps(receipt, sort_keys=True) + "\n") if not created or not receipt["ownedCleanup"]: raise RuntimeError("source fixture failed its owned cleanup proof") From 6902f8e2063b97cc012d65389c6b8d194aa3b9c8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 15:40:22 -0700 Subject: [PATCH 278/493] test(managed): bind native auth fixture to indexed generation --- .../src/provider_input_preparation.rs | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 10ade272d..88911a0e7 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -653,7 +653,7 @@ mod tests { config: vec![], mcp_capability: Some(capability.clone()), prepared_inputs: Some(PreparedProviderInputs { - generation_id: capability.grant_id, + generation_id: capability.grant_id.clone(), initial_auth_reservation: None, manifest: ProviderInputManifestReference { relative_path: ".freshell/input-manifest.json".into(), @@ -664,7 +664,13 @@ mod tests { }, config_binds: vec![], resource_mounts: vec![], - indexed_files: None, + indexed_files: Some(IndexedProviderFiles { + volume_name: indexed_provider_volume_name( + &SoulId::new(), + &capability.grant_id, + ), + aliases: vec![], + }), }), }; if restricted { From cbdb5ea96be1740604a15297f03f4ba2cb82d7ac Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 15:51:15 -0700 Subject: [PATCH 279/493] fix(managed): read native auth as the provider identity --- crates/freshell-session-host/src/main.rs | 10 +- .../src/native_auth_reader.rs | 234 ++++++++++++++++++ .../src/provider_input_preparation.rs | 159 ++++++++---- .../managed-provider-source-sandbox.py | 8 + 4 files changed, 359 insertions(+), 52 deletions(-) create mode 100644 crates/freshell-session-host/src/native_auth_reader.rs diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 19611d6f1..c92bd5c8b 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1,6 +1,7 @@ mod claude_source_observation; mod command_journal; mod control; +mod native_auth_reader; mod output_journal; mod provider_input_preparation; mod provider_secret_resolution; @@ -121,6 +122,7 @@ async fn run() -> Result<(), String> { Some("serve") => serve(&args[2..]).await, Some("worker") => worker(&args[2..]).await, Some("observe-claude-sources") if args.len() == 2 => claude_source_observation::serve_stdio(), + Some("native-auth-stdio") if args.len() == 2 => native_auth_reader::serve_stdio(), Some("observe-claude-sources-private") if args.len() == 2 => { verify_private_helper_namespace( Path::new("/home/freshell/provider"), @@ -954,6 +956,7 @@ async fn grant_execution( launch.mode.as_str(), launch.provider_secret_references.as_slice(), launch.provider_launch_context.as_ref(), + (launch.run_as_uid, launch.run_as_gid), ) }) .or_else(|| { @@ -962,10 +965,11 @@ async fn grant_execution( launch.provider.native_provider(), launch.provider_secret_references.as_slice(), launch.provider_launch_context.as_ref(), + (launch.run_as_uid, launch.run_as_gid), ) }) }); - let mut prepared_secrets = if let Some((provider, references, context)) = source { + let mut prepared_secrets = if let Some((provider, references, context, identity)) = source { let mut secrets = provider_secret_resolution::resolve_child_secrets( provider, references, @@ -978,7 +982,7 @@ async fn grant_execution( ) })?; if matches!(provider, "opencode" | "codex" | "claude" | "amplifier") { - let generated = provider_input_preparation::prepare( + let generated = provider_input_preparation::prepare_for_identity( context, &mut secrets, &provider_path_inputs, @@ -987,7 +991,9 @@ async fn grant_execution( .private_mount .join("user-provider"), &state.provider_config_paths.provider_home, + identity, ) + .await .map_err(|error| { RuntimeError::new( if error.starts_with("configuration reference selection changed") { diff --git a/crates/freshell-session-host/src/native_auth_reader.rs b/crates/freshell-session-host/src/native_auth_reader.rs new file mode 100644 index 000000000..68b8cf620 --- /dev/null +++ b/crates/freshell-session-host/src/native_auth_reader.rs @@ -0,0 +1,234 @@ +//! Selected native credentials cross only an owned provider-identity pipe. +use freshell_platform::native_auth_exchange::{ + read_selected_native_auth, validate_keys, validate_selected, MAX_AUTH_FRAME, +}; +use serde::{Deserialize, Serialize}; +use std::{ + collections::BTreeMap, + io::{Read, Write}, + path::Path, +}; + +const MAX_REQUEST: usize = 32 * 1024; + +#[derive(Deserialize, Serialize)] +struct Reply { + uid: u32, + gid: u32, + values: Result<BTreeMap<String, String>, String>, +} + +// Keep the owning reader's fixed causes; never propagate an arbitrary reply +// body, filesystem contents, or serde decoding error into host diagnostics. +fn safe_cause(cause: &str) -> &str { + match cause { + "native auth selected keys are invalid" + | "native auth HOME is unavailable" + | "native auth file is unavailable" + | "native auth file escaped owned HOME" + | "native auth file metadata failed" + | "native auth file is not regular" + | "native auth file read failed" + | "native auth file exceeds limit" + | "native auth file is not UTF-8" + | "native auth file cannot be decoded" + | "native auth selected values are invalid" + | "native auth selected values exceed limit" => cause, + _ => "native auth reader failed without classified evidence", + } +} + +pub(crate) fn serve_stdio() -> Result<(), String> { + let mut input = std::io::stdin().lock(); + let mut length = [0; 4]; + input + .read_exact(&mut length) + .map_err(|_| "native auth request unavailable")?; + let length = u32::from_be_bytes(length) as usize; + if length > MAX_REQUEST { + return Err("native auth request exceeds limit".into()); + } + let mut bytes = vec![0; length]; + input + .read_exact(&mut bytes) + .map_err(|_| "native auth request unavailable")?; + let keys: Vec<String> = + serde_json::from_slice(&bytes).map_err(|_| "native auth request invalid")?; + let mut extra = [0]; + if input + .read(&mut extra) + .map_err(|_| "native auth request incomplete")? + != 0 + { + return Err("native auth request has trailing bytes".into()); + } + validate_keys(&keys).map_err(str::to_string)?; + let home = std::env::var_os("HOME").ok_or("native auth HOME is unavailable")?; + let values = read_selected_native_auth(Path::new(&home), &keys) + .map_err(|cause| safe_cause(cause).to_string()); + let reply = Reply { + uid: unsafe { libc::geteuid() }, + gid: unsafe { libc::getegid() }, + values, + }; + let bytes = freshell_runtime_protocol::bounded_json_bytes(&reply, MAX_AUTH_FRAME) + .map_err(|_| "native auth reply exceeds limit")?; + let mut output = std::io::stdout().lock(); + output + .write_all(&(bytes.len() as u32).to_be_bytes()) + .map_err(|_| "native auth reply unavailable")?; + output + .write_all(&bytes) + .map_err(|_| "native auth reply unavailable")?; + Ok(()) +} + +pub(crate) async fn read_selected( + home: &Path, + keys: &[String], + identity: (u32, u32), +) -> Result<BTreeMap<String, String>, String> { + validate_keys(keys).map_err(str::to_string)?; + if identity == (unsafe { libc::geteuid() }, unsafe { libc::getegid() }) { + return read_selected_native_auth(home, keys).map_err(str::to_string); + } + #[cfg(not(test))] + let executable = + std::env::current_exe().map_err(|_| "native auth reader artifact unavailable")?; + #[cfg(test)] + let executable = std::path::PathBuf::from("/runtime/freshell-session-host"); + let mut child = tokio::process::Command::new("/usr/bin/setpriv") + .args(crate::provider_identity_args(identity.0, identity.1)) + .arg(executable) + .arg("native-auth-stdio") + .current_dir("/") + .env_clear() + .env("HOME", home) + .env("PATH", "/usr/bin:/bin") + .stdin(std::process::Stdio::piped()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::null()) + .spawn() + .map_err(|_| "native auth reader could not start")?; + let reader_pid = child.id(); + tracing::info!( + event = "provider.inputs.native_auth.started", + provider = "opencode", + stage = "session_host", + reader_pid, + uid = identity.0, + gid = identity.1, + "Reading selected native authentication under provider identity" + ); + let result = read_child(&mut child, keys, identity).await; + match &result { + Ok(_) => tracing::info!( + event = "provider.inputs.native_auth.completed", + provider = "opencode", + stage = "session_host", + reader_pid, + "Selected native authentication reader joined" + ), + Err(cause) => { + tracing::warn!(event="provider.inputs.native_auth.blocked", provider="opencode", stage="session_host", reader_pid, cause=%cause, "Selected native authentication unavailable") + } + } + result +} + +async fn read_child( + child: &mut tokio::process::Child, + keys: &[String], + identity: (u32, u32), +) -> Result<BTreeMap<String, String>, String> { + use freshell_runtime_protocol::{read_frame_with_limit, write_frame_with_limit}; + use tokio::io::AsyncReadExt; + let exchange = async { + let mut input = child + .stdin + .take() + .ok_or("native auth reader pipe unavailable")?; + let mut output = child + .stdout + .take() + .ok_or("native auth reader pipe unavailable")?; + write_frame_with_limit(&mut input, &keys, MAX_REQUEST) + .await + .map_err(|_| "native auth request incomplete")?; + drop(input); + let reply: Reply = read_frame_with_limit(&mut output, MAX_AUTH_FRAME) + .await + .map_err(|_| "native auth reply unavailable or exceeds limit")?; + let mut trailing = [0]; + if output + .read(&mut trailing) + .await + .map_err(|_| "native auth reader completion unavailable")? + != 0 + { + return Err("native auth reader emitted trailing bytes"); + } + let status = child + .wait() + .await + .map_err(|_| "native auth reader completion unavailable")?; + Ok((reply, status)) + }; + let completed = match tokio::time::timeout(std::time::Duration::from_secs(20), exchange).await { + Ok(result) => result.map_err(str::to_string), + Err(_) => Err("native auth reader timed out".into()), + }; + let (reply, status) = match completed { + Ok(value) => value, + Err(cause) => { + // Restricted root has no CAP_KILL. Signal this still-owned unreaped + // child under its provider identity and join before returning. + let cleanup = async { + if child + .try_wait() + .map_err(|_| "native auth reader join failed")? + .is_none() + { + let pid = child + .id() + .ok_or("native auth reader identity unavailable")? + .to_string(); + let killed = tokio::process::Command::new("/usr/bin/setpriv") + .args(crate::provider_identity_args(identity.0, identity.1)) + .args(["/usr/bin/kill", "-KILL", "--", &pid]) + .env_clear() + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .status() + .await + .map_err(|_| "native auth reader termination failed")?; + if !killed.success() + && child + .try_wait() + .map_err(|_| "native auth reader join failed")? + .is_none() + { + return Err("native auth reader termination failed"); + } + } + child + .wait() + .await + .map_err(|_| "native auth reader join failed") + }; + tokio::time::timeout(std::time::Duration::from_secs(5), cleanup) + .await + .map_err(|_| "native auth reader cleanup timed out")??; + return Err(cause); + } + }; + if !status.success() || (reply.uid, reply.gid) != identity { + return Err("native auth reader failed or returned a different identity".into()); + } + let values = reply + .values + .map_err(|cause| safe_cause(&cause).to_string())?; + validate_selected(keys, &values).map_err(str::to_string)?; + Ok(values) +} diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 88911a0e7..0b5aa0cd0 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -253,12 +253,13 @@ pub(crate) fn final_claude_source_request( Ok(Some((request, plan.observation.clone()))) } -pub fn prepare( +pub(crate) async fn prepare_for_identity( context: Option<&ProviderLaunchContext>, secrets: &mut ResolvedProviderSecrets, frozen: &ProviderPathInputs, stage: &Path, home: &Path, + identity: (u32, u32), ) -> Result<BTreeMap<String, String>, String> { use freshell_platform::opencode_recipe::{GeneratedFileRecipe, SourceLane, SourceManifest}; use sha2::{Digest, Sha256}; @@ -394,8 +395,7 @@ pub fn prepare( ) .map_err(|error| error.to_string())? } else { - freshell_platform::native_auth_exchange::read_selected_native_auth(home, &names) - .map_err(str::to_string)? + crate::native_auth_reader::read_selected(home, &names, identity).await? }; let plan = freshell_platform::opencode_config::OpencodeConfigPlan { sources: vec![], @@ -566,9 +566,27 @@ pub fn prepare( mod tests { use super::*; + async fn prepare( + context: Option<&ProviderLaunchContext>, + secrets: &mut ResolvedProviderSecrets, + frozen: &ProviderPathInputs, + stage: &Path, + home: &Path, + ) -> Result<BTreeMap<String, String>, String> { + super::prepare_for_identity( + context, + secrets, + frozen, + stage, + home, + (unsafe { libc::geteuid() }, unsafe { libc::getegid() }), + ) + .await + } + #[cfg(target_os = "linux")] - #[test] - fn provider_secret_private_reader_rejects_oversize_before_payload_read() { + #[tokio::test] + async fn provider_secret_private_reader_rejects_oversize_before_payload_read() { use private_read_observation::Observation; let owned = tempfile::tempdir().unwrap(); let private = owned.path().join("metadata"); @@ -616,7 +634,7 @@ mod tests { use sha2::{Digest, Sha256}; #[cfg(target_os = "linux")] - fn private_native_auth_case(case: &str, test: &str) { + async fn private_native_auth_case(case: &str, test: &str) { use std::os::unix::fs::{MetadataExt, PermissionsExt}; const CHILD: &str = "FRESHELL_TEST_PRIVATE_NATIVE_AUTH_ROOT"; let restricted = std::env::var_os("FRESHELL_TEST_RESTRICTED_READER").is_some(); @@ -681,13 +699,20 @@ mod tests { ); } let mut secrets = ResolvedProviderSecrets::default(); - let result = prepare( + let identity = if restricted { + (65534, 0) + } else { + (unsafe { libc::geteuid() }, unsafe { libc::getegid() }) + }; + let result = super::prepare_for_identity( Some(&context), &mut secrets, &ProviderPathInputs::default(), &stage, &home, - ); + identity, + ) + .await; match case { "selected" => { let generated = result @@ -827,40 +852,45 @@ mod tests { } #[cfg(target_os = "linux")] - #[test] - fn provider_secret_native_auth_private_home_selected() { + #[tokio::test] + async fn provider_secret_native_auth_private_home_selected() { private_native_auth_case( "selected", "provider_input_preparation::tests::provider_secret_native_auth_private_home_selected", - ); + ) + .await; } #[cfg(target_os = "linux")] - #[test] - fn provider_secret_native_auth_private_home_absent() { + #[tokio::test] + async fn provider_secret_native_auth_private_home_absent() { private_native_auth_case( "absent", "provider_input_preparation::tests::provider_secret_native_auth_private_home_absent", - ); + ) + .await; } #[cfg(target_os = "linux")] - #[test] - fn provider_secret_native_auth_private_home_foreign() { + #[tokio::test] + async fn provider_secret_native_auth_private_home_foreign() { private_native_auth_case( "foreign", "provider_input_preparation::tests::provider_secret_native_auth_private_home_foreign", - ); + ) + .await; } #[cfg(target_os = "linux")] - #[test] - fn provider_secret_native_auth_private_home_reserved() { + #[tokio::test] + async fn provider_secret_native_auth_private_home_reserved() { private_native_auth_case( "reserved", "provider_input_preparation::tests::provider_secret_native_auth_private_home_reserved", - ); + ) + .await; } - #[test] - fn provider_secret_repeated_native_source_views_release_live_storage_between_occurrences() { + #[tokio::test] + async fn provider_secret_repeated_native_source_views_release_live_storage_between_occurrences() + { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let stage = owned.path().join("stage"); @@ -924,7 +954,7 @@ mod tests { "preparedInputs":{"generationId":generation,"manifest":{"relativePath":".freshell/input-manifest.json","sha256":format!("{:x}",Sha256::digest(&bytes)),"sourceCount":0,"aliasCount":0,"recipeCount":0}} })).unwrap(); let mut secrets = ResolvedProviderSecrets::default(); - let generated = prepare(Some(&context), &mut secrets, &ProviderPathInputs::default(), &stage, &home) + let generated = prepare(Some(&context), &mut secrets, &ProviderPathInputs::default(), &stage, &home).await .expect("128 MiB cumulative reads of eight 16 MiB occurrences fit the simultaneous view profile"); assert!( generated.is_empty(), @@ -950,6 +980,7 @@ mod tests { &stage, &home, ) + .await .unwrap_err(); assert!(error.contains("byte profile"), "{error}"); assert!(refused.opencode_plan.is_none()); @@ -987,6 +1018,7 @@ mod tests { &stage, &home, ) + .await .unwrap_err(); assert!(error.contains("recipe profile limit"), "{error}"); assert!(refused.opencode_plan.is_none()); @@ -1001,8 +1033,8 @@ mod tests { } #[cfg(target_os = "linux")] - #[test] - fn provider_secret_actual_merged_profile_reaches_guarded_child() { + #[tokio::test] + async fn provider_secret_actual_merged_profile_reaches_guarded_child() { const GENERATED: &str = "FRESHELL_CONFIG_REFERENCE_0"; const RECEIPT: &str = "HOST_MERGED_PROFILE="; if std::env::var_os(GENERATED).is_some() { @@ -1149,7 +1181,7 @@ mod tests { "indexedFiles":{"volumeName":indexed_provider_volume_name(&SoulId::new(),&generation),"aliases":[]}} })).unwrap(); let mut secrets = ResolvedProviderSecrets::default(); - let result = prepare(Some(&context), &mut secrets, &frozen, &stage, &home); + let result = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).await; if accepted { let environment = result.unwrap(); assert_eq!(environment.len(), frozen_count + 1); @@ -1262,8 +1294,8 @@ mod tests { .unwrap() .is_none()); } - #[test] - fn provider_secret_claude_final_source_request_keeps_enriched_origin_and_frozen_original_selectors( + #[tokio::test] + async fn provider_secret_claude_final_source_request_keeps_enriched_origin_and_frozen_original_selectors( ) { use freshell_platform::claude_inputs::{ observation::observe, AcceptedNativeSourcePlan, ClaudeInputManifest, @@ -1361,8 +1393,8 @@ mod tests { } } - #[test] - fn provider_secret_claude_inline_settings_consumes_accepted_private_source() { + #[tokio::test] + async fn provider_secret_claude_inline_settings_consumes_accepted_private_source() { use freshell_platform::claude_inputs::{ClaudeInputManifest, INLINE_SETTINGS_SOURCE}; let owned = tempfile::tempdir().unwrap(); std::fs::create_dir_all(owned.path().join(".freshell/source-views")).unwrap(); @@ -1419,6 +1451,7 @@ mod tests { owned.path(), owned.path(), ) + .await .unwrap(); let argument = secrets.claude_settings_argument.as_deref().expect("accepted inline settings must reach the native child argument without a guessed file origin"); let output = std::process::Command::new("node").args(["-e", "const s=JSON.parse(process.argv[1]);process.stdout.write([s.env.FLAG_NONCE,s.hooks.SessionStart[0].hooks[0].command,s.hooks.SessionStart.length,s.hooks.Stop.length].join(':'))", argument]).output().unwrap(); @@ -1441,6 +1474,7 @@ mod tests { owned.path(), owned.path(), ) + .await .err() .unwrap(); assert!(error.contains("source changed")); @@ -1459,12 +1493,13 @@ mod tests { owned.path(), owned.path() ) + .await .is_err()); assert!(oversized.claude_settings_argument.is_none()); } - #[test] - fn provider_secret_claude_prepared_manifest_validates_without_original_controller_tree() { + #[tokio::test] + async fn provider_secret_claude_prepared_manifest_validates_without_original_controller_tree() { let owned = tempfile::tempdir().unwrap(); std::fs::create_dir(owned.path().join(".freshell")).unwrap(); let manifest = freshell_platform::claude_inputs::ClaudeInputManifest { @@ -1511,7 +1546,7 @@ mod tests { mcp_capability: Some(capability), }; let mut secrets = ResolvedProviderSecrets::default(); - let generated=prepare(Some(&context),&mut secrets,&ProviderPathInputs::default(),owned.path(),owned.path()) + let generated=prepare(Some(&context),&mut secrets,&ProviderPathInputs::default(),owned.path(),owned.path()).await .expect("Claude source origins validate from accepted private metadata, without rescanning controller paths"); assert!(generated.is_empty()); assert!( @@ -1532,6 +1567,7 @@ mod tests { owned.path(), owned.path() ) + .await .is_err(), "foreign source recipes cannot enter ordinary Claude metadata" ); @@ -1554,6 +1590,7 @@ mod tests { owned.path(), owned.path(), ) + .await .err() .unwrap(); assert!(error.contains("declaring origin is unrecorded")); @@ -1576,15 +1613,16 @@ mod tests { }); std::fs::create_dir(owned.path().join(".freshell/templates")).unwrap(); std::fs::write(owned.path().join(".freshell/templates/claude-0"), b"{}").unwrap(); - prepare(Some(&context), &mut secrets, &ProviderPathInputs::default(), owned.path(), owned.path()).expect("a recorded native declaring origin is consumed without reading the unavailable original controller tree"); + prepare(Some(&context), &mut secrets, &ProviderPathInputs::default(), owned.path(), owned.path()).await.expect("a recorded native declaring origin is consumed without reading the unavailable original controller tree"); assert_eq!( secrets.claude_settings_argument.as_deref(), Some("/home/freshell/provider/declaring-settings.json") ); } - #[test] - fn provider_secret_amplifier_prepared_manifest_validates_without_original_controller_tree() { + #[tokio::test] + async fn provider_secret_amplifier_prepared_manifest_validates_without_original_controller_tree( + ) { let owned = tempfile::tempdir().unwrap(); std::fs::create_dir(owned.path().join(".freshell")).unwrap(); let manifest = freshell_platform::amplifier_inputs::AmplifierInputManifest { @@ -1624,7 +1662,7 @@ mod tests { mcp_capability: Some(capability), }; let mut secrets = ResolvedProviderSecrets::default(); - let generated=prepare(Some(&context),&mut secrets,&ProviderPathInputs::default(),owned.path(),owned.path()) + let generated=prepare(Some(&context),&mut secrets,&ProviderPathInputs::default(),owned.path(),owned.path()).await .expect("Amplifier source origins validate from accepted private metadata, without rescanning controller paths"); assert!(generated.is_empty()); assert!( @@ -1645,13 +1683,14 @@ mod tests { owned.path(), owned.path() ) + .await .is_err(), "foreign source recipes cannot enter ordinary Amplifier metadata" ); } - #[test] - fn provider_secret_codex_prepared_manifest_retains_native_home_without_original_tree() { + #[tokio::test] + async fn provider_secret_codex_prepared_manifest_retains_native_home_without_original_tree() { let owned = tempfile::tempdir().unwrap(); std::fs::create_dir(owned.path().join(".freshell")).unwrap(); let manifest = freshell_platform::codex_inputs::CodexInputManifest { @@ -1699,6 +1738,7 @@ mod tests { owned.path(), owned.path(), ) + .await .expect( "accepted native Codex inputs must validate without reading original controller paths", ); @@ -1718,12 +1758,14 @@ mod tests { owned.path(), owned.path() ) + .await .unwrap_err() .contains("changed")); } - #[test] - fn provider_secret_prepared_recipe_consumes_private_frozen_inputs_without_original_tree() { + #[tokio::test] + async fn provider_secret_prepared_recipe_consumes_private_frozen_inputs_without_original_tree() + { let owned = tempfile::tempdir().unwrap(); let stage = owned.path().join("stage"); let home = owned.path().join("home"); @@ -1814,7 +1856,9 @@ mod tests { )])) .unwrap(); let mut secrets = ResolvedProviderSecrets::default(); - let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap(); + let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await + .unwrap(); assert!(secrets.opencode_plan.is_some(), "the accepted source plan must reach provider preparation without rescanning mounted templates"); assert!( secrets @@ -1841,6 +1885,7 @@ mod tests { context.prepared_inputs.as_mut().unwrap().manifest.sha256 = format!("{:x}", Sha256::digest(&bytes)); let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await .err() .unwrap_or_default(); assert!( @@ -1859,7 +1904,7 @@ mod tests { std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); context.prepared_inputs.as_mut().unwrap().manifest.sha256 = format!("{:x}", Sha256::digest(&bytes)); - let original = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + let original = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).await .expect("native selection of an approved original-form source cannot require a replacement bind"); assert_eq!( original.get("OPENCODE_CONFIG").map(String::as_str), @@ -1880,7 +1925,9 @@ mod tests { std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); context.prepared_inputs.as_mut().unwrap().manifest.sha256 = format!("{:x}", Sha256::digest(&bytes)); - let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap_err(); + let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await + .unwrap_err(); assert!(!error.contains("/unapproved/source.json")); } manifest.native_source_flags[0] = approved_flag; @@ -1902,7 +1949,9 @@ mod tests { std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); context.prepared_inputs.as_mut().unwrap().manifest.sha256 = format!("{:x}", Sha256::digest(&bytes)); - let inline = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap(); + let inline = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await + .unwrap(); assert_eq!( inline.get("OPENCODE_CONFIG_CONTENT").map(String::as_str), Some(inline_template) @@ -1913,7 +1962,7 @@ mod tests { r#"{"instructions":["{file:/unapproved/private-file}"]}"#, ) .unwrap(); - assert!(prepare(Some(&context), &mut secrets, &frozen, &stage, &home).is_err(), "changed inline template must fail before an unapproved file can reach the native child"); + assert!(prepare(Some(&context), &mut secrets, &frozen, &stage, &home).await.is_err(), "changed inline template must fail before an unapproved file can reach the native child"); manifest.sources[0].lane = SourceLane::Main; manifest.sources[0].template_sha256 = None; manifest.native_source_flags = native_flags; @@ -1926,7 +1975,7 @@ mod tests { "{file:/different-unapproved/rules}".into(), )])) .unwrap(); - assert!(prepare(Some(&context),&mut secrets,&changed,&stage,&home).is_err(),"changed lexical selection must fail without canonicalizing or reading an unavailable original tree"); + assert!(prepare(Some(&context),&mut secrets,&changed,&stage,&home).await.is_err(),"changed lexical selection must fail without canonicalizing or reading an unavailable original tree"); assert_eq!( std::fs::read_to_string(stage.join(".freshell/source-views/source-0")).unwrap(), raw @@ -1941,6 +1990,7 @@ mod tests { context.prepared_inputs.as_mut().unwrap().manifest.sha256 = format!("{:x}", Sha256::digest(&bytes)); let generated = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await .expect("direct source must use the 16 MiB document profile"); assert_eq!(generated.len(), 1); assert!(generated.contains_key("OPENCODE_CONFIG")); @@ -1956,14 +2006,20 @@ mod tests { format!("{:x}", Sha256::digest(&bytes)); let at_limit = format!("{}{{}}", " ".repeat(document_limit - 2)); std::fs::write(stage.join(".freshell/source-views/source-0"), &at_limit).unwrap(); - assert!(prepare(Some(&context), &mut secrets, &frozen, &stage, &home).is_ok()); + assert!( + prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await + .is_ok() + ); std::fs::write( stage.join(".freshell/source-views/source-0"), format!(" {at_limit}"), ) .unwrap(); assert!( - prepare(Some(&context), &mut secrets, &frozen, &stage, &home).is_err(), + prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await + .is_err(), "direct source growth above the 16 MiB profile must fail before dispatch" ); std::fs::write(stage.join(".freshell/source-views/source-0"), &large_raw).unwrap(); @@ -1976,6 +2032,7 @@ mod tests { format!("{:x}", Sha256::digest(&bytes)); assert!( prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await .unwrap_err() .contains("64 KiB"), "generated child values retain the declared stricter limit" @@ -1986,7 +2043,9 @@ mod tests { std::fs::write(stage.join(".freshell/input-manifest.json"), &bytes).unwrap(); context.prepared_inputs.as_mut().unwrap().manifest.sha256 = format!("{:x}", Sha256::digest(&bytes)); - let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home).unwrap_err(); + let error = prepare(Some(&context), &mut secrets, &frozen, &stage, &home) + .await + .unwrap_err(); assert!(error.contains("approved config bind")); assert!(!error.contains("/unapproved/native/config.json")); } diff --git a/scripts/testing/managed-provider-source-sandbox.py b/scripts/testing/managed-provider-source-sandbox.py index 055f7b350..730e2d0f5 100644 --- a/scripts/testing/managed-provider-source-sandbox.py +++ b/scripts/testing/managed-provider-source-sandbox.py @@ -41,6 +41,9 @@ def main(): expected_directory = Path(__file__).resolve().parents[2] / "target/debug/deps" if binary.parent != expected_directory.resolve(): raise RuntimeError("source fixture test artifact must belong to this worktree") + host_artifact = expected_directory.parent / "freshell-session-host" + if native_auth_reader and (not host_artifact.is_file() or host_artifact.is_symlink() or host_artifact.resolve() != host_artifact): + raise RuntimeError("native auth fixture requires its own physical compiled session host") listing = subprocess.run([str(binary), *selection, "--list"], env={"PATH": "/usr/bin:/bin"}, text=True, capture_output=True, timeout=20, check=True) @@ -59,14 +62,19 @@ def main(): created = False receipt = {"image": IMAGE, "testArtifactSha256": digest, "test": selected_test, "runId": name, "rootless": True} + if native_auth_reader: + with host_artifact.open("rb") as stream: + receipt["sessionHostArtifactSha256"] = hashlib.file_digest(stream, "sha256").hexdigest() reader = args.provider in ("claude-reader", "native-auth-reader") identity_args = ["--cap-add", "CHOWN", "--cap-add", "SETGID", "--cap-add", "SETUID", "--env", "FRESHELL_TEST_RESTRICTED_READER=1"] if reader else [] test_env = ["--env", "RUST_TEST_THREADS=4", "--env", "TOKIO_WORKER_THREADS=2"] if native_auth_reader else [] + host_mount = ["--mount", f"type=bind,src={host_artifact},dst=/runtime/freshell-session-host,readonly"] if native_auth_reader else [] try: docker("create", "--name", name, "--label", f"{LABEL}={name}", "--network", "none", "--read-only", "--user", "0:0" if reader else "65534:0", "--cap-drop", "ALL", *identity_args, *test_env, "--security-opt", "no-new-privileges:true", "--pids-limit", "128", "--memory", "2g", "--mount", f"type=bind,src={binary},dst=/runtime/source-consumer-tests,readonly", + *host_mount, "--tmpfs", "/fixture:rw,exec,nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", "--tmpfs", f"/home/freshell/provider:rw,{'exec' if args.provider == 'codex' else 'noexec'},nosuid,nodev,size=512m,uid=65534,gid=0,mode=0700", "--tmpfs", f"/tmp:rw,nosuid,nodev,noexec,size=256m,uid={0 if reader else 65534},gid=0,mode={'1777' if reader else '0700'}", From 4cfb1c8bcffdb670d4fd31a1c58a6a29191a55ed Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 16:47:04 -0700 Subject: [PATCH 280/493] test: cover managed preflight identity observation --- scripts/testing/managed-environment-create.ts | 9 +++++ .../managed-provider-environment-target.ts | 6 ++-- .../tooling/testing/runtime-sandbox.test.ts | 33 ++++++++++++++++++- 3 files changed, 44 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index b91818b32..f8daa285b 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -5,6 +5,15 @@ import path from 'node:path' import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' import { dockerSocketPath } from './runtime-sandbox.js' import { providerParityFailureDiagnostics } from '../../test/helpers/provider-parity-environment.js' +import type { ManagedRuntimeBrowserRig } from '../../test/e2e-browser/helpers/managed-runtime.js' + +export async function observeManagedEnvironmentNativeIdentity( + owner: Pick<ManagedRuntimeBrowserRig, 'inventory' | 'runtime' | 'supervisor'>, + terminalId: string, nativeSessionId: string, +) { + return (await owner.inventory()).find(row => row.terminalId === terminalId + && row.nativeSessionId === nativeSessionId && row.launchState === 'running') +} export async function readManagedEnvironmentDaemonId(): Promise<string> { const dockerHost = process.env.DOCKER_HOST diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 0cbfa82c4..b0b6aaa8b 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -13,7 +13,7 @@ import { managedEnvironmentOpencodeSelection } from '../../test/helpers/provider import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' -import { assertManagedTerminalCreated, readManagedEnvironmentDaemonId } from './managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId } from './managed-environment-create.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -286,8 +286,8 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightChild.runId, rig.runtime.runId) assert.equal(preflightChild.expectedNonce, nonce) assert.equal(preflightChild.bun, true) - const preflightView = await poll('genuine managed preflight native identity', async () => (await rig!.inventory()) - .find(row => row.terminalId === created.terminalId && row.nativeSessionId === preflightChild.nativeSessionId && row.launchState === 'running')) + const preflightView = await poll('genuine managed preflight native identity', () => + observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) assert.deepEqual(await webChildren(rig), [], 'Managed child must belong to the disposable session host, not the host web process') proof.preflight = { terminalId: created.terminalId, soulId: preflightView.soulId, incarnationId: preflightView.incarnationId, diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 64116b463..60fbb91b7 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -35,9 +35,40 @@ import { RuntimeHarness, dockerSocketPath } from '../../../../scripts/testing/ru import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' -import { assertManagedTerminalCreated, readManagedEnvironmentDaemonId } from '../../../../scripts/testing/managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId } from '../../../../scripts/testing/managed-environment-create.js' import type { ErrorMessage } from '../../../../shared/ws-protocol.js' +describe('managed environment native identity observation', () => { + it.each(['expected', 'foreign'] as const)('observes owned output before accepting %s native identity', async kind => { + const row = { soulId: 'owned-soul', terminalId: 'owned-terminal', incarnationId: 'owned-incarnation', + launchState: 'running', nativeSessionId: undefined as string | undefined } + const calls: unknown[] = [] + const runtime = { + terminalReadOutputBody: RuntimeHarness.prototype.terminalReadOutputBody, + adminOk: async (_supervisor: unknown, body: unknown) => { + calls.push(body) + row.nativeSessionId = kind === 'expected' ? 'ses_expected' : 'ses_foreign' + return { frames: [{ data: 'private output is discarded' }] } + }, + } + const owner = { inventory: async () => [row], runtime, supervisor: {} } as unknown as Parameters<typeof observeManagedEnvironmentNativeIdentity>[0] + const observed = await observeManagedEnvironmentNativeIdentity(owner, 'owned-terminal', 'ses_expected') + expect.soft(calls).toEqual([{ method: 'terminal_read_output', + params: { soulId: 'owned-soul', afterSeq: 0, maxBytes: 64 * 1024 } }]) + expect.soft(observed).toEqual(kind === 'expected' ? row : undefined) + }) + + it('leaves other terminals and stopped rows unobserved', async () => { + const adminOk = vi.fn() + const owner = { inventory: async () => [ + { soulId: 'foreign-soul', terminalId: 'other-terminal', launchState: 'running', nativeSessionId: 'ses_expected' }, + { soulId: 'stopped-soul', terminalId: 'owned-terminal', launchState: 'stopped', nativeSessionId: 'ses_expected' }, + ], runtime: { terminalReadOutputBody: RuntimeHarness.prototype.terminalReadOutputBody, adminOk }, supervisor: {} } as unknown as Parameters<typeof observeManagedEnvironmentNativeIdentity>[0] + expect(await observeManagedEnvironmentNativeIdentity(owner, 'owned-terminal', 'ses_expected')).toBeUndefined() + expect(adminOk).not.toHaveBeenCalled() + }) +}) + describe('managed environment daemon identity', () => { it('reads the exact configured Unix daemon before registering a source', async () => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-daemon-')) From 1a7241ac498e11503475d9624ca7d3513fa0bcca Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 16:50:46 -0700 Subject: [PATCH 281/493] fix: observe managed preflight output before native identity --- scripts/testing/managed-environment-create.ts | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index f8daa285b..58186b533 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -11,6 +11,13 @@ export async function observeManagedEnvironmentNativeIdentity( owner: Pick<ManagedRuntimeBrowserRig, 'inventory' | 'runtime' | 'supervisor'>, terminalId: string, nativeSessionId: string, ) { + const current = (await owner.inventory()).find(row => row.terminalId === terminalId + && row.launchState === 'running') + if (!current) return + // Native identity is published by the existing authenticated output reader; + // inventory alone only returns the stored registry snapshot. + await owner.runtime.adminOk(owner.supervisor, + owner.runtime.terminalReadOutputBody(current.soulId, 0, 64 * 1024)) return (await owner.inventory()).find(row => row.terminalId === terminalId && row.nativeSessionId === nativeSessionId && row.launchState === 'running') } From 26275dc208ee54524dddbc8795759e7f4a673a4a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 17:21:43 -0700 Subject: [PATCH 282/493] test: activate managed environment browser harness --- .../specs/managed-provider-environment-external.spec.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index aa6368988..855b7d6f7 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -33,7 +33,7 @@ async function activePane(page: Page) { } async function launch(page: Page, scenario: string) { - await page.goto(`${paired!.targetUrl}/?token=${encodeURIComponent(paired!.targetToken)}`) + await page.goto(`${paired!.targetUrl}/?token=${encodeURIComponent(paired!.targetToken)}&e2e=1`) await page.waitForFunction(() => Boolean(window.__FRESHELL_TEST_HARNESS__), undefined, { timeout: 60_000 }) await new TestHarness(page).waitForConnection(120_000) const info = await json(page, '/api/server-info') From 4bd81fbf8b961f1f9472f02d09676ddba13d1ac5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 18:31:00 -0700 Subject: [PATCH 283/493] test: protect managed preparation failures through pane reloads --- crates/freshell-freshagent/src/claude.rs | 1 + crates/freshell-freshagent/src/codex.rs | 1 + crates/freshell-freshagent/src/opencode_ws.rs | 1 + .../freshell-freshagent/src/terminal_tabs.rs | 5 +- .../freshell-protocol/src/server_messages.rs | 45 ++++ .../src/managed_mcp_capability.rs | 7 +- .../src/managed_provider_references.rs | 79 +++++++ crates/freshell-server/src/managed_runtime.rs | 7 +- .../managed_runtime_opencode_replan_tests.rs | 3 +- .../src/managed_output_tests.rs | 3 +- crates/freshell-terminal/src/registry.rs | 33 ++- crates/freshell-ws/src/create_dedupe.rs | 1 + crates/freshell-ws/src/lib.rs | 1 + .../src/managed_fixture_denies_host_pty.rs | 5 +- crates/freshell-ws/src/terminal.rs | 218 +++++++++++++++++- shared/ws-protocol.ts | 11 + src/store/paneTypes.ts | 1 + .../TerminalView.launchRetry.test.tsx | 105 ++++++++- .../client/store/panesPersistence.test.ts | 37 +++ 19 files changed, 539 insertions(+), 25 deletions(-) diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index d60ac8a85..8e0452708 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -7336,6 +7336,7 @@ impl FreshClaudeState { fn send_error(&self, request_id: &Option<String>, code: &str, message: &str) { self.broadcast(&ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index 73e641cc3..aa4e481a5 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -3652,6 +3652,7 @@ impl FreshCodexState { fn send_error(&self, request_id: &Option<String>, code: &str, message: &str) { self.broadcast(&ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, diff --git a/crates/freshell-freshagent/src/opencode_ws.rs b/crates/freshell-freshagent/src/opencode_ws.rs index 0884a5c4a..9e8d55231 100644 --- a/crates/freshell-freshagent/src/opencode_ws.rs +++ b/crates/freshell-freshagent/src/opencode_ws.rs @@ -1261,6 +1261,7 @@ impl FreshOpencodeState { fn send_error(&self, request_id: &Option<String>, code: &str, message: &str) { self.broadcast(&ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, diff --git a/crates/freshell-freshagent/src/terminal_tabs.rs b/crates/freshell-freshagent/src/terminal_tabs.rs index 0172fba9b..ceb3e7854 100644 --- a/crates/freshell-freshagent/src/terminal_tabs.rs +++ b/crates/freshell-freshagent/src/terminal_tabs.rs @@ -7123,7 +7123,10 @@ if (args.includes('app-server')) {{ _: freshell_terminal::registry::ManagedTerminalLaunch, ) -> freshell_terminal::registry::ManagedTerminalFuture< 'a, - Result<freshell_terminal::registry::ManagedTerminalDescriptor, String>, + Result< + freshell_terminal::registry::ManagedTerminalDescriptor, + freshell_terminal::registry::ManagedTerminalLaunchError, + >, > { Box::pin(async { Err("unexpected managed launch".into()) }) } diff --git a/crates/freshell-protocol/src/server_messages.rs b/crates/freshell-protocol/src/server_messages.rs index 1c8d87f7c..e444f3eeb 100644 --- a/crates/freshell-protocol/src/server_messages.rs +++ b/crates/freshell-protocol/src/server_messages.rs @@ -811,9 +811,52 @@ pub struct ConfigFallback { pub reason: ConfigFallbackReason, } +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "SCREAMING_SNAKE_CASE")] +pub enum ProviderPreparationFailureCode { + SelectedReferenceUnavailable, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum ProviderPreparationFailureStage { + Configuration, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum ProviderPreparationFailureOperation { + Resolve, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum ProviderPreparationFailureCause { + NotFound, + PermissionDenied, + Cyclic, + Unknown, +} + +/// Selected-reference metadata only. No configuration or credential payloads. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ProviderPreparationFailure { + pub code: ProviderPreparationFailureCode, + pub stage: ProviderPreparationFailureStage, + pub operation: ProviderPreparationFailureOperation, + pub cause: ProviderPreparationFailureCause, + pub workspace: String, + pub declaring_config: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub selected_path: Option<String>, +} + #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct ErrorMsg { + #[serde(skip_serializing_if = "Option::is_none")] + pub provider_preparation_failure: Option<ProviderPreparationFailure>, pub code: ErrorCode, pub message: String, pub timestamp: String, @@ -2241,6 +2284,7 @@ mod tests { // (round-1 review: the earlier `Error` sketch did not compile // against the repo type). let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, code: ErrorCode::RestoreUnavailable, message: "Session 01a0828d is still running on the server.".into(), timestamp: "2026-09-09T00:00:00Z".into(), @@ -2264,6 +2308,7 @@ mod tests { // Additive-optional: a legacy error frame with no owner fields stays // byte-identical on the wire (the frozen refusal surface). let legacy = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, code: ErrorCode::InternalError, message: "unrelated".into(), timestamp: "2026-09-09T00:00:00Z".into(), diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index cedbf9425..0831b566a 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -6576,8 +6576,11 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .map(|(path, _)| (path.clone(), std::fs::read(path).unwrap())) .collect::<Vec<_>>(); if case == "unreadable-marker" { - std::fs::set_permissions(checkout.join(".git"), std::fs::Permissions::from_mode(0o0)) - .unwrap(); + std::fs::set_permissions( + checkout.join(".git"), + std::fs::Permissions::from_mode(0o0), + ) + .unwrap(); } let sources = match case { "project-only" => vec![ClaudeSettingSource::Project], diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 266cbff7b..018ea4f09 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -50,6 +50,13 @@ impl std::fmt::Display for ReferenceError { } impl std::error::Error for ReferenceError {} +// Preserve the existing generic error until the typed producer behavior is implemented. +impl From<ReferenceError> for freshell_terminal::registry::ManagedTerminalLaunchError { + fn from(error: ReferenceError) -> Self { + error.to_string().into() + } +} + pub struct FileReferenceSelection { pub canonical_source: PathBuf, pub runtime_destination: String, @@ -1265,6 +1272,78 @@ process.stdout.write(JSON.stringify(receipts)); assert_eq!(plan.referenced_variables, vec!["NODE_EXTRA_CA_CERTS"]); assert!(!plan.template.contains("private-path-ref")); } + #[test] + fn managed_provider_references_literal_missing_path_is_actionable_without_reading_payloads() { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("opencode.json"); + let selected = temp.path().join("missing-selected.txt"); + std::fs::write( + &source, + r#"{"agent":{"fixture":{"prompt":"{file:./missing-selected.txt}"}}}"#, + ) + .unwrap(); + let raw = std::fs::read_to_string(&source).unwrap(); + let err = resolve_file_references( + &raw, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .err() + .expect("an explicit missing selection must block before dispatch"); + assert_eq!(err.operation, "resolve"); + assert_eq!( + err.cause, + "referenced file is missing, inaccessible or cyclic" + ); + assert!(err.variables.is_empty()); + assert!(err.to_string().contains(source.to_str().unwrap())); + assert!(err.to_string().contains(selected.to_str().unwrap())); + assert!(!selected.exists()); + } + + #[test] + fn managed_provider_references_literal_missing_path_keeps_typed_metadata() { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("opencode.json"); + let selected = temp.path().join("missing-selected.txt"); + std::fs::write( + &source, + r#"{"agent":{"fixture":{"prompt":"{file:./missing-selected.txt}"}}}"#, + ) + .unwrap(); + let raw = std::fs::read_to_string(&source).unwrap(); + let err = resolve_file_references( + &raw, + FileSourceOrigin { + declaring_config: &source, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .err() + .expect("an explicit missing selection must block before dispatch"); + let launch_error: freshell_terminal::registry::ManagedTerminalLaunchError = err.into(); + let failure = launch_error + .provider_preparation_failure + .expect("the real selected-reference failure keeps safe typed metadata"); + assert_eq!( + failure.code, + freshell_protocol::ProviderPreparationFailureCode::SelectedReferenceUnavailable + ); + assert_eq!( + failure.cause, + freshell_protocol::ProviderPreparationFailureCause::NotFound + ); + assert_eq!(failure.selected_path.as_deref(), selected.to_str()); + assert_eq!(failure.declaring_config, source.to_string_lossy()); + assert_eq!(failure.workspace, temp.path().to_string_lossy()); + } + #[test] fn managed_provider_references_missing_and_cyclic_targets_fail_without_echoing_candidate_values( ) { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index 7a5aca45f..ee2a74ead 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -501,7 +501,10 @@ impl ManagedTerminalController for ServerManagedRuntimeController { fn launch<'a>( &'a self, request: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>> { + ) -> ManagedTerminalFuture< + 'a, + Result<ManagedTerminalDescriptor, freshell_terminal::registry::ManagedTerminalLaunchError>, + > { Box::pin(async move { let create_key = request .create_request_id @@ -791,7 +794,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { grant_id=mcp_capability.as_ref().map(|c|c.grant_id.as_str()), stage="controller",operation="launch",%error, "Retain provider inputs until dispatch outcome is confirmed"); - return Err(error.to_string()); + return Err(error.to_string().into()); } } }; diff --git a/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs b/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs index 9ca3d57fa..be79d1218 100644 --- a/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs +++ b/crates/freshell-server/src/managed_runtime_opencode_replan_tests.rs @@ -414,6 +414,7 @@ mod uncertain_launch { }) .await .map(|_| ()) + .map_err(|error| error.to_string()) }; assert!( result.is_err(), @@ -714,7 +715,7 @@ fn managed_provider_initial_opencode_callers_replan_changed_selection_with_two_r let result=if fresh_route { fresh.clone().create_agent(HostedRestCreate {request_id:seed.clone(),provider:"opencode".into(),session_type:"freshopencode".into(),cwd:Some(workspace.to_string_lossy().into_owned()),model:None,effort:None,native_session_id:None,plugins:None,model_selection:None,permission_mode:None,sandbox:None}).await.map(|_|()).map_err(|_|"fresh refusal".to_string()) } else { - controller.launch(ManagedTerminalLaunch {spec:freshell_platform::SpawnSpec {program:"opencode".into(),args:vec![],env_overrides:BTreeMap::new(),cwd:Some(workspace.to_string_lossy().into_owned()),cols:80,rows:24},env:BTreeMap::new(),terminal_id:seed.clone(),stream_id:seed.clone(),mode:"opencode".into(),resume_session_id:None,provider_model:None,provider_reasoning_effort:None,provider_sandbox:None,provider_permission_mode:None,view_tab_id:None,view_pane_id:None,create_request_id:Some(seed.clone())}).await.map(|_|()) + controller.launch(ManagedTerminalLaunch {spec:freshell_platform::SpawnSpec {program:"opencode".into(),args:vec![],env_overrides:BTreeMap::new(),cwd:Some(workspace.to_string_lossy().into_owned()),cols:80,rows:24},env:BTreeMap::new(),terminal_id:seed.clone(),stream_id:seed.clone(),mode:"opencode".into(),resume_session_id:None,provider_model:None,provider_reasoning_effort:None,provider_sandbox:None,provider_permission_mode:None,view_tab_id:None,view_pane_id:None,create_request_id:Some(seed.clone())}).await.map(|_|()).map_err(|error| error.to_string()) }; assert_eq!(result.is_ok(),!changing && !uncertain,"actual initial caller fresh={fresh_route} changing={changing}: {result:?}"); assert_eq!(attempts.lock().await[&seed],if uncertain {1} else if changing {3} else {2}); diff --git a/crates/freshell-terminal/src/managed_output_tests.rs b/crates/freshell-terminal/src/managed_output_tests.rs index a65c195e7..120670d6d 100644 --- a/crates/freshell-terminal/src/managed_output_tests.rs +++ b/crates/freshell-terminal/src/managed_output_tests.rs @@ -31,7 +31,8 @@ impl ManagedTerminalController for Controller { fn launch<'a>( &'a self, _: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>> { + ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, ManagedTerminalLaunchError>> + { panic!("output recovery must not launch a provider") } fn input<'a>( diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 7b8fa1e8b..e9ce4d794 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -866,6 +866,35 @@ pub struct ManagedTerminalLaunch { pub type ManagedTerminalFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>; +#[derive(Debug)] +pub struct ManagedTerminalLaunchError { + pub message: String, + pub provider_preparation_failure: Option<freshell_protocol::ProviderPreparationFailure>, +} + +impl std::fmt::Display for ManagedTerminalLaunchError { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter.write_str(&self.message) + } +} + +impl std::error::Error for ManagedTerminalLaunchError {} + +impl From<String> for ManagedTerminalLaunchError { + fn from(message: String) -> Self { + Self { + message, + provider_preparation_failure: None, + } + } +} + +impl From<&str> for ManagedTerminalLaunchError { + fn from(message: &str) -> Self { + message.to_owned().into() + } +} + #[derive(Debug, Clone, PartialEq, Eq)] pub struct ManagedOutputChunk { pub seq_start: i64, @@ -941,7 +970,7 @@ pub trait ManagedTerminalController: Send + Sync { fn launch<'a>( &'a self, request: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>>; + ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, ManagedTerminalLaunchError>>; fn input<'a>( &'a self, terminal: ManagedTerminalDescriptor, @@ -4773,7 +4802,7 @@ impl TerminalRegistry { pub async fn launch_managed( &self, request: ManagedTerminalLaunch, - ) -> Result<ManagedTerminalDescriptor, String> { + ) -> Result<ManagedTerminalDescriptor, ManagedTerminalLaunchError> { let controller = self .managed_controller .read() diff --git a/crates/freshell-ws/src/create_dedupe.rs b/crates/freshell-ws/src/create_dedupe.rs index 659a7d51b..acbe6b443 100644 --- a/crates/freshell-ws/src/create_dedupe.rs +++ b/crates/freshell-ws/src/create_dedupe.rs @@ -132,6 +132,7 @@ pub enum DedupeDecision { /// the retry proceeds as a fresh create). fn waiter_error(request_id: &str) -> ServerMessage { ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, diff --git a/crates/freshell-ws/src/lib.rs b/crates/freshell-ws/src/lib.rs index 8f18738bf..8335f32ea 100644 --- a/crates/freshell-ws/src/lib.rs +++ b/crates/freshell-ws/src/lib.rs @@ -1065,6 +1065,7 @@ async fn send_error( message: &str, ) -> Result<(), axum::Error> { let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, diff --git a/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs b/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs index 52ec94fa2..6ab332d33 100644 --- a/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs +++ b/crates/freshell-ws/src/managed_fixture_denies_host_pty.rs @@ -24,7 +24,10 @@ impl ManagedTerminalController for SyntheticConfiguration { fn launch<'a>( &'a self, request: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>> { + ) -> ManagedTerminalFuture< + 'a, + Result<ManagedTerminalDescriptor, freshell_terminal::registry::ManagedTerminalLaunchError>, + > { Box::pin(async move { if request.mode != "opencode" { return Err("unexpected provider".into()); diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index 802773425..eb49a1bf3 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -949,6 +949,7 @@ async fn handle_client_text( match value.get("type").and_then(|t| t.as_str()) { Some("terminal.create") => { let reply = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -1083,6 +1084,7 @@ async fn handle_client_text( send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -1279,6 +1281,7 @@ async fn handle_client_text( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -1327,6 +1330,7 @@ async fn handle_client_text( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: Some(snap.generation), owner_epoch: Some(snap.epoch), @@ -1410,6 +1414,7 @@ async fn handle_client_text( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: Some(owner_kind_str.to_string()), owner_generation: Some(snap.generation), owner_epoch: Some(snap.epoch), @@ -1447,6 +1452,7 @@ async fn handle_client_text( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: Some(snap.generation), owner_epoch: Some(snap.epoch), @@ -1556,6 +1562,7 @@ async fn handle_client_text( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: Some(current_generation), owner_epoch: Some(current_epoch), @@ -1592,6 +1599,7 @@ async fn handle_client_text( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: Some(generation), owner_epoch: Some(ownership.boot_epoch()), @@ -2480,6 +2488,7 @@ async fn handle_client_text( send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -3134,6 +3143,7 @@ async fn send_session_reserved( owner: Option<&freshell_freshagent::ownership_lane::TerminalOwnerFields>, ) -> bool { let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: owner.map(|o| o.owner_kind.to_string()), owner_generation: owner.map(|o| o.owner_generation), owner_epoch: owner.map(|o| o.owner_epoch), @@ -3580,7 +3590,13 @@ mod managed_runtime_id_tests { fn launch<'a>( &'a self, _request: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>> { + ) -> ManagedTerminalFuture< + 'a, + Result< + ManagedTerminalDescriptor, + freshell_terminal::registry::ManagedTerminalLaunchError, + >, + > { Box::pin(async { Err("launch must not run during compatibility adoption".into()) }) } @@ -4289,6 +4305,170 @@ mod managed_resume_gate_tests { struct FixedProbe(SessionExistence); + struct MissingSelectedReference { + root: std::path::PathBuf, + calls: Arc<std::sync::atomic::AtomicUsize>, + } + + impl freshell_terminal::registry::ManagedTerminalController for MissingSelectedReference { + fn lookup_terminal<'a>( + &'a self, + _: &'a str, + _: Option<String>, + ) -> freshell_terminal::registry::ManagedTerminalFuture< + 'a, + Result<Option<freshell_terminal::registry::ManagedTerminalDescriptor>, String>, + > { + Box::pin(async { Ok(None) }) + } + fn launch<'a>( + &'a self, + request: freshell_terminal::registry::ManagedTerminalLaunch, + ) -> freshell_terminal::registry::ManagedTerminalFuture< + 'a, + Result< + freshell_terminal::registry::ManagedTerminalDescriptor, + freshell_terminal::registry::ManagedTerminalLaunchError, + >, + > { + Box::pin(async move { + self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let selected = self.root.join("missing-selected.txt"); + let io_error = std::fs::canonicalize(&selected).unwrap_err(); + assert_eq!(io_error.kind(), std::io::ErrorKind::NotFound); + assert_eq!(request.mode, "opencode"); + Err(freshell_terminal::registry::ManagedTerminalLaunchError { + message: "selected configuration reference is unavailable".into(), + provider_preparation_failure: Some(freshell_protocol::ProviderPreparationFailure { + code: freshell_protocol::ProviderPreparationFailureCode::SelectedReferenceUnavailable, + stage: freshell_protocol::ProviderPreparationFailureStage::Configuration, + operation: freshell_protocol::ProviderPreparationFailureOperation::Resolve, + cause: freshell_protocol::ProviderPreparationFailureCause::NotFound, + workspace: self.root.to_string_lossy().into_owned(), + declaring_config: self.root.join("opencode.json").to_string_lossy().into_owned(), + selected_path: Some(selected.to_string_lossy().into_owned()), + }), + }) + }) + } + fn input<'a>( + &'a self, + _: freshell_terminal::registry::ManagedTerminalDescriptor, + _: String, + ) -> freshell_terminal::registry::ManagedTerminalFuture<'a, Result<(), String>> { + panic!("a failed request has no input target") + } + fn resize<'a>( + &'a self, + _: freshell_terminal::registry::ManagedTerminalDescriptor, + _: u16, + _: u16, + ) -> freshell_terminal::registry::ManagedTerminalFuture<'a, Result<(), String>> { + panic!("a failed request has no resize target") + } + fn stop<'a>( + &'a self, + _: freshell_terminal::registry::ManagedTerminalDescriptor, + ) -> freshell_terminal::registry::ManagedTerminalFuture<'a, Result<(), String>> { + panic!("a failed request has no process") + } + fn read_output<'a>( + &'a self, + _: freshell_terminal::registry::ManagedTerminalDescriptor, + _: i64, + _: u64, + ) -> freshell_terminal::registry::ManagedTerminalFuture< + 'a, + Result<freshell_terminal::registry::ManagedOutputRead, String>, + > { + panic!("a failed request has no output target") + } + } + + #[tokio::test] + async fn managed_preparation_failure_keeps_typed_missing_reference_on_the_create_request() { + let root = tempfile::tempdir().unwrap(); + let mut state = super::pane_reconcile_gate_tests::state(); + state.registry = + freshell_terminal::TerminalRegistry::new().with_managed_environment_fixture_only(true); + state.cli_commands = Arc::new(vec![freshell_platform::CliCommandSpec { + name: "opencode".into(), + label: "OpenCode".into(), + default_cmd: "opencode".into(), + ..Default::default() + }]); + let calls = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + state + .registry + .set_managed_controller(Some(Arc::new(MissingSelectedReference { + root: root.path().into(), + calls: Arc::clone(&calls), + }))); + let conn_id = 19; + state.registry.set_managed_runtime_connection(conn_id, true); + let (mut ws_tx, mut client) = + super::pane_reconcile_gate_tests::loopback_sink_and_client().await; + let mut output = crate::create_gate::CreateOutput::Socket(&mut ws_tx); + let create: TerminalCreate = serde_json::from_value(serde_json::json!({ + "requestId": "managed-reference-failed-request", "mode": "opencode", "shell": "system", + "cwd": root.path(), + })) + .unwrap(); + let mut limiter = crate::create_limit::CreateRateLimiter::new(100, 1000); + assert!( + handle_create( + create, + None, + &mut output, + &state, + conn_id, + false, + &mut limiter, + &ConnectionIdentity::default(), + now_ms() + ) + .await + ); + drop(output); + let wire = super::pane_reconcile_gate_tests::next_text_frame(&mut client).await; + let frame: ServerMessage = serde_json::from_value(wire) + .expect("actual WS frame uses the additive serialized contract"); + client.close(None).await.unwrap(); + let ServerMessage::Error(error) = frame else { + panic!("failed request must not create a backend terminal") + }; + assert_eq!(error.code, ErrorCode::PtySpawnFailed); + assert_eq!( + error.request_id.as_deref(), + Some("managed-reference-failed-request") + ); + assert_eq!( + calls.load(std::sync::atomic::Ordering::SeqCst), + 1, + "the actual create reached the missing-file producer" + ); + assert!( + state.registry.directory().is_empty(), + "no backend or ordinary PTY is registered after refusal" + ); + let metadata = error + .provider_preparation_failure + .as_ref() + .expect("WS must retain the producer's typed failure"); + assert_eq!( + metadata.selected_path.as_deref(), + root.path().join("missing-selected.txt").to_str() + ); + assert_eq!( + metadata.cause, + freshell_protocol::ProviderPreparationFailureCause::NotFound + ); + assert!( + state.registry.directory().is_empty(), + "no backend or ordinary PTY may be fabricated for the failed request" + ); + } + impl SessionExistenceProbe for FixedProbe { fn exists(&self, _: &str, _: &str) -> SessionExistence { self.0 @@ -8419,6 +8599,7 @@ pub(crate) async fn send_create_error_with_owner( owner: Option<&freshell_freshagent::ownership_lane::TerminalOwnerFields>, ) -> bool { let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: owner.map(|o| o.owner_kind.to_string()), owner_generation: owner.map(|o| o.owner_generation), owner_epoch: owner.map(|o| o.owner_epoch), @@ -8453,6 +8634,7 @@ pub(crate) async fn send_create_error_with_stale_pair( current_generation: u64, ) -> bool { let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: Some(current_generation), owner_epoch: Some(current_epoch), @@ -8914,6 +9096,7 @@ async fn handle_attach( // never colliding with createRequestIds — see ws-client's // clearTrackedCreate-on-error behavior). AttachReply::Error(Box::new(ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -9082,6 +9265,7 @@ fn input_session_identity_mismatch_error( actual: Option<SessionLocator>, ) -> ServerMessage { ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -9120,6 +9304,7 @@ pub(crate) fn terminal_dims_in_range(cols: i64, rows: i64) -> bool { /// a `requestId` field. fn invalid_dims_error(cols: i64, rows: i64) -> ServerMessage { ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -9179,6 +9364,7 @@ fn managed_runtime_input_blocked( fn managed_runtime_error(terminal_id: &str, detail: &str) -> ServerMessage { ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -9745,6 +9931,7 @@ async fn handle_kill( ) -> bool { let unknown_terminal_error = |terminal_id: String| { ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -9871,6 +10058,7 @@ async fn handle_kill( return send(ws_tx, &msg).await; } let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -9996,6 +10184,7 @@ async fn handle_kill( return send(ws_tx, &msg).await; } let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: refused_owner_kind, owner_generation: refused_owner_generation, owner_epoch: refused_owner_epoch, @@ -10105,6 +10294,7 @@ async fn handle_kill( return send(ws_tx, &msg).await; } let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -10173,6 +10363,7 @@ async fn handle_kill( return send( ws_tx, &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -10214,6 +10405,7 @@ async fn handle_kill( if existed { if persisted_despite_error { let msg = ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, owner_kind: None, owner_generation: None, owner_epoch: None, @@ -12568,7 +12760,7 @@ mod pane_reconcile_gate_tests { tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>, >; - async fn loopback_sink_and_client() -> (WsSink, TestClient) { + pub(super) async fn loopback_sink_and_client() -> (WsSink, TestClient) { let (sink_tx, sink_rx) = tokio::sync::oneshot::channel::<WsSink>(); let sink_tx = std::sync::Arc::new(tokio::sync::Mutex::new(Some(sink_tx))); let router = axum::Router::new().route( @@ -12614,7 +12806,7 @@ mod pane_reconcile_gate_tests { (sink, client) } - async fn next_text_frame(client: &mut TestClient) -> serde_json::Value { + pub(super) async fn next_text_frame(client: &mut TestClient) -> serde_json::Value { let msg = tokio::time::timeout(std::time::Duration::from_secs(5), client.next()) .await .expect("frame within timeout") @@ -12884,7 +13076,13 @@ mod managed_output_order_tests { fn launch<'a>( &'a self, _: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>> { + ) -> ManagedTerminalFuture< + 'a, + Result< + ManagedTerminalDescriptor, + freshell_terminal::registry::ManagedTerminalLaunchError, + >, + > { Box::pin(async { Err("unused launch".into()) }) } @@ -13041,7 +13239,13 @@ mod managed_output_order_tests { fn launch<'a>( &'a self, _: ManagedTerminalLaunch, - ) -> ManagedTerminalFuture<'a, Result<ManagedTerminalDescriptor, String>> { + ) -> ManagedTerminalFuture< + 'a, + Result< + ManagedTerminalDescriptor, + freshell_terminal::registry::ManagedTerminalLaunchError, + >, + > { Box::pin(async { Err("unused".into()) }) } fn input<'a>( @@ -13396,7 +13600,7 @@ mod host_stats_dispatch_tests { tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>, >; - async fn loopback_sink_and_client() -> (WsSink, TestClient) { + pub(super) async fn loopback_sink_and_client() -> (WsSink, TestClient) { let (sink_tx, sink_rx) = tokio::sync::oneshot::channel::<WsSink>(); let sink_tx = std::sync::Arc::new(tokio::sync::Mutex::new(Some(sink_tx))); let router = axum::Router::new().route( @@ -13439,7 +13643,7 @@ mod host_stats_dispatch_tests { (sink, client) } - async fn next_text_frame(client: &mut TestClient) -> serde_json::Value { + pub(super) async fn next_text_frame(client: &mut TestClient) -> serde_json::Value { let msg = tokio::time::timeout(std::time::Duration::from_secs(5), client.next()) .await .expect("frame within timeout") diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index 1578dff78..8b8f1e04c 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -1322,7 +1322,18 @@ export type PongMessage = { timestamp: string } +export type ProviderPreparationFailure = { + code: 'SELECTED_REFERENCE_UNAVAILABLE' + stage: 'configuration' + operation: 'resolve' + cause: 'not_found' | 'permission_denied' | 'cyclic' | 'unknown' + workspace: string + declaringConfig: string + selectedPath?: string +} + export type ErrorMessage = { + providerPreparationFailure?: ProviderPreparationFailure type: 'error' code: ErrorCode message: string diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index 702fa018a..c7f09c79a 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -93,6 +93,7 @@ export type { CrashTrace } * by persistMiddleware, never persisted. */ export type LaunchFailure = { + providerPreparationFailure?: import('@shared/ws-protocol').ProviderPreparationFailure code: 'SESSION_RESERVED' | 'RESTORE_UNAVAILABLE' | 'HANDOFF_IN_PROGRESS' | 'REAP_TIMEOUT' | 'TARGET_SPAWN_FAILED' | 'STALE_GENERATION' | 'LAUNCH_FAILED' | 'SESSION_MISSING' message: string diff --git a/test/unit/client/components/TerminalView.launchRetry.test.tsx b/test/unit/client/components/TerminalView.launchRetry.test.tsx index ec042f9be..9f2e16c7b 100644 --- a/test/unit/client/components/TerminalView.launchRetry.test.tsx +++ b/test/unit/client/components/TerminalView.launchRetry.test.tsx @@ -34,6 +34,7 @@ const wsMocks = vi.hoisted(() => ({ connect: vi.fn().mockResolvedValue(undefined), onMessage: vi.fn(), onReconnect: vi.fn().mockReturnValue(() => {}), + getServerCapabilities: vi.fn(() => ({} as { managedRuntimeV1?: boolean })), })) const terminalThemeMocks = vi.hoisted(() => ({ @@ -60,6 +61,7 @@ vi.mock('@/lib/ws-client', () => ({ connect: wsMocks.connect, onMessage: wsMocks.onMessage, onReconnect: wsMocks.onReconnect, + getServerCapabilities: wsMocks.getServerCapabilities, }), })) @@ -185,7 +187,7 @@ function latestAttachRequestIdForTerminal(terminalId: string | undefined): strin return typeof attach?.attachRequestId === 'string' ? attach.attachRequestId : undefined } -function withCurrentAttachRequestId<T extends { type?: string; terminalId?: string; attachRequestId?: string }>( +function withCurrentAttachRequestId<T extends { type?: string; terminalId?: string; attachRequestId?: string; streamId?: string }>( msg: T & { __preserveMissingAttachRequestId?: boolean; __preserveMissingStreamId?: boolean }, ): T { const isStreamPayload = msg.type === 'terminal.attach.ready' @@ -255,13 +257,14 @@ function createSettingsState() { } } -function makeStore() { +function makeStore(overrides: Partial<TerminalPaneContent> = {}) { const paneContent: TerminalPaneContent = { kind: 'terminal', createRequestId: REQ, status: 'creating', mode: 'shell', shell: 'system', + ...overrides, } const root: PaneNode = { type: 'leaf', id: PANE, content: paneContent } const store = configureStore({ @@ -273,15 +276,16 @@ function makeStore() { }, preloadedState: { tabs: { + ...tabsReducer(undefined, { type: '@@INIT' }), tabs: [{ - id: TAB, mode: 'shell', status: 'running', title: 'Shell', - titleSetByUser: false, createRequestId: REQ, + id: TAB, mode: 'shell' as const, status: 'running' as const, title: 'Shell', + titleSetByUser: false, createRequestId: REQ, createdAt: 1, }], activeTabId: TAB, }, - panes: { layouts: { [TAB]: root }, activePane: { [TAB]: PANE }, paneTitles: {} }, + panes: { ...panesReducer(undefined, { type: '@@INIT' }), layouts: { [TAB]: root }, activePane: { [TAB]: PANE }, paneTitles: {} }, settings: createSettingsState(), - connection: { status: 'connected', error: null }, + connection: { ...connectionReducer(undefined, { type: '@@INIT' }), status: 'connected' as const }, }, }) return { store, paneContent } @@ -297,7 +301,7 @@ function paneStatus(store: ReturnType<typeof makeStore>['store']) { } async function renderPane(store: any, paneContent: TerminalPaneContent) { - render( + const view = render( <Provider store={store}> <TerminalView tabId={TAB} paneId={PANE} paneContent={paneContent} /> </Provider> @@ -307,6 +311,7 @@ async function renderPane(store: any, paneContent: TerminalPaneContent) { await Promise.resolve() }) expect(messageHandler).not.toBeNull() + return view } /** Anchor the launch: server acks the create, launchAttempt gets terminalId. */ @@ -342,6 +347,7 @@ describe('launch-time INVALID_TERMINAL_ID bounded retry', () => { latestAttachRequestIdByTerminal.clear() latestStreamIdByTerminal.clear() wsMocks.send.mockClear() + wsMocks.getServerCapabilities.mockReturnValue({}) wsMocks.send.mockImplementation((msg: any) => { if ( msg?.type === 'terminal.attach' @@ -394,6 +400,89 @@ describe('launch-time INVALID_TERMINAL_ID bounded retry', () => { installPerfAuditBridge(null) }) + it('keeps the real managed preparation failure on its request through reconnect without fabricating a terminal', async () => { + wsMocks.getServerCapabilities.mockReturnValue({ managedRuntimeV1: true }) + const { store, paneContent } = makeStore({ mode: 'opencode', initialCwd: '/workspace/missing-reference' }) + await renderPane(store, paneContent) + const before = sentCreates().length + expect(before).toBeGreaterThan(0) + await act(async () => messageHandler!({ type: 'error', code: 'PTY_SPAWN_FAILED', requestId: REQ, + message: 'arbitrary-private-config-marker', + providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE' as const, stage: 'configuration' as const, operation: 'resolve' as const, workspace: '/workspace', declaringConfig: '/workspace/opencode.json', + selectedPath: '/workspace/missing-reference/missing-selected.txt', + cause: 'not_found' as const }, + })) + const content = (store.getState().panes.layouts[TAB] as { content: TerminalPaneContent }).content + expect(content).toMatchObject({ createRequestId: REQ, status: 'error', launchFailure: { + code: 'LAUNCH_FAILED', retryable: true, + providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE' as const, stage: 'configuration' as const, operation: 'resolve' as const, workspace: '/workspace', declaringConfig: '/workspace/opencode.json', + selectedPath: '/workspace/missing-reference/missing-selected.txt', + cause: 'not_found' as const }, + } }) + expect(content.terminalId).toBeUndefined() + expect(content.soulId).toBeUndefined() + expect(content).not.toHaveProperty('incarnationId') + expect(content.sessionRef).toBeUndefined() + expect(JSON.stringify(content.launchFailure)).not.toContain('arbitrary-private-config-marker') + await act(async () => reconnectHandler!()) + expect(sentCreates()).toHaveLength(before) + }) + + it('keeps a saved managed preparation failure on remount and retries only after the existing explicit action', async () => { + wsMocks.getServerCapabilities.mockReturnValue({ managedRuntimeV1: true }) + const failure = Object.assign({ code: 'LAUNCH_FAILED' as const, retryable: true, + message: 'Configuration blocked: /workspace/missing-reference/missing-selected.txt: referenced file was not found' }, { + providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE' as const, stage: 'configuration' as const, operation: 'resolve' as const, workspace: '/workspace', declaringConfig: '/workspace/opencode.json', + selectedPath: '/workspace/missing-reference/missing-selected.txt', + cause: 'not_found' as const }, + }) + const { store, paneContent } = makeStore({ mode: 'opencode', status: 'error', launchFailure: failure }) + const view = await renderPane(store, paneContent) + expect(sentCreates()).toHaveLength(0) + expect(screen.getByTestId('terminal-launch-failure-card')).toHaveTextContent('missing-selected.txt') + expect(screen.queryByRole('button', { name: 'Attach to running session' })).not.toBeInTheDocument() + await act(async () => reconnectHandler!()) + expect(sentCreates()).toHaveLength(0) + fireEvent.click(screen.getByRole('button', { name: 'Retry launch' })) + const next = (store.getState().panes.layouts[TAB] as { content: TerminalPaneContent }).content + view.rerender(<Provider store={store}><TerminalView tabId={TAB} paneId={PANE} paneContent={next} /></Provider>) + await act(async () => { await Promise.resolve() }) + expect(sentCreates()).toHaveLength(1) + expect(sentCreates()[0].requestId).toBe(REQ) + expect(next.launchFailure).toBeUndefined() + expect(next.terminalId).toBeUndefined() + expect(next.sessionRef).toBeUndefined() + }) + + it('keeps ordinary untyped create failures on their existing notice-only path', async () => { + const { store, paneContent } = makeStore() + await renderPane(store, paneContent) + await act(async () => messageHandler!({ type: 'error', code: 'PTY_SPAWN_FAILED', requestId: REQ, + message: 'ordinary launch failed' })) + expect(paneStatus(store)).toBe('error') + expect((store.getState().panes.layouts[TAB] as { content: TerminalPaneContent }).content.launchFailure).toBeUndefined() + expect(screen.queryByTestId('terminal-launch-failure-card')).not.toBeInTheDocument() + }) + + it.each(['ordinary-mode', 'no-managed-capability', 'different-request'] as const)( + 'does not retain preparation metadata outside its active managed request: %s', async (control) => { + wsMocks.getServerCapabilities.mockReturnValue({ managedRuntimeV1: control !== 'no-managed-capability' }) + const { store, paneContent } = makeStore({ mode: control === 'ordinary-mode' ? 'shell' : 'opencode' }) + await renderPane(store, paneContent) + await act(async () => messageHandler!({ type: 'error', code: 'PTY_SPAWN_FAILED', + requestId: control === 'different-request' ? 'another-request' : REQ, message: 'unrelated failure', + providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE', stage: 'configuration', + operation: 'resolve', cause: 'not_found', workspace: '/workspace', + declaringConfig: '/workspace/opencode.json', selectedPath: '/workspace/missing-selected.txt' }, + })) + const content = (store.getState().panes.layouts[TAB] as { content: TerminalPaneContent }).content + expect(content.createRequestId).toBe(REQ) + expect(content.launchFailure).toBeUndefined() + expect(content.terminalId).toBeUndefined() + expect(screen.queryByTestId('terminal-launch-failure-card')).not.toBeInTheDocument() + }, + ) + it('retries terminal.create with the SAME requestId and restore:true after a launch-time INVALID_TERMINAL_ID', async () => { vi.useFakeTimers() addTerminalRestoreRequestId(REQ) // this pane is a restore round @@ -669,7 +758,7 @@ describe('launch-time INVALID_TERMINAL_ID bounded retry', () => { render(<Provider store={store}><TerminalView tabId={TAB} paneId={PANE} paneContent={content} /></Provider>) expect(screen.getAllByRole('alert')).toHaveLength(1) expect(screen.queryByTestId('terminal-launch-failure-card')).not.toBeInTheDocument() - expect(screen.queryByRole('button', { name: 'Retry', exact: true })).not.toBeInTheDocument() + expect(screen.queryByRole('button', { name: /^Retry$/ })).not.toBeInTheDocument() }) it('clears prior managed retry feedback when a different terminal conversation occupies the pane', async () => { diff --git a/test/unit/client/store/panesPersistence.test.ts b/test/unit/client/store/panesPersistence.test.ts index 8dce72aaa..a547be59d 100644 --- a/test/unit/client/store/panesPersistence.test.ts +++ b/test/unit/client/store/panesPersistence.test.ts @@ -29,6 +29,8 @@ import { } from '../../../../src/store/persistMiddleware' import { PANES_SCHEMA_VERSION } from '../../../../src/store/persistedState' import { isWellFormedPaneTree } from '../../../../src/store/paneTreeValidation' +import { flushPersistedLayoutNow } from '../../../../src/store/persistControl' +import type { LaunchFailure, TerminalPaneContent } from '../../../../src/store/paneTypes' // Delta round 3, finding 1: the flush writes THIS window's per-window layout // key (freshell.layout.v3.<layoutWindowId>) and its per-window pre- @@ -58,6 +60,41 @@ describe('Panes Persistence Integration', () => { vi.useRealTimers() }) + it.each(['managed-preparation', 'ordinary', 'owner-stamped', 'reserved'] as const)( + 'restores only the safe managed preparation failure across a real flush/load: %s', (kind) => { + const failure: LaunchFailure = Object.assign({ code: kind === 'reserved' ? 'SESSION_RESERVED' as const : 'LAUNCH_FAILED' as const, + message: kind === 'managed-preparation' ? 'arbitrary-secret-config-marker' : 'ordinary transient failure', + retryable: true }, kind === 'managed-preparation' ? { + providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE' as const, stage: 'configuration' as const, operation: 'resolve' as const, workspace: '/workspace', declaringConfig: '/workspace/opencode.json', + selectedPath: '/workspace/missing-selected.txt', + cause: 'not_found' as const }, + } : kind === 'owner-stamped' ? { ownerKind: 'terminal' as const, ownerGeneration: 7 } : {}) + const store = configureStore({ reducer: { tabs: tabsReducer, panes: panesReducer }, + middleware: getDefault => getDefault().concat(persistMiddleware as any) }) + store.dispatch(addTab({ mode: 'opencode' })) + const tabId = store.getState().tabs.tabs[0].id + store.dispatch(initLayout({ tabId, content: { kind: 'terminal', mode: 'opencode', + status: 'error', createRequestId: 'same-failed-managed-request', launchFailure: failure } satisfies TerminalPaneContent })) + store.dispatch(flushPersistedLayoutNow()) + resetPersistedLayoutCacheForTests() + resetPersistedPanesCacheForTests() + const loaded = loadPersistedPanes() + expect(loaded).toBeTruthy() + const restored = configureStore({ reducer: { tabs: tabsReducer, panes: panesReducer } }) + restored.dispatch(hydratePanes(loaded)) + const content = (restored.getState().panes.layouts[tabId] as any).content + expect(content.createRequestId).toBe('same-failed-managed-request') + expect(content.terminalId).toBeUndefined() + expect(content.soulId).toBeUndefined() + expect(content.sessionRef).toBeUndefined() + if (kind === 'managed-preparation') { + expect(content.launchFailure).toEqual({ ...failure, + message: 'Configuration blocked: /workspace/missing-selected.txt: referenced file was not found' }) + expect(JSON.stringify(localStorageMock._getStore())).not.toContain('arbitrary-secret-config-marker') + } else expect(content.launchFailure).toBeUndefined() + }, + ) + it('persists and restores panes across page refresh', () => { // 1. Create a store (simulates initial page load) const store1 = configureStore({ From edd1ca50c6129843124b3e02061b4805fb7a9c5a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 19:18:35 -0700 Subject: [PATCH 284/493] fix: retain managed configuration failures on their create requests --- .../src/managed_provider_launch_tests.rs | 8 +- .../src/managed_provider_references.rs | 97 ++++++++++++++++--- .../environment.rs | 1 + .../managed_provider_references/sources.rs | 5 + crates/freshell-ws/src/terminal.rs | 81 +++++++++++----- src/components/TerminalView.tsx | 16 ++- src/store/paneTypes.ts | 61 +++++++++++- src/store/persistMiddleware.ts | 11 ++- 8 files changed, 229 insertions(+), 51 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 220250aaf..0d5c1f433 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -2406,7 +2406,8 @@ let raw='';process.stdin.on('data',b=>raw+=b);process.stdin.on('end',()=>{ if changed_git { let error = terminal .unwrap() - .expect_err("actual final source mismatch must block terminal startup"); + .expect_err("actual final source mismatch must block terminal startup") + .message; println!( "provider_source_guard_diagnostic {}", json!({"selectionChanged":error.contains("Claude native source selection changed"),"commonDir":error.contains("git common dir"),"hostUnreachable":error.contains("host") }) @@ -4076,8 +4077,9 @@ export default {{id:'closed-directory-discovery',async server(){{ } else { std::fs::rename(&hidden, &selected).unwrap(); } - let error = - failed.expect_err("selected unavailable resource must block the actual launch"); + let error = failed + .expect_err("selected unavailable resource must block the actual launch") + .message; assert!( error.contains(field) && error.contains(if unreadable { "unreadable" } else { "missing" }) diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 018ea4f09..d782d8331 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -16,6 +16,7 @@ pub struct FileSourceOrigin<'a> { } pub struct ReferenceError { + pub preparation_failure: Option<Box<freshell_protocol::ProviderPreparationFailure>>, pub operation: &'static str, pub field: &'static str, pub source_ordinal: usize, @@ -32,6 +33,7 @@ impl std::fmt::Debug for ReferenceError { .field("declaring_source", &self.declaring_source) .field("variables", &self.variables) .field("cause", &self.cause) + .field("preparation_failure", &self.preparation_failure) .finish() } } @@ -45,15 +47,25 @@ impl std::fmt::Display for ReferenceError { self.source_ordinal, self.declaring_source.display(), self.cause - ) + )?; + if let Some(path) = self + .preparation_failure + .as_ref() + .and_then(|error| error.selected_path.as_ref()) + { + write!(f, " ({path})")?; + } + Ok(()) } } impl std::error::Error for ReferenceError {} -// Preserve the existing generic error until the typed producer behavior is implemented. impl From<ReferenceError> for freshell_terminal::registry::ManagedTerminalLaunchError { fn from(error: ReferenceError) -> Self { - error.to_string().into() + Self { + message: error.to_string(), + provider_preparation_failure: error.preparation_failure.map(|failure| *failure), + } } } @@ -173,6 +185,7 @@ pub(super) fn resolve_file_references_from_view_relocated( let substitution_events = std::cell::Cell::new(0usize); let input_error = std::cell::Cell::new(None); let fail = |ordinal, variables: Vec<String>, cause| ReferenceError { + preparation_failure: None, operation: "resolve", field: "OpenCode file", source_ordinal: ordinal, @@ -180,6 +193,35 @@ pub(super) fn resolve_file_references_from_view_relocated( variables, cause, }; + let fail_io = |ordinal, names: Vec<String>, path: &Path, error: std::io::Error, reason| { + use freshell_protocol::{ + ProviderPreparationFailure, ProviderPreparationFailureCause, + ProviderPreparationFailureCode, ProviderPreparationFailureOperation, + ProviderPreparationFailureStage, + }; + let cause = match error.kind() { + std::io::ErrorKind::NotFound => ProviderPreparationFailureCause::NotFound, + std::io::ErrorKind::PermissionDenied => { + ProviderPreparationFailureCause::PermissionDenied + } + _ if error.raw_os_error() == Some(40) => ProviderPreparationFailureCause::Cyclic, + _ => ProviderPreparationFailureCause::Unknown, + }; + let selected_path = names + .is_empty() + .then(|| path.to_string_lossy().into_owned()); + let mut failure = fail(ordinal, names, reason); + failure.preparation_failure = Some(Box::new(ProviderPreparationFailure { + code: ProviderPreparationFailureCode::SelectedReferenceUnavailable, + stage: ProviderPreparationFailureStage::Configuration, + operation: ProviderPreparationFailureOperation::Resolve, + cause, + workspace: origin.cwd.to_string_lossy().into_owned(), + declaring_config: origin.declaring_config.to_string_lossy().into_owned(), + selected_path, + })); + failure + }; let lookup = |name: &str| { let mut variables = variables.borrow_mut(); if variables.len() >= 128 && !variables.contains(name) { @@ -214,17 +256,21 @@ pub(super) fn resolve_file_references_from_view_relocated( origin.original_home, ); let physical_source = projection.relocated_files.get(&path).unwrap_or(&path); - let canonical = std::fs::canonicalize(physical_source).map_err(|_| { - fail( + let canonical = std::fs::canonicalize(physical_source).map_err(|error| { + fail_io( token.ordinal, names.clone(), + &path, + error, "referenced file is missing, inaccessible or cyclic", ) })?; - let metadata = std::fs::metadata(&canonical).map_err(|_| { - fail( + let metadata = std::fs::metadata(&canonical).map_err(|error| { + fail_io( token.ordinal, names.clone(), + &path, + error, "referenced file metadata unavailable", ) })?; @@ -235,10 +281,12 @@ pub(super) fn resolve_file_references_from_view_relocated( "referenced target must be a regular file at most 16 MiB", )); } - std::fs::File::open(&canonical).map_err(|_| { - fail( + std::fs::File::open(&canonical).map_err(|error| { + fail_io( token.ordinal, names.clone(), + &path, + error, "referenced file is unreadable", ) })?; @@ -431,6 +479,7 @@ pub fn evaluate_generated_recipe( env: &dyn Fn(&str) -> Option<String>, ) -> Result<String, ReferenceError> { freshell_platform::opencode_recipe::evaluate(recipe, env).map_err(|cause| ReferenceError { + preparation_failure: None, operation: "prepare", field: "OpenCode generated configuration", source_ordinal: 0, @@ -468,8 +517,8 @@ pub(crate) struct OpencodePreparation<'a> { pub(crate) async fn prepare_managed_opencode( request: OpencodePreparation<'_>, -) -> Result<PreparedManagedOpencode, String> { - prepare_managed_opencode_with_publisher( +) -> Result<PreparedManagedOpencode, freshell_terminal::registry::ManagedTerminalLaunchError> { + prepare_managed_opencode_with_typed_publisher( request, crate::managed_mcp_capability::issue_prepared_opencode_context, ) @@ -485,6 +534,20 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( crate::managed_mcp_capability::OpencodeEphemeralInput<'_>, ) -> Result<freshell_runtime_protocol::ProviderLaunchContext, String>, ) -> Result<PreparedManagedOpencode, String> { + prepare_managed_opencode_with_typed_publisher(request, publish) + .await + .map_err(|error| error.message) +} + +async fn prepare_managed_opencode_with_typed_publisher( + request: OpencodePreparation<'_>, + publish: impl FnOnce( + &freshell_runtime_protocol::SoulId, + &Path, + &freshell_runtime_protocol::ProviderLaunchContext, + crate::managed_mcp_capability::OpencodeEphemeralInput<'_>, + ) -> Result<freshell_runtime_protocol::ProviderLaunchContext, String>, +) -> Result<PreparedManagedOpencode, freshell_terminal::registry::ManagedTerminalLaunchError> { let OpencodePreparation { client, soul, @@ -796,7 +859,12 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( } let plan = environment .plan_sources(loaded, cwd, home) - .map_err(|error| error.to_string())?; + .map_err(|mut error| { + if let Some(failure) = error.preparation_failure.as_mut() { + failure.workspace = workspace.to_string_lossy().into_owned(); + } + freshell_terminal::registry::ManagedTerminalLaunchError::from(error) + })?; let frozen = environment.frozen_inputs(&plan)?; let frozen_names = frozen .iter() @@ -859,7 +927,7 @@ pub(crate) async fn prepare_managed_opencode_with_publisher( .await { let _ = crate::managed_mcp_capability::revoke_mcp_capability(&capability.grant_id); - return Err(error.to_string()); + return Err(error.to_string().into()); } } context @@ -931,7 +999,8 @@ pub(crate) async fn launch_initial_opencode( inherited_inline: None, inline_edits: preparation.inline_edits.clone(), }) - .await?; + .await + .map_err(|error| error.message)?; if let Some(terminal) = launch.terminal.as_mut() { terminal.provider_secret_references = prepared.references; terminal.provider_launch_context = Some(prepared.context); diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index 7de17f00e..3dc47c964 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -126,6 +126,7 @@ impl SourceEnvironment { .any(|source| source.source.lane == SourceLane::Main) { return Err(super::ReferenceError { + preparation_failure: None, operation: "prepare", field: "OpenCode native auth source", source_ordinal: 0, diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index ed9054b14..0ba8b4057 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -367,6 +367,7 @@ pub fn read_opencode_sources( let mut retained = 0usize; for (ordinal, source) in sources.into_iter().enumerate() { let fail = |cause| super::ReferenceError { + preparation_failure: None, operation: "read", field: "OpenCode source", source_ordinal: ordinal, @@ -421,6 +422,7 @@ pub fn source_environment_names( let mut events = 0usize; for (ordinal, source) in sources.iter().enumerate() { let fail = |cause| super::ReferenceError { + preparation_failure: None, operation: "select", field: "OpenCode source", source_ordinal: ordinal, @@ -515,6 +517,7 @@ pub fn plan_loaded_opencode_sources_relocated( view_budget .charge(&loaded.view) .map_err(|cause| super::ReferenceError { + preparation_failure: None, operation: "read", field: "OpenCode source", source_ordinal: 0, @@ -528,6 +531,7 @@ pub fn plan_loaded_opencode_sources_relocated( let original_form = source.lane != SourceLane::Inline && canonical_occurrences[&source.canonical_source] > 1; let fail = |cause| super::ReferenceError { + preparation_failure: None, operation: "read", field: "OpenCode source", source_ordinal: ordinal, @@ -658,6 +662,7 @@ pub fn plan_loaded_opencode_sources_relocated( lookup(planned.source.lane, &planned.source.logical_path, name) }) .map_err(|error| super::ReferenceError { + preparation_failure: None, operation: "select", field: error.field, source_ordinal: ordinal, diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index eb49a1bf3..af796ba4f 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -4408,28 +4408,29 @@ mod managed_resume_gate_tests { state.registry.set_managed_runtime_connection(conn_id, true); let (mut ws_tx, mut client) = super::pane_reconcile_gate_tests::loopback_sink_and_client().await; - let mut output = crate::create_gate::CreateOutput::Socket(&mut ws_tx); - let create: TerminalCreate = serde_json::from_value(serde_json::json!({ + { + let mut output = crate::create_gate::CreateOutput::Socket(&mut ws_tx); + let create: TerminalCreate = serde_json::from_value(serde_json::json!({ "requestId": "managed-reference-failed-request", "mode": "opencode", "shell": "system", "cwd": root.path(), })) .unwrap(); - let mut limiter = crate::create_limit::CreateRateLimiter::new(100, 1000); - assert!( - handle_create( - create, - None, - &mut output, - &state, - conn_id, - false, - &mut limiter, - &ConnectionIdentity::default(), - now_ms() - ) - .await - ); - drop(output); + let mut limiter = crate::create_limit::CreateRateLimiter::new(100, 1000); + assert!( + handle_create( + create, + None, + &mut output, + &state, + conn_id, + false, + &mut limiter, + &ConnectionIdentity::default(), + now_ms() + ) + .await + ); + } let wire = super::pane_reconcile_gate_tests::next_text_frame(&mut client).await; let frame: ServerMessage = serde_json::from_value(wire) .expect("actual WS frame uses the additive serialized contract"); @@ -6792,6 +6793,7 @@ pub(crate) async fn handle_create( // every non-negotiating connection retain the legacy local spawn path. // PIN2_PTY_SPAWN_ANCHOR: either the local PTY spawn OR the supervisor's // host-owned PTY makes the preallocated identity observable. + let mut provider_preparation_failure = None; let create_result: std::io::Result<()> = if use_managed_runtime { let managed = ManagedTerminalLaunch { spec: spec.clone(), @@ -6818,9 +6820,11 @@ pub(crate) async fn handle_create( ); Ok(()) } - Err(error) => Err(std::io::Error::other(format!( - "managed runtime launch failed: {error}" - ))), + Err(error) => { + let message = format!("managed runtime launch failed: {error}"); + provider_preparation_failure = error.provider_preparation_failure; + Err(std::io::Error::other(message)) + } } } else { // The legacy PTY spawn is synchronous; run it on the blocking pool so @@ -6932,8 +6936,16 @@ pub(crate) async fn handle_create( env_var.as_deref(), resume_session_id.is_some(), ); - return send_create_error(out, ErrorCode::PtySpawnFailed, message, &create.request_id) - .await; + return send_create_error_details( + out, + ErrorCode::PtySpawnFailed, + message, + &create.request_id, + None, + None, + provider_preparation_failure, + ) + .await; } // D8: arm the lease's TTL kill path — record the just-spawned child's pid @@ -8597,9 +8609,30 @@ pub(crate) async fn send_create_error_with_owner( request_id: &str, live_terminal_id: Option<String>, owner: Option<&freshell_freshagent::ownership_lane::TerminalOwnerFields>, +) -> bool { + send_create_error_details( + out, + code, + message, + request_id, + live_terminal_id, + owner, + None, + ) + .await +} + +async fn send_create_error_details( + out: &mut crate::create_gate::CreateOutput<'_>, + code: ErrorCode, + message: String, + request_id: &str, + live_terminal_id: Option<String>, + owner: Option<&freshell_freshagent::ownership_lane::TerminalOwnerFields>, + provider_preparation_failure: Option<freshell_protocol::ProviderPreparationFailure>, ) -> bool { let msg = ServerMessage::Error(ErrorMsg { - provider_preparation_failure: None, + provider_preparation_failure, owner_kind: owner.map(|o| o.owner_kind.to_string()), owner_generation: owner.map(|o| o.owner_generation), owner_epoch: owner.map(|o| o.owner_epoch), diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index c069d9804..531375c95 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -74,6 +74,7 @@ import { TerminalLaunchFailureCard } from '@/components/TerminalLaunchFailureCar import { FencedOwnerRecoveryActions, SessionHandoffErrorBanner } from '@/components/SessionHandoffErrorBanner' import { buildResumeContent, freshSessionTypeForPaneFlavor } from '@/lib/session-type-utils' import type { LaunchFailure } from '@/store/paneTypes' +import { buildManagedPreparationFailure, retainedManagedPreparationFailure } from '@/store/paneTypes' import { dismissTabGreen } from '@/store/turnCompletionAttention' import { focusNextTerminalSearchMatch, focusPreviousTerminalSearchMatch, loadTerminalSearch } from '@/store/terminalDirectoryThunks' import { isFatalConnectionErrorCode } from '@/store/connectionSlice' @@ -6471,7 +6472,15 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te // the card carries the attach/retry/open-as-fresh-agent actions. // Rides the same updateContent (the ref-based merge would // otherwise clobber it). - const typedLaunchFailure: LaunchFailure | undefined = msg.ownerKind !== undefined || typeof msg.ownerGeneration === 'number' + const preparationFailure = launchAttempt && msg.code === 'PTY_SPAWN_FAILED' + && mode === 'opencode' && !tid && !contentRef.current?.soulId + && !contentRef.current?.sessionRef && !contentRef.current?.resumeSessionId + && contentRef.current?.createRequestId === reqId + && ws.getServerCapabilities?.()?.managedRuntimeV1 === true + && msg.ownerKind === undefined && msg.ownerGeneration === undefined + ? buildManagedPreparationFailure(msg.providerPreparationFailure) + : undefined + const typedLaunchFailure: LaunchFailure | undefined = preparationFailure ?? (msg.ownerKind !== undefined || typeof msg.ownerGeneration === 'number' ? { code: msg.code === 'RESTORE_UNAVAILABLE' ? 'RESTORE_UNAVAILABLE' : 'LAUNCH_FAILED', message: msg.message || msg.code || 'The launch was refused.', @@ -6480,7 +6489,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te ...(typeof msg.ownerGeneration === 'number' ? { ownerGeneration: msg.ownerGeneration } : {}), ...(typeof msg.liveTerminalId === 'string' ? { terminalId: msg.liveTerminalId } : {}), } - : undefined + : undefined) updateContent({ status: 'error', streamId: undefined, @@ -6496,7 +6505,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const prefix = launchAttempt ? (launchAttempt.restore ? '[Restore failed]' : '[Launch failed]') : '[Error]' - writeLocalXtermNotice(term, `\r\n${prefix} ${msg.message || msg.code || 'Unknown error'}\r\n`) + writeLocalXtermNotice(term, `\r\n${prefix} ${preparationFailure?.message || msg.message || msg.code || 'Unknown error'}\r\n`) } const activeAttachForInvalidTerminalError = currentAttachRef.current @@ -6979,6 +6988,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te pendingSinceSeq: 0, pendingReason: 'initial_hydrate', } + if (contentRef.current && retainedManagedPreparationFailure(contentRef.current)) return sendCreate(createRequestId) } } diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index c7f09c79a..5e86f10a0 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -10,6 +10,7 @@ import type { FreshAgentRuntimeProvider, FreshAgentSessionType } from '@shared/f import type { FreshAgentStyle } from '@shared/settings' import type { SessionNameRef } from '@shared/session-names' import type { ManagedRuntimeProjectionFields } from '@shared/managed-runtime' +import type { ProviderPreparationFailure } from '@shared/ws-protocol' export type SessionLocator = SharedSessionLocator @@ -89,11 +90,12 @@ export type { CrashTrace } * TerminalView's create-error fold for refusal frames carrying the typed * owner fields (the additive `ownerKind`/`ownerGeneration` on * SESSION_RESERVED / RESTORE_UNAVAILABLE); rendered by the typed failure - * card with attach/retry/open-as-fresh-agent actions. VOLATILE — stripped - * by persistMiddleware, never persisted. + * card with attach/retry/open-as-fresh-agent actions. Owner refusals are + * volatile. A typed managed preparation failure can survive reload before + * a backend terminal exists, so reload does not repeat the failed request. */ export type LaunchFailure = { - providerPreparationFailure?: import('@shared/ws-protocol').ProviderPreparationFailure + providerPreparationFailure?: ProviderPreparationFailure code: 'SESSION_RESERVED' | 'RESTORE_UNAVAILABLE' | 'HANDOFF_IN_PROGRESS' | 'REAP_TIMEOUT' | 'TARGET_SPAWN_FAILED' | 'STALE_GENERATION' | 'LAUNCH_FAILED' | 'SESSION_MISSING' message: string @@ -117,11 +119,64 @@ export type HandoffError = { generation: number } +function normalizeProviderPreparationFailure(value: unknown): ProviderPreparationFailure | undefined { + if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined + const record = value as Record<string, unknown> + const isPath = (path: unknown): path is string => typeof path === 'string' + && path.length > 0 && !/[\u0000-\u001f]/.test(path) + if (record.code !== 'SELECTED_REFERENCE_UNAVAILABLE' || record.stage !== 'configuration' + || record.operation !== 'resolve' || !isPath(record.workspace) || !isPath(record.declaringConfig) + || !['not_found', 'permission_denied', 'cyclic', 'unknown'].includes(String(record.cause)) + || (record.selectedPath !== undefined && !isPath(record.selectedPath))) return undefined + return { + code: record.code, + stage: record.stage, + operation: record.operation, + cause: record.cause as ProviderPreparationFailure['cause'], + workspace: record.workspace, + declaringConfig: record.declaringConfig, + ...(record.selectedPath !== undefined ? { selectedPath: record.selectedPath as string } : {}), + } +} + +/** Only typed selection metadata enters the retained card; raw error text stays volatile. */ +export function buildManagedPreparationFailure(value: unknown): LaunchFailure | undefined { + const failure = normalizeProviderPreparationFailure(value) + if (!failure) return undefined + const reasons: Record<ProviderPreparationFailure['cause'], string> = { + not_found: 'referenced file was not found', + permission_denied: 'referenced file is inaccessible', + cyclic: 'referenced file is cyclic', + unknown: 'referenced file is unavailable', + } + return { + code: 'LAUNCH_FAILED', + message: `Configuration blocked: ${failure.selectedPath ?? failure.declaringConfig}: ${reasons[failure.cause]}`, + retryable: true, + providerPreparationFailure: failure, + } +} + +/** A failure before registration blocks automatic creation, without granting runtime authority. */ +export function retainedManagedPreparationFailure(content: TerminalPaneContent): LaunchFailure | undefined { + const failure = content.launchFailure + if (content.mode !== 'opencode' || content.status !== 'error' || !content.createRequestId + || content.terminalId || content.soulId || content.sessionRef || content.resumeSessionId + || failure?.code !== 'LAUNCH_FAILED' || failure.ownerKind !== undefined + || failure.ownerGeneration !== undefined || failure.terminalId !== undefined) return undefined + return buildManagedPreparationFailure(failure.providerPreparationFailure) +} + /** Sanitize an unknown value into a LaunchFailure (normalizePaneContent's whitelist gate). */ export function normalizeLaunchFailure(value: unknown): LaunchFailure | undefined { if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined const record = value as Record<string, unknown> if (typeof record.code !== 'string' || typeof record.message !== 'string') return undefined + if (record.code === 'LAUNCH_FAILED' && record.ownerKind === undefined + && record.ownerGeneration === undefined && record.terminalId === undefined) { + const failure = buildManagedPreparationFailure(record.providerPreparationFailure) + if (failure) return failure + } return { code: record.code as LaunchFailure['code'], message: record.message, diff --git a/src/store/persistMiddleware.ts b/src/store/persistMiddleware.ts index 8d87d17d1..a8c40bcae 100644 --- a/src/store/persistMiddleware.ts +++ b/src/store/persistMiddleware.ts @@ -30,7 +30,7 @@ import { getSelectedMachineId } from '@/lib/machine-identity' import { consumeArmedPreMigrationEvidenceClear } from '@/lib/recovery/layout-health' import { flushPersistedLayoutNow } from './persistControl' import { sanitizeSessionRef } from '@shared/session-contract' -import { normalizeFreshAgentEffortOverride, normalizeFreshAgentPaneModelSelection } from './paneTypes' +import { normalizeFreshAgentEffortOverride, normalizeFreshAgentPaneModelSelection, retainedManagedPreparationFailure } from './paneTypes' import { loadPersistedTabRecency, mergeTabRecencyStatesByMax, @@ -304,16 +304,19 @@ function stripTransientSessionFields(content: any): any { // scrubbed from persisted layouts on the next wholesale flush. showThinking: _legacyShowThinking, showTools: _legacyShowTools, - // kata b8ke: the typed failure surfaces are volatile — the launch - // failure card and the handoff-error banner re-derive from the live - // server state after a reload, never from a persisted snapshot. + // Owner refusals and handoff failures re-derive from live state. Only + // a typed pre-registration managed preparation failure survives below. launchFailure: _launchFailure, handoffError: _handoffError, ...rest } = content + const preparationFailure = content.kind === 'terminal' + ? retainedManagedPreparationFailure(content) + : undefined return { ...rest, + ...(preparationFailure ? { launchFailure: preparationFailure } : {}), ...(content.kind === 'fresh-agent' && !sessionRef && typeof content.serverInstanceId === 'string' && typeof content.sessionId === 'string' ? { sessionId: content.sessionId } : {}), From 25dff05ced1a9cdb071e744e1f822f8a63a45303 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 19:55:52 -0700 Subject: [PATCH 285/493] fix: retry managed preparation failures without a resume identity --- src/components/TerminalView.tsx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index 531375c95..f8f40b791 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -7344,13 +7344,13 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const retryLaunch = () => { if (managedRecoveryDecision) return - // The Relaunch discipline: a reconcile-driven respawn create re-fires - // the lifecycle effect (the reconcileEpoch bump is its ONLY re-fire - // signal — createRequestId is preserved, never re-minted). + // Preparation can fail before a session exists. Its explicit Retry + // starts that same create request without a resume identity; other + // launch failures retain the existing respawn behavior. dispatch(resetPaneForReconcileCreate({ tabId, paneId, - intent: 'respawn', + intent: retainedManagedPreparationFailure(terminalContent) ? 'fresh' : 'respawn', sessionRef: terminalContent.sessionRef, })) } From 459db7a258e6a3b9bd487e8b241c6a6d5cf4b607 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 20:00:31 -0700 Subject: [PATCH 286/493] test: protect managed browser attempt completion and retries --- scripts/testing/managed-environment-create.ts | 33 ++++++++ .../managed-provider-environment-target.ts | 23 +++--- .../tooling/testing/runtime-sandbox.test.ts | 75 ++++++++++++++++++- 3 files changed, 118 insertions(+), 13 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 58186b533..24c82180c 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -6,6 +6,39 @@ import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' import { dockerSocketPath } from './runtime-sandbox.js' import { providerParityFailureDiagnostics } from '../../test/helpers/provider-parity-environment.js' import type { ManagedRuntimeBrowserRig } from '../../test/e2e-browser/helpers/managed-runtime.js' +import type { ManagedEnvironmentFixture } from './e2e-external-target.js' + +export type ManagedEnvironmentAttempt = ManagedEnvironmentFixture & { + scenario: string + attempt: number + createRequestId?: string +} + +export type ManagedEnvironmentCompletion = ManagedEnvironmentAttempt & { + status: string + willRetry: boolean +} + +export async function serviceManagedEnvironmentScenario( + fixture: ManagedEnvironmentFixture, scenario: string, + operations: { + nextAttempt(attempt: number): Promise<{ request?: ManagedEnvironmentAttempt; completion?: ManagedEnvironmentCompletion }> + authorize(requestId: string): void + publishAuthorization(request: ManagedEnvironmentAttempt): Promise<void> + onAuthorized(request: ManagedEnvironmentAttempt): Promise<void> + waitForCompletion(request: ManagedEnvironmentAttempt): Promise<ManagedEnvironmentCompletion> + }, +) { + const { request } = await operations.nextAttempt(0) + assert(request, 'Cloud picker request is unavailable') + for (const key of ['candidateSha', 'runId', 'remoteWorkspacePath', 'expectedNonce'] as const) assert.equal(request[key], fixture[key]) + assert.equal(request.scenario, scenario) + assert.equal(typeof request.createRequestId, 'string') + assert(request.createRequestId!.length > 0) + operations.authorize(request.createRequestId!) + await operations.publishAuthorization(request) + await operations.onAuthorized(request) +} export async function observeManagedEnvironmentNativeIdentity( owner: Pick<ManagedRuntimeBrowserRig, 'inventory' | 'runtime' | 'supervisor'>, diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index b0b6aaa8b..02dd883bd 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -13,7 +13,7 @@ import { managedEnvironmentOpencodeSelection } from '../../test/helpers/provider import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' -import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId } from './managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentScenario } from './managed-environment-create.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -340,31 +340,30 @@ export async function runManagedEnvironmentTarget() { browserIdentity = await processIdentity(browser) for (const scenario of scenarios) { const cwd = path.join(selected.workspace, scenario) - const request = await poll(`Cloud ${scenario} picker request`, async () => { + await serviceManagedEnvironmentScenario(fixture, scenario, { + nextAttempt: async () => ({ request: await poll(`Cloud ${scenario} picker request`, async () => { const request = await readJson(path.join(cwd, '.managed-fixture/request.json')) if (request) return request assert.equal(browser!.exitCode, null, 'Cloud browser exited before completing its three cases') assert.equal(browser!.signalCode, null, 'Cloud browser was signaled before completing its three cases') return undefined - }, 600_000) - for (const key of ['candidateSha', 'runId', 'remoteWorkspacePath', 'expectedNonce'] as const) assert.equal(request[key], fixture[key]) - assert.equal(request.scenario, scenario) - assert.equal(typeof request.createRequestId, 'string') - assert(request.createRequestId.length > 0) - authorize(request.createRequestId) - await writeJson(path.join(cwd, '.managed-fixture/authorized.json'), request) - if (scenario === 'healthy') { + }, 600_000) }), + authorize, + publishAuthorization: request => writeJson(path.join(cwd, '.managed-fixture/authorized.json'), request), + waitForCompletion: async () => { throw new Error('Attempt completion is not yet consumed') }, + onAuthorized: async () => { if (scenario === 'healthy') { const child = await poll('actual selected Bun child', async () => readJson(path.join(cwd, '.managed-fixture/child-receipt.json'))) assert.equal(child.expectedNonce, nonce); assert.equal(child.candidateSha, fixture.candidateSha); assert.equal(child.runId, fixture.runId) assert.equal(child.bun, true); assert.equal(child.opencodeVersion, '1.18.21'); assert.match(child.bunExecutable, /opencode/) assert.equal(child.executableMode, '0711') const view = await poll('created managed native identity', async () => (await rig!.inventory()).find(row => row.terminalId === child.terminalId && row.nativeSessionId === child.nativeSessionId && row.launchState === 'running')) - const stage = await stageProof(rig, view, nonce) + const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, nativeSessionId: view.nativeSessionId, stage, child } await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt - } + } }, + }) } const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud browser final join') assert.equal(browserExit.code, 0, 'Configured Cloud browser acceptance failed') diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 60fbb91b7..a9224fb72 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -35,9 +35,82 @@ import { RuntimeHarness, dockerSocketPath } from '../../../../scripts/testing/ru import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' -import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId } from '../../../../scripts/testing/managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentScenario, type ManagedEnvironmentAttempt, type ManagedEnvironmentCompletion } from '../../../../scripts/testing/managed-environment-create.js' import type { ErrorMessage } from '../../../../shared/ws-protocol.js' +describe('managed environment browser attempt lifetime', () => { + const fixture = { candidateSha: 'a'.repeat(40), runId: 'owned-run', remoteWorkspacePath: '/owned/workspace', expectedNonce: 'owned-nonce' } + const scenario = 'healthy' + const request = (attempt: number): ManagedEnvironmentAttempt => ({ ...fixture, scenario, attempt, createRequestId: `actual-ui-${attempt}` }) + const completion = (attempt: number, willRetry: boolean): ManagedEnvironmentCompletion => ({ ...request(attempt), status: willRetry ? 'failed' : 'passed', willRetry }) + + async function ownedFixture(body: (directory: string) => Promise<void>) { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-browser-attempt-')) + try { await body(directory) } finally { await fs.rm(directory, { recursive: true, force: true }) } + } + + it('authorizes each real retry request and waits for its matching completion', async () => { + await ownedFixture(async directory => { + for (const attempt of [0, 1]) { + await fs.writeFile(path.join(directory, `request-${attempt}.json`), JSON.stringify(request(attempt))) + await fs.writeFile(path.join(directory, `completion-${attempt}.json`), JSON.stringify(completion(attempt, attempt === 0))) + } + const authorized: string[] = [], published: ManagedEnvironmentAttempt[] = [], completed: number[] = [] + await serviceManagedEnvironmentScenario(fixture, scenario, { + nextAttempt: async attempt => ({ request: JSON.parse(await fs.readFile(path.join(directory, `request-${attempt}.json`), 'utf8')) }), + authorize: id => { authorized.push(id) }, + publishAuthorization: async value => { published.push(value) }, + onAuthorized: async () => {}, + waitForCompletion: async value => { completed.push(value.attempt); return JSON.parse(await fs.readFile(path.join(directory, `completion-${value.attempt}.json`), 'utf8')) }, + }) + expect.soft(authorized).toEqual(['actual-ui-0', 'actual-ui-1']) + expect.soft(published).toEqual([request(0), request(1)]) + expect(completed).toEqual([0, 1]) + }) + }) + + it('keeps the owned target alive until the browser reports attempt completion', async () => { + await ownedFixture(async directory => { + await fs.writeFile(path.join(directory, 'request.json'), JSON.stringify(request(0))) + let finish!: (value: ManagedEnvironmentCompletion) => void + const pending = new Promise<ManagedEnvironmentCompletion>(resolve => { finish = resolve }) + let entered!: () => void + const authorized = new Promise<void>(resolve => { entered = resolve }) + let returned = false + const run = serviceManagedEnvironmentScenario(fixture, scenario, { + nextAttempt: async () => ({ request: JSON.parse(await fs.readFile(path.join(directory, 'request.json'), 'utf8')) }), + authorize: () => {}, publishAuthorization: async () => {}, onAuthorized: async () => { entered() }, waitForCompletion: () => pending, + }).then(() => { returned = true }) + try { + await authorized + await new Promise<void>(resolve => setImmediate(resolve)) + expect.soft(returned).toBe(false) + } finally { finish(completion(0, false)); await run } + expect(returned).toBe(true) + }) + }) + + it('rejects a foreign request without granting or publishing authority', async () => { + const authorize = vi.fn(), publishAuthorization = vi.fn() + await expect(serviceManagedEnvironmentScenario(fixture, scenario, { + nextAttempt: async () => ({ request: { ...request(0), runId: 'foreign-run' } }), + authorize, publishAuthorization, onAuthorized: async () => {}, waitForCompletion: async () => completion(0, false), + })).rejects.toThrow() + expect(authorize).not.toHaveBeenCalled() + expect(publishAuthorization).not.toHaveBeenCalled() + }) + + it('accepts a failed-before-picker completion without inventing a create request', async () => { + const authorize = vi.fn(), publishAuthorization = vi.fn() + await expect(serviceManagedEnvironmentScenario(fixture, scenario, { + nextAttempt: async () => ({ completion: { ...fixture, scenario, attempt: 0, status: 'failed', willRetry: false } }), + authorize, publishAuthorization, onAuthorized: async () => {}, waitForCompletion: async () => { throw new Error('No request exists') }, + })).resolves.toBeUndefined() + expect(authorize).not.toHaveBeenCalled() + expect(publishAuthorization).not.toHaveBeenCalled() + }) +}) + describe('managed environment native identity observation', () => { it.each(['expected', 'foreign'] as const)('observes owned output before accepting %s native identity', async kind => { const row = { soulId: 'owned-soul', terminalId: 'owned-terminal', incarnationId: 'owned-incarnation', From afd670ef017b6f12eba8b20439c50d1954147641 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 20:06:20 -0700 Subject: [PATCH 287/493] fix: retain managed browser target through retry completion --- scripts/testing/managed-environment-create.ts | 41 +++++++++++++++---- .../managed-provider-environment-target.ts | 40 +++++++++++------- 2 files changed, 58 insertions(+), 23 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 24c82180c..5d2ea25a1 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -29,15 +29,38 @@ export async function serviceManagedEnvironmentScenario( waitForCompletion(request: ManagedEnvironmentAttempt): Promise<ManagedEnvironmentCompletion> }, ) { - const { request } = await operations.nextAttempt(0) - assert(request, 'Cloud picker request is unavailable') - for (const key of ['candidateSha', 'runId', 'remoteWorkspacePath', 'expectedNonce'] as const) assert.equal(request[key], fixture[key]) - assert.equal(request.scenario, scenario) - assert.equal(typeof request.createRequestId, 'string') - assert(request.createRequestId!.length > 0) - operations.authorize(request.createRequestId!) - await operations.publishAuthorization(request) - await operations.onAuthorized(request) + const matchAttempt = (value: ManagedEnvironmentAttempt, attempt: number) => { + for (const key of ['candidateSha', 'runId', 'remoteWorkspacePath', 'expectedNonce'] as const) assert.equal(value[key], fixture[key]) + assert.equal(value.scenario, scenario) + assert.equal(value.attempt, attempt) + } + // The existing Cloud project permits two retries. A completion keeps this + // fixture alive; it never supplies provider or terminal authority. + for (let attempt = 0; attempt <= 2; attempt++) { + const next = await operations.nextAttempt(attempt) + let completion = next.completion + if (next.request) { + const request = next.request + matchAttempt(request, attempt) + assert.equal(typeof request.createRequestId, 'string') + assert(request.createRequestId!.length > 0) + operations.authorize(request.createRequestId!) + await operations.publishAuthorization(request) + await operations.onAuthorized(request) + completion = await operations.waitForCompletion(request) + assert.equal(completion.createRequestId, request.createRequestId) + } else { + assert(completion, 'Cloud attempt has neither request nor completion') + assert.equal(completion.createRequestId, undefined, 'Pre-picker completion cannot invent a request') + assert.notEqual(completion.status, 'passed', 'A successful case must have its real request') + } + matchAttempt(completion, attempt) + assert(['passed', 'failed', 'timedOut', 'skipped', 'interrupted'].includes(completion.status)) + assert.equal(typeof completion.willRetry, 'boolean') + if (!completion.willRetry) return + assert.notEqual(completion.status, 'passed') + assert(attempt < 2, 'Cloud attempt exceeded the existing retry count') + } } export async function observeManagedEnvironmentNativeIdentity( diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 02dd883bd..9cca2d7ff 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -341,28 +341,40 @@ export async function runManagedEnvironmentTarget() { for (const scenario of scenarios) { const cwd = path.join(selected.workspace, scenario) await serviceManagedEnvironmentScenario(fixture, scenario, { - nextAttempt: async () => ({ request: await poll(`Cloud ${scenario} picker request`, async () => { - const request = await readJson(path.join(cwd, '.managed-fixture/request.json')) - if (request) return request - assert.equal(browser!.exitCode, null, 'Cloud browser exited before completing its three cases') - assert.equal(browser!.signalCode, null, 'Cloud browser was signaled before completing its three cases') - return undefined - }, 600_000) }), - authorize, - publishAuthorization: request => writeJson(path.join(cwd, '.managed-fixture/authorized.json'), request), - waitForCompletion: async () => { throw new Error('Attempt completion is not yet consumed') }, - onAuthorized: async () => { if (scenario === 'healthy') { - const child = await poll('actual selected Bun child', async () => readJson(path.join(cwd, '.managed-fixture/child-receipt.json'))) + nextAttempt: attempt => poll(`Cloud ${scenario} attempt ${attempt} picker or completion`, async () => { + const request = await readJson(path.join(cwd, '.managed-fixture/request.json')) + if (request?.attempt === attempt) return { request } + const completion = await readJson(path.join(cwd, '.managed-fixture/completion.json')) + if (completion?.attempt === attempt) return { completion } + assert.equal(browser!.exitCode, null, 'Cloud browser exited before completing its three cases') + assert.equal(browser!.signalCode, null, 'Cloud browser was signaled before completing its three cases') + return undefined + }, attempt === 0 ? 600_000 : scenario === 'healthy' ? 360_000 : 300_000), + authorize, + publishAuthorization: request => writeJson(path.join(cwd, '.managed-fixture/authorized.json'), request), + waitForCompletion: request => poll(`Cloud ${scenario} attempt ${request.attempt} completion`, async () => { + const completion = await readJson(path.join(cwd, '.managed-fixture/completion.json')) + if (completion?.attempt === request.attempt) return completion + assert.equal(browser!.exitCode, null, 'Cloud browser exited without its attempt completion') + return undefined + }, scenario === 'healthy' ? 360_000 : 300_000), + onAuthorized: async request => { if (scenario === 'healthy') { + const { child, view } = await poll('actual selected Bun child and current request', async () => { + const child = await readJson(path.join(cwd, '.managed-fixture/child-receipt.json')) + if (!child) return undefined + const view = (await rig!.inventory()).find(row => row.terminalId === child.terminalId + && row.terminalCreateRequestId === request.createRequestId && row.nativeSessionId === child.nativeSessionId && row.launchState === 'running') + return view ? { child, view } : undefined + }) assert.equal(child.expectedNonce, nonce); assert.equal(child.candidateSha, fixture.candidateSha); assert.equal(child.runId, fixture.runId) assert.equal(child.bun, true); assert.equal(child.opencodeVersion, '1.18.21'); assert.match(child.bunExecutable, /opencode/) assert.equal(child.executableMode, '0711') - const view = await poll('created managed native identity', async () => (await rig!.inventory()).find(row => row.terminalId === child.terminalId && row.nativeSessionId === child.nativeSessionId && row.launchState === 'running')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, nativeSessionId: view.nativeSessionId, stage, child } await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt - } }, + } }, }) } const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud browser final join') From 7e83008d01c9d4e097c7e77a0308cb4bfa4e781b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 20:13:56 -0700 Subject: [PATCH 288/493] test: retain managed browser attempt completion and failed requests --- ...aged-provider-environment-external.spec.ts | 105 ++++++++++++++++-- 1 file changed, 97 insertions(+), 8 deletions(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 855b7d6f7..f9c4db1f1 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -3,14 +3,53 @@ import { resolvePairedExternalFixture } from '../../../scripts/testing/e2e-exter import { test, expect } from '../helpers/fixtures.js' import { openPanePicker } from '../helpers/pane-picker.js' import { TestHarness } from '../helpers/test-harness.js' +import type { ProviderPreparationFailure } from '../../../shared/ws-protocol.js' const paired = resolvePairedExternalFixture(process.env, true) if (paired?.kind !== 'managed-environment') throw new Error('Managed environment requires its paired owned target') const fixture = paired.fixture const headers = { 'x-auth-token': paired.targetToken } +const healthyTitle = 'executes selected resources with Bun in both managed stages and retains identity on reconnect' +const caseScenarios: Record<string, string> = { + [healthyTitle]: 'healthy', + 'missing-reference retains an actionable blocked result without a replacement conversation': 'missing-reference', + 'startup-before-identity retains an actionable blocked result without a replacement conversation': 'startup-before-identity', +} + +test.afterEach(async ({ page }, info) => { + const scenario = caseScenarios[info.title] + if (!scenario) throw new Error('Unknown managed environment case') + const cwd = `${fixture.remoteWorkspacePath}/${scenario}` + const request = await readReceipt(page, `${cwd}/.managed-fixture/request.json`) + if (info.status !== info.expectedStatus) { + const pane = await activePane(page).catch(() => null) + const diagnostic = await page.evaluate(({ terminalId, marker }) => { + const harness = window.__FRESHELL_TEST_HARNESS__ + const buffer = terminalId && harness ? harness.getTerminalBuffer(terminalId) : null + return { harnessPresent: Boolean(harness), bufferType: typeof buffer, bufferNull: buffer === null, + bufferLength: typeof buffer === 'string' ? buffer.length : null, + markerPresent: typeof buffer === 'string' ? buffer.includes(marker) : false } + }, { terminalId: pane?.content.kind === 'terminal' ? pane.content.terminalId : undefined, + marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) + console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_attempt_failed', + scenario, attempt: info.retry, status: info.status, + stage: info.annotations.filter(value => value.type === 'managed-stage').at(-1)?.description ?? 'UNKNOWN', + terminalPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.terminalId), + soulPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.soulId), + nativeIdentityPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.sessionRef), + diagnostic })) + } + const response = await page.request.post(`${paired!.targetUrl}/api/files/write`, { headers, timeout: 30_000, + data: { path: `${cwd}/.managed-fixture/completion.json`, content: JSON.stringify({ ...fixture, scenario, + attempt: info.retry, status: info.status, willRetry: info.status !== info.expectedStatus && info.retry < info.project.retries, + ...(request?.attempt === info.retry ? { createRequestId: request.createRequestId } : {}) }) } }) + expect(response.ok()).toBe(true) +}) async function json(page: Page, route: string) { const response = await page.request.get(`${paired!.targetUrl}${route}`, { headers, timeout: 30_000 }) + if (!response.ok()) console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_http_failed', + route, status: response.status() })) expect(response.ok()).toBe(true) return response.json() } @@ -32,7 +71,7 @@ async function activePane(page: Page) { }) } -async function launch(page: Page, scenario: string) { +async function launch(page: Page, scenario: string, attempt: number) { await page.goto(`${paired!.targetUrl}/?token=${encodeURIComponent(paired!.targetToken)}&e2e=1`) await page.waitForFunction(() => Boolean(window.__FRESHELL_TEST_HARNESS__), undefined, { timeout: 60_000 }) await new TestHarness(page).waitForConnection(120_000) @@ -63,13 +102,13 @@ async function launch(page: Page, scenario: string) { expect(selected.paneId).toBe(before.paneId) expect(selected.content.mode).toBe('opencode') expect(selected.content.createRequestId).toBeTruthy() - const request = { ...fixture, scenario, createRequestId: selected.content.createRequestId, paneId: selected.paneId } + const request = { ...fixture, scenario, attempt, createRequestId: selected.content.createRequestId, paneId: selected.paneId } const written = await page.request.post(`${paired!.targetUrl}/api/files/write`, { headers, data: { path: `${cwd}/.managed-fixture/request.json`, content: JSON.stringify(request) } }) expect(written.ok()).toBe(true) await expect.poll(async () => readReceipt(page, `${cwd}/.managed-fixture/authorized.json`), { timeout: 120_000 }).toMatchObject({ candidateSha: fixture.candidateSha, runId: fixture.runId, - expectedNonce: fixture.expectedNonce, createRequestId: selected.content.createRequestId }) + expectedNonce: fixture.expectedNonce, attempt, createRequestId: selected.content.createRequestId }) await page.evaluate(id => { const harness = window.__FRESHELL_TEST_HARNESS__! harness.setTerminalNetworkEffectsSuppressed(id, false) @@ -81,17 +120,21 @@ async function launch(page: Page, scenario: string) { }, selected.paneId) await expect.poll(async () => { const pane = await activePane(page) - return pane.content.kind === 'terminal' ? pane.content.terminalId : null + return pane.content.kind === 'terminal' + ? scenario === 'missing-reference' ? pane.content.launchFailure?.providerPreparationFailure : pane.content.terminalId + : null }, { timeout: 120_000 }).toBeTruthy() return { ...(await activePane(page)), cwd, createRequestId: selected.content.createRequestId } } -test('executes selected resources with Bun in both managed stages and retains identity on reconnect', async ({ page }) => { +test(healthyTitle, async ({ page }, info) => { test.setTimeout(360_000) - const pane = await launch(page, 'healthy') + info.annotations.push({ type: 'managed-stage', description: 'healthy-launch' }) + const pane = await launch(page, 'healthy', info.retry) if (pane.content.kind !== 'terminal') throw new Error('Expected the real OpenCode terminal') const terminalId = pane.content.terminalId! const marker = `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` + info.annotations.push({ type: 'managed-stage', description: 'healthy-initial-output' }) await expect.poll(() => page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.getTerminalBuffer(id), terminalId), { timeout: 120_000 }).toContain(marker) let receipt: any @@ -112,6 +155,7 @@ test('executes selected resources with Bun in both managed stages and retains id await new TestHarness(page).waitForConnection(120_000) await expect.poll(async () => (await activePane(page)).content).toMatchObject({ terminalId, sessionRef: { provider: 'opencode', sessionId: receipt.nativeSessionId } }) + info.annotations.push({ type: 'managed-stage', description: 'healthy-post-reconnect-output' }) await expect.poll(() => page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.getTerminalBuffer(id), terminalId)).toContain(marker) expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length)).toBe(beforeCreates) @@ -121,12 +165,57 @@ test('executes selected resources with Bun in both managed stages and retains id }) for (const scenario of ['missing-reference', 'startup-before-identity']) { - test(`${scenario} retains an actionable blocked result without a replacement conversation`, async ({ page }) => { + test(`${scenario} retains an actionable blocked result without a replacement conversation`, async ({ page }, info) => { test.setTimeout(300_000) - const pane = await launch(page, scenario) + info.annotations.push({ type: 'managed-stage', description: `${scenario}-launch` }) + const pane = await launch(page, scenario, info.retry) if (pane.content.kind !== 'terminal') throw new Error('Expected the retained managed terminal') + if (scenario === 'missing-reference') { + const selectedPath = `${pane.cwd}/missing-selected.txt` + const preparationFailure = { code: 'SELECTED_REFERENCE_UNAVAILABLE', stage: 'configuration', operation: 'resolve', + cause: 'not_found', workspace: pane.cwd, declaringConfig: `${pane.cwd}/opencode.json`, selectedPath } satisfies ProviderPreparationFailure + const retained = { mode: 'opencode', status: 'error', createRequestId: pane.createRequestId, + launchFailure: { code: 'LAUNCH_FAILED', providerPreparationFailure: preparationFailure } } + expect(pane.content).toMatchObject(retained) + const assertNoHandle = async () => { + const current = (await activePane(page)).content + if (current.kind !== 'terminal') throw new Error('Expected the real failed request pane') + expect(current.terminalId).toBeUndefined() + expect(current.soulId).toBeUndefined() + expect(current.sessionRef).toBeUndefined() + expect(current.resumeSessionId).toBeUndefined() + expect((await json(page, '/api/terminals')).filter((item: any) => item.cwd === pane.cwd)).toHaveLength(0) + expect(await readReceipt(page, `${pane.cwd}/.managed-fixture/child-receipt.json`)).toBeNull() + } + const diagnostic = page.locator('[data-testid="terminal-work-area"]') + await expect(diagnostic).toContainText(selectedPath) + await expect(diagnostic).toContainText(/not found/i) + await expect(diagnostic.getByRole('button', { name: 'Retry launch' })).toBeVisible() + await expect(diagnostic.getByRole('button', { name: /^Attach/ })).toHaveCount(0) + await assertNoHandle() + const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length) + expect(creates).toBe(1) + await new TestHarness(page).forceDisconnect() + await new TestHarness(page).waitForConnection(120_000) + expect((await activePane(page)).content).toMatchObject(retained) + expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) + await assertNoHandle() + await page.reload() + await page.waitForFunction(() => Boolean(window.__FRESHELL_TEST_HARNESS__), undefined, { timeout: 60_000 }) + await new TestHarness(page).waitForConnection(120_000) + expect((await activePane(page)).content).toMatchObject(retained) + expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create').length)).toBe(0) + await expect(diagnostic).toContainText(selectedPath) + await expect(diagnostic).toContainText(/not found/i) + await assertNoHandle() + return + } const terminalId = pane.content.terminalId! let row: any + info.annotations.push({ type: 'managed-stage', description: 'startup-before-identity-blocked-directory' }) await expect.poll(async () => { row = (await json(page, '/api/terminals')).find((item: any) => item.terminalId === terminalId) return row From 1c269800261f89e0a9e44617c369d4aef36b111a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 20:18:32 -0700 Subject: [PATCH 289/493] test: bind failed-pane browser checks to the real create request --- ...naged-provider-environment-external.spec.ts | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index f9c4db1f1..9188571bc 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -177,6 +177,15 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { const retained = { mode: 'opencode', status: 'error', createRequestId: pane.createRequestId, launchFailure: { code: 'LAUNCH_FAILED', providerPreparationFailure: preparationFailure } } expect(pane.content).toMatchObject(retained) + expect(await page.evaluate(requestId => window.__FRESHELL_TEST_HARNESS__!.getReceivedWsMessages?.() + .filter((message: any) => message.type === 'error' && message.requestId === requestId) + .map((message: any) => ({ code: message.code, requestId: message.requestId, + providerPreparationFailure: message.providerPreparationFailure })), pane.createRequestId)).toContainEqual({ + code: 'PTY_SPAWN_FAILED', requestId: pane.createRequestId, providerPreparationFailure: preparationFailure, + }) + expect(await page.evaluate(requestId => window.__FRESHELL_TEST_HARNESS__!.getReceivedWsMessages?.() + .filter((message: any) => message.type === 'terminal.created' && message.requestId === requestId).length, + pane.createRequestId)).toBe(0) const assertNoHandle = async () => { const current = (await activePane(page)).content if (current.kind !== 'terminal') throw new Error('Expected the real failed request pane') @@ -195,7 +204,9 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { await assertNoHandle() const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length) - expect(creates).toBe(1) + expect(await page.evaluate(requestId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.requestId === requestId).length, + pane.createRequestId)).toBe(1) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) expect((await activePane(page)).content).toMatchObject(retained) @@ -206,8 +217,9 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { await page.waitForFunction(() => Boolean(window.__FRESHELL_TEST_HARNESS__), undefined, { timeout: 60_000 }) await new TestHarness(page).waitForConnection(120_000) expect((await activePane(page)).content).toMatchObject(retained) - expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length)).toBe(0) + expect(await page.evaluate(requestId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.requestId === requestId).length, + pane.createRequestId)).toBe(0) await expect(diagnostic).toContainText(selectedPath) await expect(diagnostic).toContainText(/not found/i) await assertNoHandle() From dbca947da0ae2e9708aa11bb8c1053ae6d1a260d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 21:31:25 -0700 Subject: [PATCH 290/493] test: protect concurrent managed browser scenario lifetimes --- scripts/testing/managed-environment-create.ts | 11 +++ .../managed-provider-environment-target.ts | 17 ++-- .../tooling/testing/runtime-sandbox.test.ts | 81 ++++++++++++++++++- 3 files changed, 100 insertions(+), 9 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 5d2ea25a1..819bb21d1 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -19,6 +19,17 @@ export type ManagedEnvironmentCompletion = ManagedEnvironmentAttempt & { willRetry: boolean } +export async function serviceManagedEnvironmentCases( + fixture: ManagedEnvironmentFixture, + operations: (scenario: 'healthy' | 'missing-reference' | 'startup-before-identity') => Parameters<typeof serviceManagedEnvironmentScenario>[2], + joinBrowser: () => Promise<void>, +) { + for (const scenario of ['healthy', 'missing-reference', 'startup-before-identity'] as const) { + await serviceManagedEnvironmentScenario(fixture, scenario, operations(scenario)) + } + await joinBrowser() +} + export async function serviceManagedEnvironmentScenario( fixture: ManagedEnvironmentFixture, scenario: string, operations: { diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 9cca2d7ff..17cef1ff7 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -13,7 +13,7 @@ import { managedEnvironmentOpencodeSelection } from '../../test/helpers/provider import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' -import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentScenario } from './managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentCases } from './managed-environment-create.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -338,9 +338,9 @@ export async function runManagedEnvironmentTarget() { 'test/e2e-browser/specs/managed-provider-environment-external.spec.ts'], { cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'inherit', 'inherit'] }) browserJoin = joined(browser); browserJoin.catch(() => {}) browserIdentity = await processIdentity(browser) - for (const scenario of scenarios) { + await serviceManagedEnvironmentCases(fixture, scenario => { const cwd = path.join(selected.workspace, scenario) - await serviceManagedEnvironmentScenario(fixture, scenario, { + return { nextAttempt: attempt => poll(`Cloud ${scenario} attempt ${attempt} picker or completion`, async () => { const request = await readJson(path.join(cwd, '.managed-fixture/request.json')) if (request?.attempt === attempt) return { request } @@ -375,11 +375,12 @@ export async function runManagedEnvironmentTarget() { await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt } }, - }) - } - const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud browser final join') - assert.equal(browserExit.code, 0, 'Configured Cloud browser acceptance failed') - proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, expectedCases: 3, printedFooter: 'UNKNOWN' } + } + }, async () => { + const browserExit = await bounded(browserJoin!, 120_000, 'configured Cloud browser final join') + assert.equal(browserExit.code, 0, 'Configured Cloud browser acceptance failed') + proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, expectedCases: 3, printedFooter: 'UNKNOWN' } + }) const healthy = proof.healthy as any const original = (await rig.inventory()).find(row => row.soulId === healthy.soulId) assert(original?.containerId) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index a9224fb72..bb8cf0434 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -35,7 +35,7 @@ import { RuntimeHarness, dockerSocketPath } from '../../../../scripts/testing/ru import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' -import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentScenario, type ManagedEnvironmentAttempt, type ManagedEnvironmentCompletion } from '../../../../scripts/testing/managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentScenario, serviceManagedEnvironmentCases, type ManagedEnvironmentAttempt, type ManagedEnvironmentCompletion } from '../../../../scripts/testing/managed-environment-create.js' import type { ErrorMessage } from '../../../../shared/ws-protocol.js' describe('managed environment browser attempt lifetime', () => { @@ -109,6 +109,85 @@ describe('managed environment browser attempt lifetime', () => { expect(authorize).not.toHaveBeenCalled() expect(publishAuthorization).not.toHaveBeenCalled() }) + + it('authorizes both concurrent negative cases while healthy completion remains pending', async () => { + await ownedFixture(async directory => { + const cases = ['healthy', 'missing-reference', 'startup-before-identity'] as const + for (const selected of cases) { + await fs.mkdir(path.join(directory, selected)) + await fs.writeFile(path.join(directory, selected, 'request.json'), JSON.stringify({ + ...fixture, scenario: selected, attempt: 0, createRequestId: `actual-${selected}`, + })) + } + let release!: () => void + const pending = new Promise<void>(resolve => { release = resolve }) + const authorized: string[] = [] + const run = serviceManagedEnvironmentCases(fixture, selected => ({ + nextAttempt: async () => ({ request: JSON.parse(await fs.readFile(path.join(directory, selected, 'request.json'), 'utf8')) }), + authorize: id => { authorized.push(id) }, + publishAuthorization: async value => { await fs.writeFile(path.join(directory, selected, 'authorized.json'), JSON.stringify(value)) }, + onAuthorized: async () => {}, + waitForCompletion: async value => { + if (selected === 'healthy') await pending + return { ...value, status: 'passed', willRetry: false } + }, + }), async () => {}) + try { + await vi.waitFor(() => expect(authorized).toContain('actual-startup-before-identity')) + expect(authorized).toContain('actual-missing-reference') + for (const selected of cases) expect(JSON.parse(await fs.readFile(path.join(directory, selected, 'authorized.json'), 'utf8'))) + .toMatchObject({ scenario: selected, createRequestId: `actual-${selected}`, ...fixture }) + } finally { release(); await run } + }) + }) + + it('settles every selected case after one service fails before the browser join and cleanup', async () => { + const selected: string[] = [], settled: string[] = [] + let release!: () => void + const pending = new Promise<void>(resolve => { release = resolve }) + const failure = new Error('actual owned scenario failure') + let returned = false + const joinBrowser = vi.fn(async () => { expect(settled).toEqual(['missing-reference', 'startup-before-identity']) }) + const run = serviceManagedEnvironmentCases(fixture, scenario => ({ + nextAttempt: async () => { + selected.push(scenario) + if (scenario === 'healthy') throw failure + await pending + settled.push(scenario) + return { completion: { ...fixture, scenario, attempt: 0, status: 'failed', willRetry: false } } + }, + authorize: vi.fn(), publishAuthorization: vi.fn(), onAuthorized: vi.fn(), waitForCompletion: vi.fn(), + }), joinBrowser).then(() => undefined, error => error).finally(() => { returned = true }) + try { + await new Promise<void>(resolve => setImmediate(resolve)) + expect.soft(selected).toEqual(['healthy', 'missing-reference', 'startup-before-identity']) + expect.soft(returned).toBe(false) + expect.soft(joinBrowser).not.toHaveBeenCalled() + } finally { release() } + expect.soft(await run).toBe(failure) + expect.soft(settled).toEqual(['missing-reference', 'startup-before-identity']) + expect(joinBrowser).toHaveBeenCalledOnce() + }) + + it('waits for the actual final browser join before propagating a service failure', async () => { + let release!: () => void + const pending = new Promise<void>(resolve => { release = resolve }) + let joining = false, returned = false + const failure = new Error('owned request mismatch') + const run = serviceManagedEnvironmentCases(fixture, scenario => ({ + nextAttempt: async () => { + if (scenario === 'healthy') throw failure + return { completion: { ...fixture, scenario, attempt: 0, status: 'failed', willRetry: false } } + }, + authorize: vi.fn(), publishAuthorization: vi.fn(), onAuthorized: vi.fn(), waitForCompletion: vi.fn(), + }), async () => { joining = true; await pending }).then(() => undefined, error => error).finally(() => { returned = true }) + try { + await new Promise<void>(resolve => setImmediate(resolve)) + expect.soft(joining).toBe(true) + expect.soft(returned).toBe(false) + } finally { release() } + expect(await run).toBe(failure) + }) }) describe('managed environment native identity observation', () => { From 5cfa3e495b761c210d3a2f7827c7aefe8eabeadb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 21:44:05 -0700 Subject: [PATCH 291/493] fix: await concurrent managed browser scenarios before cleanup --- scripts/testing/managed-environment-create.ts | 14 ++-- ...aged-provider-environment-external.spec.ts | 70 ++++++++++++++++--- 2 files changed, 72 insertions(+), 12 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 819bb21d1..1ebdaf5be 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -24,10 +24,16 @@ export async function serviceManagedEnvironmentCases( operations: (scenario: 'healthy' | 'missing-reference' | 'startup-before-identity') => Parameters<typeof serviceManagedEnvironmentScenario>[2], joinBrowser: () => Promise<void>, ) { - for (const scenario of ['healthy', 'missing-reference', 'startup-before-identity'] as const) { - await serviceManagedEnvironmentScenario(fixture, scenario, operations(scenario)) - } - await joinBrowser() + const cases = ['healthy', 'missing-reference', 'startup-before-identity'] as const + const results = await Promise.allSettled(cases.map(async scenario => + serviceManagedEnvironmentScenario(fixture, scenario, operations(scenario)))) + // A failed service cannot tear down the target while another case or the + // actual configured Cloud worker still owns its attempt. + let joinFailure: unknown + try { await joinBrowser() } catch (error) { joinFailure = error } + const failure = results.find((result): result is PromiseRejectedResult => result.status === 'rejected') + if (failure) throw failure.reason + if (joinFailure !== undefined) throw joinFailure } export async function serviceManagedEnvironmentScenario( diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 9188571bc..14706ffa1 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -16,6 +16,65 @@ const caseScenarios: Record<string, string> = { 'startup-before-identity retains an actionable blocked result without a replacement conversation': 'startup-before-identity', } +async function terminalDiagnostic(page: Page, terminalId?: string) { + return page.evaluate(({ terminalId, marker }) => { + const harness = window.__FRESHELL_TEST_HARNESS__ + const buffer = terminalId && harness ? harness.getTerminalBuffer(terminalId) : null + const received = (harness?.getReceivedWsMessages?.() ?? []) as any[] + const sent = (harness?.getSentWsMessages?.() ?? []) as any[] + const writes = (harness?.getTerminalWriteEvents?.() ?? []).filter(event => event.terminalId === terminalId) + const output = received.filter(message => message.terminalId === terminalId + && ['terminal.output', 'terminal.output.batch'].includes(message.type)) + const classify = (chunks: string[]) => { + let tail = '', markerPresent = false, reset = false, alternateEntered = false, alternateExited = false + for (const chunk of chunks) { + const current = tail + chunk + markerPresent ||= current.includes(marker) + reset ||= current.includes('\x1bc') || current.includes('\x1b[2J') + alternateEntered ||= /\x1b\[\?(?:47|1047|1049)h/.test(current) + alternateExited ||= /\x1b\[\?(?:47|1047|1049)l/.test(current) + tail = current.slice(-Math.max(marker.length - 1, 16)) + } + return { chunks: chunks.length, markerPresent, reset, alternateEntered, alternateExited } + } + const attachRequests = new Set(sent.filter(message => message.terminalId === terminalId + && message.type === 'terminal.attach').map(message => message.attachRequestId)) + const attachErrors = received.filter(message => message.type === 'error' + && (message.terminalId === terminalId || typeof message.requestId === 'string' && attachRequests.has(message.requestId))) + .slice(-16).map(message => ({ type: 'error', code: ['INVALID_REQUEST', 'TERMINAL_NOT_FOUND', 'PTY_SPAWN_FAILED', + 'SESSION_RESERVED', 'INTERNAL_ERROR', 'AUTH_FAILED'].includes(message.code) ? message.code : 'UNKNOWN' })) + const metadata = (messages: any[]) => messages.filter(message => message.terminalId === terminalId && [ + 'terminal.attach', 'terminal.replay.credit', 'terminal.attach.ready', 'terminal.output.gap', + 'terminal.stream.changed', 'terminal.modes.sync', 'terminal.turn.complete', + ].includes(message.type)).slice(-32).map(message => { + const value: Record<string, string | number | boolean> = { type: message.type } + for (const key of ['sinceSeq', 'requestedSinceSeq', 'effectiveSinceSeq', 'oldestRetainedSeq', 'headSeq', + 'replayFromSeq', 'replayToSeq', 'consumedSeq', 'fromSeq', 'toSeq', 'geometryEpoch']) { + if (Number.isSafeInteger(message[key]) && message[key] >= 0) value[key] = message[key] + } + for (const key of ['attachRequestId', 'streamId']) { + if (typeof message[key] === 'string' && /^[a-zA-Z0-9._:-]{1,512}$/.test(message[key])) value[key] = message[key] + } + if (typeof message.surfaceReset === 'boolean') value.surfaceReset = message.surfaceReset + if (['viewport_hydrate', 'transport_reconnect', 'keepalive_delta'].includes(message.intent)) value.intent = message.intent + if (['geometry_authority_unknown', 'retention_lost'].includes(message.replayResetReason)) value.replayResetReason = message.replayResetReason + if (['queue_overflow', 'replay_window_exceeded', 'replay_budget_exceeded', 'handoff_boundary_reached', + 'new_pty_session', 'codex_pty_recovery', 'retention_lost', 'server_restart_incompatible_retention'].includes(message.reason)) value.reason = message.reason + return value + }) + return { harnessPresent: Boolean(harness), bufferType: typeof buffer, bufferNull: buffer === null, + bufferLength: typeof buffer === 'string' ? buffer.length : null, + markerPresent: typeof buffer === 'string' ? buffer.includes(marker) : false, + xtermBufferType: terminalId && harness ? harness.getTerminalModes(terminalId)?.bufferType ?? 'UNKNOWN' : 'UNKNOWN', + receivedOutput: classify(output.map(message => message.data).filter(value => typeof value === 'string')), + submittedWrites: classify(writes.filter(event => event.phase === 'submitted').map(event => event.data)), + completedWrites: classify(writes.filter(event => event.phase === 'written').map(event => event.data)), + sentMetadata: metadata(sent), receivedMetadata: metadata(received), attachErrors, + // These getters retain bounded history, not a complete transport trace. + receivedHistoryAtBound: received.length >= 500, writeHistoryAtBound: writes.length >= 1000 } + }, { terminalId, marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) +} + test.afterEach(async ({ page }, info) => { const scenario = caseScenarios[info.title] if (!scenario) throw new Error('Unknown managed environment case') @@ -23,14 +82,7 @@ test.afterEach(async ({ page }, info) => { const request = await readReceipt(page, `${cwd}/.managed-fixture/request.json`) if (info.status !== info.expectedStatus) { const pane = await activePane(page).catch(() => null) - const diagnostic = await page.evaluate(({ terminalId, marker }) => { - const harness = window.__FRESHELL_TEST_HARNESS__ - const buffer = terminalId && harness ? harness.getTerminalBuffer(terminalId) : null - return { harnessPresent: Boolean(harness), bufferType: typeof buffer, bufferNull: buffer === null, - bufferLength: typeof buffer === 'string' ? buffer.length : null, - markerPresent: typeof buffer === 'string' ? buffer.includes(marker) : false } - }, { terminalId: pane?.content.kind === 'terminal' ? pane.content.terminalId : undefined, - marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) + const diagnostic = await terminalDiagnostic(page, pane?.content.kind === 'terminal' ? pane.content.terminalId : undefined) console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_attempt_failed', scenario, attempt: info.retry, status: info.status, stage: info.annotations.filter(value => value.type === 'managed-stage').at(-1)?.description ?? 'UNKNOWN', @@ -151,6 +203,8 @@ test(healthyTitle, async ({ page }, info) => { expect(receipt.nativeSessionId).toBeTruthy() const beforeCreates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length) + console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.browser_before_reconnect', + scenario: 'healthy', attempt: info.retry, diagnostic: await terminalDiagnostic(page, terminalId) })) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) await expect.poll(async () => (await activePane(page)).content).toMatchObject({ terminalId, From 3d6f3ef84e388c20c73078a0beb2758337bacfaa Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 22:55:23 -0700 Subject: [PATCH 292/493] Normalize managed OpenCode failure origins --- .../src/managed_provider_references.rs | 39 ++++++++++++++++++- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index d782d8331..21167c45a 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -171,7 +171,7 @@ pub(super) fn resolve_file_references_from_view_relocated( projection: ReferenceProjection<'_>, ) -> Result<ResolvedFileReferencePlan, ReferenceError> { use freshell_platform::opencode_config::{ - file_reference_path, inspect_file_tokens, TokenTransformError, + file_reference_path, inspect_file_tokens, lexical_path, TokenTransformError, }; let mut selections: Vec<FileReferenceSelection> = Vec::new(); let mut rewritten = Vec::new(); @@ -217,7 +217,9 @@ pub(super) fn resolve_file_references_from_view_relocated( operation: ProviderPreparationFailureOperation::Resolve, cause, workspace: origin.cwd.to_string_lossy().into_owned(), - declaring_config: origin.declaring_config.to_string_lossy().into_owned(), + declaring_config: lexical_path(origin.declaring_config) + .to_string_lossy() + .into_owned(), selected_path, })); failure @@ -1413,6 +1415,39 @@ process.stdout.write(JSON.stringify(receipts)); assert_eq!(failure.workspace, temp.path().to_string_lossy()); } + #[test] + fn managed_provider_references_missing_path_normalizes_declaring_config() { + let temp = tempfile::tempdir().unwrap(); + let source = temp.path().join("opencode.json"); + let selected = temp.path().join("missing-selected.txt"); + std::fs::write( + &source, + r#"{"agent":{"fixture":{"prompt":"{file:./missing-selected.txt}"}}}"#, + ) + .unwrap(); + let raw = std::fs::read_to_string(&source).unwrap(); + let declaring_config = PathBuf::from(format!("{}/./opencode.json", temp.path().display())); + let err = resolve_file_references( + &raw, + FileSourceOrigin { + declaring_config: &declaring_config, + cwd: temp.path(), + original_home: temp.path(), + }, + &|_| None, + ) + .err() + .expect("an explicit missing selection must block before dispatch"); + let launch_error: freshell_terminal::registry::ManagedTerminalLaunchError = err.into(); + let failure = launch_error + .provider_preparation_failure + .expect("the real selected-reference failure keeps safe typed metadata"); + + assert_eq!(failure.selected_path.as_deref(), selected.to_str()); + assert_eq!(failure.declaring_config, source.to_string_lossy()); + assert_eq!(failure.workspace, temp.path().to_string_lossy()); + } + #[test] fn managed_provider_references_missing_and_cyclic_targets_fail_without_echoing_candidate_values( ) { From c9ec095304414862ab72ddb2a23680ca5138fec7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 22:55:25 -0700 Subject: [PATCH 293/493] Make managed OpenCode failure fixtures deterministic --- scripts/testing/managed-provider-environment-target.ts | 6 ++++-- test/helpers/provider-parity-environment.ts | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 17cef1ff7..37ca31f12 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -131,14 +131,16 @@ async function configure(root: string, nonce: string, candidateSha: string, runI await fs.writeFile(command, `#!/bin/sh\nprintf '%s\\n' '${JSON.stringify({ resource: nonce })}'\n`, { mode: 0o751 }) await fs.chmod(command, 0o751) await fs.writeFile(path.join(plugin, 'package.json'), '{"type":"module","exports":{"./server":"./server.mjs","./tui":"./tui.mjs"}}') - await fs.writeFile(path.join(plugin, 'server.mjs'), `export default {id:'managed-environment-server',async server(){return {async config(){if(process.cwd().endsWith('/startup-before-identity'))throw new Error('startup-fixture failed before native identity')}}}};`) + await fs.writeFile(path.join(plugin, 'server.mjs'), `export default {id:'managed-environment-server',async server(){return {}}};`) // Exact pinned API: OpenCode v1.18.21 packages/plugin/src/tui.ts and specs/tui-plugins.md. // The SDK creates only an empty local session; no prompt or model call is made. + // Fail from the TUI initializer: OpenCode logs and ignores server config-hook errors. await fs.writeFile(path.join(plugin, 'tui.mjs'), `import fs from 'node:fs';import path from 'node:path';import dependency from 'managed-fixture-dependency'; export default {id:'managed-environment-tui',async tui(api,options){ + const cwd=process.cwd();if(cwd.endsWith('/startup-before-identity')){await new Promise(resolve=>process.stderr.write('startup-fixture failed before native identity\\n',resolve));process.exit(23)} const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); - const cwd=process.cwd();const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); + const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) diff --git a/test/helpers/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts index 425513f64..d632de59f 100644 --- a/test/helpers/provider-parity-environment.ts +++ b/test/helpers/provider-parity-environment.ts @@ -6,6 +6,8 @@ export function managedEnvironmentOpencodeSelection(): Record<string, string> { return { OPENCODE_DISABLE_PROJECT_CONFIG: '1', OPENCODE_CONFIG: './opencode.json', + // Keep OpenCode's update dialog from taking over the alternate screen. + OPENCODE_DISABLE_AUTOUPDATE: '1', } } From fb0cce04806d0ec7a02acc2d7f3b49b4a191fca0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Sun, 11 Oct 2026 23:35:18 -0700 Subject: [PATCH 294/493] test: build managed acceptance client from current commit --- .../managed-provider-environment-target.ts | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 37ca31f12..f9dd182ba 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1,5 +1,5 @@ import assert from 'node:assert/strict' -import { spawn, type ChildProcess } from 'node:child_process' +import { execFileSync, spawn, type ChildProcess } from 'node:child_process' import { createHash, randomUUID } from 'node:crypto' import fs from 'node:fs/promises' import os from 'node:os' @@ -221,11 +221,27 @@ async function stageProof(rig: ManagedRuntimeBrowserRig, view: ManagedRuntimeVie const requireMode = (metadata: { mode: number }) => (metadata.mode & 0o777).toString(8).padStart(4, '0') +export function buildManagedEnvironmentClient(repoRoot: string, candidateSha: string): void { + assert.match(candidateSha, /^[a-f0-9]{40}$/, 'Managed acceptance requires a known worktree commit') + execFileSync('pnpm', ['run', 'build:client'], { + cwd: repoRoot, + env: { ...process.env, FRESHELL_BUILD_COMMIT: candidateSha }, + stdio: 'inherit', + }) +} + export async function runManagedEnvironmentTarget() { assert.equal(process.env.FRESHELL_E2E_BACKEND, 'cloud', 'Managed acceptance requires configured Cloud') assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Managed acceptance requires the robot identity') assert.equal(process.env.GCLOUD_ROBOT_ACCOUNT, 'gcloud-robot@misc-puttering-project.iam.gserviceaccount.com') const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') + const candidateSha = execFileSync('git', ['rev-parse', 'HEAD'], { + cwd: repoRoot, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'inherit'], + timeout: 5_000, + }).trim() + buildManagedEnvironmentClient(repoRoot, candidateSha) const root = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-managed-environment-')) const grants = fakeOnecliGrants(['opencode']) const nonce = randomUUID() From 86abbacde039f9a1c095ca1e679c95fa87321489 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 00:25:27 -0700 Subject: [PATCH 295/493] test: anchor managed browser acceptance at cloud dispatch --- scripts/e2e-cloud.sh | 103 ++++++++++++++++-- scripts/test/cloud-exec-id-parse.test.sh | 31 +++++- scripts/testing/cloud-execution-dispatch.ts | 47 ++++++++ .../managed-provider-environment-target.ts | 7 ++ .../testing/cloud-execution-dispatch.test.ts | 43 ++++++++ 5 files changed, 217 insertions(+), 14 deletions(-) create mode 100644 scripts/testing/cloud-execution-dispatch.ts create mode 100644 test/unit/tooling/testing/cloud-execution-dispatch.test.ts diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index aac6b5e09..2104bb859 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -155,6 +155,79 @@ account_flag() { fi } +cloud_duration_seconds() { + local remaining="$1" total=0 amount unit numeric_amount + if [[ "$remaining" =~ ^[0-9]+$ ]]; then + printf '%s' "$remaining" + return 0 + fi + while [[ -n "$remaining" ]]; do + if [[ "$remaining" =~ ^([0-9]+)(s|m|h|d)(.*)$ ]]; then + amount="${BASH_REMATCH[1]}" + unit="${BASH_REMATCH[2]}" + remaining="${BASH_REMATCH[3]}" + numeric_amount=$((10#$amount)) + case "$unit" in + s) total=$((total + numeric_amount)) ;; + m) total=$((total + numeric_amount * 60)) ;; + h) total=$((total + numeric_amount * 3600)) ;; + d) total=$((total + numeric_amount * 86400)) ;; + esac + else + return 1 + fi + done + printf '%s' "$total" +} + +write_cloud_dispatch_receipt() { + local receipt_path="${FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT:-}" + [ -n "$receipt_path" ] || return 0 + if ! [[ "$RUN_JOB_NAME" =~ ^[a-z][a-z0-9-]{0,48}$ && "$execution_id" =~ ^[A-Za-z0-9][A-Za-z0-9-]*$ ]]; then + echo "[e2e-cloud] ERROR: cannot write a dispatch receipt for an invalid job or execution name." >&2 + return 1 + fi + local temporary="${receipt_path}.tmp.$$" + local accepted_at + accepted_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)" + if ! (umask 077; printf '{"jobName":"%s","executionName":"%s","acceptedAt":"%s"}\n' \ + "$RUN_JOB_NAME" "$execution_id" "$accepted_at" > "$temporary") \ + || ! mv -f "$temporary" "$receipt_path"; then + rm -f "$temporary" + echo "[e2e-cloud] ERROR: could not publish the accepted Cloud Run execution receipt." >&2 + return 1 + fi + return 0 +} + +wait_for_cloud_execution_completion() { + local task_timeout="$1" timeout_seconds completion_time deadline describe_failures=0 + if ! timeout_seconds=$(cloud_duration_seconds "$task_timeout") || [ "$timeout_seconds" -le 0 ]; then + echo "[e2e-cloud] ERROR: unsupported Cloud Run task timeout duration '$task_timeout'." >&2 + return 1 + fi + # Allow five minutes for Cloud Run's terminal status to settle after the + # configured per-task deadline. + deadline=$((SECONDS + timeout_seconds + 300)) + echo "[e2e-cloud] Waiting for execution $execution_id to complete (task deadline: $task_timeout)." + while (( SECONDS <= deadline )); do + if completion_time=$(gcloud run jobs executions describe $(gcloud_flags) "$execution_id" \ + --format="value(status.completionTime)" 2>/dev/null); then + describe_failures=0 + if [ -n "$completion_time" ] && [ "$completion_time" != "(unset)" ]; then return 0; fi + else + describe_failures=$((describe_failures + 1)) + if [ "$describe_failures" -ge 5 ]; then + echo "[e2e-cloud] ERROR: failed to query execution $execution_id status five consecutive times." >&2 + return 1 + fi + fi + sleep 3 + done + echo "[e2e-cloud] ERROR: execution $execution_id did not reach a terminal state within its task timeout and status-settlement margin." >&2 + return 1 +} + # kata e83z: cheap live-credential check at lane start — fails in seconds, # before any build/submit work, when the resolved identity cannot mint a token. # The error names the resolved identity and its source so the failure path is @@ -682,22 +755,22 @@ cmd_run() { --memory=4Gi \ --cpu=4 - # Execute this run's job and wait for completion, capturing the exit - # status: an execute failure (quota, permissions, template error) MAY NOT - # masquerade as a test outcome. + # Start this run's job and capture the accepted execution ID immediately. + # The external managed-environment target uses the dispatch receipt below + # to start its browser acceptance window after cold image build and job + # setup, while this wrapper still waits for the exact execution to finish. echo "[e2e-cloud] Executing Cloud Run Job..." local execute_output local execute_exit=0 local execution_id="" - execute_output=$(gcloud run jobs execute $(gcloud_flags) "$RUN_JOB_NAME" --wait 2>&1) || execute_exit=$? + execute_output=$(gcloud run jobs execute $(gcloud_flags) "$RUN_JOB_NAME" --async 2>&1) || execute_exit=$? echo "$execute_output" # Extract the execution ID from the execute output. gcloud prints - # `Execution [NAME] has successfully completed.` — brackets are literal and, - # on color-capable captures, the name is wrapped in ANSI SGR codes — so strip - # escapes and allow the bracket form. (A bare `Execution \K[^ ]+` captured - # the bracket+escapes; downstream describe/logs then addressed a nonexistent - # execution and the `|| echo 0` masking below reported succeeded=0 forever.) + # `Execution [NAME] has successfully started running.` — brackets are + # literal and may contain ANSI SGR codes — so strip escapes and allow the + # bracket form. A bare `Execution \K[^ ]+` captured the bracket+escapes, + # making downstream describe/logs address a nonexistent execution. execution_id=$(echo "$execute_output" \ | sed -E 's/\x1b\[[0-9;]*m//g' \ | grep -oP 'Execution \[?\K[A-Za-z0-9][A-Za-z0-9-]*' \ @@ -722,6 +795,18 @@ cmd_run() { exit 1 fi + if [ -z "$execution_id" ]; then + echo "[e2e-cloud] ERROR: Cloud Run accepted the execute request but returned no execution ID." >&2 + exit 1 + fi + if ! write_cloud_dispatch_receipt; then + exit 1 + fi + echo "[e2e-cloud] Cloud Run execution accepted: $execution_id" + if ! wait_for_cloud_execution_completion "$timeout"; then + exit 1 + fi + # Fetch logs (requires beta track for logs read). # Capture to a variable so we can print the full output AND extract a # per-shard summary, even when some shards fail. diff --git a/scripts/test/cloud-exec-id-parse.test.sh b/scripts/test/cloud-exec-id-parse.test.sh index 1146a937a..0f898452e 100755 --- a/scripts/test/cloud-exec-id-parse.test.sh +++ b/scripts/test/cloud-exec-id-parse.test.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# Test: cloud-exec-id-parse — regression for execution-ID capture from -# `gcloud run jobs execute --wait` output. +# Test: cloud-exec-id-parse — regression for execution-ID capture from both +# `gcloud run jobs execute --wait` and `--async` output. # # gcloud prints "Execution [NAME] has successfully completed." (brackets are # literal around the name, ANSI SGR codes around it on color-capable output). @@ -46,13 +46,24 @@ touch "$FAKE_LOG" cat > "$FAKE_DIR/gcloud" << 'FAKE' #!/usr/bin/env bash echo "FAKE_GCLOUD: $@" >> "$FAKE_LOG" -# Mimic current gcloud: "Execution [NAME] ..." with literal brackets and ANSI -# SGR codes around the name (observed ngcloud output 2026-08-18). +# Mimic gcloud's accepted execution output with literal brackets and ANSI +# SGR codes around the name (observed gcloud output 2026-08-18). if [[ "$*" == *"run jobs execute"* ]]; then - printf 'Creating execution...\nExecution [\033[1mtest-exec-123\033[m] has successfully completed.\n' + if [[ "$*" == *" --async"* ]]; then + printf 'Creating execution...\nExecution [\033[1mtest-exec-123\033[m] has successfully started running.\n' + else + printf 'Creating execution...\nExecution [\033[1mtest-exec-123\033[m] has successfully completed.\n' + fi exit 0 fi if [[ "$*" == *"executions describe"* ]]; then + if [[ "$*" == *"completionTime"* ]]; then + if [[ -s "${FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT:-/nonexistent}" ]]; then + echo "dispatch receipt available before completion wait" >> "$FAKE_LOG" + fi + echo "2026-10-12T00:00:00Z" + exit 0 + fi if [[ "$*" == *"succeededCount"* ]]; then # Report every requested shard as succeeded (vitest passes per-execution # --tasks=N; e2e creates the job with --tasks=N). The wrappers require @@ -92,13 +103,23 @@ check "vitest wrapper prints success footer" grep -q "All tasks completed succes check "vitest describe targets clean execution id" \ bash -c "grep -q 'executions describe .* test-exec-123 ' '$FAKE_LOG' && ! grep -q 'executions describe .*\[' '$FAKE_LOG'" +# Confirm the Vitest wrapper still waits in its existing execution path. +check "vitest wrapper retains wait-for-completion execution mode" \ + grep -q 'run jobs execute .* --wait' "$FAKE_LOG" + # --- e2e wrapper --- rm -f "$FAKE_LOG"; touch "$FAKE_LOG" +export FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT="$FAKE_DIR/e2e-dispatch.json" +rm -f "$FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT" E2E_OUT=$(bash "$ROOT/scripts/e2e-cloud.sh" run --cloud --shards=1 2>&1) && E2E_RC=0 || E2E_RC=$? check "e2e wrapper exits 0 on green run" bash -c "[ $E2E_RC -eq 0 ]" check "e2e wrapper reports truthfully (succeeded=1)" grep -q "Succeeded tasks: 1" <<< "$E2E_OUT" check "e2e describe targets clean execution id" \ bash -c "grep -q 'executions describe .* test-exec-123 ' '$FAKE_LOG' && ! grep -q 'executions describe .*\[' '$FAKE_LOG'" +check "e2e execution uses asynchronous acceptance before its bounded completion wait" \ + bash -c "grep -q 'run jobs execute .* --async' '$FAKE_LOG' && grep -q 'dispatch receipt available before completion wait' '$FAKE_LOG'" +check "e2e wrapper records the exact accepted Cloud Run execution" \ + bash -c "jq -e '.jobName and .executionName == \"test-exec-123\" and (.acceptedAt | test(\"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\"))' '$FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT' >/dev/null" echo "" if [ "$FAILURES" -eq 0 ]; then diff --git a/scripts/testing/cloud-execution-dispatch.ts b/scripts/testing/cloud-execution-dispatch.ts new file mode 100644 index 000000000..ae5ee701a --- /dev/null +++ b/scripts/testing/cloud-execution-dispatch.ts @@ -0,0 +1,47 @@ +import fs from 'node:fs/promises' + +export interface CloudExecutionDispatch { + jobName: string + executionName: string + acceptedAt: string +} + +export const CLOUD_BUILD_AND_DISPATCH_TIMEOUT_MS = 30 * 60_000 + +const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) + +async function readDispatchReceipt(receiptPath: string): Promise<CloudExecutionDispatch | undefined> { + let contents: string + try { contents = await fs.readFile(receiptPath, 'utf8') } + catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return undefined + throw error + } + const receipt: unknown = JSON.parse(contents) + if (!receipt || typeof receipt !== 'object' || Array.isArray(receipt)) { + throw new Error('Cloud Run dispatch receipt is not an object') + } + const { jobName, executionName, acceptedAt } = receipt as Record<string, unknown> + if (typeof jobName !== 'string' || !/^[a-z][a-z0-9-]{0,48}$/.test(jobName) + || typeof executionName !== 'string' || !/^[A-Za-z0-9][A-Za-z0-9-]*$/.test(executionName) + || typeof acceptedAt !== 'string' || !/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$/.test(acceptedAt)) { + throw new Error('Cloud Run dispatch receipt does not identify an accepted job execution') + } + return { jobName, executionName, acceptedAt } +} + +export async function waitForCloudExecutionDispatch( + receiptPath: string, + isBrowserRunning: () => boolean, + timeoutMs = CLOUD_BUILD_AND_DISPATCH_TIMEOUT_MS, + pollIntervalMs = 250, +): Promise<CloudExecutionDispatch> { + const deadline = Date.now() + timeoutMs + while (Date.now() < deadline) { + const receipt = await readDispatchReceipt(receiptPath) + if (receipt) return receipt + if (!isBrowserRunning()) throw new Error('Cloud browser exited before its Cloud Run execution was accepted') + await delay(Math.min(pollIntervalMs, Math.max(1, deadline - Date.now()))) + } + throw new Error('Managed environment timed out: Cloud image build and execution dispatch') +} diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index f9dd182ba..ee65f9c01 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -14,6 +14,7 @@ import { fakeOnecliGrants } from '../../test/integration/server/provider-parity- import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentCases } from './managed-environment-create.js' +import { waitForCloudExecutionDispatch } from './cloud-execution-dispatch.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -352,10 +353,16 @@ export async function runManagedEnvironmentTarget() { delete cloudEnvironment.FRESHELL_E2E_TARGET_HOME resolvePairedExternalFixture(cloudEnvironment, true) await writeJson(path.join(root, 'handoff.json'), { fixture, origin, token: rig.info.token, tunnelIdentity }) + const cloudDispatchReceiptPath = path.join(root, 'cloud-execution-dispatch.json') + cloudEnvironment.FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT = cloudDispatchReceiptPath browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--project=chromium-external-target', 'test/e2e-browser/specs/managed-provider-environment-external.spec.ts'], { cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'inherit', 'inherit'] }) browserJoin = joined(browser); browserJoin.catch(() => {}) browserIdentity = await processIdentity(browser) + const cloudDispatch = await waitForCloudExecutionDispatch(cloudDispatchReceiptPath, + () => browser?.exitCode === null && browser.signalCode === null) + proof.cloudExecution = cloudDispatch + console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.cloud_execution_dispatched', ...cloudDispatch })) await serviceManagedEnvironmentCases(fixture, scenario => { const cwd = path.join(selected.workspace, scenario) return { diff --git a/test/unit/tooling/testing/cloud-execution-dispatch.test.ts b/test/unit/tooling/testing/cloud-execution-dispatch.test.ts new file mode 100644 index 000000000..e0e284167 --- /dev/null +++ b/test/unit/tooling/testing/cloud-execution-dispatch.test.ts @@ -0,0 +1,43 @@ +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { describe, expect, it } from 'vitest' +import { waitForCloudExecutionDispatch } from '../../../../scripts/testing/cloud-execution-dispatch.js' + +describe('managed Cloud execution dispatch barrier', () => { + it('does not start acceptance until the exact Cloud Run execution is accepted', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-cloud-dispatch-')) + const receiptPath = path.join(directory, 'dispatch.json') + let acceptanceStarted = false + try { + const acceptance = waitForCloudExecutionDispatch(receiptPath, () => true, 2_000, 5).then(receipt => { + acceptanceStarted = true + return receipt + }) + + await new Promise(resolve => setTimeout(resolve, 30)) + expect(acceptanceStarted).toBe(false) + + await fs.writeFile(receiptPath, JSON.stringify({ jobName: 'freshell-e2e-candidate-abc123', + executionName: 'candidate-exec-123', acceptedAt: '2026-10-12T00:00:00Z' })) + await expect(acceptance).resolves.toEqual({ jobName: 'freshell-e2e-candidate-abc123', + executionName: 'candidate-exec-123', acceptedAt: '2026-10-12T00:00:00Z' }) + expect(acceptanceStarted).toBe(true) + } finally { + await fs.rm(directory, { recursive: true, force: true }) + } + }) + + it('uses a published dispatch receipt even if the browser process has already exited', async () => { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-cloud-dispatch-')) + const receiptPath = path.join(directory, 'dispatch.json') + const receipt = { jobName: 'freshell-e2e-candidate-abc123', executionName: 'candidate-exec-123', + acceptedAt: '2026-10-12T00:00:00Z' } + try { + await fs.writeFile(receiptPath, JSON.stringify(receipt)) + await expect(waitForCloudExecutionDispatch(receiptPath, () => false, 2_000, 5)).resolves.toEqual(receipt) + } finally { + await fs.rm(directory, { recursive: true, force: true }) + } + }) +}) From 5ea7a882bfc251acc6abc75bc923852c5c7a3f80 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 00:49:29 -0700 Subject: [PATCH 296/493] fix: validate Cloud task timeout before dispatch --- scripts/e2e-cloud-duration.mjs | 115 +++++++++++++++++++++++ scripts/e2e-cloud.sh | 41 +++----- scripts/test/cloud-exec-id-parse.test.sh | 45 ++++++++- 3 files changed, 171 insertions(+), 30 deletions(-) create mode 100644 scripts/e2e-cloud-duration.mjs diff --git a/scripts/e2e-cloud-duration.mjs b/scripts/e2e-cloud-duration.mjs new file mode 100644 index 000000000..e824394f9 --- /dev/null +++ b/scripts/e2e-cloud-duration.mjs @@ -0,0 +1,115 @@ +const secondsPerYear = 365.2422 * 24 * 60 * 60 +const maxWaitSeconds = Number.MAX_SAFE_INTEGER - 300 + +function parseCloudDurationSeconds(value) { + if (typeof value !== 'string' || value.length === 0) { + throw new Error('duration is required') + } + + const original = value.toUpperCase() + if (/^[+-]?\d+$/.test(original)) { + return validatedSeconds(Number(original)) + } + + let duration = original + let sign = 1 + if (duration.startsWith('-')) { + sign = -1 + duration = duration.slice(1) + } else if (duration.startsWith('+')) { + duration = duration.slice(1) + } + + let timeSeparatorSeen = false + let timePartImplied = false + if (duration.startsWith('P')) { + duration = duration.slice(1) + } else { + timePartImplied = true + } + + let amount = '' + let seconds = 0 + for (let index = 0; index < duration.length; index += 1) { + const character = duration[index] + if (/[0-9]/.test(character)) { + amount += character + continue + } + if (character === '.' || character === ',') { + amount += '.' + continue + } + if (character === 'T') { + if (timeSeparatorSeen) throw new Error('duration has more than one T separator') + timeSeparatorSeen = true + timePartImplied = true + continue + } + if (amount.length === 0) throw new Error('duration unit has no preceding number') + + const component = Number(amount) * sign + if (!Number.isFinite(component)) throw new Error('duration component is out of range') + amount = '' + + if (['M', 'U', 'N'].includes(character) + && index + 2 === duration.length + && duration[index + 1] === 'S') { + const divisor = character === 'M' ? 1_000 : character === 'U' ? 1_000_000 : 1_000_000_000 + seconds += component / divisor + index += 1 + continue + } + + switch (character) { + case 'Y': + seconds += component * secondsPerYear + break + case 'W': + seconds += component * 7 * 24 * 60 * 60 + break + case 'D': + seconds += component * 24 * 60 * 60 + break + case 'H': + seconds += component * 60 * 60 + timePartImplied = true + break + case 'M': + if (timePartImplied) { + seconds += component * 60 + } else { + seconds += component * (secondsPerYear / 12) + timePartImplied = true + } + break + case 'S': + seconds += component + timePartImplied = true + break + default: + throw new Error('duration contains an unknown unit') + } + } + + const unsignedOriginal = original.replace(/^[+-]/, '') + if (amount.length > 0 && unsignedOriginal !== 'P0') { + throw new Error('duration must end with a unit') + } + + return validatedSeconds(Math.round(seconds * 10_000) / 10_000) +} + +function validatedSeconds(seconds) { + if (!Number.isFinite(seconds) || seconds < 1 || seconds > maxWaitSeconds) { + throw new Error('duration is outside the supported wait range') + } + return Math.ceil(seconds) +} + +try { + if (process.argv.length !== 3) throw new Error('expected one duration argument') + process.stdout.write(`${parseCloudDurationSeconds(process.argv[2])}\n`) +} catch { + process.exitCode = 1 +} diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index 2104bb859..2d46fbb8c 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -156,28 +156,7 @@ account_flag() { } cloud_duration_seconds() { - local remaining="$1" total=0 amount unit numeric_amount - if [[ "$remaining" =~ ^[0-9]+$ ]]; then - printf '%s' "$remaining" - return 0 - fi - while [[ -n "$remaining" ]]; do - if [[ "$remaining" =~ ^([0-9]+)(s|m|h|d)(.*)$ ]]; then - amount="${BASH_REMATCH[1]}" - unit="${BASH_REMATCH[2]}" - remaining="${BASH_REMATCH[3]}" - numeric_amount=$((10#$amount)) - case "$unit" in - s) total=$((total + numeric_amount)) ;; - m) total=$((total + numeric_amount * 60)) ;; - h) total=$((total + numeric_amount * 3600)) ;; - d) total=$((total + numeric_amount * 86400)) ;; - esac - else - return 1 - fi - done - printf '%s' "$total" + node "$ROOT/scripts/e2e-cloud-duration.mjs" "$1" } write_cloud_dispatch_receipt() { @@ -201,11 +180,7 @@ write_cloud_dispatch_receipt() { } wait_for_cloud_execution_completion() { - local task_timeout="$1" timeout_seconds completion_time deadline describe_failures=0 - if ! timeout_seconds=$(cloud_duration_seconds "$task_timeout") || [ "$timeout_seconds" -le 0 ]; then - echo "[e2e-cloud] ERROR: unsupported Cloud Run task timeout duration '$task_timeout'." >&2 - return 1 - fi + local timeout_seconds="$1" task_timeout="$2" completion_time deadline describe_failures=0 # Allow five minutes for Cloud Run's terminal status to settle after the # configured per-task deadline. deadline=$((SECONDS + timeout_seconds + 300)) @@ -455,6 +430,7 @@ cmd_run() { local local_build_flag=false local shards=1 local timeout="60m" + local timeout_seconds local -a pw_args=() while [[ $# -gt 0 ]]; do @@ -600,6 +576,15 @@ cmd_run() { "${pw_args[@]}" fi + # Parse and bound the completion-wait budget before any Cloud API call or + # job creation. Cloud Run accepts decimal and ISO-8601 duration forms; a + # post-dispatch parse failure would otherwise leave an accepted execution + # without a valid local wait deadline. + if ! timeout_seconds=$(cloud_duration_seconds "$timeout"); then + echo "[e2e-cloud] ERROR: unsupported Cloud Run task timeout duration '$timeout'." >&2 + exit 1 + fi + # Identity ladder (run lane): resolve before the image describe / build / # job calls below. `run --local` never reaches here (it exec'd above), # so the local lane stays free of GCP tooling and of the ladder's @@ -803,7 +788,7 @@ cmd_run() { exit 1 fi echo "[e2e-cloud] Cloud Run execution accepted: $execution_id" - if ! wait_for_cloud_execution_completion "$timeout"; then + if ! wait_for_cloud_execution_completion "$timeout_seconds" "$timeout"; then exit 1 fi diff --git a/scripts/test/cloud-exec-id-parse.test.sh b/scripts/test/cloud-exec-id-parse.test.sh index 0f898452e..1fd5bc02c 100755 --- a/scripts/test/cloud-exec-id-parse.test.sh +++ b/scripts/test/cloud-exec-id-parse.test.sh @@ -47,10 +47,10 @@ cat > "$FAKE_DIR/gcloud" << 'FAKE' #!/usr/bin/env bash echo "FAKE_GCLOUD: $@" >> "$FAKE_LOG" # Mimic gcloud's accepted execution output with literal brackets and ANSI -# SGR codes around the name (observed gcloud output 2026-08-18). +# SGR codes around the name (gcloud 581.0.0). if [[ "$*" == *"run jobs execute"* ]]; then if [[ "$*" == *" --async"* ]]; then - printf 'Creating execution...\nExecution [\033[1mtest-exec-123\033[m] has successfully started running.\n' + printf 'Execution [\033[1mtest-exec-123\033[m] is being started asynchronously.\n' else printf 'Creating execution...\nExecution [\033[1mtest-exec-123\033[m] has successfully completed.\n' fi @@ -121,6 +121,47 @@ check "e2e execution uses asynchronous acceptance before its bounded completion check "e2e wrapper records the exact accepted Cloud Run execution" \ bash -c "jq -e '.jobName and .executionName == \"test-exec-123\" and (.acceptedAt | test(\"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\"))' '$FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT' >/dev/null" +check_duration() { + local input="$1" expected="$2" actual + actual=$(node "$ROOT/scripts/e2e-cloud-duration.mjs" "$input" 2>/dev/null) || return 1 + [[ "$actual" == "$expected" ]] +} + +check "cloud timeout parser accepts decimal hours" check_duration "1.5h" "5400" +check "cloud timeout parser rounds a fractional mixed duration up to whole seconds" \ + check_duration "1m30.5s" "91" +check "cloud timeout parser accepts ISO time durations" check_duration "PT1H" "3600" +check "cloud timeout parser accepts ISO day and time durations" check_duration "P1DT2H" "93600" +check "cloud timeout parser treats a bare integer as seconds" check_duration "60" "60" + +check_rejected_timeout_before_job() { + local duration="$1" output + : > "$FAKE_LOG" + if output=$(bash "$ROOT/scripts/e2e-cloud.sh" run --cloud --shards=1 "--timeout=$duration" 2>&1); then + return 1 + fi + ! grep -q 'FAKE_GCLOUD: run jobs create' "$FAKE_LOG" \ + && ! grep -q 'FAKE_GCLOUD: run jobs execute' "$FAKE_LOG" \ + && grep -q 'unsupported Cloud Run task timeout duration' <<< "$output" +} + +check "cloud timeout parser rejects malformed durations before creating a job" \ + check_rejected_timeout_before_job "1hbanana" +check "cloud timeout parser rejects zero seconds before creating a job" \ + check_rejected_timeout_before_job "0" +check "cloud timeout parser rejects zero ISO duration before creating a job" \ + check_rejected_timeout_before_job "PT0S" +check "cloud timeout parser rejects values below gcloud's one-second minimum before creating a job" \ + check_rejected_timeout_before_job "0.5s" +check "cloud timeout parser rejects durations beyond the safe wait bound before creating a job" \ + check_rejected_timeout_before_job "999999999999999999999999999999999999999s" + +: > "$FAKE_LOG" +VALID_DECIMAL_OUT=$(bash "$ROOT/scripts/e2e-cloud.sh" run --cloud --shards=1 --timeout=1.5h 2>&1) && VALID_DECIMAL_RC=0 || VALID_DECIMAL_RC=$? +check "e2e wrapper accepts a decimal Cloud Run timeout" bash -c "[ $VALID_DECIMAL_RC -eq 0 ]" +check "e2e wrapper passes the accepted decimal timeout to Cloud Run" \ + grep -q 'run jobs create .* --task-timeout=1.5h' "$FAKE_LOG" + echo "" if [ "$FAILURES" -eq 0 ]; then echo "=== All checks passed ===" From 58fa752b0a71e7565c6ba858667aee3a3ff1bf08 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 00:56:23 -0700 Subject: [PATCH 297/493] fix: match gcloud duration normalization --- scripts/e2e-cloud-duration.mjs | 140 +++++++++++++++++++---- scripts/test/cloud-exec-id-parse.test.sh | 21 ++++ 2 files changed, 139 insertions(+), 22 deletions(-) diff --git a/scripts/e2e-cloud-duration.mjs b/scripts/e2e-cloud-duration.mjs index e824394f9..ac4f3d0c0 100644 --- a/scripts/e2e-cloud-duration.mjs +++ b/scripts/e2e-cloud-duration.mjs @@ -1,16 +1,118 @@ -const secondsPerYear = 365.2422 * 24 * 60 * 60 +const secondsPerDay = 24 * 60 * 60 +const daysPerYear = 365.2422 +const secondsPerYear = daysPerYear * secondsPerDay +const secondsPerMonth = secondsPerYear / 12 const maxWaitSeconds = Number.MAX_SAFE_INTEGER - 300 +function roundHalfEven(value, places = 4) { + const scale = 10 ** places + const scaled = value * scale + if (!Number.isFinite(scaled)) return value + + const sign = Math.sign(scaled) + const absolute = Math.abs(scaled) + const whole = Math.floor(absolute) + const fraction = absolute - whole + let rounded + if (fraction < 0.5) { + rounded = whole + } else if (fraction > 0.5 || whole % 2 === 1) { + rounded = whole + 1 + } else { + rounded = whole + } + return sign * rounded / scale +} + +function percolate(value) { + const whole = Math.trunc(value) + const fraction = roundHalfEven(roundHalfEven(value) - whole) + return [whole, fraction] +} + +function normalizeGcloudDuration(parts) { + let [years, yearsFraction] = percolate(parts.years) + let days = parts.days + daysPerYear * yearsFraction + + let [months, monthsFraction] = percolate(parts.months) + days += Math.trunc(daysPerYear * monthsFraction / 12) + + let [wholeDays, daysFraction] = percolate(days) + let hours = parts.hours + 24 * daysFraction + let [wholeHours, hoursFraction] = percolate(hours) + let minutes = parts.minutes + 60 * hoursFraction + let [wholeMinutes, minutesFraction] = percolate(minutes) + let seconds = parts.seconds + 60 * minutesFraction + let [wholeSeconds, secondsFraction] = percolate(seconds) + let microseconds = Math.trunc(1_000_000 * secondsFraction) + + let totalSeconds = 0 + let carry = Math.trunc(microseconds / 1_000_000) + microseconds -= carry * 1_000_000 + totalSeconds += microseconds / 1_000_000 + wholeSeconds += carry + + carry = Math.trunc(wholeSeconds / 60) + wholeSeconds -= carry * 60 + totalSeconds += wholeSeconds + wholeMinutes += carry + + carry = Math.trunc(wholeMinutes / 60) + wholeMinutes -= carry * 60 + totalSeconds += wholeMinutes * 60 + wholeHours += carry + + if (wholeDays || months || years) { + carry = Math.trunc(wholeHours / 24) + wholeHours -= carry * 24 + totalSeconds += wholeHours * 60 * 60 + wholeDays += carry + + if (wholeDays >= 366) { + wholeDays -= 366 + years += 1 + } else if (wholeDays <= -366) { + wholeDays += 366 + years -= 1 + } + carry = Math.trunc(wholeDays / daysPerYear) + wholeDays -= Math.trunc(carry * daysPerYear) + totalSeconds += wholeDays * secondsPerDay + years += carry + + carry = Math.trunc(months / 12) + months -= carry * 12 + totalSeconds += months * secondsPerMonth + years += carry + totalSeconds += years * secondsPerYear + totalSeconds = roundHalfEven(totalSeconds, 0) + microseconds / 1_000_000 + } else { + totalSeconds += wholeHours * 60 * 60 + } + + return totalSeconds +} + +function validatedSeconds(seconds) { + if (!Number.isFinite(seconds) || seconds < 1 || seconds > maxWaitSeconds) { + throw new Error('duration is outside the supported wait range') + } + return Math.ceil(seconds) +} + function parseCloudDurationSeconds(value) { if (typeof value !== 'string' || value.length === 0) { throw new Error('duration is required') } - const original = value.toUpperCase() - if (/^[+-]?\d+$/.test(original)) { - return validatedSeconds(Number(original)) + // gcloud's Duration parser first tries Python int(), which accepts surrounding + // whitespace and underscores, and treats that value as seconds. + const bareInteger = value.trim() + if (/^[+-]?[0-9]+(?:_[0-9]+)*$/.test(bareInteger)) { + return validatedSeconds(Number(bareInteger.replaceAll('_', ''))) } + const original = value.toUpperCase() let duration = original let sign = 1 if (duration.startsWith('-')) { @@ -19,6 +121,7 @@ function parseCloudDurationSeconds(value) { } else if (duration.startsWith('+')) { duration = duration.slice(1) } + if (sign < 0) throw new Error('negative durations are below Cloud Run\'s minimum') let timeSeparatorSeen = false let timePartImplied = false @@ -28,8 +131,8 @@ function parseCloudDurationSeconds(value) { timePartImplied = true } + const parts = { years: 0, months: 0, days: 0, hours: 0, minutes: 0, seconds: 0 } let amount = '' - let seconds = 0 for (let index = 0; index < duration.length; index += 1) { const character = duration[index] if (/[0-9]/.test(character)) { @@ -48,7 +151,7 @@ function parseCloudDurationSeconds(value) { } if (amount.length === 0) throw new Error('duration unit has no preceding number') - const component = Number(amount) * sign + const component = Number(amount) if (!Number.isFinite(component)) throw new Error('duration component is out of range') amount = '' @@ -56,35 +159,35 @@ function parseCloudDurationSeconds(value) { && index + 2 === duration.length && duration[index + 1] === 'S') { const divisor = character === 'M' ? 1_000 : character === 'U' ? 1_000_000 : 1_000_000_000 - seconds += component / divisor + parts.seconds += component / divisor index += 1 continue } switch (character) { case 'Y': - seconds += component * secondsPerYear + parts.years += component break case 'W': - seconds += component * 7 * 24 * 60 * 60 + parts.days += component * 7 break case 'D': - seconds += component * 24 * 60 * 60 + parts.days += component break case 'H': - seconds += component * 60 * 60 + parts.hours += component timePartImplied = true break case 'M': if (timePartImplied) { - seconds += component * 60 + parts.minutes += component } else { - seconds += component * (secondsPerYear / 12) + parts.months += component timePartImplied = true } break case 'S': - seconds += component + parts.seconds += component timePartImplied = true break default: @@ -97,14 +200,7 @@ function parseCloudDurationSeconds(value) { throw new Error('duration must end with a unit') } - return validatedSeconds(Math.round(seconds * 10_000) / 10_000) -} - -function validatedSeconds(seconds) { - if (!Number.isFinite(seconds) || seconds < 1 || seconds > maxWaitSeconds) { - throw new Error('duration is outside the supported wait range') - } - return Math.ceil(seconds) + return validatedSeconds(normalizeGcloudDuration(parts)) } try { diff --git a/scripts/test/cloud-exec-id-parse.test.sh b/scripts/test/cloud-exec-id-parse.test.sh index 1fd5bc02c..f5669bcf6 100755 --- a/scripts/test/cloud-exec-id-parse.test.sh +++ b/scripts/test/cloud-exec-id-parse.test.sh @@ -127,12 +127,31 @@ check_duration() { [[ "$actual" == "$expected" ]] } +check_duration_rejected() { + local input="$1" + ! node "$ROOT/scripts/e2e-cloud-duration.mjs" "$input" >/dev/null 2>&1 +} + check "cloud timeout parser accepts decimal hours" check_duration "1.5h" "5400" check "cloud timeout parser rounds a fractional mixed duration up to whole seconds" \ check_duration "1m30.5s" "91" +check "cloud timeout parser preserves gcloud's rounded one-second minimum" \ + check_duration "PT0.99995S" "1" check "cloud timeout parser accepts ISO time durations" check_duration "PT1H" "3600" check "cloud timeout parser accepts ISO day and time durations" check_duration "P1DT2H" "93600" +check "cloud timeout parser matches gcloud fractional-year normalization" \ + check_duration "P0.00016Y" "6308" +check "cloud timeout parser matches gcloud fractional-month normalization" \ + check_duration "P0.1M" "259200" +check "cloud timeout parser rounds gcloud fractional-day normalization up safely" \ + check_duration "P0.00016D" "18" +check "cloud timeout parser rounds gcloud fractional-week normalization up safely" \ + check_duration "P0.00016W" "96" check "cloud timeout parser treats a bare integer as seconds" check_duration "60" "60" +check "cloud timeout parser accepts whitespace around a bare integer" \ + check_duration " 60 " "60" +check "cloud timeout parser rejects calendar fractions that normalize below gcloud's minimum" \ + check_duration_rejected "P0.001M" check_rejected_timeout_before_job() { local duration="$1" output @@ -155,6 +174,8 @@ check "cloud timeout parser rejects values below gcloud's one-second minimum bef check_rejected_timeout_before_job "0.5s" check "cloud timeout parser rejects durations beyond the safe wait bound before creating a job" \ check_rejected_timeout_before_job "999999999999999999999999999999999999999s" +check "cloud timeout parser rejects calendar fractions below gcloud's minimum before creating a job" \ + check_rejected_timeout_before_job "P0.001M" : > "$FAKE_LOG" VALID_DECIMAL_OUT=$(bash "$ROOT/scripts/e2e-cloud.sh" run --cloud --shards=1 --timeout=1.5h 2>&1) && VALID_DECIMAL_RC=0 || VALID_DECIMAL_RC=$? From 9a993726b3b1ac3ba9d49ba3ef60d4fc114552bb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 01:00:58 -0700 Subject: [PATCH 298/493] fix: match gcloud binary rounding --- scripts/e2e-cloud-duration.mjs | 53 +++++++++++++++++------- scripts/test/cloud-exec-id-parse.test.sh | 14 +++++++ 2 files changed, 52 insertions(+), 15 deletions(-) diff --git a/scripts/e2e-cloud-duration.mjs b/scripts/e2e-cloud-duration.mjs index ac4f3d0c0..58a929c99 100644 --- a/scripts/e2e-cloud-duration.mjs +++ b/scripts/e2e-cloud-duration.mjs @@ -3,25 +3,48 @@ const daysPerYear = 365.2422 const secondsPerYear = daysPerYear * secondsPerDay const secondsPerMonth = secondsPerYear / 12 const maxWaitSeconds = Number.MAX_SAFE_INTEGER - 300 +const binary64Buffer = new ArrayBuffer(8) +const binary64View = new DataView(binary64Buffer) function roundHalfEven(value, places = 4) { - const scale = 10 ** places - const scaled = value * scale - if (!Number.isFinite(scaled)) return value - - const sign = Math.sign(scaled) - const absolute = Math.abs(scaled) - const whole = Math.floor(absolute) - const fraction = absolute - whole - let rounded - if (fraction < 0.5) { - rounded = whole - } else if (fraction > 0.5 || whole % 2 === 1) { - rounded = whole + 1 + if (!Number.isFinite(value) || value === 0) return value + + // Match Python round(value, places) from the exact binary64 input. Multiplying + // by 10 ** places first can erase whether a decimal half-way value lies just + // above or below the tie, which changes gcloud's component normalization. + const negative = value < 0 + binary64View.setFloat64(0, Math.abs(value), false) + const bits = binary64View.getBigUint64(0, false) + const exponentBits = Number((bits >> 52n) & 0x7ffn) + const fractionBits = bits & ((1n << 52n) - 1n) + let significand = fractionBits + let binaryExponent = -1074 + if (exponentBits !== 0) { + significand |= 1n << 52n + binaryExponent = exponentBits - 1023 - 52 + } + + let numerator = significand * (10n ** BigInt(places)) + let denominator = 1n + if (binaryExponent >= 0) { + numerator <<= BigInt(binaryExponent) } else { - rounded = whole + denominator <<= BigInt(-binaryExponent) + } + + let rounded = numerator / denominator + const remainder = numerator % denominator + const halfComparison = remainder * 2n - denominator + if (halfComparison > 0n || (halfComparison === 0n && rounded % 2n === 1n)) { + rounded += 1n + } + + let decimal = rounded.toString() + if (places > 0) { + decimal = decimal.padStart(places + 1, '0') + decimal = `${decimal.slice(0, -places)}.${decimal.slice(-places)}` } - return sign * rounded / scale + return Number(`${negative ? '-' : ''}${decimal}`) } function percolate(value) { diff --git a/scripts/test/cloud-exec-id-parse.test.sh b/scripts/test/cloud-exec-id-parse.test.sh index f5669bcf6..771d0d15f 100755 --- a/scripts/test/cloud-exec-id-parse.test.sh +++ b/scripts/test/cloud-exec-id-parse.test.sh @@ -141,6 +141,20 @@ check "cloud timeout parser accepts ISO time durations" check_duration "PT1H" "3 check "cloud timeout parser accepts ISO day and time durations" check_duration "P1DT2H" "93600" check "cloud timeout parser matches gcloud fractional-year normalization" \ check_duration "P0.00016Y" "6308" +check "cloud timeout parser preserves gcloud's lower-side year half-way rounding" \ + check_duration "P0.00005Y" "3154" +check "cloud timeout parser preserves gcloud's lower-adjacent year half-way rounding" \ + check_duration "P0.00015Y" "3154" +check "cloud timeout parser preserves gcloud's upper-side year half-way rounding" \ + check_duration "P0.00025Y" "9470" +check "cloud timeout parser preserves gcloud's upper-adjacent year half-way rounding" \ + check_duration "P0.00035Y" "9470" +check "cloud timeout parser preserves gcloud's lower-side hour half-way rounding" \ + check_duration "1.00015h" "3601" +check "cloud timeout parser preserves gcloud's upper-side hour half-way rounding" \ + check_duration "1.00025h" "3602" +check "cloud timeout parser preserves gcloud's upper-adjacent hour half-way rounding" \ + check_duration "1.00035h" "3602" check "cloud timeout parser matches gcloud fractional-month normalization" \ check_duration "P0.1M" "259200" check "cloud timeout parser rounds gcloud fractional-day normalization up safely" \ From 0d80d0ebe516da1bdfd9400687541c58793766db Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 01:14:31 -0700 Subject: [PATCH 299/493] test: record managed browser acceptance counts --- .../managed-environment-browser-summary.ts | 128 ++++++++++++++++++ .../managed-provider-environment-target.ts | 28 +++- ...anaged-environment-browser-summary.test.ts | 58 ++++++++ 3 files changed, 210 insertions(+), 4 deletions(-) create mode 100644 scripts/testing/managed-environment-browser-summary.ts create mode 100644 test/unit/tooling/testing/managed-environment-browser-summary.test.ts diff --git a/scripts/testing/managed-environment-browser-summary.ts b/scripts/testing/managed-environment-browser-summary.ts new file mode 100644 index 000000000..af1f24922 --- /dev/null +++ b/scripts/testing/managed-environment-browser-summary.ts @@ -0,0 +1,128 @@ +const summaryMarker = '[e2e-cloud] Per-shard summary:' +const summaryEndMarker = '[e2e-cloud] Succeeded tasks:' +const maxPendingLineCharacters = 16_384 +const maxSummaryLines = 8 +const ansiEscape = /\u001B\[[0-?]*[ -/]*[@-~]/g + +export type ManagedEnvironmentBrowserSummary = { + footer: string + executedCases: number + passedCases: number + failedCases: number + skippedCases: number + interruptedCases: number +} + +type Counts = Omit<ManagedEnvironmentBrowserSummary, 'footer' | 'executedCases'> + +function parseFooter(line: string): ManagedEnvironmentBrowserSummary | undefined { + const normalized = line.replace(ansiEscape, '').trim() + if (!/\b(?:failed|passed|skipped|interrupted)\b/.test(normalized)) return undefined + + const withoutDuration = normalized.replace(/\s+\([^()]*\)$/, '') + const tokens = [...withoutDuration.matchAll(/(\d+)\s+(failed|passed|skipped|interrupted)\b/g)] + if (tokens.length === 0) return undefined + const remainder = withoutDuration.replace(/\d+\s+(?:failed|passed|skipped|interrupted)\b/g, '').replace(/[\s,]/g, '') + if (remainder.length > 0) return undefined + + const counts: Counts = { passedCases: 0, failedCases: 0, skippedCases: 0, interruptedCases: 0 } + const seen = new Set<string>() + for (const token of tokens) { + const count = Number(token[1]) + const category = token[2] + if (!Number.isSafeInteger(count) || count < 0 || seen.has(category)) return undefined + seen.add(category) + counts[`${category}Cases` as keyof Counts] = count + } + + const executedCases = counts.passedCases + counts.failedCases + counts.skippedCases + counts.interruptedCases + if (!Number.isSafeInteger(executedCases) || executedCases < 1) return undefined + return { footer: normalized, executedCases, ...counts } +} + +/** + * Bounded streaming capture for the actual Playwright line-reporter footer + * printed by scripts/e2e-cloud.sh after it reads the accepted execution logs. + * It retains only the small per-shard summary section, never arbitrary test or + * environment output. + */ +export class ManagedEnvironmentBrowserSummaryCapture { + private pendingLine = '' + private inSummary = false + private finished = false + private skippingOverlongLine = false + private failure: string | undefined + private readonly summaries: ManagedEnvironmentBrowserSummary[] = [] + + write(chunk: string | Uint8Array): void { + if (this.finished) { + this.failure ??= 'Cloud browser output arrived after summary capture finished' + return + } + if (this.failure) return + + let remaining = typeof chunk === 'string' ? chunk : Buffer.from(chunk).toString('utf8') + while (remaining.length > 0) { + if (this.skippingOverlongLine) { + const newline = remaining.indexOf('\n') + if (newline < 0) return + remaining = remaining.slice(newline + 1) + this.skippingOverlongLine = false + } + + const newline = remaining.indexOf('\n') + const segmentLength = newline < 0 ? remaining.length : newline + if (this.pendingLine.length + segmentLength > maxPendingLineCharacters) { + this.pendingLine = '' + if (this.inSummary) { + this.failure = 'Cloud browser output contained an overlong Playwright summary line' + return + } + this.skippingOverlongLine = newline < 0 + } else { + const segment = newline < 0 ? remaining : remaining.slice(0, newline) + this.pendingLine += segment + if (newline < 0) return + this.consume(this.pendingLine.replace(/\r$/, '')) + this.pendingLine = '' + if (this.failure) return + } + if (newline < 0) return + remaining = remaining.slice(newline + 1) + } + } + + finish(): ManagedEnvironmentBrowserSummary { + if (this.finished) throw new Error('Cloud browser summary capture may only be finished once') + this.finished = true + if (this.failure) throw new Error(this.failure) + if (this.skippingOverlongLine && this.inSummary) { + throw new Error('Cloud browser output ended in an overlong Playwright summary line') + } + if (this.pendingLine.length > 0) this.consume(this.pendingLine.replace(/\r$/, '')) + this.pendingLine = '' + if (this.failure) throw new Error(this.failure) + if (this.summaries.length !== 1) { + throw new Error(`Expected exactly one Playwright shard summary, observed ${this.summaries.length}`) + } + return this.summaries[0]! + } + + private consume(rawLine: string): void { + const line = rawLine.replace(ansiEscape, '').trim() + if (line === summaryMarker) { + if (this.inSummary) this.failure = 'Cloud browser emitted a nested Playwright summary section' + this.inSummary = true + return + } + if (!this.inSummary) return + if (line.startsWith(summaryEndMarker)) { + this.inSummary = false + return + } + const parsed = parseFooter(rawLine) + if (!parsed) return + this.summaries.push(parsed) + if (this.summaries.length > maxSummaryLines) this.failure = 'Cloud browser emitted too many Playwright shard summaries' + } +} diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index ee65f9c01..04bfce8da 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -15,6 +15,7 @@ import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.j import { resolvePairedExternalFixture } from './e2e-external-target.js' import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentCases } from './managed-environment-create.js' import { waitForCloudExecutionDispatch } from './cloud-execution-dispatch.js' +import { ManagedEnvironmentBrowserSummaryCapture, type ManagedEnvironmentBrowserSummary } from './managed-environment-browser-summary.js' const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' @@ -58,6 +59,13 @@ function joined(child: ChildProcess): Promise<NativeExit> { }) } +function joinedAfterOutput(child: ChildProcess): Promise<NativeExit> { + return new Promise((resolve, reject) => { + child.once('error', reject) + child.once('close', (code, signal) => resolve({ code, signal })) + }) +} + async function bounded<T>(completion: Promise<T>, milliseconds: number, description: string): Promise<T> { let timer: ReturnType<typeof setTimeout> | undefined try { @@ -252,6 +260,7 @@ export async function runManagedEnvironmentTarget() { versionJoin: Promise<NativeExit> | undefined let tunnel: ChildProcess | undefined, tunnelIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, tunnelJoin: Promise<NativeExit> | undefined let browser: ChildProcess | undefined, browserIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, browserJoin: Promise<NativeExit> | undefined + const browserSummaryCapture = new ManagedEnvironmentBrowserSummaryCapture() const owners: InitialProviderVolumeAuthorization[] = [] let capturedVolumes: ReturnType<typeof captureProviderParityVolumes> = [] const proof: Record<string, unknown> = { expectedNonce: nonce } @@ -356,8 +365,11 @@ export async function runManagedEnvironmentTarget() { const cloudDispatchReceiptPath = path.join(root, 'cloud-execution-dispatch.json') cloudEnvironment.FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT = cloudDispatchReceiptPath browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--project=chromium-external-target', - 'test/e2e-browser/specs/managed-provider-environment-external.spec.ts'], { cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'inherit', 'inherit'] }) - browserJoin = joined(browser); browserJoin.catch(() => {}) + 'test/e2e-browser/specs/managed-provider-environment-external.spec.ts'], { cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'pipe', 'pipe'] }) + browser.stdout!.on('data', chunk => browserSummaryCapture.write(chunk)) + browser.stdout!.pipe(process.stdout) + browser.stderr!.pipe(process.stderr) + browserJoin = joinedAfterOutput(browser); browserJoin.catch(() => {}) browserIdentity = await processIdentity(browser) const cloudDispatch = await waitForCloudExecutionDispatch(cloudDispatchReceiptPath, () => browser?.exitCode === null && browser.signalCode === null) @@ -404,7 +416,13 @@ export async function runManagedEnvironmentTarget() { }, async () => { const browserExit = await bounded(browserJoin!, 120_000, 'configured Cloud browser final join') assert.equal(browserExit.code, 0, 'Configured Cloud browser acceptance failed') - proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, expectedCases: 3, printedFooter: 'UNKNOWN' } + const browserSummary: ManagedEnvironmentBrowserSummary = browserSummaryCapture.finish() + assert.equal(browserSummary.executedCases, scenarios.length, 'Configured Cloud ran an unexpected number of browser cases') + assert.equal(browserSummary.passedCases, scenarios.length, 'Every configured managed-environment browser case must pass') + assert.equal(browserSummary.failedCases, 0, 'Configured managed-environment browser cases failed') + assert.equal(browserSummary.skippedCases, 0, 'Configured managed-environment browser cases were skipped') + assert.equal(browserSummary.interruptedCases, 0, 'Configured managed-environment browser cases were interrupted') + proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, selectedScenarios: [...scenarios], ...browserSummary } }) const healthy = proof.healthy as any const original = (await rig.inventory()).find(row => row.soulId === healthy.soulId) @@ -426,8 +444,10 @@ export async function runManagedEnvironmentTarget() { proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, child: replacementChild } await writeJson(path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json'), proof) + const browserSummary = proof.cloudBrowser as ManagedEnvironmentBrowserSummary console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.acceptance_passed', candidateSha: fixture.candidateSha, - runId: fixture.runId, expectedBrowserCases: 3, receipt: path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json') })) + runId: fixture.runId, executedBrowserCases: browserSummary.executedCases, passedBrowserCases: browserSummary.passedCases, + receipt: path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json') })) } finally { const errors: Error[] = [] const cleanup: Record<string, unknown> = {} diff --git a/test/unit/tooling/testing/managed-environment-browser-summary.test.ts b/test/unit/tooling/testing/managed-environment-browser-summary.test.ts new file mode 100644 index 000000000..52a607feb --- /dev/null +++ b/test/unit/tooling/testing/managed-environment-browser-summary.test.ts @@ -0,0 +1,58 @@ +import { describe, expect, it } from 'vitest' +import { ManagedEnvironmentBrowserSummaryCapture } from '../../../../scripts/testing/managed-environment-browser-summary.js' + +describe('managed environment configured-browser summary', () => { + it('captures the actual one-shard Playwright footer across stream chunk boundaries', () => { + const capture = new ManagedEnvironmentBrowserSummaryCapture() + capture.write('[e2e-cloud] progress before the summary\n[e2e-cloud] Per-shard sum') + capture.write('mary:\n\u001b[32m 3 passed') + capture.write(' (42.1s)\u001b[0m\n') + capture.write('[e2e-cloud] Succeeded tasks: 1\n') + + expect(capture.finish()).toEqual({ + footer: '3 passed (42.1s)', + executedCases: 3, + passedCases: 3, + failedCases: 0, + skippedCases: 0, + interruptedCases: 0, + }) + }) + + it('reports failures and skips in a mixed Playwright footer', () => { + const capture = new ManagedEnvironmentBrowserSummaryCapture() + capture.write('[e2e-cloud] Per-shard summary:\n 1 failed, 1 skipped, 1 passed (42.1s)\n') + capture.write('[e2e-cloud] Succeeded tasks: 1\n') + + expect(capture.finish()).toEqual({ + footer: '1 failed, 1 skipped, 1 passed (42.1s)', + executedCases: 3, + passedCases: 1, + failedCases: 1, + skippedCases: 1, + interruptedCases: 0, + }) + }) + + it('rejects a missing or ambiguous per-shard footer instead of inventing a pass count', () => { + const missing = new ManagedEnvironmentBrowserSummaryCapture() + missing.write('[e2e-cloud] All tasks completed successfully.\n') + expect(() => missing.finish()).toThrow(/exactly one Playwright shard summary, observed 0/) + + const ambiguous = new ManagedEnvironmentBrowserSummaryCapture() + ambiguous.write('[e2e-cloud] Per-shard summary:\n 3 passed (42s)\n 3 passed (43s)\n') + ambiguous.write('[e2e-cloud] Succeeded tasks: 1\n') + expect(() => ambiguous.finish()).toThrow(/exactly one Playwright shard summary/) + }) + + it('ignores oversized unrelated log lines but rejects an oversized summary line', () => { + const unrelated = new ManagedEnvironmentBrowserSummaryCapture() + unrelated.write('x'.repeat(20_000)) + unrelated.write('\n[e2e-cloud] Per-shard summary:\n 3 passed (42s)\n[e2e-cloud] Succeeded tasks: 1\n') + expect(unrelated.finish().passedCases).toBe(3) + + const oversizedSummary = new ManagedEnvironmentBrowserSummaryCapture() + oversizedSummary.write(`[e2e-cloud] Per-shard summary:\n${'x'.repeat(20_000)}\n`) + expect(() => oversizedSummary.finish()).toThrow(/overlong Playwright summary line/) + }) +}) From 83f0bd8f2c145ce5a32566a312d88e936ab98ba4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 01:43:38 -0700 Subject: [PATCH 300/493] fix: preserve checkpoint scope across resumed sessions --- src/components/HistoryView.tsx | 12 +++ .../context-menu/ContextMenuProvider.tsx | 30 +++++-- src/lib/resume-session.ts | 3 + src/lib/session-handoff.ts | 3 + .../components/ContextMenuProvider.test.tsx | 88 +++++++++++++++++++ .../components/HistoryView.mobile.test.tsx | 84 +++++++++++++----- test/unit/client/components/Sidebar.test.tsx | 17 +++- test/unit/client/lib/resume-session.test.ts | 19 ++++ test/unit/client/lib/session-handoff.test.ts | 7 +- 9 files changed, 232 insertions(+), 31 deletions(-) diff --git a/src/components/HistoryView.tsx b/src/components/HistoryView.tsx index 52d96689e..37d1b8204 100644 --- a/src/components/HistoryView.tsx +++ b/src/components/HistoryView.tsx @@ -6,6 +6,8 @@ import { removeSessionFromProjects, toggleProjectExpanded } from '@/store/sessio import { api } from '@/lib/api' import { activateSessionSurface, fetchSessionWindow, type FetchSessionWindowResult } from '@/store/sessionsThunks' import { openSessionTab } from '@/store/tabsSlice' +import { findPaneContent } from '@/lib/pane-utils' +import { findPaneForSession } from '@/lib/session-utils' import { applySessionRenameCascade } from '@/store/titleSync' import { receiveSessionNameProjections, receiveSessionNames } from '@/store/sessionNamesSlice' import { renameSessionName } from '@/lib/session-names' @@ -230,6 +232,15 @@ export default function HistoryView({ onOpenSession }: { onOpenSession?: () => v const label = getProviderLabel(session.provider) const mode = (session.provider || 'claude') as CodingCliProviderName const displayName = sessionDisplayName(session, sessionNames) + const state = store.getState() + const existing = findPaneForSession( + state, + { provider: mode, sessionId: session.sessionId }, + state.connection?.serverInstanceId, + ) + const layout = existing?.paneId ? state.panes.layouts[existing.tabId] : undefined + const content = layout && existing?.paneId ? findPaneContent(layout, existing.paneId) : null + const checkpointWorkspace = content?.kind === 'fresh-agent' ? content.checkpointWorkspace : undefined dispatch(openSessionTab({ sessionId: session.sessionId, title: displayName || label, @@ -240,6 +251,7 @@ export default function HistoryView({ onOpenSession }: { onOpenSession?: () => v isSubagent: session.isSubagent, isNonInteractive: session.isNonInteractive, hasTitle: !!displayName, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), })) onOpenSession?.() } diff --git a/src/components/context-menu/ContextMenuProvider.tsx b/src/components/context-menu/ContextMenuProvider.tsx index eeba190aa..7c3375523 100644 --- a/src/components/context-menu/ContextMenuProvider.tsx +++ b/src/components/context-menu/ContextMenuProvider.tsx @@ -27,8 +27,8 @@ import { buildShareUrl } from '@/lib/utils' import { copyText } from '@/lib/clipboard' import { openExternalUrl } from '@/lib/open-url' import { triggerHapticFeedback } from '@/lib/mobile-haptics' -import { collectPaneEntries, collectTerminalIds } from '@/lib/pane-utils' -import { collectSessionRefsFromNode } from '@/lib/session-utils' +import { collectPaneEntries, collectTerminalIds, findPaneContent } from '@/lib/pane-utils' +import { collectSessionRefsFromNode, findPaneForSession } from '@/lib/session-utils' import { getTabDisplayTitle } from '@/lib/tab-title' import { getBrowserActions, getEditorActions, getTerminalActions } from '@/lib/pane-action-registry' import { buildResumeCommand, type ResumeCommandProvider } from '@/lib/coding-cli-utils' @@ -416,6 +416,20 @@ export function ContextMenuProvider({ return null }, [getProjectCollections]) + const getExistingCheckpointWorkspace = useCallback((provider: CodingCliProviderName, sessionId: string) => { + const state = appStore.getState() + const existing = findPaneForSession( + state, + { provider, sessionId }, + state.connection?.serverInstanceId, + ) + if (!existing?.paneId) return undefined + const layout = state.panes.layouts[existing.tabId] + if (!layout) return undefined + const content = findPaneContent(layout, existing.paneId) + return content?.kind === 'fresh-agent' ? content.checkpointWorkspace : undefined + }, [appStore]) + const getProjectInfo = useCallback((projectPath: string, target?: ContextTarget | null) => { for (const projects of getProjectCollections(target)) { const project = projects.find((item) => item.projectPath === projectPath) @@ -454,6 +468,7 @@ export function ContextMenuProvider({ const mode = (provider || session.provider || 'claude') as CodingCliProviderName const sessionType = (target?.kind === 'sidebar-session' ? target.sessionType : undefined) || session.sessionType || mode + const checkpointWorkspace = getExistingCheckpointWorkspace(mode, session.sessionId) const runningTerminalId = target?.kind === 'sidebar-session' && target.sessionId === sessionId ? target.runningTerminalId @@ -470,8 +485,9 @@ export function ContextMenuProvider({ isSubagent: session.isSubagent, isNonInteractive: session.isNonInteractive, hasTitle: !!session.title, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), })) - }, [dispatch, getSessionInfo, menuState?.target]) + }, [dispatch, getExistingCheckpointWorkspace, getSessionInfo, menuState?.target]) const openSessionInThisTab = useCallback((sessionId: string, provider?: string) => { const activeTabId = tabsState.activeTabId @@ -486,6 +502,7 @@ export function ContextMenuProvider({ const mode = (provider || session.provider || 'claude') as CodingCliProviderName const sessionType = (target?.kind === 'sidebar-session' ? target.sessionType : undefined) || session.sessionType || mode + const checkpointWorkspace = getExistingCheckpointWorkspace(mode, session.sessionId) const freshAgentType = resolveFreshAgentType(sessionType) const freshAgentProviderConfig = getFreshAgentProviderConfig(sessionType) const freshAgentProviderSettings = freshAgentType || freshAgentProviderConfig @@ -497,11 +514,12 @@ export function ContextMenuProvider({ sessionType, sessionId: session.sessionId, cwd: session.cwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), freshAgentProviderSettings, }), })) persistSessionMetadataOnTab(activeTabId, session, sessionType) - }, [tabsState.activeTabId, dispatch, getSessionInfo, openSessionInNewTab, menuState?.target, appSettings, persistSessionMetadataOnTab]) + }, [tabsState.activeTabId, dispatch, getExistingCheckpointWorkspace, getSessionInfo, openSessionInNewTab, menuState?.target, appSettings, persistSessionMetadataOnTab]) const renameSession = useCallback(async (sessionId: string, provider?: string, withSummary?: boolean) => { const target = menuState?.target @@ -797,6 +815,7 @@ export function ContextMenuProvider({ if (project) { for (const session of project.sessions) { const provider = (session.provider || 'claude') as CodingCliProviderName + const checkpointWorkspace = getExistingCheckpointWorkspace(provider, session.sessionId) dispatch(openSessionTab({ sessionId: session.sessionId, title: session.title || session.sessionId.slice(0, 8), @@ -808,13 +827,14 @@ export function ContextMenuProvider({ isNonInteractive: session.isNonInteractive, forceNew: true, hasTitle: !!session.title, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), })) } } setConfirmState(null) }, }) - }, [dispatch, getProjectInfo, menuState?.target]) + }, [dispatch, getExistingCheckpointWorkspace, getProjectInfo, menuState?.target]) const copyProjectPath = useCallback(async (projectPath: string) => { await copyText(projectPath) diff --git a/src/lib/resume-session.ts b/src/lib/resume-session.ts index 218a6ef00..a54f0c95b 100644 --- a/src/lib/resume-session.ts +++ b/src/lib/resume-session.ts @@ -1,6 +1,7 @@ import { findPaneForSession } from '@/lib/session-utils' import { openSessionTab, setActiveTab } from '@/store/tabsSlice' import { setActivePane } from '@/store/panesSlice' +import type { FreshAgentCheckpointWorkspace } from '@/store/paneTypes' import type { AppDispatch, RootState } from '@/store/store' export interface ResumeTarget { @@ -10,6 +11,7 @@ export interface ResumeTarget { sessionType?: string title?: string firstUserMessage?: string + checkpointWorkspace?: FreshAgentCheckpointWorkspace } /** @@ -44,6 +46,7 @@ export function resumeSessionInTab( cwd: target.cwd, title: target.title, firstUserMessage: target.firstUserMessage, + ...(target.checkpointWorkspace ? { checkpointWorkspace: target.checkpointWorkspace } : {}), }), ) onNavigate?.('terminal') diff --git a/src/lib/session-handoff.ts b/src/lib/session-handoff.ts index e143e8b5b..6d9d84ed7 100644 --- a/src/lib/session-handoff.ts +++ b/src/lib/session-handoff.ts @@ -396,6 +396,9 @@ async function runPaneSessionHandoffInternal( sessionType: handoff.owner.sessionType, sessionId: committedSessionId, cwd: resolvedCwd, + ...(latest.content.kind === 'fresh-agent' && latest.content.checkpointWorkspace + ? { checkpointWorkspace: latest.content.checkpointWorkspace } + : {}), freshAgentProviderSettings: latest.providerSettings, }), })) diff --git a/test/unit/client/components/ContextMenuProvider.test.tsx b/test/unit/client/components/ContextMenuProvider.test.tsx index 26880fed2..87bd889af 100644 --- a/test/unit/client/components/ContextMenuProvider.test.tsx +++ b/test/unit/client/components/ContextMenuProvider.test.tsx @@ -559,6 +559,7 @@ function createStoreWithOverlappingSessionWindows() { { sessionId: VALID_SESSION_ID, provider: 'claude', + sessionType: 'freshclaude', title: 'History Terminal Session', cwd: '/shared/project/history', createdAt: 1000, @@ -2168,6 +2169,21 @@ describe('ContextMenuProvider', () => { it('uses the sidebar session window for sidebar actions and preserves fresh-agent session type', async () => { const user = userEvent.setup() const store = createStoreWithSidebarWindowAgentSession() + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + store.dispatch(updatePaneContent({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'req-existing-checkpoint-scope', + resumeSessionId: VALID_SESSION_ID, + sessionRef: { provider: 'claude', sessionId: VALID_SESSION_ID }, + status: 'idle', + checkpointWorkspace, + }, + })) render( <Provider store={store}> <ContextMenuProvider @@ -2204,6 +2220,7 @@ describe('ContextMenuProvider', () => { provider: 'claude', sessionType: 'freshclaude', resumeSessionId: VALID_SESSION_ID, + checkpointWorkspace, sessionRef: { provider: 'claude', sessionId: VALID_SESSION_ID, @@ -2452,9 +2469,73 @@ describe('ContextMenuProvider', () => { }) }) + it('preserves the matching pane checkpoint workspace when opening that session in a new tab', async () => { + const user = userEvent.setup() + const store = createStoreWithSidebarWindowAgentSession() + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + store.dispatch(updatePaneContent({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'req-checkpoint-scope', + resumeSessionId: VALID_SESSION_ID, + sessionRef: { provider: 'claude', sessionId: VALID_SESSION_ID }, + status: 'idle', + checkpointWorkspace, + }, + })) + render( + <Provider store={store}> + <ContextMenuProvider + view="terminal" + onViewChange={() => {}} + onToggleSidebar={() => {}} + sidebarCollapsed={false} + > + <div + data-context={ContextIds.SidebarSession} + data-session-id={VALID_SESSION_ID} + data-provider="claude" + data-session-type="freshclaude" + > + Sidebar Agent Session + </div> + </ContextMenuProvider> + </Provider> + ) + + await user.pointer({ target: screen.getByText('Sidebar Agent Session'), keys: '[MouseRight]' }) + await user.click(screen.getByText('Open in new tab')) + + const openedTab = store.getState().tabs.tabs.find((tab) => tab.id !== 'tab-1') + expect(openedTab).toBeTruthy() + expect(store.getState().panes.layouts[openedTab!.id]).toMatchObject({ + type: 'leaf', + content: { kind: 'fresh-agent', checkpointWorkspace }, + }) + }) + it('uses the history project window for history-project actions even when sidebar has a conflicting project snapshot', async () => { const user = userEvent.setup() const store = createStoreWithOverlappingSessionWindows() + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + store.dispatch(updatePaneContent({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'req-existing-checkpoint-scope', + resumeSessionId: VALID_SESSION_ID, + sessionRef: { provider: 'claude', sessionId: VALID_SESSION_ID }, + status: 'idle', + checkpointWorkspace, + }, + })) render( <Provider store={store}> <ContextMenuProvider @@ -2483,6 +2564,13 @@ describe('ContextMenuProvider', () => { 'History Terminal Session', 'History Extra Session', ]) + const resumedLayout = openedTabs + .map((tab) => store.getState().panes.layouts[tab.id]) + .find((layout) => layout?.type === 'leaf' && layout.content.kind === 'fresh-agent') + expect(resumedLayout).toMatchObject({ + type: 'leaf', + content: { kind: 'fresh-agent', checkpointWorkspace }, + }) }) it('copies resume command from sidebar session context menu', async () => { diff --git a/test/unit/client/components/HistoryView.mobile.test.tsx b/test/unit/client/components/HistoryView.mobile.test.tsx index 717be3574..c7b8d3694 100644 --- a/test/unit/client/components/HistoryView.mobile.test.tsx +++ b/test/unit/client/components/HistoryView.mobile.test.tsx @@ -5,8 +5,10 @@ import { configureStore } from '@reduxjs/toolkit' import HistoryView from '@/components/HistoryView' import sessionsReducer from '@/store/sessionsSlice' import sessionNamesReducer, { receiveSessionNames } from '@/store/sessionNamesSlice' -import tabsReducer from '@/store/tabsSlice' -import panesReducer from '@/store/panesSlice' +import tabsReducer, { addTab } from '@/store/tabsSlice' +import panesReducer, { initLayout } from '@/store/panesSlice' + +const HISTORY_SESSION_ID = '11111111-2222-4333-8444-555555555555' // Keep api helpers stubbed; spread the real module so pure named exports // (e.g. isApiUnauthorizedError, consumed by fetchSessionWindow's rejection @@ -33,8 +35,26 @@ vi.mock('@/lib/api', async () => { } }) -function renderHistoryView(onOpenSession = vi.fn()) { +function renderHistoryView(onOpenSession = vi.fn(), sessionType?: string, historyLoaded = false) { const projectPath = '/test/project' + const projects = [ + { + projectPath, + color: '#6b7280', + sessions: [ + { + provider: 'claude', + sessionId: HISTORY_SESSION_ID, + projectPath, + lastActivityAt: Date.now(), + title: 'Test Session', + summary: 'summary', + nameRef: { kind: 'session', provider: 'claude', sessionId: HISTORY_SESSION_ID }, + ...(sessionType ? { sessionType } : {}), + }, + ], + }, + ] const store = configureStore({ reducer: { sessions: sessionsReducer, @@ -50,23 +70,8 @@ function renderHistoryView(onOpenSession = vi.fn()) { }), preloadedState: { sessions: { - projects: [ - { - projectPath, - color: '#6b7280', - sessions: [ - { - provider: 'claude', - sessionId: 'session-123', - projectPath, - lastActivityAt: Date.now(), - title: 'Test Session', - summary: 'summary', - nameRef: { kind: 'session', provider: 'claude', sessionId: 'session-123' }, - }, - ], - }, - ], + projects, + ...(historyLoaded ? { windows: { history: { projects, lastLoadedAt: 1 } } } : {}), expandedProjects: new Set([projectPath]), }, tabs: { tabs: [], activeTabId: null }, @@ -112,6 +117,43 @@ describe('HistoryView mobile behavior', () => { expect(onOpenSession).toHaveBeenCalledTimes(1) }) + it('keeps the existing pane checkpoint workspace when history reopens that session with its current flavor', async () => { + const { store } = renderHistoryView(vi.fn(), 'freshclaude', true) + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + act(() => { + store.dispatch(addTab({ + id: 'tab-source', + title: 'Existing Kilroy', + status: 'running', + mode: 'claude', + sessionRef: { provider: 'claude', sessionId: HISTORY_SESSION_ID }, + })) + store.dispatch(initLayout({ + tabId: 'tab-source', + paneId: 'pane-source', + content: { + kind: 'fresh-agent', + sessionType: 'kilroy', + provider: 'claude', + createRequestId: 'req-source', + resumeSessionId: HISTORY_SESSION_ID, + sessionRef: { provider: 'claude', sessionId: HISTORY_SESSION_ID }, + status: 'idle', + checkpointWorkspace, + }, + })) + }) + + fireEvent.click(screen.getByRole('button', { name: /open session test session/i })) + + await waitFor(() => { + expect(store.getState().panes.layouts['tab-source']).toMatchObject({ + type: 'leaf', + content: { kind: 'fresh-agent', sessionType: 'freshclaude', checkpointWorkspace }, + }) + }) + }) + it('shows a nonblocking native-sync status row in the mobile session details sheet', async () => { ;(globalThis as any).setMobileForTest(true) const { store } = renderHistoryView() @@ -124,7 +166,7 @@ describe('HistoryView mobile behavior', () => { act(() => { store.dispatch(receiveSessionNames([{ record: { - ref: { kind: 'session', provider: 'claude', sessionId: 'session-123' }, + ref: { kind: 'session', provider: 'claude', sessionId: HISTORY_SESSION_ID }, name: 'Test Session', source: 'first_message', revision: 1, diff --git a/test/unit/client/components/Sidebar.test.tsx b/test/unit/client/components/Sidebar.test.tsx index a35d6abef..7f9e691e5 100644 --- a/test/unit/client/components/Sidebar.test.tsx +++ b/test/unit/client/components/Sidebar.test.tsx @@ -4279,8 +4279,9 @@ describe('Sidebar Component - Session-Centric Display', () => { } }) - it('focuses existing pane when clicking a session already open in another tab', async () => { + it('focuses the existing fresh-agent pane with its checkpoint workspace when clicking its session', async () => { const targetSessionId = sessionId('session-already-in-pane') + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } const projects: ProjectGroup[] = [ { @@ -4292,6 +4293,7 @@ describe('Sidebar Component - Session-Centric Display', () => { lastActivityAt: Date.now(), title: 'Already in pane', cwd: '/home/user/project', + sessionType: 'freshclaude', }, ], }, @@ -4318,11 +4320,14 @@ describe('Sidebar Component - Session-Centric Display', () => { type: 'leaf', id: 'pane-2', content: { - kind: 'terminal', - mode: 'claude', + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', createRequestId: 'req-2', - status: 'running', + status: 'idle', resumeSessionId: targetSessionId, + sessionRef: { provider: 'claude', sessionId: targetSessionId }, + checkpointWorkspace, }, }, }, @@ -4350,6 +4355,10 @@ describe('Sidebar Component - Session-Centric Display', () => { expect(state.tabs.tabs).toHaveLength(2) expect(state.tabs.activeTabId).toBe('tab-2') expect(state.panes.activePane['tab-2']).toBe('pane-2') + expect(state.panes.layouts['tab-2']).toMatchObject({ + type: 'leaf', + content: { kind: 'fresh-agent', checkpointWorkspace }, + }) }) it('does not hijack a foreign copied pane when opening the local session', async () => { diff --git a/test/unit/client/lib/resume-session.test.ts b/test/unit/client/lib/resume-session.test.ts index 3f4918624..d182edcb8 100644 --- a/test/unit/client/lib/resume-session.test.ts +++ b/test/unit/client/lib/resume-session.test.ts @@ -68,4 +68,23 @@ describe('resumeSessionInTab', () => { expect(dispatch).toHaveBeenCalledWith(openSessionTabAction) expect(onNavigate).toHaveBeenCalledWith('terminal') }) + + it('preserves an explicitly carried checkpoint workspace when opening a resumed tab', () => { + findPaneForSession.mockReturnValue(undefined) + const checkpointWorkspace = { path: '/projects/selected', provenance: 'directory-picker' as const } + const target = { + provider: 'codex', + sessionId: 'thread-123', + cwd: '/home/user', + checkpointWorkspace, + } + + resumeSessionInTab(state, dispatch, target, onNavigate) + + expect(openSessionTab).toHaveBeenCalledWith(expect.objectContaining({ + sessionId: 'thread-123', + cwd: '/home/user', + checkpointWorkspace, + })) + }) }) diff --git a/test/unit/client/lib/session-handoff.test.ts b/test/unit/client/lib/session-handoff.test.ts index d0a96b57d..1401b5a2c 100644 --- a/test/unit/client/lib/session-handoff.test.ts +++ b/test/unit/client/lib/session-handoff.test.ts @@ -35,6 +35,7 @@ function buildStore(): RootState['panes'] extends never ? never : ReturnType<typ function seedRekeyedPane( store: ReturnType<typeof buildStore>, status: 'idle' | 'creating' | 'starting' = 'idle', + checkpointWorkspace?: { path: string; provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' }, ) { store.dispatch(addTab({ id: 'tab-1' })) store.dispatch(initLayout({ @@ -47,6 +48,7 @@ function seedRekeyedPane( createRequestId: 'req-handoff', sessionRef: { provider: 'claude', sessionId: OLD_SESSION_ID }, status, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), }, })) // The rekey mirror pair: the OLD key's record carries the canonical @@ -118,7 +120,8 @@ describe('b8ke ext F1: the direct handoff navigates the rekey alias chain', () = it('records the server-answered canonical owner sessionId on the fresh-agent arm', async () => { const store = buildStore() - seedRekeyedPane(store) + const checkpointWorkspace = { path: '/projects/explicit', provenance: 'directory-picker' as const } + seedRekeyedPane(store, 'idle', checkpointWorkspace) requestSessionHandoffMock.mockResolvedValue({ ok: true, operationId: 'ho-2', @@ -141,6 +144,8 @@ describe('b8ke ext F1: the direct handoff navigates the rekey alias chain', () = expect(layout.content).toMatchObject({ kind: 'fresh-agent' }) expect((layout.content as { sessionRef?: { sessionId: string } }).sessionRef) .toEqual({ provider: 'claude', sessionId: NEW_SESSION_ID }) + expect((layout.content as { checkpointWorkspace?: typeof checkpointWorkspace }).checkpointWorkspace) + .toEqual(checkpointWorkspace) }) }) From ed626d39d59f2c8fd2e76aaa1a2798dd51f4fee9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 02:11:48 -0700 Subject: [PATCH 301/493] fix: persist fresh-agent checkpoint availability --- .../fresh-agent/FreshAgentTranscript.tsx | 36 +- .../fresh-agent/FreshAgentTurnActions.tsx | 32 +- src/components/fresh-agent/FreshAgentView.tsx | 402 +++++++++++++- ...-agent-checkpoint-unavailable-rust.spec.ts | 189 +++++++ .../fresh-agent/FreshAgentTranscript.test.tsx | 24 + .../FreshAgentTurnActions.test.tsx | 35 ++ .../fresh-agent/FreshAgentView.test.tsx | 501 ++++++++++++++++-- 7 files changed, 1132 insertions(+), 87 deletions(-) create mode 100644 test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts diff --git a/src/components/fresh-agent/FreshAgentTranscript.tsx b/src/components/fresh-agent/FreshAgentTranscript.tsx index cafcd6332..25375633a 100644 --- a/src/components/fresh-agent/FreshAgentTranscript.tsx +++ b/src/components/fresh-agent/FreshAgentTranscript.tsx @@ -19,6 +19,7 @@ import { buildTurnActionItems, FreshAgentTurnActions, turnPlainText, + type FreshAgentTurnRewindAvailability, } from './FreshAgentTurnActions' import { FreshAgentActionSheet } from './FreshAgentActionSheet' import { FreshAgentTranscriptMinimap } from './FreshAgentTranscriptMinimap' @@ -799,6 +800,7 @@ type TurnActionProps = { canFork: boolean onForkFromTurn?: (turnId: string) => void onRewindToTurn?: (turn: FreshAgentTurn) => void + getRewindAvailability?: (turn: FreshAgentTurn) => FreshAgentTurnRewindAvailability | undefined /** kata 1wxv: the per-turn "undo to here" affordance (decision 3). */ canRollback?: boolean rollbackBusy?: boolean @@ -878,6 +880,7 @@ function FreshAgentTurnArticle({ // coarse→fine pointer flip stops advertising/handling long-press even though // the instance persists for the article's lifetime. const longPress = actions.onOpenActions ? longPressRef.current : null + const rewindAvailability = actions.getRewindAvailability?.(actionTurn) return ( <article className={cn( @@ -924,8 +927,31 @@ function FreshAgentTurnArticle({ onForkFromTurn={actions.onForkFromTurn} onRollbackToTurn={actions.onRollbackToTurn} onRewindToTurn={actions.onRewindToTurn} + getRewindAvailability={actions.getRewindAvailability} onOpenActions={actions.onOpenActions} /> + {rewindAvailability ? ( + <div + role={rewindAvailability.status === 'unavailable' ? 'note' : 'status'} + aria-label={rewindAvailability.status === 'unavailable' + ? 'Code rewind unavailable for this message' + : rewindAvailability.status === 'pending' + ? 'Saving code rewind checkpoint for this message' + : 'Code rewind available for this message'} + className={cn( + 'px-1 pb-1 text-xs', + rewindAvailability.status === 'unavailable' + ? 'text-amber-700 dark:text-amber-400' + : 'text-muted-foreground', + )} + > + {rewindAvailability.status === 'unavailable' + ? `Code rewind unavailable for this message: ${rewindAvailability.reason}` + : rewindAvailability.status === 'pending' + ? 'Saving the code state from before this message.' + : 'Code rewind is available for this message.'} + </div> + ) : null} {showHeader ? ( <div className="fresh-agent-turn-header mb-1 flex items-center justify-between gap-2 text-xs text-muted-foreground"> <span>{turnLabel}</span> @@ -1022,6 +1048,7 @@ export type FreshAgentTranscriptProps = { isStreaming?: boolean onForkFromTurn?: (turnId: string) => void onRewindToTurn?: (turn: FreshAgentTurn) => void + getRewindAvailability?: (turn: FreshAgentTurn) => FreshAgentTurnRewindAvailability | undefined /** kata 1wxv: rollback affordances on live turns feed TurnActionProps; the * marker bucket + its per-row redo feed the rolled-back section (decision 6). */ canRollback?: boolean @@ -1059,6 +1086,7 @@ export const FreshAgentTranscript = forwardRef<FreshAgentTranscriptHandle, Fresh isStreaming = false, onForkFromTurn, onRewindToTurn, + getRewindAvailability, canRollback = false, rollbackBusy = false, onRollbackToTurn, @@ -1168,11 +1196,12 @@ export const FreshAgentTranscript = forwardRef<FreshAgentTranscriptHandle, Fresh canFork, onForkFromTurn, onRewindToTurn, + getRewindAvailability, canRollback, rollbackBusy, onRollbackToTurn, onOpenActions: coarsePointer ? handleOpenActions : undefined, - }), [canFork, canRollback, coarsePointer, handleOpenActions, onForkFromTurn, onRewindToTurn, onRollbackToTurn, rollbackBusy]) + }), [canFork, canRollback, coarsePointer, getRewindAvailability, handleOpenActions, onForkFromTurn, onRewindToTurn, onRollbackToTurn, rollbackBusy]) // Desktop right-click surface: the global ContextMenuProvider owns every // fresh-agent contextmenu gesture and asks this pane's registered builder @@ -1193,9 +1222,10 @@ export const FreshAgentTranscript = forwardRef<FreshAgentTranscriptHandle, Fresh onForkFromTurn, onRollbackToTurn, onRewindToTurn, + getRewindAvailability, }) }) - }, [paneId, displayTurns, lineEndIndex, canFork, canRollback, rollbackBusy, onForkFromTurn, onRollbackToTurn, onRewindToTurn]) + }, [paneId, displayTurns, lineEndIndex, canFork, canRollback, rollbackBusy, onForkFromTurn, onRollbackToTurn, onRewindToTurn, getRewindAvailability]) useImperativeHandle(ref, () => ({ scrollByLine: (direction) => { @@ -1367,7 +1397,7 @@ export const FreshAgentTranscript = forwardRef<FreshAgentTranscriptHandle, Fresh {sheetTurn ? ( <FreshAgentActionSheet title={turnPlainText(sheetTurn).slice(0, 80) || getTurnLabel(sheetTurn, agentLabel)} - items={buildTurnActionItems(sheetTurn, { canFork, canRollback, rollbackBusy, onForkFromTurn, onRollbackToTurn, onRewindToTurn })} + items={buildTurnActionItems(sheetTurn, { canFork, canRollback, rollbackBusy, onForkFromTurn, onRollbackToTurn, onRewindToTurn, getRewindAvailability })} onClose={() => setSheetTurn(null)} /> ) : null} diff --git a/src/components/fresh-agent/FreshAgentTurnActions.tsx b/src/components/fresh-agent/FreshAgentTurnActions.tsx index 201d7304f..18b34e937 100644 --- a/src/components/fresh-agent/FreshAgentTurnActions.tsx +++ b/src/components/fresh-agent/FreshAgentTurnActions.tsx @@ -14,6 +14,11 @@ export function turnPlainText(turn: FreshAgentTurn): string { return text || stripSystemReminders(turn.summary ?? '') } +export type FreshAgentTurnRewindAvailability = + | { status: 'pending' } + | { status: 'available' } + | { status: 'unavailable'; reason: string } + export type TurnActionCallbacks = { canFork: boolean /** kata 1wxv: snapshot-stamped `capabilities.undo` — absent/false hides the icon. */ @@ -23,6 +28,7 @@ export type TurnActionCallbacks = { canRedo?: boolean onForkFromTurn?: (turnId: string) => void onRewindToTurn?: (turn: FreshAgentTurn) => void + getRewindAvailability?: (turn: FreshAgentTurn) => FreshAgentTurnRewindAvailability | undefined onRollbackToTurn?: (turnId: string) => void onRedoToTurn?: (turnId: string) => void } @@ -34,6 +40,9 @@ export type TurnActionCallbacks = { * surfaces never drift apart. */ export function buildTurnActionItems(turn: FreshAgentTurn, callbacks: TurnActionCallbacks): ActionSheetItem[] { + const rewindAvailability = callbacks.getRewindAvailability?.(turn) + const rewindPending = rewindAvailability?.status === 'pending' + const rewindUnavailable = rewindAvailability?.status === 'unavailable' return [ { label: 'Copy turn text', @@ -54,8 +63,10 @@ export function buildTurnActionItems(turn: FreshAgentTurn, callbacks: TurnAction run: () => callbacks.onRollbackToTurn?.(turn.turnId ?? turn.id), }, { - label: 'Rewind code to here', - disabled: callbacks.onRewindToTurn === undefined || turn.role !== 'user', + label: rewindUnavailable + ? `Rewind unavailable: ${rewindAvailability.reason}` + : rewindPending ? 'Saving code rewind checkpoint…' : 'Rewind code to here', + disabled: callbacks.onRewindToTurn === undefined || turn.role !== 'user' || rewindPending || rewindUnavailable, destructive: true, run: () => callbacks.onRewindToTurn?.(turn), }, @@ -80,6 +91,7 @@ export function FreshAgentTurnActions({ onForkFromTurn, onRollbackToTurn, onRewindToTurn, + getRewindAvailability, onOpenActions, }: TurnActionCallbacks & { turn: FreshAgentTurn @@ -87,6 +99,9 @@ export function FreshAgentTurnActions({ }) { const [copied, setCopied] = useState(false) const canRewind = onRewindToTurn !== undefined && turn.role === 'user' + const rewindAvailability = getRewindAvailability?.(turn) + const rewindPending = rewindAvailability?.status === 'pending' + const rewindUnavailable = rewindAvailability?.status === 'unavailable' const handleCopy = useCallback(async () => { const ok = await copyText(turnPlainText(turn)) @@ -139,9 +154,16 @@ export function FreshAgentTurnActions({ <button type="button" onClick={() => onRewindToTurn?.(turn)} - className="rounded p-1 text-muted-foreground transition-colors hover:bg-accent hover:text-accent-foreground" - aria-label="Rewind code to here" - title="Rewind code to the checkpoint taken when this message was sent" + disabled={rewindPending || rewindUnavailable} + className="rounded p-1 text-muted-foreground transition-colors hover:bg-accent hover:text-accent-foreground disabled:cursor-not-allowed disabled:opacity-40" + aria-label={rewindUnavailable + ? 'Rewind unavailable for this message' + : rewindPending ? 'Saving code rewind checkpoint' : 'Rewind code to here'} + title={rewindUnavailable + ? `Rewind unavailable: ${rewindAvailability.reason}` + : rewindPending + ? 'Saving the code state from before this message' + : 'Rewind code to the checkpoint taken when this message was sent'} > <History className="h-3 w-3" /> </button> diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 2a38c9b3e..581b358b9 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -9,7 +9,7 @@ import { type KeyboardEvent as ReactKeyboardEvent, } from 'react' import { nanoid } from 'nanoid' -import type { FreshAgentPaneContent } from '@/store/paneTypes' +import type { FreshAgentCheckpointOutcome, FreshAgentPaneContent } from '@/store/paneTypes' import type { PaneReconcileRequest } from '@shared/ws-protocol' import { isUnifiedAgentMode } from '@shared/session-names' import { useAppDispatch, useAppSelector, useAppStore } from '@/store/hooks' @@ -20,7 +20,7 @@ import { sendSuppressedAwareFreshAgentFrame } from '@/lib/fresh-agent-configure' import { KILL_ACK_TIMEOUT_MESSAGE, KILL_FAILED_MESSAGE, sendFreshAgentKillAndAwait, sendFreshAgentRecoveryStopAndAwait } from '@/lib/kill-ack' import { createLogger } from '@/lib/client-logger' import { getInstalledPerfAuditBridge } from '@/lib/perf-audit-bridge' -import { api, getFreshAgentModelCapabilities, getFreshAgentThreadSnapshot, setSessionMetadata } from '@/lib/api' +import { ApiError, api, getFreshAgentModelCapabilities, getFreshAgentThreadSnapshot, setSessionMetadata } from '@/lib/api' import { clearPaneCloseError, clearReconcilePendingPane, consumePaneRefreshRequest, mergePaneContent, startNewManagedRuntimeConversation, updatePaneContent } from '@/store/panesSlice' import { retryManagedConversation } from '@/lib/managed-runtime-retry' import { isManagedRuntimeRecoveryDecision, ManagedRuntimeRecoveryCard } from '@/components/ManagedRuntimeRecoveryCard' @@ -109,6 +109,7 @@ import { FreshAgentApprovalCard } from './FreshAgentApprovalCard' import { FreshAgentOpenSessionContext } from './FreshAgentItemCard' import FreshAgentQuestionBanner from './FreshAgentQuestionBanner' import { FreshAgentTranscript, type FreshAgentTranscriptHandle } from './FreshAgentTranscript' +import type { FreshAgentTurnRewindAvailability } from './FreshAgentTurnActions' import { FreshAgentComposer, type FreshAgentComposerHandle } from './FreshAgentComposer' import { FreshAgentDiffPanel } from './FreshAgentDiffPanel' import { FreshAgentSidebar } from './FreshAgentSidebar' @@ -231,7 +232,11 @@ function sameLocalEcho(a: LocalEcho | null | undefined, b: LocalEcho | null | un type PendingSendMetadata = { cwd?: string + checkpointAvailabilityScope?: string + checkpointWorkspace?: NonNullable<FreshAgentPaneContent['checkpointWorkspace']> checkpointId?: string + checkpointFailureReason?: string + checkpointFailureCode?: string submittedTurnId?: string legacyAccepted?: boolean metadataUpdateStarted?: boolean @@ -241,6 +246,125 @@ type PendingSendMetadata = { text?: string } +type CheckpointCreationEntry = CheckpointEntry & { + workspace?: NonNullable<FreshAgentPaneContent['checkpointWorkspace']> +} + +type CheckpointTurnAvailability = FreshAgentTurnRewindAvailability & { + workspace?: NonNullable<FreshAgentPaneContent['checkpointWorkspace']> + checkpointId?: string + code?: string +} + +const CHECKPOINT_CAPTURE_TIMEOUT_MS = 30_000 +const CHECKPOINT_CAPTURE_TIMEOUT_REASON = 'Code checkpoint capture timed out after 30 seconds.' +const CHECKPOINT_FAILURE_FALLBACK = 'The project workspace could not be saved before this message was sent.' + +function sanitizeCheckpointReason(value: string): string { + return value + .replace(/\u001B\[[0-?]*[ -/]*[@-~]/g, '') + .replace(/[\u0000-\u001f\u007f-\u009f]/g, ' ') + .replace(/\s+/g, ' ') + .trim() + .slice(0, 240) +} + +function isCheckpointWorkspace(value: unknown): value is NonNullable<FreshAgentPaneContent['checkpointWorkspace']> { + if (!value || typeof value !== 'object' || Array.isArray(value)) return false + const record = value as Record<string, unknown> + return typeof record.path === 'string' + && record.path.length > 0 + && (record.provenance === 'directory-picker' + || record.provenance === 'resolved-project' + || record.provenance === 'legacy-checkpoint') +} + +function checkpointFailureReason(error: unknown): string { + if (error && typeof error === 'object') { + const outer = error as { details?: unknown; code?: unknown } + const details = outer.details && typeof outer.details === 'object' + ? outer.details as { code?: unknown; details?: unknown } + : outer + if (typeof details.details === 'string') { + const reason = sanitizeCheckpointReason(details.details) + if (reason) return reason + } + } + if (error instanceof ApiError) { + const serverReason = sanitizeCheckpointReason(error.message) + if (serverReason) return serverReason + } + return CHECKPOINT_FAILURE_FALLBACK +} + +function checkpointFailureCode(error: unknown): string | undefined { + if (!error || typeof error !== 'object') return undefined + const outer = error as { details?: unknown; code?: unknown } + const details = outer.details && typeof outer.details === 'object' + ? outer.details as { code?: unknown } + : outer + const value = typeof details.code === 'string' ? details.code : outer.code + return typeof value === 'string' && /^[a-z0-9_.:-]{1,96}$/.test(value) ? value : undefined +} + +function checkpointAvailabilityKey(scope: string, kind: 'request' | 'turn', id: string): string { + return `${scope}:${kind}:${id}` +} + +function checkpointTurnIdentity(provider: FreshAgentPaneContent['provider'], id: string): string { + // Codex can split one provider turn into user and assistant display rows. + // Its accepted-turn id is the unsuffixed provider id, while each rendered + // row carries `:row-N`; match the rewind status to the user row's parent. + return provider === 'codex' ? id.replace(/:row-\d+$/, '') : id +} + +function checkpointTurnAvailabilityKey( + content: FreshAgentPaneContent, + scope: string, + id: string, +): string { + return checkpointAvailabilityKey(scope, 'turn', checkpointTurnIdentity(content.provider, id)) +} + +function checkpointAvailabilityScope(content: FreshAgentPaneContent): string { + // A provider may replace its provisional session id when it materializes + // the durable conversation. The pane create request is the stable identity + // for that conversation generation across the transition. + return content.createRequestId +} + +function checkpointOutcomeAvailability(outcome: FreshAgentCheckpointOutcome): CheckpointTurnAvailability { + return outcome.status === 'available' + ? { + status: 'available', + checkpointId: outcome.checkpointId, + ...(outcome.workspace ? { workspace: outcome.workspace } : {}), + } + : { + status: 'unavailable', + reason: outcome.reason ?? CHECKPOINT_FAILURE_FALLBACK, + ...(outcome.code ? { code: outcome.code } : {}), + ...(outcome.workspace ? { workspace: outcome.workspace } : {}), + } +} + +function checkpointAvailabilityMap(content: FreshAgentPaneContent): Map<string, CheckpointTurnAvailability> { + const scope = checkpointAvailabilityScope(content) + const availability = new Map<string, CheckpointTurnAvailability>() + for (const outcome of content.checkpointOutcomes ?? []) { + if (outcome.conversationRequestId && outcome.conversationRequestId !== content.createRequestId) continue + const value = checkpointOutcomeAvailability(outcome) + availability.set(checkpointAvailabilityKey(scope, 'request', outcome.requestId), value) + if (outcome.displayTurnId) { + availability.set(checkpointTurnAvailabilityKey(content, scope, outcome.displayTurnId), value) + } + if (outcome.nativeTurnId) { + availability.set(checkpointTurnAvailabilityKey(content, scope, outcome.nativeTurnId), value) + } + } + return availability +} + function localEchoLanded( turns: readonly FreshAgentTurn[], echo: LocalEcho, @@ -935,6 +1059,77 @@ export function FreshAgentView({ const localEchoRef = useRef<LocalEcho | null>(null) localEchoRef.current = localEcho const pendingSendMetadataRef = useRef<Map<string, PendingSendMetadata>>(new Map()) + const [checkpointAvailabilityByTurn, setCheckpointAvailabilityByTurn] = useState<Map<string, CheckpointTurnAvailability>>( + () => checkpointAvailabilityMap(paneContent), + ) + const publishCheckpointAvailability = useCallback(( + scope: string, + requestId: string | undefined, + submittedTurnId: string | undefined, + availability: CheckpointTurnAvailability, + ) => { + setCheckpointAvailabilityByTurn((current) => { + const next = new Map(current) + if (requestId) { + next.set(checkpointAvailabilityKey(scope, 'request', requestId), availability) + } + if (submittedTurnId) { + next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, submittedTurnId), availability) + } + return next + }) + if (!requestId || availability.status === 'pending') return + + const matchingTurn = submittedTurnId + ? snapshotRef.current?.turns.find((turn) => turn.role === 'user' + && checkpointTurnIdentity(paneContentRef.current.provider, getFreshAgentDisplayTurnKey(turn)) + === checkpointTurnIdentity(paneContentRef.current.provider, submittedTurnId)) + : undefined + const outcome: FreshAgentCheckpointOutcome = { + requestId, + conversationRequestId: scope, + ...(submittedTurnId ? { displayTurnId: submittedTurnId } : {}), + ...(matchingTurn?.id ? { nativeTurnId: matchingTurn.id } : {}), + status: availability.status, + ...(availability.status === 'available' && availability.checkpointId + ? { checkpointId: availability.checkpointId } + : {}), + ...(availability.workspace ? { workspace: availability.workspace } : {}), + ...(availability.status === 'unavailable' + ? { + ...(availability.code ? { code: availability.code } : {}), + reason: availability.reason, + } + : {}), + } + const state = appStore.getState() + const paneRoot = state.panes.layouts[tabId] + const latestPaneContent = paneRoot ? findPaneContent(paneRoot, paneId) : null + if (latestPaneContent?.kind !== 'fresh-agent' || latestPaneContent.createRequestId !== scope) return + const priorOutcomes = latestPaneContent.checkpointOutcomes ?? [] + const checkpointOutcomes = [ + ...priorOutcomes.filter((item) => item.requestId !== requestId), + outcome, + ].slice(-200) + dispatch(mergePaneContent({ tabId, paneId, updates: { checkpointOutcomes } })) + }, [appStore, dispatch, paneId, tabId]) + const getCheckpointAvailability = useCallback((turn: FreshAgentTurn): CheckpointTurnAvailability | undefined => { + if (turn.role !== 'user') return undefined + const scope = checkpointAvailabilityScope(paneContent) + const turnAvailability = checkpointAvailabilityByTurn.get( + checkpointTurnAvailabilityKey(paneContent, scope, getFreshAgentDisplayTurnKey(turn)), + ) + if (turnAvailability) return turnAvailability + const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId + if (typeof requestId !== 'string' || requestId.length === 0) return undefined + return checkpointAvailabilityByTurn.get(checkpointAvailabilityKey(scope, 'request', requestId)) + }, [checkpointAvailabilityByTurn, paneContent.createRequestId, paneContent.provider]) + const getRewindAvailability = useCallback((turn: FreshAgentTurn): FreshAgentTurnRewindAvailability | undefined => { + const availability = getCheckpointAvailability(turn) + if (!availability) return undefined + if (availability.status === 'unavailable') return { status: 'unavailable', reason: availability.reason } + return { status: availability.status } + }, [getCheckpointAvailability]) // Task 10: requestIds whose FRESH_AGENT_LOST_SESSION failure already fired a // retry, plus the retry requestIds themselves -- a retry is never retried, // so a resend can happen at most once per failed request (loop-proof). @@ -1436,6 +1631,29 @@ export function FreshAgentView({ const current = pendingSendMetadataRef.current.get(requestId) ?? {} const next: PendingSendMetadata = { ...current, ...patch } pendingSendMetadataRef.current.set(requestId, next) + if (next.checkpointAvailabilityScope) { + const availability: CheckpointTurnAvailability | undefined = next.checkpointId + ? { + status: 'available', + checkpointId: next.checkpointId, + ...(next.checkpointWorkspace ? { workspace: next.checkpointWorkspace } : {}), + } + : next.checkpointFailureReason + ? { + status: 'unavailable', + reason: next.checkpointFailureReason, + ...(next.checkpointFailureCode ? { code: next.checkpointFailureCode } : {}), + ...(next.checkpointWorkspace ? { workspace: next.checkpointWorkspace } : {}), + } + : checkpointAvailabilityByTurn.get(checkpointAvailabilityKey( + next.checkpointAvailabilityScope, + 'request', + requestId, + )) + if (availability) { + publishCheckpointAvailability(next.checkpointAvailabilityScope, requestId, next.submittedTurnId, availability) + } + } if ( next.metadataUpdateStarted || !next.cwd @@ -1448,6 +1666,7 @@ export function FreshAgentView({ void Promise .resolve(api.post('/api/fresh-agent/checkpoints/metadata', { cwd: next.cwd, + ...(next.checkpointWorkspace ? { workspace: next.checkpointWorkspace } : {}), id: next.checkpointId, requestId, turnId: next.submittedTurnId, @@ -1461,7 +1680,7 @@ export function FreshAgentView({ pendingSendMetadataRef.current.set(requestId, { ...latest, metadataUpdateStarted: false }) } }) - }, []) + }, [checkpointAvailabilityByTurn, publishCheckpointAvailability]) /** Builds and sends the freshAgent.send frame. Shared by the composer * submit path (sendUserText) and the lost-session retry (Task 10) so the @@ -3568,29 +3787,89 @@ export function FreshAgentView({ // Task 16: a new send starts a fresh idle-incomplete re-poll budget. idleIncompleteRetryCountRef.current = 0 const routeCwd = getFreshOpenCodeRouteCwd(current, { sessionCwd: freshOpenCodeRouteCwdRef.current }) + const checkpointCwd = agentSessionCwdRef.current ?? current.initialCwd + const checkpointScope = checkpointAvailabilityScope(current) // Retain the exact outgoing text as the resend payload (Task 10): the // lost-session retry resends from this metadata, never from the echo. - recordPendingSendMetadata(requestId, { text }) + recordPendingSendMetadata(requestId, { text, checkpointAvailabilityScope: checkpointScope }) // Checkpoint the working tree before the agent acts on this message, so - // "rewind code to here" on this turn restores the pre-turn state. Fire and - // forget: a failed snapshot must never block the send. - if (current.initialCwd) { - recordPendingSendMetadata(requestId, { cwd: current.initialCwd }) - void Promise - .resolve(api.post<CheckpointEntry>('/api/fresh-agent/checkpoints', { - cwd: current.initialCwd, + // "rewind code to here" on this turn restores the pre-turn state. The + // provider dispatch waits until capture settles, but capture failure still + // records an unavailable reason and allows the message to go through. + publishCheckpointAvailability(checkpointScope, requestId, undefined, { status: 'pending' }) + let checkpointReady: Promise<void> | undefined + if (checkpointCwd) { + recordPendingSendMetadata(requestId, { + cwd: checkpointCwd, + ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), + }) + const controller = new AbortController() + let checkpointTimer: number | undefined + let checkpointTimedOut = false + const checkpointTimeout = new Promise<never>((_resolve, reject) => { + checkpointTimer = window.setTimeout(() => { + checkpointTimedOut = true + controller.abort() + reject(new Error(CHECKPOINT_CAPTURE_TIMEOUT_REASON)) + }, CHECKPOINT_CAPTURE_TIMEOUT_MS) + }) + let checkpointRequest: Promise<CheckpointCreationEntry> + try { + checkpointRequest = Promise.resolve(api.post<CheckpointCreationEntry>('/api/fresh-agent/checkpoints', { + cwd: checkpointCwd, + ...(current.checkpointWorkspace ? { workspace: current.checkpointWorkspace } : {}), label: checkpointLabelForText(text), requestId, - })) + }, { signal: controller.signal })) + } catch (error) { + checkpointRequest = Promise.reject(error) + } + checkpointReady = Promise.race([checkpointRequest, checkpointTimeout]) .then((entry) => { + if (checkpointTimedOut) return undefined if (entry?.id) { recordPendingSendMetadata(requestId, { - cwd: current.initialCwd, + cwd: checkpointCwd, + checkpointAvailabilityScope: checkpointScope, + checkpointWorkspace: isCheckpointWorkspace(entry.workspace) + ? entry.workspace + : current.checkpointWorkspace, checkpointId: entry.id, + checkpointFailureReason: undefined, + }) + } else { + recordPendingSendMetadata(requestId, { + checkpointAvailabilityScope: checkpointScope, + checkpointFailureReason: 'The server did not return a saved code checkpoint.', + checkpointFailureCode: 'checkpoint_capture_failed', + ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), }) } + return undefined + }) + .catch((error: unknown) => { + recordPendingSendMetadata(requestId, { + checkpointAvailabilityScope: checkpointScope, + checkpointFailureReason: checkpointTimedOut + ? CHECKPOINT_CAPTURE_TIMEOUT_REASON + : checkpointFailureReason(error), + checkpointFailureCode: checkpointTimedOut + ? 'checkpoint_capture_timeout' + : checkpointFailureCode(error) ?? 'checkpoint_capture_failed', + ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), + }) + return undefined }) - .catch(() => { /* surfaced lazily when a rewind finds no checkpoint */ }) + .finally(() => { + if (checkpointTimer !== undefined) window.clearTimeout(checkpointTimer) + }) + } else { + recordPendingSendMetadata(requestId, { + checkpointAvailabilityScope: checkpointScope, + checkpointFailureReason: 'No project working directory is available for this message.', + checkpointFailureCode: 'workspace_unavailable', + ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), + }) } const isFirstMessage = !autoTitleSentRef.current && (autoTitleFreshBoundaryRef.current || snapshotConfirmsNoUserTurns) @@ -3620,7 +3899,6 @@ export function FreshAgentView({ requestId, previousTurnKeys: (snapshotRef.current?.turns ?? []).map(getTurnKey), } - sendFreshAgentSendFrame(requestId, text, routeCwd) setLocalEchoState(nextLocalEcho) dispatch(mergePaneContent({ tabId, @@ -3630,7 +3908,17 @@ export function FreshAgentView({ pendingLocalEcho: nextLocalEcho, }, })) - }, [dispatch, paneId, recordPendingSendMetadata, sendFreshAgentSendFrame, snapshotConfirmsNoUserTurns, tabId]) + const dispatchAfterCheckpoint = () => { + const latest = paneContentRef.current + if ( + checkpointAvailabilityScope(latest) !== checkpointScope + || latest.sessionId !== current.sessionId + ) return + sendFreshAgentSendFrame(requestId, text, routeCwd) + } + if (checkpointReady) void checkpointReady.then(dispatchAfterCheckpoint) + else dispatchAfterCheckpoint() + }, [dispatch, paneId, publishCheckpointAvailability, recordPendingSendMetadata, sendFreshAgentSendFrame, snapshotConfirmsNoUserTurns, tabId]) // Providers accept one active turn. Keep follow-ups until the session can // actually accept them, including across disconnects and provider failures. @@ -3679,7 +3967,7 @@ export function FreshAgentView({ // without one falls back to the LIVE session cwd (the snapshot's), so // the command runs in the session's working directory rather than the // server's user-home default when the directory is actually known. - const cwd = current.initialCwd ?? agentSessionCwdRef.current + const cwd = agentSessionCwdRef.current ?? current.initialCwd // Bind the async result to THIS conversation: the exec budget is 30 s, // and a /new (or any session replacement) mid-command must never let // the stale completion append into — and auto-send within — the new @@ -3709,33 +3997,91 @@ export function FreshAgentView({ * rewind; fork-from-turn covers the conversation half. */ const rewindToTurn = useCallback((turn: FreshAgentTurn) => { const current = paneContentRef.current - if (!current.initialCwd) { - setNotice('Rewind unavailable: this session has no working directory.') + const availability = getCheckpointAvailability(turn) + if (availability?.status === 'pending') { + setNotice('The code checkpoint for this message is still being saved.') return } - const cwd = current.initialCwd + if (availability?.status === 'unavailable') { + setNotice(`Rewind unavailable for this message: ${availability.reason}`) + return + } + const workspace = availability?.workspace ?? current.checkpointWorkspace + const cwd = agentSessionCwdRef.current ?? current.initialCwd + if (!cwd) { + const reason = 'No project working directory is available for this message.' + const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId + publishCheckpointAvailability( + checkpointAvailabilityScope(current), + typeof requestId === 'string' ? requestId : undefined, + getFreshAgentDisplayTurnKey(turn), + { status: 'unavailable', reason, code: 'workspace_unavailable', ...(workspace ? { workspace } : {}) }, + ) + setNotice(`Rewind unavailable for this message: ${reason}`) + return + } + const query = new URLSearchParams({ cwd }) + if (workspace) { + query.set('workspacePath', workspace.path) + query.set('workspaceProvenance', workspace.provenance) + } void Promise - .resolve(api.get<{ checkpoints: CheckpointEntry[] }>( - `/api/fresh-agent/checkpoints?cwd=${encodeURIComponent(cwd)}`, + .resolve(api.get<{ checkpoints: CheckpointCreationEntry[] }>( + `/api/fresh-agent/checkpoints?${query.toString()}`, )) .then((result) => { - const checkpoint = pickCheckpointForTurn(result?.checkpoints ?? [], snapshot?.turns ?? [], turn) + const checkpoint = pickCheckpointForTurn(result?.checkpoints ?? [], snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null if (!checkpoint) { - setNotice('No checkpoint found for that message (it may predate checkpointing).') + const reason = 'No code checkpoint was saved for this message.' + const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId + publishCheckpointAvailability( + checkpointAvailabilityScope(current), + typeof requestId === 'string' ? requestId : undefined, + getFreshAgentDisplayTurnKey(turn), + { status: 'unavailable', reason, code: 'checkpoint_not_found', ...(workspace ? { workspace } : {}) }, + ) + setNotice(`Rewind unavailable for this message: ${reason}`) return } const confirmed = typeof window === 'undefined' || window.confirm( `Rewind code to the state before "${checkpoint.label}"?\n\nTracked files changed since will be overwritten. Files created since are left in place. The conversation is not affected.`, ) if (!confirmed) return + const restoreWorkspace = isCheckpointWorkspace(checkpoint.workspace) + ? checkpoint.workspace + : workspace return Promise - .resolve(api.post('/api/fresh-agent/checkpoints/restore', { cwd, id: checkpoint.id })) + .resolve(api.post('/api/fresh-agent/checkpoints/restore', { + cwd, + ...(restoreWorkspace ? { workspace: restoreWorkspace } : {}), + id: checkpoint.id, + })) .then(() => setNotice(`Code rewound to before: "${checkpoint.label}"`)) }) .catch((error: unknown) => { + if (error && typeof error === 'object') { + const details = (error as { details?: unknown }).details + if (details && typeof details === 'object' && (details as { code?: unknown }).code === 'workspace_unavailable') { + const reason = checkpointFailureReason(error) + const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId + publishCheckpointAvailability( + checkpointAvailabilityScope(current), + typeof requestId === 'string' ? requestId : undefined, + getFreshAgentDisplayTurnKey(turn), + { + status: 'unavailable', + reason, + code: checkpointFailureCode(error) ?? 'checkpoint_restore_failed', + ...(workspace ? { workspace } : {}), + }, + ) + setNotice(`Rewind unavailable for this message: ${reason}`) + return + } + } setNotice(error instanceof Error ? `Rewind failed: ${error.message}` : 'Rewind failed') }) - }, [snapshot?.turns]) + }, [getCheckpointAvailability, publishCheckpointAvailability, snapshot?.turns]) // Task 6 (freshopencode TUI parity): the delegation block's "Open session" // link resumes the child durable session in its own pane, like the sidebar @@ -4185,6 +4531,7 @@ export function FreshAgentView({ showTranscriptMinimap={showTranscriptMinimap} isStreaming={isBusy} onForkFromTurn={(turnId) => sendFork(turnId)} + getRewindAvailability={getRewindAvailability} onRollbackToTurn={(turnId) => { // The busy pre-flight gate picks copy by DIRECTION (decision 7)… if (isBusy) { @@ -4210,7 +4557,7 @@ export function FreshAgentView({ } setNotice(REDO_DESTROYED_NOTICE) }} - onRewindToTurn={paneContent.initialCwd ? rewindToTurn : undefined} + onRewindToTurn={rewindToTurn} /> </div> {revealRefreshBlocking ? ( @@ -4335,6 +4682,7 @@ export function FreshAgentView({ effectiveShowTimecodes, showTranscriptMinimap, globalExpandTools, + getRewindAvailability, isBusy, isRestoring, loadError, diff --git a/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts b/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts new file mode 100644 index 000000000..a89c97a97 --- /dev/null +++ b/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts @@ -0,0 +1,189 @@ +import { execFileSync } from 'node:child_process' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { fileURLToPath } from 'node:url' +import { test, expect, createFreshE2ePage } from '../helpers/fixtures.js' +import { RustServer } from '../helpers/rust-server.js' +import { TestHarness } from '../helpers/test-harness.js' +import { openPanePicker } from '../helpers/pane-picker.js' + +const __filename = fileURLToPath(import.meta.url) +const __dirname = path.dirname(__filename) +const FAKE_CODEX_APP_SERVER_SOURCE = path.resolve( + __dirname, + '../../fixtures/coding-cli/codex-app-server/fake-app-server.mjs', +) + +async function installFakeCodexAppServer(destDir: string): Promise<string> { + await fs.mkdir(destDir, { recursive: true }) + const dest = path.join(destDir, 'fake-codex-app-server-wrapper.mjs') + const wrapper = `#!/usr/bin/env node +import { spawnSync } from 'node:child_process' +const target = ${JSON.stringify(FAKE_CODEX_APP_SERVER_SOURCE)} +const result = spawnSync(process.execPath, [target, ...process.argv.slice(2)], { stdio: 'inherit' }) +process.exit(result.status ?? 1) +` + await fs.writeFile(dest, wrapper, 'utf8') + await fs.chmod(dest, 0o755) + return dest +} + +function findFreshAgentLeaf(node: any): any { + if (!node) return null + if (node.type === 'leaf' && node.content?.kind === 'fresh-agent') return node + if (node.type === 'split') { + for (const child of node.children ?? []) { + const found = findFreshAgentLeaf(child) + if (found) return found + } + } + return null +} + +test.describe('fresh-agent checkpoint availability', () => { + test('an unreadable selected project file marks rewind unavailable and still completes the provider turn', async ({ browser }) => { + test.setTimeout(180_000) + const sharedRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-checkpoint-unavailable-')) + const projectDir = path.join(sharedRoot, 'project') + const trackedFile = path.join(projectDir, 'private-fixture.txt') + const codexHome = path.join(sharedRoot, 'codex-home') + let server: RustServer | undefined + let context: Awaited<ReturnType<typeof createFreshE2ePage>>['context'] | undefined + + try { + await fs.mkdir(projectDir, { recursive: true }) + await fs.writeFile(trackedFile, 'tracked but unreadable fixture\n', 'utf8') + execFileSync('git', ['init', projectDir], { stdio: 'ignore' }) + execFileSync('git', ['-C', projectDir, 'config', 'user.email', 'fixture@example.invalid'], { stdio: 'ignore' }) + execFileSync('git', ['-C', projectDir, 'config', 'user.name', 'Freshell fixture'], { stdio: 'ignore' }) + execFileSync('git', ['-C', projectDir, 'add', '--', 'private-fixture.txt'], { stdio: 'ignore' }) + execFileSync('git', ['-C', projectDir, 'commit', '-m', 'fixture'], { stdio: 'ignore' }) + await fs.chmod(trackedFile, 0o000) + await expect(fs.readFile(trackedFile, 'utf8')).rejects.toMatchObject({ code: 'EACCES' }) + + const fakeCodexPath = await installFakeCodexAppServer(path.join(sharedRoot, 'bin')) + server = new RustServer({ + env: { + CODEX_CMD: fakeCodexPath, + CODEX_HOME: codexHome, + FAKE_CODEX_APP_SERVER_BEHAVIOR: JSON.stringify({ recordTurns: true }), + FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES: '1', + }, + setupHome: async (homeDir) => { + const freshellDir = path.join(homeDir, '.freshell') + await fs.mkdir(freshellDir, { recursive: true }) + await fs.writeFile(path.join(freshellDir, 'config.json'), JSON.stringify({ + version: 1, + settings: { + freshAgent: { enabled: true }, + codingCli: { + enabledProviders: ['codex'], + providers: { codex: { model: 'gpt-5-codex', sandbox: 'workspace-write' } }, + }, + }, + }, null, 2)) + }, + }) + const info = await server.start() + const ownedPage = await createFreshE2ePage(browser, info) + context = ownedPage.context + const { page } = ownedPage + await page.goto(`${info.baseUrl}/?token=${info.token}&e2e=1`) + const harness = new TestHarness(page) + await harness.waitForHarness() + await harness.waitForConnection() + await page.evaluate(() => { + window.__FRESHELL_TEST_HARNESS__?.dispatch({ + type: 'connection/setAvailableClis', + payload: { claude: false, codex: true }, + }) + }) + + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshcodex$/i }).click({ force: true }) + const directoryInput = page.getByRole('combobox', { name: 'Starting directory for Freshcodex' }) + await expect(directoryInput).toBeVisible({ timeout: 10_000 }) + await directoryInput.fill(projectDir) + await directoryInput.press('Enter') + + const paneRoot = page.locator('[data-context="fresh-agent"]').last() + await expect(paneRoot).toBeVisible({ timeout: 15_000 }) + const tabId = await harness.getActiveTabId() + expect(tabId).toBeTruthy() + await expect.poll(async () => { + const layout = await harness.getPaneLayout(tabId!) + return findFreshAgentLeaf(layout)?.content?.initialCwd + }, { timeout: 10_000 }).toBe(projectDir) + await expect.poll(async () => { + const layout = await harness.getPaneLayout(tabId!) + return findFreshAgentLeaf(layout)?.content?.status + }, { timeout: 30_000 }).toBe('idle') + + const marker = `checkpoint-unavailable-${Math.random().toString(36).slice(2, 10)}` + const composer = paneRoot.getByRole('textbox', { name: 'Chat message input' }) + await composer.fill(marker) + await paneRoot.getByRole('button', { name: 'Send' }).click() + + const userTurn = paneRoot.locator('[data-turn-role="user"]', { hasText: marker }) + await expect(userTurn).toBeVisible({ timeout: 20_000 }) + const unavailable = userTurn.getByRole('note', { name: 'Code rewind unavailable for this message' }) + await expect(unavailable).toContainText('A selected project file could not be read') + await userTurn.hover() + await expect(userTurn.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + await expect(paneRoot.getByText('Fixture turn')).toBeVisible({ timeout: 30_000 }) + + let content: any + await expect.poll(async () => { + const layout = await harness.getPaneLayout(tabId!) + content = findFreshAgentLeaf(layout)?.content + return content?.status + }, { timeout: 20_000 }).toBe('idle') + expect(content.checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ + status: 'unavailable', + code: 'checkpoint_path_unreadable', + reason: expect.stringContaining('A selected project file could not be read'), + }), + ])) + expect(JSON.stringify(content.checkpointOutcomes)).not.toContain(marker) + + await page.evaluate(() => { + window.__FRESHELL_TEST_HARNESS__?.dispatch({ type: 'persist/flushNow' }) + }) + await expect.poll(() => page.evaluate(() => Object.keys(localStorage) + .filter((key) => key.startsWith('freshell.layout.v3.')) + .some((key) => localStorage.getItem(key)?.includes('checkpointOutcomes') ?? false))) + .toBe(true) + + await page.reload() + const reloadedHarness = new TestHarness(page) + await reloadedHarness.waitForHarness() + await reloadedHarness.waitForConnection() + const reloadedPane = page.locator('[data-context="fresh-agent"]').last() + await expect(reloadedPane).toBeVisible({ timeout: 20_000 }) + const reloadedUserTurn = reloadedPane.locator('[data-turn-role="user"]', { hasText: marker }) + await expect(reloadedUserTurn).toBeVisible({ timeout: 30_000 }) + await expect(reloadedUserTurn.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toContainText('A selected project file could not be read') + await reloadedUserTurn.hover() + await expect(reloadedUserTurn.getByRole('button', { name: 'Rewind unavailable for this message' })) + .toBeDisabled() + + const threadId = content.sessionId ?? content.sessionRef?.sessionId + expect(threadId).toBeTruthy() + const recordedTurnsPath = path.join(codexHome, 'fake-turns', `${threadId}.json`) + await expect.poll(async () => { + const raw = await fs.readFile(recordedTurnsPath, 'utf8').catch(() => '[]') + const turns = JSON.parse(raw) as Array<{ items?: Array<{ type?: string; content?: Array<{ text?: string }> }> }> + return turns.some((turn) => turn.items?.some((item) => item.type === 'userMessage' + && item.content?.some((part) => part.text === marker))) + }, { timeout: 20_000 }).toBe(true) + } finally { + await context?.close().catch(() => {}) + await server?.stop().catch(() => {}) + await fs.chmod(trackedFile, 0o600).catch(() => {}) + await fs.rm(sharedRoot, { recursive: true, force: true }).catch(() => {}) + } + }) +}) diff --git a/test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx index 10004aa85..ef690173b 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx @@ -2054,6 +2054,30 @@ describe('FreshAgentTranscript', () => { expect(onRewind).toHaveBeenCalledWith(expect.objectContaining({ id: 'turn-1', role: 'user' })) }) + it('shows unavailable rewind state and disables the action on the affected turn', () => { + const reason = 'Select a project folder to save code rewind.' + const onRewind = vi.fn() + render( + <FreshAgentTranscript + paneId="pane-unavailable-rewind" + turns={TURNS} + canFork={false} + onRewindToTurn={onRewind} + getRewindAvailability={(turn) => turn.turnId === 'turn-1' ? { status: 'unavailable', reason } : undefined} + />, + ) + + const userArticle = screen.getByRole('article', { name: 'You transcript turn' }) + expect(within(userArticle).getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent(`Code rewind unavailable for this message: ${reason}`) + expect(within(userArticle).getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + + const build = getFreshAgentTurnItemsBuilder('pane-unavailable-rewind') + const rewindItem = build!(0)!.find((item) => item.label.startsWith('Rewind unavailable:')) + expect(rewindItem).toMatchObject({ label: `Rewind unavailable: ${reason}`, disabled: true }) + expect(onRewind).not.toHaveBeenCalled() + }) + it('gates fork/rollback/rewind per capability and role through the registered builder', () => { const onRewind = vi.fn() render( diff --git a/test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx index 3762eefe4..3791c364b 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx @@ -49,6 +49,41 @@ describe('FreshAgentTurnActions', () => { expect(onForkFromTurn).toHaveBeenCalledWith('codex-display:v1:opaque-user-row') }) + + it('disables rewind and shows the per-message failure reason', () => { + const turn: FreshAgentTurn = { + id: 'user-turn', + turnId: 'user-turn', + role: 'user', + summary: 'prompt', + items: [{ id: 'user-text', kind: 'text', text: 'prompt' }], + } + const reason = 'Select a project folder to save code rewind.' + const onRewindToTurn = vi.fn() + const callbacks = { + canFork: false, + onRewindToTurn, + getRewindAvailability: () => ({ status: 'unavailable' as const, reason }), + } + + const rewind = buildTurnActionItems(turn, callbacks).find((item) => item.label.startsWith('Rewind unavailable:')) + expect(rewind).toMatchObject({ label: `Rewind unavailable: ${reason}`, disabled: true }) + + render( + <FreshAgentTurnActions + turn={turn} + canFork={false} + onRewindToTurn={onRewindToTurn} + getRewindAvailability={callbacks.getRewindAvailability} + />, + ) + + const button = screen.getByRole('button', { name: 'Rewind unavailable for this message' }) + expect(button).toBeDisabled() + expect(button).toHaveAttribute('title', `Rewind unavailable: ${reason}`) + fireEvent.click(button) + expect(onRewindToTurn).not.toHaveBeenCalled() + }) }) describe('rollback affordance (kata 1wxv decisions 3, 8)', () => { diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index d636dc175..fc0914725 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -296,6 +296,7 @@ const wsMock = vi.hoisted(() => ({ })) const apiMock = vi.hoisted(() => ({ + get: vi.fn(), getFreshAgentThreadSnapshot: vi.fn(), getFreshAgentModelCapabilities: vi.fn(), post: vi.fn(), @@ -321,7 +322,7 @@ vi.mock('@/lib/api', async () => { const actual = await vi.importActual<typeof import('@/lib/api')>('@/lib/api') return { ...actual, - api: { ...actual.api, post: apiMock.post }, + api: { ...actual.api, get: apiMock.get, post: apiMock.post }, getFreshAgentThreadSnapshot: apiMock.getFreshAgentThreadSnapshot, getFreshAgentModelCapabilities: apiMock.getFreshAgentModelCapabilities, requestSessionHandoff: apiMock.requestSessionHandoff, @@ -554,6 +555,8 @@ beforeEach(() => { window.localStorage.removeItem('fresh-agent-prompt-history:freshclaude') apiMock.getFreshAgentThreadSnapshot.mockReset() apiMock.getFreshAgentModelCapabilities.mockReset() + apiMock.get.mockReset() + apiMock.get.mockResolvedValue({ checkpoints: [] }) apiMock.post.mockReset() apiMock.requestSessionHandoff.mockReset() apiMock.setSessionMetadata.mockReset() @@ -3392,19 +3395,21 @@ describe('FreshAgentView', () => { }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) - expect(wsMock.send).toHaveBeenCalledWith(expect.objectContaining({ - type: 'freshAgent.send', - requestId: expect.any(String), - sessionId: 'thread-1', - sessionType: 'freshcodex', - provider: 'codex', - text: 'Ship it', - settings: { - cwd: '/repo', - model: 'gpt-5.3-codex-spark', - effort: 'max', - }, - })) + await waitFor(() => { + expect(wsMock.send).toHaveBeenCalledWith(expect.objectContaining({ + type: 'freshAgent.send', + requestId: expect.any(String), + sessionId: 'thread-1', + sessionType: 'freshcodex', + provider: 'codex', + text: 'Ship it', + settings: { + cwd: '/repo', + model: 'gpt-5.3-codex-spark', + effort: 'max', + }, + })) + }) expect(screen.queryByRole('button', { name: 'Interrupt' })).not.toBeInTheDocument() expect(screen.queryByRole('button', { name: 'Fork' })).not.toBeInTheDocument() @@ -3412,7 +3417,16 @@ describe('FreshAgentView', () => { it('uses send acknowledgements to patch checkpoints and clear local echo only on the submitted user display turn', async () => { const store = createStore() - const checkpoint = createDeferred<{ id: string; ts: number; label: string; requestId: string }>() + const checkpoint = createDeferred<{ + id: string + ts: number + label: string + requestId: string + workspace: { path: string; provenance: 'directory-picker' } + }>() + const checkpointWorkspace = { path: '/repo/project', provenance: 'directory-picker' as const } + const resolvedCheckpointWorkspace = { path: '/repo/project-root', provenance: 'directory-picker' as const } + const listedCheckpointWorkspace = { path: '/repo/legacy-checkpoint-root', provenance: 'legacy-checkpoint' as const } let onMessage: ((message: Record<string, unknown>) => void) | undefined wsMock.onMessage.mockImplementation((handler: (message: Record<string, unknown>) => void) => { onMessage = handler @@ -3431,8 +3445,29 @@ describe('FreshAgentView', () => { capabilities: { send: true, interrupt: true, fork: true }, turns: [ { - id: 'native-user-turn', - turnId: 'display-user-1', + id: 'native-user-turn:row-0', + turnId: 'display-user-1:row-0', + role: 'user', + summary: 'Ship it', + items: [{ id: 'user-text-1', kind: 'text', text: 'Ship it' }], + }, + { + id: 'native-assistant-turn', + turnId: 'display-assistant-1', + role: 'assistant', + summary: 'Done', + items: [{ id: 'assistant-text-1', kind: 'text', text: 'Done.' }], + }, + ], + }) + .mockResolvedValue({ + status: 'idle', + summary: 'answered after materialization', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [ + { + id: 'native-user-turn:row-0', + turnId: 'display-user-1:row-0', role: 'user', summary: 'Ship it', items: [{ id: 'user-text-1', kind: 'text', text: 'Ship it' }], @@ -3451,6 +3486,16 @@ describe('FreshAgentView', () => { if (url === '/api/fresh-agent/checkpoints/metadata') return Promise.resolve({ ok: true, body }) return Promise.resolve({ title: null, source: 'none' }) }) + apiMock.get.mockResolvedValue({ + checkpoints: [{ + id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + ts: 1, + label: 'Ship it', + requestId: 'req-normalized-send', + turnId: 'display-user-1', + workspace: listedCheckpointWorkspace, + }], + }) store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', @@ -3462,10 +3507,11 @@ describe('FreshAgentView', () => { sessionId: 'thread-normalized-send', status: 'idle', initialCwd: '/repo', + checkpointWorkspace, }, })) - render( + const successView = render( <Provider store={store}> <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> </Provider>, @@ -3481,55 +3527,398 @@ describe('FreshAgentView', () => { }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) - const send = sentFreshAgentMessages('freshAgent.send').at(-1) - expect(send).toMatchObject({ - type: 'freshAgent.send', - sessionId: 'thread-normalized-send', - sessionType: 'freshcodex', - provider: 'codex', - text: 'Ship it', - }) - expect(send?.requestId).toEqual(expect.any(String)) - const requestId = String(send?.requestId) expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints', { cwd: '/repo', + workspace: checkpointWorkspace, label: 'Ship it', - requestId, - }) + requestId: expect.any(String), + }, expect.objectContaining({ signal: expect.anything() })) + const checkpointRequest = apiMock.post.mock.calls.find(([url]) => url === '/api/fresh-agent/checkpoints') + const requestId = String(checkpointRequest?.[1]?.requestId) + expect(requestId).not.toBe('undefined') + expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(0) expect(screen.getByText('Ship it')).toBeInTheDocument() - expect(onMessage).toBeTypeOf('function') act(() => { - onMessage?.({ - type: 'freshAgent.send.accepted', - requestId, - submittedTurnId: 'display-user-1', - }) - }) - await act(async () => { checkpoint.resolve({ id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', ts: 1, label: 'Ship it', requestId, + workspace: resolvedCheckpointWorkspace, + }) + }) + await waitFor(() => { + expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ + type: 'freshAgent.send', + requestId, + sessionId: 'thread-normalized-send', + sessionType: 'freshcodex', + provider: 'codex', + text: 'Ship it', + })) + }) + expect(onMessage).toBeTypeOf('function') + act(() => { + onMessage?.({ + type: 'freshAgent.send.accepted', + requestId, + submittedTurnId: 'display-user-1', }) - await Promise.resolve() }) await waitFor(() => { expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/metadata', { cwd: '/repo', + workspace: resolvedCheckpointWorkspace, id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', requestId, turnId: 'display-user-1', }) }) + act(() => { + onMessage?.({ + type: 'freshAgent.session.materialized', + previousSessionId: 'thread-normalized-send', + sessionId: 'thread-materialized-send', + sessionType: 'freshcodex', + provider: 'codex', + sessionRef: { provider: 'codex', sessionId: 'thread-materialized-send' }, + }) + }) + await waitFor(() => { + expect(getFreshAgentPaneContent(store).sessionId).toBe('thread-materialized-send') + }) + expect(await screen.findByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') await waitFor(() => { expect(screen.getByText('Done.')).toBeInTheDocument() }) expect(screen.getAllByText('Ship it')).toHaveLength(1) const transcriptTurns = screen.getAllByRole('article') expect(transcriptTurns.at(-1)).toHaveTextContent('Done.') + + const persistedOutcomes = getFreshAgentPaneContent(store).checkpointOutcomes ?? [] + expect(persistedOutcomes).toEqual(expect.arrayContaining([expect.objectContaining({ + requestId, + conversationRequestId: 'req-normalized-send', + displayTurnId: 'display-user-1', + status: 'available', + checkpointId: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + workspace: resolvedCheckpointWorkspace, + })])) + expect(JSON.stringify(persistedOutcomes)).not.toContain('Ship it') + successView.unmount() + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + expect(await screen.findByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') + + vi.spyOn(window, 'confirm').mockReturnValue(true) + fireEvent.click(screen.getByRole('button', { name: 'Rewind code to here' })) + await waitFor(() => { + expect(apiMock.get).toHaveBeenCalledWith( + '/api/fresh-agent/checkpoints?cwd=%2Frepo&workspacePath=%2Frepo%2Fproject-root&workspaceProvenance=directory-picker', + ) + }) + await waitFor(() => { + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: '/repo', + workspace: listedCheckpointWorkspace, + id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + }) + }) + }) + + it('uses the live provider cwd for checkpoints when the launch cwd is stale', async () => { + const store = createStore() + const sessionId = 'thread-live-checkpoint-cwd' + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'empty', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [], + }) + store.dispatch(sessionInit({ + sessionId, + sessionType: 'freshcodex', + provider: 'codex', + cwd: '/workspace/resolved-project', + })) + apiMock.post.mockImplementation((url: string, body?: { requestId?: string }) => ( + url === '/api/fresh-agent/checkpoints' + ? Promise.resolve({ + id: 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb', + ts: 1, + label: 'Use the resolved project', + requestId: body?.requestId ?? '', + workspace: { path: '/workspace/resolved-project', provenance: 'resolved-project' }, + }) + : Promise.resolve({ ok: true }) + )) + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-stale-launch-cwd', + sessionId, + status: 'idle', + initialCwd: '/home/launch-time', + }, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) + await waitFor(() => expect(composer).not.toBeDisabled()) + fireEvent.change(composer, { target: { value: 'Use the resolved project' } }) + fireEvent.click(screen.getByRole('button', { name: 'Send' })) + + await waitFor(() => { + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints', expect.objectContaining({ + cwd: '/workspace/resolved-project', + label: 'Use the resolved project', + }), expect.objectContaining({ signal: expect.anything() })) + }) + await waitFor(() => { + expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ + sessionId, + text: 'Use the resolved project', + })) + }) + }) + + it('keeps sending when checkpoint creation fails and shows the affected message as unavailable with the server reason', async () => { + const store = createStore() + const checkpoint = createDeferred<never>() + const workspaceReason = 'Account home is not a code workspace. Select a project folder.' + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'empty', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [], + }) + apiMock.post.mockImplementation((url: string) => { + if (url === '/api/fresh-agent/checkpoints') return checkpoint.promise + return Promise.resolve({ ok: true }) + }) + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-checkpoint-failure', + sessionId: 'thread-checkpoint-failure', + status: 'idle', + initialCwd: '/account/home', + }, + })) + + const failureView = render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) + await waitFor(() => expect(composer).not.toBeDisabled()) + fireEvent.change(composer, { target: { value: 'Continue despite the checkpoint failure' } }) + fireEvent.click(screen.getByRole('button', { name: 'Send' })) + + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints', { + cwd: '/account/home', + label: 'Continue despite the checkpoint failure', + requestId: expect.any(String), + }, expect.objectContaining({ signal: expect.anything() })) + const checkpointRequest = apiMock.post.mock.calls.find(([url]) => url === '/api/fresh-agent/checkpoints') + const requestId = String(checkpointRequest?.[1]?.requestId) + expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(0) + expect(screen.getByText('Continue despite the checkpoint failure')).toBeInTheDocument() + expect(await screen.findByRole('status', { name: 'Saving code rewind checkpoint for this message' })) + .toHaveTextContent('Saving the code state from before this message.') + + await act(async () => { + checkpoint.reject(new ApiError(422, 'workspace unavailable', { + code: 'workspace_unavailable', + details: workspaceReason, + })) + await checkpoint.promise.catch(() => undefined) + }) + await waitFor(() => { + expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ + type: 'freshAgent.send', + requestId, + sessionId: 'thread-checkpoint-failure', + text: 'Continue despite the checkpoint failure', + })) + }) + const unavailable = await screen.findByRole('note', { name: 'Code rewind unavailable for this message' }) + expect(unavailable).toHaveTextContent(`Code rewind unavailable for this message: ${workspaceReason}`) + expect(screen.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + expect(screen.getByText('Continue despite the checkpoint failure')).toBeInTheDocument() + + const persistedOutcomes = getFreshAgentPaneContent(store).checkpointOutcomes ?? [] + expect(persistedOutcomes).toEqual(expect.arrayContaining([expect.objectContaining({ + requestId, + conversationRequestId: 'req-checkpoint-failure', + status: 'unavailable', + code: 'workspace_unavailable', + reason: workspaceReason, + })])) + expect(JSON.stringify(persistedOutcomes)).not.toContain('Continue despite the checkpoint failure') + + failureView.unmount() + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + expect(await screen.findByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent(workspaceReason) + expect(screen.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + }) + + it('shows a bounded server checkpoint explanation without echoing the submitted message', async () => { + const store = createStore() + const workspaceReason = 'A selected project file could not be read, so this code checkpoint is unavailable.' + const submittedText = 'Never echo this user message in checkpoint diagnostics' + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'empty', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [], + }) + apiMock.post.mockImplementation((url: string) => ( + url === '/api/fresh-agent/checkpoints' + ? Promise.reject(new ApiError(500, 'Code checkpoint unavailable', { + code: 'checkpoint_path_unreadable', + details: workspaceReason, + })) + : Promise.resolve({ ok: true }) + )) + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-checkpoint-server-error', + sessionId: 'thread-checkpoint-server-error', + status: 'idle', + initialCwd: '/repo', + }, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) + await waitFor(() => expect(composer).not.toBeDisabled()) + fireEvent.change(composer, { target: { value: submittedText } }) + fireEvent.click(screen.getByRole('button', { name: 'Send' })) + + const unavailable = await screen.findByRole('note', { name: 'Code rewind unavailable for this message' }) + expect(unavailable).toHaveTextContent(workspaceReason) + expect(unavailable).not.toHaveTextContent(submittedText) + await waitFor(() => { + expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ + type: 'freshAgent.send', + sessionId: 'thread-checkpoint-server-error', + text: submittedText, + })) + }) + }) + + it('bounds a stalled checkpoint capture and ignores its late result', async () => { + const store = createStore() + const checkpoint = createDeferred<{ id: string; ts: number; label: string; requestId: string }>() + let checkpointSignal: AbortSignal | undefined + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'empty', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [], + }) + apiMock.post.mockImplementation((url: string, _body: unknown, options?: { signal?: AbortSignal }) => { + if (url === '/api/fresh-agent/checkpoints') { + checkpointSignal = options?.signal + return checkpoint.promise + } + return Promise.resolve({ ok: true }) + }) + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-checkpoint-timeout', + sessionId: 'thread-checkpoint-timeout', + status: 'idle', + initialCwd: '/repo', + }, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) + await waitFor(() => expect(composer).not.toBeDisabled()) + vi.useFakeTimers() + try { + fireEvent.change(composer, { target: { value: 'Send even if checkpoint capture stalls' } }) + fireEvent.click(screen.getByRole('button', { name: 'Send' })) + expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(0) + + await act(async () => { + await vi.advanceTimersByTimeAsync(30_000) + }) + + expect(checkpointSignal?.aborted).toBe(true) + expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ + type: 'freshAgent.send', + sessionId: 'thread-checkpoint-timeout', + text: 'Send even if checkpoint capture stalls', + })) + expect(screen.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent('Code checkpoint capture timed out after 30 seconds.') + + await act(async () => { + checkpoint.resolve({ + id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + ts: 1, + label: 'late result', + requestId: String(apiMock.post.mock.calls.find(([url]) => url === '/api/fresh-agent/checkpoints')?.[1]?.requestId), + }) + await Promise.resolve() + }) + + expect(screen.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent('Code checkpoint capture timed out after 30 seconds.') + expect(screen.queryByRole('status', { name: 'Code rewind available for this message' })).not.toBeInTheDocument() + } finally { + vi.useRealTimers() + } }) it('persists a pending local echo so a remounted pane keeps the submitted prompt visible', async () => { @@ -3631,6 +4020,9 @@ describe('FreshAgentView', () => { }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) + await waitFor(() => { + expect(sentFreshAgentMessages('freshAgent.send').some((message) => message.text === 'hello again')).toBe(true) + }) const sendFrame = sentFreshAgentMessages('freshAgent.send').at(-1) expect(sendFrame).toBeTruthy() expect(sendFrame?.text).toBe('hello again') @@ -3729,6 +4121,9 @@ describe('FreshAgentView', () => { }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) + await waitFor(() => { + expect(sentFreshAgentMessages('freshAgent.send').some((message) => message.text === 'hello placeholder')).toBe(true) + }) const sendFrame = sentFreshAgentMessages('freshAgent.send').at(-1) expect(sendFrame?.text).toBe('hello placeholder') await waitFor(() => { @@ -3824,19 +4219,21 @@ describe('FreshAgentView', () => { }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) - expect(wsMock.send).toHaveBeenCalledWith(expect.objectContaining({ - type: 'freshAgent.send', - requestId: expect.any(String), - sessionId: 'freshopencode-req-opencode-send-policy', - sessionType: 'freshopencode', - provider: 'opencode', - text: 'Use local OpenCode policy', - settings: { - cwd: '/repo', - model: 'opencode-go/deepseek-v4-flash', - effort: 'max', - }, - })) + await waitFor(() => { + expect(wsMock.send).toHaveBeenCalledWith(expect.objectContaining({ + type: 'freshAgent.send', + requestId: expect.any(String), + sessionId: 'freshopencode-req-opencode-send-policy', + sessionType: 'freshopencode', + provider: 'opencode', + text: 'Use local OpenCode policy', + settings: { + cwd: '/repo', + model: 'opencode-go/deepseek-v4-flash', + effort: 'max', + }, + })) + }) }) it('creates Freshopencode panes with modelSelection when persisted model is absent after reload', async () => { From 0919de46a23dd0a14be773b6abed9932bbc1a974 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 02:18:51 -0700 Subject: [PATCH 302/493] fix: reject incomplete checkpoint snapshots --- crates/freshell-server/src/checkpoints.rs | 2073 ++++++++++++++++- crates/freshell-server/src/files.rs | 34 +- .../tests/code_checkpoint_permissions.rs | 581 +++++ docker/sandbox/entrypoint.sh | 46 + scripts/sandbox-test.sh | 14 + 5 files changed, 2634 insertions(+), 114 deletions(-) create mode 100644 crates/freshell-server/tests/code_checkpoint_permissions.rs diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 868ccc400..e26440357 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -1,46 +1,28 @@ -//! `POST /api/fresh-agent/checkpoints` — a faithful (create-only) port of -//! `server/fresh-agent-extras-router.ts:346-368` (`createCheckpoint`, -//! `ensureCheckpointRepo`, `checkpointGitDir`, `updateCheckpointMetadata`). +//! Code rewind checkpoints for FreshAgent panes. The routes here create, list, +//! annotate, and restore snapshots in a shadow bare Git repository under +//! `<home>/.freshell/checkpoints/<sha1-of-workspace-path>.git`; the source +//! repository's index, refs, and permissions are never changed. //! -//! ## What this is +//! A pane carries checkpoint workspace intent separately from its launch/routing +//! cwd. Directory-picker selections can name a non-Git directory; an inferred +//! workspace resolves to the enclosing Git root. An account HOME is never an +//! implicit capture scope. Older checkpoint stores remain addressable by their +//! exact legacy cwd locator. //! -//! The SPA fires this on every fresh-agent send (`src/components/fresh-agent/ -//! FreshAgentView.tsx:1583` `sendUserText`'s fire-and-forget pre-turn snapshot) so -//! "rewind code to here" can later restore the working tree to its state just -//! before a given user turn. Snapshots live in a **shadow bare git repository** -//! under `<home>/.freshell/checkpoints/<sha1-of-abs-cwd>.git` — the session cwd's -//! own git state (if any) is never touched; the shadow repo uses the cwd only as -//! `--work-tree`, so the cwd's own `.gitignore` still applies (`node_modules` and -//! friends never enter a snapshot) and a cwd that isn't itself a git repo works -//! identically (there is no "non-git cwd" degraded path in the legacy behavior — -//! `--work-tree` doesn't require the work tree to be a repo of its own). +//! Each create uses a request-local index, applies the workspace's Git ignore +//! rules, expands nested repositories and submodules into ordinary files, and +//! verifies that every selected regular file and symlink can be read. Metadata +//! is written before the shadow ref is advanced, and the ref update is the final +//! publication step. Restore and metadata routes accept only commits reachable +//! from the published ref, so failed/orphaned objects never become targets. //! -//! ## Scope (deliberately narrow — see the task's port-minimally guidance) -//! -//! Only `POST /checkpoints` (create) is ported here — the route that is 404ing -//! today and hit on every send. The sibling routes (`GET /checkpoints` list, -//! `POST /checkpoints/restore`, `POST /checkpoints/metadata`) back the "rewind -//! code to here" UI action, which is a SEPARATE, much rarer user gesture (not -//! fired on every send) — deferred to a follow-up port. The client's fire-and- -//! forget call site (`FreshAgentView.tsx:1596` `.catch(() => {})`) swallows any -//! error from this route entirely, so a partial rewind feature (no way to list or -//! restore checkpoints yet) causes no visible client error either way; it just -//! means "rewind to here" isn't wired up yet, which is already true today (404). -//! -//! ## Response shape (must match what the SPA reads) -//! -//! `src/components/fresh-agent/FreshAgentView.tsx:1583` does -//! `api.post<CheckpointEntry>('/api/fresh-agent/checkpoints', {...}).then((entry) => -//! { if (entry?.id) {...} })` — a plain untyped-at-runtime cast (no zod parse for -//! this endpoint; confirmed by grep of `src/lib/api.ts` and -//! `src/lib/fresh-agent-checkpoints.ts` — only `.id` is read). The exact legacy -//! shape (`fresh-agent-extras-router.ts:183`) is -//! `{ id: string, ts: number, label: string, requestId?: string }` — `id` is the -//! full 40-hex commit sha, `ts` is Unix seconds, `requestId` is present iff the -//! caller supplied one (JS object spread of `metadata` drops absent keys, not -//! serializes `null`/`undefined`). +//! The create response retains the legacy `{ id, ts, label, requestId? }` fields +//! and adds the resolved `{ path, provenance }` workspace locator. Expected +//! scope/readability failures return a stable code and user-safe details while +//! structured logs retain bounded internal cause and request correlation. use std::path::{Path, PathBuf}; +use std::process::Stdio; use std::sync::Arc; use std::time::{SystemTime, UNIX_EPOCH}; @@ -48,14 +30,15 @@ use std::collections::HashMap; use axum::routing::get as get_method; use axum::{ + Json, Router, extract::{Query, State}, http::{HeaderMap, StatusCode}, response::{IntoResponse, Response}, routing::post, - Json, Router, }; -use serde_json::{json, Map, Value}; +use serde_json::{Map, Value, json}; use sha1::{Digest, Sha1}; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; use tokio::process::Command; use crate::boot::{is_authed, unauthorized}; @@ -93,6 +76,425 @@ pub struct CheckpointsApiState { pub home: Arc<PathBuf>, } +#[derive(Clone, Debug, PartialEq, Eq)] +struct ResolvedCheckpointWorkspace { + path: PathBuf, + provenance: String, +} + +#[derive(Debug)] +struct CheckpointWorkspaceUnavailable { + candidate: String, + reason: String, +} + +impl CheckpointWorkspaceUnavailable { + fn response(&self) -> Response { + ( + StatusCode::UNPROCESSABLE_ENTITY, + Json(json!({ + "error": self.reason, + "code": "workspace_unavailable", + "workspace": self.candidate, + "details": self.reason, + })), + ) + .into_response() + } +} + +#[derive(Debug)] +struct CheckpointFailure { + stage: &'static str, + operation: String, + code: &'static str, + workspace: String, + path: Option<String>, + filesystem_kind: Option<String>, + errno: Option<i32>, + git_exit_status: Option<i32>, + cause: String, + user_details: &'static str, +} + +impl CheckpointFailure { + fn from_capture_detail(home: &Path, workspace: &Path, detail: &str) -> Self { + let mut operation = "capture".to_string(); + let mut path = None; + let mut cause = detail; + if let Some(after_prefix) = detail.strip_prefix("checkpoint operation ") { + if let Some((name, after_name)) = after_prefix.split_once(" failed") { + operation = name.to_string(); + if let Some(after_for) = after_name.strip_prefix(" for ") { + if let Some((path_value, cause_value)) = after_for.split_once(": ") { + path = Some(path_value.to_string()); + cause = cause_value; + } + } else if let Some(cause_value) = after_name.strip_prefix(": ") { + cause = cause_value; + } + } + } + if let Some(after_prefix) = detail.strip_prefix("Git operation ") { + if let Some((name, after_name)) = after_prefix.split_once(" failed") { + operation = name.to_string(); + cause = + after_name.trim_start_matches(|character| character == ' ' || character == ':'); + } + } else if detail.starts_with("git index update failed") { + operation = "git-index-update".to_string(); + cause = detail; + } else if detail.starts_with("failed to read checkpoint metadata") { + operation = "read-metadata".to_string(); + cause = detail; + } else if detail.starts_with("failed to persist checkpoint metadata") { + operation = "write-metadata".to_string(); + cause = detail; + } + + if path.is_none() && matches!(operation.as_str(), "read-metadata" | "write-metadata") { + path = Some( + checkpoint_metadata_path(&checkpoint_git_dir(home, workspace)) + .to_string_lossy() + .into_owned(), + ); + } + + if path.is_none() { + path = git_open_failure_path(detail); + } + + let git_exit_status = cause + .split_once("exit_status=") + .and_then(|(_, rest)| { + rest.strip_prefix("Some(") + .unwrap_or(rest) + .split(|ch: char| !ch.is_ascii_digit() && ch != '-') + .next() + }) + .and_then(|value| value.parse::<i32>().ok()); + let lowered_cause = cause.to_ascii_lowercase(); + let filesystem_kind = if lowered_cause.contains("permission denied") { + Some("PermissionDenied".to_string()) + } else if lowered_cause.contains("is a directory") { + Some("IsADirectory".to_string()) + } else { + None + }; + let errno = cause + .split_once("os error ") + .and_then(|(_, rest)| { + rest.split(|ch: char| !ch.is_ascii_digit() && ch != '-') + .next() + }) + .and_then(|value| value.parse::<i32>().ok()); + let permission_denied = cause.to_ascii_lowercase().contains("permission denied"); + let unreadable = operation == "verify-readable" + || operation == "verify-symlink" + || (permission_denied && path.is_some()); + Self { + stage: if unreadable { "readability" } else { "capture" }, + operation, + code: if unreadable { + "checkpoint_path_unreadable" + } else { + "checkpoint_capture_failed" + }, + workspace: workspace.to_string_lossy().into_owned(), + path, + filesystem_kind, + errno, + git_exit_status, + cause: bounded_log_detail(cause), + user_details: if unreadable { + "A selected project file could not be read, so this code checkpoint is unavailable." + } else { + "The project code checkpoint could not be saved." + }, + } + } + + fn log(&self, send_request_id: Option<&str>) { + if self.code == "checkpoint_path_unreadable" { + tracing::warn!( + target: "freshell_server::checkpoints", + event = "code_checkpoint.failed", + send_request_id = %send_request_id.unwrap_or(""), + stage = self.stage, + operation = %self.operation, + failure_code = self.code, + workspace = %self.workspace, + path = %self.path.as_deref().unwrap_or(""), + filesystem_kind = %self.filesystem_kind.as_deref().unwrap_or(""), + errno = self.errno.unwrap_or_default(), + git_exit_status = self.git_exit_status.unwrap_or_default(), + cause = %self.cause, + "code checkpoint failed" + ); + } else { + tracing::error!( + target: "freshell_server::checkpoints", + event = "code_checkpoint.failed", + send_request_id = %send_request_id.unwrap_or(""), + stage = self.stage, + operation = %self.operation, + failure_code = self.code, + workspace = %self.workspace, + path = %self.path.as_deref().unwrap_or(""), + filesystem_kind = %self.filesystem_kind.as_deref().unwrap_or(""), + errno = self.errno.unwrap_or_default(), + git_exit_status = self.git_exit_status.unwrap_or_default(), + cause = %self.cause, + "code checkpoint failed" + ); + } + } + + fn response(&self) -> Response { + ( + StatusCode::INTERNAL_SERVER_ERROR, + Json(json!({ + "error": "Code checkpoint unavailable", + "code": self.code, + "details": self.user_details, + "workspace": self.workspace, + })), + ) + .into_response() + } +} + +fn git_open_failure_path(detail: &str) -> Option<String> { + let (start, marker) = ["fatal: open(", "error: open("] + .iter() + .find_map(|marker| detail.find(marker).map(|start| (start, *marker)))?; + let start = start + marker.len(); + let remainder = &detail[start..]; + let (path, suffix) = match remainder.as_bytes().first().copied() { + Some(b'\'') => { + let end = remainder[1..].find("'):")? + 1; + (&remainder[1..end], &remainder[end + 3..]) + } + Some(b'\"') => { + let end = remainder[1..].find("\"):")? + 1; + (&remainder[1..end], &remainder[end + 3..]) + } + _ => { + let end = remainder.find("):")?; + (&remainder[..end], &remainder[end + 2..]) + } + }; + let path = Path::new(path); + if path.as_os_str().is_empty() + || path.is_absolute() + || path.components().any(|component| { + matches!( + component, + std::path::Component::ParentDir + | std::path::Component::RootDir + | std::path::Component::Prefix(_) + ) + }) + || !suffix.to_ascii_lowercase().contains("permission denied") + { + return None; + } + Some(path.to_string_lossy().into_owned()) +} + +fn bounded_log_detail(detail: &str) -> String { + detail + .chars() + .filter(|character| !character.is_control() || *character == ' ') + .take(240) + .collect::<String>() + .split_whitespace() + .collect::<Vec<_>>() + .join(" ") +} + +fn safe_send_request_id(request_id: Option<&str>) -> Option<&str> { + request_id.filter(|value| { + !value.is_empty() + && value.len() <= 128 + && value.bytes().all(|byte| { + byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.' | b':') + }) + }) +} + +fn log_workspace_unavailable(candidate: &str, cause: &str, send_request_id: Option<&str>) { + tracing::warn!( + target: "freshell_server::checkpoints", + event = "code_checkpoint.failed", + send_request_id = %send_request_id.unwrap_or(""), + stage = "workspace-resolution", + operation = "resolve-workspace", + failure_code = "workspace_unavailable", + workspace = candidate, + path = candidate, + cause = %bounded_log_detail(cause), + "code checkpoint workspace unavailable" + ); +} + +fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { + let mut homes = Vec::new(); + let mut add_existing = |candidate: &Path| { + if let Ok(canonical) = std::fs::canonicalize(candidate) { + if !homes.contains(&canonical) { + homes.push(canonical); + } + } + }; + if let Some(server_home) = server_home { + add_existing(server_home); + } + for variable in ["HOME", "USERPROFILE"] { + if let Some(value) = std::env::var_os(variable) { + add_existing(Path::new(&value)); + } + } + #[cfg(unix)] + if let Ok(passwd) = std::fs::read_to_string("/etc/passwd") { + for line in passwd.lines() { + let fields = line.split(':').collect::<Vec<_>>(); + if fields.len() >= 6 && !fields[5].is_empty() { + add_existing(Path::new(fields[5])); + } + } + } + #[cfg(windows)] + if let (Some(drive), Some(path)) = (std::env::var_os("HOMEDRIVE"), std::env::var_os("HOMEPATH")) + { + add_existing(&PathBuf::from(drive).join(path)); + } + homes +} + +pub fn is_os_account_home(candidate: &Path) -> bool { + let Ok(candidate) = std::fs::canonicalize(candidate) else { + return false; + }; + is_recognized_account_home(&candidate, &recognized_account_homes(None)) +} + +fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { + homes.iter().any(|home| home == path) +} + +async fn canonical_workspace(path: &Path) -> Result<PathBuf, String> { + let metadata = tokio::fs::metadata(path) + .await + .map_err(|error| format!("workspace is not accessible: {error}"))?; + if !metadata.is_dir() { + return Err("workspace is not a directory".to_string()); + } + tokio::fs::canonicalize(path) + .await + .map_err(|error| format!("workspace cannot be resolved: {error}")) +} + +async fn git_checkout_root(path: &Path) -> Option<PathBuf> { + let output = run_git(&["rev-parse", "--show-toplevel"], Some(path)) + .await + .ok()?; + tokio::fs::canonicalize(output.trim()).await.ok() +} + +async fn resolve_checkpoint_workspace( + server_home: &Path, + launch_cwd: &Path, + intent: Option<&Value>, +) -> Result<ResolvedCheckpointWorkspace, CheckpointWorkspaceUnavailable> { + let candidate = intent + .and_then(|workspace| workspace.get("path")) + .and_then(Value::as_str) + .map(str::to_string) + .unwrap_or_else(|| launch_cwd.to_string_lossy().into_owned()); + let homes = recognized_account_homes(Some(server_home)); + if let Some(workspace) = intent { + let provenance = workspace + .get("provenance") + .and_then(Value::as_str) + .filter(|value| { + matches!( + *value, + "directory-picker" | "resolved-project" | "legacy-checkpoint" + ) + }) + .map(str::to_string) + .ok_or_else(|| CheckpointWorkspaceUnavailable { + candidate: candidate.clone(), + reason: "workspace selection has no recognized provenance".to_string(), + })?; + let selected = canonical_workspace(Path::new(&candidate)) + .await + .map_err(|reason| CheckpointWorkspaceUnavailable { + candidate: candidate.clone(), + reason, + })?; + if is_recognized_account_home(&selected, &homes) { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home cannot be used as a code checkpoint workspace".to_string(), + }); + } + let path = if provenance == "legacy-checkpoint" { + selected + } else { + git_checkout_root(&selected).await.unwrap_or(selected) + }; + if is_recognized_account_home(&path, &homes) { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home cannot be used as an inferred code checkpoint workspace" + .to_string(), + }); + } + return Ok(ResolvedCheckpointWorkspace { path, provenance }); + } + + let launch = + canonical_workspace(launch_cwd) + .await + .map_err(|reason| CheckpointWorkspaceUnavailable { + candidate: candidate.clone(), + reason, + })?; + if is_recognized_account_home(&launch, &homes) { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home has no selected code checkpoint workspace".to_string(), + }); + } + let root = git_checkout_root(&launch) + .await + .ok_or_else(|| CheckpointWorkspaceUnavailable { + candidate: candidate.clone(), + reason: "no explicit workspace or Git project root is available".to_string(), + })?; + let path = root; + if is_recognized_account_home(&path, &homes) { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home cannot be used as an inferred code checkpoint workspace" + .to_string(), + }); + } + Ok(ResolvedCheckpointWorkspace { + path, + provenance: "resolved-project".to_string(), + }) +} + +fn workspace_response_value(workspace: &ResolvedCheckpointWorkspace) -> Value { + json!({ + "path": workspace.path.to_string_lossy(), + "provenance": workspace.provenance, + }) +} + /// The `/api/fresh-agent/checkpoints` sub-router. `GET` (list) and `POST` /// (create) share the one `/checkpoints` path; `restore` and `metadata` are /// their own POST-only sibling paths (`fresh-agent-extras-router.ts:230-232`). @@ -147,17 +549,32 @@ async fn post_checkpoint( .map(str::to_string); let cwd_path = Path::new(&cwd); - if tokio::fs::metadata(cwd_path).await.is_err() { - return bad_request(&format!("cwd does not exist: {cwd}")); - } + let send_request_id = safe_send_request_id(request_id.as_deref()); + let workspace = + match resolve_checkpoint_workspace(&state.home, cwd_path, body.get("workspace")).await { + Ok(workspace) => workspace, + Err(error) => { + log_workspace_unavailable(&error.candidate, &error.reason, send_request_id); + return error.response(); + } + }; - match create_checkpoint(&state.home, cwd_path, &label, request_id.as_deref()).await { - Ok(entry) => Json(entry).into_response(), - Err(message) => ( - StatusCode::INTERNAL_SERVER_ERROR, - Json(json!({ "error": message })), - ) - .into_response(), + match create_checkpoint(&state.home, &workspace.path, &label, request_id.as_deref()).await { + Ok(mut entry) => { + if let Some(fields) = entry.as_object_mut() { + fields.insert( + "workspace".to_string(), + workspace_response_value(&workspace), + ); + } + Json(entry).into_response() + } + Err(message) => { + let failure = + CheckpointFailure::from_capture_detail(&state.home, &workspace.path, &message); + failure.log(send_request_id); + failure.response() + } } } @@ -181,20 +598,109 @@ async fn create_checkpoint( let git_dir_arg = format!("--git-dir={}", git_dir.display()); let work_tree_arg = format!("--work-tree={}", cwd.display()); - run_git(&[&git_dir_arg, &work_tree_arg, "add", "-A"], Some(cwd)).await?; + let index_path = git_dir.join("index"); + let request_index = git_dir.join(format!("index.tmp-{}", uuid::Uuid::new_v4())); + let old_head = run_git(&[&git_dir_arg, "rev-parse", "--verify", "HEAD"], None) + .await + .ok() + .map(|head| head.trim().to_string()); + + if old_head.is_some() { + if tokio::fs::copy(&index_path, &request_index).await.is_err() { + let tree = run_git(&[&git_dir_arg, "rev-parse", "HEAD^{tree}"], None) + .await? + .trim() + .to_string(); + run_git_with_index(&[&git_dir_arg, "read-tree", &tree], None, &request_index).await?; + } else { + let cached_tree = + run_git_with_index(&[&git_dir_arg, "write-tree"], None, &request_index) + .await + .ok() + .map(|tree| tree.trim().to_string()); + let published_tree = run_git(&[&git_dir_arg, "rev-parse", "HEAD^{tree}"], None) + .await + .ok() + .map(|tree| tree.trim().to_string()); + if cached_tree != published_tree { + let _ = tokio::fs::remove_file(&request_index).await; + let tree = published_tree.ok_or_else(|| { + "checkpoint operation resolve-published-tree failed: current HEAD is unavailable" + .to_string() + })?; + run_git_with_index(&[&git_dir_arg, "read-tree", &tree], None, &request_index) + .await?; + } + } + } else { + run_git_with_index( + &[&git_dir_arg, "read-tree", "--empty"], + None, + &request_index, + ) + .await?; + } + + let capture = async { + verify_readable_directories(&git_dir, cwd).await?; + run_git_with_index( + &[&git_dir_arg, &work_tree_arg, "add", "-A", "--", "."], + Some(cwd), + &request_index, + ) + .await?; + expand_nested_repositories(&git_dir, cwd, &request_index).await?; + verify_checkpoint_index(&git_dir, cwd, &request_index).await?; + + let tree = run_git_with_index(&[&git_dir_arg, "write-tree"], None, &request_index) + .await? + .trim() + .to_string(); + let mut commit_args: Vec<&str> = vec![git_dir_arg.as_str()]; + commit_args.extend(CHECKPOINT_IDENTITY); + commit_args.extend(["commit-tree", tree.as_str()]); + if let Some(parent) = old_head.as_deref() { + commit_args.extend(["-p", parent]); + } + commit_args.extend(["-m", label]); + let sha = run_git(&commit_args, None).await?.trim().to_string(); - let mut commit_args: Vec<&str> = vec![&git_dir_arg, &work_tree_arg]; - commit_args.extend(CHECKPOINT_IDENTITY); - commit_args.extend(["commit", "--allow-empty", "-q", "-m", label]); - run_git(&commit_args, Some(cwd)).await?; + if let Some(request_id) = request_id { + update_checkpoint_metadata(&git_dir, &sha, request_id).await?; + } - let sha = run_git(&[&git_dir_arg, "rev-parse", "HEAD"], None) - .await? - .trim() - .to_string(); + let head_ref = run_git(&[&git_dir_arg, "symbolic-ref", "HEAD"], None) + .await? + .trim() + .to_string(); + let mut update_ref = vec![&git_dir_arg, "update-ref", head_ref.as_str(), sha.as_str()]; + if let Some(parent) = old_head.as_deref() { + update_ref.push(parent); + } + run_git(&update_ref, None).await?; - if let Some(request_id) = request_id { - update_checkpoint_metadata(&git_dir, &sha, request_id).await?; + Ok::<String, String>(sha) + } + .await; + + let sha = match capture { + Ok(sha) => sha, + Err(error) => { + let _ = tokio::fs::remove_file(&request_index).await; + return Err(error); + } + }; + + // The published tree is already complete. Losing this disposable stat cache + // can make the next request slower, but cannot hide or invalidate a target. + if let Err(error) = install_checkpoint_index(&request_index, &index_path).await { + tracing::warn!( + event = "code_checkpoint.index_cache_install_failed", + workspace = %cwd.display(), + cause = %error, + "checkpoint index cache could not be retained" + ); + let _ = tokio::fs::remove_file(&request_index).await; } let ts = SystemTime::now() @@ -216,11 +722,15 @@ async fn create_checkpoint( /// bare shadow repo on first use for this `cwd`, otherwise reuse it. async fn ensure_checkpoint_repo(home: &Path, cwd: &Path) -> Result<PathBuf, String> { let git_dir = checkpoint_git_dir(home, cwd); - if tokio::fs::metadata(git_dir.join("HEAD")).await.is_err() { - tokio::fs::create_dir_all(&git_dir) - .await - .map_err(|e| format!("failed to create checkpoint dir: {e}"))?; - run_git(&["init", "--bare", "-q", &git_dir.to_string_lossy()], None).await?; + match tokio::fs::metadata(git_dir.join("HEAD")).await { + Ok(_) => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + tokio::fs::create_dir_all(&git_dir) + .await + .map_err(|e| format!("failed to create checkpoint dir: {e}"))?; + run_git(&["init", "--bare", "-q", &git_dir.to_string_lossy()], None).await?; + } + Err(error) => return Err(format!("failed to inspect checkpoint repository: {error}")), } Ok(git_dir) } @@ -281,36 +791,703 @@ fn hex_encode(bytes: &[u8]) -> String { out } -/// `runGit` (`fresh-agent-extras-router.ts:80-94`): spawn the real `git` binary, -/// reject on non-zero exit with a legacy-shaped error message (`git <args[0]> -/// failed: <stderr>`) — legacy's `args[0]` is often a leading global flag (e.g. -/// `--git-dir=...`) rather than the subcommand name, since the identity/`--git- -/// dir`/`--work-tree` flags precede the subcommand on this router's git -/// invocations; mirrored verbatim rather than "fixed" since this message is -/// never read by the client (the one call site `.catch(() => {})`s every error). +/// Run Git without exposing the shadow-store flags or full argv in diagnostics. +/// Failures retain a safe operation name, exit status, and bounded stderr cause. async fn run_git(args: &[&str], cwd: Option<&Path>) -> Result<String, String> { + let output = invoke_git(args, cwd, None).await?; + check_git_output(args, output) +} + +async fn run_git_with_index( + args: &[&str], + cwd: Option<&Path>, + index: &Path, +) -> Result<String, String> { + let output = invoke_git(args, cwd, Some(index)).await?; + check_git_output(args, output) +} + +async fn run_git_bytes(args: &[&str], cwd: Option<&Path>) -> Result<Vec<u8>, String> { + let output = invoke_git(args, cwd, None).await?; + check_git_output_bytes(args, output) +} + +async fn run_git_with_index_bytes( + args: &[&str], + cwd: Option<&Path>, + index: &Path, +) -> Result<Vec<u8>, String> { + let output = invoke_git(args, cwd, Some(index)).await?; + check_git_output_bytes(args, output) +} + +async fn run_git_with_index_input( + args: &[&str], + cwd: Option<&Path>, + index: &Path, + input: &[u8], +) -> Result<(), String> { + let mut cmd = Command::new("git"); + cmd.args(args).stdin(Stdio::piped()); + if let Some(dir) = cwd { + cmd.current_dir(dir); + } + cmd.env("GIT_INDEX_FILE", index); + let mut child = cmd + .spawn() + .map_err(|error| format!("git update-index failed to start: {error}"))?; + if let Some(mut stdin) = child.stdin.take() { + if let Err(error) = stdin.write_all(input).await { + let _ = child.kill().await; + return Err(format!( + "git update-index failed to write path list: {error}" + )); + } + } + let output = child + .wait_with_output() + .await + .map_err(|error| format!("git update-index failed to wait: {error}"))?; + check_git_output_bytes(args, output).map(|_| ()) +} + +async fn ignored_git_directories( + git_dir: &Path, + work_tree: &Path, + directory_paths: &[Vec<u8>], +) -> Result<std::collections::HashSet<Vec<u8>>, String> { + if directory_paths.is_empty() { + return Ok(std::collections::HashSet::new()); + } + + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let work_tree_arg = format!("--work-tree={}", work_tree.display()); + let mut child = Command::new("git") + .arg(git_dir_arg) + .arg(work_tree_arg) + .args(["check-ignore", "--no-index", "--stdin", "-z"]) + .current_dir(work_tree) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .map_err(|error| format!("git check-ignore failed to start: {error}"))?; + let mut input = Vec::new(); + for path in directory_paths { + input.extend_from_slice(path); + input.push(0); + } + if let Some(mut stdin) = child.stdin.take() { + if let Err(error) = stdin.write_all(&input).await { + let _ = child.kill().await; + return Err(format!("git check-ignore failed to write paths: {error}")); + } + } + let output = child + .wait_with_output() + .await + .map_err(|error| format!("git check-ignore failed to wait: {error}"))?; + if !output.status.success() && output.status.code() != Some(1) { + return Err(format!( + "Git operation check-ignore failed (exit_status={:?}): {}", + output.status.code(), + String::from_utf8_lossy(&output.stderr).trim() + )); + } + Ok(output + .stdout + .split(|byte| *byte == 0) + .filter(|path| !path.is_empty()) + .map(<[u8]>::to_vec) + .collect()) +} + +fn git_path_bytes(path: &Path) -> Vec<u8> { + #[cfg(unix)] + { + use std::os::unix::ffi::OsStrExt; + path.as_os_str().as_bytes().to_vec() + } + #[cfg(windows)] + { + path.to_string_lossy().as_bytes().to_vec() + } +} + +fn display_relative_path(path: &Path) -> String { + if path.as_os_str().is_empty() { + ".".to_string() + } else { + path.to_string_lossy().into_owned() + } +} + +async fn verify_readable_directories(git_dir: &Path, work_tree: &Path) -> Result<(), String> { + let mut pending = vec![PathBuf::new()]; + while let Some(relative_parent) = pending.pop() { + let absolute_parent = work_tree.join(&relative_parent); + let mut entries = tokio::fs::read_dir(&absolute_parent) + .await + .map_err(|error| { + format!( + "checkpoint operation verify-readable failed for {}: {error}", + display_relative_path(&relative_parent) + ) + })?; + let mut candidates = Vec::new(); + while let Some(entry) = entries.next_entry().await.map_err(|error| { + format!( + "checkpoint operation verify-readable failed for {}: {error}", + display_relative_path(&relative_parent) + ) + })? { + if entry.file_name() == ".git" { + continue; + } + let file_type = entry.file_type().await.map_err(|error| { + let relative = relative_parent.join(entry.file_name()); + format!( + "checkpoint operation verify-readable failed for {}: {error}", + display_relative_path(&relative) + ) + })?; + if !file_type.is_dir() { + continue; + } + let relative = relative_parent.join(entry.file_name()); + let mut ignore_path = git_path_bytes(&relative); + ignore_path.push(b'/'); + candidates.push((relative, ignore_path)); + } + + let candidate_paths = candidates + .iter() + .map(|(_, path)| path.clone()) + .collect::<Vec<_>>(); + let ignored = ignored_git_directories(git_dir, work_tree, &candidate_paths).await?; + for (relative, ignore_path) in candidates { + if !ignored.contains(&ignore_path) { + pending.push(relative); + } + } + } + Ok(()) +} + +async fn invoke_git( + args: &[&str], + cwd: Option<&Path>, + index: Option<&Path>, +) -> Result<std::process::Output, String> { let mut cmd = Command::new("git"); cmd.args(args); if let Some(dir) = cwd { cmd.current_dir(dir); } - let first_arg = args.first().copied().unwrap_or(""); - let output = cmd - .output() + if let Some(index) = index { + cmd.env("GIT_INDEX_FILE", index); + } + cmd.output() .await - .map_err(|e| format!("git {first_arg} failed: {e}"))?; + .map_err(|e| format!("Git process failed to start: {e}")) +} + +fn check_git_output(args: &[&str], output: std::process::Output) -> Result<String, String> { if !output.status.success() { let stderr = String::from_utf8_lossy(&output.stderr); + let operation = git_operation_label(args); let message = if stderr.trim().is_empty() { - format!("git {first_arg} failed: exit status {}", output.status) + format!( + "Git operation {operation} failed (exit_status={:?})", + output.status.code() + ) } else { - format!("git {first_arg} failed: {}", stderr.trim()) + format!( + "Git operation {operation} failed (exit_status={:?}): {}", + output.status.code(), + stderr.trim() + ) }; return Err(message); } Ok(String::from_utf8_lossy(&output.stdout).into_owned()) } +fn check_git_output_bytes(args: &[&str], output: std::process::Output) -> Result<Vec<u8>, String> { + if !output.status.success() { + let stderr = String::from_utf8_lossy(&output.stderr); + let operation = git_operation_label(args); + let message = if stderr.trim().is_empty() { + format!( + "Git operation {operation} failed (exit_status={:?})", + output.status.code() + ) + } else { + format!( + "Git operation {operation} failed (exit_status={:?}): {}", + output.status.code(), + stderr.trim() + ) + }; + return Err(message); + } + Ok(output.stdout) +} + +fn git_operation_label(args: &[&str]) -> &'static str { + args.iter() + .rev() + .find_map(|argument| match *argument { + "add" => Some("add"), + "check-ignore" => Some("check-ignore"), + "read-tree" => Some("read-tree"), + "write-tree" => Some("write-tree"), + "commit-tree" => Some("commit-tree"), + "update-ref" => Some("update-ref"), + "ls-files" => Some("ls-files"), + "rev-parse" => Some("rev-parse"), + "symbolic-ref" => Some("symbolic-ref"), + "merge-base" => Some("merge-base"), + "log" => Some("log"), + "checkout" => Some("checkout"), + "init" => Some("init"), + "config" => Some("config"), + _ => None, + }) + .unwrap_or("other") +} + +#[derive(Clone, Debug)] +struct GitIndexEntry { + mode: Vec<u8>, + path: Vec<u8>, +} + +fn parse_git_index_entries(raw: &[u8]) -> Result<Vec<GitIndexEntry>, String> { + raw.split(|byte| *byte == 0) + .filter(|record| !record.is_empty()) + .map(|record| { + let separator = record + .iter() + .position(|byte| *byte == b'\t') + .ok_or_else(|| { + "checkpoint operation inspect-index failed: malformed Git index row".to_string() + })?; + let mode = record[..separator] + .split(|byte| byte.is_ascii_whitespace()) + .next() + .unwrap_or_default() + .to_vec(); + Ok(GitIndexEntry { + mode, + path: record[separator + 1..].to_vec(), + }) + }) + .collect() +} + +async fn checkpoint_index_entries( + git_dir: &Path, + index: &Path, +) -> Result<Vec<GitIndexEntry>, String> { + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let raw = + run_git_with_index_bytes(&[&git_dir_arg, "ls-files", "--stage", "-z"], None, index).await?; + parse_git_index_entries(&raw) +} + +#[cfg(unix)] +fn git_relative_path(path: &[u8]) -> PathBuf { + use std::os::unix::ffi::OsStrExt; + PathBuf::from(std::ffi::OsStr::from_bytes(path)) +} + +#[cfg(windows)] +fn git_relative_path(path: &[u8]) -> PathBuf { + PathBuf::from(String::from_utf8_lossy(path).as_ref()) +} + +fn git_path_ancestors(path: &[u8]) -> Vec<Vec<u8>> { + let mut ancestors = Vec::new(); + for (index, byte) in path.iter().enumerate() { + if *byte == b'/' && index > 0 { + ancestors.push(path[..index].to_vec()); + } + } + ancestors +} + +fn join_git_path(prefix: &[u8], relative: &[u8]) -> Vec<u8> { + let mut joined = Vec::with_capacity(prefix.len() + 1 + relative.len()); + joined.extend_from_slice(prefix); + joined.push(b'/'); + joined.extend_from_slice(relative); + joined +} + +fn git_path_is_at_or_below(path: &[u8], prefix: &[u8]) -> bool { + path == prefix || (path.starts_with(prefix) && path.get(prefix.len()) == Some(&b'/')) +} + +async fn directory_has_git_marker(work_tree: &Path, relative: &[u8]) -> bool { + let path = work_tree.join(git_relative_path(relative)); + let metadata = tokio::fs::symlink_metadata(&path).await; + if !metadata.is_ok_and(|metadata| metadata.is_dir() && !metadata.file_type().is_symlink()) { + return false; + } + tokio::fs::symlink_metadata(path.join(".git")).await.is_ok() +} + +async fn nested_git_roots( + repo_work_tree: &Path, + staged: &[GitIndexEntry], + selected_paths: &[Vec<u8>], +) -> Vec<Vec<u8>> { + let mut candidates = Vec::new(); + for entry in staged { + if entry.mode == b"160000" { + candidates.push(entry.path.clone()); + } + for ancestor in git_path_ancestors(&entry.path) { + if directory_has_git_marker(repo_work_tree, &ancestor).await { + candidates.push(ancestor); + } + } + } + for path in selected_paths { + for ancestor in git_path_ancestors(path) { + if directory_has_git_marker(repo_work_tree, &ancestor).await { + candidates.push(ancestor); + } + } + } + candidates.sort(); + candidates.dedup(); + candidates +} + +async fn collect_nested_snapshot_paths( + nested_root: &Path, + index_prefix: &[u8], + files: &mut Vec<Vec<u8>>, +) -> Result<(), String> { + let mut pending = vec![(nested_root.to_path_buf(), index_prefix.to_vec(), 0usize)]; + while let Some((repo_root, prefix, depth)) = pending.pop() { + if depth > 64 { + return Err(format!( + "checkpoint operation enumerate-nested-repository failed for {}: nested repository depth exceeds 64", + String::from_utf8_lossy(&prefix) + )); + } + let root_metadata = tokio::fs::symlink_metadata(&repo_root) + .await + .map_err(|error| { + format!( + "checkpoint operation enumerate-nested-repository failed for {}: {error}", + String::from_utf8_lossy(&prefix) + ) + })?; + if !root_metadata.is_dir() || root_metadata.file_type().is_symlink() { + return Err(format!( + "checkpoint operation enumerate-nested-repository failed for {}: expected a populated repository directory", + String::from_utf8_lossy(&prefix) + )); + } + + let top = run_git(&["rev-parse", "--show-toplevel"], Some(&repo_root)) + .await + .map_err(|error| { + format!( + "checkpoint operation enumerate-nested-repository failed for {}: {error}", + String::from_utf8_lossy(&prefix) + ) + })?; + let canonical_root = tokio::fs::canonicalize(&repo_root).await.map_err(|error| { + format!( + "checkpoint operation enumerate-nested-repository failed for {}: {error}", + String::from_utf8_lossy(&prefix) + ) + })?; + let canonical_git_root = tokio::fs::canonicalize(top.trim()).await.map_err(|error| { + format!( + "checkpoint operation enumerate-nested-repository failed for {}: {error}", + String::from_utf8_lossy(&prefix) + ) + })?; + if canonical_root != canonical_git_root { + return Err(format!( + "checkpoint operation enumerate-nested-repository failed for {}: Git resolves this path to a different repository root", + String::from_utf8_lossy(&prefix) + )); + } + + let staged_raw = run_git_bytes(&["ls-files", "--stage", "-z"], Some(&repo_root)) + .await + .map_err(|error| { + format!( + "checkpoint operation enumerate-nested-repository failed for {}: {error}", + String::from_utf8_lossy(&prefix) + ) + })?; + let staged = parse_git_index_entries(&staged_raw)?; + let selected_raw = run_git_bytes( + &[ + "ls-files", + "--cached", + "--others", + "--exclude-standard", + "-z", + ], + Some(&repo_root), + ) + .await + .map_err(|error| { + format!( + "checkpoint operation enumerate-nested-repository failed for {}: {error}", + String::from_utf8_lossy(&prefix) + ) + })?; + let selected = selected_raw + .split(|byte| *byte == 0) + .filter(|path| !path.is_empty()) + .map(<[u8]>::to_vec) + .collect::<Vec<_>>(); + let child_roots = nested_git_roots(&repo_root, &staged, &selected).await; + + for path in &selected { + if child_roots + .iter() + .any(|child| git_path_is_at_or_below(path, child)) + { + continue; + } + files.push(join_git_path(&prefix, path)); + } + + for child in child_roots.into_iter().rev() { + let child_prefix = join_git_path(&prefix, &child); + pending.push(( + repo_root.join(git_relative_path(&child)), + child_prefix, + depth + 1, + )); + } + } + Ok(()) +} + +async fn expand_nested_repositories( + git_dir: &Path, + work_tree: &Path, + index: &Path, +) -> Result<(), String> { + let staged = checkpoint_index_entries(git_dir, index).await?; + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let work_tree_arg = format!("--work-tree={}", work_tree.display()); + let selected_raw = run_git_with_index_bytes( + &[ + &git_dir_arg, + &work_tree_arg, + "ls-files", + "--cached", + "--others", + "--exclude-standard", + "-z", + ], + Some(work_tree), + index, + ) + .await?; + let selected = selected_raw + .split(|byte| *byte == 0) + .filter(|path| !path.is_empty()) + .map(<[u8]>::to_vec) + .collect::<Vec<_>>(); + let mut roots = nested_git_roots(work_tree, &staged, &selected).await; + if let Ok(native_raw) = run_git_bytes(&["ls-files", "--stage", "-z"], Some(work_tree)).await { + if let Ok(native_entries) = parse_git_index_entries(&native_raw) { + roots.extend( + native_entries + .into_iter() + .filter(|entry| entry.mode == b"160000") + .map(|entry| entry.path), + ); + } + } + roots.sort(); + roots.dedup(); + if roots.is_empty() { + return Ok(()); + } + + let mut files = Vec::new(); + for root in &roots { + collect_nested_snapshot_paths(&work_tree.join(git_relative_path(root)), root, &mut files) + .await?; + } + + let root_index_entries = checkpoint_index_entries(git_dir, index).await?; + let removals = root_index_entries + .iter() + .filter(|entry| { + roots + .iter() + .any(|root| git_path_is_at_or_below(&entry.path, root)) + }) + .map(|entry| entry.path.clone()) + .collect::<std::collections::BTreeSet<_>>(); + if !removals.is_empty() { + let mut input = Vec::new(); + for path in removals { + input.extend_from_slice(&path); + input.push(0); + } + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let work_tree_arg = format!("--work-tree={}", work_tree.display()); + run_git_with_index_input( + &[ + &git_dir_arg, + &work_tree_arg, + "update-index", + "-z", + "--force-remove", + "--stdin", + ], + Some(work_tree), + index, + &input, + ) + .await?; + } + + files.sort(); + files.dedup(); + if !files.is_empty() { + let mut input = Vec::new(); + for path in files { + input.extend_from_slice(&path); + input.push(0); + } + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let work_tree_arg = format!("--work-tree={}", work_tree.display()); + run_git_with_index_input( + &[ + &git_dir_arg, + &work_tree_arg, + "update-index", + "--add", + "--remove", + "-z", + "--stdin", + ], + Some(work_tree), + index, + &input, + ) + .await?; + } + Ok(()) +} + +async fn verify_checkpoint_index( + git_dir: &Path, + work_tree: &Path, + index: &Path, +) -> Result<(), String> { + let staged = checkpoint_index_entries(git_dir, index).await?; + for entry in staged { + let mode = entry.mode.as_slice(); + let path = git_relative_path(&entry.path); + if path.is_absolute() + || path.components().any(|component| { + matches!( + component, + std::path::Component::ParentDir + | std::path::Component::RootDir + | std::path::Component::Prefix(_) + ) + }) + { + return Err(format!( + "checkpoint operation inspect-index failed: unsafe captured path {}", + String::from_utf8_lossy(&entry.path) + )); + } + let absolute = work_tree.join(path); + let metadata = tokio::fs::symlink_metadata(&absolute) + .await + .map_err(|error| { + format!( + "checkpoint operation verify-readable failed for {}: {error}", + String::from_utf8_lossy(&entry.path) + ) + })?; + if mode == b"160000" { + return Err(format!( + "checkpoint operation enumerate-nested-repository failed for {}: nested repository content is not captured as ordinary files", + String::from_utf8_lossy(&entry.path) + )); + } + if mode == b"120000" { + if !metadata.file_type().is_symlink() { + return Err(format!( + "checkpoint operation verify-symlink failed for {}: indexed symlink is no longer a symlink", + String::from_utf8_lossy(&entry.path) + )); + } + tokio::fs::read_link(&absolute).await.map_err(|error| { + format!( + "checkpoint operation verify-symlink failed for {}: {error}", + String::from_utf8_lossy(&entry.path) + ) + })?; + continue; + } + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err(format!( + "checkpoint operation verify-readable failed for {}: expected a regular file", + String::from_utf8_lossy(&entry.path) + )); + } + let mut file = tokio::fs::File::open(&absolute).await.map_err(|error| { + format!( + "checkpoint operation verify-readable failed for {}: {error}", + String::from_utf8_lossy(&entry.path) + ) + })?; + let mut buffer = [0u8; 16 * 1024]; + loop { + let count = file.read(&mut buffer).await.map_err(|error| { + format!( + "checkpoint operation verify-readable failed for {}: {error}", + String::from_utf8_lossy(&entry.path) + ) + })?; + if count == 0 { + break; + } + } + } + Ok(()) +} + +async fn install_checkpoint_index( + request_index: &Path, + published_index: &Path, +) -> Result<(), String> { + if tokio::fs::try_exists(published_index) + .await + .unwrap_or(false) + { + tokio::fs::remove_file(published_index) + .await + .map_err(|error| format!("failed to remove stale checkpoint index cache: {error}"))?; + } + tokio::fs::rename(request_index, published_index) + .await + .map_err(|error| format!("failed to install checkpoint index cache: {error}")) +} + // ── checkpoint metadata (fresh-agent-extras-router.ts:107-149) ────────────── fn checkpoint_metadata_path(git_dir: &Path) -> PathBuf { @@ -473,7 +1650,36 @@ async fn resolve_checkpoint_commit(git_dir: &Path, id: &str) -> Result<String, S let git_dir_arg = format!("--git-dir={}", git_dir.display()); let revision = format!("{id}^{{commit}}"); let out = run_git(&[&git_dir_arg, "rev-parse", "--verify", &revision], None).await?; - Ok(out.trim().to_string()) + let resolved = out.trim().to_string(); + let ancestry = invoke_git( + &[ + &git_dir_arg, + "merge-base", + "--is-ancestor", + &resolved, + "HEAD", + ], + None, + None, + ) + .await?; + if ancestry.status.success() { + return Ok(resolved); + } + if ancestry.status.code() == Some(1) { + return Err("invalid checkpoint id: target is not published".to_string()); + } + check_git_output( + &[ + &git_dir_arg, + "merge-base", + "--is-ancestor", + &resolved, + "HEAD", + ], + ancestry, + )?; + unreachable!("failed merge-base command must return an error") } /// `updateCheckpointMetadata` (`fresh-agent-extras-router.ts:151-172`), full @@ -531,6 +1737,7 @@ async fn restore_checkpoint(home: &Path, cwd: &Path, id: &str) -> Result<(), Str return Err("invalid checkpoint id".to_string()); } let git_dir = ensure_checkpoint_repo(home, cwd).await?; + let published_id = resolve_checkpoint_commit(&git_dir, id).await?; let git_dir_arg = format!("--git-dir={}", git_dir.display()); let work_tree_arg = format!("--work-tree={}", cwd.display()); run_git( @@ -539,7 +1746,7 @@ async fn restore_checkpoint(home: &Path, cwd: &Path, id: &str) -> Result<(), Str &work_tree_arg, "checkout", "-q", - id, + &published_id, "--", ".", ], @@ -571,14 +1778,92 @@ async fn get_checkpoints( Some(cwd) => cwd.to_string(), None => return bad_request("cwd query parameter required"), }; - match list_checkpoints(&state.home, Path::new(&cwd)).await { - Ok(entries) => Json(json!({ "checkpoints": entries })).into_response(), - Err(message) => ( - StatusCode::INTERNAL_SERVER_ERROR, - Json(json!({ "error": message })), - ) - .into_response(), + let explicit_intent = params + .get("workspacePath") + .map(String::as_str) + .filter(|path| !path.is_empty()) + .map(|path| { + json!({ + "path": path, + "provenance": params.get("workspaceProvenance") + .map(String::as_str) + .unwrap_or("resolved-project"), + }) + }); + let workspaces = if let Some(intent) = explicit_intent { + match resolve_checkpoint_workspace(&state.home, Path::new(&cwd), Some(&intent)).await { + Ok(workspace) => vec![workspace], + Err(error) => return error.response(), + } + } else { + // Modern panes resolve nested launch directories to the Git project + // root. Old panes can still have an immutable checkpoint store keyed + // by their exact launch cwd, so include that store only when it already + // exists. An unselected non-Git directory or HOME therefore lists as + // empty and can never become an implicit capture scope. + let mut workspaces = Vec::new(); + if let Ok(workspace) = + resolve_checkpoint_workspace(&state.home, Path::new(&cwd), None).await + { + workspaces.push(workspace); + } + + let legacy_path = resolve_absolute(Path::new(&cwd)); + let candidate_for_home_check = tokio::fs::canonicalize(&legacy_path) + .await + .unwrap_or_else(|_| legacy_path.clone()); + if !is_recognized_account_home( + &candidate_for_home_check, + &recognized_account_homes(Some(&state.home)), + ) { + let already_present = workspaces + .iter() + .any(|workspace| workspace.path == legacy_path); + if !already_present + && tokio::fs::metadata(checkpoint_git_dir(&state.home, &legacy_path).join("HEAD")) + .await + .is_ok() + { + workspaces.push(ResolvedCheckpointWorkspace { + path: legacy_path, + provenance: "legacy-checkpoint".to_string(), + }); + } + } + workspaces + }; + + let mut entries = Vec::new(); + for workspace in workspaces { + match list_checkpoints(&state.home, &workspace.path).await { + Ok(mut scoped_entries) => { + for entry in &mut scoped_entries { + if let Some(fields) = entry.as_object_mut() { + fields.insert( + "workspace".to_string(), + workspace_response_value(&workspace), + ); + } + } + entries.extend(scoped_entries); + } + Err(message) => { + return ( + StatusCode::INTERNAL_SERVER_ERROR, + Json(json!({ "error": message })), + ) + .into_response(); + } + } } + entries.sort_by(|left, right| { + right["ts"] + .as_u64() + .unwrap_or_default() + .cmp(&left["ts"].as_u64().unwrap_or_default()) + }); + entries.truncate(CHECKPOINT_LIST_LIMIT as usize); + Json(json!({ "checkpoints": entries })).into_response() } /// `POST /checkpoints/metadata` (`fresh-agent-extras-router.ts:382-408`). @@ -610,13 +1895,20 @@ async fn post_checkpoint_metadata( if request_id.is_none() && turn_id.is_none() { return bad_request("requestId or turnId is required"); } - if tokio::fs::metadata(cwd).await.is_err() { - return bad_request(&format!("cwd does not exist: {cwd}")); - } + let cwd_path = Path::new(cwd); + let workspace = if let Some(intent) = body.get("workspace") { + match resolve_checkpoint_workspace(&state.home, cwd_path, Some(intent)).await { + Ok(workspace) => workspace.path, + Err(error) => return error.response(), + } + } else { + if tokio::fs::metadata(cwd_path).await.is_err() { + return bad_request(&format!("cwd does not exist: {cwd}")); + } + cwd_path.to_path_buf() + }; - match apply_checkpoint_metadata_patch(&state.home, Path::new(cwd), id, request_id, turn_id) - .await - { + match apply_checkpoint_metadata_patch(&state.home, &workspace, id, request_id, turn_id).await { Ok(entry) => Json(entry).into_response(), Err(message) => { let status = if message.contains("invalid checkpoint id") { @@ -644,11 +1936,20 @@ async fn post_checkpoint_restore( if cwd.is_empty() || id.is_empty() { return bad_request("cwd and id are required"); } - if tokio::fs::metadata(cwd).await.is_err() { - return bad_request(&format!("cwd does not exist: {cwd}")); - } + let cwd_path = Path::new(cwd); + let workspace = if let Some(intent) = body.get("workspace") { + match resolve_checkpoint_workspace(&state.home, cwd_path, Some(intent)).await { + Ok(workspace) => workspace.path, + Err(error) => return error.response(), + } + } else { + if tokio::fs::metadata(cwd_path).await.is_err() { + return bad_request(&format!("cwd does not exist: {cwd}")); + } + cwd_path.to_path_buf() + }; - match restore_checkpoint(&state.home, Path::new(cwd), id).await { + match restore_checkpoint(&state.home, &workspace, id).await { Ok(()) => Json(json!({ "restored": true })).into_response(), Err(message) => { let status = if message.contains("invalid checkpoint id") { @@ -715,10 +2016,78 @@ mod tests { assert_eq!(resolved, PathBuf::from("/a/c/d")); } + #[test] + fn git_add_permission_failure_has_structured_path_and_cause() { + let home = tempfile::tempdir().unwrap(); + let failure = CheckpointFailure::from_capture_detail( + home.path(), + Path::new("/workspace/project"), + "Git operation add failed (exit_status=Some(128)): error: open(\"private-file\"): Permission denied\nerror: unable to index file 'private-file'\nfatal: adding files failed", + ); + assert_eq!(failure.stage, "readability"); + assert_eq!(failure.operation, "add"); + assert_eq!(failure.code, "checkpoint_path_unreadable"); + assert_eq!(failure.path.as_deref(), Some("private-file")); + assert_eq!(failure.filesystem_kind.as_deref(), Some("PermissionDenied")); + assert_eq!(failure.git_exit_status, Some(128)); + assert!(failure.cause.contains("Permission denied")); + } + + #[test] + fn git_permission_failure_path_parser_rejects_unsafe_and_non_permission_paths() { + assert_eq!( + git_open_failure_path( + "Git operation add failed (exit_status=128): fatal: open('../outside'): Permission denied" + ), + None + ); + assert_eq!( + git_open_failure_path( + "Git operation add failed (exit_status=128): fatal: open(\"src/file.rs\"): No such file or directory" + ), + None + ); + } + + #[tokio::test] + async fn directory_ignore_probe_honors_workspace_gitignore_with_shadow_repo() { + let work_tree = tempfile::tempdir().unwrap(); + let shadow_parent = tempfile::tempdir().unwrap(); + let git_dir = shadow_parent.path().join("shadow.git"); + tokio::fs::write( + work_tree.path().join(".gitignore"), + "private-dir.ignored/\n", + ) + .await + .unwrap(); + tokio::fs::create_dir(work_tree.path().join("private-dir.ignored")) + .await + .unwrap(); + run_git( + &["init", "--bare", "--quiet", git_dir.to_str().unwrap()], + None, + ) + .await + .unwrap(); + + let ignored = ignored_git_directories( + &git_dir, + work_tree.path(), + &[b"private-dir.ignored/".to_vec()], + ) + .await + .unwrap(); + assert!(ignored.contains(b"private-dir.ignored/".as_slice())); + } + // ── route-level tests ──────────────────────────────────────────────── fn valid_body(cwd: &str) -> Value { - json!({ "cwd": cwd, "label": "fix the thing" }) + json!({ + "cwd": cwd, + "workspace": { "path": cwd, "provenance": "directory-picker" }, + "label": "fix the thing" + }) } #[tokio::test] @@ -769,12 +2138,9 @@ mod tests { Json(valid_body(missing.to_str().unwrap())), ) .await; - assert_eq!(resp.status(), StatusCode::BAD_REQUEST); + assert_eq!(resp.status(), StatusCode::UNPROCESSABLE_ENTITY); let value = body_json(resp).await; - assert!(value["error"] - .as_str() - .unwrap() - .starts_with("cwd does not exist:")); + assert_eq!(value["code"], json!("workspace_unavailable")); } #[tokio::test] @@ -796,6 +2162,469 @@ mod tests { assert!(value.get("requestId").is_none()); } + #[tokio::test] + async fn explicit_non_git_picker_scope_restores() { + let home_dir = tempfile::tempdir().unwrap(); + let project_dir = tempfile::tempdir().unwrap(); + let source = project_dir.path().join("source.txt"); + tokio::fs::write(&source, "before").await.unwrap(); + let workspace = json!({ + "path": project_dir.path(), + "provenance": "directory-picker", + }); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": home_dir.path(), + "workspace": workspace, + "label": "before turn", + "requestId": "scope-request", + })), + ) + .await; + let status = response.status(); + let entry = body_json(response).await; + assert_eq!(status, StatusCode::OK, "{entry}"); + assert_eq!(entry["workspace"]["path"], json!(project_dir.path())); + assert_eq!(entry["workspace"]["provenance"], json!("directory-picker")); + + tokio::fs::write(&source, "after").await.unwrap(); + let listed = get_checkpoints( + State(state(home_dir.path())), + headers_with_token("tok"), + Query(HashMap::from([ + ( + "cwd".to_string(), + home_dir.path().to_string_lossy().into_owned(), + ), + ( + "workspacePath".to_string(), + project_dir.path().to_string_lossy().into_owned(), + ), + ( + "workspaceProvenance".to_string(), + "directory-picker".to_string(), + ), + ])), + ) + .await; + let listed = body_json(listed).await; + assert_eq!(listed["checkpoints"][0]["id"], entry["id"]); + + let restored = post_checkpoint_restore( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": home_dir.path(), + "workspace": workspace, + "id": entry["id"], + })), + ) + .await; + assert_eq!(restored.status(), StatusCode::OK); + assert_eq!(tokio::fs::read_to_string(source).await.unwrap(), "before"); + } + + #[tokio::test] + async fn home_launch_without_workspace_does_not_publish_checkpoint() { + let home_dir = tempfile::tempdir().unwrap(); + tokio::fs::write(home_dir.path().join("private.txt"), "private") + .await + .unwrap(); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ "cwd": home_dir.path() })), + ) + .await; + + assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY); + let error = body_json(response).await; + assert_eq!(error["code"], json!("workspace_unavailable")); + assert_eq!(error["workspace"], json!(home_dir.path())); + let checkpoints = home_dir.path().join(".freshell/checkpoints"); + assert!(!tokio::fs::try_exists(checkpoints).await.unwrap()); + } + + #[tokio::test] + async fn picker_home_is_launch_only() { + let home_dir = tempfile::tempdir().unwrap(); + tokio::fs::write(home_dir.path().join("private.txt"), "private") + .await + .unwrap(); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": home_dir.path(), + "workspace": { + "path": home_dir.path(), + "provenance": "directory-picker", + }, + })), + ) + .await; + + assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY); + let error = body_json(response).await; + assert_eq!(error["code"], json!("workspace_unavailable")); + assert!( + !tokio::fs::try_exists(home_dir.path().join(".freshell/checkpoints")) + .await + .unwrap() + ); + } + + #[tokio::test] + async fn resolved_project_scope_independent_of_routing_cwd() { + let home_dir = tempfile::tempdir().unwrap(); + let project_dir = tempfile::tempdir().unwrap(); + let nested = project_dir.path().join("src/nested"); + tokio::fs::create_dir_all(&nested).await.unwrap(); + let source = project_dir.path().join("src/main.rs"); + tokio::fs::write(&source, "before").await.unwrap(); + run_git(&["init", "-q"], Some(project_dir.path())) + .await + .unwrap(); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ "cwd": nested })), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let entry = body_json(response).await; + assert_eq!(entry["workspace"]["path"], json!(project_dir.path())); + assert_eq!(entry["workspace"]["provenance"], json!("resolved-project")); + + tokio::fs::write(&source, "after").await.unwrap(); + let restored = post_checkpoint_restore( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": home_dir.path(), + "workspace": entry["workspace"], + "id": entry["id"], + })), + ) + .await; + assert_eq!(restored.status(), StatusCode::OK); + assert_eq!(tokio::fs::read_to_string(source).await.unwrap(), "before"); + } + + #[tokio::test] + async fn nested_clone_non_git_scope_complete_restore() { + let home_dir = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let nested = workspace.path().join("vendor/project"); + tokio::fs::create_dir_all(&nested).await.unwrap(); + run_git(&["init", "-q"], Some(&nested)).await.unwrap(); + + let ignore_file = nested.join(".gitignore"); + let tracked = nested.join("tracked.txt"); + let tracked_ignored = nested.join("tracked-ignored.txt"); + let untracked = nested.join("untracked.txt"); + let ignored_untracked = nested.join("ignored-untracked.txt"); + tokio::fs::write(&ignore_file, "tracked-ignored.txt\nignored-untracked.txt\n") + .await + .unwrap(); + tokio::fs::write(&tracked, "tracked before").await.unwrap(); + tokio::fs::write(&tracked_ignored, "ignored but tracked before") + .await + .unwrap(); + run_git(&["add", ".gitignore", "tracked.txt"], Some(&nested)) + .await + .unwrap(); + run_git(&["add", "-f", "tracked-ignored.txt"], Some(&nested)) + .await + .unwrap(); + run_git( + &[ + "-c", + "user.name=Checkpoint Test", + "-c", + "user.email=checkpoint@example.invalid", + "commit", + "-m", + "nested fixture", + ], + Some(&nested), + ) + .await + .unwrap(); + tokio::fs::write(&untracked, "untracked before") + .await + .unwrap(); + tokio::fs::write(&ignored_untracked, "ignored untracked before") + .await + .unwrap(); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": workspace.path(), + "workspace": { + "path": workspace.path(), + "provenance": "directory-picker", + }, + "label": "nested clone state", + })), + ) + .await; + let status = response.status(); + let entry = body_json(response).await; + assert_eq!(status, StatusCode::OK, "{entry}"); + let id = entry["id"].as_str().expect("checkpoint id").to_string(); + + tokio::fs::write(&tracked, "tracked after").await.unwrap(); + tokio::fs::write(&tracked_ignored, "ignored but tracked after") + .await + .unwrap(); + tokio::fs::write(&untracked, "untracked after") + .await + .unwrap(); + tokio::fs::write(&ignored_untracked, "ignored untracked after") + .await + .unwrap(); + + let restored = post_checkpoint_restore( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": workspace.path(), + "workspace": { + "path": workspace.path(), + "provenance": "directory-picker", + }, + "id": id, + })), + ) + .await; + assert_eq!( + restored.status(), + StatusCode::OK, + "{}", + body_json(restored).await + ); + assert_eq!( + tokio::fs::read_to_string(tracked).await.unwrap(), + "tracked before" + ); + assert_eq!( + tokio::fs::read_to_string(tracked_ignored).await.unwrap(), + "ignored but tracked before" + ); + assert_eq!( + tokio::fs::read_to_string(untracked).await.unwrap(), + "untracked before" + ); + assert_eq!( + tokio::fs::read_to_string(ignored_untracked).await.unwrap(), + "ignored untracked after", + "ignored untracked files stay outside the existing Git code scope" + ); + } + + #[tokio::test] + async fn submodule_working_edits_complete_restore() { + let home_dir = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let source = tempfile::tempdir().unwrap(); + let source_file = source.path().join("library.rs"); + tokio::fs::write(&source_file, "pub fn version() -> u8 { 1 }\n") + .await + .unwrap(); + run_git(&["init", "-q"], Some(source.path())).await.unwrap(); + run_git(&["add", "library.rs"], Some(source.path())) + .await + .unwrap(); + run_git( + &[ + "-c", + "user.name=Checkpoint Test", + "-c", + "user.email=checkpoint@example.invalid", + "commit", + "-m", + "submodule source", + ], + Some(source.path()), + ) + .await + .unwrap(); + + run_git(&["init", "-q"], Some(workspace.path())) + .await + .unwrap(); + run_git( + &[ + "-c", + "protocol.file.allow=always", + "submodule", + "add", + source.path().to_str().unwrap(), + "modules/library", + ], + Some(workspace.path()), + ) + .await + .unwrap(); + let submodule = workspace.path().join("modules/library"); + let original_head = run_git(&["rev-parse", "HEAD"], Some(&submodule)) + .await + .unwrap(); + let working_file = submodule.join("library.rs"); + tokio::fs::write(&working_file, "pub fn version() -> u8 { 2 }\n") + .await + .unwrap(); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ "cwd": workspace.path() })), + ) + .await; + let status = response.status(); + let entry = body_json(response).await; + assert_eq!(status, StatusCode::OK, "{entry}"); + let id = entry["id"].as_str().unwrap().to_string(); + + tokio::fs::write(&working_file, "pub fn version() -> u8 { 3 }\n") + .await + .unwrap(); + let restore = post_checkpoint_restore( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": workspace.path(), + "workspace": entry["workspace"], + "id": id, + })), + ) + .await; + assert_eq!(restore.status(), StatusCode::OK); + assert_eq!( + tokio::fs::read_to_string(working_file).await.unwrap(), + "pub fn version() -> u8 { 2 }\n" + ); + assert_eq!( + run_git(&["rev-parse", "HEAD"], Some(&submodule)) + .await + .unwrap(), + original_head, + "checkpoint restore does not move the nested repository's identity" + ); + let git_dir = checkpoint_git_dir(home_dir.path(), workspace.path()); + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let tree = run_git(&[&git_dir_arg, "ls-tree", &id, "modules/library"], None) + .await + .unwrap(); + assert!( + tree.starts_with("040000 tree "), + "nested contents must be blobs, not a gitlink: {tree}" + ); + } + + #[tokio::test] + async fn checkpoint_failed_metadata_publication_never_becomes_restore_target() { + let home_dir = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let file = workspace.path().join("source.txt"); + tokio::fs::write(&file, "current contents").await.unwrap(); + let git_dir = ensure_checkpoint_repo(home_dir.path(), workspace.path()) + .await + .unwrap(); + tokio::fs::create_dir(checkpoint_metadata_path(&git_dir)) + .await + .unwrap(); + + let response = post_checkpoint( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": workspace.path(), + "workspace": { + "path": workspace.path(), + "provenance": "directory-picker", + }, + "requestId": "metadata-write-failure", + })), + ) + .await; + assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); + + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let fsck = run_git( + &[&git_dir_arg, "fsck", "--no-reflogs", "--unreachable"], + None, + ) + .await + .unwrap(); + let orphan = fsck + .lines() + .find_map(|line| { + line.strip_prefix("unreachable commit ") + .or_else(|| line.strip_prefix("dangling commit ")) + }) + .unwrap_or_else(|| { + panic!("metadata failure should leave an unreachable commit object: {fsck}") + }) + .to_string(); + let current_head = run_git(&[&git_dir_arg, "rev-parse", "--verify", "HEAD"], None).await; + assert!( + current_head.is_err(), + "failed target must not advance the published ref" + ); + + let list = get_checkpoints( + State(state(home_dir.path())), + headers_with_token("tok"), + Query(HashMap::from([ + ( + "cwd".to_string(), + workspace.path().to_string_lossy().into_owned(), + ), + ( + "workspacePath".to_string(), + workspace.path().to_string_lossy().into_owned(), + ), + ( + "workspaceProvenance".to_string(), + "directory-picker".to_string(), + ), + ])), + ) + .await; + let entries = body_json(list).await; + assert_eq!(entries["checkpoints"], json!([])); + + tokio::fs::write(&file, "still current").await.unwrap(); + let restore = post_checkpoint_restore( + State(state(home_dir.path())), + headers_with_token("tok"), + Json(json!({ + "cwd": workspace.path(), + "workspace": { + "path": workspace.path(), + "provenance": "directory-picker", + }, + "id": orphan, + })), + ) + .await; + assert_eq!(restore.status(), StatusCode::INTERNAL_SERVER_ERROR); + assert_eq!( + tokio::fs::read_to_string(file).await.unwrap(), + "still current" + ); + } + #[tokio::test] async fn default_label_is_checkpoint_when_absent() { let home_dir = tempfile::tempdir().unwrap(); @@ -803,7 +2632,10 @@ mod tests { let resp = post_checkpoint( State(state(home_dir.path())), headers_with_token("tok"), - Json(json!({ "cwd": cwd_dir.path().to_str().unwrap() })), + Json(json!({ + "cwd": cwd_dir.path(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, + })), ) .await .into_response(); @@ -822,6 +2654,7 @@ mod tests { headers_with_token("tok"), Json(json!({ "cwd": cwd_dir.path().to_str().unwrap(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, "label": format!(" {long_label} "), })), ) @@ -842,6 +2675,7 @@ mod tests { headers_with_token("tok"), Json(json!({ "cwd": cwd_dir.path().to_str().unwrap(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, "label": "with request id", "requestId": "req-123", })), @@ -1047,7 +2881,11 @@ mod tests { let resp1 = post_checkpoint( State(state(home_dir.path())), headers_with_token("tok"), - Json(json!({ "cwd": cwd_dir.path().to_str().unwrap(), "label": "first" })), + Json(json!({ + "cwd": cwd_dir.path(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, + "label": "first" + })), ) .await .into_response(); @@ -1056,7 +2894,11 @@ mod tests { let resp2 = post_checkpoint( State(state(home_dir.path())), headers_with_token("tok"), - Json(json!({ "cwd": cwd_dir.path().to_str().unwrap(), "label": "second" })), + Json(json!({ + "cwd": cwd_dir.path(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, + "label": "second" + })), ) .await .into_response(); @@ -1089,6 +2931,7 @@ mod tests { headers_with_token("tok"), Json(json!({ "cwd": cwd_dir.path().to_str().unwrap(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, "label": "with metadata", "requestId": "req-1", })), @@ -1165,10 +3008,12 @@ mod tests { .await; assert_eq!(resp.status(), StatusCode::BAD_REQUEST); let value = body_json(resp).await; - assert!(value["error"] - .as_str() - .unwrap() - .starts_with("cwd does not exist:")); + assert!( + value["error"] + .as_str() + .unwrap() + .starts_with("cwd does not exist:") + ); } #[tokio::test] @@ -1553,7 +3398,11 @@ mod tests { &app, "POST", "/api/fresh-agent/checkpoints", - json!({ "cwd": cwd_dir.path().to_str().unwrap(), "label": "before the fix" }), + json!({ + "cwd": cwd_dir.path(), + "workspace": { "path": cwd_dir.path(), "provenance": "directory-picker" }, + "label": "before the fix" + }), ) .await; assert_eq!(status, StatusCode::OK); diff --git a/crates/freshell-server/src/files.rs b/crates/freshell-server/src/files.rs index c42f11a71..33a7c0632 100644 --- a/crates/freshell-server/src/files.rs +++ b/crates/freshell-server/src/files.rs @@ -200,8 +200,19 @@ async fn validate_dir( .unwrap_or(false) }) .unwrap_or(false); - - Json(json!({ "valid": is_dir, "resolvedPath": resolved.display })).into_response() + let checkpoint_workspace_eligible = is_dir + && resolved + .fs_path + .as_deref() + .and_then(|path| std::fs::canonicalize(path).ok()) + .is_some_and(|path| !crate::checkpoints::is_os_account_home(&path)); + + Json(json!({ + "valid": is_dir, + "resolvedPath": resolved.display, + "checkpointWorkspaceEligible": checkpoint_workspace_eligible, + })) + .into_response() } /// `GET /api/files/read?path=<p>` \u2192 `{ content, size, modifiedAt }` (`files-router.ts:85`). @@ -1394,6 +1405,7 @@ mod tests { let v = body_json(resp).await; assert_eq!(v["valid"], true); assert_eq!(v["resolvedPath"], dir); + assert_eq!(v["checkpointWorkspaceEligible"], true); let bogus = format!("{}/freshell-nonexistent-xyz", tmp.path().display()); let resp = validate_dir( State(test_state()), @@ -1404,6 +1416,24 @@ mod tests { .into_response(); let v = body_json(resp).await; assert_eq!(v["valid"], false); + assert_eq!(v["checkpointWorkspaceEligible"], false); + } + + #[allow(clippy::await_holding_lock)] + #[tokio::test] + async fn validate_dir_marks_account_home_as_launch_only() { + let _guard = env_lock(); + let home = std::env::var("HOME").expect("test environment has HOME"); + let response = validate_dir( + State(test_state()), + auth_headers(), + Json(json!({ "path": home })), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let value = body_json(response).await; + assert_eq!(value["valid"], true); + assert_eq!(value["checkpointWorkspaceEligible"], false); } /// Node's `normalizeUserPath` runs `path.posix.resolve` on POSIX inputs diff --git a/crates/freshell-server/tests/code_checkpoint_permissions.rs b/crates/freshell-server/tests/code_checkpoint_permissions.rs new file mode 100644 index 000000000..893ea5541 --- /dev/null +++ b/crates/freshell-server/tests/code_checkpoint_permissions.rs @@ -0,0 +1,581 @@ +#![cfg(all(unix, feature = "managed-runtime-v1"))] + +//! Black-box permission and publication proof for automatic code snapshots. +//! Run only through `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture +//! "cargo test -p freshell-server --features managed-runtime-v1 --locked --test +//! code_checkpoint_permissions"`; the trusted entrypoint creates the root-owned +//! unreadable sentinels and this test observes them only as the sandbox user. + +use std::fs; +use std::os::unix::fs::{MetadataExt, PermissionsExt}; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +use reqwest::{Client, StatusCode}; +use serde_json::{Value, json}; +use sha1::{Digest, Sha1}; + +const AUTH_TOKEN: &str = "checkpoint-fixture-auth-marker-do-not-log"; +const OUTGOING_MARKER: &str = "checkpoint-fixture-outgoing-text-do-not-log"; + +struct OwnedServer(Option<Child>); + +impl OwnedServer { + fn start(port: u16, home: &Path, log_dir: &Path) -> Self { + let child = Command::new(env!("CARGO_BIN_EXE_freshell-server")) + .env("PORT", port.to_string()) + .env("AUTH_TOKEN", AUTH_TOKEN) + .env("FRESHELL_HOME", home) + .env("HOME", home) + .env("FRESHELL_LOG_DIR", log_dir) + .env("FRESHELL_BIND_HOST", "127.0.0.1") + .env("FRESHELL_DISABLE_WSL_PORT_FORWARD", "1") + .env_remove("FRESHELL_MANAGED_RUNTIME_V1") + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("start the compiled Freshell server as the restricted fixture user"); + Self(Some(child)) + } + + fn child_mut(&mut self) -> &mut Child { + self.0.as_mut().expect("server child is still owned") + } + + fn stop(&mut self) { + if let Some(mut child) = self.0.take() { + let _ = child.kill(); + let _ = child.wait(); + } + } +} + +impl Drop for OwnedServer { + fn drop(&mut self) { + self.stop(); + } +} + +fn fixture_root() -> PathBuf { + let root = std::env::var_os("FRESHELL_CHECKPOINT_FIXTURE_ROOT") + .map(PathBuf::from) + .expect("run via --runtime-suite --checkpoint-fixture"); + assert_eq!(root, PathBuf::from("/tmp/freshell-checkpoint-fixture")); + root +} + +fn effective_capabilities() -> u64 { + let status = fs::read_to_string("/proc/self/status").expect("read process status"); + let line = status + .lines() + .find(|line| line.starts_with("CapEff:")) + .expect("CapEff line exists"); + u64::from_str_radix(line.split_whitespace().nth(1).unwrap(), 16).unwrap() +} + +async fn wait_for_health(port: u16, child: &mut Child) { + let client = Client::new(); + let deadline = Instant::now() + Duration::from_secs(20); + while Instant::now() < deadline { + if let Some(status) = child.try_wait().expect("poll owned server") { + panic!("server exited before health became ready: {status}"); + } + if client + .get(format!("http://127.0.0.1:{port}/api/health")) + .send() + .await + .is_ok_and(|response| response.status().is_success()) + { + return; + } + tokio::time::sleep(Duration::from_millis(50)).await; + } + panic!("server did not become healthy on loopback port {port}"); +} + +fn allocate_port() -> u16 { + let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind ephemeral port"); + listener.local_addr().expect("read local port").port() +} + +async fn post_json(client: &Client, url: &str, body: Value) -> (StatusCode, Value) { + let response = client + .post(url) + .header("x-auth-token", AUTH_TOKEN) + .json(&body) + .send() + .await + .expect("checkpoint HTTP request completes"); + let status = response.status(); + let value = response.json().await.expect("checkpoint response is JSON"); + (status, value) +} + +fn workspace_intent(path: &Path) -> Value { + json!({ "path": path, "provenance": "directory-picker" }) +} + +async fn list_workspace(client: &Client, base: &str, cwd: &Path, workspace: &Path) -> Vec<Value> { + // The closed fixture uses fixed slash-only paths without query delimiters; + // formatting avoids enabling reqwest's optional query feature solely here. + let url = format!( + "{base}/api/fresh-agent/checkpoints?cwd={}&workspacePath={}&workspaceProvenance=directory-picker", + cwd.display(), + workspace.display(), + ); + let response = client + .get(url) + .header("x-auth-token", AUTH_TOKEN) + .send() + .await + .expect("checkpoint list request completes"); + assert_eq!(response.status(), StatusCode::OK); + response.json::<Value>().await.unwrap()["checkpoints"] + .as_array() + .unwrap() + .clone() +} + +fn shadow_repo(home: &Path, workspace: &Path) -> PathBuf { + let canonical = fs::canonicalize(workspace).expect("workspace canonical path"); + let digest = format!("{:x}", Sha1::digest(canonical.to_string_lossy().as_bytes())); + home.join(".freshell") + .join("checkpoints") + .join(format!("{}.git", &digest[..16])) +} + +fn run_git(args: &[&str], cwd: &Path) -> String { + let output = Command::new("git") + .args(args) + .current_dir(cwd) + .output() + .expect("start Git fixture command"); + assert!( + output.status.success(), + "Git fixture command failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8_lossy(&output.stdout).into_owned() +} + +fn denied_file_read(path: &Path) { + let error = fs::File::open(path).expect_err("restricted source file must not be readable"); + assert_eq!(error.kind(), std::io::ErrorKind::PermissionDenied); +} + +#[tokio::test] +async fn restricted_checkpoint_routes_preserve_complete_targets_and_log_correlated_failures() { + let root = fixture_root(); + assert_ne!(unsafe { libc::geteuid() }, 0, "fixture must run as UID B"); + assert_eq!( + effective_capabilities(), + 0, + "fixture user must have no capabilities" + ); + + let home = root.join("home"); + let storage = root.join("storage"); + let project = root.join("project"); + let private_file_ignored = project.join("private-file.ignored"); + let private_file = project.join("private-file"); + let private_dir_ignored = project.join("private-dir.ignored"); + let private_dir = project.join("private-dir"); + let ignore_text = fs::read_to_string(project.join(".gitignore")).unwrap(); + assert!( + ignore_text + .lines() + .any(|line| line == "private-file.ignored") + ); + assert!( + ignore_text + .lines() + .any(|line| line == "private-dir.ignored/") + ); + + denied_file_read(&private_file_ignored); + assert_eq!( + fs::read_dir(&private_dir_ignored) + .expect_err("restricted directory must not be traversable") + .kind(), + std::io::ErrorKind::PermissionDenied + ); + let private_file_mode = fs::symlink_metadata(&private_file_ignored).unwrap(); + let private_dir_mode = fs::symlink_metadata(&private_dir_ignored).unwrap(); + assert_eq!(private_file_mode.uid(), 0); + assert_eq!(private_file_mode.mode() & 0o777, 0o600); + assert_eq!(private_dir_mode.uid(), 0); + assert_eq!(private_dir_mode.mode() & 0o777, 0o700); + + // B can change eligibility only by same-parent rename; ownership and mode + // remain those prepared by the trusted root initializer. + fs::rename(&private_file_ignored, &private_file).unwrap(); + denied_file_read(&private_file); + fs::rename(&private_file, &private_file_ignored).unwrap(); + fs::rename(&private_dir_ignored, &private_dir).unwrap(); + assert_eq!( + fs::read_dir(&private_dir) + .expect_err("renamed restricted directory remains non-traversable") + .kind(), + std::io::ErrorKind::PermissionDenied + ); + fs::rename(&private_dir, &private_dir_ignored).unwrap(); + let file_after_rename = fs::symlink_metadata(&private_file_ignored).unwrap(); + let dir_after_rename = fs::symlink_metadata(&private_dir_ignored).unwrap(); + assert_eq!( + (file_after_rename.uid(), file_after_rename.mode() & 0o777), + (0, 0o600) + ); + assert_eq!( + (dir_after_rename.uid(), dir_after_rename.mode() & 0o777), + (0, 0o700) + ); + + fs::write(project.join("main.rs"), "before\n").unwrap(); + fs::write(project.join("run.sh"), "#!/bin/sh\necho before\n").unwrap(); + fs::set_permissions(project.join("run.sh"), fs::Permissions::from_mode(0o755)).unwrap(); + std::os::unix::fs::symlink("main.rs", project.join("main-link")).unwrap(); + run_git(&["init", "-q"], &project); + let outer_status_before = run_git(&["status", "--porcelain"], &project); + + fs::create_dir_all(storage.join("logs")).unwrap(); + let port = allocate_port(); + let mut server = OwnedServer::start(port, &home, &storage.join("logs")); + wait_for_health(port, server.child_mut()).await; + + let client = Client::new(); + let base = format!("http://127.0.0.1:{port}"); + let checkpoints_url = format!("{base}/api/fresh-agent/checkpoints"); + + // HOME is launch/routing cwd only when no separate workspace intent is + // present. It must not create a home-keyed checkpoint store. + let (home_status, home_error) = post_json( + &client, + &checkpoints_url, + json!({ "cwd": home, "label": OUTGOING_MARKER, "requestId": "send-home-refused" }), + ) + .await; + assert_eq!(home_status, StatusCode::UNPROCESSABLE_ENTITY); + assert_eq!(home_error["code"], "workspace_unavailable"); + assert!(!home.join(".freshell/checkpoints").exists()); + + let (create_status, created) = post_json( + &client, + &checkpoints_url, + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "label": "readable git snapshot", + "requestId": "send-initial-success", + }), + ) + .await; + assert_eq!( + create_status, + StatusCode::OK, + "{created}; logs {}", + fs::read_to_string(storage.join("logs/rust-server.jsonl")).unwrap() + ); + let checkpoint_id = created["id"].as_str().unwrap().to_string(); + assert_eq!( + created["workspace"]["path"], + project.to_string_lossy().as_ref() + ); + + fs::write(project.join("main.rs"), "after\n").unwrap(); + fs::write(project.join("run.sh"), "#!/bin/sh\necho after\n").unwrap(); + fs::set_permissions(project.join("run.sh"), fs::Permissions::from_mode(0o644)).unwrap(); + fs::remove_file(project.join("main-link")).unwrap(); + std::os::unix::fs::symlink("run.sh", project.join("main-link")).unwrap(); + + let (restore_status, restore_body) = post_json( + &client, + &format!("{base}/api/fresh-agent/checkpoints/restore"), + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "id": checkpoint_id, + }), + ) + .await; + assert_eq!(restore_status, StatusCode::OK, "{restore_body}"); + assert_eq!( + fs::read_to_string(project.join("main.rs")).unwrap(), + "before\n" + ); + assert_eq!( + fs::metadata(project.join("run.sh")).unwrap().mode() & 0o777, + 0o755 + ); + assert_eq!( + fs::read_link(project.join("main-link")).unwrap(), + Path::new("main.rs") + ); + + let outer_status_after = run_git(&["status", "--porcelain"], &project); + assert_eq!(outer_status_before, outer_status_after); + + // Make a previously ignored, root-owned private file eligible. Git itself + // may report the denial during `add`; the route must reject publication + // with a stable, safe reason and leave the previous complete target valid. + fs::rename(&private_file_ignored, &private_file).unwrap(); + let (file_failure_status, file_failure) = post_json( + &client, + &checkpoints_url, + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "label": OUTGOING_MARKER, + "requestId": "send-private-file", + }), + ) + .await; + assert_ne!(file_failure_status, StatusCode::OK); + let file_failure_log = fs::read_to_string(storage.join("logs/rust-server.jsonl")) + .unwrap() + .lines() + .filter(|line| line.contains("send-private-file")) + .collect::<Vec<_>>() + .join("\n"); + assert_eq!( + file_failure["code"], "checkpoint_path_unreadable", + "response {file_failure}; correlated log {file_failure_log}" + ); + fs::rename(&private_file, &private_file_ignored).unwrap(); + + fs::rename(&private_dir_ignored, &private_dir).unwrap(); + let (dir_failure_status, dir_failure) = post_json( + &client, + &checkpoints_url, + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "label": OUTGOING_MARKER, + "requestId": "send-private-dir", + }), + ) + .await; + assert_ne!(dir_failure_status, StatusCode::OK); + assert_eq!(dir_failure["code"], "checkpoint_path_unreadable"); + fs::rename(&private_dir, &private_dir_ignored).unwrap(); + + let previous = list_workspace(&client, &base, &home, &project).await; + assert_eq!(previous.len(), 1); + assert_eq!(previous[0]["id"], checkpoint_id); + assert_eq!( + previous[0]["workspace"]["path"], + project.to_string_lossy().as_ref() + ); + + // A real metadata-store read failure happens after the new Git commit + // object is created. Its orphan is not published and cannot be restored. + let git_dir = shadow_repo(&home, &project); + let metadata_path = git_dir.join("freshell-checkpoint-metadata.json"); + let metadata_backup = git_dir.join("freshell-checkpoint-metadata.json.fixture-backup"); + assert!(metadata_path.is_file()); + fs::rename(&metadata_path, &metadata_backup).unwrap(); + fs::create_dir(&metadata_path).unwrap(); + let (storage_failure_status, storage_failure) = post_json( + &client, + &checkpoints_url, + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "label": OUTGOING_MARKER, + "requestId": "send-storage-failure", + }), + ) + .await; + assert_eq!(storage_failure_status, StatusCode::INTERNAL_SERVER_ERROR); + assert_eq!(storage_failure["code"], "checkpoint_capture_failed"); + fs::remove_dir(&metadata_path).unwrap(); + fs::rename(&metadata_backup, &metadata_path).unwrap(); + + let fsck = run_git( + &[ + "--git-dir", + git_dir.to_str().unwrap(), + "fsck", + "--full", + "--no-reflogs", + ], + &root, + ); + let orphan_id = fsck + .lines() + .find_map(|line| { + let mut words = line.split_whitespace(); + let relation = words.next()?; + let kind = words.next()?; + let id = words.next()?; + ((relation == "dangling" || relation == "unreachable") && kind == "commit") + .then(|| id.to_string()) + }) + .expect("failed metadata publication left an inspectable orphan commit"); + let (orphan_restore_status, _) = post_json( + &client, + &format!("{base}/api/fresh-agent/checkpoints/restore"), + json!({ "cwd": home, "workspace": workspace_intent(&project), "id": orphan_id }), + ) + .await; + assert_eq!(orphan_restore_status, StatusCode::BAD_REQUEST); + assert_eq!( + list_workspace(&client, &base, &home, &project).await.len(), + 1 + ); + + // Once the owned storage obstruction is removed, the same workspace can + // publish a later complete checkpoint without exposing the orphan. + fs::write(project.join("main.rs"), "after repaired capture\n").unwrap(); + let (repaired_status, repaired_entry) = post_json( + &client, + &checkpoints_url, + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "label": "capture after repair", + "requestId": "send-after-repair", + }), + ) + .await; + assert_eq!(repaired_status, StatusCode::OK, "{repaired_entry}"); + let repaired_list = list_workspace(&client, &base, &home, &project).await; + assert_eq!(repaired_list.len(), 2); + assert!( + repaired_list + .iter() + .any(|entry| entry["id"] == repaired_entry["id"]) + ); + fs::write(project.join("main.rs"), "changed after repaired capture\n").unwrap(); + let (repaired_restore_status, repaired_restore_body) = post_json( + &client, + &format!("{base}/api/fresh-agent/checkpoints/restore"), + json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "id": repaired_entry["id"], + }), + ) + .await; + assert_eq!( + repaired_restore_status, + StatusCode::OK, + "{repaired_restore_body}" + ); + assert_eq!( + fs::read_to_string(project.join("main.rs")).unwrap(), + "after repaired capture\n" + ); + + // A separately selected non-Git directory also uses the complete snapshot + // route while HOME remains only the launch cwd. + let non_git = root.join("non-git"); + fs::create_dir(&non_git).unwrap(); + fs::write(non_git.join("one.txt"), "one-before\n").unwrap(); + fs::write(non_git.join("two.txt"), "two-before\n").unwrap(); + let (non_git_status, non_git_entry) = post_json( + &client, + &checkpoints_url, + json!({ + "cwd": home, + "workspace": workspace_intent(&non_git), + "label": "non-git scope", + "requestId": "send-non-git", + }), + ) + .await; + assert_eq!(non_git_status, StatusCode::OK, "{non_git_entry}"); + fs::write(non_git.join("one.txt"), "one-after\n").unwrap(); + fs::write(non_git.join("two.txt"), "two-after\n").unwrap(); + let (non_git_restore_status, _) = post_json( + &client, + &format!("{base}/api/fresh-agent/checkpoints/restore"), + json!({ + "cwd": home, + "workspace": workspace_intent(&non_git), + "id": non_git_entry["id"], + }), + ) + .await; + assert_eq!(non_git_restore_status, StatusCode::OK); + assert_eq!( + fs::read_to_string(non_git.join("one.txt")).unwrap(), + "one-before\n" + ); + assert_eq!( + fs::read_to_string(non_git.join("two.txt")).unwrap(), + "two-before\n" + ); + + let logs = fs::read_to_string(storage.join("logs/rust-server.jsonl")).unwrap(); + let events = logs + .lines() + .map(|line| serde_json::from_str::<Value>(line).expect("each log line is JSON")) + .collect::<Vec<_>>(); + for send_request_id in [ + "send-private-file", + "send-private-dir", + "send-storage-failure", + ] { + let event = events + .iter() + .find(|event| { + event["event"] == "code_checkpoint.failed" + && event["send_request_id"] == send_request_id + }) + .unwrap_or_else(|| panic!("missing correlated failure event for {send_request_id}")); + assert!( + event["request_id"].as_str().is_some(), + "HTTP request span is retained" + ); + assert_eq!(event["workspace"], project.to_string_lossy().as_ref()); + assert!(event["stage"].as_str().is_some()); + assert!(event["operation"].as_str().is_some()); + assert!(event["cause"].as_str().is_some()); + } + for send_request_id in ["send-private-file", "send-private-dir"] { + let event = events + .iter() + .find(|event| { + event["event"] == "code_checkpoint.failed" + && event["send_request_id"] == send_request_id + }) + .unwrap(); + assert_eq!(event["stage"], "readability"); + assert_eq!(event["filesystem_kind"], "PermissionDenied"); + assert!( + event["cause"] + .as_str() + .unwrap() + .contains("Permission denied") + ); + let path = event["path"].as_str().unwrap(); + assert!(!path.is_empty()); + let expected_path = if send_request_id == "send-private-file" { + "private-file" + } else { + "private-dir" + }; + assert!( + path == expected_path || path.starts_with(&format!("{expected_path}/")), + "expected the path of the eligible unreadable source, got {path}" + ); + } + assert!(!logs.contains(OUTGOING_MARKER)); + assert!(!logs.contains(AUTH_TOKEN)); + + // Private source permissions remain untouched even after eligible-name + // transitions and all route operations. + let final_private_file = fs::symlink_metadata(private_file_ignored).unwrap(); + let final_private_dir = fs::symlink_metadata(private_dir_ignored).unwrap(); + assert_eq!( + (final_private_file.uid(), final_private_file.mode() & 0o777), + (0, 0o600) + ); + assert_eq!( + (final_private_dir.uid(), final_private_dir.mode() & 0o777), + (0, 0o700) + ); + + server.stop(); +} diff --git a/docker/sandbox/entrypoint.sh b/docker/sandbox/entrypoint.sh index c7faca3db..0ed75c6f6 100755 --- a/docker/sandbox/entrypoint.sh +++ b/docker/sandbox/entrypoint.sh @@ -9,6 +9,52 @@ set -euo pipefail TARGET_UID="$(id -u sandbox)" TARGET_GID="$(id -g sandbox)" +# This exact, closed fixture is used only by the checkpoint permission smoke. +# The wrapper permits it only in the networkless runtime suite; setup has no +# caller-supplied path, command, or ownership script. All observations and +# mutations happen later as the unprivileged sandbox user. +if [ "${FRESHELL_SANDBOX_CHECKPOINT_FIXTURE:-}" = "1" ]; then + if [ "${FRESHELL_SANDBOX_MODE:-}" != "runtime-suite" ] || [ "$(id -u)" -ne 0 ]; then + echo "[sandbox] checkpoint fixture requires the root runtime-suite initializer" >&2 + exit 2 + fi + CHECKPOINT_FIXTURE_ROOT="/tmp/freshell-checkpoint-fixture" + mkdir -p \ + "${CHECKPOINT_FIXTURE_ROOT}/project" \ + "${CHECKPOINT_FIXTURE_ROOT}/storage" \ + "${CHECKPOINT_FIXTURE_ROOT}/home" + chown "${TARGET_UID}:${TARGET_GID}" \ + "${CHECKPOINT_FIXTURE_ROOT}" \ + "${CHECKPOINT_FIXTURE_ROOT}/project" \ + "${CHECKPOINT_FIXTURE_ROOT}/storage" \ + "${CHECKPOINT_FIXTURE_ROOT}/home" + chmod 0755 \ + "${CHECKPOINT_FIXTURE_ROOT}" \ + "${CHECKPOINT_FIXTURE_ROOT}/project" \ + "${CHECKPOINT_FIXTURE_ROOT}/storage" \ + "${CHECKPOINT_FIXTURE_ROOT}/home" + + printf '%s\n' 'private-file.ignored' 'private-dir.ignored/' \ + >"${CHECKPOINT_FIXTURE_ROOT}/project/.gitignore" + chown "${TARGET_UID}:${TARGET_GID}" "${CHECKPOINT_FIXTURE_ROOT}/project/.gitignore" + chmod 0644 "${CHECKPOINT_FIXTURE_ROOT}/project/.gitignore" + + printf '%s\n' 'restricted checkpoint file sentinel' \ + >"${CHECKPOINT_FIXTURE_ROOT}/project/private-file.ignored" + chmod 0600 "${CHECKPOINT_FIXTURE_ROOT}/project/private-file.ignored" + chown 0:0 "${CHECKPOINT_FIXTURE_ROOT}/project/private-file.ignored" + + mkdir "${CHECKPOINT_FIXTURE_ROOT}/project/private-dir.ignored" + printf '%s\n' 'restricted checkpoint directory sentinel' \ + >"${CHECKPOINT_FIXTURE_ROOT}/project/private-dir.ignored/private.txt" + chmod 0600 "${CHECKPOINT_FIXTURE_ROOT}/project/private-dir.ignored/private.txt" + chown 0:0 "${CHECKPOINT_FIXTURE_ROOT}/project/private-dir.ignored/private.txt" + chmod 0700 "${CHECKPOINT_FIXTURE_ROOT}/project/private-dir.ignored" + chown 0:0 "${CHECKPOINT_FIXTURE_ROOT}/project/private-dir.ignored" + + export FRESHELL_CHECKPOINT_FIXTURE_ROOT="${CHECKPOINT_FIXTURE_ROOT}" +fi + cd /workspace # /home/sandbox/.cache is where npm/pnpm postinstall scripts (electron, diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index f293ea6e7..80c3ea65e 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -11,6 +11,7 @@ # scripts/sandbox-test.sh --corpus "cargo test -p freshell-sessions -- --ignored perf" # scripts/sandbox-test.sh --native-tools <manifest-path> --receipt-dir <absolute-directory> # scripts/sandbox-test.sh --runtime-suite "cargo test -p freshell-codex ..." +# scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions" # # --runtime-suite is the stricter managed-runtime lifecycle mode: no network, # no real provider corpus, no Docker/admin socket, read-only container root. @@ -91,6 +92,7 @@ if [ "${1:-}" = "--settle-owned" ]; then else MOUNT_CORPUS=0 RUNTIME_SUITE=0 + CHECKPOINT_FIXTURE=0 while [ "$#" -gt 0 ]; do case "${1}" in --corpus) @@ -101,6 +103,10 @@ else RUNTIME_SUITE=1 shift ;; + --checkpoint-fixture) + CHECKPOINT_FIXTURE=1 + shift + ;; --) shift break @@ -116,6 +122,11 @@ else exit 2 fi + if [ "${CHECKPOINT_FIXTURE}" -eq 1 ] && [ "${RUNTIME_SUITE}" -ne 1 ]; then + echo "[sandbox] --checkpoint-fixture requires --runtime-suite" >&2 + exit 2 + fi + if [ "$#" -lt 1 ]; then echo "usage: $0 [--corpus | --runtime-suite] \"<command to run inside the sandbox>\"" >&2 exit 2 @@ -358,6 +369,9 @@ if [ "${RUNTIME_SUITE}" -eq 1 ]; then --security-opt no-new-privileges -e FRESHELL_SANDBOX_MODE=runtime-suite ) + if [ "${CHECKPOINT_FIXTURE}" -eq 1 ]; then + DOCKER_ARGS+=(-e FRESHELL_SANDBOX_CHECKPOINT_FIXTURE=1) + fi fi if [ "${MOUNT_CORPUS}" -eq 1 ]; then From fc4afd037ba27dec3946e4e161dc693bbe792109 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 02:18:59 -0700 Subject: [PATCH 303/493] feat: preserve checkpoint workspace and turn outcomes --- src/components/panes/DirectoryPicker.tsx | 36 ++++- src/components/panes/PaneContainer.tsx | 14 +- src/lib/session-type-utils.ts | 9 +- src/lib/tab-registry-open.ts | 2 + src/store/paneTypes.ts | 107 +++++++++++++++ src/store/panesSlice.ts | 33 +++++ src/store/persistedState.ts | 13 +- src/store/tabsSlice.ts | 10 +- .../components/panes/DirectoryPicker.test.tsx | 57 +++++++- .../PaneContainer.createContent.test.tsx | 16 ++- .../panesSlice.checkpoint-outcomes.test.ts | 129 ++++++++++++++++++ test/unit/client/store/persistControl.test.ts | 51 +++++++ 12 files changed, 460 insertions(+), 17 deletions(-) create mode 100644 test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts diff --git a/src/components/panes/DirectoryPicker.tsx b/src/components/panes/DirectoryPicker.tsx index 8880a648e..086b49864 100644 --- a/src/components/panes/DirectoryPicker.tsx +++ b/src/components/panes/DirectoryPicker.tsx @@ -12,7 +12,10 @@ type DirectoryPickerProps = { defaultCwd?: string tabDirectories?: string[] globalDefault?: string - onConfirm: (cwd: string) => void + onConfirm: ( + cwd: string, + checkpointWorkspace?: { path: string; provenance: 'directory-picker' }, + ) => void onBack: () => void paneId?: string focusEligible?: boolean @@ -193,13 +196,22 @@ export default function DirectoryPicker({ const validationId = validationRequestIdRef.current try { - const result = await api.post<{ valid: boolean; resolvedPath?: string }>('/api/files/validate-dir', { path: nextPath }) + const result = await api.post<{ + valid: boolean + resolvedPath?: string + checkpointWorkspaceEligible?: boolean + }>('/api/files/validate-dir', { path: nextPath }) if (validationRequestIdRef.current !== validationId) return if (!result.valid) { setError('directory not found') return } - onConfirm(result.resolvedPath || nextPath) + const resolvedPath = result.resolvedPath || nextPath + if (result.checkpointWorkspaceEligible === true) { + onConfirm(resolvedPath, { path: resolvedPath, provenance: 'directory-picker' }) + } else { + onConfirm(resolvedPath) + } } catch (error) { if (validationRequestIdRef.current !== validationId) return if (isApiError(error) && error.status === 403) { @@ -229,7 +241,23 @@ export default function DirectoryPicker({ try { const result = await api.post<{ created: boolean; existed?: boolean; resolvedPath?: string }>('/api/files/mkdir', { path: nextPath }) if (createRequestIdRef.current !== createId) return - onConfirm(result.resolvedPath || nextPath) + const resolvedPath = result.resolvedPath || nextPath + const validation = await api.post<{ + valid: boolean + resolvedPath?: string + checkpointWorkspaceEligible?: boolean + }>('/api/files/validate-dir', { path: resolvedPath }) + if (createRequestIdRef.current !== createId) return + if (!validation.valid) { + setError('directory not found') + return + } + const validatedPath = validation.resolvedPath || resolvedPath + if (validation.checkpointWorkspaceEligible === true) { + onConfirm(validatedPath, { path: validatedPath, provenance: 'directory-picker' }) + } else { + onConfirm(validatedPath) + } } catch (error) { if (createRequestIdRef.current !== createId) return if (isApiError(error) && error.status === 403) { diff --git a/src/components/panes/PaneContainer.tsx b/src/components/panes/PaneContainer.tsx index a858fc68f..edb7fa7fc 100644 --- a/src/components/panes/PaneContainer.tsx +++ b/src/components/panes/PaneContainer.tsx @@ -689,7 +689,11 @@ function PickerWrapper({ return (freshAgentProviderConfig ? freshAgentProviderConfig.codingCliProvider : providerType) as CodingCliProviderName }, []) - const createContentForType = useCallback((type: PanePickerType, cwd?: string): PaneContent => { + const createContentForType = useCallback(( + type: PanePickerType, + cwd?: string, + checkpointWorkspace?: { path: string; provenance: 'directory-picker' }, + ): PaneContent => { if (typeof type === 'string' && type.startsWith('ext:')) { const extensionName = type.slice(4) return { @@ -755,6 +759,7 @@ function PickerWrapper({ plugins: freshAgentType.runtimeProvider === 'claude' ? freshAgentSettings?.defaultPlugins : undefined, style: providerSettings?.style ?? DEFAULT_FRESH_AGENT_STYLE, ...(cwd ? { initialCwd: cwd } : {}), + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), } } @@ -841,11 +846,14 @@ function PickerWrapper({ dispatch(updatePaneContent({ tabId, paneId, content: newContent })) }, [createContentForType, dispatch, tabId, paneId, extensionEntries]) - const handleDirectoryConfirm = useCallback((cwd: string) => { + const handleDirectoryConfirm = useCallback(( + cwd: string, + checkpointWorkspace?: { path: string; provenance: 'directory-picker' }, + ) => { if (step.step !== 'directory') return const providerType = step.providerType - const newContent = createContentForType(providerType, cwd) + const newContent = createContentForType(providerType, cwd, checkpointWorkspace) dispatch(updatePaneContent({ tabId, paneId, content: newContent })) // Save the selected directory for the provider diff --git a/src/lib/session-type-utils.ts b/src/lib/session-type-utils.ts index 88affe790..e7e5ef688 100644 --- a/src/lib/session-type-utils.ts +++ b/src/lib/session-type-utils.ts @@ -7,7 +7,11 @@ import { getFreshAgentProviderConfig } from '@/lib/fresh-agent-provider-utils' import { resolveFreshAgentType } from '@/lib/fresh-agent-registry' import type { FreshAgentProviderName, FreshAgentProviderSettings } from '@/lib/fresh-agent-provider-types' import type { CodingCliProviderName } from '@/store/types' -import type { FreshAgentPaneInput, TerminalPaneInput } from '@/store/paneTypes' +import type { + FreshAgentCheckpointWorkspace, + FreshAgentPaneInput, + TerminalPaneInput, +} from '@/store/paneTypes' import type { ClientExtensionEntry } from '@shared/extension-types' import { getPairedPublicSessionType, @@ -190,6 +194,7 @@ export function buildResumeContent(opts: { sessionType: string sessionId: string cwd?: string + checkpointWorkspace?: FreshAgentCheckpointWorkspace freshAgentProviderSettings?: FreshAgentProviderSettings liveTerminal?: { terminalId: string @@ -213,6 +218,7 @@ export function buildResumeContent(opts: { sessionId: opts.sessionId, }, initialCwd: opts.cwd, + ...(opts.checkpointWorkspace ? { checkpointWorkspace: opts.checkpointWorkspace } : {}), modelSelection: ps?.modelSelection, model: freshAgentType.defaultModel, ...(permissionMode ? { permissionMode } : {}), @@ -233,6 +239,7 @@ export function buildResumeContent(opts: { sessionId: opts.sessionId, }, initialCwd: opts.cwd, + ...(opts.checkpointWorkspace ? { checkpointWorkspace: opts.checkpointWorkspace } : {}), modelSelection: ps?.modelSelection, permissionMode: ps?.defaultPermissionMode ?? freshAgentProviderConfig.defaultPermissionMode, effort: ps?.effort, diff --git a/src/lib/tab-registry-open.ts b/src/lib/tab-registry-open.ts index 025290b6b..f721c5c4b 100644 --- a/src/lib/tab-registry-open.ts +++ b/src/lib/tab-registry-open.ts @@ -23,6 +23,7 @@ import { import { normalizeFreshAgentEffortOverride, normalizeFreshAgentModelSelection, + type FreshAgentCheckpointWorkspace, type PaneContentInput, type SessionLocator, } from '@/store/paneTypes' @@ -161,6 +162,7 @@ export function sanitizePaneSnapshot( ...(restoreError && !sessionRef ? { restoreError } : {}), serverInstanceId: record.serverInstanceId, initialCwd: payload.initialCwd as string | undefined, + checkpointWorkspace: payload.checkpointWorkspace as FreshAgentCheckpointWorkspace | undefined, model: payload.model as string | undefined, modelSelection: normalizeFreshAgentModelSelection(payload.modelSelection, payload.model), permissionMode: payload.permissionMode as string | undefined, diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index 5e86f10a0..62044262e 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -67,6 +67,104 @@ export function normalizeFreshAgentEffortOverride(value: unknown): string | unde return trimmed.length > 0 ? trimmed : undefined } +export type FreshAgentCheckpointOutcome = { + /** Conversation generation; changes only when the pane is deliberately replaced. */ + conversationRequestId?: string + requestId: string + displayTurnId?: string + nativeTurnId?: string + status: 'available' | 'unavailable' + checkpointId?: string + workspace?: FreshAgentCheckpointWorkspace + code?: string + reason?: string +} + +const FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT = 200 + +function checkpointOutcomeString(value: unknown, maxLength: number): string | undefined { + if (typeof value !== 'string' || value.length === 0 || value.length > maxLength) return undefined + if (/[\u0000-\u001f\u007f-\u009f]/.test(value)) return undefined + return value +} + +function checkpointOutcomeReason(value: unknown): string | undefined { + if (typeof value !== 'string') return undefined + const normalized = value + .replace(/\u001B\[[0-?]*[ -/]*[@-~]/g, '') + .replace(/[\u0000-\u001f\u007f-\u009f]/g, ' ') + .replace(/\s+/g, ' ') + .trim() + .slice(0, 240) + return normalized || undefined +} + +function checkpointOutcomeWorkspace(value: unknown): FreshAgentCheckpointWorkspace | undefined { + if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined + const candidate = value as Record<string, unknown> + const path = checkpointOutcomeString(candidate.path, 4096) + if (!path) return undefined + if (candidate.provenance !== 'directory-picker' + && candidate.provenance !== 'resolved-project' + && candidate.provenance !== 'legacy-checkpoint') return undefined + return { path, provenance: candidate.provenance } +} + +/** + * Persist only a bounded record of settled checkpoint availability. The + * whitelist deliberately excludes the submitted prompt and provider payloads. + */ +export function normalizeFreshAgentCheckpointOutcomes( + value: unknown, + conversationRequestId?: string, +): FreshAgentCheckpointOutcome[] | undefined { + if (!Array.isArray(value)) return undefined + const byRequest = new Map<string, FreshAgentCheckpointOutcome>() + for (const candidate of value) { + if (!candidate || typeof candidate !== 'object' || Array.isArray(candidate)) continue + const record = candidate as Record<string, unknown> + const requestId = checkpointOutcomeString(record.requestId, 128) + if (!requestId || !/^[A-Za-z0-9_.:-]+$/.test(requestId)) continue + const recordConversationRequestId = record.conversationRequestId === undefined + ? undefined + : checkpointOutcomeString(record.conversationRequestId, 128) + if (record.conversationRequestId !== undefined + && (!recordConversationRequestId || !/^[A-Za-z0-9_.:-]+$/.test(recordConversationRequestId))) continue + if (conversationRequestId && recordConversationRequestId + && recordConversationRequestId !== conversationRequestId) continue + if (record.status !== 'available' && record.status !== 'unavailable') continue + + const displayTurnId = checkpointOutcomeString(record.displayTurnId, 256) + const nativeTurnId = checkpointOutcomeString(record.nativeTurnId, 256) + const checkpointId = checkpointOutcomeString(record.checkpointId, 256) + const workspace = checkpointOutcomeWorkspace(record.workspace) + const code = typeof record.code === 'string' + && record.code.length <= 96 + && /^[a-z0-9_.:-]+$/.test(record.code) + ? record.code + : undefined + const reason = checkpointOutcomeReason(record.reason) + + if (record.status === 'available' && !checkpointId) continue + if (record.status === 'unavailable' && !code && !reason) continue + + byRequest.delete(requestId) + byRequest.set(requestId, { + requestId, + ...(recordConversationRequestId ? { conversationRequestId: recordConversationRequestId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + status: record.status, + ...(checkpointId ? { checkpointId } : {}), + ...(workspace ? { workspace } : {}), + ...(code ? { code } : {}), + ...(reason ? { reason } : {}), + }) + } + const normalized = [...byRequest.values()].slice(-FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) + return normalized.length > 0 ? normalized : undefined +} + export function normalizeFreshAgentModelEffortLevels(value: unknown): string[] | undefined { if (!Array.isArray(value)) { return undefined @@ -357,6 +455,10 @@ export type FreshAgentPaneContent = { /** Explicit restore failure when no canonical durable target exists. */ restoreError?: RestoreError initialCwd?: string + /** Independently selected code scope for per-turn rewind checkpoints. */ + checkpointWorkspace?: FreshAgentCheckpointWorkspace + /** Bounded settled per-turn rewind availability; prompt text is never stored here. */ + checkpointOutcomes?: FreshAgentCheckpointOutcome[] createError?: FreshAgentCreateError /** Volatile close failure, shown in the pane's dismissible yellow error banner. */ closeError?: string @@ -415,6 +517,11 @@ export type ExtensionPaneContent = { props: Record<string, unknown> } +export type FreshAgentCheckpointWorkspace = { + path: string + provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' +} + /** * Union type for all pane content types. */ diff --git a/src/store/panesSlice.ts b/src/store/panesSlice.ts index c0d86c320..a56cba339 100644 --- a/src/store/panesSlice.ts +++ b/src/store/panesSlice.ts @@ -2,6 +2,7 @@ import { createSlice, PayloadAction } from '@reduxjs/toolkit' import { nanoid } from 'nanoid' import { normalizeFreshAgentEffortOverride, + normalizeFreshAgentCheckpointOutcomes, normalizeFreshAgentModelEffortLevels, normalizeFreshAgentModelSelection, normalizeFreshAgentPendingLocalEcho, @@ -9,6 +10,8 @@ import { normalizeLaunchFailure, type DeadSessionEntry, type CrashTrace, + type FreshAgentCheckpointWorkspace, + type FreshAgentCheckpointOutcome, type FreshAgentPaneContent, type HandoffError, type LaunchFailure, @@ -72,6 +75,23 @@ function normalizeManagedRuntimeProjection( ) as ManagedRuntimeProjectionFields } +function normalizeCheckpointWorkspace(value: unknown): FreshAgentCheckpointWorkspace | undefined { + if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined + const candidate = value as Record<string, unknown> + if (typeof candidate.path !== 'string' || candidate.path.length === 0) return undefined + if (candidate.provenance !== 'directory-picker' + && candidate.provenance !== 'resolved-project' + && candidate.provenance !== 'legacy-checkpoint') return undefined + return { path: candidate.path, provenance: candidate.provenance } +} + +function normalizeCheckpointOutcomes( + value: unknown, + conversationRequestId?: string, +): FreshAgentCheckpointOutcome[] | undefined { + return normalizeFreshAgentCheckpointOutcomes(value, conversationRequestId) +} + function buildPreservedSessionRef( localContent: Extract<PaneContent, { kind: 'terminal' | 'fresh-agent' }>, _preservedResumeSessionId?: string, @@ -176,6 +196,15 @@ function normalizePaneContent( const pendingLocalEcho = normalizeFreshAgentPendingLocalEcho(rawFreshAgent.pendingLocalEcho) const modelEffortLevels = normalizeFreshAgentModelEffortLevels(rawFreshAgent.modelEffortLevels) const freshHandoffError = normalizeHandoffError(rawFreshAgent.handoffError) + const checkpointWorkspace = normalizeCheckpointWorkspace(rawFreshAgent.checkpointWorkspace) + const outcomeConversationRequestId = typeof rawFreshAgent.createRequestId === 'string' + && rawFreshAgent.createRequestId + ? rawFreshAgent.createRequestId + : previousCreateRequestId + const checkpointOutcomes = normalizeCheckpointOutcomes( + rawFreshAgent.checkpointOutcomes, + outcomeConversationRequestId, + ) const closeError = typeof input.closeError === 'string' ? input.closeError : undefined const rawModelLabel = rawFreshAgent.modelLabel const modelLabel = @@ -229,6 +258,8 @@ function normalizePaneContent( serverInstanceId: typeof input.serverInstanceId === 'string' ? input.serverInstanceId : undefined, restoreError: existingRestoreError, initialCwd: input.initialCwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), + ...(checkpointOutcomes ? { checkpointOutcomes } : {}), createError: input.createError, closeError, modelSelection: normalizeFreshAgentModelSelection( @@ -314,6 +345,8 @@ function normalizePaneContent( serverInstanceId: typeof input.serverInstanceId === 'string' ? input.serverInstanceId : undefined, ...('restoreError' in durableState && durableState.restoreError ? { restoreError: durableState.restoreError } : {}), initialCwd: input.initialCwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), + ...(checkpointOutcomes ? { checkpointOutcomes } : {}), createError: input.createError, modelSelection: normalizeFreshAgentModelSelection( (input as { modelSelection?: unknown }).modelSelection, diff --git a/src/store/persistedState.ts b/src/store/persistedState.ts index a1498926c..81cd50cae 100644 --- a/src/store/persistedState.ts +++ b/src/store/persistedState.ts @@ -20,7 +20,10 @@ import { stripScopedPaneTitleMetadata, stripSessionOwnedTabFreezeFlags, } from '@/lib/session-name-migration' -import { normalizeFreshAgentPaneModelSelection } from './paneTypes' +import { + normalizeFreshAgentCheckpointOutcomes, + normalizeFreshAgentPaneModelSelection, +} from './paneTypes' import { createLogger } from '@/lib/client-logger' import { isValidMachineIdStamp } from '@/lib/machine-identity' @@ -337,11 +340,16 @@ function normalizeTerminalContent(content: Record<string, unknown>): Record<stri } function normalizeFreshAgentContent(content: Record<string, unknown>): Record<string, unknown> { + const checkpointOutcomes = normalizeFreshAgentCheckpointOutcomes( + content.checkpointOutcomes, + typeof content.createRequestId === 'string' ? content.createRequestId : undefined, + ) const existingRestoreError = readRestoreError(content.restoreError) if (existingRestoreError) { const { sessionRef: _legacySessionRef, restoreError: _legacyRestoreError, + checkpointOutcomes: _legacyCheckpointOutcomes, // Vestigial per-pane display overrides (no writer since 2026-04): // dropped during rehydration, never constructed into pane content. showThinking: _legacyShowThinking, @@ -364,6 +372,7 @@ function normalizeFreshAgentContent(content: Record<string, unknown>): Record<st return { ...rest, restoreError: existingRestoreError, + ...(checkpointOutcomes ? { checkpointOutcomes } : {}), } } @@ -383,6 +392,7 @@ function normalizeFreshAgentContent(content: Record<string, unknown>): Record<st const { sessionRef: _legacySessionRef, restoreError: _legacyRestoreError, + checkpointOutcomes: _legacyCheckpointOutcomes, // Vestigial per-pane display overrides (no writer since 2026-04): // dropped during rehydration, never constructed into pane content. showThinking: _legacyShowThinking, @@ -410,6 +420,7 @@ function normalizeFreshAgentContent(content: Record<string, unknown>): Record<st return { ...restWithNormalizedModel, + ...(checkpointOutcomes ? { checkpointOutcomes } : {}), ...(durableState.sessionRef ? { sessionRef: durableState.sessionRef } : {}), ...(('restoreError' in durableState && durableState.restoreError) || existingRestoreError ? { restoreError: ('restoreError' in durableState && durableState.restoreError) || existingRestoreError } diff --git a/src/store/tabsSlice.ts b/src/store/tabsSlice.ts index 7a090b264..22bcac9c3 100644 --- a/src/store/tabsSlice.ts +++ b/src/store/tabsSlice.ts @@ -5,7 +5,11 @@ import { nanoid } from 'nanoid' import { closePane, initLayout, restoreLayout, removeLayout, replacePane, setPaneCloseError, updatePaneContent, updatePaneTitleByTerminalId, updatePaneTitle, markTabClosing, clearTabClosing, markPaneClosing, clearPaneClosing, hasAnyClosePending } from './panesSlice' import { clearTabAttention, clearPaneAttention } from './turnCompletionSlice.js' import { updateSessionActivity } from './sessionActivitySlice' -import type { PaneContent, PaneNode } from './paneTypes' +import type { + FreshAgentCheckpointWorkspace, + PaneContent, + PaneNode, +} from './paneTypes' import { findTabIdForSession, collectSessionRefsFromTabs, liveTerminalRowIdentity } from '@/lib/session-utils' import { getProviderLabel } from '@/lib/coding-cli-utils' import { basenameSegment } from '@shared/path-basename' @@ -1695,10 +1699,11 @@ export const reopenClosedTab = createAsyncThunk( export const openSessionTab = createAsyncThunk( 'tabs/openSessionTab', async ( - { sessionId, title, cwd, provider, sessionType, terminalId, managedRuntime, forceNew, firstUserMessage, isSubagent, isNonInteractive, hasTitle, liveTerminalOnly }: { + { sessionId, title, cwd, checkpointWorkspace, provider, sessionType, terminalId, managedRuntime, forceNew, firstUserMessage, isSubagent, isNonInteractive, hasTitle, liveTerminalOnly }: { sessionId: string title?: string cwd?: string + checkpointWorkspace?: FreshAgentCheckpointWorkspace provider?: CodingCliProviderName sessionType?: string terminalId?: string @@ -1738,6 +1743,7 @@ export const openSessionTab = createAsyncThunk( sessionType: resolvedSessionType, sessionId, cwd, + checkpointWorkspace, freshAgentProviderSettings, }) diff --git a/test/unit/client/components/panes/DirectoryPicker.test.tsx b/test/unit/client/components/panes/DirectoryPicker.test.tsx index a95cd2139..b2a514fc8 100644 --- a/test/unit/client/components/panes/DirectoryPicker.test.tsx +++ b/test/unit/client/components/panes/DirectoryPicker.test.tsx @@ -165,6 +165,11 @@ describe('DirectoryPicker', () => { it('validates and confirms directory on Enter', async () => { const { onConfirm } = renderDirectoryPicker() + mockApiPost.mockResolvedValueOnce({ + valid: true, + resolvedPath: '/tmp/selected', + checkpointWorkspaceEligible: true, + }) const input = screen.getByLabelText('Starting directory for Claude') fireEvent.change(input, { target: { value: '/tmp/selected' } }) @@ -173,7 +178,26 @@ describe('DirectoryPicker', () => { await waitFor(() => { expect(mockApiPost).toHaveBeenCalledWith('/api/files/validate-dir', { path: '/tmp/selected' }) }) - expect(onConfirm).toHaveBeenCalledWith('/resolved/path') + expect(onConfirm).toHaveBeenCalledWith('/tmp/selected', { + path: '/tmp/selected', + provenance: 'directory-picker', + }) + }) + + it('keeps account home as launch cwd without selecting it for checkpoints', async () => { + mockApiPost.mockResolvedValueOnce({ + valid: true, + resolvedPath: '/home/user', + checkpointWorkspaceEligible: false, + }) + const { onConfirm } = renderDirectoryPicker({ defaultCwd: '' }) + const input = screen.getByLabelText('Starting directory for Claude') + + fireEvent.change(input, { target: { value: '/home/user' } }) + fireEvent.keyDown(input, { key: 'Enter' }) + + await waitFor(() => expect(onConfirm).toHaveBeenCalledWith('/home/user')) + expect(onConfirm).toHaveBeenCalledTimes(1) }) it('confirms typed input on Enter when no suggestion is actively selected', async () => { @@ -322,6 +346,11 @@ describe('DirectoryPicker', () => { it('creates directory and confirms on success', async () => { mockApiGet.mockResolvedValueOnce({ directories: [] }) mockApiPost.mockResolvedValueOnce({ created: true, resolvedPath: '/tmp/new-dir' }) + mockApiPost.mockResolvedValueOnce({ + valid: true, + resolvedPath: '/tmp/new-dir', + checkpointWorkspaceEligible: true, + }) const { onConfirm } = renderDirectoryPicker({ defaultCwd: '' }) const input = screen.getByLabelText('Starting directory for Claude') @@ -333,7 +362,11 @@ describe('DirectoryPicker', () => { await waitFor(() => { expect(mockApiPost).toHaveBeenCalledWith('/api/files/mkdir', { path: '/tmp/new-dir' }) }) - expect(onConfirm).toHaveBeenCalledWith('/tmp/new-dir') + expect(mockApiPost).toHaveBeenCalledWith('/api/files/validate-dir', { path: '/tmp/new-dir' }) + expect(onConfirm).toHaveBeenCalledWith('/tmp/new-dir', { + path: '/tmp/new-dir', + provenance: 'directory-picker', + }) }) it('shows creating state while request is in flight', async () => { @@ -402,6 +435,11 @@ describe('DirectoryPicker', () => { mockApiPost .mockReturnValueOnce(new Promise((resolve) => { resolveFirst = resolve })) .mockReturnValueOnce(new Promise((resolve) => { resolveSecond = resolve })) + .mockResolvedValueOnce({ + valid: true, + resolvedPath: '/tmp/dir-b', + checkpointWorkspaceEligible: true, + }) const { onConfirm } = renderDirectoryPicker({ defaultCwd: '' }) const input = screen.getByLabelText('Starting directory for Claude') @@ -417,7 +455,10 @@ describe('DirectoryPicker', () => { resolveSecond!({ created: true, resolvedPath: '/tmp/dir-b' }) await waitFor(() => { - expect(onConfirm).toHaveBeenCalledWith('/tmp/dir-b') + expect(onConfirm).toHaveBeenCalledWith('/tmp/dir-b', { + path: '/tmp/dir-b', + provenance: 'directory-picker', + }) }) expect(onConfirm).toHaveBeenCalledTimes(1) }) @@ -451,6 +492,11 @@ describe('DirectoryPicker', () => { it('triggers create on Shift+Enter', async () => { mockApiGet.mockResolvedValueOnce({ directories: [] }) mockApiPost.mockResolvedValueOnce({ created: true, resolvedPath: '/tmp/shift-enter' }) + mockApiPost.mockResolvedValueOnce({ + valid: true, + resolvedPath: '/tmp/shift-enter', + checkpointWorkspaceEligible: true, + }) const { onConfirm } = renderDirectoryPicker({ defaultCwd: '' }) const input = screen.getByLabelText('Starting directory for Claude') @@ -460,7 +506,10 @@ describe('DirectoryPicker', () => { await waitFor(() => { expect(mockApiPost).toHaveBeenCalledWith('/api/files/mkdir', { path: '/tmp/shift-enter' }) }) - expect(onConfirm).toHaveBeenCalledWith('/tmp/shift-enter') + expect(onConfirm).toHaveBeenCalledWith('/tmp/shift-enter', { + path: '/tmp/shift-enter', + provenance: 'directory-picker', + }) }) }) diff --git a/test/unit/client/components/panes/PaneContainer.createContent.test.tsx b/test/unit/client/components/panes/PaneContainer.createContent.test.tsx index 8c26147c3..a2eb83e60 100644 --- a/test/unit/client/components/panes/PaneContainer.createContent.test.tsx +++ b/test/unit/client/components/panes/PaneContainer.createContent.test.tsx @@ -23,8 +23,9 @@ import { } from '@shared/settings' // Hoist mock functions so vi.mock can reference them -const { mockSend, mockTerminalView, saveServerSettingsPatchSpy } = vi.hoisted(() => ({ +const { mockSend, mockTerminalView, mockApiPost, saveServerSettingsPatchSpy } = vi.hoisted(() => ({ mockSend: vi.fn(), + mockApiPost: vi.fn(), mockTerminalView: vi.fn(({ tabId, paneId }: { tabId: string; paneId: string }) => ( <div data-testid={`terminal-${paneId}`}>Terminal for {tabId}/{paneId}</div> )), @@ -41,7 +42,7 @@ vi.mock('@/lib/ws-client', () => ({ vi.mock('@/lib/api', () => ({ api: { get: vi.fn().mockResolvedValue({ directories: [] }), - post: vi.fn().mockResolvedValue({ valid: true, resolvedPath: '/resolved/path' }), + post: (...args: unknown[]) => mockApiPost(...args), patch: vi.fn().mockResolvedValue({}), }, })) @@ -184,6 +185,7 @@ describe('createContentForType with ext: prefix', () => { beforeEach(() => { mockSend.mockClear() mockTerminalView.mockClear() + mockApiPost.mockReset().mockResolvedValue({ valid: true, resolvedPath: '/resolved/path' }) saveServerSettingsPatchSpy.mockClear() vi.stubGlobal('fetch', vi.fn().mockImplementation(async (url: string) => { if (url === '/api/terminals') return { ok: true, text: async () => '[]' } @@ -276,6 +278,11 @@ describe('createContentForType with ext: prefix', () => { }) it('creates fresh-agent content for freshclaude selections', async () => { + mockApiPost.mockResolvedValueOnce({ + valid: true, + resolvedPath: '/workspace/project', + checkpointWorkspaceEligible: true, + }) const node = createPickerNode('pane-1') const store = createStore( { layouts: { 'tab-1': node }, activePane: { 'tab-1': 'pane-1' } }, @@ -311,6 +318,11 @@ describe('createContentForType with ext: prefix', () => { kind: 'fresh-agent', sessionType: 'freshclaude', provider: 'claude', + initialCwd: '/workspace/project', + checkpointWorkspace: { + path: '/workspace/project', + provenance: 'directory-picker', + }, }) }) }) diff --git a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts new file mode 100644 index 000000000..219e28e75 --- /dev/null +++ b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts @@ -0,0 +1,129 @@ +import { describe, expect, it } from 'vitest' + +import panesReducer, { initLayout } from '@/store/panesSlice' +import type { PaneNode } from '@/store/paneTypes' + +describe('FreshAgent checkpoint outcomes', () => { + it('keeps bounded settled outcomes and drops transient or unrecognized fields', () => { + const state = panesReducer(undefined, initLayout({ + tabId: 'tab-checkpoints', + paneId: 'pane-checkpoints', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'create-1', + status: 'running', + checkpointOutcomes: [ + { + requestId: 'send-available', + conversationRequestId: 'create-1', + displayTurnId: 'turn-4:row-0', + nativeTurnId: 'native-user-4', + status: 'available', + checkpointId: 'checkpoint-4', + workspace: { path: '/work/project', provenance: 'directory-picker' }, + outgoingText: 'must never be retained', + }, + { + requestId: 'send-unavailable', + conversationRequestId: 'create-1', + displayTurnId: 'turn-5:row-0', + status: 'unavailable', + code: 'checkpoint_path_unreadable', + reason: 'A selected file could not be read.\nRetry after fixing permissions.', + outgoingText: 'must never be retained', + }, + { + requestId: 'send-pending', + status: 'pending', + }, + { + requestId: 'bad identity with spaces', + status: 'unavailable', + reason: 'invalid identity', + }, + { + requestId: 'stale-conversation-outcome', + conversationRequestId: 'create-replaced', + status: 'unavailable', + code: 'capture_failed', + reason: 'belongs to the previous conversation', + }, + ], + } as never, + })) + + const leaf = state.layouts['tab-checkpoints'] as Extract<PaneNode, { type: 'leaf' }> + expect(leaf.content).toMatchObject({ + kind: 'fresh-agent', + checkpointOutcomes: [ + { + requestId: 'send-available', + conversationRequestId: 'create-1', + displayTurnId: 'turn-4:row-0', + nativeTurnId: 'native-user-4', + status: 'available', + checkpointId: 'checkpoint-4', + workspace: { path: '/work/project', provenance: 'directory-picker' }, + }, + { + requestId: 'send-unavailable', + conversationRequestId: 'create-1', + displayTurnId: 'turn-5:row-0', + status: 'unavailable', + code: 'checkpoint_path_unreadable', + reason: 'A selected file could not be read. Retry after fixing permissions.', + }, + ], + }) + expect(JSON.stringify(leaf.content)).not.toContain('must never be retained') + }) + + it('keeps only the newest 200 settled outcomes and deduplicates by request id', () => { + const outcomes = Array.from({ length: 202 }, (_, index) => ({ + requestId: `send-${index}`, + displayTurnId: `turn-${index}`, + status: 'unavailable', + code: 'capture_failed', + reason: `failure ${index}`, + })) + outcomes[201] = { + requestId: 'send-201', + displayTurnId: 'turn-201', + status: 'available', + checkpointId: 'checkpoint-latest', + workspace: { path: '/work/project', provenance: 'resolved-project' }, + } as typeof outcomes[number] + outcomes.push({ + requestId: 'send-201', + displayTurnId: 'turn-201', + status: 'unavailable', + code: 'late_failure', + reason: 'later result for the same request', + }) + + const state = panesReducer(undefined, initLayout({ + tabId: 'tab-checkpoints', + paneId: 'pane-checkpoints', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'create-1', + status: 'running', + checkpointOutcomes: outcomes, + } as never, + })) + const leaf = state.layouts['tab-checkpoints'] as Extract<PaneNode, { type: 'leaf' }> + const stored = (leaf.content as { checkpointOutcomes?: Array<Record<string, unknown>> }).checkpointOutcomes! + + expect(stored).toHaveLength(200) + expect(stored[0].requestId).toBe('send-2') + expect(stored.at(-1)).toMatchObject({ + requestId: 'send-201', + status: 'unavailable', + code: 'late_failure', + }) + }) +}) diff --git a/test/unit/client/store/persistControl.test.ts b/test/unit/client/store/persistControl.test.ts index 847f8387f..2711247eb 100644 --- a/test/unit/client/store/persistControl.test.ts +++ b/test/unit/client/store/persistControl.test.ts @@ -15,6 +15,7 @@ import { getCanonicalDurableSessionId, getPreferredResumeSessionId, } from '@/store/persistControl' +import { parsePersistedLayoutRaw } from '@/store/persistedState' import tabsReducer, { addTab, updateTab } from '@/store/tabsSlice' const WINDOW_ID = 'client-persist-control-tests' @@ -64,6 +65,56 @@ describe('persistControl', () => { expect(JSON.parse(raw!).tabs.tabs[0].title).toBe('Renamed') }) + it('persists settled checkpoint outcomes and restores their bounded whitelist', () => { + const store = configureStore({ + reducer: { + freshAgent: freshAgentReducer, + tabs: tabsReducer, + panes: panesReducer, + }, + middleware: (getDefault) => getDefault().concat(persistMiddleware as any), + }) + + store.dispatch(addTab({ id: 'tab-1', title: 'Checkpoint test', mode: 'freshclaude' })) + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'create-1', + status: 'running', + checkpointOutcomes: [{ + conversationRequestId: 'create-1', + requestId: 'send-1', + displayTurnId: 'turn-4:row-0', + nativeTurnId: 'native-user-4', + status: 'unavailable', + code: 'checkpoint_path_unreadable', + reason: 'A selected file could not be read.', + outgoingText: 'never persist this prompt', + }], + } as never, + })) + store.dispatch(flushPersistedLayoutNow()) + + const raw = localStorage.getItem(OWN_LAYOUT_KEY) + expect(raw).not.toBeNull() + const parsed = parsePersistedLayoutRaw(raw!)! + const savedContent = (parsed.panes.layouts['tab-1'] as any).content + expect(savedContent.checkpointOutcomes).toEqual([{ + conversationRequestId: 'create-1', + requestId: 'send-1', + displayTurnId: 'turn-4:row-0', + nativeTurnId: 'native-user-4', + status: 'unavailable', + code: 'checkpoint_path_unreadable', + reason: 'A selected file could not be read.', + }]) + expect(raw).not.toContain('never persist this prompt') + }) + it('does not force immediate flush for unrelated session updates', () => { const store = configureStore({ reducer: { From 4f705c01ee8b4bc2660cf30168e50ef06b40c5d0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 02:35:06 -0700 Subject: [PATCH 304/493] fix: stop checkpoint git on request cancellation --- crates/freshell-server/src/checkpoints.rs | 15 +- .../tests/code_checkpoint_permissions.rs | 215 +++++++++++++++++- 2 files changed, 224 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index e26440357..422a21521 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -828,6 +828,10 @@ async fn run_git_with_index_input( input: &[u8], ) -> Result<(), String> { let mut cmd = Command::new("git"); + // Request cancellation drops this future. Tokio leaves child processes + // running by default, so every checkpoint-owned Git process must be + // configured to stop with its request instead of outliving it. + cmd.kill_on_drop(true); cmd.args(args).stdin(Stdio::piped()); if let Some(dir) = cwd { cmd.current_dir(dir); @@ -862,14 +866,17 @@ async fn ignored_git_directories( let git_dir_arg = format!("--git-dir={}", git_dir.display()); let work_tree_arg = format!("--work-tree={}", work_tree.display()); - let mut child = Command::new("git") + let mut command = Command::new("git"); + command + .kill_on_drop(true) .arg(git_dir_arg) .arg(work_tree_arg) .args(["check-ignore", "--no-index", "--stdin", "-z"]) .current_dir(work_tree) .stdin(Stdio::piped()) .stdout(Stdio::piped()) - .stderr(Stdio::piped()) + .stderr(Stdio::piped()); + let mut child = command .spawn() .map_err(|error| format!("git check-ignore failed to start: {error}"))?; let mut input = Vec::new(); @@ -980,6 +987,10 @@ async fn invoke_git( index: Option<&Path>, ) -> Result<std::process::Output, String> { let mut cmd = Command::new("git"); + // Unlike JavaScript child_process, Tokio's default is to leave a child + // alive when its owner future is dropped. HTTP cancellation must not + // detach an in-flight snapshot command. + cmd.kill_on_drop(true); cmd.args(args); if let Some(dir) = cwd { cmd.current_dir(dir); diff --git a/crates/freshell-server/tests/code_checkpoint_permissions.rs b/crates/freshell-server/tests/code_checkpoint_permissions.rs index 893ea5541..c2bec8583 100644 --- a/crates/freshell-server/tests/code_checkpoint_permissions.rs +++ b/crates/freshell-server/tests/code_checkpoint_permissions.rs @@ -6,6 +6,7 @@ //! code_checkpoint_permissions"`; the trusted entrypoint creates the root-owned //! unreadable sentinels and this test observes them only as the sandbox user. +use std::ffi::OsString; use std::fs; use std::os::unix::fs::{MetadataExt, PermissionsExt}; use std::path::{Path, PathBuf}; @@ -15,6 +16,8 @@ use std::time::{Duration, Instant}; use reqwest::{Client, StatusCode}; use serde_json::{Value, json}; use sha1::{Digest, Sha1}; +use tokio::io::AsyncWriteExt; +use tokio::net::TcpStream; const AUTH_TOKEN: &str = "checkpoint-fixture-auth-marker-do-not-log"; const OUTGOING_MARKER: &str = "checkpoint-fixture-outgoing-text-do-not-log"; @@ -22,8 +25,9 @@ const OUTGOING_MARKER: &str = "checkpoint-fixture-outgoing-text-do-not-log"; struct OwnedServer(Option<Child>); impl OwnedServer { - fn start(port: u16, home: &Path, log_dir: &Path) -> Self { - let child = Command::new(env!("CARGO_BIN_EXE_freshell-server")) + fn start(port: u16, home: &Path, log_dir: &Path, path: Option<OsString>) -> Self { + let mut command = Command::new(env!("CARGO_BIN_EXE_freshell-server")); + command .env("PORT", port.to_string()) .env("AUTH_TOKEN", AUTH_TOKEN) .env("FRESHELL_HOME", home) @@ -31,7 +35,11 @@ impl OwnedServer { .env("FRESHELL_LOG_DIR", log_dir) .env("FRESHELL_BIND_HOST", "127.0.0.1") .env("FRESHELL_DISABLE_WSL_PORT_FORWARD", "1") - .env_remove("FRESHELL_MANAGED_RUNTIME_V1") + .env_remove("FRESHELL_MANAGED_RUNTIME_V1"); + if let Some(path) = path { + command.env("PATH", path); + } + let child = command .stdout(Stdio::null()) .stderr(Stdio::null()) .spawn() @@ -51,6 +59,33 @@ impl OwnedServer { } } +struct GitBarrierCleanup { + pid_path: PathBuf, + release_path: PathBuf, + wrapper_path: PathBuf, + wrapper_dir: PathBuf, +} + +impl Drop for GitBarrierCleanup { + fn drop(&mut self) { + if let (Ok(raw_pid), Ok(wrapper)) = ( + fs::read_to_string(&self.pid_path), + self.wrapper_path.to_str().ok_or(()), + ) { + if let Ok(pid) = raw_pid.trim().parse::<i32>() { + let command_line = fs::read(format!("/proc/{pid}/cmdline")).unwrap_or_default(); + if String::from_utf8_lossy(&command_line).contains(wrapper) { + unsafe { + libc::kill(pid, libc::SIGKILL); + } + } + } + } + let _ = fs::write(&self.release_path, "release\n"); + let _ = fs::remove_dir_all(&self.wrapper_dir); + } +} + impl Drop for OwnedServer { fn drop(&mut self) { self.stop(); @@ -240,7 +275,7 @@ async fn restricted_checkpoint_routes_preserve_complete_targets_and_log_correlat fs::create_dir_all(storage.join("logs")).unwrap(); let port = allocate_port(); - let mut server = OwnedServer::start(port, &home, &storage.join("logs")); + let mut server = OwnedServer::start(port, &home, &storage.join("logs"), None); wait_for_health(port, server.child_mut()).await; let client = Client::new(); @@ -579,3 +614,175 @@ async fn restricted_checkpoint_routes_preserve_complete_targets_and_log_correlat server.stop(); } + +#[cfg(target_os = "linux")] +#[tokio::test] +async fn dropped_checkpoint_http_request_kills_git_before_publication() { + let root = fixture_root(); + assert_ne!(unsafe { libc::geteuid() }, 0, "fixture must run as UID B"); + assert_eq!(effective_capabilities(), 0); + + let home = root.join("cancel-home"); + let storage = root.join("cancel-storage"); + let project = root.join("cancel-project"); + let executable_tmp = std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") + .map(PathBuf::from) + .expect("runtime sandbox provides an executable fixture directory"); + let wrapper_dir = executable_tmp.join(format!("checkpoint-git-wrapper-{}", std::process::id())); + let barrier_dir = root.join("cancel-git-barrier"); + fs::create_dir_all(&home).unwrap(); + fs::create_dir_all(storage.join("logs")).unwrap(); + fs::create_dir_all(&project).unwrap(); + fs::create_dir_all(&wrapper_dir).unwrap(); + fs::create_dir_all(&barrier_dir).unwrap(); + fs::write( + project.join("source.txt"), + "must never publish after abort\n", + ) + .unwrap(); + run_git(&["init", "-q"], &project); + + let pid_path = barrier_dir.join("git-add.pid"); + let started_path = barrier_dir.join("git-add.started"); + let release_path = barrier_dir.join("git-add.release"); + let calls_path = barrier_dir.join("git.calls"); + let real_git = std::env::split_paths(&std::env::var_os("PATH").unwrap_or_default()) + .map(|directory| directory.join("git")) + .find(|candidate| candidate.is_file()) + .expect("sandbox PATH contains real Git"); + let wrapper = format!( + "#!/bin/sh\n\ + if [ \"$1\" = \"--version\" ]; then echo WRAPPED; exit 0; fi\n\ + target=0\n\ + add=0\n\ + printf '%s\\n' \"$*\" >> {}\n\ + for arg do\n\ + case \"$arg\" in\n\ + --work-tree=*) case \"$arg\" in *cancel-project) target=1 ;; esac ;;\n\ + add) add=1 ;;\n\ + esac\n\ + done\n\ + if [ \"$target\" -eq 1 ] && [ \"$add\" -eq 1 ]; then\n\ + printf '%s\\n' \"$$\" > {}\n\ + printf '%s\\n' started > {}\n\ + while [ ! -e {} ]; do :; done\n\ + fi\n\ + exec {} \"$@\"\n", + calls_path.display(), + pid_path.display(), + started_path.display(), + release_path.display(), + real_git.display(), + ); + let wrapper_path = wrapper_dir.join("git"); + let _barrier_cleanup = GitBarrierCleanup { + pid_path: pid_path.clone(), + release_path: release_path.clone(), + wrapper_path: wrapper_path.clone(), + wrapper_dir: wrapper_dir.clone(), + }; + fs::write(&wrapper_path, wrapper).unwrap(); + fs::set_permissions(&wrapper_path, fs::Permissions::from_mode(0o755)).unwrap(); + + let original_path = std::env::var_os("PATH").unwrap_or_default(); + let server_path = std::env::join_paths( + std::iter::once(wrapper_dir.clone()).chain(std::env::split_paths(&original_path)), + ) + .unwrap(); + let wrapper_probe = Command::new("git") + .arg("--version") + .env("PATH", &server_path) + .output() + .expect("resolve the test Git wrapper from the server PATH"); + assert_eq!( + String::from_utf8_lossy(&wrapper_probe.stdout).trim(), + "WRAPPED", + "server PATH resolves git to the barrier wrapper" + ); + let port = allocate_port(); + let mut server = OwnedServer::start(port, &home, &storage.join("logs"), Some(server_path)); + wait_for_health(port, server.child_mut()).await; + let server_pid = server.child_mut().id(); + let server_environment = fs::read(format!("/proc/{server_pid}/environ")).unwrap(); + let path_entry = server_environment + .split(|byte| *byte == 0) + .find(|entry| entry.starts_with(b"PATH=")) + .expect("server process has PATH"); + assert!( + String::from_utf8_lossy(path_entry).contains(&wrapper_dir.display().to_string()), + "server PATH includes test barrier directory: {}", + String::from_utf8_lossy(path_entry) + ); + + let body = json!({ + "cwd": home, + "workspace": workspace_intent(&project), + "label": "aborted capture", + "requestId": "send-aborted-capture", + }) + .to_string(); + let request = format!( + "POST /api/fresh-agent/checkpoints HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\nConnection: close\r\nX-Auth-Token: {AUTH_TOKEN}\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{body}", + body.len() + ); + let mut stream = TcpStream::connect(("127.0.0.1", port)) + .await + .expect("connect to checkpoint route"); + stream + .write_all(request.as_bytes()) + .await + .expect("send complete checkpoint request"); + + let deadline = Instant::now() + Duration::from_secs(10); + while !started_path.exists() && Instant::now() < deadline { + tokio::time::sleep(Duration::from_millis(10)).await; + } + assert!( + started_path.exists(), + "Git add reached the controlled barrier; Git calls were {}; server logs were {}", + fs::read_to_string(&calls_path).unwrap_or_default(), + fs::read_to_string(storage.join("logs/rust-server.jsonl")).unwrap_or_default() + ); + let git_pid = fs::read_to_string(&pid_path) + .unwrap() + .trim() + .parse::<i32>() + .unwrap(); + + // Closing the actual HTTP connection drops the Axum handler future while + // Git is still blocked before it can write this request's private index. + drop(stream); + let stopped_deadline = Instant::now() + Duration::from_secs(5); + while linux_process_state(git_pid).is_some_and(|state| state != 'Z') + && Instant::now() < stopped_deadline + { + tokio::time::sleep(Duration::from_millis(10)).await; + } + assert!( + linux_process_state(git_pid).is_none_or(|state| state == 'Z'), + "Git child {git_pid} must stop executing after the HTTP request is dropped" + ); + + let client = Client::new(); + let base = format!("http://127.0.0.1:{port}"); + let checkpoints = list_workspace(&client, &base, &home, &project).await; + assert!(checkpoints.is_empty(), "aborted capture is not published"); + tokio::time::sleep(Duration::from_millis(200)).await; + let later_checkpoints = list_workspace(&client, &base, &home, &project).await; + assert!( + later_checkpoints.is_empty(), + "aborted capture does not publish after the cancellation settles" + ); + + // Release only after the no-publication assertion so a failed test cannot + // leave the controlled child spinning in the disposable container. + fs::write(&release_path, "release\n").unwrap(); + server.stop(); +} + +#[cfg(target_os = "linux")] +fn linux_process_state(pid: i32) -> Option<char> { + let stat = fs::read_to_string(format!("/proc/{pid}/stat")).ok()?; + let (_, fields) = stat.rsplit_once(") ")?; + fields.chars().next() +} From db8edc5ed15b5d9d29b17591eb1daf7045730d84 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 02:52:27 -0700 Subject: [PATCH 305/493] Use server-owned fresh-agent checkpoint outcomes --- shared/ws-protocol.ts | 11 + src/components/HistoryView.tsx | 13 +- src/components/ResumeSessionDialog.tsx | 16 +- src/components/Sidebar.tsx | 6 + src/components/TabContent.tsx | 1 + src/components/TerminalView.tsx | 9 + .../context-menu/ContextMenuProvider.tsx | 24 +- src/components/fresh-agent/FreshAgentView.tsx | 357 ++++++------- src/lib/resume-session.ts | 32 +- src/store/paneTypes.ts | 15 +- src/store/persistedState.ts | 9 + src/store/selectors/sidebarSelectors.ts | 11 +- src/store/tabsSlice.ts | 4 + src/store/types.ts | 9 + .../components/ContextMenuProvider.test.tsx | 43 +- .../components/ResumeSessionDialog.test.tsx | 22 + test/unit/client/components/Sidebar.test.tsx | 41 ++ .../client/components/TabContent.test.tsx | 8 + .../TerminalView.lifecycle.test.tsx | 9 +- .../fresh-agent/FreshAgentView.test.tsx | 490 ++++++++++-------- .../lib/resume-session.workspace.test.ts | 94 ++++ .../panesSlice.checkpoint-outcomes.test.ts | 8 + test/unit/client/store/persistControl.test.ts | 1 + test/unit/client/store/tabsSlice.test.ts | 4 + 24 files changed, 775 insertions(+), 462 deletions(-) create mode 100644 test/unit/client/lib/resume-session.workspace.test.ts diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index 8b8f1e04c..26022244b 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -899,6 +899,16 @@ export const FreshAgentSendSchema = z.object({ sessionId: z.string().min(1), sessionType: z.enum(['freshclaude', 'freshcodex', 'kilroy', 'freshopencode']), provider: z.enum(['claude', 'codex', 'opencode']), + /** kata bdpt Task 7: one server-owned pre-turn checkpoint barrier is + * correlated with this provider attempt and dispatched as part of this + * same send. Explicit workspace scope is persisted pane intent only. */ + checkpointIntent: z.object({ + logicalAttemptId: z.string().min(1).max(128), + workspace: z.object({ + path: z.string().min(1).max(4096), + provenance: z.enum(['directory-picker', 'resolved-project', 'legacy-checkpoint']), + }).optional(), + }).optional(), /** b8ke ext r8 F5: the send's delayed-request fence (crashed-session * recovery re-claims carry it, so a stale queued send is typed-refused, * never an unfenced recreation). */ @@ -1858,6 +1868,7 @@ export type FreshAgentServerMessage = ownerKind?: 'terminal' | 'fresh-agent' ownerGeneration?: number ownerEpoch?: number } + | { type: 'freshAgent.checkpoint.outcome'; logicalAttemptId: string; requestId: string; provider: 'claude' | 'codex' | 'opencode'; sessionId: string; sessionType: 'freshclaude' | 'freshcodex' | 'kilroy' | 'freshopencode'; status: 'available' | 'unavailable'; checkpointId?: string; workspace?: { path: string; provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' }; code?: string; reason?: string } | { type: 'freshAgent.send.accepted'; requestId: string; sessionId: string; sessionType: string; provider: string; submittedTurnId?: string; cwd?: string } | { type: 'freshAgent.event'; sessionId: string; sessionType: string; provider: string; event: unknown } | { type: 'freshAgent.session.materialized'; previousSessionId: string; sessionId: string; sessionType: string; provider: string; sessionRef?: { provider: string; sessionId: string }; sessionName?: SessionNameRecord; nameRef?: SessionNameRef } diff --git a/src/components/HistoryView.tsx b/src/components/HistoryView.tsx index 37d1b8204..950515937 100644 --- a/src/components/HistoryView.tsx +++ b/src/components/HistoryView.tsx @@ -6,8 +6,7 @@ import { removeSessionFromProjects, toggleProjectExpanded } from '@/store/sessio import { api } from '@/lib/api' import { activateSessionSurface, fetchSessionWindow, type FetchSessionWindowResult } from '@/store/sessionsThunks' import { openSessionTab } from '@/store/tabsSlice' -import { findPaneContent } from '@/lib/pane-utils' -import { findPaneForSession } from '@/lib/session-utils' +import { findCheckpointWorkspaceForSession } from '@/lib/resume-session' import { applySessionRenameCascade } from '@/store/titleSync' import { receiveSessionNameProjections, receiveSessionNames } from '@/store/sessionNamesSlice' import { renameSessionName } from '@/lib/session-names' @@ -233,14 +232,8 @@ export default function HistoryView({ onOpenSession }: { onOpenSession?: () => v const mode = (session.provider || 'claude') as CodingCliProviderName const displayName = sessionDisplayName(session, sessionNames) const state = store.getState() - const existing = findPaneForSession( - state, - { provider: mode, sessionId: session.sessionId }, - state.connection?.serverInstanceId, - ) - const layout = existing?.paneId ? state.panes.layouts[existing.tabId] : undefined - const content = layout && existing?.paneId ? findPaneContent(layout, existing.paneId) : null - const checkpointWorkspace = content?.kind === 'fresh-agent' ? content.checkpointWorkspace : undefined + const checkpointWorkspace = session.checkpointWorkspace + ?? findCheckpointWorkspaceForSession(state, { provider: mode, sessionId: session.sessionId }) dispatch(openSessionTab({ sessionId: session.sessionId, title: displayName || label, diff --git a/src/components/ResumeSessionDialog.tsx b/src/components/ResumeSessionDialog.tsx index 203f2019e..862d8daef 100644 --- a/src/components/ResumeSessionDialog.tsx +++ b/src/components/ResumeSessionDialog.tsx @@ -2,7 +2,11 @@ import { useCallback, useEffect, useRef, useState } from 'react' import { createPortal } from 'react-dom' import { useStore } from 'react-redux' import { api } from '@/lib/api' -import { resumeSessionInTab, type ResumeTarget } from '@/lib/resume-session' +import { + findCheckpointWorkspaceForSession, + resumeSessionInTab, + type ResumeTarget, +} from '@/lib/resume-session' import { OVERLAY_Z } from '@/components/ui/overlay' import { useAppDispatch } from '@/store/hooks' import type { RootState } from '@/store/store' @@ -91,7 +95,15 @@ export function ResumeSessionDialog({ open, onClose, onNavigate }: ResumeSession const finishResume = useCallback( (target: ResumeTarget, note: string) => { - resumeSessionInTab(store.getState(), dispatch, target, onNavigate) + const state = store.getState() + const checkpointWorkspace = target.checkpointWorkspace + ?? findCheckpointWorkspaceForSession(state, target) + resumeSessionInTab( + state, + dispatch, + { ...target, ...(checkpointWorkspace ? { checkpointWorkspace } : {}) }, + onNavigate, + ) setPhase({ kind: 'resumed', note }) closeTimerRef.current = window.setTimeout(onClose, RESUMED_CLOSE_MS) }, diff --git a/src/components/Sidebar.tsx b/src/components/Sidebar.tsx index dc5d3cc3d..76b3ba2bf 100644 --- a/src/components/Sidebar.tsx +++ b/src/components/Sidebar.tsx @@ -9,6 +9,7 @@ import { shallowEqual } from 'react-redux' import { openSessionTab, setActiveTab, updateTab } from '@/store/tabsSlice' import { addPane, setActivePane, updatePaneTitle } from '@/store/panesSlice' import { findPaneForSession } from '@/lib/session-utils' +import { findCheckpointWorkspaceForSession } from '@/lib/resume-session' import { resolveSessionTypeConfig, buildResumeContent } from '@/lib/session-type-utils' import { getFreshAgentProviderConfig } from '@/lib/fresh-agent-provider-utils' import { resolveFreshAgentType } from '@/lib/fresh-agent-registry' @@ -477,6 +478,8 @@ export default function Sidebar({ const handleItemClick = useCallback((item: SessionItem) => { const provider = item.provider as CodingCliProviderName const state = store.getState() + const checkpointWorkspace = item.checkpointWorkspace + ?? findCheckpointWorkspaceForSession(state, { provider, sessionId: item.sessionId }) const currentActiveTabId = state.tabs.activeTabId const runningTerminalId = item.isRunning || item.soulId ? item.runningTerminalId : undefined const localServerInstanceId = state.connection.serverInstanceId @@ -524,6 +527,7 @@ export default function Sidebar({ sessionId: item.sessionId, title: item.title, cwd: item.cwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), provider, sessionType, terminalId: runningTerminalId, @@ -545,6 +549,7 @@ export default function Sidebar({ sessionId: item.sessionId, title: item.title, cwd: item.cwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), provider, sessionType, terminalId: runningTerminalId, @@ -565,6 +570,7 @@ export default function Sidebar({ sessionType, sessionId: item.sessionId, cwd: item.cwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), freshAgentProviderSettings, liveTerminal: runningTerminalId && localServerInstanceId ? { terminalId: runningTerminalId, serverInstanceId: localServerInstanceId, ...managedTerminalProjection(item) } diff --git a/src/components/TabContent.tsx b/src/components/TabContent.tsx index afd754cc8..ac4e3051f 100644 --- a/src/components/TabContent.tsx +++ b/src/components/TabContent.tsx @@ -37,6 +37,7 @@ export default function TabContent({ tabId, hidden }: TabContentProps) { sessionType: resumeSessionType, sessionId: tabSessionId, cwd: tab.initialCwd, + checkpointWorkspace: tab.checkpointWorkspace, }) } else if (tab.mode !== 'shell') { defaultContent = { diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index f8f40b791..5ca277359 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -73,6 +73,7 @@ import { TerminalStuckCard } from '@/components/TerminalStuckCard' import { TerminalLaunchFailureCard } from '@/components/TerminalLaunchFailureCard' import { FencedOwnerRecoveryActions, SessionHandoffErrorBanner } from '@/components/SessionHandoffErrorBanner' import { buildResumeContent, freshSessionTypeForPaneFlavor } from '@/lib/session-type-utils' +import { findCheckpointWorkspaceForSession } from '@/lib/resume-session' import type { LaunchFailure } from '@/store/paneTypes' import { buildManagedPreparationFailure, retainedManagedPreparationFailure } from '@/store/paneTypes' import { dismissTabGreen } from '@/store/turnCompletionAttention' @@ -7324,6 +7325,13 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const state = appStore.getState() const canonical = resolveCanonicalPaneSession(state, terminalContent) const sessionId = canonical?.sessionId ?? sessionRef.sessionId + const tabCheckpointWorkspace = state.tabs.tabs.find((candidate) => candidate.id === tabId) + const carriesThisSession = tabCheckpointWorkspace?.sessionRef?.provider === sessionRef.provider + && tabCheckpointWorkspace.sessionRef.sessionId === sessionRef.sessionId + const checkpointWorkspace = findCheckpointWorkspaceForSession(state, { + provider: sessionRef.provider, + sessionId: sessionRef.sessionId, + }) ?? (carriesThisSession ? tabCheckpointWorkspace?.checkpointWorkspace : undefined) dispatch(updatePaneContent({ tabId, paneId, @@ -7331,6 +7339,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te sessionType, sessionId, cwd: terminalContent.initialCwd, + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), ...(providerSettings ? { freshAgentProviderSettings: providerSettings } : {}), }), })) diff --git a/src/components/context-menu/ContextMenuProvider.tsx b/src/components/context-menu/ContextMenuProvider.tsx index 7c3375523..2de85d48a 100644 --- a/src/components/context-menu/ContextMenuProvider.tsx +++ b/src/components/context-menu/ContextMenuProvider.tsx @@ -27,8 +27,9 @@ import { buildShareUrl } from '@/lib/utils' import { copyText } from '@/lib/clipboard' import { openExternalUrl } from '@/lib/open-url' import { triggerHapticFeedback } from '@/lib/mobile-haptics' -import { collectPaneEntries, collectTerminalIds, findPaneContent } from '@/lib/pane-utils' +import { collectPaneEntries, collectTerminalIds } from '@/lib/pane-utils' import { collectSessionRefsFromNode, findPaneForSession } from '@/lib/session-utils' +import { findCheckpointWorkspaceForSession } from '@/lib/resume-session' import { getTabDisplayTitle } from '@/lib/tab-title' import { getBrowserActions, getEditorActions, getTerminalActions } from '@/lib/pane-action-registry' import { buildResumeCommand, type ResumeCommandProvider } from '@/lib/coding-cli-utils' @@ -417,17 +418,7 @@ export function ContextMenuProvider({ }, [getProjectCollections]) const getExistingCheckpointWorkspace = useCallback((provider: CodingCliProviderName, sessionId: string) => { - const state = appStore.getState() - const existing = findPaneForSession( - state, - { provider, sessionId }, - state.connection?.serverInstanceId, - ) - if (!existing?.paneId) return undefined - const layout = state.panes.layouts[existing.tabId] - if (!layout) return undefined - const content = findPaneContent(layout, existing.paneId) - return content?.kind === 'fresh-agent' ? content.checkpointWorkspace : undefined + return findCheckpointWorkspaceForSession(appStore.getState(), { provider, sessionId }) }, [appStore]) const getProjectInfo = useCallback((projectPath: string, target?: ContextTarget | null) => { @@ -468,7 +459,8 @@ export function ContextMenuProvider({ const mode = (provider || session.provider || 'claude') as CodingCliProviderName const sessionType = (target?.kind === 'sidebar-session' ? target.sessionType : undefined) || session.sessionType || mode - const checkpointWorkspace = getExistingCheckpointWorkspace(mode, session.sessionId) + const checkpointWorkspace = session.checkpointWorkspace + ?? getExistingCheckpointWorkspace(mode, session.sessionId) const runningTerminalId = target?.kind === 'sidebar-session' && target.sessionId === sessionId ? target.runningTerminalId @@ -502,7 +494,8 @@ export function ContextMenuProvider({ const mode = (provider || session.provider || 'claude') as CodingCliProviderName const sessionType = (target?.kind === 'sidebar-session' ? target.sessionType : undefined) || session.sessionType || mode - const checkpointWorkspace = getExistingCheckpointWorkspace(mode, session.sessionId) + const checkpointWorkspace = session.checkpointWorkspace + ?? getExistingCheckpointWorkspace(mode, session.sessionId) const freshAgentType = resolveFreshAgentType(sessionType) const freshAgentProviderConfig = getFreshAgentProviderConfig(sessionType) const freshAgentProviderSettings = freshAgentType || freshAgentProviderConfig @@ -815,7 +808,8 @@ export function ContextMenuProvider({ if (project) { for (const session of project.sessions) { const provider = (session.provider || 'claude') as CodingCliProviderName - const checkpointWorkspace = getExistingCheckpointWorkspace(provider, session.sessionId) + const checkpointWorkspace = session.checkpointWorkspace + ?? getExistingCheckpointWorkspace(provider, session.sessionId) dispatch(openSessionTab({ sessionId: session.sessionId, title: session.title || session.sessionId.slice(0, 8), diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 581b358b9..7e742df9c 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -9,7 +9,11 @@ import { type KeyboardEvent as ReactKeyboardEvent, } from 'react' import { nanoid } from 'nanoid' -import type { FreshAgentCheckpointOutcome, FreshAgentPaneContent } from '@/store/paneTypes' +import { + normalizeFreshAgentCheckpointWorkspace, + type FreshAgentCheckpointOutcome, + type FreshAgentPaneContent, +} from '@/store/paneTypes' import type { PaneReconcileRequest } from '@shared/ws-protocol' import { isUnifiedAgentMode } from '@shared/session-names' import { useAppDispatch, useAppSelector, useAppStore } from '@/store/hooks' @@ -98,7 +102,6 @@ import { } from '@/lib/fresh-agent-snapshot-thread' import { DEFAULT_FRESH_AGENT_STYLE, normalizeFreshAgentStyle } from '@shared/settings' import { - checkpointLabelForText, pickCheckpointForTurn, type CheckpointEntry, } from '@/lib/fresh-agent-checkpoints' @@ -231,15 +234,12 @@ function sameLocalEcho(a: LocalEcho | null | undefined, b: LocalEcho | null | un } type PendingSendMetadata = { - cwd?: string checkpointAvailabilityScope?: string checkpointWorkspace?: NonNullable<FreshAgentPaneContent['checkpointWorkspace']> - checkpointId?: string - checkpointFailureReason?: string - checkpointFailureCode?: string + logicalAttemptId?: string + checkpointOutcome?: CheckpointTurnAvailability submittedTurnId?: string legacyAccepted?: boolean - metadataUpdateStarted?: boolean /** The exact text of the freshAgent.send frame -- retained as the resend * payload for the lost-session retry (Task 10). Never read back from the * local echo. */ @@ -256,8 +256,6 @@ type CheckpointTurnAvailability = FreshAgentTurnRewindAvailability & { code?: string } -const CHECKPOINT_CAPTURE_TIMEOUT_MS = 30_000 -const CHECKPOINT_CAPTURE_TIMEOUT_REASON = 'Code checkpoint capture timed out after 30 seconds.' const CHECKPOINT_FAILURE_FALLBACK = 'The project workspace could not be saved before this message was sent.' function sanitizeCheckpointReason(value: string): string { @@ -269,16 +267,6 @@ function sanitizeCheckpointReason(value: string): string { .slice(0, 240) } -function isCheckpointWorkspace(value: unknown): value is NonNullable<FreshAgentPaneContent['checkpointWorkspace']> { - if (!value || typeof value !== 'object' || Array.isArray(value)) return false - const record = value as Record<string, unknown> - return typeof record.path === 'string' - && record.path.length > 0 - && (record.provenance === 'directory-picker' - || record.provenance === 'resolved-project' - || record.provenance === 'legacy-checkpoint') -} - function checkpointFailureReason(error: unknown): string { if (error && typeof error === 'object') { const outer = error as { details?: unknown; code?: unknown } @@ -1086,6 +1074,7 @@ export function FreshAgentView({ === checkpointTurnIdentity(paneContentRef.current.provider, submittedTurnId)) : undefined const outcome: FreshAgentCheckpointOutcome = { + version: 1, requestId, conversationRequestId: scope, ...(submittedTurnId ? { displayTurnId: submittedTurnId } : {}), @@ -1631,63 +1620,23 @@ export function FreshAgentView({ const current = pendingSendMetadataRef.current.get(requestId) ?? {} const next: PendingSendMetadata = { ...current, ...patch } pendingSendMetadataRef.current.set(requestId, next) - if (next.checkpointAvailabilityScope) { - const availability: CheckpointTurnAvailability | undefined = next.checkpointId - ? { - status: 'available', - checkpointId: next.checkpointId, - ...(next.checkpointWorkspace ? { workspace: next.checkpointWorkspace } : {}), - } - : next.checkpointFailureReason - ? { - status: 'unavailable', - reason: next.checkpointFailureReason, - ...(next.checkpointFailureCode ? { code: next.checkpointFailureCode } : {}), - ...(next.checkpointWorkspace ? { workspace: next.checkpointWorkspace } : {}), - } - : checkpointAvailabilityByTurn.get(checkpointAvailabilityKey( - next.checkpointAvailabilityScope, - 'request', - requestId, - )) - if (availability) { - publishCheckpointAvailability(next.checkpointAvailabilityScope, requestId, next.submittedTurnId, availability) - } - } - if ( - next.metadataUpdateStarted - || !next.cwd - || !next.checkpointId - || !next.submittedTurnId - ) { - return - } - pendingSendMetadataRef.current.set(requestId, { ...next, metadataUpdateStarted: true }) - void Promise - .resolve(api.post('/api/fresh-agent/checkpoints/metadata', { - cwd: next.cwd, - ...(next.checkpointWorkspace ? { workspace: next.checkpointWorkspace } : {}), - id: next.checkpointId, + if (next.checkpointAvailabilityScope && next.checkpointOutcome) { + publishCheckpointAvailability( + next.checkpointAvailabilityScope, requestId, - turnId: next.submittedTurnId, - })) - .then(() => { - pendingSendMetadataRef.current.delete(requestId) - }) - .catch(() => { - const latest = pendingSendMetadataRef.current.get(requestId) - if (latest) { - pendingSendMetadataRef.current.set(requestId, { ...latest, metadataUpdateStarted: false }) - } - }) - }, [checkpointAvailabilityByTurn, publishCheckpointAvailability]) + next.submittedTurnId, + next.checkpointOutcome, + ) + } + }, [publishCheckpointAvailability]) - /** Builds and sends the freshAgent.send frame. Shared by the composer - * submit path (sendUserText) and the lost-session retry (Task 10) so the - * retry frame carries exactly the same fields, plus the route cwd. */ + /** Builds and sends the freshAgent.send frame. The lost-session retry gets + * its own logical attempt because the server explicitly refused the prior + * provider attempt before accepting it. */ const sendFreshAgentSendFrame = useCallback((requestId: string, text: string, cwd?: string) => { const current = paneContentRef.current if (!current.sessionId || isManagedRuntimeRecoveryPending(current.recoverySummary)) return + const pending = pendingSendMetadataRef.current.get(requestId) // b8ke ext r8 F5: the send is a lifecycle producer — it carries the // observed (epoch, generation) fence so a queued send landing after a // crash + generation advance is typed-refused server-side, never an @@ -1699,6 +1648,10 @@ export function FreshAgentView({ sessionId: current.sessionId, sessionType: current.sessionType, provider: current.provider, + checkpointIntent: { + logicalAttemptId: pending?.logicalAttemptId ?? requestId, + ...(pending?.checkpointWorkspace ? { workspace: pending.checkpointWorkspace } : {}), + }, ...(fence ? { observedEpoch: fence.epoch, observedGeneration: fence.generation } : {}), ...(cwd ? { cwd } : {}), text, @@ -1720,7 +1673,13 @@ export function FreshAgentView({ * screen. */ const resendPendingMessage = useCallback((retryRequestId: string, text: string, cwd: string) => { if (outgoingTurnRef.current) outgoingTurnRef.current.requestId = retryRequestId - recordPendingSendMetadata(retryRequestId, { text }) + const current = paneContentRef.current + recordPendingSendMetadata(retryRequestId, { + text, + logicalAttemptId: nanoid(), + checkpointAvailabilityScope: checkpointAvailabilityScope(current), + ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), + }) sendFreshAgentSendFrame(retryRequestId, text, cwd) const echo = localEchoRef.current if (echo && echo.text === text) { @@ -2762,6 +2721,61 @@ export function FreshAgentView({ redriveAfterSessionReserved() return } + if (message.type === 'freshAgent.checkpoint.outcome') { + const requestId = message.requestId + const pending = pendingSendMetadataRef.current.get(requestId) + const current = paneContentRef.current + if ( + !pending + || message.logicalAttemptId !== pending.logicalAttemptId + || !locatorMatchesPane( + message, + current, + freshOpenCodeRouteCwdRef.current, + runtimeOwnersForLocator(), + ) + ) return + + const workspace = normalizeFreshAgentCheckpointWorkspace(message.workspace) + ?? pending.checkpointWorkspace + let checkpointOutcome: CheckpointTurnAvailability + if ( + message.status === 'available' + && typeof message.checkpointId === 'string' + && message.checkpointId.length > 0 + && message.checkpointId.length <= 256 + && !/[\u0000-\u001f\u007f-\u009f]/.test(message.checkpointId) + ) { + checkpointOutcome = { + status: 'available', + checkpointId: message.checkpointId, + ...(workspace ? { workspace } : {}), + } + } else if (message.status === 'unavailable') { + const suppliedReason = typeof message.reason === 'string' + ? sanitizeCheckpointReason(message.reason) + : '' + const suppliedCode = typeof message.code === 'string' + && /^[a-z0-9_.:-]{1,96}$/.test(message.code) + ? message.code + : undefined + checkpointOutcome = { + status: 'unavailable', + reason: suppliedReason || CHECKPOINT_FAILURE_FALLBACK, + ...(suppliedCode ? { code: suppliedCode } : {}), + ...(workspace ? { workspace } : {}), + } + } else { + checkpointOutcome = { + status: 'unavailable', + reason: 'The server did not return a saved code checkpoint.', + code: 'checkpoint_capture_failed', + ...(workspace ? { workspace } : {}), + } + } + recordPendingSendMetadata(requestId, { checkpointOutcome }) + return + } if ( message.type === 'freshAgent.send.accepted' && typeof message.requestId === 'string' @@ -2777,6 +2791,19 @@ export function FreshAgentView({ const submittedTurnId = typeof message.submittedTurnId === 'string' ? message.submittedTurnId : undefined + const pending = pendingSendMetadataRef.current.get(message.requestId) + if (pending && !pending.checkpointOutcome) { + // Older servers may accept a send without the additive outcome + // frame. Keep that turn honest and rewind disabled after acceptance. + recordPendingSendMetadata(message.requestId, { + checkpointOutcome: { + status: 'unavailable', + reason: 'The server did not confirm a code checkpoint for this message.', + code: 'checkpoint_outcome_missing', + ...(pending.checkpointWorkspace ? { workspace: pending.checkpointWorkspace } : {}), + }, + }) + } if (submittedTurnId) { recordPendingSendMetadata(message.requestId, { submittedTurnId }) if (outgoingTurnRef.current?.requestId === message.requestId) { @@ -3783,94 +3810,24 @@ export function FreshAgentView({ const current = paneContentRef.current if (!current.sessionId || isManagedRuntimeRecoveryPending(current.recoverySummary)) return const requestId = nanoid() + const logicalAttemptId = nanoid() outgoingTurnRef.current = { requestId, text, sawBusy: false, previousTurns: snapshotRef.current?.turns ?? [] } // Task 16: a new send starts a fresh idle-incomplete re-poll budget. idleIncompleteRetryCountRef.current = 0 const routeCwd = getFreshOpenCodeRouteCwd(current, { sessionCwd: freshOpenCodeRouteCwdRef.current }) - const checkpointCwd = agentSessionCwdRef.current ?? current.initialCwd const checkpointScope = checkpointAvailabilityScope(current) - // Retain the exact outgoing text as the resend payload (Task 10): the - // lost-session retry resends from this metadata, never from the echo. - recordPendingSendMetadata(requestId, { text, checkpointAvailabilityScope: checkpointScope }) - // Checkpoint the working tree before the agent acts on this message, so - // "rewind code to here" on this turn restores the pre-turn state. The - // provider dispatch waits until capture settles, but capture failure still - // records an unavailable reason and allows the message to go through. + // Retain the outgoing text only in transient retry metadata. Durable + // checkpoint outcomes contain no prompt text or provider payloads. + recordPendingSendMetadata(requestId, { + text, + logicalAttemptId, + checkpointAvailabilityScope: checkpointScope, + ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), + }) + // The server owns the pre-turn capture barrier inside this same send + // request. Pending is visible immediately; its correlated outcome arrives + // before the server forwards the message to the provider. publishCheckpointAvailability(checkpointScope, requestId, undefined, { status: 'pending' }) - let checkpointReady: Promise<void> | undefined - if (checkpointCwd) { - recordPendingSendMetadata(requestId, { - cwd: checkpointCwd, - ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), - }) - const controller = new AbortController() - let checkpointTimer: number | undefined - let checkpointTimedOut = false - const checkpointTimeout = new Promise<never>((_resolve, reject) => { - checkpointTimer = window.setTimeout(() => { - checkpointTimedOut = true - controller.abort() - reject(new Error(CHECKPOINT_CAPTURE_TIMEOUT_REASON)) - }, CHECKPOINT_CAPTURE_TIMEOUT_MS) - }) - let checkpointRequest: Promise<CheckpointCreationEntry> - try { - checkpointRequest = Promise.resolve(api.post<CheckpointCreationEntry>('/api/fresh-agent/checkpoints', { - cwd: checkpointCwd, - ...(current.checkpointWorkspace ? { workspace: current.checkpointWorkspace } : {}), - label: checkpointLabelForText(text), - requestId, - }, { signal: controller.signal })) - } catch (error) { - checkpointRequest = Promise.reject(error) - } - checkpointReady = Promise.race([checkpointRequest, checkpointTimeout]) - .then((entry) => { - if (checkpointTimedOut) return undefined - if (entry?.id) { - recordPendingSendMetadata(requestId, { - cwd: checkpointCwd, - checkpointAvailabilityScope: checkpointScope, - checkpointWorkspace: isCheckpointWorkspace(entry.workspace) - ? entry.workspace - : current.checkpointWorkspace, - checkpointId: entry.id, - checkpointFailureReason: undefined, - }) - } else { - recordPendingSendMetadata(requestId, { - checkpointAvailabilityScope: checkpointScope, - checkpointFailureReason: 'The server did not return a saved code checkpoint.', - checkpointFailureCode: 'checkpoint_capture_failed', - ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), - }) - } - return undefined - }) - .catch((error: unknown) => { - recordPendingSendMetadata(requestId, { - checkpointAvailabilityScope: checkpointScope, - checkpointFailureReason: checkpointTimedOut - ? CHECKPOINT_CAPTURE_TIMEOUT_REASON - : checkpointFailureReason(error), - checkpointFailureCode: checkpointTimedOut - ? 'checkpoint_capture_timeout' - : checkpointFailureCode(error) ?? 'checkpoint_capture_failed', - ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), - }) - return undefined - }) - .finally(() => { - if (checkpointTimer !== undefined) window.clearTimeout(checkpointTimer) - }) - } else { - recordPendingSendMetadata(requestId, { - checkpointAvailabilityScope: checkpointScope, - checkpointFailureReason: 'No project working directory is available for this message.', - checkpointFailureCode: 'workspace_unavailable', - ...(current.checkpointWorkspace ? { checkpointWorkspace: current.checkpointWorkspace } : {}), - }) - } const isFirstMessage = !autoTitleSentRef.current && (autoTitleFreshBoundaryRef.current || snapshotConfirmsNoUserTurns) if (isFirstMessage) { @@ -3908,16 +3865,11 @@ export function FreshAgentView({ pendingLocalEcho: nextLocalEcho, }, })) - const dispatchAfterCheckpoint = () => { - const latest = paneContentRef.current - if ( - checkpointAvailabilityScope(latest) !== checkpointScope - || latest.sessionId !== current.sessionId - ) return - sendFreshAgentSendFrame(requestId, text, routeCwd) - } - if (checkpointReady) void checkpointReady.then(dispatchAfterCheckpoint) - else dispatchAfterCheckpoint() + const latest = paneContentRef.current + if ( + checkpointAvailabilityScope(latest) === checkpointScope + && latest.sessionId === current.sessionId + ) sendFreshAgentSendFrame(requestId, text, routeCwd) }, [dispatch, paneId, publishCheckpointAvailability, recordPendingSendMetadata, sendFreshAgentSendFrame, snapshotConfirmsNoUserTurns, tabId]) // Providers accept one active turn. Keep follow-ups until the session can @@ -4025,14 +3977,28 @@ export function FreshAgentView({ query.set('workspacePath', workspace.path) query.set('workspaceProvenance', workspace.provenance) } - void Promise + void Promise .resolve(api.get<{ checkpoints: CheckpointCreationEntry[] }>( `/api/fresh-agent/checkpoints?${query.toString()}`, )) .then((result) => { - const checkpoint = pickCheckpointForTurn(result?.checkpoints ?? [], snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null + if (!result || !Array.isArray(result.checkpoints)) { + setNotice('Could not verify the saved code checkpoint list. Try again.') + return + } + const checkpoints = result.checkpoints + // Settled outcomes persist the exact create response across reloads. + // This is authoritative when metadata association failed: duplicate + // prompt labels cannot safely identify which snapshot belongs to the + // selected user turn. Label matching remains only for legacy turns + // without a persisted checkpoint ID. + const checkpoint = availability?.checkpointId + ? checkpoints.find((entry) => entry.id === availability.checkpointId) ?? null + : pickCheckpointForTurn(checkpoints, snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null if (!checkpoint) { - const reason = 'No code checkpoint was saved for this message.' + const reason = availability?.checkpointId + ? 'The saved code checkpoint for this message is no longer available.' + : 'No code checkpoint was saved for this message.' const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId publishCheckpointAvailability( checkpointAvailabilityScope(current), @@ -4047,9 +4013,7 @@ export function FreshAgentView({ `Rewind code to the state before "${checkpoint.label}"?\n\nTracked files changed since will be overwritten. Files created since are left in place. The conversation is not affected.`, ) if (!confirmed) return - const restoreWorkspace = isCheckpointWorkspace(checkpoint.workspace) - ? checkpoint.workspace - : workspace + const restoreWorkspace = normalizeFreshAgentCheckpointWorkspace(checkpoint.workspace) ?? workspace return Promise .resolve(api.post('/api/fresh-agent/checkpoints/restore', { cwd, @@ -4057,29 +4021,35 @@ export function FreshAgentView({ id: checkpoint.id, })) .then(() => setNotice(`Code rewound to before: "${checkpoint.label}"`)) + .catch((error: unknown) => { + if (error && typeof error === 'object') { + const details = (error as { details?: unknown }).details + if (details && typeof details === 'object' && (details as { code?: unknown }).code === 'workspace_unavailable') { + const reason = checkpointFailureReason(error) + const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId + publishCheckpointAvailability( + checkpointAvailabilityScope(current), + typeof requestId === 'string' ? requestId : undefined, + getFreshAgentDisplayTurnKey(turn), + { + status: 'unavailable', + reason, + code: checkpointFailureCode(error) ?? 'checkpoint_restore_failed', + ...(workspace ? { workspace } : {}), + }, + ) + setNotice(`Rewind unavailable for this message: ${reason}`) + return + } + } + setNotice(error instanceof Error ? `Rewind failed: ${error.message}` : 'Rewind failed') + }) }) - .catch((error: unknown) => { - if (error && typeof error === 'object') { - const details = (error as { details?: unknown }).details - if (details && typeof details === 'object' && (details as { code?: unknown }).code === 'workspace_unavailable') { - const reason = checkpointFailureReason(error) - const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId - publishCheckpointAvailability( - checkpointAvailabilityScope(current), - typeof requestId === 'string' ? requestId : undefined, - getFreshAgentDisplayTurnKey(turn), - { - status: 'unavailable', - reason, - code: checkpointFailureCode(error) ?? 'checkpoint_restore_failed', - ...(workspace ? { workspace } : {}), - }, - ) - setNotice(`Rewind unavailable for this message: ${reason}`) - return - } - } - setNotice(error instanceof Error ? `Rewind failed: ${error.message}` : 'Rewind failed') + .catch(() => { + // A failed or malformed list read is uncertain. Keep the durable + // checkpoint outcome available so the user can retry instead of + // turning a transient API failure into a permanent missing target. + setNotice('Could not check the saved code checkpoint list. Try again.') }) }, [getCheckpointAvailability, publishCheckpointAvailability, snapshot?.turns]) @@ -4094,11 +4064,12 @@ export function FreshAgentView({ sessionId, title: title ?? sessionId, cwd: freshOpenCodeRouteCwd, + ...(paneContent.checkpointWorkspace ? { checkpointWorkspace: paneContent.checkpointWorkspace } : {}), provider: 'opencode', sessionType: 'freshopencode', isSubagent: true, })) - }, [dispatch, freshOpenCodeRouteCwd]) + }, [dispatch, freshOpenCodeRouteCwd, paneContent.checkpointWorkspace]) const content = useMemo(() => { const retainedHistory = snapshot?.extensions?.[snapshot.provider]?.nativeHistoryRetention diff --git a/src/lib/resume-session.ts b/src/lib/resume-session.ts index a54f0c95b..83eb3e9df 100644 --- a/src/lib/resume-session.ts +++ b/src/lib/resume-session.ts @@ -1,8 +1,10 @@ import { findPaneForSession } from '@/lib/session-utils' +import { findPaneContent } from '@/lib/pane-utils' import { openSessionTab, setActiveTab } from '@/store/tabsSlice' import { setActivePane } from '@/store/panesSlice' -import type { FreshAgentCheckpointWorkspace } from '@/store/paneTypes' +import type { FreshAgentCheckpointWorkspace } from '@/store/types' import type { AppDispatch, RootState } from '@/store/store' +import type { CodingCliProviderName } from '@/store/types' export interface ResumeTarget { provider: string @@ -14,6 +16,30 @@ export interface ResumeTarget { checkpointWorkspace?: FreshAgentCheckpointWorkspace } +/** Preserve a session's rewind scope when a resume flow must rebuild its pane. */ +export function findCheckpointWorkspaceForSession( + state: RootState, + target: Pick<ResumeTarget, 'provider' | 'sessionId'>, +): FreshAgentCheckpointWorkspace | undefined { + const existing = findPaneForSession( + state, + { provider: target.provider as CodingCliProviderName, sessionId: target.sessionId }, + state.connection?.serverInstanceId, + ) + if (!existing) return undefined + const tab = state.tabs?.tabs?.find((candidate) => candidate.id === existing.tabId) + const layout = state.panes?.layouts?.[existing.tabId] + const content = existing.paneId && layout ? findPaneContent(layout, existing.paneId) : null + if (content?.kind === 'fresh-agent' && content.checkpointWorkspace) { + return content.checkpointWorkspace + } + // A tab can contain several sessions after splitting, so its single + // workspace is a safe fallback only while the matching tab has no layout + // and is waiting for TabContent hydration. A terminal pane in a split tab + // must never inherit another pane's checkpoint scope. + return existing.paneId ? undefined : tab?.checkpointWorkspace +} + /** * Resume a session in a tab following the sidebar's dedup convention: * if a pane already holds the session, focus it; otherwise open a new @@ -38,6 +64,8 @@ export function resumeSessionInTab( onNavigate?.('terminal') return { deduped: true } } + const checkpointWorkspace = target.checkpointWorkspace + ?? findCheckpointWorkspaceForSession(state, target) dispatch( openSessionTab({ sessionId: target.sessionId, @@ -46,7 +74,7 @@ export function resumeSessionInTab( cwd: target.cwd, title: target.title, firstUserMessage: target.firstUserMessage, - ...(target.checkpointWorkspace ? { checkpointWorkspace: target.checkpointWorkspace } : {}), + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), }), ) onNavigate?.('terminal') diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index 62044262e..dc4770a19 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -1,4 +1,5 @@ -import type { TerminalStatus, TabMode, ShellType } from './types' +import type { TerminalStatus, TabMode, ShellType, FreshAgentCheckpointWorkspace } from './types' +export type { FreshAgentCheckpointWorkspace } from './types' import { FreshAgentModelSelectionSchema, type FreshAgentModelSelection, @@ -68,6 +69,7 @@ export function normalizeFreshAgentEffortOverride(value: unknown): string | unde } export type FreshAgentCheckpointOutcome = { + version: 1 /** Conversation generation; changes only when the pane is deliberately replaced. */ conversationRequestId?: string requestId: string @@ -99,7 +101,7 @@ function checkpointOutcomeReason(value: unknown): string | undefined { return normalized || undefined } -function checkpointOutcomeWorkspace(value: unknown): FreshAgentCheckpointWorkspace | undefined { +export function normalizeFreshAgentCheckpointWorkspace(value: unknown): FreshAgentCheckpointWorkspace | undefined { if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined const candidate = value as Record<string, unknown> const path = checkpointOutcomeString(candidate.path, 4096) @@ -125,6 +127,7 @@ export function normalizeFreshAgentCheckpointOutcomes( const record = candidate as Record<string, unknown> const requestId = checkpointOutcomeString(record.requestId, 128) if (!requestId || !/^[A-Za-z0-9_.:-]+$/.test(requestId)) continue + if (record.version !== undefined && record.version !== 1) continue const recordConversationRequestId = record.conversationRequestId === undefined ? undefined : checkpointOutcomeString(record.conversationRequestId, 128) @@ -137,7 +140,7 @@ export function normalizeFreshAgentCheckpointOutcomes( const displayTurnId = checkpointOutcomeString(record.displayTurnId, 256) const nativeTurnId = checkpointOutcomeString(record.nativeTurnId, 256) const checkpointId = checkpointOutcomeString(record.checkpointId, 256) - const workspace = checkpointOutcomeWorkspace(record.workspace) + const workspace = normalizeFreshAgentCheckpointWorkspace(record.workspace) const code = typeof record.code === 'string' && record.code.length <= 96 && /^[a-z0-9_.:-]+$/.test(record.code) @@ -150,6 +153,7 @@ export function normalizeFreshAgentCheckpointOutcomes( byRequest.delete(requestId) byRequest.set(requestId, { + version: 1, requestId, ...(recordConversationRequestId ? { conversationRequestId: recordConversationRequestId } : {}), ...(displayTurnId ? { displayTurnId } : {}), @@ -517,11 +521,6 @@ export type ExtensionPaneContent = { props: Record<string, unknown> } -export type FreshAgentCheckpointWorkspace = { - path: string - provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' -} - /** * Union type for all pane content types. */ diff --git a/src/store/persistedState.ts b/src/store/persistedState.ts index 81cd50cae..a80bfd3d9 100644 --- a/src/store/persistedState.ts +++ b/src/store/persistedState.ts @@ -21,6 +21,7 @@ import { stripSessionOwnedTabFreezeFlags, } from '@/lib/session-name-migration' import { + normalizeFreshAgentCheckpointWorkspace, normalizeFreshAgentCheckpointOutcomes, normalizeFreshAgentPaneModelSelection, } from './paneTypes' @@ -54,6 +55,13 @@ const zSanitizedOptionalString = z.preprocess( const zTabMode = zSanitizedOptionalString const zCodingCliProvider = zSanitizedOptionalString +const zCheckpointWorkspace = z.preprocess( + (value) => normalizeFreshAgentCheckpointWorkspace(value), + z.object({ + path: z.string().min(1), + provenance: z.enum(['directory-picker', 'resolved-project', 'legacy-checkpoint']), + }).optional(), +) const zTab = z.object({ id: z.string().min(1), @@ -64,6 +72,7 @@ const zTab = z.object({ mode: zTabMode, codingCliProvider: zCodingCliProvider, resumeSessionId: z.string().optional(), + checkpointWorkspace: zCheckpointWorkspace, }).passthrough() // The tabs array is validated per-element (see `salvageTabs`) rather than atomically via diff --git a/src/store/selectors/sidebarSelectors.ts b/src/store/selectors/sidebarSelectors.ts index 40075f548..c11d2cea3 100644 --- a/src/store/selectors/sidebarSelectors.ts +++ b/src/store/selectors/sidebarSelectors.ts @@ -2,7 +2,14 @@ import { createSelector } from '@reduxjs/toolkit' import type { ManagedRuntimeProjectionFields } from '../../../shared/managed-runtime.js' import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal' import type { RootState } from '../store' -import type { BackgroundTerminal, CodingCliProviderName, WorktreeGrouping, ProjectGroup } from '../types' +import type { + BackgroundTerminal, + CodingCliProviderName, + FreshAgentCheckpointWorkspace, + WorktreeGrouping, + ProjectGroup, +} from '../types' +import { normalizeFreshAgentCheckpointWorkspace } from '../paneTypes' import { collectSessionRefsFromTabs } from '@/lib/session-utils' import { getFreshAgentProviderConfig } from '@/lib/fresh-agent-provider-utils' import { resolveFreshAgentType } from '@/lib/fresh-agent-registry' @@ -42,6 +49,7 @@ export interface SidebarSessionItem extends ManagedRuntimeProjectionFields { archived?: boolean timestamp: number cwd?: string + checkpointWorkspace?: FreshAgentCheckpointWorkspace hasTab: boolean ratchetedActivity?: number isRunning: boolean @@ -327,6 +335,7 @@ export function buildSessionItems( archived: session.archived, timestamp: session.lastActivityAt, cwd: session.cwd, + checkpointWorkspace: normalizeFreshAgentCheckpointWorkspace(session.checkpointWorkspace), hasTab: tabInfo?.hasTab ?? false, ratchetedActivity, isRunning: runningTerminal ? runningTerminal.isRunning : !!runningTerminalId, diff --git a/src/store/tabsSlice.ts b/src/store/tabsSlice.ts index 22bcac9c3..94c5ecf71 100644 --- a/src/store/tabsSlice.ts +++ b/src/store/tabsSlice.ts @@ -10,6 +10,7 @@ import type { PaneContent, PaneNode, } from './paneTypes' +import { normalizeFreshAgentCheckpointWorkspace } from './paneTypes' import { findTabIdForSession, collectSessionRefsFromTabs, liveTerminalRowIdentity } from '@/lib/session-utils' import { getProviderLabel } from '@/lib/coding-cli-utils' import { basenameSegment } from '@shared/path-basename' @@ -312,6 +313,7 @@ type AddTabPayload = { mode?: TabMode shell?: ShellType initialCwd?: string + checkpointWorkspace?: FreshAgentCheckpointWorkspace sessionRef?: Tab['sessionRef'] serverInstanceId?: string resumeSessionId?: string @@ -360,6 +362,7 @@ export const tabsSlice = createSlice({ mode: payload.mode || 'shell', shell: payload.shell || 'system', initialCwd: payload.initialCwd, + checkpointWorkspace: normalizeFreshAgentCheckpointWorkspace(payload.checkpointWorkspace), sessionRef, serverInstanceId: payload.serverInstanceId, resumeSessionId: undefined, @@ -1908,6 +1911,7 @@ export const openSessionTab = createAsyncThunk( mode: resolvedProvider, codingCliProvider: resolvedProvider, initialCwd: cwd, + checkpointWorkspace: desiredResumeContent.checkpointWorkspace, sessionRef: desiredResumeContent.sessionRef, sessionMetadataByKey: buildSessionMetadataByKey(), })) diff --git a/src/store/types.ts b/src/store/types.ts index 5e298b414..37de5f502 100644 --- a/src/store/types.ts +++ b/src/store/types.ts @@ -45,6 +45,11 @@ export type TabMode = 'shell' | CodingCliProviderName */ export type ShellType = 'system' | 'cmd' | 'powershell' | 'wsl' +export type FreshAgentCheckpointWorkspace = { + path: string + provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' +} + export interface SessionListMetadata { sessionType?: string firstUserMessage?: string @@ -62,6 +67,8 @@ export interface Tab extends ManagedRuntimeProjectionFields { mode: TabMode shell?: ShellType initialCwd?: string + /** Immutable code-checkpoint workspace used when a fresh-agent pane is reconstructed. */ + checkpointWorkspace?: FreshAgentCheckpointWorkspace sessionRef?: SessionLocator codexDurability?: CodexDurabilityRef serverInstanceId?: string @@ -118,6 +125,8 @@ export interface CodingCliSession { summary?: string firstUserMessage?: string cwd?: string + /** Optional client/server locator for restoring the session's rewind scope. */ + checkpointWorkspace?: FreshAgentCheckpointWorkspace archived?: boolean sourceFile?: string isSubagent?: boolean diff --git a/test/unit/client/components/ContextMenuProvider.test.tsx b/test/unit/client/components/ContextMenuProvider.test.tsx index 87bd889af..0928e4acf 100644 --- a/test/unit/client/components/ContextMenuProvider.test.tsx +++ b/test/unit/client/components/ContextMenuProvider.test.tsx @@ -307,7 +307,7 @@ function createStoreWithSession() { }) } -function createStoreWithSidebarWindowAgentSession() { +function createStoreWithSidebarWindowAgentSession(checkpointWorkspace?: { path: string; provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' }) { return configureStore({ reducer: { tabs: tabsReducer, @@ -408,6 +408,7 @@ function createStoreWithSidebarWindowAgentSession() { updatedAt: 2000, nameRef: { kind: 'session', provider: 'claude', sessionId: VALID_SESSION_ID }, sessionName: 'Sidebar Agent Session', + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), }, ], }, @@ -2457,8 +2458,9 @@ describe('ContextMenuProvider', () => { expect(store.getState().panes.layouts[openedTab!.id]).toMatchObject({ type: 'leaf', content: { - kind: 'terminal', - mode: 'claude', + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', initialCwd: '/shared/project/history', sessionRef: { provider: 'claude', @@ -2518,6 +2520,41 @@ describe('ContextMenuProvider', () => { }) }) + it('preserves a session-provided checkpoint workspace when opening it in a new tab', async () => { + const user = userEvent.setup() + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + const store = createStoreWithSidebarWindowAgentSession(checkpointWorkspace) + render( + <Provider store={store}> + <ContextMenuProvider + view="terminal" + onViewChange={() => {}} + onToggleSidebar={() => {}} + sidebarCollapsed={false} + > + <div + data-context={ContextIds.SidebarSession} + data-session-id={VALID_SESSION_ID} + data-provider="claude" + data-session-type="freshclaude" + > + Sidebar Agent Session + </div> + </ContextMenuProvider> + </Provider>, + ) + + await user.pointer({ target: screen.getByText('Sidebar Agent Session'), keys: '[MouseRight]' }) + await user.click(screen.getByText('Open in new tab')) + + const openedTab = store.getState().tabs.tabs.find((tab) => tab.id !== 'tab-1') + expect(openedTab?.checkpointWorkspace).toEqual(checkpointWorkspace) + expect(store.getState().panes.layouts[openedTab!.id]).toMatchObject({ + type: 'leaf', + content: { kind: 'fresh-agent', checkpointWorkspace }, + }) + }) + it('uses the history project window for history-project actions even when sidebar has a conflicting project snapshot', async () => { const user = userEvent.setup() const store = createStoreWithOverlappingSessionWindows() diff --git a/test/unit/client/components/ResumeSessionDialog.test.tsx b/test/unit/client/components/ResumeSessionDialog.test.tsx index 06cce7f36..3fb2a77d2 100644 --- a/test/unit/client/components/ResumeSessionDialog.test.tsx +++ b/test/unit/client/components/ResumeSessionDialog.test.tsx @@ -9,8 +9,10 @@ vi.mock('@/lib/api', () => ({ })) const resumeSessionInTab = vi.fn(() => ({ deduped: false })) +const findCheckpointWorkspaceForSession = vi.fn(() => undefined as unknown) vi.mock('@/lib/resume-session', () => ({ resumeSessionInTab: (...args: unknown[]) => resumeSessionInTab(...args), + findCheckpointWorkspaceForSession: (...args: unknown[]) => findCheckpointWorkspaceForSession(...args), })) import { ResumeSessionDialog } from '@/components/ResumeSessionDialog' @@ -276,6 +278,26 @@ describe('ResumeSessionDialog', () => { expect(resumeSessionInTab).not.toHaveBeenCalled() }) + it('passes the local checkpoint workspace through the resume-anyway fallback', async () => { + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + findCheckpointWorkspaceForSession.mockReturnValue(checkpointWorkspace) + apiPost.mockReturnValue(ok([])) + renderDialog() + typeAndResolve(V7) + const anyway = await screen.findByTestId('resume-anyway-button') + fireEvent.click(anyway) + + expect(findCheckpointWorkspaceForSession).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ + provider: 'codex', + sessionId: V7, + })) + expect(resumeSessionInTab.mock.calls[0]?.[2]).toMatchObject({ + provider: 'codex', + sessionId: V7, + checkpointWorkspace, + }) + }) + it('a NON-ready response never auto-resumes, even with a single cwd match (degraded seam)', async () => { // Pins the ordering the future 'degraded' status depends on: any non-ready // response must be handled as a retry state BEFORE match handling. diff --git a/test/unit/client/components/Sidebar.test.tsx b/test/unit/client/components/Sidebar.test.tsx index 7f9e691e5..9323319b6 100644 --- a/test/unit/client/components/Sidebar.test.tsx +++ b/test/unit/client/components/Sidebar.test.tsx @@ -4,6 +4,7 @@ import { render, screen, fireEvent, cleanup, act, waitFor, within } from '@testi import { Provider } from 'react-redux' import { configureStore } from '@reduxjs/toolkit' import Sidebar from '@/components/Sidebar' +import { collectPaneContents } from '@/lib/pane-utils' import settingsReducer, { defaultSettings } from '@/store/settingsSlice' import tabsReducer, { closeTab } from '@/store/tabsSlice' import panesReducer from '@/store/panesSlice' @@ -4205,6 +4206,46 @@ describe('Sidebar Component - Session-Centric Display', () => { }) describe('sidebar click opens pane', () => { + it.each(['tab', 'split'] as const)('preserves the checkpoint workspace when opening a fresh-agent session in %s mode', async (sessionOpenMode) => { + const targetSessionId = sessionId(`fresh-workspace-${sessionOpenMode}`) + const checkpointWorkspace = { path: '/selected/fresh-project', provenance: 'directory-picker' as const } + const projects: ProjectGroup[] = [{ + projectPath: '/selected/fresh-project', + sessions: [{ + sessionId: targetSessionId, + provider: 'claude', + sessionType: 'freshclaude', + projectPath: '/selected/fresh-project', + lastActivityAt: Date.now(), + title: 'Fresh checkpoint workspace', + cwd: '/selected/fresh-project', + checkpointWorkspace, + }], + }] + const tabs = [{ id: 'tab-1', mode: 'shell' as const }] + const panes = { + layouts: { + 'tab-1': { + type: 'leaf' as const, + id: 'pane-1', + content: { kind: 'terminal' as const, mode: 'shell' as const, createRequestId: 'req-1', status: 'running' as const }, + }, + }, + activePane: { 'tab-1': 'pane-1' }, + paneTitles: {}, + } + const store = createTestStore({ projects, tabs, panes, activeTabId: 'tab-1', sessionOpenMode }) + renderSidebar(store, []) + + await act(async () => { vi.advanceTimersByTime(100) }) + fireEvent.click(screen.getByText('Fresh checkpoint workspace').closest('button')!) + + const pane = Object.values(store.getState().panes.layouts) + .flatMap((layout) => collectPaneContents(layout)) + .find((content) => content.kind === 'fresh-agent' && content.sessionRef?.sessionId === targetSessionId) + expect(pane).toMatchObject({ kind: 'fresh-agent', checkpointWorkspace }) + }) + it('splits a new pane in the current tab when clicking a session', async () => { const projects: ProjectGroup[] = [ { diff --git a/test/unit/client/components/TabContent.test.tsx b/test/unit/client/components/TabContent.test.tsx index a584211d1..27663551b 100644 --- a/test/unit/client/components/TabContent.test.tsx +++ b/test/unit/client/components/TabContent.test.tsx @@ -26,6 +26,10 @@ interface TabConfig { provider: string sessionId: string } + checkpointWorkspace?: { + path: string + provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' + } sessionMetadataByKey?: Record<string, unknown> } @@ -58,6 +62,7 @@ function createStore(tabs: TabConfig[], options: StoreOptions = {}) { codingCliSessionId: (t as any).codingCliSessionId, resumeSessionId: t.resumeSessionId, sessionRef: t.sessionRef, + checkpointWorkspace: t.checkpointWorkspace, sessionMetadataByKey: t.sessionMetadataByKey, createRequestId: 'req-1', })), @@ -190,6 +195,7 @@ describe('TabContent', () => { }) it('restores fresh-agent default content for shell-mode no-layout tabs using persisted codingCliProvider metadata', () => { + const checkpointWorkspace = { path: '/selected/repo', provenance: 'directory-picker' as const } const store = createStore([ { id: 'tab-1', @@ -199,6 +205,7 @@ describe('TabContent', () => { provider: 'claude', sessionId: '550e8400-e29b-41d4-a716-446655440001', }, + checkpointWorkspace, sessionMetadataByKey: { 'claude:550e8400-e29b-41d4-a716-446655440001': { sessionType: 'freshclaude', @@ -224,6 +231,7 @@ describe('TabContent', () => { provider: 'claude', sessionId: '550e8400-e29b-41d4-a716-446655440001', }, + checkpointWorkspace, }), }), expect.anything(), diff --git a/test/unit/client/components/TerminalView.lifecycle.test.tsx b/test/unit/client/components/TerminalView.lifecycle.test.tsx index e8a47fb23..70eca362b 100644 --- a/test/unit/client/components/TerminalView.lifecycle.test.tsx +++ b/test/unit/client/components/TerminalView.lifecycle.test.tsx @@ -3495,6 +3495,7 @@ describe('TerminalView lifecycle updates', () => { function setupTypedPane(options: { content?: Partial<TerminalPaneContent> tabMetadata?: Record<string, { sessionType?: string }> + tabCheckpointWorkspace?: { path: string; provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' } seed?: (store: ReturnType<typeof configureStore>) => void } = {}) { const tabId = 'tab-b8ke' @@ -3533,6 +3534,10 @@ describe('TerminalView lifecycle updates', () => { title: 'Codex', titleSetByUser: false, createRequestId: 'req-b8ke', + ...(options.tabCheckpointWorkspace ? { checkpointWorkspace: options.tabCheckpointWorkspace } : {}), + ...(options.tabCheckpointWorkspace + ? { sessionRef: { provider: 'codex' as const, sessionId: TYPED_SESSION_ID } } + : {}), ...(options.tabMetadata ? { sessionMetadataByKey: options.tabMetadata } : {}), }], activeTabId: tabId, @@ -4195,7 +4200,8 @@ describe('TerminalView lifecycle updates', () => { it('b8ke ext r16 F5: the open-as-fresh-agent action resolves the retired pre-rekey key to the canonical key', async () => { const CANONICAL_ID = '77777777-8888-4999-aaaa-bbbbbbbbbbbb' - const { store } = setupTypedPane() + const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + const { store } = setupTypedPane({ tabCheckpointWorkspace: checkpointWorkspace }) // The pane holds the RETIRED pre-rekey reference; the runtime-owners // map carries the alias mirror (the server's rekey mirror frame — // the same chain the owner selection resolves through). @@ -4247,6 +4253,7 @@ describe('TerminalView lifecycle updates', () => { // The converted pane's sessionRef carries the CANONICAL key (the // retired pre-rekey key must never reach the lifecycle start). expect(leaf.content.sessionRef?.sessionId).toBe(CANONICAL_ID) + expect(leaf.content.checkpointWorkspace).toEqual(checkpointWorkspace) } else { throw new Error('the pane did not convert') } diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index fc0914725..76d2364e2 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3415,18 +3415,11 @@ describe('FreshAgentView', () => { expect(screen.queryByRole('button', { name: 'Fork' })).not.toBeInTheDocument() }) - it('uses send acknowledgements to patch checkpoints and clear local echo only on the submitted user display turn', async () => { - const store = createStore() - const checkpoint = createDeferred<{ - id: string - ts: number - label: string - requestId: string - workspace: { path: string; provenance: 'directory-picker' } - }>() + it('sends one checkpoint-intent frame and folds the server-owned outcome onto its accepted turn', async () => { + const store = createStore() const checkpointWorkspace = { path: '/repo/project', provenance: 'directory-picker' as const } const resolvedCheckpointWorkspace = { path: '/repo/project-root', provenance: 'directory-picker' as const } - const listedCheckpointWorkspace = { path: '/repo/legacy-checkpoint-root', provenance: 'legacy-checkpoint' as const } + const listedCheckpointWorkspace = resolvedCheckpointWorkspace let onMessage: ((message: Record<string, unknown>) => void) | undefined wsMock.onMessage.mockImplementation((handler: (message: Record<string, unknown>) => void) => { onMessage = handler @@ -3481,11 +3474,7 @@ describe('FreshAgentView', () => { }, ], }) - apiMock.post.mockImplementation((url: string, body: Record<string, unknown>) => { - if (url === '/api/fresh-agent/checkpoints') return checkpoint.promise - if (url === '/api/fresh-agent/checkpoints/metadata') return Promise.resolve({ ok: true, body }) - return Promise.resolve({ title: null, source: 'none' }) - }) + apiMock.post.mockResolvedValue({ ok: true }) apiMock.get.mockResolvedValue({ checkpoints: [{ id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', @@ -3527,37 +3516,43 @@ describe('FreshAgentView', () => { }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) - expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints', { - cwd: '/repo', - workspace: checkpointWorkspace, - label: 'Ship it', - requestId: expect.any(String), - }, expect.objectContaining({ signal: expect.anything() })) - const checkpointRequest = apiMock.post.mock.calls.find(([url]) => url === '/api/fresh-agent/checkpoints') - const requestId = String(checkpointRequest?.[1]?.requestId) + const sendFrame = sentFreshAgentMessages('freshAgent.send')[0] + expect(sendFrame).toMatchObject({ + type: 'freshAgent.send', + sessionId: 'thread-normalized-send', + sessionType: 'freshcodex', + provider: 'codex', + text: 'Ship it', + checkpointIntent: { + logicalAttemptId: expect.any(String), + workspace: checkpointWorkspace, + }, + }) + const requestId = String(sendFrame?.requestId) + const logicalAttemptId = String((sendFrame?.checkpointIntent as Record<string, unknown>)?.logicalAttemptId) expect(requestId).not.toBe('undefined') - expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(0) + expect(logicalAttemptId).not.toBe('undefined') + expect(requestId).not.toBe(logicalAttemptId) + expect(apiMock.post.mock.calls.some(([url]) => String(url).startsWith('/api/fresh-agent/checkpoints'))).toBe(false) expect(screen.getByText('Ship it')).toBeInTheDocument() + expect(screen.getByRole('status', { name: 'Saving code rewind checkpoint for this message' })) + .toHaveTextContent('Saving the code state from before this message.') act(() => { - checkpoint.resolve({ - id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', - ts: 1, - label: 'Ship it', - requestId, - workspace: resolvedCheckpointWorkspace, - }) - }) - await waitFor(() => { - expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ - type: 'freshAgent.send', + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', requestId, + logicalAttemptId, sessionId: 'thread-normalized-send', sessionType: 'freshcodex', provider: 'codex', - text: 'Ship it', - })) + status: 'available', + checkpointId: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + workspace: resolvedCheckpointWorkspace, + }) }) + expect(await screen.findByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') expect(onMessage).toBeTypeOf('function') act(() => { onMessage?.({ @@ -3567,15 +3562,7 @@ describe('FreshAgentView', () => { }) }) - await waitFor(() => { - expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/metadata', { - cwd: '/repo', - workspace: resolvedCheckpointWorkspace, - id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', - requestId, - turnId: 'display-user-1', - }) - }) + expect(apiMock.post.mock.calls.some(([url]) => url === '/api/fresh-agent/checkpoints/metadata')).toBe(false) act(() => { onMessage?.({ type: 'freshAgent.session.materialized', @@ -3633,7 +3620,171 @@ describe('FreshAgentView', () => { }) }) - it('uses the live provider cwd for checkpoints when the launch cwd is stale', async () => { + it('rewinds to the persisted checkpoint ID after reload when duplicate prompts have no metadata links', async () => { + const checkpointWorkspace = { path: '/repo/selected', provenance: 'directory-picker' as const } + const firstCheckpointId = '11111111111111111111111111111111' + const secondCheckpointId = '22222222222222222222222222222222' + const turns = [ + { + id: 'native-user-1', + turnId: 'display-user-1', + role: 'user' as const, + summary: 'Repeat this prompt', + items: [{ id: 'user-text-1', kind: 'text' as const, text: 'Repeat this prompt' }], + }, + { + id: 'native-user-2', + turnId: 'display-user-2', + role: 'user' as const, + summary: 'Repeat this prompt', + items: [{ id: 'user-text-2', kind: 'text' as const, text: 'Repeat this prompt' }], + }, + ] + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'duplicate prompts', + capabilities: { send: true, interrupt: true, fork: true }, + turns, + }) + // Metadata writes failed before reload, so the list has no request/turn + // association. The durable outcomes still identify each exact checkpoint. + apiMock.get.mockResolvedValue({ checkpoints: [ + { id: secondCheckpointId, ts: 2, label: 'Repeat this prompt' }, + { id: firstCheckpointId, ts: 1, label: 'Repeat this prompt' }, + ] }) + const persistedContent = JSON.parse(JSON.stringify({ + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-duplicate-prompt-conversation', + sessionId: 'thread-duplicate-prompt', + sessionRef: { provider: 'codex', sessionId: 'thread-duplicate-prompt' }, + status: 'idle', + initialCwd: '/repo', + checkpointWorkspace, + checkpointOutcomes: [ + { + version: 1, + conversationRequestId: 'req-duplicate-prompt-conversation', + requestId: 'request-first', + displayTurnId: 'display-user-1', + status: 'available', + checkpointId: secondCheckpointId, + workspace: checkpointWorkspace, + }, + { + version: 1, + conversationRequestId: 'req-duplicate-prompt-conversation', + requestId: 'request-second', + displayTurnId: 'display-user-2', + status: 'available', + checkpointId: firstCheckpointId, + workspace: checkpointWorkspace, + }, + ], + })) + const store = createStore() + store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', content: persistedContent })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + vi.spyOn(window, 'confirm').mockReturnValue(true) + const rewindButtons = await screen.findAllByRole('button', { name: 'Rewind code to here' }) + expect(rewindButtons).toHaveLength(2) + + fireEvent.click(rewindButtons[0]!) + await waitFor(() => { + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: '/repo', + workspace: checkpointWorkspace, + id: secondCheckpointId, + }) + }) + fireEvent.click(rewindButtons[1]!) + await waitFor(() => { + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: '/repo', + workspace: checkpointWorkspace, + id: firstCheckpointId, + }) + }) + }) + + it('preserves an exact available checkpoint after a transient list failure and allows retry', async () => { + const checkpointId = '33333333333333333333333333333333' + const checkpointWorkspace = { path: '/repo/selected', provenance: 'directory-picker' as const } + const persistedOutcome = { + version: 1 as const, + conversationRequestId: 'req-checkpoint-list-retry', + requestId: 'request-checkpoint-list-retry', + displayTurnId: 'display-retry-turn', + status: 'available' as const, + checkpointId, + workspace: checkpointWorkspace, + } + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'saved turn', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: 'native-retry-turn', + turnId: 'display-retry-turn', + role: 'user', + summary: 'Retry checkpoint lookup', + items: [{ id: 'retry-user-text', kind: 'text', text: 'Retry checkpoint lookup' }], + }], + }) + apiMock.get + .mockRejectedValueOnce(new ApiError(422, 'workspace unavailable', { + code: 'workspace_unavailable', + details: 'The selected project workspace is temporarily unavailable.', + })) + .mockResolvedValue({ checkpoints: [{ id: checkpointId, ts: 1, label: 'Retry checkpoint lookup', workspace: checkpointWorkspace }] }) + const persistedContent = JSON.parse(JSON.stringify({ + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-checkpoint-list-retry', + sessionId: 'thread-checkpoint-list-retry', + status: 'idle', + initialCwd: '/repo', + checkpointWorkspace, + checkpointOutcomes: [persistedOutcome], + })) as FreshAgentPaneContent + const store = createStore() + store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', content: persistedContent })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + vi.spyOn(window, 'confirm').mockReturnValue(true) + + const rewind = await screen.findByRole('button', { name: 'Rewind code to here' }) + fireEvent.click(rewind) + expect(await screen.findByText('Could not check the saved code checkpoint list. Try again.')) + .toBeInTheDocument() + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual([persistedOutcome]) + expect(screen.getByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') + expect(apiMock.post).not.toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', expect.anything()) + + fireEvent.click(rewind) + await waitFor(() => { + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: '/repo', + workspace: checkpointWorkspace, + id: checkpointId, + }) + }) + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual([persistedOutcome]) + }) + + it('does not promote a live provider cwd into an explicit checkpoint workspace', async () => { const store = createStore() const sessionId = 'thread-live-checkpoint-cwd' apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ @@ -3648,17 +3799,6 @@ describe('FreshAgentView', () => { provider: 'codex', cwd: '/workspace/resolved-project', })) - apiMock.post.mockImplementation((url: string, body?: { requestId?: string }) => ( - url === '/api/fresh-agent/checkpoints' - ? Promise.resolve({ - id: 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb', - ts: 1, - label: 'Use the resolved project', - requestId: body?.requestId ?? '', - workspace: { path: '/workspace/resolved-project', provenance: 'resolved-project' }, - }) - : Promise.resolve({ ok: true }) - )) store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', @@ -3684,34 +3824,31 @@ describe('FreshAgentView', () => { fireEvent.change(composer, { target: { value: 'Use the resolved project' } }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) - await waitFor(() => { - expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints', expect.objectContaining({ - cwd: '/workspace/resolved-project', - label: 'Use the resolved project', - }), expect.objectContaining({ signal: expect.anything() })) - }) await waitFor(() => { expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ sessionId, text: 'Use the resolved project', + checkpointIntent: { logicalAttemptId: expect.any(String) }, })) }) + expect(apiMock.post.mock.calls.some(([url]) => String(url).startsWith('/api/fresh-agent/checkpoints'))).toBe(false) }) - it('keeps sending when checkpoint creation fails and shows the affected message as unavailable with the server reason', async () => { + it('keeps the send usable and persists the server-owned unavailable outcome on its accepted turn', async () => { const store = createStore() - const checkpoint = createDeferred<never>() const workspaceReason = 'Account home is not a code workspace. Select a project folder.' + const submittedText = 'Continue despite the checkpoint failure' + let onMessage: ((message: Record<string, unknown>) => void) | undefined + wsMock.onMessage.mockImplementation((handler: (message: Record<string, unknown>) => void) => { + onMessage = handler + return () => {} + }) apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ status: 'idle', summary: 'empty', capabilities: { send: true, interrupt: true, fork: true }, turns: [], }) - apiMock.post.mockImplementation((url: string) => { - if (url === '/api/fresh-agent/checkpoints') return checkpoint.promise - return Promise.resolve({ ok: true }) - }) store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', @@ -3734,54 +3871,69 @@ describe('FreshAgentView', () => { const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) await waitFor(() => expect(composer).not.toBeDisabled()) - fireEvent.change(composer, { target: { value: 'Continue despite the checkpoint failure' } }) + fireEvent.change(composer, { target: { value: submittedText } }) fireEvent.click(screen.getByRole('button', { name: 'Send' })) - expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints', { - cwd: '/account/home', - label: 'Continue despite the checkpoint failure', - requestId: expect.any(String), - }, expect.objectContaining({ signal: expect.anything() })) - const checkpointRequest = apiMock.post.mock.calls.find(([url]) => url === '/api/fresh-agent/checkpoints') - const requestId = String(checkpointRequest?.[1]?.requestId) - expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(0) - expect(screen.getByText('Continue despite the checkpoint failure')).toBeInTheDocument() + const sendFrame = sentFreshAgentMessages('freshAgent.send').at(-1) + expect(sendFrame).toMatchObject({ + type: 'freshAgent.send', + text: submittedText, + checkpointIntent: { logicalAttemptId: expect.any(String) }, + }) + const requestId = String(sendFrame?.requestId) + const logicalAttemptId = String((sendFrame?.checkpointIntent as Record<string, unknown>)?.logicalAttemptId) + expect(screen.getByText(submittedText)).toBeInTheDocument() expect(await screen.findByRole('status', { name: 'Saving code rewind checkpoint for this message' })) .toHaveTextContent('Saving the code state from before this message.') - await act(async () => { - checkpoint.reject(new ApiError(422, 'workspace unavailable', { + expect(onMessage).toBeTypeOf('function') + act(() => { + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', + requestId, + logicalAttemptId, + sessionId: 'thread-checkpoint-failure', + sessionType: 'freshcodex', + provider: 'codex', + status: 'unavailable', code: 'workspace_unavailable', - details: workspaceReason, - })) - await checkpoint.promise.catch(() => undefined) + reason: workspaceReason, + }) }) - await waitFor(() => { - expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ - type: 'freshAgent.send', + expect(await screen.findByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent(`Code rewind unavailable for this message: ${workspaceReason}`) + expect(screen.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + + act(() => { + onMessage?.({ + type: 'freshAgent.send.accepted', requestId, sessionId: 'thread-checkpoint-failure', - text: 'Continue despite the checkpoint failure', - })) + sessionType: 'freshcodex', + provider: 'codex', + submittedTurnId: 'display-failed-turn', + }) }) - const unavailable = await screen.findByRole('note', { name: 'Code rewind unavailable for this message' }) - expect(unavailable).toHaveTextContent(`Code rewind unavailable for this message: ${workspaceReason}`) - expect(screen.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() - expect(screen.getByText('Continue despite the checkpoint failure')).toBeInTheDocument() const persistedOutcomes = getFreshAgentPaneContent(store).checkpointOutcomes ?? [] expect(persistedOutcomes).toEqual(expect.arrayContaining([expect.objectContaining({ + version: 1, requestId, conversationRequestId: 'req-checkpoint-failure', + displayTurnId: 'display-failed-turn', status: 'unavailable', code: 'workspace_unavailable', reason: workspaceReason, })])) - expect(JSON.stringify(persistedOutcomes)).not.toContain('Continue despite the checkpoint failure') + expect(JSON.stringify(persistedOutcomes)).not.toContain(submittedText) + expect(apiMock.post.mock.calls.some(([url]) => String(url).startsWith('/api/fresh-agent/checkpoints'))).toBe(false) failureView.unmount() + const serializedContent = JSON.parse(JSON.stringify(getFreshAgentPaneContent(store))) as FreshAgentPaneContent + const reloadedStore = createStore() + reloadedStore.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', content: serializedContent })) render( - <Provider store={store}> + <Provider store={reloadedStore}> <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> </Provider>, ) @@ -3790,137 +3942,6 @@ describe('FreshAgentView', () => { expect(screen.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() }) - it('shows a bounded server checkpoint explanation without echoing the submitted message', async () => { - const store = createStore() - const workspaceReason = 'A selected project file could not be read, so this code checkpoint is unavailable.' - const submittedText = 'Never echo this user message in checkpoint diagnostics' - apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ - status: 'idle', - summary: 'empty', - capabilities: { send: true, interrupt: true, fork: true }, - turns: [], - }) - apiMock.post.mockImplementation((url: string) => ( - url === '/api/fresh-agent/checkpoints' - ? Promise.reject(new ApiError(500, 'Code checkpoint unavailable', { - code: 'checkpoint_path_unreadable', - details: workspaceReason, - })) - : Promise.resolve({ ok: true }) - )) - store.dispatch(initLayout({ - tabId: 'tab-1', - paneId: 'pane-1', - content: { - kind: 'fresh-agent', - sessionType: 'freshcodex', - provider: 'codex', - createRequestId: 'req-checkpoint-server-error', - sessionId: 'thread-checkpoint-server-error', - status: 'idle', - initialCwd: '/repo', - }, - })) - - render( - <Provider store={store}> - <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> - </Provider>, - ) - - const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) - await waitFor(() => expect(composer).not.toBeDisabled()) - fireEvent.change(composer, { target: { value: submittedText } }) - fireEvent.click(screen.getByRole('button', { name: 'Send' })) - - const unavailable = await screen.findByRole('note', { name: 'Code rewind unavailable for this message' }) - expect(unavailable).toHaveTextContent(workspaceReason) - expect(unavailable).not.toHaveTextContent(submittedText) - await waitFor(() => { - expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ - type: 'freshAgent.send', - sessionId: 'thread-checkpoint-server-error', - text: submittedText, - })) - }) - }) - - it('bounds a stalled checkpoint capture and ignores its late result', async () => { - const store = createStore() - const checkpoint = createDeferred<{ id: string; ts: number; label: string; requestId: string }>() - let checkpointSignal: AbortSignal | undefined - apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ - status: 'idle', - summary: 'empty', - capabilities: { send: true, interrupt: true, fork: true }, - turns: [], - }) - apiMock.post.mockImplementation((url: string, _body: unknown, options?: { signal?: AbortSignal }) => { - if (url === '/api/fresh-agent/checkpoints') { - checkpointSignal = options?.signal - return checkpoint.promise - } - return Promise.resolve({ ok: true }) - }) - store.dispatch(initLayout({ - tabId: 'tab-1', - paneId: 'pane-1', - content: { - kind: 'fresh-agent', - sessionType: 'freshcodex', - provider: 'codex', - createRequestId: 'req-checkpoint-timeout', - sessionId: 'thread-checkpoint-timeout', - status: 'idle', - initialCwd: '/repo', - }, - })) - - render( - <Provider store={store}> - <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> - </Provider>, - ) - - const composer = await screen.findByRole('textbox', { name: 'Chat message input' }) - await waitFor(() => expect(composer).not.toBeDisabled()) - vi.useFakeTimers() - try { - fireEvent.change(composer, { target: { value: 'Send even if checkpoint capture stalls' } }) - fireEvent.click(screen.getByRole('button', { name: 'Send' })) - expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(0) - - await act(async () => { - await vi.advanceTimersByTimeAsync(30_000) - }) - - expect(checkpointSignal?.aborted).toBe(true) - expect(sentFreshAgentMessages('freshAgent.send')).toContainEqual(expect.objectContaining({ - type: 'freshAgent.send', - sessionId: 'thread-checkpoint-timeout', - text: 'Send even if checkpoint capture stalls', - })) - expect(screen.getByRole('note', { name: 'Code rewind unavailable for this message' })) - .toHaveTextContent('Code checkpoint capture timed out after 30 seconds.') - - await act(async () => { - checkpoint.resolve({ - id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', - ts: 1, - label: 'late result', - requestId: String(apiMock.post.mock.calls.find(([url]) => url === '/api/fresh-agent/checkpoints')?.[1]?.requestId), - }) - await Promise.resolve() - }) - - expect(screen.getByRole('note', { name: 'Code rewind unavailable for this message' })) - .toHaveTextContent('Code checkpoint capture timed out after 30 seconds.') - expect(screen.queryByRole('status', { name: 'Code rewind available for this message' })).not.toBeInTheDocument() - } finally { - vi.useRealTimers() - } - }) - it('persists a pending local echo so a remounted pane keeps the submitted prompt visible', async () => { const store = createStore() apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ @@ -3982,10 +4003,15 @@ describe('FreshAgentView', () => { it('re-attaches with the route cwd and resends once when a send fails with FRESH_AGENT_LOST_SESSION', async () => { const store = createStore() let onMessage: ((message: Record<string, unknown>) => void) | undefined + const reconnectHandlers: Array<() => void> = [] wsMock.onMessage.mockImplementation((handler: (message: Record<string, unknown>) => void) => { onMessage = handler return () => {} }) + wsMock.onReconnect.mockImplementation((handler: () => void) => { + reconnectHandlers.push(handler) + return () => {} + }) apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ status: 'idle', summary: 'empty', @@ -4026,9 +4052,17 @@ describe('FreshAgentView', () => { const sendFrame = sentFreshAgentMessages('freshAgent.send').at(-1) expect(sendFrame).toBeTruthy() expect(sendFrame?.text).toBe('hello again') + const initialLogicalAttemptId = (sendFrame?.checkpointIntent as Record<string, unknown>)?.logicalAttemptId + expect(initialLogicalAttemptId).toEqual(expect.any(String)) await waitFor(() => { expect(getFreshAgentPaneContent(store)).toMatchObject({ status: 'running' }) }) + + // An ambiguous transport reconnect is not permission for the client to + // release another provider send. The server joins by logicalAttemptId. + act(() => reconnectHandlers.forEach((handler) => handler())) + await act(async () => { await Promise.resolve() }) + expect(sentFreshAgentMessages('freshAgent.send')).toHaveLength(1) wsMock.send.mockClear() // Act: server rejects with the lost-session code for that request @@ -4056,6 +4090,8 @@ describe('FreshAgentView', () => { const retried = sentFreshAgentMessages('freshAgent.send').at(-1) expect(retried?.requestId).toEqual(expect.any(String)) expect(retried?.requestId).not.toBe(sendFrame?.requestId) + expect((retried?.checkpointIntent as Record<string, unknown>)?.logicalAttemptId).toEqual(expect.any(String)) + expect((retried?.checkpointIntent as Record<string, unknown>)?.logicalAttemptId).not.toBe(initialLogicalAttemptId) wsMock.send.mockClear() act(() => { onMessage?.({ diff --git a/test/unit/client/lib/resume-session.workspace.test.ts b/test/unit/client/lib/resume-session.workspace.test.ts new file mode 100644 index 000000000..2db74c877 --- /dev/null +++ b/test/unit/client/lib/resume-session.workspace.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, it } from 'vitest' +import { configureStore } from '@reduxjs/toolkit' +import connectionReducer from '@/store/connectionSlice' +import panesReducer, { initLayout } from '@/store/panesSlice' +import tabsReducer from '@/store/tabsSlice' +import { findCheckpointWorkspaceForSession } from '@/lib/resume-session' +import type { RootState } from '@/store/store' + +const sessionId = '550e8400-e29b-41d4-a716-446655440090' +const checkpointWorkspace = { path: '/selected/project', provenance: 'directory-picker' as const } + +function createStore() { + return configureStore({ + reducer: { tabs: tabsReducer, panes: panesReducer, connection: connectionReducer }, + preloadedState: { + tabs: { + tabs: [{ + id: 'tab-1', + createRequestId: 'tab-1', + title: 'Fresh session', + status: 'running' as const, + mode: 'claude' as const, + initialCwd: '/repo', + sessionRef: { provider: 'claude' as const, sessionId }, + checkpointWorkspace, + createdAt: 1, + }], + activeTabId: 'tab-1', + }, + panes: { layouts: {}, activePane: {}, paneTitles: {} }, + connection: { + status: 'ready' as const, + serverInstanceId: 'server-1', + platform: null, + availableClis: {}, + featureFlags: {}, + }, + }, + }) +} + +describe('findCheckpointWorkspaceForSession', () => { + it('reads the persisted workspace from a matching tab when its layout needs hydration', () => { + const store = createStore() + + expect(findCheckpointWorkspaceForSession(store.getState() as RootState, { + provider: 'claude', + sessionId, + })).toEqual(checkpointWorkspace) + }) + + it('reads the current workspace from a matching fresh-agent pane', () => { + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshclaude', + provider: 'claude', + createRequestId: 'req-1', + sessionId, + sessionRef: { provider: 'claude', sessionId }, + status: 'idle', + checkpointWorkspace, + }, + })) + + expect(findCheckpointWorkspaceForSession(store.getState() as RootState, { + provider: 'claude', + sessionId, + })).toEqual(checkpointWorkspace) + }) + + it('does not use a tab workspace for a different pane in the same split tab', () => { + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'terminal', + mode: 'claude', + createRequestId: 'terminal-req', + status: 'running', + sessionRef: { provider: 'claude', sessionId }, + }, + })) + + expect(findCheckpointWorkspaceForSession(store.getState() as RootState, { + provider: 'claude', + sessionId, + })).toBeUndefined() + }) +}) diff --git a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts index 219e28e75..134a5dcd8 100644 --- a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts +++ b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts @@ -43,6 +43,12 @@ describe('FreshAgent checkpoint outcomes', () => { status: 'unavailable', reason: 'invalid identity', }, + { + version: 2, + requestId: 'unsupported-version', + status: 'unavailable', + reason: 'must not be interpreted as version 1', + }, { requestId: 'stale-conversation-outcome', conversationRequestId: 'create-replaced', @@ -59,6 +65,7 @@ describe('FreshAgent checkpoint outcomes', () => { kind: 'fresh-agent', checkpointOutcomes: [ { + version: 1, requestId: 'send-available', conversationRequestId: 'create-1', displayTurnId: 'turn-4:row-0', @@ -68,6 +75,7 @@ describe('FreshAgent checkpoint outcomes', () => { workspace: { path: '/work/project', provenance: 'directory-picker' }, }, { + version: 1, requestId: 'send-unavailable', conversationRequestId: 'create-1', displayTurnId: 'turn-5:row-0', diff --git a/test/unit/client/store/persistControl.test.ts b/test/unit/client/store/persistControl.test.ts index 2711247eb..301259af2 100644 --- a/test/unit/client/store/persistControl.test.ts +++ b/test/unit/client/store/persistControl.test.ts @@ -104,6 +104,7 @@ describe('persistControl', () => { const parsed = parsePersistedLayoutRaw(raw!)! const savedContent = (parsed.panes.layouts['tab-1'] as any).content expect(savedContent.checkpointOutcomes).toEqual([{ + version: 1, conversationRequestId: 'create-1', requestId: 'send-1', displayTurnId: 'turn-4:row-0', diff --git a/test/unit/client/store/tabsSlice.test.ts b/test/unit/client/store/tabsSlice.test.ts index f6b171a66..c60078de1 100644 --- a/test/unit/client/store/tabsSlice.test.ts +++ b/test/unit/client/store/tabsSlice.test.ts @@ -1114,6 +1114,7 @@ describe('tabsSlice', () => { it('opens a Claude history row as a fresh-agent pane with resumeSessionId', async () => { const store = createOpenSessionStore() + const checkpointWorkspace = { path: '/selected/repo', provenance: 'directory-picker' as const } await store.dispatch(openSessionTab({ sessionId: VALID_CLAUDE_SESSION_ID, @@ -1121,12 +1122,14 @@ describe('tabsSlice', () => { cwd: '/repo', provider: 'claude', sessionType: 'freshclaude', + checkpointWorkspace, })) const state = store.getState() const tab = state.tabs.tabs.find((candidate) => candidate.title === 'Existing Freshclaude session') expect(tab).toBeTruthy() expect(tab?.sessionRef).toEqual({ provider: 'claude', sessionId: VALID_CLAUDE_SESSION_ID }) + expect(tab?.checkpointWorkspace).toEqual(checkpointWorkspace) const layout = state.panes.layouts[tab!.id] expect(layout.type).toBe('leaf') @@ -1136,6 +1139,7 @@ describe('tabsSlice', () => { provider: 'claude', resumeSessionId: VALID_CLAUDE_SESSION_ID, sessionRef: { provider: 'claude', sessionId: VALID_CLAUDE_SESSION_ID }, + checkpointWorkspace, }) }) From 5f972fa6064261c3eb6266750cde520fa3a3cbba Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 03:05:34 -0700 Subject: [PATCH 306/493] Add configured checkpoint browser target --- .../managed-provider-environment-target.ts | 509 +++++++++++++++++- ...code-checkpoint-workspace-external.spec.ts | 363 +++++++++++++ .../codex-app-server/fake-app-server.mjs | 7 +- .../tooling/testing/browser-selection.test.ts | 22 + ...anaged-provider-environment-target.test.ts | 30 ++ 5 files changed, 927 insertions(+), 4 deletions(-) create mode 100644 test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts create mode 100644 test/unit/tooling/testing/managed-provider-environment-target.test.ts diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 04bfce8da..8abd97317 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1,6 +1,7 @@ import assert from 'node:assert/strict' -import { execFileSync, spawn, type ChildProcess } from 'node:child_process' +import { execFileSync, spawn, spawnSync, type ChildProcess } from 'node:child_process' import { createHash, randomUUID } from 'node:crypto' +import { createRequire } from 'node:module' import fs from 'node:fs/promises' import os from 'node:os' import path from 'node:path' @@ -21,9 +22,21 @@ const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' const scenarios = ['healthy', 'missing-reference', 'startup-before-identity'] as const const ordinaryFixtureMode = 'managed-fixture-ordinary' +const checkpointImage = 'freshell-sandbox:latest' +const checkpointContainerPort = 3477 +const checkpointFixtureRoot = '/tmp/freshell-checkpoint-fixture' +const checkpointControlRequest = `${checkpointFixtureRoot}/control/request.json` +const checkpointControlAck = (id: string) => `${checkpointFixtureRoot}/control/ack-${id}.json` +export type ManagedProviderEnvironmentTargetMode = 'managed-environment' | 'code-checkpoint' const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') +export function resolveManagedProviderEnvironmentTargetMode(args: string[]): ManagedProviderEnvironmentTargetMode { + if (args.length === 0) return 'managed-environment' + if (args.length === 1 && args[0] === '--checkpoint') return 'code-checkpoint' + throw new Error('Usage: managed-provider-environment-target.ts [--checkpoint]') +} + async function poll<T>(description: string, operation: () => Promise<T | undefined>, timeout = 180_000): Promise<T> { const end = Date.now() + timeout while (Date.now() < end) { @@ -230,6 +243,226 @@ async function stageProof(rig: ManagedRuntimeBrowserRig, view: ManagedRuntimeVie const requireMode = (metadata: { mode: number }) => (metadata.mode & 0o777).toString(8).padStart(4, '0') +const checkpointTargetLauncher = String.raw`import assert from 'node:assert/strict' +import fs from 'node:fs' +import fsp from 'node:fs/promises' +import path from 'node:path' +import { execFileSync, spawn } from 'node:child_process' + +const root = '/tmp/freshell-checkpoint-fixture' +const uid = Number(process.env.FRESHELL_CHECKPOINT_FIXTURE_UID) +assert.equal(process.env.FRESHELL_CHECKPOINT_FIXTURE_ROOT, root) +assert(Number.isSafeInteger(uid) && uid > 0) +assert.equal(process.getuid(), uid, 'fixture setup must run as restricted UID B') +const status = fs.readFileSync('/proc/self/status', 'utf8') +const capabilities = status.match(/^CapEff:\s+([0-9a-f]+)$/m)?.[1] +assert.equal(capabilities, '0000000000000000', 'fixture UID B must have no effective capabilities') + +const project = path.join(root, 'project') +const home = path.join(root, 'home') +const storage = path.join(root, 'storage') +const control = path.join(root, 'control') +const deniedFile = path.join(project, 'private-file.ignored') +const deniedDirectory = path.join(project, 'private-dir.ignored') +const mode = (selected) => fs.lstatSync(selected).mode & 0o777 +function assertDeniedFile(selected) { + assert.throws(() => fs.readFileSync(selected), { code: 'EACCES' }) +} +function assertDeniedDirectory(selected) { + assert.throws(() => fs.readdirSync(selected), { code: 'EACCES' }) +} +const fileStat = fs.lstatSync(deniedFile) +const directoryStat = fs.lstatSync(deniedDirectory) +assert.equal(fileStat.uid, 0) +assert.equal(mode(deniedFile), 0o600) +assert.equal(directoryStat.uid, 0) +assert.equal(mode(deniedDirectory), 0o700) +assertDeniedFile(deniedFile) +assertDeniedDirectory(deniedDirectory) +const eligibleFile = path.join(project, 'private-file') +const eligibleDirectory = path.join(project, 'private-dir') +fs.renameSync(deniedFile, eligibleFile) +assertDeniedFile(eligibleFile) +fs.renameSync(eligibleFile, deniedFile) +fs.renameSync(deniedDirectory, eligibleDirectory) +assertDeniedDirectory(eligibleDirectory) +fs.renameSync(eligibleDirectory, deniedDirectory) +assert.deepEqual([fs.lstatSync(deniedFile).uid, mode(deniedFile)], [0, 0o600]) +assert.deepEqual([fs.lstatSync(deniedDirectory).uid, mode(deniedDirectory)], [0, 0o700]) + +fs.mkdirSync(control, { recursive: true, mode: 0o700 }) +fs.mkdirSync(path.join(project, '.freshell-checkpoint-fixture'), { recursive: true, mode: 0o700 }) +fs.writeFileSync(path.join(project, 'main.rs'), 'main before checkpoint\n') +fs.writeFileSync(path.join(project, 'notes.txt'), 'notes before checkpoint\n') +fs.writeFileSync(path.join(project, 'run.sh'), '#!/bin/sh\necho before\n', { mode: 0o755 }) +fs.symlinkSync('main.rs', path.join(project, 'main-link')) +execFileSync('git', ['init', '-q'], { cwd: project, stdio: 'ignore' }) +execFileSync('git', ['config', 'user.name', 'Freshell checkpoint fixture'], { cwd: project, stdio: 'ignore' }) +execFileSync('git', ['config', 'user.email', 'fixture@example.invalid'], { cwd: project, stdio: 'ignore' }) +execFileSync('git', ['add', '--', '.gitignore', 'main.rs', 'notes.txt', 'run.sh', 'main-link'], { cwd: project, stdio: 'ignore' }) +execFileSync('git', ['commit', '-q', '-m', 'synthetic checkpoint fixture'], { cwd: project, stdio: 'ignore' }) + +const nonGit = path.join(root, 'non-git') +fs.mkdirSync(nonGit, { mode: 0o755 }) +fs.writeFileSync(path.join(nonGit, 'one.txt'), 'one before checkpoint\n') +fs.writeFileSync(path.join(nonGit, 'two.txt'), 'two before checkpoint\n') +fs.mkdirSync(path.join(home, '.freshell'), { recursive: true, mode: 0o700 }) +fs.mkdirSync(path.join(home, '.codex'), { recursive: true, mode: 0o700 }) +fs.mkdirSync(path.join(storage, 'logs'), { recursive: true, mode: 0o700 }) +fs.writeFileSync(path.join(home, '.freshell', 'config.json'), JSON.stringify({ version: 1, settings: { + network: { configured: true, host: '127.0.0.1' }, + freshAgent: { enabled: true, initialSetupDone: true, defaultPlugins: [], providers: {} }, + codingCli: { enabledProviders: ['codex'], providers: { codex: { model: 'gpt-5-codex', sandbox: 'workspace-write' } } }, +} })) +fs.writeFileSync(path.join(control, 'initialization.json'), JSON.stringify({ restrictedUid: uid, + effectiveCapabilities: capabilities, privateFileDenied: true, privateDirectoryDenied: true, + privateFileUid: fileStat.uid, privateFileMode: '0600', privateDirectoryUid: directoryStat.uid, + privateDirectoryMode: '0700', sameParentRenameVerified: true, nonGitWorkspace: nonGit })) +console.log(JSON.stringify({ severity: 'info', event: 'code_checkpoint.target_fixture_initialized', + restrictedUid: uid, effectiveCapabilities: capabilities, privateFileDenied: true, + privateDirectoryDenied: true, sameParentRenameVerified: true })) + +const child = spawn('/checkpoint-target/freshell-server', [], { stdio: 'inherit', env: process.env }) +child.once('error', (error) => { console.error(error.message); process.exitCode = 1 }) +for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => child.kill(signal)) +const exit = await new Promise((resolve, reject) => { + child.once('error', reject) + child.once('exit', (code, signal) => resolve({ code, signal })) +}) +if (exit.signal) process.exitCode = 1 +else process.exitCode = exit.code ?? 1 +` + +const checkpointTargetControl = String.raw`import assert from 'node:assert/strict' +import fs from 'node:fs' +import path from 'node:path' + +const [action, requestId, expectedUidText] = process.argv.slice(2) +const expectedUid = Number(expectedUidText) +assert(['eligible', 'ignored'].includes(action)) +assert(/^[a-zA-Z0-9-]{1,96}$/.test(requestId ?? '')) +assert(Number.isSafeInteger(expectedUid) && expectedUid > 0) +assert.equal(process.getuid(), expectedUid, 'control must run as restricted UID B') +const capabilities = fs.readFileSync('/proc/self/status', 'utf8').match(/^CapEff:\s+([0-9a-f]+)$/m)?.[1] +assert.equal(capabilities, '0000000000000000', 'control must not receive capabilities') +const project = '/tmp/freshell-checkpoint-fixture/project' +const from = path.join(project, action === 'eligible' ? 'private-file.ignored' : 'private-file') +const to = path.join(project, action === 'eligible' ? 'private-file' : 'private-file.ignored') +if (fs.existsSync(from)) fs.renameSync(from, to) +const privatePath = action === 'eligible' ? to : path.join(project, 'private-file.ignored') +const privateStat = fs.lstatSync(privatePath) +assert.equal(privateStat.uid, 0) +assert.equal(privateStat.mode & 0o777, 0o600) +assert.throws(() => fs.readFileSync(privatePath), { code: 'EACCES' }) +const root = '/tmp/freshell-checkpoint-fixture/control' +const ack = path.join(root, 'ack-' + requestId + '.json') +fs.writeFileSync(ack, JSON.stringify({ requestId, action, restrictedUid: expectedUid, + privateFileUid: privateStat.uid, privateFileMode: '0600', privateFileDenied: true, + effectiveCapabilities: capabilities }), { flag: 'wx', mode: 0o600 }) +` + +function docker(args: string[], timeout = 30_000): string { + return execFileSync('docker', args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout }).trim() +} + +function dockerMaybe(args: string[], timeout = 10_000): string | undefined { + const result = spawnSync('docker', args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout }) + if (result.error) throw result.error + if (result.status !== 0) return undefined + return result.stdout.trim() +} + +async function stageCheckpointTarget(stage: string, repoRoot: string): Promise<void> { + await fs.mkdir(stage, { recursive: true, mode: 0o700 }) + const binary = path.join(repoRoot, 'target/debug/freshell-server') + const client = path.join(repoRoot, 'dist/client') + assert((await fs.stat(binary)).isFile(), 'Feature-enabled checkpoint server binary is missing') + assert((await fs.stat(client)).isDirectory(), 'Built client assets are missing') + await fs.copyFile(binary, path.join(stage, 'freshell-server')) + await fs.chmod(path.join(stage, 'freshell-server'), 0o555) + await fs.cp(client, path.join(stage, 'client'), { recursive: true, dereference: true, preserveTimestamps: false }) + await fs.copyFile(path.resolve(repoRoot, 'docker/sandbox/entrypoint.sh'), path.join(stage, 'sandbox-entrypoint.sh')) + await fs.chmod(path.join(stage, 'sandbox-entrypoint.sh'), 0o555) + await fs.writeFile(path.join(stage, 'launch.mjs'), checkpointTargetLauncher, { mode: 0o444 }) + await fs.writeFile(path.join(stage, 'control.mjs'), checkpointTargetControl, { mode: 0o444 }) + const fake = path.resolve(repoRoot, 'test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs') + await fs.copyFile(fake, path.join(stage, 'fake-app-server.mjs')) + await fs.chmod(path.join(stage, 'fake-app-server.mjs'), 0o444) + const require = createRequire(import.meta.url) + const wsRoot = path.dirname(require.resolve('ws')) + await fs.mkdir(path.join(stage, 'node_modules'), { mode: 0o700 }) + await fs.cp(wsRoot, path.join(stage, 'node_modules/ws'), { recursive: true, dereference: true, preserveTimestamps: false }) + await fs.chmod(path.join(stage, 'node_modules'), 0o555) +} + +function restrictedIdentity(): { uid: number; gid: number } { + const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') + if (!dockerMaybe(['image', 'inspect', '--format', '{{.Id}}', checkpointImage])) { + execFileSync(path.join(repoRoot, 'scripts/sandbox-build.sh'), { cwd: repoRoot, stdio: 'inherit' }) + } + const readIdentity = () => ({ + uid: Number(docker(['run', '--rm', '--read-only', '--network', 'none', '--entrypoint', '/usr/bin/id', + checkpointImage, '-u', 'sandbox'])), + gid: Number(docker(['run', '--rm', '--read-only', '--network', 'none', '--entrypoint', '/usr/bin/id', + checkpointImage, '-g', 'sandbox'])), + }) + let { uid, gid } = readIdentity() + if (uid !== process.getuid() || gid !== process.getgid()) { + execFileSync(path.join(repoRoot, 'scripts/sandbox-build.sh'), { cwd: repoRoot, stdio: 'inherit' }) + const rebuiltIdentity = readIdentity() + uid = rebuiltIdentity.uid + gid = rebuiltIdentity.gid + } + assert(Number.isSafeInteger(uid) && uid > 0 && Number.isSafeInteger(gid) && gid > 0, + 'Sandbox image must provide its fixed non-root restricted identity') + return { uid, gid } +} + +async function externalTargetRead(baseUrl: string, token: string, selected: string): Promise<any | undefined> { + const response = await fetch(`${baseUrl}/api/files/read?path=${encodeURIComponent(selected)}`, { + headers: { 'x-auth-token': token }, signal: AbortSignal.timeout(15_000), + }) + if (response.status === 404) return undefined + assert(response.ok, `Owned checkpoint target control read failed with HTTP ${response.status}`) + const body = await response.json() as { content?: unknown } + assert(typeof body.content === 'string') + return JSON.parse(body.content) +} + +async function serveCheckpointControls( + baseUrl: string, + token: string, + containerId: string, + identity: { uid: number; gid: number }, + fixture: { expectedNonce: string }, + browser: ChildProcess, + proof: Record<string, unknown>, +): Promise<void> { + const processed = new Set<string>() + while (browser.exitCode === null && browser.signalCode === null) { + const request = await externalTargetRead(baseUrl, token, checkpointControlRequest) + if (request && typeof request.requestId === 'string' && !processed.has(request.requestId)) { + assert.deepEqual(Object.keys(request).sort(), ['action', 'expectedNonce', 'requestId']) + assert.equal(request.expectedNonce, fixture.expectedNonce, 'Fixture control nonce did not match this owned run') + assert(['eligible', 'ignored'].includes(request.action), 'Fixture requested an unsupported private-path state') + assert.match(request.requestId, /^[a-zA-Z0-9-]{1,96}$/) + processed.add(request.requestId) + execFileSync('docker', ['exec', '--user', `${identity.uid}:${identity.gid}`, containerId, + '/usr/bin/node', '/checkpoint-target/control.mjs', request.action, request.requestId, String(identity.uid)], + { stdio: ['ignore', 'ignore', 'pipe'], timeout: 15_000 }) + const ack = await poll(`restricted fixture control acknowledgement ${request.requestId}`, async () => { + return externalTargetRead(baseUrl, token, checkpointControlAck(request.requestId)) + }, 15_000) + assert.deepEqual(ack, { requestId: request.requestId, action: request.action, restrictedUid: identity.uid, + privateFileUid: 0, privateFileMode: '0600', privateFileDenied: true, effectiveCapabilities: '0000000000000000' }) + const controls = (proof.controls ??= []) as unknown[] + controls.push({ requestId: request.requestId, action: request.action, restrictedUid: identity.uid, + privateFileDenied: true, effectiveCapabilities: ack.effectiveCapabilities }) + } + await delay(150) + } +} + export function buildManagedEnvironmentClient(repoRoot: string, candidateSha: string): void { assert.match(candidateSha, /^[a-f0-9]{40}$/, 'Managed acceptance requires a known worktree commit') execFileSync('pnpm', ['run', 'build:client'], { @@ -239,6 +472,269 @@ export function buildManagedEnvironmentClient(repoRoot: string, candidateSha: st }) } +export async function runCodeCheckpointTarget() { + assert.equal(process.env.FRESHELL_E2E_BACKEND, 'cloud', 'Checkpoint acceptance requires configured Cloud') + assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Checkpoint acceptance requires the robot identity') + assert.equal(process.env.GCLOUD_ROBOT_ACCOUNT, 'gcloud-robot@misc-puttering-project.iam.gserviceaccount.com') + const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') + const candidateSha = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: repoRoot, encoding: 'utf8', timeout: 5_000 }).trim() + assert.match(candidateSha, /^[a-f0-9]{40}$/) + const dirty = execFileSync('git', ['status', '--porcelain'], { cwd: repoRoot, encoding: 'utf8', timeout: 5_000 }).trim() + assert.equal(dirty, '', 'Checkpoint Cloud acceptance must build an exact committed candidate') + execFileSync('mise', ['exec', 'rust@1.96', '--', 'cargo', 'build', '-p', 'freshell-server', + '--features', 'managed-runtime-v1', '--locked'], { + cwd: repoRoot, env: { ...process.env, FRESHELL_BUILD_COMMIT: candidateSha }, stdio: 'inherit', + }) + buildManagedEnvironmentClient(repoRoot, candidateSha) + + const runId = randomUUID() + const expectedNonce = randomUUID() + const identity = restrictedIdentity() + assert.equal(identity.uid, process.getuid(), 'Sandbox image UID must match the invoking test owner for read-only fixture mounts') + assert.equal(identity.gid, process.getgid(), 'Sandbox image GID must match the invoking test owner for read-only fixture mounts') + const root = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-code-checkpoint-')) + const stage = path.join(root, 'artifacts') + const token = randomUUID() + const fixture = { + candidateSha, + runId, + syntheticWorkspacePath: path.join(checkpointFixtureRoot, 'project'), + syntheticHomePath: path.join(checkpointFixtureRoot, 'home'), + nonGitWorkspacePath: path.join(checkpointFixtureRoot, 'non-git'), + restrictedUid: identity.uid, + expectedNonce, + } + const proof: Record<string, unknown> = { candidateSha, runId, fixture, image: checkpointImage } + const networkName = `freshell-checkpoint-${runId.slice(0, 8)}` + const containerName = `freshell-checkpoint-${runId.slice(0, 8)}` + const containerIdPath = path.join(root, 'container.id') + const dockerEnvPath = path.join(root, 'target.env') + const cloudDispatchReceiptPath = path.join(root, 'cloud-execution-dispatch.json') + const evidenceDir = path.join(repoRoot, '.runtime-evidence', candidateSha, runId) + const receiptPath = path.join(evidenceDir, 'code-checkpoint-acceptance.json') + let networkId: string | undefined + let containerId: string | undefined + let dockerRun: ChildProcess | undefined, dockerRunIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, + dockerRunJoin: Promise<NativeExit> | undefined + let tunnel: ChildProcess | undefined, tunnelIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, + tunnelJoin: Promise<NativeExit> | undefined + let browser: ChildProcess | undefined, browserIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, + browserJoin: Promise<NativeExit> | undefined + let dockerStdout = '', dockerStderr = '' + const browserSummaryCapture = new ManagedEnvironmentBrowserSummaryCapture() + try { + await stageCheckpointTarget(stage, repoRoot) + await fs.writeFile(dockerEnvPath, [ + `AUTH_TOKEN=${token}`, + `PORT=${checkpointContainerPort}`, + 'NODE_ENV=production', + 'FRESHELL_BIND_HOST=0.0.0.0', + 'FRESHELL_DISABLE_WSL_PORT_FORWARD=1', + 'HIDE_STARTUP_TOKEN=true', + `FRESHELL_HOME=${fixture.syntheticHomePath}`, + `HOME=${fixture.syntheticHomePath}`, + `CODEX_HOME=${fixture.syntheticHomePath}/.codex`, + `FRESHELL_LOG_DIR=${checkpointFixtureRoot}/storage/logs`, + 'FRESHELL_CLIENT_DIR=/checkpoint-target/client', + 'CODEX_CMD=node /checkpoint-target/fake-app-server.mjs', + 'FAKE_CODEX_APP_SERVER_BEHAVIOR={"recordTurns":true,"uniqueThreadIds":true}', + 'FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES=1', + `FRESHELL_CHECKPOINT_FIXTURE_ROOT=${checkpointFixtureRoot}`, + `FRESHELL_CHECKPOINT_FIXTURE_UID=${identity.uid}`, + 'FRESHELL_SANDBOX_MODE=runtime-suite', + 'FRESHELL_SANDBOX_CHECKPOINT_FIXTURE=1', + '', + ].join('\n'), { mode: 0o600 }) + + networkId = docker(['network', 'create', '--driver', 'bridge', '--internal', '--label', 'freshell.owner=checkpoint-acceptance', + '--label', `freshell.run-id=${runId}`, networkName]) + assert.match(networkId, /^[a-f0-9]{64}$/) + const networkDetails = docker(['network', 'inspect', '--format', '{{.Id}} {{index .Labels "freshell.run-id"}} {{.Internal}}', networkId]) + assert.equal(networkDetails, `${networkId} ${runId} true`, 'Checkpoint network must be internally isolated and owned by this run') + + const dockerArgs = [ + 'run', '--rm', '--init', '--name', containerName, '--cidfile', containerIdPath, + '--label', 'freshell.owner=checkpoint-acceptance', '--label', `freshell.run-id=${runId}`, + '--network', networkId, '--publish', `127.0.0.1::${checkpointContainerPort}/tcp`, + '--pids-limit', '512', '--memory', '4g', '--read-only', '--security-opt', 'no-new-privileges', + '--tmpfs', '/tmp:rw,nosuid,nodev,size=512m', '--tmpfs', '/workspace:rw,nosuid,nodev,size=64m', + '--tmpfs', '/usr/local/cargo:rw,nosuid,nodev,size=256m', + '--tmpfs', '/home/sandbox/.cache:rw,nosuid,nodev,size=256m', + '--mount', `type=bind,source=${stage},target=/checkpoint-target,readonly`, + '--env-file', dockerEnvPath, '--entrypoint', '/checkpoint-target/sandbox-entrypoint.sh', + checkpointImage, '/usr/local/bin/node', '/checkpoint-target/launch.mjs', + ] + dockerRun = spawn('docker', dockerArgs, { cwd: root, stdio: ['ignore', 'pipe', 'pipe'] }) + dockerRunJoin = joinedAfterOutput(dockerRun); dockerRunJoin.catch(() => {}) + dockerRun.stdout!.on('data', (chunk: Buffer) => { + dockerStdout = (dockerStdout + chunk.toString()).slice(-262_144) + }) + dockerRun.stderr!.on('data', (chunk: Buffer) => { + dockerStderr = (dockerStderr + chunk.toString()).slice(-262_144) + }) + dockerRunIdentity = await processIdentity(dockerRun) + containerId = await poll('owned checkpoint container identity', async () => { + const contents = await fs.readFile(containerIdPath, 'utf8').catch(error => { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return undefined + throw error + }) + if (contents) return contents.trim() + assert.equal(dockerRun!.exitCode, null, 'Owned checkpoint container exited before publishing its ID') + return undefined + }, 60_000) + assert.match(containerId, /^[a-f0-9]{64}$/) + const containerDetails = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', containerId]) + assert.equal(containerDetails, `${containerId} ${runId}`, 'Checkpoint container ownership labels did not match this run') + const mapping = await poll('loopback-only ephemeral checkpoint target port', async () => { + const value = dockerMaybe(['port', containerId!, `${checkpointContainerPort}/tcp`]) + if (!value) return undefined + const match = value.match(/^127\.0\.0\.1:(\d+)$/m) + if (!match) throw new Error('Checkpoint target port was not published on host loopback only') + return Number(match[1]) + }, 60_000) + assert(Number.isSafeInteger(mapping) && mapping > 0 && mapping <= 65535) + const baseUrl = `http://127.0.0.1:${mapping}` + await poll('restricted checkpoint target health', async () => { + const response = await fetch(`${baseUrl}/api/health`, { signal: AbortSignal.timeout(5_000) }).catch(() => undefined) + return response?.ok ? true : undefined + }, 120_000) + const initialization = await poll('fixed restricted-UID initializer proof', () => + externalTargetRead(baseUrl, token, `${checkpointFixtureRoot}/control/initialization.json`), 30_000) + assert.deepEqual(initialization, { restrictedUid: identity.uid, effectiveCapabilities: '0000000000000000', + privateFileDenied: true, privateDirectoryDenied: true, privateFileUid: 0, privateFileMode: '0600', + privateDirectoryUid: 0, privateDirectoryMode: '0700', sameParentRenameVerified: true, + nonGitWorkspace: fixture.nonGitWorkspacePath }) + const serverInfoResponse = await fetch(`${baseUrl}/api/server-info`, { headers: { 'x-auth-token': token }, signal: AbortSignal.timeout(15_000) }) + assert(serverInfoResponse.ok) + const serverInfo = await serverInfoResponse.json() as { commit?: unknown; buildDirty?: unknown; runtime?: unknown } + assert.equal(serverInfo.runtime, 'rust') + assert.equal(serverInfo.commit, candidateSha) + assert.equal(serverInfo.buildDirty, false) + proof.target = { containerId, containerName, networkId, networkName, networkInternal: true, restrictedIdentity: identity, + hostLoopbackPort: mapping, serverInfo, initialization } + + const toolHome = path.join(root, 'cloudflared-home') + await fs.mkdir(toolHome, { mode: 0o700 }) + const tool = path.join(root, 'cloudflared') + const download = await fetch(`https://github.com/cloudflare/cloudflared/releases/download/${cloudflaredVersion}/cloudflared-linux-amd64`, + { signal: AbortSignal.timeout(120_000) }) + assert(download.ok) + const bytes = Buffer.from(await download.arrayBuffer()) + assert.equal(digest(bytes), cloudflaredSha, 'Pinned cloudflared checksum mismatch') + await fs.writeFile(tool, bytes, { mode: 0o700 }) + const versionText = execFileSync(tool, ['--version'], { cwd: root, env: { PATH: process.env.PATH ?? '', HOME: toolHome }, + encoding: 'utf8', timeout: 30_000 }) + assert.match(versionText, /^cloudflared version 2026\.10\.0(?:\s|$)/) + tunnel = spawn(tool, ['tunnel', '--url', baseUrl], { + cwd: root, env: { PATH: process.env.PATH ?? '', HOME: toolHome, XDG_CONFIG_HOME: toolHome }, + stdio: ['ignore', 'pipe', 'pipe'], + }) + tunnelJoin = joined(tunnel); tunnelJoin.catch(() => {}) + tunnelIdentity = await processIdentity(tunnel) + let tunnelLog = '' + const tunnelOutput = (chunk: Buffer) => { tunnelLog = (tunnelLog + chunk.toString()).slice(-65_536) } + tunnel.stdout!.on('data', tunnelOutput); tunnel.stderr!.on('data', tunnelOutput) + const origin = await poll('pinned checkpoint Quick Tunnel hostname', async () => { + assert.equal(tunnel!.exitCode, null, 'Owned checkpoint tunnel exited before publication') + return tunnelLog.match(/https:\/\/[a-z0-9-]+\.trycloudflare\.com/)?.[0] + }, 120_000) + await poll('checkpoint tunnel health', async () => { + const response = await fetch(`${origin}/api/health`, { signal: AbortSignal.timeout(15_000) }).catch(() => undefined) + return response?.ok ? true : undefined + }, 120_000) + + const cloudEnvironment: NodeJS.ProcessEnv = { ...process.env, + FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'code-checkpoint', + FRESHELL_CODE_CHECKPOINT_FIXTURE: JSON.stringify(fixture), + FRESHELL_E2E_TARGET_URL: origin, + FRESHELL_E2E_TARGET_WS_URL: origin.replace('https:', 'wss:') + '/ws', + FRESHELL_E2E_TARGET_TOKEN: token, + } + delete cloudEnvironment.FRESHELL_MANAGED_ENVIRONMENT_FIXTURE + delete cloudEnvironment.FRESHELL_E2E_TARGET_HOME + const paired = resolvePairedExternalFixture(cloudEnvironment, true) + assert.equal(paired?.kind, 'code-checkpoint') + const browserSpec = 'test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts' + cloudEnvironment.FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT = cloudDispatchReceiptPath + browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--project=chromium-external-target', browserSpec], { + cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'pipe', 'pipe'], + }) + browser.stdout!.on('data', chunk => browserSummaryCapture.write(chunk)) + browser.stdout!.pipe(process.stdout); browser.stderr!.pipe(process.stderr) + browserJoin = joinedAfterOutput(browser); browserJoin.catch(() => {}) + browserIdentity = await processIdentity(browser) + const cloudDispatch = await waitForCloudExecutionDispatch(cloudDispatchReceiptPath, + () => browser?.exitCode === null && browser.signalCode === null) + proof.cloudExecution = cloudDispatch + console.log(JSON.stringify({ severity: 'info', event: 'code_checkpoint.cloud_execution_dispatched', ...cloudDispatch })) + await serveCheckpointControls(baseUrl, token, containerId, identity, fixture, browser, proof) + const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud checkpoint browser final join') + assert.equal(browserExit.code, 0, 'Configured Cloud checkpoint browser acceptance failed') + const browserSummary = browserSummaryCapture.finish() + assert.equal(browserSummary.executedCases, 3, 'Configured Cloud must execute all three checkpoint browser cases') + assert.equal(browserSummary.passedCases, 3, 'Every configured checkpoint browser case must pass') + assert.equal(browserSummary.failedCases, 0) + assert.equal(browserSummary.skippedCases, 0) + assert.equal(browserSummary.interruptedCases, 0) + proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, ...browserSummary } + await fs.mkdir(evidenceDir, { recursive: true, mode: 0o700 }) + await writeJson(receiptPath, proof) + console.log(JSON.stringify({ severity: 'info', event: 'code_checkpoint.acceptance_passed', candidateSha, runId, + executedBrowserCases: browserSummary.executedCases, passedBrowserCases: browserSummary.passedCases, receipt: receiptPath })) + } finally { + const errors: Error[] = [] + const cleanup: Record<string, unknown> = {} + const attempt = async (operation: string, action: () => unknown | Promise<unknown>) => { + try { cleanup[operation] = await action() } + catch (error) { errors.push(new Error(`Owned checkpoint target cleanup failed: ${operation}`, { cause: error })); cleanup[operation] = 'FAILED; retain owned fixture' } + } + await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin)) + await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin)) + if (containerId) await attempt('container', async () => { + const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) + if (matching.length) { + assert.deepEqual(matching, [containerId], 'Only the exact owned checkpoint container may be stopped') + const details = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', containerId]) + assert.equal(details, `${containerId} ${runId}`, 'Refusing to clean up an unowned checkpoint container') + docker(['container', 'stop', '--time', '10', containerId], 20_000) + } + const remaining = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) + if (remaining.length) docker(['container', 'rm', '--force', containerId], 20_000) + assert.deepEqual(docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean), []) + return { containerId, settled: true } + }) + else if (dockerRun && dockerRunIdentity && dockerRunJoin) await attempt('containerIdentity', async () => { + const named = docker(['ps', '-aq', '--no-trunc', '--filter', `name=^/${containerName}$`]).split('\n').filter(Boolean) + assert.deepEqual(named, [], 'Container without a published owned ID remains unknown; retain its fixture') + }) + await attempt('dockerRunJoin', () => stopExact(dockerRun, dockerRunIdentity, dockerRunJoin)) + if (networkId) await attempt('network', async () => { + const matching = docker(['network', 'ls', '-q', '--no-trunc', '--filter', `id=${networkId}`]).split('\n').filter(Boolean) + if (matching.length) { + assert.deepEqual(matching, [networkId], 'Only the exact owned checkpoint network may be removed') + const details = docker(['network', 'inspect', '--format', '{{.Id}} {{index .Labels "freshell.run-id"}}', networkId]) + assert.equal(details, `${networkId} ${runId}`, 'Refusing to remove an unowned checkpoint network') + docker(['network', 'rm', networkId]) + } + assert.deepEqual(docker(['network', 'ls', '-q', '--no-trunc', '--filter', `id=${networkId}`]).split('\n').filter(Boolean), []) + return { networkId, settled: true } + }) + if (errors.length === 0) await attempt('fixtureDirectory', () => fs.rm(root, { recursive: true, force: true })) + proof.cleanup = cleanup + proof.knownOwnedSettled = errors.length === 0 + if (errors.length) proof.retainedFixture = root + await attempt('cleanupReceipt', async () => { + proof.targetOutput = { + stdoutTail: dockerStdout.replaceAll(token, '[redacted]').slice(-8192), + stderrTail: dockerStderr.replaceAll(token, '[redacted]').slice(-8192), + } + await fs.mkdir(evidenceDir, { recursive: true, mode: 0o700 }) + await writeJson(path.join(evidenceDir, 'code-checkpoint-cleanup.json'), proof) + }) + if (errors.length) throw new AggregateError(errors, 'Checkpoint target cleanup remains unsettled') + } +} + export async function runManagedEnvironmentTarget() { assert.equal(process.env.FRESHELL_E2E_BACKEND, 'cloud', 'Managed acceptance requires configured Cloud') assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Managed acceptance requires the robot identity') @@ -496,9 +992,16 @@ export async function runManagedEnvironmentTarget() { } } +export async function runManagedProviderEnvironmentTarget(args = process.argv.slice(2)) { + const mode = resolveManagedProviderEnvironmentTargetMode(args) + if (mode === 'code-checkpoint') return runCodeCheckpointTarget() + return runManagedEnvironmentTarget() +} + if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) { - runManagedEnvironmentTarget().catch(error => { - console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.acceptance_failed', error: error instanceof Error ? error.message : 'Unknown fixture failure' })) + runManagedProviderEnvironmentTarget().catch(error => { + const mode = process.argv.slice(2).includes('--checkpoint') ? 'code_checkpoint' : 'managed_environment' + console.error(JSON.stringify({ severity: 'error', event: `${mode}.acceptance_failed`, error: error instanceof Error ? error.message : 'Unknown fixture failure' })) process.exitCode = 1 }) } diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts new file mode 100644 index 000000000..817384fc6 --- /dev/null +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -0,0 +1,363 @@ +import { randomUUID } from 'node:crypto' +import { resolvePairedExternalFixture } from '../../../scripts/testing/e2e-external-target.js' +import { test, expect } from '../helpers/fixtures.js' +import { openPanePicker } from '../helpers/pane-picker.js' +import { TestHarness } from '../helpers/test-harness.js' + +const paired = resolvePairedExternalFixture(process.env, true) +if (paired?.kind !== 'code-checkpoint') throw new Error('Code checkpoint acceptance requires its paired owned target') +const fixture = paired.fixture +const headers = { 'x-auth-token': paired.targetToken } +const checkpointRoot = '/tmp/freshell-checkpoint-fixture' +const controlRequestPath = `${checkpointRoot}/control/request.json` + +type FixtureTurn = { + id: string + items: Array<{ type?: string; id?: string; content?: Array<{ text?: string }> }> +} + +type CheckpointOutcome = { + requestId: string + conversationRequestId?: string + displayTurnId?: string + nativeTurnId?: string + status: 'available' | 'unavailable' + checkpointId?: string + workspace?: { path: string; provenance: string } + code?: string + reason?: string +} + +async function currentFreshAgent(page: import('@playwright/test').Page): Promise<{ + tabId: string + paneId: string + content: any +}> { + return page.evaluate(() => { + const state = window.__FRESHELL_TEST_HARNESS__!.getState() + const tabId = state.tabs.activeTabId + if (!tabId) throw new Error('There is no active tab') + const root = state.panes.layouts[tabId] + const find = (node: any, found: any[] = []): any[] => { + if (!node) return found + if (node.type === 'leaf' && node.content?.kind === 'fresh-agent') found.push(node) + if (node.type === 'split') for (const child of node.children ?? []) find(child, found) + return found + } + const candidates = find(root) + if (candidates.length !== 1) throw new Error(`Expected one fresh-agent pane; found ${candidates.length}`) + const leaf = candidates[0] + return { tabId, paneId: leaf.id, content: leaf.content } + }) +} + +async function targetRead(page: import('@playwright/test').Page, selected: string): Promise<string | undefined> { + const response = await page.request.get(`${paired!.targetUrl}/api/files/read?path=${encodeURIComponent(selected)}`, + { headers, timeout: 30_000 }) + if (response.status() === 404) return undefined + expect(response.ok(), `Reading synthetic target file failed with HTTP ${response.status()}`).toBe(true) + const body = await response.json() as { content?: unknown } + expect(typeof body.content).toBe('string') + return body.content as string +} + +async function targetWrite(page: import('@playwright/test').Page, selected: string, content: string): Promise<void> { + const response = await page.request.post(`${paired!.targetUrl}/api/files/write`, { + headers, + timeout: 30_000, + data: { path: selected, content }, + }) + expect(response.ok(), `Writing synthetic target file failed with HTTP ${response.status()}`).toBe(true) +} + +async function targetJson<T>(page: import('@playwright/test').Page, selected: string): Promise<T | undefined> { + const content = await targetRead(page, selected) + return content === undefined ? undefined : JSON.parse(content) as T +} + +async function verifyTarget(page: import('@playwright/test').Page): Promise<void> { + const response = await page.request.get(`${paired!.targetUrl}/api/server-info`, { headers, timeout: 30_000 }) + expect(response.ok()).toBe(true) + const info = await response.json() as { runtime?: unknown; commit?: unknown; buildDirty?: unknown } + expect(info).toMatchObject({ runtime: 'rust', commit: fixture.candidateSha, buildDirty: false }) +} + +async function launchCodex(page: import('@playwright/test').Page, workspace: string) { + await page.goto(`${paired!.targetUrl}/?token=${encodeURIComponent(paired!.targetToken)}&e2e=1`, { timeout: 60_000 }) + const harness = new TestHarness(page) + await harness.waitForHarness(60_000) + await harness.waitForConnection(120_000) + await verifyTarget(page) + + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshcodex$/i }).click({ force: true }) + const directoryInput = page.getByRole('combobox', { name: 'Starting directory for Freshcodex' }) + await expect(directoryInput).toBeVisible({ timeout: 15_000 }) + await directoryInput.fill(workspace) + await directoryInput.press('Enter') + + const paneRoot = page.locator('[data-context="fresh-agent"]').last() + await expect(paneRoot).toBeVisible({ timeout: 30_000 }) + await expect.poll(async () => (await currentFreshAgent(page)).content.initialCwd, { timeout: 30_000 }).toBe(workspace) + await expect.poll(async () => (await currentFreshAgent(page)).content.status, { timeout: 120_000 }).toBe('idle') + await expect.poll(async () => { + const content = (await currentFreshAgent(page)).content + return content.sessionRef?.sessionId ?? content.sessionId + }, { timeout: 120_000 }).toBeTruthy() + return { paneRoot, harness, ...(await currentFreshAgent(page)) } +} + +async function sendThroughComposer( + page: import('@playwright/test').Page, + paneRoot: import('@playwright/test').Locator, + text: string, + expectedWorkspace: { path: string; provenance: 'directory-picker' } | undefined, +) { + await expect.poll(async () => (await currentFreshAgent(page)).content.status, { timeout: 60_000 }).toBe('idle') + const baseline = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.().length ?? 0) + await paneRoot.getByRole('textbox', { name: 'Chat message input' }).fill(text) + await paneRoot.getByRole('button', { name: 'Send' }).click() + + let sent: any + await expect.poll(async () => { + const frames = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() ?? []) as any[] + sent = frames.slice(baseline).find(frame => frame.type === 'freshAgent.send' && frame.text === text) + return sent + }, { timeout: 30_000 }).toBeTruthy() + expect(sent.requestId).toEqual(expect.any(String)) + expect(sent.checkpointIntent).toMatchObject({ logicalAttemptId: expect.any(String) }) + if (expectedWorkspace) { + expect(sent.checkpointIntent.workspace).toEqual(expectedWorkspace) + } else { + expect(sent.checkpointIntent).not.toHaveProperty('workspace') + } + + let accepted: any + await expect.poll(async () => { + const frames = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getReceivedWsMessages?.() ?? []) as any[] + accepted = frames.find(frame => frame.type === 'freshAgent.send.accepted' && frame.requestId === sent.requestId) + return accepted + }, { timeout: 120_000 }).toBeTruthy() + expect(accepted.submittedTurnId).toEqual(expect.any(String)) + + let outcome: CheckpointOutcome | undefined + await expect.poll(async () => { + const content = (await currentFreshAgent(page)).content + outcome = content.checkpointOutcomes?.find((entry: CheckpointOutcome) => entry.requestId === sent.requestId) + // The send barrier's first outcome is deliberately pre-forward and may + // not yet know Codex's persisted user row. Require the later updated + // native-linked projection before treating the per-turn result as ready. + return typeof outcome?.nativeTurnId === 'string' + && outcome.displayTurnId === accepted.submittedTurnId + }, { timeout: 120_000 }).toBe(true) + expect(outcome!.requestId).toBe(sent.requestId) + expect(outcome!.conversationRequestId).toBe((await currentFreshAgent(page)).content.createRequestId) + expect(outcome!.displayTurnId).toBe(accepted.submittedTurnId) + expect(outcome!.nativeTurnId).toEqual(expect.any(String)) + expect(JSON.stringify((await currentFreshAgent(page)).content.checkpointOutcomes)).not.toContain(text) + + const row = paneRoot.locator('article[data-turn-role="user"]', { hasText: text }) + await expect(row).toBeVisible({ timeout: 30_000 }) + return { sent, accepted, outcome: outcome!, row } +} + +async function assertNativeUserIdentity( + page: import('@playwright/test').Page, + outcome: CheckpointOutcome, + acceptedTurnId: string, + text: string, +): Promise<FixtureTurn> { + const content = (await currentFreshAgent(page)).content + const sessionId = content.sessionRef?.sessionId ?? content.sessionId + expect(sessionId).toEqual(expect.any(String)) + let turns: FixtureTurn[] | undefined + const selected = `${fixture.syntheticHomePath}/.codex/fake-turns/${sessionId}.json` + await expect.poll(async () => { + turns = await targetJson<FixtureTurn[]>(page, selected) + return turns?.find(turn => turn.id === acceptedTurnId) + }, { timeout: 60_000 }).toBeTruthy() + const turn = turns!.find(candidate => candidate.id === acceptedTurnId)! + const user = turn.items.find(item => item.type === 'userMessage' + && item.content?.some(part => part.text === text)) + expect(user?.id).toBe(`${acceptedTurnId}:user`) + // The persisted checkpoint outcome must point at the actual native Codex + // user-message identity. The display turn ID is a separate, stable UI join. + expect(outcome.nativeTurnId).toBe(user!.id) + return turn +} + +async function requestPrivateFileState( + page: import('@playwright/test').Page, + action: 'eligible' | 'ignored', +): Promise<void> { + const requestId = randomUUID() + const ackPath = `${checkpointRoot}/control/ack-${requestId}.json` + await targetWrite(page, controlRequestPath, JSON.stringify({ action, expectedNonce: fixture.expectedNonce, requestId })) + await expect.poll(() => targetJson(page, ackPath), { timeout: 30_000 }).toMatchObject({ + requestId, + action, + restrictedUid: fixture.restrictedUid, + privateFileUid: 0, + privateFileMode: '0600', + privateFileDenied: true, + effectiveCapabilities: '0000000000000000', + }) +} + +async function flushLayoutAndReload(page: import('@playwright/test').Page): Promise<void> { + await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.dispatch({ type: 'persist/flushNow' })) + await expect.poll(() => page.evaluate(() => Object.keys(localStorage) + .filter(key => key.startsWith('freshell.layout.v3.')) + .some(key => localStorage.getItem(key)?.includes('checkpointOutcomes') ?? false))).toBe(true) + await page.reload({ timeout: 60_000 }) + const harness = new TestHarness(page) + await harness.waitForHarness(60_000) + await harness.waitForConnection(120_000) + await expect(page.locator('[data-context="fresh-agent"]')).toBeVisible({ timeout: 30_000 }) +} + +test('Git workspace sends retain per-turn success and failure, then restore the exact successful row', async ({ page }) => { + test.setTimeout(360_000) + await verifyTarget(page) + // Cloud retries reuse the owned synthetic target. Restore the mutable code + // and the private file's ignored state so every attempt starts from the + // same readable checkpoint fixture even if a prior attempt failed midway. + await targetWrite(page, `${fixture.syntheticWorkspacePath}/main.rs`, 'main before checkpoint\n') + await targetWrite(page, `${fixture.syntheticWorkspacePath}/notes.txt`, 'notes before checkpoint\n') + await requestPrivateFileState(page, 'ignored') + const { paneRoot } = await launchCodex(page, fixture.syntheticWorkspacePath) + const before = await currentFreshAgent(page) + expect(before.content.checkpointWorkspace).toEqual({ path: fixture.syntheticWorkspacePath, provenance: 'directory-picker' }) + const originalSessionId = before.content.sessionRef?.sessionId ?? before.content.sessionId + const originalModel = before.content.model + expect(originalSessionId).toBeTruthy() + + const samePrompt = `checkpoint same prompt ${fixture.expectedNonce}` + const first = await sendThroughComposer(page, paneRoot, samePrompt, + { path: fixture.syntheticWorkspacePath, provenance: 'directory-picker' }) + expect(first.outcome).toMatchObject({ status: 'available', workspace: { + path: fixture.syntheticWorkspacePath, provenance: 'directory-picker', + } }) + expect(first.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) + await expect(first.row.getByRole('status', { name: 'Code rewind available for this message' })) + .toContainText('Code rewind is available for this message.') + await expect(first.row.getByRole('button', { name: 'Rewind code to here' })).toBeEnabled() + await assertNativeUserIdentity(page, first.outcome, first.accepted.submittedTurnId, samePrompt) + + await targetWrite(page, `${fixture.syntheticWorkspacePath}/main.rs`, 'main changed after first send\n') + await targetWrite(page, `${fixture.syntheticWorkspacePath}/notes.txt`, 'notes changed after first send\n') + await requestPrivateFileState(page, 'eligible') + const second = await sendThroughComposer(page, paneRoot, samePrompt, + { path: fixture.syntheticWorkspacePath, provenance: 'directory-picker' }) + expect(second.sent.checkpointIntent.logicalAttemptId).not.toBe(first.sent.checkpointIntent.logicalAttemptId) + expect(second.accepted.requestId).not.toBe(first.accepted.requestId) + expect(second.outcome).toMatchObject({ status: 'unavailable', code: 'checkpoint_path_unreadable' }) + expect(second.outcome.reason).toContain('A selected project file could not be read') + await expect.poll(async () => paneRoot.locator('article[data-turn-role="user"]', { hasText: samePrompt }).count()) + .toBe(2) + const firstRow = paneRoot.locator('article[data-turn-role="user"]', { hasText: samePrompt }).nth(0) + const secondRow = paneRoot.locator('article[data-turn-role="user"]', { hasText: samePrompt }).nth(1) + await expect(firstRow.getByRole('status', { name: 'Code rewind available for this message' })).toBeVisible() + await expect(secondRow.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toContainText('A selected project file could not be read') + await secondRow.hover() + await expect(secondRow.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + await assertNativeUserIdentity(page, second.outcome, second.accepted.submittedTurnId, samePrompt) + await requestPrivateFileState(page, 'ignored') + + const beforeReloadOutcomes = (await currentFreshAgent(page)).content.checkpointOutcomes as CheckpointOutcome[] + expect(beforeReloadOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ requestId: first.sent.requestId, nativeTurnId: first.outcome.nativeTurnId, + status: 'available', checkpointId: first.outcome.checkpointId }), + expect.objectContaining({ requestId: second.sent.requestId, nativeTurnId: second.outcome.nativeTurnId, + status: 'unavailable', code: 'checkpoint_path_unreadable' }), + ])) + await flushLayoutAndReload(page) + const afterReload = await currentFreshAgent(page) + expect(afterReload.content.checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ requestId: first.sent.requestId, nativeTurnId: first.outcome.nativeTurnId, + status: 'available', checkpointId: first.outcome.checkpointId }), + expect.objectContaining({ requestId: second.sent.requestId, nativeTurnId: second.outcome.nativeTurnId, + status: 'unavailable', code: 'checkpoint_path_unreadable' }), + ])) + expect(afterReload.content.sessionRef?.sessionId ?? afterReload.content.sessionId).toBe(originalSessionId) + expect(afterReload.content.model).toBe(originalModel) + + const restoredPane = page.locator('[data-context="fresh-agent"]').last() + const duplicateRows = restoredPane.locator('article[data-turn-role="user"]', { hasText: samePrompt }) + await expect(duplicateRows).toHaveCount(2, { timeout: 60_000 }) + await expect(duplicateRows.nth(0).getByRole('status', { name: 'Code rewind available for this message' })).toBeVisible() + await expect(duplicateRows.nth(1).getByRole('note', { name: 'Code rewind unavailable for this message' })).toBeVisible() + await duplicateRows.nth(0).hover() + const rewind = duplicateRows.nth(0).getByRole('button', { name: 'Rewind code to here' }) + await expect(rewind).toBeEnabled() + const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) + await rewind.click() + await dialogAccepted + await expect.poll(() => targetRead(page, `${fixture.syntheticWorkspacePath}/main.rs`)).toBe('main before checkpoint\n') + await expect.poll(() => targetRead(page, `${fixture.syntheticWorkspacePath}/notes.txt`)).toBe('notes before checkpoint\n') + await expect(restoredPane.getByText(`Code rewound to before: "${samePrompt}"`)).toBeVisible({ timeout: 30_000 }) + await expect(duplicateRows).toHaveCount(2) + const durable = await targetJson<FixtureTurn[]>(page, + `${fixture.syntheticHomePath}/.codex/fake-turns/${originalSessionId}.json`) + expect(durable).toHaveLength(2) + expect(durable?.map(turn => turn.items.find(item => item.type === 'userMessage')?.id)) + .toEqual([first.outcome.nativeTurnId, second.outcome.nativeTurnId]) +}) + +test('picker-confirmed non-Git workspace captures and restores all selected files', async ({ page }) => { + test.setTimeout(300_000) + await verifyTarget(page) + // As above, each Cloud retry starts from the original non-Git file bytes. + await targetWrite(page, `${fixture.nonGitWorkspacePath}/one.txt`, 'one before checkpoint\n') + await targetWrite(page, `${fixture.nonGitWorkspacePath}/two.txt`, 'two before checkpoint\n') + const { paneRoot, content } = await launchCodex(page, fixture.nonGitWorkspacePath) + expect(content.checkpointWorkspace).toEqual({ path: fixture.nonGitWorkspacePath, provenance: 'directory-picker' }) + const originalSessionId = content.sessionRef?.sessionId ?? content.sessionId + const marker = `checkpoint non-git ${fixture.expectedNonce}` + const turn = await sendThroughComposer(page, paneRoot, marker, + { path: fixture.nonGitWorkspacePath, provenance: 'directory-picker' }) + expect(turn.outcome).toMatchObject({ status: 'available', workspace: { + path: fixture.nonGitWorkspacePath, provenance: 'directory-picker', + } }) + expect(turn.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) + await expect(turn.row.getByRole('status', { name: 'Code rewind available for this message' })).toBeVisible() + await expect(turn.row.getByRole('button', { name: 'Rewind code to here' })).toBeEnabled() + await assertNativeUserIdentity(page, turn.outcome, turn.accepted.submittedTurnId, marker) + + await targetWrite(page, `${fixture.nonGitWorkspacePath}/one.txt`, 'one changed after send\n') + await targetWrite(page, `${fixture.nonGitWorkspacePath}/two.txt`, 'two changed after send\n') + await turn.row.hover() + const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) + await turn.row.getByRole('button', { name: 'Rewind code to here' }).click() + await dialogAccepted + await expect.poll(() => targetRead(page, `${fixture.nonGitWorkspacePath}/one.txt`)).toBe('one before checkpoint\n') + await expect.poll(() => targetRead(page, `${fixture.nonGitWorkspacePath}/two.txt`)).toBe('two before checkpoint\n') + const afterRestore = await currentFreshAgent(page) + expect(afterRestore.content.sessionRef?.sessionId ?? afterRestore.content.sessionId).toBe(originalSessionId) + expect(afterRestore.content.checkpointWorkspace).toEqual({ path: fixture.nonGitWorkspacePath, provenance: 'directory-picker' }) + await expect(paneRoot.getByText(`Code rewound to before: "${marker}"`)).toBeVisible({ timeout: 30_000 }) +}) + +test('HOME confirmation is launch-only and an accepted send has no checkpoint', async ({ page }) => { + test.setTimeout(240_000) + await verifyTarget(page) + const { paneRoot, content } = await launchCodex(page, fixture.syntheticHomePath) + expect(content.initialCwd).toBe(fixture.syntheticHomePath) + expect(content.checkpointWorkspace).toBeUndefined() + const marker = `checkpoint home ${fixture.expectedNonce}` + const turn = await sendThroughComposer(page, paneRoot, marker, undefined) + expect(turn.outcome).toMatchObject({ status: 'unavailable', code: 'workspace_unavailable' }) + expect(turn.outcome.reason).toMatch(/account home|selected code checkpoint workspace/i) + expect(turn.sent.checkpointIntent).not.toHaveProperty('workspace') + await expect(turn.row.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toContainText(/account home|selected code checkpoint workspace/i) + await turn.row.hover() + await expect(turn.row.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + + const list = await page.request.get(`${paired!.targetUrl}/api/fresh-agent/checkpoints?cwd=${encodeURIComponent(fixture.syntheticHomePath)}`, + { headers, timeout: 30_000 }) + expect(list.ok()).toBe(true) + expect(await list.json()).toEqual({ checkpoints: [] }) + expect(await targetRead(page, `${fixture.syntheticHomePath}/.freshell/checkpoints/HEAD`)).toBeUndefined() +}) diff --git a/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs b/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs index 9531a8c8c..63cae7437 100644 --- a/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs +++ b/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs @@ -304,6 +304,8 @@ function makeThreadTurnsPage(params = {}) { } } +let uniqueThreadStartCounter = 0 + function successResult(method, params) { if (method === 'initialize') { return { @@ -314,7 +316,10 @@ function successResult(method, params) { } } if (method === 'thread/start') { - const threadId = behavior.threadStartThreadId || 'thread-new-1' + const threadId = behavior.threadStartThreadId + || (behavior.uniqueThreadIds === true + ? `thread-${process.pid}-${++uniqueThreadStartCounter}` + : 'thread-new-1') const rolloutPath = behavior.threadStartRolloutPath || behavior.rolloutPath const thread = makeThread(threadId, params) if (rolloutPath) thread.path = rolloutPath diff --git a/test/unit/tooling/testing/browser-selection.test.ts b/test/unit/tooling/testing/browser-selection.test.ts index e18135be9..e6e430d67 100644 --- a/test/unit/tooling/testing/browser-selection.test.ts +++ b/test/unit/tooling/testing/browser-selection.test.ts @@ -117,6 +117,28 @@ describe('browser selection non-vacuity', () => { expect(ordinaryExplicit.status).toBe(1) }, 90_000) + it('discovers checkpoint workspace cases only through their paired dedicated Cloud project', () => { + const spec = 'code-checkpoint-workspace-external.spec.ts' + const ordinary = listedProjects(cleanEnvironment(), cloudConfig) + expect(listedSpecFiles(ordinary.output).has(spec)).toBe(false) + const fixture = JSON.stringify({ candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', + syntheticWorkspacePath: '/tmp/selected-checkpoint-fixture/project', + syntheticHomePath: '/tmp/selected-checkpoint-fixture/home', + nonGitWorkspacePath: '/tmp/selected-checkpoint-fixture/non-git', restrictedUid: 65534, + expectedNonce: 'selection-checkpoint-nonce' }) + const paired = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'code-checkpoint', + FRESHELL_CODE_CHECKPOINT_FIXTURE: fixture, FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', + FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token' }) + const selected = listedProjects(paired, cloudConfig, ['--project=chromium-external-target', spec]) + expect(selected.labels).toEqual(['chromium-external-target']) + expect(selected.tests).toBe(3) + expect(selected.files).toBe(1) + expect([...listedSpecFiles(selected.output)]).toEqual([spec]) + const ordinaryExplicit = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, + '--list', '--project=chromium', spec], { cwd: projectRoot, env: paired, encoding: 'utf8' }) + expect(ordinaryExplicit.status).toBe(1) + }, 90_000) + it.each(['missing', 'mismatched'] as const)('refuses external discovery with a %s fixture pair', pairing => { const env = cleanEnvironment({ FRESHELL_E2E_EXTERNAL_FIXTURE_KIND: 'managed-environment', FRESHELL_E2E_TARGET_URL: 'https://fixture.invalid', FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token', diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts new file mode 100644 index 000000000..016db5669 --- /dev/null +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -0,0 +1,30 @@ +import { describe, expect, it } from 'vitest' +import { + resolveManagedProviderEnvironmentTargetMode, + runManagedProviderEnvironmentTarget, +} from '../../../../scripts/testing/managed-provider-environment-target.js' + +describe('managed provider environment target mode', () => { + it('keeps the existing managed target as the default and selects checkpoint only explicitly', () => { + expect(resolveManagedProviderEnvironmentTargetMode([])).toBe('managed-environment') + expect(resolveManagedProviderEnvironmentTargetMode(['--checkpoint'])).toBe('code-checkpoint') + }) + + it.each([['--managed'], ['--checkpoint', '--checkpoint'], ['--checkpoint', 'other'], ['--unknown']])( + 'rejects unsupported or ambiguous target arguments %j', (...args) => { + expect(() => resolveManagedProviderEnvironmentTargetMode(args)).toThrow(/usage/i) + }, + ) + + it('fails closed on a local backend instead of falling back from checkpoint mode', async () => { + const priorBackend = process.env.FRESHELL_E2E_BACKEND + process.env.FRESHELL_E2E_BACKEND = 'local' + try { + await expect(runManagedProviderEnvironmentTarget(['--checkpoint'])) + .rejects.toThrow('Checkpoint acceptance requires configured Cloud') + } finally { + if (priorBackend === undefined) delete process.env.FRESHELL_E2E_BACKEND + else process.env.FRESHELL_E2E_BACKEND = priorBackend + } + }) +}) From 52e263d608cb9caf35961b3c96815ed88b85df72 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 03:31:24 -0700 Subject: [PATCH 307/493] Add provider-native checkpoint turn identities --- Cargo.lock | 7 + crates/freshell-claude-sidecar/index.mjs | 55 +++- crates/freshell-freshagent/Cargo.toml | 2 +- crates/freshell-freshagent/src/claude.rs | 41 +++ crates/freshell-freshagent/src/codex.rs | 65 +++- crates/freshell-freshagent/src/opencode_ws.rs | 256 ++++++++++++++- .../src/session_handoff/tests.rs | 2 + crates/freshell-opencode/Cargo.toml | 8 + crates/freshell-opencode/src/lib.rs | 3 +- crates/freshell-opencode/src/serve.rs | 81 ++++- .../tests/checkpoint_message_id.rs | 198 ++++++++++++ .../tests/checkpoint_message_id_contract.rs | 276 ++++++++++++++++ .../testing/opencode-checkpoint-contract.ts | 224 +++++++++++++ shared/fresh-agent-contract.ts | 3 + .../claude-checkpoint-uuid-replay.mjs | 295 ++++++++++++++++++ .../claude-checkpoint-uuid-replay.test.ts | 70 +++++ .../fixtures/fake-checkpoint-sdk.mjs | 30 ++ .../user-message-replay.test.ts | 111 +++++++ test/unit/shared/fresh-agent-turns.test.ts | 21 ++ 19 files changed, 1727 insertions(+), 21 deletions(-) create mode 100644 crates/freshell-opencode/tests/checkpoint_message_id.rs create mode 100644 crates/freshell-opencode/tests/checkpoint_message_id_contract.rs create mode 100644 scripts/testing/opencode-checkpoint-contract.ts create mode 100644 test/integration/providers/claude-checkpoint-uuid-replay.mjs create mode 100644 test/integration/providers/claude-checkpoint-uuid-replay.test.ts create mode 100644 test/unit/claude-sidecar/fixtures/fake-checkpoint-sdk.mjs create mode 100644 test/unit/claude-sidecar/user-message-replay.test.ts diff --git a/Cargo.lock b/Cargo.lock index 05138c927..8a6c50c14 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4518,6 +4518,12 @@ dependencies = [ "digest", ] +[[package]] +name = "sha1_smol" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbfa15b3dddfee50a0fff136974b3e1bde555604ba463834a7eb7deb6417705d" + [[package]] name = "sha2" version = "0.10.9" @@ -5756,6 +5762,7 @@ dependencies = [ "getrandom 0.4.3", "js-sys", "serde_core", + "sha1_smol", "wasm-bindgen", ] diff --git a/crates/freshell-claude-sidecar/index.mjs b/crates/freshell-claude-sidecar/index.mjs index fe9665a29..ef6db93d8 100644 --- a/crates/freshell-claude-sidecar/index.mjs +++ b/crates/freshell-claude-sidecar/index.mjs @@ -240,6 +240,27 @@ function handleSdkMessage(sessionId, msg) { } switch (msg.type) { + case 'user': { + // The SDK replays actual persisted user messages when this query opts in. + // Only the unique UUID that this sidecar supplied for a checkpointed send + // can resolve its attempt; assistant/tool/result frames and unrelated + // user rows are never identity evidence. + if (msg.isReplay === true && msg.message?.role === 'user' && isUuid(msg.uuid)) { + const attempt = st.pendingCheckpointAttempts.get(msg.uuid) + if (attempt) { + st.pendingCheckpointAttempts.delete(msg.uuid) + emit({ + type: 'sdk.checkpointTurnIdentity', + sessionId, + logicalAttemptId: attempt.logicalAttemptId, + requestId: attempt.requestId, + nativeTurnId: msg.uuid, + displayTurnId: msg.uuid, + }) + } + } + break + } case 'system': { if (msg.subtype === 'init') { st.cliSessionId = msg.session_id @@ -386,6 +407,7 @@ function handleCreate(req) { // finally can mint the edge for a result-less turn end. turnCompleteGate: createTurnCompleteGate(), pendingResults: 0, + pendingCheckpointAttempts: new Map(), } const { iterable, handle } = createInputStream((isCompact) => { if (isCompact) state.handedCompactLikely = true @@ -423,6 +445,10 @@ function handleCreate(req) { mcpServers: hostedMcpServers(req.hostedMcp), pathToClaudeCodeExecutable: process.env.CLAUDE_CMD || undefined, includePartialMessages: true, + // This is the SDK's native persisted-user witness. Without the replay + // flag a supplied SDKUserMessage UUID is not surfaced back to the + // sidecar, so checkpoint identity must remain unbound. + extraArgs: { 'replay-user-messages': null }, abortController: abort, env: createClaudeSdkCleanEnv(process.env), settingSources: ['user', 'project', 'local'], @@ -469,15 +495,29 @@ function handleSend(req) { // the quiesce handler can drain never-handed compacts from the input queue // and so a same-tick handoff arms the BUSY-forcing handed flag. const isCompact = /^\s*\/compact(\s|$)/.test(String(req.text ?? '')) - st.inputStream.push( - { + const attempt = req.checkpointAttempt + const checkpointAttempt = attempt && isUuid(attempt.uuid) + && typeof attempt.logicalAttemptId === 'string' && attempt.logicalAttemptId.length > 0 + && typeof attempt.requestId === 'string' && attempt.requestId.length > 0 + ? { uuid: attempt.uuid, logicalAttemptId: attempt.logicalAttemptId, requestId: attempt.requestId } + : undefined + const userMessage = { type: 'user', message: { role: 'user', content: userMessageContent(req.text, req.images) }, parent_tool_use_id: null, session_id: st.cliSessionId || 'default', - }, - isCompact, - ) + ...(checkpointAttempt ? { uuid: checkpointAttempt.uuid } : {}), + } + if (checkpointAttempt) { + // Bound stale/missing-replay entries. Eviction means unavailable identity, + // never an approximate join by request order or prompt text. + st.pendingCheckpointAttempts.delete(checkpointAttempt.uuid) + st.pendingCheckpointAttempts.set(checkpointAttempt.uuid, checkpointAttempt) + while (st.pendingCheckpointAttempts.size > 128) { + st.pendingCheckpointAttempts.delete(st.pendingCheckpointAttempts.keys().next().value) + } + } + st.inputStream.push(userMessage, isCompact) // Unified attention bookkeeping: this send is now ACCEPTED (the input-stream // push succeeded) and awaits exactly one terminal frame — a COUNTER, not a // boolean: two queued sends leave it at 2 and the first result must drop it @@ -488,6 +528,11 @@ function handleSend(req) { st.pendingResults += 1 } +function isUuid(value) { + return typeof value === 'string' + && /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(value) +} + async function handleConfigure(req) { const st = sessions.get(req.sessionId) if (!st) { diff --git a/crates/freshell-freshagent/Cargo.toml b/crates/freshell-freshagent/Cargo.toml index 8a7fe29fa..05fc4813f 100644 --- a/crates/freshell-freshagent/Cargo.toml +++ b/crates/freshell-freshagent/Cargo.toml @@ -88,7 +88,7 @@ futures-util = "0.3" # app-server sidecar spawn (`process`), and its stdio drain (`io-util`). tokio = { version = "1", features = ["rt", "macros", "sync", "time", "process", "io-util"] } # paneId / tabId / placeholder-suffix / submittedTurnId + claude-sidecar ownership-id minting. -uuid = { version = "1", features = ["v4"] } +uuid = { version = "1", features = ["v4", "v5"] } # DIAG-01 lifecycle events (session created/send accepted/turn complete/crash # detected/crash recovery/sidecar spawn+reap): flows to the global `tracing` # subscriber `freshell-server` installs (`crates/freshell-server/src/logging.rs`) diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index 8e0452708..82ce23230 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -78,6 +78,19 @@ use crate::{FreshAgentCreateDedup, FreshAgentCreateOutcome, SharedPaneIdentitySi /// The runtime provider (`AGENT_SESSION_TYPES.claude.provider`). const PROVIDER: &str = "claude"; +/// Build the sidecar-only correlation record for one checkpointed Claude +/// send. The UUID is supplied on SDKUserMessage and later accepted only when +/// the SDK replays the persisted user row carrying that same UUID. +fn checkpoint_attempt_for_send(msg: &FreshAgentSend) -> Option<Value> { + let intent = msg.checkpoint_intent.as_ref()?; + let request_id = msg.request_id.as_deref().filter(|id| !id.is_empty())?; + Some(json!({ + "uuid": uuid::Uuid::new_v4().to_string(), + "logicalAttemptId": intent.logical_attempt_id, + "requestId": request_id, + })) +} + /// b8ke ext r22 F1: the pane-scoped PROVISIONAL identity for a fresh /// create's pre-spawn claim — the createRequestId the caller already /// carries, namespaced so it can never collide with a provider-minted @@ -4448,8 +4461,12 @@ impl FreshClaudeState { }; // Address the sidecar by ITS id for this session (== the map key for created // sessions; differs for resumed-on-attach sessions, Task 6). + let checkpoint_attempt = checkpoint_attempt_for_send(&msg); let mut send_req = json!({ "type": "send", "sessionId": session.sidecar_session_id, "text": msg.text }); + if let Some(attempt) = checkpoint_attempt { + send_req["checkpointAttempt"] = attempt; + } if let Some(images) = msg.images { send_req["images"] = json!(images); } @@ -9237,6 +9254,7 @@ fn normalize_sdk_type(sdk_type: &str) -> Option<&'static str> { "sdk.session.snapshot" => "freshAgent.session.snapshot", "sdk.session.changed" => "freshAgent.session.changed", "sdk.session.init" => "freshAgent.session.init", + "sdk.checkpointTurnIdentity" => "checkpointTurnIdentity", "sdk.session.metadata" => "freshAgent.session.metadata", "sdk.assistant" => "freshAgent.assistant", "sdk.stream" => "freshAgent.stream", @@ -11509,9 +11527,32 @@ rl.on('line', (line) => { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, } } + #[test] + fn checkpoint_turn_identity_claude_send_carries_valid_uuid_and_attempt_correlation() { + let mut msg = send_msg("claude-placeholder", "same prompt"); + msg.request_id = Some("request-7".to_string()); + msg.checkpoint_intent = Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-7".to_string(), + workspace: None, + }); + + let attempt = checkpoint_attempt_for_send(&msg).expect("checkpoint attempt"); + let uuid = uuid::Uuid::parse_str(attempt["uuid"].as_str().expect("UUID string")) + .expect("valid UUID"); + assert_eq!( + uuid.get_version_num(), + 4, + "the SDK input uses a fresh v4 UUID" + ); + assert_eq!(attempt["requestId"], "request-7"); + assert_eq!(attempt["logicalAttemptId"], "attempt-7"); + assert!(checkpoint_attempt_for_send(&send_msg("legacy", "prompt")).is_none()); + } + /// kata z7j7 (claude scope honesty): a settings-bearing send whose payload /// includes `sandbox` — a knob claude has NO wire concept for (the configure /// frame carries only model/effort/permissionMode/cwd) — must surface the diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index aa4e481a5..520e48523 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -9555,6 +9555,9 @@ fn build_codex_turn_json(raw_turn: &Value, ordinal: usize) -> Result<Vec<Value>, } let row_count = rows.len(); + let unique_user_row = (rows.iter().filter(|row| row.role == "user").count() == 1) + .then(|| rows.iter().position(|row| row.role == "user")) + .flatten(); let turns = rows .into_iter() .enumerate() @@ -9567,7 +9570,7 @@ fn build_codex_turn_json(raw_turn: &Value, ordinal: usize) -> Result<Vec<Value>, format!("{turn_id}:row-{row_index}") }; let (summary, summary_kind) = summarize_codex_items(&row.items); - json!({ + let mut turn = json!({ "id": row_turn_id, "turnId": row_turn_id, "ordinal": ordinal, @@ -9576,7 +9579,11 @@ fn build_codex_turn_json(raw_turn: &Value, ordinal: usize) -> Result<Vec<Value>, "summary": summary, "summaryKind": summary_kind, "items": row.items, - }) + }); + if unique_user_row == Some(row_index) && !turn_id.is_empty() { + turn["nativeTurnId"] = json!(turn_id); + } + turn }) .collect(); @@ -12322,6 +12329,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, } } @@ -15287,6 +15295,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }) @@ -15432,6 +15441,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }) @@ -15551,6 +15561,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }) @@ -21090,6 +21101,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }) @@ -21442,6 +21454,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }) @@ -21570,6 +21583,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }) @@ -24185,6 +24199,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; @@ -24268,6 +24283,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; @@ -24363,6 +24379,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; @@ -24439,6 +24456,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; @@ -24737,6 +24755,7 @@ pub(crate) mod tests { settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; }); @@ -26626,6 +26645,45 @@ pub(crate) mod tests { assert_eq!(turns[1]["turnId"], json!("turn-multi-id:row-1")); } + #[test] + fn checkpoint_turn_identity_codex_binds_native_id_only_to_one_actual_user_row() { + let raw_turn = json!({ + "id": "turn-native-1", + "status": "completed", + "items": [ + { "id": "user-item-1", "type": "userMessage", "content": [{ "type": "text", "text": "same prompt" }] }, + { "id": "assistant-item-1", "type": "agentMessage", "text": "answer" } + ] + }); + + let turns = build_codex_turn_json(&raw_turn, 0).expect("turn builds"); + let user_rows: Vec<_> = turns.iter().filter(|turn| turn["role"] == "user").collect(); + assert_eq!(user_rows.len(), 1); + assert_eq!(user_rows[0]["id"], "turn-native-1:row-0"); + assert_eq!(user_rows[0]["turnId"], "turn-native-1:row-0"); + assert_eq!(user_rows[0]["nativeTurnId"], "turn-native-1"); + assert!(turns + .iter() + .filter(|turn| turn["role"] != "user") + .all(|turn| turn.get("nativeTurnId").is_none())); + + let ambiguous = json!({ + "id": "turn-native-ambiguous", + "status": "completed", + "items": [ + { "id": "u1", "type": "userMessage", "content": [{ "type": "text", "text": "same prompt" }] }, + { "id": "a1", "type": "agentMessage", "text": "answer" }, + { "id": "u2", "type": "userMessage", "content": [{ "type": "text", "text": "same prompt" }] } + ] + }); + let turns = build_codex_turn_json(&ambiguous, 0).expect("ambiguous turn still builds"); + assert_eq!( + turns.iter().filter(|turn| turn["role"] == "user").count(), + 2 + ); + assert!(turns.iter().all(|turn| turn.get("nativeTurnId").is_none())); + } + #[test] fn build_codex_turn_json_empty_response_synthetic_row_also_carries_assistant_role() { // A completed turn with ONLY user-role items and no assistant output gets a synthetic @@ -27073,6 +27131,7 @@ pub(crate) mod tests { }), observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }; // First send carries the initial pair. @@ -27314,6 +27373,7 @@ pub(crate) mod tests { settings: None, observed_epoch: Some(before.epoch), observed_generation: Some(before.generation), + checkpoint_intent: None, }) .await; let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(15); @@ -28300,6 +28360,7 @@ pub(crate) mod tests { settings: None, observed_epoch: Some(7), observed_generation: None, + checkpoint_intent: None, }) .await; let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(15); diff --git a/crates/freshell-freshagent/src/opencode_ws.rs b/crates/freshell-freshagent/src/opencode_ws.rs index 9e8d55231..c1676fab8 100644 --- a/crates/freshell-freshagent/src/opencode_ws.rs +++ b/crates/freshell-freshagent/src/opencode_ws.rs @@ -88,6 +88,39 @@ const PROVIDER: &str = "opencode"; /// `DEFAULT_TURN_TIMEOUT_MS` (`adapter.ts:35`). const DEFAULT_TURN_TIMEOUT: Duration = Duration::from_millis(600_000); +#[derive(Clone, Debug)] +struct CheckpointTurnIdentity { + logical_attempt_id: String, + request_id: String, + native_message_id: String, +} + +/// Derive a stable OpenCode-native message ID only for a modern attempt with a +/// valid v4 logical identity and request correlation. The same ID is regenerated +/// when a send waits behind compact, so queue/drain preserves the exact identity. +fn checkpoint_turn_identity(send: &FreshAgentSend) -> Option<CheckpointTurnIdentity> { + let intent = send.checkpoint_intent.as_ref()?; + let logical_attempt_id = intent.logical_attempt_id.trim(); + if logical_attempt_id.is_empty() { + return None; + } + let request_id = send + .request_id + .as_deref() + .filter(|request_id| !request_id.trim().is_empty())?; + Some(CheckpointTurnIdentity { + logical_attempt_id: logical_attempt_id.to_string(), + request_id: request_id.to_string(), + // The client currently uses a 21-character URL-safe nanoid for logical + // attempts. Hash that stable value into OpenCode's native message-ID shape; + // do not constrain the protocol correlation to UUID syntax. + native_message_id: format!( + "msg_{}", + uuid::Uuid::new_v5(&uuid::Uuid::NAMESPACE_OID, logical_attempt_id.as_bytes()).simple() + ), + }) +} + /// Shared, cheaply-cloneable freshopencode WS state (mergeable into `WsState`). #[derive(Clone)] pub struct FreshOpencodeState { @@ -2092,6 +2125,7 @@ impl FreshOpencodeState { already_accepted: bool, ) { let request_id = msg.request_id.clone(); + let checkpoint_identity = checkpoint_turn_identity(&msg); // materializeOrSend:334-335 -- a fresh turn starts un-aborted and un-errored; // `handle_interrupt` flips `turn_aborted` while we are parked on idle, and // the serve-stream bridge flips `turn_errored` if the turn reports an error. @@ -2635,16 +2669,36 @@ impl FreshOpencodeState { // settled outcome below (or a confirmed abort) may disarm it. let mut op_guard = op_guard; // Option<AttachGuard>, owned here let dispatch_witness = prompt_dispatched.clone(); - let mut turn_fut = Box::pin(manager.run_turn( - &real_id, - &text, - model.as_deref(), - effort.as_deref(), - DEFAULT_TURN_TIMEOUT, - route.clone(), - Some(daemon_turn_accepted.clone()), - Some(dispatch_witness), - )); + let mut turn_fut = Box::pin(async { + if let Some(identity) = checkpoint_identity.as_ref() { + manager + .run_turn_with_message_id( + &real_id, + &text, + model.as_deref(), + effort.as_deref(), + Some(&identity.native_message_id), + DEFAULT_TURN_TIMEOUT, + route.clone(), + Some(daemon_turn_accepted.clone()), + Some(dispatch_witness), + ) + .await + } else { + manager + .run_turn( + &real_id, + &text, + model.as_deref(), + effort.as_deref(), + DEFAULT_TURN_TIMEOUT, + route.clone(), + Some(daemon_turn_accepted.clone()), + Some(dispatch_witness), + ) + .await + } + }); // The compact drive's own guard-release discipline: race the // fresh dispatch witness against the drive. The guard releases // at the DISPATCH boundary — the prompt POST's request leg — @@ -2686,6 +2740,45 @@ impl FreshOpencodeState { Err(_) => {} } + // A prompt ID is only a candidate until provider history shows that exact + // ID on a role=user row. Observe after dispatch; never join by text/order. + if prompt_dispatched.load(Ordering::SeqCst) { + if let Some(identity) = checkpoint_identity.as_ref() { + match manager.list_messages(&real_id, &route).await { + Ok(messages) => { + if let Some(native_id) = freshell_opencode::observed_user_message_id( + &messages, + &identity.native_message_id, + ) { + this.broadcast(&event_frame( + &real_id, + json!({ + "type": "checkpointTurnIdentity", + "logicalAttemptId": identity.logical_attempt_id, + "requestId": identity.request_id, + "sessionId": real_id, + "nativeTurnId": native_id, + "displayTurnId": identity.native_message_id, + }), + )); + } else { + tracing::warn!(target: "freshell_freshagent::opencode", + session_id = %real_id, + request_id = %identity.request_id, + logical_attempt_id = %identity.logical_attempt_id, + "checkpoint_turn_identity_unobserved"); + } + } + Err(error) => tracing::warn!(target: "freshell_freshagent::opencode", + session_id = %real_id, + request_id = %identity.request_id, + logical_attempt_id = %identity.logical_attempt_id, + error = %error, + "checkpoint_turn_identity_history_read_failed"), + } + } + } + settle_turn_outcome( &fresh_agent, &real_id, @@ -9663,9 +9756,33 @@ mod tests { settings: None, observed_epoch, observed_generation, + checkpoint_intent: None, } } + #[test] + fn checkpoint_turn_identity_opencode_message_id_is_stable_for_nanoid_attempt() { + let logical_attempt_id = "V1StGXR8_Z5jdHi6B-myT"; + let mut send = send_msg("ses_identity", "first request"); + send.request_id = Some("request-first".into()); + send.checkpoint_intent = Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: logical_attempt_id.into(), + workspace: None, + }); + let first = checkpoint_turn_identity(&send).expect("modern send has identity"); + send.request_id = Some("request-retry".into()); + let retry = checkpoint_turn_identity(&send).expect("retry preserves logical identity"); + + assert_eq!(first.logical_attempt_id, logical_attempt_id); + assert_eq!(first.request_id, "request-first"); + assert_eq!(retry.request_id, "request-retry"); + assert_eq!(first.native_message_id, retry.native_message_id); + assert_eq!( + first.native_message_id, + "msg_55c25d96e4445f67b35ea116686e169c" + ); + } + // ── tests ──────────────────────────────────────────────────────────── #[tokio::test] @@ -16696,6 +16813,7 @@ mod tests { next_session: AtomicUsize, requests: StdMutex<Vec<RecordedRequest>>, busy_budget: StdMutex<std::collections::HashMap<String, usize>>, + persisted_message_history: StdMutex<std::collections::HashMap<String, Vec<Value>>>, summarize_outcome: SummarizeOutcome, config_body: Vec<u8>, bus_probe: StdMutex<tokio::sync::broadcast::Receiver<String>>, @@ -16774,6 +16892,7 @@ mod tests { next_session: AtomicUsize::new(0), requests: StdMutex::new(Vec::new()), busy_budget: StdMutex::new(std::collections::HashMap::new()), + persisted_message_history: StdMutex::new(std::collections::HashMap::new()), summarize_outcome, config_body, bus_probe: StdMutex::new(bus_probe), @@ -16948,7 +17067,7 @@ mod tests { .push(RecordedRequest { method: method.clone(), url: req.url.clone(), - body: body_value, + body: body_value.clone(), }); if req.url.contains("/global/health") { @@ -17110,6 +17229,19 @@ mod tests { .and_then(|rest| rest.split(['/', '?']).next()) .unwrap_or("") .to_string(); + if let Some(message_id) = body_value + .as_ref() + .and_then(|body| body.get("messageID")) + .and_then(Value::as_str) + { + self.persisted_message_history + .lock() + .expect("persisted message history mutex") + .insert( + id.clone(), + vec![json!({"info": {"id": message_id, "role": "user"}})], + ); + } self.busy_budget .lock() .expect("busy budget mutex") @@ -17122,6 +17254,28 @@ mod tests { Ok(ServeHttpResponse::new(200, b"{}".to_vec())) }); } + if method == "GET" && req.url.contains("/message") { + let id = req + .url + .split("/session/") + .nth(1) + .and_then(|rest| rest.split(['/', '?']).next()) + .unwrap_or("") + .to_string(); + let messages = self + .persisted_message_history + .lock() + .expect("persisted message history mutex") + .get(&id) + .cloned() + .unwrap_or_default(); + return Box::pin(async move { + Ok(ServeHttpResponse::new( + 200, + serde_json::to_vec(&messages).expect("history encodes as JSON"), + )) + }); + } if method == "GET" && req.url.contains("/session/status") { { let mut failures = self @@ -21934,6 +22088,7 @@ mod tests { }), observed_epoch: None, observed_generation: None, + checkpoint_intent: None, } } @@ -24896,6 +25051,85 @@ mod tests { ); } + #[tokio::test] + async fn checkpoint_turn_identity_opencode_checkpoint_message_id_survives_compact_queue() { + let summarize_gate = Arc::new(tokio::sync::Notify::new()); + let (st, http, mut rx) = compact_state_gated( + r#"{"model":null}"#, + SummarizeOutcome::OkAnswered, + Some(summarize_gate.clone()), + None, + ) + .await; + insert_compact_session(&st, "ses_checkpoint_queue", Some("prov/model")).await; + tokio::time::timeout( + std::time::Duration::from_secs(2), + st.handle_compact(compact_msg("ses_checkpoint_queue")), + ) + .await + .expect("compact registers"); + await_summarize_posted(&http).await; + + let logical_attempt_id = "V1StGXR8_Z5jdHi6B-myT"; + let expected_message_id = "msg_55c25d96e4445f67b35ea116686e169c"; + let mut send = send_msg("ses_checkpoint_queue", "queued checkpoint send"); + send.checkpoint_intent = Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: logical_attempt_id.into(), + workspace: None, + }); + tokio::time::timeout(std::time::Duration::from_secs(2), st.handle_send(send)) + .await + .expect("checkpoint send queues behind compact"); + let accepted_frames = frames_until(&mut rx, |frame| { + frame["type"] == "freshAgent.send.accepted" + && frame["requestId"] == "req-queued checkpoint send" + }) + .await; + let accepted = accepted_frames + .iter() + .find(|frame| frame["type"] == "freshAgent.send.accepted") + .expect("send acceptance is present"); + assert!(accepted["submittedTurnId"].is_null()); + assert!( + drain_frames(&mut rx) + .iter() + .all(|frame| { frame["event"]["type"] != "checkpointTurnIdentity" }), + "queue acknowledgement remains pending until the provider persists the user row" + ); + + summarize_gate.notify_waiters(); + await_prompt_posted(&http, "queued checkpoint send").await; + let prompt = http + .recorded() + .into_iter() + .find(|request| { + request.url.contains("prompt_async") + && r_body_contains(request, "queued checkpoint send") + }) + .expect("queued prompt drains through the provider transport"); + assert_eq!(prompt.body.unwrap()["messageID"], expected_message_id); + + let receipt_frames = frames_until(&mut rx, |frame| { + frame["type"] == "freshAgent.event" + && frame["event"]["type"] == "checkpointTurnIdentity" + }) + .await; + let receipt = receipt_frames + .iter() + .find(|frame| { + frame["type"] == "freshAgent.event" + && frame["event"]["type"] == "checkpointTurnIdentity" + }) + .expect("checkpoint identity receipt is present"); + assert_eq!(receipt["sessionId"], "ses_checkpoint_queue"); + assert_eq!(receipt["event"]["logicalAttemptId"], logical_attempt_id); + assert_eq!(receipt["event"]["requestId"], "req-queued checkpoint send"); + assert_eq!(receipt["event"]["nativeTurnId"], expected_message_id); + assert_eq!(receipt["event"]["displayTurnId"], expected_message_id); + assert!(receipt["event"].get("text").is_none()); + st.fresh_agent.ensure_manager().await.shutdown().await; + } + // ── merged from the send-during-compact-queue branch ── /// MUST #4 (the client contract): a fresh send arriving while older /// entries are still queued appends — never drives ahead of them. diff --git a/crates/freshell-freshagent/src/session_handoff/tests.rs b/crates/freshell-freshagent/src/session_handoff/tests.rs index 06667206b..28850b3b1 100644 --- a/crates/freshell-freshagent/src/session_handoff/tests.rs +++ b/crates/freshell-freshagent/src/session_handoff/tests.rs @@ -7528,6 +7528,7 @@ async fn opencode_handoff_stop_aborts_the_active_turn_through_the_manager_before settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; env.await_audit_row(Duration::from_secs(20), |r| { @@ -7669,6 +7670,7 @@ async fn opencode_handoff_stop_aborts_a_daemon_turn_left_running_by_a_finished_l settings: None, observed_epoch: None, observed_generation: None, + checkpoint_intent: None, }) .await; env.await_audit_row(Duration::from_secs(20), |r| { diff --git a/crates/freshell-opencode/Cargo.toml b/crates/freshell-opencode/Cargo.toml index 9d2edaa18..093c81519 100644 --- a/crates/freshell-opencode/Cargo.toml +++ b/crates/freshell-opencode/Cargo.toml @@ -30,6 +30,9 @@ publish.workspace = true # live in the next step). Default-off keeps the shared workspace build lean. default = [] real-transport = ["dep:reqwest", "dep:libc", "tokio/process", "tokio/io-util"] +# The opt-in contract requires a separately provisioned pinned OpenCode binary; it +# stays out of default workspace/pre-push behavior and reuses production transport. +real-provider-contract-tests = ["real-transport"] [dependencies] freshell-platform = { path = "../freshell-platform" } @@ -67,3 +70,8 @@ tokio = { version = "1", features = ["test-util"] } # F3 test facility: a capturing `Layer` + `tracing::subscriber::set_default` so the # malformed-inline-config warning is assertable (the freshell-codex/DIAG-01 idiom). tracing-subscriber = "0.3" + +[[test]] +name = "checkpoint_message_id_contract" +path = "tests/checkpoint_message_id_contract.rs" +required-features = ["real-provider-contract-tests"] diff --git a/crates/freshell-opencode/src/lib.rs b/crates/freshell-opencode/src/lib.rs index cc74e3b5d..65e1445b5 100644 --- a/crates/freshell-opencode/src/lib.rs +++ b/crates/freshell-opencode/src/lib.rs @@ -51,7 +51,8 @@ pub use model::{ FRESHOPENCODE_DEFAULT_EFFORT, }; pub use serve::{ - build_prompt_body, default_opencode_database, display_error_chain, is_healthy_response, + build_prompt_body, build_prompt_body_with_message_id, default_opencode_database, + display_error_chain, is_healthy_response, observed_user_message_id, resolve_opencode_database, CreatedSession, DaemonSignal, EffectiveOpencodeDatabase, Endpoint, EventSource, EventStreamHandle, ForkedSession, OpencodeServeManager, PortAllocator, ProcessSpawner, Route, ServeConfig, ServeDeps, ServeError, ServeHttp, ServeHttpError, diff --git a/crates/freshell-opencode/src/serve.rs b/crates/freshell-opencode/src/serve.rs index 1b67993d2..d6cf29356 100644 --- a/crates/freshell-opencode/src/serve.rs +++ b/crates/freshell-opencode/src/serve.rs @@ -2002,9 +2002,39 @@ impl OpencodeServeManager { route: Route, accepted_witness: Option<std::sync::Arc<std::sync::atomic::AtomicBool>>, dispatched_witness: Option<std::sync::Arc<std::sync::atomic::AtomicBool>>, + ) -> Result<(), ServeError> { + self.run_turn_with_message_id( + session_id, + text, + model, + effort, + None, + timeout, + route, + accepted_witness, + dispatched_witness, + ) + .await + } + + /// The native-identity variant of [`Self::run_turn`]. `message_id` is threaded + /// through the same production prompt and idle-settlement path; persistence remains + /// an observation made by the caller through [`Self::list_messages`]. + #[allow(clippy::too_many_arguments)] + pub async fn run_turn_with_message_id( + &self, + session_id: &str, + text: &str, + model: Option<&str>, + effort: Option<&str>, + message_id: Option<&str>, + timeout: Duration, + route: Route, + accepted_witness: Option<std::sync::Arc<std::sync::atomic::AtomicBool>>, + dispatched_witness: Option<std::sync::Arc<std::sync::atomic::AtomicBool>>, ) -> Result<(), ServeError> { let rx = self.subscribe(session_id); - let body = build_prompt_body(text, model, effort); + let body = build_prompt_body_with_message_id(text, model, effort, message_id); self.prompt_async( session_id, body, @@ -2101,6 +2131,39 @@ fn dispatch_event_on_era(inner: &Arc<Inner>, event: ParsedServeEvent, era: Optio /// round-4 R4-2: it composes subscribe + prompt_async + await_idle itself /// so the prompt POST issues INSIDE its dispatch/condemn critical section). pub fn build_prompt_body(text: &str, model: Option<&str>, effort: Option<&str>) -> Value { + build_prompt_body_with_message_id(text, model, effort, None) +} + +/// Build the production prompt body with an optional native message ID. OpenCode's +/// `PromptInput.messageID` is the identity of the persisted user message; callers must +/// only claim it after observing the matching user row in session history. +pub fn build_prompt_body_with_message_id( + text: &str, + model: Option<&str>, + effort: Option<&str>, + message_id: Option<&str>, +) -> Value { + let mut body = match build_prompt_body_without_native_id(text, model, effort) { + Value::Object(body) => body, + _ => unreachable!("prompt body builder returns an object"), + }; + if let Some(message_id) = message_id.filter(|id| is_valid_opencode_message_id(id)) { + body.insert("messageID".into(), Value::String(message_id.to_string())); + } + Value::Object(body) +} + +fn is_valid_opencode_message_id(message_id: &str) -> bool { + message_id + .strip_prefix("msg_") + .is_some_and(|suffix| suffix.len() == 32 && suffix.bytes().all(|b| b.is_ascii_hexdigit())) +} + +fn build_prompt_body_without_native_id( + text: &str, + model: Option<&str>, + effort: Option<&str>, +) -> Value { let mut body = Map::new(); body.insert( "parts".into(), @@ -2118,6 +2181,22 @@ pub fn build_prompt_body(text: &str, model: Option<&str>, effort: Option<&str>) Value::Object(body) } +/// Return the actual persisted user-message id only when exactly one row matches the +/// requested native ID and is role-qualified as a user row. +pub fn observed_user_message_id(messages: &Value, expected_message_id: &str) -> Option<String> { + if !is_valid_opencode_message_id(expected_message_id) { + return None; + } + let matches = messages.as_array()?.iter().filter_map(|message| { + let info = message.get("info")?; + (info.get("id")?.as_str()? == expected_message_id && info.get("role")?.as_str()? == "user") + .then(|| expected_message_id.to_string()) + }); + let mut matches = matches; + let found = matches.next()?; + matches.next().is_none().then_some(found) +} + /// `isHealthyResponse(body)` (`serve-manager.ts:57-59`) over the raw probe body: a JSON /// object is healthy unless `healthy === false`; a non-JSON/unparseable 2xx body is /// treated as `{}` → healthy (the reference `res.json().catch(() => ({}))`, diff --git a/crates/freshell-opencode/tests/checkpoint_message_id.rs b/crates/freshell-opencode/tests/checkpoint_message_id.rs new file mode 100644 index 000000000..aa8370145 --- /dev/null +++ b/crates/freshell-opencode/tests/checkpoint_message_id.rs @@ -0,0 +1,198 @@ +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use freshell_opencode::serve::{ + Endpoint, EventSink, EventSource, EventStreamHandle, HttpMethod, PortAllocator, ProcessSpawner, + ServeConfig, ServeDeps, ServeError, ServeHttp, ServeHttpError, ServeHttpRequest, + ServeHttpResponse, ServeProcess, SpawnRequest, +}; +use freshell_opencode::{observed_user_message_id, OpencodeServeManager}; +use serde_json::{json, Value}; + +const CHECKPOINT_MESSAGE_ID: &str = "msg_0123456789abcdef0123456789abcdef"; + +#[derive(Default)] +struct FakeProviderState { + prompt_body: Mutex<Option<Value>>, + persisted_messages: Mutex<Value>, + status_poll: AtomicUsize, +} + +struct FakeHttp(Arc<FakeProviderState>); + +impl ServeHttp for FakeHttp { + fn request<'a>( + &'a self, + request: ServeHttpRequest, + ) -> freshell_opencode::serve::BoxFuture<'a, Result<ServeHttpResponse, ServeHttpError>> { + let state = self.0.clone(); + Box::pin(async move { + if request.url.contains("/global/health") { + return Ok(ServeHttpResponse::new(200, b"{}".to_vec())); + } + if request.method == HttpMethod::Post && request.url.contains("/prompt_async") { + let body: Value = + serde_json::from_slice(request.body.as_deref().expect("prompt body is sent")) + .expect("prompt body is JSON"); + let native_id = body["messageID"] + .as_str() + .unwrap_or("provider-generated") + .to_string(); + *state.prompt_body.lock().unwrap() = Some(body); + *state.persisted_messages.lock().unwrap() = json!([{ + "info": {"id": native_id, "role": "user"}, + "parts": [{"type": "text", "text": "synthetic prompt"}] + }]); + return Ok(ServeHttpResponse::new(200, b"{}".to_vec())); + } + if request.method == HttpMethod::Get && request.url.contains("/session/status") { + let poll = state.status_poll.fetch_add(1, Ordering::SeqCst); + let body = if poll == 0 { + json!({"ses_checkpoint": {"type": "busy"}}) + } else { + json!({"ses_checkpoint": {"type": "idle"}}) + }; + return Ok(ServeHttpResponse::new( + 200, + serde_json::to_vec(&body).unwrap(), + )); + } + if request.method == HttpMethod::Get && request.url.contains("/message") { + let body = state.persisted_messages.lock().unwrap().clone(); + return Ok(ServeHttpResponse::new( + 200, + serde_json::to_vec(&body).unwrap(), + )); + } + Ok(ServeHttpResponse::new(200, b"{}".to_vec())) + }) + } +} + +struct FakeProcess; + +impl ServeProcess for FakeProcess { + fn exited(&self) -> Option<i32> { + None + } + + fn take_fatal_startup_error(&self) -> Option<String> { + None + } + + fn kill(&self) {} +} + +struct FakeSpawner; + +impl ProcessSpawner for FakeSpawner { + fn spawn(&self, _request: SpawnRequest) -> Result<Box<dyn ServeProcess>, String> { + Ok(Box::new(FakeProcess)) + } +} + +struct FakePorts; + +impl PortAllocator for FakePorts { + fn allocate(&self) -> Result<Endpoint, String> { + Ok(Endpoint { + hostname: "127.0.0.1".into(), + port: 43127, + }) + } +} + +struct FakeEventHandle; + +impl EventStreamHandle for FakeEventHandle {} + +struct FakeEvents; + +impl EventSource for FakeEvents { + fn connect(&self, _url: String, _sink: EventSink) -> Box<dyn EventStreamHandle> { + Box::new(FakeEventHandle) + } +} + +async fn fake_manager() -> (OpencodeServeManager, Arc<FakeProviderState>) { + let state = Arc::new(FakeProviderState::default()); + let manager = OpencodeServeManager::new( + ServeDeps { + spawner: Arc::new(FakeSpawner), + http: Arc::new(FakeHttp(state.clone())), + ports: Arc::new(FakePorts), + events: Arc::new(FakeEvents), + }, + ServeConfig { + health_timeout: Duration::from_secs(1), + health_probe_timeout: Duration::from_millis(100), + health_retry_interval: Duration::from_millis(1), + idle_poll_interval: Duration::from_millis(1), + request_timeout: Duration::from_secs(1), + ..ServeConfig::default() + }, + ); + (manager, state) +} + +#[tokio::test] +async fn opencode_checkpoint_message_id_in_prompt_body() -> Result<(), ServeError> { + let (manager, state) = fake_manager().await; + manager + .run_turn_with_message_id( + "ses_checkpoint", + "synthetic prompt", + Some("local/fake"), + None, + Some(CHECKPOINT_MESSAGE_ID), + Duration::from_secs(1), + None, + None, + None, + ) + .await?; + + let prompt_body = state.prompt_body.lock().unwrap().clone().unwrap(); + assert_eq!(prompt_body["messageID"], CHECKPOINT_MESSAGE_ID); + let persisted = manager.list_messages("ses_checkpoint", &None).await?; + assert_eq!( + observed_user_message_id(&persisted, CHECKPOINT_MESSAGE_ID).as_deref(), + Some(CHECKPOINT_MESSAGE_ID), + "the production prompt path is joined only after the fake serve exposes its persisted user row" + ); + manager.shutdown().await; + Ok(()) +} + +#[test] +fn opencode_checkpoint_message_id_binds_observed_native_row() { + let messages = json!([ + {"info": {"id": "msg_assistant", "role": "assistant"}}, + {"info": {"id": CHECKPOINT_MESSAGE_ID, "role": "user"}}, + ]); + assert_eq!( + observed_user_message_id(&messages, CHECKPOINT_MESSAGE_ID).as_deref(), + Some(CHECKPOINT_MESSAGE_ID) + ); + assert_eq!( + observed_user_message_id(&messages, "msg_abcdefabcdefabcdefabcdefabcdefab"), + None, + "a supplied ID is not binding evidence when history does not show it" + ); + let assistant_only = json!([{"info": {"id": CHECKPOINT_MESSAGE_ID, "role": "assistant"}}]); + assert_eq!( + observed_user_message_id(&assistant_only, CHECKPOINT_MESSAGE_ID), + None, + "an ID on an assistant row is not a submitted user turn" + ); + let ambiguous = json!([ + {"info": {"id": CHECKPOINT_MESSAGE_ID, "role": "user"}}, + {"info": {"id": CHECKPOINT_MESSAGE_ID, "role": "user"}}, + ]); + assert_eq!( + observed_user_message_id(&ambiguous, CHECKPOINT_MESSAGE_ID), + None, + "duplicate matching user rows remain ambiguous" + ); +} diff --git a/crates/freshell-opencode/tests/checkpoint_message_id_contract.rs b/crates/freshell-opencode/tests/checkpoint_message_id_contract.rs new file mode 100644 index 000000000..e7ca62eeb --- /dev/null +++ b/crates/freshell-opencode/tests/checkpoint_message_id_contract.rs @@ -0,0 +1,276 @@ +use std::fs; +use std::io::{Read, Write}; +use std::net::{TcpListener, TcpStream}; +use std::path::{Path, PathBuf}; +use std::process::Command; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::thread::{self, JoinHandle}; +use std::time::Duration; + +use freshell_opencode::serve::{ServeConfig, ServeDeps}; +use freshell_opencode::transport::{ + LoopbackPortAllocator, ReqwestEventSource, ReqwestServeHttp, TokioProcessSpawner, +}; +use freshell_opencode::{observed_user_message_id, OpencodeServeManager}; +use serde_json::json; + +const EXPECTED_VERSION: &str = "1.18.21"; +const NATIVE_MESSAGE_ID: &str = "msg_55c25d96e4445f67b35ea116686e169c"; + +struct LocalModelEndpoint { + base_url: String, + stopping: Arc<AtomicBool>, + worker: Option<JoinHandle<()>>, +} + +impl LocalModelEndpoint { + fn start() -> Self { + let listener = TcpListener::bind(("127.0.0.1", 0)).expect("bind local fake model"); + listener + .set_nonblocking(true) + .expect("local fake model listener is nonblocking"); + let address = listener.local_addr().expect("fake model address"); + let stopping = Arc::new(AtomicBool::new(false)); + let worker_stopping = stopping.clone(); + let worker = thread::spawn(move || { + while !worker_stopping.load(Ordering::SeqCst) { + match listener.accept() { + Ok((stream, _)) => respond_to_model_request(stream), + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + thread::sleep(Duration::from_millis(5)); + } + Err(_) => break, + } + } + }); + Self { + base_url: format!("http://{address}/v1"), + stopping, + worker: Some(worker), + } + } + + fn stop(&mut self) { + self.stopping.store(true, Ordering::SeqCst); + if let Some(worker) = self.worker.take() { + let _ = worker.join(); + } + } +} + +impl Drop for LocalModelEndpoint { + fn drop(&mut self) { + self.stop(); + } +} + +fn respond_to_model_request(mut stream: TcpStream) { + let _ = stream.set_read_timeout(Some(Duration::from_secs(2))); + let mut request = Vec::new(); + let mut chunk = [0u8; 4096]; + loop { + match stream.read(&mut chunk) { + Ok(0) => break, + Ok(count) => { + request.extend_from_slice(&chunk[..count]); + let Some(header_end) = request.windows(4).position(|part| part == b"\r\n\r\n") + else { + continue; + }; + let headers = String::from_utf8_lossy(&request[..header_end]); + let content_length = headers + .lines() + .find_map(|line| { + let (key, value) = line.split_once(':')?; + key.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::<usize>().ok()) + .flatten() + }) + .unwrap_or(0); + if request.len() >= header_end + 4 + content_length { + break; + } + } + Err(_) => return, + } + } + + let first_line = String::from_utf8_lossy(&request) + .lines() + .next() + .unwrap_or_default() + .to_string(); + let (status, content_type, body) = if first_line.starts_with("GET /v1/models") { + ( + "200 OK", + "application/json", + br#"{"data":[{"id":"fixture-model","object":"model","created":1,"owned_by":"freshell"}]}"#.to_vec(), + ) + } else { + let body = concat!( + "data: {\"id\":\"chatcmpl-fixture\",\"object\":\"chat.completion.chunk\",\"created\":1,\"model\":\"fixture-model\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"done\"},\"finish_reason\":null}]}\n\n", + "data: {\"id\":\"chatcmpl-fixture\",\"object\":\"chat.completion.chunk\",\"created\":1,\"model\":\"fixture-model\",\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":1,\"completion_tokens\":1,\"total_tokens\":2}}\n\n", + "data: [DONE]\n\n" + ) + .as_bytes() + .to_vec(); + ("200 OK", "text/event-stream", body) + }; + let response = format!( + "HTTP/1.1 {status}\r\ncontent-type: {content_type}\r\ncontent-length: {}\r\nconnection: close\r\n\r\n", + body.len() + ); + if stream.write_all(response.as_bytes()).is_ok() { + let _ = stream.write_all(&body); + let _ = stream.flush(); + } +} + +fn prepare_provider_roots(root: &Path) -> PathBuf { + for name in ["home", "config", "data", "state", "cache", "project"] { + fs::create_dir_all(root.join(name)).expect("create synthetic OpenCode root"); + } + root.join("project") +} + +fn assert_pinned_binary(binary: &Path, root: &Path) { + let output = Command::new(binary) + .arg("--version") + .env_clear() + .env("HOME", root.join("home")) + .env("XDG_CONFIG_HOME", root.join("config")) + .env("XDG_DATA_HOME", root.join("data")) + .env("XDG_STATE_HOME", root.join("state")) + .env("XDG_CACHE_HOME", root.join("cache")) + .env("CI", "1") + .output() + .expect("run the selected OpenCode binary"); + assert!(output.status.success(), "pinned binary --version succeeds"); + let version = String::from_utf8_lossy(&output.stdout).trim().to_string(); + assert_eq!(version, EXPECTED_VERSION, "contract requires pinned binary"); +} + +#[tokio::test] +async fn opencode_checkpoint_message_id_persists_after_prompt() { + let binary = PathBuf::from( + std::env::var_os("FRESHELL_OPENCODE_CONTRACT_BIN") + .expect("runner supplies the run-owned pinned OpenCode binary"), + ); + assert!(binary.is_absolute(), "contract binary path is absolute"); + assert!(binary.is_file(), "contract binary exists"); + let root = PathBuf::from( + std::env::var_os("FRESHELL_OPENCODE_CONTRACT_ROOT") + .expect("runner supplies the private synthetic fixture root"), + ); + assert!(root.is_absolute(), "fixture root is absolute"); + assert!(root.is_dir(), "fixture root exists"); + assert_pinned_binary(&binary, &root); + let project = prepare_provider_roots(&root); + let mut model_endpoint = LocalModelEndpoint::start(); + let config_content = json!({ + "provider": { + "freshell-contract": { + "npm": "@ai-sdk/openai-compatible", + "name": "Freshell local contract provider", + "options": { + "baseURL": model_endpoint.base_url, + "apiKey": "fixture-only" + }, + "models": { + "fixture-model": { + "name": "Synthetic fixture model", + "limit": {"context": 16384, "output": 1024} + } + } + } + } + }) + .to_string(); + let config = ServeConfig { + command: binary.display().to_string(), + env: vec![ + ("HOME".into(), root.join("home").display().to_string()), + ( + "XDG_CONFIG_HOME".into(), + root.join("config").display().to_string(), + ), + ( + "XDG_DATA_HOME".into(), + root.join("data").display().to_string(), + ), + ( + "XDG_STATE_HOME".into(), + root.join("state").display().to_string(), + ), + ( + "XDG_CACHE_HOME".into(), + root.join("cache").display().to_string(), + ), + ("CI".into(), "1".into()), + ("NO_COLOR".into(), "1".into()), + ("OPENCODE_DISABLE_AUTOUPDATE".into(), "1".into()), + ("OPENCODE_CONFIG_CONTENT".into(), config_content), + ], + health_timeout: Duration::from_secs(20), + health_probe_timeout: Duration::from_secs(2), + health_retry_interval: Duration::from_millis(100), + idle_poll_interval: Duration::from_millis(100), + required_idle_status_polls: 2, + request_timeout: Duration::from_secs(20), + ..ServeConfig::default() + }; + let manager = OpencodeServeManager::new( + ServeDeps { + spawner: Arc::new(TokioProcessSpawner), + http: Arc::new(ReqwestServeHttp::new()), + ports: Arc::new(LoopbackPortAllocator), + events: Arc::new(ReqwestEventSource::new()), + }, + config, + ); + + let operation = tokio::time::timeout(Duration::from_secs(120), async { + let project_path = project.display().to_string(); + let session = manager + .create_session(Some("checkpoint contract"), None, Some(&project_path)) + .await + .expect("create synthetic OpenCode session"); + manager + .run_turn_with_message_id( + &session.id, + "Reply with the word done.", + Some("freshell-contract/fixture-model"), + None, + Some(NATIVE_MESSAGE_ID), + Duration::from_secs(90), + Some(project_path), + None, + None, + ) + .await + .expect("local synthetic model turn settles"); + let messages = manager + .list_messages(&session.id, &None) + .await + .expect("read actual persisted OpenCode history"); + assert_eq!( + observed_user_message_id(&messages, NATIVE_MESSAGE_ID).as_deref(), + Some(NATIVE_MESSAGE_ID), + "OpenCode must persist prompt.messageID as info.id on its user row" + ); + }) + .await; + + let cleanup = tokio::time::timeout(Duration::from_secs(15), manager.shutdown()).await; + model_endpoint.stop(); + assert!( + cleanup.is_ok(), + "owned OpenCode process cleanup completes in 15s" + ); + assert!( + operation.is_ok(), + "real persistence contract completes in 120s" + ); + operation.unwrap(); +} diff --git a/scripts/testing/opencode-checkpoint-contract.ts b/scripts/testing/opencode-checkpoint-contract.ts new file mode 100644 index 000000000..3c8b1e823 --- /dev/null +++ b/scripts/testing/opencode-checkpoint-contract.ts @@ -0,0 +1,224 @@ +import { spawn } from 'node:child_process' +import { chmod, mkdir, mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import process from 'node:process' + +const PINNED_VERSION = '1.18.21' +const PINNED_PACKAGE_MANAGER = '10.34.5' +const PACKAGE_PROVISION_TIMEOUT_MS = 5 * 60 * 1000 +const CONTRACT_TIMEOUT_MS = 120 * 1000 + +type CommandResult = { + stdout: string + stderr: string + exitCode: number +} + +function isolatedEnvironment(root: string): NodeJS.ProcessEnv { + const env: NodeJS.ProcessEnv = { ...process.env } + for (const key of Object.keys(env)) { + if (/^(OPENCODE_|OPENAI_API_KEY$|ANTHROPIC_API_KEY$|GEMINI_API_KEY$)/.test(key)) { + delete env[key] + } + } + env.HOME = path.join(root, 'home') + env.XDG_CONFIG_HOME = path.join(root, 'config') + env.XDG_DATA_HOME = path.join(root, 'data') + env.XDG_STATE_HOME = path.join(root, 'state') + env.XDG_CACHE_HOME = path.join(root, 'cache') + env.CI = '1' + env.NO_COLOR = '1' + return env +} + +async function runCommand( + command: string, + args: string[], + options: { cwd: string; env: NodeJS.ProcessEnv; timeoutMs: number; label: string }, +): Promise<CommandResult> { + const child = spawn(command, args, { + cwd: options.cwd, + env: options.env, + stdio: ['ignore', 'pipe', 'pipe'], + detached: process.platform !== 'win32', + }) + let stdout = '' + let stderr = '' + child.stdout.setEncoding('utf8').on('data', (chunk: string) => { stdout += chunk }) + child.stderr.setEncoding('utf8').on('data', (chunk: string) => { stderr += chunk }) + + let timedOut = false + const timer = setTimeout(() => { + timedOut = true + if (child.pid && process.platform !== 'win32') { + try { process.kill(-child.pid, 'SIGTERM') } catch { /* already exited */ } + setTimeout(() => { + if (child.pid) { + try { process.kill(-child.pid, 'SIGKILL') } catch { /* already exited */ } + } + }, 2_000).unref() + } else { + child.kill('SIGTERM') + } + }, options.timeoutMs) + timer.unref() + + const exitCode = await new Promise<number>((resolve, reject) => { + child.once('error', reject) + child.once('close', (code) => resolve(code ?? 1)) + }).finally(() => clearTimeout(timer)) + if (timedOut) throw new Error(`${options.label} exceeded ${options.timeoutMs}ms`) + if (exitCode !== 0) { + throw new Error(`${options.label} exited ${exitCode}\n${stderr.trim() || stdout.trim()}`) + } + return { stdout, stderr, exitCode } +} + +function packageIntegrity(lock: string): string { + const packagesIndex = lock.indexOf('\npackages:') + if (packagesIndex < 0) throw new Error('pnpm lockfile has no packages section') + const packages = lock.slice(packagesIndex) + const block = packages.match( + new RegExp(`(?:^|\\n) opencode-ai@${PINNED_VERSION.replaceAll('.', '\\.')}:[\\s\\S]*?(?=\\n [^ ]|$)`), + )?.[0] + // pnpm emits this as an inline YAML map (`resolution: {integrity: ...}`), + // so the trailing map brace is not part of the SRI value. + const integrity = block?.match(/integrity:\s*(sha512-[A-Za-z0-9+/]+={0,2})(?:\s|})/)?.[1] + if (!integrity) throw new Error(`could not read opencode-ai@${PINNED_VERSION} integrity`) + return integrity +} + +async function main(): Promise<void> { + const tempRoot = await mkdtemp(path.join(os.tmpdir(), 'freshell-opencode-checkpoint-')) + await chmod(tempRoot, 0o700) + const toolRoot = path.join(tempRoot, 'tool') + await mkdir(toolRoot, { mode: 0o700 }) + for (const directory of ['home', 'config', 'data', 'state', 'cache']) { + await mkdir(path.join(tempRoot, directory), { mode: 0o700 }) + } + const env = isolatedEnvironment(tempRoot) + const receipt = { + contract: 'opencode_checkpoint_message_id_persists_after_prompt', + package: `opencode-ai@${PINNED_VERSION}`, + packageIntegrity: '', + binaryVersion: '', + testCommand: '', + testsExecuted: 0, + testsPassed: 0, + testsFailed: 0, + } + const receiptPath = path.resolve( + path.join( + path.resolve(process.cwd(), '..', '..'), + '.worktrees/.the-usual-logs/managed-provider-environment/reports/opencode-checkpoint-contract.json', + ), + ) + let failure: unknown + + try { + await writeFile(path.join(toolRoot, 'package.json'), `${JSON.stringify({ + name: 'freshell-opencode-checkpoint-contract-tool', + version: '1.0.0', + private: true, + packageManager: `pnpm@${PINNED_PACKAGE_MANAGER}`, + }, null, 2)}\n`, { mode: 0o600 }) + await writeFile(path.join(toolRoot, 'pnpm-workspace.yaml'), [ + 'packages: []', + 'allowBuilds:', + ' opencode-ai: true', + '', + ].join('\n'), { mode: 0o600 }) + + const pnpmVersion = await runCommand('pnpm', ['--version'], { + cwd: toolRoot, + env, + timeoutMs: 30_000, + label: 'pnpm version check', + }) + if (pnpmVersion.stdout.trim() !== PINNED_PACKAGE_MANAGER) { + throw new Error(`runner requires pnpm ${PINNED_PACKAGE_MANAGER}; found ${pnpmVersion.stdout.trim()}`) + } + + await runCommand('pnpm', ['add', '--save-exact', '--workspace-root', `opencode-ai@${PINNED_VERSION}`], { + cwd: toolRoot, + env, + timeoutMs: PACKAGE_PROVISION_TIMEOUT_MS, + label: 'pinned OpenCode package provisioning', + }) + await runCommand('pnpm', ['install', '--frozen-lockfile'], { + cwd: toolRoot, + env, + timeoutMs: PACKAGE_PROVISION_TIMEOUT_MS, + label: 'frozen pinned OpenCode install', + }) + + const packageJson = JSON.parse(await readFile(path.join(toolRoot, 'node_modules/opencode-ai/package.json'), 'utf8')) as { + version?: string + } + if (packageJson.version !== PINNED_VERSION) { + throw new Error(`installed opencode-ai package is ${packageJson.version ?? 'unknown'}, expected ${PINNED_VERSION}`) + } + receipt.packageIntegrity = packageIntegrity(await readFile(path.join(toolRoot, 'pnpm-lock.yaml'), 'utf8')) + + const binary = path.join(toolRoot, 'node_modules/.bin/opencode') + await realpath(binary) + const version = await runCommand(binary, ['--version'], { + cwd: toolRoot, + env, + timeoutMs: 30_000, + label: 'run pinned OpenCode version', + }) + receipt.binaryVersion = version.stdout.trim() + if (receipt.binaryVersion !== PINNED_VERSION) { + throw new Error(`selected OpenCode binary is ${receipt.binaryVersion}, expected ${PINNED_VERSION}`) + } + + const rustCommand = [ + 'mise', 'exec', 'rust@1.96', '--', 'cargo', 'test', + '-p', 'freshell-opencode', + '--features', 'real-provider-contract-tests', + '--locked', + '--test', 'checkpoint_message_id_contract', + 'opencode_checkpoint_message_id_persists_after_prompt', + '--', '--exact', '--nocapture', + ] + receipt.testCommand = rustCommand.join(' ') + const contractEnv: NodeJS.ProcessEnv = { ...process.env } + for (const key of Object.keys(contractEnv)) { + if (/^(OPENCODE_|OPENAI_API_KEY$|ANTHROPIC_API_KEY$|GEMINI_API_KEY$)/.test(key)) { + delete contractEnv[key] + } + } + const result = await runCommand(rustCommand[0]!, rustCommand.slice(1), { + cwd: process.cwd(), + env: { + ...contractEnv, + FRESHELL_OPENCODE_CONTRACT_BIN: binary, + FRESHELL_OPENCODE_CONTRACT_ROOT: tempRoot, + }, + timeoutMs: CONTRACT_TIMEOUT_MS + 15_000, + label: 'pinned OpenCode messageID persistence contract', + }) + const output = `${result.stdout}\n${result.stderr}` + const summary = output.match(/test result: ok\. (\d+) passed; (\d+) failed; (\d+) ignored;/) + if (!summary) throw new Error('contract test output has no successful positive-count result summary') + receipt.testsExecuted = Number(summary[1]) + Number(summary[2]) + Number(summary[3]) + receipt.testsPassed = Number(summary[1]) + receipt.testsFailed = Number(summary[2]) + if (receipt.testsExecuted !== 1 || receipt.testsPassed !== 1 || receipt.testsFailed !== 0) { + throw new Error(`expected exactly one passing contract test, got ${receipt.testsExecuted} executed`) + } + } catch (error) { + failure = error + } finally { + await mkdir(path.dirname(receiptPath), { recursive: true, mode: 0o700 }) + await writeFile(receiptPath, `${JSON.stringify(receipt, null, 2)}\n`, { mode: 0o600 }) + await rm(tempRoot, { recursive: true, force: true, maxRetries: 2, retryDelay: 100 }) + } + + if (failure) throw failure + process.stdout.write(`OpenCode checkpoint identity contract passed; private receipt: ${receiptPath}\n`) +} + +await main() diff --git a/shared/fresh-agent-contract.ts b/shared/fresh-agent-contract.ts index 34d4f0ea9..48f8ccf22 100644 --- a/shared/fresh-agent-contract.ts +++ b/shared/fresh-agent-contract.ts @@ -220,6 +220,9 @@ export const FreshAgentTranscriptItemSchema = z.discriminatedUnion('kind', [ export const FreshAgentTurnSchema = z.object({ id: z.string().min(1), turnId: z.string().min(1), + /** Provider-native identity for an observed user row when it differs from + * the display key or must be matched to a server-accepted submission. */ + nativeTurnId: z.string().min(1).optional(), messageId: z.string().min(1).optional(), ordinal: z.number().int().nonnegative().optional(), source: z.enum(['durable', 'live', 'server']).optional(), diff --git a/test/integration/providers/claude-checkpoint-uuid-replay.mjs b/test/integration/providers/claude-checkpoint-uuid-replay.mjs new file mode 100644 index 000000000..6f448225c --- /dev/null +++ b/test/integration/providers/claude-checkpoint-uuid-replay.mjs @@ -0,0 +1,295 @@ +import fs from 'node:fs/promises' +import path from 'node:path' +import http from 'node:http' +import { spawn, execFile } from 'node:child_process' +import { promisify } from 'node:util' +import { randomUUID } from 'node:crypto' + +const repoRoot = process.argv[2] +if (!repoRoot) throw new Error('repository root argument is required') + +const sdkDir = path.join(repoRoot, 'crates/freshell-claude-sidecar/node_modules/@anthropic-ai/claude-agent-sdk') +const sdkPackage = JSON.parse(await fs.readFile(path.join(sdkDir, 'package.json'), 'utf8')) +if (sdkPackage.version !== '0.3.237' || sdkPackage.claudeCodeVersion !== '2.1.237') { + throw new Error('the Claude checkpoint contract requires SDK 0.3.237 and CLI 2.1.237') +} + +const storeRoot = path.join(repoRoot, 'node_modules/.pnpm') +const optionalStore = (await fs.readdir(storeRoot)).find(name => + name.startsWith('@anthropic-ai+claude-agent-sdk-linux-x64@0.3.237')) +if (!optionalStore) throw new Error('the pinned Linux x64 Claude CLI package is not installed') +const bundleDir = path.join(storeRoot, optionalStore, 'node_modules/@anthropic-ai/claude-agent-sdk-linux-x64') +const bundlePackage = JSON.parse(await fs.readFile(path.join(bundleDir, 'package.json'), 'utf8')) +if (bundlePackage.version !== '0.3.237') throw new Error('the bundled Claude CLI package version is not pinned') +const nativeCli = await fs.realpath(path.join(bundleDir, 'claude')) + +const fixture = await fs.mkdtemp(path.join(repoRoot, '.tmp-claude-checkpoint-contract-')) +await fs.chmod(fixture, 0o700) +const home = path.join(fixture, 'home') +const cwd = path.join(fixture, 'workspace') +for (const directory of [home, cwd, path.join(fixture, 'tmp')]) { + await fs.mkdir(directory, { recursive: true, mode: 0o700 }) +} + +const inputUuid = randomUUID() +const promptText = randomUUID() +const responseText = randomUUID() +const apiKey = randomUUID() +const childEnv = { + HOME: home, + PATH: `${path.dirname(process.execPath)}:/usr/bin:/bin`, + TMPDIR: path.join(fixture, 'tmp'), + XDG_CONFIG_HOME: path.join(home, '.config'), + XDG_CACHE_HOME: path.join(home, '.cache'), + XDG_DATA_HOME: path.join(home, '.local/share'), + CLAUDE_CONFIG_DIR: path.join(home, '.claude'), + ANTHROPIC_API_KEY: apiKey, + FRESHELL_CLAUDE_ONECLI_API_KEY: '1', + CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: '1', + DISABLE_TELEMETRY: '1', + DISABLE_ERROR_REPORTING: '1', + DISABLE_AUTOUPDATER: '1', + CLAUDE_CODE_DISABLE_METRICS: '1', + CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY: '1', + LANG: 'C.UTF-8', +} + +let server +let query +let nativeChild +let releasePrompt +let timer +let resultMessage +let replayedUserUuid +let persistedUserUuid +let replayFlagObservedByNativeCli = false +let localProviderClosed = false +let nativeCliExited = false +let verdict = 'FAIL' +let failureStage = 'initialization' + +const closeProvider = async () => { + releasePrompt?.() + try { query?.close?.() } catch {} + if (nativeChild && !nativeCliExited) { + nativeChild.kill('SIGTERM') + await Promise.race([ + new Promise(resolve => nativeChild.once('exit', resolve)), + new Promise(resolve => setTimeout(resolve, 3000)), + ]) + if (!nativeCliExited) nativeChild.kill('SIGKILL') + await Promise.race([ + new Promise(resolve => nativeChild.once('exit', resolve)), + new Promise(resolve => setTimeout(resolve, 3000)), + ]) + } + nativeCliExited = !nativeChild || nativeCliExited + if (server) { + server.closeAllConnections() + await new Promise(resolve => server.close(resolve)) + } + localProviderClosed = !server || !server.listening +} + +try { + const { stdout: nativeVersion } = await promisify(execFile)(nativeCli, ['--version'], { + cwd, + env: childEnv, + timeout: 10_000, + maxBuffer: 8192, + }) + if (!/^2\.1\.237(?:\s|$)/.test(nativeVersion.trim())) { + throw new Error('native CLI did not report version 2.1.237') + } + const { stdout: nativeHelp } = await promisify(execFile)(nativeCli, ['--help'], { + cwd, + env: childEnv, + timeout: 10_000, + maxBuffer: 100_000, + }) + if (!nativeHelp.includes('--replay-user-messages')) throw new Error('native CLI does not advertise replay-user-messages') + + failureStage = 'local provider' + server = http.createServer(async (request, response) => { + const url = new URL(request.url ?? '/', 'http://127.0.0.1') + const chunks = [] + let size = 0 + for await (const chunk of request) { + size += chunk.length + if (size > 2 * 1024 * 1024) { + response.writeHead(413).end() + return + } + chunks.push(chunk) + } + if (chunks.length) { + try { JSON.parse(Buffer.concat(chunks).toString('utf8')) } catch { + response.writeHead(400).end() + return + } + } + if (url.pathname === '/v1/messages/count_tokens') { + response.writeHead(200, { 'content-type': 'application/json' }) + .end(JSON.stringify({ input_tokens: 1 })) + return + } + if (url.pathname !== '/v1/messages') { + response.writeHead(404).end() + return + } + + const message = { + id: `msg_${randomUUID().replaceAll('-', '')}`, + type: 'message', + role: 'assistant', + model: 'claude-sonnet-4-5-20250929', + content: [{ type: 'text', text: responseText }], + stop_reason: 'end_turn', + stop_sequence: null, + usage: { input_tokens: 1, output_tokens: 1, cache_creation_input_tokens: 0, cache_read_input_tokens: 0 }, + } + response.writeHead(200, { + 'content-type': 'text/event-stream', + 'cache-control': 'no-cache', + 'request-id': randomUUID(), + }) + const send = (type, data) => response.write(`event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`) + send('message_start', { message: { ...message, content: [], stop_reason: null, usage: { ...message.usage, output_tokens: 0 } } }) + send('content_block_start', { index: 0, content_block: { type: 'text', text: '' } }) + send('content_block_delta', { index: 0, delta: { type: 'text_delta', text: responseText } }) + send('content_block_stop', { index: 0 }) + send('message_delta', { delta: { stop_reason: 'end_turn', stop_sequence: null }, usage: { output_tokens: 1 } }) + send('message_stop', {}) + response.end() + }) + await new Promise((resolve, reject) => { + server.once('error', reject) + server.listen(0, '127.0.0.1', resolve) + }) + const providerUrl = `http://127.0.0.1:${server.address().port}` + childEnv.ANTHROPIC_BASE_URL = providerUrl + + // Match the production sidecar's clean child environment and SDK dependency. + for (const name of Object.keys(process.env)) delete process.env[name] + Object.assign(process.env, childEnv) + const { query: sdkQuery } = await import(path.join(sdkDir, 'sdk.mjs')) + const abortController = new AbortController() + timer = setTimeout(() => { + abortController.abort() + query?.close?.() + }, 45_000) + + async function* prompt() { + yield { + type: 'user', + uuid: inputUuid, + message: { role: 'user', content: promptText }, + parent_tool_use_id: null, + } + await new Promise(resolve => { releasePrompt = resolve }) + } + + failureStage = 'SDK query and replay' + query = sdkQuery({ + prompt: prompt(), + options: { + cwd, + env: childEnv, + settingSources: ['user', 'project', 'local'], + mcpServers: {}, + strictMcpConfig: true, + tools: [], + model: 'claude-sonnet-4-5-20250929', + maxTurns: 1, + systemPrompt: '', + pathToClaudeCodeExecutable: nativeCli, + extraArgs: { 'replay-user-messages': null }, + abortController, + spawnClaudeCodeProcess(options) { + if (options.command !== nativeCli || options.cwd !== cwd + || options.env.HOME !== home || options.env.ANTHROPIC_BASE_URL !== providerUrl) { + throw new Error('SDK spawned the native CLI outside the isolated local-provider fixture') + } + replayFlagObservedByNativeCli = options.args.includes('--replay-user-messages') + nativeChild = spawn(options.command, options.args, { + cwd: options.cwd, + env: options.env, + signal: options.signal, + stdio: ['pipe', 'pipe', 'pipe'], + }) + nativeChild.stderr.on('data', () => {}) + nativeChild.once('exit', () => { nativeCliExited = true }) + return nativeChild + }, + stderr() {}, + }, + }) + + for await (const message of query) { + if (message.type === 'user' && message.isReplay === true && message.message?.role === 'user') { + replayedUserUuid = message.uuid + } + if (message.type === 'result') { + resultMessage = message + releasePrompt?.() + break + } + } + clearTimeout(timer) + timer = undefined + query.close() + await closeProvider() + + failureStage = 'persisted transcript' + async function walk(directory) { + let entries + try { entries = await fs.readdir(directory, { withFileTypes: true }) } catch { return } + for (const entry of entries) { + const target = path.join(directory, entry.name) + if (entry.isDirectory()) await walk(target) + else if (entry.isFile() && entry.name.endsWith('.jsonl')) { + for (const line of (await fs.readFile(target, 'utf8')).split('\n').filter(Boolean)) { + let row + try { row = JSON.parse(line) } catch { continue } + if (row.type === 'user' && row.message?.role === 'user' && row.uuid === inputUuid) { + persistedUserUuid = row.uuid + } + } + } + } + } + await walk(home) + if (replayedUserUuid === inputUuid && persistedUserUuid === inputUuid + && resultMessage?.subtype === 'success' && resultMessage?.is_error === false + && replayFlagObservedByNativeCli && nativeCliExited && localProviderClosed) { + verdict = 'PASS' + } +} catch (error) { + process.stderr.write(`${JSON.stringify({ + severity: 'error', + component: 'claude-checkpoint-contract', + stage: failureStage, + code: error?.code ?? 'CONTRACT_FAILURE', + name: error?.name ?? 'Error', + message: String(error?.message ?? 'unknown error').slice(0, 500), + })}\n`) +} finally { + if (timer) clearTimeout(timer) + await closeProvider() + await fs.rm(fixture, { recursive: true, force: true }) +} + +const fixtureRemoved = !(await fs.stat(fixture).catch(() => null)) +process.stdout.write(`${JSON.stringify({ + verdict, + sdkVersion: sdkPackage.version, + bundledCliVersion: sdkPackage.claudeCodeVersion, + replayFlagObservedByNativeCli, + replayedUserUuid, + persistedUserUuid, + resultSubtype: resultMessage?.subtype, + nativeCliExited, + localProviderClosed, + fixtureRemoved, +})}\n`) +if (verdict !== 'PASS' || !fixtureRemoved) process.exitCode = 1 diff --git a/test/integration/providers/claude-checkpoint-uuid-replay.test.ts b/test/integration/providers/claude-checkpoint-uuid-replay.test.ts new file mode 100644 index 000000000..9b5b85d82 --- /dev/null +++ b/test/integration/providers/claude-checkpoint-uuid-replay.test.ts @@ -0,0 +1,70 @@ +// @vitest-environment node + +import { spawn, type ChildProcess } from 'node:child_process' +import path from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' + +const __dirname = path.dirname(fileURLToPath(import.meta.url)) +const REPO_ROOT = path.resolve(__dirname, '../../..') +const HARNESS = path.join(__dirname, 'claude-checkpoint-uuid-replay.mjs') +const children = new Set<ChildProcess>() + +afterEach(() => { + for (const child of children) child.kill('SIGKILL') + children.clear() +}) + +describe('Claude checkpoint UUID replay against the pinned local provider', () => { + it('replays and persists the UUID attached to the exact user prompt', async () => { + const child = spawn(process.execPath, [HARNESS, REPO_ROOT], { + cwd: REPO_ROOT, + env: { + PATH: process.env.PATH ?? '/usr/bin:/bin', + LANG: 'C.UTF-8', + }, + stdio: ['ignore', 'pipe', 'pipe'], + }) + children.add(child) + + let stdout = '' + let stderr = '' + child.stdout!.setEncoding('utf8').on('data', (chunk: string) => { stdout += chunk }) + child.stderr!.setEncoding('utf8').on('data', (chunk: string) => { stderr += chunk }) + + const exit = await new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => { + const timer = setTimeout(() => { + child.kill('SIGTERM') + reject(new Error(`Claude local-provider contract timed out; stderr=${stderr.slice(-2000)}`)) + }, 75_000) + timer.unref() + child.once('error', (error) => { + clearTimeout(timer) + reject(error) + }) + child.once('exit', (code, signal) => { + clearTimeout(timer) + resolve({ code, signal }) + }) + }) + children.delete(child) + + expect(exit, stderr).toEqual({ code: 0, signal: null }) + const lines = stdout.trim().split('\n').filter(Boolean) + expect(lines, stderr).toHaveLength(1) + const receipt = JSON.parse(lines[0]!) as Record<string, unknown> + expect(receipt).toMatchObject({ + verdict: 'PASS', + sdkVersion: '0.3.237', + bundledCliVersion: '2.1.237', + replayFlagObservedByNativeCli: true, + replayedUserUuid: expect.any(String), + persistedUserUuid: expect.any(String), + resultSubtype: 'success', + fixtureRemoved: true, + localProviderClosed: true, + nativeCliExited: true, + }) + expect(receipt.replayedUserUuid).toBe(receipt.persistedUserUuid) + }, 80_000) +}) diff --git a/test/unit/claude-sidecar/fixtures/fake-checkpoint-sdk.mjs b/test/unit/claude-sidecar/fixtures/fake-checkpoint-sdk.mjs new file mode 100644 index 000000000..0a810ca5f --- /dev/null +++ b/test/unit/claude-sidecar/fixtures/fake-checkpoint-sdk.mjs @@ -0,0 +1,30 @@ +const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i + +export function query({ prompt, options }) { + const mode = process.env.FRESHELL_FAKE_CHECKPOINT_REPLAY || 'match' + const iterable = { + async *[Symbol.asyncIterator]() { + yield { type: 'system', subtype: 'init', session_id: 'claude-native-session', model: 'fake', cwd: '/tmp' } + const input = await prompt[Symbol.asyncIterator]().next() + if (input.done) return + if (!UUID_RE.test(input.value.uuid || '')) throw new Error('SDK input did not receive a valid v4 UUID') + if (input.value.message?.role !== 'user') throw new Error('SDK input was not a user message') + if (options.extraArgs?.['replay-user-messages'] !== null) throw new Error('replay-user-messages is not enabled') + + if (mode !== 'absent') { + yield { + type: 'user', + isReplay: mode !== 'ordinary', + uuid: mode === 'mismatch' ? 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa' : input.value.uuid, + session_id: input.value.session_id, + message: { role: 'user', content: input.value.message.content }, + parent_tool_use_id: null, + } + } + yield { type: 'assistant', message: { role: 'assistant', content: [{ type: 'text', text: 'not identity evidence' }] } } + yield { type: 'result', subtype: 'success', is_error: false, usage: { input_tokens: 1, output_tokens: 1 } } + }, + close() {}, + } + return iterable +} diff --git a/test/unit/claude-sidecar/user-message-replay.test.ts b/test/unit/claude-sidecar/user-message-replay.test.ts new file mode 100644 index 000000000..f99d3122b --- /dev/null +++ b/test/unit/claude-sidecar/user-message-replay.test.ts @@ -0,0 +1,111 @@ +// @vitest-environment node + +import { afterEach, describe, expect, it } from 'vitest' +import { spawn, type ChildProcess } from 'node:child_process' +import { createInterface } from 'node:readline' +import path from 'node:path' +import { fileURLToPath } from 'node:url' + +const __dirname = path.dirname(fileURLToPath(import.meta.url)) +const REPO_ROOT = path.resolve(__dirname, '../../..') +const SIDECAR = path.join(REPO_ROOT, 'crates/freshell-claude-sidecar/index.mjs') +const FAKE_SDK = path.join(__dirname, 'fixtures/fake-checkpoint-sdk.mjs') +const children = new Set<ChildProcess>() + +afterEach(() => { + for (const child of children) child.kill('SIGKILL') + children.clear() +}) + +interface SidecarFrame { + type?: string + sessionId?: string + event?: Record<string, unknown> + [key: string]: unknown +} + +function launchSidecar(mode: 'match' | 'mismatch' | 'absent' | 'ordinary') { + const child = spawn(process.execPath, [SIDECAR], { + env: { + ...process.env, + FRESHELL_CLAUDE_SDK_QUERY_MODULE: FAKE_SDK, + FRESHELL_FAKE_CHECKPOINT_REPLAY: mode, + }, + stdio: ['pipe', 'pipe', 'pipe'], + }) + children.add(child) + const frames: SidecarFrame[] = [] + const waiters: Array<{ predicate: (frame: SidecarFrame) => boolean; resolve: (frame: SidecarFrame) => void }> = [] + const lines = createInterface({ input: child.stdout! }) + lines.on('line', (line) => { + let frame: SidecarFrame + try { frame = JSON.parse(line) as SidecarFrame } catch { return } + frames.push(frame) + for (let index = waiters.length - 1; index >= 0; index--) { + const waiter = waiters[index]! + if (waiter.predicate(frame)) { + waiters.splice(index, 1) + waiter.resolve(frame) + } + } + }) + const waitFor = (predicate: (frame: SidecarFrame) => boolean) => { + const existing = frames.find(predicate) + if (existing) return Promise.resolve(existing) + return new Promise<SidecarFrame>((resolve, reject) => { + waiters.push({ predicate, resolve }) + const timer = setTimeout(() => { + const index = waiters.findIndex((waiter) => waiter.resolve === resolve) + if (index >= 0) waiters.splice(index, 1) + reject(new Error(`timed out waiting for sidecar frame; saw ${JSON.stringify(frames)}`)) + }, 4000) + timer.unref() + }) + } + return { child, frames, waitFor } +} + +describe('Claude checkpoint identity at the SDK replay boundary', () => { + it.each([ + { mode: 'match' as const, shouldBind: true }, + { mode: 'mismatch' as const, shouldBind: false }, + { mode: 'absent' as const, shouldBind: false }, + { mode: 'ordinary' as const, shouldBind: false }, + ])('$mode replay binds only its own correlated user row', async ({ mode, shouldBind }) => { + const { child, frames, waitFor } = launchSidecar(mode) + try { + child.stdin!.write(JSON.stringify({ type: 'create', requestId: 'create-1', cwd: '/tmp' }) + '\n') + const created = await waitFor((frame) => frame.type === 'created') + const sessionId = String(created.sessionId) + const checkpointAttempt = { + uuid: 'b3eaf0e1-c8fe-4fb3-8802-b3ce52ca0b16', + requestId: 'send-1', + logicalAttemptId: 'attempt-1', + } + child.stdin!.write(JSON.stringify({ + type: 'send', sessionId, text: 'same prompt', checkpointAttempt, + }) + '\n') + await waitFor((frame) => frame.type === 'sdk.result') + + const receipts = frames.filter((frame) => frame.type === 'sdk.checkpointTurnIdentity') + if (shouldBind) { + expect(receipts).toHaveLength(1) + expect(receipts[0]).toMatchObject({ + logicalAttemptId: 'attempt-1', + requestId: 'send-1', + sessionId, + nativeTurnId: checkpointAttempt.uuid, + displayTurnId: checkpointAttempt.uuid, + }) + expect(receipts[0]).not.toHaveProperty('text') + } else { + expect(receipts).toHaveLength(0) + } + expect(frames.some((frame) => frame.type === 'sdk.assistant')).toBe(true) + } finally { + child.stdin!.write(JSON.stringify({ type: 'shutdown' }) + '\n') + await new Promise<void>((resolve) => child.once('close', () => resolve())) + children.delete(child) + } + }) +}) diff --git a/test/unit/shared/fresh-agent-turns.test.ts b/test/unit/shared/fresh-agent-turns.test.ts index f017fa2e2..31821d7a9 100644 --- a/test/unit/shared/fresh-agent-turns.test.ts +++ b/test/unit/shared/fresh-agent-turns.test.ts @@ -89,6 +89,27 @@ describe('fresh-agent display turn helpers', () => { expect(() => FreshAgentTurnSchema.parse({ ...base, summaryKind: 'bogus' })).toThrow() }) + it('accepts the optional native turn identity without changing display turn ids', () => { + const turn = FreshAgentTurnSchema.parse({ + id: 'turn-native:row-0', + turnId: 'turn-native:row-0', + nativeTurnId: 'turn-native', + role: 'user', + summary: 'same prompt', + items: [], + }) + expect(turn.turnId).toBe('turn-native:row-0') + expect(turn.nativeTurnId).toBe('turn-native') + expect(() => FreshAgentTurnSchema.parse({ + id: 'turn-native:row-0', + turnId: 'turn-native:row-0', + nativeTurnId: '', + role: 'user', + summary: 'same prompt', + items: [], + })).toThrow() + }) + it('treats only an explicit echo tag as non-authored (missing is conservative)', () => { expect(turnSummaryIsAuthored({ summaryKind: 'echo' })).toBe(false) expect(turnSummaryIsAuthored({ summaryKind: 'authored' })).toBe(true) From 8d69dc14a02167f722d636b15f3d363c714feca0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 03:48:08 -0700 Subject: [PATCH 308/493] Retry OpenCode checkpoint identity history reads --- crates/freshell-freshagent/src/opencode_ws.rs | 269 +++++++++++++++--- 1 file changed, 234 insertions(+), 35 deletions(-) diff --git a/crates/freshell-freshagent/src/opencode_ws.rs b/crates/freshell-freshagent/src/opencode_ws.rs index c1676fab8..d2a0dea6b 100644 --- a/crates/freshell-freshagent/src/opencode_ws.rs +++ b/crates/freshell-freshagent/src/opencode_ws.rs @@ -87,6 +87,11 @@ const SESSION_TYPE: &str = "freshopencode"; const PROVIDER: &str = "opencode"; /// `DEFAULT_TURN_TIMEOUT_MS` (`adapter.ts:35`). const DEFAULT_TURN_TIMEOUT: Duration = Duration::from_millis(600_000); +/// OpenCode may publish a prompt response before `/session/:id/message` reflects it. +/// Keep reconciliation to one initial read plus three short retries. +const CHECKPOINT_IDENTITY_HISTORY_ATTEMPTS: usize = 4; +const CHECKPOINT_IDENTITY_HISTORY_REQUEST_TIMEOUT: Duration = Duration::from_secs(1); +const CHECKPOINT_IDENTITY_HISTORY_RETRY_DELAY: Duration = Duration::from_millis(75); #[derive(Clone, Debug)] struct CheckpointTurnIdentity { @@ -121,6 +126,57 @@ fn checkpoint_turn_identity(send: &FreshAgentSend) -> Option<CheckpointTurnIdent }) } +/// Observe the native user row for a checkpointed OpenCode send. History reads are +/// read-only and no session lock is held here; cancelling this future drops only the +/// current GET or retry delay. No receipt is emitted unless one bounded read confirms +/// the exact native id on exactly one role=user row. +async fn observe_checkpoint_turn_identity( + manager: &OpencodeServeManager, + session_id: &str, + route: &Option<String>, + identity: &CheckpointTurnIdentity, +) -> Option<String> { + let mut last_failure = None; + for attempt in 1..=CHECKPOINT_IDENTITY_HISTORY_ATTEMPTS { + let read = tokio::time::timeout( + CHECKPOINT_IDENTITY_HISTORY_REQUEST_TIMEOUT, + manager.list_messages(session_id, route), + ) + .await; + match read { + Ok(Ok(messages)) => { + if let Some(native_id) = freshell_opencode::observed_user_message_id( + &messages, + &identity.native_message_id, + ) { + return Some(native_id); + } + last_failure = Some("matching role=user row is not visible yet".to_string()); + } + Ok(Err(error)) => last_failure = Some(error.to_string()), + Err(_) => { + last_failure = Some(format!( + "message-history read exceeded {}ms", + CHECKPOINT_IDENTITY_HISTORY_REQUEST_TIMEOUT.as_millis() + )); + } + } + + if attempt < CHECKPOINT_IDENTITY_HISTORY_ATTEMPTS { + tokio::time::sleep(CHECKPOINT_IDENTITY_HISTORY_RETRY_DELAY).await; + } + } + + tracing::warn!(target: "freshell_freshagent::opencode", + session_id, + request_id = %identity.request_id, + logical_attempt_id = %identity.logical_attempt_id, + attempts = CHECKPOINT_IDENTITY_HISTORY_ATTEMPTS, + error = last_failure.as_deref().unwrap_or("history row was not observed"), + "checkpoint_turn_identity_unobserved"); + None +} + /// Shared, cheaply-cloneable freshopencode WS state (mergeable into `WsState`). #[derive(Clone)] pub struct FreshOpencodeState { @@ -2744,37 +2800,20 @@ impl FreshOpencodeState { // ID on a role=user row. Observe after dispatch; never join by text/order. if prompt_dispatched.load(Ordering::SeqCst) { if let Some(identity) = checkpoint_identity.as_ref() { - match manager.list_messages(&real_id, &route).await { - Ok(messages) => { - if let Some(native_id) = freshell_opencode::observed_user_message_id( - &messages, - &identity.native_message_id, - ) { - this.broadcast(&event_frame( - &real_id, - json!({ - "type": "checkpointTurnIdentity", - "logicalAttemptId": identity.logical_attempt_id, - "requestId": identity.request_id, - "sessionId": real_id, - "nativeTurnId": native_id, - "displayTurnId": identity.native_message_id, - }), - )); - } else { - tracing::warn!(target: "freshell_freshagent::opencode", - session_id = %real_id, - request_id = %identity.request_id, - logical_attempt_id = %identity.logical_attempt_id, - "checkpoint_turn_identity_unobserved"); - } - } - Err(error) => tracing::warn!(target: "freshell_freshagent::opencode", - session_id = %real_id, - request_id = %identity.request_id, - logical_attempt_id = %identity.logical_attempt_id, - error = %error, - "checkpoint_turn_identity_history_read_failed"), + if let Some(native_id) = + observe_checkpoint_turn_identity(&manager, &real_id, &route, identity).await + { + this.broadcast(&event_frame( + &real_id, + json!({ + "type": "checkpointTurnIdentity", + "logicalAttemptId": identity.logical_attempt_id, + "requestId": identity.request_id, + "sessionId": real_id, + "nativeTurnId": native_id, + "displayTurnId": identity.native_message_id, + }), + )); } } } @@ -16814,6 +16853,10 @@ mod tests { requests: StdMutex<Vec<RecordedRequest>>, busy_budget: StdMutex<std::collections::HashMap<String, usize>>, persisted_message_history: StdMutex<std::collections::HashMap<String, Vec<Value>>>, + message_history_misses: StdMutex<usize>, + message_history_errors: StdMutex<usize>, + message_history_reads: AtomicUsize, + message_history_gate: StdMutex<Option<Arc<tokio::sync::Notify>>>, summarize_outcome: SummarizeOutcome, config_body: Vec<u8>, bus_probe: StdMutex<tokio::sync::broadcast::Receiver<String>>, @@ -16893,6 +16936,10 @@ mod tests { requests: StdMutex::new(Vec::new()), busy_budget: StdMutex::new(std::collections::HashMap::new()), persisted_message_history: StdMutex::new(std::collections::HashMap::new()), + message_history_misses: StdMutex::new(0), + message_history_errors: StdMutex::new(0), + message_history_reads: AtomicUsize::new(0), + message_history_gate: StdMutex::new(None), summarize_outcome, config_body, bus_probe: StdMutex::new(bus_probe), @@ -16914,6 +16961,15 @@ mod tests { self.requests.lock().expect("requests mutex").clone() } + fn arm_message_history_gate(&self) -> Arc<tokio::sync::Notify> { + let gate = Arc::new(tokio::sync::Notify::new()); + *self + .message_history_gate + .lock() + .expect("message history gate mutex") = Some(gate.clone()); + gate + } + /// send-during-compact queue (Task 3): arm the one-shot prompt /// park; the returned Notify releases the parked POST. fn arm_prompt_gate(&self) -> Arc<tokio::sync::Notify> { @@ -17255,6 +17311,43 @@ mod tests { }); } if method == "GET" && req.url.contains("/message") { + self.message_history_reads.fetch_add(1, Ordering::SeqCst); + let history_missing = { + let mut remaining = self + .message_history_misses + .lock() + .expect("message history misses mutex"); + if *remaining == 0 { + false + } else { + *remaining -= 1; + true + } + }; + if history_missing { + return Box::pin( + async move { Ok(ServeHttpResponse::new(200, b"[]".to_vec())) }, + ); + } + let history_error = { + let mut remaining = self + .message_history_errors + .lock() + .expect("message history errors mutex"); + if *remaining == 0 { + false + } else { + *remaining -= 1; + true + } + }; + if history_error { + return Box::pin(async move { + Err(ServeHttpError::Ambiguous( + "transient message-history read failure by the test knob".to_string(), + )) + }); + } let id = req .url .split("/session/") @@ -17269,7 +17362,15 @@ mod tests { .get(&id) .cloned() .unwrap_or_default(); + let history_gate = self + .message_history_gate + .lock() + .expect("message history gate mutex") + .take(); return Box::pin(async move { + if let Some(gate) = history_gate { + gate.notified().await; + } Ok(ServeHttpResponse::new( 200, serde_json::to_vec(&messages).expect("history encodes as JSON"), @@ -25062,6 +25163,14 @@ mod tests { ) .await; insert_compact_session(&st, "ses_checkpoint_queue", Some("prov/model")).await; + *http + .message_history_misses + .lock() + .expect("message history misses mutex") = 1; + *http + .message_history_errors + .lock() + .expect("message history errors mutex") = 1; tokio::time::timeout( std::time::Duration::from_secs(2), st.handle_compact(compact_msg("ses_checkpoint_queue")), @@ -25108,13 +25217,25 @@ mod tests { }) .expect("queued prompt drains through the provider transport"); assert_eq!(prompt.body.unwrap()["messageID"], expected_message_id); - - let receipt_frames = frames_until(&mut rx, |frame| { + // The compact that was already in flight has its own completion edge; + // drain it before waiting for the queued send's completion and identity. + let _compact_frames = frames_until(&mut rx, |frame| { frame["type"] == "freshAgent.event" - && frame["event"]["type"] == "checkpointTurnIdentity" + && frame["event"]["type"] == "freshAgent.turn.complete" }) .await; - let receipt = receipt_frames + let settled_frames = frames_until(&mut rx, |frame| { + frame["type"] == "freshAgent.event" + && frame["event"]["type"] == "freshAgent.turn.complete" + }) + .await; + assert_eq!( + http.message_history_reads.load(Ordering::SeqCst), + 3, + "identity reconciliation retries both an initially absent row and a transient history error before settling" + ); + + let receipt = settled_frames .iter() .find(|frame| { frame["type"] == "freshAgent.event" @@ -25130,6 +25251,84 @@ mod tests { st.fresh_agent.ensure_manager().await.shutdown().await; } + #[tokio::test] + async fn checkpoint_turn_identity_opencode_history_retry_cancels_without_leaking_reads() { + let (st, http, mut rx) = + compact_state(r#"{"model":null}"#, SummarizeOutcome::OkAnswered).await; + let manager = st.fresh_agent.ensure_manager().await; + let _history_gate = http.arm_message_history_gate(); + let identity = CheckpointTurnIdentity { + logical_attempt_id: "attempt-cancel".into(), + request_id: "request-cancel".into(), + native_message_id: "msg_0123456789abcdef0123456789abcdef".into(), + }; + let route = None; + let task_manager = manager.clone(); + let task = tokio::spawn(async move { + observe_checkpoint_turn_identity( + &task_manager, + "ses_checkpoint_cancel", + &route, + &identity, + ) + .await + }); + + tokio::time::timeout(Duration::from_secs(1), async { + while http.message_history_reads.load(Ordering::SeqCst) == 0 { + tokio::task::yield_now().await; + } + }) + .await + .expect("the bounded history read starts"); + task.abort(); + assert!( + task.await + .expect_err("the aborted reconciliation task settles") + .is_cancelled(), + "cancellation drops the in-flight read instead of leaving a retry task behind" + ); + + tokio::time::sleep(Duration::from_millis(200)).await; + assert_eq!(http.message_history_reads.load(Ordering::SeqCst), 1); + assert!(drain_frames(&mut rx).iter().all(|frame| { + frame["type"] != "freshAgent.event" + || frame["event"]["type"] != "checkpointTurnIdentity" + })); + manager.shutdown().await; + } + + #[tokio::test] + async fn checkpoint_turn_identity_opencode_history_retry_stops_at_attempt_limit() { + let (st, http, _rx) = + compact_state(r#"{"model":null}"#, SummarizeOutcome::OkAnswered).await; + let manager = st.fresh_agent.ensure_manager().await; + *http + .message_history_errors + .lock() + .expect("message history errors mutex") = CHECKPOINT_IDENTITY_HISTORY_ATTEMPTS; + let identity = CheckpointTurnIdentity { + logical_attempt_id: "attempt-bounded".into(), + request_id: "request-bounded".into(), + native_message_id: "msg_0123456789abcdef0123456789abcdef".into(), + }; + + let observed = + observe_checkpoint_turn_identity(&manager, "ses_checkpoint_bounded", &None, &identity) + .await; + + assert_eq!( + observed, None, + "failed history reads never bind a candidate id" + ); + assert_eq!( + http.message_history_reads.load(Ordering::SeqCst), + CHECKPOINT_IDENTITY_HISTORY_ATTEMPTS, + "history reconciliation performs one initial read and only the configured retries" + ); + manager.shutdown().await; + } + // ── merged from the send-during-compact-queue branch ── /// MUST #4 (the client contract): a fresh send arriving while older /// entries are still queued appends — never drives ahead of them. From a686953e8e53b9eae29ba4c0fcf22c4d7563173b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 03:57:45 -0700 Subject: [PATCH 309/493] feat: gate fresh-agent sends on checkpoint capture Persist the checkpoint outcome and logical-attempt dispatch fence before forwarding local or managed provider sends. Keep request-owned Git process trees and workspace locks under a server supervisor through timeout/drop settlement, and replay provider-confirmed turn identity with durable outcomes. Validation: restricted checkpoint runtime fixture passed 2/2; sandboxed dropped-route and timeout descendant selectors passed; focused barrier, quarantine, durable Codex receipt, delayed/read-error Codex identity, WS test compilation, and checkpoint outcome schema tests passed. Windows: default-feature x86_64-pc-windows-gnu server binary check passed. With managed-runtime-v1, cross-check remains blocked by the existing unconditional tokio::net::UnixStream import at crates/freshell-runtime-client/src/lib.rs:23. --- Cargo.lock | 1 + crates/freshell-freshagent/src/codex.rs | 454 ++- .../freshell-protocol/src/client_messages.rs | 10 +- crates/freshell-protocol/src/common.rs | 26 + .../freshell-protocol/src/server_messages.rs | 43 +- crates/freshell-server/Cargo.toml | 14 +- crates/freshell-server/src/checkpoints.rs | 3015 ++++++++++++++++- .../freshell-server/src/fresh_agent_proxy.rs | 134 +- .../src/fresh_agent_proxy_tests.rs | 10 + crates/freshell-server/src/main.rs | 17 +- crates/freshell-ws/src/auto_resume.rs | 1 + crates/freshell-ws/src/checkpoint_barrier.rs | 34 + crates/freshell-ws/src/codex_association.rs | 1 + crates/freshell-ws/src/codex_proxy_route.rs | 1 + crates/freshell-ws/src/identity_ownership.rs | 1 + crates/freshell-ws/src/invariants.rs | 1 + crates/freshell-ws/src/lib.rs | 6 + .../freshell-ws/src/opencode_association.rs | 1 + crates/freshell-ws/src/terminal.rs | 108 +- crates/freshell-ws/tests/auto_resume_e2e.rs | 1 + .../freshell-ws/tests/auto_resume_respawn.rs | 1 + .../tests/claude_session_rebind.rs | 1 + .../tests/cli_connection_scheduling.rs | 1 + .../tests/codex_managed_launch_e2e.rs | 1 + .../tests/codex_session_ref_resume.rs | 1 + .../tests/codex_sidecar_reattach_e2e.rs | 1 + crates/freshell-ws/tests/common/mod.rs | 12 + .../freshell-ws/tests/cross_kind_liveness.rs | 1 + .../tests/diag01_lifecycle_events.rs | 1 + .../tests/freshagent_claude_attach.rs | 1 + .../tests/freshagent_claude_kill_interrupt.rs | 1 + .../tests/freshagent_claude_rollback.rs | 1 + .../tests/freshagent_session_lease.rs | 1 + .../freshell-ws/tests/hello_capabilities.rs | 1 + crates/freshell-ws/tests/hello_timeout.rs | 1 + crates/freshell-ws/tests/keepalive.rs | 1 + crates/freshell-ws/tests/max_payload.rs | 1 + .../tests/opencode_switch_rebind.rs | 1 + crates/freshell-ws/tests/origin_policy.rs | 1 + crates/freshell-ws/tests/paced_replay.rs | 1 + crates/freshell-ws/tests/pane_reconcile.rs | 1 + .../tests/pane_reconcile_freshagent.rs | 1 + .../freshell-ws/tests/rest_claude_identity.rs | 1 + .../tests/rest_locator_identity.rs | 1 + .../freshell-ws/tests/rest_ws_shared_gate.rs | 1 + .../tests/restore_plan_queue_cap.rs | 1 + .../freshell-ws/tests/restore_spawn_gate.rs | 1 + .../tests/restore_stale_cwd_rehome.rs | 1 + crates/freshell-ws/tests/restore_storm.rs | 1 + .../tests/resume_validation_gate.rs | 2 + .../tests/safe08_restore_diagnostics.rs | 1 + crates/freshell-ws/tests/sessions_prefs.rs | 1 + .../freshell-ws/tests/term09_output_queue.rs | 1 + crates/freshell-ws/tests/ui_layout_sync.rs | 1 + shared/ws-protocol.ts | 33 +- .../ws-protocol.checkpoint-outcome.test.ts | 55 + 56 files changed, 3805 insertions(+), 208 deletions(-) create mode 100644 crates/freshell-ws/src/checkpoint_barrier.rs create mode 100644 test/unit/shared/ws-protocol.checkpoint-outcome.test.ts diff --git a/Cargo.lock b/Cargo.lock index 8a6c50c14..b0c105620 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1486,6 +1486,7 @@ dependencies = [ "tracing-subscriber", "unicode-normalization", "uuid", + "windows-sys 0.61.2", ] [[package]] diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index 520e48523..0b68b27b8 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -47,26 +47,26 @@ use std::sync::{Arc, Mutex as StdMutex}; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; use axum::{ + Json, Router, extract::State, http::{HeaderMap, StatusCode}, response::{IntoResponse, Response}, routing::patch, - Json, Router, }; -use serde_json::{json, Map, Value}; -use tokio::sync::{oneshot, Mutex as TokioMutex}; +use serde_json::{Map, Value, json}; +use tokio::sync::{Mutex as TokioMutex, oneshot}; use freshell_codex::launch_lifecycle::{ - allocate_loopback_port, drain_child_io, SIDECAR_START_BUDGET, + SIDECAR_START_BUDGET, allocate_loopback_port, drain_child_io, }; -use freshell_codex::launch_plan::{codex_sidecar_spawn_spec, CodexSidecarLaunchContext}; -use freshell_codex::transport::{reap_owned_codex_sidecars, TungsteniteTransport}; +use freshell_codex::launch_plan::{CodexSidecarLaunchContext, codex_sidecar_spawn_spec}; +use freshell_codex::transport::{TungsteniteTransport, reap_owned_codex_sidecars}; use freshell_codex::{ - mint_ownership_id, next_monotonic_turn_complete_at, normalize_codex_thread_status, - normalize_freshcodex_effort, normalize_freshcodex_model, read_rollout_history_mode, - to_codex_reasoning_effort, CodexAdapterEvent, CodexAppServerClient, CodexAppServerError, + CODEX_SIDECAR_OWNERSHIP_ENV, CodexAdapterEvent, CodexAppServerClient, CodexAppServerError, CodexNotification, CodexStatus, CodexSubscription, HistoryMode, StartThreadParams, - StartTurnParams, ThreadForkParams, CODEX_SIDECAR_OWNERSHIP_ENV, + StartTurnParams, ThreadForkParams, mint_ownership_id, next_monotonic_turn_complete_at, + normalize_codex_thread_status, normalize_freshcodex_effort, normalize_freshcodex_model, + read_rollout_history_mode, to_codex_reasoning_effort, }; use freshell_protocol::{ ErrorCode, ErrorMsg, FreshAgentAttach, FreshAgentCompact, FreshAgentConfigure, @@ -78,7 +78,7 @@ use freshell_protocol::{ use freshell_terminal::FrameSink; use crate::summary::{ - truncate_summary, SUMMARY_KIND_AUTHORED, SUMMARY_KIND_ECHO, TOOL_ERROR_LABEL, TOOL_RESULT_LABEL, + SUMMARY_KIND_AUTHORED, SUMMARY_KIND_ECHO, TOOL_ERROR_LABEL, TOOL_RESULT_LABEL, truncate_summary, }; use crate::{FreshAgentCreateDedup, FreshAgentCreateOutcome, SharedPaneIdentitySink}; @@ -188,6 +188,11 @@ fn locate_thread_rollout( const SESSION_TYPE: &str = "freshcodex"; /// The runtime provider (`AGENT_SESSION_TYPES.codex.provider`). const PROVIDER: &str = "codex"; +/// Each checkpoint identity read uses the app-server's existing 30-second +/// snapshot bound; two attempts keep delayed persistence and one transient read +/// failure recoverable without leaving an unbounded background poller. +const CHECKPOINT_IDENTITY_PROBE_ATTEMPTS: usize = 2; +const CHECKPOINT_IDENTITY_PROBE_RETRY_DELAY: Duration = Duration::from_millis(100); enum CodexStopMode { Close, @@ -2403,13 +2408,13 @@ impl FreshCodexState { naming_handle: Option<String>, ) { let _ = &mut own_ticket; // moved to finish_create below on the success path - // b8ke focused episode-2 round-2 F4: the claim exists — register the - // watchdog machinery NOW, before the spawn + thread/resume awaits (up - // to 45s): the cancellation slot + the settle guard (held to THIS - // handler's unwind) + the partial runtime (pid None until the spawn - // hook arms the real identity). A slow startup swept at 30s is then - // cancellable-and-confirmable, never a registration-ignored - // StaleStart fence. + // b8ke focused episode-2 round-2 F4: the claim exists — register the + // watchdog machinery NOW, before the spawn + thread/resume awaits (up + // to 45s): the cancellation slot + the settle guard (held to THIS + // handler's unwind) + the partial runtime (pid None until the spawn + // hook arms the real identity). A slow startup swept at 30s is then + // cancellable-and-confirmable, never a registration-ignored + // StaleStart fence. let resume_start_pid_slot = crate::ownership_lane::sidecar_pid_cancel_slot(); let _resume_start_cancellation = crate::ownership_lane::register_start_cancellation_for_ticket( @@ -2684,10 +2689,10 @@ impl FreshCodexState { naming_handle: Option<String>, ) { let _ = &mut own_ticket; // consumed at the commit below on the success path - // D8 (focused-ep1-r5 Finding 2): a HOLLOW `Some` (a partially - // initialized client's hello — all fields absent) behaves like `None` - // on every decision below: the park, the eviction-guard adopt, and - // the binding write's stamps. + // D8 (focused-ep1-r5 Finding 2): a HOLLOW `Some` (a partially + // initialized client's hello — all fields absent) behaves like `None` + // on every decision below: the park, the eviction-guard adopt, and + // the binding write's stamps. let provenance = provenance.filter(|p| p.is_meaningful()); // kata b8ke Task 3: the sidecar pid for the commit's owner identity // (captured before `child` moves into the exit watcher). @@ -3259,6 +3264,12 @@ impl FreshCodexState { /// self-healed crash. pub async fn handle_send(&self, msg: FreshAgentSend) { let request_id = msg.request_id.clone(); + let checkpoint_identity_request = msg.checkpoint_intent.as_ref().and_then(|intent| { + request_id + .as_deref() + .filter(|request_id| !request_id.trim().is_empty()) + .map(|request_id| (request_id.to_string(), intent.logical_attempt_id.clone())) + }); let mut session_id = msg.session_id.clone(); let cwd = msg.cwd.clone(); // b8ke ext r8 F5: the send's delayed-request fence (the new @@ -3641,13 +3652,95 @@ impl FreshCodexState { self.broadcast(&ServerMessage::FreshAgentSendAccepted( freshell_protocol::FreshAgentSendAccepted { provider: PROVIDER.to_string(), - request_id: request_id.unwrap_or_default(), - session_id, + request_id: request_id.clone().unwrap_or_default(), + session_id: session_id.clone(), session_type: SESSION_TYPE.to_string(), cwd: turn_cwd, - submitted_turn_id: Some(submitted_turn_id), + submitted_turn_id: Some(submitted_turn_id.clone()), }, )); + + if let Some((request_id, logical_attempt_id)) = checkpoint_identity_request { + self.spawn_checkpoint_identity_probe( + client, + session_id, + submitted_turn_id, + request_id, + logical_attempt_id, + ); + } + } + + fn spawn_checkpoint_identity_probe( + &self, + client: Arc<CodexAppServerClient>, + session_id: String, + submitted_turn_id: String, + request_id: String, + logical_attempt_id: String, + ) { + let state = self.clone(); + tokio::spawn(async move { + // `turn/start` acceptance is not itself proof that the native user row + // is persisted. Read the provider history and emit a receipt only when + // the exact submitted turn projects to one unique user row. Reads + // are bounded individually by the app-server client's existing + // 30-second snapshot timeout; the fixed attempt count bounds this + // service-owned probe to at most two reads. + let mut identity = None; + for attempt in 1..=CHECKPOINT_IDENTITY_PROBE_ATTEMPTS { + match client.read_thread(&session_id, true).await { + Ok(raw_thread) => { + identity = checkpoint_codex_turn_identity(&raw_thread, &submitted_turn_id); + if identity.is_some() { + break; + } + tracing::debug!(target: "freshell_freshagent::codex", + session_id = %session_id, + submitted_turn_id = %submitted_turn_id, + request_id = %request_id, + attempt, + "freshagent.codex.checkpoint_identity_row_not_yet_observed" + ); + } + Err(error) => { + tracing::warn!(target: "freshell_freshagent::codex", + session_id = %session_id, + submitted_turn_id = %submitted_turn_id, + request_id = %request_id, + attempt, + error = %error, + "freshagent.codex.checkpoint_identity_read_failed" + ); + } + } + if attempt < CHECKPOINT_IDENTITY_PROBE_ATTEMPTS { + tokio::time::sleep(CHECKPOINT_IDENTITY_PROBE_RETRY_DELAY).await; + } + } + let Some((native_turn_id, display_turn_id)) = identity else { + tracing::warn!(target: "freshell_freshagent::codex", + session_id = %session_id, + submitted_turn_id = %submitted_turn_id, + request_id = %request_id, + "freshagent.codex.checkpoint_identity_probe_exhausted" + ); + return; + }; + state.broadcast(&ServerMessage::FreshAgentEvent(FreshAgentEvent { + event: json!({ + "type": "checkpointTurnIdentity", + "logicalAttemptId": logical_attempt_id, + "requestId": request_id, + "sessionId": session_id, + "nativeTurnId": native_turn_id, + "displayTurnId": display_turn_id, + }), + provider: PROVIDER.to_string(), + session_id, + session_type: SESSION_TYPE.to_string(), + })); + }); } fn send_error(&self, request_id: &Option<String>, code: &str, message: &str) { @@ -9590,6 +9683,34 @@ fn build_codex_turn_json(raw_turn: &Value, ordinal: usize) -> Result<Vec<Value>, Ok(turns) } +fn checkpoint_codex_turn_identity( + raw_thread: &Value, + submitted_turn_id: &str, +) -> Option<(String, String)> { + let raw_turns = raw_thread.get("thread")?.get("turns")?.as_array()?; + let mut matching = raw_turns + .iter() + .filter(|turn| turn.get("id").and_then(Value::as_str) == Some(submitted_turn_id)); + let raw_turn = matching.next()?; + if matching.next().is_some() { + return None; + } + let projected = build_codex_turn_json(raw_turn, 0).ok()?; + let mut user_rows = projected + .iter() + .filter(|turn| turn.get("role").and_then(Value::as_str) == Some("user")); + let user_row = user_rows.next()?; + if user_rows.next().is_some() { + return None; + } + let native_turn_id = user_row.get("nativeTurnId")?.as_str()?; + if native_turn_id != submitted_turn_id { + return None; + } + let display_turn_id = user_row.get("turnId")?.as_str()?; + Some((native_turn_id.to_string(), display_turn_id.to_string())) +} + /// Watch an owned sidecar child to completion. Two ways out: /// /// - The child exits ON ITS OWN (crash / unexpected disconnect, never requested): self-heal @@ -11034,8 +11155,7 @@ fn codex_turn_plain_text(turn: &Value) -> String { /// the ledger) and the pane is told the two histories may disagree until the /// next refresh. Explicit RPC-rejection legs instead compensate (the provider /// ANSWERED, so the revert provably did not apply). -const CODEX_UNCERTAIN_ROLLBACK_COPY: &str = - "Undo state could not be confirmed — conversation and undo history may disagree until the next refresh."; +const CODEX_UNCERTAIN_ROLLBACK_COPY: &str = "Undo state could not be confirmed — conversation and undo history may disagree until the next refresh."; /// LBC-1: a pre-feature LEGACY thread — `thread/revert` answered /// `-32600 "…only supports paginated threads"`. Structured on the RPC code @@ -11166,7 +11286,9 @@ fn normalize_codex_permission( } match permission.as_deref() { None | Some("untrusted" | "on-failure" | "on-request" | "never") => Ok(()), - _ => Err("Choose a supported Codex approval policy: untrusted, on-failure, on-request, or never."), + _ => Err( + "Choose a supported Codex approval policy: untrusted, on-failure, on-request, or never.", + ), } } @@ -11237,8 +11359,8 @@ pub(crate) mod tests { use std::sync::{Arc, Mutex, OnceLock}; use tracing::field::{Field, Visit}; use tracing::{Event, Subscriber}; - use tracing_subscriber::layer::{Context, SubscriberExt}; use tracing_subscriber::Layer; + use tracing_subscriber::layer::{Context, SubscriberExt}; #[derive(Debug, Clone, Default)] pub struct CapturedEvent { @@ -13078,9 +13200,10 @@ pub(crate) mod tests { _ => panic!("the seed claim must grant"), }; let generation = seed_ticket.as_ref().expect("the seed ticket").generation(); - assert!(st - .commit_lane_claim_at(&mut seed_ticket, "thread-r29-f3", "thread-r29-f3", None,) - .is_ok()); + assert!( + st.commit_lane_claim_at(&mut seed_ticket, "thread-r29-f3", "thread-r29-f3", None,) + .is_ok() + ); insert_fake_session_with_real_consumer( &st, @@ -13155,9 +13278,10 @@ pub(crate) mod tests { _ => panic!("the seed claim must grant"), }; let generation = seed_ticket.as_ref().expect("the seed ticket").generation(); - assert!(st - .commit_lane_claim_at(&mut seed_ticket, "thread-ep5-r2", "thread-ep5-r2", None) - .is_ok()); + assert!( + st.commit_lane_claim_at(&mut seed_ticket, "thread-ep5-r2", "thread-ep5-r2", None) + .is_ok() + ); insert_fake_session_with_real_consumer( &st, @@ -13605,10 +13729,12 @@ pub(crate) mod tests { leases.claim(PROVIDER, session_id, "replacement", 1), crate::session_lease::FreshSessionClaim::Acquired )); - assert!(!condemned_priors - .lock() - .expect("condemned priors lock") - .contains_key(session_id)); + assert!( + !condemned_priors + .lock() + .expect("condemned priors lock") + .contains_key(session_id) + ); } /// b8ke focused review FR9: the half-fenced kill refusal carries the @@ -14030,8 +14156,8 @@ pub(crate) mod tests { /// cleaned) while the answer still reports `success:false`: the kill /// visibly fails, never masquerading as clean success nor clean failure. #[tokio::test] - async fn a_kill_whose_close_persists_despite_the_reported_error_ends_the_session_and_fails_visibly( - ) { + async fn a_kill_whose_close_persists_despite_the_reported_error_ends_the_session_and_fails_visibly() + { let (transport, _peer) = freshell_codex::new_channel_transport(); let (client, _notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -14098,8 +14224,8 @@ pub(crate) mod tests { /// row revived — then the lane parks pre-registration while the kill /// lands.) #[tokio::test(flavor = "multi_thread")] - async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_attached_thread( - ) { + async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_attached_thread() + { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd("{}"); let (st, mut rx, fake) = state_with_sink(); @@ -14207,8 +14333,8 @@ pub(crate) mod tests { /// re-check tears the registered session down and the create answers /// failed. #[tokio::test(flavor = "multi_thread")] - async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_resumed_create( - ) { + async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_resumed_create() + { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd("{}"); let (st, mut rx, fake) = state_with_sink(); @@ -14818,8 +14944,8 @@ pub(crate) mod tests { /// folded. Repeat the exact no-slack delivery window enough times to exercise the /// independent RPC waiter and notification-consumer tasks on the multi-thread runtime. #[tokio::test(flavor = "multi_thread", worker_threads = 4)] - async fn handle_send_always_broadcasts_accepted_before_an_immediate_completion_snapshot_and_chime( - ) { + async fn handle_send_always_broadcasts_accepted_before_an_immediate_completion_snapshot_and_chime() + { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -15099,8 +15225,8 @@ pub(crate) mod tests { /// between the stale and the matching completion stays BUSY_TURN; the gate /// releases EXACTLY at the id-matched completion. #[tokio::test] - async fn a_stale_prior_completion_never_clears_the_compact_window_and_only_the_matching_completion_does( - ) { + async fn a_stale_prior_completion_never_clears_the_compact_window_and_only_the_matching_completion_does() + { let (st, _rx_boot) = state_with_bus(); let (peer, mut wire) = insert_idle_compact_session(&st, "thread-cstale").await; @@ -15401,8 +15527,8 @@ pub(crate) mod tests { /// compact window so the newer turn's own matching completion can reopen the /// rollback gate. #[tokio::test] - async fn an_idless_compact_completion_inside_a_newer_turns_window_stays_busy_and_retires_the_compact( - ) { + async fn an_idless_compact_completion_inside_a_newer_turns_window_stays_busy_and_retires_the_compact() + { let (st, _rx_boot) = state_with_bus(); let (peer, mut wire) = insert_idle_compact_session(&st, "thread-cnewer-idless").await; @@ -17065,8 +17191,8 @@ pub(crate) mod tests { } #[tokio::test(flavor = "multi_thread")] - async fn handle_fork_threads_fork_archive_then_child_unarchive_resume_across_two_sidecars_and_replies_forked( - ) { + async fn handle_fork_threads_fork_archive_then_child_unarchive_resume_across_two_sidecars_and_replies_forked() + { let _guard = ENV_LOCK.lock().await; let log_path = std::env::temp_dir().join(format!( "freshell-fork-oplog-{}-{}.jsonl", @@ -17629,8 +17755,8 @@ pub(crate) mod tests { /// guard) but finds a LIVE incumbent at registration time adopts that /// incumbent — this arm must restamp exactly like the fast path. #[tokio::test] - async fn adopt_live_create_through_the_finish_create_eviction_guard_restamps_the_current_connection( - ) { + async fn adopt_live_create_through_the_finish_create_eviction_guard_restamps_the_current_connection() + { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd( &json!({ @@ -17756,8 +17882,8 @@ pub(crate) mod tests { /// incumbent's already-parked Some to None, and writes nothing (the /// ledger's keep-when-None merge preserves the row's stamps). #[tokio::test] - async fn adopt_live_create_with_no_connection_provenance_keeps_the_parked_stamps_and_writes_nothing( - ) { + async fn adopt_live_create_with_no_connection_provenance_keeps_the_parked_stamps_and_writes_nothing() + { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd("{}"); let (st, mut rx) = state_with_bus(); @@ -17907,8 +18033,8 @@ pub(crate) mod tests { /// with a parent that parks nothing the child stamps fall back to the /// parent's DURABLE ROW (the last source that knows the attribution). #[tokio::test] - async fn fork_falls_back_to_the_durable_row_when_the_fork_connection_is_hollow_and_the_park_is_empty( - ) { + async fn fork_falls_back_to_the_durable_row_when_the_fork_connection_is_hollow_and_the_park_is_empty() + { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd( &json!({ @@ -19115,8 +19241,8 @@ pub(crate) mod tests { /// latch while a NEWER turn is live (that latch belongs to the newer /// turn now) — the seed window deterministically constructed below. #[tokio::test] - async fn a_superseded_slot_completion_during_the_snapshot_seed_window_keeps_the_newer_turns_latch( - ) { + async fn a_superseded_slot_completion_during_the_snapshot_seed_window_keeps_the_newer_turns_latch() + { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -19516,8 +19642,8 @@ pub(crate) mod tests { /// publishes its snapshot without a second edge, and a subsequent /// crash rings nothing extra. #[tokio::test] - async fn system_error_status_with_a_turn_in_flight_rings_once_and_gates_the_matching_completion( - ) { + async fn system_error_status_with_a_turn_in_flight_rings_once_and_gates_the_matching_completion() + { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -20665,7 +20791,9 @@ pub(crate) mod tests { assert_eq!(frames[0]["event"]["code"], json!("UNSUPPORTED_CAPABILITY")); assert_eq!( frames[0]["event"]["message"], - json!("Redo is not supported for freshcodex (codex history revert is destructive; there is no redo primitive).") + json!( + "Redo is not supported for freshcodex (codex history revert is destructive; there is no redo primitive)." + ) ); assert!( fake.rollbacks.lock().unwrap().is_empty(), @@ -21217,8 +21345,8 @@ pub(crate) mod tests { } #[tokio::test] - async fn handle_rollback_double_undo_within_one_epoch_keeps_the_marker_bucket_in_conversation_order( - ) { + async fn handle_rollback_double_undo_within_one_epoch_keeps_the_marker_bucket_in_conversation_order() + { // Whole-branch review Minor-3: the codex marker bucket is CONVERSATION // order (the plan :165 union rule wins over the codex bullet at :166). // Two sequential undos inside ONE epoch (no submission between) remove @@ -21402,8 +21530,8 @@ pub(crate) mod tests { } #[tokio::test] - async fn handle_rollback_undo_send_undo_undo_freezes_epoch_zero_and_orders_the_new_epoch_ascending( - ) { + async fn handle_rollback_undo_send_undo_undo_freezes_epoch_zero_and_orders_the_new_epoch_ascending() + { // Delta-r1 F8's codified multi-epoch case: undo → send (decision 5 // destroy) → undo → undo ⇒ the marker bucket is the FIRST epoch's // frozen prefix, then the NEW epoch's entries in conversation order @@ -24170,8 +24298,8 @@ pub(crate) mod tests { // _post_recovery_snapshot` (the other test mutating the process-global // `CODEX_CMD`/`FAKE_CODEX_APP_SERVER_BEHAVIOR` env vars) for the test's ENTIRE // duration, not just around individual calls. - async fn send_after_unrequested_crash_resumes_the_same_thread_id_and_completes_with_no_error_frame( - ) { + async fn send_after_unrequested_crash_resumes_the_same_thread_id_and_completes_with_no_error_frame() + { let _guard = ENV_LOCK.lock().await; let (st, mut rx) = state_with_bus(); @@ -24419,8 +24547,8 @@ pub(crate) mod tests { /// session mapped under its OLD id, still marked exited, for a future retry (mirroring /// the pre-existing `RespawnFailed` contract for a `thread/start` failure). #[tokio::test(flavor = "multi_thread")] - async fn send_after_crash_with_transient_resume_failure_reports_respawn_failed_and_stays_exited( - ) { + async fn send_after_crash_with_transient_resume_failure_reports_respawn_failed_and_stays_exited() + { let _guard = ENV_LOCK.lock().await; let (st, mut rx) = state_with_bus(); @@ -24840,8 +24968,8 @@ pub(crate) mod tests { #[tokio::test(flavor = "multi_thread")] // Intentional: `_guard` must span every `.await` in this test to serialize it against // every other env-var-mutating test in this module for its ENTIRE duration. - async fn attach_after_unrequested_crash_respawns_the_sidecar_and_emits_a_post_recovery_snapshot( - ) { + async fn attach_after_unrequested_crash_respawns_the_sidecar_and_emits_a_post_recovery_snapshot() + { let _guard = ENV_LOCK.lock().await; let (st, mut rx) = state_with_bus(); let capture = tracing_capture::capture_by_session("attach-recovery-marker-unused"); @@ -25361,7 +25489,8 @@ pub(crate) mod tests { "no broadcast error may target the discarded old id: {frame}" ); assert_eq!( - frame["sessionId"], json!("parent-new-mint"), + frame["sessionId"], + json!("parent-new-mint"), "the degradation frame rides the new id, like the crate's own keying invariant: {frame}" ); } @@ -26662,10 +26791,26 @@ pub(crate) mod tests { assert_eq!(user_rows[0]["id"], "turn-native-1:row-0"); assert_eq!(user_rows[0]["turnId"], "turn-native-1:row-0"); assert_eq!(user_rows[0]["nativeTurnId"], "turn-native-1"); - assert!(turns - .iter() - .filter(|turn| turn["role"] != "user") - .all(|turn| turn.get("nativeTurnId").is_none())); + assert!( + turns + .iter() + .filter(|turn| turn["role"] != "user") + .all(|turn| turn.get("nativeTurnId").is_none()) + ); + let raw_thread = json!({"thread":{"turns":[raw_turn.clone()]}}); + assert_eq!( + checkpoint_codex_turn_identity(&raw_thread, "turn-native-1"), + Some(( + "turn-native-1".to_string(), + "turn-native-1:row-0".to_string() + )), + "the receipt identity comes from the unique provider-persisted user row" + ); + assert_eq!( + checkpoint_codex_turn_identity(&raw_thread, "some-other-turn"), + None, + "a snapshot row from another send cannot satisfy this request" + ); let ambiguous = json!({ "id": "turn-native-ambiguous", @@ -26682,6 +26827,152 @@ pub(crate) mod tests { 2 ); assert!(turns.iter().all(|turn| turn.get("nativeTurnId").is_none())); + assert_eq!( + checkpoint_codex_turn_identity( + &json!({"thread":{"turns":[ambiguous]}}), + "turn-native-ambiguous" + ), + None, + "an ambiguous multi-user provider turn cannot emit a native identity receipt" + ); + } + + #[tokio::test] + async fn checkpoint_identity_probe_retries_until_delayed_user_row_is_persisted() { + let (transport, peer) = freshell_codex::new_channel_transport(); + let (client, _notifs) = CodexAppServerClient::connect(transport); + let (state, mut rx) = state_with_bus(); + state.spawn_checkpoint_identity_probe( + Arc::new(client), + "thread-checkpoint-receipt".to_string(), + "turn-persisted-user".to_string(), + "request-checkpoint-1".to_string(), + "logical-checkpoint-1".to_string(), + ); + + let (initialize_id, initialize_method, _) = peer.expect_request().await; + assert_eq!(initialize_method, "initialize"); + peer.respond( + &initialize_id, + json!({ "userAgent": "test", "codexHome": "/h", "platformFamily": "u", "platformOs": "l" }), + ); + let _ = peer.expect_notification().await; + let (first_read_id, first_read_method, first_read_params) = peer.expect_request().await; + assert_eq!(first_read_method, "thread/read"); + assert_eq!(first_read_params["threadId"], "thread-checkpoint-receipt"); + assert_eq!(first_read_params["includeTurns"], true); + peer.respond( + &first_read_id, + json!({ + "thread": { + "id": "thread-checkpoint-receipt", + "status": "running", + "turns": [] + } + }), + ); + + let (read_id, read_method, read_params) = peer.expect_request().await; + assert_eq!(read_method, "thread/read"); + assert_eq!(read_params["threadId"], "thread-checkpoint-receipt"); + assert_eq!(read_params["includeTurns"], true); + peer.respond( + &read_id, + json!({ + "thread": { + "id": "thread-checkpoint-receipt", + "status": "idle", + "turns": [{ + "id": "turn-persisted-user", + "status": "completed", + "items": [ + { "id": "user-row", "type": "userMessage", "content": [ + { "type": "text", "text": "prompt is not copied into receipt" } + ] }, + { "id": "assistant-row", "type": "agentMessage", "text": "answer" } + ] + }] + } + }), + ); + + let frame = tokio::time::timeout(Duration::from_secs(1), rx.recv()) + .await + .expect("provider identity receipt was broadcast") + .expect("broadcast bus remains open"); + let frame: Value = serde_json::from_str(&frame).expect("receipt is JSON"); + assert_eq!(frame["type"], "freshAgent.event"); + assert_eq!(frame["provider"], "codex"); + assert_eq!(frame["sessionId"], "thread-checkpoint-receipt"); + assert_eq!(frame["event"]["type"], "checkpointTurnIdentity"); + assert_eq!(frame["event"]["requestId"], "request-checkpoint-1"); + assert_eq!(frame["event"]["logicalAttemptId"], "logical-checkpoint-1"); + assert_eq!(frame["event"]["nativeTurnId"], "turn-persisted-user"); + assert_eq!(frame["event"]["displayTurnId"], "turn-persisted-user:row-0"); + assert!( + frame["event"].get("text").is_none(), + "checkpoint identity receipts never carry prompt text" + ); + } + + #[tokio::test] + async fn checkpoint_identity_probe_retries_after_a_thread_read_error() { + let (transport, peer) = freshell_codex::new_channel_transport(); + let (client, _notifs) = CodexAppServerClient::connect(transport); + let (state, mut rx) = state_with_bus(); + state.spawn_checkpoint_identity_probe( + Arc::new(client), + "thread-checkpoint-read-retry".to_string(), + "turn-persisted-after-retry".to_string(), + "request-checkpoint-retry".to_string(), + "logical-checkpoint-retry".to_string(), + ); + + let (initialize_id, initialize_method, _) = peer.expect_request().await; + assert_eq!(initialize_method, "initialize"); + peer.respond( + &initialize_id, + json!({ "userAgent": "test", "codexHome": "/h", "platformFamily": "u", "platformOs": "l" }), + ); + let _ = peer.expect_notification().await; + let (first_read_id, first_read_method, _) = peer.expect_request().await; + assert_eq!(first_read_method, "thread/read"); + peer.respond_error(&first_read_id, -32000, "temporary history read failure"); + + let (second_read_id, second_read_method, _) = peer.expect_request().await; + assert_eq!(second_read_method, "thread/read"); + peer.respond( + &second_read_id, + json!({ + "thread": { + "id": "thread-checkpoint-read-retry", + "status": "idle", + "turns": [{ + "id": "turn-persisted-after-retry", + "status": "completed", + "items": [ + { "id": "user-row", "type": "userMessage", "content": [ + { "type": "text", "text": "prompt is excluded" } + ] }, + { "id": "assistant-row", "type": "agentMessage", "text": "answer" } + ] + }] + } + }), + ); + + let frame = tokio::time::timeout(Duration::from_secs(1), rx.recv()) + .await + .expect("retry emitted the provider identity receipt") + .expect("broadcast bus remains open"); + let frame: Value = serde_json::from_str(&frame).expect("receipt is JSON"); + assert_eq!(frame["event"]["type"], "checkpointTurnIdentity"); + assert_eq!(frame["event"]["requestId"], "request-checkpoint-retry"); + assert_eq!( + frame["event"]["logicalAttemptId"], + "logical-checkpoint-retry" + ); + assert_eq!(frame["event"]["nativeTurnId"], "turn-persisted-after-retry"); } #[test] @@ -28879,7 +29170,10 @@ mod provider_exec_tests { "managed fresh Codex must reject its actual full argv+env before spawn" ); } else { - assert!(result.is_ok(),"valid selected fresh Codex inputs must execute the actual app-server handshake"); + assert!( + result.is_ok(), + "valid selected fresh Codex inputs must execute the actual app-server handshake" + ); } return; } diff --git a/crates/freshell-protocol/src/client_messages.rs b/crates/freshell-protocol/src/client_messages.rs index a2ab717ca..4b432adf3 100644 --- a/crates/freshell-protocol/src/client_messages.rs +++ b/crates/freshell-protocol/src/client_messages.rs @@ -8,8 +8,9 @@ use serde_json::Value; use std::collections::BTreeMap; use crate::common::{ - double_option, AgentProvider, CodexDurability, PermissionMode, Sandbox, SessionLocator, - SessionType, Shell, StringOrNumber, TerminalAttachIntent, TerminalAttachPriority, + AgentProvider, CodexDurability, FreshAgentCheckpointIntent, PermissionMode, Sandbox, + SessionLocator, SessionType, Shell, StringOrNumber, TerminalAttachIntent, + TerminalAttachPriority, double_option, }; /// A message sent from a client to the server. @@ -989,6 +990,11 @@ pub struct FreshAgentSend { pub observed_epoch: Option<u64>, #[serde(skip_serializing_if = "Option::is_none")] pub observed_generation: Option<u64>, + /// Modern send/capture barrier identity. The server captures and settles + /// this attempt before dispatching the provider send; legacy frames omit + /// it and retain their existing behavior. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub checkpoint_intent: Option<FreshAgentCheckpointIntent>, } #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] diff --git a/crates/freshell-protocol/src/common.rs b/crates/freshell-protocol/src/common.rs index d349c8c9d..f7468691e 100644 --- a/crates/freshell-protocol/src/common.rs +++ b/crates/freshell-protocol/src/common.rs @@ -153,6 +153,32 @@ pub enum SessionType { Freshopencode, } +/// Immutable scope carried with one automatic pre-turn code checkpoint. +/// Provenance records how a non-Git directory became an explicit scope; the +/// server still resolves and validates the path before capture. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct FreshAgentCheckpointWorkspace { + pub path: String, + pub provenance: CheckpointWorkspaceProvenance, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum CheckpointWorkspaceProvenance { + DirectoryPicker, + ResolvedProject, + LegacyCheckpoint, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct FreshAgentCheckpointIntent { + pub logical_attempt_id: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub workspace: Option<FreshAgentCheckpointWorkspace>, +} + /// Sandbox policy shared by terminal and fresh-agent launch requests. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "kebab-case")] diff --git a/crates/freshell-protocol/src/server_messages.rs b/crates/freshell-protocol/src/server_messages.rs index e444f3eeb..30d0a8c9e 100644 --- a/crates/freshell-protocol/src/server_messages.rs +++ b/crates/freshell-protocol/src/server_messages.rs @@ -10,8 +10,8 @@ use std::collections::HashMap; use crate::common::{ AgentProvider, AmplifierActivityRecord, ClaudeActivityRecord, CodexActivityRecord, - CodexDurability, ErrorCode, OpencodeActivityRecord, SessionLocator, TerminalMetaRecord, - TurnCompletionSnapshot, + CodexDurability, ErrorCode, FreshAgentCheckpointWorkspace, OpencodeActivityRecord, + SessionLocator, SessionType, TerminalMetaRecord, TurnCompletionSnapshot, }; use crate::session_names::{SessionNameRecord, SessionNameRef, SessionNameUpdated}; use crate::settings::ServerSettings; @@ -66,6 +66,8 @@ pub enum ServerMessage { ExtensionsRegistry(ExtensionsRegistry), #[serde(rename = "freshAgent.create.failed")] FreshAgentCreateFailed(FreshAgentCreateFailed), + #[serde(rename = "freshAgent.checkpoint.outcome")] + FreshAgentCheckpointOutcome(FreshAgentCheckpointOutcome), #[serde(rename = "freshAgent.created")] FreshAgentCreated(FreshAgentCreated), #[serde(rename = "freshAgent.event")] @@ -207,7 +209,7 @@ pub enum ServerMessage { /// The exact `type` discriminants of every server→client message, in the frozen /// inventory's order. This is the T0 conformance checklist. -pub const SERVER_MESSAGE_TYPES: [&str; 69] = [ +pub const SERVER_MESSAGE_TYPES: [&str; 70] = [ "amplifier.activity.list.response", "amplifier.activity.updated", "claude.activity.list.response", @@ -226,6 +228,7 @@ pub const SERVER_MESSAGE_TYPES: [&str; 69] = [ "extension.server.starting", "extension.server.stopped", "extensions.registry", + "freshAgent.checkpoint.outcome", "freshAgent.create.failed", "freshAgent.created", "freshAgent.event", @@ -279,6 +282,40 @@ pub const SERVER_MESSAGE_TYPES: [&str; 69] = [ "ui.command", ]; +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum FreshAgentCheckpointStatus { + Available, + Unavailable, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct FreshAgentCheckpointOutcome { + pub logical_attempt_id: String, + pub request_id: String, + pub provider: AgentProvider, + pub session_id: String, + pub session_type: SessionType, + pub status: FreshAgentCheckpointStatus, + #[serde(skip_serializing_if = "Option::is_none")] + pub checkpoint_id: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub workspace: Option<FreshAgentCheckpointWorkspace>, + #[serde(skip_serializing_if = "Option::is_none")] + pub code: Option<String>, + #[serde(skip_serializing_if = "Option::is_none")] + pub reason: Option<String>, + /// Provider-native user row identity, added only after the provider confirms + /// the actual persisted/replayed row for this logical attempt. + #[serde(skip_serializing_if = "Option::is_none")] + pub native_turn_id: Option<String>, + /// Actual display-row key joined to `nativeTurnId`, when the provider + /// projection has materialized that row. + #[serde(skip_serializing_if = "Option::is_none")] + pub display_turn_id: Option<String>, +} + /// Supervisor startup-scan readiness carried by /// [`ServerMessage::RuntimeInventoryChanged`]. Mirrors /// `ManagedRuntimeInitialScanStateSchema` in `shared/managed-runtime.ts`. diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index 3fd72c348..fe403e92f 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -135,6 +135,7 @@ socket2 = { version = "0.6", features = ["all"] } hyper = "1" hyper-util = { version = "0.1", features = ["tokio", "server", "server-auto", "service"] } tower = { version = "0.5", features = ["util"] } +futures-util = "0.3" # The from-the-first-byte redaction scrub (DIAG-03): matches `"...token...":"…"` # and `"cookie":"…"` JSON fields (and a raw `Cookie:`/`Set-Cookie:` header # fragment, belt-and-braces) regardless of which call site produced them. @@ -152,6 +153,18 @@ unicode-normalization = "=0.1.25" # lock's own unix-only implementation. libc = "0.2" +[target.'cfg(windows)'.dependencies] +# Checkpoint Git children are started suspended, assigned to a kill-on-close +# Job Object, and only then resumed. This owns filters/hooks and other child +# processes through timeout settlement on native Windows. +windows-sys = { version = "=0.61.2", features = [ + "Win32_Foundation", + "Win32_Security", + "Win32_System_Diagnostics_ToolHelp", + "Win32_System_JobObjects", + "Win32_System_Threading", +] } + [target.'cfg(unix)'.dev-dependencies] freshell-supervisor = { path = "../freshell-supervisor" } @@ -168,7 +181,6 @@ rusqlite = { version = "0.31", features = ["bundled"] } # against the compiled binary (same version freshell-ws's own tests already # use, `crates/freshell-ws/Cargo.toml`) — no mock, the actual wire protocol. tokio-tungstenite = "0.24" -futures-util = "0.3" # NET-09: black-box config-preservation test requires reqwest with json encoding # (main [dependencies] reqwest is feature-gated for stream+rustls only). reqwest = { version = "0.13", default-features = false, features = ["json", "rustls"] } diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 422a21521..65f5ef5f5 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -23,10 +23,11 @@ use std::path::{Path, PathBuf}; use std::process::Stdio; -use std::sync::Arc; -use std::time::{SystemTime, UNIX_EPOCH}; +use std::sync::{Arc, Mutex as StdMutex, Weak}; +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; -use std::collections::HashMap; +use std::collections::{BTreeMap, HashMap, HashSet}; +use std::sync::atomic::{AtomicBool, Ordering}; use axum::routing::get as get_method; use axum::{ @@ -36,44 +37,2171 @@ use axum::{ response::{IntoResponse, Response}, routing::post, }; +use serde::{Deserialize, Serialize}; use serde_json::{Map, Value, json}; use sha1::{Digest, Sha1}; use tokio::io::{AsyncReadExt, AsyncWriteExt}; -use tokio::process::Command; +use tokio::process::{Child, Command}; +use tokio::sync::{Mutex, OwnedMutexGuard, watch}; + +use freshell_protocol::{ + AgentProvider, CheckpointWorkspaceProvenance, FreshAgentCheckpointOutcome, + FreshAgentCheckpointStatus, FreshAgentCheckpointWorkspace, FreshAgentSend, ServerMessage, + SessionType, +}; use crate::boot::{is_authed, unauthorized}; -/// `CHECKPOINT_LABEL_LIMIT` (`fresh-agent-extras-router.ts:349` inline `.slice(0, -/// 120)` on the label — mirrored here as a named constant for clarity, matching -/// the client's own `CHECKPOINT_LABEL_LIMIT` in `src/lib/fresh-agent-checkpoints.ts:6`). -/// NOTE: truncation here is by Unicode scalar value (Rust `char`), not by UTF-16 -/// code unit as JS `String.prototype.slice` counts — astral-plane characters -/// (rare in a checkpoint label, which is derived from the outgoing message text) -/// would truncate one char later here than in JS. Not fixed: a byte-for-byte port -/// of JS's UTF-16 indexing has no ergonomic Rust equivalent and the divergence is -/// unobservable for the overwhelming majority of real labels (ASCII/BMP text). -const CHECKPOINT_LABEL_LIMIT: usize = 120; +/// `CHECKPOINT_LABEL_LIMIT` (`fresh-agent-extras-router.ts:349` inline `.slice(0, +/// 120)` on the label — mirrored here as a named constant for clarity, matching +/// the client's own `CHECKPOINT_LABEL_LIMIT` in `src/lib/fresh-agent-checkpoints.ts:6`). +/// NOTE: truncation here is by Unicode scalar value (Rust `char`), not by UTF-16 +/// code unit as JS `String.prototype.slice` counts — astral-plane characters +/// (rare in a checkpoint label, which is derived from the outgoing message text) +/// would truncate one char later here than in JS. Not fixed: a byte-for-byte port +/// of JS's UTF-16 indexing has no ergonomic Rust equivalent and the divergence is +/// unobservable for the overwhelming majority of real labels (ASCII/BMP text). +const CHECKPOINT_LABEL_LIMIT: usize = 120; + +/// `CHECKPOINT_IDENTITY` (`fresh-agent-extras-router.ts:73`): the shadow repo's +/// commit author/committer identity, so a checkpoint commit never depends on (or +/// pollutes) the real user's global git config. +const CHECKPOINT_IDENTITY: [&str; 4] = [ + "-c", + "user.name=freshell", + "-c", + "user.email=checkpoints@freshell.local", +]; + +/// Shared state for the `/api/fresh-agent/checkpoints` route. +#[derive(Clone)] +pub struct CheckpointsApiState { + pub auth_token: Arc<String>, + /// The resolved home directory whose `.freshell/checkpoints/` holds every + /// shadow repo (mirrors `os.homedir()` in `checkpointGitDir`, + /// `fresh-agent-extras-router.ts:76`). `Arc`-shared, not re-resolved per + /// request, matching this crate's convention for boot-resolved paths (see + /// `main.rs`'s `session_metadata_dir`). + pub home: Arc<PathBuf>, + /// Same service instance used by the WebSocket local and managed send + /// paths, so REST capture, listing and restore share workspace locks and + /// durable attempt outcomes. + pub service: Arc<CheckpointService>, +} + +const CHECKPOINT_AUTO_BUDGET: Duration = Duration::from_secs(15); +const CHECKPOINT_EXPLICIT_BUDGET: Duration = Duration::from_secs(120); +const CHECKPOINT_TIMEOUT_COOLDOWN: Duration = Duration::from_secs(60); +const CHECKPOINT_COOLDOWN_LIMIT: usize = 128; +const CHECKPOINT_OUTCOME_STORE_VERSION: u8 = 1; +const CHECKPOINT_OUTCOME_STORE_LIMIT: usize = 8192; + +#[derive(Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +struct StoredCheckpointOutcome { + version: u8, + outcome: FreshAgentCheckpointOutcome, + request_ids: Vec<String>, + session_aliases: Vec<String>, + #[serde(default)] + dispatch_started: bool, + #[serde(default = "capture_settled_by_default")] + capture_settled: bool, + submitted_turn_id: Option<String>, + created_at_ms: u64, +} + +fn capture_settled_by_default() -> bool { + true +} + +#[derive(Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +struct CheckpointOutcomeStore { + version: u8, + outcomes: BTreeMap<String, StoredCheckpointOutcome>, +} + +struct CheckpointAttemptJob { + outcome: watch::Sender<Option<StoredCheckpointOutcome>>, + finished: watch::Sender<bool>, +} + +#[derive(Clone)] +struct CheckpointCooldown { + expires_at: Instant, +} + +struct CaptureRequestCancellation { + sender: watch::Sender<bool>, +} + +impl Drop for CaptureRequestCancellation { + fn drop(&mut self) { + self.sender.send_replace(true); + } +} + +enum ExplicitCaptureError { + WorkspaceUnavailable(CheckpointWorkspaceUnavailable), + ResolutionTimeout(String), + WorkspaceBusy(ResolvedCheckpointWorkspace), + Capture { + workspace: ResolvedCheckpointWorkspace, + message: String, + }, + Unsettled { + candidate: PathBuf, + workspace: Option<ResolvedCheckpointWorkspace>, + message: String, + }, + Cancelled, + ServiceStopped, +} + +async fn wait_for_capture_cancellation(receiver: &mut watch::Receiver<bool>) { + if !*receiver.borrow() { + let _ = receiver.changed().await; + } +} + +async fn cancel_and_settle_capture_task<T>( + control: &CheckpointCaptureControl, + task: &mut tokio::task::JoinHandle<T>, +) -> Result<(), String> { + if let Err(error) = control.cancel_processes().await { + tracing::error!( + event = "code_checkpoint.process_termination_failed", + cause = %error, + "checkpoint child termination was not confirmed" + ); + } + // Let the runner reap a killed direct child normally. If it is still in a + // filesystem future after its process groups/jobs are gone, abort that + // remaining future rather than letting it outlive the attempt deadline. + if tokio::time::timeout(Duration::from_secs(1), &mut *task) + .await + .is_err() + { + task.abort(); + let _ = task.await; + } + control + .wait_for_process_groups() + .await + .map_err(|error| error.to_string()) +} + +/// Shared checkpoint route/service state. Automatic WebSocket sends and the +/// explicit REST capture path use the same workspace serialization and +/// published shadow repository. Outcomes are persisted separately from the +/// checkpoint metadata patch, so a lost client connection can recover the +/// result after reattach. +pub struct CheckpointService { + home: Arc<PathBuf>, + broadcast_tx: Arc<tokio::sync::broadcast::Sender<String>>, + workspace_locks: StdMutex<HashMap<PathBuf, Weak<Mutex<()>>>>, + outcome_gate: Mutex<()>, + attempt_jobs: Mutex<HashMap<String, Arc<CheckpointAttemptJob>>>, + cooldowns: StdMutex<HashMap<PathBuf, CheckpointCooldown>>, + in_memory_outcomes: StdMutex<BTreeMap<String, StoredCheckpointOutcome>>, +} + +tokio::task_local! { + static CHECKPOINT_CAPTURE_CONTEXT: Arc<CheckpointCaptureControl>; +} + +#[derive(Default)] +struct CheckpointCaptureControl { + cancelled: AtomicBool, + #[cfg(unix)] + process_groups: StdMutex<HashSet<i32>>, + #[cfg(windows)] + jobs: StdMutex<Vec<Arc<WindowsJob>>>, + #[cfg(test)] + test_settlement_gate: StdMutex<Option<Arc<TestSettlementGate>>>, +} + +#[cfg(test)] +struct TestSettlementGate { + entered: tokio::sync::Notify, + release: tokio::sync::Notify, +} + +impl CheckpointCaptureControl { + fn spawn_child(self: &Arc<Self>, command: &mut Command) -> std::io::Result<CheckpointChild> { + #[cfg(unix)] + { + use std::os::unix::process::CommandExt; + let mut groups = self + .process_groups + .lock() + .expect("checkpoint process group set poisoned"); + if self.cancelled.load(Ordering::SeqCst) { + return Err(std::io::Error::new( + std::io::ErrorKind::Interrupted, + "checkpoint capture was cancelled", + )); + } + command.as_std_mut().process_group(0); + command.kill_on_drop(true); + let child = command.spawn()?; + let group = child + .id() + .and_then(|pid| i32::try_from(pid).ok()) + .ok_or_else(|| { + std::io::Error::other("checkpoint child process has no process group id") + })?; + groups.insert(group); + return Ok(CheckpointChild { + child: Some(child), + control: Some(Arc::clone(self)), + process_group: Some(group), + #[cfg(windows)] + job: None, + }); + } + #[cfg(windows)] + { + use std::os::windows::process::CommandExt; + use windows_sys::Win32::System::Threading::CREATE_SUSPENDED; + + let mut jobs = self.jobs.lock().expect("checkpoint job list poisoned"); + if self.cancelled.load(Ordering::SeqCst) { + return Err(std::io::Error::new( + std::io::ErrorKind::Interrupted, + "checkpoint capture was cancelled", + )); + } + let job = Arc::new(WindowsJob::new()?); + command.as_std_mut().creation_flags(CREATE_SUSPENDED); + command.kill_on_drop(true); + let mut child = command.spawn()?; + let process_id = child.id().ok_or_else(|| { + std::io::Error::other("checkpoint child process has no process id") + })?; + if let Err(error) = job.assign_suspended_process(process_id) { + job.terminate(); + let _ = child.start_kill(); + return Err(error); + } + jobs.push(Arc::clone(&job)); + Ok(CheckpointChild { + child: Some(child), + control: Some(Arc::clone(self)), + process_group: None, + job: Some(job), + }) + } + #[cfg(not(any(unix, windows)))] + { + if self.cancelled.load(Ordering::SeqCst) { + return Err(std::io::Error::new( + std::io::ErrorKind::Interrupted, + "checkpoint capture was cancelled", + )); + } + command.kill_on_drop(true); + let child = command.spawn()?; + Ok(CheckpointChild { + child: Some(child), + control: Some(Arc::clone(self)), + process_group: None, + #[cfg(windows)] + job: None, + }) + } + } + + async fn cancel_processes(&self) -> std::io::Result<()> { + self.cancelled.store(true, Ordering::SeqCst); + #[cfg(unix)] + { + let groups = self + .process_groups + .lock() + .expect("checkpoint process group set poisoned") + .iter() + .copied() + .collect::<Vec<_>>(); + for group in &groups { + signal_process_group(*group, libc::SIGTERM); + } + if !groups.is_empty() { + tokio::time::sleep(Duration::from_millis(100)).await; + } + for group in groups { + signal_process_group(group, libc::SIGKILL); + } + } + #[cfg(windows)] + { + let jobs = self + .jobs + .lock() + .expect("checkpoint job list poisoned") + .clone(); + for job in jobs { + job.terminate(); + } + } + Ok(()) + } + + async fn wait_for_process_groups(&self) -> std::io::Result<()> { + #[cfg(test)] + let test_gate = { + self.test_settlement_gate + .lock() + .expect("checkpoint test settlement gate poisoned") + .clone() + }; + #[cfg(test)] + if let Some(gate) = test_gate { + gate.entered.notify_one(); + gate.release.notified().await; + } + #[cfg(unix)] + { + let deadline = Instant::now() + Duration::from_secs(5); + loop { + let has_groups = { + let mut groups = self + .process_groups + .lock() + .expect("checkpoint process group set poisoned"); + groups.retain(|group| process_group_exists(*group)); + !groups.is_empty() + }; + if !has_groups { + return Ok(()); + } + if Instant::now() >= deadline { + return Err(std::io::Error::new( + std::io::ErrorKind::TimedOut, + "checkpoint process group did not settle within five seconds", + )); + } + tokio::time::sleep(Duration::from_millis(25)).await; + } + } + #[cfg(windows)] + { + let jobs = self + .jobs + .lock() + .expect("checkpoint job list poisoned") + .clone(); + for job in jobs { + job.settle().await?; + self.jobs + .lock() + .expect("checkpoint job list poisoned") + .retain(|candidate| !Arc::ptr_eq(candidate, &job)); + } + Ok(()) + } + #[cfg(not(any(unix, windows)))] + { + Ok(()) + } + } + + #[cfg(unix)] + async fn settle_process_group(&self, group: i32) -> std::io::Result<()> { + let deadline = Instant::now() + Duration::from_secs(5); + if process_group_exists(group) { + signal_process_group(group, libc::SIGTERM); + for _ in 0..4 { + if !process_group_exists(group) { + break; + } + tokio::time::sleep(Duration::from_millis(25)).await; + } + if process_group_exists(group) { + signal_process_group(group, libc::SIGKILL); + } + while process_group_exists(group) { + if Instant::now() >= deadline { + return Err(std::io::Error::new( + std::io::ErrorKind::TimedOut, + "checkpoint child process group did not settle within five seconds", + )); + } + tokio::time::sleep(Duration::from_millis(25)).await; + } + } + self.process_groups + .lock() + .expect("checkpoint process group set poisoned") + .remove(&group); + Ok(()) + } +} + +struct CheckpointChild { + child: Option<Child>, + control: Option<Arc<CheckpointCaptureControl>>, + process_group: Option<i32>, + #[cfg(windows)] + job: Option<Arc<WindowsJob>>, +} + +impl CheckpointChild { + async fn wait_with_output(mut self) -> std::io::Result<std::process::Output> { + let child = self + .child + .take() + .expect("checkpoint child must be present before wait") + .wait_with_output() + .await?; + self.settle().await?; + Ok(child) + } + + async fn terminate_and_wait(&mut self) { + if let Some(mut child) = self.child.take() { + let _ = child.kill().await; + let _ = child.wait().await; + } + let _ = self.settle().await; + } + + async fn settle(&mut self) -> std::io::Result<()> { + #[cfg(unix)] + if let (Some(control), Some(group)) = (&self.control, self.process_group) { + control.settle_process_group(group).await?; + } + #[cfg(windows)] + if let (Some(control), Some(job)) = (&self.control, &self.job) { + job.settle().await?; + control + .jobs + .lock() + .expect("checkpoint job list poisoned") + .retain(|candidate| !Arc::ptr_eq(candidate, job)); + } + Ok(()) + } +} + +fn checkpoint_child(command: &mut Command) -> std::io::Result<CheckpointChild> { + CHECKPOINT_CAPTURE_CONTEXT + .try_with(|control| control.spawn_child(command)) + .unwrap_or_else(|_| { + Err(std::io::Error::other( + "checkpoint Git process requires the server-owned capture runner", + )) + }) +} + +/// Non-capture checkpoint operations (listing, metadata lookup, and restore) +/// still use the same child wrapper, but do not have a capture supervisor. +/// Capture code is guarded by `create_checkpoint` and can only use the strict +/// `checkpoint_child` path above. +fn checkpoint_read_only_child(command: &mut Command) -> std::io::Result<CheckpointChild> { + match CHECKPOINT_CAPTURE_CONTEXT.try_with(|control| control.spawn_child(command)) { + Ok(result) => result, + Err(_) => { + command.kill_on_drop(true); + command.spawn().map(|child| CheckpointChild { + child: Some(child), + control: None, + process_group: None, + #[cfg(windows)] + job: None, + }) + } + } +} + +#[cfg(unix)] +fn signal_process_group(group: i32, signal: i32) { + // Negative PID targets the complete group, including Git filters/hooks. + let _ = unsafe { libc::kill(-group, signal) }; +} + +#[cfg(unix)] +fn process_group_exists(group: i32) -> bool { + let result = unsafe { libc::kill(-group, 0) }; + result == 0 || std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM) +} + +#[cfg(windows)] +struct WindowsJob { + handle: windows_sys::Win32::Foundation::HANDLE, +} + +#[cfg(windows)] +unsafe impl Send for WindowsJob {} +#[cfg(windows)] +unsafe impl Sync for WindowsJob {} + +#[cfg(windows)] +impl WindowsJob { + fn new() -> std::io::Result<Self> { + use windows_sys::Win32::System::JobObjects::{ + CreateJobObjectW, JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, + JOBOBJECT_EXTENDED_LIMIT_INFORMATION, JobObjectExtendedLimitInformation, + SetInformationJobObject, + }; + + let handle = unsafe { CreateJobObjectW(std::ptr::null(), std::ptr::null()) }; + if handle.is_null() { + return Err(std::io::Error::last_os_error()); + } + let mut limits = JOBOBJECT_EXTENDED_LIMIT_INFORMATION::default(); + limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE; + let configured = unsafe { + SetInformationJobObject( + handle, + JobObjectExtendedLimitInformation, + std::ptr::addr_of!(limits).cast(), + std::mem::size_of::<JOBOBJECT_EXTENDED_LIMIT_INFORMATION>() as u32, + ) + }; + if configured == 0 { + let error = std::io::Error::last_os_error(); + unsafe { windows_sys::Win32::Foundation::CloseHandle(handle) }; + return Err(error); + } + Ok(Self { handle }) + } + + fn assign_suspended_process(&self, process_id: u32) -> std::io::Result<()> { + use windows_sys::Win32::Foundation::CloseHandle; + use windows_sys::Win32::System::JobObjects::AssignProcessToJobObject; + use windows_sys::Win32::System::Threading::{ + OpenProcess, PROCESS_SET_QUOTA, PROCESS_TERMINATE, + }; + + let process = unsafe { OpenProcess(PROCESS_SET_QUOTA | PROCESS_TERMINATE, 0, process_id) }; + if process.is_null() { + return Err(std::io::Error::last_os_error()); + } + let assigned = unsafe { AssignProcessToJobObject(self.handle, process) }; + let assign_error = (assigned == 0).then(std::io::Error::last_os_error); + unsafe { CloseHandle(process) }; + if let Some(error) = assign_error { + return Err(error); + } + + let thread_id = find_process_thread(process_id)?; + let thread = unsafe { + windows_sys::Win32::System::Threading::OpenThread( + windows_sys::Win32::System::Threading::THREAD_SUSPEND_RESUME, + 0, + thread_id, + ) + }; + if thread.is_null() { + return Err(std::io::Error::last_os_error()); + } + let resumed = unsafe { windows_sys::Win32::System::Threading::ResumeThread(thread) }; + unsafe { CloseHandle(thread) }; + if resumed == u32::MAX { + return Err(std::io::Error::last_os_error()); + } + Ok(()) + } + + fn active_processes(&self) -> std::io::Result<u32> { + use windows_sys::Win32::System::JobObjects::{ + JOBOBJECT_BASIC_ACCOUNTING_INFORMATION, JobObjectBasicAccountingInformation, + QueryInformationJobObject, + }; + + let mut accounting = JOBOBJECT_BASIC_ACCOUNTING_INFORMATION::default(); + let queried = unsafe { + QueryInformationJobObject( + self.handle, + JobObjectBasicAccountingInformation, + std::ptr::addr_of_mut!(accounting).cast(), + std::mem::size_of::<JOBOBJECT_BASIC_ACCOUNTING_INFORMATION>() as u32, + std::ptr::null_mut(), + ) + }; + if queried == 0 { + return Err(std::io::Error::last_os_error()); + } + Ok(accounting.ActiveProcesses) + } + + fn terminate(&self) { + let terminated = + unsafe { windows_sys::Win32::System::JobObjects::TerminateJobObject(self.handle, 1) }; + if terminated == 0 { + tracing::error!( + event = "code_checkpoint.job_termination_failed", + cause = %std::io::Error::last_os_error(), + "checkpoint child job could not be terminated" + ); + } + } + + async fn settle(&self) -> std::io::Result<()> { + let active_processes = match self.active_processes() { + Ok(active_processes) => active_processes, + Err(error) => { + tracing::error!( + event = "code_checkpoint.job_process_query_failed", + cause = %error, + "checkpoint child job activity could not be queried; terminating it" + ); + u32::MAX + } + }; + if active_processes > 0 { + self.terminate(); + } else { + return Ok(()); + } + let handle = self.handle as usize; + let wait_result = tokio::task::spawn_blocking(move || unsafe { + windows_sys::Win32::System::Threading::WaitForSingleObject( + handle as windows_sys::Win32::Foundation::HANDLE, + 5_000, + ) + }) + .await + .map_err(|error| { + std::io::Error::other(format!("checkpoint job wait task failed: {error}")) + })?; + match wait_result { + windows_sys::Win32::Foundation::WAIT_OBJECT_0 => Ok(()), + windows_sys::Win32::Foundation::WAIT_TIMEOUT => Err(std::io::Error::new( + std::io::ErrorKind::TimedOut, + "checkpoint job did not settle within five seconds", + )), + windows_sys::Win32::Foundation::WAIT_FAILED => Err(std::io::Error::last_os_error()), + status => Err(std::io::Error::other(format!( + "checkpoint job wait returned unexpected status {status}" + ))), + } + } +} + +#[cfg(windows)] +impl Drop for WindowsJob { + fn drop(&mut self) { + unsafe { windows_sys::Win32::Foundation::CloseHandle(self.handle) }; + } +} + +#[cfg(windows)] +fn find_process_thread(process_id: u32) -> std::io::Result<u32> { + use windows_sys::Win32::Foundation::{CloseHandle, INVALID_HANDLE_VALUE}; + use windows_sys::Win32::System::Diagnostics::ToolHelp::{ + CreateToolhelp32Snapshot, TH32CS_SNAPTHREAD, THREADENTRY32, Thread32First, Thread32Next, + }; + + let snapshot = unsafe { CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0) }; + if snapshot == INVALID_HANDLE_VALUE { + return Err(std::io::Error::last_os_error()); + } + let mut entry = THREADENTRY32 { + dwSize: std::mem::size_of::<THREADENTRY32>() as u32, + ..Default::default() + }; + let mut found = None; + let mut has_entry = unsafe { Thread32First(snapshot, &mut entry) }; + while has_entry != 0 { + if entry.th32OwnerProcessID == process_id { + found = Some(entry.th32ThreadID); + break; + } + has_entry = unsafe { Thread32Next(snapshot, &mut entry) }; + } + unsafe { CloseHandle(snapshot) }; + found.ok_or_else(|| std::io::Error::other("suspended Git process thread was not found")) +} + +impl CheckpointService { + pub fn new( + home: Arc<PathBuf>, + broadcast_tx: Arc<tokio::sync::broadcast::Sender<String>>, + ) -> Self { + Self { + home, + broadcast_tx, + workspace_locks: StdMutex::new(HashMap::new()), + outcome_gate: Mutex::new(()), + attempt_jobs: Mutex::new(HashMap::new()), + cooldowns: StdMutex::new(HashMap::new()), + in_memory_outcomes: StdMutex::new(BTreeMap::new()), + } + } + + pub fn start_observer(self: &Arc<Self>) { + let mut receiver = self.broadcast_tx.subscribe(); + let service = Arc::downgrade(self); + tokio::spawn(async move { + loop { + match receiver.recv().await { + Ok(frame) => { + let Some(service) = service.upgrade() else { + break; + }; + service.observe_broadcast_frame(&frame).await; + } + Err(tokio::sync::broadcast::error::RecvError::Lagged(count)) => { + tracing::warn!( + event = "code_checkpoint.outcome_observer_lagged", + skipped_frames = count, + "checkpoint outcome observer skipped broadcast frames" + ); + } + Err(tokio::sync::broadcast::error::RecvError::Closed) => break, + } + } + }); + } + + fn outcome_store_path(&self) -> PathBuf { + self.home + .join(".freshell") + .join("checkpoints") + .join("attempt-outcomes-v1.json") + } + + fn workspace_lock(&self, workspace: &Path) -> Arc<Mutex<()>> { + let key = resolve_absolute(workspace); + let mut locks = self + .workspace_locks + .lock() + .expect("checkpoint workspace lock map poisoned"); + locks.retain(|_, lock| lock.strong_count() > 0); + if let Some(lock) = locks.get(&key).and_then(Weak::upgrade) { + return lock; + } + let lock = Arc::new(Mutex::new(())); + locks.insert(key, Arc::downgrade(&lock)); + lock + } + + async fn lock_workspace( + &self, + workspace: &Path, + budget: Duration, + ) -> Result<OwnedMutexGuard<()>, ()> { + tokio::time::timeout(budget, self.workspace_lock(workspace).lock_owned()) + .await + .map_err(|_| ()) + } + + fn quarantine_workspace( + &self, + workspace: PathBuf, + lock: OwnedMutexGuard<()>, + control: Arc<CheckpointCaptureControl>, + request_id: Option<String>, + stage: &'static str, + initial_cause: String, + ) -> tokio::task::JoinHandle<()> { + tokio::spawn(async move { + // Keep the lock alive for the whole retry loop. A caller must never + // release workspace serialization while a child may still mutate + // the shadow repository. + let _workspace_lock = lock; + tracing::error!( + event = "code_checkpoint.workspace_quarantined", + workspace = %workspace.display(), + request_id = request_id.as_deref().unwrap_or(""), + stage, + cause = %initial_cause, + "checkpoint child settlement is uncertain; workspace remains locked" + ); + let mut retry = 0_u64; + loop { + retry = retry.saturating_add(1); + if let Err(error) = control.cancel_processes().await { + tracing::error!( + event = "code_checkpoint.quarantine_termination_failed", + workspace = %workspace.display(), + request_id = request_id.as_deref().unwrap_or(""), + stage, + retry, + cause = %error, + "checkpoint process termination retry failed" + ); + } + match control.wait_for_process_groups().await { + Ok(()) => { + tracing::warn!( + event = "code_checkpoint.workspace_quarantine_released", + workspace = %workspace.display(), + request_id = request_id.as_deref().unwrap_or(""), + stage, + retry, + "checkpoint children settled; workspace lock released" + ); + break; + } + Err(error) => { + tracing::error!( + event = "code_checkpoint.quarantine_settlement_pending", + workspace = %workspace.display(), + request_id = request_id.as_deref().unwrap_or(""), + stage, + retry, + cause = %error, + "checkpoint workspace remains locked until child settlement is confirmed" + ); + tokio::time::sleep(Duration::from_secs(1)).await; + } + } + } + }) + } + + async fn read_outcome_store(&self) -> Result<CheckpointOutcomeStore, String> { + let path = self.outcome_store_path(); + let raw = match tokio::fs::read(&path).await { + Ok(raw) => raw, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return Ok(CheckpointOutcomeStore { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcomes: BTreeMap::new(), + }); + } + Err(error) => { + return Err(format!( + "failed to read checkpoint attempt outcomes: {error}" + )); + } + }; + let store: CheckpointOutcomeStore = serde_json::from_slice(&raw) + .map_err(|error| format!("invalid checkpoint attempt outcome store: {error}"))?; + if store.version != CHECKPOINT_OUTCOME_STORE_VERSION { + return Err("unsupported checkpoint attempt outcome store version".to_string()); + } + Ok(store) + } + + async fn write_outcome_store(&self, store: &CheckpointOutcomeStore) -> Result<(), String> { + let path = self.outcome_store_path(); + let parent = path + .parent() + .ok_or_else(|| "checkpoint outcome store has no parent directory".to_string())?; + tokio::fs::create_dir_all(parent) + .await + .map_err(|error| format!("failed to create checkpoint outcome directory: {error}"))?; + let temporary = parent.join(format!( + "attempt-outcomes-v1.json.tmp-{}", + uuid::Uuid::new_v4() + )); + let serialized = serde_json::to_vec_pretty(store) + .map_err(|error| format!("failed to serialize checkpoint outcomes: {error}"))?; + #[cfg(unix)] + let write_result = { + let mut options = tokio::fs::OpenOptions::new(); + options.write(true).create_new(true).mode(0o600); + match options.open(&temporary).await { + Ok(mut file) => match file.write_all(&serialized).await { + Ok(()) => file + .sync_all() + .await + .map_err(|error| format!("failed to flush checkpoint outcomes: {error}")), + Err(error) => Err(format!("failed to write checkpoint outcomes: {error}")), + }, + Err(error) => Err(format!("failed to create checkpoint outcome file: {error}")), + } + }; + #[cfg(not(unix))] + let write_result = tokio::fs::write(&temporary, &serialized) + .await + .map_err(|error| format!("failed to write checkpoint outcomes: {error}")); + if let Err(error) = write_result { + let _ = tokio::fs::remove_file(&temporary).await; + return Err(error); + } + if let Err(error) = tokio::fs::rename(&temporary, &path).await { + let _ = tokio::fs::remove_file(&temporary).await; + return Err(format!("failed to publish checkpoint outcomes: {error}")); + } + Ok(()) + } + + async fn persist_outcome( + &self, + key: &str, + outcome: StoredCheckpointOutcome, + ) -> Result<(), String> { + let _guard = self.outcome_gate.lock().await; + let mut store = self.read_outcome_store().await?; + store.outcomes.insert(key.to_string(), outcome.clone()); + while store.outcomes.len() > CHECKPOINT_OUTCOME_STORE_LIMIT { + let oldest = store + .outcomes + .iter() + .min_by_key(|(_, item)| item.created_at_ms) + .map(|(key, _)| key.clone()); + let Some(oldest) = oldest else { break }; + store.outcomes.remove(&oldest); + } + self.write_outcome_store(&store).await?; + self.in_memory_outcomes + .lock() + .expect("checkpoint outcome memory cache poisoned") + .insert(key.to_string(), outcome); + Ok(()) + } + + async fn find_outcome(&self, key: &str) -> Result<Option<StoredCheckpointOutcome>, String> { + if let Some(outcome) = self + .in_memory_outcomes + .lock() + .expect("checkpoint outcome memory cache poisoned") + .get(key) + .cloned() + { + return Ok(Some(outcome)); + } + let _guard = self.outcome_gate.lock().await; + let store = self.read_outcome_store().await?; + let outcome = store.outcomes.get(key).cloned(); + if let Some(outcome) = &outcome { + self.in_memory_outcomes + .lock() + .expect("checkpoint outcome memory cache poisoned") + .insert(key.to_string(), outcome.clone()); + } + Ok(outcome) + } + + async fn replay_for_identity( + &self, + provider: AgentProvider, + session_id: &str, + session_type: SessionType, + ) { + let records = match self.read_outcome_store().await { + Ok(store) => store.outcomes, + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_store_read_failed", + cause = %error, + "durable checkpoint outcomes could not be replayed" + ); + self.in_memory_outcomes + .lock() + .expect("checkpoint outcome memory cache poisoned") + .clone() + } + }; + for record in records.values().filter(|record| { + record.outcome.provider == provider + && record.outcome.session_type == session_type + && record + .session_aliases + .iter() + .any(|alias| alias == session_id) + }) { + if record.request_ids.is_empty() { + self.broadcast_outcome(record.outcome.clone()); + } else { + for request_id in &record.request_ids { + let mut outcome = record.outcome.clone(); + outcome.request_id = request_id.clone(); + self.broadcast_outcome(outcome); + } + } + } + } + + fn broadcast_outcome(&self, outcome: FreshAgentCheckpointOutcome) { + let message = ServerMessage::FreshAgentCheckpointOutcome(outcome); + if let Ok(frame) = serde_json::to_string(&message) { + let _ = self.broadcast_tx.send(frame); + } + } + + async fn observe_broadcast_frame(&self, frame: &str) { + let Ok(message) = serde_json::from_str::<Value>(frame) else { + return; + }; + match message.get("type").and_then(Value::as_str) { + Some("freshAgent.send.accepted") => self.observe_accepted(&message).await, + Some("freshAgent.session.materialized") => self.observe_materialized(&message).await, + Some("freshAgent.event") => self.observe_turn_identity(&message).await, + _ => {} + } + } + + async fn update_matching_outcomes<Matches, Update>( + &self, + mut matches: Matches, + mut update: Update, + ) -> Result<Vec<StoredCheckpointOutcome>, String> + where + Matches: FnMut(&StoredCheckpointOutcome) -> bool, + Update: FnMut(&mut StoredCheckpointOutcome), + { + let _guard = self.outcome_gate.lock().await; + let mut store = self.read_outcome_store().await?; + let mut changed = Vec::new(); + for record in store.outcomes.values_mut() { + if matches(record) { + update(record); + changed.push(record.clone()); + } + } + if !changed.is_empty() { + self.write_outcome_store(&store).await?; + let mut cache = self + .in_memory_outcomes + .lock() + .expect("checkpoint outcome memory cache poisoned"); + for (key, record) in &store.outcomes { + cache.insert(key.clone(), record.clone()); + } + } + Ok(changed) + } + + async fn observe_accepted(&self, message: &Value) { + let Some(request_id) = message.get("requestId").and_then(Value::as_str) else { + return; + }; + let Some(provider) = message + .get("provider") + .and_then(Value::as_str) + .and_then(parse_agent_provider) + else { + return; + }; + let Some(session_type) = message + .get("sessionType") + .and_then(Value::as_str) + .and_then(parse_session_type) + else { + return; + }; + let Some(session_id) = message.get("sessionId").and_then(Value::as_str) else { + return; + }; + let submitted_turn_id = message + .get("submittedTurnId") + .and_then(Value::as_str) + .map(str::to_string); + let updated = self + .update_matching_outcomes( + |record| { + record.outcome.provider == provider + && record.outcome.session_type == session_type + && record.request_ids.iter().any(|alias| alias == request_id) + && record + .session_aliases + .iter() + .any(|alias| alias == session_id) + }, + |record| record.submitted_turn_id = submitted_turn_id.clone(), + ) + .await; + if let Err(error) = updated { + tracing::error!( + event = "code_checkpoint.accepted_identity_persist_failed", + request_id, + cause = %error, + "accepted turn identity could not be persisted" + ); + } + } + + async fn observe_materialized(&self, message: &Value) { + let Some(previous) = message.get("previousSessionId").and_then(Value::as_str) else { + return; + }; + let Some(session_id) = message.get("sessionId").and_then(Value::as_str) else { + return; + }; + let Some(provider) = message + .get("provider") + .and_then(Value::as_str) + .and_then(parse_agent_provider) + else { + return; + }; + let Some(session_type) = message + .get("sessionType") + .and_then(Value::as_str) + .and_then(parse_session_type) + else { + return; + }; + let updated = self + .update_matching_outcomes( + |record| { + record.outcome.provider == provider + && record.outcome.session_type == session_type + && record.session_aliases.iter().any(|alias| alias == previous) + }, + |record| { + if !record + .session_aliases + .iter() + .any(|alias| alias == session_id) + { + record.session_aliases.push(session_id.to_string()); + } + }, + ) + .await; + if let Err(error) = updated { + tracing::error!( + event = "code_checkpoint.session_alias_persist_failed", + cause = %error, + "checkpoint session materialization alias could not be persisted" + ); + } + } + + async fn observe_turn_identity(&self, message: &Value) { + let Some(event) = message.get("event") else { + return; + }; + if event.get("type").and_then(Value::as_str) != Some("checkpointTurnIdentity") { + return; + } + let Some(logical_attempt_id) = event.get("logicalAttemptId").and_then(Value::as_str) else { + return; + }; + let Some(request_id) = event.get("requestId").and_then(Value::as_str) else { + return; + }; + let Some(native_turn_id) = event.get("nativeTurnId").and_then(Value::as_str) else { + return; + }; + let session_id = event + .get("sessionId") + .and_then(Value::as_str) + .or_else(|| message.get("sessionId").and_then(Value::as_str)); + let Some(session_id) = session_id else { return }; + let display_turn_id = event + .get("displayTurnId") + .and_then(Value::as_str) + .map(str::to_string); + let Some(provider) = message + .get("provider") + .and_then(Value::as_str) + .and_then(parse_agent_provider) + else { + return; + }; + let Some(session_type) = message + .get("sessionType") + .and_then(Value::as_str) + .and_then(parse_session_type) + else { + return; + }; + let updated = self + .update_matching_outcomes( + |record| { + record.outcome.logical_attempt_id == logical_attempt_id + && record.outcome.provider == provider + && record.outcome.session_type == session_type + && record.request_ids.iter().any(|alias| alias == request_id) + && record + .session_aliases + .iter() + .any(|alias| alias == session_id) + && record + .outcome + .native_turn_id + .as_deref() + .is_none_or(|existing| existing == native_turn_id) + }, + |record| { + record.outcome.native_turn_id = Some(native_turn_id.to_string()); + if display_turn_id.is_some() { + record.outcome.display_turn_id = display_turn_id.clone(); + } + }, + ) + .await; + match updated { + Ok(records) => { + for record in records { + if record.request_ids.is_empty() { + self.broadcast_outcome(record.outcome.clone()); + } else { + for request_id in &record.request_ids { + let mut outcome = record.outcome.clone(); + outcome.request_id = request_id.clone(); + self.broadcast_outcome(outcome); + } + } + } + } + Err(error) => tracing::error!( + event = "code_checkpoint.turn_identity_persist_failed", + logical_attempt_id, + cause = %error, + "provider checkpoint turn identity could not be persisted" + ), + } + } -/// `CHECKPOINT_IDENTITY` (`fresh-agent-extras-router.ts:73`): the shadow repo's -/// commit author/committer identity, so a checkpoint commit never depends on (or -/// pollutes) the real user's global git config. -const CHECKPOINT_IDENTITY: [&str; 4] = [ - "-c", - "user.name=freshell", - "-c", - "user.email=checkpoints@freshell.local", -]; + fn active_cooldown(&self, workspace: &Path) -> Option<Duration> { + let now = Instant::now(); + let mut cooldowns = self + .cooldowns + .lock() + .expect("checkpoint cooldown map poisoned"); + cooldowns.retain(|_, cooldown| cooldown.expires_at > now); + cooldowns + .get(workspace) + .map(|cooldown| cooldown.expires_at.saturating_duration_since(now)) + } -/// Shared state for the `/api/fresh-agent/checkpoints` route. -#[derive(Clone)] -pub struct CheckpointsApiState { - pub auth_token: Arc<String>, - /// The resolved home directory whose `.freshell/checkpoints/` holds every - /// shadow repo (mirrors `os.homedir()` in `checkpointGitDir`, - /// `fresh-agent-extras-router.ts:76`). `Arc`-shared, not re-resolved per - /// request, matching this crate's convention for boot-resolved paths (see - /// `main.rs`'s `session_metadata_dir`). - pub home: Arc<PathBuf>, + fn establish_cooldown(&self, workspace: &Path) { + let now = Instant::now(); + let mut cooldowns = self + .cooldowns + .lock() + .expect("checkpoint cooldown map poisoned"); + cooldowns.retain(|_, cooldown| cooldown.expires_at > now); + cooldowns.insert( + workspace.to_path_buf(), + CheckpointCooldown { + expires_at: now + CHECKPOINT_TIMEOUT_COOLDOWN, + }, + ); + while cooldowns.len() > CHECKPOINT_COOLDOWN_LIMIT { + let oldest = cooldowns + .iter() + .min_by_key(|(_, cooldown)| cooldown.expires_at) + .map(|(path, _)| path.clone()); + let Some(oldest) = oldest else { break }; + cooldowns.remove(&oldest); + } + } + + fn clear_cooldown(&self, workspace: &Path) { + self.cooldowns + .lock() + .expect("checkpoint cooldown map poisoned") + .remove(workspace); + } + + fn new_record( + &self, + send: &FreshAgentSend, + logical_attempt_id: &str, + request_id: &str, + workspace: Option<(&ResolvedCheckpointWorkspace, &str)>, + result: Result<String, (&'static str, String)>, + ) -> StoredCheckpointOutcome { + let (status, checkpoint_id, code, reason, resolved_workspace) = match result { + Ok(checkpoint_id) => ( + FreshAgentCheckpointStatus::Available, + Some(checkpoint_id), + None, + None, + workspace.map(|(workspace, _)| FreshAgentCheckpointWorkspace { + path: workspace.path.to_string_lossy().into_owned(), + provenance: workspace_provenance(&workspace.provenance), + }), + ), + Err((code, reason)) => ( + FreshAgentCheckpointStatus::Unavailable, + None, + Some(code.to_string()), + Some(reason), + workspace.map(|(workspace, _)| FreshAgentCheckpointWorkspace { + path: workspace.path.to_string_lossy().into_owned(), + provenance: workspace_provenance(&workspace.provenance), + }), + ), + }; + let mut request_ids = vec![request_id.to_string()]; + request_ids.truncate(16); + let mut session_aliases = vec![send.session_id.clone()]; + session_aliases.truncate(16); + StoredCheckpointOutcome { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcome: FreshAgentCheckpointOutcome { + logical_attempt_id: logical_attempt_id.to_string(), + request_id: request_id.to_string(), + provider: send.provider, + session_id: send.session_id.clone(), + session_type: send.session_type, + status, + checkpoint_id, + workspace: resolved_workspace, + code, + reason, + native_turn_id: None, + display_turn_id: None, + }, + request_ids, + session_aliases, + dispatch_started: false, + capture_settled: true, + submitted_turn_id: None, + created_at_ms: SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap_or_default() + .as_millis() as u64, + } + } + + async fn capture_automatic( + &self, + send: &FreshAgentSend, + request_id: &str, + logical_attempt_id: &str, + ) -> StoredCheckpointOutcome { + let started_at = Instant::now(); + let launch_cwd = send + .settings + .as_ref() + .and_then(|settings| settings.cwd.as_deref()) + .or(send.cwd.as_deref()); + let Some(launch_cwd) = launch_cwd else { + return self.new_record( + send, + logical_attempt_id, + request_id, + None, + Err(( + "workspace_unavailable", + "No selected code workspace is available for this send.".into(), + )), + ); + }; + let intent = send + .checkpoint_intent + .as_ref() + .and_then(|intent| serde_json::to_value(intent.workspace.as_ref()).ok()) + .and_then(|value| if value.is_null() { None } else { Some(value) }); + let resolve_control = Arc::new(CheckpointCaptureControl::default()); + let resolve_control_for_task = Arc::clone(&resolve_control); + let home = Arc::clone(&self.home); + let launch_cwd = launch_cwd.to_string(); + let mut resolve_task = tokio::spawn(CHECKPOINT_CAPTURE_CONTEXT.scope( + resolve_control_for_task, + async move { + resolve_checkpoint_workspace(&home, Path::new(&launch_cwd), intent.as_ref()).await + }, + )); + let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); + let workspace = match tokio::time::timeout(remaining, &mut resolve_task).await { + Ok(Ok(Ok(workspace))) => workspace, + Ok(Ok(Err(error))) => { + log_workspace_unavailable(&error.candidate, &error.reason, Some(request_id)); + return self.new_record( + send, + logical_attempt_id, + request_id, + None, + Err(( + "workspace_unavailable", + "No eligible code workspace is available for this send.".into(), + )), + ); + } + Ok(Err(error)) => { + return self.new_record( + send, + logical_attempt_id, + request_id, + None, + Err(( + "workspace_resolution_failed", + format!("Workspace resolution failed; your message will still be sent ({error})."), + )), + ); + } + Err(_) => { + let settlement = + cancel_and_settle_capture_task(&resolve_control, &mut resolve_task).await; + let mut record = self.new_record( + send, + logical_attempt_id, + request_id, + None, + Err(match &settlement { + Ok(()) => ( + "workspace_resolution_timeout", + "Workspace resolution took too long; your message will still be sent.".into(), + ), + Err(cause) => ( + "workspace_resolution_unsettled", + format!("Workspace resolution timed out and its child process could not be confirmed settled; the send was withheld ({cause})."), + ), + }), + ); + record.capture_settled = settlement.is_ok(); + return record; + } + }; + if let Some(remaining) = self.active_cooldown(&workspace.path) { + return self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(( + "capture_timeout_cooldown", + format!("Code snapshot capture is paused briefly after a timeout; retry in {} seconds.", remaining.as_secs().saturating_add(1)), + )), + ); + } + + let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); + if remaining.is_zero() { + return self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(("workspace_resolution_timeout", "Workspace resolution used the available snapshot time; your message will still be sent.".into())), + ); + } + let lock = match self.lock_workspace(&workspace.path, remaining).await { + Ok(lock) => lock, + Err(()) => { + return self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(("workspace_lock_timeout", "This workspace is busy with another snapshot; your message will still be sent.".into())), + ); + } + }; + let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); + if remaining.is_zero() { + drop(lock); + return self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(("workspace_lock_timeout", "This workspace is busy with another snapshot; your message will still be sent.".into())), + ); + } + + match self + .capture_with_budget( + &workspace, + "Automatic pre-turn snapshot", + request_id, + remaining, + None, + ) + .await + { + CaptureAttempt::Published(checkpoint_id) => { + self.clear_cooldown(&workspace.path); + drop(lock); + self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Ok(checkpoint_id), + ) + } + CaptureAttempt::TimedOut(published) => { + self.establish_cooldown(&workspace.path); + drop(lock); + if let Some(checkpoint_id) = published { + self.clear_cooldown(&workspace.path); + self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Ok(checkpoint_id), + ) + } else { + self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(( + "capture_timeout", + "The code snapshot took too long; your message will still be sent." + .into(), + )), + ) + } + } + CaptureAttempt::Cancelled(published) => { + drop(lock); + if let Some(checkpoint_id) = published { + self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Ok(checkpoint_id), + ) + } else { + self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(( + "capture_cancelled", + "The code snapshot was cancelled; your message will still be sent." + .into(), + )), + ) + } + } + CaptureAttempt::Unsettled { + message: error, + control, + } => { + let mut record = self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err(( + "capture_unsettled", + format!("The code snapshot could not be confirmed settled; the send was withheld ({error})."), + )), + ); + record.capture_settled = false; + let _quarantine = self.quarantine_workspace( + workspace.path.clone(), + lock, + control, + Some(request_id.to_string()), + "automatic_capture", + error, + ); + record + } + CaptureAttempt::Failed(error) => { + drop(lock); + let failure = + CheckpointFailure::from_capture_detail(&self.home, &workspace.path, &error); + failure.log(Some(request_id)); + self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Err((failure.code, failure.user_details.to_string())), + ) + } + } + } + + async fn capture_explicit( + self: &Arc<Self>, + launch_cwd: PathBuf, + workspace_intent: Option<Value>, + label: String, + request_id: Option<String>, + ) -> Result<(Value, ResolvedCheckpointWorkspace), ExplicitCaptureError> { + let deadline = Instant::now() + CHECKPOINT_EXPLICIT_BUDGET; + let (cancel_tx, cancel_rx) = watch::channel(false); + let cancellation_guard = CaptureRequestCancellation { sender: cancel_tx }; + let service = Arc::clone(self); + let (settled_tx, settled_rx) = tokio::sync::oneshot::channel::< + Result<(Value, ResolvedCheckpointWorkspace), ExplicitCaptureError>, + >(); + tokio::spawn(async move { + let result = service + .capture_explicit_supervisor( + launch_cwd, + workspace_intent, + label, + request_id, + deadline, + cancel_rx, + ) + .await; + let _ = settled_tx.send(result); + }); + let result = settled_rx + .await + .unwrap_or(Err(ExplicitCaptureError::ServiceStopped)); + drop(cancellation_guard); + result + } + + async fn capture_explicit_supervisor( + &self, + launch_cwd: PathBuf, + workspace_intent: Option<Value>, + label: String, + request_id: Option<String>, + deadline: Instant, + mut cancel_rx: watch::Receiver<bool>, + ) -> Result<(Value, ResolvedCheckpointWorkspace), ExplicitCaptureError> { + let resolve_control = Arc::new(CheckpointCaptureControl::default()); + let resolve_control_for_task = Arc::clone(&resolve_control); + let launch_cwd_for_task = launch_cwd.clone(); + let home = Arc::clone(&self.home); + let mut resolve_task = tokio::spawn(CHECKPOINT_CAPTURE_CONTEXT.scope( + resolve_control_for_task, + async move { + resolve_checkpoint_workspace(&home, &launch_cwd_for_task, workspace_intent.as_ref()) + .await + }, + )); + let remaining = deadline.saturating_duration_since(Instant::now()); + if remaining.is_zero() { + let candidate = launch_cwd.clone(); + return match cancel_and_settle_capture_task(&resolve_control, &mut resolve_task).await { + Ok(()) => Err(ExplicitCaptureError::ResolutionTimeout( + candidate.to_string_lossy().into_owned(), + )), + Err(message) => Err(ExplicitCaptureError::Unsettled { + candidate, + workspace: None, + message, + }), + }; + } + + let workspace = tokio::select! { + biased; + _ = wait_for_capture_cancellation(&mut cancel_rx) => { + let candidate = launch_cwd.clone(); + return match cancel_and_settle_capture_task(&resolve_control, &mut resolve_task).await { + Ok(()) => Err(ExplicitCaptureError::Cancelled), + Err(message) => Err(ExplicitCaptureError::Unsettled { + candidate, + workspace: None, + message, + }), + }; + } + result = tokio::time::timeout(remaining, &mut resolve_task) => { + match result { + Ok(Ok(Ok(workspace))) => workspace, + Ok(Ok(Err(error))) => return Err(ExplicitCaptureError::WorkspaceUnavailable(error)), + Ok(Err(error)) => return Err(ExplicitCaptureError::Capture { + workspace: ResolvedCheckpointWorkspace { + path: resolve_absolute(&launch_cwd), + provenance: "resolved-project".to_string(), + }, + message: format!("workspace resolution task failed: {error}"), + }), + Err(_) => { + let candidate = launch_cwd.clone(); + return match cancel_and_settle_capture_task(&resolve_control, &mut resolve_task).await { + Ok(()) => Err(ExplicitCaptureError::ResolutionTimeout( + candidate.to_string_lossy().into_owned(), + )), + Err(message) => Err(ExplicitCaptureError::Unsettled { + candidate, + workspace: None, + message, + }), + }; + } + } + } + }; + + let remaining = deadline.saturating_duration_since(Instant::now()); + if remaining.is_zero() { + return Err(ExplicitCaptureError::ResolutionTimeout( + launch_cwd.to_string_lossy().into_owned(), + )); + } + let lock = tokio::select! { + biased; + _ = wait_for_capture_cancellation(&mut cancel_rx) => { + return Err(ExplicitCaptureError::Cancelled); + } + lock = self.lock_workspace(&workspace.path, remaining) => match lock { + Ok(lock) => lock, + Err(()) => return Err(ExplicitCaptureError::WorkspaceBusy(workspace)), + } + }; + let remaining = deadline.saturating_duration_since(Instant::now()); + if remaining.is_zero() { + drop(lock); + return Err(ExplicitCaptureError::WorkspaceBusy(workspace)); + } + let result = self + .capture_with_budget( + &workspace, + &label, + request_id.as_deref().unwrap_or(""), + remaining, + Some(cancel_rx), + ) + .await; + match result { + CaptureAttempt::Published(checkpoint_id) => { + self.clear_cooldown(&workspace.path); + drop(lock); + Ok(( + explicit_capture_response(checkpoint_id, &label, request_id.as_deref()), + workspace, + )) + } + CaptureAttempt::TimedOut(Some(checkpoint_id)) => { + self.clear_cooldown(&workspace.path); + drop(lock); + Ok(( + explicit_capture_response(checkpoint_id, &label, request_id.as_deref()), + workspace, + )) + } + CaptureAttempt::TimedOut(None) => { + self.establish_cooldown(&workspace.path); + drop(lock); + Err(ExplicitCaptureError::Capture { + workspace, + message: "checkpoint capture timed out".to_string(), + }) + } + CaptureAttempt::Cancelled(_) => { + drop(lock); + Err(ExplicitCaptureError::Cancelled) + } + CaptureAttempt::Unsettled { message, control } => { + let candidate = workspace.path.clone(); + let _quarantine = self.quarantine_workspace( + candidate.clone(), + lock, + control, + request_id.clone(), + "explicit_capture", + message.clone(), + ); + Err(ExplicitCaptureError::Unsettled { + candidate, + workspace: Some(workspace), + message, + }) + } + CaptureAttempt::Failed(error) => { + drop(lock); + Err(ExplicitCaptureError::Capture { + workspace, + message: error, + }) + } + } + } + + async fn capture_with_budget( + &self, + workspace: &ResolvedCheckpointWorkspace, + label: &str, + request_id: &str, + budget: Duration, + mut cancellation: Option<watch::Receiver<bool>>, + ) -> CaptureAttempt { + // The task is owned by this server service, not by the socket/HTTP + // request future. On budget expiry the scoped runner is terminated, + // reaped, and reconciled under the workspace lock before this returns. + let control = Arc::new(CheckpointCaptureControl::default()); + let home = Arc::clone(&self.home); + let workspace_path = workspace.path.clone(); + let label = label.to_string(); + let request_id_owned = request_id.to_string(); + let task_control = Arc::clone(&control); + let mut task = tokio::spawn(CHECKPOINT_CAPTURE_CONTEXT.scope(task_control, async move { + create_checkpoint( + &home, + &workspace_path, + &label, + if request_id_owned.is_empty() { + None + } else { + Some(request_id_owned.as_str()) + }, + ) + .await + })); + let result = if let Some(receiver) = cancellation.as_mut() { + tokio::select! { + biased; + _ = wait_for_capture_cancellation(receiver) => { + match cancel_and_settle_capture_task(&control, &mut task).await { + Ok(()) => { + let published = self.reconcile_published_request(workspace, request_id).await; + return CaptureAttempt::Cancelled(published); + } + Err(error) => { + return CaptureAttempt::Unsettled { + message: error, + control: Arc::clone(&control), + }; + } + } + } + result = tokio::time::timeout(budget, &mut task) => result, + } + } else { + tokio::time::timeout(budget, &mut task).await + }; + match result { + Ok(Ok(Ok(entry))) => { + CaptureAttempt::Published(entry["id"].as_str().unwrap_or_default().to_string()) + } + Ok(Ok(Err(error))) => { + if let Err(settlement_error) = control.wait_for_process_groups().await { + return CaptureAttempt::Unsettled { + message: format!( + "Git operation failed ({error}); child settlement failed ({settlement_error})" + ), + control: Arc::clone(&control), + }; + } + if let Some(published) = self + .reconcile_published_request(workspace, request_id) + .await + { + CaptureAttempt::Published(published) + } else { + CaptureAttempt::Failed(error) + } + } + Ok(Err(error)) => { + if let Err(settlement_error) = control.cancel_processes().await { + tracing::error!( + event = "code_checkpoint.process_termination_failed", + cause = %settlement_error, + "checkpoint child termination was not confirmed after a capture task failure" + ); + } + if let Err(settlement_error) = control.wait_for_process_groups().await { + return CaptureAttempt::Unsettled { + message: format!( + "capture task failed ({error}); child settlement failed ({settlement_error})" + ), + control: Arc::clone(&control), + }; + } + if let Some(published) = self + .reconcile_published_request(workspace, request_id) + .await + { + CaptureAttempt::Published(published) + } else { + CaptureAttempt::Failed(format!("checkpoint capture task failed: {error}")) + } + } + Err(_) => { + if let Err(error) = cancel_and_settle_capture_task(&control, &mut task).await { + return CaptureAttempt::Unsettled { + message: error, + control: Arc::clone(&control), + }; + } + let published = self + .reconcile_published_request(workspace, request_id) + .await; + CaptureAttempt::TimedOut(published) + } + } + } + + async fn reconcile_published_request( + &self, + workspace: &ResolvedCheckpointWorkspace, + request_id: &str, + ) -> Option<String> { + if request_id.is_empty() { + return None; + } + list_checkpoints(&self.home, &workspace.path) + .await + .ok() + .and_then(|entries| { + entries + .into_iter() + .find(|entry| entry["requestId"].as_str() == Some(request_id)) + .and_then(|entry| entry["id"].as_str().map(str::to_string)) + }) + } +} + +enum CaptureAttempt { + Published(String), + Failed(String), + TimedOut(Option<String>), + Cancelled(Option<String>), + Unsettled { + message: String, + control: Arc<CheckpointCaptureControl>, + }, +} + +fn explicit_capture_response( + checkpoint_id: String, + label: &str, + request_id: Option<&str>, +) -> Value { + let ts = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap_or_default() + .as_secs(); + let mut response = Map::new(); + response.insert("id".to_string(), json!(checkpoint_id)); + response.insert("ts".to_string(), json!(ts)); + response.insert("label".to_string(), json!(label)); + if let Some(request_id) = request_id { + response.insert("requestId".to_string(), json!(request_id)); + } + Value::Object(response) +} + +fn checkpoint_attempt_key(send: &FreshAgentSend, logical_attempt_id: &str) -> String { + let provider = serde_json::to_string(&send.provider).unwrap_or_default(); + let session_type = serde_json::to_string(&send.session_type).unwrap_or_default(); + format!( + "{provider}:{session_type}:{}:{logical_attempt_id}", + send.session_id + ) +} + +fn parse_agent_provider(value: &str) -> Option<AgentProvider> { + match value { + "claude" => Some(AgentProvider::Claude), + "codex" => Some(AgentProvider::Codex), + "opencode" => Some(AgentProvider::Opencode), + _ => None, + } +} + +fn parse_session_type(value: &str) -> Option<SessionType> { + match value { + "freshclaude" => Some(SessionType::Freshclaude), + "freshcodex" => Some(SessionType::Freshcodex), + "kilroy" => Some(SessionType::Kilroy), + "freshopencode" => Some(SessionType::Freshopencode), + _ => None, + } +} + +fn workspace_provenance(value: &str) -> CheckpointWorkspaceProvenance { + match value { + "directory-picker" => CheckpointWorkspaceProvenance::DirectoryPicker, + "legacy-checkpoint" => CheckpointWorkspaceProvenance::LegacyCheckpoint, + _ => CheckpointWorkspaceProvenance::ResolvedProject, + } +} + +impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for CheckpointService { + fn barrier_and_forward( + self: Arc<Self>, + send: FreshAgentSend, + forward: futures_util::future::BoxFuture<'static, ()>, + ) -> futures_util::future::BoxFuture<'static, ()> { + Box::pin(async move { + let Some(intent) = send.checkpoint_intent.clone() else { + forward.await; + return; + }; + let Some(request_id) = send + .request_id + .clone() + .filter(|request_id| !request_id.trim().is_empty()) + else { + tracing::warn!( + event = "code_checkpoint.send_missing_request_id", + provider = ?send.provider, + session_id = %send.session_id, + logical_attempt_id = %intent.logical_attempt_id, + "checkpoint send was refused because its request identity is missing" + ); + return; + }; + let logical_attempt_id = intent.logical_attempt_id.trim().to_string(); + let key = checkpoint_attempt_key(&send, &logical_attempt_id); + + let (job, leader) = { + let mut jobs = self.attempt_jobs.lock().await; + if let Some(job) = jobs.get(&key) { + (Arc::clone(job), false) + } else { + let (outcome, _) = watch::channel(None); + let (finished, _) = watch::channel(false); + let job = Arc::new(CheckpointAttemptJob { outcome, finished }); + jobs.insert(key.clone(), Arc::clone(&job)); + (job, true) + } + }; + + if !leader { + let mut outcome_rx = job.outcome.subscribe(); + while outcome_rx.borrow().is_none() { + if outcome_rx.changed().await.is_err() { + return; + } + } + let outcome_snapshot = outcome_rx.borrow().clone(); + if let Some(mut record) = outcome_snapshot { + if !record.request_ids.iter().any(|alias| alias == &request_id) + && record.request_ids.len() < 16 + { + record.request_ids.push(request_id.clone()); + let _ = self.persist_outcome(&key, record.clone()).await; + } + record.outcome.request_id = request_id.clone(); + self.broadcast_outcome(record.outcome); + } + let mut finished_rx = job.finished.subscribe(); + while !*finished_rx.borrow() { + if finished_rx.changed().await.is_err() { + break; + } + } + return; + } + + let (record, allow_forward) = match self.find_outcome(&key).await { + Ok(Some(mut record)) => { + let added_alias = !record.request_ids.iter().any(|alias| alias == &request_id) + && record.request_ids.len() < 16; + if added_alias { + record.request_ids.push(request_id.clone()); + } + if !record.capture_settled { + if added_alias { + if let Err(error) = self.persist_outcome(&key, record.clone()).await { + tracing::error!( + event = "code_checkpoint.outcome_alias_persist_failed", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + cause = %error, + "checkpoint request alias could not be persisted" + ); + } + } + (record, false) + } else if record.dispatch_started { + if added_alias { + if let Err(error) = self.persist_outcome(&key, record.clone()).await { + tracing::error!( + event = "code_checkpoint.outcome_alias_persist_failed", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + cause = %error, + "checkpoint request alias could not be persisted" + ); + } + } + // A durable dispatch fence is terminal for this logical + // attempt. An explicit retry must use a new attempt ID. + (record, false) + } else { + // Persist the admission fence before forwarding. A crash + // after this point can lose a provider send, but cannot + // replay the same logical attempt twice. + record.dispatch_started = true; + match self.persist_outcome(&key, record.clone()).await { + Ok(()) => (record, true), + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_persist_failed", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + cause = %error, + "checkpoint outcome and dispatch fence could not be saved; provider send was withheld" + ); + record.dispatch_started = false; + record.outcome.status = FreshAgentCheckpointStatus::Unavailable; + record.outcome.checkpoint_id = None; + record.outcome.code = Some("outcome_store_unavailable".into()); + record.outcome.reason = Some( + "The checkpoint result could not be saved; this turn will remain unavailable.".into(), + ); + (record, false) + } + } + } + } + Ok(None) => { + let mut record = self + .capture_automatic(&send, &request_id, &logical_attempt_id) + .await; + record.dispatch_started = record.capture_settled; + match self.persist_outcome(&key, record.clone()).await { + Ok(()) => { + let allow_forward = record.capture_settled; + (record, allow_forward) + } + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_persist_failed", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + cause = %error, + "checkpoint outcome and dispatch fence could not be saved; provider send was withheld" + ); + record.dispatch_started = false; + record.outcome.status = FreshAgentCheckpointStatus::Unavailable; + record.outcome.checkpoint_id = None; + record.outcome.code = Some("outcome_store_unavailable".into()); + record.outcome.reason = Some( + "The checkpoint result could not be saved; this turn will remain unavailable.".into(), + ); + (record, false) + } + } + } + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_store_read_failed", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + cause = %error, + "checkpoint attempt could not be safely deduplicated" + ); + ( + self.new_record( + &send, + &logical_attempt_id, + &request_id, + None, + Err(( + "outcome_store_unavailable", + "The checkpoint result could not be saved; this turn will remain unavailable.".into(), + )), + ), + false, + ) + } + }; + let mut record = record; + record.outcome.request_id = request_id.clone(); + self.broadcast_outcome(record.outcome.clone()); + job.outcome.send_replace(Some(record)); + if allow_forward { + forward.await; + } + job.finished.send_replace(true); + let mut jobs = self.attempt_jobs.lock().await; + if jobs + .get(&key) + .is_some_and(|current| Arc::ptr_eq(current, &job)) + { + jobs.remove(&key); + } + }) + } + + fn replay_for_session( + self: Arc<Self>, + provider: AgentProvider, + session_id: String, + session_type: SessionType, + ) -> futures_util::future::BoxFuture<'static, ()> { + Box::pin(async move { + self.replay_for_identity(provider, &session_id, session_type) + .await; + }) + } } #[derive(Clone, Debug, PartialEq, Eq)] @@ -548,19 +2676,18 @@ async fn post_checkpoint( .filter(|s| !s.is_empty()) .map(str::to_string); - let cwd_path = Path::new(&cwd); let send_request_id = safe_send_request_id(request_id.as_deref()); - let workspace = - match resolve_checkpoint_workspace(&state.home, cwd_path, body.get("workspace")).await { - Ok(workspace) => workspace, - Err(error) => { - log_workspace_unavailable(&error.candidate, &error.reason, send_request_id); - return error.response(); - } - }; - - match create_checkpoint(&state.home, &workspace.path, &label, request_id.as_deref()).await { - Ok(mut entry) => { + match state + .service + .capture_explicit( + PathBuf::from(&cwd), + body.get("workspace").cloned(), + label, + request_id.clone(), + ) + .await + { + Ok((mut entry, workspace)) => { if let Some(fields) = entry.as_object_mut() { fields.insert( "workspace".to_string(), @@ -569,12 +2696,78 @@ async fn post_checkpoint( } Json(entry).into_response() } - Err(message) => { + Err(ExplicitCaptureError::WorkspaceUnavailable(error)) => { + log_workspace_unavailable(&error.candidate, &error.reason, send_request_id); + error.response() + } + Err(ExplicitCaptureError::ResolutionTimeout(candidate)) => { + tracing::warn!( + event = "code_checkpoint.workspace_resolution_timeout", + request_id = ?send_request_id, + candidate = %candidate, + "checkpoint workspace resolution exceeded its budget" + ); + let failure = CheckpointFailure::from_capture_detail( + &state.home, + Path::new(&candidate), + "workspace resolution timed out", + ); + failure.log(send_request_id); + failure.response() + } + Err(ExplicitCaptureError::WorkspaceBusy(workspace)) => { + tracing::warn!( + event = "code_checkpoint.workspace_lock_timeout", + request_id = ?send_request_id, + workspace = %workspace.path.display(), + "checkpoint workspace stayed busy until the request budget expired" + ); + ( + StatusCode::SERVICE_UNAVAILABLE, + Json(json!({ "error": "checkpoint workspace lock wait timed out" })), + ) + .into_response() + } + Err(ExplicitCaptureError::Capture { workspace, message }) => { let failure = CheckpointFailure::from_capture_detail(&state.home, &workspace.path, &message); failure.log(send_request_id); failure.response() } + Err(ExplicitCaptureError::Unsettled { + candidate, + workspace, + message, + }) => { + let path = workspace + .as_ref() + .map(|workspace| workspace.path.as_path()) + .unwrap_or(candidate.as_path()); + tracing::error!( + event = "code_checkpoint.capture_settlement_failed", + request_id = ?send_request_id, + workspace = %path.display(), + cause = %message, + "checkpoint child settlement could not be confirmed; no provider dispatch is involved" + ); + let failure = CheckpointFailure::from_capture_detail( + &state.home, + path, + "checkpoint child settlement could not be confirmed", + ); + failure.log(send_request_id); + failure.response() + } + Err(ExplicitCaptureError::Cancelled) => ( + StatusCode::SERVICE_UNAVAILABLE, + Json(json!({ "error": "checkpoint capture was cancelled" })), + ) + .into_response(), + Err(ExplicitCaptureError::ServiceStopped) => ( + StatusCode::INTERNAL_SERVER_ERROR, + Json(json!({ "error": "checkpoint capture service stopped unexpectedly" })), + ) + .into_response(), } } @@ -594,6 +2787,11 @@ async fn create_checkpoint( label: &str, request_id: Option<&str>, ) -> Result<Value, String> { + let capture_started_at = Instant::now(); + if CHECKPOINT_CAPTURE_CONTEXT.try_with(|_| ()).is_err() { + return Err("checkpoint capture requires the server-owned capture runner".to_string()); + } + let repo_prepare_started_at = Instant::now(); let git_dir = ensure_checkpoint_repo(home, cwd).await?; let git_dir_arg = format!("--git-dir={}", git_dir.display()); let work_tree_arg = format!("--work-tree={}", cwd.display()); @@ -640,18 +2838,38 @@ async fn create_checkpoint( ) .await?; } + let repo_prepare_ms = repo_prepare_started_at.elapsed().as_millis() as u64; + let mut stage_ms = BTreeMap::<&'static str, u64>::new(); let capture = async { + let stage_started_at = Instant::now(); verify_readable_directories(&git_dir, cwd).await?; + stage_ms.insert("readability", stage_started_at.elapsed().as_millis() as u64); + + let stage_started_at = Instant::now(); run_git_with_index( &[&git_dir_arg, &work_tree_arg, "add", "-A", "--", "."], Some(cwd), &request_index, ) .await?; + stage_ms.insert("git_add", stage_started_at.elapsed().as_millis() as u64); + + let stage_started_at = Instant::now(); expand_nested_repositories(&git_dir, cwd, &request_index).await?; - verify_checkpoint_index(&git_dir, cwd, &request_index).await?; + stage_ms.insert( + "nested_expansion", + stage_started_at.elapsed().as_millis() as u64, + ); + + let stage_started_at = Instant::now(); + let eligible_file_count = verify_checkpoint_index(&git_dir, cwd, &request_index).await?; + stage_ms.insert( + "index_validation", + stage_started_at.elapsed().as_millis() as u64, + ); + let stage_started_at = Instant::now(); let tree = run_git_with_index(&[&git_dir_arg, "write-tree"], None, &request_index) .await? .trim() @@ -665,10 +2883,35 @@ async fn create_checkpoint( commit_args.extend(["-m", label]); let sha = run_git(&commit_args, None).await?.trim().to_string(); + let changed_file_count = run_git_bytes( + &[ + &git_dir_arg, + "diff-tree", + "--root", + "--no-commit-id", + "--name-only", + "-r", + "-z", + &sha, + ], + None, + ) + .await? + .split(|byte| *byte == 0) + .filter(|path| !path.is_empty()) + .count(); + stage_ms.insert( + "tree_and_commit", + stage_started_at.elapsed().as_millis() as u64, + ); + if let Some(request_id) = request_id { + let stage_started_at = Instant::now(); update_checkpoint_metadata(&git_dir, &sha, request_id).await?; + stage_ms.insert("metadata", stage_started_at.elapsed().as_millis() as u64); } + let stage_started_at = Instant::now(); let head_ref = run_git(&[&git_dir_arg, "symbolic-ref", "HEAD"], None) .await? .trim() @@ -678,13 +2921,17 @@ async fn create_checkpoint( update_ref.push(parent); } run_git(&update_ref, None).await?; + stage_ms.insert( + "ref_publication", + stage_started_at.elapsed().as_millis() as u64, + ); - Ok::<String, String>(sha) + Ok::<(String, usize, usize), String>((sha, eligible_file_count, changed_file_count)) } .await; - let sha = match capture { - Ok(sha) => sha, + let (sha, eligible_file_count, changed_file_count) = match capture { + Ok(capture) => capture, Err(error) => { let _ = tokio::fs::remove_file(&request_index).await; return Err(error); @@ -693,6 +2940,7 @@ async fn create_checkpoint( // The published tree is already complete. Losing this disposable stat cache // can make the next request slower, but cannot hide or invalidate a target. + let index_cache_started_at = Instant::now(); if let Err(error) = install_checkpoint_index(&request_index, &index_path).await { tracing::warn!( event = "code_checkpoint.index_cache_install_failed", @@ -702,6 +2950,27 @@ async fn create_checkpoint( ); let _ = tokio::fs::remove_file(&request_index).await; } + let index_cache_install_ms = index_cache_started_at.elapsed().as_millis() as u64; + + tracing::info!( + event = "code_checkpoint.capture_succeeded", + workspace = %cwd.display(), + request_id = ?safe_send_request_id(request_id), + checkpoint_id = %sha, + eligible_file_count, + changed_file_count, + repo_prepare_ms, + readability_ms = stage_ms.get("readability").copied().unwrap_or_default(), + git_add_ms = stage_ms.get("git_add").copied().unwrap_or_default(), + nested_expansion_ms = stage_ms.get("nested_expansion").copied().unwrap_or_default(), + index_validation_ms = stage_ms.get("index_validation").copied().unwrap_or_default(), + tree_and_commit_ms = stage_ms.get("tree_and_commit").copied().unwrap_or_default(), + metadata_ms = stage_ms.get("metadata").copied().unwrap_or_default(), + ref_publication_ms = stage_ms.get("ref_publication").copied().unwrap_or_default(), + index_cache_install_ms, + total_ms = capture_started_at.elapsed().as_millis() as u64, + "checkpoint capture published successfully" + ); let ts = SystemTime::now() .duration_since(UNIX_EPOCH) @@ -832,17 +3101,19 @@ async fn run_git_with_index_input( // running by default, so every checkpoint-owned Git process must be // configured to stop with its request instead of outliving it. cmd.kill_on_drop(true); - cmd.args(args).stdin(Stdio::piped()); + cmd.args(args) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); if let Some(dir) = cwd { cmd.current_dir(dir); } cmd.env("GIT_INDEX_FILE", index); - let mut child = cmd - .spawn() + let mut child = checkpoint_child(&mut cmd) .map_err(|error| format!("git update-index failed to start: {error}"))?; - if let Some(mut stdin) = child.stdin.take() { + if let Some(mut stdin) = child.child.as_mut().and_then(|child| child.stdin.take()) { if let Err(error) = stdin.write_all(input).await { - let _ = child.kill().await; + child.terminate_and_wait().await; return Err(format!( "git update-index failed to write path list: {error}" )); @@ -876,17 +3147,16 @@ async fn ignored_git_directories( .stdin(Stdio::piped()) .stdout(Stdio::piped()) .stderr(Stdio::piped()); - let mut child = command - .spawn() + let mut child = checkpoint_child(&mut command) .map_err(|error| format!("git check-ignore failed to start: {error}"))?; let mut input = Vec::new(); for path in directory_paths { input.extend_from_slice(path); input.push(0); } - if let Some(mut stdin) = child.stdin.take() { + if let Some(mut stdin) = child.child.as_mut().and_then(|child| child.stdin.take()) { if let Err(error) = stdin.write_all(&input).await { - let _ = child.kill().await; + child.terminate_and_wait().await; return Err(format!("git check-ignore failed to write paths: {error}")); } } @@ -992,15 +3262,22 @@ async fn invoke_git( // detach an in-flight snapshot command. cmd.kill_on_drop(true); cmd.args(args); + // `wait_with_output` only captures the output streams that were explicitly + // piped. Keep Git output available for both normal values (tree/ref ids) + // and actionable structured failure causes instead of inheriting/discarding + // it in the server process. + cmd.stdout(Stdio::piped()).stderr(Stdio::piped()); if let Some(dir) = cwd { cmd.current_dir(dir); } if let Some(index) = index { cmd.env("GIT_INDEX_FILE", index); } - cmd.output() + checkpoint_read_only_child(&mut cmd) + .map_err(|e| format!("Git process failed to start: {e}"))? + .wait_with_output() .await - .map_err(|e| format!("Git process failed to start: {e}")) + .map_err(|e| format!("Git process failed to wait: {e}")) } fn check_git_output(args: &[&str], output: std::process::Output) -> Result<String, String> { @@ -1404,9 +3681,9 @@ async fn verify_checkpoint_index( git_dir: &Path, work_tree: &Path, index: &Path, -) -> Result<(), String> { +) -> Result<usize, String> { let staged = checkpoint_index_entries(git_dir, index).await?; - for entry in staged { + for entry in &staged { let mode = entry.mode.as_slice(); let path = git_relative_path(&entry.path); if path.is_absolute() @@ -1479,7 +3756,7 @@ async fn verify_checkpoint_index( } } } - Ok(()) + Ok(staged.len()) } async fn install_checkpoint_index( @@ -1598,12 +3875,41 @@ fn is_valid_checkpoint_id(id: &str) -> bool { /// error — mirrors the two `try {} catch { return [] }` guards in legacy. async fn list_checkpoints(home: &Path, cwd: &Path) -> Result<Vec<Value>, String> { let git_dir = checkpoint_git_dir(home, cwd); - if tokio::fs::metadata(git_dir.join("HEAD")).await.is_err() { - return Ok(Vec::new()); + match tokio::fs::symlink_metadata(git_dir.join("HEAD")).await { + Ok(_) => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return match tokio::fs::symlink_metadata(&git_dir).await { + Err(directory_error) if directory_error.kind() == std::io::ErrorKind::NotFound => { + Ok(Vec::new()) + } + Ok(_) => Err("checkpoint repository exists but its HEAD is missing".to_string()), + Err(directory_error) => Err(format!( + "failed to inspect checkpoint repository: {directory_error}" + )), + }; + } + Err(error) => { + return Err(format!( + "failed to inspect checkpoint repository HEAD: {error}" + )); + } } let git_dir_arg = format!("--git-dir={}", git_dir.display()); + // A successfully inspected but unborn shadow repository is the one + // legitimate `git log` failure that means "no checkpoints yet". Probe + // HEAD first and accept only Git's quiet missing-revision status; corrupt + // refs, unreadable storage, and all other Git failures must remain errors + // so callers cannot erase a previously Available turn as an empty list. + let head_args = [&git_dir_arg, "rev-parse", "--verify", "--quiet", "HEAD"]; + let head = invoke_git(&head_args, None, None).await?; + if !head.status.success() { + if head.status.code() == Some(1) { + return Ok(Vec::new()); + } + check_git_output(&head_args, head)?; + } let limit = CHECKPOINT_LIST_LIMIT.to_string(); - let raw = match run_git( + let raw = run_git( &[ &git_dir_arg, "log", @@ -1613,12 +3919,7 @@ async fn list_checkpoints(home: &Path, cwd: &Path) -> Result<Vec<Value>, String> ], None, ) - .await - { - Ok(raw) => raw, - // Empty repo (no commits yet) — `git log` errors on a HEAD-less repo. - Err(_) => return Ok(Vec::new()), - }; + .await?; let metadata = read_checkpoint_metadata(&git_dir).await?; let mut entries = Vec::new(); for line in raw.split('\n') { @@ -1811,7 +4112,10 @@ async fn get_checkpoints( // root. Old panes can still have an immutable checkpoint store keyed // by their exact launch cwd, so include that store only when it already // exists. An unselected non-Git directory or HOME therefore lists as - // empty and can never become an implicit capture scope. + // empty and can never become an implicit capture scope. Include the + // exact launch cwd as a candidate even when its store is absent; the + // shared workspace lock makes the missing-store check race-free with + // the first create. let mut workspaces = Vec::new(); if let Ok(workspace) = resolve_checkpoint_workspace(&state.home, Path::new(&cwd), None).await @@ -1830,11 +4134,7 @@ async fn get_checkpoints( let already_present = workspaces .iter() .any(|workspace| workspace.path == legacy_path); - if !already_present - && tokio::fs::metadata(checkpoint_git_dir(&state.home, &legacy_path).join("HEAD")) - .await - .is_ok() - { + if !already_present { workspaces.push(ResolvedCheckpointWorkspace { path: legacy_path, provenance: "legacy-checkpoint".to_string(), @@ -1846,6 +4146,20 @@ async fn get_checkpoints( let mut entries = Vec::new(); for workspace in workspaces { + let _workspace_lock = match state + .service + .lock_workspace(&workspace.path, CHECKPOINT_EXPLICIT_BUDGET) + .await + { + Ok(lock) => lock, + Err(()) => { + return ( + StatusCode::SERVICE_UNAVAILABLE, + Json(json!({ "error": "checkpoint workspace is busy" })), + ) + .into_response(); + } + }; match list_checkpoints(&state.home, &workspace.path).await { Ok(mut scoped_entries) => { for entry in &mut scoped_entries { @@ -1919,6 +4233,20 @@ async fn post_checkpoint_metadata( cwd_path.to_path_buf() }; + let _workspace_lock = match state + .service + .lock_workspace(&workspace, CHECKPOINT_EXPLICIT_BUDGET) + .await + { + Ok(lock) => lock, + Err(()) => { + return ( + StatusCode::SERVICE_UNAVAILABLE, + Json(json!({ "error": "checkpoint workspace is busy" })), + ) + .into_response(); + } + }; match apply_checkpoint_metadata_patch(&state.home, &workspace, id, request_id, turn_id).await { Ok(entry) => Json(entry).into_response(), Err(message) => { @@ -1960,6 +4288,20 @@ async fn post_checkpoint_restore( cwd_path.to_path_buf() }; + let _workspace_lock = match state + .service + .lock_workspace(&workspace, CHECKPOINT_EXPLICIT_BUDGET) + .await + { + Ok(lock) => lock, + Err(()) => { + return ( + StatusCode::SERVICE_UNAVAILABLE, + Json(json!({ "error": "checkpoint workspace is busy" })), + ) + .into_response(); + } + }; match restore_checkpoint(&state.home, &workspace, id).await { Ok(()) => Json(json!({ "restored": true })).into_response(), Err(message) => { @@ -1984,12 +4326,309 @@ mod tests { } fn state(home: &Path) -> CheckpointsApiState { + let (broadcast_tx, _) = tokio::sync::broadcast::channel(8); + let home = Arc::new(home.to_path_buf()); CheckpointsApiState { auth_token: Arc::new("tok".to_string()), - home: Arc::new(home.to_path_buf()), + home: Arc::clone(&home), + service: Arc::new(CheckpointService::new(home, Arc::new(broadcast_tx))), } } + #[tokio::test] + async fn unsettled_capture_quarantine_holds_workspace_lock_until_settlement() { + let home_dir = tempfile::tempdir().unwrap(); + let workspace_dir = tempfile::tempdir().unwrap(); + let state = state(home_dir.path()); + let service = Arc::clone(&state.service); + let workspace = workspace_dir.path().to_path_buf(); + let lock = service + .lock_workspace(&workspace, Duration::from_secs(1)) + .await + .unwrap(); + let control = Arc::new(CheckpointCaptureControl::default()); + let gate = Arc::new(TestSettlementGate { + entered: tokio::sync::Notify::new(), + release: tokio::sync::Notify::new(), + }); + *control + .test_settlement_gate + .lock() + .expect("test settlement gate mutex poisoned") = Some(Arc::clone(&gate)); + + let quarantine = service.quarantine_workspace( + workspace.clone(), + lock, + Arc::clone(&control), + Some("unsettled-attempt".to_string()), + "automatic_capture_test", + "simulated uncertain child settlement".to_string(), + ); + tokio::time::timeout(Duration::from_secs(1), gate.entered.notified()) + .await + .expect("quarantine began checking child settlement"); + + assert!( + service + .lock_workspace(&workspace, Duration::from_millis(30)) + .await + .is_err(), + "another capture must not enter while child settlement is unconfirmed" + ); + + gate.release.notify_one(); + tokio::time::timeout(Duration::from_secs(1), quarantine) + .await + .expect("quarantine released after settlement was confirmed") + .expect("quarantine worker completed"); + let later_lock = service + .lock_workspace(&workspace, Duration::from_secs(1)) + .await + .expect("workspace lock becomes available after settlement"); + drop(later_lock); + } + + #[cfg(unix)] + #[tokio::test] + async fn checkpoint_timeout_settles_spawned_descendant_before_returning() { + let dir = tempfile::tempdir().unwrap(); + let started = dir.path().join("descendant-started"); + let marker = dir.path().join("late-publish"); + let shell_script = format!( + "(sleep 0.8; printf late > '{}') & printf started > '{}'; wait", + marker.display(), + started.display(), + ); + let control = Arc::new(CheckpointCaptureControl::default()); + let mut command = Command::new("sh"); + command + .arg("-c") + .arg(shell_script) + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + let child = control.spawn_child(&mut command).unwrap(); + let child_task = tokio::spawn(async move { child.wait_with_output().await }); + tokio::time::timeout(Duration::from_secs(1), async { + while !started.exists() { + tokio::time::sleep(Duration::from_millis(5)).await; + } + }) + .await + .expect("the child started its delayed descendant"); + + control.cancel_processes().await.unwrap(); + child_task.await.unwrap().unwrap(); + control.wait_for_process_groups().await.unwrap(); + tokio::time::sleep(Duration::from_millis(900)).await; + assert!( + !marker.exists(), + "a killed Git descendant must not publish after checkpoint settlement" + ); + assert!(control.process_groups.lock().unwrap().is_empty()); + } + + #[cfg(windows)] + #[tokio::test] + async fn checkpoint_timeout_settles_windows_job_descendant_before_returning() { + let dir = tempfile::tempdir().unwrap(); + let started = dir.path().join("descendant-started.txt"); + let marker = dir.path().join("late-publish.txt"); + let started_path = started.to_string_lossy().replace('\'', "''"); + let marker_path = marker.to_string_lossy().replace('\'', "''"); + let script = format!( + "start \"\" /B powershell -NoProfile -Command \"Set-Content -NoNewline -Path '{started_path}' -Value started; Start-Sleep -Seconds 1; Set-Content -NoNewline -Path '{marker_path}' -Value late\" & ping -n 60 127.0.0.1 > nul" + ); + let control = Arc::new(CheckpointCaptureControl::default()); + let mut command = Command::new("cmd.exe"); + command + .args(["/C", &script]) + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + let child = control.spawn_child(&mut command).unwrap(); + let child_task = tokio::spawn(async move { child.wait_with_output().await }); + tokio::time::timeout(Duration::from_secs(10), async { + while !started.exists() { + tokio::time::sleep(Duration::from_millis(20)).await; + } + }) + .await + .expect("the Windows child started its Job-owned descendant"); + + control.cancel_processes().await.unwrap(); + child_task.await.unwrap().unwrap(); + control.wait_for_process_groups().await.unwrap(); + tokio::time::sleep(Duration::from_millis(1200)).await; + assert!( + !marker.exists(), + "a terminated Job descendant cannot publish after checkpoint settlement" + ); + assert!(control.jobs.lock().unwrap().is_empty()); + } + + #[tokio::test] + async fn accepted_logical_attempt_is_never_forwarded_again() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-1".into(), + session_type: SessionType::Freshopencode, + text: "the request body is never stored in checkpoint outcomes".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "logical-attempt-1".into(), + workspace: None, + }), + }; + + let dispatches_for_first = Arc::clone(&dispatches); + service + .clone() + .barrier_and_forward( + make_send("request-1"), + Box::pin(async move { + dispatches_for_first.fetch_add(1, Ordering::SeqCst); + }), + ) + .await; + service + .observe_accepted(&json!({ + "requestId": "request-1", + "provider": "opencode", + "sessionId": "session-1", + "sessionType": "freshopencode", + "submittedTurnId": "provider-turn-1" + })) + .await; + + let dispatches_for_second = Arc::clone(&dispatches); + service + .clone() + .barrier_and_forward( + make_send("request-2"), + Box::pin(async move { + dispatches_for_second.fetch_add(1, Ordering::SeqCst); + }), + ) + .await; + + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + let key = format!("\"opencode\":\"freshopencode\":session-1:logical-attempt-1"); + let record = service.find_outcome(&key).await.unwrap().unwrap(); + assert!(record.dispatch_started); + assert_eq!(record.submitted_turn_id.as_deref(), Some("provider-turn-1")); + assert!(record.request_ids.contains(&"request-2".to_string())); + } + + #[tokio::test] + async fn codex_identity_receipt_is_durable_and_replayed_after_service_reload() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + let workspace = ResolvedCheckpointWorkspace { + path: PathBuf::from("/work/codex-project"), + provenance: "directory-picker".to_string(), + }; + let send = FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "codex-session".to_string(), + session_type: SessionType::Freshcodex, + text: "do not persist this prompt".to_string(), + cwd: Some("/work/codex-project".to_string()), + images: None, + request_id: Some("codex-request-1".to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "codex-logical-attempt-1".to_string(), + workspace: None, + }), + }; + let key = checkpoint_attempt_key(&send, "codex-logical-attempt-1"); + let mut record = service.new_record( + &send, + "codex-logical-attempt-1", + "codex-request-1", + Some((&workspace, "")), + Ok("checkpoint-sha".to_string()), + ); + record.dispatch_started = true; + record.submitted_turn_id = Some("codex-native-turn-1".to_string()); + service.persist_outcome(&key, record).await.unwrap(); + + service + .observe_turn_identity(&json!({ + "provider": "codex", + "sessionId": "codex-session", + "sessionType": "freshcodex", + "event": { + "type": "checkpointTurnIdentity", + "requestId": "codex-request-1", + "logicalAttemptId": "codex-logical-attempt-1", + "sessionId": "codex-session", + "nativeTurnId": "codex-native-turn-1", + "displayTurnId": "codex-native-turn-1:row-0" + } + })) + .await; + + let persisted = service.read_outcome_store().await.unwrap(); + let saved = persisted.outcomes.get(&key).unwrap(); + assert_eq!( + saved.outcome.native_turn_id.as_deref(), + Some("codex-native-turn-1") + ); + assert_eq!( + saved.outcome.display_turn_id.as_deref(), + Some("codex-native-turn-1:row-0") + ); + assert!( + !serde_json::to_string(&persisted) + .unwrap() + .contains("do not persist this prompt"), + "durable checkpoint identity state never stores the sent prompt" + ); + + let (replay_tx, mut replay_rx) = tokio::sync::broadcast::channel(8); + let reloaded = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(replay_tx), + )); + reloaded + .clone() + .replay_for_session( + AgentProvider::Codex, + "codex-session".to_string(), + SessionType::Freshcodex, + ) + .await; + let frame = replay_rx.recv().await.unwrap(); + let replayed: Value = serde_json::from_str(&frame).unwrap(); + assert_eq!(replayed["type"], "freshAgent.checkpoint.outcome"); + assert_eq!(replayed["logicalAttemptId"], "codex-logical-attempt-1"); + assert_eq!(replayed["nativeTurnId"], "codex-native-turn-1"); + assert_eq!(replayed["displayTurnId"], "codex-native-turn-1:row-0"); + } + async fn body_json(resp: Response) -> Value { let body = axum::body::to_bytes(resp.into_body(), usize::MAX) .await @@ -2081,13 +4720,17 @@ mod tests { .await .unwrap(); - let ignored = ignored_git_directories( - &git_dir, - work_tree.path(), - &[b"private-dir.ignored/".to_vec()], - ) - .await - .unwrap(); + let ignored = CHECKPOINT_CAPTURE_CONTEXT + .scope( + Arc::new(CheckpointCaptureControl::default()), + ignored_git_directories( + &git_dir, + work_tree.path(), + &[b"private-dir.ignored/".to_vec()], + ), + ) + .await + .unwrap(); assert!(ignored.contains(b"private-dir.ignored/".as_slice())); } @@ -2793,6 +5436,125 @@ mod tests { assert_eq!(resp.status(), StatusCode::OK); } + #[cfg(unix)] + #[tokio::test] + async fn dropped_http_capture_holds_workspace_until_descendant_is_settled() { + use std::os::unix::fs::PermissionsExt; + use tower::ServiceExt; + + let home_dir = tempfile::tempdir().unwrap(); + let cwd_dir = tempfile::tempdir().unwrap(); + let started = home_dir.path().join("filter-descendant-started"); + let late_write = home_dir.path().join("filter-descendant-late-write"); + let filter = home_dir.path().join("blocking-clean-filter.sh"); + let filter_contents = format!( + "#!/bin/sh\n( trap '' TERM; printf started > '{}'; sleep 1; printf late > '{}' ) &\nwait\ncat\n", + started.display(), + late_write.display(), + ); + tokio::fs::write(&filter, filter_contents).await.unwrap(); + let mut permissions = tokio::fs::metadata(&filter).await.unwrap().permissions(); + permissions.set_mode(0o755); + tokio::fs::set_permissions(&filter, permissions) + .await + .unwrap(); + tokio::fs::write( + cwd_dir.path().join(".gitattributes"), + "*.slow filter=blocking\n", + ) + .await + .unwrap(); + tokio::fs::write(cwd_dir.path().join("payload.slow"), "checkpoint data\n") + .await + .unwrap(); + + let git_dir = ensure_checkpoint_repo(home_dir.path(), cwd_dir.path()) + .await + .unwrap(); + let git_dir_arg = format!("--git-dir={}", git_dir.display()); + let filter_command = filter.to_string_lossy().into_owned(); + run_git( + &[ + &git_dir_arg, + "config", + "filter.blocking.clean", + &filter_command, + ], + None, + ) + .await + .unwrap(); + run_git( + &[&git_dir_arg, "config", "filter.blocking.required", "true"], + None, + ) + .await + .unwrap(); + + let state = state(home_dir.path()); + let service = Arc::clone(&state.service); + let workspace = cwd_dir.path().to_path_buf(); + let workspace_text = workspace.to_string_lossy().into_owned(); + let app = router(state); + let request = axum::http::Request::builder() + .method("POST") + .uri("/api/fresh-agent/checkpoints") + .header("content-type", "application/json") + .header("x-auth-token", "tok") + .body(axum::body::Body::from( + serde_json::to_vec(&json!({ + "cwd": workspace_text, + "workspace": { + "path": workspace_text, + "provenance": "directory-picker" + }, + "requestId": "dropped-http-capture" + })) + .unwrap(), + )) + .unwrap(); + let request_task = tokio::spawn(app.oneshot(request)); + tokio::time::timeout(Duration::from_secs(3), async { + while !started.exists() { + tokio::time::sleep(Duration::from_millis(5)).await; + } + }) + .await + .expect("the Git clean filter started its descendant"); + + request_task.abort(); + let _ = request_task.await; + + let during_settlement = service + .lock_workspace(&workspace, Duration::from_millis(30)) + .await; + assert!( + during_settlement.is_err(), + "the workspace lock must remain owned while the descendant is being killed and reaped" + ); + + let settled_lock = tokio::time::timeout( + Duration::from_secs(3), + service.lock_workspace(&workspace, Duration::from_secs(3)), + ) + .await + .expect("the HTTP cancellation supervisor settles") + .expect("the workspace is released after settlement"); + drop(settled_lock); + + tokio::time::sleep(Duration::from_millis(1100)).await; + assert!( + !late_write.exists(), + "a descendant cannot continue writing after the route has settled cancellation" + ); + let entries = list_checkpoints(home_dir.path(), &workspace).await.unwrap(); + assert!( + !entries + .iter() + .any(|entry| { entry["requestId"].as_str() == Some("dropped-http-capture") }) + ); + } + // ── is_valid_checkpoint_id ────────────────────────────────────────── #[test] @@ -2867,6 +5629,40 @@ mod tests { assert_eq!(value["checkpoints"], json!([])); } + #[tokio::test] + async fn list_returns_empty_for_an_initialized_but_unborn_shadow_repository() { + let home_dir = tempfile::tempdir().unwrap(); + let cwd_dir = tempfile::tempdir().unwrap(); + ensure_checkpoint_repo(home_dir.path(), cwd_dir.path()) + .await + .unwrap(); + + assert!( + list_checkpoints(home_dir.path(), cwd_dir.path()) + .await + .unwrap() + .is_empty() + ); + } + + #[tokio::test] + async fn list_reports_an_existing_shadow_repository_with_missing_head_as_corrupt() { + let home_dir = tempfile::tempdir().unwrap(); + let cwd_dir = tempfile::tempdir().unwrap(); + let git_dir = ensure_checkpoint_repo(home_dir.path(), cwd_dir.path()) + .await + .unwrap(); + tokio::fs::remove_file(git_dir.join("HEAD")).await.unwrap(); + + let result = list_checkpoints(home_dir.path(), cwd_dir.path()).await; + + assert!( + result + .unwrap_err() + .contains("checkpoint repository exists but its HEAD is missing") + ); + } + #[tokio::test] async fn list_does_not_require_cwd_to_exist_on_disk() { // Legacy GET /checkpoints has no fsp.access(cwd) guard (unlike POST @@ -2933,6 +5729,51 @@ mod tests { assert!(checkpoints[0]["ts"].as_u64().unwrap() > 0); } + #[tokio::test] + async fn list_storage_error_is_not_reported_as_an_empty_checkpoint_history() { + let home_dir = tempfile::tempdir().unwrap(); + let cwd_dir = tempfile::tempdir().unwrap(); + let first = CHECKPOINT_CAPTURE_CONTEXT + .scope( + Arc::new(CheckpointCaptureControl::default()), + create_checkpoint(home_dir.path(), cwd_dir.path(), "first", Some("req-1")), + ) + .await + .unwrap(); + let git_dir = checkpoint_git_dir(home_dir.path(), cwd_dir.path()); + let head_ref = run_git( + &[ + &format!("--git-dir={}", git_dir.display()), + "symbolic-ref", + "HEAD", + ], + None, + ) + .await + .unwrap(); + let ref_path = git_dir.join(head_ref.trim()); + tokio::fs::write(&ref_path, format!("{}\n", "0".repeat(40))) + .await + .unwrap(); + + let response = get_checkpoints( + State(state(home_dir.path())), + headers_with_token("tok"), + list_query(cwd_dir.path().to_str().unwrap()), + ) + .await; + + assert_eq!(first["id"].as_str().unwrap().len(), 40); + assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); + let body = body_json(response).await; + assert!( + body["error"] + .as_str() + .unwrap_or_default() + .contains("Git operation log failed") + ); + } + #[tokio::test] async fn list_includes_persisted_request_id_metadata() { let home_dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 38aea5cb4..ed8ff54c7 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -12,8 +12,8 @@ use freshell_freshagent::naming::{admit_pending_projection, session_projection, use freshell_protocol::session_names::NamedProvider; use freshell_protocol::{ AgentProvider, FreshAgentCreateFailed, FreshAgentCreated, FreshAgentEvent, FreshAgentForked, - FreshAgentKilled, FreshAgentRecoveryStopped, ServerMessage, SessionLocator, SessionType, - StringOrNumber, + FreshAgentKilled, FreshAgentRecoveryStopped, FreshAgentSend, ServerMessage, SessionLocator, + SessionType, StringOrNumber, }; use freshell_runtime_client::RuntimeClient; use freshell_runtime_protocol::{ @@ -54,6 +54,8 @@ pub(crate) struct HostedFreshAgentProxy { pollers: Mutex<HashSet<SoulId>>, fixture_modes: HashSet<String>, naming: OnceLock<Arc<dyn SessionNaming>>, + checkpoint_barrier: OnceLock<Arc<dyn freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier>>, + ownership: OnceLock<Arc<freshell_ownership::RuntimeOwnershipRegistry>>, } struct RollbackInvocation { @@ -87,9 +89,89 @@ impl HostedFreshAgentProxy { pollers: Mutex::new(HashSet::new()), fixture_modes, naming: OnceLock::new(), + checkpoint_barrier: OnceLock::new(), + ownership: OnceLock::new(), }))) } + pub(crate) fn set_checkpoint_barrier( + &self, + barrier: Arc<dyn freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier>, + ) { + let _ = self.checkpoint_barrier.set(barrier); + } + + pub(crate) fn set_ownership( + &self, + ownership: Arc<freshell_ownership::RuntimeOwnershipRegistry>, + ) { + let _ = self.ownership.set(ownership); + } + + fn post_barrier_admission(&self, message: &FreshAgentSend) -> Result<(), &'static str> { + let (epoch, generation) = match (message.observed_epoch, message.observed_generation) { + (None, None) => return Ok(()), + (Some(epoch), Some(generation)) => (epoch, generation), + _ => return Err("fresh-agent ownership fence is incomplete"), + }; + let Some(ownership) = self.ownership.get() else { + return Ok(()); + }; + let provider = provider_wire(&message.provider); + let canonical_id = ownership.resolve_canonical(&provider, &message.session_id); + let current = ownership.observe(&provider, &canonical_id); + if current.epoch != epoch || current.generation != generation { + return Err("fresh-agent session changed while the checkpoint was being captured"); + } + match current.state { + freshell_ownership::OwnershipState::Live { owner, .. } + if owner.kind == freshell_ownership::RuntimeOwnerKind::FreshAgent => Ok(()), + _ => Err("fresh-agent session is no longer available for this send"), + } + } + + async fn dispatch_send(&self, message: FreshAgentSend) { + let provider = message.provider; + let session_id = message.session_id.clone(); + let session_type = message.session_type; + if let Err(reason) = self.post_barrier_admission(&message) { + self.error_event_code( + &provider, + &session_id, + session_type, + "FRESH_AGENT_STALE_SESSION", + reason, + ); + return; + } + let request_id = message + .request_id + .as_deref() + .and_then(|value| RequestId::parse(value.to_string()).ok()) + .unwrap_or_else(RequestId::new); + let result = match self + .resolve_soul(&provider, session_type, &session_id) + .await + { + Some(soul) => { + let settings = turn_settings(message.settings, message.cwd); + self.client + .fresh_agent_send(request_id, soul, message.text, settings) + .await + .map(|_| ()) + } + None => Err(freshell_runtime_client::ClientError::UnexpectedResult), + }; + if result.is_err() { + self.error_event( + &provider, + &session_id, + session_type, + "managed fresh-agent input was not accepted", + ); + } + } + pub(crate) fn set_session_naming(&self, naming: Arc<dyn SessionNaming>) { let _ = self.naming.set(naming); } @@ -114,6 +196,15 @@ impl HostedFreshAgentProxy { } } HostedFreshAgentCommand::Attach(message) => { + if let Some(barrier) = self.checkpoint_barrier.get() { + Arc::clone(barrier) + .replay_for_session( + message.provider, + message.session_id.clone(), + message.session_type, + ) + .await; + } if let Some(soul) = self .resolve_soul(&message.provider, message.session_type, &message.session_id) .await @@ -142,34 +233,19 @@ impl HostedFreshAgentProxy { } } HostedFreshAgentCommand::Send(message) => { - let provider = message.provider; - let session_id = message.session_id.clone(); - let session_type = message.session_type; - let request_id = message - .request_id - .as_deref() - .and_then(|value| RequestId::parse(value.to_string()).ok()) - .unwrap_or_else(RequestId::new); - let result = match self - .resolve_soul(&provider, session_type, &session_id) - .await - { - Some(soul) => { - let settings = turn_settings(message.settings, message.cwd); - self.client - .fresh_agent_send(request_id, soul, message.text, settings) - .await - .map(|_| ()) + if message.checkpoint_intent.is_some() { + if let Some(barrier) = self.checkpoint_barrier.get() { + let proxy = Arc::clone(&self); + let capture_send = message.clone(); + let forward = Box::pin(async move { proxy.dispatch_send(message).await }); + Arc::clone(barrier) + .barrier_and_forward(capture_send, forward) + .await; + } else { + self.dispatch_send(message).await; } - None => Err(freshell_runtime_client::ClientError::UnexpectedResult), - }; - if result.is_err() { - self.error_event( - &provider, - &session_id, - session_type, - "managed fresh-agent input was not accepted", - ); + } else { + self.dispatch_send(message).await; } } HostedFreshAgentCommand::Interrupt(message) => { diff --git a/crates/freshell-server/src/fresh_agent_proxy_tests.rs b/crates/freshell-server/src/fresh_agent_proxy_tests.rs index 83bd9e6e0..6d97084ef 100644 --- a/crates/freshell-server/src/fresh_agent_proxy_tests.rs +++ b/crates/freshell-server/src/fresh_agent_proxy_tests.rs @@ -69,6 +69,8 @@ fn hosted_created_frame_uses_the_public_request_identity() { pollers: Mutex::new(HashSet::new()), fixture_modes: HashSet::new(), naming: OnceLock::new(), + checkpoint_barrier: OnceLock::new(), + ownership: OnceLock::new(), }; proxy.forward_host_event( AgentEvent::Provider { @@ -101,6 +103,8 @@ async fn managed_recovery_stop_refuses_without_contacting_the_host() { pollers: Mutex::new(HashSet::new()), fixture_modes: HashSet::new(), naming: OnceLock::new(), + checkpoint_barrier: OnceLock::new(), + ownership: OnceLock::new(), }); proxy @@ -375,6 +379,8 @@ async fn managed_provider_fork_rekeys_the_same_soul_without_launch_or_stop() { pollers: Mutex::new(HashSet::new()), fixture_modes: HashSet::new(), naming: std::sync::OnceLock::new(), + checkpoint_barrier: std::sync::OnceLock::new(), + ownership: std::sync::OnceLock::new(), }); Arc::clone(&proxy) .handle(HostedFreshAgentCommand::Fork(FreshAgentFork { @@ -698,6 +704,8 @@ async fn rollback_commands_keep_direction_target_and_request_fences() { pollers: Mutex::new(HashSet::new()), fixture_modes: HashSet::new(), naming: OnceLock::new(), + checkpoint_barrier: OnceLock::new(), + ownership: OnceLock::new(), }); let message = serde_json::json!({"provider":"codex","sessionId":"public-thread","sessionType":"freshcodex", "requestId":"rollback-ui-request","mode":"toTurn","turnId":"turn-selected","cwd":"/workspace"}); @@ -763,6 +771,8 @@ fn snapshot_outage_proxy(socket: &Path) -> Arc<HostedFreshAgentProxy> { pollers: Mutex::new(HashSet::new()), fixture_modes: HashSet::new(), naming: OnceLock::new(), + checkpoint_barrier: OnceLock::new(), + ownership: OnceLock::new(), }) } diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index 5487ae679..1c4ac987c 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -1003,6 +1003,14 @@ async fn main() -> ExitCode { // socket — the original `WsHandler.broadcast`. Capacity is generous so a paced // fresh-agent turn's handful of broadcasts never laps a briefly-busy consumer. let broadcast_tx = Arc::new(tokio::sync::broadcast::channel::<String>(1024).0); + let checkpoint_home = Arc::new(home.clone().unwrap_or_else(|| PathBuf::from("."))); + let checkpoint_service = Arc::new(checkpoints::CheckpointService::new( + Arc::clone(&checkpoint_home), + Arc::clone(&broadcast_tx), + )); + let checkpoint_barrier: Arc<dyn freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier> = + checkpoint_service.clone(); + checkpoint_service.start_observer(); // The shared UI-screenshot broker over that same bus: `POST /api/screenshots` // registers a request + broadcasts `screenshot.capture`; the `/ws` loop routes @@ -1178,6 +1186,8 @@ async fn main() -> ExitCode { }); #[cfg(feature = "managed-runtime-v1")] if let Some(gateway) = hosted_fresh_gateway.as_ref() { + gateway.set_checkpoint_barrier(Arc::clone(&checkpoint_barrier)); + gateway.set_ownership(Arc::clone(&ownership)); fresh_agent_state .set_hosted_rest_gateway(gateway.clone()) .unwrap_or_else(|error| { @@ -2280,6 +2290,9 @@ async fn main() -> ExitCode { interest: host_stats_interest.clone(), collector: Some(host_stats_collector.clone()), }, + checkpoint_send: freshell_ws::checkpoint_barrier::WsCheckpointState { + service: Some(Arc::clone(&checkpoint_barrier)), + }, terminals_revision: Arc::clone(&terminals_revision), sessions_revision: Arc::clone(&sessions_revision), cli_commands: Arc::clone(&cli_commands), @@ -3001,7 +3014,8 @@ async fn main() -> ExitCode { // back to the cwd-relative `.` the other home-relative state above uses. let checkpoints_state = checkpoints::CheckpointsApiState { auth_token: Arc::clone(&auth_token), - home: Arc::new(home.clone().unwrap_or_else(|| PathBuf::from("."))), + home: Arc::clone(&checkpoint_home), + service: Arc::clone(&checkpoint_service), }; // `POST /api/fresh-agent/attachments` (`fresh-agent-extras-router.ts:260-287`): @@ -4667,6 +4681,7 @@ mod sessions_sweep_tests { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/src/auto_resume.rs b/crates/freshell-ws/src/auto_resume.rs index 50fbaec78..fbb0ff511 100644 --- a/crates/freshell-ws/src/auto_resume.rs +++ b/crates/freshell-ws/src/auto_resume.rs @@ -3049,6 +3049,7 @@ mod tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: std::sync::Arc::new(Vec::new()), diff --git a/crates/freshell-ws/src/checkpoint_barrier.rs b/crates/freshell-ws/src/checkpoint_barrier.rs new file mode 100644 index 000000000..ddfd36e44 --- /dev/null +++ b/crates/freshell-ws/src/checkpoint_barrier.rs @@ -0,0 +1,34 @@ +//! Narrow service seam for the modern fresh-agent send/capture barrier. +//! +//! The server owns the captured attempt and only polls the provider-forward +//! future after capture publication or failure has fully settled. Callers +//! pass the forward operation as a future so duplicate logical attempts can +//! join one server-owned job without dispatching twice. + +use std::sync::Arc; + +use futures::future::BoxFuture; +use freshell_protocol::{AgentProvider, FreshAgentSend, SessionType}; + +pub trait FreshAgentCheckpointBarrier: Send + Sync { + fn barrier_and_forward( + self: Arc<Self>, + send: FreshAgentSend, + forward: BoxFuture<'static, ()>, + ) -> BoxFuture<'static, ()>; + + /// Replays durable per-turn results when a pane reattaches after missing + /// the pre-forward outcome broadcast. Implementations filter by provider + /// session identity and never infer an outcome from transcript text. + fn replay_for_session( + self: Arc<Self>, + provider: AgentProvider, + session_id: String, + session_type: SessionType, + ) -> BoxFuture<'static, ()>; +} + +#[derive(Clone, Default)] +pub struct WsCheckpointState { + pub service: Option<Arc<dyn FreshAgentCheckpointBarrier>>, +} diff --git a/crates/freshell-ws/src/codex_association.rs b/crates/freshell-ws/src/codex_association.rs index c68e0b039..c9b47f0d2 100644 --- a/crates/freshell-ws/src/codex_association.rs +++ b/crates/freshell-ws/src/codex_association.rs @@ -310,6 +310,7 @@ mod tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: StdArc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: StdArc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: StdArc::new(Vec::new()), diff --git a/crates/freshell-ws/src/codex_proxy_route.rs b/crates/freshell-ws/src/codex_proxy_route.rs index ea2e2c58a..5b265f17e 100644 --- a/crates/freshell-ws/src/codex_proxy_route.rs +++ b/crates/freshell-ws/src/codex_proxy_route.rs @@ -290,6 +290,7 @@ mod tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: StdArc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: StdArc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: StdArc::new(Vec::new()), diff --git a/crates/freshell-ws/src/identity_ownership.rs b/crates/freshell-ws/src/identity_ownership.rs index 743fe164a..89a4616d5 100644 --- a/crates/freshell-ws/src/identity_ownership.rs +++ b/crates/freshell-ws/src/identity_ownership.rs @@ -895,6 +895,7 @@ mod tests { screenshots: crate::screenshot::ScreenshotBroker::new(state_broadcast_tx()), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/src/invariants.rs b/crates/freshell-ws/src/invariants.rs index 20bd009b9..7fb15471d 100644 --- a/crates/freshell-ws/src/invariants.rs +++ b/crates/freshell-ws/src/invariants.rs @@ -1197,6 +1197,7 @@ mod tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: std::sync::Arc::new(Vec::new()), diff --git a/crates/freshell-ws/src/lib.rs b/crates/freshell-ws/src/lib.rs index 8335f32ea..1e6d7c047 100644 --- a/crates/freshell-ws/src/lib.rs +++ b/crates/freshell-ws/src/lib.rs @@ -37,6 +37,7 @@ pub fn ready_build_id() -> Option<String> { pub mod activity; pub mod auto_resume; pub mod backpressure; +pub mod checkpoint_barrier; pub mod claude_signal; pub mod claude_truth; pub mod codex_association; @@ -259,6 +260,10 @@ pub struct WsState { /// gain exactly one `host_stats: Default::default()` arm each (the plan's /// >~6-site sweep rule). See [`crate::host_stats_collector`]. pub host_stats: crate::host_stats_collector::WsHostStatsState, + /// Server-owned pre-turn checkpoint capture barrier. The server wires the + /// same service here and into its managed fresh-agent gateway; test + /// states leave it absent so legacy sends keep their existing behavior. + pub checkpoint_send: crate::checkpoint_barrier::WsCheckpointState, /// The handler-scoped monotonic `terminals.changed` revision counter /// (`ws-handler.ts:566` `terminalsRevision`). SHARED with the REST /// `/api/terminals` PATCH/DELETE broadcasts (`terminals::TerminalsState`), @@ -1118,6 +1123,7 @@ pub(crate) fn test_ws_state() -> WsState { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/src/opencode_association.rs b/crates/freshell-ws/src/opencode_association.rs index 03eebe5cd..c81e9b628 100644 --- a/crates/freshell-ws/src/opencode_association.rs +++ b/crates/freshell-ws/src/opencode_association.rs @@ -575,6 +575,7 @@ mod tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: StdArc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: StdArc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: StdArc::new(Vec::new()), diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index af796ba4f..31e20aa8b 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -1053,6 +1053,41 @@ async fn handle_client_text( if let Some(reply) = fresh_agent_control_refusal(&message) { return send(ws_tx, &reply).await; } + if let ClientMessage::FreshAgentSend(fresh_send) = &message { + if fresh_send.checkpoint_intent.is_some() + && fresh_send + .request_id + .as_deref() + .is_none_or(|request_id| request_id.trim().is_empty()) + { + tracing::warn!( + event = "code_checkpoint.send_missing_request_id", + provider = ?fresh_send.provider, + session_id = %fresh_send.session_id, + "modern checkpoint send was rejected before capture or provider dispatch" + ); + return send( + ws_tx, + &ServerMessage::Error(ErrorMsg { + provider_preparation_failure: None, + code: ErrorCode::InvalidMessage, + message: "requestId is required when checkpointIntent is present".into(), + timestamp: crate::now_iso(), + actual_session_ref: None, + expected_session_ref: None, + request_id: None, + retry_after_ms: None, + terminal_exit_code: None, + terminal_id: None, + live_terminal_id: None, + owner_kind: None, + owner_generation: None, + owner_epoch: None, + }), + ) + .await; + } + } // Explicit managed fresh-agent rollout: every fresh-agent operation is // consumed by the external gateway when installed. It is never allowed // to fall through and mint a competing web-owned provider writer. @@ -1928,25 +1963,61 @@ async fn handle_client_text( // same pattern as the other `freshAgent.*` arms. `_` keeps swallowing only // `Amplifier` (no fresh-agent runtime, same as the `FreshAgentSend` arm). ClientMessage::FreshAgentAttach(attach) => { + let checkpoint_replay = state.checkpoint_send.service.clone(); match attach.provider { freshell_protocol::AgentProvider::Codex => { let fresh_codex = state.fresh_codex.clone(); tokio::spawn( - async move { fresh_codex.handle_attach(attach).await } + async move { + if let Some(service) = checkpoint_replay { + Arc::clone(&service) + .replay_for_session( + attach.provider, + attach.session_id.clone(), + attach.session_type, + ) + .await; + } + fresh_codex.handle_attach(attach).await + } .instrument(tracing::Span::current()), ); } freshell_protocol::AgentProvider::Claude => { let fresh_claude = state.fresh_claude.clone(); + let checkpoint_replay = state.checkpoint_send.service.clone(); tokio::spawn( - async move { fresh_claude.handle_attach(attach).await } + async move { + if let Some(service) = checkpoint_replay { + Arc::clone(&service) + .replay_for_session( + attach.provider, + attach.session_id.clone(), + attach.session_type, + ) + .await; + } + fresh_claude.handle_attach(attach).await + } .instrument(tracing::Span::current()), ); } freshell_protocol::AgentProvider::Opencode => { let fresh_opencode = state.fresh_opencode.clone(); + let checkpoint_replay = state.checkpoint_send.service.clone(); tokio::spawn( - async move { fresh_opencode.handle_attach(attach).await } + async move { + if let Some(service) = checkpoint_replay { + Arc::clone(&service) + .replay_for_session( + attach.provider, + attach.session_id.clone(), + attach.session_type, + ) + .await; + } + fresh_opencode.handle_attach(attach).await + } .instrument(tracing::Span::current()), ); } @@ -1955,6 +2026,32 @@ async fn handle_client_text( true } ClientMessage::FreshAgentSend(send) => { + if send.checkpoint_intent.is_some() { + if let Some(service) = state.checkpoint_send.service.clone() { + let captured_send = send.clone(); + let forward: futures::future::BoxFuture<'static, ()> = match send.provider { + freshell_protocol::AgentProvider::Codex => { + let fresh_codex = state.fresh_codex.clone(); + Box::pin(async move { fresh_codex.handle_send(send).await }) + } + freshell_protocol::AgentProvider::Claude => { + let fresh_claude = state.fresh_claude.clone(); + Box::pin(async move { fresh_claude.handle_send(send).await }) + } + freshell_protocol::AgentProvider::Opencode => { + let fresh_opencode = state.fresh_opencode.clone(); + Box::pin(async move { fresh_opencode.handle_send(send).await }) + } + freshell_protocol::AgentProvider::Amplifier => Box::pin(async {}), + }; + tokio::spawn( + Arc::clone(&service) + .barrier_and_forward(captured_send, forward) + .instrument(tracing::Span::current()), + ); + return true; + } + } match send.provider { freshell_protocol::AgentProvider::Codex => { let fresh_codex = state.fresh_codex.clone(); @@ -11608,6 +11705,7 @@ mod terminals_changed_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), @@ -11807,6 +11905,7 @@ mod terminal_kill_stop_wedge_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), @@ -12240,6 +12339,7 @@ mod terminal_meta_created_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: std::sync::Arc::new(Vec::new()), @@ -12884,6 +12984,7 @@ mod pane_reconcile_gate_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), @@ -13721,6 +13822,7 @@ mod host_stats_dispatch_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats, + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/auto_resume_e2e.rs b/crates/freshell-ws/tests/auto_resume_e2e.rs index d7e54e27e..00ac94433 100644 --- a/crates/freshell-ws/tests/auto_resume_e2e.rs +++ b/crates/freshell-ws/tests/auto_resume_e2e.rs @@ -523,6 +523,7 @@ async fn spawn_server_with_hub_and_ownership( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), diff --git a/crates/freshell-ws/tests/auto_resume_respawn.rs b/crates/freshell-ws/tests/auto_resume_respawn.rs index ede3284bf..568bb9b83 100644 --- a/crates/freshell-ws/tests/auto_resume_respawn.rs +++ b/crates/freshell-ws/tests/auto_resume_respawn.rs @@ -401,6 +401,7 @@ fn respawn_state_with_probe( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![common::sleeper_cli_spec("amplifier")]), diff --git a/crates/freshell-ws/tests/claude_session_rebind.rs b/crates/freshell-ws/tests/claude_session_rebind.rs index 3b0b11609..faa8b2437 100644 --- a/crates/freshell-ws/tests/claude_session_rebind.rs +++ b/crates/freshell-ws/tests/claude_session_rebind.rs @@ -215,6 +215,7 @@ async fn spawn_server_returning_state( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), diff --git a/crates/freshell-ws/tests/cli_connection_scheduling.rs b/crates/freshell-ws/tests/cli_connection_scheduling.rs index 4b0338771..b75ecd916 100644 --- a/crates/freshell-ws/tests/cli_connection_scheduling.rs +++ b/crates/freshell-ws/tests/cli_connection_scheduling.rs @@ -120,6 +120,7 @@ fn state(probe: Arc<ParkedProbe>, missing_program: &std::path::Path) -> WsState screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![cli]), diff --git a/crates/freshell-ws/tests/codex_managed_launch_e2e.rs b/crates/freshell-ws/tests/codex_managed_launch_e2e.rs index bcb9ab56a..9823e3a18 100644 --- a/crates/freshell-ws/tests/codex_managed_launch_e2e.rs +++ b/crates/freshell-ws/tests/codex_managed_launch_e2e.rs @@ -217,6 +217,7 @@ async fn spawn_server() -> ( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![codex_cli_spec()]), diff --git a/crates/freshell-ws/tests/codex_session_ref_resume.rs b/crates/freshell-ws/tests/codex_session_ref_resume.rs index 6bee85ba6..397e7bf8a 100644 --- a/crates/freshell-ws/tests/codex_session_ref_resume.rs +++ b/crates/freshell-ws/tests/codex_session_ref_resume.rs @@ -143,6 +143,7 @@ async fn spawn_server() -> (String, freshell_terminal::TerminalRegistry) { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![codex_cli_spec()]), diff --git a/crates/freshell-ws/tests/codex_sidecar_reattach_e2e.rs b/crates/freshell-ws/tests/codex_sidecar_reattach_e2e.rs index dc710d794..e20868e89 100644 --- a/crates/freshell-ws/tests/codex_sidecar_reattach_e2e.rs +++ b/crates/freshell-ws/tests/codex_sidecar_reattach_e2e.rs @@ -426,6 +426,7 @@ async fn spawn_server() -> (String, freshell_terminal::TerminalRegistry) { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![codex_cli_spec()]), diff --git a/crates/freshell-ws/tests/common/mod.rs b/crates/freshell-ws/tests/common/mod.rs index 5e7441a19..65e97b698 100644 --- a/crates/freshell-ws/tests/common/mod.rs +++ b/crates/freshell-ws/tests/common/mod.rs @@ -192,6 +192,7 @@ pub async fn spawn_server_with_specs_and_shared_settings( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -273,6 +274,7 @@ pub async fn spawn_server_with_specs( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -611,6 +613,7 @@ pub async fn spawn_server_with_specs_and_naming( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -691,6 +694,7 @@ pub async fn spawn_server_with_specs_and_auto_resume_rx( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -799,6 +803,7 @@ pub async fn spawn_server_with_specs_hub_and_state( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -888,6 +893,7 @@ pub async fn spawn_server_with_specs_and_state( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -980,6 +986,7 @@ pub async fn spawn_server_with_ledger( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -1073,6 +1080,7 @@ pub async fn spawn_server_with_ledger_and_state( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -1162,6 +1170,7 @@ pub async fn spawn_server_with_specs_and_activity( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -1249,6 +1258,7 @@ pub async fn spawn_server_with_specs_activity_and_codex_locator( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -1366,6 +1376,7 @@ pub async fn spawn_server_with_specs_activity_codex_locator_and_naming( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), @@ -1479,6 +1490,7 @@ pub async fn spawn_server_with_create_protect_probes( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/cross_kind_liveness.rs b/crates/freshell-ws/tests/cross_kind_liveness.rs index fbd2239fa..08f39e99b 100644 --- a/crates/freshell-ws/tests/cross_kind_liveness.rs +++ b/crates/freshell-ws/tests/cross_kind_liveness.rs @@ -564,6 +564,7 @@ async fn build_ws_state_with_probe( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), diff --git a/crates/freshell-ws/tests/diag01_lifecycle_events.rs b/crates/freshell-ws/tests/diag01_lifecycle_events.rs index 85318dba7..ea5792196 100644 --- a/crates/freshell-ws/tests/diag01_lifecycle_events.rs +++ b/crates/freshell-ws/tests/diag01_lifecycle_events.rs @@ -241,6 +241,7 @@ async fn spawn_server(ping_interval_ms: u64) -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/freshagent_claude_attach.rs b/crates/freshell-ws/tests/freshagent_claude_attach.rs index f4e088e00..ffb2f3895 100644 --- a/crates/freshell-ws/tests/freshagent_claude_attach.rs +++ b/crates/freshell-ws/tests/freshagent_claude_attach.rs @@ -203,6 +203,7 @@ async fn spawn_server() -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/freshagent_claude_kill_interrupt.rs b/crates/freshell-ws/tests/freshagent_claude_kill_interrupt.rs index a78c1a296..2894a497a 100644 --- a/crates/freshell-ws/tests/freshagent_claude_kill_interrupt.rs +++ b/crates/freshell-ws/tests/freshagent_claude_kill_interrupt.rs @@ -203,6 +203,7 @@ async fn spawn_server() -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/freshagent_claude_rollback.rs b/crates/freshell-ws/tests/freshagent_claude_rollback.rs index f46a58ba2..6f6cab3ad 100644 --- a/crates/freshell-ws/tests/freshagent_claude_rollback.rs +++ b/crates/freshell-ws/tests/freshagent_claude_rollback.rs @@ -609,6 +609,7 @@ async fn spawn_server_with_rollback_rig( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/freshagent_session_lease.rs b/crates/freshell-ws/tests/freshagent_session_lease.rs index 5e415ede7..34a636bc6 100644 --- a/crates/freshell-ws/tests/freshagent_session_lease.rs +++ b/crates/freshell-ws/tests/freshagent_session_lease.rs @@ -307,6 +307,7 @@ async fn spawn_server() -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/hello_capabilities.rs b/crates/freshell-ws/tests/hello_capabilities.rs index 723537934..a8b382d69 100644 --- a/crates/freshell-ws/tests/hello_capabilities.rs +++ b/crates/freshell-ws/tests/hello_capabilities.rs @@ -83,6 +83,7 @@ async fn spawn_server() -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/hello_timeout.rs b/crates/freshell-ws/tests/hello_timeout.rs index 83facd2ea..739aafe18 100644 --- a/crates/freshell-ws/tests/hello_timeout.rs +++ b/crates/freshell-ws/tests/hello_timeout.rs @@ -85,6 +85,7 @@ async fn spawn_server(hello_timeout_ms: u64) -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/keepalive.rs b/crates/freshell-ws/tests/keepalive.rs index 65b4371b2..f3fe7c2ad 100644 --- a/crates/freshell-ws/tests/keepalive.rs +++ b/crates/freshell-ws/tests/keepalive.rs @@ -86,6 +86,7 @@ async fn spawn_server( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/max_payload.rs b/crates/freshell-ws/tests/max_payload.rs index cf0937802..1cfce3f53 100644 --- a/crates/freshell-ws/tests/max_payload.rs +++ b/crates/freshell-ws/tests/max_payload.rs @@ -86,6 +86,7 @@ async fn spawn_server(ws_max_payload_bytes: usize) -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/opencode_switch_rebind.rs b/crates/freshell-ws/tests/opencode_switch_rebind.rs index 85fc91ece..80c2e1c51 100644 --- a/crates/freshell-ws/tests/opencode_switch_rebind.rs +++ b/crates/freshell-ws/tests/opencode_switch_rebind.rs @@ -274,6 +274,7 @@ async fn spawn_server_returning_state( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(cli_commands), diff --git a/crates/freshell-ws/tests/origin_policy.rs b/crates/freshell-ws/tests/origin_policy.rs index c6bb7d3e7..abb869d0e 100644 --- a/crates/freshell-ws/tests/origin_policy.rs +++ b/crates/freshell-ws/tests/origin_policy.rs @@ -77,6 +77,7 @@ async fn spawn_server(allowed_origins: Vec<String>) -> (String, String) { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/paced_replay.rs b/crates/freshell-ws/tests/paced_replay.rs index ef23f476e..a08776dde 100644 --- a/crates/freshell-ws/tests/paced_replay.rs +++ b/crates/freshell-ws/tests/paced_replay.rs @@ -307,6 +307,7 @@ async fn spawn_server_with( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/pane_reconcile.rs b/crates/freshell-ws/tests/pane_reconcile.rs index 6f70ffddd..101ca555a 100644 --- a/crates/freshell-ws/tests/pane_reconcile.rs +++ b/crates/freshell-ws/tests/pane_reconcile.rs @@ -158,6 +158,7 @@ async fn spawn_server_with_probe( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/pane_reconcile_freshagent.rs b/crates/freshell-ws/tests/pane_reconcile_freshagent.rs index 0ca2caa47..e09625aba 100644 --- a/crates/freshell-ws/tests/pane_reconcile_freshagent.rs +++ b/crates/freshell-ws/tests/pane_reconcile_freshagent.rs @@ -228,6 +228,7 @@ async fn spawn_server_with_probe(probe: Arc<StubProbe>) -> Server { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/rest_claude_identity.rs b/crates/freshell-ws/tests/rest_claude_identity.rs index 260f6196f..b5ab0bd57 100644 --- a/crates/freshell-ws/tests/rest_claude_identity.rs +++ b/crates/freshell-ws/tests/rest_claude_identity.rs @@ -115,6 +115,7 @@ async fn spawn_merged_server() -> Harness { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::clone(&cli_commands), diff --git a/crates/freshell-ws/tests/rest_locator_identity.rs b/crates/freshell-ws/tests/rest_locator_identity.rs index 7821884da..43a389848 100644 --- a/crates/freshell-ws/tests/rest_locator_identity.rs +++ b/crates/freshell-ws/tests/rest_locator_identity.rs @@ -119,6 +119,7 @@ async fn spawn_merged_server( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::clone(&cli_commands), diff --git a/crates/freshell-ws/tests/rest_ws_shared_gate.rs b/crates/freshell-ws/tests/rest_ws_shared_gate.rs index 6be3960f3..f3170a9bb 100644 --- a/crates/freshell-ws/tests/rest_ws_shared_gate.rs +++ b/crates/freshell-ws/tests/rest_ws_shared_gate.rs @@ -155,6 +155,7 @@ async fn spawn_combined_server( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/restore_plan_queue_cap.rs b/crates/freshell-ws/tests/restore_plan_queue_cap.rs index c5142bd21..4a761850c 100644 --- a/crates/freshell-ws/tests/restore_plan_queue_cap.rs +++ b/crates/freshell-ws/tests/restore_plan_queue_cap.rs @@ -130,6 +130,7 @@ async fn spawn_server( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![ diff --git a/crates/freshell-ws/tests/restore_spawn_gate.rs b/crates/freshell-ws/tests/restore_spawn_gate.rs index 852137a71..2baba5920 100644 --- a/crates/freshell-ws/tests/restore_spawn_gate.rs +++ b/crates/freshell-ws/tests/restore_spawn_gate.rs @@ -139,6 +139,7 @@ async fn spawn_server( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![ diff --git a/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs b/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs index 25f51a3ae..6eb6c686d 100644 --- a/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs +++ b/crates/freshell-ws/tests/restore_stale_cwd_rehome.rs @@ -49,6 +49,7 @@ async fn spawn_server_with_default_cwd( pane_ledger: std::sync::Arc::new(freshell_ws::pane_ledger::PaneLedger::disabled()), identity: freshell_ws::identity::TerminalIdentityRegistry::new(), host_stats: Default::default(), + checkpoint_send: Default::default(), auth_token: Arc::clone(&auth_token), server_instance_id: Arc::new("srv-test".to_string()), boot_id: Arc::new("boot-test".to_string()), diff --git a/crates/freshell-ws/tests/restore_storm.rs b/crates/freshell-ws/tests/restore_storm.rs index 873cbcae9..c198b5dd6 100644 --- a/crates/freshell-ws/tests/restore_storm.rs +++ b/crates/freshell-ws/tests/restore_storm.rs @@ -136,6 +136,7 @@ async fn spawn_server( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![ diff --git a/crates/freshell-ws/tests/resume_validation_gate.rs b/crates/freshell-ws/tests/resume_validation_gate.rs index 90016c529..60de4a133 100644 --- a/crates/freshell-ws/tests/resume_validation_gate.rs +++ b/crates/freshell-ws/tests/resume_validation_gate.rs @@ -172,6 +172,7 @@ async fn spawn_server_with_probe( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![ @@ -370,6 +371,7 @@ async fn spawn_managed_codex_server_with_probe( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(vec![codex_cli_spec()]), diff --git a/crates/freshell-ws/tests/safe08_restore_diagnostics.rs b/crates/freshell-ws/tests/safe08_restore_diagnostics.rs index 705985db6..4578063a4 100644 --- a/crates/freshell-ws/tests/safe08_restore_diagnostics.rs +++ b/crates/freshell-ws/tests/safe08_restore_diagnostics.rs @@ -172,6 +172,7 @@ async fn spawn_server() -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/sessions_prefs.rs b/crates/freshell-ws/tests/sessions_prefs.rs index 58bf2ff3d..7fd98dcf8 100644 --- a/crates/freshell-ws/tests/sessions_prefs.rs +++ b/crates/freshell-ws/tests/sessions_prefs.rs @@ -90,6 +90,7 @@ async fn spawn_server() -> ( screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: interest.clone(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/term09_output_queue.rs b/crates/freshell-ws/tests/term09_output_queue.rs index 6e690b056..08b20daec 100644 --- a/crates/freshell-ws/tests/term09_output_queue.rs +++ b/crates/freshell-ws/tests/term09_output_queue.rs @@ -201,6 +201,7 @@ async fn spawn_server(term09: Term09Config) -> String { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-ws/tests/ui_layout_sync.rs b/crates/freshell-ws/tests/ui_layout_sync.rs index 4486b3787..88e164251 100644 --- a/crates/freshell-ws/tests/ui_layout_sync.rs +++ b/crates/freshell-ws/tests/ui_layout_sync.rs @@ -105,6 +105,7 @@ async fn spawn_server() -> (String, String, LayoutStore) { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index 26022244b..48c2dcda8 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -903,7 +903,7 @@ export const FreshAgentSendSchema = z.object({ * correlated with this provider attempt and dispatched as part of this * same send. Explicit workspace scope is persisted pane intent only. */ checkpointIntent: z.object({ - logicalAttemptId: z.string().min(1).max(128), + logicalAttemptId: z.string().trim().min(1).max(128), workspace: z.object({ path: z.string().min(1).max(4096), provenance: z.enum(['directory-picker', 'resolved-project', 'legacy-checkpoint']), @@ -927,6 +927,14 @@ export const FreshAgentSendSchema = z.object({ mediaType: z.string(), data: z.string(), })).optional(), +}).superRefine((send, context) => { + if (send.checkpointIntent && !send.requestId?.trim()) { + context.addIssue({ + code: z.ZodIssueCode.custom, + path: ['requestId'], + message: 'requestId is required when checkpointIntent is present', + }) + } }) export const FreshAgentInterruptSchema = z.object({ @@ -1858,6 +1866,27 @@ export type SdkRestoreFailureCode = | 'RESTORE_DIVERGED' | 'RESTORE_STALE_REVISION' +export const FreshAgentCheckpointOutcomeSchema = z.object({ + type: z.literal('freshAgent.checkpoint.outcome'), + logicalAttemptId: z.string().min(1), + requestId: z.string().min(1), + provider: z.enum(['claude', 'codex', 'opencode']), + sessionId: z.string().min(1), + sessionType: z.enum(['freshclaude', 'freshcodex', 'kilroy', 'freshopencode']), + status: z.enum(['available', 'unavailable']), + checkpointId: z.string().min(1).optional(), + workspace: z.object({ + path: z.string().min(1), + provenance: z.enum(['directory-picker', 'resolved-project', 'legacy-checkpoint']), + }).strict().optional(), + code: z.string().optional(), + reason: z.string().optional(), + nativeTurnId: z.string().min(1).optional(), + displayTurnId: z.string().min(1).optional(), +}).strict() + +export type FreshAgentCheckpointOutcome = z.infer<typeof FreshAgentCheckpointOutcomeSchema> + export type FreshAgentServerMessage = | { type: 'freshAgent.created'; requestId: string; sessionId: string; sessionType: string; provider: string; runtimeProvider: string; sessionRef?: { provider: string; sessionId: string }; sessionName?: SessionNameRecord; nameRef?: SessionNameRef } | { type: 'freshAgent.create.failed'; requestId: string; code: string; message: string; retryable?: boolean @@ -1868,7 +1897,7 @@ export type FreshAgentServerMessage = ownerKind?: 'terminal' | 'fresh-agent' ownerGeneration?: number ownerEpoch?: number } - | { type: 'freshAgent.checkpoint.outcome'; logicalAttemptId: string; requestId: string; provider: 'claude' | 'codex' | 'opencode'; sessionId: string; sessionType: 'freshclaude' | 'freshcodex' | 'kilroy' | 'freshopencode'; status: 'available' | 'unavailable'; checkpointId?: string; workspace?: { path: string; provenance: 'directory-picker' | 'resolved-project' | 'legacy-checkpoint' }; code?: string; reason?: string } + | FreshAgentCheckpointOutcome | { type: 'freshAgent.send.accepted'; requestId: string; sessionId: string; sessionType: string; provider: string; submittedTurnId?: string; cwd?: string } | { type: 'freshAgent.event'; sessionId: string; sessionType: string; provider: string; event: unknown } | { type: 'freshAgent.session.materialized'; previousSessionId: string; sessionId: string; sessionType: string; provider: string; sessionRef?: { provider: string; sessionId: string }; sessionName?: SessionNameRecord; nameRef?: SessionNameRef } diff --git a/test/unit/shared/ws-protocol.checkpoint-outcome.test.ts b/test/unit/shared/ws-protocol.checkpoint-outcome.test.ts new file mode 100644 index 000000000..79e49a2b7 --- /dev/null +++ b/test/unit/shared/ws-protocol.checkpoint-outcome.test.ts @@ -0,0 +1,55 @@ +import { describe, expect, it } from 'vitest' +import { + FreshAgentCheckpointOutcomeSchema, + FreshAgentSendSchema, +} from '../../../shared/ws-protocol.js' + +describe('FreshAgentCheckpointOutcomeSchema', () => { + const base = { + type: 'freshAgent.checkpoint.outcome', + logicalAttemptId: 'attempt-1', + requestId: 'request-1', + provider: 'opencode', + sessionId: 'session-1', + sessionType: 'freshopencode', + status: 'available', + checkpointId: 'a'.repeat(40), + } as const + + it('accepts optional native and display turn identities', () => { + expect(FreshAgentCheckpointOutcomeSchema.parse({ + ...base, + nativeTurnId: 'native-turn-1', + displayTurnId: 'display-turn-1', + })).toMatchObject({ + nativeTurnId: 'native-turn-1', + displayTurnId: 'display-turn-1', + }) + expect(FreshAgentCheckpointOutcomeSchema.parse(base).nativeTurnId).toBeUndefined() + }) + + it('rejects malformed or unrecognized identity fields', () => { + expect(() => FreshAgentCheckpointOutcomeSchema.parse({ ...base, nativeTurnId: 42 })).toThrow() + expect(() => FreshAgentCheckpointOutcomeSchema.parse({ ...base, displayTurnId: '' })).toThrow() + expect(() => FreshAgentCheckpointOutcomeSchema.parse({ ...base, providerTurnId: 'legacy' })).toThrow() + }) + + it('requires request correlation for a checkpoint-bearing send', () => { + const send = { + type: 'freshAgent.send', + sessionId: 'session-1', + sessionType: 'freshopencode', + provider: 'opencode', + checkpointIntent: { logicalAttemptId: 'attempt-1' }, + text: 'hello', + } + expect(FreshAgentSendSchema.safeParse(send).success).toBe(false) + expect(FreshAgentSendSchema.parse({ ...send, requestId: 'request-1' }).requestId).toBe('request-1') + expect(FreshAgentSendSchema.safeParse({ ...send, requestId: ' ' }).success).toBe(false) + expect(FreshAgentSendSchema.safeParse({ + ...send, + requestId: 'request-1', + checkpointIntent: { logicalAttemptId: ' ' }, + }).success).toBe(false) + }) +}) From 20977fc9547aa88132ed4e49f60aa5c91f599cc6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 03:13:17 -0700 Subject: [PATCH 310/493] Bind checkpoint outcomes to provider user rows --- src/components/fresh-agent/FreshAgentView.tsx | 38 ++++++++++--- .../fresh-agent/FreshAgentView.test.tsx | 55 ++++++++++++++++--- 2 files changed, 78 insertions(+), 15 deletions(-) diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 7e742df9c..16e4f1b9b 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -254,6 +254,9 @@ type CheckpointTurnAvailability = FreshAgentTurnRewindAvailability & { workspace?: NonNullable<FreshAgentPaneContent['checkpointWorkspace']> checkpointId?: string code?: string + /** Exact provider identity, present only after observing the real user row. */ + nativeTurnId?: string + displayTurnId?: string } const CHECKPOINT_FAILURE_FALLBACK = 'The project workspace could not be saved before this message was sent.' @@ -327,12 +330,16 @@ function checkpointOutcomeAvailability(outcome: FreshAgentCheckpointOutcome): Ch status: 'available', checkpointId: outcome.checkpointId, ...(outcome.workspace ? { workspace: outcome.workspace } : {}), + ...(outcome.nativeTurnId ? { nativeTurnId: outcome.nativeTurnId } : {}), + ...(outcome.displayTurnId ? { displayTurnId: outcome.displayTurnId } : {}), } : { status: 'unavailable', reason: outcome.reason ?? CHECKPOINT_FAILURE_FALLBACK, ...(outcome.code ? { code: outcome.code } : {}), ...(outcome.workspace ? { workspace: outcome.workspace } : {}), + ...(outcome.nativeTurnId ? { nativeTurnId: outcome.nativeTurnId } : {}), + ...(outcome.displayTurnId ? { displayTurnId: outcome.displayTurnId } : {}), } } @@ -1064,21 +1071,22 @@ export function FreshAgentView({ if (submittedTurnId) { next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, submittedTurnId), availability) } + if (availability.displayTurnId) { + next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, availability.displayTurnId), availability) + } + if (availability.nativeTurnId) { + next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, availability.nativeTurnId), availability) + } return next }) if (!requestId || availability.status === 'pending') return - const matchingTurn = submittedTurnId - ? snapshotRef.current?.turns.find((turn) => turn.role === 'user' - && checkpointTurnIdentity(paneContentRef.current.provider, getFreshAgentDisplayTurnKey(turn)) - === checkpointTurnIdentity(paneContentRef.current.provider, submittedTurnId)) - : undefined const outcome: FreshAgentCheckpointOutcome = { version: 1, requestId, conversationRequestId: scope, - ...(submittedTurnId ? { displayTurnId: submittedTurnId } : {}), - ...(matchingTurn?.id ? { nativeTurnId: matchingTurn.id } : {}), + ...(availability.displayTurnId ? { displayTurnId: availability.displayTurnId } : {}), + ...(availability.nativeTurnId ? { nativeTurnId: availability.nativeTurnId } : {}), status: availability.status, ...(availability.status === 'available' && availability.checkpointId ? { checkpointId: availability.checkpointId } @@ -2738,6 +2746,16 @@ export function FreshAgentView({ const workspace = normalizeFreshAgentCheckpointWorkspace(message.workspace) ?? pending.checkpointWorkspace + const checkpointTurnId = (value: unknown): string | undefined => ( + typeof value === 'string' + && value.length > 0 + && value.length <= 256 + && !/[\u0000-\u001f\u007f-\u009f]/.test(value) + ? value + : undefined + ) + const nativeTurnId = checkpointTurnId(message.nativeTurnId) + const displayTurnId = checkpointTurnId(message.displayTurnId) let checkpointOutcome: CheckpointTurnAvailability if ( message.status === 'available' @@ -2750,6 +2768,8 @@ export function FreshAgentView({ status: 'available', checkpointId: message.checkpointId, ...(workspace ? { workspace } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), } } else if (message.status === 'unavailable') { const suppliedReason = typeof message.reason === 'string' @@ -2764,6 +2784,8 @@ export function FreshAgentView({ reason: suppliedReason || CHECKPOINT_FAILURE_FALLBACK, ...(suppliedCode ? { code: suppliedCode } : {}), ...(workspace ? { workspace } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), } } else { checkpointOutcome = { @@ -2771,6 +2793,8 @@ export function FreshAgentView({ reason: 'The server did not return a saved code checkpoint.', code: 'checkpoint_capture_failed', ...(workspace ? { workspace } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), } } recordPendingSendMetadata(requestId, { checkpointOutcome }) diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index 76d2364e2..f01f75e12 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3438,8 +3438,9 @@ describe('FreshAgentView', () => { capabilities: { send: true, interrupt: true, fork: true }, turns: [ { - id: 'native-user-turn:row-0', - turnId: 'display-user-1:row-0', + id: 'codex-turn-17:row-0', + turnId: 'codex-turn-17:row-0', + nativeTurnId: 'codex-turn-17', role: 'user', summary: 'Ship it', items: [{ id: 'user-text-1', kind: 'text', text: 'Ship it' }], @@ -3459,8 +3460,9 @@ describe('FreshAgentView', () => { capabilities: { send: true, interrupt: true, fork: true }, turns: [ { - id: 'native-user-turn:row-0', - turnId: 'display-user-1:row-0', + id: 'codex-turn-17:row-0', + turnId: 'codex-turn-17:row-0', + nativeTurnId: 'codex-turn-17', role: 'user', summary: 'Ship it', items: [{ id: 'user-text-1', kind: 'text', text: 'Ship it' }], @@ -3558,7 +3560,27 @@ describe('FreshAgentView', () => { onMessage?.({ type: 'freshAgent.send.accepted', requestId, - submittedTurnId: 'display-user-1', + submittedTurnId: 'codex-turn-17', + }) + }) + const acceptedOutcome = getFreshAgentPaneContent(store).checkpointOutcomes?.find( + (outcome) => outcome.requestId === requestId, + ) + expect(acceptedOutcome).not.toHaveProperty('nativeTurnId') + expect(acceptedOutcome).not.toHaveProperty('displayTurnId') + act(() => { + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', + requestId, + logicalAttemptId, + sessionId: 'thread-normalized-send', + sessionType: 'freshcodex', + provider: 'codex', + status: 'available', + checkpointId: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + workspace: resolvedCheckpointWorkspace, + nativeTurnId: 'codex-turn-17', + displayTurnId: 'codex-turn-17:row-0', }) }) @@ -3589,7 +3611,8 @@ describe('FreshAgentView', () => { expect(persistedOutcomes).toEqual(expect.arrayContaining([expect.objectContaining({ requestId, conversationRequestId: 'req-normalized-send', - displayTurnId: 'display-user-1', + displayTurnId: 'codex-turn-17:row-0', + nativeTurnId: 'codex-turn-17', status: 'available', checkpointId: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', workspace: resolvedCheckpointWorkspace, @@ -3911,7 +3934,22 @@ describe('FreshAgentView', () => { sessionId: 'thread-checkpoint-failure', sessionType: 'freshcodex', provider: 'codex', - submittedTurnId: 'display-failed-turn', + submittedTurnId: 'native-failed-turn', + }) + }) + act(() => { + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', + requestId, + logicalAttemptId, + sessionId: 'thread-checkpoint-failure', + sessionType: 'freshcodex', + provider: 'codex', + status: 'unavailable', + code: 'workspace_unavailable', + reason: workspaceReason, + nativeTurnId: 'native-failed-turn', + displayTurnId: 'native-failed-turn:row-0', }) }) @@ -3920,7 +3958,8 @@ describe('FreshAgentView', () => { version: 1, requestId, conversationRequestId: 'req-checkpoint-failure', - displayTurnId: 'display-failed-turn', + displayTurnId: 'native-failed-turn:row-0', + nativeTurnId: 'native-failed-turn', status: 'unavailable', code: 'workspace_unavailable', reason: workspaceReason, From 838fa8303a93e0c44c26fb0b5269d84b0e2c8467 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 04:21:58 -0700 Subject: [PATCH 311/493] fix: persist checkpoint results across pane recovery --- crates/freshell-freshagent/src/codex.rs | 148 +++++++------- crates/freshell-opencode/src/lib.rs | 11 +- .../freshell-protocol/src/client_messages.rs | 6 +- crates/freshell-server/src/checkpoints.rs | 150 +++++++++----- .../freshell-server/src/fresh_agent_proxy.rs | 8 +- crates/freshell-server/src/main.rs | 2 +- .../tests/code_checkpoint_permissions.rs | 36 ++-- crates/freshell-ws/src/checkpoint_barrier.rs | 2 +- crates/freshell-ws/src/terminal.rs | 14 +- src/components/fresh-agent/FreshAgentView.tsx | 81 ++++++-- ...code-checkpoint-workspace-external.spec.ts | 16 +- ...-agent-checkpoint-unavailable-rust.spec.ts | 124 ++++++++++++ .../fresh-agent/FreshAgentView.test.tsx | 191 ++++++++++++++++++ 13 files changed, 601 insertions(+), 188 deletions(-) diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index 0b68b27b8..c5d87348a 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -47,26 +47,26 @@ use std::sync::{Arc, Mutex as StdMutex}; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; use axum::{ - Json, Router, extract::State, http::{HeaderMap, StatusCode}, response::{IntoResponse, Response}, routing::patch, + Json, Router, }; -use serde_json::{Map, Value, json}; -use tokio::sync::{Mutex as TokioMutex, oneshot}; +use serde_json::{json, Map, Value}; +use tokio::sync::{oneshot, Mutex as TokioMutex}; use freshell_codex::launch_lifecycle::{ - SIDECAR_START_BUDGET, allocate_loopback_port, drain_child_io, + allocate_loopback_port, drain_child_io, SIDECAR_START_BUDGET, }; -use freshell_codex::launch_plan::{CodexSidecarLaunchContext, codex_sidecar_spawn_spec}; -use freshell_codex::transport::{TungsteniteTransport, reap_owned_codex_sidecars}; +use freshell_codex::launch_plan::{codex_sidecar_spawn_spec, CodexSidecarLaunchContext}; +use freshell_codex::transport::{reap_owned_codex_sidecars, TungsteniteTransport}; use freshell_codex::{ - CODEX_SIDECAR_OWNERSHIP_ENV, CodexAdapterEvent, CodexAppServerClient, CodexAppServerError, + mint_ownership_id, next_monotonic_turn_complete_at, normalize_codex_thread_status, + normalize_freshcodex_effort, normalize_freshcodex_model, read_rollout_history_mode, + to_codex_reasoning_effort, CodexAdapterEvent, CodexAppServerClient, CodexAppServerError, CodexNotification, CodexStatus, CodexSubscription, HistoryMode, StartThreadParams, - StartTurnParams, ThreadForkParams, mint_ownership_id, next_monotonic_turn_complete_at, - normalize_codex_thread_status, normalize_freshcodex_effort, normalize_freshcodex_model, - read_rollout_history_mode, to_codex_reasoning_effort, + StartTurnParams, ThreadForkParams, CODEX_SIDECAR_OWNERSHIP_ENV, }; use freshell_protocol::{ ErrorCode, ErrorMsg, FreshAgentAttach, FreshAgentCompact, FreshAgentConfigure, @@ -78,7 +78,7 @@ use freshell_protocol::{ use freshell_terminal::FrameSink; use crate::summary::{ - SUMMARY_KIND_AUTHORED, SUMMARY_KIND_ECHO, TOOL_ERROR_LABEL, TOOL_RESULT_LABEL, truncate_summary, + truncate_summary, SUMMARY_KIND_AUTHORED, SUMMARY_KIND_ECHO, TOOL_ERROR_LABEL, TOOL_RESULT_LABEL, }; use crate::{FreshAgentCreateDedup, FreshAgentCreateOutcome, SharedPaneIdentitySink}; @@ -2408,13 +2408,13 @@ impl FreshCodexState { naming_handle: Option<String>, ) { let _ = &mut own_ticket; // moved to finish_create below on the success path - // b8ke focused episode-2 round-2 F4: the claim exists — register the - // watchdog machinery NOW, before the spawn + thread/resume awaits (up - // to 45s): the cancellation slot + the settle guard (held to THIS - // handler's unwind) + the partial runtime (pid None until the spawn - // hook arms the real identity). A slow startup swept at 30s is then - // cancellable-and-confirmable, never a registration-ignored - // StaleStart fence. + // b8ke focused episode-2 round-2 F4: the claim exists — register the + // watchdog machinery NOW, before the spawn + thread/resume awaits (up + // to 45s): the cancellation slot + the settle guard (held to THIS + // handler's unwind) + the partial runtime (pid None until the spawn + // hook arms the real identity). A slow startup swept at 30s is then + // cancellable-and-confirmable, never a registration-ignored + // StaleStart fence. let resume_start_pid_slot = crate::ownership_lane::sidecar_pid_cancel_slot(); let _resume_start_cancellation = crate::ownership_lane::register_start_cancellation_for_ticket( @@ -2689,10 +2689,10 @@ impl FreshCodexState { naming_handle: Option<String>, ) { let _ = &mut own_ticket; // consumed at the commit below on the success path - // D8 (focused-ep1-r5 Finding 2): a HOLLOW `Some` (a partially - // initialized client's hello — all fields absent) behaves like `None` - // on every decision below: the park, the eviction-guard adopt, and - // the binding write's stamps. + // D8 (focused-ep1-r5 Finding 2): a HOLLOW `Some` (a partially + // initialized client's hello — all fields absent) behaves like `None` + // on every decision below: the park, the eviction-guard adopt, and + // the binding write's stamps. let provenance = provenance.filter(|p| p.is_meaningful()); // kata b8ke Task 3: the sidecar pid for the commit's owner identity // (captured before `child` moves into the exit watcher). @@ -11359,8 +11359,8 @@ pub(crate) mod tests { use std::sync::{Arc, Mutex, OnceLock}; use tracing::field::{Field, Visit}; use tracing::{Event, Subscriber}; - use tracing_subscriber::Layer; use tracing_subscriber::layer::{Context, SubscriberExt}; + use tracing_subscriber::Layer; #[derive(Debug, Clone, Default)] pub struct CapturedEvent { @@ -13200,10 +13200,9 @@ pub(crate) mod tests { _ => panic!("the seed claim must grant"), }; let generation = seed_ticket.as_ref().expect("the seed ticket").generation(); - assert!( - st.commit_lane_claim_at(&mut seed_ticket, "thread-r29-f3", "thread-r29-f3", None,) - .is_ok() - ); + assert!(st + .commit_lane_claim_at(&mut seed_ticket, "thread-r29-f3", "thread-r29-f3", None,) + .is_ok()); insert_fake_session_with_real_consumer( &st, @@ -13278,10 +13277,9 @@ pub(crate) mod tests { _ => panic!("the seed claim must grant"), }; let generation = seed_ticket.as_ref().expect("the seed ticket").generation(); - assert!( - st.commit_lane_claim_at(&mut seed_ticket, "thread-ep5-r2", "thread-ep5-r2", None) - .is_ok() - ); + assert!(st + .commit_lane_claim_at(&mut seed_ticket, "thread-ep5-r2", "thread-ep5-r2", None) + .is_ok()); insert_fake_session_with_real_consumer( &st, @@ -13729,12 +13727,10 @@ pub(crate) mod tests { leases.claim(PROVIDER, session_id, "replacement", 1), crate::session_lease::FreshSessionClaim::Acquired )); - assert!( - !condemned_priors - .lock() - .expect("condemned priors lock") - .contains_key(session_id) - ); + assert!(!condemned_priors + .lock() + .expect("condemned priors lock") + .contains_key(session_id)); } /// b8ke focused review FR9: the half-fenced kill refusal carries the @@ -14156,8 +14152,8 @@ pub(crate) mod tests { /// cleaned) while the answer still reports `success:false`: the kill /// visibly fails, never masquerading as clean success nor clean failure. #[tokio::test] - async fn a_kill_whose_close_persists_despite_the_reported_error_ends_the_session_and_fails_visibly() - { + async fn a_kill_whose_close_persists_despite_the_reported_error_ends_the_session_and_fails_visibly( + ) { let (transport, _peer) = freshell_codex::new_channel_transport(); let (client, _notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -14224,8 +14220,8 @@ pub(crate) mod tests { /// row revived — then the lane parks pre-registration while the kill /// lands.) #[tokio::test(flavor = "multi_thread")] - async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_attached_thread() - { + async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_attached_thread( + ) { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd("{}"); let (st, mut rx, fake) = state_with_sink(); @@ -14333,8 +14329,8 @@ pub(crate) mod tests { /// re-check tears the registered session down and the create answers /// failed. #[tokio::test(flavor = "multi_thread")] - async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_resumed_create() - { + async fn a_kill_landing_between_the_commit_and_the_registration_still_closes_the_resumed_create( + ) { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd("{}"); let (st, mut rx, fake) = state_with_sink(); @@ -14944,8 +14940,8 @@ pub(crate) mod tests { /// folded. Repeat the exact no-slack delivery window enough times to exercise the /// independent RPC waiter and notification-consumer tasks on the multi-thread runtime. #[tokio::test(flavor = "multi_thread", worker_threads = 4)] - async fn handle_send_always_broadcasts_accepted_before_an_immediate_completion_snapshot_and_chime() - { + async fn handle_send_always_broadcasts_accepted_before_an_immediate_completion_snapshot_and_chime( + ) { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -15225,8 +15221,8 @@ pub(crate) mod tests { /// between the stale and the matching completion stays BUSY_TURN; the gate /// releases EXACTLY at the id-matched completion. #[tokio::test] - async fn a_stale_prior_completion_never_clears_the_compact_window_and_only_the_matching_completion_does() - { + async fn a_stale_prior_completion_never_clears_the_compact_window_and_only_the_matching_completion_does( + ) { let (st, _rx_boot) = state_with_bus(); let (peer, mut wire) = insert_idle_compact_session(&st, "thread-cstale").await; @@ -15527,8 +15523,8 @@ pub(crate) mod tests { /// compact window so the newer turn's own matching completion can reopen the /// rollback gate. #[tokio::test] - async fn an_idless_compact_completion_inside_a_newer_turns_window_stays_busy_and_retires_the_compact() - { + async fn an_idless_compact_completion_inside_a_newer_turns_window_stays_busy_and_retires_the_compact( + ) { let (st, _rx_boot) = state_with_bus(); let (peer, mut wire) = insert_idle_compact_session(&st, "thread-cnewer-idless").await; @@ -17191,8 +17187,8 @@ pub(crate) mod tests { } #[tokio::test(flavor = "multi_thread")] - async fn handle_fork_threads_fork_archive_then_child_unarchive_resume_across_two_sidecars_and_replies_forked() - { + async fn handle_fork_threads_fork_archive_then_child_unarchive_resume_across_two_sidecars_and_replies_forked( + ) { let _guard = ENV_LOCK.lock().await; let log_path = std::env::temp_dir().join(format!( "freshell-fork-oplog-{}-{}.jsonl", @@ -17755,8 +17751,8 @@ pub(crate) mod tests { /// guard) but finds a LIVE incumbent at registration time adopts that /// incumbent — this arm must restamp exactly like the fast path. #[tokio::test] - async fn adopt_live_create_through_the_finish_create_eviction_guard_restamps_the_current_connection() - { + async fn adopt_live_create_through_the_finish_create_eviction_guard_restamps_the_current_connection( + ) { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd( &json!({ @@ -17882,8 +17878,8 @@ pub(crate) mod tests { /// incumbent's already-parked Some to None, and writes nothing (the /// ledger's keep-when-None merge preserves the row's stamps). #[tokio::test] - async fn adopt_live_create_with_no_connection_provenance_keeps_the_parked_stamps_and_writes_nothing() - { + async fn adopt_live_create_with_no_connection_provenance_keeps_the_parked_stamps_and_writes_nothing( + ) { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd("{}"); let (st, mut rx) = state_with_bus(); @@ -18033,8 +18029,8 @@ pub(crate) mod tests { /// with a parent that parks nothing the child stamps fall back to the /// parent's DURABLE ROW (the last source that knows the attribution). #[tokio::test] - async fn fork_falls_back_to_the_durable_row_when_the_fork_connection_is_hollow_and_the_park_is_empty() - { + async fn fork_falls_back_to_the_durable_row_when_the_fork_connection_is_hollow_and_the_park_is_empty( + ) { let _guard = ENV_LOCK.lock().await; configure_fake_codex_cmd( &json!({ @@ -19241,8 +19237,8 @@ pub(crate) mod tests { /// latch while a NEWER turn is live (that latch belongs to the newer /// turn now) — the seed window deterministically constructed below. #[tokio::test] - async fn a_superseded_slot_completion_during_the_snapshot_seed_window_keeps_the_newer_turns_latch() - { + async fn a_superseded_slot_completion_during_the_snapshot_seed_window_keeps_the_newer_turns_latch( + ) { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -19642,8 +19638,8 @@ pub(crate) mod tests { /// publishes its snapshot without a second edge, and a subsequent /// crash rings nothing extra. #[tokio::test] - async fn system_error_status_with_a_turn_in_flight_rings_once_and_gates_the_matching_completion() - { + async fn system_error_status_with_a_turn_in_flight_rings_once_and_gates_the_matching_completion( + ) { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, notifs) = CodexAppServerClient::connect(transport); let client = Arc::new(client); @@ -21345,8 +21341,8 @@ pub(crate) mod tests { } #[tokio::test] - async fn handle_rollback_double_undo_within_one_epoch_keeps_the_marker_bucket_in_conversation_order() - { + async fn handle_rollback_double_undo_within_one_epoch_keeps_the_marker_bucket_in_conversation_order( + ) { // Whole-branch review Minor-3: the codex marker bucket is CONVERSATION // order (the plan :165 union rule wins over the codex bullet at :166). // Two sequential undos inside ONE epoch (no submission between) remove @@ -21530,8 +21526,8 @@ pub(crate) mod tests { } #[tokio::test] - async fn handle_rollback_undo_send_undo_undo_freezes_epoch_zero_and_orders_the_new_epoch_ascending() - { + async fn handle_rollback_undo_send_undo_undo_freezes_epoch_zero_and_orders_the_new_epoch_ascending( + ) { // Delta-r1 F8's codified multi-epoch case: undo → send (decision 5 // destroy) → undo → undo ⇒ the marker bucket is the FIRST epoch's // frozen prefix, then the NEW epoch's entries in conversation order @@ -24298,8 +24294,8 @@ pub(crate) mod tests { // _post_recovery_snapshot` (the other test mutating the process-global // `CODEX_CMD`/`FAKE_CODEX_APP_SERVER_BEHAVIOR` env vars) for the test's ENTIRE // duration, not just around individual calls. - async fn send_after_unrequested_crash_resumes_the_same_thread_id_and_completes_with_no_error_frame() - { + async fn send_after_unrequested_crash_resumes_the_same_thread_id_and_completes_with_no_error_frame( + ) { let _guard = ENV_LOCK.lock().await; let (st, mut rx) = state_with_bus(); @@ -24547,8 +24543,8 @@ pub(crate) mod tests { /// session mapped under its OLD id, still marked exited, for a future retry (mirroring /// the pre-existing `RespawnFailed` contract for a `thread/start` failure). #[tokio::test(flavor = "multi_thread")] - async fn send_after_crash_with_transient_resume_failure_reports_respawn_failed_and_stays_exited() - { + async fn send_after_crash_with_transient_resume_failure_reports_respawn_failed_and_stays_exited( + ) { let _guard = ENV_LOCK.lock().await; let (st, mut rx) = state_with_bus(); @@ -24968,8 +24964,8 @@ pub(crate) mod tests { #[tokio::test(flavor = "multi_thread")] // Intentional: `_guard` must span every `.await` in this test to serialize it against // every other env-var-mutating test in this module for its ENTIRE duration. - async fn attach_after_unrequested_crash_respawns_the_sidecar_and_emits_a_post_recovery_snapshot() - { + async fn attach_after_unrequested_crash_respawns_the_sidecar_and_emits_a_post_recovery_snapshot( + ) { let _guard = ENV_LOCK.lock().await; let (st, mut rx) = state_with_bus(); let capture = tracing_capture::capture_by_session("attach-recovery-marker-unused"); @@ -26791,12 +26787,10 @@ pub(crate) mod tests { assert_eq!(user_rows[0]["id"], "turn-native-1:row-0"); assert_eq!(user_rows[0]["turnId"], "turn-native-1:row-0"); assert_eq!(user_rows[0]["nativeTurnId"], "turn-native-1"); - assert!( - turns - .iter() - .filter(|turn| turn["role"] != "user") - .all(|turn| turn.get("nativeTurnId").is_none()) - ); + assert!(turns + .iter() + .filter(|turn| turn["role"] != "user") + .all(|turn| turn.get("nativeTurnId").is_none())); let raw_thread = json!({"thread":{"turns":[raw_turn.clone()]}}); assert_eq!( checkpoint_codex_turn_identity(&raw_thread, "turn-native-1"), diff --git a/crates/freshell-opencode/src/lib.rs b/crates/freshell-opencode/src/lib.rs index 65e1445b5..512f8b783 100644 --- a/crates/freshell-opencode/src/lib.rs +++ b/crates/freshell-opencode/src/lib.rs @@ -52,10 +52,9 @@ pub use model::{ }; pub use serve::{ build_prompt_body, build_prompt_body_with_message_id, default_opencode_database, - display_error_chain, is_healthy_response, observed_user_message_id, - resolve_opencode_database, CreatedSession, DaemonSignal, EffectiveOpencodeDatabase, Endpoint, - EventSource, EventStreamHandle, ForkedSession, OpencodeServeManager, PortAllocator, - ProcessSpawner, Route, ServeConfig, ServeDeps, ServeError, ServeHttp, ServeHttpError, - ServeHttpRequest, ServeHttpResponse, ServeProcess, SessionSignal, SpawnRequest, - OPENCODE_SIDECAR_OWNERSHIP_ENV, + display_error_chain, is_healthy_response, observed_user_message_id, resolve_opencode_database, + CreatedSession, DaemonSignal, EffectiveOpencodeDatabase, Endpoint, EventSource, + EventStreamHandle, ForkedSession, OpencodeServeManager, PortAllocator, ProcessSpawner, Route, + ServeConfig, ServeDeps, ServeError, ServeHttp, ServeHttpError, ServeHttpRequest, + ServeHttpResponse, ServeProcess, SessionSignal, SpawnRequest, OPENCODE_SIDECAR_OWNERSHIP_ENV, }; diff --git a/crates/freshell-protocol/src/client_messages.rs b/crates/freshell-protocol/src/client_messages.rs index 4b432adf3..b18997051 100644 --- a/crates/freshell-protocol/src/client_messages.rs +++ b/crates/freshell-protocol/src/client_messages.rs @@ -8,9 +8,9 @@ use serde_json::Value; use std::collections::BTreeMap; use crate::common::{ - AgentProvider, CodexDurability, FreshAgentCheckpointIntent, PermissionMode, Sandbox, - SessionLocator, SessionType, Shell, StringOrNumber, TerminalAttachIntent, - TerminalAttachPriority, double_option, + double_option, AgentProvider, CodexDurability, FreshAgentCheckpointIntent, PermissionMode, + Sandbox, SessionLocator, SessionType, Shell, StringOrNumber, TerminalAttachIntent, + TerminalAttachPriority, }; /// A message sent from a client to the server. diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 65f5ef5f5..23b8739af 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -31,18 +31,18 @@ use std::sync::atomic::{AtomicBool, Ordering}; use axum::routing::get as get_method; use axum::{ - Json, Router, extract::{Query, State}, http::{HeaderMap, StatusCode}, response::{IntoResponse, Response}, routing::post, + Json, Router, }; use serde::{Deserialize, Serialize}; -use serde_json::{Map, Value, json}; +use serde_json::{json, Map, Value}; use sha1::{Digest, Sha1}; use tokio::io::{AsyncReadExt, AsyncWriteExt}; use tokio::process::{Child, Command}; -use tokio::sync::{Mutex, OwnedMutexGuard, watch}; +use tokio::sync::{watch, Mutex, OwnedMutexGuard}; use freshell_protocol::{ AgentProvider, CheckpointWorkspaceProvenance, FreshAgentCheckpointOutcome, @@ -541,9 +541,8 @@ unsafe impl Sync for WindowsJob {} impl WindowsJob { fn new() -> std::io::Result<Self> { use windows_sys::Win32::System::JobObjects::{ - CreateJobObjectW, JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, - JOBOBJECT_EXTENDED_LIMIT_INFORMATION, JobObjectExtendedLimitInformation, - SetInformationJobObject, + CreateJobObjectW, JobObjectExtendedLimitInformation, SetInformationJobObject, + JOBOBJECT_EXTENDED_LIMIT_INFORMATION, JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, }; let handle = unsafe { CreateJobObjectW(std::ptr::null(), std::ptr::null()) }; @@ -607,8 +606,8 @@ impl WindowsJob { fn active_processes(&self) -> std::io::Result<u32> { use windows_sys::Win32::System::JobObjects::{ - JOBOBJECT_BASIC_ACCOUNTING_INFORMATION, JobObjectBasicAccountingInformation, - QueryInformationJobObject, + JobObjectBasicAccountingInformation, QueryInformationJobObject, + JOBOBJECT_BASIC_ACCOUNTING_INFORMATION, }; let mut accounting = JOBOBJECT_BASIC_ACCOUNTING_INFORMATION::default(); @@ -692,7 +691,7 @@ impl Drop for WindowsJob { fn find_process_thread(process_id: u32) -> std::io::Result<u32> { use windows_sys::Win32::Foundation::{CloseHandle, INVALID_HANDLE_VALUE}; use windows_sys::Win32::System::Diagnostics::ToolHelp::{ - CreateToolhelp32Snapshot, TH32CS_SNAPTHREAD, THREADENTRY32, Thread32First, Thread32Next, + CreateToolhelp32Snapshot, Thread32First, Thread32Next, TH32CS_SNAPTHREAD, THREADENTRY32, }; let snapshot = unsafe { CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0) }; @@ -1359,7 +1358,22 @@ impl CheckpointService { .as_ref() .and_then(|settings| settings.cwd.as_deref()) .or(send.cwd.as_deref()); - let Some(launch_cwd) = launch_cwd else { + let intent = send + .checkpoint_intent + .as_ref() + .and_then(|intent| serde_json::to_value(intent.workspace.as_ref()).ok()) + .and_then(|value| if value.is_null() { None } else { Some(value) }); + // An explicitly selected checkpoint workspace is independent from the + // provider's launch cwd. Use it only as the resolver anchor when a + // resumed pane no longer has its original cwd; never write it back to + // `send` or change the provider's working directory. + let workspace_anchor = launch_cwd.or_else(|| { + send.checkpoint_intent + .as_ref() + .and_then(|intent| intent.workspace.as_ref()) + .map(|workspace| workspace.path.as_str()) + }); + let Some(workspace_anchor) = workspace_anchor else { return self.new_record( send, logical_attempt_id, @@ -1371,19 +1385,15 @@ impl CheckpointService { )), ); }; - let intent = send - .checkpoint_intent - .as_ref() - .and_then(|intent| serde_json::to_value(intent.workspace.as_ref()).ok()) - .and_then(|value| if value.is_null() { None } else { Some(value) }); let resolve_control = Arc::new(CheckpointCaptureControl::default()); let resolve_control_for_task = Arc::clone(&resolve_control); let home = Arc::clone(&self.home); - let launch_cwd = launch_cwd.to_string(); + let workspace_anchor = workspace_anchor.to_string(); let mut resolve_task = tokio::spawn(CHECKPOINT_CAPTURE_CONTEXT.scope( resolve_control_for_task, async move { - resolve_checkpoint_workspace(&home, Path::new(&launch_cwd), intent.as_ref()).await + resolve_checkpoint_workspace(&home, Path::new(&workspace_anchor), intent.as_ref()) + .await }, )); let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); @@ -4903,6 +4913,66 @@ mod tests { assert!(!tokio::fs::try_exists(checkpoints).await.unwrap()); } + #[tokio::test] + async fn explicit_checkpoint_workspace_can_be_captured_without_a_launch_cwd() { + let home_dir = tempfile::tempdir().unwrap(); + let workspace_dir = tempfile::tempdir().unwrap(); + let fixture = workspace_dir.path().join("selected-project.txt"); + tokio::fs::write(&fixture, "workspace selected independently of launch cwd\n") + .await + .unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(8); + let service = CheckpointService::new( + Arc::new(home_dir.path().to_path_buf()), + Arc::new(broadcast_tx), + ); + let send = FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "thread-no-launch-cwd".to_string(), + session_type: SessionType::Freshcodex, + text: "continue from the selected project".to_string(), + cwd: None, + images: None, + request_id: Some("request-no-launch-cwd".to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-no-launch-cwd".to_string(), + workspace: Some(FreshAgentCheckpointWorkspace { + path: workspace_dir.path().to_string_lossy().into_owned(), + provenance: CheckpointWorkspaceProvenance::DirectoryPicker, + }), + }), + }; + + let outcome = service + .capture_automatic(&send, "request-no-launch-cwd", "attempt-no-launch-cwd") + .await; + + assert_eq!( + send.cwd, None, + "capture must not synthesize provider launch cwd" + ); + assert_eq!( + outcome.outcome.status, + FreshAgentCheckpointStatus::Available, + "an explicit selected workspace is sufficient for checkpoint capture" + ); + assert!(outcome.outcome.checkpoint_id.is_some()); + assert_eq!( + outcome.outcome.workspace, + Some(FreshAgentCheckpointWorkspace { + path: workspace_dir.path().to_string_lossy().into_owned(), + provenance: CheckpointWorkspaceProvenance::DirectoryPicker, + }), + ); + assert_eq!( + tokio::fs::read_to_string(fixture).await.unwrap(), + "workspace selected independently of launch cwd\n" + ); + } + #[tokio::test] async fn picker_home_is_launch_only() { let home_dir = tempfile::tempdir().unwrap(); @@ -5548,11 +5618,9 @@ mod tests { "a descendant cannot continue writing after the route has settled cancellation" ); let entries = list_checkpoints(home_dir.path(), &workspace).await.unwrap(); - assert!( - !entries - .iter() - .any(|entry| { entry["requestId"].as_str() == Some("dropped-http-capture") }) - ); + assert!(!entries + .iter() + .any(|entry| { entry["requestId"].as_str() == Some("dropped-http-capture") })); } // ── is_valid_checkpoint_id ────────────────────────────────────────── @@ -5637,12 +5705,10 @@ mod tests { .await .unwrap(); - assert!( - list_checkpoints(home_dir.path(), cwd_dir.path()) - .await - .unwrap() - .is_empty() - ); + assert!(list_checkpoints(home_dir.path(), cwd_dir.path()) + .await + .unwrap() + .is_empty()); } #[tokio::test] @@ -5656,11 +5722,9 @@ mod tests { let result = list_checkpoints(home_dir.path(), cwd_dir.path()).await; - assert!( - result - .unwrap_err() - .contains("checkpoint repository exists but its HEAD is missing") - ); + assert!(result + .unwrap_err() + .contains("checkpoint repository exists but its HEAD is missing")); } #[tokio::test] @@ -5766,12 +5830,10 @@ mod tests { assert_eq!(first["id"].as_str().unwrap().len(), 40); assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); let body = body_json(response).await; - assert!( - body["error"] - .as_str() - .unwrap_or_default() - .contains("Git operation log failed") - ); + assert!(body["error"] + .as_str() + .unwrap_or_default() + .contains("Git operation log failed")); } #[tokio::test] @@ -5860,12 +5922,10 @@ mod tests { .await; assert_eq!(resp.status(), StatusCode::BAD_REQUEST); let value = body_json(resp).await; - assert!( - value["error"] - .as_str() - .unwrap() - .starts_with("cwd does not exist:") - ); + assert!(value["error"] + .as_str() + .unwrap() + .starts_with("cwd does not exist:")); } #[tokio::test] diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index ed8ff54c7..0d1063c0d 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -54,7 +54,8 @@ pub(crate) struct HostedFreshAgentProxy { pollers: Mutex<HashSet<SoulId>>, fixture_modes: HashSet<String>, naming: OnceLock<Arc<dyn SessionNaming>>, - checkpoint_barrier: OnceLock<Arc<dyn freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier>>, + checkpoint_barrier: + OnceLock<Arc<dyn freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier>>, ownership: OnceLock<Arc<freshell_ownership::RuntimeOwnershipRegistry>>, } @@ -125,7 +126,10 @@ impl HostedFreshAgentProxy { } match current.state { freshell_ownership::OwnershipState::Live { owner, .. } - if owner.kind == freshell_ownership::RuntimeOwnerKind::FreshAgent => Ok(()), + if owner.kind == freshell_ownership::RuntimeOwnerKind::FreshAgent => + { + Ok(()) + } _ => Err("fresh-agent session is no longer available for this send"), } } diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index 1c4ac987c..ee6fefdb5 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -4681,7 +4681,7 @@ mod sessions_sweep_tests { screenshots: freshell_ws::screenshot::ScreenshotBroker::new(Arc::clone(&broadcast_tx)), subagent_interest: Default::default(), host_stats: Default::default(), - checkpoint_send: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/crates/freshell-server/tests/code_checkpoint_permissions.rs b/crates/freshell-server/tests/code_checkpoint_permissions.rs index c2bec8583..a0d21f035 100644 --- a/crates/freshell-server/tests/code_checkpoint_permissions.rs +++ b/crates/freshell-server/tests/code_checkpoint_permissions.rs @@ -14,7 +14,7 @@ use std::process::{Child, Command, Stdio}; use std::time::{Duration, Instant}; use reqwest::{Client, StatusCode}; -use serde_json::{Value, json}; +use serde_json::{json, Value}; use sha1::{Digest, Sha1}; use tokio::io::AsyncWriteExt; use tokio::net::TcpStream; @@ -217,16 +217,12 @@ async fn restricted_checkpoint_routes_preserve_complete_targets_and_log_correlat let private_dir_ignored = project.join("private-dir.ignored"); let private_dir = project.join("private-dir"); let ignore_text = fs::read_to_string(project.join(".gitignore")).unwrap(); - assert!( - ignore_text - .lines() - .any(|line| line == "private-file.ignored") - ); - assert!( - ignore_text - .lines() - .any(|line| line == "private-dir.ignored/") - ); + assert!(ignore_text + .lines() + .any(|line| line == "private-file.ignored")); + assert!(ignore_text + .lines() + .any(|line| line == "private-dir.ignored/")); denied_file_read(&private_file_ignored); assert_eq!( @@ -476,11 +472,9 @@ async fn restricted_checkpoint_routes_preserve_complete_targets_and_log_correlat assert_eq!(repaired_status, StatusCode::OK, "{repaired_entry}"); let repaired_list = list_workspace(&client, &base, &home, &project).await; assert_eq!(repaired_list.len(), 2); - assert!( - repaired_list - .iter() - .any(|entry| entry["id"] == repaired_entry["id"]) - ); + assert!(repaired_list + .iter() + .any(|entry| entry["id"] == repaired_entry["id"])); fs::write(project.join("main.rs"), "changed after repaired capture\n").unwrap(); let (repaired_restore_status, repaired_restore_body) = post_json( &client, @@ -578,12 +572,10 @@ async fn restricted_checkpoint_routes_preserve_complete_targets_and_log_correlat .unwrap(); assert_eq!(event["stage"], "readability"); assert_eq!(event["filesystem_kind"], "PermissionDenied"); - assert!( - event["cause"] - .as_str() - .unwrap() - .contains("Permission denied") - ); + assert!(event["cause"] + .as_str() + .unwrap() + .contains("Permission denied")); let path = event["path"].as_str().unwrap(); assert!(!path.is_empty()); let expected_path = if send_request_id == "send-private-file" { diff --git a/crates/freshell-ws/src/checkpoint_barrier.rs b/crates/freshell-ws/src/checkpoint_barrier.rs index ddfd36e44..aaa8b9976 100644 --- a/crates/freshell-ws/src/checkpoint_barrier.rs +++ b/crates/freshell-ws/src/checkpoint_barrier.rs @@ -7,8 +7,8 @@ use std::sync::Arc; -use futures::future::BoxFuture; use freshell_protocol::{AgentProvider, FreshAgentSend, SessionType}; +use futures::future::BoxFuture; pub trait FreshAgentCheckpointBarrier: Send + Sync { fn barrier_and_forward( diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index 31e20aa8b..80e65bd3f 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -1980,7 +1980,7 @@ async fn handle_client_text( } fresh_codex.handle_attach(attach).await } - .instrument(tracing::Span::current()), + .instrument(tracing::Span::current()), ); } freshell_protocol::AgentProvider::Claude => { @@ -1999,7 +1999,7 @@ async fn handle_client_text( } fresh_claude.handle_attach(attach).await } - .instrument(tracing::Span::current()), + .instrument(tracing::Span::current()), ); } freshell_protocol::AgentProvider::Opencode => { @@ -2018,7 +2018,7 @@ async fn handle_client_text( } fresh_opencode.handle_attach(attach).await } - .instrument(tracing::Span::current()), + .instrument(tracing::Span::current()), ); } _ => {} @@ -11705,7 +11705,7 @@ mod terminals_changed_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), - checkpoint_send: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), @@ -11905,7 +11905,7 @@ mod terminal_kill_stop_wedge_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), - checkpoint_send: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), @@ -12339,7 +12339,7 @@ mod terminal_meta_created_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), - checkpoint_send: Default::default(), + checkpoint_send: Default::default(), terminals_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: std::sync::Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: std::sync::Arc::new(Vec::new()), @@ -12984,7 +12984,7 @@ mod pane_reconcile_gate_tests { screenshots: crate::screenshot::ScreenshotBroker::new(broadcast_tx), subagent_interest: Default::default(), host_stats: Default::default(), - checkpoint_send: Default::default(), + checkpoint_send: Default::default(), terminals_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), sessions_revision: Arc::new(std::sync::atomic::AtomicI64::new(0)), cli_commands: Arc::new(Vec::new()), diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 16e4f1b9b..80d3ef7c8 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -360,6 +360,27 @@ function checkpointAvailabilityMap(content: FreshAgentPaneContent): Map<string, return availability } +function checkpointAvailabilityMatchesTurn( + provider: FreshAgentPaneContent['provider'], + availability: CheckpointTurnAvailability, + turn: FreshAgentTurn, +): boolean { + if (turn.role !== 'user') return false + const displayTurnId = getFreshAgentDisplayTurnKey(turn) + if (availability.displayTurnId && availability.displayTurnId !== displayTurnId) return false + if (availability.nativeTurnId) { + if (turn.nativeTurnId) { + if (availability.nativeTurnId !== turn.nativeTurnId) return false + } else if ( + checkpointTurnIdentity(provider, availability.nativeTurnId) + !== checkpointTurnIdentity(provider, displayTurnId) + ) { + return false + } + } + return Boolean(availability.displayTurnId || availability.nativeTurnId) +} + function localEchoLanded( turns: readonly FreshAgentTurn[], echo: LocalEcho, @@ -1113,10 +1134,21 @@ export function FreshAgentView({ const getCheckpointAvailability = useCallback((turn: FreshAgentTurn): CheckpointTurnAvailability | undefined => { if (turn.role !== 'user') return undefined const scope = checkpointAvailabilityScope(paneContent) + const displayTurnId = getFreshAgentDisplayTurnKey(turn) const turnAvailability = checkpointAvailabilityByTurn.get( - checkpointTurnAvailabilityKey(paneContent, scope, getFreshAgentDisplayTurnKey(turn)), + checkpointTurnAvailabilityKey(paneContent, scope, displayTurnId), ) - if (turnAvailability) return turnAvailability + if (turnAvailability && checkpointAvailabilityMatchesTurn(paneContent.provider, turnAvailability, turn)) { + return turnAvailability + } + if (turn.nativeTurnId) { + const nativeAvailability = checkpointAvailabilityByTurn.get( + checkpointTurnAvailabilityKey(paneContent, scope, turn.nativeTurnId), + ) + if (nativeAvailability && checkpointAvailabilityMatchesTurn(paneContent.provider, nativeAvailability, turn)) { + return nativeAvailability + } + } const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId if (typeof requestId !== 'string' || requestId.length === 0) return undefined return checkpointAvailabilityByTurn.get(checkpointAvailabilityKey(scope, 'request', requestId)) @@ -2733,19 +2765,6 @@ export function FreshAgentView({ const requestId = message.requestId const pending = pendingSendMetadataRef.current.get(requestId) const current = paneContentRef.current - if ( - !pending - || message.logicalAttemptId !== pending.logicalAttemptId - || !locatorMatchesPane( - message, - current, - freshOpenCodeRouteCwdRef.current, - runtimeOwnersForLocator(), - ) - ) return - - const workspace = normalizeFreshAgentCheckpointWorkspace(message.workspace) - ?? pending.checkpointWorkspace const checkpointTurnId = (value: unknown): string | undefined => ( typeof value === 'string' && value.length > 0 @@ -2756,6 +2775,21 @@ export function FreshAgentView({ ) const nativeTurnId = checkpointTurnId(message.nativeTurnId) const displayTurnId = checkpointTurnId(message.displayTurnId) + const hasProviderTurnIdentity = Boolean(nativeTurnId || displayTurnId) + if ( + (pending && message.logicalAttemptId !== pending.logicalAttemptId) + || (!pending && !hasProviderTurnIdentity) + || !locatorMatchesPane( + message, + current, + freshOpenCodeRouteCwdRef.current, + runtimeOwnersForLocator(), + ) + ) return + + const workspace = normalizeFreshAgentCheckpointWorkspace(message.workspace) + ?? pending?.checkpointWorkspace + ?? current.checkpointWorkspace let checkpointOutcome: CheckpointTurnAvailability if ( message.status === 'available' @@ -2797,7 +2831,20 @@ export function FreshAgentView({ ...(displayTurnId ? { displayTurnId } : {}), } } - recordPendingSendMetadata(requestId, { checkpointOutcome }) + if (pending) { + recordPendingSendMetadata(requestId, { checkpointOutcome }) + } else { + // Replayed outcomes arrive after remount, when the transient send + // map is empty. The provider IDs are stamped only after observing + // the exact persisted user row, so they safely bind the durable + // result to its transcript row without prompt or ordinal matching. + publishCheckpointAvailability( + checkpointAvailabilityScope(current), + requestId, + undefined, + checkpointOutcome, + ) + } return } if ( @@ -3983,7 +4030,7 @@ export function FreshAgentView({ return } const workspace = availability?.workspace ?? current.checkpointWorkspace - const cwd = agentSessionCwdRef.current ?? current.initialCwd + const cwd = agentSessionCwdRef.current ?? current.initialCwd ?? workspace?.path if (!cwd) { const reason = 'No project working directory is available for this message.' const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 817384fc6..868934889 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -147,13 +147,13 @@ async function sendThroughComposer( // The send barrier's first outcome is deliberately pre-forward and may // not yet know Codex's persisted user row. Require the later updated // native-linked projection before treating the per-turn result as ready. - return typeof outcome?.nativeTurnId === 'string' - && outcome.displayTurnId === accepted.submittedTurnId + return outcome?.nativeTurnId === accepted.submittedTurnId + && outcome.displayTurnId === `${accepted.submittedTurnId}:row-0` }, { timeout: 120_000 }).toBe(true) expect(outcome!.requestId).toBe(sent.requestId) expect(outcome!.conversationRequestId).toBe((await currentFreshAgent(page)).content.createRequestId) - expect(outcome!.displayTurnId).toBe(accepted.submittedTurnId) - expect(outcome!.nativeTurnId).toEqual(expect.any(String)) + expect(outcome!.displayTurnId).toBe(`${accepted.submittedTurnId}:row-0`) + expect(outcome!.nativeTurnId).toBe(accepted.submittedTurnId) expect(JSON.stringify((await currentFreshAgent(page)).content.checkpointOutcomes)).not.toContain(text) const row = paneRoot.locator('article[data-turn-role="user"]', { hasText: text }) @@ -180,9 +180,11 @@ async function assertNativeUserIdentity( const user = turn.items.find(item => item.type === 'userMessage' && item.content?.some(part => part.text === text)) expect(user?.id).toBe(`${acceptedTurnId}:user`) - // The persisted checkpoint outcome must point at the actual native Codex - // user-message identity. The display turn ID is a separate, stable UI join. - expect(outcome.nativeTurnId).toBe(user!.id) + // The persisted checkpoint outcome must point at the provider turn identity + // and the projected user-row identity. The raw user-message item has its own + // item id, which is not either checkpoint join key. + expect(outcome.nativeTurnId).toBe(acceptedTurnId) + expect(outcome.displayTurnId).toBe(`${acceptedTurnId}:row-0`) return turn } diff --git a/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts b/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts index a89c97a97..151bb635b 100644 --- a/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts +++ b/test/e2e-browser/specs/fresh-agent-checkpoint-unavailable-rust.spec.ts @@ -42,6 +42,130 @@ function findFreshAgentLeaf(node: any): any { } test.describe('fresh-agent checkpoint availability', () => { + test('a selected non-Git workspace keeps its successful checkpoint through reload and restores it', async ({ browser }) => { + test.setTimeout(180_000) + const sharedRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-checkpoint-restore-')) + const projectDir = path.join(sharedRoot, 'selected-workspace') + const sourceFile = path.join(projectDir, 'state.txt') + const codexHome = path.join(sharedRoot, 'codex-home') + let server: RustServer | undefined + let context: Awaited<ReturnType<typeof createFreshE2ePage>>['context'] | undefined + + try { + await fs.mkdir(projectDir, { recursive: true }) + await fs.writeFile(sourceFile, 'before checkpoint\n', 'utf8') + const fakeCodexPath = await installFakeCodexAppServer(path.join(sharedRoot, 'bin')) + server = new RustServer({ + env: { + CODEX_CMD: fakeCodexPath, + CODEX_HOME: codexHome, + FAKE_CODEX_APP_SERVER_BEHAVIOR: JSON.stringify({ recordTurns: true }), + FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES: '1', + }, + setupHome: async (homeDir) => { + const freshellDir = path.join(homeDir, '.freshell') + await fs.mkdir(freshellDir, { recursive: true }) + await fs.writeFile(path.join(freshellDir, 'config.json'), JSON.stringify({ + version: 1, + settings: { + freshAgent: { enabled: true }, + codingCli: { + enabledProviders: ['codex'], + providers: { codex: { model: 'gpt-5-codex', sandbox: 'workspace-write' } }, + }, + }, + }, null, 2)) + }, + }) + const info = await server.start() + const ownedPage = await createFreshE2ePage(browser, info) + context = ownedPage.context + const { page } = ownedPage + await page.goto(`${info.baseUrl}/?token=${info.token}&e2e=1`) + const harness = new TestHarness(page) + await harness.waitForHarness() + await harness.waitForConnection() + await page.evaluate(() => { + window.__FRESHELL_TEST_HARNESS__?.dispatch({ + type: 'connection/setAvailableClis', + payload: { claude: false, codex: true }, + }) + }) + + const picker = await openPanePicker(page) + await picker.getByRole('button', { name: /^Freshcodex$/i }).click({ force: true }) + const directoryInput = page.getByRole('combobox', { name: 'Starting directory for Freshcodex' }) + await expect(directoryInput).toBeVisible({ timeout: 10_000 }) + await directoryInput.fill(projectDir) + await directoryInput.press('Enter') + + const paneRoot = page.locator('[data-context="fresh-agent"]').last() + await expect(paneRoot).toBeVisible({ timeout: 15_000 }) + const tabId = await harness.getActiveTabId() + expect(tabId).toBeTruthy() + await expect.poll(async () => { + const layout = await harness.getPaneLayout(tabId!) + return findFreshAgentLeaf(layout)?.content?.initialCwd + }, { timeout: 10_000 }).toBe(projectDir) + await expect.poll(async () => { + const layout = await harness.getPaneLayout(tabId!) + return findFreshAgentLeaf(layout)?.content?.status + }, { timeout: 30_000 }).toBe('idle') + + const marker = `checkpoint-restore-${Math.random().toString(36).slice(2, 10)}` + const composer = paneRoot.getByRole('textbox', { name: 'Chat message input' }) + await composer.fill(marker) + await paneRoot.getByRole('button', { name: 'Send' }).click() + + const userTurn = paneRoot.locator('[data-turn-role="user"]', { hasText: marker }) + await expect(userTurn).toBeVisible({ timeout: 20_000 }) + await expect(userTurn.getByRole('status', { name: 'Code rewind available for this message' })) + .toContainText('Code rewind is available for this message.') + await expect(paneRoot.getByText('Fixture turn')).toBeVisible({ timeout: 30_000 }) + + let content: any + await expect.poll(async () => { + const layout = await harness.getPaneLayout(tabId!) + content = findFreshAgentLeaf(layout)?.content + return content?.status + }, { timeout: 20_000 }).toBe('idle') + const outcome = content.checkpointOutcomes?.find((entry: any) => entry.status === 'available') + expect(outcome?.checkpointId).toMatch(/^[a-f0-9]{40}$/) + expect(outcome?.workspace).toEqual({ path: projectDir, provenance: 'directory-picker' }) + expect(JSON.stringify(content.checkpointOutcomes)).not.toContain(marker) + + await fs.writeFile(sourceFile, 'after checkpoint\n', 'utf8') + await page.evaluate(() => { + window.__FRESHELL_TEST_HARNESS__?.dispatch({ type: 'persist/flushNow' }) + }) + await page.reload() + const reloadedHarness = new TestHarness(page) + await reloadedHarness.waitForHarness() + await reloadedHarness.waitForConnection() + const reloadedPane = page.locator('[data-context="fresh-agent"]').last() + await expect(reloadedPane).toBeVisible({ timeout: 20_000 }) + const reloadedTurn = reloadedPane.locator('[data-turn-role="user"]', { hasText: marker }) + await expect(reloadedTurn).toBeVisible({ timeout: 30_000 }) + await expect(reloadedTurn.getByRole('status', { name: 'Code rewind available for this message' })) + .toContainText('Code rewind is available for this message.') + await reloadedTurn.hover() + const rewind = reloadedTurn.getByRole('button', { name: 'Rewind code to here' }) + await expect(rewind).toBeEnabled() + const dialogAccepted = page.waitForEvent('dialog').then((dialog) => dialog.accept()) + await rewind.click() + await dialogAccepted + + await expect.poll(() => fs.readFile(sourceFile, 'utf8'), { timeout: 20_000 }) + .toBe('before checkpoint\n') + await expect(reloadedPane.getByText(`Code rewound to before: "${marker}"`)) + .toBeVisible({ timeout: 10_000 }) + } finally { + await context?.close().catch(() => {}) + await server?.stop().catch(() => {}) + await fs.rm(sharedRoot, { recursive: true, force: true }).catch(() => {}) + } + }) + test('an unreadable selected project file marks rewind unavailable and still completes the provider turn', async ({ browser }) => { test.setTimeout(180_000) const sharedRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-checkpoint-unavailable-')) diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index f01f75e12..36d644c51 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3643,6 +3643,194 @@ describe('FreshAgentView', () => { }) }) + it('accepts a replayed checkpoint outcome after reload when it carries an exact provider row identity', async () => { + const checkpointId = 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' + const workspace = { path: '/repo/selected', provenance: 'directory-picker' as const } + const displayTurnId = 'codex-replayed-turn:row-0' + const nativeTurnId = 'codex-replayed-turn' + const row = { + id: displayTurnId, + turnId: displayTurnId, + nativeTurnId, + role: 'user' as const, + summary: 'A transcript row already stored by the provider', + items: [{ id: 'replayed-user-text', kind: 'text' as const, text: 'A transcript row already stored by the provider' }], + } + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'reloaded conversation', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [row, { + id: 'codex-replayed-turn:row-1', + turnId: 'codex-replayed-turn:row-1', + nativeTurnId, + role: 'user', + summary: 'A second projection row sharing the provider turn', + items: [{ id: 'replayed-decoy-text', kind: 'text', text: 'A second projection row sharing the provider turn' }], + }], + }) + let onMessage: ((message: Record<string, unknown>) => void) | undefined + wsMock.onMessage.mockImplementation((handler: (message: Record<string, unknown>) => void) => { + onMessage = handler + return () => {} + }) + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-replayed-conversation', + sessionId: 'thread-replayed-conversation', + sessionRef: { provider: 'codex', sessionId: 'thread-replayed-conversation' }, + status: 'idle', + checkpointWorkspace: workspace, + }, + })) + + const view = render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + expect(await screen.findByText('A transcript row already stored by the provider')).toBeInTheDocument() + expect(onMessage).toBeTypeOf('function') + + act(() => { + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', + requestId: 'request-replayed-turn', + logicalAttemptId: 'attempt-replayed-turn', + sessionId: 'thread-replayed-conversation', + sessionType: 'freshcodex', + provider: 'codex', + status: 'available', + checkpointId, + workspace, + }) + }) + expect(getFreshAgentPaneContent(store).checkpointOutcomes ?? []).toHaveLength(0) + expect(screen.queryByRole('status', { name: 'Code rewind available for this message' })).not.toBeInTheDocument() + + // The first broadcast was missed while this pane was detached. The server + // replay has no in-memory pending send metadata to rely on after remount. + act(() => { + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', + requestId: 'request-replayed-turn', + logicalAttemptId: 'attempt-replayed-turn', + sessionId: 'thread-replayed-conversation', + sessionType: 'freshcodex', + provider: 'codex', + status: 'available', + checkpointId, + workspace, + nativeTurnId, + displayTurnId, + }) + }) + + const exactReplayUserRows = screen.getAllByRole('article', { name: 'You transcript turn' }) + expect(exactReplayUserRows).toHaveLength(2) + expect(await within(exactReplayUserRows[0]!).findByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') + expect(within(exactReplayUserRows[1]!).queryByRole('status', { name: 'Code rewind available for this message' })) + .not.toBeInTheDocument() + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ + requestId: 'request-replayed-turn', + conversationRequestId: 'req-replayed-conversation', + nativeTurnId, + displayTurnId, + checkpointId, + status: 'available', + }), + ])) + + view.unmount() + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + expect(await screen.findByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') + }) + + it('uses the selected checkpoint workspace to rewind when a resumed pane has no launch cwd', async () => { + const checkpointId = 'cccccccccccccccccccccccccccccccc' + const workspace = { path: '/repo/selected-without-launch-cwd', provenance: 'directory-picker' as const } + const displayTurnId = 'resume-row-1' + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'resumed conversation', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: displayTurnId, + turnId: displayTurnId, + nativeTurnId: 'native-resume-row-1', + role: 'user', + summary: 'Restore the selected workspace', + items: [{ id: 'resume-user-text', kind: 'text', text: 'Restore the selected workspace' }], + }], + }) + apiMock.get.mockResolvedValue({ checkpoints: [{ + id: checkpointId, + ts: 1, + label: 'Restore the selected workspace', + workspace, + }] }) + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-resumed-without-cwd', + sessionId: 'thread-resumed-without-cwd', + sessionRef: { provider: 'codex', sessionId: 'thread-resumed-without-cwd' }, + status: 'idle', + checkpointWorkspace: workspace, + checkpointOutcomes: [{ + version: 1, + conversationRequestId: 'req-resumed-without-cwd', + requestId: 'request-resume-row-1', + displayTurnId, + nativeTurnId: 'native-resume-row-1', + status: 'available', + checkpointId, + workspace, + }], + }, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + vi.spyOn(window, 'confirm').mockReturnValue(true) + const turn = await screen.findByText('Restore the selected workspace') + const article = turn.closest('[data-turn-role="user"]') + expect(article).not.toBeNull() + fireEvent.click(within(article as HTMLElement).getByRole('button', { name: 'Rewind code to here' })) + + await waitFor(() => { + expect(apiMock.get).toHaveBeenCalledWith( + '/api/fresh-agent/checkpoints?cwd=%2Frepo%2Fselected-without-launch-cwd&workspacePath=%2Frepo%2Fselected-without-launch-cwd&workspaceProvenance=directory-picker', + ) + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: '/repo/selected-without-launch-cwd', + workspace, + id: checkpointId, + }) + }) + }) + it('rewinds to the persisted checkpoint ID after reload when duplicate prompts have no metadata links', async () => { const checkpointWorkspace = { path: '/repo/selected', provenance: 'directory-picker' as const } const firstCheckpointId = '11111111111111111111111111111111' @@ -3923,6 +4111,9 @@ describe('FreshAgentView', () => { reason: workspaceReason, }) }) + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ requestId, status: 'unavailable', reason: workspaceReason }), + ])) expect(await screen.findByRole('note', { name: 'Code rewind unavailable for this message' })) .toHaveTextContent(`Code rewind unavailable for this message: ${workspaceReason}`) expect(screen.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() From eb8b6a231c0d2067cea4e8e9fbe2db3bb53be478 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 04:29:14 -0700 Subject: [PATCH 312/493] fix: initialize hosted sends without checkpoint intent --- crates/freshell-session-host/src/providers/fresh_agent.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index 1b3b6bd9b..391712f70 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -495,6 +495,7 @@ impl FreshAgentTransport for HostedTransport { permission_mode: profile.permission_mode, sandbox: profile.sandbox.as_deref().and_then(parse_sandbox), }), + checkpoint_intent: None, }; match &self.state { ProviderState::Claude(state) => state.handle_send(send).await, From 8a6cbfaabe3bf8d2c2d9ea1e5d4cd0d2d2e1e4da Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 04:59:03 -0700 Subject: [PATCH 313/493] fix: retain managed preparation failures across reload --- src/lib/pane-reconcile.ts | 5 ++++ ...aged-provider-environment-external.spec.ts | 22 ++++++++++++-- test/unit/client/lib/pane-reconcile.test.ts | 30 +++++++++++++++++++ 3 files changed, 54 insertions(+), 3 deletions(-) diff --git a/src/lib/pane-reconcile.ts b/src/lib/pane-reconcile.ts index b3b737b7f..543956e95 100644 --- a/src/lib/pane-reconcile.ts +++ b/src/lib/pane-reconcile.ts @@ -37,6 +37,7 @@ import { setPaneRestoreError, setReconcileWarming, } from '@/store/panesSlice' +import { retainedManagedPreparationFailure } from '@/store/paneTypes' import { clearSessionLost } from '@/store/freshAgentSlice' import type { PaneOwnerDivergence } from '@/store/selectors/runtimeOwner' import { derivePaneTitle } from '@/lib/derivePaneTitle' @@ -147,6 +148,10 @@ function effectiveReconcileSessionRef( function toReconcilePane(tabId: string, paneId: string, content: TerminalPaneContent): ReconcilePane | null { // Panes without a createRequestId cannot be reconciled — skip. if (!content.createRequestId) return null + // A failed managed preparation never registered a backend terminal. Its + // retained error is the durable result; asking the server to reconcile the + // same createRequestId would classify it as fresh and silently retry it. + if (retainedManagedPreparationFailure(content)) return null const sessionRef = effectiveReconcileSessionRef(content.sessionRef, content.resumeSessionId, content.mode) return { paneKey: paneKeyFor(tabId, paneId), diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 14706ffa1..5d62e133c 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -75,6 +75,20 @@ async function terminalDiagnostic(page: Page, terminalId?: string) { }, { terminalId, marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) } +async function hasCompletedOutputWrite(page: Page, terminalId: string, marker: string, after = 0) { + return page.evaluate(({ terminalId, marker, after }) => { + const events = window.__FRESHELL_TEST_HARNESS__?.getTerminalWriteEvents?.() ?? [] + let tail = '' + for (const event of events) { + if (event.terminalId !== terminalId || event.phase !== 'written' || event.at < after) continue + const current = tail + event.data + if (current.includes(marker)) return true + tail = current.slice(-Math.max(marker.length - 1, 16)) + } + return false + }, { terminalId, marker, after }) +} + test.afterEach(async ({ page }, info) => { const scenario = caseScenarios[info.title] if (!scenario) throw new Error('Unknown managed environment case') @@ -187,8 +201,8 @@ test(healthyTitle, async ({ page }, info) => { const terminalId = pane.content.terminalId! const marker = `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` info.annotations.push({ type: 'managed-stage', description: 'healthy-initial-output' }) - await expect.poll(() => page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.getTerminalBuffer(id), terminalId), - { timeout: 120_000 }).toContain(marker) + await expect.poll(() => hasCompletedOutputWrite(page, terminalId, marker), + { timeout: 120_000 }).toBe(true) let receipt: any await expect.poll(async () => { receipt = await readReceipt(page, `${pane.cwd}/.managed-fixture/target-receipt.json`) @@ -205,12 +219,14 @@ test(healthyTitle, async ({ page }, info) => { .filter((message: any) => message.type === 'terminal.create').length) console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.browser_before_reconnect', scenario: 'healthy', attempt: info.retry, diagnostic: await terminalDiagnostic(page, terminalId) })) + const reconnectStartedAt = await page.evaluate(() => performance.now()) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) await expect.poll(async () => (await activePane(page)).content).toMatchObject({ terminalId, sessionRef: { provider: 'opencode', sessionId: receipt.nativeSessionId } }) info.annotations.push({ type: 'managed-stage', description: 'healthy-post-reconnect-output' }) - await expect.poll(() => page.evaluate(id => window.__FRESHELL_TEST_HARNESS__!.getTerminalBuffer(id), terminalId)).toContain(marker) + await expect.poll(() => hasCompletedOutputWrite(page, terminalId, marker, reconnectStartedAt), + { timeout: 120_000 }).toBe(true) expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length)).toBe(beforeCreates) const rows = await json(page, '/api/terminals') diff --git a/test/unit/client/lib/pane-reconcile.test.ts b/test/unit/client/lib/pane-reconcile.test.ts index dfc5766c2..8bfe88de9 100644 --- a/test/unit/client/lib/pane-reconcile.test.ts +++ b/test/unit/client/lib/pane-reconcile.test.ts @@ -178,6 +178,36 @@ describe('buildReconcileRequest', () => { expect(buildReconcileRequest(emptyState())).toBeNull() }) + it('does not reconcile a persisted managed preparation failure as a fresh create', () => { + let state = emptyPanesState() + state = addTerminalPane(state, 'tab1', 'blocked', { + mode: 'opencode', + status: 'error', + createRequestId: 'failed-managed-create', + launchFailure: { + code: 'LAUNCH_FAILED', + message: 'Configuration blocked: /workspace/missing.txt: referenced file was not found', + retryable: true, + providerPreparationFailure: { + code: 'SELECTED_REFERENCE_UNAVAILABLE', + stage: 'configuration', + operation: 'resolve', + cause: 'not_found', + workspace: '/workspace', + declaringConfig: '/workspace/opencode.json', + selectedPath: '/workspace/missing.txt', + }, + }, + }) + state = addTerminalPane(state, 'tab2', 'healthy', { + mode: 'opencode', status: 'running', terminalId: 'terminal-healthy', + }) + + const request = buildReconcileRequest(asRootState(state)) + expect(request?.panes.map(pane => pane.paneKey)).toEqual(['tab2:healthy']) + expect(buildReconcileRequestForPanes(asRootState(state), [{ tabId: 'tab1', paneId: 'blocked' }])).toBeNull() + }) + it('caps at 200 panes with a console.error breadcrumb', () => { const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}) let panes = emptyPanesState() From 21301413c24b616a5e4929c40762a55feeff943e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 05:16:43 -0700 Subject: [PATCH 314/493] fix: normalize Cloud Playwright summary logs --- scripts/e2e-cloud.sh | 7 +++-- scripts/testing/e2e-cloud-log-summary.mjs | 18 ++++++++++++ .../testing/e2e-cloud-log-summary.test.ts | 28 +++++++++++++++++++ 3 files changed, 51 insertions(+), 2 deletions(-) create mode 100644 scripts/testing/e2e-cloud-log-summary.mjs create mode 100644 test/unit/tooling/testing/e2e-cloud-log-summary.test.ts diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index 2d46fbb8c..fdaa2209a 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -864,10 +864,13 @@ cmd_run() { # Extract and display a per-shard summary from the Playwright output. # Each shard's entrypoint prints "Shard X/Y assignment" and Playwright's # line reporter prints a final " N passed (duration)" or - # " N failed, M passed (duration)" summary line. + # " N failed, M passed (duration)" summary line. Cloud Logging prefixes + # each returned line with its timestamp and can preserve terminal cursor + # controls from Playwright, so normalize those before the target validates + # the footer's pass/fail/skip counts. echo "" echo "[e2e-cloud] Per-shard summary:" - echo "$log_output" | grep -E '(\[e2e-entrypoint\] Shard [0-9]+/[0-9]+ assignment|^\s+[0-9]+ (passed|failed))' || true + printf '%s\n' "$log_output" | node "$ROOT/scripts/testing/e2e-cloud-log-summary.mjs" # Check execution status — transient describe errors right after # `execute --wait` are a real flake class, so retry briefly; a PERMANENT diff --git a/scripts/testing/e2e-cloud-log-summary.mjs b/scripts/testing/e2e-cloud-log-summary.mjs new file mode 100644 index 000000000..5c689e038 --- /dev/null +++ b/scripts/testing/e2e-cloud-log-summary.mjs @@ -0,0 +1,18 @@ +import { readFileSync } from 'node:fs' +import path from 'node:path' +import { pathToFileURL } from 'node:url' + +const ansiEscape = /\u001B\[[0-?]*[ -/]*[@-~]/g +const cloudLogTimestamp = /^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}(?:\.\d+)? / + +export function extractE2eCloudPlaywrightSummaryLines(output) { + return output.split(/\r?\n/) + .map(line => line.replace(cloudLogTimestamp, '').replace(ansiEscape, '')) + .filter(line => /^\[e2e-entrypoint\] Shard [0-9]+\/[0-9]+ assignment\b/.test(line) + || /^\s+[0-9]+ (?:failed|passed|skipped|interrupted)\b/.test(line)) +} + +if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) { + const lines = extractE2eCloudPlaywrightSummaryLines(readFileSync(0, 'utf8')) + if (lines.length > 0) process.stdout.write(`${lines.join('\n')}\n`) +} diff --git a/test/unit/tooling/testing/e2e-cloud-log-summary.test.ts b/test/unit/tooling/testing/e2e-cloud-log-summary.test.ts new file mode 100644 index 000000000..31a5635b6 --- /dev/null +++ b/test/unit/tooling/testing/e2e-cloud-log-summary.test.ts @@ -0,0 +1,28 @@ +import { spawnSync } from 'node:child_process' +import path from 'node:path' +import { describe, expect, it } from 'vitest' +import { extractE2eCloudPlaywrightSummaryLines } from '../../../../scripts/testing/e2e-cloud-log-summary.mjs' + +const cliPath = path.resolve(import.meta.dirname, '../../../../scripts/testing/e2e-cloud-log-summary.mjs') + +describe('Cloud Playwright log summary extraction', () => { + it('removes Cloud Logging timestamps and terminal control codes before selecting the footer', () => { + const output = [ + '2026-10-12 12:12:13 [e2e-entrypoint] Shard 1/1 assignment: tests 1-3', + '2026-10-12 12:12:54 \u001b[1A\u001b[2K 3 passed (39.8s)\u001b[0m', + '2026-10-12 12:12:54 worker task completed successfully', + '2026-10-12 12:12:55 1 skipped (0.1s)', + ].join('\n') + + const expected = [ + '[e2e-entrypoint] Shard 1/1 assignment: tests 1-3', + ' 3 passed (39.8s)', + ' 1 skipped (0.1s)', + ] + expect(extractE2eCloudPlaywrightSummaryLines(output)).toEqual(expected) + + const cli = spawnSync(process.execPath, [cliPath], { input: output, encoding: 'utf8' }) + expect(cli.status, cli.stderr).toBe(0) + expect(cli.stdout.trimEnd().split('\n')).toEqual(expected) + }) +}) From 88a28d0132a65e05525d5f1fc7d2655a2c9dfe80 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 05:47:33 -0700 Subject: [PATCH 315/493] fix: capture live provider resume evidence --- .../src/provider_preparation_tests.rs | 141 +++++++++++++++++- crates/freshell-supervisor/src/recovery.rs | 107 +++++++++++-- 2 files changed, 231 insertions(+), 17 deletions(-) diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs index d9efb098a..ba33c954a 100644 --- a/crates/freshell-supervisor/src/provider_preparation_tests.rs +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -20,6 +20,7 @@ struct Daemon { host_tasks: Vec<tokio::task::JoinHandle<()>>, reject_inputs: bool, consumed_rejection: bool, + recovery_probes: Vec<String>, } struct Backend(Arc<Mutex<Daemon>>, std::path::PathBuf); @@ -211,10 +212,17 @@ impl RuntimeBackend for Backend { HostCommand::ProbeRecovery { resume_spec: Some(spec), .. - } => HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { - evidence_revision: spec.evidence_revision, - resume_spec: spec, - }), + } => { + state + .lock() + .unwrap() + .recovery_probes + .push(spec.provider_session.native_session_id.clone()); + HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { + evidence_revision: spec.evidence_revision, + resume_spec: spec, + }) + } command => panic!("unexpected synthetic host command {command:?}"), }; write_frame( @@ -372,6 +380,131 @@ async fn control_listener(supervisor: &Supervisor) -> tokio::task::JoinHandle<() task } +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn live_opencode_probe_persists_resume_evidence_before_enclosure_exit() { + let temp = tempfile::tempdir().unwrap(); + let registry_root = temp.path().join("registry"); + let registry = Registry::open(®istry_root, None).unwrap(); + let soul = SoulId::new(); + let daemon = Arc::new(Mutex::new(Daemon::default())); + let runtime_root = temp.path().join("runtime"); + let host_binary_path = temp.path().join("synthetic-host"); + std::fs::create_dir_all(&runtime_root).unwrap(); + std::fs::write(&host_binary_path, "synthetic-host-artifact").unwrap(); + + let config = SupervisorConfig { + runtime_root: runtime_root.clone(), + control_socket_path: temp.path().join("control.sock"), + host_binary_path: host_binary_path.clone(), + image_ref: format!("sha256:{}", "a".repeat(64)), + test_run_id: "live-resume-capture".into(), + control_secret: "synthetic-control-secret".into(), + lifecycle_log: temp.path().join("lifecycle.jsonl"), + admission: crate::admission::AdmissionPolicy::default(), + }; + let backend = Arc::new(Backend(daemon.clone(), registry_root)); + let supervisor = Supervisor::new(registry.clone(), backend.clone(), config).unwrap(); + let terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"terminal-live-resume", + "streamId":"stream-live-resume", + "mode":"opencode", + "program":"opencode", + "args":[], + "env":{"HOME":"/home/freshell/provider"}, + "cwd":temp.path(), + "runAsUid":65534, + "runAsGid":0, + "cols":80, + "rows":24, + "projectKey":"project-live-resume", + "workspacePath":temp.path(), + "providerLaunchContext":context(&soul, 1), + "providerSecretReferences":references() + })) + .unwrap(); + let prepared = registry + .prepare_launch(crate::registry::LaunchPreparation { + soul_id: soul.clone(), + provider: "opencode".into(), + provider_store_id: "store-exact".into(), + native_session_id: None, + creation_seed_ref: "seed-live-resume".into(), + request_id: RequestId::new(), + payload_digest: "live-resume-capture".into(), + requested_limits: limits(), + profile: RuntimeProfile::Custom, + project_key: "project-live-resume".into(), + fixture: None, + terminal: Some(terminal), + fresh_agent: None, + view_intent: None, + admission: crate::admission::AdmissionPolicy::default(), + }) + .await + .unwrap(); + let incarnation_id = prepared.incarnation_id.clone(); + let runtime_dir = runtime_root.join(incarnation_id.as_str()); + std::fs::create_dir_all(&runtime_dir).unwrap(); + std::fs::write(runtime_dir.join("secret"), b"synthetic-host-secret").unwrap(); + registry + .commit_created( + incarnation_id.clone(), + crate::registry::BackendCreatedRecord { + daemon_id: DockerDaemonId::parse("synthetic-daemon").unwrap(), + container_id: "d".repeat(64), + image_ref: format!("sha256:{}", "a".repeat(64)), + runtime_dir: runtime_dir.clone(), + host_binary_path, + immutable_config_digest: "e".repeat(64), + }, + ) + .await + .unwrap(); + let boot = HostBootId::new(); + registry + .commit_execution_grant(incarnation_id.clone(), boot, limits()) + .await + .unwrap(); + registry.mark_running(incarnation_id.clone()).await.unwrap(); + registry + .record_native_session(soul.clone(), incarnation_id.clone(), "ses_exact".into()) + .await + .unwrap(); + let handle = registry.owned_handle(incarnation_id).await.unwrap(); + backend.start_host(&handle).await.unwrap(); + + let live = supervisor.probe_recovery(soul.clone()).await.unwrap(); + assert!(matches!(live, RecoveryProbe::ReattachReady { .. })); + let durable = registry.recovery_context(soul.clone()).await.unwrap(); + assert_eq!( + durable + .resume_spec + .as_ref() + .map(|spec| spec.provider_session.native_session_id.as_str()), + Some("ses_exact"), + "a healthy live probe must capture exact provider-store evidence before the enclosure can disappear" + ); + assert_eq!(daemon.lock().unwrap().recovery_probes, ["ses_exact"]); + + daemon + .lock() + .unwrap() + .running + .remove(durable.prior_handle.incarnation_id()); + let after_exit = supervisor.probe_recovery(soul).await.unwrap(); + assert!(matches!( + after_exit, + RecoveryProbe::ResumeReady { resume_spec, .. } + if resume_spec.provider_session.native_session_id == "ses_exact" + )); + + let tasks = std::mem::take(&mut daemon.lock().unwrap().host_tasks); + for task in tasks { + task.abort(); + let _ = task.await; + } +} + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn provider_accepted_completion_retries_after_registry_success_on_fresh_healthy_startup() { for failure in ["controller", "publisher", "retire"] { diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index 8882bd7d3..23994689b 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -59,7 +59,7 @@ impl Supervisor { RecoveryBlockReason::OldRuntimeNotEmpty, &message, None, - )) + )); } Err(error) => { return Ok(blocked( @@ -67,7 +67,7 @@ impl Supervisor { RecoveryBlockReason::ProviderUnavailable, &error.to_string(), Some(2_000), - )) + )); } }; @@ -110,6 +110,10 @@ impl Supervisor { .map_err(map_registry)?; } if !status.exited { + if context.provider != "shell" && context.resume_spec.is_none() { + self.capture_live_resume_evidence(&context, &host, &status) + .await; + } return Ok(live_reattach_probe(&context, &status)); } if context.provider == "shell" { @@ -156,7 +160,7 @@ impl Supervisor { RecoveryBlockReason::ProviderUnavailable, &error.message, Some(2_000), - )) + )); } } } @@ -179,6 +183,67 @@ impl Supervisor { )) } + /// Capture a verified exact-session resume proof while the provider store + /// is still reachable. A live process remains reattachable even when the + /// provider cannot produce a proof yet; the lifecycle event records that + /// gap so an eventual whole-enclosure exit does not look like a mystery. + async fn capture_live_resume_evidence( + &self, + context: &RecoveryContext, + host: &crate::service::AuthenticatedHost, + status: &crate::service::HostStatus, + ) { + let capture = match resume_catalog::exact_resume_candidate( + context, + status.native_session_id.as_deref(), + ) { + Ok(Some(candidate)) => { + match self.probe_host_store(context, host, Some(candidate)).await { + Ok(RecoveryProbe::ResumeReady { resume_spec, .. }) => { + match self + .registry + .store_verified_resume_spec(context.soul_id.clone(), *resume_spec) + .await + { + Ok(stored) => serde_json::json!({ + "result": "captured", + "evidenceRevision": stored.evidence_revision, + }), + Err(error) => serde_json::json!({ + "result": "persist_failed", + "error": error.to_string(), + }), + } + } + Ok(probe) => serde_json::json!({ + "result": "not_verified", + "probe": probe, + }), + Err(error) => serde_json::json!({ + "result": "probe_failed", + "errorCode": error.code, + "error": error.message, + }), + } + } + Ok(None) => serde_json::json!({ "result": "identity_unavailable" }), + Err(probe) => serde_json::json!({ + "result": "candidate_blocked", + "probe": probe, + }), + }; + crate::service::append_event( + &self.config.lifecycle_log, + "supervisor.recovery.live_resume_evidence", + serde_json::json!({ + "soulId": context.soul_id, + "incarnationId": context.prior_handle.incarnation_id(), + "provider": context.provider, + "capture": capture, + }), + ); + } + async fn probe_host_store( &self, context: &RecoveryContext, @@ -264,10 +329,18 @@ impl Supervisor { { return Ok(RecoveryResult { outcome: RecoveryOutcome::Blocked, - probe: Some(blocked(RecoveryPath::NativeResume, RecoveryBlockReason::RetryBudget, - "automatic recovery budget is exhausted; repair the dependency and explicitly retry", None)), - view, prior_incarnation_id: None, attempt_id: None, - expected_native_session_id: None, observed_native_session_id: None, incident_id: None, + probe: Some(blocked( + RecoveryPath::NativeResume, + RecoveryBlockReason::RetryBudget, + "automatic recovery budget is exhausted; repair the dependency and explicitly retry", + None, + )), + view, + prior_incarnation_id: None, + attempt_id: None, + expected_native_session_id: None, + observed_native_session_id: None, + incident_id: None, }); } } @@ -958,7 +1031,7 @@ impl Supervisor { return Err(RuntimeError::new( RuntimeErrorCode::InvalidRequest, "legacy loss preservation requires a definitive-negative probe", - )) + )); } }; let (_, mut preserved_evidence) = @@ -1331,8 +1404,12 @@ impl Supervisor { }); } if context.dispatched_command_count == 0 { - return Err(blocked(RecoveryPath::PristineSeed, RecoveryBlockReason::AmbiguousIdentity, - "initialization/native identity is uncertain; no durable input dispatch establishes pristine-seed unavailability", None)); + return Err(blocked( + RecoveryPath::PristineSeed, + RecoveryBlockReason::AmbiguousIdentity, + "initialization/native identity is uncertain; no durable input dispatch establishes pristine-seed unavailability", + None, + )); } RecoveryPathEvidence { path: RecoveryPath::PristineSeed, @@ -1962,8 +2039,12 @@ fn offline_probe(context: &RecoveryContext) -> Result<RecoveryProbe, RuntimeErro ), }); } - Ok(blocked(RecoveryPath::NativeResume, RecoveryBlockReason::AmbiguousIdentity, - "native identity/resume metadata is unrecorded or unverified; provider store absence has not been established", None)) + Ok(blocked( + RecoveryPath::NativeResume, + RecoveryBlockReason::AmbiguousIdentity, + "native identity/resume metadata is unrecorded or unverified; provider store absence has not been established", + None, + )) } fn downtime_bucket(elapsed_ms: u64) -> &'static str { @@ -2092,7 +2173,7 @@ fn install_prepared_provider_context( return Err(RuntimeError::new( RuntimeErrorCode::InvalidRequest, "complete provider replacement requires exactly one workload", - )) + )); } } Ok(()) From 910c32b04823bee40b22cf032109332d34f4e5b4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 05:47:43 -0700 Subject: [PATCH 316/493] test: tolerate loaded native auth callback --- crates/freshell-supervisor/src/native_auth_lookup_tests.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index aa5924f55..89639fe88 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -562,7 +562,7 @@ async fn provider_native_auth_nested_callback_uses_current_attempt_without_reloc envelope }; let reply = tokio::time::timeout( - std::time::Duration::from_millis(250), + std::time::Duration::from_secs(2), supervisor.dispatch(request(soul.clone(), key.clone())), ) .await From 3cc328a56de66beac55736d512f280eec69211a9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 07:05:29 -0700 Subject: [PATCH 317/493] Fix managed checkpoint identity and retry recovery --- .../freshell-agent-runtime/src/host_actor.rs | 42 +- .../src/host_actor_tests.rs | 83 + crates/freshell-runtime-client/src/lib.rs | 55 + crates/freshell-runtime-protocol/src/lib.rs | 68 + crates/freshell-server/src/checkpoints.rs | 1760 +++++++++++++++-- .../freshell-server/src/fresh_agent_proxy.rs | 12 +- .../src/fresh_agent_proxy_rest.rs | 2 +- crates/freshell-session-host/src/main.rs | 32 +- .../src/providers/fresh_agent.rs | 81 +- .../src/claude_source_observation.rs | 20 +- .../src/native_auth_lookup_tests.rs | 7 +- .../src/provider_preparation_tests.rs | 78 +- crates/freshell-supervisor/src/recovery.rs | 46 + crates/freshell-supervisor/src/service.rs | 15 +- docs/development/managed-runtime.md | 35 + src/components/fresh-agent/FreshAgentView.tsx | 88 +- test/e2e-browser/fixtures/fake-opencode.cjs | 23 +- ...code-checkpoint-workspace-external.spec.ts | 7 +- .../managed-native-history/codex-tools.json | 1 + .../fresh-agent/FreshAgentView.test.tsx | 83 +- .../fake-opencode-fixture.test.ts | 36 + 21 files changed, 2368 insertions(+), 206 deletions(-) diff --git a/crates/freshell-agent-runtime/src/host_actor.rs b/crates/freshell-agent-runtime/src/host_actor.rs index 03a52ef8e..2a157e170 100644 --- a/crates/freshell-agent-runtime/src/host_actor.rs +++ b/crates/freshell-agent-runtime/src/host_actor.rs @@ -201,6 +201,18 @@ pub trait FreshAgentTransport: Send + Sync { text: &str, profile: &FreshAgentProfile, ) -> Result<DispatchAck, DispatchFailure>; + /// Dispatches a send whose checkpoint attempt identity must reach the + /// provider-native turn receipt. Legacy transports remain compatible and + /// ignore the optional metadata by delegating to `dispatch`. + async fn dispatch_with_checkpoint_identity( + &self, + request_id: &RequestId, + text: &str, + profile: &FreshAgentProfile, + _logical_attempt_id: Option<&str>, + ) -> Result<DispatchAck, DispatchFailure> { + self.dispatch(request_id, text, profile).await + } async fn resolve_permission( &self, decision_id: &str, @@ -571,12 +583,33 @@ impl FreshAgentHostActor { request_id: RequestId, text: String, settings: Option<FreshAgentTurnSettings>, + ) -> Result<CommandState, ActorError> { + self.dispatch_with_checkpoint_identity(request_id, text, settings, None) + .await + } + + pub async fn dispatch_with_checkpoint_identity( + &self, + request_id: RequestId, + text: String, + settings: Option<FreshAgentTurnSettings>, + logical_attempt_id: Option<String>, ) -> Result<CommandState, ActorError> { // One provider conversation has one ordered input lane. Keep the // durability transition and provider acceptance boundary within the // same fence so concurrent gateways cannot become double writers. let _dispatch_guard = self.dispatch_lock.lock().await; - let command_material = serde_json::to_vec(&(text.as_str(), &settings)).map_err(persist)?; + // Preserve the exact historical digest for legacy sends so a command + // journal created before this field existed remains idempotent after + // upgrade. A modern checkpointed send additionally fences the logical + // attempt identity into durable request-ID conflict detection. + let command_material = match logical_attempt_id.as_deref() { + Some(logical_attempt_id) => { + serde_json::to_vec(&(text.as_str(), &settings, logical_attempt_id)) + } + None => serde_json::to_vec(&(text.as_str(), &settings)), + } + .map_err(persist)?; let digest = digest(&command_material); let dispatch_profile; { @@ -618,7 +651,12 @@ impl FreshAgentHostActor { let result = self .transport - .dispatch(&request_id, &text, &dispatch_profile) + .dispatch_with_checkpoint_identity( + &request_id, + &text, + &dispatch_profile, + logical_attempt_id.as_deref(), + ) .await; let mut state = self.state.lock().await; let command = state diff --git a/crates/freshell-agent-runtime/src/host_actor_tests.rs b/crates/freshell-agent-runtime/src/host_actor_tests.rs index 14069eae4..102648484 100644 --- a/crates/freshell-agent-runtime/src/host_actor_tests.rs +++ b/crates/freshell-agent-runtime/src/host_actor_tests.rs @@ -335,6 +335,7 @@ struct FakeTransport { starts: AtomicUsize, start_profiles: std::sync::Mutex<Vec<FreshAgentProfile>>, dispatches: AtomicUsize, + checkpoint_attempts: std::sync::Mutex<Vec<Option<String>>>, decisions: AtomicUsize, stops: AtomicUsize, ambiguous_dispatch: bool, @@ -434,6 +435,7 @@ impl FakeTransport { starts: AtomicUsize::new(0), start_profiles: std::sync::Mutex::new(Vec::new()), dispatches: AtomicUsize::new(0), + checkpoint_attempts: std::sync::Mutex::new(Vec::new()), decisions: AtomicUsize::new(0), stops: AtomicUsize::new(0), ambiguous_dispatch: false, @@ -448,6 +450,7 @@ impl FakeTransport { starts: AtomicUsize::new(0), start_profiles: std::sync::Mutex::new(Vec::new()), dispatches: AtomicUsize::new(0), + checkpoint_attempts: std::sync::Mutex::new(Vec::new()), decisions: AtomicUsize::new(0), stops: AtomicUsize::new(0), ambiguous_dispatch: false, @@ -484,6 +487,19 @@ impl FreshAgentTransport for FakeTransport { }) } } + async fn dispatch_with_checkpoint_identity( + &self, + request_id: &RequestId, + text: &str, + profile: &FreshAgentProfile, + logical_attempt_id: Option<&str>, + ) -> Result<DispatchAck, DispatchFailure> { + self.checkpoint_attempts + .lock() + .unwrap() + .push(logical_attempt_id.map(str::to_owned)); + self.dispatch(request_id, text, profile).await + } async fn resolve_permission( &self, _decision_id: &str, @@ -882,6 +898,73 @@ async fn ambiguous_accepted_input_is_never_dispatched_twice() { assert_eq!(transport.dispatches.load(Ordering::SeqCst), 1); } +#[tokio::test] +async fn checkpoint_attempt_identity_survives_recovery_and_fences_request_id_reuse() { + let dir = tempfile::tempdir().unwrap(); + let transport = FakeTransport::new("checkpoint-native"); + let actor = FreshAgentHostActor::open( + dir.path(), + profile(FreshProvider::Opencode, "checkpoint-store", None), + transport.clone(), + ) + .await + .unwrap(); + let request = RequestId::parse("checkpoint-command").unwrap(); + assert_eq!( + actor + .dispatch_with_checkpoint_identity( + request.clone(), + "checkpointed work".into(), + None, + Some("attempt-one".into()), + ) + .await, + Ok(CommandState::ProviderAcked) + ); + drop(actor); + + let recovered = FreshAgentHostActor::open( + dir.path(), + profile( + FreshProvider::Opencode, + "checkpoint-store", + Some("checkpoint-native"), + ), + transport.clone(), + ) + .await + .unwrap(); + assert_eq!( + recovered + .dispatch_with_checkpoint_identity( + request.clone(), + "checkpointed work".into(), + None, + Some("attempt-one".into()), + ) + .await, + Ok(CommandState::ProviderAcked), + "retrying the durable request with the same identity returns its recorded result" + ); + assert_eq!( + recovered + .dispatch_with_checkpoint_identity( + request, + "checkpointed work".into(), + None, + Some("attempt-two".into()), + ) + .await, + Err(ActorError::RequestConflict), + "one durable request ID cannot be rebound to another logical attempt" + ); + assert_eq!(transport.dispatches.load(Ordering::SeqCst), 1); + assert_eq!( + *transport.checkpoint_attempts.lock().unwrap(), + vec![Some("attempt-one".into())] + ); +} + #[tokio::test] async fn one_state_directory_has_only_one_writer() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-runtime-client/src/lib.rs b/crates/freshell-runtime-client/src/lib.rs index faa4c850f..3d4ee3441 100644 --- a/crates/freshell-runtime-client/src/lib.rs +++ b/crates/freshell-runtime-client/src/lib.rs @@ -709,6 +709,7 @@ impl RuntimeClient { soul_id: SoulId, text: String, settings: Option<freshell_runtime_protocol::FreshAgentTurnSettings>, + checkpoint_logical_attempt_id: Option<String>, ) -> Result<freshell_runtime_protocol::CommandState, ClientError> { let epoch = self.current_epoch().await?; match self @@ -718,6 +719,7 @@ impl RuntimeClient { soul_id, text, settings, + checkpoint_logical_attempt_id, expected_control_epoch: Some(epoch), }), ) @@ -1203,6 +1205,59 @@ mod tests { } } + #[tokio::test] + async fn fresh_agent_send_preserves_checkpoint_identity_on_the_control_request() { + let dir = tempfile::tempdir().unwrap(); + let socket = dir.path().join("control.sock"); + let listener = UnixListener::bind(&socket).unwrap(); + let server = tokio::spawn(async move { + let (mut stream, _) = listener.accept().await.unwrap(); + let envelope: Envelope<AdminCommand> = read_frame(&mut stream).await.unwrap(); + let AdminCommand::FreshAgentSend(request) = &envelope.body else { + panic!("expected managed fresh-agent send: {envelope:?}"); + }; + assert_eq!( + request.soul_id, + SoulId::parse("checkpoint-send-soul").unwrap() + ); + assert_eq!(request.text, "continue with the requested change"); + assert_eq!( + request.checkpoint_logical_attempt_id.as_deref(), + Some("attempt-control-1") + ); + assert_eq!(request.expected_control_epoch, Some(19)); + write_frame( + &mut stream, + &AdminReply { + request_id: envelope.request_id, + result: Ok(AdminResult::FreshAgentCommand { + state: freshell_runtime_protocol::CommandState::ProviderAcked, + }), + }, + ) + .await + .unwrap(); + }); + let client = RuntimeClient::new(&socket, "0123456789abcdef"); + *client.control_epoch.write().await = Some(19); + + let state = client + .fresh_agent_send( + RequestId::parse("request-checkpoint-send").unwrap(), + SoulId::parse("checkpoint-send-soul").unwrap(), + "continue with the requested change".into(), + None, + Some("attempt-control-1".into()), + ) + .await + .unwrap(); + server.await.unwrap(); + assert_eq!( + state, + freshell_runtime_protocol::CommandState::ProviderAcked + ); + } + fn source_snapshot(kind: FreshAgentReadKind) -> serde_json::Value { let live = matches!(kind, FreshAgentReadKind::Snapshot); serde_json::json!({ diff --git a/crates/freshell-runtime-protocol/src/lib.rs b/crates/freshell-runtime-protocol/src/lib.rs index ee9a4d0e7..3803e0d8b 100644 --- a/crates/freshell-runtime-protocol/src/lib.rs +++ b/crates/freshell-runtime-protocol/src/lib.rs @@ -2478,6 +2478,12 @@ pub struct FreshAgentSendRequest { pub text: String, #[serde(default, skip_serializing_if = "Option::is_none")] pub settings: Option<FreshAgentTurnSettings>, + /// Opaque checkpoint attempt identity, when this send crossed the modern + /// server-owned snapshot barrier. It is passed through the managed host to + /// the provider so the native turn receipt can be joined without text or + /// ordinal matching. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub checkpoint_logical_attempt_id: Option<String>, #[serde(skip_serializing_if = "Option::is_none")] pub expected_control_epoch: Option<u64>, } @@ -3425,6 +3431,8 @@ pub enum HostCommand { text: String, #[serde(default, skip_serializing_if = "Option::is_none")] settings: Option<FreshAgentTurnSettings>, + #[serde(default, skip_serializing_if = "Option::is_none")] + checkpoint_logical_attempt_id: Option<String>, }, FreshAgentFork { incarnation_id: IncarnationId, @@ -3685,6 +3693,66 @@ where mod tests { use super::*; + #[test] + fn hosted_checkpoint_send_identity_is_optional_and_backward_compatible() { + let request: FreshAgentSendRequest = serde_json::from_value(serde_json::json!({ + "soulId": "soul-hosted", + "text": "continue", + "checkpointLogicalAttemptId": "attempt-hosted-1" + })) + .unwrap(); + assert_eq!( + request.checkpoint_logical_attempt_id.as_deref(), + Some("attempt-hosted-1") + ); + + let legacy_request: FreshAgentSendRequest = serde_json::from_value(serde_json::json!({ + "soulId": "soul-hosted", + "text": "legacy continue" + })) + .unwrap(); + assert_eq!(legacy_request.checkpoint_logical_attempt_id, None); + assert_eq!( + serde_json::to_value(&legacy_request).unwrap(), + serde_json::json!({ "soulId": "soul-hosted", "text": "legacy continue" }) + ); + + let host_command: HostCommand = serde_json::from_value(serde_json::json!({ + "method": "fresh_agent_send", + "params": { + "incarnation_id": "incarnation-hosted", + "request_id": "request-hosted", + "text": "continue", + "checkpoint_logical_attempt_id": "attempt-hosted-1" + } + })) + .unwrap(); + assert!(matches!( + host_command, + HostCommand::FreshAgentSend { + checkpoint_logical_attempt_id: Some(ref id), + .. + } if id == "attempt-hosted-1" + )); + + let legacy_host_command: HostCommand = serde_json::from_value(serde_json::json!({ + "method": "fresh_agent_send", + "params": { + "incarnation_id": "incarnation-hosted", + "request_id": "request-hosted", + "text": "legacy continue" + } + })) + .unwrap(); + assert!(matches!( + legacy_host_command, + HostCommand::FreshAgentSend { + checkpoint_logical_attempt_id: None, + .. + } + )); + } + #[test] fn provider_preparation_serialization_stops_before_materializing_oversized_metadata() { use std::sync::atomic::{AtomicUsize, Ordering}; diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 23b8739af..67bde06cb 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -95,6 +95,63 @@ const CHECKPOINT_TIMEOUT_COOLDOWN: Duration = Duration::from_secs(60); const CHECKPOINT_COOLDOWN_LIMIT: usize = 128; const CHECKPOINT_OUTCOME_STORE_VERSION: u8 = 1; const CHECKPOINT_OUTCOME_STORE_LIMIT: usize = 8192; +const CHECKPOINT_DISPATCH_FENCE_LIMIT: usize = CHECKPOINT_OUTCOME_STORE_LIMIT; +const CHECKPOINT_VOLATILE_FENCE_LIMIT: usize = 1024; + +#[derive(Clone, Copy)] +struct CheckpointAttemptFence { + dispatch_started: bool, + capture_settled: bool, +} + +#[derive(Default)] +struct CheckpointOutcomeCache { + outcomes: HashMap<String, StoredCheckpointOutcome>, + snapshot_loaded: bool, + store_file_observed: bool, + snapshot_complete: bool, +} + +impl CheckpointOutcomeCache { + fn replace_from_store(&mut self, store: &CheckpointOutcomeStore) { + self.snapshot_loaded = true; + self.store_file_observed = true; + self.snapshot_complete = store.outcomes.len() <= CHECKPOINT_DISPATCH_FENCE_LIMIT; + self.outcomes = store + .outcomes + .iter() + .take(CHECKPOINT_DISPATCH_FENCE_LIMIT) + .map(|(key, record)| (key.clone(), record.clone())) + .collect(); + } + + fn observe_absent_store(&mut self) { + self.snapshot_loaded = true; + self.snapshot_complete = true; + } + + fn known_attempt(&self, key: &str) -> Option<CheckpointAttemptFence> { + self.outcomes.get(key).map(|record| CheckpointAttemptFence { + dispatch_started: record.dispatch_started, + capture_settled: record.capture_settled, + }) + } + + fn cached_outcome(&self, key: &str) -> Option<StoredCheckpointOutcome> { + self.outcomes.get(key).cloned() + } + + fn cached_outcomes(&self) -> BTreeMap<String, StoredCheckpointOutcome> { + self.outcomes + .iter() + .map(|(key, outcome)| (key.clone(), outcome.clone())) + .collect() + } + + fn can_rule_out_unknown_attempt(&self) -> bool { + self.snapshot_loaded && self.snapshot_complete + } +} #[derive(Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] @@ -204,7 +261,10 @@ pub struct CheckpointService { outcome_gate: Mutex<()>, attempt_jobs: Mutex<HashMap<String, Arc<CheckpointAttemptJob>>>, cooldowns: StdMutex<HashMap<PathBuf, CheckpointCooldown>>, - in_memory_outcomes: StdMutex<BTreeMap<String, StoredCheckpointOutcome>>, + durable_outcome_cache: StdMutex<CheckpointOutcomeCache>, + volatile_outcomes: StdMutex<BTreeMap<String, StoredCheckpointOutcome>>, + #[cfg(test)] + test_capture_attempt_override: StdMutex<Option<CaptureAttempt>>, } tokio::task_local! { @@ -727,7 +787,10 @@ impl CheckpointService { outcome_gate: Mutex::new(()), attempt_jobs: Mutex::new(HashMap::new()), cooldowns: StdMutex::new(HashMap::new()), - in_memory_outcomes: StdMutex::new(BTreeMap::new()), + durable_outcome_cache: StdMutex::new(CheckpointOutcomeCache::default()), + volatile_outcomes: StdMutex::new(BTreeMap::new()), + #[cfg(test)] + test_capture_attempt_override: StdMutex::new(None), } } @@ -763,6 +826,82 @@ impl CheckpointService { .join("attempt-outcomes-v1.json") } + fn outcome_store_presence_marker_path(&self) -> PathBuf { + self.home + .join(".freshell") + .join("checkpoints") + .join("attempt-outcomes-v1.initialized") + } + + async fn ensure_outcome_store_presence_marker(&self) -> Result<(), String> { + let path = self.outcome_store_presence_marker_path(); + #[cfg(unix)] + let open_result = { + let mut options = tokio::fs::OpenOptions::new(); + options.write(true).create_new(true).mode(0o600); + options.open(&path).await + }; + #[cfg(not(unix))] + let open_result = tokio::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&path) + .await; + + let mut marker = match open_result { + Ok(marker) => marker, + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => return Ok(()), + Err(error) => { + return Err(format!( + "failed to create checkpoint outcome presence marker: {error}" + )); + } + }; + marker + .write_all(b"freshell checkpoint outcomes initialized v1\n") + .await + .map_err(|error| { + format!("failed to write checkpoint outcome presence marker: {error}") + })?; + marker.sync_all().await.map_err(|error| { + format!("failed to flush checkpoint outcome presence marker: {error}") + })?; + #[cfg(unix)] + { + let parent = path.parent().ok_or_else(|| { + "checkpoint outcome presence marker has no parent directory".to_owned() + })?; + tokio::fs::File::open(parent) + .await + .map_err(|error| { + format!("failed to open checkpoint outcome directory for flush: {error}") + })? + .sync_all() + .await + .map_err(|error| { + format!("failed to flush checkpoint outcome directory: {error}") + })?; + } + Ok(()) + } + + async fn outcome_store_presence_marker_exists(&self) -> Result<bool, String> { + match tokio::fs::symlink_metadata(self.outcome_store_presence_marker_path()).await { + Ok(_) => Ok(true), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(error) => Err(format!( + "failed to inspect checkpoint outcome presence marker: {error}" + )), + } + } + + fn mark_outcome_store_observed(&self) { + self.durable_outcome_cache + .lock() + .expect("checkpoint dispatch fences mutex poisoned") + .store_file_observed = true; + } + fn workspace_lock(&self, workspace: &Path) -> Arc<Mutex<()>> { let key = resolve_absolute(workspace); let mut locks = self @@ -858,22 +997,59 @@ impl CheckpointService { let raw = match tokio::fs::read(&path).await { Ok(raw) => raw, Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + let marker_exists = match self.outcome_store_presence_marker_exists().await { + Ok(exists) => exists, + Err(marker_error) => { + self.mark_outcome_store_observed(); + return Err(marker_error); + } + }; + let mut fences = self + .durable_outcome_cache + .lock() + .expect("checkpoint dispatch fences mutex poisoned"); + if marker_exists || fences.store_file_observed { + fences.store_file_observed = true; + return Err( + "checkpoint attempt outcome store is missing after its prior presence could not be ruled out".into(), + ); + } + fences.observe_absent_store(); return Ok(CheckpointOutcomeStore { version: CHECKPOINT_OUTCOME_STORE_VERSION, outcomes: BTreeMap::new(), }); } Err(error) => { + self.mark_outcome_store_observed(); + let marker_error = self + .ensure_outcome_store_presence_marker() + .await + .err() + .map(|error| format!("; {error}")) + .unwrap_or_default(); return Err(format!( - "failed to read checkpoint attempt outcomes: {error}" + "failed to read checkpoint attempt outcomes: {error}{marker_error}" )); } }; + self.mark_outcome_store_observed(); + if let Err(error) = self.ensure_outcome_store_presence_marker().await { + tracing::error!( + event = "code_checkpoint.outcome_presence_marker_write_failed", + cause = %error, + "checkpoint outcome store was readable but its durable presence marker could not be written" + ); + } let store: CheckpointOutcomeStore = serde_json::from_slice(&raw) .map_err(|error| format!("invalid checkpoint attempt outcome store: {error}"))?; if store.version != CHECKPOINT_OUTCOME_STORE_VERSION { return Err("unsupported checkpoint attempt outcome store version".to_string()); } + self.durable_outcome_cache + .lock() + .expect("checkpoint dispatch fences mutex poisoned") + .replace_from_store(&store); Ok(store) } @@ -885,6 +1061,7 @@ impl CheckpointService { tokio::fs::create_dir_all(parent) .await .map_err(|error| format!("failed to create checkpoint outcome directory: {error}"))?; + self.ensure_outcome_store_presence_marker().await?; let temporary = parent.join(format!( "attempt-outcomes-v1.json.tmp-{}", uuid::Uuid::new_v4() @@ -918,6 +1095,10 @@ impl CheckpointService { let _ = tokio::fs::remove_file(&temporary).await; return Err(format!("failed to publish checkpoint outcomes: {error}")); } + self.durable_outcome_cache + .lock() + .expect("checkpoint dispatch fences mutex poisoned") + .replace_from_store(store); Ok(()) } @@ -939,33 +1120,113 @@ impl CheckpointService { store.outcomes.remove(&oldest); } self.write_outcome_store(&store).await?; - self.in_memory_outcomes + self.volatile_outcomes .lock() - .expect("checkpoint outcome memory cache poisoned") - .insert(key.to_string(), outcome); + .expect("volatile checkpoint outcomes mutex poisoned") + .remove(key); Ok(()) } + fn attempt_state_for_store_failure(&self, key: &str) -> (Option<CheckpointAttemptFence>, bool) { + let fences = self + .durable_outcome_cache + .lock() + .expect("checkpoint dispatch fences mutex poisoned"); + ( + fences.known_attempt(key), + fences.can_rule_out_unknown_attempt(), + ) + } + + fn remember_volatile_outcome(&self, key: &str, outcome: StoredCheckpointOutcome) -> bool { + let mut outcomes = self + .volatile_outcomes + .lock() + .expect("volatile checkpoint outcomes mutex poisoned"); + if !outcomes.contains_key(key) && outcomes.len() >= CHECKPOINT_VOLATILE_FENCE_LIMIT { + return false; + } + outcomes.insert(key.to_string(), outcome); + true + } + + fn mark_outcome_store_failure( + record: &mut StoredCheckpointOutcome, + operation: &'static str, + ) -> bool { + let allow_forward = record.capture_settled; + record.dispatch_started = allow_forward; + record.outcome.status = FreshAgentCheckpointStatus::Unavailable; + record.outcome.checkpoint_id = None; + record.outcome.code = Some("outcome_store_unavailable".into()); + record.outcome.reason = Some(if allow_forward { + format!( + "{operation}; rewind for this message is unknown. Your message will still be sent." + ) + } else { + format!("{operation}, and the pre-turn snapshot is still settling; your message was withheld.") + }); + allow_forward + } + + fn mark_known_attempt_store_failure( + record: &mut StoredCheckpointOutcome, + state: CheckpointAttemptFence, + ) { + record.dispatch_started = state.dispatch_started; + record.capture_settled = state.capture_settled; + record.outcome.status = FreshAgentCheckpointStatus::Unavailable; + record.outcome.checkpoint_id = None; + record.outcome.code = Some("outcome_store_unavailable".into()); + record.outcome.reason = Some(if !state.capture_settled { + "Rewind status could not be read, and the pre-turn snapshot is still settling; your message was withheld.".into() + } else { + "Rewind status could not be read; an earlier dispatch may already have started, so this retry was withheld to prevent a duplicate.".into() + }); + } + + fn mark_unverified_attempt_store_failure(record: &mut StoredCheckpointOutcome) { + record.dispatch_started = false; + record.capture_settled = false; + record.outcome.status = FreshAgentCheckpointStatus::Unavailable; + record.outcome.checkpoint_id = None; + record.outcome.code = Some("outcome_store_unavailable".into()); + record.outcome.reason = Some( + "Rewind status could not be read, and prior send history could not be verified; your message was withheld.".into(), + ); + } + + fn mark_volatile_fence_capacity_failure(record: &mut StoredCheckpointOutcome) { + record.dispatch_started = false; + record.outcome.status = FreshAgentCheckpointStatus::Unavailable; + record.outcome.checkpoint_id = None; + record.outcome.code = Some("outcome_store_unavailable".into()); + record.outcome.reason = Some( + "Rewind status could not be saved, and local duplicate protection is full; your message was withheld.".into(), + ); + } + async fn find_outcome(&self, key: &str) -> Result<Option<StoredCheckpointOutcome>, String> { if let Some(outcome) = self - .in_memory_outcomes + .volatile_outcomes .lock() - .expect("checkpoint outcome memory cache poisoned") + .expect("volatile checkpoint outcomes mutex poisoned") .get(key) .cloned() { return Ok(Some(outcome)); } + if let Some(outcome) = self + .durable_outcome_cache + .lock() + .expect("durable checkpoint outcome cache mutex poisoned") + .cached_outcome(key) + { + return Ok(Some(outcome)); + } let _guard = self.outcome_gate.lock().await; let store = self.read_outcome_store().await?; - let outcome = store.outcomes.get(key).cloned(); - if let Some(outcome) = &outcome { - self.in_memory_outcomes - .lock() - .expect("checkpoint outcome memory cache poisoned") - .insert(key.to_string(), outcome.clone()); - } - Ok(outcome) + Ok(store.outcomes.get(key).cloned()) } async fn replay_for_identity( @@ -974,18 +1235,29 @@ impl CheckpointService { session_id: &str, session_type: SessionType, ) { - let records = match self.read_outcome_store().await { - Ok(store) => store.outcomes, - Err(error) => { - tracing::error!( - event = "code_checkpoint.outcome_store_read_failed", - cause = %error, - "durable checkpoint outcomes could not be replayed" - ); - self.in_memory_outcomes - .lock() - .expect("checkpoint outcome memory cache poisoned") - .clone() + let records = { + let _guard = self.outcome_gate.lock().await; + match self.read_outcome_store().await { + Ok(store) => store.outcomes, + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_store_read_failed", + cause = %error, + "durable checkpoint outcomes could not be replayed" + ); + let mut cached = self + .durable_outcome_cache + .lock() + .expect("durable checkpoint outcome cache mutex poisoned") + .cached_outcomes(); + cached.extend( + self.volatile_outcomes + .lock() + .expect("volatile checkpoint outcomes mutex poisoned") + .clone(), + ); + cached + } } }; for record in records.values().filter(|record| { @@ -1047,13 +1319,6 @@ impl CheckpointService { } if !changed.is_empty() { self.write_outcome_store(&store).await?; - let mut cache = self - .in_memory_outcomes - .lock() - .expect("checkpoint outcome memory cache poisoned"); - for (key, record) in &store.outcomes { - cache.insert(key.clone(), record.clone()); - } } Ok(changed) } @@ -1093,8 +1358,19 @@ impl CheckpointService { .session_aliases .iter() .any(|alias| alias == session_id) + && submitted_turn_id.as_deref().is_none_or(|submitted| { + record + .outcome + .native_turn_id + .as_deref() + .is_none_or(|native| native == submitted) + }) + }, + |record| { + if submitted_turn_id.is_some() { + record.submitted_turn_id = submitted_turn_id.clone(); + } }, - |record| record.submitted_turn_id = submitted_turn_id.clone(), ) .await; if let Err(error) = updated { @@ -1205,6 +1481,10 @@ impl CheckpointService { .session_aliases .iter() .any(|alias| alias == session_id) + && record + .submitted_turn_id + .as_deref() + .is_none_or(|submitted| submitted == native_turn_id) && record .outcome .native_turn_id @@ -1381,7 +1661,7 @@ impl CheckpointService { None, Err(( "workspace_unavailable", - "No selected code workspace is available for this send.".into(), + "No selected code workspace is available for a snapshot; rewind is unavailable, and your message will still be sent.".into(), )), ); }; @@ -1408,7 +1688,7 @@ impl CheckpointService { None, Err(( "workspace_unavailable", - "No eligible code workspace is available for this send.".into(), + "No eligible code workspace is available for a snapshot; rewind is unavailable, and your message will still be sent.".into(), )), ); } @@ -1580,7 +1860,7 @@ impl CheckpointService { )), ); record.capture_settled = false; - let _quarantine = self.quarantine_workspace( + let quarantine = self.quarantine_workspace( workspace.path.clone(), lock, control, @@ -1588,6 +1868,40 @@ impl CheckpointService { "automatic_capture", error, ); + match quarantine.await { + Ok(()) => { + if let Some(checkpoint_id) = self + .reconcile_published_request(&workspace, request_id) + .await + { + return self.new_record( + send, + logical_attempt_id, + request_id, + Some((&workspace, "")), + Ok(checkpoint_id), + ); + } + record.capture_settled = true; + record.outcome.code = Some("capture_failed".into()); + record.outcome.reason = Some( + "The code snapshot could not be completed; rewind is unavailable, and your message will still be sent." + .into(), + ); + } + Err(settlement_error) => { + tracing::error!( + event = "code_checkpoint.quarantine_task_failed", + workspace = %workspace.path.display(), + request_id, + cause = %settlement_error, + "checkpoint child settlement was not confirmed; the provider send remains withheld" + ); + record.outcome.reason = Some(format!( + "The code snapshot could not be confirmed settled; your message was withheld ({settlement_error})." + )); + } + } record } CaptureAttempt::Failed(error) => { @@ -1809,6 +2123,16 @@ impl CheckpointService { budget: Duration, mut cancellation: Option<watch::Receiver<bool>>, ) -> CaptureAttempt { + #[cfg(test)] + if let Some(attempt) = self + .test_capture_attempt_override + .lock() + .expect("checkpoint test capture override mutex poisoned") + .take() + { + return attempt; + } + // The task is owned by this server service, not by the socket/HTTP // request future. On budget expiry the scoped runner is terminated, // reaped, and reconciled under the workspace lock before this returns. @@ -2069,7 +2393,8 @@ impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for Checkpoint return; } - let (record, allow_forward) = match self.find_outcome(&key).await { + let mut volatile_fallback = false; + let (record, mut allow_forward) = match self.find_outcome(&key).await { Ok(Some(mut record)) => { let added_alias = !record.request_ids.iter().any(|alias| alias == &request_id) && record.request_ids.len() < 16; @@ -2112,21 +2437,21 @@ impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for Checkpoint match self.persist_outcome(&key, record.clone()).await { Ok(()) => (record, true), Err(error) => { + let allow_forward = Self::mark_outcome_store_failure( + &mut record, + "Rewind status could not be saved", + ); + volatile_fallback = true; tracing::error!( event = "code_checkpoint.outcome_persist_failed", send_request_id = %request_id, logical_attempt_id = %logical_attempt_id, cause = %error, - "checkpoint outcome and dispatch fence could not be saved; provider send was withheld" - ); - record.dispatch_started = false; - record.outcome.status = FreshAgentCheckpointStatus::Unavailable; - record.outcome.checkpoint_id = None; - record.outcome.code = Some("outcome_store_unavailable".into()); - record.outcome.reason = Some( - "The checkpoint result could not be saved; this turn will remain unavailable.".into(), + dispatch_allowed = allow_forward, + dispatch_fence = if allow_forward { "process_local" } else { "none" }, + "checkpoint outcome and durable dispatch fence could not be saved" ); - (record, false) + (record, allow_forward) } } } @@ -2142,49 +2467,87 @@ impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for Checkpoint (record, allow_forward) } Err(error) => { + let allow_forward = Self::mark_outcome_store_failure( + &mut record, + "Rewind status could not be saved", + ); + volatile_fallback = true; tracing::error!( event = "code_checkpoint.outcome_persist_failed", send_request_id = %request_id, logical_attempt_id = %logical_attempt_id, cause = %error, - "checkpoint outcome and dispatch fence could not be saved; provider send was withheld" - ); - record.dispatch_started = false; - record.outcome.status = FreshAgentCheckpointStatus::Unavailable; - record.outcome.checkpoint_id = None; - record.outcome.code = Some("outcome_store_unavailable".into()); - record.outcome.reason = Some( - "The checkpoint result could not be saved; this turn will remain unavailable.".into(), + dispatch_allowed = allow_forward, + dispatch_fence = if allow_forward { "process_local" } else { "none" }, + "checkpoint outcome and durable dispatch fence could not be saved" ); - (record, false) + (record, allow_forward) } } } Err(error) => { + let (known_attempt, can_rule_out_prior_attempt) = + self.attempt_state_for_store_failure(&key); tracing::error!( event = "code_checkpoint.outcome_store_read_failed", send_request_id = %request_id, logical_attempt_id = %logical_attempt_id, cause = %error, - "checkpoint attempt could not be safely deduplicated" + known_dispatched_attempt = known_attempt.is_some_and(|state| state.dispatch_started), + known_unsettled_attempt = known_attempt.is_some_and(|state| !state.capture_settled), + can_rule_out_prior_attempt, + "durable checkpoint attempt history is unavailable; dispatch depends on the last trusted snapshot" ); - ( - self.new_record( - &send, - &logical_attempt_id, - &request_id, - None, - Err(( - "outcome_store_unavailable", - "The checkpoint result could not be saved; this turn will remain unavailable.".into(), - )), - ), - false, - ) + let mut record = self.new_record( + &send, + &logical_attempt_id, + &request_id, + None, + Err(( + "outcome_store_unavailable", + "Rewind status could not be read".into(), + )), + ); + let allow_forward = match known_attempt { + Some(state) if !state.capture_settled || state.dispatch_started => { + Self::mark_known_attempt_store_failure(&mut record, state); + false + } + Some(_) => { + volatile_fallback = true; + Self::mark_outcome_store_failure( + &mut record, + "Rewind status could not be read", + ) + } + None if can_rule_out_prior_attempt => { + volatile_fallback = true; + Self::mark_outcome_store_failure( + &mut record, + "Rewind status could not be read", + ) + } + None => { + Self::mark_unverified_attempt_store_failure(&mut record); + false + } + }; + (record, allow_forward) } }; let mut record = record; record.outcome.request_id = request_id.clone(); + if volatile_fallback && !self.remember_volatile_outcome(&key, record.clone()) { + Self::mark_volatile_fence_capacity_failure(&mut record); + allow_forward = false; + tracing::error!( + event = "code_checkpoint.volatile_fence_capacity_exhausted", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + capacity = CHECKPOINT_VOLATILE_FENCE_LIMIT, + "checkpoint send was withheld because its process-local duplicate fence could not be retained" + ); + } self.broadcast_outcome(record.outcome.clone()); job.outcome.send_replace(Some(record)); if allow_forward { @@ -4546,86 +4909,1061 @@ mod tests { } #[tokio::test] - async fn codex_identity_receipt_is_durable_and_replayed_after_service_reload() { + async fn outcome_store_read_failure_without_a_trusted_baseline_withholds_send() { use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + tokio::fs::create_dir_all(&store_path).await.unwrap(); let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); - let service = - CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); - let workspace = ResolvedCheckpointWorkspace { - path: PathBuf::from("/work/codex-project"), - provenance: "directory-picker".to_string(), - }; - let send = FreshAgentSend { - provider: AgentProvider::Codex, - session_id: "codex-session".to_string(), - session_type: SessionType::Freshcodex, - text: "do not persist this prompt".to_string(), - cwd: Some("/work/codex-project".to_string()), + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-read-failure".into(), + session_type: SessionType::Freshopencode, + text: "send despite unavailable rewind status".into(), + cwd: None, images: None, - request_id: Some("codex-request-1".to_string()), + request_id: Some(request_id.into()), settings: None, observed_epoch: None, observed_generation: None, checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { - logical_attempt_id: "codex-logical-attempt-1".to_string(), + logical_attempt_id: "attempt-read-failure".into(), workspace: None, }), }; - let key = checkpoint_attempt_key(&send, "codex-logical-attempt-1"); - let mut record = service.new_record( - &send, - "codex-logical-attempt-1", - "codex-request-1", - Some((&workspace, "")), - Ok("checkpoint-sha".to_string()), - ); - record.dispatch_started = true; - record.submitted_turn_id = Some("codex-native-turn-1".to_string()); - service.persist_outcome(&key, record).await.unwrap(); + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; service - .observe_turn_identity(&json!({ - "provider": "codex", - "sessionId": "codex-session", - "sessionType": "freshcodex", - "event": { - "type": "checkpointTurnIdentity", - "requestId": "codex-request-1", - "logicalAttemptId": "codex-logical-attempt-1", - "sessionId": "codex-session", - "nativeTurnId": "codex-native-turn-1", - "displayTurnId": "codex-native-turn-1:row-0" - } - })) + .clone() + .barrier_and_forward( + make_send("request-read-failure-1"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + service + .clone() + .barrier_and_forward( + make_send("request-read-failure-2"), + make_forward(Arc::clone(&dispatches)), + ) .await; - let persisted = service.read_outcome_store().await.unwrap(); - let saved = persisted.outcomes.get(&key).unwrap(); - assert_eq!( - saved.outcome.native_turn_id.as_deref(), - Some("codex-native-turn-1") - ); assert_eq!( - saved.outcome.display_turn_id.as_deref(), - Some("codex-native-turn-1:row-0") - ); - assert!( - !serde_json::to_string(&persisted) - .unwrap() - .contains("do not persist this prompt"), - "durable checkpoint identity state never stores the sent prompt" + dispatches.load(Ordering::SeqCst), + 0, + "an unreadable store at startup cannot prove that this attempt was never dispatched" ); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["type"], "freshAgent.checkpoint.outcome"); + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "outcome_store_unavailable"); + assert!(outcome["reason"] + .as_str() + .unwrap() + .contains("prior send history could not be verified")); + assert!(outcome["reason"].as_str().unwrap().contains("withheld")); + } - let (replay_tx, mut replay_rx) = tokio::sync::broadcast::channel(8); - let reloaded = Arc::new(CheckpointService::new( + #[tokio::test] + async fn malformed_seen_store_disappearing_does_not_establish_empty_baseline() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + tokio::fs::create_dir_all(store_path.parent().unwrap()) + .await + .unwrap(); + tokio::fs::write(&store_path, b"not a checkpoint outcome store") + .await + .unwrap(); + + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( Arc::new(home.path().to_path_buf()), - Arc::new(replay_tx), + Arc::new(broadcast_tx), )); - reloaded - .clone() - .replay_for_session( + let key = "\"opencode\":\"freshopencode\":session-malformed-store:attempt-malformed-store"; + assert!(service.find_outcome(key).await.is_err()); + tokio::fs::remove_file(&store_path).await.unwrap(); + + let dispatches = Arc::new(AtomicUsize::new(0)); + let send = FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-malformed-store".into(), + session_type: SessionType::Freshopencode, + text: "do not assume a deleted malformed store was empty".into(), + cwd: None, + images: None, + request_id: Some("request-malformed-store".into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-malformed-store".into(), + workspace: None, + }), + }; + service + .clone() + .barrier_and_forward( + send, + Box::pin({ + let dispatches = Arc::clone(&dispatches); + async move { + dispatches.fetch_add(1, Ordering::SeqCst); + } + }), + ) + .await; + + assert_eq!(dispatches.load(Ordering::SeqCst), 0); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert!(outcome["reason"] + .as_str() + .unwrap() + .contains("prior send history could not be verified")); + } + + #[tokio::test] + async fn durable_dispatch_fence_blocks_retry_when_outcome_store_becomes_unreadable() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-read-after-dispatch".into(), + session_type: SessionType::Freshopencode, + text: "retry must not duplicate an already dispatched turn".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-read-after-dispatch".into(), + workspace: None, + }), + }; + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + + service + .clone() + .barrier_and_forward( + make_send("request-read-after-dispatch-1"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + + tokio::fs::remove_file(&store_path).await.unwrap(); + tokio::fs::create_dir(&store_path).await.unwrap(); + service + .clone() + .barrier_and_forward( + make_send("request-read-after-dispatch-2"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + + assert_eq!( + dispatches.load(Ordering::SeqCst), + 1, + "an unreadable durable history cannot turn a known dispatched attempt into a new send" + ); + } + + #[tokio::test] + async fn loaded_durable_dispatch_fence_blocks_retry_after_store_is_removed() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + let (seed_broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let seed_service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(seed_broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-loaded-dispatch".into(), + session_type: SessionType::Freshopencode, + text: "retry must not duplicate a durable dispatch from a prior service".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-loaded-dispatch".into(), + workspace: None, + }), + }; + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + + seed_service + .clone() + .barrier_and_forward( + make_send("request-loaded-dispatch-1"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let key = "\"opencode\":\"freshopencode\":session-loaded-dispatch:attempt-loaded-dispatch"; + let loaded = service.find_outcome(key).await.unwrap().unwrap(); + assert!(loaded.dispatch_started); + assert!(loaded.capture_settled); + + tokio::fs::remove_file(&store_path).await.unwrap(); + assert!(service.read_outcome_store().await.is_err()); + service + .clone() + .barrier_and_forward( + make_send("request-loaded-dispatch-2"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["requestId"], "request-loaded-dispatch-2"); + } + + #[tokio::test] + async fn durable_outcome_presence_marker_blocks_retry_after_restart_and_store_deletion() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + let (seed_broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let seed_service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(seed_broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-restart-store-deletion".into(), + session_type: SessionType::Freshopencode, + text: "do not duplicate after restart if the durable fence disappears".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-restart-store-deletion".into(), + workspace: None, + }), + }; + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + + seed_service + .clone() + .barrier_and_forward( + make_send("request-restart-store-deletion-1"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + tokio::fs::remove_file(&store_path).await.unwrap(); + + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + service + .clone() + .barrier_and_forward( + make_send("request-restart-store-deletion-2"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + + assert_eq!( + dispatches.load(Ordering::SeqCst), + 1, + "a restarted service must not treat a deleted durable history as first use" + ); + } + + #[tokio::test] + async fn cached_durable_outcome_replays_when_store_becomes_unreadable() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let send = FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-cache-replay".into(), + session_type: SessionType::Freshopencode, + text: "replay the saved outcome after a transient store error".into(), + cwd: None, + images: None, + request_id: Some("request-cache-replay".into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-cache-replay".into(), + workspace: None, + }), + }; + service + .clone() + .barrier_and_forward( + send, + Box::pin({ + let dispatches = Arc::clone(&dispatches); + async move { + dispatches.fetch_add(1, Ordering::SeqCst); + } + }), + ) + .await; + tokio::fs::remove_file(&store_path).await.unwrap(); + tokio::fs::create_dir(&store_path).await.unwrap(); + + service + .replay_for_identity( + AgentProvider::Opencode, + "session-cache-replay", + SessionType::Freshopencode, + ) + .await; + + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["requestId"], "request-cache-replay"); + assert_eq!(outcome["logicalAttemptId"], "attempt-cache-replay"); + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + } + + #[tokio::test] + async fn loaded_unsettled_attempt_stays_withheld_after_store_becomes_unreadable() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + let (seed_broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let seed_service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(seed_broadcast_tx), + )); + let seed_send = FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-loaded-unsettled".into(), + session_type: SessionType::Freshopencode, + text: "do not release this send until capture settlement is known".into(), + cwd: None, + images: None, + request_id: Some("request-loaded-unsettled-1".into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-loaded-unsettled".into(), + workspace: None, + }), + }; + let key = + "\"opencode\":\"freshopencode\":session-loaded-unsettled:attempt-loaded-unsettled"; + let mut unsettled = seed_service.new_record( + &seed_send, + "attempt-loaded-unsettled", + "request-loaded-unsettled-1", + None, + Err(( + "checkpoint_settlement_pending", + "snapshot still settling".into(), + )), + ); + unsettled.capture_settled = false; + unsettled.dispatch_started = false; + seed_service.persist_outcome(key, unsettled).await.unwrap(); + + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let loaded = service.find_outcome(key).await.unwrap().unwrap(); + assert!(!loaded.dispatch_started); + assert!(!loaded.capture_settled); + + tokio::fs::remove_file(&store_path).await.unwrap(); + tokio::fs::create_dir(&store_path).await.unwrap(); + assert!(service.read_outcome_store().await.is_err()); + let dispatches = Arc::new(AtomicUsize::new(0)); + let retry = FreshAgentSend { + request_id: Some("request-loaded-unsettled-2".into()), + ..seed_send + }; + service + .clone() + .barrier_and_forward( + retry, + Box::pin({ + let dispatches = Arc::clone(&dispatches); + async move { + dispatches.fetch_add(1, Ordering::SeqCst); + } + }), + ) + .await; + + assert_eq!(dispatches.load(Ordering::SeqCst), 0); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["requestId"], "request-loaded-unsettled-2"); + } + + #[cfg(unix)] + #[tokio::test] + async fn outcome_store_write_failure_forwards_once_and_reports_unknown_rewind() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::os::unix::fs::PermissionsExt; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_dir = home.path().join(".freshell/checkpoints"); + tokio::fs::create_dir_all(&store_dir).await.unwrap(); + let store_path = store_dir.join("attempt-outcomes-v1.json"); + let empty_store = CheckpointOutcomeStore { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcomes: BTreeMap::new(), + }; + tokio::fs::write(&store_path, serde_json::to_vec(&empty_store).unwrap()) + .await + .unwrap(); + let mut permissions = tokio::fs::metadata(&store_dir).await.unwrap().permissions(); + permissions.set_mode(0o555); + tokio::fs::set_permissions(&store_dir, permissions) + .await + .unwrap(); + + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-write-failure".into(), + session_type: SessionType::Freshopencode, + text: "send despite unavailable rewind status".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-write-failure".into(), + workspace: None, + }), + }; + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + + service + .clone() + .barrier_and_forward( + make_send("request-write-failure-1"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + service + .clone() + .barrier_and_forward( + make_send("request-write-failure-2"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + let mut permissions = tokio::fs::metadata(&store_dir).await.unwrap().permissions(); + permissions.set_mode(0o755); + tokio::fs::set_permissions(&store_dir, permissions) + .await + .unwrap(); + + assert_eq!( + dispatches.load(Ordering::SeqCst), + 1, + "a store write error must allow one send and fence its duplicate" + ); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["type"], "freshAgent.checkpoint.outcome"); + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "outcome_store_unavailable"); + assert!(outcome["reason"].as_str().unwrap().contains("rewind")); + assert!(outcome["reason"].as_str().unwrap().contains("sent")); + } + + #[tokio::test] + async fn volatile_fence_capacity_fails_closed_without_duplicate_dispatch() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_path = home + .path() + .join(".freshell/checkpoints/attempt-outcomes-v1.json"); + let store_dir = store_path.parent().unwrap(); + tokio::fs::create_dir_all(store_dir).await.unwrap(); + let empty_store = CheckpointOutcomeStore { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcomes: BTreeMap::new(), + }; + tokio::fs::write(&store_path, serde_json::to_vec(&empty_store).unwrap()) + .await + .unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx.clone()), + )); + assert!(service + .read_outcome_store() + .await + .unwrap() + .outcomes + .is_empty()); + tokio::fs::remove_file(&store_path).await.unwrap(); + tokio::fs::create_dir(&store_path).await.unwrap(); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |attempt: usize, request_id: String| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-fence-capacity".into(), + session_type: SessionType::Freshopencode, + text: "send only when duplicate protection is retained".into(), + cwd: None, + images: None, + request_id: Some(request_id), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: format!("attempt-fence-capacity-{attempt}"), + workspace: None, + }), + }; + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + + for attempt in 0..CHECKPOINT_VOLATILE_FENCE_LIMIT { + service + .clone() + .barrier_and_forward( + make_send(attempt, format!("request-fence-capacity-{attempt}")), + make_forward(Arc::clone(&dispatches)), + ) + .await; + } + assert_eq!( + dispatches.load(Ordering::SeqCst), + CHECKPOINT_VOLATILE_FENCE_LIMIT + ); + let current_len = service + .volatile_outcomes + .lock() + .expect("volatile checkpoint outcomes mutex poisoned") + .len(); + assert_eq!( + current_len, CHECKPOINT_VOLATILE_FENCE_LIMIT, + "process-local fences must never exceed their configured capacity" + ); + + let mut outcome_rx = broadcast_tx.subscribe(); + service + .clone() + .barrier_and_forward( + make_send( + CHECKPOINT_VOLATILE_FENCE_LIMIT, + "request-fence-capacity-overflow".into(), + ), + make_forward(Arc::clone(&dispatches)), + ) + .await; + service + .clone() + .barrier_and_forward( + make_send( + CHECKPOINT_VOLATILE_FENCE_LIMIT, + "request-fence-capacity-overflow-retry".into(), + ), + make_forward(Arc::clone(&dispatches)), + ) + .await; + + assert_eq!( + dispatches.load(Ordering::SeqCst), + CHECKPOINT_VOLATILE_FENCE_LIMIT, + "an attempt without retained local duplicate protection must stay withheld" + ); + for _ in 0..2 { + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "outcome_store_unavailable"); + assert!(outcome["reason"].as_str().unwrap().contains("withheld")); + } + } + + #[cfg(unix)] + #[tokio::test] + async fn durable_outcomes_do_not_consume_volatile_fence_capacity() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::os::unix::fs::PermissionsExt; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let store_dir = home.path().join(".freshell/checkpoints"); + tokio::fs::create_dir_all(&store_dir).await.unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let mut outcomes = BTreeMap::new(); + for index in 0..CHECKPOINT_VOLATILE_FENCE_LIMIT { + let request_id = format!("durable-request-{index}"); + let send = FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: format!("durable-session-{index}"), + session_type: SessionType::Freshopencode, + text: "not stored".into(), + cwd: None, + images: None, + request_id: Some(request_id.clone()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: None, + }; + let mut record = service.new_record( + &send, + &format!("durable-attempt-{index}"), + &request_id, + None, + Err(("workspace_unavailable", "No workspace".into())), + ); + record.dispatch_started = true; + outcomes.insert(format!("durable-key-{index}"), record); + } + service + .write_outcome_store(&CheckpointOutcomeStore { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcomes, + }) + .await + .unwrap(); + service + .observe_accepted(&json!({ + "requestId": "durable-request-0", + "provider": "opencode", + "sessionId": "durable-session-0", + "sessionType": "freshopencode", + "submittedTurnId": "durable-turn-0" + })) + .await; + + let durable_count = service.read_outcome_store().await.unwrap().outcomes.len(); + assert_eq!(durable_count, CHECKPOINT_VOLATILE_FENCE_LIMIT); + let cached_durable_count = service + .volatile_outcomes + .lock() + .expect("volatile checkpoint outcomes mutex poisoned") + .len(); + assert_eq!( + cached_durable_count, 0, + "durable outcomes remain on disk and must not consume volatile fence slots" + ); + + let mut permissions = tokio::fs::metadata(&store_dir).await.unwrap().permissions(); + permissions.set_mode(0o555); + tokio::fs::set_permissions(&store_dir, permissions) + .await + .unwrap(); + let dispatches = Arc::new(AtomicUsize::new(0)); + let send = FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-needs-volatile-fence".into(), + session_type: SessionType::Freshopencode, + text: "send with a new volatile fence slot".into(), + cwd: None, + images: None, + request_id: Some("request-needs-volatile-fence".into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-needs-volatile-fence".into(), + workspace: None, + }), + }; + service + .clone() + .barrier_and_forward( + send, + Box::pin({ + let dispatches = Arc::clone(&dispatches); + async move { + dispatches.fetch_add(1, Ordering::SeqCst); + } + }), + ) + .await; + let mut permissions = tokio::fs::metadata(&store_dir).await.unwrap().permissions(); + permissions.set_mode(0o755); + tokio::fs::set_permissions(&store_dir, permissions) + .await + .unwrap(); + + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + let volatile_count = service + .volatile_outcomes + .lock() + .expect("volatile checkpoint outcomes mutex poisoned") + .len(); + assert_eq!( + volatile_count, 1, + "only the failed durable write should consume a volatile fence slot" + ); + } + + #[tokio::test] + async fn settled_workspace_failure_forwards_once_and_reports_unavailable_rewind() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-workspace-failure".into(), + session_type: SessionType::Freshopencode, + text: "send despite missing workspace".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-workspace-failure".into(), + workspace: None, + }), + }; + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + + service + .clone() + .barrier_and_forward( + make_send("request-workspace-failure-1"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + service + .clone() + .barrier_and_forward( + make_send("request-workspace-failure-2"), + make_forward(Arc::clone(&dispatches)), + ) + .await; + + assert_eq!( + dispatches.load(Ordering::SeqCst), + 1, + "a settled workspace failure must not suppress or duplicate the provider send" + ); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["type"], "freshAgent.checkpoint.outcome"); + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "workspace_unavailable"); + assert!(outcome["reason"] + .as_str() + .unwrap() + .contains("message will still be sent")); + } + + #[tokio::test] + async fn pending_send_forwards_once_after_quarantine_confirms_settlement() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + run_git(&["init", "-q"], Some(workspace.path())) + .await + .unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = broadcast_tx.subscribe(); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let gate = Arc::new(TestSettlementGate { + entered: tokio::sync::Notify::new(), + release: tokio::sync::Notify::new(), + }); + let control = Arc::new(CheckpointCaptureControl::default()); + *control + .test_settlement_gate + .lock() + .expect("checkpoint test settlement gate poisoned") = Some(Arc::clone(&gate)); + *service + .test_capture_attempt_override + .lock() + .expect("checkpoint test capture override mutex poisoned") = + Some(CaptureAttempt::Unsettled { + message: "simulated uncertain child settlement".into(), + control, + }); + + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-quarantine-settlement".into(), + session_type: SessionType::Freshopencode, + text: "continue after snapshot child settles".into(), + cwd: Some(workspace.path().to_string_lossy().into_owned()), + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "attempt-quarantine-settlement".into(), + workspace: None, + }), + }; + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_forward = |dispatches: Arc<AtomicUsize>| { + Box::pin(async move { + dispatches.fetch_add(1, Ordering::SeqCst); + }) as futures_util::future::BoxFuture<'static, ()> + }; + let first = tokio::spawn(service.clone().barrier_and_forward( + make_send("request-quarantine-settlement-1"), + make_forward(Arc::clone(&dispatches)), + )); + + tokio::time::timeout(Duration::from_secs(1), gate.entered.notified()) + .await + .expect("automatic capture entered the existing quarantine settlement gate"); + assert_eq!(dispatches.load(Ordering::SeqCst), 0); + assert!( + service + .lock_workspace(workspace.path(), Duration::from_millis(30)) + .await + .is_err(), + "the provider send must stay behind the quarantined workspace lock" + ); + + let duplicate = tokio::spawn(service.clone().barrier_and_forward( + make_send("request-quarantine-settlement-2"), + make_forward(Arc::clone(&dispatches)), + )); + tokio::task::yield_now().await; + assert_eq!(dispatches.load(Ordering::SeqCst), 0); + gate.release.notify_one(); + + tokio::time::timeout(Duration::from_secs(2), first) + .await + .expect("settled pending send completed") + .expect("pending send task completed"); + tokio::time::timeout(Duration::from_secs(2), duplicate) + .await + .expect("duplicate joined the pending send") + .expect("duplicate task completed"); + + assert_eq!( + dispatches.load(Ordering::SeqCst), + 1, + "a settled quarantined attempt forwards once despite a duplicate request" + ); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["type"], "freshAgent.checkpoint.outcome"); + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "capture_failed"); + assert!(outcome["reason"] + .as_str() + .unwrap() + .contains("message will still be sent")); + let key = format!( + "\"opencode\":\"freshopencode\":session-quarantine-settlement:attempt-quarantine-settlement" + ); + let saved = service.find_outcome(&key).await.unwrap().unwrap(); + assert!(saved.capture_settled); + assert!(saved.dispatch_started); + } + + #[tokio::test] + async fn codex_identity_receipt_is_durable_and_replayed_after_service_reload() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + let workspace = ResolvedCheckpointWorkspace { + path: PathBuf::from("/work/codex-project"), + provenance: "directory-picker".to_string(), + }; + let send = FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "codex-session".to_string(), + session_type: SessionType::Freshcodex, + text: "do not persist this prompt".to_string(), + cwd: Some("/work/codex-project".to_string()), + images: None, + request_id: Some("codex-request-1".to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "codex-logical-attempt-1".to_string(), + workspace: None, + }), + }; + let key = checkpoint_attempt_key(&send, "codex-logical-attempt-1"); + let mut record = service.new_record( + &send, + "codex-logical-attempt-1", + "codex-request-1", + Some((&workspace, "")), + Ok("checkpoint-sha".to_string()), + ); + record.dispatch_started = true; + record.submitted_turn_id = Some("codex-native-turn-1".to_string()); + service.persist_outcome(&key, record).await.unwrap(); + + service + .observe_turn_identity(&json!({ + "provider": "codex", + "sessionId": "codex-session", + "sessionType": "freshcodex", + "event": { + "type": "checkpointTurnIdentity", + "requestId": "codex-request-1", + "logicalAttemptId": "codex-logical-attempt-1", + "sessionId": "codex-session", + "nativeTurnId": "codex-native-turn-1", + "displayTurnId": "codex-native-turn-1:row-0" + } + })) + .await; + + let persisted = service.read_outcome_store().await.unwrap(); + let saved = persisted.outcomes.get(&key).unwrap(); + assert_eq!( + saved.outcome.native_turn_id.as_deref(), + Some("codex-native-turn-1") + ); + assert_eq!( + saved.outcome.display_turn_id.as_deref(), + Some("codex-native-turn-1:row-0") + ); + assert!( + !serde_json::to_string(&persisted) + .unwrap() + .contains("do not persist this prompt"), + "durable checkpoint identity state never stores the sent prompt" + ); + + let (replay_tx, mut replay_rx) = tokio::sync::broadcast::channel(8); + let reloaded = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(replay_tx), + )); + reloaded + .clone() + .replay_for_session( AgentProvider::Codex, "codex-session".to_string(), SessionType::Freshcodex, @@ -4639,6 +5977,166 @@ mod tests { assert_eq!(replayed["displayTurnId"], "codex-native-turn-1:row-0"); } + #[tokio::test] + async fn codex_checkpoint_identity_must_match_accepted_provider_turn_id_in_either_event_order() + { + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + let workspace = ResolvedCheckpointWorkspace { + path: PathBuf::from("/work/codex-project"), + provenance: "directory-picker".to_string(), + }; + let send = |request_id: &str, attempt_id: &str| FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "codex-session".to_string(), + session_type: SessionType::Freshcodex, + text: "identity mismatch must not enable rewind".to_string(), + cwd: Some("/work/codex-project".to_string()), + images: None, + request_id: Some(request_id.to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: attempt_id.to_string(), + workspace: None, + }), + }; + let identity_event = |request_id: &str, attempt_id: &str, native_turn_id: &str| { + json!({ + "provider": "codex", + "sessionId": "codex-session", + "sessionType": "freshcodex", + "event": { + "type": "checkpointTurnIdentity", + "requestId": request_id, + "logicalAttemptId": attempt_id, + "sessionId": "codex-session", + "nativeTurnId": native_turn_id, + "displayTurnId": format!("{native_turn_id}:row-0") + } + }) + }; + + let first_send = send("request-identity-first", "attempt-identity-first"); + let first_key = checkpoint_attempt_key(&first_send, "attempt-identity-first"); + let mut first = service.new_record( + &first_send, + "attempt-identity-first", + "request-identity-first", + Some((&workspace, "")), + Ok("checkpoint-first".to_string()), + ); + first.dispatch_started = true; + first.submitted_turn_id = Some("accepted-turn-first".to_string()); + service.persist_outcome(&first_key, first).await.unwrap(); + + service + .observe_turn_identity(&identity_event( + "request-identity-first", + "attempt-identity-first", + "different-turn-first", + )) + .await; + let persisted = service.read_outcome_store().await.unwrap(); + let first_saved = persisted.outcomes.get(&first_key).unwrap(); + assert_eq!(first_saved.outcome.native_turn_id, None); + assert_eq!(first_saved.outcome.display_turn_id, None); + + service + .observe_turn_identity(&identity_event( + "request-identity-first", + "attempt-identity-first", + "accepted-turn-first", + )) + .await; + let persisted = service.read_outcome_store().await.unwrap(); + assert_eq!( + persisted + .outcomes + .get(&first_key) + .unwrap() + .outcome + .native_turn_id + .as_deref(), + Some("accepted-turn-first") + ); + + let second_send = send("request-identity-second", "attempt-identity-second"); + let second_key = checkpoint_attempt_key(&second_send, "attempt-identity-second"); + let mut second = service.new_record( + &second_send, + "attempt-identity-second", + "request-identity-second", + Some((&workspace, "")), + Ok("checkpoint-second".to_string()), + ); + second.dispatch_started = true; + service.persist_outcome(&second_key, second).await.unwrap(); + service + .observe_turn_identity(&identity_event( + "request-identity-second", + "attempt-identity-second", + "identity-before-acceptance", + )) + .await; + service + .observe_accepted(&json!({ + "requestId": "request-identity-second", + "provider": "codex", + "sessionId": "codex-session", + "sessionType": "freshcodex", + "submittedTurnId": "different-accepted-turn" + })) + .await; + + let persisted = service.read_outcome_store().await.unwrap(); + let second_saved = persisted.outcomes.get(&second_key).unwrap(); + assert_eq!( + second_saved.outcome.native_turn_id.as_deref(), + Some("identity-before-acceptance") + ); + assert_eq!( + second_saved.submitted_turn_id, None, + "a conflicting later acceptance cannot be joined to this checkpoint" + ); + + let third_send = send("request-identity-third", "attempt-identity-third"); + let third_key = checkpoint_attempt_key(&third_send, "attempt-identity-third"); + let mut third = service.new_record( + &third_send, + "attempt-identity-third", + "request-identity-third", + Some((&workspace, "")), + Ok("checkpoint-third".to_string()), + ); + third.dispatch_started = true; + third.submitted_turn_id = Some("accepted-turn-third".to_string()); + service.persist_outcome(&third_key, third).await.unwrap(); + service + .observe_accepted(&json!({ + "requestId": "request-identity-third", + "provider": "codex", + "sessionId": "codex-session", + "sessionType": "freshcodex" + })) + .await; + + let persisted = service.read_outcome_store().await.unwrap(); + assert_eq!( + persisted + .outcomes + .get(&third_key) + .unwrap() + .submitted_turn_id + .as_deref(), + Some("accepted-turn-third"), + "an accepted event without an optional native ID must preserve exact identity already recorded" + ); + } + async fn body_json(resp: Response) -> Value { let body = axum::body::to_bytes(resp.into_body(), usize::MAX) .await diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 0d1063c0d..495de83ef 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -138,6 +138,10 @@ impl HostedFreshAgentProxy { let provider = message.provider; let session_id = message.session_id.clone(); let session_type = message.session_type; + let checkpoint_logical_attempt_id = message + .checkpoint_intent + .as_ref() + .map(|intent| intent.logical_attempt_id.clone()); if let Err(reason) = self.post_barrier_admission(&message) { self.error_event_code( &provider, @@ -160,7 +164,13 @@ impl HostedFreshAgentProxy { Some(soul) => { let settings = turn_settings(message.settings, message.cwd); self.client - .fresh_agent_send(request_id, soul, message.text, settings) + .fresh_agent_send( + request_id, + soul, + message.text, + settings, + checkpoint_logical_attempt_id, + ) .await .map(|_| ()) } diff --git a/crates/freshell-server/src/fresh_agent_proxy_rest.rs b/crates/freshell-server/src/fresh_agent_proxy_rest.rs index 1495dcc3f..25d6334dd 100644 --- a/crates/freshell-server/src/fresh_agent_proxy_rest.rs +++ b/crates/freshell-server/src/fresh_agent_proxy_rest.rs @@ -176,7 +176,7 @@ impl HostedFreshAgentRestGateway for HostedFreshAgentProxy { .map_err(|_| ())?; let request_id = RequestId::parse(request.request_id).map_err(|_| ())?; self.client - .fresh_agent_send(request_id, soul.clone(), request.text, None) + .fresh_agent_send(request_id, soul.clone(), request.text, None, None) .await .map_err(|_| ())?; let completed = wait_for_completion( diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index c92bd5c8b..44b00bdc7 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -563,6 +563,7 @@ async fn dispatch( request_id, text, settings, + checkpoint_logical_attempt_id, } => { ensure_incarnation(&incarnation_id, state)?; let actor = state @@ -572,7 +573,12 @@ async fn dispatch( .clone() .ok_or_else(unsupported_fresh_agent)?; let command_state = actor - .dispatch(request_id, text, settings) + .dispatch_with_checkpoint_identity( + request_id, + text, + settings, + checkpoint_logical_attempt_id, + ) .await .map_err(map_actor_error)?; let native_session_id = actor.profile().await.native_session_id; @@ -5752,6 +5758,7 @@ mod tests { struct RpcFixtureTransport { dispatches: AtomicUsize, + checkpoint_attempts: std::sync::Mutex<Vec<Option<String>>>, decisions: AtomicUsize, operations: AtomicUsize, captures: AtomicUsize, @@ -5778,6 +5785,23 @@ mod tests { }) } + async fn dispatch_with_checkpoint_identity( + &self, + request_id: &RequestId, + _text: &str, + _profile: &FreshAgentProfile, + logical_attempt_id: Option<&str>, + ) -> Result<DispatchAck, DispatchFailure> { + self.dispatches.fetch_add(1, Ordering::SeqCst); + self.checkpoint_attempts + .lock() + .unwrap() + .push(logical_attempt_id.map(str::to_owned)); + Ok(DispatchAck { + provider_ack_id: Some(request_id.to_string()), + }) + } + async fn resolve_permission( &self, _decision_id: &str, @@ -5874,6 +5898,7 @@ mod tests { let state = test_host_state(); let transport = Arc::new(RpcFixtureTransport { dispatches: AtomicUsize::new(0), + checkpoint_attempts: std::sync::Mutex::new(Vec::new()), decisions: AtomicUsize::new(0), operations: AtomicUsize::new(0), captures: AtomicUsize::new(0), @@ -5916,6 +5941,7 @@ mod tests { request_id: request_id.clone(), text: "perform fixture work".into(), settings: None, + checkpoint_logical_attempt_id: Some("managed-attempt-rpc-1".into()), }, ), &state, @@ -5930,6 +5956,10 @@ mod tests { } if native == "fixture-native-thread" )); + assert_eq!( + *transport.checkpoint_attempts.lock().unwrap(), + vec![Some("managed-attempt-rpc-1".into())] + ); // A gateway reconnect is only a new cursor. The same request and the // same decision are idempotent at the host-owned actor. diff --git a/crates/freshell-session-host/src/providers/fresh_agent.rs b/crates/freshell-session-host/src/providers/fresh_agent.rs index 391712f70..4230df498 100644 --- a/crates/freshell-session-host/src/providers/fresh_agent.rs +++ b/crates/freshell-session-host/src/providers/fresh_agent.rs @@ -14,9 +14,10 @@ use freshell_codex::launch_plan::CodexSidecarLaunchContext; use freshell_freshagent::rollback_record::{RollbackDirection, RollbackModeReq, RollbackRequest}; use freshell_freshagent::{FreshAgentState, FreshClaudeState, FreshCodexState, FreshOpencodeState}; use freshell_protocol::{ - AgentProvider, FreshAgentApprovalRespond, FreshAgentCompact, FreshAgentCreate, FreshAgentFork, - FreshAgentInterrupt, FreshAgentKill, FreshAgentQuestionRespond, FreshAgentSend, - FreshAgentSendSettings, ServerMessage, SessionLocator, SessionType, StringOrNumber, + AgentProvider, FreshAgentApprovalRespond, FreshAgentCheckpointIntent, FreshAgentCompact, + FreshAgentCreate, FreshAgentFork, FreshAgentInterrupt, FreshAgentKill, + FreshAgentQuestionRespond, FreshAgentSend, FreshAgentSendSettings, ServerMessage, + SessionLocator, SessionType, StringOrNumber, }; use freshell_runtime_protocol::{ AgentEvent, FreshAgentCapture, FreshAgentFixtureTransport, FreshAgentLaunchSpec, @@ -461,6 +462,17 @@ impl FreshAgentTransport for HostedTransport { request_id: &RequestId, text: &str, profile: &FreshAgentProfile, + ) -> Result<DispatchAck, DispatchFailure> { + self.dispatch_with_checkpoint_identity(request_id, text, profile, None) + .await + } + + async fn dispatch_with_checkpoint_identity( + &self, + request_id: &RequestId, + text: &str, + profile: &FreshAgentProfile, + logical_attempt_id: Option<&str>, ) -> Result<DispatchAck, DispatchFailure> { #[cfg(test)] if let Some(delegate) = self.test_delegate.as_ref() { @@ -495,7 +507,12 @@ impl FreshAgentTransport for HostedTransport { permission_mode: profile.permission_mode, sandbox: profile.sandbox.as_deref().and_then(parse_sandbox), }), - checkpoint_intent: None, + checkpoint_intent: logical_attempt_id.map(|logical_attempt_id| { + FreshAgentCheckpointIntent { + logical_attempt_id: logical_attempt_id.to_string(), + workspace: None, + } + }), }; match &self.state { ProviderState::Claude(state) => state.handle_send(send).await, @@ -1514,12 +1531,30 @@ lines.on('line', line=>{ let pre = actor.snapshot().await; let spawned_before_send = audit.exists(); let first = actor - .dispatch( + .dispatch_with_checkpoint_identity( RequestId::parse("first-owned-send").unwrap(), "First owned prompt".into(), None, + Some("managed-opencode-attempt-1".into()), ) .await; + let checkpoint_identity = tokio::time::timeout(Duration::from_secs(10), async { + loop { + let events = actor.read_events(0, 256).await.events; + if let Some(payload) = events.iter().find_map(|entry| match &entry.event { + AgentEvent::Provider { payload } + if payload["event"]["type"] == "checkpointTurnIdentity" => + { + Some(payload.clone()) + } + _ => None, + }) { + break payload; + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await; let identity = tokio::time::timeout(Duration::from_secs(10), async { loop { if let Some(native) = actor.profile().await.native_session_id { @@ -1556,6 +1591,14 @@ lines.on('line', line=>{ .await; let final_native = actor.profile().await.native_session_id; transport.stop().await.unwrap(); + let checkpoint_identity = match checkpoint_identity { + Ok(payload) => payload, + Err(_) => panic!( + "hosted OpenCode send must preserve checkpoint identity through the provider; actor events: {:#?}; fixture audit: {}", + actor.read_events(0, 256).await, + std::fs::read_to_string(&audit).unwrap_or_default() + ), + }; let rows: Vec<Value> = std::fs::read_to_string(&audit) .unwrap_or_default() .lines() @@ -1568,7 +1611,35 @@ lines.on('line', line=>{ assert_eq!(pre.unwrap()["threadId"], registered); first.unwrap(); second.unwrap(); + assert_eq!( + checkpoint_identity["event"]["logicalAttemptId"], + "managed-opencode-attempt-1" + ); + assert_eq!( + checkpoint_identity["event"]["requestId"], + "first-owned-send" + ); + assert!(checkpoint_identity["event"]["nativeTurnId"] + .as_str() + .is_some_and(|id| id.starts_with("msg_"))); observed_prompts.unwrap(); + let identity_receipt_count = actor + .read_events(0, 256) + .await + .events + .iter() + .filter(|entry| { + matches!( + &entry.event, + AgentEvent::Provider { payload } + if payload["event"]["type"] == "checkpointTurnIdentity" + ) + }) + .count(); + assert_eq!( + identity_receipt_count, 1, + "the second legacy send must not create a checkpoint identity receipt" + ); let native = identity.unwrap(); assert_eq!(live.unwrap()["threadId"], native); assert_eq!(final_native.as_deref(), Some(native.as_str())); diff --git a/crates/freshell-supervisor/src/claude_source_observation.rs b/crates/freshell-supervisor/src/claude_source_observation.rs index 0bdacfa4c..b431bc06b 100644 --- a/crates/freshell-supervisor/src/claude_source_observation.rs +++ b/crates/freshell-supervisor/src/claude_source_observation.rs @@ -11,12 +11,16 @@ impl Supervisor { ) -> Result<freshell_platform::claude_inputs::observation::ObservationReply, RuntimeError> { request.validate()?; let denied = || { - RuntimeError::new(RuntimeErrorCode::InvalidRequest, - "Claude source observation requires its current creation or recovery attempt; retain inputs and reprepare") + RuntimeError::new( + RuntimeErrorCode::InvalidRequest, + "Claude source observation requires its current creation or recovery attempt; retain inputs and reprepare", + ) }; let unavailable = || { - RuntimeError::new(RuntimeErrorCode::HostUnreachable, - "Claude native source evidence is unavailable; retain inputs and repair the source before dispatch") + RuntimeError::new( + RuntimeErrorCode::HostUnreachable, + "Claude native source evidence is unavailable; retain inputs and repair the source before dispatch", + ) }; let mut recovery = None; let owner = match &request.authority { @@ -463,10 +467,14 @@ mod tests { .await .expect("nested Claude source lookup must not reacquire the held soul lifecycle lock") .result; - assert_eq!(backend.2.load(std::sync::atomic::Ordering::SeqCst),usize::from(mode!="candidate-unrecorded-prior"),"unrecorded prior inputs must block before the backend; valid provenance reaches it once"); + assert_eq!( + backend.2.load(std::sync::atomic::Ordering::SeqCst), + usize::from(mode != "candidate-unrecorded-prior"), + "unrecorded prior inputs must block before the backend; valid provenance reaches it once" + ); assert_eq!( result.is_ok(), - matches!(mode,"live"|"candidate"), + matches!(mode, "live" | "candidate"), "{mode}: changed scoped authority after IPC cannot qualify source evidence: {result:?}" ); if let Err(error) = result { diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index 89639fe88..3d622bc9b 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -67,8 +67,11 @@ async fn provider_claude_creation_source_operation_requires_reader_evidence_afte let mut lookup = Envelope::new(RequestId::new(), ControlRole::Web, command); lookup.auth = Some(secret.into()); let result = supervisor.dispatch(lookup).await.result; - assert_eq!(result.unwrap_err().code, RuntimeErrorCode::HostUnreachable, - "proved initial volume absence must not substitute for unavailable source-selection evidence"); + assert_eq!( + result.unwrap_err().code, + RuntimeErrorCode::HostUnreachable, + "proved initial volume absence must not substitute for unavailable source-selection evidence" + ); assert_eq!( backend.0.load(Ordering::SeqCst), 3, diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs index ba33c954a..04e0ede17 100644 --- a/crates/freshell-supervisor/src/provider_preparation_tests.rs +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -21,6 +21,7 @@ struct Daemon { reject_inputs: bool, consumed_rejection: bool, recovery_probes: Vec<String>, + recovery_probe_available: bool, } struct Backend(Arc<Mutex<Daemon>>, std::path::PathBuf); @@ -213,15 +214,29 @@ impl RuntimeBackend for Backend { resume_spec: Some(spec), .. } => { + let mut state = state.lock().unwrap(); state - .lock() - .unwrap() .recovery_probes .push(spec.provider_session.native_session_id.clone()); - HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { - evidence_revision: spec.evidence_revision, - resume_spec: spec, - }) + if state.recovery_probe_available { + HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { + evidence_revision: spec.evidence_revision, + resume_spec: spec, + }) + } else { + HostResult::RecoveryProbe(RecoveryProbe::Blocked { + path: RecoveryPath::NativeResume, + reason: RecoveryBlockReason::StoreUnreadable, + retry_hint: RetryHint { + automatic_after_ms: None, + manual_retry: true, + repair: None, + }, + evidence: vec![ + "synthetic provider store is temporarily unavailable".into(), + ], + }) + } } command => panic!("unexpected synthetic host command {command:?}"), }; @@ -381,7 +396,7 @@ async fn control_listener(supervisor: &Supervisor) -> tokio::task::JoinHandle<() } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] -async fn live_opencode_probe_persists_resume_evidence_before_enclosure_exit() { +async fn live_opencode_probe_coalesces_failed_evidence_reads_then_recovers_before_exit() { let temp = tempfile::tempdir().unwrap(); let registry_root = temp.path().join("registry"); let registry = Registry::open(®istry_root, None).unwrap(); @@ -402,6 +417,7 @@ async fn live_opencode_probe_persists_resume_evidence_before_enclosure_exit() { lifecycle_log: temp.path().join("lifecycle.jsonl"), admission: crate::admission::AdmissionPolicy::default(), }; + let lifecycle_log = config.lifecycle_log.clone(); let backend = Arc::new(Backend(daemon.clone(), registry_root)); let supervisor = Supervisor::new(registry.clone(), backend.clone(), config).unwrap(); let terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ @@ -475,6 +491,46 @@ async fn live_opencode_probe_persists_resume_evidence_before_enclosure_exit() { let live = supervisor.probe_recovery(soul.clone()).await.unwrap(); assert!(matches!(live, RecoveryProbe::ReattachReady { .. })); + let after_first_probe = registry.recovery_context(soul.clone()).await.unwrap(); + assert_eq!( + after_first_probe + .resume_spec + .as_ref() + .map(|spec| spec.provider_session.native_session_id.as_str()), + None, + "an unverified provider-store read must not become durable resume evidence" + ); + assert_eq!(daemon.lock().unwrap().recovery_probes, ["ses_exact"]); + let count_live_resume_events = || { + std::fs::read_to_string(&lifecycle_log) + .unwrap_or_default() + .lines() + .filter_map(|line| serde_json::from_str::<serde_json::Value>(line).ok()) + .filter(|event| event["event"] == "supervisor.recovery.live_resume_evidence") + .count() + }; + assert_eq!(count_live_resume_events(), 1); + + let repeated_live = supervisor.probe_recovery(soul.clone()).await.unwrap(); + assert!(matches!(repeated_live, RecoveryProbe::ReattachReady { .. })); + assert_eq!( + daemon.lock().unwrap().recovery_probes, + ["ses_exact"], + "unchanged live identity must not repeat a failing provider-store read on every poll" + ); + assert_eq!( + count_live_resume_events(), + 1, + "unchanged failures should not append duplicate lifecycle events" + ); + + tokio::time::sleep( + crate::recovery::LIVE_RESUME_EVIDENCE_RETRY_INTERVAL + Duration::from_millis(25), + ) + .await; + daemon.lock().unwrap().recovery_probe_available = true; + let retried_live = supervisor.probe_recovery(soul.clone()).await.unwrap(); + assert!(matches!(retried_live, RecoveryProbe::ReattachReady { .. })); let durable = registry.recovery_context(soul.clone()).await.unwrap(); assert_eq!( durable @@ -482,9 +538,13 @@ async fn live_opencode_probe_persists_resume_evidence_before_enclosure_exit() { .as_ref() .map(|spec| spec.provider_session.native_session_id.as_str()), Some("ses_exact"), - "a healthy live probe must capture exact provider-store evidence before the enclosure can disappear" + "a later verified exact-session read must become durable resume evidence" ); - assert_eq!(daemon.lock().unwrap().recovery_probes, ["ses_exact"]); + assert_eq!( + daemon.lock().unwrap().recovery_probes, + ["ses_exact", "ses_exact"] + ); + assert_eq!(count_live_resume_events(), 2); daemon .lock() diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index 23994689b..f2b2de932 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -28,6 +28,12 @@ use freshell_runtime_protocol::{ use sha2::{Digest, Sha256}; use tokio::time::{sleep, Duration, Instant}; +#[cfg(test)] +pub(crate) const LIVE_RESUME_EVIDENCE_RETRY_INTERVAL: Duration = Duration::from_secs(2); +#[cfg(not(test))] +pub(crate) const LIVE_RESUME_EVIDENCE_RETRY_INTERVAL: Duration = Duration::from_secs(30); +const LIVE_RESUME_EVIDENCE_ATTEMPT_CACHE_MAX_ENTRIES: usize = 1_024; + impl Supervisor { pub(crate) async fn probe_recovery( &self, @@ -193,6 +199,10 @@ impl Supervisor { host: &crate::service::AuthenticatedHost, status: &crate::service::HostStatus, ) { + if !self.live_resume_evidence_capture_due(context, status).await { + return; + } + let capture = match resume_catalog::exact_resume_candidate( context, status.native_session_id.as_deref(), @@ -244,6 +254,41 @@ impl Supervisor { ); } + async fn live_resume_evidence_capture_due( + &self, + context: &RecoveryContext, + status: &crate::service::HostStatus, + ) -> bool { + let key = ( + context.soul_id.as_str().to_string(), + context.prior_handle.incarnation_id().as_str().to_string(), + status + .native_session_id + .clone() + .or_else(|| context.native_session_id.clone()), + ); + let now = Instant::now(); + let mut attempts = self.live_resume_evidence_attempts.lock().await; + attempts.retain(|_, attempted_at| { + now.duration_since(*attempted_at) < LIVE_RESUME_EVIDENCE_RETRY_INTERVAL + }); + if attempts.contains_key(&key) { + return false; + } + + if attempts.len() >= LIVE_RESUME_EVIDENCE_ATTEMPT_CACHE_MAX_ENTRIES { + if let Some(oldest_key) = attempts + .iter() + .min_by_key(|(_, attempted_at)| **attempted_at) + .map(|(key, _)| key.clone()) + { + attempts.remove(&oldest_key); + } + } + attempts.insert(key, now); + true + } + async fn probe_host_store( &self, context: &RecoveryContext, @@ -1724,6 +1769,7 @@ impl Supervisor { request_id: request_id.clone(), text: command.data, settings: Some(fresh_agent_replay_settings(agent)), + checkpoint_logical_attempt_id: None, } } else { HostCommand::TerminalInput { diff --git a/crates/freshell-supervisor/src/service.rs b/crates/freshell-supervisor/src/service.rs index 0ca498701..39efaf62c 100644 --- a/crates/freshell-supervisor/src/service.rs +++ b/crates/freshell-supervisor/src/service.rs @@ -102,6 +102,7 @@ type ProviderPathInputBindings = HashMap< Option<Instant>, ), >; +pub(crate) type LiveResumeEvidenceKey = (String, String, Option<String>); #[derive(Clone)] pub struct Supervisor { @@ -112,6 +113,8 @@ pub struct Supervisor { /// remain independently recoverable. Stop intentionally does not take this /// lock: its durable intent revision must be able to preempt recovery. pub(crate) recovery_locks: Arc<Mutex<HashMap<String, Arc<Mutex<()>>>>>, + /// Bounds repeated provider-store probes for an unchanged live identity. + pub(crate) live_resume_evidence_attempts: Arc<Mutex<HashMap<LiveResumeEvidenceKey, Instant>>>, auth_lookup_authorities: Arc<StdMutex<HashMap<String, AuthLookupAuthority>>>, provider_path_inputs: Arc<StdMutex<ProviderPathInputBindings>>, } @@ -160,6 +163,7 @@ impl Supervisor { backend, config: Arc::new(config), recovery_locks: Arc::new(Mutex::new(HashMap::new())), + live_resume_evidence_attempts: Arc::new(Mutex::new(HashMap::new())), auth_lookup_authorities: Arc::new(StdMutex::new(HashMap::new())), provider_path_inputs: Arc::new(StdMutex::new(HashMap::new())), }) @@ -505,6 +509,7 @@ impl Supervisor { request.soul_id, request.text, request.settings, + request.checkpoint_logical_attempt_id, ) .await?; Ok(AdminResult::FreshAgentCommand { state }) @@ -1451,6 +1456,7 @@ impl Supervisor { soul_id: SoulId, text: String, settings: Option<freshell_runtime_protocol::FreshAgentTurnSettings>, + checkpoint_logical_attempt_id: Option<String>, ) -> Result<CommandState, RuntimeError> { if text.len() > freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES / 2 { return Err(RuntimeError::new( @@ -1471,7 +1477,13 @@ impl Supervisor { "soul is not a hosted fresh-agent", )); } - let settings_correlation = serde_json::to_vec(&settings).map_err(|_| { + let settings_correlation = match checkpoint_logical_attempt_id.as_deref() { + Some(logical_attempt_id) => { + serde_json::to_vec(&(settings.as_ref(), logical_attempt_id)) + } + None => serde_json::to_vec(&settings), + } + .map_err(|_| { RuntimeError::new( RuntimeErrorCode::InvalidRequest, "fresh-agent settings could not be correlated", @@ -1530,6 +1542,7 @@ impl Supervisor { request_id: request_id.clone(), text, settings, + checkpoint_logical_attempt_id, }, ) .await; diff --git a/docs/development/managed-runtime.md b/docs/development/managed-runtime.md index fc8db8f54..d3f072a9c 100644 --- a/docs/development/managed-runtime.md +++ b/docs/development/managed-runtime.md @@ -345,6 +345,41 @@ durable session-host app-server. The provider still runs in a soul-owned home; OpenCode runs one pinned runtime per soul on private loopback, with native identity learned inside that runtime. The web controller does not own provider processes or reach into private provider state. + +Provider resources are copied with the shared bounded copier during both +controller preparation and session-host staging. Copies retain executable bits +for scripts and binaries while using private permissions; the source files are +never chmodded. Configuration file references are resolved against the file +that declares them, including relative paths and supported environment +references, then staged under the provider home. A missing, unreadable, cyclic, +or oversized reference fails preparation with a typed explanation instead of +being silently removed. Native conversation databases and credential sources +are excluded from ordinary resource copying and continue to use their dedicated +store or secret-reference paths. + +Modern fresh-agent sends may include a checkpoint attempt ID. The server takes +and verifies the pre-turn code snapshot for the selected workspace, then +records an outcome tied to that exact send before forwarding it to the provider. +A completed snapshot failure leaves rewind unavailable for that message but +does not discard the message. If snapshot work has not settled, the bounded +barrier keeps that send queued while the child is quarantined. Once settlement +is confirmed, the same send is forwarded exactly once with rewind unavailable; +if settlement cannot be confirmed, the send remains withheld. Durable outcome +storage read failures preserve the last complete attempt snapshot: known +dispatched or unsettled sends stay withheld, while a new settled attempt can +use a bounded process-local duplicate fence. If the store cannot be read before +any trusted snapshot is available, the send is withheld because prior dispatch +cannot be ruled out. Once a store has been observed or failed to read, a later +missing file remains a read failure. A restart during a process-local fallback +loses that temporary fence, so an unreadable store at startup fails closed. +The durable presence marker also keeps a removed outcome file from appearing +to be a first-use store after restart; restore the outcome file to resume sends +when that marker is present. +Provider identity is joined from persisted native IDs: Codex +associates its native turn ID only with the matching user row, Claude uses the +SDK replay UUID, and OpenCode uses the persisted message ID. Display text and +row order are not identity keys. A replayed turn without an exact identity +keeps rewind unavailable, including after the pane is remounted. The managed fresh-agent adapter routes create/send, interrupt, permission resolutions, compact, rollback, event history, read-only transcript capture, and supported provider-native fork operations through the session host. The Rust web diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 80d3ef7c8..20e69da8b 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -302,19 +302,11 @@ function checkpointAvailabilityKey(scope: string, kind: 'request' | 'turn', id: return `${scope}:${kind}:${id}` } -function checkpointTurnIdentity(provider: FreshAgentPaneContent['provider'], id: string): string { - // Codex can split one provider turn into user and assistant display rows. - // Its accepted-turn id is the unsuffixed provider id, while each rendered - // row carries `:row-N`; match the rewind status to the user row's parent. - return provider === 'codex' ? id.replace(/:row-\d+$/, '') : id -} - -function checkpointTurnAvailabilityKey( - content: FreshAgentPaneContent, - scope: string, - id: string, -): string { - return checkpointAvailabilityKey(scope, 'turn', checkpointTurnIdentity(content.provider, id)) +function checkpointTurnAvailabilityKey(scope: string, id: string): string { + // Display row IDs and provider-native turn IDs are separate aliases. Keep + // display IDs exact here; the Codex row-0 transition is handled explicitly + // when matching an outcome to a row, so row-1 cannot inherit row-0 state. + return checkpointAvailabilityKey(scope, 'turn', id) } function checkpointAvailabilityScope(content: FreshAgentPaneContent): string { @@ -351,10 +343,10 @@ function checkpointAvailabilityMap(content: FreshAgentPaneContent): Map<string, const value = checkpointOutcomeAvailability(outcome) availability.set(checkpointAvailabilityKey(scope, 'request', outcome.requestId), value) if (outcome.displayTurnId) { - availability.set(checkpointTurnAvailabilityKey(content, scope, outcome.displayTurnId), value) + availability.set(checkpointTurnAvailabilityKey(scope, outcome.displayTurnId), value) } if (outcome.nativeTurnId) { - availability.set(checkpointTurnAvailabilityKey(content, scope, outcome.nativeTurnId), value) + availability.set(checkpointTurnAvailabilityKey(scope, outcome.nativeTurnId), value) } } return availability @@ -367,18 +359,38 @@ function checkpointAvailabilityMatchesTurn( ): boolean { if (turn.role !== 'user') return false const displayTurnId = getFreshAgentDisplayTurnKey(turn) - if (availability.displayTurnId && availability.displayTurnId !== displayTurnId) return false - if (availability.nativeTurnId) { - if (turn.nativeTurnId) { - if (availability.nativeTurnId !== turn.nativeTurnId) return false - } else if ( - checkpointTurnIdentity(provider, availability.nativeTurnId) - !== checkpointTurnIdentity(provider, displayTurnId) - ) { - return false - } + const nativeIdentityMatches = Boolean( + availability.nativeTurnId + && turn.nativeTurnId + && availability.nativeTurnId === turn.nativeTurnId, + ) + if (availability.displayTurnId === displayTurnId) { + return !availability.nativeTurnId || !turn.nativeTurnId || nativeIdentityMatches } - return Boolean(availability.displayTurnId || availability.nativeTurnId) + + if (!availability.nativeTurnId || !nativeIdentityMatches) return false + if (provider === 'codex') { + // During an in-progress Codex turn the accepted identity can arrive before + // transcript projection has split the native turn into row-0 (user) and + // row-1 (assistant). Once materialized, only row-0 represents that user + // message; a different row sharing the same native ID is not a match. + return availability.displayTurnId === availability.nativeTurnId + && displayTurnId === `${availability.nativeTurnId}:row-0` + } + return availability.displayTurnId === undefined +} + +const MODERN_TURN_CHECKPOINT_UNAVAILABLE = 'No exact code checkpoint is available for this message.' + +function isModernCheckpointTurn(turn: FreshAgentTurn): boolean { + // Current provider snapshots mark materialized/live turns with their source; + // providers that expose an exact native identity are modern even when an + // older server omitted that source field. Older snapshots with neither keep + // the legacy label-based rewind path. + return Boolean(turn.nativeTurnId) + || turn.source === 'durable' + || turn.source === 'live' + || turn.source === 'server' } function localEchoLanded( @@ -1090,13 +1102,13 @@ export function FreshAgentView({ next.set(checkpointAvailabilityKey(scope, 'request', requestId), availability) } if (submittedTurnId) { - next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, submittedTurnId), availability) + next.set(checkpointTurnAvailabilityKey(scope, submittedTurnId), availability) } if (availability.displayTurnId) { - next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, availability.displayTurnId), availability) + next.set(checkpointTurnAvailabilityKey(scope, availability.displayTurnId), availability) } if (availability.nativeTurnId) { - next.set(checkpointTurnAvailabilityKey(paneContentRef.current, scope, availability.nativeTurnId), availability) + next.set(checkpointTurnAvailabilityKey(scope, availability.nativeTurnId), availability) } return next }) @@ -1136,14 +1148,14 @@ export function FreshAgentView({ const scope = checkpointAvailabilityScope(paneContent) const displayTurnId = getFreshAgentDisplayTurnKey(turn) const turnAvailability = checkpointAvailabilityByTurn.get( - checkpointTurnAvailabilityKey(paneContent, scope, displayTurnId), + checkpointTurnAvailabilityKey(scope, displayTurnId), ) if (turnAvailability && checkpointAvailabilityMatchesTurn(paneContent.provider, turnAvailability, turn)) { return turnAvailability } if (turn.nativeTurnId) { const nativeAvailability = checkpointAvailabilityByTurn.get( - checkpointTurnAvailabilityKey(paneContent, scope, turn.nativeTurnId), + checkpointTurnAvailabilityKey(scope, turn.nativeTurnId), ) if (nativeAvailability && checkpointAvailabilityMatchesTurn(paneContent.provider, nativeAvailability, turn)) { return nativeAvailability @@ -1155,7 +1167,11 @@ export function FreshAgentView({ }, [checkpointAvailabilityByTurn, paneContent.createRequestId, paneContent.provider]) const getRewindAvailability = useCallback((turn: FreshAgentTurn): FreshAgentTurnRewindAvailability | undefined => { const availability = getCheckpointAvailability(turn) - if (!availability) return undefined + if (!availability) { + return isModernCheckpointTurn(turn) + ? { status: 'unavailable', reason: MODERN_TURN_CHECKPOINT_UNAVAILABLE } + : undefined + } if (availability.status === 'unavailable') return { status: 'unavailable', reason: availability.reason } return { status: availability.status } }, [getCheckpointAvailability]) @@ -4029,6 +4045,10 @@ export function FreshAgentView({ setNotice(`Rewind unavailable for this message: ${availability.reason}`) return } + if (!availability && isModernCheckpointTurn(turn)) { + setNotice(`Rewind unavailable for this message: ${MODERN_TURN_CHECKPOINT_UNAVAILABLE}`) + return + } const workspace = availability?.workspace ?? current.checkpointWorkspace const cwd = agentSessionCwdRef.current ?? current.initialCwd ?? workspace?.path if (!cwd) { @@ -4065,7 +4085,9 @@ export function FreshAgentView({ // without a persisted checkpoint ID. const checkpoint = availability?.checkpointId ? checkpoints.find((entry) => entry.id === availability.checkpointId) ?? null - : pickCheckpointForTurn(checkpoints, snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null + : isModernCheckpointTurn(turn) + ? null + : pickCheckpointForTurn(checkpoints, snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null if (!checkpoint) { const reason = availability?.checkpointId ? 'The saved code checkpoint for this message is no longer available.' diff --git a/test/e2e-browser/fixtures/fake-opencode.cjs b/test/e2e-browser/fixtures/fake-opencode.cjs index 4ecc5e8b9..8ef2f204f 100755 --- a/test/e2e-browser/fixtures/fake-opencode.cjs +++ b/test/e2e-browser/fixtures/fake-opencode.cjs @@ -120,6 +120,10 @@ function nextMessageSequence(db, sessionId) { return next } +function isValidMessageId(messageId) { + return typeof messageId === 'string' && /^msg_[0-9a-f]{32}$/i.test(messageId) +} + function sessionModel() { return JSON.stringify({ providerID: 'opencode', modelID: 'fake-opencode' }) } @@ -333,7 +337,9 @@ function appendTuiParityMessages(input) { const sequence = nextMessageSequence(db, input.sessionId) const userTime = Date.now() const assistantTime = userTime + 1 - const userMessageId = `msg_${input.sessionId}_${sequence}_user` + const userMessageId = isValidMessageId(input.messageId) + ? input.messageId + : `msg_${input.sessionId}_${sequence}_user` const assistantMessageId = `msg_${input.sessionId}_${sequence + 1}_assistant` insertMessage(db, { messageId: userMessageId, sessionId: input.sessionId, role: 'user', now: userTime }) insertPart(db, { @@ -404,7 +410,7 @@ function appendTuiParityMessages(input) { }) db.prepare('UPDATE session SET time_updated = ? WHERE id = ?').run(assistantTime, input.sessionId) seedTuiParityChildSessions({ parentSessionId: input.sessionId, directory: existing.directory }) - return { promptText: input.promptText, userMessageId, assistantMessageId, assistantTime } + return { promptText: input.promptText, userMessageId, assistantMessageId, assistantTime, sequence } } finally { db.close() } @@ -999,7 +1005,9 @@ function appendPromptMessages(input) { .filter(Boolean) .join('\n') const responseText = process.env.FAKE_OPENCODE_RESPONSE_TEXT || `Fake OpenCode response: ${promptText}` - const userMessageId = `msg_${input.sessionId}_${sequence}_user` + const userMessageId = isValidMessageId(input.messageId) + ? input.messageId + : `msg_${input.sessionId}_${sequence}_user` const assistantMessageId = `msg_${input.sessionId}_${sequence + 1}_assistant` insertTextMessage(db, { sessionId: input.sessionId, @@ -1061,7 +1069,7 @@ function appendPromptMessages(input) { data: { type: 'reasoning', text: 'waiting for the provider response' }, }) db.prepare('UPDATE session SET time_updated = ? WHERE id = ?').run(erroredTime, input.sessionId) - return { promptText, responseText, userMessageId, assistantMessageId, assistantTime: erroredTime } + return { promptText, responseText, userMessageId, assistantMessageId, assistantTime: erroredTime, sequence } } insertTextMessage(db, { sessionId: input.sessionId, @@ -1072,7 +1080,7 @@ function appendPromptMessages(input) { now: assistantTime, }) db.prepare('UPDATE session SET time_updated = ? WHERE id = ?').run(assistantTime, input.sessionId) - return { promptText, responseText, userMessageId, assistantMessageId, assistantTime } + return { promptText, responseText, userMessageId, assistantMessageId, assistantTime, sequence } } finally { db.close() } @@ -1314,12 +1322,13 @@ const server = http.createServer(async (req, res) => { const appended = tuiParityEnabled ? appendTuiParityMessages({ sessionId, + messageId: body.messageID, promptText: parts .map((part) => typeof part?.text === 'string' ? part.text : '') .filter(Boolean) .join('\n'), }) - : appendPromptMessages({ sessionId, parts }) + : appendPromptMessages({ sessionId, parts, messageId: body.messageID }) if (!appended) { emitSessionIdle(sessionId, { routeDirectory: directory, @@ -1334,7 +1343,7 @@ const server = http.createServer(async (req, res) => { // so the rollback spec reverts a PATCH-CARRYING turn and proves the // working tree stays byte-identical (conversation-only rollback). const patchTurn = Number.parseInt(process.env.FAKE_OPENCODE_PATCH_TURN ?? '', 10) - const userSeq = Number(/^msg_.+_(\d+)_user$/.exec(appended.userMessageId)?.[1]) + const userSeq = appended.sequence if (Number.isInteger(patchTurn) && patchTurn > 0 && (userSeq + 1) / 2 === patchTurn) { const patchPath = path.join(session.directory, 'patch-target.txt') fs.writeFileSync(patchPath, `fake patch from turn ${patchTurn} (${appended.userMessageId})\n`) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 868934889..55ef7c888 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -303,8 +303,13 @@ test('Git workspace sends retain per-turn success and failure, then restore the const durable = await targetJson<FixtureTurn[]>(page, `${fixture.syntheticHomePath}/.codex/fake-turns/${originalSessionId}.json`) expect(durable).toHaveLength(2) - expect(durable?.map(turn => turn.items.find(item => item.type === 'userMessage')?.id)) + // The provider-native turn ID is the durable turn's raw `id`. The nested + // userMessage.id is a distinct item ID (`<turnId>:user`) and must never be + // used as the checkpoint identity join key. + expect(durable?.map(turn => turn.id)) .toEqual([first.outcome.nativeTurnId, second.outcome.nativeTurnId]) + expect(durable?.map(turn => turn.items.find(item => item.type === 'userMessage')?.id)) + .toEqual(durable?.map(turn => `${turn.id}:user`)) }) test('picker-confirmed non-Git workspace captures and restores all selected files', async ({ page }) => { diff --git a/test/fixtures/managed-native-history/codex-tools.json b/test/fixtures/managed-native-history/codex-tools.json index 3cb910f2d..d0ca8fcc4 100644 --- a/test/fixtures/managed-native-history/codex-tools.json +++ b/test/fixtures/managed-native-history/codex-tools.json @@ -43,6 +43,7 @@ "role": "user", "summary": "Saved coding request", "summaryKind": "echo", + "nativeTurnId": "native-turn", "items": [ { "id": "user-native:part:0", diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index 36d644c51..8539b6c74 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3646,12 +3646,17 @@ describe('FreshAgentView', () => { it('accepts a replayed checkpoint outcome after reload when it carries an exact provider row identity', async () => { const checkpointId = 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' const workspace = { path: '/repo/selected', provenance: 'directory-picker' as const } - const displayTurnId = 'codex-replayed-turn:row-0' const nativeTurnId = 'codex-replayed-turn' + // A provider may report the accepted identity while its user row is still + // projected under the raw turn ID. Once the assistant row arrives, Codex + // gives the user row the stable display key `${turnId}:row-0`. + const receiptDisplayTurnId = nativeTurnId + const renderedUserDisplayTurnId = `${nativeTurnId}:row-0` const row = { - id: displayTurnId, - turnId: displayTurnId, + id: renderedUserDisplayTurnId, + turnId: renderedUserDisplayTurnId, nativeTurnId, + source: 'durable' as const, role: 'user' as const, summary: 'A transcript row already stored by the provider', items: [{ id: 'replayed-user-text', kind: 'text' as const, text: 'A transcript row already stored by the provider' }], @@ -3664,6 +3669,7 @@ describe('FreshAgentView', () => { id: 'codex-replayed-turn:row-1', turnId: 'codex-replayed-turn:row-1', nativeTurnId, + source: 'durable', role: 'user', summary: 'A second projection row sharing the provider turn', items: [{ id: 'replayed-decoy-text', kind: 'text', text: 'A second projection row sharing the provider turn' }], @@ -3728,7 +3734,7 @@ describe('FreshAgentView', () => { checkpointId, workspace, nativeTurnId, - displayTurnId, + displayTurnId: receiptDisplayTurnId, }) }) @@ -3738,12 +3744,16 @@ describe('FreshAgentView', () => { .toHaveTextContent('Code rewind is available for this message.') expect(within(exactReplayUserRows[1]!).queryByRole('status', { name: 'Code rewind available for this message' })) .not.toBeInTheDocument() + expect(within(exactReplayUserRows[1]!).getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent('No exact code checkpoint is available for this message.') + expect(within(exactReplayUserRows[1]!).getByRole('button', { name: 'Rewind unavailable for this message' })) + .toBeDisabled() expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual(expect.arrayContaining([ expect.objectContaining({ requestId: 'request-replayed-turn', conversationRequestId: 'req-replayed-conversation', nativeTurnId, - displayTurnId, + displayTurnId: receiptDisplayTurnId, checkpointId, status: 'available', }), @@ -3755,8 +3765,69 @@ describe('FreshAgentView', () => { <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> </Provider>, ) - expect(await screen.findByRole('status', { name: 'Code rewind available for this message' })) + const remountedUserRows = await screen.findAllByRole('article', { name: 'You transcript turn' }) + expect(remountedUserRows).toHaveLength(2) + expect(await within(remountedUserRows[0]!).findByRole('status', { name: 'Code rewind available for this message' })) .toHaveTextContent('Code rewind is available for this message.') + expect(within(remountedUserRows[0]!).getByRole('button', { name: 'Rewind code to here' })).toBeEnabled() + expect(within(remountedUserRows[1]!).queryByRole('status', { name: 'Code rewind available for this message' })) + .not.toBeInTheDocument() + expect(within(remountedUserRows[1]!).getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent('No exact code checkpoint is available for this message.') + expect(within(remountedUserRows[1]!).getByRole('button', { name: 'Rewind unavailable for this message' })) + .toBeDisabled() + }) + + it('keeps an unbound modern turn unavailable instead of matching its prompt to another checkpoint', async () => { + const duplicatePrompt = 'Apply this repeated prompt' + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'unbound modern turn', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: 'modern-unbound-turn:row-0', + turnId: 'modern-unbound-turn:row-0', + nativeTurnId: 'modern-unbound-turn', + source: 'durable', + role: 'user', + summary: duplicatePrompt, + items: [{ id: 'modern-unbound-user-text', kind: 'text', text: duplicatePrompt }], + }], + }) + apiMock.get.mockResolvedValue({ checkpoints: [{ + id: 'cccccccccccccccccccccccccccccccc', + ts: 1, + label: duplicatePrompt, + }] }) + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-unbound-modern-conversation', + sessionId: 'thread-unbound-modern', + status: 'idle', + initialCwd: '/repo', + }, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const [userRow] = await screen.findAllByRole('article', { name: 'You transcript turn' }) + expect(within(userRow!).getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toHaveTextContent('No exact code checkpoint is available for this message.') + const rewindButton = within(userRow!).getByRole('button', { name: 'Rewind unavailable for this message' }) + expect(rewindButton).toBeDisabled() + fireEvent.click(rewindButton) + expect(apiMock.get).not.toHaveBeenCalledWith(expect.stringContaining('/api/fresh-agent/checkpoints?')) + expect(apiMock.post).not.toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', expect.anything()) }) it('uses the selected checkpoint workspace to rewind when a resumed pane has no launch cwd', async () => { diff --git a/test/unit/provider-fixtures/fake-opencode-fixture.test.ts b/test/unit/provider-fixtures/fake-opencode-fixture.test.ts index ee651c4a0..4802a02e2 100644 --- a/test/unit/provider-fixtures/fake-opencode-fixture.test.ts +++ b/test/unit/provider-fixtures/fake-opencode-fixture.test.ts @@ -122,4 +122,40 @@ describe('fake-opencode fixture fork sequence parity (ep3-r1 F3)', () => { .map((m) => (m.parts ?? []).filter((p) => p.type === 'text').map((p) => p.text).join('\n')) expect(userTexts).toEqual(['turn one', 'turn two', 'follow-up']) }, 30_000) + + it('persists the native user message ID supplied to prompt_async', async () => { + const sessionId = 'ses_checkpoint_identity' + const seeded = spawnSync( + process.execPath, + [FIXTURE, 'run', 'seed prompt', '--session', sessionId], + { + env: { + ...process.env, + XDG_DATA_HOME: xdgDataHome, + FAKE_OPENCODE_AUDIT_LOG: auditLog, + }, + }, + ) + expect(seeded.status).toBe(0) + + const headers = { 'content-type': 'application/json', 'x-opencode-directory': projectRoot } + const messageId = 'msg_0123456789abcdef0123456789abcdef' + const promptRes = await fetch(`http://127.0.0.1:${port}/session/${sessionId}/prompt_async`, { + method: 'POST', + headers, + body: JSON.stringify({ + parts: [{ type: 'text', text: 'checkpointed prompt' }], + messageID: messageId, + }), + }) + expect(promptRes.status).toBe(200) + + const messagesRes = await fetch(`http://127.0.0.1:${port}/session/${sessionId}/message`, { headers }) + expect(messagesRes.status).toBe(200) + const messages = (await messagesRes.json()) as FakeMessage[] + const userMessage = messages.find((message) => message.info.id === messageId) + + expect(userMessage?.info.role).toBe('user') + expect(userMessage?.parts?.map((part) => part.text)).toContain('checkpointed prompt') + }, 30_000) }) From 3b459413eab2eef68f722908eab9ed6e8fb54d8c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 07:28:21 -0700 Subject: [PATCH 318/493] Fix restricted checkpoint fixture permissions --- .../managed-provider-environment-target.ts | 34 ++++++++++++++++--- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 8abd97317..7af4fb623 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -372,6 +372,16 @@ function dockerMaybe(args: string[], timeout = 10_000): string | undefined { return result.stdout.trim() } +async function makeStageTreeReadable(directory: string): Promise<void> { + await fs.chmod(directory, 0o755) + for (const entry of await fs.readdir(directory, { withFileTypes: true })) { + const selected = path.join(directory, entry.name) + if (entry.isDirectory()) await makeStageTreeReadable(selected) + else if (entry.isFile()) await fs.chmod(selected, 0o444) + else throw new Error(`Unexpected non-file in staged checkpoint input: ${selected}`) + } +} + async function stageCheckpointTarget(stage: string, repoRoot: string): Promise<void> { await fs.mkdir(stage, { recursive: true, mode: 0o700 }) const binary = path.join(repoRoot, 'target/debug/freshell-server') @@ -381,18 +391,32 @@ async function stageCheckpointTarget(stage: string, repoRoot: string): Promise<v await fs.copyFile(binary, path.join(stage, 'freshell-server')) await fs.chmod(path.join(stage, 'freshell-server'), 0o555) await fs.cp(client, path.join(stage, 'client'), { recursive: true, dereference: true, preserveTimestamps: false }) + await makeStageTreeReadable(path.join(stage, 'client')) await fs.copyFile(path.resolve(repoRoot, 'docker/sandbox/entrypoint.sh'), path.join(stage, 'sandbox-entrypoint.sh')) await fs.chmod(path.join(stage, 'sandbox-entrypoint.sh'), 0o555) - await fs.writeFile(path.join(stage, 'launch.mjs'), checkpointTargetLauncher, { mode: 0o444 }) - await fs.writeFile(path.join(stage, 'control.mjs'), checkpointTargetControl, { mode: 0o444 }) + const launcher = path.join(stage, 'launch.mjs') + await fs.writeFile(launcher, checkpointTargetLauncher, { mode: 0o444 }) + await fs.chmod(launcher, 0o444) + const control = path.join(stage, 'control.mjs') + await fs.writeFile(control, checkpointTargetControl, { mode: 0o444 }) + await fs.chmod(control, 0o444) const fake = path.resolve(repoRoot, 'test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs') await fs.copyFile(fake, path.join(stage, 'fake-app-server.mjs')) await fs.chmod(path.join(stage, 'fake-app-server.mjs'), 0o444) const require = createRequire(import.meta.url) const wsRoot = path.dirname(require.resolve('ws')) - await fs.mkdir(path.join(stage, 'node_modules'), { mode: 0o700 }) - await fs.cp(wsRoot, path.join(stage, 'node_modules/ws'), { recursive: true, dereference: true, preserveTimestamps: false }) - await fs.chmod(path.join(stage, 'node_modules'), 0o555) + const nodeModules = path.join(stage, 'node_modules') + await fs.mkdir(nodeModules, { mode: 0o700 }) + const stagedWebSocket = path.join(nodeModules, 'ws') + await fs.cp(wsRoot, stagedWebSocket, { recursive: true, dereference: true, preserveTimestamps: false }) + await makeStageTreeReadable(stagedWebSocket) + // Rootless Docker maps container UIDs into a private host range, so the + // numerically matching sandbox UID still cannot traverse a host-owned 0700 + // bind-mount root. The mount itself is read-only; grant traversal at the + // root and keep copied package contents readable so the restricted process + // can load its entrypoint and dependencies. + await fs.chmod(nodeModules, 0o755) + await fs.chmod(stage, 0o711) } function restrictedIdentity(): { uid: number; gid: number } { From 8402cf3e4e840c54517f8e31026fcb20e1176339 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 07:32:22 -0700 Subject: [PATCH 319/493] Preserve checkpoint target failure evidence --- .../managed-provider-environment-target.ts | 22 ++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 7af4fb623..01e45840e 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -545,6 +545,7 @@ export async function runCodeCheckpointTarget() { let browser: ChildProcess | undefined, browserIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, browserJoin: Promise<NativeExit> | undefined let dockerStdout = '', dockerStderr = '' + let primaryFailure: unknown const browserSummaryCapture = new ManagedEnvironmentBrowserSummaryCapture() try { await stageCheckpointTarget(stage, repoRoot) @@ -624,13 +625,16 @@ export async function runCodeCheckpointTarget() { }, 120_000) const initialization = await poll('fixed restricted-UID initializer proof', () => externalTargetRead(baseUrl, token, `${checkpointFixtureRoot}/control/initialization.json`), 30_000) + proof.initialization = initialization assert.deepEqual(initialization, { restrictedUid: identity.uid, effectiveCapabilities: '0000000000000000', privateFileDenied: true, privateDirectoryDenied: true, privateFileUid: 0, privateFileMode: '0600', privateDirectoryUid: 0, privateDirectoryMode: '0700', sameParentRenameVerified: true, nonGitWorkspace: fixture.nonGitWorkspacePath }) const serverInfoResponse = await fetch(`${baseUrl}/api/server-info`, { headers: { 'x-auth-token': token }, signal: AbortSignal.timeout(15_000) }) + proof.serverInfoStatus = serverInfoResponse.status assert(serverInfoResponse.ok) const serverInfo = await serverInfoResponse.json() as { commit?: unknown; buildDirty?: unknown; runtime?: unknown } + proof.serverInfo = serverInfo assert.equal(serverInfo.runtime, 'rust') assert.equal(serverInfo.commit, candidateSha) assert.equal(serverInfo.buildDirty, false) @@ -705,6 +709,9 @@ export async function runCodeCheckpointTarget() { await writeJson(receiptPath, proof) console.log(JSON.stringify({ severity: 'info', event: 'code_checkpoint.acceptance_passed', candidateSha, runId, executedBrowserCases: browserSummary.executedCases, passedBrowserCases: browserSummary.passedCases, receipt: receiptPath })) + } catch (error) { + primaryFailure = error + throw error } finally { const errors: Error[] = [] const cleanup: Record<string, unknown> = {} @@ -747,6 +754,13 @@ export async function runCodeCheckpointTarget() { proof.cleanup = cleanup proof.knownOwnedSettled = errors.length === 0 if (errors.length) proof.retainedFixture = root + if (primaryFailure !== undefined) { + const message = primaryFailure instanceof Error ? primaryFailure.message : String(primaryFailure) + proof.primaryFailure = { + name: primaryFailure instanceof Error ? primaryFailure.name : 'UnknownError', + message: message.replaceAll(token, '[redacted]'), + } + } await attempt('cleanupReceipt', async () => { proof.targetOutput = { stdoutTail: dockerStdout.replaceAll(token, '[redacted]').slice(-8192), @@ -755,7 +769,13 @@ export async function runCodeCheckpointTarget() { await fs.mkdir(evidenceDir, { recursive: true, mode: 0o700 }) await writeJson(path.join(evidenceDir, 'code-checkpoint-cleanup.json'), proof) }) - if (errors.length) throw new AggregateError(errors, 'Checkpoint target cleanup remains unsettled') + if (errors.length) { + const primaryMessage = primaryFailure === undefined + ? '' + : `: ${primaryFailure instanceof Error ? primaryFailure.message : String(primaryFailure)}` + const failures = primaryFailure === undefined ? errors : [primaryFailure, ...errors] + throw new AggregateError(failures, `Checkpoint target failed${primaryMessage}; cleanup remains unsettled`) + } } } From 8943a0efe401538627fa51c006536a4af6bc148d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 07:41:54 -0700 Subject: [PATCH 320/493] Keep checkpoint target isolated under rootless Docker --- .../managed-provider-environment-target.ts | 126 +++++++++++++++++- ...anaged-provider-environment-target.test.ts | 85 ++++++++++++ 2 files changed, 207 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 01e45840e..252bc76c6 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -24,6 +24,7 @@ const scenarios = ['healthy', 'missing-reference', 'startup-before-identity'] as const ordinaryFixtureMode = 'managed-fixture-ordinary' const checkpointImage = 'freshell-sandbox:latest' const checkpointContainerPort = 3477 +const checkpointForwarderPort = 3478 const checkpointFixtureRoot = '/tmp/freshell-checkpoint-fixture' const checkpointControlRequest = `${checkpointFixtureRoot}/control/request.json` const checkpointControlAck = (id: string) => `${checkpointFixtureRoot}/control/ack-${id}.json` @@ -31,6 +32,37 @@ export type ManagedProviderEnvironmentTargetMode = 'managed-environment' | 'code const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') +export const checkpointTcpForwarderSource = String.raw` +const net = require('node:net') +const listenHost = process.env.FRESHELL_CHECKPOINT_FORWARDER_HOST +const listenPort = Number(process.env.FRESHELL_CHECKPOINT_FORWARDER_PORT) +const upstreamHost = process.env.FRESHELL_CHECKPOINT_UPSTREAM_HOST +const upstreamPort = Number(process.env.FRESHELL_CHECKPOINT_UPSTREAM_PORT) +if (!listenHost || !Number.isSafeInteger(listenPort) || listenPort < 1 || listenPort > 65535 || + !upstreamHost || !Number.isSafeInteger(upstreamPort) || upstreamPort < 1 || upstreamPort > 65535) { + console.error(JSON.stringify({ severity: 'error', event: 'code_checkpoint.forwarder_config_invalid' })) + process.exit(2) +} +const server = net.createServer(client => { + const upstream = net.createConnection({ host: upstreamHost, port: upstreamPort }) + client.pipe(upstream) + upstream.pipe(client) + client.once('error', () => upstream.destroy()) + upstream.once('error', error => { + console.error(JSON.stringify({ severity: 'warn', event: 'code_checkpoint.forwarder_upstream_failed', code: error.code ?? 'unknown' })) + client.destroy() + }) + client.once('close', () => upstream.destroy()) +}) +server.once('error', error => { + console.error(JSON.stringify({ severity: 'error', event: 'code_checkpoint.forwarder_listen_failed', code: error.code ?? 'unknown' })) + process.exitCode = 1 +}) +server.listen(listenPort, listenHost, () => { + console.log(JSON.stringify({ severity: 'info', event: 'code_checkpoint.forwarder_listening', listenHost, listenPort, upstreamHost, upstreamPort })) +}) +` + export function resolveManagedProviderEnvironmentTargetMode(args: string[]): ManagedProviderEnvironmentTargetMode { if (args.length === 0) return 'managed-environment' if (args.length === 1 && args[0] === '--checkpoint') return 'code-checkpoint' @@ -372,6 +404,33 @@ function dockerMaybe(args: string[], timeout = 10_000): string | undefined { return result.stdout.trim() } +function assertDirectEgressBlocked(containerId: string, subject: string): Record<string, unknown> { + const probe = String.raw` +const net = require('node:net') +let settled = false +const socket = net.createConnection({ host: '1.1.1.1', port: 443 }) +const blocked = reason => { + if (settled) return + settled = true + console.log(JSON.stringify({ blocked: true, reason })) + socket.destroy() +} +socket.setTimeout(1500) +socket.once('connect', () => { + settled = true + console.error(JSON.stringify({ blocked: false })) + socket.destroy() + process.exitCode = 1 +}) +socket.once('error', error => blocked(error.code ?? 'connection-failed')) +socket.once('timeout', () => blocked('timeout')) +` + const result = docker(['exec', containerId, '/usr/local/bin/node', '-e', probe], 5_000) + const evidence = JSON.parse(result) as Record<string, unknown> + assert.equal(evidence.blocked, true, `${subject} unexpectedly reached the external egress probe`) + return evidence +} + async function makeStageTreeReadable(directory: string): Promise<void> { await fs.chmod(directory, 0o755) for (const entry of await fs.readdir(directory, { withFileTypes: true })) { @@ -530,14 +589,18 @@ export async function runCodeCheckpointTarget() { } const proof: Record<string, unknown> = { candidateSha, runId, fixture, image: checkpointImage } const networkName = `freshell-checkpoint-${runId.slice(0, 8)}` + const forwarderNetworkName = `freshell-checkpoint-forwarder-${runId.slice(0, 8)}` const containerName = `freshell-checkpoint-${runId.slice(0, 8)}` + const forwarderContainerName = `freshell-checkpoint-forwarder-${runId.slice(0, 8)}` const containerIdPath = path.join(root, 'container.id') const dockerEnvPath = path.join(root, 'target.env') const cloudDispatchReceiptPath = path.join(root, 'cloud-execution-dispatch.json') const evidenceDir = path.join(repoRoot, '.runtime-evidence', candidateSha, runId) const receiptPath = path.join(evidenceDir, 'code-checkpoint-acceptance.json') let networkId: string | undefined + let forwarderNetworkId: string | undefined let containerId: string | undefined + let forwarderContainerId: string | undefined let dockerRun: ChildProcess | undefined, dockerRunIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, dockerRunJoin: Promise<NativeExit> | undefined let tunnel: ChildProcess | undefined, tunnelIdentity: Awaited<ReturnType<typeof processIdentity>> | undefined, @@ -576,11 +639,19 @@ export async function runCodeCheckpointTarget() { assert.match(networkId, /^[a-f0-9]{64}$/) const networkDetails = docker(['network', 'inspect', '--format', '{{.Id}} {{index .Labels "freshell.run-id"}} {{.Internal}}', networkId]) assert.equal(networkDetails, `${networkId} ${runId} true`, 'Checkpoint network must be internally isolated and owned by this run') + forwarderNetworkId = docker(['network', 'create', '--driver', 'bridge', + '--opt', 'com.docker.network.bridge.enable_ip_masquerade=false', + '--label', 'freshell.owner=checkpoint-acceptance', '--label', `freshell.run-id=${runId}`, forwarderNetworkName]) + assert.match(forwarderNetworkId, /^[a-f0-9]{64}$/) + const forwarderNetworkDetails = docker(['network', 'inspect', '--format', + '{{.Id}} {{index .Labels "freshell.run-id"}} {{.Internal}} {{index .Options "com.docker.network.bridge.enable_ip_masquerade"}}', forwarderNetworkId]) + assert.equal(forwarderNetworkDetails, `${forwarderNetworkId} ${runId} false false`, + 'Loopback forwarder network must disable outbound masquerading and be owned by this run') const dockerArgs = [ 'run', '--rm', '--init', '--name', containerName, '--cidfile', containerIdPath, '--label', 'freshell.owner=checkpoint-acceptance', '--label', `freshell.run-id=${runId}`, - '--network', networkId, '--publish', `127.0.0.1::${checkpointContainerPort}/tcp`, + '--network', networkId, '--pids-limit', '512', '--memory', '4g', '--read-only', '--security-opt', 'no-new-privileges', '--tmpfs', '/tmp:rw,nosuid,nodev,size=512m', '--tmpfs', '/workspace:rw,nosuid,nodev,size=64m', '--tmpfs', '/usr/local/cargo:rw,nosuid,nodev,size=256m', @@ -610,11 +681,33 @@ export async function runCodeCheckpointTarget() { assert.match(containerId, /^[a-f0-9]{64}$/) const containerDetails = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', containerId]) assert.equal(containerDetails, `${containerId} ${runId}`, 'Checkpoint container ownership labels did not match this run') + const targetAddress = docker(['container', 'inspect', '--format', `{{(index .NetworkSettings.Networks "${networkName}").IPAddress}}`, containerId]) + assert.match(targetAddress, /^(?:\d{1,3}\.){3}\d{1,3}$/, 'Restricted checkpoint target did not receive an IPv4 address') + const targetEgress = assertDirectEgressBlocked(containerId, 'Restricted checkpoint target') + + const forwarderArgs = [ + 'run', '--detach', '--rm', '--init', '--name', forwarderContainerName, + '--label', 'freshell.owner=checkpoint-acceptance', '--label', `freshell.run-id=${runId}`, + '--network', forwarderNetworkId, '--publish', `127.0.0.1::${checkpointForwarderPort}/tcp`, + '--pids-limit', '128', '--memory', '128m', '--read-only', '--security-opt', 'no-new-privileges', '--cap-drop', 'ALL', + '--env', 'FRESHELL_CHECKPOINT_FORWARDER_HOST=0.0.0.0', + '--env', `FRESHELL_CHECKPOINT_FORWARDER_PORT=${checkpointForwarderPort}`, + '--env', `FRESHELL_CHECKPOINT_UPSTREAM_HOST=${targetAddress}`, + '--env', `FRESHELL_CHECKPOINT_UPSTREAM_PORT=${checkpointContainerPort}`, + '--entrypoint', '/usr/local/bin/node', checkpointImage, '-e', checkpointTcpForwarderSource, + ] + forwarderContainerId = docker(forwarderArgs) + assert.match(forwarderContainerId, /^[a-f0-9]{64}$/, 'Loopback forwarder did not publish its owned container identity') + const forwarderDetails = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', forwarderContainerId]) + assert.equal(forwarderDetails, `${forwarderContainerId} ${runId}`, 'Loopback forwarder ownership labels did not match this run') + docker(['network', 'connect', networkId, forwarderContainerId]) + const forwarderEgress = assertDirectEgressBlocked(forwarderContainerId, 'Loopback forwarder') + const mapping = await poll('loopback-only ephemeral checkpoint target port', async () => { - const value = dockerMaybe(['port', containerId!, `${checkpointContainerPort}/tcp`]) + const value = dockerMaybe(['port', forwarderContainerId!, `${checkpointForwarderPort}/tcp`]) if (!value) return undefined const match = value.match(/^127\.0\.0\.1:(\d+)$/m) - if (!match) throw new Error('Checkpoint target port was not published on host loopback only') + if (!match) throw new Error('Checkpoint target forwarder port was not published on host loopback only') return Number(match[1]) }, 60_000) assert(Number.isSafeInteger(mapping) && mapping > 0 && mapping <= 65535) @@ -639,7 +732,8 @@ export async function runCodeCheckpointTarget() { assert.equal(serverInfo.commit, candidateSha) assert.equal(serverInfo.buildDirty, false) proof.target = { containerId, containerName, networkId, networkName, networkInternal: true, restrictedIdentity: identity, - hostLoopbackPort: mapping, serverInfo, initialization } + targetAddress, targetEgress, forwarderContainerId, forwarderContainerName, forwarderNetworkId, forwarderNetworkName, + forwarderNetworkMasquerade: false, forwarderEgress, hostLoopbackPort: mapping, serverInfo, initialization } const toolHome = path.join(root, 'cloudflared-home') await fs.mkdir(toolHome, { mode: 0o700 }) @@ -721,6 +815,19 @@ export async function runCodeCheckpointTarget() { } await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin)) await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin)) + if (forwarderContainerId) await attempt('forwarderContainer', async () => { + const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) + if (matching.length) { + assert.deepEqual(matching, [forwarderContainerId], 'Only the exact owned checkpoint forwarder may be stopped') + const details = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', forwarderContainerId]) + assert.equal(details, `${forwarderContainerId} ${runId}`, 'Refusing to clean up an unowned checkpoint forwarder') + docker(['container', 'stop', '--timeout', '10', forwarderContainerId], 20_000) + } + const remaining = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) + if (remaining.length) docker(['container', 'rm', '--force', forwarderContainerId], 20_000) + assert.deepEqual(docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean), []) + return { containerId: forwarderContainerId, settled: true } + }) if (containerId) await attempt('container', async () => { const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) if (matching.length) { @@ -750,6 +857,17 @@ export async function runCodeCheckpointTarget() { assert.deepEqual(docker(['network', 'ls', '-q', '--no-trunc', '--filter', `id=${networkId}`]).split('\n').filter(Boolean), []) return { networkId, settled: true } }) + if (forwarderNetworkId) await attempt('forwarderNetwork', async () => { + const matching = docker(['network', 'ls', '-q', '--no-trunc', '--filter', `id=${forwarderNetworkId}`]).split('\n').filter(Boolean) + if (matching.length) { + assert.deepEqual(matching, [forwarderNetworkId], 'Only the exact owned checkpoint forwarder network may be removed') + const details = docker(['network', 'inspect', '--format', '{{.Id}} {{index .Labels "freshell.run-id"}}', forwarderNetworkId]) + assert.equal(details, `${forwarderNetworkId} ${runId}`, 'Refusing to remove an unowned checkpoint forwarder network') + docker(['network', 'rm', forwarderNetworkId]) + } + assert.deepEqual(docker(['network', 'ls', '-q', '--no-trunc', '--filter', `id=${forwarderNetworkId}`]).split('\n').filter(Boolean), []) + return { networkId: forwarderNetworkId, settled: true } + }) if (errors.length === 0) await attempt('fixtureDirectory', () => fs.rm(root, { recursive: true, force: true })) proof.cleanup = cleanup proof.knownOwnedSettled = errors.length === 0 diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 016db5669..05d924e9d 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -1,15 +1,100 @@ +import { spawn } from 'node:child_process' +import { createConnection, createServer, type AddressInfo } from 'node:net' import { describe, expect, it } from 'vitest' import { + checkpointTcpForwarderSource, resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, } from '../../../../scripts/testing/managed-provider-environment-target.js' +async function listenOnLoopback(server: ReturnType<typeof createServer>): Promise<number> { + await new Promise<void>((resolve, reject) => { + server.once('error', reject) + server.listen(0, '127.0.0.1', resolve) + }) + return (server.address() as AddressInfo).port +} + describe('managed provider environment target mode', () => { it('keeps the existing managed target as the default and selects checkpoint only explicitly', () => { expect(resolveManagedProviderEnvironmentTargetMode([])).toBe('managed-environment') expect(resolveManagedProviderEnvironmentTargetMode(['--checkpoint'])).toBe('code-checkpoint') }) + it('forwards one fixed TCP target while preserving streamed client data', async () => { + const upstream = createServer(socket => { + socket.on('data', chunk => socket.write(chunk)) + }) + const upstreamPort = await listenOnLoopback(upstream) + const listenProbe = createServer() + const listenPort = await listenOnLoopback(listenProbe) + await new Promise<void>(resolve => listenProbe.close(() => resolve())) + + const forwarder = spawn(process.execPath, ['-e', checkpointTcpForwarderSource], { + env: { + ...process.env, + FRESHELL_CHECKPOINT_FORWARDER_HOST: '127.0.0.1', + FRESHELL_CHECKPOINT_FORWARDER_PORT: String(listenPort), + FRESHELL_CHECKPOINT_UPSTREAM_HOST: '127.0.0.1', + FRESHELL_CHECKPOINT_UPSTREAM_PORT: String(upstreamPort), + }, + stdio: ['ignore', 'pipe', 'pipe'], + }) + let stdout = '' + let stderr = '' + forwarder.stdout.on('data', chunk => { stdout += chunk.toString() }) + forwarder.stderr.on('data', chunk => { stderr += chunk.toString() }) + try { + await new Promise<void>((resolve, reject) => { + const timer = setTimeout(() => reject(new Error(`forwarder did not become ready: ${stdout} ${stderr}`)), 5_000) + forwarder.stdout.on('data', () => { + if (!stdout.includes('code_checkpoint.forwarder_listening')) return + clearTimeout(timer) + resolve() + }) + forwarder.once('error', error => { + clearTimeout(timer) + reject(error) + }) + forwarder.once('exit', (code, signal) => { + clearTimeout(timer) + reject(new Error(`forwarder exited before ready (${code ?? signal}): ${stdout} ${stderr}`)) + }) + }) + + const payload = Buffer.from('checkpoint-forwarder-stream-proof') + const received = await new Promise<Buffer>((resolve, reject) => { + const chunks: Buffer[] = [] + let byteLength = 0 + const client = createConnection({ host: '127.0.0.1', port: listenPort }, () => client.write(payload)) + const timer = setTimeout(() => { + client.destroy() + reject(new Error('forwarded TCP stream timed out')) + }, 5_000) + client.on('data', chunk => { + chunks.push(Buffer.from(chunk)) + byteLength += chunk.length + if (byteLength >= payload.length) { + clearTimeout(timer) + resolve(Buffer.concat(chunks)) + client.destroy() + } + }) + client.once('error', error => { + clearTimeout(timer) + reject(error) + }) + }) + expect(received).toEqual(payload) + } finally { + if (forwarder.exitCode === null && forwarder.signalCode === null) { + forwarder.kill('SIGTERM') + await new Promise<void>(resolve => forwarder.once('exit', () => resolve())) + } + await new Promise<void>(resolve => upstream.close(() => resolve())) + } + }) + it.each([['--managed'], ['--checkpoint', '--checkpoint'], ['--checkpoint', 'other'], ['--unknown']])( 'rejects unsupported or ambiguous target arguments %j', (...args) => { expect(() => resolveManagedProviderEnvironmentTargetMode(args)).toThrow(/usage/i) From 4e7b8c4daa7317832a0cb9f96df9ec590a7846a2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 08:03:15 -0700 Subject: [PATCH 321/493] Preflight checkpoint controls and settle owned cloud retries --- .../managed-provider-environment-target.ts | 124 +++++++++++++++--- ...anaged-provider-environment-target.test.ts | 45 +++++++ 2 files changed, 152 insertions(+), 17 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 252bc76c6..2ed29bdaf 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -88,11 +88,12 @@ async function readJson(selected: string): Promise<any | undefined> { catch (error) { if ((error as NodeJS.ErrnoException).code === 'ENOENT') return; throw error } } -async function processIdentity(child: { pid?: number }) { +export async function processIdentity(child: { pid?: number }) { assert(child.pid) const raw = await fs.readFile(`/proc/${child.pid}/stat`, 'utf8') const fields = raw.slice(raw.lastIndexOf(')') + 2).trim().split(/\s+/) - return { pid: child.pid, parentPid: Number(fields[1]), birthTicks: fields[19], cwd: await fs.readlink(`/proc/${child.pid}/cwd`) } + return { pid: child.pid, parentPid: Number(fields[1]), processGroupId: Number(fields[2]), sessionId: Number(fields[3]), + birthTicks: fields[19], cwd: await fs.readlink(`/proc/${child.pid}/cwd`) } } type NativeExit = { code: number | null; signal: NodeJS.Signals | null } @@ -120,18 +121,30 @@ async function bounded<T>(completion: Promise<T>, milliseconds: number, descript } finally { clearTimeout(timer) } } -async function stopExact(child: ChildProcess | undefined, identity: Awaited<ReturnType<typeof processIdentity>> | undefined, - completion: Promise<NativeExit> | undefined) { +export async function stopExact(child: ChildProcess | undefined, identity: Awaited<ReturnType<typeof processIdentity>> | undefined, + completion: Promise<NativeExit> | undefined, ownedProcessGroup = false) { if (!child || !completion) return if (child.exitCode === null && child.signalCode === null) { - try { - assert(identity, 'Live owned child has no captured identity; retain its fixture') - const current = await processIdentity(child) - assert.deepEqual(current, identity, 'Owned child identity changed; retain it') - child.kill('SIGTERM') - } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } - try { return await bounded(completion, 15_000, 'owned child graceful join') } - catch (error) { + assert(identity, 'Live owned child has no captured identity; retain its fixture') + let current: Awaited<ReturnType<typeof processIdentity>> | undefined + try { current = await processIdentity(child) } + catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + if (current) assert.deepEqual(current, identity, 'Owned child identity changed; retain it') + if (ownedProcessGroup) signalOwnedProcessGroup(identity, 'SIGTERM') + else if (current) child.kill('SIGTERM') + } + try { return await bounded(completion, 15_000, 'owned child graceful join') } + catch (error) { + if (ownedProcessGroup) { + assert(identity, 'Detached owned child group has no captured identity; retain its fixture') + if (identity.processGroupId !== identity.pid || identity.sessionId !== identity.pid) { + throw new Error('Detached child no longer identifies its unique owned process group; retain its fixture', { cause: error }) + } + if (child.exitCode === null && child.signalCode === null) { + assert.deepEqual(await processIdentity(child), identity, 'Owned child identity changed before group escalation; retain it') + } + signalOwnedProcessGroup(identity, 'SIGKILL') + } else { if (child.exitCode !== null || child.signalCode !== null) throw error assert.deepEqual(await processIdentity(child), identity, 'Owned child identity changed before escalation; retain it') child.kill('SIGKILL') @@ -140,6 +153,16 @@ async function stopExact(child: ChildProcess | undefined, identity: Awaited<Retu return bounded(completion, 15_000, 'owned child final join') } +function signalOwnedProcessGroup(identity: Awaited<ReturnType<typeof processIdentity>>, signal: NodeJS.Signals): boolean { + assert.equal(identity.processGroupId, identity.pid, 'Owned detached process group must match its captured leader') + assert.equal(identity.sessionId, identity.pid, 'Owned detached session must match its captured leader') + try { process.kill(-identity.processGroupId, signal); return true } + catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ESRCH') return false + throw error + } +} + async function terminalDirectory(rig: ManagedRuntimeBrowserRig): Promise<any[]> { const response = await fetch(`${rig.info.baseUrl}/api/terminals`, { headers: { 'x-auth-token': rig.info.token }, signal: AbortSignal.timeout(30_000) }) @@ -404,6 +427,20 @@ function dockerMaybe(args: string[], timeout = 10_000): string | undefined { return result.stdout.trim() } +function cancelOwnedCloudExecution(executionName: string, candidateSha: string): Record<string, unknown> { + assert.match(executionName, new RegExp(`^freshell-e2e-${candidateSha.slice(0, 12)}-[a-z0-9-]+$`), + 'Refusing to cancel a Cloud execution not created for this checkpoint candidate') + const account = process.env.FRESHELL_GCP_ACCOUNT ?? process.env.GCLOUD_IDENT + const args = ['run', 'jobs', 'executions', 'cancel', executionName, '--quiet', + `--project=${process.env.FRESHELL_GCP_PROJECT ?? 'misc-puttering-project'}`, + `--region=${process.env.FRESHELL_GCP_REGION ?? 'us-west1'}`] + if (account) args.push(`--account=${account}`) + const result = spawnSync('gcloud', args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], timeout: 30_000 }) + if (result.error) throw result.error + if (result.status !== 0) throw new Error(`Owned Cloud checkpoint execution cancellation failed: ${(result.stderr ?? '').trim().slice(-1500)}`) + return { executionName, cancelled: true } +} + function assertDirectEgressBlocked(containerId: string, subject: string): Record<string, unknown> { const probe = String.raw` const net = require('node:net') @@ -512,6 +549,52 @@ async function externalTargetRead(baseUrl: string, token: string, selected: stri return JSON.parse(body.content) } +async function externalTargetWrite(baseUrl: string, token: string, selected: string, content: string): Promise<void> { + const response = await fetch(`${baseUrl}/api/files/write`, { + method: 'POST', + headers: { 'x-auth-token': token, 'content-type': 'application/json' }, + body: JSON.stringify({ path: selected, content }), + signal: AbortSignal.timeout(15_000), + }) + assert(response.ok, `Owned checkpoint target control write failed with HTTP ${response.status}`) +} + +function runCheckpointControl(containerId: string, identity: { uid: number; gid: number }, action: 'eligible' | 'ignored', requestId: string): void { + try { + execFileSync('docker', ['exec', '--user', `${identity.uid}:${identity.gid}`, containerId, + '/usr/bin/node', '/checkpoint-target/control.mjs', action, requestId, String(identity.uid)], + { stdio: ['ignore', 'ignore', 'pipe'], timeout: 15_000 }) + } catch (error) { + const commandError = error as NodeJS.ErrnoException & { status?: number | null; signal?: NodeJS.Signals | null; stderr?: Buffer | string } + const stderr = Buffer.isBuffer(commandError.stderr) ? commandError.stderr.toString('utf8') : commandError.stderr + const details = [ + commandError.status === null || commandError.status === undefined ? undefined : `exit=${commandError.status}`, + commandError.signal ? `signal=${commandError.signal}` : undefined, + stderr?.trim() ? `stderr=${stderr.trim().slice(-2048)}` : undefined, + ].filter(Boolean).join('; ') + throw new Error(`Restricted checkpoint control ${requestId} (${action}) failed${details ? `: ${details}` : ''}`, { cause: error }) + } +} + +async function verifyCheckpointControl( + baseUrl: string, + token: string, + containerId: string, + identity: { uid: number; gid: number }, + fixture: { expectedNonce: string }, + action: 'eligible' | 'ignored', +): Promise<Record<string, unknown>> { + const requestId = randomUUID() + await externalTargetWrite(baseUrl, token, checkpointControlRequest, JSON.stringify({ action, expectedNonce: fixture.expectedNonce, requestId })) + runCheckpointControl(containerId, identity, action, requestId) + const ack = await poll(`restricted fixture control acknowledgement ${requestId}`, async () => + externalTargetRead(baseUrl, token, checkpointControlAck(requestId)), 15_000) + assert.deepEqual(ack, { requestId, action, restrictedUid: identity.uid, privateFileUid: 0, + privateFileMode: '0600', privateFileDenied: true, effectiveCapabilities: '0000000000000000' }) + return { requestId, action, restrictedUid: identity.uid, privateFileDenied: true, + effectiveCapabilities: ack.effectiveCapabilities } +} + async function serveCheckpointControls( baseUrl: string, token: string, @@ -530,9 +613,7 @@ async function serveCheckpointControls( assert(['eligible', 'ignored'].includes(request.action), 'Fixture requested an unsupported private-path state') assert.match(request.requestId, /^[a-zA-Z0-9-]{1,96}$/) processed.add(request.requestId) - execFileSync('docker', ['exec', '--user', `${identity.uid}:${identity.gid}`, containerId, - '/usr/bin/node', '/checkpoint-target/control.mjs', request.action, request.requestId, String(identity.uid)], - { stdio: ['ignore', 'ignore', 'pipe'], timeout: 15_000 }) + runCheckpointControl(containerId, identity, request.action, request.requestId) const ack = await poll(`restricted fixture control acknowledgement ${request.requestId}`, async () => { return externalTargetRead(baseUrl, token, checkpointControlAck(request.requestId)) }, 15_000) @@ -734,6 +815,10 @@ export async function runCodeCheckpointTarget() { proof.target = { containerId, containerName, networkId, networkName, networkInternal: true, restrictedIdentity: identity, targetAddress, targetEgress, forwarderContainerId, forwarderContainerName, forwarderNetworkId, forwarderNetworkName, forwarderNetworkMasquerade: false, forwarderEgress, hostLoopbackPort: mapping, serverInfo, initialization } + proof.localControlPreflight = [] + for (const action of ['ignored', 'eligible', 'ignored'] as const) { + (proof.localControlPreflight as unknown[]).push(await verifyCheckpointControl(baseUrl, token, containerId, identity, fixture, action)) + } const toolHome = path.join(root, 'cloudflared-home') await fs.mkdir(toolHome, { mode: 0o700 }) @@ -779,7 +864,7 @@ export async function runCodeCheckpointTarget() { const browserSpec = 'test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts' cloudEnvironment.FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT = cloudDispatchReceiptPath browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--project=chromium-external-target', browserSpec], { - cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'pipe', 'pipe'], + cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'pipe', 'pipe'], detached: true, }) browser.stdout!.on('data', chunk => browserSummaryCapture.write(chunk)) browser.stdout!.pipe(process.stdout); browser.stderr!.pipe(process.stderr) @@ -813,7 +898,12 @@ export async function runCodeCheckpointTarget() { try { cleanup[operation] = await action() } catch (error) { errors.push(new Error(`Owned checkpoint target cleanup failed: ${operation}`, { cause: error })); cleanup[operation] = 'FAILED; retain owned fixture' } } - await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin)) + const cloudExecution = proof.cloudExecution as { executionName?: unknown } | undefined + if (primaryFailure !== undefined && browser?.exitCode === null && browser.signalCode === null + && typeof cloudExecution?.executionName === 'string') { + await attempt('cloudExecution', () => cancelOwnedCloudExecution(cloudExecution.executionName as string, candidateSha)) + } + await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin, true)) await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin)) if (forwarderContainerId) await attempt('forwarderContainer', async () => { const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 05d924e9d..9be165926 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -1,10 +1,13 @@ import { spawn } from 'node:child_process' +import fs from 'node:fs/promises' import { createConnection, createServer, type AddressInfo } from 'node:net' import { describe, expect, it } from 'vitest' import { checkpointTcpForwarderSource, + processIdentity, resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, + stopExact, } from '../../../../scripts/testing/managed-provider-environment-target.js' async function listenOnLoopback(server: ReturnType<typeof createServer>): Promise<number> { @@ -95,6 +98,48 @@ describe('managed provider environment target mode', () => { } }) + it('settles the exact detached child process group when a wrapper exits before its child', async () => { + const parentSource = `const {spawn}=require('node:child_process');const grandchild=spawn(process.execPath,['-e','setInterval(()=>{},1000)'],{stdio:'inherit'});process.stdout.write(JSON.stringify({pid:grandchild.pid})+'\\n');setInterval(()=>{},1000)` + const wrapper = spawn(process.execPath, ['-e', parentSource], { + stdio: ['ignore', 'pipe', 'pipe'], + detached: true, + }) + let stdout = '' + let stderr = '' + wrapper.stdout.on('data', chunk => { stdout += chunk.toString() }) + wrapper.stderr.on('data', chunk => { stderr += chunk.toString() }) + const closed = new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => { + wrapper.once('error', reject) + wrapper.once('close', (code, signal) => resolve({ code, signal })) + }) + const childPid = await new Promise<number>((resolve, reject) => { + const timer = setTimeout(() => reject(new Error(`wrapper did not publish its child PID: ${stdout} ${stderr}`)), 5_000) + wrapper.stdout.on('data', () => { + const match = stdout.match(/\{"pid":(\d+)\}/) + if (!match) return + clearTimeout(timer) + resolve(Number(match[1])) + }) + wrapper.once('error', error => { + clearTimeout(timer) + reject(error) + }) + wrapper.once('exit', (code, signal) => { + clearTimeout(timer) + reject(new Error(`wrapper exited before publishing its child PID (${code ?? signal}): ${stdout} ${stderr}`)) + }) + }) + const identity = await processIdentity(wrapper) + expect(identity.processGroupId).toBe(wrapper.pid) + expect(identity.sessionId).toBe(wrapper.pid) + await stopExact(wrapper, identity, closed, true) + expect(wrapper.signalCode).toBe('SIGTERM') + const childState = await fs.readFile(`/proc/${childPid}/stat`, 'utf8').then(raw => + raw.slice(raw.lastIndexOf(')') + 2).trim().split(/\s+/)[0], + ).catch(() => undefined) + expect(childState === undefined || childState === 'Z').toBe(true) + }) + it.each([['--managed'], ['--checkpoint', '--checkpoint'], ['--checkpoint', 'other'], ['--unknown']])( 'rejects unsupported or ambiguous target arguments %j', (...args) => { expect(() => resolveManagedProviderEnvironmentTargetMode(args)).toThrow(/usage/i) From 16bee1ee4cc05538ddc192bb73646920cb73d968 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 08:05:37 -0700 Subject: [PATCH 322/493] Run checkpoint controls with sandbox Node path --- .../managed-provider-environment-target.ts | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 2ed29bdaf..d3781881a 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -562,15 +562,20 @@ async function externalTargetWrite(baseUrl: string, token: string, selected: str function runCheckpointControl(containerId: string, identity: { uid: number; gid: number }, action: 'eligible' | 'ignored', requestId: string): void { try { execFileSync('docker', ['exec', '--user', `${identity.uid}:${identity.gid}`, containerId, - '/usr/bin/node', '/checkpoint-target/control.mjs', action, requestId, String(identity.uid)], - { stdio: ['ignore', 'ignore', 'pipe'], timeout: 15_000 }) + '/usr/local/bin/node', '/checkpoint-target/control.mjs', action, requestId, String(identity.uid)], + { stdio: ['ignore', 'pipe', 'pipe'], timeout: 15_000 }) } catch (error) { - const commandError = error as NodeJS.ErrnoException & { status?: number | null; signal?: NodeJS.Signals | null; stderr?: Buffer | string } + const commandError = error as NodeJS.ErrnoException & { + status?: number | null; signal?: NodeJS.Signals | null; stderr?: Buffer | string; stdout?: Buffer | string + } const stderr = Buffer.isBuffer(commandError.stderr) ? commandError.stderr.toString('utf8') : commandError.stderr + const stdout = Buffer.isBuffer(commandError.stdout) ? commandError.stdout.toString('utf8') : commandError.stdout const details = [ commandError.status === null || commandError.status === undefined ? undefined : `exit=${commandError.status}`, commandError.signal ? `signal=${commandError.signal}` : undefined, stderr?.trim() ? `stderr=${stderr.trim().slice(-2048)}` : undefined, + stdout?.trim() ? `stdout=${stdout.trim().slice(-1024)}` : undefined, + !stderr?.trim() && !stdout?.trim() ? commandError.message : undefined, ].filter(Boolean).join('; ') throw new Error(`Restricted checkpoint control ${requestId} (${action}) failed${details ? `: ${details}` : ''}`, { cause: error }) } @@ -741,7 +746,7 @@ export async function runCodeCheckpointTarget() { '--env-file', dockerEnvPath, '--entrypoint', '/checkpoint-target/sandbox-entrypoint.sh', checkpointImage, '/usr/local/bin/node', '/checkpoint-target/launch.mjs', ] - dockerRun = spawn('docker', dockerArgs, { cwd: root, stdio: ['ignore', 'pipe', 'pipe'] }) + dockerRun = spawn('docker', dockerArgs, { cwd: root, stdio: ['ignore', 'pipe', 'pipe'], detached: true }) dockerRunJoin = joinedAfterOutput(dockerRun); dockerRunJoin.catch(() => {}) dockerRun.stdout!.on('data', (chunk: Buffer) => { dockerStdout = (dockerStdout + chunk.toString()).slice(-262_144) @@ -834,7 +839,7 @@ export async function runCodeCheckpointTarget() { assert.match(versionText, /^cloudflared version 2026\.10\.0(?:\s|$)/) tunnel = spawn(tool, ['tunnel', '--url', baseUrl], { cwd: root, env: { PATH: process.env.PATH ?? '', HOME: toolHome, XDG_CONFIG_HOME: toolHome }, - stdio: ['ignore', 'pipe', 'pipe'], + stdio: ['ignore', 'pipe', 'pipe'], detached: true, }) tunnelJoin = joined(tunnel); tunnelJoin.catch(() => {}) tunnelIdentity = await processIdentity(tunnel) @@ -904,7 +909,7 @@ export async function runCodeCheckpointTarget() { await attempt('cloudExecution', () => cancelOwnedCloudExecution(cloudExecution.executionName as string, candidateSha)) } await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin, true)) - await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin)) + await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin, true)) if (forwarderContainerId) await attempt('forwarderContainer', async () => { const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) if (matching.length) { @@ -935,7 +940,7 @@ export async function runCodeCheckpointTarget() { const named = docker(['ps', '-aq', '--no-trunc', '--filter', `name=^/${containerName}$`]).split('\n').filter(Boolean) assert.deepEqual(named, [], 'Container without a published owned ID remains unknown; retain its fixture') }) - await attempt('dockerRunJoin', () => stopExact(dockerRun, dockerRunIdentity, dockerRunJoin)) + await attempt('dockerRunJoin', () => stopExact(dockerRun, dockerRunIdentity, dockerRunJoin, true)) if (networkId) await attempt('network', async () => { const matching = docker(['network', 'ls', '-q', '--no-trunc', '--filter', `id=${networkId}`]).split('\n').filter(Boolean) if (matching.length) { From a595b6bc8ade1470d52605fd2be10e88b6483bff Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 08:17:29 -0700 Subject: [PATCH 323/493] Clear preflight requests before checkpoint browser run --- .../managed-provider-environment-target.ts | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index d3781881a..9d3dcf09d 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -824,6 +824,17 @@ export async function runCodeCheckpointTarget() { for (const action of ['ignored', 'eligible', 'ignored'] as const) { (proof.localControlPreflight as unknown[]).push(await verifyCheckpointControl(baseUrl, token, containerId, identity, fixture, action)) } + await externalTargetWrite(baseUrl, token, checkpointControlRequest, '{}') + proof.localControlPreflightRequestCleared = true + if (process.env.FRESHELL_CODE_CHECKPOINT_PREFLIGHT_ONLY === '1') { + proof.localPreflightOnly = true + await fs.mkdir(evidenceDir, { recursive: true, mode: 0o700 }) + const preflightReceipt = path.join(evidenceDir, 'code-checkpoint-preflight.json') + await writeJson(preflightReceipt, proof) + console.log(JSON.stringify({ severity: 'info', event: 'code_checkpoint.local_preflight_passed', candidateSha, runId, + controlActions: ['ignored', 'eligible', 'ignored'], receipt: preflightReceipt })) + return + } const toolHome = path.join(root, 'cloudflared-home') await fs.mkdir(toolHome, { mode: 0o700 }) @@ -901,7 +912,11 @@ export async function runCodeCheckpointTarget() { const cleanup: Record<string, unknown> = {} const attempt = async (operation: string, action: () => unknown | Promise<unknown>) => { try { cleanup[operation] = await action() } - catch (error) { errors.push(new Error(`Owned checkpoint target cleanup failed: ${operation}`, { cause: error })); cleanup[operation] = 'FAILED; retain owned fixture' } + catch (error) { + errors.push(new Error(`Owned checkpoint target cleanup failed: ${operation}`, { cause: error })) + const message = error instanceof Error ? error.message : String(error) + cleanup[operation] = { settled: false, error: message.replaceAll(token, '[redacted]').slice(-2048) } + } } const cloudExecution = proof.cloudExecution as { executionName?: unknown } | undefined if (primaryFailure !== undefined && browser?.exitCode === null && browser.signalCode === null From 6efee3368036ea99bfe98bdcbdd324e9c7906611 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 08:18:54 -0700 Subject: [PATCH 324/493] Handle exited checkpoint process during join --- .../managed-provider-environment-target.ts | 21 +++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 9d3dcf09d..f7b861c0c 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -121,6 +121,11 @@ async function bounded<T>(completion: Promise<T>, milliseconds: number, descript } finally { clearTimeout(timer) } } +function processIsGone(error: unknown): boolean { + const code = (error as NodeJS.ErrnoException).code + return code === 'ENOENT' || code === 'ESRCH' +} + export async function stopExact(child: ChildProcess | undefined, identity: Awaited<ReturnType<typeof processIdentity>> | undefined, completion: Promise<NativeExit> | undefined, ownedProcessGroup = false) { if (!child || !completion) return @@ -128,7 +133,7 @@ export async function stopExact(child: ChildProcess | undefined, identity: Await assert(identity, 'Live owned child has no captured identity; retain its fixture') let current: Awaited<ReturnType<typeof processIdentity>> | undefined try { current = await processIdentity(child) } - catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + catch (error) { if (!processIsGone(error)) throw error } if (current) assert.deepEqual(current, identity, 'Owned child identity changed; retain it') if (ownedProcessGroup) signalOwnedProcessGroup(identity, 'SIGTERM') else if (current) child.kill('SIGTERM') @@ -141,13 +146,21 @@ export async function stopExact(child: ChildProcess | undefined, identity: Await throw new Error('Detached child no longer identifies its unique owned process group; retain its fixture', { cause: error }) } if (child.exitCode === null && child.signalCode === null) { - assert.deepEqual(await processIdentity(child), identity, 'Owned child identity changed before group escalation; retain it') + let current: Awaited<ReturnType<typeof processIdentity>> | undefined + try { current = await processIdentity(child) } + catch (identityError) { if (!processIsGone(identityError)) throw identityError } + if (current) assert.deepEqual(current, identity, 'Owned child identity changed before group escalation; retain it') } signalOwnedProcessGroup(identity, 'SIGKILL') } else { if (child.exitCode !== null || child.signalCode !== null) throw error - assert.deepEqual(await processIdentity(child), identity, 'Owned child identity changed before escalation; retain it') - child.kill('SIGKILL') + let current: Awaited<ReturnType<typeof processIdentity>> | undefined + try { current = await processIdentity(child) } + catch (identityError) { if (!processIsGone(identityError)) throw identityError } + if (current) { + assert.deepEqual(current, identity, 'Owned child identity changed before escalation; retain it') + child.kill('SIGKILL') + } } } return bounded(completion, 15_000, 'owned child final join') From cdb45580c61f84655a6f948da57b503c68796ab3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 08:36:47 -0700 Subject: [PATCH 325/493] style(test): remove trailing blank line --- test/unit/tooling/run-rust-tests-fixture.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/test/unit/tooling/run-rust-tests-fixture.ts b/test/unit/tooling/run-rust-tests-fixture.ts index bbf726fe8..4d4f50150 100644 --- a/test/unit/tooling/run-rust-tests-fixture.ts +++ b/test/unit/tooling/run-rust-tests-fixture.ts @@ -83,4 +83,3 @@ export function fixture() { cleanup() { rmSync(root, { recursive: true, force: true }); rmSync(artifacts, { recursive: true, force: true }) }, } } - From a77691c021720786b03987394499ab069062aa8b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 08:36:53 -0700 Subject: [PATCH 326/493] fix(testing): respect target rate limits in checkpoint acceptance --- .../managed-provider-environment-target.ts | 41 ++++++++++++++----- ...anaged-provider-environment-target.test.ts | 23 +++++++++++ 2 files changed, 54 insertions(+), 10 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index f7b861c0c..b17fbaac6 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -28,6 +28,8 @@ const checkpointForwarderPort = 3478 const checkpointFixtureRoot = '/tmp/freshell-checkpoint-fixture' const checkpointControlRequest = `${checkpointFixtureRoot}/control/request.json` const checkpointControlAck = (id: string) => `${checkpointFixtureRoot}/control/ack-${id}.json` +const checkpointControlPollIntervalMs = 1_000 +const targetReadRateLimitRetries = 3 export type ManagedProviderEnvironmentTargetMode = 'managed-environment' | 'code-checkpoint' const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') @@ -551,15 +553,32 @@ function restrictedIdentity(): { uid: number; gid: number } { return { uid, gid } } -async function externalTargetRead(baseUrl: string, token: string, selected: string): Promise<any | undefined> { - const response = await fetch(`${baseUrl}/api/files/read?path=${encodeURIComponent(selected)}`, { - headers: { 'x-auth-token': token }, signal: AbortSignal.timeout(15_000), - }) - if (response.status === 404) return undefined - assert(response.ok, `Owned checkpoint target control read failed with HTTP ${response.status}`) - const body = await response.json() as { content?: unknown } - assert(typeof body.content === 'string') - return JSON.parse(body.content) +export async function externalTargetRead( + baseUrl: string, + token: string, + selected: string, + dependencies: { fetcher?: typeof fetch; wait?: (milliseconds: number) => Promise<void> } = {}, +): Promise<any | undefined> { + const fetchTarget = dependencies.fetcher ?? fetch + const wait = dependencies.wait ?? delay + let retries = 0 + while (true) { + const response = await fetchTarget(`${baseUrl}/api/files/read?path=${encodeURIComponent(selected)}`, { + headers: { 'x-auth-token': token }, signal: AbortSignal.timeout(15_000), + }) + if (response.status === 429 && retries < targetReadRateLimitRetries) { + const retryAfterSeconds = Number(response.headers.get('retry-after')) + await response.body?.cancel() + await wait(Number.isFinite(retryAfterSeconds) && retryAfterSeconds > 0 ? retryAfterSeconds * 1_000 : 1_000) + retries += 1 + continue + } + if (response.status === 404) return undefined + assert(response.ok, `Owned checkpoint target control read failed with HTTP ${response.status}`) + const body = await response.json() as { content?: unknown } + assert(typeof body.content === 'string') + return JSON.parse(body.content) + } } async function externalTargetWrite(baseUrl: string, token: string, selected: string, content: string): Promise<void> { @@ -641,7 +660,9 @@ async function serveCheckpointControls( controls.push({ requestId: request.requestId, action: request.action, restrictedUid: identity.uid, privateFileDenied: true, effectiveCapabilities: ack.effectiveCapabilities }) } - await delay(150) + // Each read consumes the target's authenticated API budget shared with + // browser traffic. Keep fixture-control polling below that request rate. + await delay(checkpointControlPollIntervalMs) } } diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 9be165926..29084b12b 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -4,6 +4,7 @@ import { createConnection, createServer, type AddressInfo } from 'node:net' import { describe, expect, it } from 'vitest' import { checkpointTcpForwarderSource, + externalTargetRead, processIdentity, resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, @@ -19,6 +20,28 @@ async function listenOnLoopback(server: ReturnType<typeof createServer>): Promis } describe('managed provider environment target mode', () => { + it('waits for the target Retry-After before retrying a rate-limited control read', async () => { + const responses = [ + new Response('rate limited', { status: 429, headers: { 'retry-after': '2' } }), + new Response(JSON.stringify({ content: JSON.stringify({ requestId: 'request-1' }) }), { status: 200 }), + ] + const waits: number[] = [] + let calls = 0 + const fetcher: typeof fetch = async () => { + calls += 1 + return responses.shift()! + } + + const result = await externalTargetRead('http://127.0.0.1:1234', 'token', '/control/request.json', { + fetcher, + wait: async milliseconds => { waits.push(milliseconds) }, + }) + + expect(result).toEqual({ requestId: 'request-1' }) + expect(calls).toBe(2) + expect(waits).toEqual([2_000]) + }) + it('keeps the existing managed target as the default and selects checkpoint only explicitly', () => { expect(resolveManagedProviderEnvironmentTargetMode([])).toBe('managed-environment') expect(resolveManagedProviderEnvironmentTargetMode(['--checkpoint'])).toBe('code-checkpoint') From 74d5375042c68eb0dd79d3883b9e12ed42fef40b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 09:05:35 -0700 Subject: [PATCH 327/493] fix(testing): expose Codex in checkpoint target --- .../managed-provider-environment-target.ts | 16 +++++++++++++++- .../code-checkpoint-workspace-external.spec.ts | 18 +++++++++++++++++- 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index b17fbaac6..9e001fb9b 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -503,6 +503,13 @@ async function stageCheckpointTarget(stage: string, repoRoot: string): Promise<v await fs.chmod(path.join(stage, 'freshell-server'), 0o555) await fs.cp(client, path.join(stage, 'client'), { recursive: true, dereference: true, preserveTimestamps: false }) await makeStageTreeReadable(path.join(stage, 'client')) + // The restricted server runs outside the repository cwd, so stage the real + // Codex CLI manifest that makes the provider visible to the pane picker. + const codexExtension = path.join(stage, 'extensions', 'codex-cli') + await fs.mkdir(path.dirname(codexExtension), { recursive: true, mode: 0o755 }) + await fs.cp(path.join(repoRoot, 'extensions', 'codex-cli'), codexExtension, + { recursive: true, dereference: true, preserveTimestamps: false }) + await makeStageTreeReadable(path.join(stage, 'extensions')) await fs.copyFile(path.resolve(repoRoot, 'docker/sandbox/entrypoint.sh'), path.join(stage, 'sandbox-entrypoint.sh')) await fs.chmod(path.join(stage, 'sandbox-entrypoint.sh'), 0o555) const launcher = path.join(stage, 'launch.mjs') @@ -514,6 +521,12 @@ async function stageCheckpointTarget(stage: string, repoRoot: string): Promise<v const fake = path.resolve(repoRoot, 'test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs') await fs.copyFile(fake, path.join(stage, 'fake-app-server.mjs')) await fs.chmod(path.join(stage, 'fake-app-server.mjs'), 0o444) + // CLI discovery runs `which` on CODEX_CMD, so expose the multi-argument fake + // app-server command through a discoverable single-word executable. + const fakeCodex = path.join(stage, 'fake-codex') + await fs.writeFile(fakeCodex, + '#!/bin/sh\nexec /usr/local/bin/node /checkpoint-target/fake-app-server.mjs "$@"\n', { mode: 0o555 }) + await fs.chmod(fakeCodex, 0o555) const require = createRequire(import.meta.url) const wsRoot = path.dirname(require.resolve('ws')) const nodeModules = path.join(stage, 'node_modules') @@ -739,12 +752,13 @@ export async function runCodeCheckpointTarget() { 'FRESHELL_BIND_HOST=0.0.0.0', 'FRESHELL_DISABLE_WSL_PORT_FORWARD=1', 'HIDE_STARTUP_TOKEN=true', + 'FRESHELL_EXTENSIONS_DIR=/checkpoint-target/extensions', `FRESHELL_HOME=${fixture.syntheticHomePath}`, `HOME=${fixture.syntheticHomePath}`, `CODEX_HOME=${fixture.syntheticHomePath}/.codex`, `FRESHELL_LOG_DIR=${checkpointFixtureRoot}/storage/logs`, 'FRESHELL_CLIENT_DIR=/checkpoint-target/client', - 'CODEX_CMD=node /checkpoint-target/fake-app-server.mjs', + 'CODEX_CMD=/checkpoint-target/fake-codex', 'FAKE_CODEX_APP_SERVER_BEHAVIOR={"recordTurns":true,"uniqueThreadIds":true}', 'FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES=1', `FRESHELL_CHECKPOINT_FIXTURE_ROOT=${checkpointFixtureRoot}`, diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 55ef7c888..fe701dd5a 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -80,6 +80,19 @@ async function verifyTarget(page: import('@playwright/test').Page): Promise<void expect(response.ok()).toBe(true) const info = await response.json() as { runtime?: unknown; commit?: unknown; buildDirty?: unknown } expect(info).toMatchObject({ runtime: 'rust', commit: fixture.candidateSha, buildDirty: false }) + + const platformResponse = await page.request.get(`${paired!.targetUrl}/api/platform`, { headers, timeout: 30_000 }) + expect(platformResponse.ok()).toBe(true) + const platform = await platformResponse.json() as { availableClis?: Record<string, boolean> } + expect(platform.availableClis?.codex, + 'The synthetic target must expose Codex so Freshcodex is selectable in the pane picker').toBe(true) + + const extensionsResponse = await page.request.get(`${paired!.targetUrl}/api/extensions`, { headers, timeout: 30_000 }) + expect(extensionsResponse.ok()).toBe(true) + const extensions = await extensionsResponse.json() as Array<{ name?: unknown; category?: unknown }> + expect(extensions).toEqual(expect.arrayContaining([ + expect.objectContaining({ name: 'codex', category: 'cli' }), + ])) } async function launchCodex(page: import('@playwright/test').Page, workspace: string) { @@ -90,7 +103,10 @@ async function launchCodex(page: import('@playwright/test').Page, workspace: str await verifyTarget(page) const picker = await openPanePicker(page) - await picker.getByRole('button', { name: /^Freshcodex$/i }).click({ force: true }) + const freshcodex = picker.getByRole('button', { name: /^Freshcodex$/i }) + await expect(freshcodex, + 'The enabled Codex runtime must have a visible Freshcodex option before launch').toBeVisible({ timeout: 10_000 }) + await freshcodex.click({ force: true }) const directoryInput = page.getByRole('combobox', { name: 'Starting directory for Freshcodex' }) await expect(directoryInput).toBeVisible({ timeout: 15_000 }) await directoryInput.fill(workspace) From a0c8f1dda7df4e9710ef780831171641f26264da Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 09:13:39 -0700 Subject: [PATCH 328/493] test(managed-provider): fix source mode under restrictive umask --- .../freshell-server/src/managed_mcp_capability/source_stage.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 8b3464874..5eb60398e 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -381,6 +381,7 @@ mod configured_resource_tests { } let sound = external.join("selected.wav"); std::fs::write(&sound, b"RIFF-selected-wave-bytes").unwrap(); + std::fs::set_permissions(&sound, std::fs::Permissions::from_mode(0o644)).unwrap(); std::fs::write(tool.join("resource"), "mcp-resource").unwrap(); std::fs::write( tool.join("node_modules/dependency/index.js"), From f8e69761b3f235580de4b0632ddc6413f17c7bcc Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 09:23:25 -0700 Subject: [PATCH 329/493] test(checkpoint): align fixture and verify websocket barrier --- crates/freshell-ws/src/terminal.rs | 122 ++++++++++++++++++ .../managed-provider-environment-target.ts | 4 + ...code-checkpoint-workspace-external.spec.ts | 2 + 3 files changed, 128 insertions(+) diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index 80e65bd3f..b41fd8774 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -13008,6 +13008,128 @@ mod pane_reconcile_gate_tests { } } + struct GatedCheckpointSend { + seen_send: tokio::sync::Mutex<Option<freshell_protocol::FreshAgentSend>>, + entered: tokio::sync::Mutex<Option<tokio::sync::oneshot::Sender<()>>>, + release: tokio::sync::Mutex<Option<tokio::sync::oneshot::Receiver<()>>>, + } + + impl crate::checkpoint_barrier::FreshAgentCheckpointBarrier for GatedCheckpointSend { + fn barrier_and_forward( + self: Arc<Self>, + send: freshell_protocol::FreshAgentSend, + forward: futures::future::BoxFuture<'static, ()>, + ) -> futures::future::BoxFuture<'static, ()> { + Box::pin(async move { + *self.seen_send.lock().await = Some(send); + if let Some(entered) = self.entered.lock().await.take() { + let _ = entered.send(()); + } + if let Some(release) = self.release.lock().await.take() { + let _ = release.await; + } + forward.await; + }) + } + + fn replay_for_session( + self: Arc<Self>, + _provider: freshell_protocol::AgentProvider, + _session_id: String, + _session_type: freshell_protocol::SessionType, + ) -> futures::future::BoxFuture<'static, ()> { + Box::pin(async {}) + } + } + + #[tokio::test] + async fn checkpoint_intent_send_is_gated_by_server_barrier_before_opencode_dispatch() { + let (mut ws_tx, _client) = loopback_sink_and_client().await; + let mut state = state(); + let mut provider_events = state.broadcast_tx.subscribe(); + let (entered_tx, entered_rx) = tokio::sync::oneshot::channel(); + let (release_tx, release_rx) = tokio::sync::oneshot::channel(); + let barrier = Arc::new(GatedCheckpointSend { + seen_send: tokio::sync::Mutex::new(None), + entered: tokio::sync::Mutex::new(Some(entered_tx)), + release: tokio::sync::Mutex::new(Some(release_rx)), + }); + state.checkpoint_send.service = Some(barrier.clone()); + let conn_sink: FrameSink = std::sync::Arc::new(|_| {}); + let (interactive_create_tx, _interactive_create_rx) = + mpsc::channel::<interactive_creates::Job>(1); + let (_cancel_tx, create_cancel_rx) = tokio::sync::watch::channel(false); + let mut host_stats_last_refresh_at = None; + let mut conn_identity = ConnectionIdentity::default(); + + let keep_open = handle_client_text( + r#"{"type":"freshAgent.send","provider":"opencode","sessionId":"missing-opencode-session","sessionType":"freshopencode","text":"checkpoint barrier probe","requestId":"checkpoint-barrier-request","checkpointIntent":{"logicalAttemptId":"checkpoint-barrier-attempt","workspace":{"path":"/tmp/checkpoint-barrier-probe","provenance":"directory-picker"}}}"#, + &mut ws_tx, + &state, + 1, + &conn_sink, + false, + false, + false, + false, + &interactive_create_tx, + &create_cancel_rx, + &mut host_stats_last_refresh_at, + &mut conn_identity, + &mut Default::default(), + &None, + ) + .await; + assert!( + keep_open, + "a checkpointed send must keep the websocket open" + ); + tokio::time::timeout(std::time::Duration::from_secs(5), entered_rx) + .await + .expect("checkpoint barrier was invoked") + .expect("barrier entry signal"); + + let captured = barrier + .seen_send + .lock() + .await + .clone() + .expect("barrier receives the original send"); + assert_eq!( + captured.request_id.as_deref(), + Some("checkpoint-barrier-request") + ); + assert_eq!( + captured.checkpoint_intent, + Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "checkpoint-barrier-attempt".to_string(), + workspace: Some(freshell_protocol::FreshAgentCheckpointWorkspace { + path: "/tmp/checkpoint-barrier-probe".to_string(), + provenance: freshell_protocol::CheckpointWorkspaceProvenance::DirectoryPicker, + }), + }) + ); + assert!( + matches!( + provider_events.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + ), + "provider dispatch must remain blocked until the barrier settles" + ); + + release_tx.send(()).expect("release provider dispatch"); + let event = tokio::time::timeout(std::time::Duration::from_secs(5), provider_events.recv()) + .await + .expect("provider dispatch produces an event after release") + .expect("provider event channel remains open"); + let event: serde_json::Value = serde_json::from_str(&event).expect("provider event JSON"); + assert_eq!(event["type"], "error"); + assert_eq!(event["requestId"], "checkpoint-barrier-request"); + assert!(event["message"] + .as_str() + .is_some_and(|message| message.starts_with("SESSION_NOT_FOUND:"))); + } + #[tokio::test] async fn rejected_unknown_tabs_push_does_not_replace_connection_identity() { let (mut ws_tx, mut client) = loopback_sink_and_client().await; diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 9e001fb9b..3e3cee92a 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -324,6 +324,10 @@ const uid = Number(process.env.FRESHELL_CHECKPOINT_FIXTURE_UID) assert.equal(process.env.FRESHELL_CHECKPOINT_FIXTURE_ROOT, root) assert(Number.isSafeInteger(uid) && uid > 0) assert.equal(process.getuid(), uid, 'fixture setup must run as restricted UID B') +// The sandbox entrypoint's gosu handoff resets HOME to /home/sandbox. Restore +// the explicitly staged synthetic account home so validate-dir exercises the +// same account-home exclusion that this fixture advertises to the browser. +process.env.HOME = path.join(root, 'home') const status = fs.readFileSync('/proc/self/status', 'utf8') const capabilities = status.match(/^CapEff:\s+([0-9a-f]+)$/m)?.[1] assert.equal(capabilities, '0000000000000000', 'fixture UID B must have no effective capabilities') diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index fe701dd5a..d72f8c33e 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -259,6 +259,7 @@ test('Git workspace sends retain per-turn success and failure, then restore the expect(first.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) await expect(first.row.getByRole('status', { name: 'Code rewind available for this message' })) .toContainText('Code rewind is available for this message.') + await first.row.hover() await expect(first.row.getByRole('button', { name: 'Rewind code to here' })).toBeEnabled() await assertNativeUserIdentity(page, first.outcome, first.accepted.submittedTurnId, samePrompt) @@ -345,6 +346,7 @@ test('picker-confirmed non-Git workspace captures and restores all selected file } }) expect(turn.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) await expect(turn.row.getByRole('status', { name: 'Code rewind available for this message' })).toBeVisible() + await turn.row.hover() await expect(turn.row.getByRole('button', { name: 'Rewind code to here' })).toBeEnabled() await assertNativeUserIdentity(page, turn.outcome, turn.accepted.submittedTurnId, marker) From c15c74fb2a225be4da5ccae7af1353e8bf6533b4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 09:42:01 -0700 Subject: [PATCH 330/493] test(checkpoint): align cloud acceptance with turn labels --- ...code-checkpoint-workspace-external.spec.ts | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index d72f8c33e..a546f4eed 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -172,7 +172,10 @@ async function sendThroughComposer( expect(outcome!.nativeTurnId).toBe(accepted.submittedTurnId) expect(JSON.stringify((await currentFreshAgent(page)).content.checkpointOutcomes)).not.toContain(text) - const row = paneRoot.locator('article[data-turn-role="user"]', { hasText: text }) + // Repeated prompts are intentional in this acceptance case. The freshly + // accepted turn is the newest matching user row; requiring the locator to + // be unique would make the second send fail before its identity assertions. + const row = paneRoot.locator('article[data-turn-role="user"]', { hasText: text }).last() await expect(row).toBeVisible({ timeout: 30_000 }) return { sent, accepted, outcome: outcome!, row } } @@ -310,12 +313,13 @@ test('Git workspace sends retain per-turn success and failure, then restore the await duplicateRows.nth(0).hover() const rewind = duplicateRows.nth(0).getByRole('button', { name: 'Rewind code to here' }) await expect(rewind).toBeEnabled() + const rewindNotice = expect(restoredPane.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) + .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) await rewind.click() - await dialogAccepted + await Promise.all([dialogAccepted, rewindNotice]) await expect.poll(() => targetRead(page, `${fixture.syntheticWorkspacePath}/main.rs`)).toBe('main before checkpoint\n') await expect.poll(() => targetRead(page, `${fixture.syntheticWorkspacePath}/notes.txt`)).toBe('notes before checkpoint\n') - await expect(restoredPane.getByText(`Code rewound to before: "${samePrompt}"`)).toBeVisible({ timeout: 30_000 }) await expect(duplicateRows).toHaveCount(2) const durable = await targetJson<FixtureTurn[]>(page, `${fixture.syntheticHomePath}/.codex/fake-turns/${originalSessionId}.json`) @@ -353,15 +357,16 @@ test('picker-confirmed non-Git workspace captures and restores all selected file await targetWrite(page, `${fixture.nonGitWorkspacePath}/one.txt`, 'one changed after send\n') await targetWrite(page, `${fixture.nonGitWorkspacePath}/two.txt`, 'two changed after send\n') await turn.row.hover() + const rewindNotice = expect(paneRoot.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) + .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) await turn.row.getByRole('button', { name: 'Rewind code to here' }).click() - await dialogAccepted + await Promise.all([dialogAccepted, rewindNotice]) await expect.poll(() => targetRead(page, `${fixture.nonGitWorkspacePath}/one.txt`)).toBe('one before checkpoint\n') await expect.poll(() => targetRead(page, `${fixture.nonGitWorkspacePath}/two.txt`)).toBe('two before checkpoint\n') const afterRestore = await currentFreshAgent(page) expect(afterRestore.content.sessionRef?.sessionId ?? afterRestore.content.sessionId).toBe(originalSessionId) expect(afterRestore.content.checkpointWorkspace).toEqual({ path: fixture.nonGitWorkspacePath, provenance: 'directory-picker' }) - await expect(paneRoot.getByText(`Code rewound to before: "${marker}"`)).toBeVisible({ timeout: 30_000 }) }) test('HOME confirmation is launch-only and an accepted send has no checkpoint', async ({ page }) => { @@ -373,10 +378,10 @@ test('HOME confirmation is launch-only and an accepted send has no checkpoint', const marker = `checkpoint home ${fixture.expectedNonce}` const turn = await sendThroughComposer(page, paneRoot, marker, undefined) expect(turn.outcome).toMatchObject({ status: 'unavailable', code: 'workspace_unavailable' }) - expect(turn.outcome.reason).toMatch(/account home|selected code checkpoint workspace/i) + expect(turn.outcome.reason).toMatch(/no eligible code workspace is available/i) expect(turn.sent.checkpointIntent).not.toHaveProperty('workspace') await expect(turn.row.getByRole('note', { name: 'Code rewind unavailable for this message' })) - .toContainText(/account home|selected code checkpoint workspace/i) + .toContainText(turn.outcome.reason!) await turn.row.hover() await expect(turn.row.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() From 492439094e784022db3936895eb2d729c2fd258f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 10:01:50 -0700 Subject: [PATCH 331/493] docs(plan): correct managed checkpoint verification paths --- docs/plans/2026-10-05-managed-provider-environment.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 6400993e9..9ef088a8b 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -348,7 +348,7 @@ Managed output observations distinguish Running, active Recovering, Blocked with Then the runner supplies the absolute selected binary via `FRESHELL_OPENCODE_CONTRACT_BIN` and its owned fixture locator to `mise exec rust@1.96 -- cargo test -p freshell-opencode --features real-provider-contract-tests --locked --test checkpoint_message_id_contract opencode_checkpoint_message_id_persists_after_prompt -- --exact --nocapture`. The test independently validates the selected binary/version, starts the real production serve/prompt path with isolated synthetic configuration and local model server, and reads actual native persisted history. Use a 120-second whole-contract execution deadline (including real serve startup, synthetic prompt completion and history observation) and a 15-second owned-process cleanup deadline; timeout is a failing result, never a skip. Bound package provisioning separately to five minutes and fail rather than using an ambient binary. Own fixture PIDs and finish termination/reaping in `finally`, preserving only a private nonsecret command/version/integrity/pass-count receipt outside the tracked tree. Gate requires exactly one executed/passed contract test and zero failures/ignored tests; zero matches or missing/wrong binary is a failure. This dedicated lane remains mandatory before readiness and on final committed candidate HEAD, alongside the default fake tests. It does not imply enabling real-provider tests in the default CI/prepush lane. -**Named identity regression tests:** Add `test/unit/claude-sidecar/user-message-replay.test.ts` to execute the real sidecar queue/SDK-input handler with an injected synthetic SDK transport and verify `claude_checkpoint_uuid_replay_binds_persisted_user`: the supplied valid UUID/request correlation survives actual user replay, mismatched/absent replay stays unknown, and unrelated assistant/result events cannot bind it. Add `test/integration/server/claude-checkpoint-uuid-replay.test.ts` to run the pinned SDK/bundled CLI with an isolated synthetic local provider endpoint, no model-network or real grants, and assert the same UUID in both replay and persisted user transcript. Reuse the synthetic probe arrangement recorded at `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`; a mock-input-only test is insufficient for this integration gate. Add default fake-only `crates/freshell-opencode/tests/checkpoint_message_id.rs` with `opencode_checkpoint_message_id_in_prompt_body` and `opencode_checkpoint_message_id_binds_observed_native_row`: execute the production prompt/transport path with injected HTTP/history fakes and assert its outgoing native ID and observed user-row binding without any provider binary. Add separate opt-in `crates/freshell-opencode/tests/checkpoint_message_id_contract.rs` with `opencode_checkpoint_message_id_persists_after_prompt`: execute that production path against pinned OpenCode 1.18.21 in an isolated synthetic store and local model endpoint, then read native persisted history and assert `messageID == info.id`. The fake target guards production behavior in default Cargo/CI/prepush; only the actual pinned contract target proves native persistence. Separately add `checkpoint_turn_identity::opencode_checkpoint_message_id_survives_compact_queue` in `crates/freshell-freshagent/src/opencode_ws.rs`, where the compact queue actually lives: execute queue/drain/prompt and bind the observed persisted user row, rather than a queue acknowledgement or rewritten fixture ID. Freshagent tests named `checkpoint_turn_identity` also cover the production Codex native/display user join, modern duplicate text, aliases, unknown results and queue handling. Server tests named `managed_checkpoint_send_barrier` cover feature-gated post-capture admission, stale fences, one dispatch after failure and settlement/duplicate joining. Each named selector must execute a positive test count after Task 7's changes. +**Named identity regression tests:** Add `test/unit/claude-sidecar/user-message-replay.test.ts` to execute the real sidecar queue/SDK-input handler with an injected synthetic SDK transport and verify `claude_checkpoint_uuid_replay_binds_persisted_user`: the supplied valid UUID/request correlation survives actual user replay, mismatched/absent replay stays unknown, and unrelated assistant/result events cannot bind it. Add `test/integration/providers/claude-checkpoint-uuid-replay.test.ts` to run the pinned SDK/bundled CLI with an isolated synthetic local provider endpoint, no model-network or real grants, and assert the same UUID in both replay and persisted user transcript. Reuse the synthetic probe arrangement recorded at `/home/dan/code/freshell/.worktrees/.the-usual-logs/managed-provider-environment/reports/sdk-uuid-preflight.md`; a mock-input-only test is insufficient for this integration gate. Add default fake-only `crates/freshell-opencode/tests/checkpoint_message_id.rs` with `opencode_checkpoint_message_id_in_prompt_body` and `opencode_checkpoint_message_id_binds_observed_native_row`: execute the production prompt/transport path with injected HTTP/history fakes and assert its outgoing native ID and observed user-row binding without any provider binary. Add separate opt-in `crates/freshell-opencode/tests/checkpoint_message_id_contract.rs` with `opencode_checkpoint_message_id_persists_after_prompt`: execute that production path against pinned OpenCode 1.18.21 in an isolated synthetic store and local model endpoint, then read native persisted history and assert `messageID == info.id`. The fake target guards production behavior in default Cargo/CI/prepush; only the actual pinned contract target proves native persistence. Separately add `checkpoint_turn_identity::opencode_checkpoint_message_id_survives_compact_queue` in `crates/freshell-freshagent/src/opencode_ws.rs`, where the compact queue actually lives: execute queue/drain/prompt and bind the observed persisted user row, rather than a queue acknowledgement or rewritten fixture ID. Freshagent tests named `checkpoint_turn_identity` also cover the production Codex native/display user join, modern duplicate text, aliases, unknown results and queue handling. Server service-barrier cases include `checkpoints::tests::durable_dispatch_fence_blocks_retry_when_outcome_store_becomes_unreadable` and `checkpoints::tests::pending_send_forwards_once_after_quarantine_confirms_settlement`. The actual WebSocket dispatch gate is `terminal::pane_reconcile_gate_tests::checkpoint_intent_send_is_gated_by_server_barrier_before_opencode_dispatch` in `freshell-ws`; each named selector must execute a positive test count after Task 7's changes. - [ ] **Step 1: Establish failing behavioral tests before implementation.** Extend Rust checkpoint route tests to select a Git checkout root from a nested launch cwd, select an explicit workspace while launch cwd is an account home, refuse an automatic home scope, and accept an explicitly selected non-Git workspace. Assert actual checkpoint tree/file restoration rather than request shape alone. Add the named picker producer/normalization/route tests from Step 2, including actual validate-dir HOME/non-home results and mkdir-confirm reuse. Add route tests showing failed staging/metadata publication creates no new list/restore target and does not alter prior successful checkpoints. Add the real restricted-UID black-box failure fixture from Step 6; a mode-bit fixture executed as root is not a red test. Extend component tests by exercising composer send, acknowledgements and actual rendered actions for pending/success/failed/unavailable results, stable duplicate prompts, metadata timing, reload/remount, queued sends and scope changes. The assertion that a pre-turn snapshot finishes before provider dispatch must fail against the current fire-and-forget path. Add the ten timeout/cooldown/lock-budget service cases named in Step 4 before implementing that behavior: their controllable runner/clock must expose repeated full-wait behavior as RED, not just a missing field/module compile failure. Verify these tests fail for the intended current behaviors and record red output. @@ -393,14 +393,14 @@ Managed output observations distinguish Running, active Recovering, Blocked with Through the real picker/composer and real checkpoint routes, send a successful first user turn, observe enabled rewind for that exact durable row, mutate files, create a failed second turn with the same text and an inaccessible eligible path, observe the second message accepted and usable interaction with a disabled rewind/reason, and assert the earlier success stays available. Restore the first turn through UI confirmation and verify filesystem contents, conversation/model/session identity, and later turn history remain intact. Separately create a pane using the actual DirectoryPicker confirmation of a synthetic non-Git directory (no `.git`), send an accepted user turn, verify its resolved scope/exact row has enabled rewind, mutate at least two eligible files, restore that row through UI confirmation and verify both original contents and the unchanged non-Git scope. This tests picker provenance through client send, server resolution, publication and restore; API-only seeding is insufficient. Separately create a HOME-launched pane through the actual DirectoryPicker: account-home confirmation sets launch cwd only, sends succeed with a clear unavailable reason and no home checkpoint. The project and non-Git panes are created by picker confirmation of their own directories, carry the separate workspace record and snapshot/restore only that selected code. Assert actual persisted pane intent and resolved server result; do not claim a same-pane HOME launch plus independent project selection that the current UI cannot express. Component/route/service tests above cover independence from a later different launch/routing cwd. Include remount/reload so failure state cannot turn into a successful duplicate-label target. Wait on stable acknowledged rows/results; do not sleep around optimistic echoes, skip a failed case, or count a zero-test selector. Record actual executed/passed counts from the configured cloud receipt. The previous reachability preflight proves transport only; it does not satisfy this browser proof. - [ ] **Step 8: Green, refactor and focused checkpoint commit.** Refactor shared scope/availability selection only after red tests pass; remove stale create-only/deferred comments in `checkpoints.rs` because all sibling routes now exist. Run affected component/checkpoint selection tests using the repo's `test:vitest` path, Rust route and black-box smoke lanes, and the actual configured-cloud browser spec. Integrate with the periodic task-boundary gate and final exact-HEAD broad gate after all Tasks 1–7 join. Validate no foreign files or permission changes entered the diff. Record Task 7 start/end commits and commit focused, compiling logical checkpoints with their behavioral tests, for example scope/complete publication, settled send barrier and native identity, then per-turn UI/restricted fixtures/browser acceptance. Include all required interface/callsite wiring in each checkpoint; do not commit broken intermediates or force this whole task into one commit. The fresh task specification-plus-quality review covers the entire start..end range and descriptive test evidence for every checkpoint. Update the kata's criterion evidence via the parent workflow. No PR creation or live restart without their separate explicit approvals. - Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked managed_checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked checkpoint_send_barrier`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id` (at least two fake-only tests); `pnpm exec tsx scripts/testing/opencode-checkpoint-contract.ts` (exactly one real pinned persistence test, using the required-features command above); `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/server/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx test/unit/client/components/panes/DirectoryPicker.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/store/panesSlice.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/pane-reconcile.fresh-agent.test.ts test/unit/client/lib/tab-registry-open.test.ts --config config/vitest/vitest.config.ts`. Require positive named scope/producer cases and actual server `files::tests` validate-dir eligibility behavior in addition to the `checkpoints` gate. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, picker-confirmed HOME with no scope and separate project/non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. + Focused commands (all after Task 7 implementation): `mise exec rust@1.96 -- cargo build -p freshell-server --features managed-runtime-v1 --locked`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints`; `mise exec rust@1.96 -- cargo test -p freshell-server --features managed-runtime-v1 --locked checkpoints::tests::durable_dispatch_fence_blocks_retry_when_outcome_store_becomes_unreadable`; `mise exec rust@1.96 -- cargo test -p freshell-ws --locked --lib terminal::pane_reconcile_gate_tests::checkpoint_intent_send_is_gated_by_server_barrier_before_opencode_dispatch`; `mise exec rust@1.96 -- cargo test -p freshell-freshagent --locked checkpoint_turn_identity`; `mise exec rust@1.96 -- cargo test -p freshell-opencode --locked --test checkpoint_message_id` (at least two fake-only tests); `pnpm exec tsx scripts/testing/opencode-checkpoint-contract.ts` (exactly one real pinned persistence test, using the required-features command above); `pnpm run test:vitest run test/unit/claude-sidecar/user-message-replay.test.ts test/integration/providers/claude-checkpoint-uuid-replay.test.ts --config config/vitest/vitest.config.ts`; require positive executed counts for every selector and retain the named UUID replay/persistence receipt; `pnpm run test:vitest run test/unit/client/lib/fresh-agent-checkpoints.test.ts test/unit/client/components/fresh-agent/FreshAgentView.test.tsx test/unit/client/components/fresh-agent/FreshAgentTranscript.test.tsx test/unit/client/components/fresh-agent/FreshAgentTurnActions.test.tsx test/unit/client/components/panes/DirectoryPicker.test.tsx test/unit/client/components/panes/PaneContainer.test.tsx test/unit/client/components/panes/PaneContainer.createContent.test.tsx test/unit/client/store/panesSlice.test.ts test/unit/client/lib/session-type-utils.test.ts test/unit/client/lib/pane-reconcile.fresh-agent.test.ts test/unit/client/lib/tab-registry-open.test.ts --config config/vitest/vitest.config.ts`. Require positive named scope/producer cases and actual server `files::tests` validate-dir eligibility behavior in addition to the `checkpoints` gate. Add named black-box test `crates/freshell-server/tests/code_checkpoint_permissions.rs` and run `scripts/sandbox-test.sh --runtime-suite --checkpoint-fixture "cargo test -p freshell-server --features managed-runtime-v1 --locked --test code_checkpoint_permissions"`; expected actual restricted route failures preserve previous complete targets and source permissions. New cloud-legal spec `test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` runs with `GCLOUD_ROBOT_REQUIRE=1 pnpm exec tsx scripts/testing/managed-provider-environment-target.ts --checkpoint`. That closed driver mode creates the restricted paired fixture, validates its exact owned target and sets kind `code-checkpoint` plus `FRESHELL_CODE_CHECKPOINT_FIXTURE` before `pnpm run test:e2e --project=chromium-external-target test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts` on configured cloud. Expected positive executed/pass counts for successful/failed turns, permission denial, picker-confirmed HOME with no scope and separate project/non-Git capture/restore; no skipped/zero-test coverage or absent-target fallback. ## Acceptance and completion boundary 1. Map freshell#bdpt's exact latest 15 criteria to the executed task/test evidence in the parent run record; the user's requested provider and recovery additions remain mandatory. Do not close the kata or claim full completion with unresolved schema/reference or cloud managed browser coverage. 2. All code tasks require recorded behavioral red, green, refactor and affected-test evidence. Synthetic files/commands are intentional fixtures; production preparation, grants, ownership, terminal transport and browser lifecycle are real. No static prose/config hash tests qualify. 3. Periodic broad gate: record the execution-start commit and most recent passing broad-gate commit. At every passed task/specification-plus-quality-review boundary, count non-plan commits since execution start or the last broad gate (whichever is later). If the count is at least 10, run `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed environment passed-task periodic acceptance" pnpm run test` on that committed boundary before starting another task; record HEAD/result and reset the count only after a pass. Plan/report-only commits do not increment it. This rule preserves the-usual periodic gate and does not replace final acceptance. -4. After all seven tasks' commits and specification-plus-quality reviews, finish the worktree clean and record the final committed HEAD. On that exact HEAD, after Task 7, run the feature-enabled server build and `checkpoints`, `managed_checkpoint_send_barrier`, `managed_runtime` and `managed_provider` tests, plus `managed_runtime_api::stop_contract_tests` with a nonzero count. Run the named SDK UUID replay/persisted-transcript and OpenCode prompt/compact-queue `messageID` persistence commands from Task 7; record their real executed/passed counts and candidate HEAD. Default Cargo/CI does not enable `managed-runtime-v1`, so default-suite success is insufficient for these managed paths. Preserve actual paired configured-cloud environment/checkpoint browser receipts after their final affecting source changes. Then run the separate required no-write typecheck `pnpm run typecheck:client`, followed by the final coordinated broad gate `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed provider environment and recovery final committed HEAD acceptance" pnpm run test`. These commands are on the final committed candidate, not a precommit Task 6 or Task 7 tree; preserve separately needed focused managed-feature, integration, browser and sandbox evidence from the tasks. Record command/results and verified HEAD, then push the feature branch with required prepush checks. A subsequent source/review fix requires a new focused commit and appropriate renewed verification of that changed final HEAD; do not rerun an unchanged broad gate merely because a task boundary passed. Stop before PR creation until explicit approval. Do not deploy/restart the live server while finishing the branch. +4. After all seven tasks' commits and specification-plus-quality reviews, finish the worktree clean and record the final committed HEAD. On that exact HEAD, after Task 7, run the feature-enabled server build and `checkpoints` tests, the named durable-dispatch-fence server test, `managed_runtime` and `managed_provider` tests, the named WebSocket send-barrier test, and `managed_runtime_api::stop_contract_tests` with a nonzero count. Run the named SDK UUID replay/persisted-transcript and OpenCode prompt/compact-queue `messageID` persistence commands from Task 7; record their real executed/passed counts and candidate HEAD. Default Cargo/CI does not enable `managed-runtime-v1`, so default-suite success is insufficient for these managed paths. Preserve actual paired configured-cloud environment/checkpoint browser receipts after their final affecting source changes. Then run the separate required no-write typecheck `pnpm run typecheck:client`, followed by the final coordinated broad gate `GCLOUD_ROBOT_REQUIRE=1 FRESHELL_TEST_SUMMARY="managed provider environment and recovery final committed HEAD acceptance" pnpm run test`. These commands are on the final committed candidate, not a precommit Task 6 or Task 7 tree; preserve separately needed focused managed-feature, integration, browser and sandbox evidence from the tasks. Record command/results and verified HEAD, then push the feature branch with required prepush checks. A subsequent source/review fix requires a new focused commit and appropriate renewed verification of that changed final HEAD; do not rerun an unchanged broad gate merely because a task boundary passed. Stop before PR creation until explicit approval. Do not deploy/restart the live server while finishing the branch. 5. Once separately approved, follow the managed rollout runbook, preserve registry/runtime/image/secrets roots, and recheck original OpenCode failure via Freshell MCP. Assert usable startup and existing configuration without reading/logging secret contents. This operational residual is accepted pending approval, whereas missing implementation or configured-cloud test evidence is not. ## Dispatcher self-review From 8dffca3e6ea3f412df100a387c1d7b4f416d666f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 10:33:08 -0700 Subject: [PATCH 332/493] fix(testing): package Cloud Vitest dependencies --- docker/cloud-run/Dockerfile | 3 +++ scripts/testing/managed-provider-environment-target.ts | 2 +- test/{integration/server => helpers}/provider-parity-onecli.ts | 0 test/integration/server/fresh-agent-parity.test.ts | 2 +- test/integration/server/managed-provider-parity.test.ts | 2 +- 5 files changed, 6 insertions(+), 3 deletions(-) rename test/{integration/server => helpers}/provider-parity-onecli.ts (100%) diff --git a/docker/cloud-run/Dockerfile b/docker/cloud-run/Dockerfile index c98ba3ea5..5df621a5a 100644 --- a/docker/cloud-run/Dockerfile +++ b/docker/cloud-run/Dockerfile @@ -110,6 +110,9 @@ COPY --from=rust-builder /usr/local/rustup /usr/local/rustup COPY --from=node-builder /app/dist/client ./dist/client COPY --from=node-builder /app/dist/tools ./dist/tools COPY --from=node-builder /app/node_modules ./node_modules +# The pinned Claude contract exercises the isolated sidecar package directly. +# Its node_modules links back to the root pnpm store copied above. +COPY --from=node-builder /app/crates/freshell-claude-sidecar/node_modules ./crates/freshell-claude-sidecar/node_modules # Fail the image build if the copied test toolchain cannot execute in this # Debian runtime. A container-level test checks the configured runtime user. diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 3e3cee92a..eda721805 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -11,7 +11,7 @@ import { WS_PROTOCOL_VERSION, type ClientMessage } from '../../shared/ws-protoco import { ManagedRuntimeBrowserRig, type ManagedRuntimeView } from '../../test/e2e-browser/helpers/managed-runtime.js' import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../test/helpers/provider-parity-broker.js' import { managedEnvironmentOpencodeSelection } from '../../test/helpers/provider-parity-environment.js' -import { fakeOnecliGrants } from '../../test/integration/server/provider-parity-onecli.js' +import { fakeOnecliGrants } from '../../test/helpers/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentCases } from './managed-environment-create.js' diff --git a/test/integration/server/provider-parity-onecli.ts b/test/helpers/provider-parity-onecli.ts similarity index 100% rename from test/integration/server/provider-parity-onecli.ts rename to test/helpers/provider-parity-onecli.ts diff --git a/test/integration/server/fresh-agent-parity.test.ts b/test/integration/server/fresh-agent-parity.test.ts index 85de31105..b94f1cab8 100644 --- a/test/integration/server/fresh-agent-parity.test.ts +++ b/test/integration/server/fresh-agent-parity.test.ts @@ -7,7 +7,7 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest' import { WS_PROTOCOL_VERSION } from '../../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig } from '../../e2e-browser/helpers/managed-runtime.js' -import { exposeGrantsToHarness, fakeOnecliGrants, withMissingGrant } from './provider-parity-onecli.js' +import { exposeGrantsToHarness, fakeOnecliGrants, withMissingGrant } from '../../helpers/provider-parity-onecli.js' const root = path.resolve(import.meta.dirname, '../../..') const providers = [ diff --git a/test/integration/server/managed-provider-parity.test.ts b/test/integration/server/managed-provider-parity.test.ts index 1a8ab3c13..3544c98db 100644 --- a/test/integration/server/managed-provider-parity.test.ts +++ b/test/integration/server/managed-provider-parity.test.ts @@ -9,7 +9,7 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest' import { WS_PROTOCOL_VERSION } from '../../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig } from '../../e2e-browser/helpers/managed-runtime.js' -import { exposeGrantsToHarness, fakeOnecliGrants, withMissingGrant } from './provider-parity-onecli.js' +import { exposeGrantsToHarness, fakeOnecliGrants, withMissingGrant } from '../../helpers/provider-parity-onecli.js' import { makeProviderParityResourcesReadable, prepareProviderParityUserRoots, providerParityCommands, providerParityFailureDiagnostics, writeProviderParityEnvironment } from '../../helpers/provider-parity-environment.js' import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../helpers/provider-parity-broker.js' import type { InitialProviderVolumeAuthorization } from '../../../scripts/testing/runtime-test-broker.js' From ab430688320e7ac54b6ac91dbc03185b939c0bb1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 10:46:35 -0700 Subject: [PATCH 333/493] fix(checkpoints): satisfy workspace clippy --- crates/freshell-server/src/checkpoints.rs | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 67bde06cb..3156ab589 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -313,13 +313,13 @@ impl CheckpointCaptureControl { std::io::Error::other("checkpoint child process has no process group id") })?; groups.insert(group); - return Ok(CheckpointChild { + Ok(CheckpointChild { child: Some(child), control: Some(Arc::clone(self)), process_group: Some(group), #[cfg(windows)] job: None, - }); + }) } #[cfg(windows)] { @@ -2639,8 +2639,7 @@ impl CheckpointFailure { if let Some(after_prefix) = detail.strip_prefix("Git operation ") { if let Some((name, after_name)) = after_prefix.split_once(" failed") { operation = name.to_string(); - cause = - after_name.trim_start_matches(|character| character == ' ' || character == ':'); + cause = after_name.trim_start_matches([' ', ':']); } } else if detail.starts_with("git index update failed") { operation = "git-index-update".to_string(); @@ -4901,7 +4900,7 @@ mod tests { .await; assert_eq!(dispatches.load(Ordering::SeqCst), 1); - let key = format!("\"opencode\":\"freshopencode\":session-1:logical-attempt-1"); + let key = "\"opencode\":\"freshopencode\":session-1:logical-attempt-1".to_string(); let record = service.find_outcome(&key).await.unwrap().unwrap(); assert!(record.dispatch_started); assert_eq!(record.submitted_turn_id.as_deref(), Some("provider-turn-1")); @@ -5875,9 +5874,7 @@ mod tests { .as_str() .unwrap() .contains("message will still be sent")); - let key = format!( - "\"opencode\":\"freshopencode\":session-quarantine-settlement:attempt-quarantine-settlement" - ); + let key = "\"opencode\":\"freshopencode\":session-quarantine-settlement:attempt-quarantine-settlement".to_string(); let saved = service.find_outcome(&key).await.unwrap().unwrap(); assert!(saved.capture_settled); assert!(saved.dispatch_started); From ef67af089618099d8d9b5a6d64c97559d2e32c83 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 11:47:04 -0700 Subject: [PATCH 334/493] fix(protocol): freeze checkpoint outcome schemas --- crates/freshell-protocol/tests/inventory.rs | 12 +- port/contract/managed-runtime.schema.json | 57 ++++ port/contract/ws-message-inventory.json | 3 +- port/contract/ws-protocol.schema.json | 297 ++++++++++++++++++- port/contract/ws-server-messages.schema.json | 266 ++++++++++++++++- test/unit/port/ws-contract-freeze.test.ts | 1 + 6 files changed, 626 insertions(+), 10 deletions(-) diff --git a/crates/freshell-protocol/tests/inventory.rs b/crates/freshell-protocol/tests/inventory.rs index c628732a1..5563e1b5e 100644 --- a/crates/freshell-protocol/tests/inventory.rs +++ b/crates/freshell-protocol/tests/inventory.rs @@ -48,12 +48,12 @@ fn server_types_match_inventory_exactly() { let inv = inventory(); assert_eq!( inv["serverToClient"]["count"].as_u64(), - Some(69), - "inventory declares 69 server→client types" + Some(70), + "inventory declares 70 server→client types" ); let expected = json_type_set(&inv["serverToClient"]["types"]); let actual: BTreeSet<String> = SERVER_MESSAGE_TYPES.iter().map(|s| s.to_string()).collect(); - assert_eq!(actual.len(), 69, "crate declares 69 server types (no dups)"); + assert_eq!(actual.len(), 70, "crate declares 70 server types (no dups)"); assert_eq!( actual, expected, "SERVER_MESSAGE_TYPES must equal the frozen inventory (no missing/extra)" @@ -61,14 +61,14 @@ fn server_types_match_inventory_exactly() { } #[test] -fn combined_surface_is_111() { +fn combined_surface_is_112() { let all = all_message_types(); - assert_eq!(all.len(), 111, "42 client + 69 server = 111 discriminants"); + assert_eq!(all.len(), 112, "42 client + 70 server = 112 discriminants"); // sorted + unique let unique: BTreeSet<&str> = all.iter().copied().collect(); assert_eq!( unique.len(), - 111, + 112, "no discriminant collides across directions" ); } diff --git a/port/contract/managed-runtime.schema.json b/port/contract/managed-runtime.schema.json index 5e3699c57..37559f203 100644 --- a/port/contract/managed-runtime.schema.json +++ b/port/contract/managed-runtime.schema.json @@ -438,6 +438,18 @@ "minimum": 0, "type": "integer" }, + "freshAgentCreateRequestId": { + "type": "string" + }, + "freshAgentRuntimeVariant": { + "type": "string" + }, + "freshAgentSessionId": { + "type": "string" + }, + "freshAgentSessionType": { + "type": "string" + }, "hostBootId": { "type": "string" }, @@ -1372,6 +1384,15 @@ ], "type": "string" }, + "runtimeStatus": { + "enum": [ + "running", + "recovering", + "blocked", + "exited" + ], + "type": "string" + }, "soulId": { "maxLength": 160, "minLength": 1, @@ -1989,6 +2010,18 @@ "minimum": 0, "type": "integer" }, + "freshAgentCreateRequestId": { + "type": "string" + }, + "freshAgentRuntimeVariant": { + "type": "string" + }, + "freshAgentSessionId": { + "type": "string" + }, + "freshAgentSessionType": { + "type": "string" + }, "hostBootId": { "type": "string" }, @@ -2324,6 +2357,18 @@ "minimum": 0, "type": "integer" }, + "freshAgentCreateRequestId": { + "type": "string" + }, + "freshAgentRuntimeVariant": { + "type": "string" + }, + "freshAgentSessionId": { + "type": "string" + }, + "freshAgentSessionType": { + "type": "string" + }, "hostBootId": { "type": "string" }, @@ -2631,6 +2676,18 @@ "minimum": 0, "type": "integer" }, + "freshAgentCreateRequestId": { + "type": "string" + }, + "freshAgentRuntimeVariant": { + "type": "string" + }, + "freshAgentSessionId": { + "type": "string" + }, + "freshAgentSessionType": { + "type": "string" + }, "hostBootId": { "type": "string" }, diff --git a/port/contract/ws-message-inventory.json b/port/contract/ws-message-inventory.json index 4fdf6fac4..51d04eb16 100644 --- a/port/contract/ws-message-inventory.json +++ b/port/contract/ws-message-inventory.json @@ -49,7 +49,7 @@ "description": "Auto-generated from shared/ws-protocol.ts. DO NOT EDIT BY HAND. Regenerate with `pnpm run contract:generate`. `type` discriminants for every message in each direction, resolved from the ClientMessage and ServerMessage union types via the TypeScript type checker.", "generator": "port/contract/generate-ws-contract.ts", "serverToClient": { - "count": 69, + "count": 70, "types": [ "amplifier.activity.list.response", "amplifier.activity.updated", @@ -69,6 +69,7 @@ "extension.server.starting", "extension.server.stopped", "extensions.registry", + "freshAgent.checkpoint.outcome", "freshAgent.create.failed", "freshAgent.created", "freshAgent.event", diff --git a/port/contract/ws-protocol.schema.json b/port/contract/ws-protocol.schema.json index 3d3b05087..2b72ca5ad 100644 --- a/port/contract/ws-protocol.schema.json +++ b/port/contract/ws-protocol.schema.json @@ -2,7 +2,7 @@ "description": "Auto-generated from shared/ws-protocol.ts. DO NOT EDIT BY HAND. Regenerate with `pnpm run contract:generate`. Each entry in `schemas` is a self-contained JSON Schema for one exported Zod schema. The wire contract is frozen for the Rust port — changing it is out of scope.", "generator": "port/contract/generate-ws-contract.ts", "jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema", - "schemaCount": 85, + "schemaCount": 88, "schemas": { "AmplifierActivityListResponseSchema": { "$schema": "https://json-schema.org/draft/2020-12/schema", @@ -2055,6 +2055,43 @@ { "additionalProperties": false, "properties": { + "checkpointIntent": { + "additionalProperties": false, + "properties": { + "logicalAttemptId": { + "maxLength": 128, + "minLength": 1, + "type": "string" + }, + "workspace": { + "additionalProperties": false, + "properties": { + "path": { + "maxLength": 4096, + "minLength": 1, + "type": "string" + }, + "provenance": { + "enum": [ + "directory-picker", + "resolved-project", + "legacy-checkpoint" + ], + "type": "string" + } + }, + "required": [ + "path", + "provenance" + ], + "type": "object" + } + }, + "required": [ + "logicalAttemptId" + ], + "type": "object" + }, "cwd": { "type": "string" }, @@ -3285,6 +3322,102 @@ ], "type": "object" }, + "FreshAgentCheckpointOutcomeSchema": { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "additionalProperties": false, + "properties": { + "checkpointId": { + "minLength": 1, + "type": "string" + }, + "code": { + "type": "string" + }, + "displayTurnId": { + "minLength": 1, + "type": "string" + }, + "logicalAttemptId": { + "minLength": 1, + "type": "string" + }, + "nativeTurnId": { + "minLength": 1, + "type": "string" + }, + "provider": { + "enum": [ + "claude", + "codex", + "opencode" + ], + "type": "string" + }, + "reason": { + "type": "string" + }, + "requestId": { + "minLength": 1, + "type": "string" + }, + "sessionId": { + "minLength": 1, + "type": "string" + }, + "sessionType": { + "enum": [ + "freshclaude", + "freshcodex", + "kilroy", + "freshopencode" + ], + "type": "string" + }, + "status": { + "enum": [ + "available", + "unavailable" + ], + "type": "string" + }, + "type": { + "const": "freshAgent.checkpoint.outcome", + "type": "string" + }, + "workspace": { + "additionalProperties": false, + "properties": { + "path": { + "minLength": 1, + "type": "string" + }, + "provenance": { + "enum": [ + "directory-picker", + "resolved-project", + "legacy-checkpoint" + ], + "type": "string" + } + }, + "required": [ + "path", + "provenance" + ], + "type": "object" + } + }, + "required": [ + "type", + "logicalAttemptId", + "requestId", + "provider", + "sessionId", + "sessionType", + "status" + ], + "type": "object" + }, "FreshAgentClientMessageSchema": { "$schema": "https://json-schema.org/draft/2020-12/schema", "oneOf": [ @@ -3505,6 +3638,43 @@ { "additionalProperties": false, "properties": { + "checkpointIntent": { + "additionalProperties": false, + "properties": { + "logicalAttemptId": { + "maxLength": 128, + "minLength": 1, + "type": "string" + }, + "workspace": { + "additionalProperties": false, + "properties": { + "path": { + "maxLength": 4096, + "minLength": 1, + "type": "string" + }, + "provenance": { + "enum": [ + "directory-picker", + "resolved-project", + "legacy-checkpoint" + ], + "type": "string" + } + }, + "required": [ + "path", + "provenance" + ], + "type": "object" + } + }, + "required": [ + "logicalAttemptId" + ], + "type": "object" + }, "cwd": { "type": "string" }, @@ -4660,6 +4830,94 @@ ], "type": "object" }, + "FreshAgentRecoveryStopSchema": { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "additionalProperties": false, + "properties": { + "observedEpoch": { + "maximum": 9007199254740991, + "minimum": 0, + "type": "integer" + }, + "observedGeneration": { + "maximum": 9007199254740991, + "minimum": 0, + "type": "integer" + }, + "provider": { + "const": "codex", + "type": "string" + }, + "requestId": { + "minLength": 1, + "type": "string" + }, + "sessionId": { + "minLength": 1, + "type": "string" + }, + "sessionType": { + "const": "freshcodex", + "type": "string" + }, + "type": { + "const": "freshAgent.recovery.stop", + "type": "string" + } + }, + "required": [ + "type", + "requestId", + "sessionId", + "sessionType", + "provider" + ], + "type": "object" + }, + "FreshAgentRecoveryStoppedSchema": { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "additionalProperties": false, + "properties": { + "code": { + "type": "string" + }, + "message": { + "type": "string" + }, + "provider": { + "const": "codex", + "type": "string" + }, + "requestId": { + "minLength": 1, + "type": "string" + }, + "sessionId": { + "minLength": 1, + "type": "string" + }, + "sessionType": { + "const": "freshcodex", + "type": "string" + }, + "success": { + "type": "boolean" + }, + "type": { + "const": "freshAgent.recovery.stopped", + "type": "string" + } + }, + "required": [ + "type", + "requestId", + "sessionId", + "sessionType", + "provider", + "success" + ], + "type": "object" + }, "FreshAgentRedoSchema": { "$schema": "https://json-schema.org/draft/2020-12/schema", "additionalProperties": false, @@ -4731,6 +4989,43 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "additionalProperties": false, "properties": { + "checkpointIntent": { + "additionalProperties": false, + "properties": { + "logicalAttemptId": { + "maxLength": 128, + "minLength": 1, + "type": "string" + }, + "workspace": { + "additionalProperties": false, + "properties": { + "path": { + "maxLength": 4096, + "minLength": 1, + "type": "string" + }, + "provenance": { + "enum": [ + "directory-picker", + "resolved-project", + "legacy-checkpoint" + ], + "type": "string" + } + }, + "required": [ + "path", + "provenance" + ], + "type": "object" + } + }, + "required": [ + "logicalAttemptId" + ], + "type": "object" + }, "cwd": { "type": "string" }, diff --git a/port/contract/ws-server-messages.schema.json b/port/contract/ws-server-messages.schema.json index 775fed3e4..53779693a 100644 --- a/port/contract/ws-server-messages.schema.json +++ b/port/contract/ws-server-messages.schema.json @@ -3,7 +3,7 @@ "description": "Auto-generated from shared/ws-protocol.ts. DO NOT EDIT BY HAND. Regenerate with `pnpm run contract:generate`. Each entry is a JSON Schema for one server→client message shape, keyed by its `type` discriminant and synthesized from the ServerMessage union via the TypeScript type checker. Opaque values (unknown / index signatures / any) are intentionally permissive — see nondeterministic-fields.md. The wire contract is frozen for the Rust port.", "generator": "port/contract/generate-ws-contract.ts", "jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema", - "messageCount": 69, + "messageCount": 70, "messages": { "amplifier.activity.list.response": { "$schema": "https://json-schema.org/draft/2020-12/schema", @@ -615,6 +615,50 @@ ], "type": "string" }, + "providerPreparationFailure": { + "additionalProperties": false, + "properties": { + "cause": { + "enum": [ + "unknown", + "not_found", + "permission_denied", + "cyclic" + ], + "type": "string" + }, + "code": { + "const": "SELECTED_REFERENCE_UNAVAILABLE", + "type": "string" + }, + "declaringConfig": { + "type": "string" + }, + "operation": { + "const": "resolve", + "type": "string" + }, + "selectedPath": { + "type": "string" + }, + "stage": { + "const": "configuration", + "type": "string" + }, + "workspace": { + "type": "string" + } + }, + "required": [ + "cause", + "code", + "declaringConfig", + "operation", + "stage", + "workspace" + ], + "type": "object" + }, "requestId": { "type": "string" }, @@ -887,6 +931,95 @@ ], "type": "object" }, + "freshAgent.checkpoint.outcome": { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "additionalProperties": false, + "properties": { + "checkpointId": { + "type": "string" + }, + "code": { + "type": "string" + }, + "displayTurnId": { + "type": "string" + }, + "logicalAttemptId": { + "type": "string" + }, + "nativeTurnId": { + "type": "string" + }, + "provider": { + "enum": [ + "codex", + "claude", + "opencode" + ], + "type": "string" + }, + "reason": { + "type": "string" + }, + "requestId": { + "type": "string" + }, + "sessionId": { + "type": "string" + }, + "sessionType": { + "enum": [ + "freshclaude", + "freshcodex", + "kilroy", + "freshopencode" + ], + "type": "string" + }, + "status": { + "enum": [ + "available", + "unavailable" + ], + "type": "string" + }, + "type": { + "const": "freshAgent.checkpoint.outcome", + "type": "string" + }, + "workspace": { + "additionalProperties": false, + "properties": { + "path": { + "type": "string" + }, + "provenance": { + "enum": [ + "directory-picker", + "resolved-project", + "legacy-checkpoint" + ], + "type": "string" + } + }, + "required": [ + "path", + "provenance" + ], + "type": "object" + } + }, + "required": [ + "logicalAttemptId", + "provider", + "requestId", + "sessionId", + "sessionType", + "status", + "type" + ], + "type": "object" + }, "freshAgent.create.failed": { "$schema": "https://json-schema.org/draft/2020-12/schema", "additionalProperties": false, @@ -4554,6 +4687,12 @@ "exitCode": { "type": "number" }, + "exitCodeAvailable": { + "type": "boolean" + }, + "managedRetained": { + "type": "boolean" + }, "terminalId": { "type": "string" }, @@ -4835,6 +4974,9 @@ "description": { "type": "string" }, + "incarnationId": { + "type": "string" + }, "lastActivityAt": { "type": "number" }, @@ -4888,13 +5030,72 @@ } ] }, + "recoverySummary": { + "additionalProperties": false, + "properties": { + "allocationState": { + "enum": [ + "allocated", + "materializing", + "verified_durable", + "degraded" + ], + "type": "string" + }, + "attemptId": { + "type": "string" + }, + "desiredState": { + "enum": [ + "running", + "stopped" + ], + "type": "string" + }, + "durabilityState": { + "enum": [ + "unknown", + "live_only", + "resume_captured", + "checkpoint_captured", + "intrinsically_non_resumable" + ], + "type": "string" + }, + "incidentId": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "recoveryState": { + "enum": [ + "live", + "blocked", + "recovering", + "stopped", + "lost" + ], + "type": "string" + } + }, + "required": [ + "allocationState", + "desiredState", + "durabilityState", + "recoveryState" + ], + "type": "object" + }, "resumeTargetIsSubagent": { "type": "boolean" }, "runtimeStatus": { "enum": [ + "blocked", "running", - "recovering" + "recovering", + "exited" ], "type": "string" }, @@ -5000,6 +5201,9 @@ ], "type": "object" }, + "soulId": { + "type": "string" + }, "status": { "enum": [ "running", @@ -5486,11 +5690,69 @@ "reason": { "type": "string" }, + "recoverySummary": { + "additionalProperties": false, + "properties": { + "allocationState": { + "enum": [ + "allocated", + "materializing", + "verified_durable", + "degraded" + ], + "type": "string" + }, + "attemptId": { + "type": "string" + }, + "desiredState": { + "enum": [ + "running", + "stopped" + ], + "type": "string" + }, + "durabilityState": { + "enum": [ + "unknown", + "live_only", + "resume_captured", + "checkpoint_captured", + "intrinsically_non_resumable" + ], + "type": "string" + }, + "incidentId": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "recoveryState": { + "enum": [ + "live", + "blocked", + "recovering", + "stopped", + "lost" + ], + "type": "string" + } + }, + "required": [ + "allocationState", + "desiredState", + "durabilityState", + "recoveryState" + ], + "type": "object" + }, "resumeCycles": { "type": "number" }, "status": { "enum": [ + "blocked", "running", "recovering", "exited" diff --git a/test/unit/port/ws-contract-freeze.test.ts b/test/unit/port/ws-contract-freeze.test.ts index 0d1337c06..232a1432e 100644 --- a/test/unit/port/ws-contract-freeze.test.ts +++ b/test/unit/port/ws-contract-freeze.test.ts @@ -25,6 +25,7 @@ const ZOD_BACKED_SERVER_MESSAGES = [ 'claude.activity.updated', 'codex.activity.list.response', 'codex.activity.updated', + 'freshAgent.checkpoint.outcome', 'hoststats.refresh.response', 'hoststats.snapshot', 'opencode.activity.list.response', From 9dd15704f0f89ad21adbb6263ef006157e3ac660 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 11:47:08 -0700 Subject: [PATCH 335/493] test(sandbox): use executable temp for process fixtures --- crates/freshell-server/src/checkpoints.rs | 12 +++++++++++- .../freshagent-wireshape-differential.test.ts | 15 +++++++++++---- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 3156ab589..d6d10857f 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -7011,7 +7011,17 @@ mod tests { let cwd_dir = tempfile::tempdir().unwrap(); let started = home_dir.path().join("filter-descendant-started"); let late_write = home_dir.path().join("filter-descendant-late-write"); - let filter = home_dir.path().join("blocking-clean-filter.sh"); + // Runtime-suite sandboxes mount `/tmp` without executable permission. + // Keep this deliberately executable Git filter in the sandbox-owned + // executable temp volume when one is configured. + let executable_tmp = std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") + .map(PathBuf::from) + .unwrap_or_else(|| home_dir.path().to_path_buf()); + let filter_dir = tempfile::Builder::new() + .prefix("freshell-checkpoint-filter-") + .tempdir_in(executable_tmp) + .unwrap(); + let filter = filter_dir.path().join("blocking-clean-filter.sh"); let filter_contents = format!( "#!/bin/sh\n( trap '' TERM; printf started > '{}'; sleep 1; printf late > '{}' ) &\nwait\ncat\n", started.display(), diff --git a/test/unit/port/oracle/freshagent-wireshape-differential.test.ts b/test/unit/port/oracle/freshagent-wireshape-differential.test.ts index 3a8050a21..b8a69afb2 100644 --- a/test/unit/port/oracle/freshagent-wireshape-differential.test.ts +++ b/test/unit/port/oracle/freshagent-wireshape-differential.test.ts @@ -120,6 +120,7 @@ async function captureCodexFlow( ): Promise<{ handle: ExternalServerHandle; capture: FreshAgentCapture }> { const handle = await startExternalServer({ provider: 'freshagent-wireshape', + verbose: process.env.FRESHELL_ORACLE_VERBOSE === '1', env: { CODEX_CMD: fakeCodexPath, FAKE_CODEX_APP_SERVER_BEHAVIOR: codexBehaviorEnv(), @@ -154,12 +155,17 @@ async function captureCodexFlow( sessionType: 'freshcodex', provider: 'codex', }) - const created = await client.waitForServerMessage( - (m) => m.type === 'freshAgent.created' + const createOutcome = await client.waitForServerMessage( + (m) => (m.type === 'freshAgent.created' || m.type === 'freshAgent.create.failed') && (m.parsed as { requestId?: unknown }).requestId === 'wireshape-create-1', 20_000, - 'freshAgent.created', + 'freshAgent.created or freshAgent.create.failed', ) + expect( + createOutcome.type, + `fresh-agent create failed: ${JSON.stringify(createOutcome.parsed)}`, + ).toBe('freshAgent.created') + const created = createOutcome const sessionId = (created.parsed as { sessionId: string }).sessionId expect(sessionId, 'freshAgent.created must carry a non-empty sessionId').toBeTruthy() @@ -267,7 +273,8 @@ describe('Fresh-agent wire-shape determinism (Rust, codex)', () => { it( 'emits byte-identical freshAgent envelopes/inner-event shapes on create+send+attach', async () => { - sharedRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-wireshape-differential-')) + const executableTempDir = process.env.FRESHELL_TEST_EXECUTABLE_TMP_DIR ?? os.tmpdir() + sharedRoot = await fs.mkdtemp(path.join(executableTempDir, 'freshell-wireshape-differential-')) const fakeCodexPath = await installFakeCodexAppServer(path.join(sharedRoot, 'bin')) const firstRun = await captureCodexFlow(fakeCodexPath) From 3335ed9effc1acbc8dfabc61f9f4dafab7df355a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 16:48:36 -0700 Subject: [PATCH 336/493] fix: include enabled OpenCode MCP command resources --- .../opencode_resources.rs | 180 ++++++++++++++++- .../managed_provider_references/sources.rs | 182 ++++++++++++++++++ ...2026-10-05-managed-provider-environment.md | 2 +- .../testing/managed-provider-launch-broker.ts | 18 +- .../managed-provider-launch-sandbox.py | 17 ++ 5 files changed, 389 insertions(+), 10 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index cc5788b73..351e8cd32 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -3,6 +3,8 @@ use super::sources::PlannedSourceOccurrence; use std::path::{Path, PathBuf}; +const FRESHELL_IMAGE_MCP_SERVER_PATH: &str = "/opt/freshell-mcp/server.js"; + #[derive(Clone, Copy)] enum Token { String(usize, usize), @@ -93,7 +95,7 @@ fn fields(raw: &str) -> Result<Vec<(&'static str, String)>, &'static str> { } // Read only member spans: selected nested schemas never deserialize or rewrite -// an entire source document, nor interpret command arrays as file selections. +// an entire source document, nor interpret unrelated command arguments as paths. type ObjectMember = (std::ops::Range<usize>, std::ops::Range<usize>); fn object_members(raw: &str, start: usize) -> Result<Vec<ObjectMember>, &'static str> { @@ -203,12 +205,12 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' for (name, range) in object_members(mcp, 0)?.into_iter().rev() { let name = serde_json::from_str::<String>(&mcp[name]) .map_err(|_| "invalid selected MCP server name")?; - if names.insert(name) { - winners.push(range); + if names.insert(name.clone()) { + winners.push((name, range)); } } drop(names); - for range in winners.into_iter().rev() { + for (name, range) in winners.into_iter().rev() { let entry = &mcp[range]; let entries = object_members(entry, 0)?; if member(entry, &entries, "type") @@ -219,16 +221,128 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' { continue; } + // Keep each server's cwd adjacent to its command references so a + // different server's cwd cannot accidentally satisfy a missing path. + result.push(("OpenCode mcp.entry", String::new())); if let Some(value) = member(entry, &entries, "cwd") { if let Some(path) = selected_string(value)? { result.push(("OpenCode mcp.cwd", path)); } } + if let Some(value) = member(entry, &entries, "command") { + let command = string_array(value)?; + // Freshell's exact server command is supplied by the managed + // runtime image. Its /opt path is not a host resource to copy. + let image_command = name == "freshell" + && command.len() == 2 + && command[0] == "node" + && command[1] == FRESHELL_IMAGE_MCP_SERVER_PATH; + if !image_command { + for (index, argument) in command.iter().enumerate() { + if (index == 0 && is_explicit_command_path(argument)) + || is_script_path(argument) + { + result.push(("OpenCode mcp.command", argument.clone())); + } + } + let inline_code = command.iter().any(|argument| { + matches!(argument.as_str(), "-e" | "-c" | "--eval" | "--print") + }); + let bare_script_arguments = command + .iter() + .skip(1) + .filter(|argument| !argument.starts_with('-') && !argument.is_empty()) + .collect::<Vec<_>>(); + if command_uses_script_runtime(command.first().map(String::as_str)) + && !inline_code + && bare_script_arguments.len() == 1 + && !is_script_path(bare_script_arguments[0]) + { + result.push(("OpenCode mcp.command", bare_script_arguments[0].clone())); + } + } + } + } + } + Ok(result) +} + +fn string_array(raw: &str) -> Result<Vec<String>, &'static str> { + let mut cursor = Cursor { raw, at: 0 }; + if !matches!(cursor.next(), Some(Token::Punctuation(b'['))) { + return Ok(Vec::new()); + } + let mut depth = 1usize; + let mut result = Vec::new(); + while depth != 0 { + match cursor.next().ok_or("incomplete selected command array")? { + Token::Punctuation(b'[' | b'{') => { + depth += 1; + if depth > 64 { + return Err("selected command exceeds shared depth profile"); + } + } + Token::Punctuation(b']' | b'}') => depth -= 1, + Token::String(start, end) if depth == 1 => { + if result.len() >= 100_000 || end - start > 64 * 1024 { + return Err("configured resource selection profile exceeded"); + } + result.push( + serde_json::from_str(&cursor.raw[start..end]) + .map_err(|_| "invalid selected command argument")?, + ); + } + _ => {} } } Ok(result) } +fn is_explicit_command_path(argument: &str) -> bool { + !argument.starts_with('-') + && (Path::new(argument).is_absolute() + || argument.starts_with("./") + || argument.starts_with("../")) +} + +fn is_script_path(argument: &str) -> bool { + if argument.starts_with('-') || argument.starts_with('@') { + return false; + } + Path::new(argument).extension().is_some_and(|extension| { + matches!( + extension.to_string_lossy().to_ascii_lowercase().as_str(), + "js" | "mjs" | "cjs" | "ts" | "mts" | "cts" | "py" | "sh" | "rb" + ) + }) +} + +fn command_uses_script_runtime(executable: Option<&str>) -> bool { + let Some(executable) = executable.and_then(|executable| Path::new(executable).file_name()) + else { + return false; + }; + let executable = executable.to_string_lossy().to_ascii_lowercase(); + let executable = executable.strip_suffix(".exe").unwrap_or(&executable); + matches!( + executable, + "node" + | "nodejs" + | "bun" + | "deno" + | "tsx" + | "ts-node" + | "python" + | "python3" + | "ruby" + | "bash" + | "sh" + | "zsh" + | "pwsh" + | "powershell" + ) +} + fn resource_array( cursor: &mut Cursor<'_>, field: &'static str, @@ -332,7 +446,12 @@ pub fn select( nested_fields(raw, source.source.lane == super::sources::SourceLane::Tui) .map_err(|cause| fail("OpenCode resource", cause))?, ); + let mut current_mcp_working_dir: Option<PathBuf> = None; for (field, spelling) in selections { + if field == "OpenCode mcp.entry" { + current_mcp_working_dir = None; + continue; + } // Private file-token payloads already use the accepted grant plan. if spelling.contains("{file:") { continue; @@ -392,7 +511,7 @@ pub fn select( } else { continue; } // Native package identity remains native. - } else if field == "OpenCode mcp.cwd" { + } else if field == "OpenCode mcp.cwd" || field == "OpenCode mcp.command" { // Native path.resolve(instance cwd, raw): no HOME or URL expansion. PathBuf::from(&spelling) } else { @@ -463,6 +582,17 @@ pub fn select( .join(path); } else if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { path = source.instance_cwd.join(path); + } else if field == "OpenCode mcp.command" { + path = if let Some(working_dir) = ¤t_mcp_working_dir { + working_dir.join(path) + } else { + source + .instance_cwd + .ancestors() + .map(|base| base.join(&path)) + .find(|candidate| std::fs::symlink_metadata(candidate).is_ok()) + .unwrap_or_else(|| source.instance_cwd.join(path)) + }; } else { path = source .instance_cwd @@ -474,7 +604,10 @@ pub fn select( } if matches!( field, - "OpenCode skills.paths" | "OpenCode mcp.cwd" | "OpenCode attention.sounds" + "OpenCode skills.paths" + | "OpenCode mcp.cwd" + | "OpenCode mcp.command" + | "OpenCode attention.sounds" ) { // Native resolution resolution normalizes .. before deciding // whether the selected resource is supplied by the workspace. @@ -489,6 +622,28 @@ pub fn select( insert(path, field)?; continue; } + if field == "OpenCode mcp.command" { + if !canonical.is_file() { + return Err(fail( + field, + "selected command resource is not a regular file", + )); + } + // The server's selected cwd is already copied as a directory. + // Avoid an overlapping file/package mount for a script it contains; + // besides being redundant, that shape cannot be authorized by the + // closed launch fixture. Canonical resolution keeps symlinks that + // escape the cwd on the explicit-resource path below. + if current_mcp_working_dir + .as_ref() + .is_some_and(|working_dir| canonical.starts_with(working_dir)) + { + continue; + } + // Retain the exact executable/script path as well as its bounded + // package closure below when it belongs to an installed package. + insert(canonical.clone(), field)?; + } if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { if !canonical.is_dir() { return Err(fail( @@ -496,6 +651,9 @@ pub fn select( "selected directory resource is not a directory", )); } + if field == "OpenCode mcp.cwd" { + current_mcp_working_dir = Some(canonical); + } insert(path, field)?; continue; } @@ -506,7 +664,12 @@ pub fn select( if global.is_some_and(|global| path.starts_with(global.join("plugins"))) { continue; } - let mut scope = if canonical.is_dir() { + let mut scope = if field == "OpenCode mcp.command" { + canonical + .parent() + .ok_or_else(|| fail(field, "command has no containing scope"))? + .to_owned() + } else if canonical.is_dir() { path.clone() } else { path.parent() @@ -525,6 +688,9 @@ pub fn select( } } if scope == home || scope == Path::new("/") { + if field == "OpenCode mcp.command" { + continue; + } return Err(fail(field, "select a bounded plugin or package directory")); } insert(scope.clone(), field)?; diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 0ba8b4057..41a6e5e7a 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1113,4 +1113,186 @@ mod tests { "generated child names cannot shadow user or source-specific auth values" ); } + + #[test] + fn review_r6_local_mcp_command_file_arguments_are_selected() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = owned.path().join("outside-tools"); + let absolute_script = tools.join("absolute-mcp.js"); + let relative_script = workspace.join("scripts/relative-mcp.js"); + let env_script = tools.join("env-mcp.js"); + let symlink_target = tools.join("symlink-target.js"); + let workspace_symlink = workspace.join("scripts/symlinked-mcp.js"); + let cwd_script = workspace.join("mcp-cwd/cwd-mcp.js"); + let extensionless_script = tools.join("extensionless-mcp"); + let missing_disabled_script = tools.join("missing-disabled.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(relative_script.parent().unwrap()).unwrap(); + std::fs::create_dir_all(cwd_script.parent().unwrap()).unwrap(); + for script in [ + &absolute_script, + &relative_script, + &env_script, + &symlink_target, + &cwd_script, + &extensionless_script, + ] { + std::fs::write(script, "process.stdout.write('mcp fixture')\n").unwrap(); + } + std::os::unix::fs::symlink(&symlink_target, &workspace_symlink).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "absolute": { + "type": "local", + "command": ["node", absolute_script], + "enabled": true, + }, + "relative": { + "type": "local", + "command": ["node", "scripts/relative-mcp.js"], + "enabled": true, + }, + "workspace-symlink": { + "type": "local", + "command": ["node", "scripts/symlinked-mcp.js"], + "enabled": true, + }, + "env": { + "type": "local", + "command": ["node", "{env:MCP_SERVER_PATH}"], + "enabled": true, + }, + "cwd": { + "type": "local", + "cwd": "mcp-cwd", + "command": ["node", "cwd-mcp.js"], + "enabled": true, + }, + "extensionless": { + "type": "local", + "command": ["node", extensionless_script], + "enabled": true, + }, + "package": { + "type": "local", + "command": ["npx", "-y", "@example/mcp-server"], + "enabled": true, + }, + "disabled": { + "type": "local", + "command": ["node", missing_disabled_script], + "enabled": false, + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, name| { + (name == "MCP_SERVER_PATH").then(|| env_script.to_string_lossy().into_owned()) + }, + ) + .unwrap(); + let selected = &plan[0].configured_resources; + + for script in [ + &absolute_script, + &relative_script, + &symlink_target, + &env_script, + &extensionless_script, + ] { + assert!( + selected.contains(&script.canonicalize().unwrap()), + "enabled local MCP command file must be available to the managed child: {}", + script.display() + ); + } + assert!( + selected.contains(&cwd_script.parent().unwrap().canonicalize().unwrap()), + "the selected local MCP cwd carries scripts below it" + ); + assert!( + !selected.contains(&cwd_script.canonicalize().unwrap()), + "scripts already carried by the selected cwd do not get overlapping mounts" + ); + assert!(!selected.contains(&missing_disabled_script)); + assert!( + plan[0] + .references + .path_variables + .contains(&"MCP_SERVER_PATH".to_string()), + "command path environment references are resolved through the planned source" + ); + } + + #[test] + fn freshell_image_mcp_command_is_not_selected_as_a_host_resource() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let local_script = external.join("user-mcp.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&external).unwrap(); + std::fs::write(&local_script, "process.stdout.write('user mcp')\n").unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "freshell": { + "type": "local", + "command": ["node", "/opt/freshell-mcp/server.js"], + }, + "user": { + "type": "local", + "command": ["node", local_script], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .expect("the image-provided Freshell MCP is not a host path reference"); + + assert!(plan[0] + .configured_resources + .contains(&local_script.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .iter() + .any(|path| path.starts_with("/opt/freshell-mcp") + || path == Path::new("/opt/package.json"))); + } } diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 9ef088a8b..772c97c6b 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -75,7 +75,7 @@ R means an ordinary selected complete resource/configuration; S means private re | OpenCode file substitution | Native `{file:...}` after the single `{env:...}` pass, including nested path variables and introduced complete tokens | S payloads. Absolute path as selected; `~/` from original HOME; relative from logical declaring parent, inline from instance cwd. Preserve trim/escaping and comment-prefix exclusion, source occurrence and main/TUI auth distinctions. | | OpenCode plugins, resources, directories | Main/TUI `plugin[]` string or `[specifier, options]` local path/file URL; discovered `plugin`/`plugins`; `agent`/`agents`, `command`/`commands`, `mode`/`modes`; `skill`/`skills`, HOME/project `.claude/skills` and `.agents/skills`; selected package metadata/locks, installed `node_modules` | R complete skill/plugin/package/dependency units. Local plugin spec from declaring config parent. Auto resources from each discovered config directory. Preserve installed Bun/Node resolution and writable selected config roots; network reinstall cannot prove transported dependency completeness. | | OpenCode instruction/skill selection | `instructions[]`, global `AGENTS.md`, HOME `.claude/CLAUDE.md`, project `AGENTS.md`/`CLAUDE.md`/`CONTEXT.md`; `skills.paths[]`, `skills.urls[]` | R file/glob instructions: `~/` HOME, absolute at its parent, relative searched upward from instance cwd. Skill paths use instance cwd for relative forms; preserve whole selected directory. URLs stay URLs. Respect native disable flags and first-applicable discovery. | -| OpenCode TUI/commands | Global/explicit `OPENCODE_TUI_CONFIG`, root-first ancestor and `.opencode` `tui.json{,c}`; `attention.sounds.*`; `mcp.<name>.cwd`; MCP/formatter/LSP `command[]` | TUI relative files/sounds/plugin paths use declaring TUI parent; no authEnv. TUI ancestor search has no worktree stop. MCP cwd is workspace-relative R directory. Commands are C, not file schemas; document-level native substitution still applies. | +| OpenCode TUI/commands | Global/explicit `OPENCODE_TUI_CONFIG`, root-first ancestor and `.opencode` `tui.json{,c}`; `attention.sounds.*`; enabled local `mcp.<name>.cwd` and literal executable/script paths from its `command[]`; MCP/formatter/LSP command arrays | TUI relative files/sounds/plugin paths use declaring TUI parent; no authEnv. TUI ancestor search has no worktree stop. MCP cwd is workspace-relative R directory. Preserve command arrays and ordinary args unchanged; select only explicit local executable paths and recognized script operands from enabled local MCP entries. Registry/package identities, flags, remote MCP entries and disabled servers stay native. | | Codex 0.147.0 config layers | `$CODEX_HOME/config.toml`, selected `<name>.config.toml`, cwd `config.toml`, ancestor `.codex/config.toml`; `model_instructions_file`, `experimental_compact_prompt_file`, `model_catalog_json`, `agents.<role>.config_file` | R typed input files resolve from their declaring layer parent with native HOME expansion; recursively loaded role config retains its own source parent. Preserve trust gating and layer precedence; never treat cwd fallback as config origin. | | Codex skills/plugins | `skills.config[].path`; layer `skills`, legacy `$CODEX_HOME/skills`, HOME/cwd-to-project `.agents/skills`; selected `plugins/cache`, `plugins/data`; manifest path forms `skills`, `mcpServers`, `hooks`, `apps` | Path is an enable/disable selector, not a new discovery root. R complete discovered skill trees and selected installed plugin roots. Plugin paths from plugin root; retain inline MCP/hook forms and dependency context. | | Codex commands | `mcp_servers.<name>.command`, `.args`, `.cwd`, `notify[]`, hook commands | Commands/argv are C. MCP cwd follows its pinned `LegacyAppPathString` loader semantics, resolved from the launch working directory when relative, rather than pretending it is a typed per-layer input-file path. Preserve actual native command semantics with fixture execution; do not rewrite arbitrary args. | diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index 384e54fd9..c163904fe 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -92,9 +92,23 @@ const timer=setInterval(()=>{ })+'\n',{mode:0o600}) }) } - } catch { + } catch (error) { clearInterval(timer) - fs.writeFileSync(path.join(root,'broker-failure'),'closed fixture authorization failed',{mode:0o600}) + const cause=error instanceof Error?error.message:String(error) + const failurePath=path.join(root,'broker-failure') + fs.writeFileSync(failurePath,cause,{mode:0o600}) + try { + const detail:Record<string,string>={cause} + if (error && typeof error==='object') { + const rejected=error as {targetPath?:unknown;preparationStageRoot?:unknown} + if (typeof rejected.targetPath==='string') detail.targetPath=diagnosticPath(rejected.targetPath)??'<invalid>' + if (typeof rejected.preparationStageRoot==='string') detail.preparationStageRoot=diagnosticPath(rejected.preparationStageRoot)??'<invalid>' + } + fs.writeFileSync(failurePath,JSON.stringify(detail),{mode:0o600}) + } catch { + // Preserve the fixed cause even if optional path diagnostics cannot be + // serialized; the outer sandbox records the broker's state and logs. + } void broker.close() process.exitCode=1 } diff --git a/scripts/testing/managed-provider-launch-sandbox.py b/scripts/testing/managed-provider-launch-sandbox.py index 7a4901602..3a0dbfd55 100644 --- a/scripts/testing/managed-provider-launch-sandbox.py +++ b/scripts/testing/managed-provider-launch-sandbox.py @@ -228,6 +228,19 @@ def create(name,*arguments): exit_code=int(waited.stdout.strip()) outcome=docker("logs",controller,check=False) receipt["exitCode"]=exit_code + broker_state=docker("container","inspect",broker_name,check=False) + if broker_state.returncode==0: + state=json.loads(broker_state.stdout)[0]["State"] + broker_logs=docker("logs",broker_name,check=False) + receipt["brokerDiagnostics"]={ + "containerStatus":state.get("Status"), + "containerExitCode":state.get("ExitCode"), + "containerError":state.get("Error"), + "logTail":(broker_logs.stdout+broker_logs.stderr)[-4096:], + "handoffFiles":sorted(path.name for path in (root/"handoff").iterdir()), + "brokerFailure":(root/"broker-failure").read_text() if (root/"broker-failure").is_file() else None, + "brokerSocketExists":(root/"broker.sock").exists(), + } if args.provider=="claude": receipt["claudeFlavor"]=args.claude_flavor receipt["claudeSettingsForm"]=args.claude_settings_form @@ -274,6 +287,10 @@ def create(name,*arguments): if not 2<=len(candidates)<=6 or len(authorized)!=len(candidates) or observed!=authorized or len(observers)!=len(candidates) or any(value.get("uid")!=65534 or value.get("gid")!=0 or value.get("backendCompleteAfterCleanup") is not True for value in observers): receipt["exitCode"]=2;receipt["failureClass"]="candidate_observer_count_mismatch" else: + broker_failure=root/"broker-failure" + if broker_failure.is_file(): + receipt["brokerFailure"]=broker_failure.read_text() + print(f"closed fixture broker failure: {receipt['brokerFailure']}",flush=True) receipt["exitCode"]=2;receipt["failureClass"]="broker_receipt_unavailable" finally: # Get only this run's objects, then inspect exact labels and use From 4811242b8e767be4a76ace9aa4048624eacef0fd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 16:48:41 -0700 Subject: [PATCH 337/493] fix: preserve explicit checkpoint scope and log failures --- crates/freshell-server/src/checkpoints.rs | 431 ++++++++++++++++-- crates/freshell-server/src/files.rs | 14 +- .../components/panes/DirectoryPicker.test.tsx | 9 +- 3 files changed, 413 insertions(+), 41 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index d6d10857f..7b8527754 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -1654,6 +1654,14 @@ impl CheckpointService { .map(|workspace| workspace.path.as_str()) }); let Some(workspace_anchor) = workspace_anchor else { + log_automatic_checkpoint_failure( + Some(request_id), + "workspace_unavailable", + "workspace-resolution", + "resolve-workspace", + None, + "No selected code workspace is available for a snapshot", + ); return self.new_record( send, logical_attempt_id, @@ -1669,11 +1677,16 @@ impl CheckpointService { let resolve_control_for_task = Arc::clone(&resolve_control); let home = Arc::clone(&self.home); let workspace_anchor = workspace_anchor.to_string(); + let workspace_anchor_for_task = workspace_anchor.clone(); let mut resolve_task = tokio::spawn(CHECKPOINT_CAPTURE_CONTEXT.scope( resolve_control_for_task, async move { - resolve_checkpoint_workspace(&home, Path::new(&workspace_anchor), intent.as_ref()) - .await + resolve_checkpoint_workspace( + &home, + Path::new(&workspace_anchor_for_task), + intent.as_ref(), + ) + .await }, )); let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); @@ -1693,6 +1706,14 @@ impl CheckpointService { ); } Ok(Err(error)) => { + log_automatic_checkpoint_failure( + Some(request_id), + "workspace_resolution_failed", + "workspace-resolution", + "resolve-workspace", + Some(Path::new(&workspace_anchor)), + &error.to_string(), + ); return self.new_record( send, logical_attempt_id, @@ -1707,6 +1728,24 @@ impl CheckpointService { Err(_) => { let settlement = cancel_and_settle_capture_task(&resolve_control, &mut resolve_task).await; + let (failure_code, cause) = match &settlement { + Ok(()) => ( + "workspace_resolution_timeout", + "Workspace resolution exceeded the automatic snapshot budget".to_string(), + ), + Err(cause) => ( + "workspace_resolution_unsettled", + format!("Workspace resolution timed out and its child process could not be confirmed settled: {cause}"), + ), + }; + log_automatic_checkpoint_failure( + Some(request_id), + failure_code, + "workspace-resolution", + "resolve-workspace", + Some(Path::new(&workspace_anchor)), + &cause, + ); let mut record = self.new_record( send, logical_attempt_id, @@ -1728,6 +1767,17 @@ impl CheckpointService { } }; if let Some(remaining) = self.active_cooldown(&workspace.path) { + log_automatic_checkpoint_failure( + Some(request_id), + "capture_timeout_cooldown", + "capture", + "automatic-pre-turn-snapshot", + Some(&workspace.path), + &format!( + "capture is paused for {} seconds after a timeout", + remaining.as_secs() + ), + ); return self.new_record( send, logical_attempt_id, @@ -1742,6 +1792,14 @@ impl CheckpointService { let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); if remaining.is_zero() { + log_automatic_checkpoint_failure( + Some(request_id), + "workspace_resolution_timeout", + "workspace-resolution", + "resolve-workspace", + Some(&workspace.path), + "Workspace resolution used the available snapshot time", + ); return self.new_record( send, logical_attempt_id, @@ -1753,6 +1811,14 @@ impl CheckpointService { let lock = match self.lock_workspace(&workspace.path, remaining).await { Ok(lock) => lock, Err(()) => { + log_automatic_checkpoint_failure( + Some(request_id), + "workspace_lock_timeout", + "capture", + "lock-workspace", + Some(&workspace.path), + "Workspace lock was not available before the snapshot budget expired", + ); return self.new_record( send, logical_attempt_id, @@ -1765,6 +1831,14 @@ impl CheckpointService { let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); if remaining.is_zero() { drop(lock); + log_automatic_checkpoint_failure( + Some(request_id), + "workspace_lock_timeout", + "capture", + "lock-workspace", + Some(&workspace.path), + "Workspace lock acquisition used the available snapshot time", + ); return self.new_record( send, logical_attempt_id, @@ -1808,6 +1882,14 @@ impl CheckpointService { Ok(checkpoint_id), ) } else { + log_automatic_checkpoint_failure( + Some(request_id), + "capture_timeout", + "capture", + "automatic-pre-turn-snapshot", + Some(&workspace.path), + "Code snapshot capture exceeded its automatic budget", + ); self.new_record( send, logical_attempt_id, @@ -1832,6 +1914,14 @@ impl CheckpointService { Ok(checkpoint_id), ) } else { + log_automatic_checkpoint_failure( + Some(request_id), + "capture_cancelled", + "capture", + "automatic-pre-turn-snapshot", + Some(&workspace.path), + "Code snapshot capture was cancelled before publishing a checkpoint", + ); self.new_record( send, logical_attempt_id, @@ -1860,6 +1950,7 @@ impl CheckpointService { )), ); record.capture_settled = false; + let capture_failure_cause = error.clone(); let quarantine = self.quarantine_workspace( workspace.path.clone(), lock, @@ -1902,6 +1993,27 @@ impl CheckpointService { )); } } + let failure_cause = if record.outcome.code.as_deref() == Some("capture_failed") { + capture_failure_cause.as_str() + } else { + record + .outcome + .reason + .as_deref() + .unwrap_or(capture_failure_cause.as_str()) + }; + log_automatic_checkpoint_failure( + Some(request_id), + record + .outcome + .code + .as_deref() + .unwrap_or("capture_unsettled"), + "capture", + "automatic-pre-turn-snapshot", + Some(&workspace.path), + failure_cause, + ); record } CaptureAttempt::Failed(error) => { @@ -2823,6 +2935,31 @@ fn safe_send_request_id(request_id: Option<&str>) -> Option<&str> { }) } +fn log_automatic_checkpoint_failure( + send_request_id: Option<&str>, + failure_code: &str, + stage: &'static str, + operation: &'static str, + workspace: Option<&Path>, + cause: &str, +) { + let workspace = workspace + .map(|path| path.to_string_lossy().into_owned()) + .unwrap_or_default(); + tracing::warn!( + target: "freshell_server::checkpoints", + event = "code_checkpoint.failed", + send_request_id = %safe_send_request_id(send_request_id).unwrap_or(""), + stage, + operation, + failure_code, + workspace = %workspace, + path = %workspace, + cause = %bounded_log_detail(cause), + "automatic code checkpoint unavailable" + ); +} + fn log_workspace_unavailable(candidate: &str, cause: &str, send_request_id: Option<&str>) { tracing::warn!( target: "freshell_server::checkpoints", @@ -2872,13 +3009,6 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { homes } -pub fn is_os_account_home(candidate: &Path) -> bool { - let Ok(candidate) = std::fs::canonicalize(candidate) else { - return false; - }; - is_recognized_account_home(&candidate, &recognized_account_homes(None)) -} - fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { homes.iter().any(|home| home == path) } @@ -2934,7 +3064,8 @@ async fn resolve_checkpoint_workspace( candidate: candidate.clone(), reason, })?; - if is_recognized_account_home(&selected, &homes) { + let explicitly_selected_home = provenance == "directory-picker"; + if is_recognized_account_home(&selected, &homes) && !explicitly_selected_home { return Err(CheckpointWorkspaceUnavailable { candidate, reason: "account home cannot be used as a code checkpoint workspace".to_string(), @@ -2945,7 +3076,7 @@ async fn resolve_checkpoint_workspace( } else { git_checkout_root(&selected).await.unwrap_or(selected) }; - if is_recognized_account_home(&path, &homes) { + if is_recognized_account_home(&path, &homes) && !explicitly_selected_home { return Err(CheckpointWorkspaceUnavailable { candidate, reason: "account home cannot be used as an inferred code checkpoint workspace" @@ -3167,6 +3298,10 @@ async fn create_checkpoint( let git_dir = ensure_checkpoint_repo(home, cwd).await?; let git_dir_arg = format!("--git-dir={}", git_dir.display()); let work_tree_arg = format!("--work-tree={}", cwd.display()); + let excluded_checkpoint_storage = checkpoint_storage_exclusion(&git_dir, cwd)?; + let excluded_storage_pathspec = excluded_checkpoint_storage + .as_deref() + .map(|path| format!(":(exclude,top,literal){}", path.display())); let index_path = git_dir.join("index"); let request_index = git_dir.join(format!("index.tmp-{}", uuid::Uuid::new_v4())); @@ -3215,16 +3350,37 @@ async fn create_checkpoint( let capture = async { let stage_started_at = Instant::now(); - verify_readable_directories(&git_dir, cwd).await?; + if let Some(path) = excluded_checkpoint_storage.as_deref() { + // A user may explicitly select the server's own account home. Its + // checkpoint database is runtime state nested under that directory, + // not part of the selected workspace snapshot. + let pathspec = format!(":(top,literal){}", path.display()); + run_git_with_index( + &[ + &git_dir_arg, + &work_tree_arg, + "rm", + "--cached", + "-r", + "-f", + "--ignore-unmatch", + "--", + &pathspec, + ], + Some(cwd), + &request_index, + ) + .await?; + } + verify_readable_directories(&git_dir, cwd, excluded_checkpoint_storage.as_deref()).await?; stage_ms.insert("readability", stage_started_at.elapsed().as_millis() as u64); let stage_started_at = Instant::now(); - run_git_with_index( - &[&git_dir_arg, &work_tree_arg, "add", "-A", "--", "."], - Some(cwd), - &request_index, - ) - .await?; + let mut add_arguments = vec![&git_dir_arg, &work_tree_arg, "add", "-A", "--", "."]; + if let Some(pathspec) = excluded_storage_pathspec.as_deref() { + add_arguments.push(pathspec); + } + run_git_with_index(&add_arguments, Some(cwd), &request_index).await?; stage_ms.insert("git_add", stage_started_at.elapsed().as_millis() as u64); let stage_started_at = Instant::now(); @@ -3235,7 +3391,13 @@ async fn create_checkpoint( ); let stage_started_at = Instant::now(); - let eligible_file_count = verify_checkpoint_index(&git_dir, cwd, &request_index).await?; + let eligible_file_count = verify_checkpoint_index( + &git_dir, + cwd, + &request_index, + excluded_checkpoint_storage.as_deref(), + ) + .await?; stage_ms.insert( "index_validation", stage_started_at.elapsed().as_millis() as u64, @@ -3391,6 +3553,29 @@ fn checkpoint_git_dir(home: &Path, cwd: &Path) -> PathBuf { .join(format!("{short}.git")) } +fn checkpoint_storage_exclusion( + git_dir: &Path, + work_tree: &Path, +) -> Result<Option<PathBuf>, String> { + let storage = git_dir.parent().ok_or_else(|| { + "checkpoint operation resolve-storage failed: storage path has no parent".to_string() + })?; + if work_tree.starts_with(storage) { + return Err( + "checkpoint operation resolve-storage failed: workspace overlaps Freshell checkpoint storage" + .to_string(), + ); + } + match storage.strip_prefix(work_tree) { + Ok(relative) if relative.as_os_str().is_empty() => Err( + "checkpoint operation resolve-storage failed: workspace is Freshell checkpoint storage" + .to_string(), + ), + Ok(relative) => Ok(Some(relative.to_owned())), + Err(_) => Ok(None), + } +} + /// Lexical (no symlink-following) equivalent of Node's `path.resolve(cwd)`: an /// already-absolute path is normalized (`.`/`..` segments collapsed); a relative /// one is resolved against the process's current directory first. Node's @@ -3571,7 +3756,11 @@ fn display_relative_path(path: &Path) -> String { } } -async fn verify_readable_directories(git_dir: &Path, work_tree: &Path) -> Result<(), String> { +async fn verify_readable_directories( + git_dir: &Path, + work_tree: &Path, + excluded_checkpoint_storage: Option<&Path>, +) -> Result<(), String> { let mut pending = vec![PathBuf::new()]; while let Some(relative_parent) = pending.pop() { let absolute_parent = work_tree.join(&relative_parent); @@ -3604,6 +3793,9 @@ async fn verify_readable_directories(git_dir: &Path, work_tree: &Path) -> Result continue; } let relative = relative_parent.join(entry.file_name()); + if excluded_checkpoint_storage.is_some_and(|excluded| relative.starts_with(excluded)) { + continue; + } let mut ignore_path = git_path_bytes(&relative); ignore_path.push(b'/'); candidates.push((relative, ignore_path)); @@ -4053,6 +4245,7 @@ async fn verify_checkpoint_index( git_dir: &Path, work_tree: &Path, index: &Path, + excluded_checkpoint_storage: Option<&Path>, ) -> Result<usize, String> { let staged = checkpoint_index_entries(git_dir, index).await?; for entry in &staged { @@ -4073,7 +4266,13 @@ async fn verify_checkpoint_index( String::from_utf8_lossy(&entry.path) )); } - let absolute = work_tree.join(path); + let absolute = work_tree.join(&path); + if excluded_checkpoint_storage.is_some_and(|excluded| path.starts_with(excluded)) { + return Err(format!( + "checkpoint operation inspect-index failed: internal checkpoint storage was staged at {}", + String::from_utf8_lossy(&entry.path) + )); + } let metadata = tokio::fs::symlink_metadata(&absolute) .await .map_err(|error| { @@ -4691,6 +4890,66 @@ async fn post_checkpoint_restore( mod tests { use super::*; + #[derive(Clone)] + struct ReviewR6CaptureLogs( + Arc<std::sync::Mutex<Vec<std::collections::BTreeMap<String, String>>>>, + ); + + #[derive(Default)] + struct ReviewR6Fields(std::collections::BTreeMap<String, String>); + + impl tracing::field::Visit for ReviewR6Fields { + fn record_str(&mut self, field: &tracing::field::Field, value: &str) { + self.0.insert(field.name().to_string(), value.to_string()); + } + + fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) { + self.0 + .insert(field.name().to_string(), format!("{value:?}")); + } + } + + impl<S: tracing::Subscriber> tracing_subscriber::Layer<S> for ReviewR6CaptureLogs { + fn on_event( + &self, + event: &tracing::Event<'_>, + _: tracing_subscriber::layer::Context<'_, S>, + ) { + if event.metadata().target() != "freshell_server::checkpoints" { + return; + } + let mut fields = ReviewR6Fields::default(); + event.record(&mut fields); + self.0.lock().unwrap().push(fields.0); + } + } + + fn review_r6_send( + cwd: Option<String>, + workspace: Option<&Path>, + request_id: &str, + ) -> FreshAgentSend { + FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "review-r6-session".to_string(), + session_type: SessionType::Freshopencode, + text: "private prompt marker must not enter checkpoint logs".to_string(), + cwd, + images: None, + request_id: Some(request_id.to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "review-r6-attempt".to_string(), + workspace: workspace.map(|path| FreshAgentCheckpointWorkspace { + path: path.to_string_lossy().into_owned(), + provenance: CheckpointWorkspaceProvenance::DirectoryPicker, + }), + }), + } + } + fn headers_with_token(token: &str) -> HeaderMap { let mut headers = HeaderMap::new(); headers.insert("x-auth-token", token.parse().unwrap()); @@ -6386,6 +6645,111 @@ mod tests { assert_eq!(tokio::fs::read_to_string(source).await.unwrap(), "before"); } + #[tokio::test] + async fn review_r6_explicit_picker_account_home_is_a_valid_scope() { + let home = tempfile::tempdir().unwrap(); + let intent = json!({ + "path": home.path(), + "provenance": "directory-picker", + }); + + let resolved = resolve_checkpoint_workspace(home.path(), home.path(), Some(&intent)) + .await + .expect("explicit directory-picker selection defines the account-home scope"); + + assert_eq!(resolved.path, home.path().canonicalize().unwrap()); + assert_eq!(resolved.provenance, "directory-picker"); + } + + #[tokio::test] + async fn review_r6_automatic_checkpoint_missing_workspace_is_logged() { + use tracing::instrument::WithSubscriber; + use tracing_subscriber::prelude::*; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(8); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + let events = Arc::new(std::sync::Mutex::new(Vec::new())); + let subscriber = tracing_subscriber::registry().with(ReviewR6CaptureLogs(events.clone())); + let send = review_r6_send(None, None, "review-r6-no-workspace"); + + let outcome = service + .capture_automatic(&send, "review-r6-no-workspace", "review-r6-attempt") + .with_subscriber(subscriber) + .await; + + assert_eq!( + outcome.outcome.status, + FreshAgentCheckpointStatus::Unavailable + ); + let events = events.lock().unwrap(); + let failure = events + .iter() + .find(|event| { + event + .get("event") + .is_some_and(|value| value == "code_checkpoint.failed") + }) + .expect("automatic unavailable outcome must emit one structured failure event"); + assert_eq!(failure["send_request_id"], "review-r6-no-workspace"); + assert_eq!(failure["stage"], "workspace-resolution"); + assert_eq!(failure["operation"], "resolve-workspace"); + assert_eq!(failure["failure_code"], "workspace_unavailable"); + assert!(!serde_json::to_string(&*events) + .unwrap() + .contains("private prompt marker")); + } + + #[tokio::test] + async fn review_r6_automatic_checkpoint_cooldown_is_logged() { + use tracing::instrument::WithSubscriber; + use tracing_subscriber::prelude::*; + + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(8); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + service.establish_cooldown(workspace.path()); + let events = Arc::new(std::sync::Mutex::new(Vec::new())); + let subscriber = tracing_subscriber::registry().with(ReviewR6CaptureLogs(events.clone())); + let send = review_r6_send( + Some(workspace.path().to_string_lossy().into_owned()), + Some(workspace.path()), + "review-r6-cooldown", + ); + + let outcome = service + .capture_automatic(&send, "review-r6-cooldown", "review-r6-attempt") + .with_subscriber(subscriber) + .await; + + assert_eq!( + outcome.outcome.status, + FreshAgentCheckpointStatus::Unavailable + ); + let events = events.lock().unwrap(); + let failure = events + .iter() + .find(|event| { + event + .get("failure_code") + .is_some_and(|value| value == "capture_timeout_cooldown") + }) + .expect("cooldown unavailable outcome must emit a structured failure event"); + assert_eq!(failure["send_request_id"], "review-r6-cooldown"); + assert_eq!(failure["stage"], "capture"); + assert_eq!(failure["operation"], "automatic-pre-turn-snapshot"); + assert_eq!( + failure["workspace"], + workspace.path().to_string_lossy().as_ref() + ); + assert!(!serde_json::to_string(&*events) + .unwrap() + .contains("private prompt marker")); + } + #[tokio::test] async fn home_launch_without_workspace_does_not_publish_checkpoint() { let home_dir = tempfile::tempdir().unwrap(); @@ -6469,7 +6833,7 @@ mod tests { } #[tokio::test] - async fn picker_home_is_launch_only() { + async fn explicit_picker_home_is_snapshotted_without_the_internal_store() { let home_dir = tempfile::tempdir().unwrap(); tokio::fs::write(home_dir.path().join("private.txt"), "private") .await @@ -6488,14 +6852,21 @@ mod tests { ) .await; - assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY); - let error = body_json(response).await; - assert_eq!(error["code"], json!("workspace_unavailable")); - assert!( - !tokio::fs::try_exists(home_dir.path().join(".freshell/checkpoints")) - .await - .unwrap() - ); + let status = response.status(); + let entry = body_json(response).await; + assert_eq!(status, StatusCode::OK, "{entry}"); + assert_eq!(entry["workspace"]["path"], json!(home_dir.path())); + assert_eq!(entry["workspace"]["provenance"], json!("directory-picker")); + + let git_dir = checkpoint_git_dir(home_dir.path(), home_dir.path()); + let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) + .await + .expect("published picker-home checkpoint index is readable"); + assert!(index.iter().any(|entry| entry.path == b"private.txt")); + assert!(!index.iter().any(|entry| { + entry.path.starts_with(b".freshell/checkpoints/") + || entry.path == b".freshell/checkpoints" + })); } #[tokio::test] diff --git a/crates/freshell-server/src/files.rs b/crates/freshell-server/src/files.rs index 33a7c0632..8141d3086 100644 --- a/crates/freshell-server/src/files.rs +++ b/crates/freshell-server/src/files.rs @@ -200,12 +200,10 @@ async fn validate_dir( .unwrap_or(false) }) .unwrap_or(false); - let checkpoint_workspace_eligible = is_dir - && resolved - .fs_path - .as_deref() - .and_then(|path| std::fs::canonicalize(path).ok()) - .is_some_and(|path| !crate::checkpoints::is_os_account_home(&path)); + // This flag answers whether an explicit directory-picker selection can + // define a snapshot scope. Automatic capture still rejects an unselected + // account home in the checkpoint workspace resolver. + let checkpoint_workspace_eligible = is_dir; Json(json!({ "valid": is_dir, @@ -1421,7 +1419,7 @@ mod tests { #[allow(clippy::await_holding_lock)] #[tokio::test] - async fn validate_dir_marks_account_home_as_launch_only() { + async fn review_r6_validate_dir_allows_explicit_account_home_scope() { let _guard = env_lock(); let home = std::env::var("HOME").expect("test environment has HOME"); let response = validate_dir( @@ -1433,7 +1431,7 @@ mod tests { assert_eq!(response.status(), StatusCode::OK); let value = body_json(response).await; assert_eq!(value["valid"], true); - assert_eq!(value["checkpointWorkspaceEligible"], false); + assert_eq!(value["checkpointWorkspaceEligible"], true); } /// Node's `normalizeUserPath` runs `path.posix.resolve` on POSIX inputs diff --git a/test/unit/client/components/panes/DirectoryPicker.test.tsx b/test/unit/client/components/panes/DirectoryPicker.test.tsx index b2a514fc8..247a26c00 100644 --- a/test/unit/client/components/panes/DirectoryPicker.test.tsx +++ b/test/unit/client/components/panes/DirectoryPicker.test.tsx @@ -184,11 +184,11 @@ describe('DirectoryPicker', () => { }) }) - it('keeps account home as launch cwd without selecting it for checkpoints', async () => { + it('passes explicit account-home selection through as checkpoint scope', async () => { mockApiPost.mockResolvedValueOnce({ valid: true, resolvedPath: '/home/user', - checkpointWorkspaceEligible: false, + checkpointWorkspaceEligible: true, }) const { onConfirm } = renderDirectoryPicker({ defaultCwd: '' }) const input = screen.getByLabelText('Starting directory for Claude') @@ -196,7 +196,10 @@ describe('DirectoryPicker', () => { fireEvent.change(input, { target: { value: '/home/user' } }) fireEvent.keyDown(input, { key: 'Enter' }) - await waitFor(() => expect(onConfirm).toHaveBeenCalledWith('/home/user')) + await waitFor(() => expect(onConfirm).toHaveBeenCalledWith('/home/user', { + path: '/home/user', + provenance: 'directory-picker', + })) expect(onConfirm).toHaveBeenCalledTimes(1) }) From c919cac612bca1f984292b8f6c63fe25a8f78ff4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 16:48:44 -0700 Subject: [PATCH 338/493] test: correct runtime inventory and cli output capture --- test/unit/architecture/fresh-agent-only-runtime.test.ts | 2 -- test/unit/cli/action-capabilities.test.ts | 4 +++- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/test/unit/architecture/fresh-agent-only-runtime.test.ts b/test/unit/architecture/fresh-agent-only-runtime.test.ts index 83772f4bf..fbdd26a74 100644 --- a/test/unit/architecture/fresh-agent-only-runtime.test.ts +++ b/test/unit/architecture/fresh-agent-only-runtime.test.ts @@ -12,8 +12,6 @@ const productionRuntimeRoots = ['src', 'shared'] as const const retiredNodeBackendPaths = [ 'server', 'test/server', - 'test/unit/server', - 'test/integration/server', 'test/integration/session-repair.test.ts', 'tsconfig.server.json', 'dist/server', diff --git a/test/unit/cli/action-capabilities.test.ts b/test/unit/cli/action-capabilities.test.ts index 622f86c87..fc9b98b32 100644 --- a/test/unit/cli/action-capabilities.test.ts +++ b/test/unit/cli/action-capabilities.test.ts @@ -35,7 +35,9 @@ async function runCli(args: string[], url: string): Promise<{ code: number | nul let stderr = '' child.stdout.on('data', (chunk) => { stdout += String(chunk) }) child.stderr.on('data', (chunk) => { stderr += String(chunk) }) - const [code] = await once(child, 'exit') as [number | null] + // Wait for stdio to finish draining too; `exit` can arrive while buffered + // stderr diagnostics are still being delivered to these listeners. + const [code] = await once(child, 'close') as [number | null] return { code, stdout, stderr } } From 9993da0371acbe9e0fafc201ecdaf0d2f85cf60b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 17:12:32 -0700 Subject: [PATCH 339/493] fix(checkpoints): preserve explicit account-home scope --- crates/freshell-server/src/checkpoints.rs | 102 ++++++++++++++++++++-- 1 file changed, 95 insertions(+), 7 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 7b8527754..a4b75fa4f 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -3064,14 +3064,15 @@ async fn resolve_checkpoint_workspace( candidate: candidate.clone(), reason, })?; - let explicitly_selected_home = provenance == "directory-picker"; + let explicitly_selected_home = + provenance == "directory-picker" && is_recognized_account_home(&selected, &homes); if is_recognized_account_home(&selected, &homes) && !explicitly_selected_home { return Err(CheckpointWorkspaceUnavailable { candidate, reason: "account home cannot be used as a code checkpoint workspace".to_string(), }); } - let path = if provenance == "legacy-checkpoint" { + let path = if provenance == "legacy-checkpoint" || explicitly_selected_home { selected } else { git_checkout_root(&selected).await.unwrap_or(selected) @@ -3560,17 +3561,22 @@ fn checkpoint_storage_exclusion( let storage = git_dir.parent().ok_or_else(|| { "checkpoint operation resolve-storage failed: storage path has no parent".to_string() })?; - if work_tree.starts_with(storage) { + let home = storage.parent().and_then(Path::parent).ok_or_else(|| { + "checkpoint operation resolve-storage failed: storage path has no home anchor".to_string() + })?; + let canonical_home = std::fs::canonicalize(home).map_err(|_| { + "checkpoint operation resolve-storage failed: home anchor cannot be resolved".to_string() + })?; + // Canonicalize the home anchor so aliases compare against the canonical + // workspace, then retain the logical suffix for Git's pathspec. + let storage = canonical_home.join(".freshell").join("checkpoints"); + if work_tree.starts_with(&storage) { return Err( "checkpoint operation resolve-storage failed: workspace overlaps Freshell checkpoint storage" .to_string(), ); } match storage.strip_prefix(work_tree) { - Ok(relative) if relative.as_os_str().is_empty() => Err( - "checkpoint operation resolve-storage failed: workspace is Freshell checkpoint storage" - .to_string(), - ), Ok(relative) => Ok(Some(relative.to_owned())), Err(_) => Ok(None), } @@ -6661,6 +6667,42 @@ mod tests { assert_eq!(resolved.provenance, "directory-picker"); } + #[tokio::test] + async fn explicit_picker_home_keeps_exact_scope_when_git_root_is_its_parent() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + tokio::fs::create_dir(&home).await.unwrap(); + run_git(&["init", "-q"], Some(owned.path())).await.unwrap(); + let intent = json!({ + "path": home, + "provenance": "directory-picker", + }); + + let resolved = resolve_checkpoint_workspace(&home, &home, Some(&intent)) + .await + .expect("explicit account-home selection is an eligible scope"); + + assert_eq!(resolved.path, home.canonicalize().unwrap()); + } + + #[tokio::test] + async fn picker_subdirectory_of_account_home_git_repo_is_rejected() { + let home = tempfile::tempdir().unwrap(); + let selected = home.path().join("project"); + tokio::fs::create_dir(&selected).await.unwrap(); + run_git(&["init", "-q"], Some(home.path())).await.unwrap(); + let intent = json!({ + "path": selected, + "provenance": "directory-picker", + }); + + let error = resolve_checkpoint_workspace(home.path(), &selected, Some(&intent)) + .await + .expect_err("selecting a child must not authorize its account-home Git root"); + + assert!(error.reason.contains("account home"), "{}", error.reason); + } + #[tokio::test] async fn review_r6_automatic_checkpoint_missing_workspace_is_logged() { use tracing::instrument::WithSubscriber; @@ -6869,6 +6911,52 @@ mod tests { })); } + #[cfg(unix)] + #[tokio::test] + async fn explicit_picker_home_symlink_excludes_internal_store_and_rejects_store_scope() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("actual-home"); + let alias = owned.path().join("home-alias"); + tokio::fs::create_dir(&home).await.unwrap(); + std::os::unix::fs::symlink(&home, &alias).unwrap(); + tokio::fs::write(home.join("private.txt"), "private") + .await + .unwrap(); + + let response = post_checkpoint( + State(state(&alias)), + headers_with_token("tok"), + Json(json!({ + "cwd": alias, + "workspace": { + "path": alias, + "provenance": "directory-picker", + }, + })), + ) + .await; + + let status = response.status(); + let entry = body_json(response).await; + assert_eq!(status, StatusCode::OK, "{entry}"); + let git_dir = checkpoint_git_dir(&alias, &home); + let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) + .await + .expect("published aliased-home checkpoint index is readable"); + assert!(index.iter().any(|entry| entry.path == b"private.txt")); + assert!(!index.iter().any(|entry| { + entry.path.starts_with(b".freshell/checkpoints/") + || entry.path == b".freshell/checkpoints" + })); + + let nested_workspace = home.join(".freshell/checkpoints/nested-workspace"); + tokio::fs::create_dir_all(&nested_workspace).await.unwrap(); + let canonical_nested = tokio::fs::canonicalize(&nested_workspace).await.unwrap(); + let error = checkpoint_storage_exclusion(&git_dir, &canonical_nested) + .expect_err("the aliased checkpoint store cannot become a workspace"); + assert!(error.contains("overlaps Freshell checkpoint storage")); + } + #[tokio::test] async fn resolved_project_scope_independent_of_routing_cwd() { let home_dir = tempfile::tempdir().unwrap(); From 335f448197a82ed1cef3d1a023587bcf8a0063ca Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 17:12:36 -0700 Subject: [PATCH 340/493] fix(opencode): stage actual MCP script operands --- .../src/managed_provider_launch_tests.rs | 15 +- .../opencode_resources.rs | 138 +++++++++++++++--- .../managed_provider_references/sources.rs | 22 +++ 3 files changed, 151 insertions(+), 24 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_launch_tests.rs b/crates/freshell-server/src/managed_provider_launch_tests.rs index 0d5c1f433..175ba25ea 100644 --- a/crates/freshell-server/src/managed_provider_launch_tests.rs +++ b/crates/freshell-server/src/managed_provider_launch_tests.rs @@ -3777,9 +3777,11 @@ export default {{id:'closed-shared-{label}',async tui(api,options){{ ) .unwrap(); let mcp_cwd_receipt = fixture.join("workspace/native-mcp-cwd-consumed.txt"); - std::fs::write(mcp_cwd.join("server.cjs"), format!(r#"const fs=require('node:fs'),readline=require('node:readline'); -fs.writeFileSync({},fs.readFileSync('resource','utf8')+':'+require('native-mcp-dependency')); -readline.createInterface({{input:process.stdin}}).on('line',line=>{{const q=JSON.parse(line);if(q.id===undefined)return;const result=q.method==='initialize'?{{protocolVersion:q.params.protocolVersion,capabilities:{{tools:{{}}}},serverInfo:{{name:'closed-cwd-probe',version:'1'}}}}:q.method==='tools/list'?{{tools:[]}}:{{}};process.stdout.write(JSON.stringify({{jsonrpc:'2.0',id:q.id,result}})+'\n');}}); + let external_mcp_command = external.join("extensionless-mcp-server"); + std::fs::write(&external_mcp_command, format!(r#"const fs=require('node:fs'),readline=require('node:readline'),path=require('node:path'); +const requireFromCwd=require('node:module').createRequire(path.join(process.cwd(),'mcp-cwd-entry.cjs')); +const receipt={}; +readline.createInterface({{input:process.stdin}}).on('line',line=>{{const q=JSON.parse(line);if(q.id===undefined)return;let result={{}};if(q.method==='initialize'){{fs.writeFileSync(receipt,JSON.stringify({{arguments:process.argv.slice(2),resource:fs.readFileSync('resource','utf8')+':'+requireFromCwd('native-mcp-dependency')}}));result={{protocolVersion:q.params.protocolVersion,capabilities:{{tools:{{}}}},serverInfo:{{name:'closed-cwd-probe',version:'1'}}}};}}else if(q.method==='tools/list'){{result={{tools:[]}};}}process.stdout.write(JSON.stringify({{jsonrpc:'2.0',id:q.id,result}})+'\n');}}); "#,serde_json::to_string(&mcp_cwd_receipt).unwrap())).unwrap(); let external_receipt = fixture.join("workspace/native-external-consumed.json"); let external_plugin = r#"import fs from 'node:fs';import crypto from 'node:crypto';import path from 'node:path';import {execFileSync} from 'node:child_process'; @@ -3793,7 +3795,7 @@ export default async (input,options)=>({config:async(config)=>{ if(!skill)throw Error('selected native skill unavailable'); const output=execFileSync(path.join(path.dirname(skill.location),'scripts/run')); await input.client._client.get({url:'/mcp'}); - const mcp=digest(fs.readFileSync(options.mcpReceipt)); + const mcp=JSON.parse(fs.readFileSync(options.mcpReceipt,'utf8')); fs.writeFileSync(options.receipt,JSON.stringify({adjacent:digest(adjacent),child:digest(child),parent:digest(parent),resource:digest(fs.readFileSync(new URL('./resource.txt',import.meta.url))),instruction:digest(fs.readFileSync(options.instruction)),selected:config.instructions.includes(options.instruction),nativeLiveViews:config.agent.liveCapacity.description==='native-live-capacity-nonce',nativeSkill:true,skillContent:digest(skill.content),skillExecution:digest(output),nativeMcpCwd:mcp})); } catch(error) {fs.writeFileSync(options.receipt+'.failure',JSON.stringify({operation:'native_skill_consumer',type:error instanceof TypeError?'TypeError':error.code==='ENOENT'?'ENOENT':'native_skill_unavailable'}));} },0); @@ -3803,8 +3805,7 @@ export default async (input,options)=>({config:async(config)=>{ let mut original: Value = serde_json::from_slice(&std::fs::read(&original_config).unwrap()).unwrap(); original["plugin"].as_array_mut().unwrap().push(json!([reqwest::Url::from_file_path(package.join("index.mjs")).unwrap().to_string(), {"receipt":external_receipt,"instruction":external_instruction,"mcpReceipt":mcp_cwd_receipt}])); - original["mcp"]["closed-cwd-probe"] = - json!({"type":"local","command":["node","server.cjs"],"cwd":"~/mcp-cwd"}); + original["mcp"]["closed-cwd-probe"] = json!({"type":"local","command":["node",external_mcp_command,"--config","prod"],"cwd":"~/mcp-cwd"}); original["instructions"] = json!([external_instruction]); original["skills"] = json!({"paths":[selected_skills,"../../scenarios/external-native-resources/selected-skills"]}); // Native ConfigParse/jsonc-parser keeps only the final resource member. @@ -3828,7 +3829,7 @@ export default async (input,options)=>({config:async(config)=>{ ); std::fs::write(&original_config, duplicate_resources.as_bytes()).unwrap(); let original_bytes = std::fs::read(&original_config).unwrap(); - let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeLiveViews":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":format!("{:x}",Sha256::digest(b"native-mcp-resource:native-mcp-dependency"))}); + let expected_external = json!({"adjacent":format!("{:x}",Sha256::digest(b"native-adjacent-import")),"child":format!("{:x}",Sha256::digest(b"native-child-dependency")),"parent":format!("{:x}",Sha256::digest(b"native-parent-dependency")),"resource":format!("{:x}",Sha256::digest(b"native-adjacent-resource")),"instruction":format!("{:x}",Sha256::digest(b"native-external-instruction")),"selected":true,"nativeLiveViews":true,"nativeSkill":true,"skillContent":format!("{:x}",Sha256::digest(b"Execute the adjacent synthetic resources.\n")),"skillExecution":format!("{:x}",Sha256::digest(b"native-skill-resource:native-skill-dependency")),"nativeMcpCwd":{"arguments":["--config","prod"],"resource":"native-mcp-resource:native-mcp-dependency"}}); // Five logical native sources share three immutable 16 MiB raw views. // Their 48 MiB retained controller views fit its simultaneous profile; // the host must not turn 80 MiB of sequential reads into a live limit. diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 351e8cd32..1c60b14b1 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -238,27 +238,26 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' && command[0] == "node" && command[1] == FRESHELL_IMAGE_MCP_SERVER_PATH; if !image_command { + let script_index = script_operand_index(&command); + let script_runtime = + command_uses_script_runtime(command.first().map(String::as_str)); for (index, argument) in command.iter().enumerate() { - if (index == 0 && is_explicit_command_path(argument)) - || is_script_path(argument) - { + let script_file = if script_runtime { + script_index.is_some_and(|script| { + index > 0 && index <= script && is_script_path(argument) + }) + } else { + is_script_path(argument) + }; + if (index == 0 && is_explicit_command_path(argument)) || script_file { result.push(("OpenCode mcp.command", argument.clone())); } } - let inline_code = command.iter().any(|argument| { - matches!(argument.as_str(), "-e" | "-c" | "--eval" | "--print") - }); - let bare_script_arguments = command - .iter() - .skip(1) - .filter(|argument| !argument.starts_with('-') && !argument.is_empty()) - .collect::<Vec<_>>(); - if command_uses_script_runtime(command.first().map(String::as_str)) - && !inline_code - && bare_script_arguments.len() == 1 - && !is_script_path(bare_script_arguments[0]) - { - result.push(("OpenCode mcp.command", bare_script_arguments[0].clone())); + if let Some(script_index) = script_index { + let script = &command[script_index]; + if !is_script_path(script) { + result.push(("OpenCode mcp.command", script.clone())); + } } } } @@ -343,6 +342,111 @@ fn command_uses_script_runtime(executable: Option<&str>) -> bool { ) } +fn script_operand_index(command: &[String]) -> Option<usize> { + let executable = command + .first()? + .rsplit(['/', '\\']) + .next()? + .to_ascii_lowercase(); + let executable = executable.strip_suffix(".exe").unwrap_or(&executable); + if !command_uses_script_runtime(Some(executable)) { + return None; + } + let mut index = 1; + if matches!(executable, "bun" | "deno") && command.get(index).is_some_and(|arg| arg == "run") { + index += 1; + } + while index < command.len() { + let argument = command[index].as_str(); + if argument == "--" { + return command + .get(index + 1) + .filter(|argument| !argument.is_empty()) + .map(|_| index + 1); + } + let option = argument.split_once('=').map_or(argument, |(name, _)| name); + if matches!(executable, "node" | "nodejs" | "bun" | "tsx" | "ts-node") { + if matches!(option, "-e" | "--eval" | "-p" | "--print") { + return None; + } + if matches!( + option, + "-r" | "--require" + | "--import" + | "--loader" + | "--experimental-loader" + | "--conditions" + | "--title" + | "--redirect-warnings" + | "--diagnostic-dir" + | "--env-file" + | "--env-file-if-exists" + | "--watch-path" + | "--tsconfig" + ) && !argument.contains('=') + { + index = (index + 2).min(command.len()); + continue; + } + } else if matches!(executable, "python" | "python3") { + if matches!(argument, "-c" | "-m") { + return None; + } + if matches!(argument, "-W" | "-X") { + index = (index + 2).min(command.len()); + continue; + } + if argument == "-" { + return None; + } + } else if executable == "ruby" { + if argument == "-e" { + return None; + } + if matches!(argument, "-r" | "-I" | "-C" | "-E" | "-F") { + index = (index + 2).min(command.len()); + continue; + } + } else if matches!(executable, "bash" | "sh" | "zsh") { + if argument == "-c" { + return None; + } + if matches!( + argument, + "-o" | "+o" | "-O" | "+O" | "--rcfile" | "--init-file" + ) { + index = (index + 2).min(command.len()); + continue; + } + } else if matches!(executable, "pwsh" | "powershell") { + if matches!(argument.to_ascii_lowercase().as_str(), "-command" | "-c") { + return None; + } + if matches!(argument.to_ascii_lowercase().as_str(), "-file" | "-f") { + return (index + 1 < command.len()).then_some(index + 1); + } + if matches!( + argument.to_ascii_lowercase().as_str(), + "-executionpolicy" + | "-workingdirectory" + | "-inputformat" + | "-outputformat" + | "-configurationname" + | "-configurationfile" + ) { + index = (index + 2).min(command.len()); + continue; + } + } + if argument.starts_with('-') || argument.is_empty() { + index += 1; + continue; + } + return Some(index); + } + None +} + fn resource_array( cursor: &mut Cursor<'_>, field: &'static str, diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 41a6e5e7a..b403fc580 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1127,6 +1127,9 @@ mod tests { let workspace_symlink = workspace.join("scripts/symlinked-mcp.js"); let cwd_script = workspace.join("mcp-cwd/cwd-mcp.js"); let extensionless_script = tools.join("extensionless-mcp"); + let extensionless_args_script = tools.join("extensionless-with-args"); + let preload_script = tools.join("preload.js"); + let trailing_decoy_script = tools.join("trailing-decoy.js"); let missing_disabled_script = tools.join("missing-disabled.js"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); @@ -1140,6 +1143,9 @@ mod tests { &symlink_target, &cwd_script, &extensionless_script, + &extensionless_args_script, + &preload_script, + &trailing_decoy_script, ] { std::fs::write(script, "process.stdout.write('mcp fixture')\n").unwrap(); } @@ -1180,6 +1186,16 @@ mod tests { "command": ["node", extensionless_script], "enabled": true, }, + "extensionless-with-trailing-options": { + "type": "local", + "command": ["node", extensionless_args_script, "--config", "prod"], + "enabled": true, + }, + "extensionless-after-runtime-option": { + "type": "local", + "command": ["node", "--require", preload_script, extensionless_args_script, "--config", trailing_decoy_script], + "enabled": true, + }, "package": { "type": "local", "command": ["npx", "-y", "@example/mcp-server"], @@ -1218,6 +1234,8 @@ mod tests { &symlink_target, &env_script, &extensionless_script, + &extensionless_args_script, + &preload_script, ] { assert!( selected.contains(&script.canonicalize().unwrap()), @@ -1225,6 +1243,10 @@ mod tests { script.display() ); } + assert!( + !selected.contains(&trailing_decoy_script.canonicalize().unwrap()), + "arguments after the selected script are values, not additional scripts" + ); assert!( selected.contains(&cwd_script.parent().unwrap().canonicalize().unwrap()), "the selected local MCP cwd carries scripts below it" From 88daa21eca324a5a472f376c2b6dfbd57ac60a9b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 17:12:40 -0700 Subject: [PATCH 341/493] fix(testing): include broker path diagnostics --- .../testing/managed-provider-launch-broker.ts | 2 +- scripts/testing/runtime-test-broker.ts | 2 +- .../testing/runtime-test-broker.test.ts | 69 ++++++++++++++++++- 3 files changed, 69 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-provider-launch-broker.ts b/scripts/testing/managed-provider-launch-broker.ts index c163904fe..c92993ad6 100644 --- a/scripts/testing/managed-provider-launch-broker.ts +++ b/scripts/testing/managed-provider-launch-broker.ts @@ -2,7 +2,7 @@ //! the native startup fixture receives this broker's restricted proxy socket. import fs from 'node:fs' import path from 'node:path' -import { RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from './runtime-test-broker.js' +import { diagnosticPath, RestrictedDockerBroker, type ClaudeCandidateAuthorization, type PreparedInputAuthorization } from './runtime-test-broker.js' const root=process.env.FRESHELL_PROVIDER_LAUNCH_FIXTURE ?? '' if (path.dirname(root)!=='/tmp' || !/^fln-[a-f0-9]{16}$/.test(path.basename(root))) { diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 661a5e266..259e58ffb 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -14,7 +14,7 @@ class PreparedInputTargetError extends Error { } } -function diagnosticPath(value: string): string | undefined { +export function diagnosticPath(value: string): string | undefined { return value.length <= 4096 && path.isAbsolute(value) && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) ? value : undefined } diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index cde503baf..10a7d9af7 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -1,5 +1,5 @@ -import { createHash } from 'node:crypto' -import { spawnSync } from 'node:child_process' +import { createHash, randomBytes } from 'node:crypto' +import { spawn, spawnSync } from 'node:child_process' import os from 'node:os' import fs from 'node:fs/promises' import http from 'node:http' @@ -78,6 +78,71 @@ it('classifies owned provider publication failures without retaining error bodie expect(JSON.stringify(diagnostic)).not.toContain(secret) }) +it('serializes rejected target paths in managed provider launch broker failures', async () => { + const root = path.join('/tmp', `fln-${randomBytes(8).toString('hex')}`) + const imageRef = 'sha256:967627e41734aa903e824e4be22849c7f22ce0e69487ce61bb72004c4817f89c' + const hostBinaryPath = path.join(root, 'freshell-session-host') + const authorization = (installationId: string, soulId: string) => ({ + installationId, + soulId, + providerVolumeName: `freshell-provider-${createHash('sha256').update(`${installationId}\0${soulId}`).digest('hex').slice(0, 24)}`, + imageRef, + hostBinaryPath, + binarySha256: 'a'.repeat(64), + daemonId: 'fixture-daemon', + }) + let child: ReturnType<typeof spawn> | undefined + try { + await fs.mkdir(path.join(root, 'handoff'), { recursive: true, mode: 0o700 }) + await fs.mkdir(path.join(root, 'candidate-handoff'), { mode: 0o700 }) + const owner = authorization('install-one', 'soul-one') + await fs.writeFile(path.join(root, 'broker-configuration.json'), JSON.stringify({ + provider: 'opencode', claudeFlavor: 'freshclaude', imageRef, hostBinaryPath, + initialVolumes: [owner, authorization('install-two', 'soul-two')], runId: 'launch-diagnostic-test', + })) + await fs.writeFile(path.join(root, 'handoff', 'attempt-one.json'), JSON.stringify({ + ...owner, incarnationId: 'incarnation-one', generationId: 'grant-one', + stageRoot: path.join(root, 'scenarios/scenario/control/provider-roots/grant-one'), + configurationBinds: [{ sourceRelativePath: '.freshell/templates/source-0', effectiveDestination: '/outside/rejected-target' }], + resourceMounts: [], indexedSecretSources: [], + })) + + child = spawn(process.execPath, [ + path.resolve('node_modules/tsx/dist/cli.mjs'), path.resolve('scripts/testing/managed-provider-launch-broker.ts'), + ], { env: { ...process.env, FRESHELL_PROVIDER_LAUNCH_FIXTURE: root }, stdio: ['ignore', 'ignore', 'pipe'] }) + let stderr = '' + child.stderr.setEncoding('utf8').on('data', (chunk: string) => { stderr += chunk }) + const closed = new Promise<number | null>((resolve, reject) => { + child!.once('error', reject) + child!.once('close', (code) => resolve(code)) + }) + const failurePath = path.join(root, 'broker-failure') + const deadline = Date.now() + 10_000 + let failureExists = false + while (Date.now() < deadline) { + try { await fs.access(failurePath); failureExists = true; break } catch {} + if (child.exitCode !== null) break + await new Promise((resolve) => setTimeout(resolve, 20)) + } + if (!failureExists && child.exitCode === null) { + child.kill('SIGKILL') + await closed + throw new Error(`managed provider launch broker did not write a failure receipt: ${stderr}`) + } + const exitCode = await closed + expect(exitCode, stderr).toBe(1) + const failure = JSON.parse(await fs.readFile(failurePath, 'utf8')) + expect(failure).toEqual({ + cause: 'prepared input target is outside selected fixture roots', + targetPath: '/outside/rejected-target', + preparationStageRoot: path.join(root, 'scenarios/scenario/control/provider-roots/grant-one'), + }) + } finally { + if (child && child.exitCode === null) child.kill('SIGKILL') + await fs.rm(root, { recursive: true, force: true }) + } +}) + it('executes selected OpenCode resources after fixture readability preparation', async () => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'freshell-parity-readable-')) try { From c48feda3da1630512eb6ecff5baea3100077dd8f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 17:12:44 -0700 Subject: [PATCH 342/493] test(checkpoints): verify selected HOME rewind in browser --- ...code-checkpoint-workspace-external.spec.ts | 49 ++++++++++++++----- 1 file changed, 36 insertions(+), 13 deletions(-) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index a546f4eed..c67a0e52c 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -1,4 +1,5 @@ -import { randomUUID } from 'node:crypto' +import { createHash, randomUUID } from 'node:crypto' +import { resolve } from 'node:path' import { resolvePairedExternalFixture } from '../../../scripts/testing/e2e-external-target.js' import { test, expect } from '../helpers/fixtures.js' import { openPanePicker } from '../helpers/pane-picker.js' @@ -369,25 +370,47 @@ test('picker-confirmed non-Git workspace captures and restores all selected file expect(afterRestore.content.checkpointWorkspace).toEqual({ path: fixture.nonGitWorkspacePath, provenance: 'directory-picker' }) }) -test('HOME confirmation is launch-only and an accepted send has no checkpoint', async ({ page }) => { +test('picker-confirmed HOME captures and restores a harmless home file', async ({ page }) => { test.setTimeout(240_000) await verifyTarget(page) + const homeFile = `${fixture.syntheticHomePath}/checkpoint-home-file.txt` + await targetWrite(page, homeFile, 'home file before checkpoint\n') const { paneRoot, content } = await launchCodex(page, fixture.syntheticHomePath) expect(content.initialCwd).toBe(fixture.syntheticHomePath) - expect(content.checkpointWorkspace).toBeUndefined() + expect(content.checkpointWorkspace).toEqual({ + path: fixture.syntheticHomePath, + provenance: 'directory-picker', + }) const marker = `checkpoint home ${fixture.expectedNonce}` - const turn = await sendThroughComposer(page, paneRoot, marker, undefined) - expect(turn.outcome).toMatchObject({ status: 'unavailable', code: 'workspace_unavailable' }) - expect(turn.outcome.reason).toMatch(/no eligible code workspace is available/i) - expect(turn.sent.checkpointIntent).not.toHaveProperty('workspace') - await expect(turn.row.getByRole('note', { name: 'Code rewind unavailable for this message' })) - .toContainText(turn.outcome.reason!) + const turn = await sendThroughComposer(page, paneRoot, marker, + { path: fixture.syntheticHomePath, provenance: 'directory-picker' }) + expect(turn.outcome).toMatchObject({ status: 'available', workspace: { + path: fixture.syntheticHomePath, provenance: 'directory-picker', + } }) + expect(turn.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) + await expect(turn.row.getByRole('status', { name: 'Code rewind available for this message' })) + .toBeVisible() + await assertNativeUserIdentity(page, turn.outcome, turn.accepted.submittedTurnId, marker) + + await targetWrite(page, homeFile, 'home file changed after send\n') await turn.row.hover() - await expect(turn.row.getByRole('button', { name: 'Rewind unavailable for this message' })).toBeDisabled() + const rewindNotice = expect(paneRoot.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) + .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) + await turn.row.getByRole('button', { name: 'Rewind code to here' }).click() + await Promise.all([dialogAccepted, rewindNotice]) + await expect.poll(() => targetRead(page, homeFile)).toBe('home file before checkpoint\n') - const list = await page.request.get(`${paired!.targetUrl}/api/fresh-agent/checkpoints?cwd=${encodeURIComponent(fixture.syntheticHomePath)}`, + const query = new URLSearchParams({ + cwd: fixture.syntheticHomePath, + workspacePath: fixture.syntheticHomePath, + workspaceProvenance: 'directory-picker', + }) + const list = await page.request.get(`${paired!.targetUrl}/api/fresh-agent/checkpoints?${query}`, { headers, timeout: 30_000 }) expect(list.ok()).toBe(true) - expect(await list.json()).toEqual({ checkpoints: [] }) - expect(await targetRead(page, `${fixture.syntheticHomePath}/.freshell/checkpoints/HEAD`)).toBeUndefined() + expect(await list.json()).toMatchObject({ checkpoints: [expect.objectContaining({ id: turn.outcome.checkpointId })] }) + const workspaceKey = createHash('sha1').update(resolve(fixture.syntheticHomePath)).digest('hex').slice(0, 16) + expect(await targetRead(page, + `${fixture.syntheticHomePath}/.freshell/checkpoints/${workspaceKey}.git/HEAD`)).toBeTruthy() }) From 72f44339ea4c69e878e34c174790e1a5473cf31b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 17:25:13 -0700 Subject: [PATCH 343/493] fix(opencode): accept Bun MCP package scripts --- .../opencode_resources.rs | 30 ++++++++++- .../managed_provider_references/sources.rs | 52 +++++++++++++++++++ 2 files changed, 80 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 1c60b14b1..ccbc6dae6 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -244,7 +244,10 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' for (index, argument) in command.iter().enumerate() { let script_file = if script_runtime { script_index.is_some_and(|script| { - index > 0 && index <= script && is_script_path(argument) + index > 0 + && index <= script + && is_script_path(argument) + && !is_bun_package_script_operand(&command, index) }) } else { is_script_path(argument) @@ -255,7 +258,9 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' } if let Some(script_index) = script_index { let script = &command[script_index]; - if !is_script_path(script) { + if !is_script_path(script) + && !is_bun_package_script_operand(&command, script_index) + { result.push(("OpenCode mcp.command", script.clone())); } } @@ -316,6 +321,27 @@ fn is_script_path(argument: &str) -> bool { }) } +fn is_bun_package_script_operand(command: &[String], index: usize) -> bool { + let Some(executable) = command + .first() + .and_then(|executable| Path::new(executable).file_name()) + else { + return false; + }; + let executable = executable.to_string_lossy().to_ascii_lowercase(); + let executable = executable.strip_suffix(".exe").unwrap_or(&executable); + let Some(argument) = command.get(index) else { + return false; + }; + executable == "bun" + && command.get(1).is_some_and(|argument| argument == "run") + && !argument.starts_with('-') + && !argument.starts_with('.') + && !Path::new(argument).is_absolute() + && !argument.contains(['/', '\\']) + && !is_script_path(argument) +} + fn command_uses_script_runtime(executable: Option<&str>) -> bool { let Some(executable) = executable.and_then(|executable| Path::new(executable).file_name()) else { diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index b403fc580..1a873d062 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1265,6 +1265,58 @@ mod tests { ); } + #[test] + fn review_r7_bun_package_script_is_not_mistaken_for_a_file_path() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let package = owned.path().join("outside-mcp-package"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&package).unwrap(); + std::fs::write( + package.join("package.json"), + r#"{"scripts":{"mcp":"node ./server.js"}}"#, + ) + .unwrap(); + std::fs::write(package.join("server.js"), "process.exit(0)\n").unwrap(); + + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "package-script": { + "type": "local", + "cwd": package, + "command": ["bun", "run", "mcp"], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .expect("bun run <package-script> is a command, not a path to copy"); + + let selected = &plan[0].configured_resources; + assert!(selected.contains(&package.canonicalize().unwrap())); + assert!(package.join("package.json").is_file()); + assert!(package.join("server.js").is_file()); + assert!(!selected.contains(&workspace.join("mcp"))); + } + #[test] fn freshell_image_mcp_command_is_not_selected_as_a_host_resource() { let owned = tempfile::tempdir().unwrap(); From 17c03eb66108440821e6487d3e6b6b0428b5c59d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 17:44:11 -0700 Subject: [PATCH 344/493] fix(checkpoints): honor resolved session workspace scope --- crates/freshell-server/Cargo.toml | 3 + crates/freshell-server/src/checkpoints.rs | 564 ++++++++++++++++-- src/lib/api.ts | 134 +++-- src/store/sessionsThunks.ts | 1 + test/unit/client/lib/api.test.ts | 44 ++ test/unit/client/store/sessionsThunks.test.ts | 34 ++ 6 files changed, 669 insertions(+), 111 deletions(-) diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index fe403e92f..a2fa29b8c 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -160,6 +160,9 @@ libc = "0.2" windows-sys = { version = "=0.61.2", features = [ "Win32_Foundation", "Win32_Security", + "Win32_Security_Authorization", + "Win32_System_Environment", + "Win32_System_Registry", "Win32_System_Diagnostics_ToolHelp", "Win32_System_JobObjects", "Win32_System_Threading", diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index a4b75fa4f..cc27f7f32 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -2987,30 +2987,270 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { if let Some(server_home) = server_home { add_existing(server_home); } - for variable in ["HOME", "USERPROFILE"] { + if let Some(user_home) = std::env::home_dir() { + add_existing(&user_home); + for directory in [".claude", ".codex", ".opencode", ".amplifier"] { + add_existing(&user_home.join(directory)); + } + } + for variable in [ + "CLAUDE_HOME", + "CLAUDE_CONFIG_DIR", + "CODEX_HOME", + "OPENCODE_CONFIG_DIR", + "FRESHELL_AMPLIFIER_HOME", + ] { if let Some(value) = std::env::var_os(variable) { add_existing(Path::new(&value)); } } - #[cfg(unix)] - if let Ok(passwd) = std::fs::read_to_string("/etc/passwd") { - for line in passwd.lines() { - let fields = line.split(':').collect::<Vec<_>>(); - if fields.len() >= 6 && !fields[5].is_empty() { - add_existing(Path::new(fields[5])); + homes +} + +fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { + homes.iter().any(|home| home == path) +} + +fn is_account_home_for_owner( + path: &Path, + known_homes: &[PathBuf], + account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), +) -> Result<bool, String> { + if is_recognized_account_home(path, known_homes) { + return Ok(true); + } + let owner_home = account_home_for_directory(path)?; + Ok(owner_home == path) +} + +#[cfg(unix)] +fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { + use std::ffi::{CStr, OsString}; + use std::os::unix::ffi::OsStringExt; + use std::os::unix::fs::MetadataExt; + + let owner = std::fs::metadata(path) + .map_err(|error| format!("workspace owner cannot be read: {error}"))? + .uid(); + let mut capacity = 16 * 1024; + loop { + let mut account: libc::passwd = unsafe { std::mem::zeroed() }; + let mut buffer = vec![0u8; capacity]; + let mut result = std::ptr::null_mut(); + let status = unsafe { + libc::getpwuid_r( + owner, + &mut account, + buffer.as_mut_ptr().cast(), + buffer.len(), + &mut result, + ) + }; + if status == libc::ERANGE { + if capacity >= 1024 * 1024 { + return Err("workspace owner's account record exceeds the lookup bound".into()); } + capacity *= 2; + continue; + } + if status != 0 { + return Err(format!( + "workspace owner's account home cannot be resolved (getpwuid_r status={status})" + )); + } + if result.is_null() || account.pw_dir.is_null() { + return Err("workspace owner's account has no resolvable home directory".into()); + } + let home = unsafe { CStr::from_ptr(account.pw_dir) }; + if home.to_bytes().is_empty() { + return Err("workspace owner's account has an empty home directory".into()); } + let home = PathBuf::from(OsString::from_vec(home.to_bytes().to_vec())); + return std::fs::canonicalize(&home).map_err(|error| { + format!("workspace owner's account home cannot be resolved: {error}") + }); } - #[cfg(windows)] - if let (Some(drive), Some(path)) = (std::env::var_os("HOMEDRIVE"), std::env::var_os("HOMEPATH")) +} + +#[cfg(windows)] +fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { + use std::os::windows::ffi::OsStrExt; + use windows_sys::Win32::Foundation::{LocalFree, ERROR_SUCCESS, HLOCAL}; + use windows_sys::Win32::Security::Authorization::{ + ConvertSidToStringSidW, GetNamedSecurityInfoW, SE_FILE_OBJECT, + }; + use windows_sys::Win32::Security::{OWNER_SECURITY_INFORMATION, PSECURITY_DESCRIPTOR, PSID}; + use windows_sys::Win32::System::Environment::ExpandEnvironmentStringsW; + use windows_sys::Win32::System::Registry::{ + RegCloseKey, RegOpenKeyExW, RegQueryValueExW, HKEY, HKEY_LOCAL_MACHINE, KEY_READ, + REG_EXPAND_SZ, REG_SZ, + }; + + struct LocalAllocation(HLOCAL); + impl Drop for LocalAllocation { + fn drop(&mut self) { + if !self.0.is_null() { + unsafe { + LocalFree(self.0); + } + } + } + } + struct RegistryKey(HKEY); + impl Drop for RegistryKey { + fn drop(&mut self) { + if !self.0.is_null() { + unsafe { + RegCloseKey(self.0); + } + } + } + } + fn wide_string(pointer: *const u16) -> Result<String, String> { + if pointer.is_null() { + return Err("Windows account-home lookup returned an empty value".into()); + } + let mut length = 0usize; + while length < 4096 && unsafe { *pointer.add(length) } != 0 { + length += 1; + } + if length == 4096 { + return Err("Windows account-home lookup exceeded its string bound".into()); + } + String::from_utf16(unsafe { std::slice::from_raw_parts(pointer, length) }) + .map_err(|_| "Windows account-home lookup returned invalid UTF-16".into()) + } + + let path_wide = path + .as_os_str() + .encode_wide() + .chain(Some(0)) + .collect::<Vec<_>>(); + let mut owner_sid: PSID = std::ptr::null_mut(); + let mut descriptor: PSECURITY_DESCRIPTOR = std::ptr::null_mut(); + let status = unsafe { + GetNamedSecurityInfoW( + path_wide.as_ptr(), + SE_FILE_OBJECT, + OWNER_SECURITY_INFORMATION, + &mut owner_sid, + std::ptr::null_mut(), + std::ptr::null_mut(), + std::ptr::null_mut(), + &mut descriptor, + ) + }; + if status != ERROR_SUCCESS { + return Err(format!( + "workspace owner SID cannot be read (Windows error {status})" + )); + } + let _descriptor = LocalAllocation(descriptor.cast()); + let mut sid_string = std::ptr::null_mut(); + if unsafe { ConvertSidToStringSidW(owner_sid, &mut sid_string) } == 0 { + return Err("workspace owner SID cannot be formatted".into()); + } + let _sid_string = LocalAllocation(sid_string.cast()); + let sid = wide_string(sid_string)?; + let profile_key = + format!("SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\ProfileList\\{sid}"); + let profile_key = profile_key + .encode_utf16() + .chain(Some(0)) + .collect::<Vec<_>>(); + let mut raw_key: HKEY = std::ptr::null_mut(); + let status = unsafe { + RegOpenKeyExW( + HKEY_LOCAL_MACHINE, + profile_key.as_ptr(), + 0, + KEY_READ, + &mut raw_key, + ) + }; + if status != ERROR_SUCCESS { + return Err(format!( + "workspace owner SID has no readable Windows profile record (error {status})" + )); + } + let profile_key = RegistryKey(raw_key); + let value_name = "ProfileImagePath" + .encode_utf16() + .chain(Some(0)) + .collect::<Vec<_>>(); + let mut value_type = 0; + let mut byte_length = 0; + let status = unsafe { + RegQueryValueExW( + profile_key.0, + value_name.as_ptr(), + std::ptr::null(), + &mut value_type, + std::ptr::null_mut(), + &mut byte_length, + ) + }; + if status != ERROR_SUCCESS + || byte_length == 0 + || byte_length > 64 * 1024 + || byte_length % 2 != 0 { - add_existing(&PathBuf::from(drive).join(path)); + return Err(format!( + "workspace owner Windows profile path cannot be read (error {status})" + )); } - homes + if value_type != REG_SZ && value_type != REG_EXPAND_SZ { + return Err("workspace owner Windows profile path has an unsupported registry type".into()); + } + let mut value = vec![0u16; byte_length as usize / 2]; + let status = unsafe { + RegQueryValueExW( + profile_key.0, + value_name.as_ptr(), + std::ptr::null(), + &mut value_type, + value.as_mut_ptr().cast(), + &mut byte_length, + ) + }; + if status != ERROR_SUCCESS || byte_length as usize > value.len() * 2 { + return Err(format!( + "workspace owner Windows profile path read failed (error {status})" + )); + } + value.truncate(byte_length as usize / 2); + while value.last() == Some(&0) { + value.pop(); + } + let profile = String::from_utf16(&value) + .map_err(|_| "workspace owner Windows profile path is invalid UTF-16".to_string())?; + let profile = if value_type == REG_EXPAND_SZ { + let source = profile.encode_utf16().chain(Some(0)).collect::<Vec<_>>(); + let required = + unsafe { ExpandEnvironmentStringsW(source.as_ptr(), std::ptr::null_mut(), 0) }; + if required == 0 || required > 32 * 1024 { + return Err("workspace owner Windows profile path cannot be expanded".into()); + } + let mut expanded = vec![0u16; required as usize]; + let written = + unsafe { ExpandEnvironmentStringsW(source.as_ptr(), expanded.as_mut_ptr(), required) }; + if written == 0 || written > required { + return Err("workspace owner Windows profile path expansion failed".into()); + } + expanded.truncate(written.saturating_sub(1) as usize); + String::from_utf16(&expanded).map_err(|_| { + "workspace owner Windows profile path expansion is invalid UTF-16".to_string() + })? + } else { + profile + }; + std::fs::canonicalize(PathBuf::from(profile)) + .map_err(|error| format!("workspace owner Windows profile cannot be resolved: {error}")) } -fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { - homes.iter().any(|home| home == path) +#[cfg(not(any(unix, windows)))] +fn account_home_for_directory(_path: &Path) -> Result<PathBuf, String> { + Err("workspace owner account-home lookup is unavailable on this platform".into()) } async fn canonical_workspace(path: &Path) -> Result<PathBuf, String> { @@ -3025,17 +3265,58 @@ async fn canonical_workspace(path: &Path) -> Result<PathBuf, String> { .map_err(|error| format!("workspace cannot be resolved: {error}")) } -async fn git_checkout_root(path: &Path) -> Option<PathBuf> { - let output = run_git(&["rev-parse", "--show-toplevel"], Some(path)) +async fn git_checkout_root(path: &Path) -> Result<Option<PathBuf>, String> { + let output = match run_git(&["rev-parse", "--show-toplevel"], Some(path)).await { + Ok(output) => output, + Err(error) if error.contains("not a git repository") => { + if has_git_marker_in_ancestors(path)? { + return Err(error); + } + return Ok(None); + } + Err(error) => return Err(error), + }; + tokio::fs::canonicalize(output.trim()) .await - .ok()?; - tokio::fs::canonicalize(output.trim()).await.ok() + .map(Some) + .map_err(|error| format!("Git reported a checkout root that cannot be resolved: {error}")) +} + +fn has_git_marker_in_ancestors(path: &Path) -> Result<bool, String> { + for ancestor in path.ancestors() { + match std::fs::symlink_metadata(ancestor.join(".git")) { + Ok(_) => return Ok(true), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => { + return Err(format!( + "Git checkout marker cannot be inspected at {}: {error}", + ancestor.display() + )); + } + } + } + Ok(false) } async fn resolve_checkpoint_workspace( server_home: &Path, launch_cwd: &Path, intent: Option<&Value>, +) -> Result<ResolvedCheckpointWorkspace, CheckpointWorkspaceUnavailable> { + resolve_checkpoint_workspace_with_account_home( + server_home, + launch_cwd, + intent, + &account_home_for_directory, + ) + .await +} + +async fn resolve_checkpoint_workspace_with_account_home( + server_home: &Path, + launch_cwd: &Path, + intent: Option<&Value>, + account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), ) -> Result<ResolvedCheckpointWorkspace, CheckpointWorkspaceUnavailable> { let candidate = intent .and_then(|workspace| workspace.get("path")) @@ -3064,26 +3345,70 @@ async fn resolve_checkpoint_workspace( candidate: candidate.clone(), reason, })?; - let explicitly_selected_home = - provenance == "directory-picker" && is_recognized_account_home(&selected, &homes); - if is_recognized_account_home(&selected, &homes) && !explicitly_selected_home { - return Err(CheckpointWorkspaceUnavailable { - candidate, - reason: "account home cannot be used as a code checkpoint workspace".to_string(), + if provenance == "legacy-checkpoint" { + return Ok(ResolvedCheckpointWorkspace { + path: selected, + provenance, }); } - let path = if provenance == "legacy-checkpoint" || explicitly_selected_home { - selected - } else { - git_checkout_root(&selected).await.unwrap_or(selected) - }; - if is_recognized_account_home(&path, &homes) && !explicitly_selected_home { + let selected_home = + match is_account_home_for_owner(&selected, &homes, account_home_for_directory) { + Ok(is_home) => Some(is_home), + Err(_reason) if provenance == "directory-picker" => None, + Err(reason) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: format!("workspace account scope is unknown: {reason}"), + }); + } + }; + if provenance == "directory-picker" && selected_home == Some(true) { + return Ok(ResolvedCheckpointWorkspace { + path: selected, + provenance, + }); + } + if selected_home == Some(true) { return Err(CheckpointWorkspaceUnavailable { candidate, reason: "account home cannot be used as an inferred code checkpoint workspace" .to_string(), }); } + let path = match git_checkout_root(&selected).await { + Ok(Some(root)) if provenance == "directory-picker" && root == selected => selected, + Ok(Some(root)) => { + if selected_home.is_none() { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "workspace account scope is unknown; the selected directory cannot be widened to its Git root".into(), + }); + } + match is_account_home_for_owner(&root, &homes, account_home_for_directory) { + Ok(true) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home cannot be used as an inferred code checkpoint workspace".into(), + }); + } + Ok(false) => root, + Err(reason) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: format!("Git checkout account scope is unknown: {reason}"), + }); + } + } + } + Ok(None) if provenance == "directory-picker" => selected, + Ok(None) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "the selected session checkout is no longer a Git project".into(), + }); + } + Err(reason) => return Err(CheckpointWorkspaceUnavailable { candidate, reason }), + }; return Ok(ResolvedCheckpointWorkspace { path, provenance }); } @@ -3094,25 +3419,49 @@ async fn resolve_checkpoint_workspace( candidate: candidate.clone(), reason, })?; - if is_recognized_account_home(&launch, &homes) { - return Err(CheckpointWorkspaceUnavailable { - candidate, - reason: "account home has no selected code checkpoint workspace".to_string(), - }); + match is_account_home_for_owner(&launch, &homes, account_home_for_directory) { + Ok(true) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home has no selected code checkpoint workspace".to_string(), + }); + } + Ok(false) => {} + Err(reason) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: format!("workspace account scope is unknown: {reason}"), + }); + } } - let root = git_checkout_root(&launch) - .await - .ok_or_else(|| CheckpointWorkspaceUnavailable { - candidate: candidate.clone(), - reason: "no explicit workspace or Git project root is available".to_string(), - })?; + let root = match git_checkout_root(&launch).await { + Ok(Some(root)) => root, + Ok(None) => { + return Err(CheckpointWorkspaceUnavailable { + candidate: candidate.clone(), + reason: "no explicit workspace or Git project root is available".to_string(), + }); + } + Err(reason) => { + return Err(CheckpointWorkspaceUnavailable { candidate, reason }); + } + }; let path = root; - if is_recognized_account_home(&path, &homes) { - return Err(CheckpointWorkspaceUnavailable { - candidate, - reason: "account home cannot be used as an inferred code checkpoint workspace" - .to_string(), - }); + match is_account_home_for_owner(&path, &homes, account_home_for_directory) { + Ok(true) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home cannot be used as an inferred code checkpoint workspace" + .to_string(), + }); + } + Ok(false) => {} + Err(reason) => { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: format!("Git checkout account scope is unknown: {reason}"), + }); + } } Ok(ResolvedCheckpointWorkspace { path, @@ -3564,12 +3913,14 @@ fn checkpoint_storage_exclusion( let home = storage.parent().and_then(Path::parent).ok_or_else(|| { "checkpoint operation resolve-storage failed: storage path has no home anchor".to_string() })?; - let canonical_home = std::fs::canonicalize(home).map_err(|_| { + std::fs::canonicalize(home).map_err(|_| { "checkpoint operation resolve-storage failed: home anchor cannot be resolved".to_string() })?; - // Canonicalize the home anchor so aliases compare against the canonical - // workspace, then retain the logical suffix for Git's pathspec. - let storage = canonical_home.join(".freshell").join("checkpoints"); + // Resolve the actual directory after following links in either the home + // alias or `.freshell`; Git traverses the target's real in-worktree path. + let storage = std::fs::canonicalize(storage).map_err(|error| { + format!("checkpoint operation resolve-storage failed: checkpoint directory cannot be resolved: {error}") + })?; if work_tree.starts_with(&storage) { return Err( "checkpoint operation resolve-storage failed: workspace overlaps Freshell checkpoint storage" @@ -4704,10 +5055,14 @@ async fn get_checkpoints( let candidate_for_home_check = tokio::fs::canonicalize(&legacy_path) .await .unwrap_or_else(|_| legacy_path.clone()); - if !is_recognized_account_home( + let known_homes = recognized_account_homes(Some(&state.home)); + let allow_legacy_scope = is_account_home_for_owner( &candidate_for_home_check, - &recognized_account_homes(Some(&state.home)), - ) { + &known_homes, + &account_home_for_directory, + ) + .is_ok_and(|is_home| !is_home); + if allow_legacy_scope { let already_present = workspaces .iter() .any(|workspace| workspace.path == legacy_path); @@ -6703,6 +7058,63 @@ mod tests { assert!(error.reason.contains("account home"), "{}", error.reason); } + #[tokio::test] + async fn picker_git_lookup_error_does_not_fall_back_to_a_partial_subdirectory() { + let owned = tempfile::tempdir().unwrap(); + let account_home = owned.path().join("account-home"); + let selected = account_home.join("project/src"); + tokio::fs::create_dir_all(&selected).await.unwrap(); + run_git(&["init", "-q"], Some(&account_home)).await.unwrap(); + tokio::fs::write(account_home.join(".git/config"), "[core\n") + .await + .unwrap(); + let intent = json!({ + "path": selected, + "provenance": "directory-picker", + }); + + let error = resolve_checkpoint_workspace(owned.path(), &selected, Some(&intent)) + .await + .expect_err("a failed Git lookup must not authorize a partial nested snapshot"); + + assert!( + error.reason.contains("Git operation rev-parse failed"), + "{}", + error.reason + ); + assert!(error.reason.contains("bad config line"), "{}", error.reason); + } + + #[tokio::test] + async fn inferred_foreign_account_home_with_dotfiles_git_is_rejected_by_owner_identity() { + let owned = tempfile::tempdir().unwrap(); + let server_home = owned.path().join("server-profile"); + let foreign_home = owned.path().join("nss-profile"); + tokio::fs::create_dir_all(&server_home).await.unwrap(); + tokio::fs::create_dir_all(&foreign_home).await.unwrap(); + run_git(&["init", "-q"], Some(&foreign_home)).await.unwrap(); + let foreign_home = foreign_home.canonicalize().unwrap(); + let owner_home = foreign_home.clone(); + let account_home_for_directory = move |path: &Path| { + if path == owner_home { + Ok(owner_home.clone()) + } else { + Err("synthetic NSS owner has no resolvable home".to_string()) + } + }; + + let error = resolve_checkpoint_workspace_with_account_home( + &server_home, + &foreign_home, + None, + &account_home_for_directory, + ) + .await + .expect_err("a foreign account's dotfiles checkout cannot become an inferred scope"); + + assert!(error.reason.contains("account home"), "{}", error.reason); + } + #[tokio::test] async fn review_r6_automatic_checkpoint_missing_workspace_is_logged() { use tracing::instrument::WithSubscriber; @@ -6957,6 +7369,52 @@ mod tests { assert!(error.contains("overlaps Freshell checkpoint storage")); } + #[cfg(unix)] + #[tokio::test] + async fn explicit_picker_home_excludes_checkpoint_store_through_in_home_freshell_symlink() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let real_freshell_dir = home.join("runtime-state"); + tokio::fs::create_dir_all(&real_freshell_dir).await.unwrap(); + tokio::fs::write(real_freshell_dir.join("unrelated-state.txt"), "keep") + .await + .unwrap(); + tokio::fs::write(home.join("private.txt"), "private") + .await + .unwrap(); + std::os::unix::fs::symlink(&real_freshell_dir, home.join(".freshell")).unwrap(); + let workspace = home.canonicalize().unwrap(); + + let response = post_checkpoint( + State(state(&home)), + headers_with_token("tok"), + Json(json!({ + "cwd": workspace, + "workspace": { + "path": workspace, + "provenance": "directory-picker", + }, + })), + ) + .await; + let status = response.status(); + let entry = body_json(response).await; + assert_eq!(status, StatusCode::OK, "{entry}"); + + let git_dir = checkpoint_git_dir(&home, &home); + let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) + .await + .expect("published picker-home checkpoint index is readable"); + assert!(index.iter().any(|entry| entry.path == b"private.txt")); + assert!(index + .iter() + .any(|entry| entry.path == b"runtime-state/unrelated-state.txt")); + assert!(!index.iter().any(|entry| { + entry.path.starts_with(b"runtime-state/checkpoints/") + || entry.path == b"runtime-state/checkpoints" + })); + } + #[tokio::test] async fn resolved_project_scope_independent_of_routing_cwd() { let home_dir = tempfile::tempdir().unwrap(); diff --git a/src/lib/api.ts b/src/lib/api.ts index 36099857a..0f6952a9c 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -8,7 +8,11 @@ import { FreshAgentApiContractError } from '@/lib/fresh-agent-api-error' import { getClientPerfConfig, isClientPerfLoggingEnabled, logClientPerf } from '@/lib/perf-logger' import { getAuthToken } from '@/lib/auth' import { sanitizeSessionLocators } from '@/lib/session-utils' -import type { SessionLocator } from '@/store/paneTypes' +import { + normalizeFreshAgentCheckpointWorkspace, + type FreshAgentCheckpointWorkspace, + type SessionLocator, +} from '@/store/paneTypes' import type { RecoveryInventory } from '@/lib/recovery/types' import type { Machine } from '@/lib/machine-identity' import { @@ -735,6 +739,7 @@ export type SearchResult = { provider: CodingCliProviderName projectPath: string checkoutPath?: string + checkpointWorkspace?: FreshAgentCheckpointWorkspace title?: string summary?: string sessionType?: string @@ -828,38 +833,44 @@ function groupDirectoryItemsAsProjects( // selectors see it. Absent map (older server) → no color, exactly the // pre-SESSION-05 behavior. ...(projectColors?.[projectPath] ? { color: projectColors[projectPath] } : {}), - sessions: sessions.map((item) => ({ - provider: item.provider, - sessionId: item.sessionId, - projectPath: item.projectPath, - ...(item.checkoutPath ? { checkoutPath: item.checkoutPath } : {}), - lastActivityAt: item.lastActivityAt, - createdAt: item.createdAt, - archived: item.archived, - cwd: item.cwd, - title: item.title, - summary: item.summary, - isSubagent: item.isSubagent, - isNonInteractive: item.isNonInteractive, - isRunning: item.isRunning, - runningTerminalId: item.runningTerminalId, - liveTerminalOnly: item.liveTerminalOnly, - firstUserMessage: item.firstUserMessage, - sessionType: item.sessionType, - // STATUS-STRIP: window rows carry live usage so the fresh-agent strip - // context meter reads the same data the sidebar fetches. - ...(item.tokenUsage ? { tokenUsage: item.tokenUsage } : {}), - // b5fb: the reviewed reset flow's provenance fields (allowlist mapper — - // forward explicitly or they never reach Redux). - ...(item.titleOverridden ? { titleOverridden: true } : {}), - ...(item.providerTitle !== undefined ? { providerTitle: item.providerTitle } : {}), - ...(item.titleOverrideSource ? { titleOverrideSource: item.titleOverrideSource } : {}), - // Unified agent names (Task 2 projection): the row's canonical ref + - // last-known name — a fresh second client's sessionNames cache - // bootstraps from these (its ready-time read races its hydration). - ...(item.nameRef ? { nameRef: item.nameRef } : {}), - ...(item.sessionName !== undefined ? { sessionName: item.sessionName } : {}), - })), + sessions: sessions.map((item) => { + const checkpointWorkspace = normalizeFreshAgentCheckpointWorkspace( + item.checkoutPath ? { path: item.checkoutPath, provenance: 'resolved-project' } : undefined, + ) + return { + provider: item.provider, + sessionId: item.sessionId, + projectPath: item.projectPath, + ...(item.checkoutPath ? { checkoutPath: item.checkoutPath } : {}), + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), + lastActivityAt: item.lastActivityAt, + createdAt: item.createdAt, + archived: item.archived, + cwd: item.cwd, + title: item.title, + summary: item.summary, + isSubagent: item.isSubagent, + isNonInteractive: item.isNonInteractive, + isRunning: item.isRunning, + runningTerminalId: item.runningTerminalId, + liveTerminalOnly: item.liveTerminalOnly, + firstUserMessage: item.firstUserMessage, + sessionType: item.sessionType, + // STATUS-STRIP: window rows carry live usage so the fresh-agent strip + // context meter reads the same data the sidebar fetches. + ...(item.tokenUsage ? { tokenUsage: item.tokenUsage } : {}), + // b5fb: the reviewed reset flow's provenance fields (allowlist mapper — + // forward explicitly or they never reach Redux). + ...(item.titleOverridden ? { titleOverridden: true } : {}), + ...(item.providerTitle !== undefined ? { providerTitle: item.providerTitle } : {}), + ...(item.titleOverrideSource ? { titleOverrideSource: item.titleOverrideSource } : {}), + // Unified agent names (Task 2 projection): the row's canonical ref + + // last-known name — a fresh second client's sessionNames cache + // bootstraps from these (its ready-time read races its hydration). + ...(item.nameRef ? { nameRef: item.nameRef } : {}), + ...(item.sessionName !== undefined ? { sessionName: item.sessionName } : {}), + } + }), })) } @@ -1110,31 +1121,38 @@ export async function searchSessions(options: SearchOptions): Promise<SearchResp })) as ReadModelSessionDirectoryPage const response: SearchResponse = { - results: page.items.map((item) => ({ - sessionId: item.sessionId, - provider: item.provider, - projectPath: item.projectPath, - title: item.title, - summary: item.summary, - matchedIn: item.matchedIn === 'firstUserMessage' ? 'userMessage' : item.matchedIn ?? 'title', - snippet: item.snippet, - lastActivityAt: item.lastActivityAt, - createdAt: item.createdAt, - archived: item.archived, - cwd: item.cwd, - sessionType: item.sessionType, - firstUserMessage: item.firstUserMessage, - isSubagent: item.isSubagent, - isNonInteractive: item.isNonInteractive, - isRunning: item.isRunning, - runningTerminalId: item.runningTerminalId, - liveTerminalOnly: item.liveTerminalOnly, - ...(item.tokenUsage ? { tokenUsage: item.tokenUsage } : {}), - // b5fb: forward the reset-flow provenance fields (allowlist mapper). - ...(item.titleOverridden ? { titleOverridden: true } : {}), - ...(item.providerTitle !== undefined ? { providerTitle: item.providerTitle } : {}), - ...(item.titleOverrideSource ? { titleOverrideSource: item.titleOverrideSource } : {}), - })), + results: page.items.map((item) => { + const checkpointWorkspace = normalizeFreshAgentCheckpointWorkspace( + item.checkoutPath ? { path: item.checkoutPath, provenance: 'resolved-project' } : undefined, + ) + return { + sessionId: item.sessionId, + provider: item.provider, + projectPath: item.projectPath, + ...(item.checkoutPath ? { checkoutPath: item.checkoutPath } : {}), + ...(checkpointWorkspace ? { checkpointWorkspace } : {}), + title: item.title, + summary: item.summary, + matchedIn: item.matchedIn === 'firstUserMessage' ? 'userMessage' : item.matchedIn ?? 'title', + snippet: item.snippet, + lastActivityAt: item.lastActivityAt, + createdAt: item.createdAt, + archived: item.archived, + cwd: item.cwd, + sessionType: item.sessionType, + firstUserMessage: item.firstUserMessage, + isSubagent: item.isSubagent, + isNonInteractive: item.isNonInteractive, + isRunning: item.isRunning, + runningTerminalId: item.runningTerminalId, + liveTerminalOnly: item.liveTerminalOnly, + ...(item.tokenUsage ? { tokenUsage: item.tokenUsage } : {}), + // b5fb: forward the reset-flow provenance fields (allowlist mapper). + ...(item.titleOverridden ? { titleOverridden: true } : {}), + ...(item.providerTitle !== undefined ? { providerTitle: item.providerTitle } : {}), + ...(item.titleOverrideSource ? { titleOverrideSource: item.titleOverrideSource } : {}), + } + }), tier, query, totalScanned: page.items.length, diff --git a/src/store/sessionsThunks.ts b/src/store/sessionsThunks.ts index af5544d2d..4711e782e 100644 --- a/src/store/sessionsThunks.ts +++ b/src/store/sessionsThunks.ts @@ -91,6 +91,7 @@ function searchResultsToProjects( sessionId: result.sessionId, projectPath: result.projectPath, ...(result.checkoutPath ? { checkoutPath: result.checkoutPath } : {}), + ...(result.checkpointWorkspace ? { checkpointWorkspace: result.checkpointWorkspace } : {}), lastActivityAt: result.lastActivityAt, createdAt: result.createdAt, archived: result.archived, diff --git a/test/unit/client/lib/api.test.ts b/test/unit/client/lib/api.test.ts index 3dbcbbd7f..3929672d2 100644 --- a/test/unit/client/lib/api.test.ts +++ b/test/unit/client/lib/api.test.ts @@ -983,6 +983,50 @@ describe('searchSessions cursor pagination', () => { expect(response.hasMore).toBe(false) }) + it('preserves linked-worktree checkout scope for session opens from snapshots and search', async () => { + const checkoutPath = '/repo/.worktrees/feature-a' + mockFetch + .mockResolvedValueOnce(mockJson({ + items: [{ + sessionId: 'session-worktree', + provider: 'codex', + projectPath: '/repo', + checkoutPath, + cwd: '/home/user', + isRunning: false, + lastActivityAt: 1_000, + }], + nextCursor: null, + revision: 1, + })) + .mockResolvedValueOnce(mockJson({ + items: [{ + sessionId: 'session-worktree', + provider: 'codex', + projectPath: '/repo', + checkoutPath, + cwd: '/home/user', + matchedIn: 'title', + isRunning: false, + lastActivityAt: 1_000, + }], + nextCursor: null, + revision: 2, + })) + + const snapshot = await fetchSidebarSessionsSnapshot() + const search = await searchSessions({ query: 'worktree' }) + + expect(snapshot.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual({ + path: checkoutPath, + provenance: 'resolved-project', + }) + expect(search.results[0]?.checkpointWorkspace).toEqual({ + path: checkoutPath, + provenance: 'resolved-project', + }) + }) + it('forwards the cursor to the session-directory request when paginating a search', async () => { mockFetch.mockResolvedValueOnce(mockJson({ items: [], diff --git a/test/unit/client/store/sessionsThunks.test.ts b/test/unit/client/store/sessionsThunks.test.ts index 07393b7c8..21f99e5a2 100644 --- a/test/unit/client/store/sessionsThunks.test.ts +++ b/test/unit/client/store/sessionsThunks.test.ts @@ -341,6 +341,40 @@ describe('sessionsThunks', () => { }) }) + it('keeps inferred linked-worktree checkpoint scope on search session rows', async () => { + searchSessions.mockResolvedValueOnce({ + results: [{ + provider: 'codex', + sessionId: 'session-worktree', + projectPath: '/repo', + checkoutPath: '/repo/.worktrees/feature-a', + checkpointWorkspace: { + path: '/repo/.worktrees/feature-a', + provenance: 'resolved-project', + }, + matchedIn: 'title', + lastActivityAt: 1_000, + }], + tier: 'title', + query: 'worktree', + totalScanned: 1, + }) + + const store = createStore() + store.dispatch(setActiveSessionSurface('sidebar')) + + await store.dispatch(fetchSessionWindow({ + surface: 'sidebar', + priority: 'visible', + query: 'worktree', + }) as any) + + expect(store.getState().sessions.windows.sidebar.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual({ + path: '/repo/.worktrees/feature-a', + provenance: 'resolved-project', + }) + }) + it('keeps tier changes and clearing a non-empty query in the visible search intent', async () => { const tierChange = createDeferred<any>() const clearSearch = createDeferred<any>() From 87531f3c1110280cd2a67028e2b3b3c7730ed1ad Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 18:02:31 -0700 Subject: [PATCH 345/493] fix(managed): resolve checkpoint scope and Bun scripts --- .../opencode_resources.rs | 170 ++++++++++++++---- .../managed_provider_references/sources.rs | 145 +++++++++++++++ src/lib/api.ts | 16 +- test/unit/client/lib/api.test.ts | 45 +++++ 4 files changed, 340 insertions(+), 36 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index ccbc6dae6..94cdfd8b7 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1,9 +1,11 @@ //! Finite native ordinary resource fields. Keep raw documents and every //! unrelated string intact; these selections are paths, never private payloads. use super::sources::PlannedSourceOccurrence; +use std::io::Read; use std::path::{Path, PathBuf}; const FRESHELL_IMAGE_MCP_SERVER_PATH: &str = "/opt/freshell-mcp/server.js"; +const MAX_BUN_PACKAGE_JSON_BYTES: u64 = 16 * 1024 * 1024; #[derive(Clone, Copy)] enum Token { @@ -247,7 +249,7 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' index > 0 && index <= script && is_script_path(argument) - && !is_bun_package_script_operand(&command, index) + && !is_bun_run_bare_operand(&command, index) }) } else { is_script_path(argument) @@ -258,9 +260,9 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' } if let Some(script_index) = script_index { let script = &command[script_index]; - if !is_script_path(script) - && !is_bun_package_script_operand(&command, script_index) - { + if is_bun_run_bare_operand(&command, script_index) { + result.push(("OpenCode bun run operand", script.clone())); + } else if !is_script_path(script) { result.push(("OpenCode mcp.command", script.clone())); } } @@ -321,7 +323,7 @@ fn is_script_path(argument: &str) -> bool { }) } -fn is_bun_package_script_operand(command: &[String], index: usize) -> bool { +fn is_bun_run_bare_operand(command: &[String], index: usize) -> bool { let Some(executable) = command .first() .and_then(|executable| Path::new(executable).file_name()) @@ -335,11 +337,54 @@ fn is_bun_package_script_operand(command: &[String], index: usize) -> bool { }; executable == "bun" && command.get(1).is_some_and(|argument| argument == "run") - && !argument.starts_with('-') - && !argument.starts_with('.') + && is_bare_bun_script_name(argument) +} + +fn is_bare_bun_script_name(argument: &str) -> bool { + !argument.is_empty() + && !argument.starts_with(['-', '.']) && !Path::new(argument).is_absolute() && !argument.contains(['/', '\\']) - && !is_script_path(argument) +} + +fn bun_package_script_root( + working_dir: &Path, + script_name: &str, +) -> Result<Option<PathBuf>, &'static str> { + for directory in working_dir.ancestors().take(64) { + if directory == Path::new("/") { + break; + } + let manifest = directory.join("package.json"); + let metadata = match std::fs::metadata(&manifest) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => return Err("Bun package.json is unavailable"), + }; + if !metadata.is_file() { + return Err("Bun package.json is not a regular file"); + } + if metadata.len() > MAX_BUN_PACKAGE_JSON_BYTES { + return Err("Bun package.json exceeds the 16 MiB script lookup limit"); + } + let mut bytes = Vec::with_capacity(metadata.len() as usize); + std::fs::File::open(&manifest) + .map_err(|_| "Bun package.json is unreadable")? + .take(MAX_BUN_PACKAGE_JSON_BYTES + 1) + .read_to_end(&mut bytes) + .map_err(|_| "Bun package.json could not be read")?; + if bytes.len() as u64 > MAX_BUN_PACKAGE_JSON_BYTES { + return Err("Bun package.json exceeds the 16 MiB script lookup limit"); + } + let package: serde_json::Value = + serde_json::from_slice(&bytes).map_err(|_| "Bun package.json is not valid JSON")?; + let has_script = package + .get("scripts") + .and_then(serde_json::Value::as_object) + .is_some_and(|scripts| scripts.contains_key(script_name)); + return Ok(has_script.then(|| directory.to_path_buf())); + } + Ok(None) } fn command_uses_script_runtime(executable: Option<&str>) -> bool { @@ -541,6 +586,28 @@ fn checked(path: &Path, field: &'static str, _source: &Path) -> Result<PathBuf, Ok(canonical) } +fn insert_resource( + result: &mut std::collections::BTreeSet<PathBuf>, + resource_path_bytes: &mut usize, + path: PathBuf, + field: &'static str, +) -> Result<(), ResourceError> { + if !result.contains(&path) { + let bytes = path.as_os_str().len(); + if result.len() >= 100_000 + || bytes > (64 * 1024 * 1024usize).saturating_sub(*resource_path_bytes) + { + return Err(ResourceError { + field, + cause: "configured resource selection exceeds shared entry or metadata profile", + }); + } + *resource_path_bytes += bytes; + result.insert(path); + } + Ok(()) +} + /// Return complete ordinary units and native installed ancestor lookup units. /// Exact mounts retain native field spellings and their declaring origins. pub fn select( @@ -553,22 +620,6 @@ pub fn select( let fail = |field: &'static str, cause: &'static str| ResourceError { field, cause }; let mut result = std::collections::BTreeSet::new(); let mut resource_path_bytes = 0usize; - let mut insert = |path: PathBuf, field| { - if !result.contains(&path) { - let bytes = path.as_os_str().len(); - if result.len() >= 100_000 - || bytes > (64 * 1024 * 1024usize).saturating_sub(resource_path_bytes) - { - return Err(fail( - field, - "configured resource selection exceeds shared entry or metadata profile", - )); - } - resource_path_bytes += bytes; - result.insert(path); - } - Ok(()) - }; let mut variables = std::collections::BTreeSet::new(); let raw = source.references.unexpanded_source(); let mut selections = fields(raw).map_err(|cause| fail("OpenCode resource", cause))?; @@ -577,9 +628,12 @@ pub fn select( .map_err(|cause| fail("OpenCode resource", cause))?, ); let mut current_mcp_working_dir: Option<PathBuf> = None; + let mut current_mcp_resource_path: Option<PathBuf> = None; for (field, spelling) in selections { + let mut field = field; if field == "OpenCode mcp.entry" { current_mcp_working_dir = None; + current_mcp_resource_path = None; continue; } // Private file-token payloads already use the accepted grant plan. @@ -617,6 +671,56 @@ pub fn select( if introduced_file.get() { continue; } + if field == "OpenCode bun run operand" { + let working_dir = current_mcp_resource_path + .as_deref() + .unwrap_or(&source.instance_cwd); + if is_bare_bun_script_name(&spelling) { + if let Some(package_root) = bun_package_script_root(working_dir, &spelling) + .map_err(|cause| fail("OpenCode mcp.command", cause))? + { + let canonical_package_root = + checked(&package_root, "OpenCode mcp.command", declaring)?; + if !canonical_package_root.is_dir() { + return Err(fail( + "OpenCode mcp.command", + "Bun package script root is not a directory", + )); + } + let canonical_home = std::fs::canonicalize(home).ok(); + let canonical_working_dir = current_mcp_working_dir + .clone() + .or_else(|| std::fs::canonicalize(working_dir).ok()); + if canonical_package_root == Path::new("/") + || (canonical_home.as_deref() == Some(canonical_package_root.as_path()) + && canonical_working_dir.as_deref() + != Some(canonical_package_root.as_path())) + { + return Err(fail( + "OpenCode mcp.command", + "Bun package script requires copying an unbounded account or system root", + )); + } + if let Some(cwd_resource_path) = ¤t_mcp_resource_path { + if cwd_resource_path != &package_root + && cwd_resource_path.starts_with(&package_root) + && result.remove(cwd_resource_path) + { + resource_path_bytes = resource_path_bytes + .saturating_sub(cwd_resource_path.as_os_str().len()); + } + } + insert_resource( + &mut result, + &mut resource_path_bytes, + package_root, + "OpenCode mcp.command", + )?; + continue; + } + } + field = "OpenCode mcp.command"; + } if field != "OpenCode attention.sounds" && field != "OpenCode mcp.cwd" && (spelling.starts_with("http://") || spelling.starts_with("https://")) @@ -688,7 +792,7 @@ pub fn select( if !canonical.is_file() { return Err(fail(field, "selected file resource is not a regular file")); } - insert(selected, field)?; + insert_resource(&mut result, &mut resource_path_bytes, selected, field)?; matched = true; } if matched { @@ -749,7 +853,7 @@ pub fn select( if !canonical.is_file() { return Err(fail(field, "selected file resource is not a regular file")); } - insert(path, field)?; + insert_resource(&mut result, &mut resource_path_bytes, path, field)?; continue; } if field == "OpenCode mcp.command" { @@ -772,7 +876,12 @@ pub fn select( } // Retain the exact executable/script path as well as its bounded // package closure below when it belongs to an installed package. - insert(canonical.clone(), field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical.clone(), + field, + )?; } if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { if !canonical.is_dir() { @@ -783,8 +892,9 @@ pub fn select( } if field == "OpenCode mcp.cwd" { current_mcp_working_dir = Some(canonical); + current_mcp_resource_path = Some(path.clone()); } - insert(path, field)?; + insert_resource(&mut result, &mut resource_path_bytes, path, field)?; continue; } // The native global plugin discovery unit is already selected in the @@ -823,7 +933,7 @@ pub fn select( } return Err(fail(field, "select a bounded plugin or package directory")); } - insert(scope.clone(), field)?; + insert_resource(&mut result, &mut resource_path_bytes, scope.clone(), field)?; for ancestor in scope.parent().into_iter().flat_map(Path::ancestors) { if ancestor == Path::new("/") { break; @@ -840,7 +950,7 @@ pub fn select( match std::fs::symlink_metadata(&candidate) { Ok(_) => { checked(&candidate, field, declaring)?; - insert(candidate, field)?; + insert_resource(&mut result, &mut resource_path_bytes, candidate, field)?; } Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} Err(_) => { diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 1a873d062..5ae4af919 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1317,6 +1317,151 @@ mod tests { assert!(!selected.contains(&workspace.join("mcp"))); } + #[test] + fn review_r8_bun_package_script_name_with_extension_uses_nearest_manifest() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let package = owned.path().join("outside-mcp-package"); + let nested = package.join("nested"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&nested).unwrap(); + std::fs::write( + package.join("package.json"), + r#"{"scripts":{"mcp.js":"node ./server.js"}}"#, + ) + .unwrap(); + std::fs::write(package.join("server.js"), "process.exit(0)\n").unwrap(); + + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "package-script": { + "type": "local", + "cwd": nested, + "command": ["bun", "run", "mcp.js"], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .expect("bun run gives the matching package script precedence over a same-named file"); + + assert!(plan[0] + .configured_resources + .contains(&package.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&nested.canonicalize().unwrap())); + assert!(package.join("package.json").is_file()); + assert!(package.join("server.js").is_file()); + assert!(!package.join("mcp.js").exists()); + } + + #[test] + fn bun_run_bare_script_file_remains_a_resource_without_matching_package_script() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(workspace.join("package.json"), r#"{"scripts":{}}"#).unwrap(); + std::fs::write(workspace.join("mcp.js"), "process.exit(0)\n").unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "file-command": { + "type": "local", + "command": ["bun", "run", "mcp.js"], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .expect("without a matching package script, a bare Bun file operand remains selectable"); + + assert!(plan[0] + .configured_resources + .contains(&workspace.join("mcp.js").canonicalize().unwrap())); + } + + #[test] + fn bun_run_explicit_relative_file_wins_over_same_named_package_script() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + workspace.join("package.json"), + r#"{"scripts":{"mcp.js":"echo package-script"}}"#, + ) + .unwrap(); + std::fs::write(workspace.join("mcp.js"), "process.exit(0)\n").unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "file-command": { + "type": "local", + "command": ["bun", "run", "./mcp.js"], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .expect("Bun treats an explicit ./ operand as a source file"); + + assert!(plan[0] + .configured_resources + .contains(&workspace.join("mcp.js").canonicalize().unwrap())); + } + #[test] fn freshell_image_mcp_command_is_not_selected_as_a_host_resource() { let owned = tempfile::tempdir().unwrap(); diff --git a/src/lib/api.ts b/src/lib/api.ts index 0f6952a9c..3e5890787 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -834,9 +834,7 @@ function groupDirectoryItemsAsProjects( // pre-SESSION-05 behavior. ...(projectColors?.[projectPath] ? { color: projectColors[projectPath] } : {}), sessions: sessions.map((item) => { - const checkpointWorkspace = normalizeFreshAgentCheckpointWorkspace( - item.checkoutPath ? { path: item.checkoutPath, provenance: 'resolved-project' } : undefined, - ) + const checkpointWorkspace = checkpointWorkspaceForDirectoryItem(item) return { provider: item.provider, sessionId: item.sessionId, @@ -874,6 +872,14 @@ function groupDirectoryItemsAsProjects( })) } +function checkpointWorkspaceForDirectoryItem( + item: Pick<ReadModelSessionDirectoryItem, 'checkoutPath' | 'projectPath'>, +): FreshAgentCheckpointWorkspace | undefined { + const path = item.checkoutPath || item.projectPath + if (!path || path === 'unknown' || path.startsWith('terminal:')) return undefined + return normalizeFreshAgentCheckpointWorkspace({ path, provenance: 'resolved-project' }) +} + export async function setSessionMetadata( provider: string, sessionId: string, @@ -1122,9 +1128,7 @@ export async function searchSessions(options: SearchOptions): Promise<SearchResp const response: SearchResponse = { results: page.items.map((item) => { - const checkpointWorkspace = normalizeFreshAgentCheckpointWorkspace( - item.checkoutPath ? { path: item.checkoutPath, provenance: 'resolved-project' } : undefined, - ) + const checkpointWorkspace = checkpointWorkspaceForDirectoryItem(item) return { sessionId: item.sessionId, provider: item.provider, diff --git a/test/unit/client/lib/api.test.ts b/test/unit/client/lib/api.test.ts index 3929672d2..40aa0aec0 100644 --- a/test/unit/client/lib/api.test.ts +++ b/test/unit/client/lib/api.test.ts @@ -1027,6 +1027,51 @@ describe('searchSessions cursor pagination', () => { }) }) + it('uses the resolved project path when an ordinary session launch cwd differs', async () => { + const items = [ + { + sessionId: 'session-ordinary', + provider: 'codex', + projectPath: '/repo', + cwd: '/home/user', + isRunning: false, + lastActivityAt: 1_000, + }, + { + sessionId: 'session-unknown', + provider: 'codex', + projectPath: 'unknown', + isRunning: false, + lastActivityAt: 900, + }, + { + sessionId: 'terminal-only', + provider: 'codex', + projectPath: 'terminal:term-42', + liveTerminalOnly: true, + isRunning: true, + lastActivityAt: 800, + }, + ] + mockFetch + .mockResolvedValueOnce(mockJson({ items, nextCursor: null, revision: 1 })) + .mockResolvedValueOnce(mockJson({ items, nextCursor: null, revision: 2 })) + + const snapshot = await fetchSidebarSessionsSnapshot() + const search = await searchSessions({ query: 'ordinary' }) + const ordinarySnapshot = snapshot.projects + .flatMap((project) => project.sessions) + .find((session) => session.sessionId === 'session-ordinary') + const terminalSearch = search.results.find((session) => session.sessionId === 'terminal-only') + + expect(ordinarySnapshot?.checkpointWorkspace).toEqual({ path: '/repo', provenance: 'resolved-project' }) + expect(search.results.find((session) => session.sessionId === 'session-ordinary')?.checkpointWorkspace) + .toEqual({ path: '/repo', provenance: 'resolved-project' }) + expect(snapshot.projects.flatMap((project) => project.sessions) + .find((session) => session.sessionId === 'session-unknown')?.checkpointWorkspace).toBeUndefined() + expect(terminalSearch?.checkpointWorkspace).toBeUndefined() + }) + it('forwards the cursor to the session-directory request when paginating a search', async () => { mockFetch.mockResolvedValueOnce(mockJson({ items: [], From 721cc298aaedf10f7875adb9fb815cf49ee44546 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 18:16:58 -0700 Subject: [PATCH 346/493] fix(managed): retain Bun ancestor dependencies --- .../managed_mcp_capability/source_stage.rs | 127 ++++++++++++++++++ .../opencode_resources.rs | 91 ++++++++----- 2 files changed, 186 insertions(+), 32 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 5eb60398e..9c6538b91 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -476,6 +476,133 @@ mod configured_resource_tests { } } + #[test] + fn bun_package_script_dependency_in_parent_node_modules_runs_after_both_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let package = external.join("mcp-package"); + let nested_cwd = package.join("nested"); + let dependency = external.join("node_modules/parent-dependency"); + for path in [&home, &workspace, &nested_cwd, &dependency] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + package.join("package.json"), + r#"{"scripts":{"mcp.js":"node ./server.js"}}"#, + ) + .unwrap(); + std::fs::write( + package.join("server.js"), + "process.stdout.write(require('parent-dependency'))\n", + ) + .unwrap(); + std::fs::write( + dependency.join("index.js"), + "module.exports='parent dependency loaded'\n", + ) + .unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "package-script": { + "type": "local", + "cwd": nested_cwd, + "command": ["bun", "run", "mcp.js"], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let selected = &plan[0].configured_resources; + assert!(selected.contains(&package.canonicalize().unwrap())); + assert!(selected.contains(&external.join("node_modules").canonicalize().unwrap())); + assert!(!selected.contains(&nested_cwd.canonicalize().unwrap())); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let stage = owned.path().join("stage"); + let host = owned.path().join("host"); + let stage_selections = units + .iter() + .map(|(source, mount)| CopySelection { + source: source.clone(), + destination_relative: mount.stage_relative_path.clone().into(), + }) + .collect::<Vec<_>>(); + copy_environment(&stage_selections, &stage, &[], Default::default()).unwrap(); + let host_selections = units + .iter() + .map(|(_, mount)| CopySelection { + source: stage.join(&mount.stage_relative_path), + destination_relative: mount.provider_relative_path.clone().into(), + }) + .collect::<Vec<_>>(); + copy_environment(&host_selections, &host, &[], Default::default()).unwrap(); + + // Managed mounts retain the selected resources' original relative + // topology, so package resolution can walk from the package into its + // parent's node_modules directory. + let provider = owned.path().join("provider"); + let provider_selections = units + .iter() + .map(|(_, mount)| CopySelection { + source: host.join(&mount.provider_relative_path), + destination_relative: Path::new(&mount.effective_destination) + .strip_prefix(owned.path()) + .unwrap() + .to_path_buf(), + }) + .collect::<Vec<_>>(); + copy_environment(&provider_selections, &provider, &[], Default::default()).unwrap(); + let staged_package = provider.join(package.strip_prefix(owned.path()).unwrap()); + let staged_manifest: serde_json::Value = + serde_json::from_slice(&std::fs::read(staged_package.join("package.json")).unwrap()) + .unwrap(); + let staged_script = staged_manifest["scripts"]["mcp.js"].as_str().unwrap(); + let hidden = owned.path().join("hidden-external"); + std::fs::rename(&external, &hidden).unwrap(); + // Execute the package manifest's actual command after both copy + // stages, without making the Rust test depend on a host Bun install. + let output = std::process::Command::new("sh") + .args(["-c", staged_script]) + .current_dir(&staged_package) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output() + .unwrap(); + std::fs::rename(&hidden, &external).unwrap(); + assert!( + output.status.success(), + "the staged Bun package script must resolve its parent-installed dependency: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"parent dependency loaded"); + } + #[test] fn managed_provider_nested_attention_and_literal_tilde_keep_native_origins_after_both_copies() { for scenario in ["nested", "top-replacement", "tilde"] { diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 94cdfd8b7..22020de21 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -608,6 +608,48 @@ fn insert_resource( Ok(()) } +fn insert_installed_package_ancestor_resources( + scope: &Path, + home: &Path, + declaring: &Path, + result: &mut std::collections::BTreeSet<PathBuf>, + resource_path_bytes: &mut usize, + field: &'static str, +) -> Result<(), ResourceError> { + for ancestor in scope.parent().into_iter().flat_map(Path::ancestors) { + if ancestor == Path::new("/") { + break; + } + for name in [ + "node_modules", + "package.json", + "package-lock.json", + "bun.lock", + "bun.lockb", + "pnpm-lock.yaml", + ] { + let candidate = ancestor.join(name); + match std::fs::symlink_metadata(&candidate) { + Ok(_) => { + checked(&candidate, field, declaring)?; + insert_resource(result, resource_path_bytes, candidate, field)?; + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => { + return Err(ResourceError { + field, + cause: "installed plugin lookup context is unavailable", + }) + } + } + } + if ancestor == home { + break; + } + } + Ok(()) +} + /// Return complete ordinary units and native installed ancestor lookup units. /// Exact mounts retain native field spellings and their declaring origins. pub fn select( @@ -713,7 +755,15 @@ pub fn select( insert_resource( &mut result, &mut resource_path_bytes, - package_root, + package_root.clone(), + "OpenCode mcp.command", + )?; + insert_installed_package_ancestor_resources( + &package_root, + home, + declaring, + &mut result, + &mut resource_path_bytes, "OpenCode mcp.command", )?; continue; @@ -934,37 +984,14 @@ pub fn select( return Err(fail(field, "select a bounded plugin or package directory")); } insert_resource(&mut result, &mut resource_path_bytes, scope.clone(), field)?; - for ancestor in scope.parent().into_iter().flat_map(Path::ancestors) { - if ancestor == Path::new("/") { - break; - } - for name in [ - "node_modules", - "package.json", - "package-lock.json", - "bun.lock", - "bun.lockb", - "pnpm-lock.yaml", - ] { - let candidate = ancestor.join(name); - match std::fs::symlink_metadata(&candidate) { - Ok(_) => { - checked(&candidate, field, declaring)?; - insert_resource(&mut result, &mut resource_path_bytes, candidate, field)?; - } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} - Err(_) => { - return Err(fail( - field, - "installed plugin lookup context is unavailable", - )) - } - } - } - if ancestor == home { - break; - } - } + insert_installed_package_ancestor_resources( + &scope, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; } Ok(( result.into_iter().collect(), From ffd0bd33555f170580a0de250c62f0a990cfd053 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 18:17:05 -0700 Subject: [PATCH 347/493] test(checkpoints): cover linked-worktree session rewind --- ...2026-10-05-managed-provider-environment.md | 2 +- scripts/testing/e2e-external-target.ts | 3 +- .../managed-provider-environment-target.ts | 10 +- ...code-checkpoint-workspace-external.spec.ts | 93 ++++++++++++++++++- .../server/e2e-cloud-external-target.test.ts | 3 +- .../tooling/testing/browser-selection.test.ts | 3 +- 6 files changed, 107 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-10-05-managed-provider-environment.md b/docs/plans/2026-10-05-managed-provider-environment.md index 772c97c6b..fb092b761 100644 --- a/docs/plans/2026-10-05-managed-provider-environment.md +++ b/docs/plans/2026-10-05-managed-provider-environment.md @@ -320,7 +320,7 @@ Managed output observations distinguish Running, active Recovering, Blocked with **Interfaces:** Existing inventory `runtimeStatus` admits `blocked` and the existing Rust exited process state where final tail is still draining; pane recovery projection uses `error` for Blocked/Lost, `exited` for Stopped, `recovering` only for active recovery, Running only after verified live recovery. Retain recoverySummary reason/attempt/incident and existing retry controls. External browser consumes `FRESHELL_E2E_TARGET_URL`, `FRESHELL_E2E_TARGET_WS_URL`, `FRESHELL_E2E_TARGET_TOKEN`, optional `FRESHELL_E2E_TARGET_TIMEOUT_MS`, and the narrow nonsecret `FRESHELL_MANAGED_ENVIRONMENT_FIXTURE` JSON parameter through allowlisted cloud forwarding. That parameter contains candidate SHA, run ID, remote workspace path and expected nonce only; ownership/cleanup receipt remains on the target host. Do not forward TARGET_HOME: cloud `/app` is a different filesystem. Candidate provenance comes from authenticated `/api/server-info` commit/buildDirty; actual managed readiness comes from `/api/runtime/readiness`, negotiated ready.managedRuntimeV1 and the created terminal/soul/incarnation inventory, not an invented server-info flag. -**External-target selection contract (also Task 7):** Ordinary Chromium/Firefox/WebKit projects exclude exactly `managed-provider-environment-external.spec.ts` and `code-checkpoint-workspace-external.spec.ts` in shared match-all discovery. Do not add them to `CLOUD_SKIP_SPECS`, discover them and call `test.skip`, or count absence from the ordinary broad suite as coverage. Add the dedicated `chromium-external-target` project only when `FRESHELL_E2E_EXTERNAL_FIXTURE_KIND` is the closed value `managed-environment` or `code-checkpoint` and its corresponding validated fixture JSON plus target URL/token are paired. Its `testMatch` selects only that kind's named spec and does not inherit the ordinary external-spec ignore. Explicit selection without a complete matching fixture is an error before dispatch/config use; its driver cannot fall back to an owned ordinary server or an ambient external/live target. Task 7's `FRESHELL_CODE_CHECKPOINT_FIXTURE` JSON contains candidate SHA, run ID, synthetic workspace/home/non-Git fixture paths, restricted UID and expected nonce only. Forward only these two new names and the kind through the cloud env-file allowlist. The wrapper's shard/spec discovery and job invocation must use the same positively selected project/fixture contract, otherwise zero discovered tests fail. Extend fake-gcloud dispatch tests and Playwright discovery tests by executing configuration/list selection: ordinary cloud lists neither external spec, a correctly paired kind lists/runs its exact nonzero spec, and missing/mismatched pairs fail. Record actual pass counts separately from the final ordinary broad receipt. +**External-target selection contract (also Task 7):** Ordinary Chromium/Firefox/WebKit projects exclude exactly `managed-provider-environment-external.spec.ts` and `code-checkpoint-workspace-external.spec.ts` in shared match-all discovery. Do not add them to `CLOUD_SKIP_SPECS`, discover them and call `test.skip`, or count absence from the ordinary broad suite as coverage. Add the dedicated `chromium-external-target` project only when `FRESHELL_E2E_EXTERNAL_FIXTURE_KIND` is the closed value `managed-environment` or `code-checkpoint` and its corresponding validated fixture JSON plus target URL/token are paired. Its `testMatch` selects only that kind's named spec and does not inherit the ordinary external-spec ignore. Explicit selection without a complete matching fixture is an error before dispatch/config use; its driver cannot fall back to an owned ordinary server or an ambient external/live target. Task 7's `FRESHELL_CODE_CHECKPOINT_FIXTURE` JSON contains candidate SHA, run ID, synthetic workspace, linked-worktree checkout, home and non-Git fixture paths, restricted UID and expected nonce only. Forward only these two new names and the kind through the cloud env-file allowlist. The wrapper's shard/spec discovery and job invocation must use the same positively selected project/fixture contract, otherwise zero discovered tests fail. Extend fake-gcloud dispatch tests and Playwright discovery tests by executing configuration/list selection: ordinary cloud lists neither external spec, a correctly paired kind lists/runs its exact nonzero spec, and missing/mismatched pairs fail. Record actual pass counts separately from the final ordinary broad receipt. - [ ] **Step 1: Add failing UI and real managed browser tests.** Extend unit cases from running and recovering through Blocked/Lost/Stopped, asserting exact projected status and retained summary/identity. Extend cloud-compatible contextual UI spec to show actual shell startup output, final blocked/lost/stopped status, existing retry controls, retained output and successful replacement attach; assert no silent extra `terminal.create`. These tests cover UI rendering/state and remain explicitly narrower than complete managed startup. Feed blocked plain/paged directory rows through the real API/thunk/store flow, render Overview and Background Sessions, and assert the same terminal remains visible with its blocked reason/recovery data. Exercise existing Overview rename/fenced kill and Background attach/fenced kill controls as applicable; replay opens the retained output with input refused, rather than creating a replacement or fabricating running state. Assert counts exclude blocked processes and successful retry changes that same row to running. The contextual browser spec visits these actual surfaces and opens retained replay; it must preserve final diagnostics and existing controls rather than merely asserting an inventory object. diff --git a/scripts/testing/e2e-external-target.ts b/scripts/testing/e2e-external-target.ts index b41d41049..93c822c76 100644 --- a/scripts/testing/e2e-external-target.ts +++ b/scripts/testing/e2e-external-target.ts @@ -17,6 +17,7 @@ export type CodeCheckpointFixture = { candidateSha: string runId: string syntheticWorkspacePath: string + linkedWorktreePath: string syntheticHomePath: string nonGitWorkspacePath: string restrictedUid: number @@ -55,7 +56,7 @@ export function resolvePairedExternalFixture( try { value = JSON.parse(env[matching]!) } catch { fail('fixture JSON') } if (!value || typeof value !== 'object' || Array.isArray(value)) fail('fixture JSON') const paths = kind === 'managed-environment' ? ['remoteWorkspacePath'] - : ['syntheticWorkspacePath', 'syntheticHomePath', 'nonGitWorkspacePath'] + : ['syntheticWorkspacePath', 'linkedWorktreePath', 'syntheticHomePath', 'nonGitWorkspacePath'] const fields = ['candidateSha', 'runId', 'expectedNonce', ...paths, ...(kind === 'code-checkpoint' ? ['restrictedUid'] : [])] if (Object.keys(value).length !== fields.length || Object.keys(value).some(key => !fields.includes(key))) fail('fixture fields') if (typeof value.candidateSha !== 'string' || !/^[a-f0-9]{40}$/.test(value.candidateSha)) fail('candidate SHA') diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index eda721805..914cf0788 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -333,6 +333,7 @@ const capabilities = status.match(/^CapEff:\s+([0-9a-f]+)$/m)?.[1] assert.equal(capabilities, '0000000000000000', 'fixture UID B must have no effective capabilities') const project = path.join(root, 'project') +const linkedWorktree = path.join(root, 'linked-worktree') const home = path.join(root, 'home') const storage = path.join(root, 'storage') const control = path.join(root, 'control') @@ -366,15 +367,21 @@ assert.deepEqual([fs.lstatSync(deniedDirectory).uid, mode(deniedDirectory)], [0, fs.mkdirSync(control, { recursive: true, mode: 0o700 }) fs.mkdirSync(path.join(project, '.freshell-checkpoint-fixture'), { recursive: true, mode: 0o700 }) +fs.mkdirSync(path.join(project, 'nested'), { recursive: true, mode: 0o755 }) fs.writeFileSync(path.join(project, 'main.rs'), 'main before checkpoint\n') fs.writeFileSync(path.join(project, 'notes.txt'), 'notes before checkpoint\n') +fs.writeFileSync(path.join(project, 'nested/rewind.txt'), 'nested before checkpoint\n') fs.writeFileSync(path.join(project, 'run.sh'), '#!/bin/sh\necho before\n', { mode: 0o755 }) fs.symlinkSync('main.rs', path.join(project, 'main-link')) execFileSync('git', ['init', '-q'], { cwd: project, stdio: 'ignore' }) execFileSync('git', ['config', 'user.name', 'Freshell checkpoint fixture'], { cwd: project, stdio: 'ignore' }) execFileSync('git', ['config', 'user.email', 'fixture@example.invalid'], { cwd: project, stdio: 'ignore' }) -execFileSync('git', ['add', '--', '.gitignore', 'main.rs', 'notes.txt', 'run.sh', 'main-link'], { cwd: project, stdio: 'ignore' }) +execFileSync('git', ['add', '--', '.gitignore', 'main.rs', 'notes.txt', 'run.sh', 'main-link', 'nested/rewind.txt'], { cwd: project, stdio: 'ignore' }) execFileSync('git', ['commit', '-q', '-m', 'synthetic checkpoint fixture'], { cwd: project, stdio: 'ignore' }) +execFileSync('git', ['worktree', 'add', '-q', '-b', 'checkpoint-e2e-linked-worktree', linkedWorktree, 'HEAD'], { + cwd: project, + stdio: 'ignore', +}) const nonGit = path.join(root, 'non-git') fs.mkdirSync(nonGit, { mode: 0o755 }) @@ -719,6 +726,7 @@ export async function runCodeCheckpointTarget() { candidateSha, runId, syntheticWorkspacePath: path.join(checkpointFixtureRoot, 'project'), + linkedWorktreePath: path.join(checkpointFixtureRoot, 'linked-worktree'), syntheticHomePath: path.join(checkpointFixtureRoot, 'home'), nonGitWorkspacePath: path.join(checkpointFixtureRoot, 'non-git'), restrictedUid: identity.uid, diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index c67a0e52c..f7182b141 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -128,7 +128,7 @@ async function sendThroughComposer( page: import('@playwright/test').Page, paneRoot: import('@playwright/test').Locator, text: string, - expectedWorkspace: { path: string; provenance: 'directory-picker' } | undefined, + expectedWorkspace: { path: string; provenance: 'directory-picker' | 'resolved-project' } | undefined, ) { await expect.poll(async () => (await currentFreshAgent(page)).content.status, { timeout: 60_000 }).toBe('idle') const baseline = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.().length ?? 0) @@ -164,7 +164,8 @@ async function sendThroughComposer( // The send barrier's first outcome is deliberately pre-forward and may // not yet know Codex's persisted user row. Require the later updated // native-linked projection before treating the per-turn result as ready. - return outcome?.nativeTurnId === accepted.submittedTurnId + return outcome !== undefined + && outcome.nativeTurnId === accepted.submittedTurnId && outcome.displayTurnId === `${accepted.submittedTurnId}:row-0` }, { timeout: 120_000 }).toBe(true) expect(outcome!.requestId).toBe(sent.requestId) @@ -334,6 +335,94 @@ test('Git workspace sends retain per-turn success and failure, then restore the .toEqual(durable?.map(turn => `${turn.id}:user`)) }) +test('opening a linked-worktree session from History rewinds its resolved checkout', async ({ page }) => { + test.setTimeout(360_000) + await verifyTarget(page) + const checkout = fixture.linkedWorktreePath + const launchDirectory = `${checkout}/nested` + const sessionText = `linked-worktree history session ${fixture.expectedNonce}` + await targetWrite(page, `${checkout}/main.rs`, 'main before history checkpoint\n') + await targetWrite(page, `${launchDirectory}/rewind.txt`, 'nested before history checkpoint\n') + + const { paneRoot } = await launchCodex(page, launchDirectory) + const initial = await currentFreshAgent(page) + const sessionId = initial.content.sessionRef?.sessionId ?? initial.content.sessionId + expect(initial.content.checkpointWorkspace).toEqual({ path: launchDirectory, provenance: 'directory-picker' }) + expect(sessionId).toEqual(expect.any(String)) + await sendThroughComposer(page, paneRoot, sessionText, { + path: launchDirectory, + provenance: 'directory-picker', + }) + + type DirectoryItem = { + sessionId: string + projectPath: string + checkoutPath?: string + cwd?: string + } + let directoryItem: DirectoryItem | undefined + await expect.poll(async () => { + const response = await page.request.get( + `${paired!.targetUrl}/api/session-directory?priority=visible&limit=50`, + { headers }, + ) + expect(response.ok()).toBe(true) + const payload = await response.json() as { items: DirectoryItem[] } + directoryItem = payload.items.find((item) => item.sessionId === sessionId) + return directoryItem + }, { timeout: 60_000, intervals: [500, 1000, 2000] }).toBeTruthy() + expect(directoryItem).toMatchObject({ + projectPath: fixture.syntheticWorkspacePath, + checkoutPath: checkout, + cwd: launchDirectory, + }) + + // History must open from its directory projection; keeping the seed pane + // open would exercise the already-open-pane path and preserve its explicit + // picker scope instead of testing ordinary session restoration. + await paneRoot.getByRole('button', { name: 'Close pane' }).click() + await expect(page.locator('[data-context="fresh-agent"]')).toHaveCount(0, { timeout: 30_000 }) + await page.locator('button[title="Projects (Ctrl+B P)"]').click() + const projectHeader = page.locator(`[data-project-path="${fixture.syntheticWorkspacePath}"]`) + await expect(projectHeader).toBeVisible({ timeout: 30_000 }) + await projectHeader.click() + const historyRow = page.locator(`[data-context="history-session"][data-session-id="${sessionId}"]`) + await expect(historyRow).toBeVisible({ timeout: 30_000 }) + await historyRow.getByRole('button', { name: 'Open session' }).click() + + await expect.poll(async () => (await currentFreshAgent(page)).content.checkpointWorkspace) + .toEqual({ path: checkout, provenance: 'resolved-project' }) + const reopenedPane = page.locator('[data-context="fresh-agent"]').last() + const beforeMain = 'main before linked-worktree rewind\n' + const beforeNested = 'nested before linked-worktree rewind\n' + await targetWrite(page, `${checkout}/main.rs`, beforeMain) + await targetWrite(page, `${launchDirectory}/rewind.txt`, beforeNested) + + const turnText = `linked-worktree resolved-scope rewind ${fixture.expectedNonce}` + const turn = await sendThroughComposer(page, reopenedPane, turnText, { + path: checkout, + provenance: 'resolved-project', + }) + expect(turn.outcome).toMatchObject({ + status: 'available', + workspace: { path: checkout, provenance: 'resolved-project' }, + }) + expect(turn.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) + + await targetWrite(page, `${checkout}/main.rs`, 'main changed after linked-worktree turn\n') + await targetWrite(page, `${launchDirectory}/rewind.txt`, 'nested changed after linked-worktree turn\n') + await turn.row.hover() + const rewind = turn.row.getByRole('button', { name: 'Rewind code to here' }) + await expect(rewind).toBeEnabled() + const rewindNotice = expect(reopenedPane.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) + .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + const dialogAccepted = page.waitForEvent('dialog').then((dialog) => dialog.accept()) + await rewind.click() + await Promise.all([dialogAccepted, rewindNotice]) + await expect.poll(() => targetRead(page, `${checkout}/main.rs`)).toBe(beforeMain) + await expect.poll(() => targetRead(page, `${launchDirectory}/rewind.txt`)).toBe(beforeNested) +}) + test('picker-confirmed non-Git workspace captures and restores all selected files', async ({ page }) => { test.setTimeout(300_000) await verifyTarget(page) diff --git a/test/unit/server/e2e-cloud-external-target.test.ts b/test/unit/server/e2e-cloud-external-target.test.ts index 5aa6cd77e..6aef8d74d 100644 --- a/test/unit/server/e2e-cloud-external-target.test.ts +++ b/test/unit/server/e2e-cloud-external-target.test.ts @@ -11,7 +11,8 @@ const targetToken = 'synthetic token: quote " newline\nvalue' const managedFixture = { candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', remoteWorkspacePath: '/tmp/synthetic/workspace', expectedNonce: 'synthetic-environment-nonce' } const checkpointFixture = { candidateSha: 'a'.repeat(40), runId: managedFixture.runId, - syntheticWorkspacePath: '/tmp/synthetic/workspace', syntheticHomePath: '/tmp/synthetic/home', + syntheticWorkspacePath: '/tmp/synthetic/workspace', linkedWorktreePath: '/tmp/synthetic/linked-worktree', + syntheticHomePath: '/tmp/synthetic/home', nonGitWorkspacePath: '/tmp/synthetic/non-git', restrictedUid: 65534, expectedNonce: 'synthetic-checkpoint-nonce' } async function dispatch(overrides: Record<string, string | undefined>) { diff --git a/test/unit/tooling/testing/browser-selection.test.ts b/test/unit/tooling/testing/browser-selection.test.ts index e6e430d67..9b9775ae2 100644 --- a/test/unit/tooling/testing/browser-selection.test.ts +++ b/test/unit/tooling/testing/browser-selection.test.ts @@ -123,6 +123,7 @@ describe('browser selection non-vacuity', () => { expect(listedSpecFiles(ordinary.output).has(spec)).toBe(false) const fixture = JSON.stringify({ candidateSha: 'a'.repeat(40), runId: '11111111-1111-4111-8111-111111111111', syntheticWorkspacePath: '/tmp/selected-checkpoint-fixture/project', + linkedWorktreePath: '/tmp/selected-checkpoint-fixture/linked-worktree', syntheticHomePath: '/tmp/selected-checkpoint-fixture/home', nonGitWorkspacePath: '/tmp/selected-checkpoint-fixture/non-git', restrictedUid: 65534, expectedNonce: 'selection-checkpoint-nonce' }) @@ -131,7 +132,7 @@ describe('browser selection non-vacuity', () => { FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token' }) const selected = listedProjects(paired, cloudConfig, ['--project=chromium-external-target', spec]) expect(selected.labels).toEqual(['chromium-external-target']) - expect(selected.tests).toBe(3) + expect(selected.tests).toBe(4) expect(selected.files).toBe(1) expect([...listedSpecFiles(selected.output)]).toEqual([spec]) const ordinaryExplicit = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, From 37f63f8479f35c855292a7433281bb793a1c33fd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 18:50:42 -0700 Subject: [PATCH 348/493] fix: preserve checkpoint workspace across session history --- .../freshell-server/src/session_directory.rs | 70 ++++++++ .../freshell-server/src/session_metadata.rs | 167 +++++++++++++++++- shared/read-models.ts | 4 + src/components/fresh-agent/FreshAgentView.tsx | 20 ++- src/lib/api.ts | 17 +- ...code-checkpoint-workspace-external.spec.ts | 89 ++++++++++ test/unit/client/lib/api.test.ts | 44 +++++ 7 files changed, 393 insertions(+), 18 deletions(-) diff --git a/crates/freshell-server/src/session_directory.rs b/crates/freshell-server/src/session_directory.rs index 95490a8ff..d68d906ed 100644 --- a/crates/freshell-server/src/session_directory.rs +++ b/crates/freshell-server/src/session_directory.rs @@ -744,6 +744,7 @@ async fn session_directory( apply_naming_projection(items, state.identity.naming().as_ref()).await; match apply_query(items, &query, &identities) { Ok(mut page) => { + apply_checkpoint_workspace_projection(&mut page, &metadata_entries); page["revision"] = json!(revision); page["snapshotSeq"] = json!(snapshot_seq); page["serverInstance"] = json!(state.server_instance.as_str()); @@ -1576,6 +1577,44 @@ fn apply_session_metadata( .collect() } +/// Add the explicitly selected checkpoint scope to the same directory rows +/// used by History and search. Only the validated picker provenance is +/// projected; inferred project paths continue to be resolved by the client. +fn apply_checkpoint_workspace_projection( + page: &mut Value, + metadata: &std::collections::HashMap<String, Value>, +) { + let Some(items) = page.get_mut("items").and_then(Value::as_array_mut) else { + return; + }; + for item in items { + let (Some(provider), Some(session_id)) = ( + item.get("provider").and_then(Value::as_str), + item.get("sessionId").and_then(Value::as_str), + ) else { + continue; + }; + let Some(workspace) = metadata + .get(&format!("{provider}:{session_id}")) + .and_then(|entry| entry.get("checkpointWorkspace")) + else { + continue; + }; + let (Some(path), Some("directory-picker")) = ( + workspace.get("path").and_then(Value::as_str), + workspace.get("provenance").and_then(Value::as_str), + ) else { + continue; + }; + if !path.is_empty() && path.encode_utf16().count() <= 4096 && !path.contains('\0') { + item["checkpointWorkspace"] = json!({ + "path": path, + "provenance": "directory-picker", + }); + } + } +} + // ── Sidebar join (Fix Spec: Session Naming Cluster, SYMPTOM 1) ───────────── // // Ports `toItems`/`joinRunningState`/`buildLiveTerminalSessionItem`/ @@ -3364,6 +3403,37 @@ mod tests { assert!(out[0].title_overridden); } + #[test] + fn explicit_checkpoint_workspace_projects_to_history_and_search_items() { + let workspace = json!({ + "path": "/tmp/non-git-workspace", + "provenance": "directory-picker", + }); + let mut page = json!({ + "items": [ + { "provider": "codex", "sessionId": "session-1" }, + { "provider": "codex", "sessionId": "session-2" }, + ], + }); + let metadata = std::collections::HashMap::from([ + ( + "codex:session-1".to_string(), + json!({ "checkpointWorkspace": workspace.clone() }), + ), + ( + "codex:session-2".to_string(), + json!({ + "checkpointWorkspace": { "path": "/tmp/project", "provenance": "resolved-project" } + }), + ), + ]); + + apply_checkpoint_workspace_projection(&mut page, &metadata); + + assert_eq!(page["items"][0]["checkpointWorkspace"], workspace); + assert!(page["items"][1].get("checkpointWorkspace").is_none()); + } + #[test] fn canonical_identity_prefers_its_own_user_state_and_falls_back_to_a_safe_legacy_identity() { let item = DirItem { diff --git a/crates/freshell-server/src/session_metadata.rs b/crates/freshell-server/src/session_metadata.rs index 6c0e51fc5..b2fd23efb 100644 --- a/crates/freshell-server/src/session_metadata.rs +++ b/crates/freshell-server/src/session_metadata.rs @@ -162,6 +162,27 @@ impl SessionMetadataStore { session_id: &str, session_type: &str, session_type_source: Option<&str>, + ) -> std::io::Result<bool> { + self.set_with_checkpoint_workspace( + provider, + session_id, + session_type, + session_type_source, + None, + ) + .await + } + + /// Merge the explicit workspace provenance selected when a fresh-agent + /// session was created. Later flavor-only writes pass `None`, preserving + /// this field along with the other metadata already stored for the session. + pub async fn set_with_checkpoint_workspace( + &self, + provider: &str, + session_id: &str, + session_type: &str, + session_type_source: Option<&str>, + checkpoint_workspace: Option<&Value>, ) -> std::io::Result<bool> { let mut guard = self.inner.lock().await; let mut data = Self::load_locked(&mut guard, &self.path).await; @@ -195,6 +216,9 @@ impl SessionMetadataStore { // one (matching the JS spread: an absent key in `entry` never touches `next`). let mut next = existing.clone(); next.insert("sessionType".to_string(), json!(session_type)); + if let Some(workspace) = checkpoint_workspace { + next.insert("checkpointWorkspace".to_string(), workspace.clone()); + } if let Some(source) = session_type_source { let existing_type = existing.get("sessionType").and_then(Value::as_str); @@ -398,8 +422,8 @@ async fn post_session_metadata( })), ) .into_response(), - Ok((provider, session_id, session_type, session_type_source)) => { - match apply_session_metadata( + Ok((provider, session_id, session_type, session_type_source, checkpoint_workspace)) => { + match apply_session_metadata_with_checkpoint_workspace( &state.store, &state.broadcast_tx, &state.sessions_revision, @@ -407,6 +431,7 @@ async fn post_session_metadata( &session_id, &session_type, session_type_source.as_deref(), + checkpoint_workspace.as_ref(), ) .await { @@ -435,11 +460,49 @@ pub async fn apply_session_metadata( session_type: &str, session_type_source: Option<&str>, ) -> std::io::Result<bool> { - let changed = store - .set(provider, session_id, session_type, session_type_source) - .await?; + apply_session_metadata_with_checkpoint_workspace( + store, + broadcast_tx, + sessions_revision, + provider, + session_id, + session_type, + session_type_source, + None, + ) + .await +} + +async fn apply_session_metadata_with_checkpoint_workspace( + store: &SessionMetadataStore, + broadcast_tx: &tokio::sync::broadcast::Sender<String>, + sessions_revision: &std::sync::atomic::AtomicI64, + provider: &str, + session_id: &str, + session_type: &str, + session_type_source: Option<&str>, + checkpoint_workspace: Option<&Value>, +) -> std::io::Result<bool> { + let changed = match checkpoint_workspace { + Some(workspace) => { + store + .set_with_checkpoint_workspace( + provider, + session_id, + session_type, + session_type_source, + Some(workspace), + ) + .await? + } + None => { + store + .set(provider, session_id, session_type, session_type_source) + .await? + } + }; // W5 fix-forward: broadcast `sessions.changed` directly for a - // metadata write that actually changed the persisted `sessionType` + // metadata write that actually changed the persisted entry // (mirroring `sessions::patch_session`'s GAP-1 fix). Guarded on // `changed` -- the store's own no-op comparison (semantic `Map` // equality, see `SessionMetadataStore::set`'s doc comment) is the @@ -460,7 +523,7 @@ pub async fn apply_session_metadata( /// match; the client only reads the top-level `error` string on a 400). fn validate_post_body( body: &Value, -) -> Result<(String, String, String, Option<String>), Vec<Value>> { +) -> Result<(String, String, String, Option<String>, Option<Value>), Vec<Value>> { let mut issues = Vec::new(); let provider = body.get("provider").and_then(Value::as_str); @@ -496,6 +559,37 @@ fn validate_post_body( .push(json!({ "path": ["sessionTypeSource"], "message": "Invalid sessionTypeSource" })); } + let checkpoint_workspace = body.get("checkpointWorkspace"); + let checkpoint_workspace_value = match checkpoint_workspace { + None => None, + Some(Value::Object(workspace)) => { + let path = workspace.get("path").and_then(Value::as_str); + let provenance = workspace.get("provenance").and_then(Value::as_str); + if path.is_some_and(|path| { + !path.is_empty() && path.encode_utf16().count() <= 4096 && !path.contains('\0') + }) && provenance == Some("directory-picker") + { + Some(json!({ + "path": path.unwrap(), + "provenance": "directory-picker", + })) + } else { + issues.push(json!({ + "path": ["checkpointWorkspace"], + "message": "Expected a bounded directory-picker workspace", + })); + None + } + } + Some(_) => { + issues.push(json!({ + "path": ["checkpointWorkspace"], + "message": "Expected a bounded directory-picker workspace", + })); + None + } + }; + if !issues.is_empty() { return Err(issues); } @@ -507,6 +601,7 @@ fn validate_post_body( session_type_source .and_then(Value::as_str) .map(str::to_string), + checkpoint_workspace_value, )) } @@ -769,6 +864,64 @@ mod tests { assert_eq!(entry["sessionType"], json!("freshcodex")); } + #[tokio::test] + async fn explicit_checkpoint_workspace_survives_later_materialized_metadata_write() { + let dir = tempfile::tempdir().unwrap(); + let store = SessionMetadataStore::new(dir.path()); + let workspace = json!({ + "path": "/tmp/non-git-workspace", + "provenance": "directory-picker", + }); + let mut selected = valid_body(); + selected["checkpointWorkspace"] = workspace.clone(); + let selected_response = post_session_metadata( + State(state(store.clone())), + headers_with_token("tok"), + Json(selected), + ) + .await; + assert_eq!(selected_response.status(), StatusCode::OK); + assert_eq!( + store.get("codex", "sess-1").await.unwrap()["checkpointWorkspace"], + workspace + ); + + let mut refreshed = valid_body(); + refreshed["sessionTypeSource"] = json!("materialized"); + let _ = post_session_metadata( + State(state(store.clone())), + headers_with_token("tok"), + Json(refreshed), + ) + .await; + assert_eq!( + store.get("codex", "sess-1").await.unwrap()["checkpointWorkspace"], + workspace + ); + let reloaded = SessionMetadataStore::new(dir.path()); + assert_eq!( + reloaded.get("codex", "sess-1").await.unwrap()["checkpointWorkspace"], + workspace + ); + } + + #[tokio::test] + async fn checkpoint_workspace_rejects_non_picker_provenance() { + let dir = tempfile::tempdir().unwrap(); + let mut body = valid_body(); + body["checkpointWorkspace"] = json!({ + "path": "/tmp/project", + "provenance": "resolved-project", + }); + let response = post_session_metadata( + State(state(SessionMetadataStore::new(dir.path()))), + headers_with_token("tok"), + Json(body), + ) + .await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + } + #[tokio::test] async fn second_identical_post_reports_changed_false() { let dir = tempfile::tempdir().unwrap(); diff --git a/shared/read-models.ts b/shared/read-models.ts index 9e0058535..28c3c58cd 100644 --- a/shared/read-models.ts +++ b/shared/read-models.ts @@ -54,6 +54,10 @@ export const SessionDirectoryItemSchema = z.object({ provider: z.string().min(1), projectPath: z.string().min(1), checkoutPath: z.string().optional(), + checkpointWorkspace: z.object({ + path: z.string().min(1).max(4096), + provenance: z.literal('directory-picker'), + }).optional(), title: z.string().optional(), summary: z.string().optional(), snippet: z.string().optional(), diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 20e69da8b..aec12c76a 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -536,17 +536,21 @@ function getCreatedResumeSessionId( return message.sessionId } -function persistDurableFreshAgentFlavor(message: { - provider: string - sessionId?: string - sessionType: string - sessionRef?: { provider: string; sessionId: string } -}) { +function persistDurableFreshAgentFlavor( + message: { + provider: string + sessionId?: string + sessionType: string + sessionRef?: { provider: string; sessionId: string } + }, + checkpointWorkspace?: FreshAgentPaneContent['checkpointWorkspace'], +) { const provider = message.sessionRef?.provider ?? message.provider const sessionId = message.sessionRef?.sessionId ?? message.sessionId if (!provider || !sessionId || !isDurableProviderSessionId(provider, sessionId)) return setSessionMetadata(provider, sessionId, message.sessionType, { sessionTypeSource: 'materialized', + ...(checkpointWorkspace?.provenance === 'directory-picker' ? { checkpointWorkspace } : {}), }).catch((err) => { log.warn({ event: 'fresh_agent_session_metadata_tag_failed', @@ -2693,7 +2697,7 @@ export function FreshAgentView({ releasePendingRebind() clearReserveRedrive() // Task 14: a completed create ends the reservation window const current = paneContentRef.current - persistDurableFreshAgentFlavor(message) + persistDurableFreshAgentFlavor(message, current.checkpointWorkspace) dispatch(updatePaneContent({ tabId, paneId, @@ -2750,7 +2754,7 @@ export function FreshAgentView({ sessionId: message.sessionId, sessionType: message.sessionType, sessionRef, - }) + }, current.checkpointWorkspace) migratePendingAutoTitle(current.sessionId, message.sessionId, message.provider, message.sessionType) requestSnapshotRefresh('materialized') dispatch(updatePaneContent({ diff --git a/src/lib/api.ts b/src/lib/api.ts index 3e5890787..c356d724b 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -873,9 +873,14 @@ function groupDirectoryItemsAsProjects( } function checkpointWorkspaceForDirectoryItem( - item: Pick<ReadModelSessionDirectoryItem, 'checkoutPath' | 'projectPath'>, + item: Pick<ReadModelSessionDirectoryItem, 'checkoutPath' | 'projectPath' | 'checkpointWorkspace'>, ): FreshAgentCheckpointWorkspace | undefined { - const path = item.checkoutPath || item.projectPath + if (item.checkoutPath) { + return normalizeFreshAgentCheckpointWorkspace({ path: item.checkoutPath, provenance: 'resolved-project' }) + } + const explicitWorkspace = normalizeFreshAgentCheckpointWorkspace(item.checkpointWorkspace) + if (explicitWorkspace?.provenance === 'directory-picker') return explicitWorkspace + const path = item.projectPath if (!path || path === 'unknown' || path.startsWith('terminal:')) return undefined return normalizeFreshAgentCheckpointWorkspace({ path, provenance: 'resolved-project' }) } @@ -884,13 +889,19 @@ export async function setSessionMetadata( provider: string, sessionId: string, sessionType: string, - options: { sessionTypeSource?: 'explicit' | 'materialized' } = {}, + options: { + sessionTypeSource?: 'explicit' | 'materialized' + checkpointWorkspace?: FreshAgentCheckpointWorkspace + } = {}, ): Promise<void> { await api.post('/api/session-metadata', { provider, sessionId, sessionType, sessionTypeSource: options.sessionTypeSource ?? 'explicit', + ...(options.checkpointWorkspace?.provenance === 'directory-picker' + ? { checkpointWorkspace: options.checkpointWorkspace } + : {}), }) } diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index f7182b141..cf7bc259e 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -239,6 +239,85 @@ async function flushLayoutAndReload(page: import('@playwright/test').Page): Prom await expect(page.locator('[data-context="fresh-agent"]')).toBeVisible({ timeout: 30_000 }) } +async function openSessionFromHistory( + page: import('@playwright/test').Page, + paneRoot: import('@playwright/test').Locator, + projectPath: string, + sessionId: string, + checkpointWorkspace: { path: string; provenance: 'directory-picker' }, +) { + await expect.poll(async () => { + const response = await page.request.get( + `${paired!.targetUrl}/api/session-directory?priority=visible&limit=50`, + { headers }, + ) + expect(response.ok()).toBe(true) + const payload = await response.json() as { items: Array<{ + sessionId: string + projectPath: string + checkpointWorkspace?: { path: string; provenance: string } + }> } + return payload.items.find((item) => item.sessionId === sessionId) + }, { timeout: 60_000, intervals: [500, 1000, 2000] }).toMatchObject({ + projectPath, + checkpointWorkspace, + }) + + // Closing the seed pane makes History restore from its directory projection, + // which proves the explicit workspace provenance survived session metadata. + await paneRoot.getByRole('button', { name: 'Close pane' }).click() + await expect(page.locator('[data-context="fresh-agent"]')).toHaveCount(0, { timeout: 30_000 }) + await page.locator('button[title="Projects (Ctrl+B P)"]').click() + const projectHeader = page.locator(`[data-project-path="${projectPath}"]`) + await expect(projectHeader).toBeVisible({ timeout: 30_000 }) + await projectHeader.click() + const historyRow = page.locator(`[data-context="history-session"][data-session-id="${sessionId}"]`) + await expect(historyRow).toBeVisible({ timeout: 30_000 }) + await historyRow.getByRole('button', { name: 'Open session' }).click() + + const reopenedPane = page.locator('[data-context="fresh-agent"]').last() + await expect.poll(async () => (await currentFreshAgent(page)).content.checkpointWorkspace) + .toEqual(checkpointWorkspace) + return reopenedPane +} + +async function verifyPickerWorkspaceReopensForRewind( + page: import('@playwright/test').Page, + workspacePath: string, + fileName: string, + label: string, +): Promise<void> { + await verifyTarget(page) + const selectedWorkspace = { path: workspacePath, provenance: 'directory-picker' as const } + const file = `${workspacePath}/${fileName}` + await targetWrite(page, file, `${label} before first turn\n`) + const { paneRoot } = await launchCodex(page, workspacePath) + const initial = await currentFreshAgent(page) + const sessionId = initial.content.sessionRef?.sessionId ?? initial.content.sessionId + expect(sessionId).toEqual(expect.any(String)) + const initialTurn = await sendThroughComposer(page, paneRoot, `${label} seed ${fixture.expectedNonce}`, selectedWorkspace) + expect(initialTurn.outcome.status).toBe('available') + + const reopenedPane = await openSessionFromHistory( + page, paneRoot, workspacePath, sessionId, selectedWorkspace, + ) + const beforeTurn = `${label} before reopened turn\n` + await targetWrite(page, file, beforeTurn) + const turn = await sendThroughComposer(page, reopenedPane, `${label} reopened ${fixture.expectedNonce}`, selectedWorkspace) + expect(turn.outcome).toMatchObject({ status: 'available', workspace: selectedWorkspace }) + + await targetWrite(page, file, `${label} changed after reopened turn\n`) + await turn.row.hover() + const rewind = turn.row.getByRole('button', { name: 'Rewind code to here' }) + await expect(rewind).toBeEnabled() + const rewindNotice = expect(reopenedPane.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) + .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + const dialogAccepted = page.waitForEvent('dialog').then((dialog) => dialog.accept()) + await rewind.click() + await Promise.all([dialogAccepted, rewindNotice]) + await expect.poll(() => targetRead(page, file)).toBe(beforeTurn) +} + test('Git workspace sends retain per-turn success and failure, then restore the exact successful row', async ({ page }) => { test.setTimeout(360_000) await verifyTarget(page) @@ -459,6 +538,16 @@ test('picker-confirmed non-Git workspace captures and restores all selected file expect(afterRestore.content.checkpointWorkspace).toEqual({ path: fixture.nonGitWorkspacePath, provenance: 'directory-picker' }) }) +test('History reopening preserves a non-Git picker workspace for later rewind', async ({ page }) => { + test.setTimeout(360_000) + await verifyPickerWorkspaceReopensForRewind(page, fixture.nonGitWorkspacePath, 'one.txt', 'non-Git') +}) + +test('History reopening preserves an explicitly selected HOME workspace for later rewind', async ({ page }) => { + test.setTimeout(360_000) + await verifyPickerWorkspaceReopensForRewind(page, fixture.syntheticHomePath, 'checkpoint-home-file.txt', 'HOME') +}) + test('picker-confirmed HOME captures and restores a harmless home file', async ({ page }) => { test.setTimeout(240_000) await verifyTarget(page) diff --git a/test/unit/client/lib/api.test.ts b/test/unit/client/lib/api.test.ts index 40aa0aec0..7f38122c7 100644 --- a/test/unit/client/lib/api.test.ts +++ b/test/unit/client/lib/api.test.ts @@ -1072,6 +1072,28 @@ describe('searchSessions cursor pagination', () => { expect(terminalSearch?.checkpointWorkspace).toBeUndefined() }) + it('restores an explicitly selected non-Git workspace from snapshots and search', async () => { + const workspace = { path: '/tmp/non-git-workspace', provenance: 'directory-picker' as const } + const item = { + sessionId: 'session-non-git', + provider: 'codex', + projectPath: '/tmp/non-git-workspace', + checkpointWorkspace: workspace, + cwd: '/tmp/non-git-workspace', + isRunning: false, + lastActivityAt: 1_000, + } + mockFetch + .mockResolvedValueOnce(mockJson({ items: [item], nextCursor: null, revision: 1 })) + .mockResolvedValueOnce(mockJson({ items: [item], nextCursor: null, revision: 2 })) + + const snapshot = await fetchSidebarSessionsSnapshot() + const search = await searchSessions({ query: 'non-git' }) + + expect(snapshot.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual(workspace) + expect(search.results[0]?.checkpointWorkspace).toEqual(workspace) + }) + it('forwards the cursor to the session-directory request when paginating a search', async () => { mockFetch.mockResolvedValueOnce(mockJson({ items: [], @@ -1141,6 +1163,28 @@ describe('setSessionMetadata()', () => { ) }) + it('persists an explicitly selected checkpoint workspace with materialized session metadata', async () => { + mockFetch.mockResolvedValueOnce({ ok: true, text: () => Promise.resolve('') }) + + await setSessionMetadata('codex', 'session-non-git', 'freshcodex', { + sessionTypeSource: 'materialized', + checkpointWorkspace: { path: '/tmp/non-git-workspace', provenance: 'directory-picker' }, + }) + + expect(mockFetch).toHaveBeenCalledWith( + '/api/session-metadata', + expect.objectContaining({ + body: JSON.stringify({ + provider: 'codex', + sessionId: 'session-non-git', + sessionType: 'freshcodex', + sessionTypeSource: 'materialized', + checkpointWorkspace: { path: '/tmp/non-git-workspace', provenance: 'directory-picker' }, + }), + }), + ) + }) + it('sends auth token in headers', async () => { mockFetch.mockResolvedValueOnce({ ok: true, From e209f7ecd90f3fbbee4909905181b1f7a9f8fc50 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 18:50:47 -0700 Subject: [PATCH 349/493] test: verify nested OpenCode MCP startup --- .../managed-provider-environment-target.ts | 106 ++++++++++++++++-- 1 file changed, 99 insertions(+), 7 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 914cf0788..f6a15aa1d 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -65,6 +65,55 @@ server.listen(listenPort, listenHost, () => { }) ` +const managedMcpServerSource = String.raw`import { writeFileSync } from 'node:fs' +import dependency from 'managed-fixture-mcp-dependency' + +const [receiptPath, expectedNonce] = process.argv.slice(2) +if (!receiptPath || !expectedNonce) process.exit(12) +writeFileSync(receiptPath, JSON.stringify({ + expectedNonce, + dependency, + bun: typeof Bun === 'object', + bunVersion: Bun.version, + launchCwd: process.cwd(), + configuredCwd: process.env.FRESHELL_EXPECTED_MCP_CWD, + executable: process.execPath, + scriptPath: new URL(import.meta.url).pathname, +})) + +let buffered = '' +process.stdin.setEncoding('utf8') +process.stdin.on('data', (chunk) => { + buffered += chunk + while (true) { + const boundary = buffered.indexOf('\n') + if (boundary < 0) break + const line = buffered.slice(0, boundary) + buffered = buffered.slice(boundary + 1) + let request + try { request = JSON.parse(line) } catch { continue } + if (!request || !Object.hasOwn(request, 'id')) continue + let result + if (request.method === 'initialize') { + result = { + protocolVersion: request.params?.protocolVersion ?? '2024-11-05', + capabilities: { tools: {} }, + serverInfo: { name: 'freshell-managed-bun-fixture', version: '1.0.0' }, + } + } else if (request.method === 'tools/list') { + result = { tools: [{ name: 'fixture_probe', description: 'Managed Bun launch proof', inputSchema: { type: 'object', properties: {} } }] } + } else if (request.method === 'ping') { + result = {} + } else { + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, + error: { code: -32601, message: 'Method not found' } }) + '\n') + continue + } + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n') + } +}) +` + export function resolveManagedProviderEnvironmentTargetMode(args: string[]): ManagedProviderEnvironmentTargetMode { if (args.length === 0) return 'managed-environment' if (args.length === 1 && args[0] === '--checkpoint') return 'code-checkpoint' @@ -203,7 +252,12 @@ async function webChildren(rig: ManagedRuntimeBrowserRig): Promise<number[]> { async function configure(root: string, nonce: string, candidateSha: string, runId: string) { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') const resources = path.join(workspace, 'selected-resources'), plugin = path.join(resources, 'fixture-plugin') + const mcpPackage = path.join(resources, 'fixture-mcp') + const mcpNested = path.join(mcpPackage, 'nested') + const mcpParentDependency = path.join(resources, 'node_modules/managed-fixture-mcp-dependency') await fs.mkdir(path.join(plugin, 'node_modules/managed-fixture-dependency'), { recursive: true, mode: 0o755 }) + await fs.mkdir(mcpNested, { recursive: true, mode: 0o755 }) + await fs.mkdir(mcpParentDependency, { recursive: true, mode: 0o755 }) await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) await fs.chmod(home, 0o700) await fs.chmod(root, 0o751) @@ -236,6 +290,17 @@ export default {id:'managed-environment-tui',async tui(api,options){ const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) + await writeJson(path.join(mcpPackage, 'package.json'), { + type: 'module', + scripts: { 'mcp.js': 'bun ./server.mjs' }, + }) + await fs.writeFile(path.join(mcpPackage, 'server.mjs'), managedMcpServerSource) + await writeJson(path.join(mcpParentDependency, 'package.json'), { + name: 'managed-fixture-mcp-dependency', + type: 'module', + exports: './index.mjs', + }) + await fs.writeFile(path.join(mcpParentDependency, 'index.mjs'), `export default ${JSON.stringify(nonce)};`) const absoluteRule = path.join(resources, 'absolute-rule.txt') await fs.writeFile(absoluteRule, nonce, { mode: 0o600 }) const options = { expectedNonce: nonce, candidateSha, runId } @@ -247,10 +312,21 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.chmod(cwd, 0o777) await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) + const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') await writeJson(path.join(cwd, 'opencode.json'), { agent: { fixtureRelative: { - prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } } }) + prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } }, + mcp: { + managed_fixture: { + type: 'local', + command: ['bun', 'run', 'mcp.js', mcpReceipt, nonce], + cwd: mcpNested, + enabled: true, + environment: { FRESHELL_EXPECTED_MCP_CWD: mcpNested }, + }, + }, + }) } - return { home, xdg, workspace, command } + return { home, xdg, workspace, command, mcpPackage, mcpNested } } async function socketCreate(rig: ManagedRuntimeBrowserRig, requestId: string, mode: 'shell' | 'opencode', cwd: string, @@ -939,7 +1015,7 @@ export async function runCodeCheckpointTarget() { assert.equal(paired?.kind, 'code-checkpoint') const browserSpec = 'test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts' cloudEnvironment.FRESHELL_E2E_CLOUD_DISPATCH_RECEIPT = cloudDispatchReceiptPath - browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--project=chromium-external-target', browserSpec], { + browser = spawn('pnpm', ['run', 'test:e2e', '--shards=1', '--workers=1', '--project=chromium-external-target', browserSpec], { cwd: repoRoot, env: cloudEnvironment, stdio: ['ignore', 'pipe', 'pipe'], detached: true, }) browser.stdout!.on('data', chunk => browserSummaryCapture.write(chunk)) @@ -954,8 +1030,8 @@ export async function runCodeCheckpointTarget() { const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud checkpoint browser final join') assert.equal(browserExit.code, 0, 'Configured Cloud checkpoint browser acceptance failed') const browserSummary = browserSummaryCapture.finish() - assert.equal(browserSummary.executedCases, 3, 'Configured Cloud must execute all three checkpoint browser cases') - assert.equal(browserSummary.passedCases, 3, 'Every configured checkpoint browser case must pass') + assert.equal(browserSummary.executedCases, 6, 'Configured Cloud must execute all six checkpoint browser cases') + assert.equal(browserSummary.passedCases, 6, 'Every configured checkpoint browser case must pass') assert.equal(browserSummary.failedCases, 0) assert.equal(browserSummary.skippedCases, 0) assert.equal(browserSummary.interruptedCases, 0) @@ -1142,12 +1218,20 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightChild.runId, rig.runtime.runId) assert.equal(preflightChild.expectedNonce, nonce) assert.equal(preflightChild.bun, true) + const preflightMcp = await poll('OpenCode starts the nested Bun package-script MCP', async () => + readJson(path.join(selected.workspace, 'preflight/.managed-fixture/mcp-receipt.json'))) + assert.equal(preflightMcp.expectedNonce, nonce) + assert.equal(preflightMcp.dependency, nonce, 'MCP must load its dependency from the staged parent node_modules') + assert.equal(preflightMcp.bun, true, 'OpenCode must run the local MCP package script under Bun') + assert.equal(preflightMcp.launchCwd, selected.mcpNested, 'OpenCode must honor the nested MCP cwd') + assert.equal(preflightMcp.configuredCwd, selected.mcpNested) + assert(preflightMcp.scriptPath.endsWith('/fixture-mcp/server.mjs')) const preflightView = await poll('genuine managed preflight native identity', () => observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) assert.deepEqual(await webChildren(rig), [], 'Managed child must belong to the disposable session host, not the host web process') proof.preflight = { terminalId: created.terminalId, soulId: preflightView.soulId, incarnationId: preflightView.incarnationId, - stage: await stageProof(rig, preflightView, nonce), child: preflightChild } + stage: await stageProof(rig, preflightView, nonce), child: preflightChild, mcpServer: preflightMcp } const toolHome = path.join(root, 'tools') await fs.mkdir(toolHome, { mode: 0o700 }) const tool = path.join(toolHome, 'cloudflared') @@ -1234,9 +1318,17 @@ export async function runManagedEnvironmentTarget() { assert.equal(child.expectedNonce, nonce); assert.equal(child.candidateSha, fixture.candidateSha); assert.equal(child.runId, fixture.runId) assert.equal(child.bun, true); assert.equal(child.opencodeVersion, '1.18.21'); assert.match(child.bunExecutable, /opencode/) assert.equal(child.executableMode, '0711') + const mcpServer = await poll('Cloud OpenCode starts the nested Bun package-script MCP', async () => + readJson(path.join(cwd, '.managed-fixture/mcp-receipt.json'))) + assert.equal(mcpServer.expectedNonce, nonce) + assert.equal(mcpServer.dependency, nonce, 'OpenCode MCP must load the staged parent dependency') + assert.equal(mcpServer.bun, true, 'OpenCode must run the local MCP package script under Bun') + assert.equal(mcpServer.launchCwd, selected.mcpNested, 'OpenCode must honor the nested MCP cwd') + assert.equal(mcpServer.configuredCwd, selected.mcpNested) + assert(mcpServer.scriptPath.endsWith('/fixture-mcp/server.mjs')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, - nativeSessionId: view.nativeSessionId, stage, child } + nativeSessionId: view.nativeSessionId, stage, child, mcpServer } await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt } }, From 84749dfe01cfe29a0cabf5ab63f6b2eaea0f7091 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 19:02:30 -0700 Subject: [PATCH 350/493] feat(runtime): include pinned Bun for MCP commands --- docker/runtime/Dockerfile | 3 +++ docker/runtime/README.md | 5 +++-- docker/runtime/provider-versions.json | 6 ++++++ scripts/testing/managed-provider-environment-target.ts | 2 ++ 4 files changed, 14 insertions(+), 2 deletions(-) diff --git a/docker/runtime/Dockerfile b/docker/runtime/Dockerfile index f9026dd9d..4a4d95814 100644 --- a/docker/runtime/Dockerfile +++ b/docker/runtime/Dockerfile @@ -36,6 +36,7 @@ FROM ubuntu@sha256:33ceb71981b602c1a7443a53469e4dba065f7503eab3078a2d7a57a2ab987 ARG CLAUDE_CODE_VERSION=2.1.263 ARG CODEX_VERSION=0.147.0 ARG OPENCODE_VERSION=1.18.21 +ARG BUN_VERSION=1.4.3 ARG AMPLIFIER_APP_CLI_COMMIT=1873aa980535c99a743b17172e4231833f6c8741 ARG AMPLIFIER_UV_VERSION=0.8.22 ARG AMPLIFIER_PROVIDER_VLLM_COMMIT=e1846e907b0ce90b3e8748346244e548fa0d3241 @@ -51,6 +52,7 @@ RUN apt-get update \ "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" \ "@openai/codex@${CODEX_VERSION}" \ "opencode-ai@${OPENCODE_VERSION}" \ + "bun@${BUN_VERSION}" \ && python3 -m venv /opt/uv \ && /opt/uv/bin/pip install --no-cache-dir "uv==${AMPLIFIER_UV_VERSION}" \ && git clone --filter=blob:none https://github.com/microsoft/amplifier-app-cli /opt/amplifier-src \ @@ -69,6 +71,7 @@ RUN apt-get update \ && claude --version | grep -F "${CLAUDE_CODE_VERSION}" \ && codex --version | grep -F "${CODEX_VERSION}" \ && opencode --version | grep -Fx "${OPENCODE_VERSION}" \ + && bun --version | grep -Fx "${BUN_VERSION}" \ && /opt/amplifier-src/.venv/bin/python -c 'import importlib.metadata as m; assert m.version("amplifier-app-cli") == "0.1.1"' \ && /opt/amplifier-src/.venv/bin/python -c 'import importlib.metadata as m; assert m.version("openai") == "2.8.1"; assert m.version("openai-harmony") == "0.0.8"; assert m.version("amplifier-module-provider-vllm") == "1.0.0"' \ && /lib/x86_64-linux-gnu/libc.so.6 | grep -F '2.39' diff --git a/docker/runtime/README.md b/docker/runtime/README.md index 91eb92b22..983f01b80 100644 --- a/docker/runtime/README.md +++ b/docker/runtime/README.md @@ -1,7 +1,7 @@ # Managed runtime image and deployment contract Phase 2 uses a digest-pinned workload image based on Node 22.23.2 with exact -Claude Code 2.1.263 and OpenCode 1.18.21 installs. `provider-versions.json` is the machine-readable +Claude Code 2.1.263, OpenCode 1.18.21, and Bun 1.4.3 installs. `provider-versions.json` is the machine-readable version receipt and `Dockerfile` must reproduce it; floating `latest` tags are not accepted by the supervisor or live gate. The runtime harness disables BuildKit provenance/SBOM attestations for this local workload build so unrelated @@ -49,7 +49,8 @@ while Node's environment proxy is enabled, then verifies Freshell's API request did not reach the proxy. The controller supplies each incarnation's scoped MCP grant in the provider environment. Provider MCP configuration, including user entries, -keeps its ordinary semantics. +keeps its ordinary semantics. Bun is available for local MCP commands that run +package scripts with `bun run`. Provider credentials and MCP capability bytes are never placed in durable launch records or terminal environment snapshots. Managed providers obtain diff --git a/docker/runtime/provider-versions.json b/docker/runtime/provider-versions.json index 41ca6ed64..75c7ee2b3 100644 --- a/docker/runtime/provider-versions.json +++ b/docker/runtime/provider-versions.json @@ -11,6 +11,12 @@ "digest": "sha256:83f487e0a63425e5b4d146fb5e5be574bcbe1b7b843d3ebafdd95eaf7767a7e5" }, "node": "22.23.2", + "runtimeTools": { + "bun": { + "package": "bun", + "version": "1.4.3" + } + }, "providers": { "claude": { "package": "@anthropic-ai/claude-code", diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index f6a15aa1d..920ae8450 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1223,6 +1223,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightMcp.expectedNonce, nonce) assert.equal(preflightMcp.dependency, nonce, 'MCP must load its dependency from the staged parent node_modules') assert.equal(preflightMcp.bun, true, 'OpenCode must run the local MCP package script under Bun') + assert.equal(preflightMcp.bunVersion, '1.4.3', 'Managed OpenCode MCP must use the pinned Bun runtime') assert.equal(preflightMcp.launchCwd, selected.mcpNested, 'OpenCode must honor the nested MCP cwd') assert.equal(preflightMcp.configuredCwd, selected.mcpNested) assert(preflightMcp.scriptPath.endsWith('/fixture-mcp/server.mjs')) @@ -1323,6 +1324,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(mcpServer.expectedNonce, nonce) assert.equal(mcpServer.dependency, nonce, 'OpenCode MCP must load the staged parent dependency') assert.equal(mcpServer.bun, true, 'OpenCode must run the local MCP package script under Bun') + assert.equal(mcpServer.bunVersion, '1.4.3', 'Managed OpenCode MCP must use the pinned Bun runtime') assert.equal(mcpServer.launchCwd, selected.mcpNested, 'OpenCode must honor the nested MCP cwd') assert.equal(mcpServer.configuredCwd, selected.mcpNested) assert(mcpServer.scriptPath.endsWith('/fixture-mcp/server.mjs')) From 3ff81b08cc3725402c5c3fdd09d7e9c33b50af0c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 19:13:52 -0700 Subject: [PATCH 351/493] test: assert nested Bun MCP launch cwd correctly --- scripts/testing/managed-provider-environment-target.ts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 920ae8450..241c4a099 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -76,6 +76,7 @@ writeFileSync(receiptPath, JSON.stringify({ bun: typeof Bun === 'object', bunVersion: Bun.version, launchCwd: process.cwd(), + bunCommandCwd: process.env.npm_config_local_prefix, configuredCwd: process.env.FRESHELL_EXPECTED_MCP_CWD, executable: process.execPath, scriptPath: new URL(import.meta.url).pathname, @@ -1224,7 +1225,8 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightMcp.dependency, nonce, 'MCP must load its dependency from the staged parent node_modules') assert.equal(preflightMcp.bun, true, 'OpenCode must run the local MCP package script under Bun') assert.equal(preflightMcp.bunVersion, '1.4.3', 'Managed OpenCode MCP must use the pinned Bun runtime') - assert.equal(preflightMcp.launchCwd, selected.mcpNested, 'OpenCode must honor the nested MCP cwd') + assert.equal(preflightMcp.launchCwd, selected.mcpPackage, 'Bun runs a package script from its package root') + assert.equal(preflightMcp.bunCommandCwd, selected.mcpNested, 'OpenCode must launch Bun from the configured nested MCP cwd') assert.equal(preflightMcp.configuredCwd, selected.mcpNested) assert(preflightMcp.scriptPath.endsWith('/fixture-mcp/server.mjs')) const preflightView = await poll('genuine managed preflight native identity', () => @@ -1325,7 +1327,8 @@ export async function runManagedEnvironmentTarget() { assert.equal(mcpServer.dependency, nonce, 'OpenCode MCP must load the staged parent dependency') assert.equal(mcpServer.bun, true, 'OpenCode must run the local MCP package script under Bun') assert.equal(mcpServer.bunVersion, '1.4.3', 'Managed OpenCode MCP must use the pinned Bun runtime') - assert.equal(mcpServer.launchCwd, selected.mcpNested, 'OpenCode must honor the nested MCP cwd') + assert.equal(mcpServer.launchCwd, selected.mcpPackage, 'Bun runs a package script from its package root') + assert.equal(mcpServer.bunCommandCwd, selected.mcpNested, 'OpenCode must launch Bun from the configured nested MCP cwd') assert.equal(mcpServer.configuredCwd, selected.mcpNested) assert(mcpServer.scriptPath.endsWith('/fixture-mcp/server.mjs')) const stage = await stageProof(rig!, view, nonce) From c99dec065bd4157153c878513ca589c1ff0bb335 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 19:54:29 -0700 Subject: [PATCH 352/493] test: prove managed MCP staging and checkpoint history --- crates/freshell-server/src/checkpoints.rs | 42 +++++ .../managed-provider-environment-target.ts | 15 +- ...code-checkpoint-workspace-external.spec.ts | 20 ++- .../codex-app-server/fake-app-server.mjs | 28 +++- .../fake-codex-app-server-history.test.ts | 155 ++++++++++++++++++ ...anaged-provider-environment-target.test.ts | 9 + 6 files changed, 253 insertions(+), 16 deletions(-) create mode 100644 test/unit/tooling/testing/fake-codex-app-server-history.test.ts diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index cc27f7f32..540d8c086 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -7311,6 +7311,48 @@ mod tests { assert_eq!(status, StatusCode::OK, "{entry}"); assert_eq!(entry["workspace"]["path"], json!(home_dir.path())); assert_eq!(entry["workspace"]["provenance"], json!("directory-picker")); + let checkpoint_id = entry["id"].as_str().unwrap().to_string(); + + tokio::fs::write(home_dir.path().join("private.txt"), "changed after capture") + .await + .unwrap(); + restore_checkpoint(home_dir.path(), home_dir.path(), &checkpoint_id) + .await + .expect("the selected HOME checkpoint restores"); + assert_eq!( + tokio::fs::read_to_string(home_dir.path().join("private.txt")) + .await + .unwrap(), + "private" + ); + + let list_params = HashMap::from([ + ( + "cwd".to_string(), + home_dir.path().to_string_lossy().into_owned(), + ), + ( + "workspacePath".to_string(), + home_dir.path().to_string_lossy().into_owned(), + ), + ( + "workspaceProvenance".to_string(), + "directory-picker".to_string(), + ), + ]); + let listed = get_checkpoints( + State(state(home_dir.path())), + headers_with_token("tok"), + Query(list_params), + ) + .await; + assert_eq!(listed.status(), StatusCode::OK); + let listed = body_json(listed).await; + assert!(listed["checkpoints"] + .as_array() + .unwrap() + .iter() + .any(|checkpoint| { checkpoint["id"].as_str() == Some(checkpoint_id.as_str()) })); let git_dir = checkpoint_git_dir(home_dir.path(), home_dir.path()); let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 241c4a099..442c9d6fe 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -250,12 +250,19 @@ async function webChildren(rig: ManagedRuntimeBrowserRig): Promise<number[]> { return [...children].sort((left, right) => left - right) } -async function configure(root: string, nonce: string, candidateSha: string, runId: string) { - const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg'), workspace = path.join(root, 'workspace') - const resources = path.join(workspace, 'selected-resources'), plugin = path.join(resources, 'fixture-plugin') +export function managedEnvironmentFixtureLayout(root: string) { + const workspace = path.join(root, 'workspace') + const resources = path.join(root, 'selected-resources') + const plugin = path.join(resources, 'fixture-plugin') const mcpPackage = path.join(resources, 'fixture-mcp') const mcpNested = path.join(mcpPackage, 'nested') const mcpParentDependency = path.join(resources, 'node_modules/managed-fixture-mcp-dependency') + return { workspace, resources, plugin, mcpPackage, mcpNested, mcpParentDependency } +} + +async function configure(root: string, nonce: string, candidateSha: string, runId: string) { + const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg') + const { workspace, resources, plugin, mcpPackage, mcpNested, mcpParentDependency } = managedEnvironmentFixtureLayout(root) await fs.mkdir(path.join(plugin, 'node_modules/managed-fixture-dependency'), { recursive: true, mode: 0o755 }) await fs.mkdir(mcpNested, { recursive: true, mode: 0o755 }) await fs.mkdir(mcpParentDependency, { recursive: true, mode: 0o755 }) @@ -848,7 +855,7 @@ export async function runCodeCheckpointTarget() { `FRESHELL_LOG_DIR=${checkpointFixtureRoot}/storage/logs`, 'FRESHELL_CLIENT_DIR=/checkpoint-target/client', 'CODEX_CMD=/checkpoint-target/fake-codex', - 'FAKE_CODEX_APP_SERVER_BEHAVIOR={"recordTurns":true,"uniqueThreadIds":true}', + 'FAKE_CODEX_APP_SERVER_BEHAVIOR={"recordTurns":true,"uniqueThreadIds":true,"persistRolloutUserMessages":true}', 'FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES=1', `FRESHELL_CHECKPOINT_FIXTURE_ROOT=${checkpointFixtureRoot}`, `FRESHELL_CHECKPOINT_FIXTURE_UID=${identity.uid}`, diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index cf7bc259e..374801859 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -570,6 +570,18 @@ test('picker-confirmed HOME captures and restores a harmless home file', async ( .toBeVisible() await assertNativeUserIdentity(page, turn.outcome, turn.accepted.submittedTurnId, marker) + const query = new URLSearchParams({ + cwd: fixture.syntheticHomePath, + workspacePath: fixture.syntheticHomePath, + workspaceProvenance: 'directory-picker', + }) + const listUrl = `${paired!.targetUrl}/api/fresh-agent/checkpoints?${query}` + const capturedList = await page.request.get(listUrl, { headers, timeout: 30_000 }) + expect(capturedList.ok()).toBe(true) + expect(await capturedList.json()).toMatchObject({ + checkpoints: [expect.objectContaining({ id: turn.outcome.checkpointId })], + }) + await targetWrite(page, homeFile, 'home file changed after send\n') await turn.row.hover() const rewindNotice = expect(paneRoot.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) @@ -579,13 +591,7 @@ test('picker-confirmed HOME captures and restores a harmless home file', async ( await Promise.all([dialogAccepted, rewindNotice]) await expect.poll(() => targetRead(page, homeFile)).toBe('home file before checkpoint\n') - const query = new URLSearchParams({ - cwd: fixture.syntheticHomePath, - workspacePath: fixture.syntheticHomePath, - workspaceProvenance: 'directory-picker', - }) - const list = await page.request.get(`${paired!.targetUrl}/api/fresh-agent/checkpoints?${query}`, - { headers, timeout: 30_000 }) + const list = await page.request.get(listUrl, { headers, timeout: 30_000 }) expect(list.ok()).toBe(true) expect(await list.json()).toMatchObject({ checkpoints: [expect.objectContaining({ id: turn.outcome.checkpointId })] }) const workspaceKey = createHash('sha1').update(resolve(fixture.syntheticHomePath)).digest('hex').slice(0, 16) diff --git a/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs b/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs index 63cae7437..282968780 100644 --- a/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs +++ b/test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs @@ -183,17 +183,32 @@ function ensureDurableArtifact(threadId) { // session_meta first line, the shape the real codex rollout writer produces // and the Rust indexer parses (parse_codex_session_content requires an id + // cwd -- the R10b cwd-less exclusion gate skips files without one). - fs.writeFileSync(thread.path, JSON.stringify({ - timestamp: now.toISOString(), - type: 'session_meta', - payload: { id: threadId, cwd: process.cwd(), createdAt: now.toISOString() }, - }) + '\n', 'utf8') + try { + fs.writeFileSync(thread.path, JSON.stringify({ + timestamp: now.toISOString(), + type: 'session_meta', + payload: { id: threadId, cwd: process.cwd(), createdAt: now.toISOString() }, + }) + '\n', { encoding: 'utf8', flag: 'wx' }) + } catch (error) { + if (error?.code !== 'EEXIST') throw error + } return { codexHome, thread, } } +function appendDurableUserMessage(thread, params) { + const message = Array.isArray(params?.input) + ? params.input.map((part) => (part && typeof part.text === 'string' ? part.text : '')).filter(Boolean).join('\n') + : '' + fs.appendFileSync(thread.path, JSON.stringify({ + timestamp: new Date().toISOString(), + type: 'event_msg', + payload: { type: 'user_message', message }, + }) + '\n', 'utf8') +} + function writeBytes(stream, totalBytes, chunkSize = 16 * 1024) { if (!Number.isFinite(totalBytes) || totalBytes <= 0) { return Promise.resolve() @@ -994,6 +1009,9 @@ wss.on('connection', (socket) => { } if (method === 'turn/start' && message.params?.threadId) { const { thread } = ensureDurableArtifact(message.params.threadId) + if (behavior.persistRolloutUserMessages) { + appendDurableUserMessage(thread, message.params) + } const rolloutPath = thread.path const rolloutParent = path.dirname(rolloutPath) for (const [watchId, watchedPath] of watches) { diff --git a/test/unit/tooling/testing/fake-codex-app-server-history.test.ts b/test/unit/tooling/testing/fake-codex-app-server-history.test.ts new file mode 100644 index 000000000..bff28681d --- /dev/null +++ b/test/unit/tooling/testing/fake-codex-app-server-history.test.ts @@ -0,0 +1,155 @@ +import { spawn, type ChildProcess } from 'node:child_process' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { fileURLToPath } from 'node:url' +import WebSocket from 'ws' +import { describe, expect, it } from 'vitest' + +const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..') +const fakeAppServer = path.join(repoRoot, 'test/fixtures/coding-cli/codex-app-server/fake-app-server.mjs') + +const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) + +async function waitForPort(portFile: string, child: ChildProcess): Promise<number> { + const deadline = Date.now() + 5_000 + let stderr = '' + child.stderr?.on('data', chunk => { stderr = (stderr + chunk.toString()).slice(-4_000) }) + while (Date.now() < deadline) { + if (child.exitCode !== null || child.signalCode !== null) { + throw new Error(`fake Codex app-server exited before listening: ${stderr}`) + } + const raw = await fs.readFile(portFile, 'utf8').catch(error => { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return undefined + throw error + }) + if (raw) { + const port = Number(raw.trim()) + if (Number.isInteger(port) && port > 0 && port <= 65535) return port + throw new Error('fake Codex app-server published an invalid port') + } + await delay(20) + } + throw new Error(`fake Codex app-server did not publish its port: ${stderr}`) +} + +async function connectWebSocket(url: string): Promise<WebSocket> { + return new Promise((resolve, reject) => { + const socket = new WebSocket(url) + const timer = setTimeout(() => { + socket.terminate() + reject(new Error('fake Codex app-server websocket connection timed out')) + }, 5_000) + socket.once('open', () => { + clearTimeout(timer) + resolve(socket) + }) + socket.once('error', error => { + clearTimeout(timer) + reject(error) + }) + }) +} + +async function rpc(socket: WebSocket, id: string, method: string, params: Record<string, unknown>) { + return new Promise<Record<string, any>>((resolve, reject) => { + const timer = setTimeout(() => { + socket.off('message', onMessage) + reject(new Error(`fake Codex app-server ${method} timed out`)) + }, 5_000) + const onMessage = (raw: WebSocket.RawData) => { + let frame: Record<string, any> + try { frame = JSON.parse(raw.toString()) } + catch { return } + if (frame.id !== id) return + clearTimeout(timer) + socket.off('message', onMessage) + if (frame.error) reject(new Error(String(frame.error.message ?? method))) + else resolve(frame) + } + socket.on('message', onMessage) + socket.send(JSON.stringify({ jsonrpc: '2.0', id, method, params }), error => { + if (!error) return + clearTimeout(timer) + socket.off('message', onMessage) + reject(error) + }) + }) +} + +async function stop(child: ChildProcess | undefined): Promise<void> { + if (!child || child.exitCode !== null || child.signalCode !== null) return + const exited = new Promise<void>(resolve => child.once('exit', () => resolve())) + child.kill('SIGTERM') + const stopped = await Promise.race([exited.then(() => true), delay(3_000).then(() => false)]) + if (!stopped) { + child.kill('SIGKILL') + await exited + } +} + +describe('fake Codex app-server history transcript', () => { + it('persists the submitted user message in the rollout so History can display the session', async () => { + const owned = await fs.mkdtemp(path.join(os.tmpdir(), 'fake-codex-history-')) + const workspace = path.join(owned, 'workspace') + const codexHome = path.join(owned, '.codex') + const portFile = path.join(owned, 'port') + const threadId = 'thread-history-visible' + const prompts = ['Visible checkpoint history title', 'A later visible history message'] + await fs.mkdir(workspace) + let child: ChildProcess | undefined + let socket: WebSocket | undefined + try { + child = spawn(process.execPath, [fakeAppServer, '--listen', 'ws://127.0.0.1:0'], { + cwd: workspace, + env: { + ...process.env, + CODEX_HOME: codexHome, + FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES: '1', + FAKE_CODEX_APP_SERVER_PORT_FILE: portFile, + FAKE_CODEX_APP_SERVER_BEHAVIOR: JSON.stringify({ + threadStartThreadId: threadId, + persistRolloutUserMessages: true, + }), + }, + stdio: ['ignore', 'ignore', 'pipe'], + }) + const port = await waitForPort(portFile, child) + socket = await connectWebSocket(`ws://127.0.0.1:${port}`) + await rpc(socket, 'initialize', 'initialize', {}) + socket.send(JSON.stringify({ jsonrpc: '2.0', method: 'initialized', params: {} })) + const started = await rpc(socket, 'thread-start', 'thread/start', { cwd: workspace }) + const rolloutPath = started.result?.thread?.path + expect(rolloutPath).toEqual(expect.any(String)) + for (const [index, prompt] of prompts.entries()) { + await rpc(socket, `turn-start-${index}`, 'turn/start', { + threadId, + input: [{ type: 'text', text: prompt }], + }) + } + + let records: Array<Record<string, any>> = [] + const deadline = Date.now() + 5_000 + while (Date.now() < deadline) { + const raw = await fs.readFile(rolloutPath, 'utf8').catch(error => { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return '' + throw error + }) + records = raw.split('\n').filter(Boolean).map(line => JSON.parse(line)) + const messages = records + .filter(record => record.type === 'event_msg' && record.payload?.type === 'user_message') + .map(record => record.payload.message) + if (messages.length === prompts.length) break + await delay(20) + } + expect(records[0]?.type).toBe('session_meta') + expect(records + .filter(record => record.type === 'event_msg' && record.payload?.type === 'user_message') + .map(record => record.payload.message)).toEqual(prompts) + } finally { + socket?.close() + await stop(child) + await fs.rm(owned, { recursive: true, force: true }) + } + }, 15_000) +}) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 29084b12b..6ea1d755a 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -1,10 +1,12 @@ import { spawn } from 'node:child_process' import fs from 'node:fs/promises' import { createConnection, createServer, type AddressInfo } from 'node:net' +import path from 'node:path' import { describe, expect, it } from 'vitest' import { checkpointTcpForwarderSource, externalTargetRead, + managedEnvironmentFixtureLayout, processIdentity, resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, @@ -47,6 +49,13 @@ describe('managed provider environment target mode', () => { expect(resolveManagedProviderEnvironmentTargetMode(['--checkpoint'])).toBe('code-checkpoint') }) + it('keeps the OpenCode MCP package and parent dependency outside the mounted workspace', () => { + const fixture = managedEnvironmentFixtureLayout('/tmp/managed-environment-fixture') + const packageRelativeToWorkspace = path.relative(fixture.workspace, fixture.mcpPackage) + expect(packageRelativeToWorkspace.startsWith(`..${path.sep}`) || path.isAbsolute(packageRelativeToWorkspace)).toBe(true) + expect(fixture.mcpParentDependency).toBe(path.join(fixture.resources, 'node_modules/managed-fixture-mcp-dependency')) + }) + it('forwards one fixed TCP target while preserving streamed client data', async () => { const upstream = createServer(socket => { socket.on('data', chunk => socket.write(chunk)) From d9f41bcdeb1ad37a8b90a4ab97fdc78f0c89f250 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:16:56 -0700 Subject: [PATCH 353/493] fix: stage indexed OpenCode files for managed launch --- .../src/provider_inputs.rs | 40 ++++ .../managed_mcp_capability/source_stage.rs | 128 +++++++++++ .../src/provider_input_preparation.rs | 2 + .../freshell-supervisor/src/backend/docker.rs | 203 +++++++++++++++--- .../src/backend/input_publisher.rs | 60 +++++- .../src/backend/prepared_inputs.rs | 1 + .../src/native_auth_lookup_tests.rs | 1 + .../src/provider_preparation_tests.rs | 1 + crates/freshell-supervisor/src/recovery.rs | 1 + 9 files changed, 406 insertions(+), 31 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 2759b8576..5895fed20 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -188,6 +188,16 @@ pub struct IndexedProviderFiles { pub volume_name: String, #[serde(default)] pub aliases: Vec<IndexedProviderFileAlias>, + /// Paths inside the private prepared generation used to source indexed + /// payloads when the original host paths are outside the runtime host. + /// Missing is retained for older persisted launch specifications. + #[serde(default)] + #[serde(skip_serializing_if = "Option::is_none")] + pub staged_source_paths: Option<Vec<String>>, +} + +pub fn indexed_source_stage_path(source_id: u16) -> String { + format!(".freshell/indexed-source-files/source-{source_id}") } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -335,6 +345,13 @@ impl PreparedProviderInputs { }) || files.aliases.len() > 256 || files.aliases.len() != self.manifest.alias_count as usize + || files.staged_source_paths.as_ref().is_some_and(|paths| { + paths.len() != self.manifest.source_count as usize + || paths + .iter() + .enumerate() + .any(|(id, path)| path != &indexed_source_stage_path(id as u16)) + }) { return Err(invalid("indexed file publication")); } @@ -606,6 +623,7 @@ mod tests { effective_destination: "/fixture/other".into(), }, ], + staged_source_paths: None, }), } } @@ -670,6 +688,28 @@ mod tests { ); } #[test] + fn provider_prepared_indexed_source_paths_are_fixed_and_match_the_source_count() { + let mut value = prepared(); + value.indexed_files.as_mut().unwrap().staged_source_paths = Some(vec![ + ".freshell/indexed-source-files/source-0".into(), + ".freshell/indexed-source-files/source-1".into(), + ]); + value.validate(Some(&capability())).unwrap(); + + value + .indexed_files + .as_mut() + .unwrap() + .staged_source_paths + .as_mut() + .unwrap()[1] = ".freshell/indexed-source-files/../outside".into(); + assert!(value.validate(Some(&capability())).is_err()); + + value.indexed_files.as_mut().unwrap().staged_source_paths = + Some(vec![".freshell/indexed-source-files/source-0".into()]); + assert!(value.validate(Some(&capability())).is_err()); + } + #[test] fn provider_prepared_mounts_preserve_home_and_control_parents() { for destination in [ "/home/freshell/provider", diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 9c6538b91..568e68671 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -34,6 +34,69 @@ mod configured_resource_tests { } } + #[test] + fn managed_provider_stages_external_indexed_config_payloads_in_the_private_generation() { + use crate::managed_mcp_capability::OpencodeEphemeralInput; + use crate::managed_provider_references::sources::{ + plan_opencode_sources, DiscoveryRole, SourceOccurrence, + }; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&external).unwrap(); + let selected = external.join("absolute-rule.txt"); + std::fs::write(&selected, "external prompt payload").unwrap(); + let config = home.join("opencode.json"); + let raw = serde_json::json!({"agent":{"fixture":{"prompt":format!("{{file:{}}}", selected.display())}}}).to_string(); + std::fs::write(&config, &raw).unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Home, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let stage_root = owned.path().join("provider-generation"); + std::fs::create_dir(&stage_root).unwrap(); + + stage( + &stage_root, + &home, + &workspace, + &plan, + OpencodeEphemeralInput::default(), + ) + .unwrap(); + + std::fs::rename(&external, owned.path().join("external-hidden")).unwrap(); + assert_eq!( + std::fs::read(stage_root.join(".freshell/indexed-source-files/source-0")).unwrap(), + b"external prompt payload", + "the supervisor must receive selected bytes from the private generation, not the original host path" + ); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + assert_eq!( + std::fs::metadata(stage_root.join(".freshell/indexed-source-files/source-0")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o600 + ); + } + } + #[test] fn managed_provider_duplicate_resource_members_select_only_native_winners_after_both_copies() { use std::os::unix::fs::PermissionsExt; @@ -1309,11 +1372,69 @@ pub(super) fn stage( canonical_source, }); } + stage_indexed_sources(root, &manifest.files)?; let metadata = freshell_runtime_protocol::bounded_json_bytes(&manifest, 64 * 1024 * 1024) .map_err(|_| "source manifest metadata profile exceeded")?; write_private_bytes(&root.join(".freshell/input-manifest.json"), &metadata) } +fn stage_indexed_sources(root: &Path, files: &[SelectedSourceFile]) -> Result<(), String> { + use std::io::Read; + let source_root = root.join(".freshell/indexed-source-files"); + private_directory(&source_root)?; + let mut total_bytes = 0u64; + for (expected, source) in files.iter().enumerate() { + if usize::from(source.source_id) != expected { + return Err("source indexes are not contiguous".into()); + } + let path = Path::new(&source.canonical_source); + let canonical = std::fs::canonicalize(path) + .map_err(|_| "selected configuration payload is unavailable before staging")?; + let metadata = std::fs::symlink_metadata(path) + .map_err(|_| "selected configuration payload metadata is unavailable")?; + if canonical != path || !metadata.is_file() || metadata.file_type().is_symlink() { + return Err("selected configuration payload must be a canonical regular file".into()); + } + let file = std::fs::File::open(path) + .map_err(|_| "selected configuration payload cannot be opened for staging")?; + let opened = file + .metadata() + .map_err(|_| "selected configuration payload metadata is unavailable")?; + if !opened.is_file() + || opened.len() > freshell_platform::provider_input_exchange::MAX_FILE_BYTES + { + return Err("selected configuration payload exceeds the indexed file profile".into()); + } + total_bytes = total_bytes + .checked_add(opened.len()) + .filter(|bytes| *bytes <= freshell_platform::provider_input_exchange::MAX_TOTAL_BYTES) + .ok_or("selected configuration payloads exceed the indexed total profile")?; + let mut bytes = Vec::with_capacity(opened.len() as usize); + file.take(freshell_platform::provider_input_exchange::MAX_FILE_BYTES + 1) + .read_to_end(&mut bytes) + .map_err(|_| "selected configuration payload cannot be read for staging")?; + if bytes.len() as u64 > freshell_platform::provider_input_exchange::MAX_FILE_BYTES + || bytes.len() as u64 != opened.len() + { + return Err( + "selected configuration payload changed or exceeded its indexed profile".into(), + ); + } + let current = std::fs::canonicalize(path) + .map_err(|_| "selected configuration payload changed during staging")?; + let current_metadata = std::fs::symlink_metadata(path) + .map_err(|_| "selected configuration payload changed during staging")?; + if current != canonical || current_metadata.file_type().is_symlink() { + return Err("selected configuration payload changed during staging".into()); + } + let destination = root.join(freshell_runtime_protocol::indexed_source_stage_path( + source.source_id, + )); + write_private_bytes(&destination, &bytes)?; + } + Ok(()) +} + pub(super) struct DescriptionRoots<'a> { pub root: &'a Path, pub home: &'a Path, @@ -1606,6 +1727,13 @@ pub(super) fn describe( indexed_files: (!sources.is_empty()).then(|| IndexedProviderFiles { volume_name: indexed_provider_volume_name(soul, &capability.grant_id), aliases, + staged_source_paths: Some( + (0..sources.len()) + .map(|source_id| { + freshell_runtime_protocol::indexed_source_stage_path(source_id as u16) + }) + .collect(), + ), }), }; inputs diff --git a/crates/freshell-session-host/src/provider_input_preparation.rs b/crates/freshell-session-host/src/provider_input_preparation.rs index 0b5aa0cd0..2574d125d 100644 --- a/crates/freshell-session-host/src/provider_input_preparation.rs +++ b/crates/freshell-session-host/src/provider_input_preparation.rs @@ -688,6 +688,7 @@ mod tests { &capability.grant_id, ), aliases: vec![], + staged_source_paths: None, }), }), }; @@ -1839,6 +1840,7 @@ mod tests { indexed_files: Some(IndexedProviderFiles { volume_name: indexed_provider_volume_name(&SoulId::new(), &capability.grant_id), aliases: vec![], + staged_source_paths: None, }), }), preparation: ProviderPreparation::Opencode { diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 19b585b0e..4946dd762 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -155,6 +155,28 @@ fn workload_mounts( return Err("terminal cwd must be inside the approved workspace".into()); } reject_management_path(&workspace)?; + let provider_user_root = + if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { + Some(staged_provider_root_for( + &capability_directory_for(purpose)?, + &capability.grant_id, + purpose, + )?) + } else if context.is_some() { + let home = std::env::var_os("HOME"); + let user_root_exists = home.as_ref().is_some_and(|home| { + provider_user_root_path(Path::new(home), provider).is_some_and(|path| path.exists()) + }); + if let Some(home) = home.as_ref().filter(|_| user_root_exists) { + Some(approved_user_provider_root(Path::new(home), provider)?) + } else if context.is_some_and(user_provider_config_referenced) { + return Err("referenced provider user root is unavailable".into()); + } else { + None + } + } else { + None + }; let git_common_dir = git_common_path .map(|raw| -> Result<PathBuf, String> { let canonical = canonical_dir(Path::new(raw), "git common dir")?; @@ -187,6 +209,10 @@ fn workload_mounts( let indexed = context .and_then(|context| context.prepared_inputs.as_ref()) .is_some_and(|inputs| inputs.indexed_files.is_some()); + let staged_source_paths = context + .and_then(|context| context.prepared_inputs.as_ref()) + .and_then(|inputs| inputs.indexed_files.as_ref()) + .and_then(|files| files.staged_source_paths.as_deref()); let mut source_id = 0u16; for (index, secret) in secret_references.iter().enumerate() { if purpose == MountPurpose::Adopt { @@ -197,9 +223,46 @@ fn workload_mounts( } continue; } - let source = std::fs::canonicalize(&secret.source_path) - .map_err(|error| format!("provider secret reference: {error}"))?; - if !source.is_file() || source != Path::new(&secret.source_path) { + let staged_source = if secret.profile.is_configuration_file() { + staged_source_paths + .and_then(|paths| paths.get(usize::from(source_id))) + .map(|relative| -> Result<PathBuf, String> { + let root = provider_user_root + .as_deref() + .ok_or("prepared indexed source root is unavailable")?; + let path = root.join(relative); + let canonical = std::fs::canonicalize(&path) + .map_err(|error| format!("prepared indexed source: {error}"))?; + let metadata = std::fs::symlink_metadata(&path) + .map_err(|error| format!("prepared indexed source metadata: {error}"))?; + if canonical != path || !metadata.is_file() || metadata.file_type().is_symlink() + { + return Err( + "prepared indexed source must be a canonical regular file".into() + ); + } + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = metadata.permissions().mode(); + if mode & 0o007 != 0 || mode & 0o020 != 0 { + return Err("prepared indexed source must be private".into()); + } + } + Ok(canonical) + }) + .transpose()? + } else { + None + }; + let source_is_staged = staged_source.is_some(); + let source = if let Some(source) = staged_source { + source + } else { + std::fs::canonicalize(&secret.source_path) + .map_err(|error| format!("provider secret reference: {error}"))? + }; + if !source.is_file() || !source_is_staged && source != Path::new(&secret.source_path) { return Err("provider secret reference must be a canonical regular file".into()); } #[cfg(unix)] @@ -216,35 +279,15 @@ fn workload_mounts( return Err("provider OneCLI grant must be private".into()); } } - reject_management_path(&source)?; + if !source_is_staged { + reject_management_path(&source)?; + } provider_secret_files.push(source); if indexed && secret.profile.is_configuration_file() { indexed_secret_sources.insert(index, source_id); source_id += 1; } } - let provider_user_root = - if let Some(capability) = context.and_then(|context| context.mcp_capability.as_ref()) { - Some(staged_provider_root_for( - &capability_directory_for(purpose)?, - &capability.grant_id, - purpose, - )?) - } else if context.is_some() { - let home = std::env::var_os("HOME"); - let user_root_exists = home.as_ref().is_some_and(|home| { - provider_user_root_path(Path::new(home), provider).is_some_and(|path| path.exists()) - }); - if let Some(home) = home.as_ref().filter(|_| user_root_exists) { - Some(approved_user_provider_root(Path::new(home), provider)?) - } else if context.is_some_and(user_provider_config_referenced) { - return Err("referenced provider user root is unavailable".into()); - } else { - None - } - } else { - None - }; let mcp_capability_file = context .and_then(|context| context.mcp_capability.as_ref()) .map(|capability| { @@ -997,4 +1040,112 @@ mod provider_secret_tests { assert!(durable.contains(&canonical_grant.to_string_lossy().to_string())); assert!(!durable.contains("fixture-secret-bytes")); } + + #[cfg(unix)] + #[test] + fn indexed_configuration_reference_uses_its_private_generation_source() { + const CHILD: &str = "FRESHELL_INDEXED_STAGED_SOURCE_TEST_CHILD"; + if std::env::var_os(CHILD).is_none() { + let status = std::process::Command::new("/usr/bin/timeout") + .args(["--kill-after=1s", "20s"]) + .arg(std::env::current_exe().unwrap()) + .args([ + "--exact", + "backend::docker::provider_secret_tests::indexed_configuration_reference_uses_its_private_generation_source", + "--nocapture", + ]) + .env(CHILD, "1") + .status() + .unwrap(); + assert!(status.success(), "isolated indexed source mapping failed"); + return; + } + + use freshell_runtime_protocol::{ + IndexedProviderFiles, McpCapabilityReference, PreparedProviderInputs, + ProviderInputManifestReference, ProviderLaunchContext, ProviderPreparation, + ProviderSecretProfile, ProviderSecretReference, + }; + use sha2::{Digest, Sha256}; + use std::collections::BTreeMap; + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let capabilities = owned.path().join("mcp-capabilities"); + let stage = owned.path().join("provider-roots/grant-indexed"); + let workspace = owned.path().join("workspace"); + let indexed_source = stage.join(".freshell/indexed-source-files/source-0"); + std::fs::create_dir_all(&capabilities).unwrap(); + std::fs::create_dir_all(stage.join(".freshell/indexed-source-files")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let manifest = b"{}"; + std::fs::write(stage.join(".freshell/input-manifest.json"), manifest).unwrap(); + std::fs::write(&indexed_source, b"staged prompt payload").unwrap(); + std::fs::set_permissions(&indexed_source, std::fs::Permissions::from_mode(0o640)).unwrap(); + let capability = McpCapabilityReference { + grant_id: "grant-indexed".into(), + endpoint: "http://host.docker.internal:3344".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let capability_file = capabilities.join("grant-indexed.json"); + std::fs::write(&capability_file, b"{}").unwrap(); + std::fs::set_permissions(&capability_file, std::fs::Permissions::from_mode(0o600)).unwrap(); + std::env::set_var("FRESHELL_RUNTIME_CAPABILITY_DIR", &capabilities); + let context = ProviderLaunchContext { + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + config: vec![], + mcp_capability: Some(capability.clone()), + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: format!("{:x}", Sha256::digest(manifest)), + source_count: 1, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: Some(IndexedProviderFiles { + volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), + aliases: vec![], + staged_source_paths: Some(vec![ + freshell_runtime_protocol::indexed_source_stage_path(0), + ]), + }), + }), + }; + context.validate("opencode").unwrap(); + let workspace = std::fs::canonicalize(workspace) + .unwrap() + .to_string_lossy() + .into_owned(); + let mut terminal: TerminalLaunchSpec = serde_json::from_value(serde_json::json!({ + "terminalId":"terminal-indexed-source","streamId":"stream-indexed-source", + "mode":"opencode","program":"opencode","cwd":workspace.clone(),"workspacePath":workspace, + "runAsUid":65534,"runAsGid":0,"cols":80,"rows":24,"projectKey":"indexed-source", + "providerLaunchContext":context + })).unwrap(); + terminal.provider_secret_references = vec![ProviderSecretReference { + source_path: owned + .path() + .join("host-only/original-prompt.txt") + .display() + .to_string(), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(".freshell/references/0".into()), + }]; + + let mounts = terminal_mounts(&terminal).unwrap(); + + assert_eq!(mounts.provider_secret_files, vec![indexed_source]); + assert_eq!(mounts.indexed_secret_sources, BTreeMap::from([(0, 0)])); + } } diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index cfb04f68f..f7039d73f 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -77,6 +77,30 @@ fn labels(spec: &CreateRuntimeSpec, generation: &str) -> Value { "com.freshell.input-generation":generation}) } +fn indexed_payload_sources( + mounts: &docker::TerminalMounts, + source_count: usize, +) -> Result<Vec<PathBuf>, BackendError> { + let mut sources = vec![None; source_count]; + for (reference_index, source_id) in &mounts.indexed_secret_sources { + let source = mounts + .provider_secret_files + .get(*reference_index) + .cloned() + .ok_or_else(|| fail("input publisher source index unavailable"))?; + let target = sources + .get_mut(usize::from(*source_id)) + .ok_or_else(|| fail("input publisher source ID unavailable"))?; + if target.replace(source).is_some() { + return Err(fail("input publisher source ID is duplicated")); + } + } + sources + .into_iter() + .collect::<Option<Vec<_>>>() + .ok_or_else(|| fail("input publisher indexed source set is incomplete")) +} + fn filter_query(value: Value) -> String { serde_json::to_vec(&value) .expect("finite Docker filter") @@ -595,11 +619,7 @@ impl DockerEngineBackend { inputs .validate_references(references) .map_err(|_| fail("input publisher source identity mismatch"))?; - let sources = references - .iter() - .filter(|reference| reference.profile.is_configuration_file()) - .map(|reference| PathBuf::from(&reference.source_path)) - .collect::<Vec<_>>(); + let sources = indexed_payload_sources(mounts, inputs.manifest.source_count as usize)?; let soul = spec.soul_id.to_string(); { let held = self @@ -1143,6 +1163,36 @@ impl DockerEngineBackend { mod tests { use super::*; + #[test] + fn indexed_payload_publication_uses_the_prepared_source_paths_in_source_id_order() { + let mounts = docker::TerminalMounts { + workspace: "/workspace".into(), + git_common_dir: None, + provider_bootstrap_files: vec![], + provider_secret_files: vec![ + "/onecli.env".into(), + "/private-generation/source-1".into(), + "/private-generation/source-0".into(), + ], + indexed_secret_sources: BTreeMap::from([(1, 1), (2, 0)]), + provider_user_root: None, + mcp_capability_file: None, + input_manifest: None, + config_binds: vec![], + resource_mounts: vec![], + indexed_files: None, + }; + + assert_eq!( + indexed_payload_sources(&mounts, 2).unwrap(), + vec![ + PathBuf::from("/private-generation/source-0"), + PathBuf::from("/private-generation/source-1"), + ] + ); + assert!(indexed_payload_sources(&mounts, 3).is_err()); + } + #[tokio::test] async fn provider_input_publisher_commit_without_indexed_descriptor_needs_no_daemon() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs index 7818fbe8a..875b6ba05 100644 --- a/crates/freshell-supervisor/src/backend/prepared_inputs.rs +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -183,6 +183,7 @@ mod tests { logical_destination: "/outside/credential".into(), effective_destination: "/outside/credential".into(), }], + staged_source_paths: None, }), }; let descriptors = mount_descriptors(&mounts, "owned-provider").unwrap(); diff --git a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs index 3d622bc9b..a44453d71 100644 --- a/crates/freshell-supervisor/src/native_auth_lookup_tests.rs +++ b/crates/freshell-supervisor/src/native_auth_lookup_tests.rs @@ -643,6 +643,7 @@ async fn provider_native_auth_nested_callback_uses_current_attempt_without_reloc effective_destination: format!("/fixture/source-{index}"), }) .collect(), + staged_source_paths: None, }), }); let references = (0..6) diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs index 04e0ede17..525ca8dd1 100644 --- a/crates/freshell-supervisor/src/provider_preparation_tests.rs +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -74,6 +74,7 @@ fn context(soul: &SoulId, ordinal: usize) -> ProviderLaunchContext { logical_destination: "/home/freshell/provider/.freshell/references/0".into(), effective_destination: "/home/freshell/provider/.freshell/references/0".into(), }], + staged_source_paths: None, }), }), } diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index f2b2de932..cf0fbe08c 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -2609,6 +2609,7 @@ mod tests { &generation, ), aliases: vec![], + staged_source_paths: None, }), }); let complete = freshell_runtime_protocol::PreparedProviderContext { From c32818587c87cac7bcca94801e963dd391f073de Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:45:57 -0700 Subject: [PATCH 354/493] fix: stream indexed provider files without persistent copies --- Cargo.lock | 14 ++ .../managed_mcp_capability/source_stage.rs | 152 ++++++------- crates/freshell-supervisor/Cargo.toml | 1 + .../freshell-supervisor/src/backend/docker.rs | 38 +++- .../src/backend/input_publisher.rs | 200 ++++++++++++++++-- 5 files changed, 293 insertions(+), 112 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index b0c105620..2ba46712a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1553,6 +1553,7 @@ dependencies = [ "thiserror 2.0.18", "tokio", "tracing", + "tracing-subscriber", "uuid", ] @@ -5540,6 +5541,16 @@ dependencies = [ "tracing-core", ] +[[package]] +name = "tracing-serde" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "704b1aeb7be0d0a84fc9828cae51dab5970fee5088f83d1dd7ee6f6246fc6ff1" +dependencies = [ + "serde", + "tracing-core", +] + [[package]] name = "tracing-subscriber" version = "0.3.23" @@ -5550,12 +5561,15 @@ dependencies = [ "nu-ansi-term", "once_cell", "regex-automata", + "serde", + "serde_json", "sharded-slab", "smallvec", "thread_local", "tracing", "tracing-core", "tracing-log", + "tracing-serde", ] [[package]] diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 568e68671..4562fa3d5 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -35,7 +35,22 @@ mod configured_resource_tests { } #[test] - fn managed_provider_stages_external_indexed_config_payloads_in_the_private_generation() { + fn managed_provider_keeps_external_indexed_payloads_out_of_persistent_generation() { + fn tree_contains_bytes(path: &Path, expected: &[u8]) -> bool { + std::fs::read_dir(path).unwrap().any(|entry| { + let entry = entry.unwrap(); + let metadata = entry.metadata().unwrap(); + if metadata.is_dir() { + tree_contains_bytes(&entry.path(), expected) + } else { + std::fs::read(entry.path()) + .unwrap() + .windows(expected.len()) + .any(|window| window == expected) + } + }) + } + use crate::managed_mcp_capability::OpencodeEphemeralInput; use crate::managed_provider_references::sources::{ plan_opencode_sources, DiscoveryRole, SourceOccurrence, @@ -49,7 +64,7 @@ mod configured_resource_tests { std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(&external).unwrap(); let selected = external.join("absolute-rule.txt"); - std::fs::write(&selected, "external prompt payload").unwrap(); + std::fs::write(&selected, "PRIVATE_CONFIG_REFERENCE_CONTENT").unwrap(); let config = home.join("opencode.json"); let raw = serde_json::json!({"agent":{"fixture":{"prompt":format!("{{file:{}}}", selected.display())}}}).to_string(); std::fs::write(&config, &raw).unwrap(); @@ -77,24 +92,52 @@ mod configured_resource_tests { ) .unwrap(); - std::fs::rename(&external, owned.path().join("external-hidden")).unwrap(); - assert_eq!( - std::fs::read(stage_root.join(".freshell/indexed-source-files/source-0")).unwrap(), - b"external prompt payload", - "the supervisor must receive selected bytes from the private generation, not the original host path" + assert!( + selected.is_file(), + "the durable grant remains a path reference" ); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - assert_eq!( - std::fs::metadata(stage_root.join(".freshell/indexed-source-files/source-0")) - .unwrap() - .permissions() - .mode() - & 0o777, - 0o600 - ); - } + assert!( + !stage_root.join(".freshell/indexed-source-files").exists(), + "referenced file contents must flow through the existing tmpfs publisher, not persist in the provider generation" + ); + assert!( + !tree_contains_bytes(&stage_root, b"PRIVATE_CONFIG_REFERENCE_CONTENT"), + "referenced bytes must not be copied elsewhere in the persistent provider generation" + ); + + std::fs::write(stage_root.join(".freshell/resource-mounts.json"), b"[]").unwrap(); + let secret_references = [freshell_runtime_protocol::ProviderSecretReference { + source_path: selected.display().to_string(), + profile: freshell_runtime_protocol::ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(".freshell/references/0".into()), + }]; + let capability = freshell_runtime_protocol::McpCapabilityReference { + grant_id: "grant-indexed-source".into(), + endpoint: "http://host.docker.internal:3344".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let prepared = describe( + DescriptionRoots { + root: &stage_root, + home: &home, + config_root: None, + workspace: &workspace, + }, + &plan, + &freshell_runtime_protocol::SoulId::parse("soul-indexed-source").unwrap(), + &capability, + None, + &secret_references, + ) + .unwrap(); + prepared.validate_references(&secret_references).unwrap(); + assert!(prepared + .indexed_files + .as_ref() + .unwrap() + .staged_source_paths + .is_none()); } #[test] @@ -1372,69 +1415,11 @@ pub(super) fn stage( canonical_source, }); } - stage_indexed_sources(root, &manifest.files)?; let metadata = freshell_runtime_protocol::bounded_json_bytes(&manifest, 64 * 1024 * 1024) .map_err(|_| "source manifest metadata profile exceeded")?; write_private_bytes(&root.join(".freshell/input-manifest.json"), &metadata) } -fn stage_indexed_sources(root: &Path, files: &[SelectedSourceFile]) -> Result<(), String> { - use std::io::Read; - let source_root = root.join(".freshell/indexed-source-files"); - private_directory(&source_root)?; - let mut total_bytes = 0u64; - for (expected, source) in files.iter().enumerate() { - if usize::from(source.source_id) != expected { - return Err("source indexes are not contiguous".into()); - } - let path = Path::new(&source.canonical_source); - let canonical = std::fs::canonicalize(path) - .map_err(|_| "selected configuration payload is unavailable before staging")?; - let metadata = std::fs::symlink_metadata(path) - .map_err(|_| "selected configuration payload metadata is unavailable")?; - if canonical != path || !metadata.is_file() || metadata.file_type().is_symlink() { - return Err("selected configuration payload must be a canonical regular file".into()); - } - let file = std::fs::File::open(path) - .map_err(|_| "selected configuration payload cannot be opened for staging")?; - let opened = file - .metadata() - .map_err(|_| "selected configuration payload metadata is unavailable")?; - if !opened.is_file() - || opened.len() > freshell_platform::provider_input_exchange::MAX_FILE_BYTES - { - return Err("selected configuration payload exceeds the indexed file profile".into()); - } - total_bytes = total_bytes - .checked_add(opened.len()) - .filter(|bytes| *bytes <= freshell_platform::provider_input_exchange::MAX_TOTAL_BYTES) - .ok_or("selected configuration payloads exceed the indexed total profile")?; - let mut bytes = Vec::with_capacity(opened.len() as usize); - file.take(freshell_platform::provider_input_exchange::MAX_FILE_BYTES + 1) - .read_to_end(&mut bytes) - .map_err(|_| "selected configuration payload cannot be read for staging")?; - if bytes.len() as u64 > freshell_platform::provider_input_exchange::MAX_FILE_BYTES - || bytes.len() as u64 != opened.len() - { - return Err( - "selected configuration payload changed or exceeded its indexed profile".into(), - ); - } - let current = std::fs::canonicalize(path) - .map_err(|_| "selected configuration payload changed during staging")?; - let current_metadata = std::fs::symlink_metadata(path) - .map_err(|_| "selected configuration payload changed during staging")?; - if current != canonical || current_metadata.file_type().is_symlink() { - return Err("selected configuration payload changed during staging".into()); - } - let destination = root.join(freshell_runtime_protocol::indexed_source_stage_path( - source.source_id, - )); - write_private_bytes(&destination, &bytes)?; - } - Ok(()) -} - pub(super) struct DescriptionRoots<'a> { pub root: &'a Path, pub home: &'a Path, @@ -1727,13 +1712,12 @@ pub(super) fn describe( indexed_files: (!sources.is_empty()).then(|| IndexedProviderFiles { volume_name: indexed_provider_volume_name(soul, &capability.grant_id), aliases, - staged_source_paths: Some( - (0..sources.len()) - .map(|source_id| { - freshell_runtime_protocol::indexed_source_stage_path(source_id as u16) - }) - .collect(), - ), + // New launches keep only the original configuration file path in + // ProviderSecretReference. The supervisor streams bytes directly + // into the provider's bounded tmpfs input volume; duplicating the + // payload in this durable generation could persist credentials. + // `Some(...)` remains supported when adopting older descriptors. + staged_source_paths: None, }), }; inputs diff --git a/crates/freshell-supervisor/Cargo.toml b/crates/freshell-supervisor/Cargo.toml index 70f627b71..15547f1a6 100644 --- a/crates/freshell-supervisor/Cargo.toml +++ b/crates/freshell-supervisor/Cargo.toml @@ -29,3 +29,4 @@ libc = "0.2" [dev-dependencies] tempfile = "3" +tracing-subscriber = { version = "0.3", default-features = false, features = ["fmt", "json"] } diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 4946dd762..57a618f69 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -1043,7 +1043,7 @@ mod provider_secret_tests { #[cfg(unix)] #[test] - fn indexed_configuration_reference_uses_its_private_generation_source() { + fn indexed_configuration_reference_uses_original_path_and_preserves_legacy_staged_sources() { const CHILD: &str = "FRESHELL_INDEXED_STAGED_SOURCE_TEST_CHILD"; if std::env::var_os(CHILD).is_none() { let status = std::process::Command::new("/usr/bin/timeout") @@ -1051,7 +1051,7 @@ mod provider_secret_tests { .arg(std::env::current_exe().unwrap()) .args([ "--exact", - "backend::docker::provider_secret_tests::indexed_configuration_reference_uses_its_private_generation_source", + "backend::docker::provider_secret_tests::indexed_configuration_reference_uses_original_path_and_preserves_legacy_staged_sources", "--nocapture", ]) .env(CHILD, "1") @@ -1075,13 +1075,17 @@ mod provider_secret_tests { let stage = owned.path().join("provider-roots/grant-indexed"); let workspace = owned.path().join("workspace"); let indexed_source = stage.join(".freshell/indexed-source-files/source-0"); + let original_source = owned.path().join("host-only/original-prompt.txt"); std::fs::create_dir_all(&capabilities).unwrap(); std::fs::create_dir_all(stage.join(".freshell/indexed-source-files")).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(original_source.parent().unwrap()).unwrap(); let manifest = b"{}"; std::fs::write(stage.join(".freshell/input-manifest.json"), manifest).unwrap(); std::fs::write(&indexed_source, b"staged prompt payload").unwrap(); std::fs::set_permissions(&indexed_source, std::fs::Permissions::from_mode(0o640)).unwrap(); + std::fs::write(&original_source, b"original prompt payload").unwrap(); + let original_source = std::fs::canonicalize(original_source).unwrap(); let capability = McpCapabilityReference { grant_id: "grant-indexed".into(), endpoint: "http://host.docker.internal:3344".into(), @@ -1116,9 +1120,7 @@ mod provider_secret_tests { indexed_files: Some(IndexedProviderFiles { volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), aliases: vec![], - staged_source_paths: Some(vec![ - freshell_runtime_protocol::indexed_source_stage_path(0), - ]), + staged_source_paths: None, }), }), }; @@ -1134,18 +1136,32 @@ mod provider_secret_tests { "providerLaunchContext":context })).unwrap(); terminal.provider_secret_references = vec![ProviderSecretReference { - source_path: owned - .path() - .join("host-only/original-prompt.txt") - .display() - .to_string(), + source_path: original_source.display().to_string(), profile: ProviderSecretProfile::OpencodeConfigurationFile, provider_relative_path: Some(".freshell/references/0".into()), }]; let mounts = terminal_mounts(&terminal).unwrap(); - assert_eq!(mounts.provider_secret_files, vec![indexed_source]); + assert_eq!(mounts.provider_secret_files, vec![original_source]); assert_eq!(mounts.indexed_secret_sources, BTreeMap::from([(0, 0)])); + + let mut legacy = terminal; + legacy + .provider_launch_context + .as_mut() + .unwrap() + .prepared_inputs + .as_mut() + .unwrap() + .indexed_files + .as_mut() + .unwrap() + .staged_source_paths = + Some(vec![freshell_runtime_protocol::indexed_source_stage_path( + 0, + )]); + let legacy_mounts = terminal_mounts(&legacy).unwrap(); + assert_eq!(legacy_mounts.provider_secret_files, vec![indexed_source]); } } diff --git a/crates/freshell-supervisor/src/backend/input_publisher.rs b/crates/freshell-supervisor/src/backend/input_publisher.rs index f7039d73f..5e00a3a1b 100644 --- a/crates/freshell-supervisor/src/backend/input_publisher.rs +++ b/crates/freshell-supervisor/src/backend/input_publisher.rs @@ -10,6 +10,71 @@ fn fail(cause: &'static str) -> BackendError { BackendError::Unavailable(cause.into()) } +fn source_io_failure( + source_id: usize, + path: &Path, + operation: &'static str, + error: std::io::Error, +) -> BackendError { + tracing::error!( + event = "provider.input.source.failed", + stage = "input_publication", + source_id, + operation, + source_path = %path.display(), + error_kind = ?error.kind(), + os_error = error.raw_os_error(), + error = %error, + ); + BackendError::Unavailable(format!( + "provider input source {} failed during {operation}: {error}", + path.display() + )) +} + +fn source_state_failure( + source_id: usize, + path: &Path, + operation: &'static str, + reason: &'static str, +) -> BackendError { + tracing::error!( + event = "provider.input.source.failed", + stage = "input_publication", + source_id, + operation, + source_path = %path.display(), + failure = reason, + ); + BackendError::Unavailable(format!( + "provider input source {} failed during {operation}: {reason}", + path.display() + )) +} + +fn source_limit_failure( + source_id: usize, + path: &Path, + operation: &'static str, + observed: u64, + permitted: u64, +) -> BackendError { + tracing::error!( + event = "provider.input.source.failed", + stage = "input_publication", + source_id, + operation, + source_path = %path.display(), + failure = "payload_exceeds_profile", + observed, + permitted, + ); + BackendError::Unavailable(format!( + "provider input source {} exceeded its profile during {operation}: {observed} bytes, permitted {permitted}", + path.display() + )) +} + pub(super) struct HeldPublisher { soul: String, incarnation: String, @@ -142,34 +207,65 @@ fn environment(value: &Value) -> Option<BTreeMap<&str, &str>> { } async fn open_approved_source( + source_id: usize, path: &Path, remaining: u64, ) -> Result<tokio::io::Take<tokio::fs::File>, BackendError> { - if std::fs::canonicalize(path).map_err(|_| fail("approved input source unavailable"))? != path { - return Err(fail("approved input source selection changed")); + if std::fs::canonicalize(path) + .map_err(|error| source_io_failure(source_id, path, "canonicalize", error))? + != path + { + return Err(source_state_failure( + source_id, + path, + "canonicalize", + "selected path changed", + )); } let limit = MAX_FILE_BYTES.min(remaining); let metadata = tokio::fs::metadata(path) .await - .map_err(|_| fail("approved input source metadata failed"))?; + .map_err(|error| source_io_failure(source_id, path, "metadata", error))?; if !metadata.is_file() { - return Err(fail("approved input source is not regular")); + return Err(source_state_failure( + source_id, + path, + "metadata", + "source is not a regular file", + )); } if metadata.len() > limit { - return Err(fail("approved input source actual bytes exceed profile")); + return Err(source_limit_failure( + source_id, + path, + "preflight", + metadata.len(), + limit, + )); } let file = tokio::fs::File::open(path) .await - .map_err(|_| fail("approved input source read failed"))?; + .map_err(|error| source_io_failure(source_id, path, "open", error))?; let opened = file .metadata() .await - .map_err(|_| fail("approved input source metadata failed"))?; + .map_err(|error| source_io_failure(source_id, path, "opened_metadata", error))?; if !opened.is_file() { - return Err(fail("approved input source is not regular")); + return Err(source_state_failure( + source_id, + path, + "opened_metadata", + "source is not a regular file", + )); } if opened.len() > limit { - return Err(fail("approved input source actual bytes exceed profile")); + return Err(source_limit_failure( + source_id, + path, + "opened_preflight", + opened.len(), + limit, + )); } Ok(file.take(limit + 1)) } @@ -810,7 +906,8 @@ impl DockerEngineBackend { let mut total = 0u64; for (source_id, path) in sources.iter().enumerate() { let mut file = - open_approved_source(path, MAX_TOTAL_BYTES.saturating_sub(total)).await?; + open_approved_source(source_id, path, MAX_TOTAL_BYTES.saturating_sub(total)) + .await?; request( &mut stream, &Request::Open { @@ -825,14 +922,29 @@ impl DockerEngineBackend { let read = file .read(&mut chunk) .await - .map_err(|_| fail("approved input source read failed"))?; + .map_err(|error| source_io_failure(source_id, path, "read", error))?; if read == 0 { break; } count += read as u64; total += read as u64; - if count > MAX_FILE_BYTES || total > MAX_TOTAL_BYTES { - return Err(fail("approved input source actual bytes exceed profile")); + if count > MAX_FILE_BYTES { + return Err(source_limit_failure( + source_id, + path, + "stream", + count, + MAX_FILE_BYTES, + )); + } + if total > MAX_TOTAL_BYTES { + return Err(source_limit_failure( + source_id, + path, + "stream_total", + total, + MAX_TOTAL_BYTES, + )); } request( &mut stream, @@ -1163,6 +1275,58 @@ impl DockerEngineBackend { mod tests { use super::*; + #[derive(Clone)] + struct CapturedLogWriter(std::sync::Arc<std::sync::Mutex<Vec<u8>>>); + + impl std::io::Write for CapturedLogWriter { + fn write(&mut self, bytes: &[u8]) -> std::io::Result<usize> { + self.0.lock().unwrap().extend_from_slice(bytes); + Ok(bytes.len()) + } + + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + + impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for CapturedLogWriter { + type Writer = Self; + + fn make_writer(&'a self) -> Self::Writer { + self.clone() + } + } + + #[tokio::test(flavor = "current_thread")] + async fn provider_input_source_failure_logs_path_and_filesystem_cause_without_contents() { + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("selected-secret-file"); + std::fs::write(&source, "NEVER_LOG_THIS_SECRET_CONTENT").unwrap(); + std::fs::remove_file(&source).unwrap(); + let captured = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let subscriber = tracing_subscriber::fmt() + .json() + .without_time() + .with_writer(CapturedLogWriter(captured.clone())) + .finish(); + let _guard = tracing::subscriber::set_default(subscriber); + + let error = open_approved_source(0, &source, MAX_TOTAL_BYTES) + .await + .unwrap_err(); + + assert!(error.to_string().contains(&source.display().to_string())); + assert!(error.to_string().contains("No such file or directory")); + let captured = captured.lock().unwrap(); + let event: serde_json::Value = serde_json::from_slice(&captured).unwrap(); + assert_eq!(event["fields"]["event"], "provider.input.source.failed"); + assert_eq!(event["fields"]["source_id"], 0); + assert_eq!(event["fields"]["source_path"], source.display().to_string()); + assert_eq!(event["fields"]["operation"], "canonicalize"); + assert_eq!(event["fields"]["error_kind"], "NotFound"); + assert!(!String::from_utf8_lossy(&captured).contains("NEVER_LOG_THIS_SECRET_CONTENT")); + } + #[test] fn indexed_payload_publication_uses_the_prepared_source_paths_in_source_id_order() { let mounts = docker::TerminalMounts { @@ -1257,7 +1421,7 @@ mod tests { ) .unwrap(); drop(file); - let result = open_approved_source(&source, remaining).await; + let result = open_approved_source(0, &source, remaining).await; let rejected = match result { Err(error) => { assert!(error.to_string().contains("exceed")); @@ -1279,7 +1443,9 @@ mod tests { let file = std::fs::File::create(&source).unwrap(); file.set_len(MAX_FILE_BYTES).unwrap(); drop(file); - let mut reader = open_approved_source(&source, MAX_FILE_BYTES).await.unwrap(); + let mut reader = open_approved_source(0, &source, MAX_FILE_BYTES) + .await + .unwrap(); let mut bytes = [0; MAX_CHUNK_BYTES]; let mut observed = 0u64; loop { @@ -1293,12 +1459,12 @@ mod tests { assert_eq!(observed, MAX_FILE_BYTES); std::fs::write(&source, []).unwrap(); - let mut reader = open_approved_source(&source, 0).await.unwrap(); + let mut reader = open_approved_source(0, &source, 0).await.unwrap(); assert_eq!(reader.read(&mut [0]).await.unwrap(), 0); // Growth after acceptance remains bounded by the same one-byte // lookahead used by the publisher's streaming count checks. - let mut reader = open_approved_source(&source, 1).await.unwrap(); + let mut reader = open_approved_source(0, &source, 1).await.unwrap(); std::fs::OpenOptions::new() .write(true) .open(&source) From 70e7022993086b4606651d992b24f9c549e89006 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:46:03 -0700 Subject: [PATCH 355/493] test: authorize selected provider input roots --- .../managed-provider-environment-target.ts | 3 ++- scripts/testing/runtime-sandbox.ts | 16 +++++++++++- scripts/testing/runtime-test-broker.ts | 14 +++++++++++ .../tooling/testing/runtime-sandbox.test.ts | 5 +++- .../testing/runtime-test-broker.test.ts | 25 ++++++++++++++++--- 5 files changed, 57 insertions(+), 6 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 442c9d6fe..b9c9c9e04 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1184,7 +1184,8 @@ export async function runManagedEnvironmentTarget() { FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE: '', FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE: '' } rig = new ManagedRuntimeBrowserRig(repoRoot, 2, { ...environment, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY: '1' }, environment, 'release', { enabledProviders: ['opencode'] }, home, - { workspaceRoots: [workspace], bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) + { workspaceRoots: [workspace], providerInputRoots: [managedEnvironmentFixtureLayout(root).resources], + bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) const selected = await configure(root, nonce, rig.runtime.candidateSha, rig.runtime.runId) await rig.start() webIdentity = await processIdentity(rig.web.processEvidence()) diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index 9f77506ff..4670b34e1 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -98,6 +98,7 @@ export function phase2BootstrapFiles( export type RuntimeFixtureInputs = { workspaceRoots: string[] + providerInputRoots?: string[] bootstrapFiles: string[] } @@ -164,7 +165,9 @@ export class RuntimeHarness { fixtureInputs?: RuntimeFixtureInputs) { this.repoRoot = fs.realpathSync(repoRoot) this.fixtureInputs = fixtureInputs && { - workspaceRoots: [...fixtureInputs.workspaceRoots], bootstrapFiles: [...fixtureInputs.bootstrapFiles], + workspaceRoots: [...fixtureInputs.workspaceRoots], + providerInputRoots: fixtureInputs.providerInputRoots ? [...fixtureInputs.providerInputRoots] : undefined, + bootstrapFiles: [...fixtureInputs.bootstrapFiles], } this.phase = phase this.runId = runId @@ -240,6 +243,7 @@ export class RuntimeHarness { allowedImageRefs: new Set([this.imageRef]), allowTerminalWorkloads: this.phase >= 2, allowedWorkspaceRoots: new Set(this.phase >= 2 ? this.phase2WorkspaceRoots() : []), + allowedProviderInputRoots: new Set(this.phase >= 2 ? this.phase2ProviderInputRoots() : []), allowedProviderUserRoots: new Set(this.phase >= 2 ? this.phase2ProviderUserRoots() : []), allowedBootstrapFiles: new Set(this.phase >= 2 ? this.phase2BootstrapFiles() : []), testRunId: this.runId, @@ -477,6 +481,10 @@ export class RuntimeHarness { if (root === this.testRoot || root === this.buildDir) continue args.push('-v', `${root}:${root}:ro`) } + for (const root of this.phase2ProviderInputRoots()) { + if (root === this.testRoot || root === this.buildDir || this.phase2WorkspaceRoots().includes(root)) continue + args.push('-v', `${root}:${root}:ro`) + } } if (this.phase >= 2) { for (const credentialFile of this.phase2BootstrapFiles()) { @@ -1112,6 +1120,12 @@ export class RuntimeHarness { return [...roots] } + private phase2ProviderInputRoots(): string[] { + if (!this.fixtureInputs?.providerInputRoots) return [] + return this.fixtureInputs.providerInputRoots.map(selected => + validateOwnedFixturePath(selected, 'directory', 'provider input fixture root')) + } + private phase2ProviderUserRoots(): string[] { const home = this.providerFixtureHome ?? os.homedir() const validateFixtureDirectory = (selected: string) => validateOwnedFixturePath(selected, 'directory', 'provider fixture root') diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index 259e58ffb..fa288fd86 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -130,6 +130,7 @@ export type BrokerEvent = { targetPath?: string preparationStageRoot?: string selectedWorkspaceRoots?: string[] + selectedProviderInputRoots?: string[] destructive: boolean unsafeAttempt: boolean } @@ -142,6 +143,7 @@ export type RestrictedDockerBrokerPolicy = { allowedImageRefs: Set<string> allowTerminalWorkloads?: boolean allowedWorkspaceRoots?: Set<string> + allowedProviderInputRoots?: Set<string> allowedProviderUserRoots?: Set<string> allowedBootstrapFiles?: Set<string> allowedHostGatewayAddresses?: Set<string> @@ -245,6 +247,7 @@ export class RestrictedDockerBroker { && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) && (value === input.workspacePath || isStrictDescendant(value, input.workspacePath) || mappedProviderDestination(value) + || this.isAllowedProviderInputPath(value) || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) const unit = (value: string, prefix: string, file = false) => typeof value === 'string' && new RegExp(`^${prefix}/[a-zA-Z0-9_-]{1,128}${file ? '/source' : ''}$`).test(value) @@ -317,6 +320,7 @@ export class RestrictedDockerBroker { const destination = (value: string) => path.isAbsolute(value) && path.normalize(value) === value && !/[\x00-\x1f\x7f]/.test(value) && (this.isAllowedWorkspacePath(value) + || this.isAllowedProviderInputPath(value) || mappedProviderDestination(value) || [...this.policy.allowedProviderUserRoots ?? []].some((root) => value === root || isStrictDescendant(value, root))) const targets = new Set<string>() @@ -1394,6 +1398,8 @@ export class RestrictedDockerBroker { preparationStageRoot: diagnosticPath(error.preparationStageRoot), selectedWorkspaceRoots: [...this.policy.allowedWorkspaceRoots ?? []] .filter(root => diagnosticPath(root) !== undefined), + selectedProviderInputRoots: [...this.policy.allowedProviderInputRoots ?? []] + .filter(root => diagnosticPath(root) !== undefined), } : {} this.record({ method: 'POST', url, decision: 'block', reason: `fixture publication unavailable at ${phase}: ${category}`, ...target, destructive: false, unsafeAttempt: false }) @@ -1501,6 +1507,14 @@ export class RestrictedDockerBroker { return false } + private isAllowedProviderInputPath(candidate: string): boolean { + const roots = this.policy.allowedProviderInputRoots ?? new Set<string>() + for (const root of roots) { + if (candidate === root || isStrictDescendant(candidate, root)) return true + } + return false + } + private isAllowedExchangeSource(source: unknown, prefix: 'freshell-inputs' | 'freshell-auth'): source is string { if (typeof source !== 'string' || path.normalize(source) !== source || !new RegExp(`^${prefix}-[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$`).test(path.basename(source))) return false diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index bb8cf0434..d08214c3e 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -579,6 +579,7 @@ describe('closed managed environment fixture inputs', () => { const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-supervisor-input-')) const repository = path.join(directory, 'repository'), home = path.join(directory, 'home') const xdg = path.join(home, 'fixture-xdg'), workspace = path.join(directory, 'workspace') + const providerInputs = path.join(directory, 'selected-provider-inputs') const grant = path.join(directory, 'grant.env'), socket = path.join(directory, 'daemon.sock') const previousDockerHost = process.env.DOCKER_HOST const daemon = http.createServer((_request, response) => response.writeHead(500).end()) @@ -589,6 +590,7 @@ describe('closed managed environment fixture inputs', () => { await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true }) await fs.chmod(home, 0o700) await fs.mkdir(workspace) + await fs.mkdir(providerInputs) await fs.writeFile(grant, 'synthetic-closed-grant', { mode: 0o600 }) const host = path.join(directory, 'host') await fs.writeFile(host, 'owned fixture host artifact') @@ -596,7 +598,7 @@ describe('closed managed environment fixture inputs', () => { process.env.DOCKER_HOST = `unix://${socket}` const rig = new ManagedRuntimeBrowserRig(repository, 2, { HOME: home, XDG_CONFIG_HOME: xdg }, { HOME: home, XDG_CONFIG_HOME: xdg }, 'test', { enabledProviders: ['opencode'] }, home, - { workspaceRoots: [workspace], bootstrapFiles: [grant], defaultCwd: workspace }) + { workspaceRoots: [workspace], providerInputRoots: [providerInputs], bootstrapFiles: [grant], defaultCwd: workspace }) harness = rig.runtime useOwnedHarnessArtifacts(harness, host) await harness.prepare() @@ -606,6 +608,7 @@ describe('closed managed environment fixture inputs', () => { expect(fakeDocker.supervisorCommands).toHaveLength(1) const dispatched = fakeDocker.supervisorCommands[0] expect(dispatched).toContain(`${workspace}:${workspace}:ro`) + expect(dispatched).toContain(`${providerInputs}:${providerInputs}:ro`) expect(dispatched).toContain(`${grant}:${grant}:ro`) expect(dispatched).toContain(`HOME=${home}`) expect(dispatched).toContain(`XDG_CONFIG_HOME=${xdg}`) diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 10a7d9af7..f80638c1a 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -1069,14 +1069,31 @@ describe('fixed indexed publisher broker',()=>{ expect(forwarded).toHaveLength(before) const diagnostic = instance.eventsSnapshot().find(event => event.reason === 'fixture publication unavailable at prepared_authorization: target_root') expect(diagnostic).toMatchObject({ targetPath: destination, preparationStageRoot: input.stageRoot, - selectedWorkspaceRoots: ['/workspace'] }) + selectedWorkspaceRoots: ['/workspace'], selectedProviderInputRoots: ['/selected-provider-inputs'] }) const events = JSON.stringify(instance.eventsSnapshot()) expect(events).not.toContain('PRIVATE_SCOPE_CONTENT_SENTINEL') expect(events).not.toContain('PRIVATE_CONFIG_CONTENT_SENTINEL') expect(instance.receipts()).toEqual([]) - }, undefined, true) + }, undefined, true, new Set(['/selected-provider-inputs'])) }, ) + it('authorizes selected external provider input targets without authorizing them as workspaces', async () => { + await withHttpBroker(async ({ request, broker: instance }) => { + const selectedRoot = '/selected-provider-inputs' + const input = { ...publisherAuthorization(), resourceMounts: [{ + providerRelativePath: '.freshell/provider-roots/fixture-mcp', + effectiveDestination: path.join(selectedRoot, 'fixture-mcp'), + kind: 'directory' as const, + }] } + const selectedAsWorkspace = createBody([ + ...ordinaryBinds.slice(0, 3), + `${selectedRoot}:${selectedRoot}:rw`, + ]) + const result = await request('POST', '/v1.47/containers/create?name=external-input-as-workspace', JSON.parse(selectedAsWorkspace.toString())) + expect(result.status).toBe(403) + expect(() => instance.authorizePreparedInputs(input)).not.toThrow() + }, undefined, false, new Set(['/selected-provider-inputs'])) + }) it.each(['exited', 'running', 'foreign-owner', 'changed-image', 'changed-host', 'foreign-capability'] as const)( 'qualifies a replacement publication against its retained owner (%s)', async (state) => { await withHttpBroker(async ({ directory, request, forwarded, broker: instance, setRuntimeRunning }) => { @@ -1546,7 +1563,8 @@ describe('fixed indexed publisher broker',()=>{ }); }); -async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise<void>,gateways?:Set<string>, physicalStage = false) { +async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise<void>,gateways?:Set<string>, physicalStage = false, + allowedProviderInputRoots = new Set<string>()) { const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-broker-history-')) const socket = path.join(dir, 'docker.sock') const proxy = path.join(dir, 'broker.sock') @@ -1642,6 +1660,7 @@ async function withHttpBroker(run: (fixture: BrokerHttpFixture) => Promise<void> allowedImageRefs: new Set(['sha256:fixture-image', 'sha256:other-approved-image']), allowTerminalWorkloads: true, allowedWorkspaceRoots: new Set(['/workspace']), + allowedProviderInputRoots, testRunId: 'run-one', logPath: path.join(dir, 'broker.jsonl'), }) From 1a1b90aa0de1883b2ae50a3f8a87fea7d0166017 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:23:46 -0700 Subject: [PATCH 356/493] fix(checkpoints): keep account home out of snapshot scope --- crates/freshell-server/src/checkpoints.rs | 157 ++++++------------ crates/freshell-server/src/files.rs | 70 +++++++- ...code-checkpoint-workspace-external.spec.ts | 69 ++++---- .../components/panes/DirectoryPicker.test.tsx | 9 +- 4 files changed, 145 insertions(+), 160 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 540d8c086..45f8a8f85 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -3007,6 +3007,15 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { homes } +fn is_account_home_checkpoint_scope(path: &Path, server_home: &Path) -> bool { + if std::fs::canonicalize(server_home).is_ok_and(|home| home == path) { + return true; + } + crate::session_directory::provider_home() + .and_then(|home| std::fs::canonicalize(home).ok()) + .is_some_and(|home| home == path) +} + fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { homes.iter().any(|home| home == path) } @@ -3345,6 +3354,12 @@ async fn resolve_checkpoint_workspace_with_account_home( candidate: candidate.clone(), reason, })?; + if is_account_home_checkpoint_scope(&selected, server_home) { + return Err(CheckpointWorkspaceUnavailable { + candidate, + reason: "account home cannot be used as a code checkpoint workspace".to_string(), + }); + } if provenance == "legacy-checkpoint" { return Ok(ResolvedCheckpointWorkspace { path: selected, @@ -7007,23 +7022,22 @@ mod tests { } #[tokio::test] - async fn review_r6_explicit_picker_account_home_is_a_valid_scope() { + async fn explicit_picker_account_home_is_not_a_checkpoint_scope() { let home = tempfile::tempdir().unwrap(); let intent = json!({ "path": home.path(), "provenance": "directory-picker", }); - let resolved = resolve_checkpoint_workspace(home.path(), home.path(), Some(&intent)) + let error = resolve_checkpoint_workspace(home.path(), home.path(), Some(&intent)) .await - .expect("explicit directory-picker selection defines the account-home scope"); + .expect_err("an account-home picker choice only defines launch cwd"); - assert_eq!(resolved.path, home.path().canonicalize().unwrap()); - assert_eq!(resolved.provenance, "directory-picker"); + assert!(error.reason.contains("account home"), "{}", error.reason); } #[tokio::test] - async fn explicit_picker_home_keeps_exact_scope_when_git_root_is_its_parent() { + async fn explicit_picker_home_is_rejected_even_when_git_root_is_its_parent() { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); tokio::fs::create_dir(&home).await.unwrap(); @@ -7033,11 +7047,11 @@ mod tests { "provenance": "directory-picker", }); - let resolved = resolve_checkpoint_workspace(&home, &home, Some(&intent)) + let error = resolve_checkpoint_workspace(&home, &home, Some(&intent)) .await - .expect("explicit account-home selection is an eligible scope"); + .expect_err("a Git parent must not turn account HOME into a snapshot scope"); - assert_eq!(resolved.path, home.canonicalize().unwrap()); + assert!(error.reason.contains("account home"), "{}", error.reason); } #[tokio::test] @@ -7287,7 +7301,7 @@ mod tests { } #[tokio::test] - async fn explicit_picker_home_is_snapshotted_without_the_internal_store() { + async fn explicit_picker_home_is_rejected_without_creating_checkpoint_storage() { let home_dir = tempfile::tempdir().unwrap(); tokio::fs::write(home_dir.path().join("private.txt"), "private") .await @@ -7307,67 +7321,20 @@ mod tests { .await; let status = response.status(); - let entry = body_json(response).await; - assert_eq!(status, StatusCode::OK, "{entry}"); - assert_eq!(entry["workspace"]["path"], json!(home_dir.path())); - assert_eq!(entry["workspace"]["provenance"], json!("directory-picker")); - let checkpoint_id = entry["id"].as_str().unwrap().to_string(); - - tokio::fs::write(home_dir.path().join("private.txt"), "changed after capture") - .await - .unwrap(); - restore_checkpoint(home_dir.path(), home_dir.path(), &checkpoint_id) - .await - .expect("the selected HOME checkpoint restores"); - assert_eq!( - tokio::fs::read_to_string(home_dir.path().join("private.txt")) + let error = body_json(response).await; + assert_eq!(status, StatusCode::UNPROCESSABLE_ENTITY, "{error}"); + assert_eq!(error["code"], json!("workspace_unavailable")); + assert!(error["error"].as_str().unwrap().contains("account home")); + assert!( + !tokio::fs::try_exists(home_dir.path().join(".freshell/checkpoints")) .await - .unwrap(), - "private" + .unwrap() ); - - let list_params = HashMap::from([ - ( - "cwd".to_string(), - home_dir.path().to_string_lossy().into_owned(), - ), - ( - "workspacePath".to_string(), - home_dir.path().to_string_lossy().into_owned(), - ), - ( - "workspaceProvenance".to_string(), - "directory-picker".to_string(), - ), - ]); - let listed = get_checkpoints( - State(state(home_dir.path())), - headers_with_token("tok"), - Query(list_params), - ) - .await; - assert_eq!(listed.status(), StatusCode::OK); - let listed = body_json(listed).await; - assert!(listed["checkpoints"] - .as_array() - .unwrap() - .iter() - .any(|checkpoint| { checkpoint["id"].as_str() == Some(checkpoint_id.as_str()) })); - - let git_dir = checkpoint_git_dir(home_dir.path(), home_dir.path()); - let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) - .await - .expect("published picker-home checkpoint index is readable"); - assert!(index.iter().any(|entry| entry.path == b"private.txt")); - assert!(!index.iter().any(|entry| { - entry.path.starts_with(b".freshell/checkpoints/") - || entry.path == b".freshell/checkpoints" - })); } #[cfg(unix)] #[tokio::test] - async fn explicit_picker_home_symlink_excludes_internal_store_and_rejects_store_scope() { + async fn explicit_picker_home_symlink_is_rejected_without_creating_checkpoint_storage() { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("actual-home"); let alias = owned.path().join("home-alias"); @@ -7391,18 +7358,14 @@ mod tests { .await; let status = response.status(); - let entry = body_json(response).await; - assert_eq!(status, StatusCode::OK, "{entry}"); - let git_dir = checkpoint_git_dir(&alias, &home); - let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) + let error = body_json(response).await; + assert_eq!(status, StatusCode::UNPROCESSABLE_ENTITY, "{error}"); + assert_eq!(error["code"], json!("workspace_unavailable")); + assert!(!tokio::fs::try_exists(home.join(".freshell/checkpoints")) .await - .expect("published aliased-home checkpoint index is readable"); - assert!(index.iter().any(|entry| entry.path == b"private.txt")); - assert!(!index.iter().any(|entry| { - entry.path.starts_with(b".freshell/checkpoints/") - || entry.path == b".freshell/checkpoints" - })); + .unwrap()); + let git_dir = checkpoint_git_dir(&alias, &home); let nested_workspace = home.join(".freshell/checkpoints/nested-workspace"); tokio::fs::create_dir_all(&nested_workspace).await.unwrap(); let canonical_nested = tokio::fs::canonicalize(&nested_workspace).await.unwrap(); @@ -7413,7 +7376,7 @@ mod tests { #[cfg(unix)] #[tokio::test] - async fn explicit_picker_home_excludes_checkpoint_store_through_in_home_freshell_symlink() { + async fn checkpoint_storage_exclusion_follows_in_home_freshell_symlink() { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let real_freshell_dir = home.join("runtime-state"); @@ -7421,40 +7384,18 @@ mod tests { tokio::fs::write(real_freshell_dir.join("unrelated-state.txt"), "keep") .await .unwrap(); - tokio::fs::write(home.join("private.txt"), "private") - .await - .unwrap(); std::os::unix::fs::symlink(&real_freshell_dir, home.join(".freshell")).unwrap(); - let workspace = home.canonicalize().unwrap(); - - let response = post_checkpoint( - State(state(&home)), - headers_with_token("tok"), - Json(json!({ - "cwd": workspace, - "workspace": { - "path": workspace, - "provenance": "directory-picker", - }, - })), - ) - .await; - let status = response.status(); - let entry = body_json(response).await; - assert_eq!(status, StatusCode::OK, "{entry}"); + let project = home.join("project"); + tokio::fs::create_dir(&project).await.unwrap(); + let git_dir = checkpoint_git_dir(&home, &project); + tokio::fs::create_dir_all(&git_dir).await.unwrap(); + let stored_workspace = git_dir.join("nested-workspace"); + tokio::fs::create_dir_all(&stored_workspace).await.unwrap(); + let canonical_nested = tokio::fs::canonicalize(&stored_workspace).await.unwrap(); - let git_dir = checkpoint_git_dir(&home, &home); - let index = checkpoint_index_entries(&git_dir, &git_dir.join("index")) - .await - .expect("published picker-home checkpoint index is readable"); - assert!(index.iter().any(|entry| entry.path == b"private.txt")); - assert!(index - .iter() - .any(|entry| entry.path == b"runtime-state/unrelated-state.txt")); - assert!(!index.iter().any(|entry| { - entry.path.starts_with(b"runtime-state/checkpoints/") - || entry.path == b"runtime-state/checkpoints" - })); + let error = checkpoint_storage_exclusion(&git_dir, &canonical_nested) + .expect_err("a workspace below the symlinked checkpoint store must be rejected"); + assert!(error.contains("overlaps Freshell checkpoint storage")); } #[tokio::test] diff --git a/crates/freshell-server/src/files.rs b/crates/freshell-server/src/files.rs index 8141d3086..d695e2bb5 100644 --- a/crates/freshell-server/src/files.rs +++ b/crates/freshell-server/src/files.rs @@ -200,10 +200,13 @@ async fn validate_dir( .unwrap_or(false) }) .unwrap_or(false); - // This flag answers whether an explicit directory-picker selection can - // define a snapshot scope. Automatic capture still rejects an unselected - // account home in the checkpoint workspace resolver. - let checkpoint_workspace_eligible = is_dir; + // Account homes select a launch cwd only. Canonicalize both sides so a + // symlink to HOME cannot turn it into an explicit snapshot scope. + let checkpoint_workspace_eligible = is_dir + && resolved + .fs_path + .as_deref() + .is_some_and(|path| !is_checkpoint_home(Path::new(path))); Json(json!({ "valid": is_dir, @@ -213,6 +216,24 @@ async fn validate_dir( .into_response() } +fn is_checkpoint_home(path: &Path) -> bool { + let Some(selected) = std::fs::canonicalize(path).ok() else { + return false; + }; + + let mut home_candidates = crate::session_directory::provider_home() + .into_iter() + .collect::<Vec<_>>(); + if let Some(configured_home) = std::env::var_os("FRESHELL_HOME") { + home_candidates.push(PathBuf::from(configured_home)); + } + + home_candidates + .iter() + .filter_map(|home| std::fs::canonicalize(home).ok()) + .any(|home| home == selected) +} + /// `GET /api/files/read?path=<p>` \u2192 `{ content, size, modifiedAt }` (`files-router.ts:85`). /// /// The retained `EditorPane` opens a file with this (`EditorPane.tsx:368`). A @@ -1419,19 +1440,54 @@ mod tests { #[allow(clippy::await_holding_lock)] #[tokio::test] - async fn review_r6_validate_dir_allows_explicit_account_home_scope() { + async fn validate_dir_keeps_account_home_as_launch_directory_only() { let _guard = env_lock(); - let home = std::env::var("HOME").expect("test environment has HOME"); + let home = tempfile::tempdir().unwrap(); + let home_path = home.path().to_string_lossy().into_owned(); + let project = home.path().join("project"); + std::fs::create_dir(&project).unwrap(); + let _env = EnvGuard::set(&[ + ("HOME", Some(home_path.as_str())), + ("FRESHELL_HOME", None), + ("USERPROFILE", None), + ]); let response = validate_dir( State(test_state()), auth_headers(), - Json(json!({ "path": home })), + Json(json!({ "path": home_path })), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let value = body_json(response).await; + assert_eq!(value["valid"], true); + assert_eq!(value["checkpointWorkspaceEligible"], false); + + let response = validate_dir( + State(test_state()), + auth_headers(), + Json(json!({ "path": project })), ) .await; assert_eq!(response.status(), StatusCode::OK); let value = body_json(response).await; assert_eq!(value["valid"], true); assert_eq!(value["checkpointWorkspaceEligible"], true); + + #[cfg(unix)] + { + let home_alias = home.path().join("home-alias"); + std::os::unix::fs::symlink(home.path(), &home_alias).unwrap(); + let response = validate_dir( + State(test_state()), + auth_headers(), + Json(json!({ "path": home_alias })), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let value = body_json(response).await; + assert_eq!(value["valid"], true); + assert_eq!(value["checkpointWorkspaceEligible"], false); + } } /// Node's `normalizeUserPath` runs `path.posix.resolve` on POSIX inputs diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 374801859..d0cf4974f 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -543,58 +543,49 @@ test('History reopening preserves a non-Git picker workspace for later rewind', await verifyPickerWorkspaceReopensForRewind(page, fixture.nonGitWorkspacePath, 'one.txt', 'non-Git') }) -test('History reopening preserves an explicitly selected HOME workspace for later rewind', async ({ page }) => { - test.setTimeout(360_000) - await verifyPickerWorkspaceReopensForRewind(page, fixture.syntheticHomePath, 'checkpoint-home-file.txt', 'HOME') -}) - -test('picker-confirmed HOME captures and restores a harmless home file', async ({ page }) => { +test('picker-confirmed HOME sets launch cwd without creating a checkpoint scope', async ({ page }) => { test.setTimeout(240_000) await verifyTarget(page) const homeFile = `${fixture.syntheticHomePath}/checkpoint-home-file.txt` await targetWrite(page, homeFile, 'home file before checkpoint\n') const { paneRoot, content } = await launchCodex(page, fixture.syntheticHomePath) expect(content.initialCwd).toBe(fixture.syntheticHomePath) - expect(content.checkpointWorkspace).toEqual({ - path: fixture.syntheticHomePath, - provenance: 'directory-picker', - }) + expect(content.checkpointWorkspace).toBeUndefined() const marker = `checkpoint home ${fixture.expectedNonce}` - const turn = await sendThroughComposer(page, paneRoot, marker, - { path: fixture.syntheticHomePath, provenance: 'directory-picker' }) - expect(turn.outcome).toMatchObject({ status: 'available', workspace: { - path: fixture.syntheticHomePath, provenance: 'directory-picker', - } }) - expect(turn.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) - await expect(turn.row.getByRole('status', { name: 'Code rewind available for this message' })) - .toBeVisible() + const turn = await sendThroughComposer(page, paneRoot, marker, undefined) + expect(turn.outcome).toMatchObject({ status: 'unavailable', code: 'workspace_unavailable' }) + expect(turn.outcome.reason).toContain('account home') + await expect(turn.row.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toContainText('account home') + await turn.row.hover() + await expect(turn.row.getByRole('button', { name: 'Rewind unavailable for this message' })) + .toBeDisabled() await assertNativeUserIdentity(page, turn.outcome, turn.accepted.submittedTurnId, marker) - const query = new URLSearchParams({ - cwd: fixture.syntheticHomePath, - workspacePath: fixture.syntheticHomePath, - workspaceProvenance: 'directory-picker', - }) - const listUrl = `${paired!.targetUrl}/api/fresh-agent/checkpoints?${query}` - const capturedList = await page.request.get(listUrl, { headers, timeout: 30_000 }) - expect(capturedList.ok()).toBe(true) - expect(await capturedList.json()).toMatchObject({ - checkpoints: [expect.objectContaining({ id: turn.outcome.checkpointId })], - }) - await targetWrite(page, homeFile, 'home file changed after send\n') - await turn.row.hover() - const rewindNotice = expect(paneRoot.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) - .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) - const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) - await turn.row.getByRole('button', { name: 'Rewind code to here' }).click() - await Promise.all([dialogAccepted, rewindNotice]) - await expect.poll(() => targetRead(page, homeFile)).toBe('home file before checkpoint\n') + await flushLayoutAndReload(page) + const reopened = await currentFreshAgent(page) + expect(reopened.content.initialCwd).toBe(fixture.syntheticHomePath) + expect(reopened.content.checkpointWorkspace).toBeUndefined() + expect(reopened.content.checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ + requestId: turn.sent.requestId, + status: 'unavailable', + code: 'workspace_unavailable', + }), + ])) + const restoredRow = page.locator('[data-context="fresh-agent"]').last() + .locator('article[data-turn-role="user"]', { hasText: marker }) + await expect(restoredRow.getByRole('note', { name: 'Code rewind unavailable for this message' })) + .toContainText('account home') + const query = new URLSearchParams({ cwd: fixture.syntheticHomePath }) + const listUrl = `${paired!.targetUrl}/api/fresh-agent/checkpoints?${query}` const list = await page.request.get(listUrl, { headers, timeout: 30_000 }) expect(list.ok()).toBe(true) - expect(await list.json()).toMatchObject({ checkpoints: [expect.objectContaining({ id: turn.outcome.checkpointId })] }) + expect(await list.json()).toMatchObject({ checkpoints: [] }) const workspaceKey = createHash('sha1').update(resolve(fixture.syntheticHomePath)).digest('hex').slice(0, 16) expect(await targetRead(page, - `${fixture.syntheticHomePath}/.freshell/checkpoints/${workspaceKey}.git/HEAD`)).toBeTruthy() + `${fixture.syntheticHomePath}/.freshell/checkpoints/${workspaceKey}.git/HEAD`)).toBeUndefined() + expect(await targetRead(page, homeFile)).toBe('home file changed after send\n') }) diff --git a/test/unit/client/components/panes/DirectoryPicker.test.tsx b/test/unit/client/components/panes/DirectoryPicker.test.tsx index 247a26c00..b38fd7dad 100644 --- a/test/unit/client/components/panes/DirectoryPicker.test.tsx +++ b/test/unit/client/components/panes/DirectoryPicker.test.tsx @@ -184,11 +184,11 @@ describe('DirectoryPicker', () => { }) }) - it('passes explicit account-home selection through as checkpoint scope', async () => { + it('confirms account-home selection as launch cwd without checkpoint scope', async () => { mockApiPost.mockResolvedValueOnce({ valid: true, resolvedPath: '/home/user', - checkpointWorkspaceEligible: true, + checkpointWorkspaceEligible: false, }) const { onConfirm } = renderDirectoryPicker({ defaultCwd: '' }) const input = screen.getByLabelText('Starting directory for Claude') @@ -196,10 +196,7 @@ describe('DirectoryPicker', () => { fireEvent.change(input, { target: { value: '/home/user' } }) fireEvent.keyDown(input, { key: 'Enter' }) - await waitFor(() => expect(onConfirm).toHaveBeenCalledWith('/home/user', { - path: '/home/user', - provenance: 'directory-picker', - })) + await waitFor(() => expect(onConfirm).toHaveBeenCalledWith('/home/user')) expect(onConfirm).toHaveBeenCalledTimes(1) }) From fbe80e853a095b7274088eb8df39289508d45342 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:32:22 -0700 Subject: [PATCH 357/493] fix(checkpoints): reject owner account homes as scopes --- crates/freshell-server/src/checkpoints.rs | 86 +++++++++++++++++------ crates/freshell-server/src/files.rs | 45 ++++++++---- 2 files changed, 95 insertions(+), 36 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 45f8a8f85..5321274f6 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -2999,6 +2999,7 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { "CODEX_HOME", "OPENCODE_CONFIG_DIR", "FRESHELL_AMPLIFIER_HOME", + "FRESHELL_HOME", ] { if let Some(value) = std::env::var_os(variable) { add_existing(Path::new(&value)); @@ -3008,12 +3009,22 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { } fn is_account_home_checkpoint_scope(path: &Path, server_home: &Path) -> bool { - if std::fs::canonicalize(server_home).is_ok_and(|home| home == path) { - return true; - } - crate::session_directory::provider_home() - .and_then(|home| std::fs::canonicalize(home).ok()) - .is_some_and(|home| home == path) + is_account_home_checkpoint_path_with_server_home(path, Some(server_home)) +} + +pub(crate) fn is_account_home_checkpoint_path(path: &Path) -> bool { + is_account_home_checkpoint_path_with_server_home(path, None) +} + +fn is_account_home_checkpoint_path_with_server_home( + path: &Path, + server_home: Option<&Path>, +) -> bool { + let Ok(path) = std::fs::canonicalize(path) else { + return false; + }; + let homes = recognized_account_homes(server_home); + is_account_home_for_owner(&path, &homes, &account_home_for_directory).unwrap_or(false) } fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { @@ -3360,16 +3371,17 @@ async fn resolve_checkpoint_workspace_with_account_home( reason: "account home cannot be used as a code checkpoint workspace".to_string(), }); } - if provenance == "legacy-checkpoint" { - return Ok(ResolvedCheckpointWorkspace { - path: selected, - provenance, - }); - } let selected_home = match is_account_home_for_owner(&selected, &homes, account_home_for_directory) { Ok(is_home) => Some(is_home), - Err(_reason) if provenance == "directory-picker" => None, + Err(_reason) + if matches!( + provenance.as_str(), + "directory-picker" | "legacy-checkpoint" + ) => + { + None + } Err(reason) => { return Err(CheckpointWorkspaceUnavailable { candidate, @@ -3377,17 +3389,16 @@ async fn resolve_checkpoint_workspace_with_account_home( }); } }; - if provenance == "directory-picker" && selected_home == Some(true) { - return Ok(ResolvedCheckpointWorkspace { - path: selected, - provenance, - }); - } if selected_home == Some(true) { return Err(CheckpointWorkspaceUnavailable { candidate, - reason: "account home cannot be used as an inferred code checkpoint workspace" - .to_string(), + reason: "account home cannot be used as a code checkpoint workspace".to_string(), + }); + } + if provenance == "legacy-checkpoint" { + return Ok(ResolvedCheckpointWorkspace { + path: selected, + provenance, }); } let path = match git_checkout_root(&selected).await { @@ -7036,6 +7047,39 @@ mod tests { assert!(error.reason.contains("account home"), "{}", error.reason); } + #[tokio::test] + async fn picker_owner_account_home_is_rejected_when_configured_home_differs() { + let owned = tempfile::tempdir().unwrap(); + let configured_home = owned.path().join("managed-home"); + let account_home = owned.path().join("account-home"); + tokio::fs::create_dir_all(&configured_home).await.unwrap(); + tokio::fs::create_dir_all(&account_home).await.unwrap(); + let account_home = account_home.canonicalize().unwrap(); + let owner_home = account_home.clone(); + let account_home_for_directory = move |path: &Path| { + if path == owner_home { + Ok(owner_home.clone()) + } else { + Err("synthetic owner has no resolvable home".to_string()) + } + }; + let intent = json!({ + "path": account_home, + "provenance": "directory-picker", + }); + + let error = resolve_checkpoint_workspace_with_account_home( + &configured_home, + intent["path"].as_str().map(Path::new).unwrap(), + Some(&intent), + &account_home_for_directory, + ) + .await + .expect_err("a picker-confirmed passwd-entry home is never a checkpoint scope"); + + assert!(error.reason.contains("account home"), "{}", error.reason); + } + #[tokio::test] async fn explicit_picker_home_is_rejected_even_when_git_root_is_its_parent() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/files.rs b/crates/freshell-server/src/files.rs index d695e2bb5..d1738b038 100644 --- a/crates/freshell-server/src/files.rs +++ b/crates/freshell-server/src/files.rs @@ -217,21 +217,7 @@ async fn validate_dir( } fn is_checkpoint_home(path: &Path) -> bool { - let Some(selected) = std::fs::canonicalize(path).ok() else { - return false; - }; - - let mut home_candidates = crate::session_directory::provider_home() - .into_iter() - .collect::<Vec<_>>(); - if let Some(configured_home) = std::env::var_os("FRESHELL_HOME") { - home_candidates.push(PathBuf::from(configured_home)); - } - - home_candidates - .iter() - .filter_map(|home| std::fs::canonicalize(home).ok()) - .any(|home| home == selected) + crate::checkpoints::is_account_home_checkpoint_path(path) } /// `GET /api/files/read?path=<p>` \u2192 `{ content, size, modifiedAt }` (`files-router.ts:85`). @@ -1490,6 +1476,35 @@ mod tests { } } + #[cfg(unix)] + #[allow(clippy::await_holding_lock)] + #[tokio::test] + async fn validate_dir_excludes_passwd_home_when_home_env_is_redirected() { + let _guard = env_lock(); + let redirected_home = tempfile::tempdir().unwrap(); + let redirected_home_path = redirected_home.path().to_string_lossy().into_owned(); + let account_home = + std::fs::canonicalize(crate::session_directory::passwd_entry_home()).unwrap(); + let account_home_path = account_home.to_string_lossy().into_owned(); + let _env = EnvGuard::set(&[ + ("HOME", Some(redirected_home_path.as_str())), + ("FRESHELL_HOME", None), + ("USERPROFILE", None), + ]); + + let response = validate_dir( + State(test_state()), + auth_headers(), + Json(json!({ "path": account_home_path })), + ) + .await; + + assert_eq!(response.status(), StatusCode::OK); + let value = body_json(response).await; + assert_eq!(value["valid"], true); + assert_eq!(value["checkpointWorkspaceEligible"], false); + } + /// Node's `normalizeUserPath` runs `path.posix.resolve` on POSIX inputs /// (path-utils.ts:69-70), collapsing `.`/`..` segments LEXICALLY — so /// `<tmp>/nope/../real` reaches the filesystem as `<tmp>/real` and From f31298ae7b426d20827abd6a9077b8be0fbb4f10 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:32:47 -0700 Subject: [PATCH 358/493] test(checkpoints): close provider pane through its shell --- .../code-checkpoint-workspace-external.spec.ts | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index d0cf4974f..9d07127bc 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -239,6 +239,15 @@ async function flushLayoutAndReload(page: import('@playwright/test').Page): Prom await expect(page.locator('[data-context="fresh-agent"]')).toBeVisible({ timeout: 30_000 }) } +async function closePaneForContent(paneRoot: import('@playwright/test').Locator): Promise<void> { + // PaneHeader is a sibling of the provider content, so its close button is + // owned by the nearest pane shell rather than nested inside `paneRoot`. + await paneRoot + .locator('xpath=ancestor::*[@data-pane-shell="true"][1]') + .getByRole('button', { name: 'Close pane' }) + .click() +} + async function openSessionFromHistory( page: import('@playwright/test').Page, paneRoot: import('@playwright/test').Locator, @@ -265,7 +274,7 @@ async function openSessionFromHistory( // Closing the seed pane makes History restore from its directory projection, // which proves the explicit workspace provenance survived session metadata. - await paneRoot.getByRole('button', { name: 'Close pane' }).click() + await closePaneForContent(paneRoot) await expect(page.locator('[data-context="fresh-agent"]')).toHaveCount(0, { timeout: 30_000 }) await page.locator('button[title="Projects (Ctrl+B P)"]').click() const projectHeader = page.locator(`[data-project-path="${projectPath}"]`) @@ -459,7 +468,7 @@ test('opening a linked-worktree session from History rewinds its resolved checko // History must open from its directory projection; keeping the seed pane // open would exercise the already-open-pane path and preserve its explicit // picker scope instead of testing ordinary session restoration. - await paneRoot.getByRole('button', { name: 'Close pane' }).click() + await closePaneForContent(paneRoot) await expect(page.locator('[data-context="fresh-agent"]')).toHaveCount(0, { timeout: 30_000 }) await page.locator('button[title="Projects (Ctrl+B P)"]').click() const projectHeader = page.locator(`[data-project-path="${fixture.syntheticWorkspacePath}"]`) From d068c74fd816f182983711829ce4896c82421ee4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:43:03 -0700 Subject: [PATCH 359/493] fix(checkpoints): fail closed on unknown workspace owners --- crates/freshell-server/src/checkpoints.rs | 88 ++++++++++++----------- crates/freshell-server/src/files.rs | 21 +++++- 2 files changed, 67 insertions(+), 42 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 5321274f6..e4df3c212 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -3008,23 +3008,14 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { homes } -fn is_account_home_checkpoint_scope(path: &Path, server_home: &Path) -> bool { - is_account_home_checkpoint_path_with_server_home(path, Some(server_home)) -} - -pub(crate) fn is_account_home_checkpoint_path(path: &Path) -> bool { - is_account_home_checkpoint_path_with_server_home(path, None) -} - -fn is_account_home_checkpoint_path_with_server_home( +pub(crate) fn account_home_checkpoint_path_with_lookup( path: &Path, - server_home: Option<&Path>, -) -> bool { - let Ok(path) = std::fs::canonicalize(path) else { - return false; - }; - let homes = recognized_account_homes(server_home); - is_account_home_for_owner(&path, &homes, &account_home_for_directory).unwrap_or(false) + account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), +) -> Result<bool, String> { + let path = std::fs::canonicalize(path) + .map_err(|error| format!("checkpoint workspace cannot be resolved: {error}"))?; + let homes = recognized_account_homes(None); + is_account_home_for_owner(&path, &homes, account_home_for_directory) } fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { @@ -3044,7 +3035,7 @@ fn is_account_home_for_owner( } #[cfg(unix)] -fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { +pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { use std::ffi::{CStr, OsString}; use std::os::unix::ffi::OsStringExt; use std::os::unix::fs::MetadataExt; @@ -3093,7 +3084,7 @@ fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { } #[cfg(windows)] -fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { +pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { use std::os::windows::ffi::OsStrExt; use windows_sys::Win32::Foundation::{LocalFree, ERROR_SUCCESS, HLOCAL}; use windows_sys::Win32::Security::Authorization::{ @@ -3269,7 +3260,7 @@ fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { } #[cfg(not(any(unix, windows)))] -fn account_home_for_directory(_path: &Path) -> Result<PathBuf, String> { +pub(crate) fn account_home_for_directory(_path: &Path) -> Result<PathBuf, String> { Err("workspace owner account-home lookup is unavailable on this platform".into()) } @@ -3365,23 +3356,9 @@ async fn resolve_checkpoint_workspace_with_account_home( candidate: candidate.clone(), reason, })?; - if is_account_home_checkpoint_scope(&selected, server_home) { - return Err(CheckpointWorkspaceUnavailable { - candidate, - reason: "account home cannot be used as a code checkpoint workspace".to_string(), - }); - } let selected_home = match is_account_home_for_owner(&selected, &homes, account_home_for_directory) { - Ok(is_home) => Some(is_home), - Err(_reason) - if matches!( - provenance.as_str(), - "directory-picker" | "legacy-checkpoint" - ) => - { - None - } + Ok(is_home) => is_home, Err(reason) => { return Err(CheckpointWorkspaceUnavailable { candidate, @@ -3389,7 +3366,7 @@ async fn resolve_checkpoint_workspace_with_account_home( }); } }; - if selected_home == Some(true) { + if selected_home { return Err(CheckpointWorkspaceUnavailable { candidate, reason: "account home cannot be used as a code checkpoint workspace".to_string(), @@ -3404,12 +3381,6 @@ async fn resolve_checkpoint_workspace_with_account_home( let path = match git_checkout_root(&selected).await { Ok(Some(root)) if provenance == "directory-picker" && root == selected => selected, Ok(Some(root)) => { - if selected_home.is_none() { - return Err(CheckpointWorkspaceUnavailable { - candidate, - reason: "workspace account scope is unknown; the selected directory cannot be widened to its Git root".into(), - }); - } match is_account_home_for_owner(&root, &homes, account_home_for_directory) { Ok(true) => { return Err(CheckpointWorkspaceUnavailable { @@ -7080,6 +7051,41 @@ mod tests { assert!(error.reason.contains("account home"), "{}", error.reason); } + #[tokio::test] + async fn picker_scope_is_unavailable_when_owner_home_cannot_be_resolved() { + let owned = tempfile::tempdir().unwrap(); + let server_home = owned.path().join("managed-home"); + let selected = owned.path().join("unknown-account-home"); + tokio::fs::create_dir_all(&server_home).await.unwrap(); + tokio::fs::create_dir_all(&selected).await.unwrap(); + let account_home_for_directory = + |_path: &Path| Err("synthetic NSS lookup failure".to_string()); + let intent = json!({ + "path": selected, + "provenance": "directory-picker", + }); + + let error = resolve_checkpoint_workspace_with_account_home( + &server_home, + intent["path"].as_str().map(Path::new).unwrap(), + Some(&intent), + &account_home_for_directory, + ) + .await + .expect_err("an unknown owner home must not become a non-Git snapshot scope"); + + assert!( + error.reason.contains("account scope is unknown"), + "{}", + error.reason + ); + assert!( + error.reason.contains("synthetic NSS lookup failure"), + "{}", + error.reason + ); + } + #[tokio::test] async fn explicit_picker_home_is_rejected_even_when_git_root_is_its_parent() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/files.rs b/crates/freshell-server/src/files.rs index d1738b038..856dc0b1c 100644 --- a/crates/freshell-server/src/files.rs +++ b/crates/freshell-server/src/files.rs @@ -217,7 +217,15 @@ async fn validate_dir( } fn is_checkpoint_home(path: &Path) -> bool { - crate::checkpoints::is_account_home_checkpoint_path(path) + is_checkpoint_home_with_lookup(path, &crate::checkpoints::account_home_for_directory) +} + +fn is_checkpoint_home_with_lookup( + path: &Path, + account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), +) -> bool { + crate::checkpoints::account_home_checkpoint_path_with_lookup(path, account_home_for_directory) + .unwrap_or(true) } /// `GET /api/files/read?path=<p>` \u2192 `{ content, size, modifiedAt }` (`files-router.ts:85`). @@ -1505,6 +1513,17 @@ mod tests { assert_eq!(value["checkpointWorkspaceEligible"], false); } + #[test] + fn picker_scope_is_ineligible_when_owner_account_cannot_be_resolved() { + let workspace = tempfile::tempdir().unwrap(); + let account_lookup = |_path: &Path| Err("synthetic NSS lookup failure".to_string()); + + assert!(is_checkpoint_home_with_lookup( + workspace.path(), + &account_lookup + )); + } + /// Node's `normalizeUserPath` runs `path.posix.resolve` on POSIX inputs /// (path-utils.ts:69-70), collapsing `.`/`..` segments LEXICALLY — so /// `<tmp>/nope/../real` reaches the filesystem as `<tmp>/real` and From 41bba38e2ed1ad8ced1e37e89e35a3c893e9ba09 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:52:20 -0700 Subject: [PATCH 360/493] fix(checkpoints): allow group-owned workspace scopes --- crates/freshell-server/src/checkpoints.rs | 105 +++++++++++++++++++--- crates/freshell-server/src/files.rs | 13 ++- 2 files changed, 105 insertions(+), 13 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index e4df3c212..ad8cc892f 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -3010,7 +3010,7 @@ fn recognized_account_homes(server_home: Option<&Path>) -> Vec<PathBuf> { pub(crate) fn account_home_checkpoint_path_with_lookup( path: &Path, - account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), + account_home_for_directory: &AccountHomeForDirectory, ) -> Result<bool, String> { let path = std::fs::canonicalize(path) .map_err(|error| format!("checkpoint workspace cannot be resolved: {error}"))?; @@ -3018,6 +3018,9 @@ pub(crate) fn account_home_checkpoint_path_with_lookup( is_account_home_for_owner(&path, &homes, account_home_for_directory) } +pub(crate) type AccountHomeForDirectory = + dyn Fn(&Path) -> Result<Option<PathBuf>, String> + Send + Sync; + fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { homes.iter().any(|home| home == path) } @@ -3025,17 +3028,19 @@ fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { fn is_account_home_for_owner( path: &Path, known_homes: &[PathBuf], - account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), + account_home_for_directory: &AccountHomeForDirectory, ) -> Result<bool, String> { if is_recognized_account_home(path, known_homes) { return Ok(true); } - let owner_home = account_home_for_directory(path)?; + let Some(owner_home) = account_home_for_directory(path)? else { + return Ok(false); + }; Ok(owner_home == path) } #[cfg(unix)] -pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { +pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, String> { use std::ffi::{CStr, OsString}; use std::os::unix::ffi::OsStringExt; use std::os::unix::fs::MetadataExt; @@ -3077,20 +3082,24 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> return Err("workspace owner's account has an empty home directory".into()); } let home = PathBuf::from(OsString::from_vec(home.to_bytes().to_vec())); - return std::fs::canonicalize(&home).map_err(|error| { + return std::fs::canonicalize(&home).map(Some).map_err(|error| { format!("workspace owner's account home cannot be resolved: {error}") }); } } #[cfg(windows)] -pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> { +pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, String> { use std::os::windows::ffi::OsStrExt; - use windows_sys::Win32::Foundation::{LocalFree, ERROR_SUCCESS, HLOCAL}; + use windows_sys::Win32::Foundation::{GetLastError, LocalFree, ERROR_SUCCESS, HLOCAL}; use windows_sys::Win32::Security::Authorization::{ ConvertSidToStringSidW, GetNamedSecurityInfoW, SE_FILE_OBJECT, }; - use windows_sys::Win32::Security::{OWNER_SECURITY_INFORMATION, PSECURITY_DESCRIPTOR, PSID}; + use windows_sys::Win32::Security::{ + LookupAccountSidW, SidTypeAlias, SidTypeComputer, SidTypeDomain, SidTypeGroup, + SidTypeUnknown, SidTypeUser, SidTypeWellKnownGroup, OWNER_SECURITY_INFORMATION, + PSECURITY_DESCRIPTOR, PSID, + }; use windows_sys::Win32::System::Environment::ExpandEnvironmentStringsW; use windows_sys::Win32::System::Registry::{ RegCloseKey, RegOpenKeyExW, RegQueryValueExW, HKEY, HKEY_LOCAL_MACHINE, KEY_READ, @@ -3157,6 +3166,48 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> )); } let _descriptor = LocalAllocation(descriptor.cast()); + + // A directory may be owned by a Windows group (for example, the local + // Administrators group). Groups do not have user profiles, so a missing + // ProfileList entry for those SIDs means "not a home", not an unresolved + // home. Keep lookup failures and unsupported SID kinds distinct so user + // profile paths still fail closed when they cannot be verified. + let mut owner_name = vec![0u16; 1024]; + let mut owner_name_length = owner_name.len() as u32; + let mut domain_name = vec![0u16; 1024]; + let mut domain_name_length = domain_name.len() as u32; + let mut sid_type = SidTypeUnknown; + if unsafe { + LookupAccountSidW( + std::ptr::null(), + owner_sid, + owner_name.as_mut_ptr(), + &mut owner_name_length, + domain_name.as_mut_ptr(), + &mut domain_name_length, + &mut sid_type, + ) + } == 0 + { + let error = unsafe { GetLastError() }; + return Err(format!( + "workspace owner account type cannot be resolved (Windows error {error})" + )); + } + if sid_type != SidTypeUser { + if sid_type == SidTypeAlias + || sid_type == SidTypeComputer + || sid_type == SidTypeDomain + || sid_type == SidTypeGroup + || sid_type == SidTypeWellKnownGroup + { + return Ok(None); + } + return Err(format!( + "workspace owner SID has an unsupported account type ({sid_type})" + )); + } + let mut sid_string = std::ptr::null_mut(); if unsafe { ConvertSidToStringSidW(owner_sid, &mut sid_string) } == 0 { return Err("workspace owner SID cannot be formatted".into()); @@ -3256,11 +3307,12 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<PathBuf, String> profile }; std::fs::canonicalize(PathBuf::from(profile)) + .map(Some) .map_err(|error| format!("workspace owner Windows profile cannot be resolved: {error}")) } #[cfg(not(any(unix, windows)))] -pub(crate) fn account_home_for_directory(_path: &Path) -> Result<PathBuf, String> { +pub(crate) fn account_home_for_directory(_path: &Path) -> Result<Option<PathBuf>, String> { Err("workspace owner account-home lookup is unavailable on this platform".into()) } @@ -3327,7 +3379,7 @@ async fn resolve_checkpoint_workspace_with_account_home( server_home: &Path, launch_cwd: &Path, intent: Option<&Value>, - account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), + account_home_for_directory: &AccountHomeForDirectory, ) -> Result<ResolvedCheckpointWorkspace, CheckpointWorkspaceUnavailable> { let candidate = intent .and_then(|workspace| workspace.get("path")) @@ -7029,7 +7081,7 @@ mod tests { let owner_home = account_home.clone(); let account_home_for_directory = move |path: &Path| { if path == owner_home { - Ok(owner_home.clone()) + Ok(Some(owner_home.clone())) } else { Err("synthetic owner has no resolvable home".to_string()) } @@ -7086,6 +7138,35 @@ mod tests { ); } + #[tokio::test] + async fn picked_non_git_scope_without_a_user_profile_remains_valid() { + let owned = tempfile::tempdir().unwrap(); + let server_home = owned.path().join("managed-home"); + let workspace = owned.path().join("group-owned-project"); + tokio::fs::create_dir_all(&server_home).await.unwrap(); + tokio::fs::create_dir_all(&workspace).await.unwrap(); + let account_home_for_directory = |_path: &Path| Ok(None); + let intent = json!({ + "path": workspace, + "provenance": "directory-picker", + }); + + let resolved = resolve_checkpoint_workspace_with_account_home( + &server_home, + intent["path"].as_str().map(Path::new).unwrap(), + Some(&intent), + &account_home_for_directory, + ) + .await + .expect("a group-owned non-Git project has no user-home profile to reject"); + + assert_eq!( + resolved.path, + PathBuf::from(intent["path"].as_str().unwrap()) + ); + assert_eq!(resolved.provenance, "directory-picker"); + } + #[tokio::test] async fn explicit_picker_home_is_rejected_even_when_git_root_is_its_parent() { let owned = tempfile::tempdir().unwrap(); @@ -7161,7 +7242,7 @@ mod tests { let owner_home = foreign_home.clone(); let account_home_for_directory = move |path: &Path| { if path == owner_home { - Ok(owner_home.clone()) + Ok(Some(owner_home.clone())) } else { Err("synthetic NSS owner has no resolvable home".to_string()) } diff --git a/crates/freshell-server/src/files.rs b/crates/freshell-server/src/files.rs index 856dc0b1c..345ab2189 100644 --- a/crates/freshell-server/src/files.rs +++ b/crates/freshell-server/src/files.rs @@ -222,7 +222,7 @@ fn is_checkpoint_home(path: &Path) -> bool { fn is_checkpoint_home_with_lookup( path: &Path, - account_home_for_directory: &(dyn Fn(&Path) -> Result<PathBuf, String> + Send + Sync), + account_home_for_directory: &crate::checkpoints::AccountHomeForDirectory, ) -> bool { crate::checkpoints::account_home_checkpoint_path_with_lookup(path, account_home_for_directory) .unwrap_or(true) @@ -1524,6 +1524,17 @@ mod tests { )); } + #[test] + fn picker_scope_stays_eligible_for_an_owner_without_a_user_profile() { + let workspace = tempfile::tempdir().unwrap(); + let no_user_profile = |_path: &Path| Ok(None); + + assert!(!is_checkpoint_home_with_lookup( + workspace.path(), + &no_user_profile + )); + } + /// Node's `normalizeUserPath` runs `path.posix.resolve` on POSIX inputs /// (path-utils.ts:69-70), collapsing `.`/`..` segments LEXICALLY — so /// `<tmp>/nope/../real` reaches the filesystem as `<tmp>/real` and From c9c25694dce65b0dc5312e90d8802149df266ed3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:01:23 -0700 Subject: [PATCH 361/493] fix(checkpoints): inspect profiles for well-known Windows SIDs --- crates/freshell-server/src/checkpoints.rs | 103 +++++++++++++++++----- 1 file changed, 83 insertions(+), 20 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index ad8cc892f..0f3ba5f1d 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -3021,6 +3021,26 @@ pub(crate) fn account_home_checkpoint_path_with_lookup( pub(crate) type AccountHomeForDirectory = dyn Fn(&Path) -> Result<Option<PathBuf>, String> + Send + Sync; +#[cfg(any(windows, test))] +#[derive(Clone, Copy)] +enum WindowsAccountOwnerKind { + User, + KnownNonUser, +} + +#[cfg(any(windows, test))] +fn windows_account_home_for_owner_kind( + owner_kind: WindowsAccountOwnerKind, + profile: Option<PathBuf>, +) -> Result<Option<PathBuf>, String> { + match owner_kind { + WindowsAccountOwnerKind::User => profile + .map(Some) + .ok_or_else(|| "workspace owner SID has no readable Windows profile record".into()), + WindowsAccountOwnerKind::KnownNonUser => Ok(profile), + } +} + fn is_recognized_account_home(path: &Path, homes: &[PathBuf]) -> bool { homes.iter().any(|home| home == path) } @@ -3091,7 +3111,9 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, #[cfg(windows)] pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, String> { use std::os::windows::ffi::OsStrExt; - use windows_sys::Win32::Foundation::{GetLastError, LocalFree, ERROR_SUCCESS, HLOCAL}; + use windows_sys::Win32::Foundation::{ + GetLastError, LocalFree, ERROR_FILE_NOT_FOUND, ERROR_PATH_NOT_FOUND, ERROR_SUCCESS, HLOCAL, + }; use windows_sys::Win32::Security::Authorization::{ ConvertSidToStringSidW, GetNamedSecurityInfoW, SE_FILE_OBJECT, }; @@ -3167,11 +3189,10 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, } let _descriptor = LocalAllocation(descriptor.cast()); - // A directory may be owned by a Windows group (for example, the local - // Administrators group). Groups do not have user profiles, so a missing - // ProfileList entry for those SIDs means "not a home", not an unresolved - // home. Keep lookup failures and unsupported SID kinds distinct so user - // profile paths still fail closed when they cannot be verified. + // A directory may be owned by a Windows group, which normally has no + // profile. Some well-known group SIDs (notably SYSTEM) do have registered + // profiles, so inspect ProfileList for every recognized SID type before + // deciding that a known non-user owner has no profile. let mut owner_name = vec![0u16; 1024]; let mut owner_name_length = owner_name.len() as u32; let mut domain_name = vec![0u16; 1024]; @@ -3194,20 +3215,20 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, "workspace owner account type cannot be resolved (Windows error {error})" )); } - if sid_type != SidTypeUser { - if sid_type == SidTypeAlias - || sid_type == SidTypeComputer - || sid_type == SidTypeDomain - || sid_type == SidTypeGroup - || sid_type == SidTypeWellKnownGroup - { - return Ok(None); - } + let owner_kind = if sid_type == SidTypeUser { + WindowsAccountOwnerKind::User + } else if sid_type == SidTypeAlias + || sid_type == SidTypeComputer + || sid_type == SidTypeDomain + || sid_type == SidTypeGroup + || sid_type == SidTypeWellKnownGroup + { + WindowsAccountOwnerKind::KnownNonUser + } else { return Err(format!( "workspace owner SID has an unsupported account type ({sid_type})" )); - } - + }; let mut sid_string = std::ptr::null_mut(); if unsafe { ConvertSidToStringSidW(owner_sid, &mut sid_string) } == 0 { return Err("workspace owner SID cannot be formatted".into()); @@ -3231,6 +3252,11 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, ) }; if status != ERROR_SUCCESS { + if matches!(owner_kind, WindowsAccountOwnerKind::KnownNonUser) + && (status == ERROR_FILE_NOT_FOUND || status == ERROR_PATH_NOT_FOUND) + { + return windows_account_home_for_owner_kind(owner_kind, None); + } return Err(format!( "workspace owner SID has no readable Windows profile record (error {status})" )); @@ -3306,9 +3332,9 @@ pub(crate) fn account_home_for_directory(path: &Path) -> Result<Option<PathBuf>, } else { profile }; - std::fs::canonicalize(PathBuf::from(profile)) - .map(Some) - .map_err(|error| format!("workspace owner Windows profile cannot be resolved: {error}")) + let profile = std::fs::canonicalize(PathBuf::from(profile)) + .map_err(|error| format!("workspace owner Windows profile cannot be resolved: {error}"))?; + windows_account_home_for_owner_kind(owner_kind, Some(profile)) } #[cfg(not(any(unix, windows)))] @@ -7167,6 +7193,43 @@ mod tests { assert_eq!(resolved.provenance, "directory-picker"); } + #[test] + fn windows_well_known_group_with_a_profile_entry_is_still_an_account_home() { + let profile = tempfile::tempdir().unwrap(); + let profile_path = profile.path().to_path_buf(); + + let resolved_home = windows_account_home_for_owner_kind( + WindowsAccountOwnerKind::KnownNonUser, + Some(profile_path.clone()), + ) + .unwrap(); + assert_eq!( + resolved_home, + Some(profile_path.clone()), + "a well-known group SID such as SYSTEM can still have a registered profile" + ); + let resolved_home = resolved_home.unwrap(); + let lookup = move |_path: &Path| Ok(Some(resolved_home.clone())); + assert!( + account_home_checkpoint_path_with_lookup(profile.path(), &lookup).unwrap(), + "a selected path matching that registered profile must remain ineligible" + ); + } + + #[test] + fn windows_known_non_user_without_a_profile_remains_a_valid_workspace_owner() { + assert_eq!( + windows_account_home_for_owner_kind(WindowsAccountOwnerKind::KnownNonUser, None) + .unwrap(), + None + ); + } + + #[test] + fn windows_user_without_a_profile_fails_closed() { + assert!(windows_account_home_for_owner_kind(WindowsAccountOwnerKind::User, None).is_err()); + } + #[tokio::test] async fn explicit_picker_home_is_rejected_even_when_git_root_is_its_parent() { let owned = tempfile::tempdir().unwrap(); From 69ec18858938bfe904dca2d04caa038b9341eea3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:23:24 -0700 Subject: [PATCH 362/493] test(checkpoints): cover native Windows SYSTEM profile --- .github/workflows/electron-build.yml | 6 +++ crates/freshell-server/src/checkpoints.rs | 41 ++++++++++++++++++- crates/freshell-server/src/migrations.rs | 1 + crates/freshell-server/src/network.rs | 3 ++ .../src/session_name_migration_tests.rs | 1 + 5 files changed, 51 insertions(+), 1 deletion(-) diff --git a/.github/workflows/electron-build.yml b/.github/workflows/electron-build.yml index eb6d1c7b3..ba386fa43 100644 --- a/.github/workflows/electron-build.yml +++ b/.github/workflows/electron-build.yml @@ -100,6 +100,12 @@ jobs: if: matrix.os == 'windows-2022' run: cargo test --locked -p freshell-platform --lib cli_launch::cli_argv_goldens_file::native_windows_generated_claude_hooks_work_in_shells + # Exercise the real Windows owner-SID and ProfileList APIs for SYSTEM's + # profile. The Linux Rust/browser jobs cannot cover this Windows branch. + - name: Test Windows SYSTEM checkpoint-home rejection + if: matrix.os == 'windows-2022' + run: cargo test --locked -p freshell-server --bin freshell-server checkpoints::tests::windows_system_profile_is_rejected_as_checkpoint_scope + - name: Run Electron unit tests run: pnpm run test:electron diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 0f3ba5f1d..08d1248ba 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -26,7 +26,9 @@ use std::process::Stdio; use std::sync::{Arc, Mutex as StdMutex, Weak}; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; -use std::collections::{BTreeMap, HashMap, HashSet}; +#[cfg(unix)] +use std::collections::HashSet; +use std::collections::{BTreeMap, HashMap}; use std::sync::atomic::{AtomicBool, Ordering}; use axum::routing::get as get_method; @@ -7230,6 +7232,43 @@ mod tests { assert!(windows_account_home_for_owner_kind(WindowsAccountOwnerKind::User, None).is_err()); } + #[cfg(windows)] + #[tokio::test] + async fn windows_system_profile_is_rejected_as_checkpoint_scope() { + let system_root = std::env::var_os("SystemRoot").expect("Windows sets SystemRoot"); + let system_profile = PathBuf::from(system_root) + .join("System32") + .join("config") + .join("systemprofile"); + let canonical_profile = std::fs::canonicalize(&system_profile) + .expect("the Windows SYSTEM profile directory exists"); + let server_home = tempfile::tempdir().unwrap(); + let lookup = |path: &Path| account_home_for_directory(path); + + let owner_home = account_home_for_directory(&system_profile) + .expect("native owner SID and ProfileList lookups succeed") + .expect("the well-known SYSTEM SID has a registered profile"); + assert_eq!(owner_home, canonical_profile); + assert!(account_home_checkpoint_path_with_lookup(&system_profile, &lookup).unwrap()); + + let intent = json!({ + "path": system_profile, + "provenance": "directory-picker", + }); + let launch_cwd = Path::new(intent["path"].as_str().unwrap()); + assert!( + resolve_checkpoint_workspace_with_account_home( + server_home.path(), + launch_cwd, + Some(&intent), + &lookup, + ) + .await + .is_err(), + "a selected SYSTEM profile must never become a checkpoint workspace" + ); + } + #[tokio::test] async fn explicit_picker_home_is_rejected_even_when_git_root_is_its_parent() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/migrations.rs b/crates/freshell-server/src/migrations.rs index bcbf1a12f..d34475823 100644 --- a/crates/freshell-server/src/migrations.rs +++ b/crates/freshell-server/src/migrations.rs @@ -345,6 +345,7 @@ mod tests { /// disk must NOT be recorded as complete -- Node aborts before /// markMigrationDone and retries next boot. Self-skips when the process /// can write through a read-only dir (e.g. root/CAP_DAC_OVERRIDE). + #[cfg(unix)] #[tokio::test] async fn cleanup_skips_marker_when_clears_cannot_persist() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/network.rs b/crates/freshell-server/src/network.rs index 4417d459b..187ed0333 100644 --- a/crates/freshell-server/src/network.rs +++ b/crates/freshell-server/src/network.rs @@ -2590,6 +2590,7 @@ mod tests { /// failures are retried; any [`ScenarioError::Product`] violation panics /// immediately, so the falsifying power holds on the FIRST attempt while /// transient environment noise is retried away. + #[cfg(unix)] #[tokio::test] async fn configure_rolls_back_the_listener_when_persist_fails() { for attempt in 1..=5 { @@ -2615,6 +2616,7 @@ mod tests { /// Failures are classified per [`ScenarioError`]: only documented /// transient socket artifacts come back as `Env`; every in-memory /// product invariant comes back as `Product` and fails the test fast. + #[cfg(unix)] async fn rollback_scenario_once() -> Result<(), ScenarioError> { use axum::body::Body; use axum::http::Request; @@ -2938,6 +2940,7 @@ mod tests { ); } + #[cfg(unix)] #[tokio::test] async fn disable_keeps_loopback_and_reports_error_when_persist_fails() { // FAIL-SAFE counterpart of Task 2.3's rollback test (Task 6.1 #9): when the diff --git a/crates/freshell-server/src/session_name_migration_tests.rs b/crates/freshell-server/src/session_name_migration_tests.rs index 236fa8a38..5040b7d14 100644 --- a/crates/freshell-server/src/session_name_migration_tests.rs +++ b/crates/freshell-server/src/session_name_migration_tests.rs @@ -1086,6 +1086,7 @@ async fn cleanup_removes_only_title_fields_and_preserves_row_data() { /// Fault after the canonical commit before cleanup: the receipt stands, a /// later boot finishes the cleanup idempotently, and no duplicate /// consolidation runs. +#[cfg(unix)] #[tokio::test] async fn fault_after_commit_before_cleanup_finishes_idempotently() { let (tmp, home) = fresh_home(); From d317797fd9c7b5622188e96616588030fcdbef32 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:23:26 -0700 Subject: [PATCH 363/493] refactor(session-metadata): name validated update fields --- .../freshell-server/src/session_metadata.rs | 83 ++++++++++++------- 1 file changed, 51 insertions(+), 32 deletions(-) diff --git a/crates/freshell-server/src/session_metadata.rs b/crates/freshell-server/src/session_metadata.rs index b2fd23efb..a05094060 100644 --- a/crates/freshell-server/src/session_metadata.rs +++ b/crates/freshell-server/src/session_metadata.rs @@ -422,16 +422,18 @@ async fn post_session_metadata( })), ) .into_response(), - Ok((provider, session_id, session_type, session_type_source, checkpoint_workspace)) => { + Ok(post) => { match apply_session_metadata_with_checkpoint_workspace( &state.store, &state.broadcast_tx, &state.sessions_revision, - &provider, - &session_id, - &session_type, - session_type_source.as_deref(), - checkpoint_workspace.as_ref(), + SessionMetadataUpdate { + provider: &post.provider, + session_id: &post.session_id, + session_type: &post.session_type, + session_type_source: post.session_type_source.as_deref(), + checkpoint_workspace: post.checkpoint_workspace.as_ref(), + }, ) .await { @@ -464,40 +466,51 @@ pub async fn apply_session_metadata( store, broadcast_tx, sessions_revision, - provider, - session_id, - session_type, - session_type_source, - None, + SessionMetadataUpdate { + provider, + session_id, + session_type, + session_type_source, + checkpoint_workspace: None, + }, ) .await } +struct SessionMetadataUpdate<'a> { + provider: &'a str, + session_id: &'a str, + session_type: &'a str, + session_type_source: Option<&'a str>, + checkpoint_workspace: Option<&'a Value>, +} + async fn apply_session_metadata_with_checkpoint_workspace( store: &SessionMetadataStore, broadcast_tx: &tokio::sync::broadcast::Sender<String>, sessions_revision: &std::sync::atomic::AtomicI64, - provider: &str, - session_id: &str, - session_type: &str, - session_type_source: Option<&str>, - checkpoint_workspace: Option<&Value>, + update: SessionMetadataUpdate<'_>, ) -> std::io::Result<bool> { - let changed = match checkpoint_workspace { + let changed = match update.checkpoint_workspace { Some(workspace) => { store .set_with_checkpoint_workspace( - provider, - session_id, - session_type, - session_type_source, + update.provider, + update.session_id, + update.session_type, + update.session_type_source, Some(workspace), ) .await? } None => { store - .set(provider, session_id, session_type, session_type_source) + .set( + update.provider, + update.session_id, + update.session_type, + update.session_type_source, + ) .await? } }; @@ -521,9 +534,15 @@ async fn apply_session_metadata_with_checkpoint_workspace( /// zod-issue-shaped `{path, message}` objects on failure (schema-approximate — the /// reference's exact zod issue shape is not part of any client contract this port must /// match; the client only reads the top-level `error` string on a 400). -fn validate_post_body( - body: &Value, -) -> Result<(String, String, String, Option<String>, Option<Value>), Vec<Value>> { +struct ValidatedSessionMetadataPost { + provider: String, + session_id: String, + session_type: String, + session_type_source: Option<String>, + checkpoint_workspace: Option<Value>, +} + +fn validate_post_body(body: &Value) -> Result<ValidatedSessionMetadataPost, Vec<Value>> { let mut issues = Vec::new(); let provider = body.get("provider").and_then(Value::as_str); @@ -594,15 +613,15 @@ fn validate_post_body( return Err(issues); } - Ok(( - provider.unwrap().to_string(), - session_id.unwrap().to_string(), - session_type.unwrap().to_string(), - session_type_source + Ok(ValidatedSessionMetadataPost { + provider: provider.unwrap().to_string(), + session_id: session_id.unwrap().to_string(), + session_type: session_type.unwrap().to_string(), + session_type_source: session_type_source .and_then(Value::as_str) .map(str::to_string), - checkpoint_workspace_value, - )) + checkpoint_workspace: checkpoint_workspace_value, + }) } #[cfg(test)] From 089a8d54093b20ce00914d62990f38c333036f1a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:51:53 -0700 Subject: [PATCH 364/493] fix(checkpoints): preserve explicit workspace in history --- src/lib/api.ts | 7 ++- ...code-checkpoint-workspace-external.spec.ts | 27 +++++----- test/unit/client/lib/api.test.ts | 49 ++++++++++++++----- .../tooling/testing/browser-selection.test.ts | 2 +- 4 files changed, 58 insertions(+), 27 deletions(-) diff --git a/src/lib/api.ts b/src/lib/api.ts index c356d724b..dfa9c6142 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -875,11 +875,14 @@ function groupDirectoryItemsAsProjects( function checkpointWorkspaceForDirectoryItem( item: Pick<ReadModelSessionDirectoryItem, 'checkoutPath' | 'projectPath' | 'checkpointWorkspace'>, ): FreshAgentCheckpointWorkspace | undefined { + // The picked scope is independent of launch cwd. `checkoutPath` is a + // presentation value that can fall back to cwd, so it cannot override a + // separately persisted picker choice. + const explicitWorkspace = normalizeFreshAgentCheckpointWorkspace(item.checkpointWorkspace) + if (explicitWorkspace?.provenance === 'directory-picker') return explicitWorkspace if (item.checkoutPath) { return normalizeFreshAgentCheckpointWorkspace({ path: item.checkoutPath, provenance: 'resolved-project' }) } - const explicitWorkspace = normalizeFreshAgentCheckpointWorkspace(item.checkpointWorkspace) - if (explicitWorkspace?.provenance === 'directory-picker') return explicitWorkspace const path = item.projectPath if (!path || path === 'unknown' || path.startsWith('terminal:')) return undefined return normalizeFreshAgentCheckpointWorkspace({ path, provenance: 'resolved-project' }) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 9d07127bc..67107685a 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -282,7 +282,7 @@ async function openSessionFromHistory( await projectHeader.click() const historyRow = page.locator(`[data-context="history-session"][data-session-id="${sessionId}"]`) await expect(historyRow).toBeVisible({ timeout: 30_000 }) - await historyRow.getByRole('button', { name: 'Open session' }).click() + await historyRow.getByRole('button', { name: 'Open session', exact: true }).click() const reopenedPane = page.locator('[data-context="fresh-agent"]').last() await expect.poll(async () => (await currentFreshAgent(page)).content.checkpointWorkspace) @@ -423,7 +423,7 @@ test('Git workspace sends retain per-turn success and failure, then restore the .toEqual(durable?.map(turn => `${turn.id}:user`)) }) -test('opening a linked-worktree session from History rewinds its resolved checkout', async ({ page }) => { +test('opening a linked-worktree session from History preserves picker scope and rewinds its checkout', async ({ page }) => { test.setTimeout(360_000) await verifyTarget(page) const checkout = fixture.linkedWorktreePath @@ -447,6 +447,7 @@ test('opening a linked-worktree session from History rewinds its resolved checko projectPath: string checkoutPath?: string cwd?: string + checkpointWorkspace?: { path: string; provenance: string } } let directoryItem: DirectoryItem | undefined await expect.poll(async () => { @@ -463,11 +464,11 @@ test('opening a linked-worktree session from History rewinds its resolved checko projectPath: fixture.syntheticWorkspacePath, checkoutPath: checkout, cwd: launchDirectory, + checkpointWorkspace: { path: launchDirectory, provenance: 'directory-picker' }, }) - // History must open from its directory projection; keeping the seed pane - // open would exercise the already-open-pane path and preserve its explicit - // picker scope instead of testing ordinary session restoration. + // History must restore the persisted picker scope from its directory + // projection; keeping the seed pane open would exercise only live pane state. await closePaneForContent(paneRoot) await expect(page.locator('[data-context="fresh-agent"]')).toHaveCount(0, { timeout: 30_000 }) await page.locator('button[title="Projects (Ctrl+B P)"]').click() @@ -476,10 +477,10 @@ test('opening a linked-worktree session from History rewinds its resolved checko await projectHeader.click() const historyRow = page.locator(`[data-context="history-session"][data-session-id="${sessionId}"]`) await expect(historyRow).toBeVisible({ timeout: 30_000 }) - await historyRow.getByRole('button', { name: 'Open session' }).click() + await historyRow.getByRole('button', { name: 'Open session', exact: true }).click() await expect.poll(async () => (await currentFreshAgent(page)).content.checkpointWorkspace) - .toEqual({ path: checkout, provenance: 'resolved-project' }) + .toEqual({ path: launchDirectory, provenance: 'directory-picker' }) const reopenedPane = page.locator('[data-context="fresh-agent"]').last() const beforeMain = 'main before linked-worktree rewind\n' const beforeNested = 'nested before linked-worktree rewind\n' @@ -488,12 +489,12 @@ test('opening a linked-worktree session from History rewinds its resolved checko const turnText = `linked-worktree resolved-scope rewind ${fixture.expectedNonce}` const turn = await sendThroughComposer(page, reopenedPane, turnText, { - path: checkout, - provenance: 'resolved-project', + path: launchDirectory, + provenance: 'directory-picker', }) expect(turn.outcome).toMatchObject({ status: 'available', - workspace: { path: checkout, provenance: 'resolved-project' }, + workspace: { path: checkout, provenance: 'directory-picker' }, }) expect(turn.outcome.checkpointId).toMatch(/^[a-f0-9]{40}$/) @@ -563,9 +564,9 @@ test('picker-confirmed HOME sets launch cwd without creating a checkpoint scope' const marker = `checkpoint home ${fixture.expectedNonce}` const turn = await sendThroughComposer(page, paneRoot, marker, undefined) expect(turn.outcome).toMatchObject({ status: 'unavailable', code: 'workspace_unavailable' }) - expect(turn.outcome.reason).toContain('account home') + expect(turn.outcome.reason).toContain('No eligible code workspace') await expect(turn.row.getByRole('note', { name: 'Code rewind unavailable for this message' })) - .toContainText('account home') + .toContainText('rewind is unavailable') await turn.row.hover() await expect(turn.row.getByRole('button', { name: 'Rewind unavailable for this message' })) .toBeDisabled() @@ -586,7 +587,7 @@ test('picker-confirmed HOME sets launch cwd without creating a checkpoint scope' const restoredRow = page.locator('[data-context="fresh-agent"]').last() .locator('article[data-turn-role="user"]', { hasText: marker }) await expect(restoredRow.getByRole('note', { name: 'Code rewind unavailable for this message' })) - .toContainText('account home') + .toContainText('rewind is unavailable') const query = new URLSearchParams({ cwd: fixture.syntheticHomePath }) const listUrl = `${paired!.targetUrl}/api/fresh-agent/checkpoints?${query}` diff --git a/test/unit/client/lib/api.test.ts b/test/unit/client/lib/api.test.ts index 7f38122c7..55c88f983 100644 --- a/test/unit/client/lib/api.test.ts +++ b/test/unit/client/lib/api.test.ts @@ -983,8 +983,12 @@ describe('searchSessions cursor pagination', () => { expect(response.hasMore).toBe(false) }) - it('preserves linked-worktree checkout scope for session opens from snapshots and search', async () => { + it('preserves an explicitly picked linked-worktree subdirectory from snapshots and search', async () => { const checkoutPath = '/repo/.worktrees/feature-a' + const workspace = { + path: `${checkoutPath}/nested`, + provenance: 'directory-picker' as const, + } mockFetch .mockResolvedValueOnce(mockJson({ items: [{ @@ -992,6 +996,7 @@ describe('searchSessions cursor pagination', () => { provider: 'codex', projectPath: '/repo', checkoutPath, + checkpointWorkspace: workspace, cwd: '/home/user', isRunning: false, lastActivityAt: 1_000, @@ -1005,6 +1010,7 @@ describe('searchSessions cursor pagination', () => { provider: 'codex', projectPath: '/repo', checkoutPath, + checkpointWorkspace: workspace, cwd: '/home/user', matchedIn: 'title', isRunning: false, @@ -1017,14 +1023,8 @@ describe('searchSessions cursor pagination', () => { const snapshot = await fetchSidebarSessionsSnapshot() const search = await searchSessions({ query: 'worktree' }) - expect(snapshot.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual({ - path: checkoutPath, - provenance: 'resolved-project', - }) - expect(search.results[0]?.checkpointWorkspace).toEqual({ - path: checkoutPath, - provenance: 'resolved-project', - }) + expect(snapshot.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual(workspace) + expect(search.results[0]?.checkpointWorkspace).toEqual(workspace) }) it('uses the resolved project path when an ordinary session launch cwd differs', async () => { @@ -1073,13 +1073,15 @@ describe('searchSessions cursor pagination', () => { }) it('restores an explicitly selected non-Git workspace from snapshots and search', async () => { + const checkoutPath = '/repo/.worktrees/feature-a' const workspace = { path: '/tmp/non-git-workspace', provenance: 'directory-picker' as const } const item = { sessionId: 'session-non-git', provider: 'codex', - projectPath: '/tmp/non-git-workspace', + projectPath: '/repo', + checkoutPath, checkpointWorkspace: workspace, - cwd: '/tmp/non-git-workspace', + cwd: checkoutPath, isRunning: false, lastActivityAt: 1_000, } @@ -1090,6 +1092,31 @@ describe('searchSessions cursor pagination', () => { const snapshot = await fetchSidebarSessionsSnapshot() const search = await searchSessions({ query: 'non-git' }) + expect(snapshot.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual(workspace) + expect(search.results[0]?.checkpointWorkspace).toEqual(workspace) + expect(snapshot.projects[0]?.sessions[0]?.checkoutPath).toBe(checkoutPath) + expect(search.results[0]?.checkoutPath).toBe(checkoutPath) + }) + + it('does not treat a cwd-fallback checkoutPath as proof that a picked workspace belongs to Git', async () => { + const workspace = { path: '/home/user/non-git-project', provenance: 'directory-picker' as const } + const item = { + sessionId: 'session-cwd-fallback', + provider: 'codex', + projectPath: '/repo', + checkoutPath: '/home/user', + checkpointWorkspace: workspace, + cwd: '/home/user', + isRunning: false, + lastActivityAt: 1_000, + } + mockFetch + .mockResolvedValueOnce(mockJson({ items: [item], nextCursor: null, revision: 1 })) + .mockResolvedValueOnce(mockJson({ items: [item], nextCursor: null, revision: 2 })) + + const snapshot = await fetchSidebarSessionsSnapshot() + const search = await searchSessions({ query: 'cwd fallback' }) + expect(snapshot.projects[0]?.sessions[0]?.checkpointWorkspace).toEqual(workspace) expect(search.results[0]?.checkpointWorkspace).toEqual(workspace) }) diff --git a/test/unit/tooling/testing/browser-selection.test.ts b/test/unit/tooling/testing/browser-selection.test.ts index 9b9775ae2..aea580c71 100644 --- a/test/unit/tooling/testing/browser-selection.test.ts +++ b/test/unit/tooling/testing/browser-selection.test.ts @@ -132,7 +132,7 @@ describe('browser selection non-vacuity', () => { FRESHELL_E2E_TARGET_TOKEN: 'synthetic-selection-token' }) const selected = listedProjects(paired, cloudConfig, ['--project=chromium-external-target', spec]) expect(selected.labels).toEqual(['chromium-external-target']) - expect(selected.tests).toBe(4) + expect(selected.tests).toBe(5) expect(selected.files).toBe(1) expect([...listedSpecFiles(selected.output)]).toEqual([spec]) const ordinaryExplicit = spawnSync(process.execPath, [playwrightCli, 'test', '--config', cloudConfig, From aa19e6f8b76ab434b70fe4a453be266ca031a195 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:51:56 -0700 Subject: [PATCH 365/493] fix(managed): settle terminal status from lifecycle --- .../src/managed_output_tests.rs | 200 ++++++++++++++++++ crates/freshell-terminal/src/registry.rs | 20 +- 2 files changed, 215 insertions(+), 5 deletions(-) diff --git a/crates/freshell-terminal/src/managed_output_tests.rs b/crates/freshell-terminal/src/managed_output_tests.rs index 120670d6d..d673eee01 100644 --- a/crates/freshell-terminal/src/managed_output_tests.rs +++ b/crates/freshell-terminal/src/managed_output_tests.rs @@ -123,6 +123,13 @@ fn fixture() -> (TerminalRegistry, Arc<Controller>) { (registry, controller) } +fn collect_messages() -> (FrameSink, Arc<std::sync::Mutex<Vec<ServerMessage>>>) { + let messages = Arc::new(std::sync::Mutex::new(Vec::new())); + let captured = Arc::clone(&messages); + let sink: FrameSink = Arc::new(move |frame| captured.lock().unwrap().push(frame)); + (sink, messages) +} + #[test] fn managed_source_epoch_changes_are_announced_to_both_views_before_new_output() { let (registry, controller) = fixture(); @@ -364,3 +371,196 @@ fn managed_output_paced_exit_preserves_explicit_code_availability() { assert_eq!(registry.managed_attached_to(77), vec!["terminal-stable"]); } } + +fn lifecycle_with_recovery_state( + state: ManagedLifecycleState, + recovery_state: &str, + desired_state: &str, +) -> ManagedLifecycle { + ManagedLifecycle { + state, + recovery_summary: Some(freshell_protocol::ManagedRecoverySummary { + desired_state: desired_state.into(), + recovery_state: recovery_state.into(), + reason: Some("fixture final recovery state".into()), + attempt_id: Some("attempt-fixture".into()), + incident_id: Some("incident-fixture".into()), + durability_state: "retained".into(), + allocation_state: "retained".into(), + }), + } +} + +#[test] +fn final_managed_lifecycle_marks_facade_exited_without_inventing_exit_code() { + for (state, recovery_state, desired_state, runtime_status) in [ + ( + ManagedLifecycleState::Blocked, + "blocked", + "running", + freshell_protocol::RuntimeStatus::Blocked, + ), + ( + ManagedLifecycleState::Ended, + "lost", + "running", + freshell_protocol::RuntimeStatus::Exited, + ), + ( + ManagedLifecycleState::Ended, + "stopped", + "stopped", + freshell_protocol::RuntimeStatus::Exited, + ), + ] { + let (registry, _) = fixture(); + assert!(registry.ingest_managed_output( + "terminal-stable", + 207, + 207, + "retained final output".into(), + )); + registry.publish_managed_lifecycle( + "terminal-stable", + Some(lifecycle_with_recovery_state( + state, + recovery_state, + desired_state, + )), + ); + + let row = ®istry.inventory()[0]; + assert_eq!(row.status, TerminalRunStatus::Exited); + assert_eq!(row.runtime_status, Some(runtime_status)); + assert_eq!( + row.recovery_summary + .as_ref() + .map(|summary| summary.recovery_state.as_str()), + Some(recovery_state) + ); + assert!(!registry.is_live("terminal-stable")); + assert!(registry.exists("terminal-stable")); + assert!(registry.directory()[0] + .snapshot + .contains("retained final output")); + + let (sink, seen) = collect_messages(); + assert!( + registry + .attach( + "terminal-stable", + 77, + sink, + Some("final-lifecycle-attach".into()), + 0, + false, + false, + None, + None, + None, + PacedAttachOptions::default(), + ) + .found + ); + let frames = seen.lock().unwrap(); + assert!(frames.iter().any(|frame| matches!(frame, + ServerMessage::TerminalOutput(output) if output.data.contains("retained final output")))); + assert!(frames.iter().any(|frame| matches!(frame, + ServerMessage::TerminalExit(exit) + if exit.managed_retained == Some(true) && exit.exit_code_available.is_none()))); + } +} + +#[test] +fn managed_completion_after_ended_lifecycle_publishes_the_observed_exit_code() { + let (registry, _) = fixture(); + let (sink, seen) = collect_messages(); + assert!( + registry + .attach( + "terminal-stable", + 78, + sink, + Some("completion-after-lifecycle".into()), + 206, + false, + false, + None, + None, + None, + PacedAttachOptions::default(), + ) + .found + ); + seen.lock().unwrap().clear(); + + registry.publish_managed_lifecycle( + "terminal-stable", + Some(lifecycle_with_recovery_state( + ManagedLifecycleState::Ended, + "lost", + "running", + )), + ); + assert!(!registry.is_live("terminal-stable")); + assert!(!seen + .lock() + .unwrap() + .iter() + .any(|frame| matches!(frame, ServerMessage::TerminalExit(_)))); + + assert!(registry.finish_managed_exit("terminal-stable", 17)); + assert!(seen.lock().unwrap().iter().any(|frame| matches!(frame, + ServerMessage::TerminalExit(exit) + if exit.exit_code == 17 + && exit.exit_code_available == Some(true) + && exit.managed_retained == Some(true)))); +} + +#[test] +fn managed_blocked_lifecycle_can_transition_through_recovery_back_to_running() { + let (registry, _) = fixture(); + registry.publish_managed_lifecycle( + "terminal-stable", + Some(lifecycle_with_recovery_state( + ManagedLifecycleState::Blocked, + "blocked", + "running", + )), + ); + assert!(!registry.is_live("terminal-stable")); + + registry.publish_managed_lifecycle( + "terminal-stable", + Some(lifecycle_with_recovery_state( + ManagedLifecycleState::Recovering, + "recovering", + "running", + )), + ); + assert!(registry.is_live("terminal-stable")); + assert_eq!( + registry.inventory()[0].runtime_status, + Some(freshell_protocol::RuntimeStatus::Recovering) + ); + + registry.publish_managed_lifecycle( + "terminal-stable", + Some(lifecycle_with_recovery_state( + ManagedLifecycleState::Running, + "live", + "running", + )), + ); + let row = ®istry.inventory()[0]; + assert!(registry.is_live("terminal-stable")); + assert_eq!(row.status, TerminalRunStatus::Running); + assert_eq!( + row.runtime_status, + Some(freshell_protocol::RuntimeStatus::Running) + ); + assert_eq!( + row.recovery_summary.as_ref().unwrap().recovery_state, + "live" + ); +} diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index e9ce4d794..74144ce4c 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -5044,10 +5044,15 @@ impl TerminalRegistry { { return; } - if lifecycle.state == ManagedLifecycleState::Running { - state.status = TerminalRunStatus::Running; - state.exit_code = None; - state.managed_exit_code_available = None; + match lifecycle.state { + ManagedLifecycleState::Running | ManagedLifecycleState::Recovering => { + state.status = TerminalRunStatus::Running; + state.exit_code = None; + state.managed_exit_code_available = None; + } + ManagedLifecycleState::Blocked | ManagedLifecycleState::Ended => { + state.status = TerminalRunStatus::Exited; + } } let summary = lifecycle.recovery_summary.clone(); let message = ServerMessage::TerminalStatus(freshell_protocol::TerminalStatus { @@ -5108,7 +5113,12 @@ impl TerminalRegistry { }; let mut state = shared.lock().expect("terminal lock"); if state.status == TerminalRunStatus::Exited { - return true; + match (state.managed_exit_code_available, code) { + (Some(true), _) | (Some(false), None) => return true, + // A lifecycle-only final verdict has no exit-code evidence. + // Allow a later completion observation to publish the code. + (Some(false), Some(_)) | (None, _) => {} + } } state.status = TerminalRunStatus::Exited; state.exit_code = Some(exit_code); From ccfa16f86f929b70ab3ad8adcb7ecb45965f00df Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 23:04:34 -0700 Subject: [PATCH 366/493] fix(managed): broadcast final lifecycle changes --- .../src/managed_output_tests.rs | 4 +- crates/freshell-terminal/src/registry.rs | 13 ++- .../src/managed_retained_socket_tests.rs | 7 +- crates/freshell-ws/src/terminal.rs | 98 +++++++++++++++++-- 4 files changed, 106 insertions(+), 16 deletions(-) diff --git a/crates/freshell-terminal/src/managed_output_tests.rs b/crates/freshell-terminal/src/managed_output_tests.rs index d673eee01..0d8b8755c 100644 --- a/crates/freshell-terminal/src/managed_output_tests.rs +++ b/crates/freshell-terminal/src/managed_output_tests.rs @@ -467,7 +467,9 @@ fn final_managed_lifecycle_marks_facade_exited_without_inventing_exit_code() { ServerMessage::TerminalOutput(output) if output.data.contains("retained final output")))); assert!(frames.iter().any(|frame| matches!(frame, ServerMessage::TerminalExit(exit) - if exit.managed_retained == Some(true) && exit.exit_code_available.is_none()))); + if exit.managed_retained == Some(true) + && exit.exit_code == -1 + && exit.exit_code_available == Some(false)))); } } diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 74144ce4c..8ae520509 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -5032,17 +5032,17 @@ impl TerminalRegistry { &self, terminal_id: &str, lifecycle: Option<ManagedLifecycle>, - ) { + ) -> bool { let Some(lifecycle) = lifecycle else { - return; + return false; }; let Some(shared) = self.shared_for(terminal_id) else { - return; + return false; }; let mut state = shared.lock().expect("terminal lock"); if state.managed_identity.is_none() || state.managed_lifecycle.as_ref() == Some(&lifecycle) { - return; + return false; } match lifecycle.state { ManagedLifecycleState::Running | ManagedLifecycleState::Recovering => { @@ -5052,6 +5052,10 @@ impl TerminalRegistry { } ManagedLifecycleState::Blocked | ManagedLifecycleState::Ended => { state.status = TerminalRunStatus::Exited; + if state.managed_exit_code_available != Some(true) { + state.exit_code = Some(-1); + state.managed_exit_code_available = Some(false); + } } } let summary = lifecycle.recovery_summary.clone(); @@ -5069,6 +5073,7 @@ impl TerminalRegistry { for subscriber in state.subscribers.values() { (subscriber.sink)(message.clone()); } + true } /// Error cause is classified before storage/logging; shared by every socket. diff --git a/crates/freshell-ws/src/managed_retained_socket_tests.rs b/crates/freshell-ws/src/managed_retained_socket_tests.rs index c377578f1..220690173 100644 --- a/crates/freshell-ws/src/managed_retained_socket_tests.rs +++ b/crates/freshell-ws/src/managed_retained_socket_tests.rs @@ -17,7 +17,12 @@ async fn managed_socket_windows( }); state .registry - .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { final_head }))); + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { + final_head, + lifecycle: None, + exit_code: Some(17), + native_session_id: Some("ses-short-lived".into()), + }))); let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let url = format!("ws://{}/ws", listener.local_addr().unwrap()); let token = state.auth_token.to_string(); diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index b41fd8774..ac28d23b1 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -8394,8 +8394,9 @@ use crate::terminal_meta::record_for_create as terminal_meta_record_for_create; /// revision}` to every authenticated connection via the broadcast bus. /// /// Wired call sites mirror the reference: `terminal.create` success/failed-delivery -/// (`ws:2553`/`ws:2570`) and valid `terminal.kill` (`ws:2988`). NOT wired: the -/// natural-exit path — the original broadcasts on exit only for +/// (`ws:2553`/`ws:2570`), valid `terminal.kill` (`ws:2988`), and managed lifecycle +/// projection changes so terminal-list views see final recovery status. NOT wired: +/// a plain local PTY natural-exit path — the original broadcasts on exit only for /// `recoverableForRestore` terminals (`ws:571-578`, session-repair subsystem, /// unported), and the live capture (`port/oracle/robustness/exit-orig.json`) shows /// no `terminals.changed` on a plain exit. `recoverableTerminalIds` never applies @@ -9016,7 +9017,7 @@ async fn refresh_managed_output_and_associate( crate::opencode_association::associate_managed_session(state, terminal_id, session_id) .await; } - state + let lifecycle_changed = state .registry .publish_managed_lifecycle(terminal_id, read.lifecycle); let delivered = state @@ -9046,6 +9047,9 @@ async fn refresh_managed_output_and_associate( state.registry.finish_managed_exit(terminal_id, exit_code); } } + if lifecycle_changed { + broadcast_terminals_changed(state); + } Ok(()) } @@ -13312,12 +13316,15 @@ mod pane_reconcile_gate_tests { mod managed_output_order_tests { use super::*; use freshell_terminal::registry::{ - ManagedOutputChunk, ManagedOutputRead, ManagedTerminalController, - ManagedTerminalDescriptor, ManagedTerminalFuture, + ManagedLifecycle, ManagedLifecycleState, ManagedOutputChunk, ManagedOutputRead, + ManagedTerminalController, ManagedTerminalDescriptor, ManagedTerminalFuture, }; struct ExitedOpenCodeRead { final_head: i64, + lifecycle: Option<ManagedLifecycle>, + exit_code: Option<i64>, + native_session_id: Option<String>, } impl ManagedTerminalController for ExitedOpenCodeRead { @@ -13374,7 +13381,7 @@ mod managed_output_order_tests { ) -> ManagedTerminalFuture<'a, Result<ManagedOutputRead, String>> { Box::pin(async move { Ok(ManagedOutputRead { - lifecycle: None, + lifecycle: self.lifecycle.clone(), completion: None, stream_epoch: Some("S-managed-short-lived".into()), incarnation_id: Some("incarnation-short-lived".into()), @@ -13382,8 +13389,8 @@ mod managed_output_order_tests { truncated: false, retained_from_seq: 1, head_seq: self.final_head, - exit_code: Some(17), - native_session_id: Some("ses-short-lived".into()), + exit_code: self.exit_code, + native_session_id: self.native_session_id.clone(), chunks: if after < self.final_head { vec![ManagedOutputChunk { seq_start: after + 1, @@ -13418,7 +13425,12 @@ mod managed_output_order_tests { }); state .registry - .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { final_head: 1 }))); + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { + final_head: 1, + lifecycle: None, + exit_code: Some(17), + native_session_id: Some("ses-short-lived".into()), + }))); refresh_managed_output_and_associate(&state, terminal_id, 1024) .await @@ -13459,7 +13471,12 @@ mod managed_output_order_tests { }); state .registry - .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { final_head: 3 }))); + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { + final_head: 3, + lifecycle: None, + exit_code: Some(17), + native_session_id: Some("ses-short-lived".into()), + }))); for delivered in 1..=3 { refresh_managed_output_and_associate(&state, terminal, 65536) .await @@ -13481,6 +13498,67 @@ mod managed_output_order_tests { assert_eq!(state.registry.directory()[0].snapshot.len(), 3 * 65536); } + #[tokio::test] + async fn managed_final_lifecycle_broadcasts_terminal_directory_changes_once() { + let state = super::pane_reconcile_gate_tests::state(); + let terminal_id = "T-managed-blocked"; + let mut events = state.broadcast_tx.subscribe(); + state.registry.register_managed(ManagedTerminalDescriptor { + soul_id: "soul-blocked".into(), + incarnation_id: "incarnation-blocked".into(), + terminal_id: terminal_id.into(), + stream_id: "S-managed-blocked".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: Some("ses-blocked".into()), + create_request_id: Some("req-blocked".into()), + }); + state + .registry + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { + final_head: 1, + lifecycle: Some(ManagedLifecycle { + state: ManagedLifecycleState::Blocked, + recovery_summary: Some(freshell_protocol::ManagedRecoverySummary { + desired_state: "running".into(), + recovery_state: "blocked".into(), + reason: Some("provider preparation failed".into()), + attempt_id: Some("attempt-blocked".into()), + incident_id: Some("incident-blocked".into()), + durability_state: "retained".into(), + allocation_state: "retained".into(), + }), + }), + exit_code: None, + native_session_id: None, + }))); + + refresh_managed_output_and_associate(&state, terminal_id, 1024) + .await + .expect("lifecycle refresh succeeds"); + assert_eq!( + state.registry.directory()[0].status, + freshell_protocol::TerminalRunStatus::Exited + ); + assert_eq!( + state.registry.directory()[0].runtime_status, + Some(freshell_protocol::RuntimeStatus::Blocked) + ); + let changed: serde_json::Value = serde_json::from_str(&events.try_recv().unwrap()).unwrap(); + assert_eq!( + changed, + serde_json::json!({ "type": "terminals.changed", "revision": 1 }) + ); + + refresh_managed_output_and_associate(&state, terminal_id, 1024) + .await + .expect("unchanged lifecycle refresh succeeds"); + assert!(matches!( + events.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + )); + } + include!("managed_retained_socket_tests.rs"); struct FailedRead(StdMutex<Option<String>>); From 5034cbf144e4facda00af69578f7122c904ecd70 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 23:16:17 -0700 Subject: [PATCH 367/493] fix(managed): notify exits after lifecycle-only outcomes --- .../src/managed_output_tests.rs | 56 ++++++++++ crates/freshell-terminal/src/registry.rs | 53 ++++++++- .../src/managed_retained_socket_tests.rs | 2 + crates/freshell-ws/src/terminal.rs | 105 ++++++++++++++++-- 4 files changed, 203 insertions(+), 13 deletions(-) diff --git a/crates/freshell-terminal/src/managed_output_tests.rs b/crates/freshell-terminal/src/managed_output_tests.rs index 0d8b8755c..d70e2d736 100644 --- a/crates/freshell-terminal/src/managed_output_tests.rs +++ b/crates/freshell-terminal/src/managed_output_tests.rs @@ -519,6 +519,62 @@ fn managed_completion_after_ended_lifecycle_publishes_the_observed_exit_code() { && exit.managed_retained == Some(true)))); } +#[test] +fn managed_unknown_completion_after_ended_lifecycle_notifies_attached_viewer_once() { + let (registry, _) = fixture(); + let (sink, seen) = collect_messages(); + assert!( + registry + .attach( + "terminal-stable", + 79, + sink, + Some("unknown-completion-after-lifecycle".into()), + 206, + false, + false, + None, + None, + None, + PacedAttachOptions::default(), + ) + .found + ); + seen.lock().unwrap().clear(); + + registry.publish_managed_lifecycle( + "terminal-stable", + Some(lifecycle_with_recovery_state( + ManagedLifecycleState::Ended, + "lost", + "running", + )), + ); + assert!(!seen + .lock() + .unwrap() + .iter() + .any(|frame| matches!(frame, ServerMessage::TerminalExit(_)))); + + assert!(registry.finish_managed_completion("terminal-stable", None)); + { + let frames = seen.lock().unwrap(); + assert!(frames.iter().any(|frame| matches!(frame, + ServerMessage::TerminalExit(exit) + if exit.exit_code == -1 + && exit.exit_code_available == Some(false) + && exit.managed_retained == Some(true)))); + } + + seen.lock().unwrap().clear(); + assert!(registry.finish_managed_completion("terminal-stable", None)); + assert!(!seen + .lock() + .unwrap() + .iter() + .any(|frame| matches!(frame, ServerMessage::TerminalExit(_)))); +} + #[test] fn managed_blocked_lifecycle_can_transition_through_recovery_back_to_running() { let (registry, _) = fixture(); diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 8ae520509..7d5c2e3e8 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -583,6 +583,9 @@ struct TerminalShared { managed_identity: Option<(String, String)>, managed_lifecycle: Option<ManagedLifecycle>, managed_exit_code_available: Option<bool>, + /// Whether a completion event has already been fanned out to attached viewers. + /// Lifecycle-only final states set code availability without notifying exits. + managed_exit_notified: bool, managed_output_failure: Option<String>, /// The exit code from the last `terminal.exit` fan-out (kill or natural exit). /// `None` while `status == Running`. Kept so a client that attaches AFTER the @@ -949,6 +952,36 @@ pub struct ManagedOutputRead { pub chunks: Vec<ManagedOutputChunk>, } +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ManagedOutputRefreshError { + pub message: String, + /// A cached lifecycle verdict changed even though retained output was unreadable. + pub lifecycle_changed: bool, +} + +impl From<String> for ManagedOutputRefreshError { + fn from(message: String) -> Self { + Self { + message, + lifecycle_changed: false, + } + } +} + +impl From<&str> for ManagedOutputRefreshError { + fn from(message: &str) -> Self { + Self::from(message.to_string()) + } +} + +impl std::fmt::Display for ManagedOutputRefreshError { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter.write_str(&self.message) + } +} + +impl std::error::Error for ManagedOutputRefreshError {} + /// Async control seam implemented by `freshell-server` with /// `freshell-runtime-client`. Keeping the trait here lets the existing registry /// remain the browser-facing replay/fan-out owner while the web server has no PTY, @@ -2472,6 +2505,7 @@ impl TerminalRegistry { managed_identity: None, managed_lifecycle: None, managed_exit_code_available: None, + managed_exit_notified: false, managed_output_failure: None, terminal_id: terminal_id.clone(), stream_id: stream_id.clone(), @@ -4691,6 +4725,7 @@ impl TerminalRegistry { )), managed_lifecycle: None, managed_exit_code_available: None, + managed_exit_notified: false, managed_output_failure: None, terminal_id: descriptor.terminal_id.clone(), stream_id: descriptor.stream_id.clone(), @@ -4869,7 +4904,7 @@ impl TerminalRegistry { &self, terminal_id: &str, max_bytes: u64, - ) -> Result<ManagedOutputRead, String> { + ) -> Result<ManagedOutputRead, ManagedOutputRefreshError> { let (terminal, flag) = { let inner = self.inner.lock().expect("registry lock"); let handle = inner @@ -4917,11 +4952,14 @@ impl TerminalRegistry { { Ok(read) => read, Err(error) => { - self.publish_managed_lifecycle( + let lifecycle_changed = self.publish_managed_lifecycle( terminal_id, controller.lifecycle_observation(&terminal).await, ); - return Err(error); + return Err(ManagedOutputRefreshError { + message: error, + lifecycle_changed, + }); } }; let mut announcements = Vec::new(); @@ -5049,6 +5087,7 @@ impl TerminalRegistry { state.status = TerminalRunStatus::Running; state.exit_code = None; state.managed_exit_code_available = None; + state.managed_exit_notified = false; } ManagedLifecycleState::Blocked | ManagedLifecycleState::Ended => { state.status = TerminalRunStatus::Exited; @@ -5117,17 +5156,18 @@ impl TerminalRegistry { Arc::clone(&handle.shared) }; let mut state = shared.lock().expect("terminal lock"); - if state.status == TerminalRunStatus::Exited { + if state.status == TerminalRunStatus::Exited && state.managed_exit_notified { match (state.managed_exit_code_available, code) { (Some(true), _) | (Some(false), None) => return true, - // A lifecycle-only final verdict has no exit-code evidence. - // Allow a later completion observation to publish the code. + // Allow a later completion observation to publish a known code + // after a lifecycle-only verdict or an unknown-code completion. (Some(false), Some(_)) | (None, _) => {} } } state.status = TerminalRunStatus::Exited; state.exit_code = Some(exit_code); state.managed_exit_code_available = Some(code.is_some()); + state.managed_exit_notified = true; let now = now_ms(); state.last_activity_at = now; state.last_output_activity_at = now; @@ -5203,6 +5243,7 @@ impl TerminalRegistry { managed_identity: None, managed_lifecycle: None, managed_exit_code_available: None, + managed_exit_notified: false, managed_output_failure: None, terminal_id: opts.terminal_id.clone(), stream_id: opts.stream_id, diff --git a/crates/freshell-ws/src/managed_retained_socket_tests.rs b/crates/freshell-ws/src/managed_retained_socket_tests.rs index 220690173..3adf9b84d 100644 --- a/crates/freshell-ws/src/managed_retained_socket_tests.rs +++ b/crates/freshell-ws/src/managed_retained_socket_tests.rs @@ -22,6 +22,8 @@ async fn managed_socket_windows( lifecycle: None, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), + read_error: None, + error_lifecycle: None, }))); let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let url = format!("ws://{}/ws", listener.local_addr().unwrap()); diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index ac28d23b1..ce6ad9c50 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -8985,7 +8985,10 @@ async fn refresh_managed_output_and_associate( .refresh_managed_output(terminal_id, max_bytes), ) .await - .map_err(|_| "managed output read timed out".to_string()) + .map_err(|_| freshell_terminal::registry::ManagedOutputRefreshError { + message: "managed output read timed out".into(), + lifecycle_changed: false, + }) .and_then(|read| read); let read = match result { Ok(read) => { @@ -8995,13 +8998,17 @@ async fn refresh_managed_output_and_associate( read } Err(error) => { - let cause = if error.contains("timed out") || error.contains("timeout") { + if error.lifecycle_changed { + broadcast_terminals_changed(state); + } + let cause = if error.message.contains("timed out") || error.message.contains("timeout") + { "timeout" - } else if error.contains("identity") || error.contains("ownership") { + } else if error.message.contains("identity") || error.message.contains("ownership") { "identity" - } else if error.contains("frame") - || error.contains("sequence") - || error.contains("bounds") + } else if error.message.contains("frame") + || error.message.contains("sequence") + || error.message.contains("bounds") { "invalid_output" } else { @@ -9010,7 +9017,7 @@ async fn refresh_managed_output_and_associate( state .registry .observe_managed_output_failure(terminal_id, Some(cause)); - return Err(error); + return Err(error.message); } }; if let Some(session_id) = read.native_session_id.as_deref() { @@ -13325,9 +13332,18 @@ mod managed_output_order_tests { lifecycle: Option<ManagedLifecycle>, exit_code: Option<i64>, native_session_id: Option<String>, + read_error: Option<String>, + error_lifecycle: Option<ManagedLifecycle>, } impl ManagedTerminalController for ExitedOpenCodeRead { + fn lifecycle_observation<'a>( + &'a self, + _: &'a ManagedTerminalDescriptor, + ) -> ManagedTerminalFuture<'a, Option<ManagedLifecycle>> { + Box::pin(std::future::ready(self.error_lifecycle.clone())) + } + fn lookup_terminal<'a>( &'a self, _: &'a str, @@ -13379,6 +13395,10 @@ mod managed_output_order_tests { after: i64, _: u64, ) -> ManagedTerminalFuture<'a, Result<ManagedOutputRead, String>> { + if let Some(error) = &self.read_error { + let error = error.clone(); + return Box::pin(async move { Err(error) }); + } Box::pin(async move { Ok(ManagedOutputRead { lifecycle: self.lifecycle.clone(), @@ -13430,6 +13450,8 @@ mod managed_output_order_tests { lifecycle: None, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), + read_error: None, + error_lifecycle: None, }))); refresh_managed_output_and_associate(&state, terminal_id, 1024) @@ -13476,6 +13498,8 @@ mod managed_output_order_tests { lifecycle: None, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), + read_error: None, + error_lifecycle: None, }))); for delivered in 1..=3 { refresh_managed_output_and_associate(&state, terminal, 65536) @@ -13531,6 +13555,8 @@ mod managed_output_order_tests { }), exit_code: None, native_session_id: None, + read_error: None, + error_lifecycle: None, }))); refresh_managed_output_and_associate(&state, terminal_id, 1024) @@ -13559,6 +13585,71 @@ mod managed_output_order_tests { )); } + #[tokio::test] + async fn managed_cached_final_lifecycle_broadcasts_when_output_read_fails() { + let state = super::pane_reconcile_gate_tests::state(); + let terminal_id = "T-managed-blocked-read-error"; + let mut events = state.broadcast_tx.subscribe(); + state.registry.register_managed(ManagedTerminalDescriptor { + soul_id: "soul-blocked-read-error".into(), + incarnation_id: "incarnation-blocked-read-error".into(), + terminal_id: terminal_id.into(), + stream_id: "S-managed-blocked-read-error".into(), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: Some("ses-blocked-read-error".into()), + create_request_id: Some("req-blocked-read-error".into()), + }); + let blocked_lifecycle = ManagedLifecycle { + state: ManagedLifecycleState::Blocked, + recovery_summary: Some(freshell_protocol::ManagedRecoverySummary { + desired_state: "running".into(), + recovery_state: "blocked".into(), + reason: Some("host_unreachable".into()), + attempt_id: Some("attempt-blocked-read-error".into()), + incident_id: Some("incident-blocked-read-error".into()), + durability_state: "retained".into(), + allocation_state: "retained".into(), + }), + }; + state + .registry + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { + final_head: 0, + lifecycle: None, + exit_code: None, + native_session_id: None, + read_error: Some("managed output read failed".into()), + error_lifecycle: Some(blocked_lifecycle), + }))); + + assert!( + refresh_managed_output_and_associate(&state, terminal_id, 1024) + .await + .unwrap_err() + .contains("managed output read failed") + ); + assert_eq!( + state.registry.directory()[0].runtime_status, + Some(freshell_protocol::RuntimeStatus::Blocked) + ); + let changed: serde_json::Value = serde_json::from_str(&events.try_recv().unwrap()).unwrap(); + assert_eq!( + changed, + serde_json::json!({ "type": "terminals.changed", "revision": 1 }) + ); + + assert!( + refresh_managed_output_and_associate(&state, terminal_id, 1024) + .await + .is_err() + ); + assert!(matches!( + events.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + )); + } + include!("managed_retained_socket_tests.rs"); struct FailedRead(StdMutex<Option<String>>); From 962c40a192da3fb8314c21b12422dc67da52b313 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Mon, 12 Oct 2026 23:26:37 -0700 Subject: [PATCH 368/493] fix(managed): observe lifecycle after output timeout --- crates/freshell-terminal/src/registry.rs | 19 +++ .../src/managed_retained_socket_tests.rs | 1 + crates/freshell-ws/src/terminal.rs | 149 +++++++++++++----- 3 files changed, 128 insertions(+), 41 deletions(-) diff --git a/crates/freshell-terminal/src/registry.rs b/crates/freshell-terminal/src/registry.rs index 7d5c2e3e8..3fcd666a3 100644 --- a/crates/freshell-terminal/src/registry.rs +++ b/crates/freshell-terminal/src/registry.rs @@ -5061,6 +5061,25 @@ impl TerminalRegistry { Ok(read) } + /// Refresh a managed terminal's cached lifecycle without reading retained + /// output. This is used when the output transport times out before its + /// error path can publish the controller's cached recovery verdict. + pub async fn observe_managed_lifecycle(&self, terminal_id: &str) -> bool { + let Some(terminal) = self.managed_descriptor(terminal_id) else { + return false; + }; + let controller = self + .managed_controller + .read() + .expect("managed controller lock") + .clone(); + let Some(controller) = controller else { + return false; + }; + let lifecycle = controller.lifecycle_observation(&terminal).await; + self.publish_managed_lifecycle(terminal_id, lifecycle) + } + /// Managed equivalent of `finish_pty_exit`: the OS process is already /// gone and the supervisor-owned enclosure has been stopped; retain the /// facade/replay tail but mark it exited and tell attached viewers once. diff --git a/crates/freshell-ws/src/managed_retained_socket_tests.rs b/crates/freshell-ws/src/managed_retained_socket_tests.rs index 3adf9b84d..953f814d5 100644 --- a/crates/freshell-ws/src/managed_retained_socket_tests.rs +++ b/crates/freshell-ws/src/managed_retained_socket_tests.rs @@ -22,6 +22,7 @@ async fn managed_socket_windows( lifecycle: None, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), + read_pending: false, read_error: None, error_lifecycle: None, }))); diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index ce6ad9c50..ede548503 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -8978,18 +8978,28 @@ async fn refresh_managed_output_and_associate( if state.registry.managed_descriptor(terminal_id).is_none() { return Ok(()); } - let result = tokio::time::timeout( + let result = match tokio::time::timeout( std::time::Duration::from_millis(500), state .registry .refresh_managed_output(terminal_id, max_bytes), ) .await - .map_err(|_| freshell_terminal::registry::ManagedOutputRefreshError { - message: "managed output read timed out".into(), - lifecycle_changed: false, - }) - .and_then(|read| read); + { + Ok(read) => read, + Err(_) => { + let lifecycle_changed = tokio::time::timeout( + std::time::Duration::from_millis(100), + state.registry.observe_managed_lifecycle(terminal_id), + ) + .await + .unwrap_or(false); + Err(freshell_terminal::registry::ManagedOutputRefreshError { + message: "managed output read timed out".into(), + lifecycle_changed, + }) + } + }; let read = match result { Ok(read) => { state @@ -13332,6 +13342,7 @@ mod managed_output_order_tests { lifecycle: Option<ManagedLifecycle>, exit_code: Option<i64>, native_session_id: Option<String>, + read_pending: bool, read_error: Option<String>, error_lifecycle: Option<ManagedLifecycle>, } @@ -13395,6 +13406,9 @@ mod managed_output_order_tests { after: i64, _: u64, ) -> ManagedTerminalFuture<'a, Result<ManagedOutputRead, String>> { + if self.read_pending { + return Box::pin(std::future::pending()); + } if let Some(error) = &self.read_error { let error = error.clone(); return Box::pin(async move { Err(error) }); @@ -13429,6 +13443,49 @@ mod managed_output_order_tests { } } + fn install_blocked_read_controller( + state: &WsState, + suffix: &str, + read_pending: bool, + read_error: Option<&str>, + ) -> String { + let terminal_id = format!("T-managed-{suffix}"); + state.registry.register_managed(ManagedTerminalDescriptor { + soul_id: format!("soul-{suffix}"), + incarnation_id: format!("incarnation-{suffix}"), + terminal_id: terminal_id.clone(), + stream_id: format!("S-managed-{suffix}"), + mode: "opencode".into(), + cwd: "/workspace".into(), + resume_session_id: Some(format!("ses-{suffix}")), + create_request_id: Some(format!("req-{suffix}")), + }); + let blocked_lifecycle = ManagedLifecycle { + state: ManagedLifecycleState::Blocked, + recovery_summary: Some(freshell_protocol::ManagedRecoverySummary { + desired_state: "running".into(), + recovery_state: "blocked".into(), + reason: Some("host_unreachable".into()), + attempt_id: Some(format!("attempt-{suffix}")), + incident_id: Some(format!("incident-{suffix}")), + durability_state: "retained".into(), + allocation_state: "retained".into(), + }), + }; + state + .registry + .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { + final_head: 0, + lifecycle: None, + exit_code: None, + native_session_id: None, + read_pending, + read_error: read_error.map(str::to_owned), + error_lifecycle: Some(blocked_lifecycle), + }))); + terminal_id + } + #[tokio::test] async fn native_identity_and_exit_in_one_read_bind_before_exit() { let state = super::pane_reconcile_gate_tests::state(); @@ -13450,6 +13507,7 @@ mod managed_output_order_tests { lifecycle: None, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), + read_pending: false, read_error: None, error_lifecycle: None, }))); @@ -13498,6 +13556,7 @@ mod managed_output_order_tests { lifecycle: None, exit_code: Some(17), native_session_id: Some("ses-short-lived".into()), + read_pending: false, read_error: None, error_lifecycle: None, }))); @@ -13555,6 +13614,7 @@ mod managed_output_order_tests { }), exit_code: None, native_session_id: None, + read_pending: false, read_error: None, error_lifecycle: None, }))); @@ -13588,43 +13648,16 @@ mod managed_output_order_tests { #[tokio::test] async fn managed_cached_final_lifecycle_broadcasts_when_output_read_fails() { let state = super::pane_reconcile_gate_tests::state(); - let terminal_id = "T-managed-blocked-read-error"; + let terminal_id = install_blocked_read_controller( + &state, + "blocked-read-error", + false, + Some("managed output read failed"), + ); let mut events = state.broadcast_tx.subscribe(); - state.registry.register_managed(ManagedTerminalDescriptor { - soul_id: "soul-blocked-read-error".into(), - incarnation_id: "incarnation-blocked-read-error".into(), - terminal_id: terminal_id.into(), - stream_id: "S-managed-blocked-read-error".into(), - mode: "opencode".into(), - cwd: "/workspace".into(), - resume_session_id: Some("ses-blocked-read-error".into()), - create_request_id: Some("req-blocked-read-error".into()), - }); - let blocked_lifecycle = ManagedLifecycle { - state: ManagedLifecycleState::Blocked, - recovery_summary: Some(freshell_protocol::ManagedRecoverySummary { - desired_state: "running".into(), - recovery_state: "blocked".into(), - reason: Some("host_unreachable".into()), - attempt_id: Some("attempt-blocked-read-error".into()), - incident_id: Some("incident-blocked-read-error".into()), - durability_state: "retained".into(), - allocation_state: "retained".into(), - }), - }; - state - .registry - .set_managed_controller(Some(Arc::new(ExitedOpenCodeRead { - final_head: 0, - lifecycle: None, - exit_code: None, - native_session_id: None, - read_error: Some("managed output read failed".into()), - error_lifecycle: Some(blocked_lifecycle), - }))); assert!( - refresh_managed_output_and_associate(&state, terminal_id, 1024) + refresh_managed_output_and_associate(&state, &terminal_id, 1024) .await .unwrap_err() .contains("managed output read failed") @@ -13640,7 +13673,41 @@ mod managed_output_order_tests { ); assert!( - refresh_managed_output_and_associate(&state, terminal_id, 1024) + refresh_managed_output_and_associate(&state, &terminal_id, 1024) + .await + .is_err() + ); + assert!(matches!( + events.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + )); + } + + #[tokio::test] + async fn managed_cached_final_lifecycle_broadcasts_when_output_read_times_out() { + let state = super::pane_reconcile_gate_tests::state(); + let terminal_id = + install_blocked_read_controller(&state, "blocked-read-timeout", true, None); + let mut events = state.broadcast_tx.subscribe(); + + assert!( + refresh_managed_output_and_associate(&state, &terminal_id, 1024) + .await + .unwrap_err() + .contains("timed out") + ); + assert_eq!( + state.registry.directory()[0].runtime_status, + Some(freshell_protocol::RuntimeStatus::Blocked) + ); + let changed: serde_json::Value = serde_json::from_str(&events.try_recv().unwrap()).unwrap(); + assert_eq!( + changed, + serde_json::json!({ "type": "terminals.changed", "revision": 1 }) + ); + + assert!( + refresh_managed_output_and_associate(&state, &terminal_id, 1024) .await .is_err() ); From 27da07837d94d1df0f4f7310b401405e45f14565 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:14:47 -0700 Subject: [PATCH 369/493] Bound OpenCode MCP command resources --- .../opencode_resources.rs | 38 ++++++++++++++----- .../managed_provider_references/sources.rs | 23 +++++++++++ 2 files changed, 52 insertions(+), 9 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 22020de21..8892b9129 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -932,6 +932,34 @@ pub fn select( canonical.clone(), field, )?; + let command_parent = canonical + .parent() + .ok_or_else(|| fail(field, "command has no containing scope"))?; + if let Some(package_root) = command_parent + .ancestors() + .take_while(|ancestor| *ancestor != home && ancestor.parent().is_some()) + .find(|ancestor| ancestor.join("package.json").is_file()) + { + let package_root = package_root.to_owned(); + insert_resource( + &mut result, + &mut resource_path_bytes, + package_root.clone(), + field, + )?; + insert_installed_package_ancestor_resources( + &package_root, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + // Without a nearby package manifest, the executable is the only + // bounded resource we can infer. Selecting its arbitrary parent + // can copy system directories such as /usr/bin into the runtime. + continue; } if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { if !canonical.is_dir() { @@ -954,12 +982,7 @@ pub fn select( if global.is_some_and(|global| path.starts_with(global.join("plugins"))) { continue; } - let mut scope = if field == "OpenCode mcp.command" { - canonical - .parent() - .ok_or_else(|| fail(field, "command has no containing scope"))? - .to_owned() - } else if canonical.is_dir() { + let mut scope = if canonical.is_dir() { path.clone() } else { path.parent() @@ -978,9 +1001,6 @@ pub fn select( } } if scope == home || scope == Path::new("/") { - if field == "OpenCode mcp.command" { - continue; - } return Err(fail(field, "select a bounded plugin or package directory")); } insert_resource(&mut result, &mut resource_path_bytes, scope.clone(), field)?; diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 5ae4af919..9fc74cfcc 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1120,6 +1120,9 @@ mod tests { let home = owned.path().join("home"); let workspace = owned.path().join("workspace"); let tools = owned.path().join("outside-tools"); + let host_command_dir = owned.path().join("host-system-bin"); + let host_command = host_command_dir.join("env"); + let unrelated_host_command = host_command_dir.join("unrelated-tool"); let absolute_script = tools.join("absolute-mcp.js"); let relative_script = workspace.join("scripts/relative-mcp.js"); let env_script = tools.join("env-mcp.js"); @@ -1134,10 +1137,13 @@ mod tests { std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&host_command_dir).unwrap(); std::fs::create_dir_all(relative_script.parent().unwrap()).unwrap(); std::fs::create_dir_all(cwd_script.parent().unwrap()).unwrap(); for script in [ &absolute_script, + &host_command, + &unrelated_host_command, &relative_script, &env_script, &symlink_target, @@ -1160,6 +1166,11 @@ mod tests { "command": ["node", absolute_script], "enabled": true, }, + "absolute-runtime": { + "type": "local", + "command": [host_command], + "enabled": true, + }, "relative": { "type": "local", "command": ["node", "scripts/relative-mcp.js"], @@ -1243,6 +1254,18 @@ mod tests { script.display() ); } + assert!( + selected.contains(&host_command.canonicalize().unwrap()), + "an absolute MCP runtime executable remains available to the managed child" + ); + assert!( + !selected.contains(&host_command_dir.canonicalize().unwrap()), + "an absolute system executable must not select its unbounded containing directory" + ); + assert!( + !selected.contains(&unrelated_host_command.canonicalize().unwrap()), + "an absolute system executable must not copy unrelated sibling commands" + ); assert!( !selected.contains(&trailing_decoy_script.canonicalize().unwrap()), "arguments after the selected script are values, not additional scripts" From 6d391a98c174fdcd6efd4e586149ad6bdc0714ca Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:14:54 -0700 Subject: [PATCH 370/493] Preserve blocked managed terminal status --- src/components/TerminalView.tsx | 32 +++++++++++++++---- src/lib/managed-terminal.ts | 11 +++++++ src/lib/session-type-utils.ts | 4 +-- src/store/tabsSlice.ts | 10 +++--- .../TerminalView.exitBanner.test.tsx | 26 ++++++++++++--- .../client/lib/session-type-utils.test.ts | 30 +++++++++++++++++ test/unit/client/store/tabsSlice.test.ts | 31 ++++++++++++++++++ 7 files changed, 128 insertions(+), 16 deletions(-) diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index 5ca277359..905c76f2d 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -5970,12 +5970,23 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te }, msg.status === 'running' || msg.status === 'recovering' ? msg.status : 'exited') updateContent({ status, runtimeStatus: msg.status, recoverySummary: msg.recoverySummary }) const statusTab = tabRef.current - if (statusTab) dispatch(updateTab({ id: statusTab.id, updates: { status } })) + if (statusTab) dispatch(updateTab({ id: statusTab.id, updates: { + status, + runtimeStatus: msg.status, + recoverySummary: msg.recoverySummary, + } })) } else if (msg.status === 'running' || msg.status === 'recovering') { - const status = managedTerminalPaneStatus(contentRef.current ?? {}, msg.status) + const terminal = contentRef.current + const status = managedTerminalPaneStatus(terminal ?? {}, msg.status) updateContent({ status }) const statusTab = tabRef.current - if (statusTab) dispatch(updateTab({ id: statusTab.id, updates: { status } })) + if (statusTab) dispatch(updateTab({ id: statusTab.id, updates: { + status, + ...(terminal?.soulId ? { + runtimeStatus: status === msg.status ? msg.status : terminal.runtimeStatus, + recoverySummary: terminal.recoverySummary, + } : {}), + } })) } } return @@ -6042,10 +6053,19 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te clearQuarantineRepair() dispatch(clearPaneRuntimeActivity({ paneId: paneIdRef.current })) discardPendingInput('terminal_gone') - const status = managedTerminalPaneStatus(contentRef.current ?? {}, 'exited') - updateContent({ status, runtimeStatus: 'exited' }) + const terminal = contentRef.current + const status = managedTerminalPaneStatus(terminal ?? {}, 'exited') + const runtimeStatus = terminal?.runtimeStatus === 'blocked' + || terminal?.recoverySummary?.recoveryState === 'blocked' + ? 'blocked' + : 'exited' + updateContent({ status, runtimeStatus }) const exitTab = tabRef.current - if (exitTab) dispatch(updateTab({ id: exitTab.id, updates: { status } })) + if (exitTab) dispatch(updateTab({ id: exitTab.id, updates: { + status, + runtimeStatus, + recoverySummary: terminal?.recoverySummary, + } })) return } diff --git a/src/lib/managed-terminal.ts b/src/lib/managed-terminal.ts index 161e996ad..abcd05a99 100644 --- a/src/lib/managed-terminal.ts +++ b/src/lib/managed-terminal.ts @@ -3,6 +3,7 @@ import type { TerminalStatus } from '../store/types.js' /** A retained process ending does not erase the supervisor's final verdict. */ export function managedTerminalPaneStatus(terminal: ManagedRuntimeProjectionFields, fallback: TerminalStatus): TerminalStatus { + if (terminal.runtimeStatus === 'blocked') return 'error' if (!terminal.soulId || !terminal.recoverySummary) return fallback const { recoveryState, desiredState } = terminal.recoverySummary if (recoveryState === 'blocked' || recoveryState === 'lost') return 'error' @@ -10,6 +11,16 @@ export function managedTerminalPaneStatus(terminal: ManagedRuntimeProjectionFiel return fallback } +/** Status for a pane reconstructed from a retained managed terminal projection. */ +export function managedTerminalResumePaneStatus(terminal: ManagedRuntimeProjectionFields): TerminalStatus { + const fallback = terminal.runtimeStatus === 'exited' + ? 'exited' + : terminal.runtimeStatus === 'recovering' + ? 'recovering' + : 'running' + return managedTerminalPaneStatus(terminal, fallback) +} + export function managedTerminalRecoveryLabel(terminal: ManagedRuntimeProjectionFields): string | undefined { if (!terminal.soulId) return undefined switch (terminal.recoverySummary?.recoveryState) { diff --git a/src/lib/session-type-utils.ts b/src/lib/session-type-utils.ts index e7e5ef688..8c44e4532 100644 --- a/src/lib/session-type-utils.ts +++ b/src/lib/session-type-utils.ts @@ -1,5 +1,5 @@ import type { ManagedRuntimeProjectionFields } from '../../shared/managed-runtime.js' -import { managedTerminalProjection } from './managed-terminal.js' +import { managedTerminalProjection, managedTerminalResumePaneStatus } from './managed-terminal.js' import type { ComponentType } from 'react' import { PROVIDER_ICONS, DefaultProviderIcon } from '@/components/icons/provider-icons' import { isNonShellMode, getProviderLabel } from '@/lib/coding-cli-utils' @@ -256,7 +256,7 @@ export function buildResumeContent(opts: { ? { terminalId: opts.liveTerminal.terminalId, serverInstanceId: opts.liveTerminal.serverInstanceId, - status: opts.liveTerminal.runtimeStatus === 'exited' ? 'exited' as const : 'running' as const, + status: managedTerminalResumePaneStatus(opts.liveTerminal), ...managedTerminalProjection(opts.liveTerminal), } : {}), diff --git a/src/store/tabsSlice.ts b/src/store/tabsSlice.ts index 94c5ecf71..bab9e5e9b 100644 --- a/src/store/tabsSlice.ts +++ b/src/store/tabsSlice.ts @@ -15,7 +15,7 @@ import { findTabIdForSession, collectSessionRefsFromTabs, liveTerminalRowIdentit import { getProviderLabel } from '@/lib/coding-cli-utils' import { basenameSegment } from '@shared/path-basename' import { buildResumeContent } from '@/lib/session-type-utils' -import { managedTerminalProjection } from '@/lib/managed-terminal' +import { managedTerminalProjection, managedTerminalResumePaneStatus } from '@/lib/managed-terminal' import { getFreshAgentProviderConfig, getFreshAgentProviderLabel } from '@/lib/fresh-agent-provider-utils' import { resolveFreshAgentType } from '@/lib/fresh-agent-registry' import { recordClosedTabSnapshot, pushReopenEntry, popReopenEntry } from './tabRegistrySlice' @@ -359,6 +359,7 @@ export const tabsSlice = createSlice({ description: payload.description, codingCliProvider, status: payload.status || 'creating', + runtimeStatus: payload.runtimeStatus, mode: payload.mode || 'shell', shell: payload.shell || 'system', initialCwd: payload.initialCwd, @@ -1826,13 +1827,14 @@ export const openSessionTab = createAsyncThunk( return } } - // Running terminals are always terminal panes (fresh-agent uses SDK, not PTY) + // Retained terminals are always terminal panes (fresh-agent uses SDK, not PTY). const tabId = nanoid() + const status = managedTerminalResumePaneStatus(managedRuntime ?? {}) dispatch(addTab({ id: tabId, // Coding agents name by working directory; provider label is the fallback. title: title || (cwd ? basenameSegment(cwd) : null) || getProviderLabel(resolvedProvider, extensions), - status: managedRuntime?.runtimeStatus === 'exited' ? 'exited' : 'running', + status, ...managedTerminalProjection(managedRuntime), mode: resolvedProvider, codingCliProvider: resolvedProvider, @@ -1849,7 +1851,7 @@ export const openSessionTab = createAsyncThunk( serverInstanceId: localServerInstanceId, sessionRef: liveTerminalOnly ? undefined : desiredResumeContent.sessionRef, initialCwd: cwd, - status: managedRuntime?.runtimeStatus === 'exited' ? 'exited' : 'running', + status, ...managedTerminalProjection(managedRuntime), }, })) diff --git a/test/unit/client/components/TerminalView.exitBanner.test.tsx b/test/unit/client/components/TerminalView.exitBanner.test.tsx index b578bac2e..61c5c28cc 100644 --- a/test/unit/client/components/TerminalView.exitBanner.test.tsx +++ b/test/unit/client/components/TerminalView.exitBanner.test.tsx @@ -2,7 +2,7 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import { act, render, screen, fireEvent, cleanup } from '@testing-library/react' import { configureStore } from '@reduxjs/toolkit' import { Provider, useSelector } from 'react-redux' -import tabsReducer from '@/store/tabsSlice' +import tabsReducer, { updateTab } from '@/store/tabsSlice' import panesReducer from '@/store/panesSlice' import settingsReducer, { defaultSettings } from '@/store/settingsSlice' import connectionReducer from '@/store/connectionSlice' @@ -383,13 +383,18 @@ describe('TerminalView exited-pane error banner', () => { expect(paneState(store)).toMatchObject({ status, runtimeStatus, recoverySummary: summary, terminalId: 'retained-managed', streamId: 'retained-epoch', createRequestId: REQ, sessionRef: { provider: 'codex', sessionId: SESSION_ID } }) - expect(store.getState().tabs.tabs[0].status).toBe(status) + expect(store.getState().tabs.tabs[0]).toMatchObject({ status, runtimeStatus, recoverySummary: summary }) }) it.each([ ['blocked', 'error'], ['lost', 'error'], ['stopped', 'exited'], ] as const)('keeps an authoritative %s final status when an unqualified running frame arrives later', async (recoveryState, status) => { const { store, paneContent } = makeStore({ mode: 'codex', status, recoveryState }) + act(() => store.dispatch(updateTab({ id: TAB, updates: { + soulId: 'managed-soul', + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', + recoverySummary: paneContent.recoverySummary, + } }))) act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { ...paneContent, terminalId: 'retained-managed', streamId: 'retained-epoch', runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', @@ -397,8 +402,12 @@ describe('TerminalView exited-pane error banner', () => { await renderPane(store, paneState(store)) act(() => messageHandler!({ type: 'terminal.status', terminalId: 'retained-managed', status: 'running' })) expect(paneState(store)).toMatchObject({ status, recoverySummary: { recoveryState }, + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', terminalId: 'retained-managed', streamId: 'retained-epoch', createRequestId: REQ }) - expect(store.getState().tabs.tabs[0].status).toBe(status) + expect(store.getState().tabs.tabs[0]).toMatchObject({ + status, + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', + }) }) it.each([ @@ -406,6 +415,11 @@ describe('TerminalView exited-pane error banner', () => { ] as const)('retains the %s final status when replay completes with unavailable exit evidence', async (recoveryState, status) => { const { store, paneContent } = makeStore({ mode: 'codex', status, recoveryState, lifecycle: { lastTerminalId: 'retained-managed', exit: { exitCode: 9, at: Date.now() } } }) + act(() => store.dispatch(updateTab({ id: TAB, updates: { + soulId: 'managed-soul', + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', + recoverySummary: paneContent.recoverySummary, + } }))) act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { ...paneContent, terminalId: 'retained-managed', streamId: 'retained-epoch', runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', @@ -414,8 +428,12 @@ describe('TerminalView exited-pane error banner', () => { act(() => messageHandler!({ type: 'terminal.exit', terminalId: 'retained-managed', exitCode: -1, managedRetained: true, exitCodeAvailable: false })) expect(paneState(store)).toMatchObject({ status, recoverySummary: { recoveryState }, + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', terminalId: 'retained-managed', streamId: 'retained-epoch', createRequestId: REQ }) - expect(store.getState().tabs.tabs[0].status).toBe(status) + expect(store.getState().tabs.tabs[0]).toMatchObject({ + status, + runtimeStatus: recoveryState === 'blocked' ? 'blocked' : 'exited', + }) expect(selectExitRecordFrom(store.getState().terminalLifecycle, PANE)).toBeUndefined() expect(screen.queryByText(/code -1/)).toBeNull() }) diff --git a/test/unit/client/lib/session-type-utils.test.ts b/test/unit/client/lib/session-type-utils.test.ts index 9195ce3a1..ecef5c0fc 100644 --- a/test/unit/client/lib/session-type-utils.test.ts +++ b/test/unit/client/lib/session-type-utils.test.ts @@ -179,6 +179,36 @@ describe('buildResumeContent', () => { expect('liveTerminal' in content).toBe(false) }) + it('preserves the blocked state when building a resumed managed terminal pane', () => { + const recoverySummary = { + desiredState: 'running' as const, + recoveryState: 'blocked' as const, + reason: 'host_unreachable', + durabilityState: 'resume_captured' as const, + allocationState: 'verified_durable' as const, + } + const content = buildResumeContent({ + sessionType: 'codex', + sessionId: 'blocked-session', + liveTerminal: { + terminalId: 'term-blocked', + serverInstanceId: 'srv-local', + soulId: 'blocked-soul', + incarnationId: 'blocked-incarnation', + runtimeStatus: 'blocked', + recoverySummary, + }, + }) + + expect(content).toMatchObject({ + kind: 'terminal', + terminalId: 'term-blocked', + status: 'error', + runtimeStatus: 'blocked', + recoverySummary, + }) + }) + it('fresh-agent panes have no terminalId', () => { const content = buildResumeContent({ sessionType: 'freshclaude', diff --git a/test/unit/client/store/tabsSlice.test.ts b/test/unit/client/store/tabsSlice.test.ts index c60078de1..7580562fd 100644 --- a/test/unit/client/store/tabsSlice.test.ts +++ b/test/unit/client/store/tabsSlice.test.ts @@ -1477,6 +1477,37 @@ describe('tabsSlice', () => { } }) + it('reopens a blocked managed terminal with its blocked status', async () => { + const store = createOpenSessionStore() + const recoverySummary = { + desiredState: 'running' as const, + recoveryState: 'blocked' as const, + reason: 'host_unreachable', + durabilityState: 'resume_captured' as const, + allocationState: 'verified_durable' as const, + } + + await store.dispatch(openSessionTab({ + sessionId: 'blocked-session', + provider: 'codex', + terminalId: 'term-blocked', + managedRuntime: { + soulId: 'blocked-soul', + incarnationId: 'blocked-incarnation', + runtimeStatus: 'blocked', + recoverySummary, + }, + })) + + const state = store.getState() + const tab = state.tabs.tabs[0] + expect(tab).toMatchObject({ status: 'error', soulId: 'blocked-soul', runtimeStatus: 'blocked', recoverySummary }) + expect(state.panes.layouts[tab.id]).toMatchObject({ + type: 'leaf', + content: { status: 'error', soulId: 'blocked-soul', runtimeStatus: 'blocked', recoverySummary }, + }) + }) + it('uses capitalized provider label for codex tab title', async () => { const store = configureStore({ reducer: { From bea16e8131d45fcd738cb45cd3024b4f2ba2d22f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:34:10 -0700 Subject: [PATCH 371/493] test: stabilize checkpoint browser readiness --- .../testing/managed-provider-environment-target.ts | 4 ++-- .../specs/code-checkpoint-workspace-external.spec.ts | 11 ++++++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index b9c9c9e04..24941fc74 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1038,8 +1038,8 @@ export async function runCodeCheckpointTarget() { const browserExit = await bounded(browserJoin, 120_000, 'configured Cloud checkpoint browser final join') assert.equal(browserExit.code, 0, 'Configured Cloud checkpoint browser acceptance failed') const browserSummary = browserSummaryCapture.finish() - assert.equal(browserSummary.executedCases, 6, 'Configured Cloud must execute all six checkpoint browser cases') - assert.equal(browserSummary.passedCases, 6, 'Every configured checkpoint browser case must pass') + assert.equal(browserSummary.executedCases, 5, 'Configured Cloud must execute all five checkpoint browser cases') + assert.equal(browserSummary.passedCases, 5, 'Every configured checkpoint browser case must pass') assert.equal(browserSummary.failedCases, 0) assert.equal(browserSummary.skippedCases, 0) assert.equal(browserSummary.interruptedCases, 0) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 67107685a..8fe5f7641 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -116,11 +116,16 @@ async function launchCodex(page: import('@playwright/test').Page, workspace: str const paneRoot = page.locator('[data-context="fresh-agent"]').last() await expect(paneRoot).toBeVisible({ timeout: 30_000 }) await expect.poll(async () => (await currentFreshAgent(page)).content.initialCwd, { timeout: 30_000 }).toBe(workspace) - await expect.poll(async () => (await currentFreshAgent(page)).content.status, { timeout: 120_000 }).toBe('idle') await expect.poll(async () => { const content = (await currentFreshAgent(page)).content - return content.sessionRef?.sessionId ?? content.sessionId - }, { timeout: 120_000 }).toBeTruthy() + return { + status: content.status, + sessionId: content.sessionRef?.sessionId ?? content.sessionId ?? null, + } + }, { timeout: 120_000 }).toMatchObject({ + status: 'idle', + sessionId: expect.any(String), + }) return { paneRoot, harness, ...(await currentFreshAgent(page)) } } From 87ffb02cb6c08ad902533909cdf20fbd586aff47 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:59:56 -0700 Subject: [PATCH 372/493] fix(managed): stage MCP script closure and refresh status --- .../managed_mcp_capability/source_stage.rs | 200 ++++++++++++++++++ .../opencode_resources.rs | 66 ++++-- .../managed_provider_references/sources.rs | 20 +- src/components/TerminalView.tsx | 2 +- .../TerminalView.exitBanner.test.tsx | 28 +++ 5 files changed, 292 insertions(+), 24 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 4562fa3d5..e21bfc416 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -34,6 +34,48 @@ mod configured_resource_tests { } } + fn copy_selected_units_through_provider( + owned_root: &Path, + units: &[( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + ) -> std::path::PathBuf { + let stage = owned_root.join("stage"); + let stage_selections = units + .iter() + .map(|(source, mount)| CopySelection { + source: source.clone(), + destination_relative: mount.stage_relative_path.clone().into(), + }) + .collect::<Vec<_>>(); + copy_environment(&stage_selections, &stage, &[], Default::default()).unwrap(); + + let host = owned_root.join("host"); + let host_selections = units + .iter() + .map(|(_, mount)| CopySelection { + source: stage.join(&mount.stage_relative_path), + destination_relative: mount.provider_relative_path.clone().into(), + }) + .collect::<Vec<_>>(); + copy_environment(&host_selections, &host, &[], Default::default()).unwrap(); + + let provider = owned_root.join("provider"); + let provider_selections = units + .iter() + .map(|(_, mount)| CopySelection { + source: host.join(&mount.provider_relative_path), + destination_relative: Path::new(&mount.effective_destination) + .strip_prefix(owned_root) + .unwrap() + .to_path_buf(), + }) + .collect::<Vec<_>>(); + copy_environment(&provider_selections, &provider, &[], Default::default()).unwrap(); + provider + } + #[test] fn managed_provider_keeps_external_indexed_payloads_out_of_persistent_generation() { fn tree_contains_bytes(path: &Path, expected: &[u8]) -> bool { @@ -582,6 +624,164 @@ mod configured_resource_tests { } } + #[test] + fn absolute_mcp_executable_runs_after_all_resource_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let command = external.join("mcp-executable"); + for path in [&home, &workspace, &external] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + &command, + "#!/bin/sh\nprintf 'absolute MCP executable started\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "absolute": { "type": "local", "command": [command] }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let canonical_command = command.canonicalize().unwrap(); + assert!(plan[0].configured_resources.contains(&canonical_command)); + assert!(!plan[0] + .configured_resources + .contains(&external.canonicalize().unwrap())); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(command.strip_prefix(owned.path()).unwrap()); + assert_ne!( + std::fs::metadata(&provider_command) + .unwrap() + .permissions() + .mode() + & 0o100, + 0, + "the managed command must remain executable by its owner after every provider resource copy" + ); + let hidden = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden).unwrap(); + let output = execute_selected_resource(&provider_command); + std::fs::rename(&hidden, &external).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the copied absolute MCP command must start: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"absolute MCP executable started\n"); + } + + #[test] + fn standalone_mcp_script_imports_sibling_after_all_resource_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let command = external.join("server.js"); + let helper = external.join("handler.js"); + for path in [&home, &workspace, &external] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write(&command, "process.stdout.write(require('./handler.js'))\n").unwrap(); + std::fs::write(&helper, "module.exports = 'sibling MCP helper loaded'\n").unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "standalone": { + "type": "local", + "command": ["node", command], + }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let canonical_external = external.canonicalize().unwrap(); + assert!( + plan[0].configured_resources.contains(&canonical_external), + "a standalone script must select its bounded sibling-import directory" + ); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(command.strip_prefix(owned.path()).unwrap()); + let provider_helper = provider.join(helper.strip_prefix(owned.path()).unwrap()); + assert!( + provider_helper.is_file(), + "the sibling import must be copied" + ); + + let hidden = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg(&provider_command) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output(); + std::fs::rename(&hidden, &external).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the staged MCP script must load its copied sibling: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"sibling MCP helper loaded"); + } + #[test] fn bun_package_script_dependency_in_parent_node_modules_runs_after_both_copies() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 8892b9129..40371b93d 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -252,9 +252,14 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' && !is_bun_run_bare_operand(&command, index) }) } else { - is_script_path(argument) + index == 0 && is_script_path(argument) }; - if (index == 0 && is_explicit_command_path(argument)) || script_file { + let script_operand = script_runtime + && script_index == Some(index) + && !is_bun_run_bare_operand(&command, index); + if script_file || script_operand { + result.push(("OpenCode mcp.script", argument.clone())); + } else if index == 0 && is_explicit_command_path(argument) { result.push(("OpenCode mcp.command", argument.clone())); } } @@ -262,8 +267,6 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' let script = &command[script_index]; if is_bun_run_bare_operand(&command, script_index) { result.push(("OpenCode bun run operand", script.clone())); - } else if !is_script_path(script) { - result.push(("OpenCode mcp.command", script.clone())); } } } @@ -769,7 +772,11 @@ pub fn select( continue; } } - field = "OpenCode mcp.command"; + field = if is_script_path(&spelling) { + "OpenCode mcp.script" + } else { + "OpenCode mcp.command" + }; } if field != "OpenCode attention.sounds" && field != "OpenCode mcp.cwd" @@ -795,7 +802,10 @@ pub fn select( } else { continue; } // Native package identity remains native. - } else if field == "OpenCode mcp.cwd" || field == "OpenCode mcp.command" { + } else if field == "OpenCode mcp.cwd" + || field == "OpenCode mcp.command" + || field == "OpenCode mcp.script" + { // Native path.resolve(instance cwd, raw): no HOME or URL expansion. PathBuf::from(&spelling) } else { @@ -866,7 +876,7 @@ pub fn select( .join(path); } else if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { path = source.instance_cwd.join(path); - } else if field == "OpenCode mcp.command" { + } else if field == "OpenCode mcp.command" || field == "OpenCode mcp.script" { path = if let Some(working_dir) = ¤t_mcp_working_dir { working_dir.join(path) } else { @@ -891,6 +901,7 @@ pub fn select( "OpenCode skills.paths" | "OpenCode mcp.cwd" | "OpenCode mcp.command" + | "OpenCode mcp.script" | "OpenCode attention.sounds" ) { // Native resolution resolution normalizes .. before deciding @@ -906,11 +917,11 @@ pub fn select( insert_resource(&mut result, &mut resource_path_bytes, path, field)?; continue; } - if field == "OpenCode mcp.command" { + if field == "OpenCode mcp.command" || field == "OpenCode mcp.script" { if !canonical.is_file() { return Err(fail( field, - "selected command resource is not a regular file", + "selected MCP command file is not a regular file", )); } // The server's selected cwd is already copied as a directory. @@ -924,14 +935,6 @@ pub fn select( { continue; } - // Retain the exact executable/script path as well as its bounded - // package closure below when it belongs to an installed package. - insert_resource( - &mut result, - &mut resource_path_bytes, - canonical.clone(), - field, - )?; let command_parent = canonical .parent() .ok_or_else(|| fail(field, "command has no containing scope"))?; @@ -955,10 +958,33 @@ pub fn select( &mut resource_path_bytes, field, )?; + continue; + } + if field == "OpenCode mcp.script" { + let canonical_home = std::fs::canonicalize(home).ok(); + if command_parent == Path::new("/") + || canonical_home.as_deref() == Some(command_parent) + { + return Err(fail( + field, + "standalone MCP script requires copying an unbounded account or system directory", + )); + } + // A standalone script may import sibling files without a + // package manifest. Copy its containing directory as the + // bounded module-resolution unit; copy_environment enforces + // the shared entry and byte caps before publication. + insert_resource( + &mut result, + &mut resource_path_bytes, + command_parent.to_owned(), + field, + )?; + continue; } - // Without a nearby package manifest, the executable is the only - // bounded resource we can infer. Selecting its arbitrary parent - // can copy system directories such as /usr/bin into the runtime. + // A runtime executable without a package manifest is an exact-file + // resource. Selecting its parent could copy system directories. + insert_resource(&mut result, &mut resource_path_bytes, canonical, field)?; continue; } if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 9fc74cfcc..a8393ff74 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1155,6 +1155,16 @@ mod tests { ] { std::fs::write(script, "process.stdout.write('mcp fixture')\n").unwrap(); } + std::fs::write(&host_command, "#!/bin/sh\nprintf 'mcp fixture\\n'\n").unwrap(); + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&host_command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let direct_command = std::process::Command::new(&host_command).output(); + assert!( + direct_command + .as_ref() + .is_ok_and(|output| output.status.success() && output.stdout == b"mcp fixture\n"), + "the absolute MCP executable fixture must actually start: {direct_command:?}" + ); std::os::unix::fs::symlink(&symlink_target, &workspace_symlink).unwrap(); let config = workspace.join("opencode.json"); std::fs::write( @@ -1248,9 +1258,13 @@ mod tests { &extensionless_args_script, &preload_script, ] { + let canonical_script = script.canonicalize().unwrap(); assert!( - selected.contains(&script.canonicalize().unwrap()), - "enabled local MCP command file must be available to the managed child: {}", + selected.iter().any(|resource| { + resource == &canonical_script + || (resource.is_dir() && canonical_script.starts_with(resource)) + }), + "enabled local MCP command script must be available through its selected resource: {}", script.display() ); } @@ -1530,7 +1544,7 @@ mod tests { assert!(plan[0] .configured_resources - .contains(&local_script.canonicalize().unwrap())); + .contains(&external.canonicalize().unwrap())); assert!(!plan[0] .configured_resources .iter() diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index 905c76f2d..8372a7990 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -5966,7 +5966,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te if (msg.terminalId === tid) { if (contentRef.current?.soulId && msg.recoverySummary) { const status = managedTerminalPaneStatus({ - ...contentRef.current, recoverySummary: msg.recoverySummary, + ...contentRef.current, runtimeStatus: msg.status, recoverySummary: msg.recoverySummary, }, msg.status === 'running' || msg.status === 'recovering' ? msg.status : 'exited') updateContent({ status, runtimeStatus: msg.status, recoverySummary: msg.recoverySummary }) const statusTab = tabRef.current diff --git a/test/unit/client/components/TerminalView.exitBanner.test.tsx b/test/unit/client/components/TerminalView.exitBanner.test.tsx index 61c5c28cc..4007603fb 100644 --- a/test/unit/client/components/TerminalView.exitBanner.test.tsx +++ b/test/unit/client/components/TerminalView.exitBanner.test.tsx @@ -386,6 +386,34 @@ describe('TerminalView exited-pane error banner', () => { expect(store.getState().tabs.tabs[0]).toMatchObject({ status, runtimeStatus, recoverySummary: summary }) }) + it('lets a fresh live recovery summary clear a previously blocked pane and tab status', async () => { + const { store, paneContent } = makeStore({ mode: 'codex', status: 'error', recoveryState: 'blocked' }) + act(() => store.dispatch(updateTab({ id: TAB, updates: { + soulId: 'managed-soul', + runtimeStatus: 'blocked', + recoverySummary: paneContent.recoverySummary, + } }))) + act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { + ...paneContent, + terminalId: 'retained-managed', + streamId: 'retained-epoch', + runtimeStatus: 'blocked', + } }))) + await renderPane(store, paneState(store)) + + const liveSummary = { ...paneContent.recoverySummary!, recoveryState: 'live' as const } + act(() => messageHandler!({ type: 'terminal.status', terminalId: 'retained-managed', + status: 'running', recoverySummary: liveSummary })) + + expect(paneState(store)).toMatchObject({ + status: 'running', runtimeStatus: 'running', recoverySummary: liveSummary, + terminalId: 'retained-managed', streamId: 'retained-epoch', + }) + expect(store.getState().tabs.tabs[0]).toMatchObject({ + status: 'running', runtimeStatus: 'running', recoverySummary: liveSummary, + }) + }) + it.each([ ['blocked', 'error'], ['lost', 'error'], ['stopped', 'exited'], ] as const)('keeps an authoritative %s final status when an unqualified running frame arrives later', async (recoveryState, status) => { From 3acbdf83a4c2c927ae7fedd62211717f1257203f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:13:22 -0700 Subject: [PATCH 373/493] fix(managed): preserve direct MCP executables and modules --- .../managed_mcp_capability/source_stage.rs | 157 +++++++++++++++++- .../opencode_resources.rs | 69 ++++++-- 2 files changed, 207 insertions(+), 19 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index e21bfc416..db7d6b461 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -68,8 +68,13 @@ mod configured_resource_tests { source: host.join(&mount.provider_relative_path), destination_relative: Path::new(&mount.effective_destination) .strip_prefix(owned_root) - .unwrap() - .to_path_buf(), + .map(Path::to_path_buf) + .or_else(|_| { + Path::new(&mount.effective_destination) + .strip_prefix("/home/freshell/provider") + .map(|relative| Path::new(".provider-home").join(relative)) + }) + .expect("test provider destinations must map into the temporary root"), }) .collect::<Vec<_>>(); copy_environment(&provider_selections, &provider, &[], Default::default()).unwrap(); @@ -704,6 +709,74 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"absolute MCP executable started\n"); } + #[test] + fn direct_executable_script_in_home_runs_after_all_resource_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.sh"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\nprintf 'home MCP executable started\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!(plan[0] + .configured_resources + .contains(&command.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/mcp.sh"); + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = execute_selected_resource(&provider_command); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the direct executable selected under HOME must still start: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"home MCP executable started\n"); + } + #[test] fn standalone_mcp_script_imports_sibling_after_all_resource_copies() { let owned = tempfile::tempdir().unwrap(); @@ -782,6 +855,86 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"sibling MCP helper loaded"); } + #[test] + fn standalone_node_mcp_script_loads_dependency_from_home_node_modules() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let script_dir = home.join("scripts"); + let node_modules = home.join("node_modules"); + let dependency = node_modules.join("parent-dependency"); + let command = script_dir.join("server.js"); + for path in [&script_dir, &dependency, &workspace] { + std::fs::create_dir_all(path).unwrap(); + } + std::fs::write( + &command, + "process.stdout.write(require('parent-dependency'))\n", + ) + .unwrap(); + std::fs::write( + dependency.join("index.js"), + "module.exports = 'ancestor MCP dependency loaded'\n", + ) + .unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "standalone": { "type": "local", "command": ["node", command] }, + }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!( + plan[0] + .configured_resources + .contains(&node_modules.canonicalize().unwrap()), + "the standalone Node script must select ancestor node_modules" + ); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/scripts/server.js"); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .arg(&provider_command) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap()) + .output(); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the managed Node script must resolve its copied ancestor dependency: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"ancestor MCP dependency loaded"); + } + #[test] fn bun_package_script_dependency_in_parent_node_modules_runs_after_both_copies() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 40371b93d..7a0d07516 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -243,6 +243,13 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' let script_index = script_operand_index(&command); let script_runtime = command_uses_script_runtime(command.first().map(String::as_str)); + let node_script_field = + if command_uses_node_module_resolution(command.first().map(String::as_str)) + { + "OpenCode mcp.node-script" + } else { + "OpenCode mcp.script" + }; for (index, argument) in command.iter().enumerate() { let script_file = if script_runtime { script_index.is_some_and(|script| { @@ -257,10 +264,10 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' let script_operand = script_runtime && script_index == Some(index) && !is_bun_run_bare_operand(&command, index); - if script_file || script_operand { - result.push(("OpenCode mcp.script", argument.clone())); - } else if index == 0 && is_explicit_command_path(argument) { + if index == 0 && is_explicit_command_path(argument) { result.push(("OpenCode mcp.command", argument.clone())); + } else if script_file || script_operand { + result.push((node_script_field, argument.clone())); } } if let Some(script_index) = script_index { @@ -416,6 +423,30 @@ fn command_uses_script_runtime(executable: Option<&str>) -> bool { ) } +fn command_uses_node_module_resolution(executable: Option<&str>) -> bool { + let Some(executable) = executable.and_then(|executable| Path::new(executable).file_name()) + else { + return false; + }; + let executable = executable.to_string_lossy().to_ascii_lowercase(); + let executable = executable.strip_suffix(".exe").unwrap_or(&executable); + matches!( + executable, + "node" | "nodejs" | "bun" | "deno" | "tsx" | "ts-node" + ) +} + +fn is_mcp_command_path_field(field: &str) -> bool { + matches!( + field, + "OpenCode mcp.command" | "OpenCode mcp.script" | "OpenCode mcp.node-script" + ) +} + +fn is_mcp_script_field(field: &str) -> bool { + matches!(field, "OpenCode mcp.script" | "OpenCode mcp.node-script") +} + fn script_operand_index(command: &[String]) -> Option<usize> { let executable = command .first()? @@ -773,7 +804,7 @@ pub fn select( } } field = if is_script_path(&spelling) { - "OpenCode mcp.script" + "OpenCode mcp.node-script" } else { "OpenCode mcp.command" }; @@ -802,10 +833,7 @@ pub fn select( } else { continue; } // Native package identity remains native. - } else if field == "OpenCode mcp.cwd" - || field == "OpenCode mcp.command" - || field == "OpenCode mcp.script" - { + } else if field == "OpenCode mcp.cwd" || is_mcp_command_path_field(field) { // Native path.resolve(instance cwd, raw): no HOME or URL expansion. PathBuf::from(&spelling) } else { @@ -876,7 +904,7 @@ pub fn select( .join(path); } else if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { path = source.instance_cwd.join(path); - } else if field == "OpenCode mcp.command" || field == "OpenCode mcp.script" { + } else if is_mcp_command_path_field(field) { path = if let Some(working_dir) = ¤t_mcp_working_dir { working_dir.join(path) } else { @@ -898,12 +926,9 @@ pub fn select( } if matches!( field, - "OpenCode skills.paths" - | "OpenCode mcp.cwd" - | "OpenCode mcp.command" - | "OpenCode mcp.script" - | "OpenCode attention.sounds" - ) { + "OpenCode skills.paths" | "OpenCode mcp.cwd" | "OpenCode attention.sounds" + ) || is_mcp_command_path_field(field) + { // Native resolution resolution normalizes .. before deciding // whether the selected resource is supplied by the workspace. path = freshell_platform::opencode_config::lexical_path(&path); @@ -917,7 +942,7 @@ pub fn select( insert_resource(&mut result, &mut resource_path_bytes, path, field)?; continue; } - if field == "OpenCode mcp.command" || field == "OpenCode mcp.script" { + if is_mcp_command_path_field(field) { if !canonical.is_file() { return Err(fail( field, @@ -960,7 +985,7 @@ pub fn select( )?; continue; } - if field == "OpenCode mcp.script" { + if is_mcp_script_field(field) { let canonical_home = std::fs::canonicalize(home).ok(); if command_parent == Path::new("/") || canonical_home.as_deref() == Some(command_parent) @@ -970,6 +995,16 @@ pub fn select( "standalone MCP script requires copying an unbounded account or system directory", )); } + if field == "OpenCode mcp.node-script" { + insert_installed_package_ancestor_resources( + command_parent, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } // A standalone script may import sibling files without a // package manifest. Copy its containing directory as the // bounded module-resolution unit; copy_environment enforces From 179b8031d38ce4efd175488f9529bb6cc8642f92 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:21:08 -0700 Subject: [PATCH 374/493] fix(managed): stage direct MCP script siblings --- .../managed_mcp_capability/source_stage.rs | 71 +++++++++++++++++++ .../opencode_resources.rs | 54 ++++++++++++-- 2 files changed, 121 insertions(+), 4 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index db7d6b461..cb620830a 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -777,6 +777,77 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"home MCP executable started\n"); } + #[test] + fn direct_mcp_shell_script_loads_sibling_after_home_copy() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start.sh"); + let helper = tools.join("helper.sh"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\n. \"$SCRIPT_DIR/helper.sh\"\nprintf '%s\\n' \"$MCP_HELPER_RESULT\"\n", + ) + .unwrap(); + std::fs::write(&helper, "MCP_HELPER_RESULT='sibling shell helper loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!( + plan[0] + .configured_resources + .contains(&tools.canonicalize().unwrap()), + "a direct MCP script must retain its bounded sibling-resource directory" + ); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/tools/start.sh"); + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = execute_selected_resource(&provider_command); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the directly launched managed script must load its sibling helper: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"sibling shell helper loaded\n"); + } + #[test] fn standalone_mcp_script_imports_sibling_after_all_resource_copies() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 7a0d07516..750b074e4 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -265,7 +265,16 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' && script_index == Some(index) && !is_bun_run_bare_operand(&command, index); if index == 0 && is_explicit_command_path(argument) { - result.push(("OpenCode mcp.command", argument.clone())); + if is_script_path(argument) { + let field = if is_node_script_path(argument) { + "OpenCode mcp.node-direct-script" + } else { + "OpenCode mcp.direct-script" + }; + result.push((field, argument.clone())); + } else { + result.push(("OpenCode mcp.command", argument.clone())); + } } else if script_file || script_operand { result.push((node_script_field, argument.clone())); } @@ -436,15 +445,48 @@ fn command_uses_node_module_resolution(executable: Option<&str>) -> bool { ) } +fn is_node_script_path(path: &str) -> bool { + Path::new(path).extension().is_some_and(|extension| { + matches!( + extension.to_string_lossy().to_ascii_lowercase().as_str(), + "js" | "mjs" | "cjs" | "ts" | "mts" | "cts" + ) + }) +} + fn is_mcp_command_path_field(field: &str) -> bool { matches!( field, - "OpenCode mcp.command" | "OpenCode mcp.script" | "OpenCode mcp.node-script" + "OpenCode mcp.command" + | "OpenCode mcp.script" + | "OpenCode mcp.node-script" + | "OpenCode mcp.direct-script" + | "OpenCode mcp.node-direct-script" ) } fn is_mcp_script_field(field: &str) -> bool { - matches!(field, "OpenCode mcp.script" | "OpenCode mcp.node-script") + matches!( + field, + "OpenCode mcp.script" + | "OpenCode mcp.node-script" + | "OpenCode mcp.direct-script" + | "OpenCode mcp.node-direct-script" + ) +} + +fn is_mcp_direct_script_field(field: &str) -> bool { + matches!( + field, + "OpenCode mcp.direct-script" | "OpenCode mcp.node-direct-script" + ) +} + +fn is_mcp_node_script_field(field: &str) -> bool { + matches!( + field, + "OpenCode mcp.node-script" | "OpenCode mcp.node-direct-script" + ) } fn script_operand_index(command: &[String]) -> Option<usize> { @@ -990,12 +1032,16 @@ pub fn select( if command_parent == Path::new("/") || canonical_home.as_deref() == Some(command_parent) { + if is_mcp_direct_script_field(field) { + insert_resource(&mut result, &mut resource_path_bytes, canonical, field)?; + continue; + } return Err(fail( field, "standalone MCP script requires copying an unbounded account or system directory", )); } - if field == "OpenCode mcp.node-script" { + if is_mcp_node_script_field(field) { insert_installed_package_ancestor_resources( command_parent, home, From 64890b1eb38f9aa28f28fa812b55fcdae6dc4483 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:56:01 -0700 Subject: [PATCH 375/493] fix(managed): stage direct MCP script dependencies --- .../managed_mcp_capability/source_stage.rs | 177 +++++- .../opencode_resources.rs | 512 +++++++++++++++++- .../managed-provider-environment-target.ts | 63 ++- ...aged-provider-environment-external.spec.ts | 6 + 4 files changed, 744 insertions(+), 14 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index cb620830a..ffadc79f2 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -630,7 +630,7 @@ mod configured_resource_tests { } #[test] - fn absolute_mcp_executable_runs_after_all_resource_copies() { + fn absolute_mcp_shebang_command_runs_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; let owned = tempfile::tempdir().unwrap(); @@ -719,11 +719,13 @@ mod configured_resource_tests { let command = home.join("mcp.sh"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); + let helper = home.join("helper.sh"); std::fs::write( &command, - "#!/bin/sh\nprintf 'home MCP executable started\\n'\n", + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\n. \"$SCRIPT_DIR/helper.sh\"\nprintf '%s\\n' \"$MCP_HELPER_RESULT\"\n", ) .unwrap(); + std::fs::write(&helper, "MCP_HELPER_RESULT='home sibling helper loaded'\n").unwrap(); std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); let config = workspace.join("opencode.json"); std::fs::write( @@ -750,6 +752,10 @@ mod configured_resource_tests { assert!(plan[0] .configured_resources .contains(&command.canonicalize().unwrap())); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap()), + "a direct script in HOME must retain its explicitly sourced sibling without selecting all of HOME"); assert!(!plan[0] .configured_resources .contains(&home.canonicalize().unwrap())); @@ -774,7 +780,166 @@ mod configured_resource_tests { "the direct executable selected under HOME must still start: {:?}", output.stderr ); - assert_eq!(output.stdout, b"home MCP executable started\n"); + assert_eq!(output.stdout, b"home sibling helper loaded\n"); + } + + #[test] + fn extensionless_mcp_shebang_script_loads_sibling_after_all_resource_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let helper = tools.join("helper.sh"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\n. \"$SCRIPT_DIR/helper.sh\"\nprintf '%s\\n' \"$MCP_HELPER_RESULT\"\n", + ) + .unwrap(); + std::fs::write( + &helper, + "MCP_HELPER_RESULT='extensionless sibling helper loaded'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!(plan[0] + .configured_resources + .contains(&command.canonicalize().unwrap())); + assert!( + plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap()), + "an extensionless executable shebang command must retain its sourced sibling" + ); + assert!( + !plan[0] + .configured_resources + .contains(&tools.canonicalize().unwrap()), + "a direct command must not copy unrelated siblings from its containing directory" + ); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/tools/start"); + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = execute_selected_resource(&provider_command); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the extensionless managed shebang command must load its sibling helper: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"extensionless sibling helper loaded\n"); + } + + #[test] + fn direct_node_script_in_home_loads_relative_module_after_all_resource_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("start"); + let helper = home.join("helper.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nprocess.stdout.write(require('./helper.js'))\n", + ) + .unwrap(); + std::fs::write( + &helper, + "module.exports = 'node sibling helper loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap()), + "an account-home Node script must retain its imported sibling without selecting all of HOME"); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/start"); + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = execute_selected_resource(&provider_command); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!( + output.status.success(), + "the managed Node command must load its selected relative module: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"node sibling helper loaded\n"); } #[test] @@ -1411,11 +1576,7 @@ mod configured_resource_tests { let projected = host.join(&unit.1.provider_relative_path).join("probe"); let hidden = owned.path().join("hidden-original-skills"); std::fs::rename(&skill_root, &hidden).unwrap(); - let output = std::process::Command::new(projected.join("scripts/run")) - .env_clear() - .env("PATH", std::env::var_os("PATH").unwrap()) - .output() - .unwrap(); + let output = execute_selected_resource(&projected.join("scripts/run")).unwrap(); std::fs::rename(&hidden, &skill_root).unwrap(); assert!(output.status.success(), "copied selected skill executable must consume installed dependencies and adjacent resources"); assert_eq!(output.stdout, b"skill-resource:skill-dependency"); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 750b074e4..de270c1da 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -462,6 +462,8 @@ fn is_mcp_command_path_field(field: &str) -> bool { | "OpenCode mcp.node-script" | "OpenCode mcp.direct-script" | "OpenCode mcp.node-direct-script" + | "OpenCode mcp.shebang-script" + | "OpenCode mcp.shebang-node-script" ) } @@ -472,23 +474,458 @@ fn is_mcp_script_field(field: &str) -> bool { | "OpenCode mcp.node-script" | "OpenCode mcp.direct-script" | "OpenCode mcp.node-direct-script" + | "OpenCode mcp.shebang-script" + | "OpenCode mcp.shebang-node-script" ) } fn is_mcp_direct_script_field(field: &str) -> bool { matches!( field, - "OpenCode mcp.direct-script" | "OpenCode mcp.node-direct-script" + "OpenCode mcp.direct-script" + | "OpenCode mcp.node-direct-script" + | "OpenCode mcp.shebang-script" + | "OpenCode mcp.shebang-node-script" ) } fn is_mcp_node_script_field(field: &str) -> bool { matches!( field, - "OpenCode mcp.node-script" | "OpenCode mcp.node-direct-script" + "OpenCode mcp.node-script" + | "OpenCode mcp.node-direct-script" + | "OpenCode mcp.shebang-node-script" + ) +} + +fn is_mcp_shebang_script_field(field: &str) -> bool { + matches!( + field, + "OpenCode mcp.shebang-script" | "OpenCode mcp.shebang-node-script" + ) +} + +fn shebang_interpreter_names(path: &Path) -> Option<Vec<String>> { + let mut prefix = [0; 4096]; + let file = std::fs::File::open(path).ok()?; + let length = file.take(prefix.len() as u64).read(&mut prefix).ok()?; + let first_line = prefix[..length].split(|byte| *byte == b'\n').next()?; + let shebang = first_line.strip_prefix(b"#!")?; + Some( + std::str::from_utf8(shebang) + .ok()? + .split_ascii_whitespace() + .filter_map(|part| Path::new(part).file_name()) + .map(|part| part.to_string_lossy().to_ascii_lowercase()) + .collect(), ) } +fn shebang_script_field(path: &Path) -> Option<&'static str> { + let interpreter_names = shebang_interpreter_names(path)?; + if interpreter_names + .iter() + .any(|name| matches!(name.as_str(), "node" | "nodejs" | "bun" | "deno")) + { + Some("OpenCode mcp.shebang-node-script") + } else { + Some("OpenCode mcp.shebang-script") + } +} + +fn is_shell_script(path: &Path) -> bool { + if path + .extension() + .is_some_and(|extension| extension.to_string_lossy().eq_ignore_ascii_case("sh")) + { + return true; + } + shebang_interpreter_names(path).is_some_and(|names| { + names.iter().any(|name| { + matches!( + name.as_str(), + "sh" | "bash" | "dash" | "zsh" | "ksh" | "ash" + ) + }) + }) +} + +fn shell_directory_variables(source: &str) -> std::collections::BTreeSet<String> { + source + .lines() + .filter_map(|line| { + let line = line + .trim_start() + .strip_prefix("export ") + .unwrap_or(line.trim_start()); + let (name, value) = line.split_once('=')?; + let valid_name = !name.is_empty() + && name + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_'); + (valid_name && value.contains("dirname") && value.contains("$0")) + .then(|| name.to_owned()) + }) + .collect() +} + +fn shell_source_operand(line: &str) -> Option<String> { + let line = line.trim_start(); + let rest = if let Some(rest) = line.strip_prefix("source") { + if !rest.starts_with(char::is_whitespace) { + return None; + } + rest + } else if let Some(rest) = line.strip_prefix('.') { + if !rest.starts_with(char::is_whitespace) { + return None; + } + rest + } else { + return None; + } + .trim_start(); + let first = rest.chars().next()?; + if first == '\'' || first == '"' { + let end = rest[1..].find(first)? + 1; + Some(rest[1..end].to_owned()) + } else { + rest.split(|character: char| character.is_whitespace() || ";|&".contains(character)) + .next() + .filter(|operand| !operand.is_empty()) + .map(str::to_owned) + } +} + +fn shell_source_path( + script_parent: &Path, + operand: &str, + directory_variables: &std::collections::BTreeSet<String>, +) -> Result<Option<PathBuf>, &'static str> { + if operand.starts_with('/') { + return Ok(None); + } + let mut relative = operand; + if let Some(variable) = operand.strip_prefix("${") { + let Some((name, suffix)) = variable.split_once('}') else { + return Ok(None); + }; + if !directory_variables.contains(name) { + return Err("direct MCP shell script has an unresolved dynamic source path"); + } + relative = suffix.strip_prefix('/').unwrap_or(suffix); + } else if let Some(variable) = operand.strip_prefix('$') { + let (name, suffix) = variable + .split_once('/') + .ok_or("direct MCP shell script has an unresolved dynamic source path")?; + if !directory_variables.contains(name) { + return Err("direct MCP shell script has an unresolved dynamic source path"); + } + relative = suffix; + } + if relative.is_empty() || relative.contains(['$', '*', '?']) { + return Err("direct MCP shell script has an unresolved dynamic source path"); + } + Ok(Some(script_parent.join(relative))) +} + +fn shell_directory_relative_paths( + line: &str, + directory_variables: &std::collections::BTreeSet<String>, +) -> std::collections::BTreeSet<String> { + let mut result = std::collections::BTreeSet::new(); + for variable in directory_variables { + for prefix in [format!("${variable}/"), format!("${{{variable}}}/")] { + let mut remaining = line; + while let Some(start) = remaining.find(&prefix) { + let suffix = &remaining[start + prefix.len()..]; + let length = suffix + .find(|character: char| { + character.is_whitespace() || "\"'`;|&)".contains(character) + }) + .unwrap_or(suffix.len()); + let relative = &suffix[..length]; + if !relative.is_empty() && !relative.contains(['$', '*', '?']) { + result.insert(relative.to_owned()); + } + remaining = &suffix[length..]; + if remaining.is_empty() { + break; + } + } + } + } + result +} + +enum JavaScriptToken { + Identifier(String), + String(Option<String>), + Punctuation(u8), +} + +fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> { + const MAX_TOKENS: usize = 100_000; + let bytes = source.as_bytes(); + let mut at = 0usize; + let mut result = Vec::new(); + while at < bytes.len() { + if bytes[at].is_ascii_whitespace() { + at += 1; + continue; + } + if bytes.get(at..at + 2) == Some(b"//") { + while bytes.get(at).is_some_and(|byte| *byte != b'\n') { + at += 1; + } + continue; + } + if bytes.get(at..at + 2) == Some(b"/*") { + at += 2; + while at < bytes.len() && bytes.get(at..at + 2) != Some(b"*/") { + at += 1; + } + if at == bytes.len() { + return Err("direct MCP Node script has an incomplete comment"); + } + at += 2; + continue; + } + let start = at; + let byte = bytes[at]; + if byte == b'\'' || byte == b'"' { + let quote = byte; + at += 1; + let content_start = at; + let mut escaped = false; + while at < bytes.len() && bytes[at] != quote { + if bytes[at] == b'\\' { + escaped = true; + at = (at + 2).min(bytes.len()); + } else { + at += 1; + } + } + if at == bytes.len() { + return Err("direct MCP Node script has an unterminated string"); + } + let value = (!escaped).then(|| source[content_start..at].to_owned()); + at += 1; + result.push(JavaScriptToken::String(value)); + } else if byte == b'`' { + at += 1; + while at < bytes.len() && bytes[at] != b'`' { + if bytes[at] == b'\\' { + at = (at + 2).min(bytes.len()); + } else { + at += 1; + } + } + if at == bytes.len() { + return Err("direct MCP Node script has an unterminated template string"); + } + at += 1; + result.push(JavaScriptToken::String(None)); + } else if byte == b'_' || byte == b'$' || byte.is_ascii_alphabetic() { + at += 1; + while bytes + .get(at) + .is_some_and(|byte| *byte == b'_' || *byte == b'$' || byte.is_ascii_alphanumeric()) + { + at += 1; + } + result.push(JavaScriptToken::Identifier(source[start..at].to_owned())); + } else { + at += 1; + result.push(JavaScriptToken::Punctuation(byte)); + } + if result.len() > MAX_TOKENS { + return Err("direct MCP Node dependency scan exceeds its token bound"); + } + } + Ok(result) +} + +fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static str> { + let tokens = javascript_tokens(source)?; + let mut result = std::collections::BTreeSet::new(); + for (index, token) in tokens.iter().enumerate() { + let module = match token { + JavaScriptToken::Identifier(name) if name == "require" || name == "import" => { + match tokens.get(index + 1) { + Some(JavaScriptToken::Punctuation(b'(')) => tokens.get(index + 2), + Some(JavaScriptToken::String(_)) if name == "import" => tokens.get(index + 1), + _ => None, + } + } + JavaScriptToken::Identifier(name) if name == "from" => tokens.get(index + 1), + _ => None, + }; + if let Some(JavaScriptToken::String(Some(specifier))) = module { + if specifier.starts_with("./") || specifier.starts_with("../") { + result.insert(specifier.clone()); + } + } + } + Ok(result.into_iter().collect()) +} + +fn resolve_node_relative_module(parent: &Path, specifier: &str) -> Option<PathBuf> { + let base = parent.join(specifier); + let mut candidates = vec![base.clone()]; + if base.extension().is_none() { + for extension in ["js", "mjs", "cjs", "json", "node", "ts", "mts", "cts"] { + candidates.push(base.with_extension(extension)); + } + } + if base.is_dir() { + for index in ["index.js", "index.mjs", "index.cjs", "index.json"] { + candidates.push(base.join(index)); + } + } + candidates.into_iter().find(|candidate| candidate.is_file()) +} + +fn insert_node_script_dependencies( + script: &Path, + scope: &Path, + declaring: &Path, + result: &mut std::collections::BTreeSet<PathBuf>, + resource_path_bytes: &mut usize, + field: &'static str, +) -> Result<(), ResourceError> { + const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; + const MAX_SCRIPT_FILES: usize = 256; + let fail = |cause| ResourceError { field, cause }; + let mut pending = vec![script.to_owned()]; + let mut visited = std::collections::BTreeSet::new(); + let mut total_bytes = 0u64; + while let Some(current) = pending.pop() { + let canonical = checked(¤t, field, declaring)?; + if !canonical.is_file() || !visited.insert(canonical.clone()) { + continue; + } + if visited.len() > MAX_SCRIPT_FILES { + return Err(fail( + "direct MCP Node dependency closure exceeds its entry bound", + )); + } + let metadata = std::fs::metadata(&canonical) + .map_err(|_| fail("direct MCP Node script metadata unavailable"))?; + total_bytes = total_bytes.saturating_add(metadata.len()); + if metadata.len() > MAX_SCRIPT_BYTES || total_bytes > MAX_SCRIPT_BYTES { + return Err(fail( + "direct MCP Node dependency closure exceeds its byte bound", + )); + } + if !canonical.extension().is_none_or(|extension| { + is_node_script_path(&format!("file.{}", extension.to_string_lossy())) + }) { + continue; + } + let source = std::fs::read_to_string(&canonical) + .map_err(|_| fail("direct MCP Node script is not readable UTF-8"))?; + let script_parent = canonical + .parent() + .ok_or_else(|| fail("direct MCP Node script has no containing directory"))?; + for specifier in node_relative_module_specifiers(&source).map_err(fail)? { + let candidate = resolve_node_relative_module(script_parent, &specifier) + .ok_or_else(|| fail("direct MCP Node relative module is missing"))?; + let dependency = checked(&candidate, field, declaring)?; + if !dependency.is_file() { + return Err(fail("direct MCP Node module is not a regular file")); + } + if !dependency.starts_with(scope) { + return Err(fail( + "direct MCP Node module escapes its selected script scope", + )); + } + insert_resource(result, resource_path_bytes, dependency.clone(), field)?; + pending.push(dependency); + } + } + Ok(()) +} + +fn insert_shell_script_dependencies( + script: &Path, + scope: &Path, + declaring: &Path, + result: &mut std::collections::BTreeSet<PathBuf>, + resource_path_bytes: &mut usize, + field: &'static str, +) -> Result<(), ResourceError> { + const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; + const MAX_SCRIPT_FILES: usize = 256; + let fail = |cause| ResourceError { field, cause }; + let mut pending = vec![script.to_owned()]; + let mut visited = std::collections::BTreeSet::new(); + let mut total_bytes = 0u64; + while let Some(current) = pending.pop() { + let canonical = checked(¤t, field, declaring)?; + if !canonical.is_file() || !visited.insert(canonical.clone()) { + continue; + } + if visited.len() > MAX_SCRIPT_FILES { + return Err(fail( + "direct MCP shell dependency closure exceeds its entry bound", + )); + } + let metadata = std::fs::metadata(&canonical) + .map_err(|_| fail("direct MCP shell script metadata unavailable"))?; + total_bytes = total_bytes.saturating_add(metadata.len()); + if metadata.len() > MAX_SCRIPT_BYTES || total_bytes > MAX_SCRIPT_BYTES { + return Err(fail( + "direct MCP shell dependency closure exceeds its byte bound", + )); + } + let source = std::fs::read_to_string(&canonical) + .map_err(|_| fail("direct MCP shell script is not readable UTF-8"))?; + let directory_variables = shell_directory_variables(&source); + if directory_variables.len() > 128 { + return Err(fail( + "direct MCP shell dependency closure exceeds its variable bound", + )); + } + let script_parent = canonical + .parent() + .ok_or_else(|| fail("direct MCP shell script has no containing directory"))?; + for line in source + .lines() + .filter(|line| !line.trim_start().starts_with('#')) + { + let mut candidates = std::collections::BTreeSet::new(); + if let Some(operand) = shell_source_operand(line) { + if let Some(candidate) = + shell_source_path(script_parent, &operand, &directory_variables) + .map_err(fail)? + { + candidates.insert(candidate); + } + } + candidates.extend( + shell_directory_relative_paths(line, &directory_variables) + .into_iter() + .map(|relative| script_parent.join(relative)), + ); + for candidate in candidates { + let dependency = checked(&candidate, field, declaring)?; + if !dependency.is_file() { + return Err(fail("direct MCP shell source is not a regular file")); + } + if !dependency.starts_with(scope) { + return Err(fail( + "direct MCP shell source escapes its selected script scope", + )); + } + insert_resource(result, resource_path_bytes, dependency.clone(), field)?; + pending.push(dependency); + } + } + } + Ok(()) +} + fn script_operand_index(command: &[String]) -> Option<usize> { let executable = command .first()? @@ -976,6 +1413,11 @@ pub fn select( path = freshell_platform::opencode_config::lexical_path(&path); } let canonical = checked(&path, field, declaring)?; + if field == "OpenCode mcp.command" { + if let Some(script_field) = shebang_script_field(&canonical) { + field = script_field; + } + } variables.extend(names.into_inner()); if field == "OpenCode instructions" || field == "OpenCode attention.sounds" { if !canonical.is_file() { @@ -1027,13 +1469,77 @@ pub fn select( )?; continue; } + if is_mcp_shebang_script_field(field) { + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical.clone(), + field, + )?; + if is_mcp_node_script_field(field) { + insert_installed_package_ancestor_resources( + command_parent, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + insert_node_script_dependencies( + &canonical, + command_parent, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + if is_shell_script(&canonical) { + insert_shell_script_dependencies( + &canonical, + command_parent, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + continue; + } if is_mcp_script_field(field) { let canonical_home = std::fs::canonicalize(home).ok(); if command_parent == Path::new("/") || canonical_home.as_deref() == Some(command_parent) { if is_mcp_direct_script_field(field) { - insert_resource(&mut result, &mut resource_path_bytes, canonical, field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical.clone(), + field, + )?; + if is_mcp_node_script_field(field) { + insert_node_script_dependencies( + &canonical, + command_parent, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + if canonical_home.as_deref() == Some(command_parent) + && is_shell_script(&canonical) + { + insert_shell_script_dependencies( + &canonical, + command_parent, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } continue; } return Err(fail( diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 24941fc74..acae822d9 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -115,6 +115,38 @@ process.stdin.on('data', (chunk) => { }) ` +const directMcpServerSource = String.raw`const fs = require('node:fs') +const [receiptPath, expectedNonce] = process.argv.slice(2) +if (!receiptPath || !expectedNonce) process.exit(12) +fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper: process.env.MANAGED_DIRECT_MCP_HELPER, + cwd: process.cwd(), scriptPath: process.argv[1] })) +let buffered = '' +process.stdin.setEncoding('utf8') +process.stdin.on('data', chunk => { + buffered += chunk + while (true) { + const boundary = buffered.indexOf('\n') + if (boundary < 0) break + const line = buffered.slice(0, boundary) + buffered = buffered.slice(boundary + 1) + let request + try { request = JSON.parse(line) } catch { continue } + if (!request || !Object.hasOwn(request, 'id')) continue + let result + if (request.method === 'initialize') result = { protocolVersion: request.params?.protocolVersion ?? '2024-11-05', + capabilities: { tools: {} }, serverInfo: { name: 'freshell-direct-script-fixture', version: '1.0.0' } } + else if (request.method === 'tools/list') result = { tools: [] } + else if (request.method === 'ping') result = {} + else { + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, + error: { code: -32601, message: 'Method not found' } }) + '\n') + continue + } + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n') + } +}) +` + export function resolveManagedProviderEnvironmentTargetMode(args: string[]): ManagedProviderEnvironmentTargetMode { if (args.length === 0) return 'managed-environment' if (args.length === 1 && args[0] === '--checkpoint') return 'code-checkpoint' @@ -255,16 +287,18 @@ export function managedEnvironmentFixtureLayout(root: string) { const resources = path.join(root, 'selected-resources') const plugin = path.join(resources, 'fixture-plugin') const mcpPackage = path.join(resources, 'fixture-mcp') + const directMcp = path.join(resources, 'direct-mcp') const mcpNested = path.join(mcpPackage, 'nested') const mcpParentDependency = path.join(resources, 'node_modules/managed-fixture-mcp-dependency') - return { workspace, resources, plugin, mcpPackage, mcpNested, mcpParentDependency } + return { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } } async function configure(root: string, nonce: string, candidateSha: string, runId: string) { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg') - const { workspace, resources, plugin, mcpPackage, mcpNested, mcpParentDependency } = managedEnvironmentFixtureLayout(root) + const { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } = managedEnvironmentFixtureLayout(root) await fs.mkdir(path.join(plugin, 'node_modules/managed-fixture-dependency'), { recursive: true, mode: 0o755 }) await fs.mkdir(mcpNested, { recursive: true, mode: 0o755 }) + await fs.mkdir(directMcp, { recursive: true, mode: 0o755 }) await fs.mkdir(mcpParentDependency, { recursive: true, mode: 0o755 }) await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) await fs.chmod(home, 0o700) @@ -303,6 +337,14 @@ export default {id:'managed-environment-tui',async tui(api,options){ scripts: { 'mcp.js': 'bun ./server.mjs' }, }) await fs.writeFile(path.join(mcpPackage, 'server.mjs'), managedMcpServerSource) + const directCommand = path.join(directMcp, 'start') + await fs.writeFile(directCommand, + '#!/bin/sh\nSCRIPT_DIR=$(dirname "$0")\n. "$SCRIPT_DIR/helper.sh"\nexec node "$SCRIPT_DIR/server.js" "$@"\n', + { mode: 0o751 }) + await fs.chmod(directCommand, 0o751) + await fs.writeFile(path.join(directMcp, 'helper.sh'), + `export MANAGED_DIRECT_MCP_HELPER='${nonce}'\n`) + await fs.writeFile(path.join(directMcp, 'server.js'), directMcpServerSource) await writeJson(path.join(mcpParentDependency, 'package.json'), { name: 'managed-fixture-mcp-dependency', type: 'module', @@ -331,6 +373,11 @@ export default {id:'managed-environment-tui',async tui(api,options){ enabled: true, environment: { FRESHELL_EXPECTED_MCP_CWD: mcpNested }, }, + direct_script_fixture: { + type: 'local', + command: [directCommand, path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'), nonce], + enabled: true, + }, }, }) } @@ -1237,6 +1284,11 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightMcp.bunCommandCwd, selected.mcpNested, 'OpenCode must launch Bun from the configured nested MCP cwd') assert.equal(preflightMcp.configuredCwd, selected.mcpNested) assert(preflightMcp.scriptPath.endsWith('/fixture-mcp/server.mjs')) + const preflightDirectMcp = await poll('OpenCode starts the direct extensionless MCP through its sibling helper', async () => + readJson(path.join(selected.workspace, 'preflight/.managed-fixture/direct-mcp-receipt.json'))) + assert.equal(preflightDirectMcp.expectedNonce, nonce) + assert.equal(preflightDirectMcp.helper, nonce) + assert(preflightDirectMcp.scriptPath.endsWith('/direct-mcp/server.js')) const preflightView = await poll('genuine managed preflight native identity', () => observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) @@ -1339,9 +1391,14 @@ export async function runManagedEnvironmentTarget() { assert.equal(mcpServer.bunCommandCwd, selected.mcpNested, 'OpenCode must launch Bun from the configured nested MCP cwd') assert.equal(mcpServer.configuredCwd, selected.mcpNested) assert(mcpServer.scriptPath.endsWith('/fixture-mcp/server.mjs')) + const directMcpServer = await poll('Cloud OpenCode starts the direct extensionless MCP through its sibling helper', async () => + readJson(path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'))) + assert.equal(directMcpServer.expectedNonce, nonce) + assert.equal(directMcpServer.helper, nonce) + assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.js')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, - nativeSessionId: view.nativeSessionId, stage, child, mcpServer } + nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer } await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt } }, diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 5d62e133c..032dbcab1 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -212,6 +212,12 @@ test(healthyTitle, async ({ page }, info) => { terminalId, bun: true, dependency: fixture.expectedNonce, resource: fixture.expectedNonce, absoluteReference: fixture.expectedNonce, relativeReference: fixture.expectedNonce, executableMode: '0711', } }) + let directMcpReceipt: any + await expect.poll(async () => { + directMcpReceipt = await readReceipt(page, `${pane.cwd}/.managed-fixture/direct-mcp-receipt.json`) + return directMcpReceipt + }, { timeout: 120_000 }).toMatchObject({ expectedNonce: fixture.expectedNonce, helper: fixture.expectedNonce, cwd: pane.cwd }) + expect(directMcpReceipt.scriptPath).toMatch(/\/direct-mcp\/server\.js$/) expect(receipt.soulId).toBeTruthy() expect(receipt.incarnationId).toBeTruthy() expect(receipt.nativeSessionId).toBeTruthy() From 06157afaf08a0b94bca812c04d21b3171fb0b12e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:56:06 -0700 Subject: [PATCH 376/493] chore: use default request id fallback --- crates/freshell-server/src/fresh_agent_proxy.rs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 495de83ef..1d0b5fd98 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -156,7 +156,7 @@ impl HostedFreshAgentProxy { .request_id .as_deref() .and_then(|value| RequestId::parse(value.to_string()).ok()) - .unwrap_or_else(RequestId::new); + .unwrap_or_default(); let result = match self .resolve_soul(&provider, session_type, &session_id) .await @@ -337,7 +337,7 @@ impl HostedFreshAgentProxy { .request_id .as_deref() .and_then(|value| RequestId::parse(value.to_string()).ok()) - .unwrap_or_else(RequestId::new); + .unwrap_or_default(); let result = match self .resolve_soul(&message.provider, message.session_type, &message.session_id) .await @@ -372,7 +372,7 @@ impl HostedFreshAgentProxy { .request_id .as_deref() .and_then(|value| RequestId::parse(value.to_string()).ok()) - .unwrap_or_else(RequestId::new); + .unwrap_or_default(); let result = match self .resolve_soul(&provider, session_type, &session_id) .await From d8c9d81eb595c8d80028a0fe7d4855e84c3ee640 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:56:10 -0700 Subject: [PATCH 377/493] test: tolerate transient copied-script busy errors --- crates/freshell-server/src/managed_mcp_capability.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 0831b566a..be74bc253 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -4097,7 +4097,7 @@ except Exception as error: ".codex/skills/freshell-discovered-agents/fixture/scripts/run", ]; for relative in resources { - let output = Command::new(projection.join(relative)).output().unwrap(); + let output = execute_copied_script(&projection.join(relative)).unwrap(); assert!(output.status.success(), "{output:?}"); assert_eq!(output.stdout, b"resource-nonce\n"); } From 472f7e4ea793b68a85770c00dad332d74ab2cf91 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:26:45 -0700 Subject: [PATCH 378/493] fix(managed): close direct MCP script dependencies --- .../managed_mcp_capability/source_stage.rs | 264 +++++++++++++ .../opencode_resources.rs | 352 ++++++++++++++++-- 2 files changed, 576 insertions(+), 40 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index ffadc79f2..34816f4dc 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -81,6 +81,67 @@ mod configured_resource_tests { provider } + fn plan_and_stage_direct_mcp_command( + owned_root: &Path, + home: &Path, + workspace: &Path, + command: &Path, + ) -> (Vec<PlannedSourceOccurrence>, std::path::PathBuf) { + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + workspace, + home, + &|_, _, _| None, + ) + .unwrap(); + let units = + resource_units(home, &home.join(".config/opencode"), workspace, &plan, None).unwrap(); + let command = command.canonicalize().unwrap(); + let mount = units + .iter() + .find(|(source, _)| source == &command) + .map(|(_, mount)| mount) + .expect("the direct MCP command must be in the resource plan"); + let effective_destination = Path::new(&mount.effective_destination); + let provider_relative = effective_destination + .strip_prefix(owned_root) + .map(Path::to_path_buf) + .or_else(|_| { + effective_destination + .strip_prefix("/home/freshell/provider") + .map(|relative| Path::new(".provider-home").join(relative)) + }) + .expect("the test provider destination must map into the temporary root"); + let provider = copy_selected_units_through_provider(owned_root, &units); + (plan, provider.join(provider_relative)) + } + + fn execute_with_home_hidden( + owned_root: &Path, + home: &Path, + provider_command: &Path, + ) -> std::process::Output { + let hidden_home = owned_root.join("home-hidden"); + std::fs::rename(home, &hidden_home).unwrap(); + let output = execute_selected_resource(provider_command); + std::fs::rename(&hidden_home, home).unwrap(); + output.unwrap() + } + #[test] fn managed_provider_keeps_external_indexed_payloads_out_of_persistent_generation() { fn tree_contains_bytes(path: &Path, expected: &[u8]) -> bool { @@ -942,6 +1003,209 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"node sibling helper loaded\n"); } + #[test] + fn direct_node_script_in_home_keeps_home_node_modules_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("start"); + let dependency = home.join("node_modules/home-fixture/index.js"); + std::fs::create_dir_all(dependency.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nprocess.stdout.write(require('home-fixture'))\n", + ) + .unwrap(); + std::fs::write( + &dependency, + "module.exports = 'home node_modules loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&home.join("node_modules").canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"home node_modules loaded\n"); + } + + #[test] + fn direct_node_script_loads_parent_relative_module_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let command = external.join("mcp/bin/start"); + let helper = external.join("mcp/lib/helper.js"); + std::fs::create_dir_all(command.parent().unwrap()).unwrap(); + std::fs::create_dir_all(helper.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nprocess.stdout.write(require('../lib/helper.js'))\n", + ) + .unwrap(); + std::fs::write( + &helper, + "module.exports = 'parent-relative module loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&external.join("mcp").canonicalize().unwrap())); + + let hidden_external = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden_external).unwrap(); + let output = execute_selected_resource(&provider_command).unwrap(); + std::fs::rename(&hidden_external, &external).unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"parent-relative module loaded\n"); + } + + #[test] + fn direct_python_shebang_loads_sibling_module_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let helper = tools.join("helper.py"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env python3\nimport helper\nprint(helper.VALUE)\n", + ) + .unwrap(); + std::fs::write(&helper, "VALUE = 'python sibling loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"python sibling loaded\n"); + } + + #[test] + fn direct_ruby_shebang_keeps_relative_module_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let helper = tools.join("helper.rb"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env ruby\nrequire_relative 'helper'\nputs VALUE\n", + ) + .unwrap(); + std::fs::write(&helper, "VALUE = 'ruby sibling loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + assert!(provider_command.is_file()); + assert!(provider_command.with_file_name("helper.rb").is_file()); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + } + + #[test] + fn shell_shebang_keeps_binary_support_resources_without_parsing_them() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let data = tools.join("data.bin"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\ncat \"$SCRIPT_DIR/data.bin\"\n", + ) + .unwrap(); + let expected = [0, 0xff, 0x81, b'R', b'I', b'F', b'F']; + std::fs::write(&data, expected).unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&data.canonicalize().unwrap())); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, expected); + } + + #[test] + fn shell_shebang_resolves_nested_quote_dirname_source_expression() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let helper = tools.join("helper.sh"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/bash\nsource \"$(dirname \"$0\")/helper.sh\"\nprintf '%s\\n' \"$VALUE\"\n", + ) + .unwrap(); + std::fs::write(&helper, "VALUE='nested quote source loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"nested quote source loaded\n"); + } + #[test] fn direct_mcp_shell_script_loads_sibling_after_home_copy() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index de270c1da..69e5abf4e 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -550,6 +550,38 @@ fn is_shell_script(path: &Path) -> bool { }) } +fn is_python_script(path: &Path) -> bool { + path.extension().is_some_and(|extension| { + matches!( + extension.to_string_lossy().to_ascii_lowercase().as_str(), + "py" | "pyw" + ) + }) || shebang_interpreter_names(path).is_some_and(|names| { + names + .iter() + .any(|name| matches!(name.as_str(), "python" | "python2" | "python3")) + }) +} + +fn is_ruby_script(path: &Path) -> bool { + path.extension() + .is_some_and(|extension| extension.to_string_lossy().eq_ignore_ascii_case("rb")) + || shebang_interpreter_names(path) + .is_some_and(|names| names.iter().any(|name| name == "ruby")) +} + +fn direct_script_dependency_scope(command_parent: &Path, home: &Path) -> PathBuf { + let canonical_home = std::fs::canonicalize(home).unwrap_or_else(|_| home.to_owned()); + if canonical_home != Path::new("/") && command_parent.starts_with(&canonical_home) { + return canonical_home; + } + command_parent + .parent() + .filter(|parent| *parent != Path::new("/")) + .unwrap_or(command_parent) + .to_owned() +} + fn shell_directory_variables(source: &str) -> std::collections::BTreeSet<String> { source .lines() @@ -585,6 +617,10 @@ fn shell_source_operand(line: &str) -> Option<String> { return None; } .trim_start(); + if let Some(rest) = rest.strip_prefix("\"$(dirname \"$0\")/") { + let end = rest.find('\"')?; + return Some(format!("$(dirname \"$0\")/{}", &rest[..end])); + } let first = rest.chars().next()?; if first == '\'' || first == '"' { let end = rest[1..].find(first)? + 1; @@ -605,6 +641,13 @@ fn shell_source_path( if operand.starts_with('/') { return Ok(None); } + const SCRIPT_DIRECTORY_SUBSTITUTION: &str = "$(dirname \"$0\")/"; + if let Some(relative) = operand.strip_prefix(SCRIPT_DIRECTORY_SUBSTITUTION) { + if relative.is_empty() || relative.contains(['$', '*', '?']) { + return Err("direct MCP shell script has an unresolved dynamic source path"); + } + return Ok(Some(script_parent.join(relative))); + } let mut relative = operand; if let Some(variable) = operand.strip_prefix("${") { let Some((name, suffix)) = variable.split_once('}') else { @@ -786,58 +829,232 @@ fn resolve_node_relative_module(parent: &Path, specifier: &str) -> Option<PathBu candidates.into_iter().find(|candidate| candidate.is_file()) } -fn insert_node_script_dependencies( +fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> { + fn is_static_module_name(name: &str) -> bool { + !name.is_empty() + && !name.starts_with('.') + && name + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_' || byte == b'.') + } + + let mut result = std::collections::BTreeSet::new(); + for line in source.lines() { + let statement = line.split('#').next().unwrap_or_default().trim(); + if let Some(imports) = statement.strip_prefix("import ") { + for item in imports.split(',') { + let name = item.split_ascii_whitespace().next().unwrap_or_default(); + if is_static_module_name(name) { + result.insert(name.to_owned()); + } + } + } else if let Some(imports) = statement.strip_prefix("from ") { + let Some((module, imported)) = imports.split_once(" import ") else { + continue; + }; + let module = module.trim(); + if !is_static_module_name(module) { + continue; + } + result.insert(module.to_owned()); + for item in imported + .trim() + .trim_start_matches('(') + .trim_end_matches(')') + .split(',') + { + let name = item + .trim() + .split_ascii_whitespace() + .next() + .unwrap_or_default(); + if is_static_module_name(name) { + result.insert(format!("{module}.{name}")); + } + } + } + } + result +} + +fn resolve_python_local_modules(parent: &Path, specifier: &str) -> Vec<PathBuf> { + let components = specifier.split('.').collect::<Vec<_>>(); + if components + .iter() + .any(|component| component.is_empty() || component.contains(['/', '\\'])) + { + return Vec::new(); + } + let mut result = std::collections::BTreeSet::new(); + let mut package = parent.to_owned(); + for component in &components[..components.len().saturating_sub(1)] { + package.push(component); + let init = package.join("__init__.py"); + if init.is_file() { + result.insert(init); + } + } + let mut module = parent.to_owned(); + for component in &components { + module.push(component); + } + for candidate in [ + module.with_extension("py"), + module.with_extension("pyw"), + module.join("__init__.py"), + ] { + if candidate.is_file() { + result.insert(candidate); + } + } + result.into_iter().collect() +} + +fn ruby_relative_require_specifiers(source: &str) -> Result<Vec<String>, &'static str> { + let mut result = std::collections::BTreeSet::new(); + for line in source.lines() { + let statement = line.trim_start(); + let Some(mut operand) = statement.strip_prefix("require_relative") else { + continue; + }; + if operand + .chars() + .next() + .is_some_and(|character| !character.is_ascii_whitespace() && character != '(') + { + continue; + } + operand = operand + .trim_start() + .strip_prefix('(') + .unwrap_or(operand.trim_start()); + operand = operand.trim_start(); + let Some(quote) = operand + .chars() + .next() + .filter(|quote| matches!(quote, '\'' | '"')) + else { + return Err("direct MCP Ruby script has a dynamic require_relative path"); + }; + let end = operand[1..] + .find(quote) + .ok_or("direct MCP Ruby script has an unterminated require_relative path")? + + 1; + let specifier = &operand[1..end]; + if specifier.is_empty() || specifier.contains(['$', '#', '*', '?']) { + return Err("direct MCP Ruby script has a dynamic require_relative path"); + } + result.insert(specifier.to_owned()); + } + Ok(result.into_iter().collect()) +} + +fn resolve_ruby_relative_module(parent: &Path, specifier: &str) -> Option<PathBuf> { + let base = parent.join(specifier); + [base.clone(), base.with_extension("rb")] + .into_iter() + .find(|candidate| candidate.is_file()) +} + +#[derive(Clone, Copy)] +enum ScriptDependencyRuntime { + Node, + Python, + Ruby, +} + +fn direct_script_dependency_paths( + parent: &Path, + source: &str, + runtime: ScriptDependencyRuntime, +) -> Result<Vec<PathBuf>, &'static str> { + match runtime { + ScriptDependencyRuntime::Node => node_relative_module_specifiers(source)? + .into_iter() + .map(|specifier| { + resolve_node_relative_module(parent, &specifier) + .ok_or("direct MCP Node relative module is missing") + }) + .collect(), + ScriptDependencyRuntime::Python => Ok(python_import_specifiers(source) + .into_iter() + .flat_map(|specifier| resolve_python_local_modules(parent, &specifier)) + .collect()), + ScriptDependencyRuntime::Ruby => ruby_relative_require_specifiers(source)? + .into_iter() + .map(|specifier| { + resolve_ruby_relative_module(parent, &specifier) + .ok_or("direct MCP Ruby relative module is missing") + }) + .collect(), + } +} + +fn is_script_dependency_source(path: &Path, runtime: ScriptDependencyRuntime) -> bool { + let Some(extension) = path.extension() else { + return false; + }; + let extension = extension.to_string_lossy(); + match runtime { + ScriptDependencyRuntime::Node => is_node_script_path(&format!("file.{extension}")), + ScriptDependencyRuntime::Python => { + matches!(extension.to_ascii_lowercase().as_str(), "py" | "pyw") + } + ScriptDependencyRuntime::Ruby => extension.eq_ignore_ascii_case("rb"), + } +} + +fn insert_direct_script_dependencies( script: &Path, scope: &Path, declaring: &Path, result: &mut std::collections::BTreeSet<PathBuf>, resource_path_bytes: &mut usize, field: &'static str, + runtime: ScriptDependencyRuntime, ) -> Result<(), ResourceError> { const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; const MAX_SCRIPT_FILES: usize = 256; let fail = |cause| ResourceError { field, cause }; let mut pending = vec![script.to_owned()]; let mut visited = std::collections::BTreeSet::new(); + let mut parsed = std::collections::BTreeSet::new(); let mut total_bytes = 0u64; while let Some(current) = pending.pop() { let canonical = checked(¤t, field, declaring)?; if !canonical.is_file() || !visited.insert(canonical.clone()) { continue; } - if visited.len() > MAX_SCRIPT_FILES { + if current != script && !is_script_dependency_source(&canonical, runtime) { + continue; + } + if !parsed.insert(canonical.clone()) { + continue; + } + if parsed.len() > MAX_SCRIPT_FILES { return Err(fail( - "direct MCP Node dependency closure exceeds its entry bound", + "direct MCP dependency closure exceeds its entry bound", )); } let metadata = std::fs::metadata(&canonical) - .map_err(|_| fail("direct MCP Node script metadata unavailable"))?; + .map_err(|_| fail("direct MCP script metadata unavailable"))?; total_bytes = total_bytes.saturating_add(metadata.len()); if metadata.len() > MAX_SCRIPT_BYTES || total_bytes > MAX_SCRIPT_BYTES { - return Err(fail( - "direct MCP Node dependency closure exceeds its byte bound", - )); - } - if !canonical.extension().is_none_or(|extension| { - is_node_script_path(&format!("file.{}", extension.to_string_lossy())) - }) { - continue; + return Err(fail("direct MCP dependency closure exceeds its byte bound")); } let source = std::fs::read_to_string(&canonical) - .map_err(|_| fail("direct MCP Node script is not readable UTF-8"))?; - let script_parent = canonical + .map_err(|_| fail("direct MCP script is not readable UTF-8"))?; + let parent = canonical .parent() - .ok_or_else(|| fail("direct MCP Node script has no containing directory"))?; - for specifier in node_relative_module_specifiers(&source).map_err(fail)? { - let candidate = resolve_node_relative_module(script_parent, &specifier) - .ok_or_else(|| fail("direct MCP Node relative module is missing"))?; + .ok_or_else(|| fail("direct MCP script has no containing directory"))?; + for candidate in direct_script_dependency_paths(parent, &source, runtime).map_err(fail)? { let dependency = checked(&candidate, field, declaring)?; if !dependency.is_file() { - return Err(fail("direct MCP Node module is not a regular file")); + return Err(fail("direct MCP dependency is not a regular file")); } if !dependency.starts_with(scope) { return Err(fail( - "direct MCP Node module escapes its selected script scope", + "direct MCP dependency escapes its selected script scope", )); } insert_resource(result, resource_path_bytes, dependency.clone(), field)?; @@ -858,15 +1075,23 @@ fn insert_shell_script_dependencies( const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; const MAX_SCRIPT_FILES: usize = 256; let fail = |cause| ResourceError { field, cause }; - let mut pending = vec![script.to_owned()]; + let mut pending = vec![(script.to_owned(), true)]; let mut visited = std::collections::BTreeSet::new(); + let mut parsed = std::collections::BTreeSet::new(); let mut total_bytes = 0u64; - while let Some(current) = pending.pop() { + while let Some((current, parse_as_script)) = pending.pop() { let canonical = checked(¤t, field, declaring)?; - if !canonical.is_file() || !visited.insert(canonical.clone()) { + if !canonical.is_file() { continue; } - if visited.len() > MAX_SCRIPT_FILES { + if !visited.insert(canonical.clone()) && !parse_as_script { + continue; + } + let parse_as_script = parse_as_script || is_shell_script(&canonical); + if !parse_as_script || !parsed.insert(canonical.clone()) { + continue; + } + if parsed.len() > MAX_SCRIPT_FILES { return Err(fail( "direct MCP shell dependency closure exceeds its entry bound", )); @@ -894,21 +1119,21 @@ fn insert_shell_script_dependencies( .lines() .filter(|line| !line.trim_start().starts_with('#')) { - let mut candidates = std::collections::BTreeSet::new(); + let mut candidates = std::collections::BTreeMap::new(); if let Some(operand) = shell_source_operand(line) { if let Some(candidate) = shell_source_path(script_parent, &operand, &directory_variables) .map_err(fail)? { - candidates.insert(candidate); + candidates.insert(candidate, true); } } - candidates.extend( - shell_directory_relative_paths(line, &directory_variables) - .into_iter() - .map(|relative| script_parent.join(relative)), - ); - for candidate in candidates { + for relative in shell_directory_relative_paths(line, &directory_variables) { + candidates + .entry(script_parent.join(relative)) + .or_insert(false); + } + for (candidate, parse_as_script) in candidates { let dependency = checked(&candidate, field, declaring)?; if !dependency.is_file() { return Err(fail("direct MCP shell source is not a regular file")); @@ -919,7 +1144,7 @@ fn insert_shell_script_dependencies( )); } insert_resource(result, resource_path_bytes, dependency.clone(), field)?; - pending.push(dependency); + pending.push((dependency, parse_as_script)); } } } @@ -1447,6 +1672,7 @@ pub fn select( let command_parent = canonical .parent() .ok_or_else(|| fail(field, "command has no containing scope"))?; + let dependency_scope = direct_script_dependency_scope(command_parent, home); if let Some(package_root) = command_parent .ancestors() .take_while(|ancestor| *ancestor != home && ancestor.parent().is_some()) @@ -1478,30 +1704,53 @@ pub fn select( )?; if is_mcp_node_script_field(field) { insert_installed_package_ancestor_resources( - command_parent, + &canonical, home, declaring, &mut result, &mut resource_path_bytes, field, )?; - insert_node_script_dependencies( + insert_direct_script_dependencies( &canonical, - command_parent, + &dependency_scope, declaring, &mut result, &mut resource_path_bytes, field, + ScriptDependencyRuntime::Node, )?; } if is_shell_script(&canonical) { insert_shell_script_dependencies( &canonical, - command_parent, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + if is_python_script(&canonical) { + insert_direct_script_dependencies( + &canonical, + &dependency_scope, declaring, &mut result, &mut resource_path_bytes, field, + ScriptDependencyRuntime::Python, + )?; + } + if is_ruby_script(&canonical) { + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Ruby, )?; } continue; @@ -1519,13 +1768,14 @@ pub fn select( field, )?; if is_mcp_node_script_field(field) { - insert_node_script_dependencies( + insert_direct_script_dependencies( &canonical, - command_parent, + &dependency_scope, declaring, &mut result, &mut resource_path_bytes, field, + ScriptDependencyRuntime::Node, )?; } if canonical_home.as_deref() == Some(command_parent) @@ -1533,11 +1783,33 @@ pub fn select( { insert_shell_script_dependencies( &canonical, - command_parent, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + if is_python_script(&canonical) { + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Python, + )?; + } + if is_ruby_script(&canonical) { + insert_direct_script_dependencies( + &canonical, + &dependency_scope, declaring, &mut result, &mut resource_path_bytes, field, + ScriptDependencyRuntime::Ruby, )?; } continue; From d7dcbb960e2ad39ca50166824231bb473f36d959 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:26:50 -0700 Subject: [PATCH 379/493] fix(recovery): preserve managed error after retained exit --- src/components/TerminalView.tsx | 6 +++++- .../components/TerminalView.exitBanner.test.tsx | 16 ++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index 8372a7990..42f9626a8 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -6054,7 +6054,11 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te dispatch(clearPaneRuntimeActivity({ paneId: paneIdRef.current })) discardPendingInput('terminal_gone') const terminal = contentRef.current - const status = managedTerminalPaneStatus(terminal ?? {}, 'exited') + // A retained exit can race the recovery projection on reconnect. + // Keep an already-rendered managed failure visible when this frame + // has no supervisor summary to replace it with. + const statusFallback = terminal?.status === 'error' ? 'error' : 'exited' + const status = managedTerminalPaneStatus(terminal ?? {}, statusFallback) const runtimeStatus = terminal?.runtimeStatus === 'blocked' || terminal?.recoverySummary?.recoveryState === 'blocked' ? 'blocked' diff --git a/test/unit/client/components/TerminalView.exitBanner.test.tsx b/test/unit/client/components/TerminalView.exitBanner.test.tsx index 4007603fb..83efa39a7 100644 --- a/test/unit/client/components/TerminalView.exitBanner.test.tsx +++ b/test/unit/client/components/TerminalView.exitBanner.test.tsx @@ -355,6 +355,22 @@ describe('TerminalView exited-pane error banner', () => { expect(paneState(store)).toMatchObject({terminalId:'retained-managed',streamId:'retained-epoch',createRequestId:REQ,status:'exited',soulId:'managed-soul'}) }) + it('preserves a managed recovery error when retained exit arrives without its projection', async () => { + const { store, paneContent } = makeStore({ mode: 'opencode', status: 'error', managed: true, withSessionRef: false }) + act(() => store.dispatch(updatePaneContent({ tabId: TAB, paneId: PANE, content: { + ...paneContent, terminalId: 'retained-managed', streamId: 'retained-epoch', + } }))) + await renderPane(store, paneState(store)) + + act(() => messageHandler!({ type: 'terminal.exit', terminalId: 'retained-managed', + exitCode: -1, managedRetained: true, exitCodeAvailable: false })) + + expect(paneState(store)).toMatchObject({ + terminalId: 'retained-managed', streamId: 'retained-epoch', status: 'error', soulId: 'managed-soul', + }) + expect(store.getState().tabs.tabs[0]).toMatchObject({ status: 'error' }) + }) + it('does not display a compatibility exit code when managed completion reports it unavailable', async () => { const {store,paneContent}=makeStore({mode:'codex',status:'running',managed:true, lifecycle:{lastTerminalId:'retained-managed',exit:{exitCode:9,at:Date.now()}}}) From fa4326bb7d3eac0df2997d1c94f33f6738aeca1d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:51:36 -0700 Subject: [PATCH 380/493] fix(managed): resolve bounded local script imports --- .../managed_mcp_capability/source_stage.rs | 178 +++++++++++++++- .../opencode_resources.rs | 201 ++++++++++++++---- 2 files changed, 332 insertions(+), 47 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 34816f4dc..29b007188 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -13,11 +13,21 @@ mod configured_resource_tests { use crate::managed_provider_references::sources::*; use freshell_platform::provider_environment::{copy_environment, CopySelection}; - fn execute_selected_resource(path: &Path) -> std::io::Result<std::process::Output> { + fn execute_selected_resource_with_path( + path: &Path, + path_prefix: Option<&Path>, + ) -> std::io::Result<std::process::Output> { + let inherited_path = std::env::var_os("PATH").unwrap_or_default(); + let search_path = if let Some(prefix) = path_prefix { + let paths = std::iter::once(prefix.to_owned()) + .chain(std::env::split_paths(&inherited_path)) + .collect::<Vec<_>>(); + std::env::join_paths(paths).expect("the test PATH entries must be valid") + } else { + inherited_path + }; let mut command = std::process::Command::new(path); - command - .env_clear() - .env("PATH", std::env::var_os("PATH").unwrap()); + command.env_clear().env("PATH", search_path); // Match the existing copied-script helper's finite spawn retry, while // retaining this consumer's isolated environment. ETXTBSY has no child. let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); @@ -34,6 +44,10 @@ mod configured_resource_tests { } } + fn execute_selected_resource(path: &Path) -> std::io::Result<std::process::Output> { + execute_selected_resource_with_path(path, None) + } + fn copy_selected_units_through_provider( owned_root: &Path, units: &[( @@ -134,10 +148,19 @@ mod configured_resource_tests { owned_root: &Path, home: &Path, provider_command: &Path, + ) -> std::process::Output { + execute_with_home_hidden_using_path(owned_root, home, provider_command, None) + } + + fn execute_with_home_hidden_using_path( + owned_root: &Path, + home: &Path, + provider_command: &Path, + path_prefix: Option<&Path>, ) -> std::process::Output { let hidden_home = owned_root.join("home-hidden"); std::fs::rename(home, &hidden_home).unwrap(); - let output = execute_selected_resource(provider_command); + let output = execute_selected_resource_with_path(provider_command, path_prefix); std::fs::rename(&hidden_home, home).unwrap(); output.unwrap() } @@ -1083,6 +1106,51 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"parent-relative module loaded\n"); } + #[test] + fn direct_node_script_resolves_directory_package_main_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let command = external.join("mcp/bin/start"); + let package = external.join("mcp/helpers"); + let manifest = package.join("package.json"); + let entry = package.join("lib/entry.cjs"); + std::fs::create_dir_all(command.parent().unwrap()).unwrap(); + std::fs::create_dir_all(entry.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nprocess.stdout.write(require('../helpers'))\n", + ) + .unwrap(); + std::fs::write(&manifest, r#"{"main":"lib/entry.cjs"}"#).unwrap(); + std::fs::write( + &entry, + "module.exports = 'directory package main loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&manifest.canonicalize().unwrap())); + assert!(plan[0] + .configured_resources + .contains(&entry.canonicalize().unwrap())); + let hidden_external = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden_external).unwrap(); + let output = execute_selected_resource(&provider_command).unwrap(); + std::fs::rename(&hidden_external, &external).unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"directory package main loaded\n"); + } + #[test] fn direct_python_shebang_loads_sibling_module_after_all_copies() { use std::os::unix::fs::PermissionsExt; @@ -1113,6 +1181,82 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"python sibling loaded\n"); } + #[test] + fn direct_python_package_relative_import_loads_nested_module_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let package = tools.join("pkg"); + let command = tools.join("start"); + let helper = package.join("helper.py"); + std::fs::create_dir_all(&package).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env python3\nimport pkg\nprint(pkg.helper.VALUE)\n", + ) + .unwrap(); + std::fs::write(package.join("__init__.py"), "from . import helper\n").unwrap(); + std::fs::write(&helper, "VALUE = 'nested package helper loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"nested package helper loaded\n"); + } + + #[test] + fn direct_python_versioned_shebang_loads_sibling_module_after_all_copies() { + use std::os::unix::fs::{symlink, PermissionsExt}; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let helper = tools.join("helper.py"); + let bin = owned.path().join("bin"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&bin).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env python3.12\nimport helper\nprint(helper.VALUE)\n", + ) + .unwrap(); + std::fs::write(&helper, "VALUE = 'versioned python loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let inherited_path = std::env::var_os("PATH").unwrap_or_default(); + let interpreter = std::env::split_paths(&inherited_path) + .map(|directory| directory.join("python3")) + .find(|candidate| candidate.is_file()) + .expect("python3 is required by the existing direct Python script test"); + symlink( + std::fs::canonicalize(interpreter).unwrap(), + bin.join("python3.12"), + ) + .unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + let output = + execute_with_home_hidden_using_path(owned.path(), &home, &provider_command, Some(&bin)); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"versioned python loaded\n"); + } + #[test] fn direct_ruby_shebang_keeps_relative_module_after_all_copies() { use std::os::unix::fs::PermissionsExt; @@ -1143,6 +1287,30 @@ mod configured_resource_tests { assert!(!plan[0] .configured_resources .contains(&home.canonicalize().unwrap())); + + let bin = owned.path().join("bin"); + std::fs::create_dir_all(&bin).unwrap(); + let ruby_available = std::process::Command::new("ruby") + .arg("--version") + .output() + .is_ok_and(|output| output.status.success()); + if !ruby_available { + let ruby_shim = bin.join("ruby"); + std::fs::write( + &ruby_shim, + "#!/bin/sh\nscript=\"$1\"\ngrep -F \"require_relative 'helper'\" \"$script\" >/dev/null || exit 10\ngrep -F \"VALUE = 'ruby sibling loaded'\" \"$(dirname \"$script\")/helper.rb\" >/dev/null || exit 11\nprintf 'ruby sibling loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&ruby_shim, std::fs::Permissions::from_mode(0o755)).unwrap(); + } + let output = execute_with_home_hidden_using_path( + owned.path(), + &home, + &provider_command, + (!ruby_available).then_some(bin.as_path()), + ); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"ruby sibling loaded\n"); } #[test] diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 69e5abf4e..8d9225411 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -6,6 +6,7 @@ use std::path::{Path, PathBuf}; const FRESHELL_IMAGE_MCP_SERVER_PATH: &str = "/opt/freshell-mcp/server.js"; const MAX_BUN_PACKAGE_JSON_BYTES: u64 = 16 * 1024 * 1024; +const MAX_NODE_PACKAGE_JSON_BYTES: u64 = 1024 * 1024; #[derive(Clone, Copy)] enum Token { @@ -551,16 +552,23 @@ fn is_shell_script(path: &Path) -> bool { } fn is_python_script(path: &Path) -> bool { + fn is_python_interpreter(name: &str) -> bool { + matches!(name, "python" | "python2" | "python3") + || name.strip_prefix("python").is_some_and(|version| { + !version.is_empty() + && version.split('.').all(|component| { + !component.is_empty() && component.bytes().all(|byte| byte.is_ascii_digit()) + }) + }) + } + path.extension().is_some_and(|extension| { matches!( extension.to_string_lossy().to_ascii_lowercase().as_str(), "py" | "pyw" ) - }) || shebang_interpreter_names(path).is_some_and(|names| { - names - .iter() - .any(|name| matches!(name.as_str(), "python" | "python2" | "python3")) - }) + }) || shebang_interpreter_names(path) + .is_some_and(|names| names.iter().any(|name| is_python_interpreter(name))) } fn is_ruby_script(path: &Path) -> bool { @@ -813,29 +821,106 @@ fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static Ok(result.into_iter().collect()) } -fn resolve_node_relative_module(parent: &Path, specifier: &str) -> Option<PathBuf> { - let base = parent.join(specifier); - let mut candidates = vec![base.clone()]; - if base.extension().is_none() { - for extension in ["js", "mjs", "cjs", "json", "node", "ts", "mts", "cts"] { - candidates.push(base.with_extension(extension)); - } +fn read_node_package_manifest( + directory: &Path, +) -> Result<Option<(PathBuf, Option<PathBuf>)>, &'static str> { + let manifest = directory.join("package.json"); + let metadata = match std::fs::metadata(&manifest) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(_) => return Err("direct MCP Node package.json metadata is unavailable"), + }; + if !metadata.is_file() { + return Err("direct MCP Node package.json is not a regular file"); + } + if metadata.len() > MAX_NODE_PACKAGE_JSON_BYTES { + return Err("direct MCP Node package.json exceeds its read limit"); + } + let mut bytes = Vec::with_capacity(metadata.len() as usize); + std::fs::File::open(&manifest) + .map_err(|_| "direct MCP Node package.json is unreadable")? + .take(MAX_NODE_PACKAGE_JSON_BYTES + 1) + .read_to_end(&mut bytes) + .map_err(|_| "direct MCP Node package.json could not be read")?; + if bytes.len() as u64 > MAX_NODE_PACKAGE_JSON_BYTES { + return Err("direct MCP Node package.json exceeds its read limit"); } - if base.is_dir() { + let package: serde_json::Value = serde_json::from_slice(&bytes) + .map_err(|_| "direct MCP Node package.json is not valid JSON")?; + let main = package + .get("main") + .and_then(serde_json::Value::as_str) + .filter(|main| !main.is_empty()) + .map(|main| directory.join(main)); + Ok(Some((manifest, main))) +} + +fn resolve_node_relative_module( + parent: &Path, + specifier: &str, +) -> Result<Option<Vec<PathBuf>>, &'static str> { + fn resolve(base: &Path, depth: usize) -> Result<Option<Vec<PathBuf>>, &'static str> { + const EXTENSIONS: [&str; 8] = ["js", "mjs", "cjs", "json", "node", "ts", "mts", "cts"]; + if depth > 8 { + return Err("direct MCP Node package main resolution exceeds its depth bound"); + } + if base.is_file() { + return Ok(Some(vec![base.to_owned()])); + } + if base.extension().is_none() { + for extension in EXTENSIONS { + let candidate = base.with_extension(extension); + if candidate.is_file() { + return Ok(Some(vec![candidate])); + } + } + } + if !base.is_dir() { + return Ok(None); + } + + let package = read_node_package_manifest(base)?; + if let Some((manifest, Some(main))) = package.as_ref() { + if let Some(mut files) = resolve(main, depth + 1)? { + files.insert(0, manifest.clone()); + return Ok(Some(files)); + } + } + for index in ["index.js", "index.mjs", "index.cjs", "index.json"] { - candidates.push(base.join(index)); + let candidate = base.join(index); + if candidate.is_file() { + let mut files = vec![candidate]; + if let Some((manifest, _)) = package { + files.insert(0, manifest); + } + return Ok(Some(files)); + } } + Ok(None) } - candidates.into_iter().find(|candidate| candidate.is_file()) + + resolve(&parent.join(specifier), 0) } fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> { - fn is_static_module_name(name: &str) -> bool { + fn is_identifier(name: &str) -> bool { !name.is_empty() - && !name.starts_with('.') && name .bytes() - .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_' || byte == b'.') + .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_') + } + + fn is_module_name(name: &str, allow_relative: bool) -> bool { + let dots = name.bytes().take_while(|byte| *byte == b'.').count(); + if dots > 0 { + if !allow_relative { + return false; + } + let remainder = &name[dots..]; + return remainder.is_empty() || remainder.split('.').all(is_identifier); + } + !name.is_empty() && name.split('.').all(is_identifier) } let mut result = std::collections::BTreeSet::new(); @@ -844,7 +929,7 @@ fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> if let Some(imports) = statement.strip_prefix("import ") { for item in imports.split(',') { let name = item.split_ascii_whitespace().next().unwrap_or_default(); - if is_static_module_name(name) { + if is_module_name(name, false) { result.insert(name.to_owned()); } } @@ -853,22 +938,34 @@ fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> continue; }; let module = module.trim(); - if !is_static_module_name(module) { + if !is_module_name(module, true) { continue; } - result.insert(module.to_owned()); - for item in imported + let imported = imported .trim() .trim_start_matches('(') - .trim_end_matches(')') - .split(',') - { + .trim_end_matches(')'); + if module.bytes().all(|byte| byte == b'.') { + for item in imported.split(',') { + let name = item + .trim() + .split_ascii_whitespace() + .next() + .unwrap_or_default(); + if is_identifier(name) { + result.insert(format!("{module}{name}")); + } + } + continue; + } + result.insert(module.to_owned()); + for item in imported.split(',') { let name = item .trim() .split_ascii_whitespace() .next() .unwrap_or_default(); - if is_static_module_name(name) { + if is_identifier(name) { result.insert(format!("{module}.{name}")); } } @@ -878,7 +975,20 @@ fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> } fn resolve_python_local_modules(parent: &Path, specifier: &str) -> Vec<PathBuf> { - let components = specifier.split('.').collect::<Vec<_>>(); + let relative_levels = specifier.bytes().take_while(|byte| *byte == b'.').count(); + let module = &specifier[relative_levels..]; + let root = if relative_levels == 0 { + parent + } else if let Some(root) = parent.ancestors().nth(relative_levels - 1) { + root + } else { + return Vec::new(); + }; + let components = if module.is_empty() { + Vec::new() + } else { + module.split('.').collect::<Vec<_>>() + }; if components .iter() .any(|component| component.is_empty() || component.contains(['/', '\\'])) @@ -886,7 +996,7 @@ fn resolve_python_local_modules(parent: &Path, specifier: &str) -> Vec<PathBuf> return Vec::new(); } let mut result = std::collections::BTreeSet::new(); - let mut package = parent.to_owned(); + let mut package = root.to_owned(); for component in &components[..components.len().saturating_sub(1)] { package.push(component); let init = package.join("__init__.py"); @@ -894,15 +1004,20 @@ fn resolve_python_local_modules(parent: &Path, specifier: &str) -> Vec<PathBuf> result.insert(init); } } - let mut module = parent.to_owned(); + let mut module_path = root.to_owned(); for component in &components { - module.push(component); + module_path.push(component); } - for candidate in [ - module.with_extension("py"), - module.with_extension("pyw"), - module.join("__init__.py"), - ] { + let candidates = if components.is_empty() { + vec![root.join("__init__.py")] + } else { + vec![ + module_path.with_extension("py"), + module_path.with_extension("pyw"), + module_path.join("__init__.py"), + ] + }; + for candidate in candidates { if candidate.is_file() { result.insert(candidate); } @@ -969,13 +1084,15 @@ fn direct_script_dependency_paths( runtime: ScriptDependencyRuntime, ) -> Result<Vec<PathBuf>, &'static str> { match runtime { - ScriptDependencyRuntime::Node => node_relative_module_specifiers(source)? - .into_iter() - .map(|specifier| { - resolve_node_relative_module(parent, &specifier) - .ok_or("direct MCP Node relative module is missing") - }) - .collect(), + ScriptDependencyRuntime::Node => { + let mut result = Vec::new(); + for specifier in node_relative_module_specifiers(source)? { + let mut files = resolve_node_relative_module(parent, &specifier)? + .ok_or("direct MCP Node relative module is missing")?; + result.append(&mut files); + } + Ok(result) + } ScriptDependencyRuntime::Python => Ok(python_import_specifiers(source) .into_iter() .flat_map(|specifier| resolve_python_local_modules(parent, &specifier)) From 54fd0acbe38045322808102edc4dcfd6ec384da5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 04:01:47 -0700 Subject: [PATCH 381/493] fix(managed): handle recursive local script imports --- .../managed_mcp_capability/source_stage.rs | 81 +++++++++++++++++++ .../opencode_resources.rs | 68 ++++++++++++++-- 2 files changed, 144 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 29b007188..d149f1cbb 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1151,6 +1151,51 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"directory package main loaded\n"); } + #[test] + fn direct_node_script_falls_back_to_index_for_directory_package_main_dot() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let external = owned.path().join("external"); + let command = external.join("mcp/bin/start"); + let package = external.join("mcp/helpers"); + let manifest = package.join("package.json"); + let index = package.join("index.js"); + std::fs::create_dir_all(command.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&package).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nprocess.stdout.write(require('../helpers'))\n", + ) + .unwrap(); + std::fs::write(&manifest, r#"{"main":"."}"#).unwrap(); + std::fs::write( + &index, + "module.exports = 'directory index fallback loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&manifest.canonicalize().unwrap())); + assert!(plan[0] + .configured_resources + .contains(&index.canonicalize().unwrap())); + let hidden_external = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden_external).unwrap(); + let output = execute_selected_resource(&provider_command).unwrap(); + std::fs::rename(&hidden_external, &external).unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"directory index fallback loaded\n"); + } + #[test] fn direct_python_shebang_loads_sibling_module_after_all_copies() { use std::os::unix::fs::PermissionsExt; @@ -1213,6 +1258,42 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"nested package helper loaded\n"); } + #[test] + fn direct_python_multiline_package_import_loads_nested_module_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let package = tools.join("pkg"); + let command = tools.join("start"); + let helper = package.join("helper.py"); + std::fs::create_dir_all(&package).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env python3\nimport pkg\nprint(pkg.helper.VALUE)\n", + ) + .unwrap(); + std::fs::write( + package.join("__init__.py"), + "from . import (\n helper,\n)\n", + ) + .unwrap(); + std::fs::write(&helper, "VALUE = 'multiline package helper loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"multiline package helper loaded\n"); + } + #[test] fn direct_python_versioned_shebang_loads_sibling_module_after_all_copies() { use std::os::unix::fs::{symlink, PermissionsExt}; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 8d9225411..41b65bfc3 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -859,7 +859,11 @@ fn resolve_node_relative_module( parent: &Path, specifier: &str, ) -> Result<Option<Vec<PathBuf>>, &'static str> { - fn resolve(base: &Path, depth: usize) -> Result<Option<Vec<PathBuf>>, &'static str> { + fn resolve( + base: &Path, + depth: usize, + visited_directories: &std::collections::BTreeSet<PathBuf>, + ) -> Result<Option<Vec<PathBuf>>, &'static str> { const EXTENSIONS: [&str; 8] = ["js", "mjs", "cjs", "json", "node", "ts", "mts", "cts"]; if depth > 8 { return Err("direct MCP Node package main resolution exceeds its depth bound"); @@ -879,9 +883,16 @@ fn resolve_node_relative_module( return Ok(None); } + let canonical_directory = std::fs::canonicalize(base).unwrap_or_else(|_| base.to_owned()); + if visited_directories.contains(&canonical_directory) { + return Ok(None); + } + let mut next_visited_directories = visited_directories.clone(); + next_visited_directories.insert(canonical_directory); + let package = read_node_package_manifest(base)?; if let Some((manifest, Some(main))) = package.as_ref() { - if let Some(mut files) = resolve(main, depth + 1)? { + if let Some(mut files) = resolve(main, depth + 1, &next_visited_directories)? { files.insert(0, manifest.clone()); return Ok(Some(files)); } @@ -900,7 +911,54 @@ fn resolve_node_relative_module( Ok(None) } - resolve(&parent.join(specifier), 0) + resolve( + &parent.join(specifier), + 0, + &std::collections::BTreeSet::new(), + ) +} + +fn python_import_statements(source: &str) -> Vec<String> { + fn parenthesis_delta(source: &str) -> isize { + source.chars().fold(0, |depth, character| match character { + '(' => depth + 1, + ')' => depth - 1, + _ => depth, + }) + } + + let mut statements = Vec::new(); + let mut pending: Option<(String, isize)> = None; + for raw_line in source.lines() { + let line = raw_line.split('#').next().unwrap_or_default().trim(); + if line.is_empty() { + continue; + } + if let Some((statement, depth)) = &mut pending { + statement.push(' '); + statement.push_str(line); + *depth += parenthesis_delta(line); + if *depth <= 0 { + statements.push(std::mem::take(statement)); + pending = None; + } + continue; + } + let is_parenthesized_import = line.starts_with("from ") + && line + .split_once(" import ") + .is_some_and(|(_, imported)| imported.trim_start().starts_with('(')); + let depth = parenthesis_delta(line); + if is_parenthesized_import && depth > 0 { + pending = Some((line.to_owned(), depth)); + } else { + statements.push(line.to_owned()); + } + } + if let Some((statement, _)) = pending { + statements.push(statement); + } + statements } fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> { @@ -924,8 +982,8 @@ fn python_import_specifiers(source: &str) -> std::collections::BTreeSet<String> } let mut result = std::collections::BTreeSet::new(); - for line in source.lines() { - let statement = line.split('#').next().unwrap_or_default().trim(); + for statement in python_import_statements(source) { + let statement = statement.trim(); if let Some(imports) = statement.strip_prefix("import ") { for item in imports.split(',') { let name = item.split_ascii_whitespace().next().unwrap_or_default(); From f67fcae3e11e9bcf74a02f2de45c12cc94fcab0b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 04:10:47 -0700 Subject: [PATCH 382/493] fix(ui): clear stale managed launch failures --- src/components/TerminalView.tsx | 21 ++++++++++--- src/lib/recovery/managed-runtime-recovery.ts | 6 ++++ .../TerminalView.launchRetry.test.tsx | 24 ++++++++++++++ .../unit/lib/managed-runtime-recovery.test.ts | 31 +++++++++++++++++++ 4 files changed, 78 insertions(+), 4 deletions(-) diff --git a/src/components/TerminalView.tsx b/src/components/TerminalView.tsx index 42f9626a8..872a57d48 100644 --- a/src/components/TerminalView.tsx +++ b/src/components/TerminalView.tsx @@ -667,6 +667,11 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const content = root ? findPaneContent(root, paneId) : null return content?.kind === 'terminal' ? content.closeError : undefined }) + const storedLaunchFailure = useAppSelector((s) => { + const root = s.panes.layouts[tabId] + const content = root ? findPaneContent(root, paneId) : null + return content?.kind === 'terminal' ? content.launchFailure : undefined + }) const tab = useAppSelector((s) => s.tabs.tabs.find((t) => t.id === tabId)) const tabHasSinglePane = useAppSelector((s) => s.panes.layouts[tabId]?.type === 'leaf') const activeTabId = useAppSelector((s) => s.tabs.activeTabId) @@ -5968,7 +5973,12 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const status = managedTerminalPaneStatus({ ...contentRef.current, runtimeStatus: msg.status, recoverySummary: msg.recoverySummary, }, msg.status === 'running' || msg.status === 'recovering' ? msg.status : 'exited') - updateContent({ status, runtimeStatus: msg.status, recoverySummary: msg.recoverySummary }) + updateContent({ + status, + runtimeStatus: msg.status, + recoverySummary: msg.recoverySummary, + ...(status === 'running' || status === 'recovering' ? { launchFailure: undefined } : {}), + }) const statusTab = tabRef.current if (statusTab) dispatch(updateTab({ id: statusTab.id, updates: { status, @@ -5978,7 +5988,10 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te } else if (msg.status === 'running' || msg.status === 'recovering') { const terminal = contentRef.current const status = managedTerminalPaneStatus(terminal ?? {}, msg.status) - updateContent({ status }) + updateContent({ + status, + ...(status === 'running' || status === 'recovering' ? { launchFailure: undefined } : {}), + }) const statusTab = tabRef.current if (statusTab) dispatch(updateTab({ id: statusTab.id, updates: { status, @@ -7372,7 +7385,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te // Managed recovery and owner divergence own the current decision. An old // launch failure must not add a competing alert or obsolete retry action. const typedLaunchFailure = !managedRecoveryDecision && freshAgentOwnerDivergence === null - ? terminalContent.launchFailure + ? storedLaunchFailure : undefined const retryLaunch = () => { @@ -7390,7 +7403,7 @@ function TerminalView({ tabId, paneId, paneContent, hidden, focusEpoch = 0 }: Te const attachToNamedTerminal = () => { if (managedRecoveryDecision) return - const terminalId = terminalContent.launchFailure?.terminalId + const terminalId = storedLaunchFailure?.terminalId if (!terminalId) return dispatch(applyReattachToLiveTerminal({ tabId, paneId, terminalId })) } diff --git a/src/lib/recovery/managed-runtime-recovery.ts b/src/lib/recovery/managed-runtime-recovery.ts index 777b16bc1..2f06cf2a3 100644 --- a/src/lib/recovery/managed-runtime-recovery.ts +++ b/src/lib/recovery/managed-runtime-recovery.ts @@ -179,6 +179,12 @@ function updateExistingContent( return { ...existing, ...definedOnly(managedContent), + // An authoritative live/recovering supervisor projection proves a + // prior launch refusal was superseded by a successful retry. Clear its + // volatile card payload so it cannot render over the current terminal. + ...(managedContent.status === 'running' || managedContent.status === 'recovering' + ? { launchFailure: undefined } + : {}), ...(preserveLiveStreamId ? { streamId: existing.streamId } : {}), // Preserve a user pane's stable create key when the supervisor record // predates that field. Otherwise use the authoritative managed key. diff --git a/test/unit/client/components/TerminalView.launchRetry.test.tsx b/test/unit/client/components/TerminalView.launchRetry.test.tsx index 9f2e16c7b..5fe7da05b 100644 --- a/test/unit/client/components/TerminalView.launchRetry.test.tsx +++ b/test/unit/client/components/TerminalView.launchRetry.test.tsx @@ -464,6 +464,30 @@ describe('launch-time INVALID_TERMINAL_ID bounded retry', () => { expect(screen.queryByTestId('terminal-launch-failure-card')).not.toBeInTheDocument() }) + it('clears a stale launch-failure card when the current terminal reports running', async () => { + const failure = { code: 'LAUNCH_FAILED' as const, message: 'Previous create attempt failed', retryable: true } + const { store, paneContent } = makeStore({ + mode: 'opencode', + status: 'error', + terminalId: 'terminal-retried-live', + soulId: 'managed-soul-retried-live', + launchFailure: failure, + }) + await renderPane(store, paneContent) + expect(screen.getByTestId('terminal-launch-failure-card')).toBeInTheDocument() + + await act(async () => messageHandler!({ + type: 'terminal.status', + terminalId: 'terminal-retried-live', + status: 'running', + })) + + const content = (store.getState().panes.layouts[TAB] as { content: TerminalPaneContent }).content + expect(content.status).toBe('running') + expect(content.launchFailure).toBeUndefined() + expect(screen.queryByTestId('terminal-launch-failure-card')).not.toBeInTheDocument() + }) + it.each(['ordinary-mode', 'no-managed-capability', 'different-request'] as const)( 'does not retain preparation metadata outside its active managed request: %s', async (control) => { wsMocks.getServerCapabilities.mockReturnValue({ managedRuntimeV1: control !== 'no-managed-capability' }) diff --git a/test/unit/lib/managed-runtime-recovery.test.ts b/test/unit/lib/managed-runtime-recovery.test.ts index 8b5c0d66c..7462f29c2 100644 --- a/test/unit/lib/managed-runtime-recovery.test.ts +++ b/test/unit/lib/managed-runtime-recovery.test.ts @@ -245,6 +245,37 @@ describe('managed runtime recovery merge', () => { expect(content.initialCwd).toBe('/workspace/real') }) + it('clears a stale typed launch failure when the supervisor reports the managed terminal live', () => { + const state = baseState() + state.panes.layouts['user-tab'].content = { + kind: 'terminal', + createRequestId: 'create-one', + terminalId: 'terminal-one', + streamId: 'stream-one', + status: 'error', + mode: 'opencode', + shell: 'system', + soulId: 'soul-one', + launchFailure: { code: 'LAUNCH_FAILED', message: 'Previous create attempt failed', retryable: true }, + } + const store = storeWithState(state) + const liveSnapshot = snapshot([soul({ launchState: 'running', recoveryState: 'live' })]) + const plan = buildManagedRuntimeMergePlan(liveSnapshot, store.getState() as any) + + expect(plan.updates).toHaveLength(1) + expect(plan.updates[0].content).toMatchObject({ + status: 'running', + runtimeStatus: 'running', + recoverySummary: { recoveryState: 'live' }, + }) + expect((plan.updates[0].content as any).launchFailure).toBeUndefined() + + applyManagedRuntimeMergePlan(store as any, plan) + const content = (store.getState().panes.layouts['user-tab'] as any).content + expect(content.status).toBe('running') + expect(content.launchFailure).toBeUndefined() + }) + it('preserves the live host stream epoch when a same-terminal inventory refresh arrives', () => { const state = baseState() state.panes.layouts['user-tab'].content = { From ef10fcc94116ea32f194ffc76d70f95384f962d4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 04:29:30 -0700 Subject: [PATCH 383/493] fix(managed): stage bounded script resources from home --- .../managed_mcp_capability/source_stage.rs | 135 ++++++++++++++- .../opencode_resources.rs | 157 +++++++++--------- 2 files changed, 208 insertions(+), 84 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index d149f1cbb..b1197689a 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -100,12 +100,28 @@ mod configured_resource_tests { home: &Path, workspace: &Path, command: &Path, + ) -> (Vec<PlannedSourceOccurrence>, std::path::PathBuf) { + plan_and_stage_mcp_command_for_resource( + owned_root, + home, + workspace, + &[command.to_string_lossy().into_owned()], + command, + ) + } + + fn plan_and_stage_mcp_command_for_resource( + owned_root: &Path, + home: &Path, + workspace: &Path, + command: &[String], + selected_resource: &Path, ) -> (Vec<PlannedSourceOccurrence>, std::path::PathBuf) { let config = workspace.join("opencode.json"); std::fs::write( &config, serde_json::json!({ - "mcp": { "direct": { "type": "local", "command": [command] } }, + "mcp": { "direct": { "type": "local", "command": command } }, }) .to_string(), ) @@ -124,12 +140,12 @@ mod configured_resource_tests { .unwrap(); let units = resource_units(home, &home.join(".config/opencode"), workspace, &plan, None).unwrap(); - let command = command.canonicalize().unwrap(); + let selected_resource = selected_resource.canonicalize().unwrap(); let mount = units .iter() - .find(|(source, _)| source == &command) + .find(|(source, _)| source == &selected_resource) .map(|(_, mount)| mount) - .expect("the direct MCP command must be in the resource plan"); + .expect("the selected MCP script must be in the resource plan"); let effective_destination = Path::new(&mount.effective_destination); let provider_relative = effective_destination .strip_prefix(owned_root) @@ -1063,6 +1079,90 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"home node_modules loaded\n"); } + #[test] + fn node_command_script_in_home_keeps_home_node_modules_after_all_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let dependency = home.join("node_modules/home-fixture/index.js"); + std::fs::create_dir_all(dependency.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(&command, "process.stdout.write(require('home-fixture'))\n").unwrap(); + std::fs::write( + &dependency, + "module.exports = 'runtime node_modules loaded\\n'\n", + ) + .unwrap(); + + let command_args = vec!["node".to_owned(), command.to_string_lossy().into_owned()]; + let (plan, provider_script) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command_args, + &command, + ); + assert!(plan[0] + .configured_resources + .contains(&home.join("node_modules").canonicalize().unwrap())); + assert!(plan[0] + .configured_resources + .contains(&command.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .arg(&provider_script) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"runtime node_modules loaded\n"); + } + + #[test] + fn node_shebang_javascript_script_in_home_keeps_home_node_modules_after_all_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let dependency = home.join("node_modules/home-fixture/index.js"); + std::fs::create_dir_all(dependency.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nprocess.stdout.write(require('home-fixture'))\n", + ) + .unwrap(); + std::fs::write( + &dependency, + "module.exports = 'javascript shebang node_modules loaded\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&home.join("node_modules").canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"javascript shebang node_modules loaded\n"); + } + #[test] fn direct_node_script_loads_parent_relative_module_after_all_copies() { use std::os::unix::fs::PermissionsExt; @@ -1425,6 +1525,33 @@ mod configured_resource_tests { assert_eq!(output.stdout, expected); } + #[test] + fn shell_shebang_keeps_binary_support_resource_from_inline_dirname_expression() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let data = tools.join("data.bin"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(&command, "#!/bin/sh\ncat \"$(dirname \"$0\")/data.bin\"\n").unwrap(); + let expected = [0, 0xff, 0x81, b'R', b'I', b'F', b'F']; + std::fs::write(&data, expected).unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&data.canonicalize().unwrap())); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, expected); + } + #[test] fn shell_shebang_resolves_nested_quote_dirname_source_expression() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 41b65bfc3..d453ed409 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -480,16 +480,6 @@ fn is_mcp_script_field(field: &str) -> bool { ) } -fn is_mcp_direct_script_field(field: &str) -> bool { - matches!( - field, - "OpenCode mcp.direct-script" - | "OpenCode mcp.node-direct-script" - | "OpenCode mcp.shebang-script" - | "OpenCode mcp.shebang-node-script" - ) -} - fn is_mcp_node_script_field(field: &str) -> bool { matches!( field, @@ -685,24 +675,26 @@ fn shell_directory_relative_paths( directory_variables: &std::collections::BTreeSet<String>, ) -> std::collections::BTreeSet<String> { let mut result = std::collections::BTreeSet::new(); + let mut prefixes = std::collections::BTreeSet::new(); for variable in directory_variables { - for prefix in [format!("${variable}/"), format!("${{{variable}}}/")] { - let mut remaining = line; - while let Some(start) = remaining.find(&prefix) { - let suffix = &remaining[start + prefix.len()..]; - let length = suffix - .find(|character: char| { - character.is_whitespace() || "\"'`;|&)".contains(character) - }) - .unwrap_or(suffix.len()); - let relative = &suffix[..length]; - if !relative.is_empty() && !relative.contains(['$', '*', '?']) { - result.insert(relative.to_owned()); - } - remaining = &suffix[length..]; - if remaining.is_empty() { - break; - } + prefixes.insert(format!("${variable}/")); + prefixes.insert(format!("${{{variable}}}/")); + } + prefixes.insert("$(dirname \"$0\")/".to_owned()); + for prefix in prefixes { + let mut remaining = line; + while let Some(start) = remaining.find(&prefix) { + let suffix = &remaining[start + prefix.len()..]; + let length = suffix + .find(|character: char| character.is_whitespace() || "\"'`;|&)".contains(character)) + .unwrap_or(suffix.len()); + let relative = &suffix[..length]; + if !relative.is_empty() && !relative.contains(['$', '*', '?']) { + result.insert(relative.to_owned()); + } + remaining = &suffix[length..]; + if remaining.is_empty() { + break; } } } @@ -1935,64 +1927,69 @@ pub fn select( if command_parent == Path::new("/") || canonical_home.as_deref() == Some(command_parent) { - if is_mcp_direct_script_field(field) { - insert_resource( + // HOME and the filesystem root cannot be copied as a + // script's containing directory. Select the entry point + // itself and only its bounded language-specific closure. + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical.clone(), + field, + )?; + if is_mcp_node_script_field(field) { + insert_installed_package_ancestor_resources( + &canonical, + home, + declaring, &mut result, &mut resource_path_bytes, - canonical.clone(), field, )?; - if is_mcp_node_script_field(field) { - insert_direct_script_dependencies( - &canonical, - &dependency_scope, - declaring, - &mut result, - &mut resource_path_bytes, - field, - ScriptDependencyRuntime::Node, - )?; - } - if canonical_home.as_deref() == Some(command_parent) - && is_shell_script(&canonical) - { - insert_shell_script_dependencies( - &canonical, - &dependency_scope, - declaring, - &mut result, - &mut resource_path_bytes, - field, - )?; - } - if is_python_script(&canonical) { - insert_direct_script_dependencies( - &canonical, - &dependency_scope, - declaring, - &mut result, - &mut resource_path_bytes, - field, - ScriptDependencyRuntime::Python, - )?; - } - if is_ruby_script(&canonical) { - insert_direct_script_dependencies( - &canonical, - &dependency_scope, - declaring, - &mut result, - &mut resource_path_bytes, - field, - ScriptDependencyRuntime::Ruby, - )?; - } - continue; + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Node, + )?; } - return Err(fail( - field, - "standalone MCP script requires copying an unbounded account or system directory", - )); + if canonical_home.as_deref() == Some(command_parent) + && is_shell_script(&canonical) + { + insert_shell_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + } + if is_python_script(&canonical) { + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Python, + )?; + } + if is_ruby_script(&canonical) { + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Ruby, + )?; + } + continue; } if is_mcp_node_script_field(field) { insert_installed_package_ancestor_resources( From 4120eca4b452999bad7ac25f31cbd7d545d136cf Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:40:15 -0700 Subject: [PATCH 384/493] fix(managed): preserve configured script resource closures --- .../opencode_resources.rs | 233 ++++++++++++++++-- .../managed_provider_references/sources.rs | 86 +++++++ .../managed-provider-environment-target.ts | 34 ++- scripts/testing/runtime-sandbox.ts | 5 +- src/App.tsx | 14 +- ...aged-provider-environment-external.spec.ts | 4 +- test/fixtures/providers/parity-provider.mjs | 67 ++++- test/helpers/provider-parity-environment.ts | 28 +++ .../components/App.ws-bootstrap.test.tsx | 36 +++ .../components/component-edge-cases.test.tsx | 5 +- .../testing/runtime-test-broker.test.ts | 33 ++- 11 files changed, 509 insertions(+), 36 deletions(-) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index d453ed409..12592ee80 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -172,7 +172,11 @@ fn selected_string(raw: &str) -> Result<Option<String>, &'static str> { .map(Some) .map_err(|_| "invalid selected resource path") } -fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &'static str> { +fn nested_fields( + raw: &str, + tui: bool, + skip_owned_freshell_mcp: bool, +) -> Result<Vec<(&'static str, String)>, &'static str> { let top = object_members(raw, 0)?; let mut result = Vec::new(); if tui { @@ -209,6 +213,9 @@ fn nested_fields(raw: &str, tui: bool) -> Result<Vec<(&'static str, String)>, &' let name = serde_json::from_str::<String>(&mcp[name]) .map_err(|_| "invalid selected MCP server name")?; if names.insert(name.clone()) { + if skip_owned_freshell_mcp && name == "freshell" { + continue; + } winners.push((name, range)); } } @@ -701,6 +708,30 @@ fn shell_directory_relative_paths( result } +fn shell_line_invokes_node(line: &str) -> bool { + let mut command = line.trim_start(); + while let Some(rest) = command + .strip_prefix("exec ") + .or_else(|| command.strip_prefix("command ")) + { + command = rest.trim_start(); + } + let executable = command + .split_ascii_whitespace() + .next() + .unwrap_or_default() + .trim_matches(['"', '\'']); + let name = Path::new(executable) + .file_name() + .unwrap_or_default() + .to_string_lossy() + .to_ascii_lowercase(); + matches!( + name.strip_suffix(".exe").unwrap_or(&name), + "node" | "nodejs" + ) +} + enum JavaScriptToken { Identifier(String), String(Option<String>), @@ -813,6 +844,54 @@ fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static Ok(result.into_iter().collect()) } +fn node_relative_resource_specifiers( + source: &str, +) -> Result<std::collections::BTreeSet<String>, &'static str> { + let tokens = javascript_tokens(source)?; + let mut result = std::collections::BTreeSet::new(); + for index in 0..tokens.len().saturating_sub(10) { + let Some(JavaScriptToken::Identifier(new_keyword)) = tokens.get(index) else { + continue; + }; + let Some(JavaScriptToken::Identifier(url_constructor)) = tokens.get(index + 1) else { + continue; + }; + let Some(JavaScriptToken::Punctuation(b'(')) = tokens.get(index + 2) else { + continue; + }; + let Some(JavaScriptToken::String(Some(specifier))) = tokens.get(index + 3) else { + continue; + }; + let is_import_meta_url = matches!( + tokens.get(index + 4), + Some(JavaScriptToken::Punctuation(b',')) + ) && matches!(tokens.get(index + 5), Some(JavaScriptToken::Identifier(name)) if name == "import") + && matches!( + tokens.get(index + 6), + Some(JavaScriptToken::Punctuation(b'.')) + ) + && matches!(tokens.get(index + 7), Some(JavaScriptToken::Identifier(name)) if name == "meta") + && matches!( + tokens.get(index + 8), + Some(JavaScriptToken::Punctuation(b'.')) + ) + && matches!(tokens.get(index + 9), Some(JavaScriptToken::Identifier(name)) if name == "url") + && matches!( + tokens.get(index + 10), + Some(JavaScriptToken::Punctuation(b')')) + ); + if new_keyword == "new" + && url_constructor == "URL" + && is_import_meta_url + && (specifier.starts_with("./") || specifier.starts_with("../")) + && !specifier.contains(['?', '#', '\\', '$']) + { + result.insert(specifier.clone()); + } + } + Ok(result) +} + fn read_node_package_manifest( directory: &Path, ) -> Result<Option<(PathBuf, Option<PathBuf>)>, &'static str> { @@ -1135,13 +1214,18 @@ fn direct_script_dependency_paths( ) -> Result<Vec<PathBuf>, &'static str> { match runtime { ScriptDependencyRuntime::Node => { - let mut result = Vec::new(); + let mut result = std::collections::BTreeSet::new(); for specifier in node_relative_module_specifiers(source)? { - let mut files = resolve_node_relative_module(parent, &specifier)? + let files = resolve_node_relative_module(parent, &specifier)? .ok_or("direct MCP Node relative module is missing")?; - result.append(&mut files); + result.extend(files); } - Ok(result) + result.extend( + node_relative_resource_specifiers(source)? + .into_iter() + .map(|specifier| parent.join(specifier)), + ); + Ok(result.into_iter().collect()) } ScriptDependencyRuntime::Python => Ok(python_import_specifiers(source) .into_iter() @@ -1224,7 +1308,13 @@ fn insert_direct_script_dependencies( "direct MCP dependency escapes its selected script scope", )); } - insert_resource(result, resource_path_bytes, dependency.clone(), field)?; + insert_resource( + result, + resource_path_bytes, + dependency.clone(), + field, + declaring, + )?; pending.push(dependency); } } @@ -1292,15 +1382,18 @@ fn insert_shell_script_dependencies( shell_source_path(script_parent, &operand, &directory_variables) .map_err(fail)? { - candidates.insert(candidate, true); + candidates.insert(candidate, (true, false)); } } + let invokes_node = shell_line_invokes_node(line); for relative in shell_directory_relative_paths(line, &directory_variables) { + let parse_as_node_script = invokes_node && is_node_script_path(&relative); candidates .entry(script_parent.join(relative)) - .or_insert(false); + .and_modify(|kind: &mut (bool, bool)| kind.1 |= parse_as_node_script) + .or_insert((false, parse_as_node_script)); } - for (candidate, parse_as_script) in candidates { + for (candidate, (parse_as_script, parse_as_node_script)) in candidates { let dependency = checked(&candidate, field, declaring)?; if !dependency.is_file() { return Err(fail("direct MCP shell source is not a regular file")); @@ -1310,7 +1403,32 @@ fn insert_shell_script_dependencies( "direct MCP shell source escapes its selected script scope", )); } - insert_resource(result, resource_path_bytes, dependency.clone(), field)?; + insert_resource( + result, + resource_path_bytes, + dependency.clone(), + field, + declaring, + )?; + if parse_as_node_script { + insert_installed_package_ancestor_resources( + &dependency, + scope, + declaring, + result, + resource_path_bytes, + field, + )?; + insert_direct_script_dependencies( + &dependency, + scope, + declaring, + result, + resource_path_bytes, + field, + ScriptDependencyRuntime::Node, + )?; + } pending.push((dependency, parse_as_script)); } } @@ -1496,7 +1614,40 @@ fn insert_resource( resource_path_bytes: &mut usize, path: PathBuf, field: &'static str, + declaring: &Path, ) -> Result<(), ResourceError> { + if path.is_dir() { + let resource_leaf = path + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()); + tracing::info!( + event = "provider.opencode.resource.directory_selected", + field, + resource_leaf, + source_config_leaf = declaring + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()), + source_config_parent_leaf = declaring + .parent() + .and_then(Path::file_name) + .map(|leaf| leaf.to_string_lossy().into_owned()), + source_config_grandparent_leaf = declaring + .parent() + .and_then(Path::parent) + .and_then(Path::file_name) + .map(|leaf| leaf.to_string_lossy().into_owned()), + resource_parent_leaf = path + .parent() + .and_then(Path::file_name) + .map(|leaf| leaf.to_string_lossy().into_owned()), + resource_grandparent_leaf = path + .parent() + .and_then(Path::parent) + .and_then(Path::file_name) + .map(|leaf| leaf.to_string_lossy().into_owned()), + "Selected OpenCode directory resource" + ); + } if !result.contains(&path) { let bytes = path.as_os_str().len(); if result.len() >= 100_000 @@ -1537,7 +1688,7 @@ fn insert_installed_package_ancestor_resources( match std::fs::symlink_metadata(&candidate) { Ok(_) => { checked(&candidate, field, declaring)?; - insert_resource(result, resource_path_bytes, candidate, field)?; + insert_resource(result, resource_path_bytes, candidate, field, declaring)?; } Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} Err(_) => { @@ -1569,10 +1720,21 @@ pub fn select( let mut resource_path_bytes = 0usize; let mut variables = std::collections::BTreeSet::new(); let raw = source.references.unexpanded_source(); + let skip_owned_freshell_mcp = source.source.lane == super::sources::SourceLane::Main + && !freshell_platform::opencode_config::source_owns_freshell_mcp( + raw, + &source.source.logical_path, + &source.instance_cwd, + ) + .map_err(|_| fail("OpenCode resource", "project MCP ownership check failed"))?; let mut selections = fields(raw).map_err(|cause| fail("OpenCode resource", cause))?; selections.extend( - nested_fields(raw, source.source.lane == super::sources::SourceLane::Tui) - .map_err(|cause| fail("OpenCode resource", cause))?, + nested_fields( + raw, + source.source.lane == super::sources::SourceLane::Tui, + skip_owned_freshell_mcp, + ) + .map_err(|cause| fail("OpenCode resource", cause))?, ); let mut current_mcp_working_dir: Option<PathBuf> = None; let mut current_mcp_resource_path: Option<PathBuf> = None; @@ -1662,6 +1824,7 @@ pub fn select( &mut resource_path_bytes, package_root.clone(), "OpenCode mcp.command", + declaring, )?; insert_installed_package_ancestor_resources( &package_root, @@ -1751,7 +1914,13 @@ pub fn select( if !canonical.is_file() { return Err(fail(field, "selected file resource is not a regular file")); } - insert_resource(&mut result, &mut resource_path_bytes, selected, field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + selected, + field, + declaring, + )?; matched = true; } if matched { @@ -1815,7 +1984,13 @@ pub fn select( if !canonical.is_file() { return Err(fail(field, "selected file resource is not a regular file")); } - insert_resource(&mut result, &mut resource_path_bytes, path, field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + path, + field, + declaring, + )?; continue; } if is_mcp_command_path_field(field) { @@ -1851,6 +2026,7 @@ pub fn select( &mut resource_path_bytes, package_root.clone(), field, + declaring, )?; insert_installed_package_ancestor_resources( &package_root, @@ -1868,6 +2044,7 @@ pub fn select( &mut resource_path_bytes, canonical.clone(), field, + declaring, )?; if is_mcp_node_script_field(field) { insert_installed_package_ancestor_resources( @@ -1935,6 +2112,7 @@ pub fn select( &mut resource_path_bytes, canonical.clone(), field, + declaring, )?; if is_mcp_node_script_field(field) { insert_installed_package_ancestor_resources( @@ -2010,12 +2188,19 @@ pub fn select( &mut resource_path_bytes, command_parent.to_owned(), field, + declaring, )?; continue; } // A runtime executable without a package manifest is an exact-file // resource. Selecting its parent could copy system directories. - insert_resource(&mut result, &mut resource_path_bytes, canonical, field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical, + field, + declaring, + )?; continue; } if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { @@ -2029,7 +2214,13 @@ pub fn select( current_mcp_working_dir = Some(canonical); current_mcp_resource_path = Some(path.clone()); } - insert_resource(&mut result, &mut resource_path_bytes, path, field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + path, + field, + declaring, + )?; continue; } // The native global plugin discovery unit is already selected in the @@ -2060,7 +2251,13 @@ pub fn select( if scope == home || scope == Path::new("/") { return Err(fail(field, "select a bounded plugin or package directory")); } - insert_resource(&mut result, &mut resource_path_bytes, scope.clone(), field)?; + insert_resource( + &mut result, + &mut resource_path_bytes, + scope.clone(), + field, + declaring, + )?; insert_installed_package_ancestor_resources( &scope, home, diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index a8393ff74..7e7717efc 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1551,4 +1551,90 @@ mod tests { .any(|path| path.starts_with("/opt/freshell-mcp") || path == Path::new("/opt/package.json"))); } + + #[test] + fn generated_project_freshell_mcp_is_not_selected_but_user_replacement_is() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let checkout = owned.path().join("freshell-checkout"); + let config = workspace.join(".opencode/opencode.json"); + let state = workspace.join(".opencode/.freshell-mcp-state.json"); + let loader = checkout.join("node_modules/tsx/dist/loader.mjs"); + let server = checkout.join("tools/freshell-mcp/server.ts"); + let user_root = owned.path().join("user-mcp"); + let user_script = user_root.join("server.mjs"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(config.parent().unwrap()).unwrap(); + std::fs::create_dir_all(loader.parent().unwrap()).unwrap(); + std::fs::create_dir_all(server.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&user_root).unwrap(); + std::fs::write( + checkout.join("package.json"), + r#"{"name":"freshell","type":"module"}"#, + ) + .unwrap(); + std::fs::write(&loader, "export {}\n").unwrap(); + std::fs::write(&server, "export {}\n").unwrap(); + std::fs::write(&user_script, "export {}\n").unwrap(); + let owned_command = serde_json::json!([ + "node", + "--import", + loader.to_string_lossy(), + server.to_string_lossy() + ]); + std::fs::write( + &config, + serde_json::json!({ + "mcp": {"freshell": {"type": "local", "command": owned_command}} + }) + .to_string(), + ) + .unwrap(); + std::fs::write( + &state, + serde_json::json!({"createdEntry": true, "ownedCommand": owned_command}).to_string(), + ) + .unwrap(); + + let plan_for = || { + plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap() + }; + + let generated = plan_for(); + assert!( + !generated[0] + .configured_resources + .iter() + .any(|path| path.starts_with(&checkout)), + "managed OpenCode already provides Freshell MCP from its runtime image" + ); + + let user_command = serde_json::json!(["node", user_script]); + std::fs::write( + &config, + serde_json::json!({ + "mcp": {"freshell": {"type": "local", "command": user_command}} + }) + .to_string(), + ) + .unwrap(); + let replaced = plan_for(); + let canonical_user_root = user_root.canonicalize().unwrap(); + assert!(replaced[0] + .configured_resources + .iter() + .any(|path| path == &canonical_user_root || path.starts_with(&canonical_user_root))); + } } diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index acae822d9..221031466 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -115,11 +115,14 @@ process.stdin.on('data', (chunk) => { }) ` -const directMcpServerSource = String.raw`const fs = require('node:fs') +const directMcpServerSource = String.raw`import fs from 'node:fs' +import sibling from './helper.mjs' +import dependency from 'managed-direct-fixture-dependency' +const schema = JSON.parse(fs.readFileSync(new URL('./schema.json', import.meta.url), 'utf8')) const [receiptPath, expectedNonce] = process.argv.slice(2) if (!receiptPath || !expectedNonce) process.exit(12) fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper: process.env.MANAGED_DIRECT_MCP_HELPER, - cwd: process.cwd(), scriptPath: process.argv[1] })) + sibling, dependency, schemaNonce: schema.nonce, cwd: process.cwd(), scriptPath: process.argv[1] })) let buffered = '' process.stdin.setEncoding('utf8') process.stdin.on('data', chunk => { @@ -300,6 +303,8 @@ async function configure(root: string, nonce: string, candidateSha: string, runI await fs.mkdir(mcpNested, { recursive: true, mode: 0o755 }) await fs.mkdir(directMcp, { recursive: true, mode: 0o755 }) await fs.mkdir(mcpParentDependency, { recursive: true, mode: 0o755 }) + const directMcpDependency = path.join(directMcp, 'node_modules/managed-direct-fixture-dependency') + await fs.mkdir(directMcpDependency, { recursive: true, mode: 0o755 }) await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) await fs.chmod(home, 0o700) await fs.chmod(root, 0o751) @@ -339,12 +344,20 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.writeFile(path.join(mcpPackage, 'server.mjs'), managedMcpServerSource) const directCommand = path.join(directMcp, 'start') await fs.writeFile(directCommand, - '#!/bin/sh\nSCRIPT_DIR=$(dirname "$0")\n. "$SCRIPT_DIR/helper.sh"\nexec node "$SCRIPT_DIR/server.js" "$@"\n', + '#!/bin/sh\nSCRIPT_DIR=$(dirname "$0")\n. "$SCRIPT_DIR/helper.sh"\nexec node "$SCRIPT_DIR/server.mjs" "$@"\n', { mode: 0o751 }) await fs.chmod(directCommand, 0o751) await fs.writeFile(path.join(directMcp, 'helper.sh'), `export MANAGED_DIRECT_MCP_HELPER='${nonce}'\n`) - await fs.writeFile(path.join(directMcp, 'server.js'), directMcpServerSource) + await fs.writeFile(path.join(directMcp, 'helper.mjs'), `export default ${JSON.stringify(nonce)};\n`) + await fs.writeFile(path.join(directMcp, 'schema.json'), JSON.stringify({ nonce })) + await writeJson(path.join(directMcpDependency, 'package.json'), { + name: 'managed-direct-fixture-dependency', + type: 'module', + exports: './index.mjs', + }) + await fs.writeFile(path.join(directMcpDependency, 'index.mjs'), `export default ${JSON.stringify(nonce)};\n`) + await fs.writeFile(path.join(directMcp, 'server.mjs'), directMcpServerSource) await writeJson(path.join(mcpParentDependency, 'package.json'), { name: 'managed-fixture-mcp-dependency', type: 'module', @@ -1288,13 +1301,17 @@ export async function runManagedEnvironmentTarget() { readJson(path.join(selected.workspace, 'preflight/.managed-fixture/direct-mcp-receipt.json'))) assert.equal(preflightDirectMcp.expectedNonce, nonce) assert.equal(preflightDirectMcp.helper, nonce) - assert(preflightDirectMcp.scriptPath.endsWith('/direct-mcp/server.js')) + assert.equal(preflightDirectMcp.sibling, nonce) + assert.equal(preflightDirectMcp.dependency, nonce) + assert.equal(preflightDirectMcp.schemaNonce, nonce) + assert(preflightDirectMcp.scriptPath.endsWith('/direct-mcp/server.mjs')) const preflightView = await poll('genuine managed preflight native identity', () => observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) assert.deepEqual(await webChildren(rig), [], 'Managed child must belong to the disposable session host, not the host web process') proof.preflight = { terminalId: created.terminalId, soulId: preflightView.soulId, incarnationId: preflightView.incarnationId, - stage: await stageProof(rig, preflightView, nonce), child: preflightChild, mcpServer: preflightMcp } + stage: await stageProof(rig, preflightView, nonce), child: preflightChild, mcpServer: preflightMcp, + directMcpServer: preflightDirectMcp } const toolHome = path.join(root, 'tools') await fs.mkdir(toolHome, { mode: 0o700 }) const tool = path.join(toolHome, 'cloudflared') @@ -1395,7 +1412,10 @@ export async function runManagedEnvironmentTarget() { readJson(path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'))) assert.equal(directMcpServer.expectedNonce, nonce) assert.equal(directMcpServer.helper, nonce) - assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.js')) + assert.equal(directMcpServer.sibling, nonce) + assert.equal(directMcpServer.dependency, nonce) + assert.equal(directMcpServer.schemaNonce, nonce) + assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.mjs')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer } diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index 4670b34e1..20f3240e7 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -473,7 +473,10 @@ export class RuntimeHarness { '-v', `${volumeName}:/var/lib/freshell-supervisor:rw`, ] if (this.phase >= 2) { - if (this.fixtureInputs && this.providerFixtureHome !== undefined) { + if (this.providerFixtureHome !== undefined) { + // The supervisor resolves provider configuration-file references in + // this explicitly supplied synthetic HOME before it can mount them + // into a workload. Keep ambient host homes unmounted. const home = validateOwnedFixturePath(this.providerFixtureHome, 'directory', 'provider fixture root') args.push('-v', `${home}:${home}:ro`) } diff --git a/src/App.tsx b/src/App.tsx index 4f310d225..4507b5b7d 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -8,7 +8,7 @@ import { patchSessionRunningStateFromTerminalMeta, resetWsSnapshotReceived, } from '@/store/sessionsSlice' -import { addTab, closeTab, reopenClosedTab, switchToNextTab, switchToPrevTab } from '@/store/tabsSlice' +import { addTab, closeTab, reopenClosedTab, switchToNextTab, switchToPrevTab, updateTab } from '@/store/tabsSlice' import { ApiError, api, @@ -110,7 +110,7 @@ import { updateSettingsLocal } from '@/store/settingsSlice' import { setTerminalMetaSnapshot, upsertTerminalMeta, removeTerminalMeta } from '@/store/terminalMetaSlice' import { clearAllReconcilePendingPanes, clearDeadSessionAdjudication, clearDeadTerminals, clearReconcileWarming, clearTerminalLiveHandles, setReconcilePendingPanes, updatePaneContent } from '@/store/panesSlice' import { collectPaneEntries } from '@/lib/pane-utils' -import { isRetainedManagedTerminal, managedTerminalProjection } from '@/lib/managed-terminal.js' +import { isRetainedManagedTerminal, managedTerminalProjection, managedTerminalResumePaneStatus } from '@/lib/managed-terminal.js' import { addTerminalFreshRecoveryRequestId, addTerminalRestoreRequestId, setPaneReconcileActive } from '@/lib/terminal-restore' import { reconcileTerminalSessionAssociation } from '@/lib/terminal-session-association' import { foldTerminalInventoryTitles } from '@/lib/terminal-inventory-titles' @@ -1645,16 +1645,24 @@ export default function App() { }) } if (isRetainedManagedTerminal(terminal)) { + const managedStatus = managedTerminalResumePaneStatus(terminal) + const isLive = managedStatus === 'running' || managedStatus === 'recovering' for (const [tabId, layout] of Object.entries(appStore.getState().panes.layouts)) { + let matchedTerminal = false for (const { paneId, content } of collectPaneEntries(layout)) { if (content.kind === 'terminal' && content.terminalId === terminal.terminalId) { + matchedTerminal = true dispatch(updatePaneContent({ tabId, paneId, content: { ...content, ...managedTerminalProjection(terminal), - ...(terminal.status === 'exited' ? { status: 'exited' } : {}), + status: managedStatus, + ...(isLive ? { launchFailure: undefined } : {}), } })) } } + if (matchedTerminal && isLive) { + dispatch(updateTab({ id: tabId, updates: { status: managedStatus } })) + } } } } diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 032dbcab1..6227910ff 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -217,7 +217,9 @@ test(healthyTitle, async ({ page }, info) => { directMcpReceipt = await readReceipt(page, `${pane.cwd}/.managed-fixture/direct-mcp-receipt.json`) return directMcpReceipt }, { timeout: 120_000 }).toMatchObject({ expectedNonce: fixture.expectedNonce, helper: fixture.expectedNonce, cwd: pane.cwd }) - expect(directMcpReceipt.scriptPath).toMatch(/\/direct-mcp\/server\.js$/) + expect(directMcpReceipt).toMatchObject({ sibling: fixture.expectedNonce, + dependency: fixture.expectedNonce, schemaNonce: fixture.expectedNonce }) + expect(directMcpReceipt.scriptPath).toMatch(/\/direct-mcp\/server\.mjs$/) expect(receipt.soulId).toBeTruthy() expect(receipt.incarnationId).toBeTruthy() expect(receipt.nativeSessionId).toBeTruthy() diff --git a/test/fixtures/providers/parity-provider.mjs b/test/fixtures/providers/parity-provider.mjs index cf5979561..9d54f32a0 100644 --- a/test/fixtures/providers/parity-provider.mjs +++ b/test/fixtures/providers/parity-provider.mjs @@ -13,8 +13,69 @@ const argv = process.argv.slice(2) const cwd = process.cwd() const launchId = randomUUID() +function parseJsonc(raw) { + const chars = [...raw] + let stripped = '' + let inString = false + for (let i = 0; i < chars.length;) { + const current = chars[i] + if (inString) { + stripped += current + if (current === '\\' && i + 1 < chars.length) stripped += chars[++i] + else if (current === '"') inString = false + i++ + } else if (current === '"') { + inString = true + stripped += current + i++ + } else if (current === '/' && chars[i + 1] === '/') { + stripped += ' ' + i += 2 + while (i < chars.length && chars[i] !== '\n' && chars[i] !== '\r') { + stripped += ' ' + i++ + } + } else if (current === '/' && chars[i + 1] === '*') { + stripped += ' ' + i += 2 + let closed = false + while (i < chars.length) { + if (chars[i] === '*' && chars[i + 1] === '/') { + stripped += ' ' + i += 2 + closed = true + break + } + stripped += chars[i] === '\n' || chars[i] === '\r' ? chars[i] : ' ' + i++ + } + if (!closed) throw new SyntaxError('unterminated JSONC block comment') + } else { + stripped += current + i++ + } + } + + const withoutTrailingCommas = [...stripped] + inString = false + for (let i = 0; i < withoutTrailingCommas.length; i++) { + const current = withoutTrailingCommas[i] + if (inString) { + if (current === '\\') i++ + else if (current === '"') inString = false + } else if (current === '"') { + inString = true + } else if (current === ',') { + let next = i + 1 + while (next < withoutTrailingCommas.length && /\s/.test(withoutTrailingCommas[next])) next++ + if (withoutTrailingCommas[next] === '}' || withoutTrailingCommas[next] === ']') withoutTrailingCommas[i] = ' ' + } + } + return JSON.parse(withoutTrailingCommas.join('')) +} + function readJson(file) { - try { return JSON.parse(fs.readFileSync(file, 'utf8')) } catch { return null } + try { return parseJsonc(fs.readFileSync(file, 'utf8')) } catch { return null } } function existing(relative) { @@ -135,7 +196,7 @@ export async function observeOpenCodeParitySelection(providerHome, workspace, tu if (Object.keys(project.mcp).length === 0) delete project.mcp; } const tuiConfig = tuiPath ? fs.readFileSync(tuiPath, 'utf8') : null; - const tui = tuiConfig ? JSON.parse(tuiConfig) : null; + const tui = tuiConfig ? parseJsonc(tuiConfig) : null; const spec = tui?.plugin?.find(value => typeof value === 'string' && (value.endsWith('/freshell-rebind-plugin.ts') || value.endsWith('/.freshell/opencode'))); let rebindPlugin = null; @@ -159,6 +220,7 @@ export async function observeOpenCodeParitySelection(providerHome, workspace, tu return { projectedProjectConfig: project ? JSON.stringify(project) : null, tuiConfig, + tuiConfigParsed: tui, rebindPlugin, }; } @@ -188,6 +250,7 @@ async function runProvider() { const selectedConfigReferences = provider === 'opencode' ? (() => { const selected = readJson(path.join(providerHome, 'opencode.jsonc'))?.agent?.parity + if (selected?.prompt == null && selected?.description == null) return [] return [selected?.prompt, selected?.description].map(value => { if (typeof value !== 'string') throw new Error('configured file reference missing') return value.replace(/\{file:([^}]+)\}/g, (_match, file) => fs.readFileSync(path.resolve(providerHome, file), 'utf8')) diff --git a/test/helpers/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts index d632de59f..94b6b0d2d 100644 --- a/test/helpers/provider-parity-environment.ts +++ b/test/helpers/provider-parity-environment.ts @@ -46,6 +46,34 @@ export function providerParityFailureDiagnostics(logs: string) { } } return { + directoryResourceSelections: events.filter(value => value.event === 'provider.opencode.resource.directory_selected') + .slice(-32).map(value => ({ + field: typeof value.field === 'string' ? value.field : undefined, + resourceLeaf: typeof value.resource_leaf === 'string' ? value.resource_leaf : undefined, + sourceConfigLeaf: typeof value.source_config_leaf === 'string' ? value.source_config_leaf : undefined, + sourceConfigParentLeaf: typeof value.source_config_parent_leaf === 'string' + ? value.source_config_parent_leaf : undefined, + sourceConfigGrandparentLeaf: typeof value.source_config_grandparent_leaf === 'string' + ? value.source_config_grandparent_leaf : undefined, + resourceParentLeaf: typeof value.resource_parent_leaf === 'string' ? value.resource_parent_leaf : undefined, + resourceGrandparentLeaf: typeof value.resource_grandparent_leaf === 'string' + ? value.resource_grandparent_leaf : undefined, + })), + resourceCopyFailures: events.filter(value => value.event === 'provider.environment.copy.failed') + .slice(-16).map(value => ({ + provider: typeof value.provider === 'string' ? value.provider : undefined, + operation: typeof value.operation === 'string' ? value.operation : undefined, + relativePath: typeof value.relative_path === 'string' ? value.relative_path : undefined, + sourceScope: typeof value.resource_source_scope === 'string' ? value.resource_source_scope : undefined, + sourceLeaf: typeof value.resource_source_leaf === 'string' ? value.resource_source_leaf : undefined, + sourceOrigin: typeof value.resource_source_origin === 'string' ? value.resource_source_origin : undefined, + sourceLane: typeof value.resource_source_lane === 'string' ? value.resource_source_lane : undefined, + sourceConfigLeaf: typeof value.resource_source_config_leaf === 'string' ? value.resource_source_config_leaf : undefined, + sourceConfigScope: typeof value.resource_source_config_scope === 'string' ? value.resource_source_config_scope : undefined, + limitKind: typeof value.limit_kind === 'string' ? value.limit_kind : undefined, + observed: typeof value.observed === 'number' ? value.observed : undefined, + permitted: typeof value.permitted === 'number' ? value.permitted : undefined, + })), transactionEvents: events.flatMap(value => { return ['provider.projection.transaction.started', 'provider.projection.transaction.failed', 'provider.projection.transaction.rollback', 'provider.projection.transaction.committed'].includes(value.event) diff --git a/test/unit/client/components/App.ws-bootstrap.test.tsx b/test/unit/client/components/App.ws-bootstrap.test.tsx index 29e204002..f2baacf38 100644 --- a/test/unit/client/components/App.ws-bootstrap.test.tsx +++ b/test/unit/client/components/App.ws-bootstrap.test.tsx @@ -1061,6 +1061,42 @@ describe('App WS bootstrap recovery', () => { expect(layout.content).toMatchObject({terminalId:'retained-managed',streamId:'retained-epoch',createRequestId:'retained-create',status:'exited'}) }) + it('clears a stale managed launch failure when inventory confirms the retained terminal is live', async () => { + const store = createStore({ + tabs: [{ id: 'inventory-live-tab', mode: 'opencode', status: 'error' }], + panes: { + layouts: { + 'inventory-live-tab': { + type: 'leaf', id: 'inventory-live-pane', content: { + kind: 'terminal', mode: 'opencode', status: 'error', + terminalId: 'inventory-live-terminal', streamId: 'inventory-live-stream', + createRequestId: 'inventory-live-create', soulId: 'inventory-live-soul', + launchFailure: { code: 'LAUNCH_FAILED', message: 'old launch failed', retryable: true }, + }, + }, + }, + activePane: { 'inventory-live-tab': 'inventory-live-pane' }, + }, + }) + render(<Provider store={store}><App /></Provider>) + await waitFor(() => expect(messageHandler).toBeTypeOf('function')) + + act(() => messageHandler?.({ + type: 'terminal.inventory', + terminals: [{ + terminalId: 'inventory-live-terminal', mode: 'opencode', status: 'running', + runtimeStatus: 'running', soulId: 'inventory-live-soul', incarnationId: 'inventory-live-incarnation', + }], + })) + + const layout = store.getState().panes.layouts['inventory-live-tab'] + expect(layout.type).toBe('leaf') + if (layout.type !== 'leaf') throw new Error('expected a leaf pane') + expect(layout.content).toMatchObject({ status: 'running', runtimeStatus: 'running' }) + expect(layout.content.launchFailure).toBeUndefined() + expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-live-tab')?.status).toBe('running') + }) + it('recovers an OpenCode sessionRef from inventory before clearing a stale live handle', async () => { const store = createStore({ tabs: [{ diff --git a/test/unit/client/components/component-edge-cases.test.tsx b/test/unit/client/components/component-edge-cases.test.tsx index c409df748..54a132e8b 100644 --- a/test/unit/client/components/component-edge-cases.test.tsx +++ b/test/unit/client/components/component-edge-cases.test.tsx @@ -701,11 +701,12 @@ describe('Component Edge Cases', () => { expect(screen.getByRole('button', { name: 'Kill' })).toBeEnabled() fireEvent.click(screen.getByRole('button', { name: 'Attach' })) const [tab] = store.getState().tabs.tabs - expect(tab).toMatchObject({ soulId: 'retained-soul', mode: 'codex', status: 'exited' }) + const expectedStatus = runtimeStatus === 'blocked' ? 'error' : 'exited' + expect(tab).toMatchObject({ soulId: 'retained-soul', mode: 'codex', status: expectedStatus }) expect(store.getState().tabs.activeTabId).toBe(tab.id) expect(store.getState().panes.layouts[tab.id]).toMatchObject({ type: 'leaf', - content: { terminalId: 'retained-terminal', soulId: 'retained-soul', status: 'exited', runtimeStatus }, + content: { terminalId: 'retained-terminal', soulId: 'retained-soul', status: expectedStatus, runtimeStatus }, }) }, ) diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index f80638c1a..62ae42c74 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -41,11 +41,17 @@ process.stdout.write(JSON.stringify(await observeOpenCodeParitySelection(...proc freshell: { type: 'local', command: ['actual-mcp', 'argument'] } } })) else await fs.unlink(projected).catch(() => {}) const spec = pathToFileURL(managed ? packageRoot : entry).href - await fs.writeFile(selectedTui, JSON.stringify({ theme: 'selected', plugin: ['file:///selected/user-plugin.js', spec] })) + const selectedTuiConfig = `{ + // OpenCode accepts JSONC, including comments and trailing commas. + "theme": "selected", + "plugin": ["file:///selected/user-plugin.js", ${JSON.stringify(spec)},], +}` + await fs.writeFile(selectedTui, selectedTuiConfig) const observed = inspect() expect(JSON.parse(observed.projectedProjectConfig)).toEqual(userConfig) expect(observed.rebindPlugin).toMatchObject({ spec, hello: true, sessionId: 'ses_parity' }) - expect(JSON.parse(observed.tuiConfig)).toEqual({ theme: 'selected', plugin: ['file:///selected/user-plugin.js', spec] }) + expect(observed.tuiConfig).toBe(selectedTuiConfig) + expect(observed.tuiConfigParsed).toEqual({ theme: 'selected', plugin: ['file:///selected/user-plugin.js', spec] }) } const configuredBridge = { ...userConfig, mcp: { ...userConfig.mcp, freshell: { type: 'local', command: ['selected-user-bridge'] } } } @@ -63,6 +69,16 @@ it('classifies owned provider publication failures without retaining error bodie 'null', 'false', '[]', 'not-json', JSON.stringify({ event: 'provider.projection.transaction.started', stage: 'stage' }), JSON.stringify({ event: 'provider.projection.transaction.failed', stage: 'publish', cause: `provider transaction install .config/${secret}: Invalid cross-device link (os error 18)` }), + JSON.stringify({ event: 'provider.opencode.resource.directory_selected', field: 'OpenCode mcp.cwd', + resource_leaf: 'workspace', source_config_leaf: 'opencode.json', source_config_parent_leaf: '.opencode', + source_config_grandparent_leaf: 'project', resource_parent_leaf: 'parent', + resource_grandparent_leaf: 'grandparent' }), + JSON.stringify({ event: 'provider.environment.copy.failed', provider: 'opencode', operation: 'plan', + relative_path: '.freshell/resource-units/root-0/target/debug/deps', resource_source_scope: 'external', + resource_source_leaf: 'selected-resource', resource_source_origin: 'configured_resource', + resource_source_lane: 'main', resource_source_config_leaf: 'opencode.jsonc', + resource_source_config_scope: 'provider_config', + limit_kind: 'entries', observed: 100_001, permitted: 100_000 }), JSON.stringify({ event: 'session_host.fatal', error: `provider refresh control ${secret}` }), JSON.stringify({ event: 'session_host.fatal', error: secret }), ].join('\n') @@ -75,6 +91,19 @@ it('classifies owned provider publication failures without retaining error bodie { event: 'provider.projection.transaction.started', stage: 'stage' }, { event: 'provider.projection.transaction.failed', stage: 'publish' }, ]) + expect(diagnostic.resourceCopyFailures).toEqual([{ + provider: 'opencode', operation: 'plan', + relativePath: '.freshell/resource-units/root-0/target/debug/deps', + sourceScope: 'external', sourceLeaf: 'selected-resource', + sourceOrigin: 'configured_resource', sourceLane: 'main', sourceConfigLeaf: 'opencode.jsonc', + sourceConfigScope: 'provider_config', + limitKind: 'entries', observed: 100_001, permitted: 100_000, + }]) + expect(diagnostic.directoryResourceSelections).toEqual([ + { field: 'OpenCode mcp.cwd', resourceLeaf: 'workspace', sourceConfigLeaf: 'opencode.json', + sourceConfigParentLeaf: '.opencode', sourceConfigGrandparentLeaf: 'project', resourceParentLeaf: 'parent', + resourceGrandparentLeaf: 'grandparent' }, + ]) expect(JSON.stringify(diagnostic)).not.toContain(secret) }) From d627b20b4c6eb201864bbd1442b22f2c05050511 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:40:22 -0700 Subject: [PATCH 385/493] fix(recovery): retire stale provider config placeholders --- .../src/provider_mountpoints.rs | 344 ++++++++++++- .../src/managed_mcp_capability.rs | 448 ++++++++++++++++- .../managed_mcp_capability/source_stage.rs | 451 +++++++++++++++++- .../src/backend/provider_mountpoints.rs | 46 +- .../server/managed-provider-parity.test.ts | 232 ++++++++- 5 files changed, 1457 insertions(+), 64 deletions(-) diff --git a/crates/freshell-platform/src/provider_mountpoints.rs b/crates/freshell-platform/src/provider_mountpoints.rs index 023807653..1e65a6b8d 100644 --- a/crates/freshell-platform/src/provider_mountpoints.rs +++ b/crates/freshell-platform/src/provider_mountpoints.rs @@ -2,6 +2,10 @@ use serde::{Deserialize, Serialize}; use std::path::Path; +const RESOURCE_ROOTS: &str = ".freshell/provider-roots"; +const RESOURCE_PLACEHOLDER_MANIFEST: &str = ".freshell/provider-resource-mountpoints.json"; +const MAX_PLACEHOLDER_MANIFEST_BYTES: u64 = 64 * 1024 * 1024; + #[derive(Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct Mountpoint { @@ -9,6 +13,20 @@ pub struct Mountpoint { pub directory: bool, } +fn write_result( + writer: &mut impl std::io::Write, + result: Result<(), &'static str>, +) -> std::io::Result<()> { + match result { + Ok(()) => writer.write_all(b"ok"), + Err(reason) => { + writer.write_all(b"er")?; + writer.write_all(&(reason.len() as u32).to_be_bytes())?; + writer.write_all(reason.as_bytes()) + } + } +} + #[cfg(unix)] pub fn prepare(home: &Path, selected: &[Mountpoint]) -> Result<(), &'static str> { prepare_for_identity(home, selected, 65534, 0) @@ -21,10 +39,11 @@ fn prepare_for_identity( gid: u32, ) -> Result<(), &'static str> { use std::{ - collections::BTreeMap, + collections::{BTreeMap, BTreeSet}, os::unix::fs::{DirBuilderExt, MetadataExt, OpenOptionsExt}, }; let mut paths = BTreeMap::new(); + let mut current_resource_files = BTreeSet::new(); let mut bytes = 0usize; for entry in selected { let relative = Path::new(&entry.relative_path); @@ -52,6 +71,12 @@ fn prepare_for_identity( { return Err("provider mountpoint kinds conflict"); } + if !entry.directory + && relative.starts_with(RESOURCE_ROOTS) + && relative.components().count() > Path::new(RESOURCE_ROOTS).components().count() + { + current_resource_files.insert(relative.to_owned()); + } if paths.len() > 100_000 { return Err("provider mountpoint entry profile exceeded"); } @@ -62,6 +87,9 @@ fn prepare_for_identity( return Err("provider mountpoint metadata profile exceeded"); } } + if paths.insert(Path::new(".freshell").to_owned(), true) == Some(false) { + return Err("provider mountpoint kinds conflict"); + } let metadata = std::fs::symlink_metadata(home).map_err(|_| "provider HOME mountpoint unavailable")?; if !metadata.is_dir() @@ -85,6 +113,35 @@ fn prepare_for_identity( } else if metadata.uid() != uid || metadata.gid() != gid { return Err("provider HOME ownership changed"); } + let previous_resource_files = read_resource_placeholder_manifest(home, uid, gid)?; + let mut retired = Vec::new(); + for relative in previous_resource_files.difference(¤t_resource_files) { + verify_owned_physical_parents(home, relative, uid, gid)?; + match std::fs::symlink_metadata(home.join(relative)) { + Ok(metadata) + if !metadata.file_type().is_symlink() + && metadata.is_file() + && metadata.uid() == uid + && metadata.gid() == gid + && metadata.len() == 0 => retired.push(relative.clone()), + Ok(metadata) + if !metadata.file_type().is_symlink() + && metadata.is_file() + && metadata.uid() == uid + && metadata.gid() == gid => { + return Err( + "retired provider resource mountpoint is not empty; retain inputs and reprepare", + ) + } + Ok(_) => { + return Err( + "retired provider resource mountpoint kind or ownership changed; retain inputs and reprepare", + ) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("retired provider resource mountpoint inspection failed"), + } + } // Inspection precedes every mutation, so a conflicting retained leaf never // causes a partially populated preparation or ownership adoption. for (relative, directory) in &paths { @@ -104,9 +161,10 @@ fn prepare_for_identity( } } let mut created = Vec::new(); + let mut created_resource_files = BTreeSet::new(); let result = (|| { for (relative, directory) in paths { - let path = home.join(relative); + let path = home.join(&relative); if std::fs::symlink_metadata(&path).is_ok() { continue; } @@ -122,9 +180,41 @@ fn prepare_for_identity( .mode(0o600) .open(&path) .map_err(|_| "provider file mountpoint creation failed")?; + if !directory + && Path::new(&relative).starts_with(RESOURCE_ROOTS) + && Path::new(&relative).components().count() + > Path::new(RESOURCE_ROOTS).components().count() + { + created_resource_files.insert(relative.clone()); + } } created.push(path); } + for relative in &retired { + let path = home.join(relative); + let metadata = std::fs::symlink_metadata(&path) + .map_err(|_| "retired provider resource mountpoint changed during preparation")?; + if metadata.file_type().is_symlink() + || !metadata.is_file() + || metadata.uid() != uid + || metadata.gid() != gid + || metadata.len() != 0 + { + return Err("retired provider resource mountpoint changed during preparation"); + } + std::fs::remove_file(&path) + .map_err(|_| "retired provider resource mountpoint removal failed")?; + } + let retained_resource_files = previous_resource_files + .intersection(¤t_resource_files) + .cloned() + .chain(created_resource_files.iter().cloned()) + .collect::<BTreeSet<_>>(); + // The fixed helper intentionally has CHOWN but not DAC_OVERRIDE. Write + // the manifest while it can still traverse a root-owned initial HOME; + // transferring HOME ownership first makes mode-0700 directories + // inaccessible to the helper's still-root identity. + write_resource_placeholder_manifest(home, &retained_resource_files, uid, gid, initial)?; for path in created .iter() .rev() @@ -151,10 +241,172 @@ fn prepare_for_identity( result } +#[cfg(unix)] +fn read_resource_placeholder_manifest( + home: &Path, + uid: u32, + gid: u32, +) -> Result<std::collections::BTreeSet<std::path::PathBuf>, &'static str> { + use std::os::unix::fs::MetadataExt; + let path = home.join(RESOURCE_PLACEHOLDER_MANIFEST); + let metadata = match std::fs::symlink_metadata(&path) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Default::default()), + Err(_) => return Err("provider resource mountpoint metadata unavailable"), + }; + if metadata.file_type().is_symlink() + || !metadata.is_file() + || metadata.uid() != uid + || metadata.gid() != gid + || metadata.len() > MAX_PLACEHOLDER_MANIFEST_BYTES + { + return Err("provider resource mountpoint metadata changed or exceeds its profile"); + } + let bytes = + std::fs::read(&path).map_err(|_| "provider resource mountpoint metadata unavailable")?; + let entries: Vec<String> = serde_json::from_slice(&bytes) + .map_err(|_| "provider resource mountpoint metadata invalid")?; + let mut result = std::collections::BTreeSet::new(); + for entry in entries { + let relative = std::path::PathBuf::from(&entry); + if relative.as_os_str().is_empty() + || !relative.starts_with(RESOURCE_ROOTS) + || relative.components().count() + <= std::path::Path::new(RESOURCE_ROOTS).components().count() + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + || !result.insert(relative) + { + return Err("provider resource mountpoint metadata invalid"); + } + } + Ok(result) +} + +#[cfg(unix)] +fn verify_owned_physical_parents( + home: &Path, + relative: &Path, + uid: u32, + gid: u32, +) -> Result<(), &'static str> { + use std::os::unix::fs::MetadataExt; + let mut parent = home.to_owned(); + for component in relative.parent().into_iter().flat_map(Path::components) { + parent.push(component.as_os_str()); + match std::fs::symlink_metadata(&parent) { + Ok(metadata) + if !metadata.file_type().is_symlink() + && metadata.is_dir() + && metadata.uid() == uid + && metadata.gid() == gid => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), + _ => return Err("retired provider resource mountpoint parent changed"), + } + } + Ok(()) +} + +#[cfg(unix)] +fn write_resource_placeholder_manifest( + home: &Path, + entries: &std::collections::BTreeSet<std::path::PathBuf>, + uid: u32, + gid: u32, + allow_initial_root_parent: bool, +) -> Result<(), &'static str> { + use std::{ + io::Write, + os::unix::{ + ffi::OsStrExt, + fs::{MetadataExt, OpenOptionsExt, PermissionsExt}, + }, + sync::atomic::{AtomicU64, Ordering}, + }; + static NEXT_TEMP: AtomicU64 = AtomicU64::new(0); + let serialized = entries + .iter() + .map(|entry| { + entry + .to_str() + .ok_or("provider resource mountpoint metadata invalid") + }) + .collect::<Result<Vec<_>, _>>()?; + let bytes = serde_json::to_vec(&serialized) + .map_err(|_| "provider resource mountpoint metadata serialization failed")?; + if bytes.len() as u64 > MAX_PLACEHOLDER_MANIFEST_BYTES { + return Err("provider resource mountpoint metadata profile exceeded"); + } + let parent = home.join(".freshell"); + let parent_metadata = + std::fs::symlink_metadata(&parent).map_err(|error| match error.kind() { + std::io::ErrorKind::NotFound => { + "provider resource mountpoint metadata parent missing after preparation" + } + std::io::ErrorKind::PermissionDenied => { + "provider resource mountpoint metadata parent inaccessible" + } + _ => "provider resource mountpoint metadata parent inspection failed", + })?; + let expected_owner = parent_metadata.uid() == uid && parent_metadata.gid() == gid; + let initial_root_owner = + allow_initial_root_parent && parent_metadata.uid() == 0 && parent_metadata.gid() == 0; + if parent_metadata.file_type().is_symlink() + || !parent_metadata.is_dir() + || !(expected_owner || initial_root_owner) + { + return Err("provider resource mountpoint metadata parent changed"); + } + let manifest = home.join(RESOURCE_PLACEHOLDER_MANIFEST); + for _ in 0..8 { + let sequence = NEXT_TEMP.fetch_add(1, Ordering::Relaxed); + let candidate = parent.join(format!( + ".provider-resource-mountpoints-{}-{sequence}.tmp", + std::process::id() + )); + match std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&candidate) + { + Ok(mut file) => { + let result = (|| { + file.write_all(&bytes) + .map_err(|_| "provider resource mountpoint metadata write failed")?; + file.sync_all() + .map_err(|_| "provider resource mountpoint metadata sync failed")?; + std::fs::set_permissions(&candidate, std::fs::Permissions::from_mode(0o600)) + .map_err(|_| "provider resource mountpoint metadata permissions failed")?; + let name = std::ffi::CString::new(candidate.as_os_str().as_bytes()) + .map_err(|_| "provider resource mountpoint metadata path invalid")?; + if unsafe { libc::lchown(name.as_ptr(), uid, gid) } != 0 { + return Err("provider resource mountpoint metadata ownership failed"); + } + std::fs::rename(&candidate, &manifest) + .map_err(|_| "provider resource mountpoint metadata publish failed")?; + std::fs::File::open(&parent) + .and_then(|directory| directory.sync_all()) + .map_err(|_| "provider resource mountpoint metadata parent sync failed")?; + Ok(()) + })(); + if result.is_err() { + let _ = std::fs::remove_file(&candidate); + } + return result; + } + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue, + Err(_) => return Err("provider resource mountpoint metadata temporary unavailable"), + } + } + Err("provider resource mountpoint metadata temporary unavailable") +} + #[cfg(unix)] pub fn serve(home: &Path, socket: &Path) -> Result<(), &'static str> { use std::{ - io::{Read, Write}, + io::Read, os::unix::{fs::MetadataExt, net::UnixStream}, }; let root = @@ -192,10 +444,10 @@ pub fn serve(home: &Path, socket: &Path) -> Result<(), &'static str> { .map_err(|_| "provider mountpoint request incomplete")?; let selected: Vec<Mountpoint> = serde_json::from_slice(&bytes).map_err(|_| "provider mountpoint metadata invalid")?; - prepare(home, &selected)?; - stream - .write_all(b"ok") - .map_err(|_| "provider mountpoint acknowledgement failed") + let prepared = prepare(home, &selected); + write_result(&mut stream, prepared) + .map_err(|_| "provider mountpoint acknowledgement failed")?; + prepared } #[cfg(all(test, unix))] @@ -238,6 +490,84 @@ mod tests { ); } + #[test] + fn provider_mountpoints_remove_only_retired_empty_resource_placeholders() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir(&home).unwrap(); + let root = ".freshell/provider-roots/project"; + let file = format!("{root}/opencode.json"); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + prepare_for_identity( + &home, + &[ + Mountpoint { + relative_path: root.into(), + directory: true, + }, + Mountpoint { + relative_path: file.clone(), + directory: false, + }, + ], + uid, + gid, + ) + .unwrap(); + let placeholder = home.join(&file); + let unrelated_empty = home.join(root).join("user-empty.json"); + std::fs::write(&unrelated_empty, b"").unwrap(); + assert_eq!(std::fs::metadata(&placeholder).unwrap().len(), 0); + + prepare_for_identity(&home, &[], uid, gid).unwrap(); + assert!(!placeholder.exists()); + assert!(unrelated_empty.exists()); + assert!(home.join(root).is_dir()); + + prepare_for_identity( + &home, + &[ + Mountpoint { + relative_path: root.into(), + directory: true, + }, + Mountpoint { + relative_path: file.clone(), + directory: false, + }, + ], + uid, + gid, + ) + .unwrap(); + std::fs::write(&placeholder, b"provider-owned").unwrap(); + let error = prepare_for_identity( + &home, + &[Mountpoint { + relative_path: root.into(), + directory: true, + }], + uid, + gid, + ) + .expect_err("retirement must not delete nonempty provider data"); + assert!(error.contains("not empty")); + assert_eq!(std::fs::read(&placeholder).unwrap(), b"provider-owned"); + } + + #[test] + fn provider_mountpoint_failure_acknowledgement_carries_its_static_reason() { + let mut encoded = Vec::new(); + write_result(&mut encoded, Err("preparation refused safely")).unwrap(); + assert_eq!(&encoded[..2], b"er"); + let length = u32::from_be_bytes(encoded[2..6].try_into().unwrap()) as usize; + assert_eq!(length, encoded.len() - 6); + assert_eq!( + std::str::from_utf8(&encoded[6..]).unwrap(), + "preparation refused safely" + ); + } + #[test] fn provider_mountpoints_refuse_symlink_parents_before_creating_other_paths() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index be74bc253..e3a765355 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -63,6 +63,7 @@ pub(crate) struct OpencodeEphemeralInput<'a> { pub launch_context: Option<&'a freshell_runtime_protocol::ProviderLaunchContext>, pub frozen_inputs: Option<&'a [String]>, pub native_source_flags: Option<&'a [freshell_platform::opencode_recipe::NativeSourceFlag]>, + pub prior_resource_mounts: Option<&'a [freshell_runtime_protocol::ProviderResourceMount]>, pub rebind_enabled: Option<bool>, pub inline_edits: Option<&'a freshell_platform::opencode_config::OwnedConfigEdits>, } @@ -325,6 +326,89 @@ struct CapabilityCallback { provider_path_inputs: Option<freshell_runtime_protocol::ProviderPathInputs>, } +struct ReissuedOpencodeSources { + sources: Vec<crate::managed_provider_references::sources::SourceOccurrence>, + native_source_flags: Vec<freshell_platform::opencode_recipe::NativeSourceFlag>, +} + +fn selected_opencode_sources_for_reissue( + manifest: &freshell_platform::opencode_recipe::SourceManifest, +) -> Result<ReissuedOpencodeSources, String> { + use freshell_platform::opencode_recipe::SourceLane as ManifestLane; + + let mut sources = Vec::new(); + let mut new_ordinal_by_old = HashMap::new(); + for (ordinal, source) in manifest.sources.iter().enumerate() { + if source.lane == ManifestLane::Inline { + continue; + } + let explicitly_selected = manifest.native_source_flags.iter().any(|flag| { + flag.source_ordinal + .is_some_and(|value| usize::from(value) == ordinal) + }); + let canonical = match std::fs::canonicalize(&source.logical_path) { + Ok(canonical) => canonical, + Err(error) if error.kind() == std::io::ErrorKind::NotFound && !explicitly_selected => { + // Native discovery can legitimately change while the provider is down. + // A missing discovered file is no longer part of the replacement input. + continue; + } + Err(_) => { + return Err(format!( + "prior selected configuration source {ordinal} is unavailable; repair it before reissue" + )); + } + }; + new_ordinal_by_old.insert(ordinal, sources.len()); + sources.push( + crate::managed_provider_references::sources::SourceOccurrence { + logical_path: source.logical_path.clone(), + canonical_source: canonical, + role: source.discovery_role, + lane: match source.lane { + ManifestLane::Main => { + crate::managed_provider_references::sources::SourceLane::Main + } + ManifestLane::Tui => { + crate::managed_provider_references::sources::SourceLane::Tui + } + ManifestLane::Inline => { + crate::managed_provider_references::sources::SourceLane::Inline + } + }, + }, + ); + } + + let inline_will_be_reinserted = manifest + .sources + .iter() + .any(|source| source.lane == ManifestLane::Inline); + let inline_insert_index = sources + .iter() + .position(|source| { + source.lane == crate::managed_provider_references::sources::SourceLane::Tui + }) + .unwrap_or(sources.len()); + let mut native_source_flags = manifest.native_source_flags.clone(); + for flag in &mut native_source_flags { + let Some(old_ordinal) = flag.source_ordinal else { + continue; + }; + let Some(new_ordinal) = new_ordinal_by_old.get(&usize::from(old_ordinal)).copied() else { + return Err("prior native source selector references an unavailable source".into()); + }; + let new_ordinal = new_ordinal + + usize::from(inline_will_be_reinserted && new_ordinal >= inline_insert_index); + flag.source_ordinal = + Some(u16::try_from(new_ordinal).map_err(|_| "native source ordinal overflow")?); + } + Ok(ReissuedOpencodeSources { + sources, + native_source_flags, + }) +} + async fn apply_prepared_callback( bytes: &[u8], expected: &str, @@ -700,18 +784,13 @@ async fn apply_prepared_callback( .map(str::to_owned) .or_else(|| std::env::var(name).ok()) }; - let selected_sources=accepted_manifest.as_ref().map(|manifest|{ - manifest.sources.iter().enumerate().filter(|(_, source)| source.lane != freshell_platform::opencode_recipe::SourceLane::Inline).map(|(ordinal,source)|{ - let canonical=std::fs::canonicalize(&source.logical_path) - .map_err(|_|format!("prior selected configuration source {ordinal} is unavailable; repair it before reissue"))?; - Ok(crate::managed_provider_references::sources::SourceOccurrence { - logical_path:source.logical_path.clone(),canonical_source:canonical,role:source.discovery_role, - lane:match source.lane {freshell_platform::opencode_recipe::SourceLane::Main=>crate::managed_provider_references::sources::SourceLane::Main, - freshell_platform::opencode_recipe::SourceLane::Tui=>crate::managed_provider_references::sources::SourceLane::Tui, - freshell_platform::opencode_recipe::SourceLane::Inline=>crate::managed_provider_references::sources::SourceLane::Inline}, - }) - }).collect::<Result<Vec<_>,String>>() - }).transpose()?; + let (selected_sources, reissued_native_source_flags) = + if let Some(manifest) = &accepted_manifest { + let selected = selected_opencode_sources_for_reissue(manifest)?; + (Some(selected.sources), Some(selected.native_source_flags)) + } else { + (None, None) + }; let inherited_inline = accepted_manifest .as_ref() .map(|manifest| { @@ -742,6 +821,11 @@ async fn apply_prepared_callback( }) .transpose()? .flatten(); + let prior_resource_mounts = callback + .prior_context + .as_ref() + .and_then(|context| context.prepared_inputs.as_ref()) + .map(|inputs| inputs.resource_mounts.as_slice()); let prepared = crate::managed_provider_references::prepare_managed_opencode_with_publisher( crate::managed_provider_references::OpencodePreparation { client, @@ -763,9 +847,7 @@ async fn apply_prepared_callback( selected_resource_roots: accepted_manifest .as_ref() .map(|manifest| manifest.resource_roots.clone()), - native_source_flags: accepted_manifest - .as_ref() - .map(|manifest| manifest.native_source_flags.clone()), + native_source_flags: reissued_native_source_flags, rebind_enabled: accepted_manifest .as_ref() .map(|manifest| manifest.rebind_enabled), @@ -777,6 +859,7 @@ async fn apply_prepared_callback( |soul, home, context, input| { let input = OpencodeEphemeralInput { launch_context: Some(context), + prior_resource_mounts, ..input }; let capability = store.issue_replacement_with_inputs( @@ -1554,6 +1637,7 @@ fn stage_provider_root( .iter() .map(|reference| PathBuf::from(&reference.source_path)), ); + let mut retained_resource_sources = None; if let Some(plan) = opencode_input.source_plan { // Indexed payloads remain exclusively in the private input volume, // even when an ordinary package scope also contains their files. @@ -1564,7 +1648,7 @@ fn stage_provider_root( .iter() .map(|selection| selection.canonical_source.clone()) })); - let units = source_stage::resource_units( + let mut units = source_stage::resource_units( home, &source, opencode_input @@ -1574,10 +1658,48 @@ fn stage_provider_root( plan, opencode_input.resource_roots, )?; - selections.extend(units.iter().map(|(source, mount)| CopySelection { - source: source.clone(), - destination_relative: mount.stage_relative_path.clone().into(), - })); + retained_resource_sources = if provider == "opencode" && prior_stage.is_some() { + let retained = source_stage::merge_recovery_resource_mounts( + &mut units, + opencode_input.prior_resource_mounts.unwrap_or_default(), + )?; + if retained.is_empty() { + None + } else { + let mut directory: Option<PathBuf> = None; + let mut retained_sources = HashMap::<PathBuf, PathBuf>::new(); + for (index, mount) in retained.into_iter().enumerate() { + let stage_path = PathBuf::from(&mount.stage_relative_path); + let source = if let Some(source) = + source_stage::current_source_for_resource_mount(&units, &mount) + { + source + } else if let Some(source) = retained_sources.get(&stage_path) { + source.clone() + } else { + let source_root = match &directory { + Some(directory) => directory.clone(), + None => { + let directory = + temporary.join(".freshell/retained-resource-sources"); + private_directory(&directory)?; + directory + } + }; + let source = source_root.join(index.to_string()); + private_directory(&source)?; + directory = Some(source_root); + retained_sources.insert(stage_path, source.clone()); + source + }; + units.push((source, mount)); + } + directory + } + } else { + None + }; + selections.extend(source_stage::resource_copy_selections(&units)?); let descriptors = units .into_iter() .map(|(_, mount)| mount) @@ -1807,10 +1929,145 @@ fn stage_provider_root( provider, selections = selections.len() ); - let report=copy_environment(&selections,&temporary,&excluded,CopyLimits::default()).map_err(|error| { - tracing::error!(event="provider.environment.copy.failed",stage="controller",provider,operation=error.operation,relative_path=%error.relative_path.display(),limit_kind=error.limit_kind,observed=error.observed,permitted=error.permitted,elapsed_ms=started.elapsed().as_millis() as u64,corrective_message=%error.corrective_message); - error.to_string() - })?; + let report = copy_environment(&selections, &temporary, &excluded, CopyLimits::default()) + .map_err(|error| { + let failing_selection = selections.iter().find(|selection| { + error + .relative_path + .starts_with(&selection.destination_relative) + }); + let resource_source_scope = failing_selection.map(|selection| { + let selected = &selection.source; + let workspace = opencode_input.workspace.or(opencode_input.cwd); + if selected.starts_with(home) { + "provider_home" + } else if workspace.is_some_and(|workspace| selected.starts_with(workspace)) { + "workspace" + } else if selected.starts_with(&source) { + "provider_config" + } else { + "external" + } + }); + let resource_source_leaf = failing_selection + .and_then(|selection| selection.source.file_name()) + .map(|leaf| leaf.to_string_lossy().into_owned()); + let selected_source = + failing_selection.map(|selection| selection.source.as_path()); + let same_selected_source = |candidate: &Path| { + selected_source.is_some_and(|selected| { + candidate == selected + || std::fs::canonicalize(candidate).ok().as_deref() == Some(selected) + }) + }; + let configured_source_origin = opencode_input.source_plan.and_then(|plan| { + plan.iter().find_map(|planned| { + planned + .configured_resources + .iter() + .any(|resource| same_selected_source(resource)) + .then(|| { + let lane = match planned.source.lane { + crate::managed_provider_references::sources::SourceLane::Main => { + "main" + } + crate::managed_provider_references::sources::SourceLane::Tui => { + "tui" + } + crate::managed_provider_references::sources::SourceLane::Inline => { + "inline" + } + }; + let config_leaf = planned + .source + .logical_path + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()); + let config_path = &planned.source.logical_path; + let workspace = opencode_input.workspace.or(opencode_input.cwd); + let scope = if config_path.starts_with(home) { + "provider_home" + } else if workspace + .is_some_and(|workspace| config_path.starts_with(workspace)) + { + "workspace" + } else if opencode_input + .config_root + .is_some_and(|root| config_path.starts_with(root)) + { + "provider_config" + } else { + "external" + }; + ("configured_resource", Some(lane), config_leaf, scope) + }) + }) + }); + let discovered_root_origin = opencode_input.resource_roots.and_then(|roots| { + roots + .iter() + .find(|root| same_selected_source(&root.logical_path)) + .map(|root| { + let role = match root.role { + crate::managed_provider_references::sources::DiscoveryRole::Global => { + "global_root" + } + crate::managed_provider_references::sources::DiscoveryRole::Home => { + "provider_home_root" + } + crate::managed_provider_references::sources::DiscoveryRole::Workspace => { + "workspace_root" + } + crate::managed_provider_references::sources::DiscoveryRole::Unknown => { + "unknown_root" + } + }; + let root_leaf = root + .logical_path + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()); + (role, root_leaf) + }) + }); + let ( + resource_source_origin, + resource_source_lane, + resource_source_config_leaf, + resource_source_config_scope, + ) = configured_source_origin + .map(|(origin, lane, config_leaf, config_scope)| { + (origin, lane, config_leaf, Some(config_scope)) + }) + .unwrap_or_else(|| { + discovered_root_origin + .map(|(origin, root_leaf)| (origin, None, root_leaf, None)) + .unwrap_or(("unclassified", None, None, None)) + }); + tracing::error!( + event = "provider.environment.copy.failed", + stage = "controller", + provider, + operation = error.operation, + relative_path = %error.relative_path.display(), + resource_source_scope, + resource_source_leaf, + resource_source_origin, + resource_source_lane, + resource_source_config_leaf, + resource_source_config_scope, + limit_kind = error.limit_kind, + observed = error.observed, + permitted = error.permitted, + elapsed_ms = started.elapsed().as_millis() as u64, + corrective_message = %error.corrective_message + ); + error.to_string() + })?; + if let Some(directory) = retained_resource_sources { + std::fs::remove_dir_all(directory).map_err(|error| { + format!("retained provider resource staging cleanup failed: {error}") + })?; + } tracing::info!( event = "provider.environment.copy.completed", stage = "controller", @@ -3313,6 +3570,149 @@ fn execute_copied_script(path: &Path) -> std::io::Result<std::process::Output> { mod tests { use super::*; + fn opencode_source_view( + logical_path: PathBuf, + lane: freshell_platform::opencode_recipe::SourceLane, + ordinal: usize, + ) -> freshell_platform::opencode_recipe::SourceView { + use freshell_platform::opencode_recipe::{DiscoveryRole, SourceView}; + + SourceView { + logical_path, + discovery_role: DiscoveryRole::Workspace, + native_destination: None, + lane, + unexpanded_relative_path: format!("sources/{ordinal}.json"), + template_relative_path: format!("sources/{ordinal}.json"), + template_sha256: None, + generated_name: None, + occurrences: vec![], + referenced_variables: vec![], + path_variables: vec![], + original_form: false, + } + } + + fn opencode_manifest( + sources: Vec<(PathBuf, freshell_platform::opencode_recipe::SourceLane)>, + native_source_flags: Vec<freshell_platform::opencode_recipe::NativeSourceFlag>, + ) -> freshell_platform::opencode_recipe::SourceManifest { + freshell_platform::opencode_recipe::SourceManifest { + version: 1, + cwd: PathBuf::from("/workspace"), + original_home: PathBuf::from("/home/original"), + global_config_root: None, + sources: sources + .into_iter() + .enumerate() + .map(|(ordinal, (path, lane))| opencode_source_view(path, lane, ordinal)) + .collect(), + files: vec![], + frozen_inputs: vec![], + user_owns_freshell_mcp: false, + native_source_flags, + resource_roots: vec![], + rebind_enabled: true, + inline_edits: Default::default(), + } + } + + #[test] + fn opencode_reissue_omits_a_removed_discovered_source() { + let owned = tempfile::tempdir().unwrap(); + let removed_source = owned.path().join("opencode.json"); + let manifest = opencode_manifest( + vec![( + removed_source.clone(), + freshell_platform::opencode_recipe::SourceLane::Main, + )], + vec![], + ); + + let selected = selected_opencode_sources_for_reissue(&manifest).unwrap(); + + assert!(selected.sources.is_empty()); + } + + #[test] + fn opencode_reissue_still_requires_a_removed_explicit_source() { + let owned = tempfile::tempdir().unwrap(); + let removed_source = owned.path().join("selected.json"); + let manifest = opencode_manifest( + vec![( + removed_source, + freshell_platform::opencode_recipe::SourceLane::Main, + )], + vec![freshell_platform::opencode_recipe::NativeSourceFlag { + kind: freshell_platform::opencode_recipe::NativeSourceFlagKind::Main, + runtime_destination: "/provider/selected.json".into(), + source_ordinal: Some(0), + }], + ); + + let error = selected_opencode_sources_for_reissue(&manifest) + .err() + .expect("a removed explicit source must block reissue"); + + assert_eq!( + error, + "prior selected configuration source 0 is unavailable; repair it before reissue" + ); + } + + #[test] + fn opencode_reissue_does_not_ignore_non_missing_source_errors() { + let owned = tempfile::tempdir().unwrap(); + let parent_file = owned.path().join("not-a-directory"); + std::fs::write(&parent_file, "file").unwrap(); + let manifest = opencode_manifest( + vec![( + parent_file.join("selected.json"), + freshell_platform::opencode_recipe::SourceLane::Main, + )], + vec![], + ); + + let error = selected_opencode_sources_for_reissue(&manifest) + .err() + .expect("non-missing filesystem errors must block reissue"); + + assert_eq!( + error, + "prior selected configuration source 0 is unavailable; repair it before reissue" + ); + } + + #[test] + fn opencode_reissue_remaps_native_selector_after_removing_prior_sources() { + use freshell_platform::opencode_recipe::{ + NativeSourceFlag, NativeSourceFlagKind, SourceLane, + }; + + let owned = tempfile::tempdir().unwrap(); + let removed_source = owned.path().join("removed.json"); + let tui_source = owned.path().join("tui.jsonc"); + std::fs::write(&tui_source, r#"{"theme":"dark"}"#).unwrap(); + let manifest = opencode_manifest( + vec![ + (removed_source, SourceLane::Main), + (PathBuf::from("OPENCODE_CONFIG_CONTENT"), SourceLane::Inline), + (tui_source.clone(), SourceLane::Tui), + ], + vec![NativeSourceFlag { + kind: NativeSourceFlagKind::Tui, + runtime_destination: "/provider/tui.jsonc".into(), + source_ordinal: Some(2), + }], + ); + + let selected = selected_opencode_sources_for_reissue(&manifest).unwrap(); + + assert_eq!(selected.sources.len(), 1); + assert_eq!(selected.sources[0].logical_path, tui_source); + assert_eq!(selected.native_source_flags[0].source_ordinal, Some(1)); + } + #[cfg(target_os = "linux")] #[test] fn managed_mcp_preparation_journal_partial_write_preserves_prior_and_startup_ownership() { diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index b1197689a..d664fb362 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1079,6 +1079,101 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"home node_modules loaded\n"); } + #[test] + fn direct_node_script_in_home_keeps_static_import_meta_url_data_after_all_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.mjs"); + let schema = home.join("schema.json"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "import { readFileSync } from 'node:fs';\nprocess.stdout.write(readFileSync(new URL('./schema.json', import.meta.url), 'utf8'));\n", + ) + .unwrap(); + std::fs::write(&schema, "static schema loaded\n").unwrap(); + + let (plan, provider_script) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&schema.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .arg(&provider_script) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"static schema loaded\n"); + } + + #[test] + fn shell_mcp_wrapper_stages_and_runs_node_server_module_closure() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let command = tools.join("start"); + let server = tools.join("server.js"); + let helper = tools.join("helper.cjs"); + let package = tools.join("node_modules/shell-fixture"); + std::fs::create_dir_all(&package).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\nexec node \"$SCRIPT_DIR/server.js\"\n", + ) + .unwrap(); + std::fs::write( + &server, + "const helper = require('./helper.cjs');\nconst installed = require('shell-fixture');\nprocess.stdout.write(`${helper}:${installed}\\n`);\n", + ) + .unwrap(); + std::fs::write(&helper, "module.exports = 'sibling-loaded';\n").unwrap(); + std::fs::write( + package.join("package.json"), + r#"{"name":"shell-fixture","main":"index.cjs"}"#, + ) + .unwrap(); + std::fs::write( + package.join("index.cjs"), + "module.exports = 'package-loaded';\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + for required in [&command, &server, &helper, &tools.join("node_modules")] { + assert!( + plan[0] + .configured_resources + .contains(&required.canonicalize().unwrap()), + "shell-launched Node closure omitted {}", + required.display() + ); + } + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"sibling-loaded:package-loaded\n"); + } + #[test] fn node_command_script_in_home_keeps_home_node_modules_after_all_copies() { let owned = tempfile::tempdir().unwrap(); @@ -2360,6 +2455,184 @@ mod configured_resource_tests { ); assert_eq!(std::fs::read_to_string(config).unwrap(), raw); } + + #[test] + fn resource_mount_identity_does_not_shift_when_another_root_is_added() { + use freshell_platform::opencode_recipe::{DiscoveryRole, DiscoveryRoot}; + + let owned = tempfile::tempdir().unwrap(); + let extra = owned.path().join("a-extra-resource"); + let home = owned.path().join("z-provider-home"); + let global = home.join(".config/opencode"); + let workspace = owned.path().join("workspace"); + for path in [&extra, &global, &workspace] { + std::fs::create_dir_all(path).unwrap(); + } + let global_root = DiscoveryRoot { + logical_path: global.clone(), + role: DiscoveryRole::Global, + }; + let extra_root = DiscoveryRoot { + logical_path: extra, + role: DiscoveryRole::Home, + }; + + let before = resource_units( + &home, + &global, + &workspace, + &[], + Some(std::slice::from_ref(&global_root)), + ) + .unwrap(); + let after = resource_units( + &home, + &global, + &workspace, + &[], + Some(&[extra_root, global_root]), + ) + .unwrap(); + let before_global = before + .iter() + .find(|(_, mount)| mount.logical_destination.ends_with("/.config/opencode")) + .unwrap(); + let after_global = after + .iter() + .find(|(_, mount)| mount.logical_destination.ends_with("/.config/opencode")) + .unwrap(); + assert_eq!( + before_global.1.provider_relative_path, after_global.1.provider_relative_path, + "adding a different resource root must not reassign the provider-owned subpath" + ); + } + + #[test] + fn recovery_reuses_matching_home_mount_and_retains_dropped_home_directories() { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + + let previous = ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-legacy".into(), + provider_relative_path: ".freshell/provider-roots/root-legacy".into(), + logical_destination: "/home/freshell/provider/.config/opencode".into(), + effective_destination: "/home/freshell/provider/.config/opencode".into(), + kind: ProviderResourceKind::Directory, + }; + let mut current = vec![( + "/synthetic/current-opencode-root".into(), + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-new".into(), + provider_relative_path: ".freshell/provider-roots/root-new".into(), + logical_destination: previous.logical_destination.clone(), + effective_destination: previous.effective_destination.clone(), + kind: ProviderResourceKind::Directory, + }, + )]; + let retained = + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)).unwrap(); + assert_eq!( + current[0].1.stage_relative_path, + previous.stage_relative_path + ); + assert_eq!( + current[0].1.provider_relative_path, + previous.provider_relative_path + ); + assert!( + retained.is_empty(), + "the current alias already retains this root" + ); + + let mut no_current_roots = Vec::new(); + let retained = + merge_recovery_resource_mounts(&mut no_current_roots, std::slice::from_ref(&previous)) + .unwrap(); + assert_eq!(retained, vec![previous]); + } + + #[test] + fn recovery_rejects_nested_retained_and_current_directory_mounts() { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + + let mount = |name: &str, destination: &str| ProviderResourceMount { + stage_relative_path: format!(".freshell/resource-units/{name}"), + provider_relative_path: format!(".freshell/provider-roots/{name}"), + logical_destination: destination.into(), + effective_destination: destination.into(), + kind: ProviderResourceKind::Directory, + }; + + for (current_destination, previous_destination) in [ + ( + "/home/freshell/provider/.config/opencode/plugins", + "/home/freshell/provider/.config/opencode", + ), + ( + "/home/freshell/provider/.config/opencode", + "/home/freshell/provider/.config/opencode/plugins", + ), + ] { + let previous = mount("previous", previous_destination); + let mut current = vec![( + "/synthetic/current-resource".into(), + mount("current", current_destination), + )]; + let error = + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)) + .expect_err("nested directory mount roots must preserve the prior generation"); + assert!(error.contains("overlapping") && error.contains("reprepare")); + } + + let previous = mount("previous", "/home/freshell/provider/.config/opencode"); + let mut prefix_sibling = vec![( + "/synthetic/current-resource".into(), + mount("current", "/home/freshell/provider/.config/opencode-extra"), + )]; + let retained = + merge_recovery_resource_mounts(&mut prefix_sibling, std::slice::from_ref(&previous)) + .unwrap(); + assert_eq!(retained, vec![previous]); + } + + #[test] + fn recovery_resource_copy_selections_deduplicate_shared_alias_subpaths() { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + + let owned = tempfile::tempdir().unwrap(); + let source = owned.path().join("shared"); + std::fs::create_dir(&source).unwrap(); + std::fs::write(source.join("resource.txt"), "shared-resource").unwrap(); + let mount = |destination: &str| ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-shared".into(), + provider_relative_path: ".freshell/provider-roots/root-shared".into(), + logical_destination: destination.into(), + effective_destination: destination.into(), + kind: ProviderResourceKind::Directory, + }; + let mut aliases = vec![(source, mount("/workspace/current"))]; + let prior = mount("/home/freshell/provider/.config/opencode"); + let retained = + merge_recovery_resource_mounts(&mut aliases, std::slice::from_ref(&prior)).unwrap(); + assert_eq!(retained, vec![prior]); + let source = current_source_for_resource_mount(&aliases, &retained[0]) + .expect("the selected alias already supplies this physical subpath"); + aliases.push((source, retained[0].clone())); + + let selections = resource_copy_selections(&aliases).unwrap(); + assert_eq!(aliases.len(), 2, "both aliases must remain mountable"); + assert_eq!(selections.len(), 1, "the shared staged tree is copied once"); + let stage = owned.path().join("stage"); + copy_environment(&selections, &stage, &[], Default::default()).unwrap(); + assert_eq!( + std::fs::read_to_string( + stage + .join(".freshell/resource-units/root-shared") + .join("resource.txt") + ) + .unwrap(), + "shared-resource" + ); + } } /// Selected discovery roots are complete writable units, independent of the @@ -2378,6 +2651,7 @@ pub(super) fn resource_units( String, > { use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + use sha2::{Digest, Sha256}; let mut roots = BTreeMap::new(); for root in accepted_roots.unwrap_or_default() { if root.role == crate::managed_provider_references::sources::DiscoveryRole::Unknown { @@ -2435,19 +2709,27 @@ pub(super) fn resource_units( } } let mut units = Vec::new(); - let mut identities = BTreeMap::new(); + let mut identities = BTreeMap::<std::path::PathBuf, String>::new(); for ((logical, role), canonical) in roots { - let next = identities.len(); - let ordinal = *identities.entry(canonical.clone()).or_insert(next); + let name = if let Some(name) = identities.get(&canonical) { + name.clone() + } else { + let canonical_text = canonical + .to_str() + .ok_or("selected OpenCode resource root is not UTF-8")?; + let name = format!("root-{:x}", Sha256::digest(canonical_text.as_bytes())); + identities.insert(canonical.clone(), name.clone()); + name + }; let kind = if canonical.is_file() { ProviderResourceKind::File } else { ProviderResourceKind::Directory }; let name = if kind == ProviderResourceKind::File { - format!("root-{ordinal}/source") + format!("{name}/source") } else { - format!("root-{ordinal}") + name }; let destination = freshell_platform::opencode_config::lexical_path( &crate::managed_provider_references::sources::runtime_directory_destination( @@ -2486,6 +2768,165 @@ pub(super) fn resource_units( Ok(units) } +/// Preserve managed OpenCode HOME aliases across recovery. Resource contents +/// live under provider-volume subpaths, so dropping a no-longer-selected alias +/// would hide provider-created files left beside the previously selected data. +pub(super) fn merge_recovery_resource_mounts( + current: &mut [( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + previous: &[freshell_runtime_protocol::ProviderResourceMount], +) -> Result<Vec<freshell_runtime_protocol::ProviderResourceMount>, String> { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + const PROVIDER_HOME: &str = "/home/freshell/provider"; + + fn paired_resource_paths(mount: &ProviderResourceMount) -> bool { + let Some(stage) = mount + .stage_relative_path + .strip_prefix(".freshell/resource-units/") + else { + return false; + }; + let Some(provider) = mount + .provider_relative_path + .strip_prefix(".freshell/provider-roots/") + else { + return false; + }; + stage == provider + && !stage.is_empty() + && [ + Path::new(&mount.stage_relative_path), + Path::new(&mount.provider_relative_path), + ] + .iter() + .all(|path| { + !path.is_absolute() + && path + .components() + .all(|part| matches!(part, std::path::Component::Normal(_))) + }) + } + + for (_, mount) in current.iter_mut() { + if let Some(prior) = previous + .iter() + .find(|prior| prior.effective_destination == mount.effective_destination) + { + if prior.kind != mount.kind || !paired_resource_paths(prior) { + return Err( + "prior OpenCode resource mount changed type or identity; retain inputs and reprepare" + .into(), + ); + } + mount.stage_relative_path = prior.stage_relative_path.clone(); + mount.provider_relative_path = prior.provider_relative_path.clone(); + } + } + + let current_destinations = current + .iter() + .map(|(_, mount)| mount.effective_destination.as_str()) + .collect::<std::collections::HashSet<_>>(); + let mut retained_destinations = std::collections::HashSet::new(); + let mut retained = Vec::new(); + for prior in previous { + if current_destinations.contains(prior.effective_destination.as_str()) + || prior.kind != ProviderResourceKind::Directory + || !Path::new(&prior.effective_destination).starts_with(PROVIDER_HOME) + { + continue; + } + if !paired_resource_paths(prior) { + return Err( + "prior OpenCode resource mount identity is invalid; retain inputs and reprepare" + .into(), + ); + } + if retained_destinations.insert(prior.effective_destination.clone()) { + retained.push(prior.clone()); + } + } + let merged = current + .iter() + .map(|(_, mount)| mount) + .chain(retained.iter()) + .collect::<Vec<_>>(); + for (index, left) in merged.iter().enumerate() { + if left.kind != ProviderResourceKind::Directory { + continue; + } + for right in merged.iter().skip(index + 1) { + if right.kind != ProviderResourceKind::Directory + || left.effective_destination == right.effective_destination + { + continue; + } + let left_path = Path::new(&left.effective_destination); + let right_path = Path::new(&right.effective_destination); + if left_path.starts_with(right_path) || right_path.starts_with(left_path) { + return Err( + "overlapping retained OpenCode resource mount destinations; retain inputs and reprepare" + .into(), + ); + } + } + } + Ok(retained) +} + +/// Build one physical staging copy per accepted resource subpath while leaving +/// every logical alias descriptor available for the runtime mount plan. +pub(super) fn resource_copy_selections( + units: &[( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], +) -> Result<Vec<freshell_platform::provider_environment::CopySelection>, String> { + use freshell_platform::provider_environment::CopySelection; + let mut sources = BTreeMap::<std::path::PathBuf, std::path::PathBuf>::new(); + let mut selections = Vec::new(); + for (source, mount) in units { + let destination = Path::new(&mount.stage_relative_path).to_owned(); + let canonical = std::fs::canonicalize(source) + .map_err(|_| "selected OpenCode resource stage source is unavailable")?; + match sources.get(&destination) { + Some(prior) if prior == &canonical => continue, + Some(_) => { + return Err( + "shared OpenCode resource subpath has conflicting sources; retain inputs and reprepare" + .into(), + ) + } + None => { + sources.insert(destination.clone(), canonical); + selections.push(CopySelection { + source: source.clone(), + destination_relative: destination, + }); + } + } + } + Ok(selections) +} + +pub(super) fn current_source_for_resource_mount( + current: &[( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + mount: &freshell_runtime_protocol::ProviderResourceMount, +) -> Option<std::path::PathBuf> { + current + .iter() + .find(|(_, candidate)| { + candidate.stage_relative_path == mount.stage_relative_path + && candidate.provider_relative_path == mount.provider_relative_path + }) + .map(|(source, _)| source.clone()) +} + pub(super) fn read_source_view(root: &Path, relative: &str) -> Result<String, String> { use std::io::Read; if Path::new(relative) diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index f97d9c42b..a090af2bc 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -161,7 +161,7 @@ impl DockerEngineBackend { daemon: &DockerDaemonId, ) -> Result<(), BackendError> { let selected = selections(mounts)?; - if selected.is_empty() { + if selected.is_empty() && spec.prior_provider_owner.is_none() { return Ok(()); } if spec.initial_input_reservation.is_none() { @@ -333,10 +333,31 @@ impl DockerEngineBackend { .read_exact(&mut acknowledgement) .await .map_err(|_| fail("provider mountpoint acknowledgement unavailable"))?; - if &acknowledgement != b"ok" { - return Err(fail("provider mountpoint acknowledgement invalid")); + match &acknowledgement { + b"ok" => Ok(()), + b"er" => { + let mut length = [0; 4]; + stream + .read_exact(&mut length) + .await + .map_err(|_| fail("provider mountpoint failure reason unavailable"))?; + let length = u32::from_be_bytes(length) as usize; + if length == 0 || length > 4096 { + return Err(fail("provider mountpoint failure reason exceeds limit")); + } + let mut reason = vec![0; length]; + stream + .read_exact(&mut reason) + .await + .map_err(|_| fail("provider mountpoint failure reason incomplete"))?; + let reason = std::str::from_utf8(&reason) + .map_err(|_| fail("provider mountpoint failure reason invalid"))?; + Err(BackendError::Unavailable(format!( + "provider mountpoint preparation failed: {reason}" + ))) + } + _ => Err(fail("provider mountpoint acknowledgement invalid")), } - Ok(()) }; let wait = async { let response = self @@ -354,7 +375,9 @@ impl DockerEngineBackend { } Ok(()) }; - tokio::try_join!(exchange, wait)?; + let (exchange_result, wait_result) = tokio::join!(exchange, wait); + exchange_result?; + wait_result?; if self.daemon_id().await? != *target.daemon { return Err(fail( "provider mountpoint daemon changed before main create", @@ -398,6 +421,7 @@ mod tests { for mode in [ "prepared", "empty", + "preparation-error", "lost-create-ack", "wrong-caps", "cleanup-still-present", @@ -478,6 +502,7 @@ mod tests { let path=home.join(entry.relative_path);std::fs::create_dir_all(path.parent().unwrap()).unwrap(); if entry.directory {std::fs::create_dir_all(path).unwrap();} else {std::fs::File::create(path).unwrap();} } + if mode=="preparation-error" {channel.write_all(b"er").await.unwrap();channel.write_u32(31).await.unwrap();channel.write_all(b"test preparation failure reason").await.unwrap();return;} channel.write_all(b"ok").await.unwrap(); })); (204,json!({})) @@ -537,7 +562,16 @@ mod tests { matches!(mode, "prepared" | "empty"), "{mode}: candidate preparation must settle with exact owned absence: {result:?}" ); - if matches!(mode, "prepared" | "empty" | "lost-create-ack" | "timeout") { + if mode == "preparation-error" { + assert!( + format!("{result:?}").contains("test preparation failure reason"), + "the helper's bounded failure reason must reach the caller: {result:?}" + ); + } + if matches!( + mode, + "prepared" | "empty" | "preparation-error" | "lost-create-ack" | "timeout" + ) { assert!(helper.is_none()); assert!(removed.load(std::sync::atomic::Ordering::SeqCst)); } diff --git a/test/integration/server/managed-provider-parity.test.ts b/test/integration/server/managed-provider-parity.test.ts index 3544c98db..113a90b99 100644 --- a/test/integration/server/managed-provider-parity.test.ts +++ b/test/integration/server/managed-provider-parity.test.ts @@ -1,4 +1,5 @@ import fs from 'node:fs' +import os from 'node:os' import path from 'node:path' import { createHash, randomUUID } from 'node:crypto' import http from 'node:http' @@ -113,6 +114,20 @@ class TerminalWire { throw new Error(`managed output replay timed out repeatedly for ${terminalId}`) } + async requestAttachOnce(terminalId: string): Promise<'ready' | 'runtime-unavailable'> { + const attachRequestId = `provider-parity-attach-${randomUUID()}` + const fromIndex = this.frames.length + this.send({ type: 'terminal.attach', terminalId, intent: 'viewport_hydrate', + cols: 80, rows: 24, sinceSeq: 0, attachRequestId, priority: 'background' }) + const frame = await this.wait(message => message.terminalId === terminalId && ( + (message.type === 'terminal.attach.ready' + && (!message.attachRequestId || message.attachRequestId === attachRequestId)) + || (message.type === 'error' && message.code === 'INTERNAL_ERROR' + && message.message?.includes('managed output read timed out')) + ), 10_000, fromIndex) + return frame.type === 'terminal.attach.ready' ? 'ready' : 'runtime-unavailable' + } + async close(): Promise<void> { if (this.socket.readyState === WebSocket.CLOSED) return await new Promise<void>(resolve => { @@ -152,6 +167,64 @@ function ordinaryProviderArgs(argv: string[], provider: Provider): string[] { return result } +function providerParityConfigPath(value: string, roots: Record<string, string>): string { + let candidate = value + if (candidate.startsWith('file:')) { + try { candidate = new URL(candidate).pathname } catch { return '<invalid-file-url>' } + } + if (path.isAbsolute(candidate)) { + for (const [label, root] of Object.entries(roots)) { + const relative = path.relative(root, candidate) + if (relative === '' || (!relative.startsWith(`..${path.sep}`) && relative !== '..' && !path.isAbsolute(relative))) { + return `${label}/${relative || '.'}` + } + } + return `external/${path.basename(candidate)}` + } + return candidate +} + +function providerParityConfigProbe(file: string, label: string, roots: Record<string, string>) { + if (!fs.existsSync(file)) return { label, exists: false } + try { + const config = JSON.parse(fs.readFileSync(file, 'utf8')) as Record<string, any> + const mcp = config.mcp && typeof config.mcp === 'object' + ? Object.entries(config.mcp).map(([name, value]) => { + const entry = value as Record<string, any> + return { + name, + type: entry.type, + cwd: typeof entry.cwd === 'string' ? providerParityConfigPath(entry.cwd, roots) : undefined, + commandPaths: Array.isArray(entry.command) + ? entry.command.filter((item): item is string => typeof item === 'string' + && (path.isAbsolute(item) || item.startsWith('file:') || /\.(?:js|mjs|cjs|ts|py|sh)$/.test(item))) + .map(item => providerParityConfigPath(item, roots)) + : undefined, + } + }) + : [] + return { + label, + exists: true, + pluginPaths: Array.isArray(config.plugin) + ? config.plugin.filter((item): item is string => typeof item === 'string') + .map(item => providerParityConfigPath(item, roots)) + : undefined, + instructionPaths: Array.isArray(config.instructions) + ? config.instructions.filter((item): item is string => typeof item === 'string') + .map(item => providerParityConfigPath(item, roots)) + : undefined, + skillPaths: Array.isArray(config.skills?.paths) + ? config.skills.paths.filter((item: unknown): item is string => typeof item === 'string') + .map((item: string) => providerParityConfigPath(item, roots)) + : undefined, + mcp, + } + } catch { + return { label, exists: true, parseableJson: false } + } +} + function normalized(record: RecordRow) { const env = record.env as Record<string, unknown> const tuiConfig = record.provider === 'opencode' && typeof record.tuiConfig === 'string' @@ -247,14 +320,73 @@ describe('ordinary and managed terminal provider parity', () => { const projectOpencodeConfig = () => JSON.stringify({ plugin: [pathToFileURL(path.join(workspace, 'parity-plugin.js')).href], }) + function authorizeManagedRequest(requestId: string): void { + const authorization = authorizeProviderParityRequest(rig.runtime.broker, initialProviderSource, requestId) + registeredVolumes.push({ ...authorization, requestId }) + rig.runtime.writeIncident('provider-parity-registered-requests', registeredVolumes) + } + function providerOwnedMountEvidence(containerId: string): unknown { + const source = `const fs = require('node:fs') +const path = require('node:path') +const root = '/home/freshell/provider/.freshell/provider-roots' +const alias = '/home/freshell/provider/.config/opencode/plugins' +const owned = path.join(alias, 'provider-owned.txt') +const files = {} +for (const entry of fs.readdirSync(root, { withFileTypes: true })) { + if (!entry.isDirectory()) continue + const candidate = path.join(root, entry.name, 'plugins/provider-owned.txt') + if (fs.existsSync(candidate)) files[entry.name] = fs.readFileSync(candidate, 'utf8') +} +let aliasRealPath = null +try { aliasRealPath = fs.realpathSync(alias) } catch {} +let directFile = null +try { directFile = fs.readFileSync(owned, 'utf8') } catch {} +process.stdout.write(JSON.stringify({ files, aliasExists: fs.existsSync(alias), aliasRealPath, directFile }))` + const guest = JSON.parse(rig.ownedProviderExec(containerId, ['node', '-e', source])) + const inspect = rig.runtime.inspectContainer(containerId) + const mounts = Array.isArray(inspect?.HostConfig?.Mounts) ? inspect.HostConfig.Mounts + .map((mount: any) => ({ + type: mount.Type, destination: mount.Target, sourceLeaf: typeof mount.Source === 'string' + ? path.basename(mount.Source) : undefined, + subpath: mount.VolumeOptions?.Subpath, + })) : [] + const mountedResources = Array.isArray(inspect?.Mounts) ? inspect.Mounts + .filter((mount: any) => mount.Type === 'volume' && mount.Destination?.startsWith('/home/freshell/provider/')) + .map((mount: any) => ({ destination: mount.Destination, name: mount.Name, rw: mount.RW })) : [] + const projectTarget = path.join(workspace, '.opencode') + const projectMount = Array.isArray(inspect?.HostConfig?.Mounts) + ? inspect.HostConfig.Mounts.find((mount: any) => mount.Target === projectTarget) + : undefined + const projectSubpath = projectMount?.VolumeOptions?.Subpath + const projectState = typeof projectSubpath === 'string' + ? JSON.parse(rig.ownedProviderExec(containerId, ['node', '-e', ` +const fs=require('node:fs'),path=require('node:path'); +const root=path.join('/home/freshell/provider',${JSON.stringify(projectSubpath)}); +const file=path.join(root,'opencode.json'); +let value=null;try{value=fs.readFileSync(file,'utf8')}catch{} +let metadata=null;try{const s=fs.statSync(file);metadata={size:s.size,mode:s.mode&511,ino:s.ino}}catch{} +process.stdout.write(JSON.stringify({rootExists:fs.existsSync(root),fileExists:fs.existsSync(file),value,metadata})); +`])) + : null + const mountedStage = Array.isArray(inspect?.Mounts) + ? inspect.Mounts.find((mount: any) => mount.Destination === '/run/freshell-private/user-provider')?.Source + : undefined + const stagedResourceMounts = typeof mountedStage === 'string' + && fs.existsSync(path.join(mountedStage, '.freshell/resource-mounts.json')) + ? JSON.parse(fs.readFileSync(path.join(mountedStage, '.freshell/resource-mounts.json'), 'utf8')) : null + const workspaceConfig = path.join(workspace, '.opencode/opencode.json') + const workspaceSource = fs.existsSync(workspaceConfig) + ? { exists: true, bytes: fs.statSync(workspaceConfig).size } : { exists: false } + return { guest, mounts, mountedResources, stagedResourceMounts, projectState, workspaceSource } + } beforeAll(async () => { inheritedFreshellUrl = process.env.FRESHELL_URL delete process.env.FRESHELL_URL - workspace = fs.mkdtempSync(path.join(root, '.provider-parity-')) + workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'freshell-provider-parity-')) // Select this synthetic project, without inheriting repository-local skills. execFileSync('git', ['init', '--quiet', workspace]) - webHome = fs.mkdtempSync(path.join(root, '.provider-parity-home-')) + webHome = fs.mkdtempSync(path.join(os.tmpdir(), 'freshell-provider-parity-home-')) onecli = fakeOnecliGrants(providers) restoreOnecliEnvironment = exposeGrantsToHarness(onecli.serverEnv) xdgConfigHome = path.join(webHome, 'custom-xdg') @@ -320,11 +452,15 @@ describe('ordinary and managed terminal provider parity', () => { // cpSync creates a private destination directory. Restore only the owned // project leaf's shared mode so the provider user can traverse its mount. fs.chmodSync(workspace, 0o777) - makeProviderParityResourcesReadable(path.join(workspace, 'node_modules')) + makeProviderParityResourcesReadable(workspace) + fs.chmodSync(workspace, 0o777) fs.rmSync(preparedWorkspace, { recursive: true }) fs.writeFileSync(path.join(workspace, 'opencode.jsonc'), projectOpencodeConfig()) fs.writeFileSync(path.join(workspace, '.opencode/opencode.json'), projectOpencodeConfig()) fs.writeFileSync(path.join(workspace, 'user-tui.jsonc'), JSON.stringify({ plugin: [pathToFileURL(path.join(workspace, 'user-selected-tui.js')).href] })) + for (const filename of ['opencode.jsonc', '.opencode/opencode.json', 'user-tui.jsonc']) { + fs.chmodSync(path.join(workspace, filename), 0o644) + } const health = await rig.runtime.adminOk(rig.supervisor, { method: 'health' }) expect(health.kind).toBe('health') const daemon = await new Promise<{ ID: string }>((resolve, reject) => { @@ -440,12 +576,43 @@ describe('ordinary and managed terminal provider parity', () => { try { ;({ terminalId } = await wire.create(provider, workspace, managed ? requestId => { - managedRequest = authorizeProviderParityRequest(rig.runtime.broker, initialProviderSource, requestId) - registeredVolumes.push({ ...managedRequest, requestId }) - rig.runtime.writeIncident('provider-parity-registered-requests', registeredVolumes) + authorizeManagedRequest(requestId) + managedRequest = registeredVolumes.at(-1) } : undefined)) } catch (error) { + const serverLog = path.join(rig.info.logsDir, 'rust-server.jsonl') + const failureDiagnostics = providerParityFailureDiagnostics( + `${Object.values(rig.web.capturedOutput()).join('\n')}\n${fs.existsSync(serverLog) + ? fs.readFileSync(serverLog, 'utf8') : ''}`, + ) + rig.runtime.writeIncident(`provider-parity-${provider}-create-failure`, { + provider, + managed, + requestedCwd: workspace, + configInputs: provider === 'opencode' ? [ + ['workspace-opencode-json', path.join(workspace, 'opencode.json')], + ['workspace-opencode-jsonc', path.join(workspace, 'opencode.jsonc')], + ['workspace-tui-json', path.join(workspace, 'tui.json')], + ['workspace-tui-jsonc', path.join(workspace, 'tui.jsonc')], + ['project-opencode-json', path.join(workspace, '.opencode/opencode.json')], + ['project-opencode-jsonc', path.join(workspace, '.opencode/opencode.jsonc')], + ['project-tui-json', path.join(workspace, '.opencode/tui.json')], + ['project-tui-jsonc', path.join(workspace, '.opencode/tui.jsonc')], + ...['config.json', 'opencode.json', 'opencode.jsonc', 'tui.json', 'tui.jsonc'] + .map(name => [`provider-${name}`, path.join(xdgConfigHome, 'opencode', name)] as const), + ['home-opencode-json', path.join(webHome, '.opencode/opencode.json')], + ['home-opencode-jsonc', path.join(webHome, '.opencode/opencode.jsonc')], + ].map(([label, file]) => providerParityConfigProbe(file, label, { + workspace, home: webHome, config: xdgConfigHome, checkout: root, + })) : [], + observedLaunchCwds: wire.recordRows() + .filter(row => row.provider === provider && row.kind === 'launch') + .map(row => row.cwd), + resourceCopyFailures: failureDiagnostics.resourceCopyFailures, + directoryResourceSelections: failureDiagnostics.directoryResourceSelections, + error: String(error), + }) if (managedRequest) { const receipt = rig.runtime.broker.receipts().findLast(value => value.soulId === managedRequest!.soulId && value.workspacePath === workspace) if (receipt) { @@ -475,6 +642,7 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c )).catch(error => { throw new Error(`${String(error)}; records: ${JSON.stringify(wire.recordRows())}; output: ${wire.terminalOutput()}`) }) + expect(launch.cwd, `${provider} ${managed ? 'managed' : 'ordinary'} launch cwd`).toBe(workspace) observed.push(launch) expect(launch.selectedEnvironment).toEqual({ dependency: 'installed-dependency-nonce', resource: 'selected-resource-nonce', entries: 4201, @@ -638,7 +806,7 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c it('resumed OpenCode child sees refreshed config and keeps provider-owned state', async () => { const wire = await TerminalWire.connect(rig.info, true) sockets.push(wire) - const { terminalId } = await wire.create('opencode', workspace) + const { terminalId } = await wire.create('opencode', workspace, authorizeManagedRequest) const first = await waitFor('initial OpenCode launch', () => wire.recordRows().find(row => ( row.provider === 'opencode' && row.kind === 'launch' ))) @@ -661,6 +829,7 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c if (!before.containerId) throw new Error('initial OpenCode container missing') rig.ownedProviderExec(before.containerId, ['sh', '-c', 'printf provider-owned-state > /home/freshell/provider/.config/opencode/plugins/provider-owned.txt']) + const beforeProviderOwnedMount = providerOwnedMountEvidence(before.containerId) fs.writeFileSync(path.join(xdgConfigHome, 'opencode/new-provider.jsonc'), '{"replacement":true}\n') fs.writeFileSync(path.join(xdgConfigHome, 'opencode/opencode.jsonc'), '{"replacementOriginal":true}\n') fs.rmSync(path.join(xdgConfigHome, 'opencode/plugins/parity-plugin.js')) @@ -668,9 +837,15 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c const editedTui = '{ // changed before replacement\n "theme": "light"\n}\n' fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), editedTui) rig.runtime.killOwnedRuntimeExact(before.containerId) + await wire.requestAttachOnce(terminalId) const after = await waitFor('replacement OpenCode incarnation', async () => { - const view = await rig.runningViewForTerminal(terminalId) - return view?.containerId && view.incarnationId !== before.incarnationId ? view : undefined + const rows = (await rig.inventory()).filter((row: any) => row.terminalId === terminalId) + const blocked = rows.find((row: any) => row.recoveryState === 'blocked') + if (blocked) { + throw new Error(`OpenCode replacement is blocked: ${blocked.recoveryReason}; evidenceRevision=${blocked.evidenceRevision}`) + } + return rows.find((row: any) => row.launchState === 'running' + && row.containerId && row.incarnationId !== before.incarnationId) }, 300_000).catch(async error => { const inventory = await rig.inventory() const supervisorLogs = rig.runtime.containerLogs(rig.supervisor.containerId) @@ -685,14 +860,17 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c const resumed = await waitFor('resumed OpenCode child observation', () => wire.recordRows().find(row => ( row.provider === 'opencode' && row.kind === 'launch' && row.launchId !== first.launchId )), 90_000) + const afterProviderOwnedMount = providerOwnedMountEvidence(after.containerId) expect(resumed.providerConfig['new-provider.jsonc']).toEqual({ replacement: true }) expect(resumed.providerConfig['opencode.jsonc']).toEqual({ replacementOriginal: true }) expect(resumed.providerPlugin).toBeNull() - expect(resumed.projectConfig.dotOpencodeJson).toBeNull() + expect(resumed.projectConfig.dotOpencodeJson, + JSON.stringify({ projectConfig: resumed.projectConfig, afterProviderOwnedMount })).toBeNull() expect(resumed.projectedProjectConfig).toBeNull() - expect(resumed.providerOwned).toBe('provider-owned-state') - expect(JSON.parse(resumed.tuiConfig)).toMatchObject({ theme: 'light' }) - expect(JSON.parse(resumed.tuiConfig).plugin.some((plugin: string) => plugin.endsWith('/freshell-rebind-plugin.ts'))).toBe(true) + expect(resumed.providerOwned, + JSON.stringify({ beforeProviderOwnedMount, afterProviderOwnedMount })).toBe('provider-owned-state') + expect(resumed.tuiConfigParsed).toMatchObject({ theme: 'light' }) + expect(resumed.rebindPlugin).toMatchObject({ entrypoint: 'freshell-rebind-plugin.ts', hello: true, sessionId: 'ses_parity' }) expect(fs.readFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), 'utf8')).toBe(editedTui) expect(resumed.tuiConfig).not.toBe(first.tuiConfig) const resumedMcp = await waitFor('resumed OpenCode MCP probe', () => wire.recordRows().filter(row => ( @@ -706,7 +884,7 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c it('relays a managed OpenCode session switch and restores that native session', async () => { const wire = await TerminalWire.connect(rig.info, true) sockets.push(wire) - const { terminalId } = await wire.create('opencode', workspace) + const { terminalId } = await wire.create('opencode', workspace, authorizeManagedRequest) wire.send({ type: 'terminal.input', terminalId, data: 'session switch setup\r' }) const initial = await waitFor('initial managed OpenCode identity', () => wire.recordRows().find(row => ( row.kind === 'identity' && row.terminalId === terminalId @@ -768,7 +946,7 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c it('recovers OpenCode inline and home JSONC TUI inputs after web restart with its container down', async () => { const wire = await TerminalWire.connect(rig.info, true) sockets.push(wire) - const { terminalId, requestId } = await wire.create('opencode', workspace) + const { terminalId, requestId } = await wire.create('opencode', workspace, authorizeManagedRequest) const first = await waitFor('initial OpenCode launch before web restart', () => wire.recordRows().find(row => ( row.provider === 'opencode' && row.kind === 'launch' ))) @@ -776,6 +954,10 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c await waitFor('OpenCode native identity before web restart', () => wire.recordRows().find(row => ( row.kind === 'identity' && row.terminalId === terminalId ))) + await waitFor('OpenCode resume evidence before web restart', async () => { + const view = (await rig.inventory()).find((row: any) => row.terminalId === terminalId) + return view?.durabilityState === 'resume_captured' ? view : undefined + }, 45_000) const before = await waitFor('OpenCode container before web restart', async () => { const view = await rig.runningViewForTerminal(terminalId) return view?.containerId ? view : undefined @@ -786,11 +968,6 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), editedTui) rig.runtime.killOwnedRuntimeExact(before.containerId) await rig.restartWebGracefully() - const after = await waitFor('replacement after web restart', async () => { - const view = await rig.runningViewForTerminal(terminalId) - return view?.containerId && view.incarnationId !== before.incarnationId ? view : undefined - }, 300_000) - expect(after.soulId).toBe(before.soulId) const recoveredWire = await TerminalWire.connect(rig.info, true) sockets.push(recoveredWire) const reconcileId = `provider-parity-reconcile-${randomUUID()}` @@ -802,6 +979,17 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c const reconcile = await recoveredWire.wait(frame => frame.type === 'pane.reconcile.result' && frame.reconcileId === reconcileId, 30_000) expect(reconcile.verdicts[0].verdict).toBe('attach') + await recoveredWire.requestAttachOnce(terminalId) + const after = await waitFor('replacement after web restart', async () => { + const rows = (await rig.inventory()).filter((row: any) => row.terminalId === terminalId) + const blocked = rows.find((row: any) => row.recoveryState === 'blocked') + if (blocked) { + throw new Error(`OpenCode web-restart recovery is blocked: ${blocked.recoveryReason}; evidenceRevision=${blocked.evidenceRevision}`) + } + return rows.find((row: any) => row.launchState === 'running' + && row.containerId && row.incarnationId !== before.incarnationId) + }, 300_000) + expect(after.soulId).toBe(before.soulId) await recoveredWire.attach(terminalId) const resumed = await waitFor('resumed OpenCode child after web restart', () => recoveredWire.recordRows().find(row => ( row.provider === 'opencode' && row.kind === 'launch' && row.launchId !== first.launchId @@ -810,8 +998,8 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c }) expect(resumed.inlineConfig).toBe(first.inlineConfig) expect(resumed.inlineConfig).toMatch(/^[a-f0-9]{64}$/) - expect(JSON.parse(resumed.tuiConfig)).toMatchObject({ theme: 'solarized' }) - expect(JSON.parse(resumed.tuiConfig).plugin.some((plugin: string) => plugin.endsWith('/freshell-rebind-plugin.ts'))).toBe(true) + expect(resumed.tuiConfigParsed).toMatchObject({ theme: 'solarized' }) + expect(resumed.rebindPlugin).toMatchObject({ entrypoint: 'freshell-rebind-plugin.ts', hello: true, sessionId: 'ses_parity' }) expect(fs.readFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), 'utf8')).toBe(editedTui) expect(resumed.tuiConfig).not.toBe(first.tuiConfig) const resumedMcp = await waitFor('resumed OpenCode MCP after web restart', () => recoveredWire.recordRows().find(row => ( From 6e5e70eacfbc6b0af67acccc7528c5b1a4e125aa Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:40:24 -0700 Subject: [PATCH 386/493] fix(managed): identify missing provider secret references --- .../freshell-supervisor/src/backend/docker.rs | 42 ++++++++++++++++++- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/crates/freshell-supervisor/src/backend/docker.rs b/crates/freshell-supervisor/src/backend/docker.rs index 57a618f69..f7db3cb03 100644 --- a/crates/freshell-supervisor/src/backend/docker.rs +++ b/crates/freshell-supervisor/src/backend/docker.rs @@ -259,8 +259,13 @@ fn workload_mounts( let source = if let Some(source) = staged_source { source } else { - std::fs::canonicalize(&secret.source_path) - .map_err(|error| format!("provider secret reference: {error}"))? + std::fs::canonicalize(&secret.source_path).map_err(|error| { + format!( + "provider secret reference {} ({:?}): {error}", + index + 1, + secret.profile + ) + })? }; if !source.is_file() || !source_is_staged && source != Path::new(&secret.source_path) { return Err("provider secret reference must be a canonical regular file".into()); @@ -1041,6 +1046,39 @@ mod provider_secret_tests { assert!(!durable.contains("fixture-secret-bytes")); } + #[test] + fn missing_provider_secret_reports_its_profile_without_exposing_the_source_path() { + use freshell_runtime_protocol::{ProviderSecretProfile, ProviderSecretReference}; + + let owned = tempfile::tempdir().unwrap(); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(&workspace).unwrap(); + let missing = owned.path().join("private/opencode.env"); + let error = match workload_mounts( + WorkloadPaths { + workspace_path: workspace.to_str().unwrap(), + cwd_path: workspace.to_str().unwrap(), + git_common_path: None, + }, + &[], + &[ProviderSecretReference { + provider_relative_path: None, + source_path: missing.to_string_lossy().into_owned(), + profile: ProviderSecretProfile::OpencodeOnecliEnvironment, + }], + None, + "opencode", + MountPurpose::Create, + ) { + Err(error) => error, + Ok(_) => panic!("a missing grant must prevent provider launch"), + }; + + assert!(error.contains("OpencodeOnecliEnvironment"), "{error}"); + assert!(error.contains("No such file or directory"), "{error}"); + assert!(!error.contains(&missing.to_string_lossy().to_string())); + } + #[cfg(unix)] #[test] fn indexed_configuration_reference_uses_original_path_and_preserves_legacy_staged_sources() { From 611e82aafc50b773cefdf6a6efc21cedcb54f271 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:40:27 -0700 Subject: [PATCH 387/493] chore: remove stale Freshell automation skill link --- .codex/skills/freshell-automation-tmux-style | 1 - 1 file changed, 1 deletion(-) delete mode 120000 .codex/skills/freshell-automation-tmux-style diff --git a/.codex/skills/freshell-automation-tmux-style b/.codex/skills/freshell-automation-tmux-style deleted file mode 120000 index 919d8dbcc..000000000 --- a/.codex/skills/freshell-automation-tmux-style +++ /dev/null @@ -1 +0,0 @@ -../../.claude/skills/freshell-automation-tmux-style \ No newline at end of file From fdc70593ff736925233070b793b4e01b122f7453 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:24:56 -0700 Subject: [PATCH 388/493] fix(ui): persist final managed inventory status --- src/App.tsx | 2 +- .../components/App.ws-bootstrap.test.tsx | 50 +++++++++++++++++++ 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/src/App.tsx b/src/App.tsx index 4507b5b7d..280807c59 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -1660,7 +1660,7 @@ export default function App() { } })) } } - if (matchedTerminal && isLive) { + if (matchedTerminal) { dispatch(updateTab({ id: tabId, updates: { status: managedStatus } })) } } diff --git a/test/unit/client/components/App.ws-bootstrap.test.tsx b/test/unit/client/components/App.ws-bootstrap.test.tsx index f2baacf38..6b7cbaadb 100644 --- a/test/unit/client/components/App.ws-bootstrap.test.tsx +++ b/test/unit/client/components/App.ws-bootstrap.test.tsx @@ -1097,6 +1097,56 @@ describe('App WS bootstrap recovery', () => { expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-live-tab')?.status).toBe('running') }) + it.each([ + { recoveryState: 'blocked', runtimeStatus: 'blocked', expectedStatus: 'error' }, + { recoveryState: 'lost', runtimeStatus: 'exited', expectedStatus: 'error' }, + { recoveryState: 'stopped', runtimeStatus: 'exited', expectedStatus: 'exited' }, + ] as const)( + 'persists the final $recoveryState managed status from a disconnected terminal inventory', + async ({ recoveryState, runtimeStatus, expectedStatus }) => { + const store = createStore({ + tabs: [{ id: `inventory-${recoveryState}-tab`, mode: 'opencode', status: 'running' }], + panes: { + layouts: { + [`inventory-${recoveryState}-tab`]: { + type: 'leaf', id: `inventory-${recoveryState}-pane`, content: { + kind: 'terminal', mode: 'opencode', status: 'running', + terminalId: `inventory-${recoveryState}-terminal`, + streamId: `inventory-${recoveryState}-stream`, + createRequestId: `inventory-${recoveryState}-create`, + soulId: `inventory-${recoveryState}-soul`, + }, + }, + }, + activePane: { [`inventory-${recoveryState}-tab`]: `inventory-${recoveryState}-pane` }, + }, + }) + render(<Provider store={store}><App /></Provider>) + await waitFor(() => expect(messageHandler).toBeTypeOf('function')) + + act(() => messageHandler?.({ + type: 'terminal.inventory', + terminals: [{ + terminalId: `inventory-${recoveryState}-terminal`, mode: 'opencode', status: 'exited', + runtimeStatus, soulId: `inventory-${recoveryState}-soul`, incarnationId: `inventory-${recoveryState}-incarnation`, + recoverySummary: { + desiredState: recoveryState === 'stopped' ? 'stopped' : 'running', + recoveryState, + durabilityState: 'live_only', + allocationState: 'degraded', + }, + }], + })) + + const layout = store.getState().panes.layouts[`inventory-${recoveryState}-tab`] + expect(layout.type).toBe('leaf') + if (layout.type !== 'leaf') throw new Error('expected a leaf pane') + expect(layout.content.status).toBe(expectedStatus) + expect(store.getState().tabs.tabs.find(tab => tab.id === `inventory-${recoveryState}-tab`)?.status) + .toBe(expectedStatus) + }, + ) + it('recovers an OpenCode sessionRef from inventory before clearing a stale live handle', async () => { const store = createStore({ tabs: [{ From e7e2daabe043ebd756b03fc7234a150273dd9293 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:24:58 -0700 Subject: [PATCH 389/493] fix(managed): retain Node directory resources --- .../managed_mcp_capability/source_stage.rs | 53 ++++++++++++++++++- .../opencode_resources.rs | 11 ++-- 2 files changed, 59 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index d664fb362..7fe2888cf 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -28,8 +28,9 @@ mod configured_resource_tests { }; let mut command = std::process::Command::new(path); command.env_clear().env("PATH", search_path); - // Match the existing copied-script helper's finite spawn retry, while - // retaining this consumer's isolated environment. ETXTBSY has no child. + // Retain this consumer's isolated environment while retrying a brief + // executable-copy race. A shebang's `/usr/bin/env` can report ETXTBSY + // in stderr after the script process itself has already started. let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); loop { match command.output() { @@ -39,6 +40,16 @@ mod configured_resource_tests { { std::thread::sleep(std::time::Duration::from_millis(5)); } + Ok(output) + if !output.status.success() + && output + .stderr + .windows(b"Text file busy".len()) + .any(|window| window == b"Text file busy") + && std::time::Instant::now() < deadline => + { + std::thread::sleep(std::time::Duration::from_millis(5)); + } result => return result, } } @@ -1117,6 +1128,44 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"static schema loaded\n"); } + #[test] + fn direct_node_script_in_home_keeps_static_import_meta_url_directory_after_all_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.mjs"); + let templates = home.join("templates"); + std::fs::create_dir_all(&templates).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "import { readdirSync } from 'node:fs';\nprocess.stdout.write(readdirSync(new URL('./templates/', import.meta.url)).join(','));\n", + ) + .unwrap(); + std::fs::write(templates.join("welcome.md"), "welcome").unwrap(); + + let (plan, provider_script) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&templates.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .arg(&provider_script) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"welcome.md"); + } + #[test] fn shell_mcp_wrapper_stages_and_runs_node_server_module_closure() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 12592ee80..9cf7b6704 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1300,8 +1300,11 @@ fn insert_direct_script_dependencies( .ok_or_else(|| fail("direct MCP script has no containing directory"))?; for candidate in direct_script_dependency_paths(parent, &source, runtime).map_err(fail)? { let dependency = checked(&candidate, field, declaring)?; - if !dependency.is_file() { - return Err(fail("direct MCP dependency is not a regular file")); + let dependency_is_file = dependency.is_file(); + if !dependency_is_file && !dependency.is_dir() { + return Err(fail( + "direct MCP dependency is neither a regular file nor directory", + )); } if !dependency.starts_with(scope) { return Err(fail( @@ -1315,7 +1318,9 @@ fn insert_direct_script_dependencies( field, declaring, )?; - pending.push(dependency); + if dependency_is_file { + pending.push(dependency); + } } } Ok(()) From 1df5b525e862a159c16bbd755b16bff5d6fd627e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:25:00 -0700 Subject: [PATCH 390/493] fix(managed): reissue after discovered roots disappear --- .../src/managed_mcp_capability.rs | 103 +++++++++++++++++- scripts/testing/runtime-sandbox.ts | 2 +- test/helpers/provider-parity-environment.ts | 2 - .../server/managed-provider-parity.test.ts | 6 +- 4 files changed, 103 insertions(+), 10 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e3a765355..94ff8d0c3 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -329,6 +329,7 @@ struct CapabilityCallback { struct ReissuedOpencodeSources { sources: Vec<crate::managed_provider_references::sources::SourceOccurrence>, native_source_flags: Vec<freshell_platform::opencode_recipe::NativeSourceFlag>, + resource_roots: Vec<freshell_platform::opencode_recipe::DiscoveryRoot>, } fn selected_opencode_sources_for_reissue( @@ -403,9 +404,39 @@ fn selected_opencode_sources_for_reissue( flag.source_ordinal = Some(u16::try_from(new_ordinal).map_err(|_| "native source ordinal overflow")?); } + let mut resource_roots = Vec::new(); + for root in &manifest.resource_roots { + let logical_path = freshell_platform::opencode_config::lexical_path(&root.logical_path); + let explicitly_selected = manifest.native_source_flags.iter().any(|flag| { + flag.kind == freshell_platform::opencode_recipe::NativeSourceFlagKind::Directory + && Path::new(&flag.runtime_destination) == logical_path + }); + match std::fs::canonicalize(&root.logical_path) { + Ok(canonical) if canonical.is_dir() => resource_roots.push(root.clone()), + Ok(_) => { + return Err( + "prior selected OpenCode resource root is not a directory; repair it before reissue" + .into(), + ); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound && !explicitly_selected => { + // A discovered resource root can disappear while the provider is + // stopped. Reissue from the roots that still exist; explicit + // OPENCODE_CONFIG_DIR remains a user-selected required input. + continue; + } + Err(_) => { + return Err( + "prior selected OpenCode resource root is unavailable; repair it before reissue" + .into(), + ); + } + } + } Ok(ReissuedOpencodeSources { sources, native_source_flags, + resource_roots, }) } @@ -784,12 +815,16 @@ async fn apply_prepared_callback( .map(str::to_owned) .or_else(|| std::env::var(name).ok()) }; - let (selected_sources, reissued_native_source_flags) = + let (selected_sources, reissued_native_source_flags, selected_resource_roots) = if let Some(manifest) = &accepted_manifest { let selected = selected_opencode_sources_for_reissue(manifest)?; - (Some(selected.sources), Some(selected.native_source_flags)) + ( + Some(selected.sources), + Some(selected.native_source_flags), + Some(selected.resource_roots), + ) } else { - (None, None) + (None, None, None) }; let inherited_inline = accepted_manifest .as_ref() @@ -844,9 +879,7 @@ async fn apply_prepared_callback( ), selected_sources, accepted_global_root: accepted_manifest.as_ref().map(|manifest| manifest.global_config_root.clone().ok_or_else(|| "accepted global configuration root is unrecorded; retain prior inputs and reprepare before dispatch".to_string())).transpose()?, - selected_resource_roots: accepted_manifest - .as_ref() - .map(|manifest| manifest.resource_roots.clone()), + selected_resource_roots, native_source_flags: reissued_native_source_flags, rebind_enabled: accepted_manifest .as_ref() @@ -3634,6 +3667,64 @@ mod tests { assert!(selected.sources.is_empty()); } + #[test] + fn opencode_reissue_omits_removed_discovered_resource_roots() { + use freshell_platform::opencode_recipe::{DiscoveryRole, DiscoveryRoot}; + + let owned = tempfile::tempdir().unwrap(); + let removed_root = owned.path().join("removed-project/.opencode"); + let retained_root = owned.path().join("retained-project/.opencode"); + std::fs::create_dir_all(&retained_root).unwrap(); + let mut manifest = opencode_manifest(vec![], vec![]); + manifest.resource_roots = vec![ + DiscoveryRoot { + logical_path: removed_root, + role: DiscoveryRole::Workspace, + }, + DiscoveryRoot { + logical_path: retained_root.clone(), + role: DiscoveryRole::Workspace, + }, + ]; + + let selected = selected_opencode_sources_for_reissue(&manifest).unwrap(); + + assert_eq!(selected.resource_roots.len(), 1); + assert_eq!(selected.resource_roots[0].logical_path, retained_root); + assert!(selected.resource_roots[0].role == DiscoveryRole::Workspace); + } + + #[test] + fn opencode_reissue_still_requires_a_removed_explicit_resource_root() { + use freshell_platform::opencode_recipe::{ + DiscoveryRole, DiscoveryRoot, NativeSourceFlag, NativeSourceFlagKind, + }; + + let owned = tempfile::tempdir().unwrap(); + let removed_root = owned.path().join("selected-config-dir"); + let mut manifest = opencode_manifest( + vec![], + vec![NativeSourceFlag { + kind: NativeSourceFlagKind::Directory, + runtime_destination: removed_root.to_string_lossy().into_owned(), + source_ordinal: None, + }], + ); + manifest.resource_roots = vec![DiscoveryRoot { + logical_path: removed_root, + role: DiscoveryRole::Workspace, + }]; + + let error = selected_opencode_sources_for_reissue(&manifest) + .err() + .expect("a removed explicit config directory must block reissue"); + + assert_eq!( + error, + "prior selected OpenCode resource root is unavailable; repair it before reissue" + ); + } + #[test] fn opencode_reissue_still_requires_a_removed_explicit_source() { let owned = tempfile::tempdir().unwrap(); diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index 20f3240e7..b0c06ddb7 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -430,7 +430,7 @@ export class RuntimeHarness { // Linux Unix-domain sockets have a small sun_path budget. Runtime paths // include a long incarnation UUID plus `/host.sock`, so keep the durable // runtime root intentionally short and deterministic across restarts. - const runtimeKey = `${scenarioId.slice(0, 6)}-${createHash('sha256').update(scenarioId).digest('hex').slice(0, 8)}` + const runtimeKey = `${scenarioId.slice(0, 5)}-${createHash('sha256').update(scenarioId).digest('hex').slice(0, 8)}` const runtimeRoot = path.join(this.testRoot, 'r', runtimeKey, 'x') fs.mkdirSync(controlDir, { recursive: true, mode: 0o700 }) fs.mkdirSync(runtimeRoot, { recursive: true, mode: 0o700 }) diff --git a/test/helpers/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts index 94b6b0d2d..f3e041c20 100644 --- a/test/helpers/provider-parity-environment.ts +++ b/test/helpers/provider-parity-environment.ts @@ -134,8 +134,6 @@ export function providerParityCommands(workspace: string): Record<string, string ])) return { ...commands, - OPENCODE_DISABLE_PROJECT_CONFIG: '1', - OPENCODE_CONFIG_DIR: path.join(workspace, '.opencode'), OPENCODE_CONFIG: path.join(workspace, 'opencode.jsonc'), } } diff --git a/test/integration/server/managed-provider-parity.test.ts b/test/integration/server/managed-provider-parity.test.ts index 113a90b99..d71a3c004 100644 --- a/test/integration/server/managed-provider-parity.test.ts +++ b/test/integration/server/managed-provider-parity.test.ts @@ -442,6 +442,10 @@ process.stdout.write(JSON.stringify({rootExists:fs.existsSync(root),fileExists:f // The worker environment selects installed commands and owned OpenCode roots. const providerEnvironment = providerParityCommands(path.join(rig.runtime.testRoot, 'workspace')) inheritedProviderEnvironment = Object.fromEntries(Object.keys(providerEnvironment).map(key => [key, process.env[key]])) + for (const key of ['OPENCODE_CONFIG_DIR', 'OPENCODE_DISABLE_PROJECT_CONFIG']) { + inheritedProviderEnvironment[key] = process.env[key] + delete process.env[key] + } Object.assign(process.env, providerEnvironment) await rig.start() // prepare() recreates this run's root. Populate the project afterwards so @@ -833,7 +837,7 @@ process.stdout.write(JSON.stringify({uid:process.getuid(),gid:process.getgid(),c fs.writeFileSync(path.join(xdgConfigHome, 'opencode/new-provider.jsonc'), '{"replacement":true}\n') fs.writeFileSync(path.join(xdgConfigHome, 'opencode/opencode.jsonc'), '{"replacementOriginal":true}\n') fs.rmSync(path.join(xdgConfigHome, 'opencode/plugins/parity-plugin.js')) - fs.rmSync(path.join(workspace, '.opencode/opencode.json')) + fs.rmSync(path.join(workspace, '.opencode'), { recursive: true, force: true }) const editedTui = '{ // changed before replacement\n "theme": "light"\n}\n' fs.writeFileSync(path.join(xdgConfigHome, 'opencode/tui.jsonc'), editedTui) rig.runtime.killOwnedRuntimeExact(before.containerId) From 34347cdecc26ecc9a75337c04abd59c49dc4d951 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:35:20 -0700 Subject: [PATCH 391/493] fix(managed): recognize env-launched Node resources --- .../managed_mcp_capability/source_stage.rs | 4 +- .../opencode_resources.rs | 78 ++++++++++++++----- 2 files changed, 59 insertions(+), 23 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 7fe2888cf..0b38a3b15 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1101,7 +1101,7 @@ mod configured_resource_tests { std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "import { readFileSync } from 'node:fs';\nprocess.stdout.write(readFileSync(new URL('./schema.json', import.meta.url), 'utf8'));\n", + "import { readFileSync } from 'node:fs';\nprocess.stdout.write(readFileSync(new URL('./schema.json', import.meta.url,), 'utf8'));\n", ) .unwrap(); std::fs::write(&schema, "static schema loaded\n").unwrap(); @@ -1182,7 +1182,7 @@ mod configured_resource_tests { std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\nexec node \"$SCRIPT_DIR/server.js\"\n", + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\nexec env node \"$SCRIPT_DIR/server.js\"\n", ) .unwrap(); std::fs::write( diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 9cf7b6704..934fa7d33 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -709,27 +709,57 @@ fn shell_directory_relative_paths( } fn shell_line_invokes_node(line: &str) -> bool { - let mut command = line.trim_start(); - while let Some(rest) = command - .strip_prefix("exec ") - .or_else(|| command.strip_prefix("command ")) - { - command = rest.trim_start(); + fn executable_name(token: &str) -> String { + let token = token.trim_matches(['"', '\'']); + let name = Path::new(token) + .file_name() + .unwrap_or_default() + .to_string_lossy() + .to_ascii_lowercase(); + name.strip_suffix(".exe").unwrap_or(&name).to_owned() } - let executable = command + + let words = line .split_ascii_whitespace() - .next() - .unwrap_or_default() - .trim_matches(['"', '\'']); - let name = Path::new(executable) - .file_name() - .unwrap_or_default() - .to_string_lossy() - .to_ascii_lowercase(); - matches!( - name.strip_suffix(".exe").unwrap_or(&name), - "node" | "nodejs" - ) + .map(|word| word.trim_matches(['"', '\''])) + .collect::<Vec<_>>(); + let mut index = 0; + while let Some(word) = words.get(index) { + if matches!(*word, "exec" | "command") || word.contains('=') { + index += 1; + } else { + break; + } + } + let Some(executable) = words.get(index) else { + return false; + }; + match executable_name(executable).as_str() { + "node" | "nodejs" => return true, + "env" => index += 1, + _ => return false, + } + + while let Some(argument) = words.get(index) { + if matches!(*argument, "-S" | "--split-string") { + return words + .get(index + 1) + .is_some_and(|command| shell_line_invokes_node(command)); + } + if let Some(command) = argument.strip_prefix("--split-string=") { + return shell_line_invokes_node(command); + } + if matches!(*argument, "-u" | "--unset" | "-C" | "--chdir" | "--argv0") { + index += 2; + continue; + } + if argument.starts_with('-') || argument.contains('=') { + index += 1; + continue; + } + return matches!(executable_name(argument).as_str(), "node" | "nodejs"); + } + false } enum JavaScriptToken { @@ -876,10 +906,16 @@ fn node_relative_resource_specifiers( Some(JavaScriptToken::Punctuation(b'.')) ) && matches!(tokens.get(index + 9), Some(JavaScriptToken::Identifier(name)) if name == "url") - && matches!( + && (matches!( tokens.get(index + 10), Some(JavaScriptToken::Punctuation(b')')) - ); + ) || (matches!( + tokens.get(index + 10), + Some(JavaScriptToken::Punctuation(b',')) + ) && matches!( + tokens.get(index + 11), + Some(JavaScriptToken::Punctuation(b')')) + ))); if new_keyword == "new" && url_constructor == "URL" && is_import_meta_url From 7670ee80acd2519c70a461b6ddfdb574da6f70bb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:35:22 -0700 Subject: [PATCH 392/493] fix(ui): summarize mixed managed pane status --- src/App.tsx | 18 ++++++- src/lib/managed-terminal.ts | 8 +++ .../components/App.ws-bootstrap.test.tsx | 50 +++++++++++++++++++ 3 files changed, 74 insertions(+), 2 deletions(-) diff --git a/src/App.tsx b/src/App.tsx index 280807c59..03de29836 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -110,7 +110,12 @@ import { updateSettingsLocal } from '@/store/settingsSlice' import { setTerminalMetaSnapshot, upsertTerminalMeta, removeTerminalMeta } from '@/store/terminalMetaSlice' import { clearAllReconcilePendingPanes, clearDeadSessionAdjudication, clearDeadTerminals, clearReconcileWarming, clearTerminalLiveHandles, setReconcilePendingPanes, updatePaneContent } from '@/store/panesSlice' import { collectPaneEntries } from '@/lib/pane-utils' -import { isRetainedManagedTerminal, managedTerminalProjection, managedTerminalResumePaneStatus } from '@/lib/managed-terminal.js' +import { + aggregateTerminalTabStatus, + isRetainedManagedTerminal, + managedTerminalProjection, + managedTerminalResumePaneStatus, +} from '@/lib/managed-terminal.js' import { addTerminalFreshRecoveryRequestId, addTerminalRestoreRequestId, setPaneReconcileActive } from '@/lib/terminal-restore' import { reconcileTerminalSessionAssociation } from '@/lib/terminal-session-association' import { foldTerminalInventoryTitles } from '@/lib/terminal-inventory-titles' @@ -1621,6 +1626,7 @@ export default function App() { } if (msg.type === 'terminal.inventory') { const terminals = Array.isArray(msg.terminals) ? msg.terminals : [] + const tabsWithManagedInventory = new Set<string>() const terminalMeta = Array.isArray(msg.terminalMeta) ? msg.terminalMeta : [] const terminalMetaRequestedAt = Date.now() const previousTerminalMeta = appStore.getState().terminalMeta?.byTerminalId ?? {} @@ -1661,11 +1667,19 @@ export default function App() { } } if (matchedTerminal) { - dispatch(updateTab({ id: tabId, updates: { status: managedStatus } })) + tabsWithManagedInventory.add(tabId) } } } } + for (const tabId of tabsWithManagedInventory) { + const layout = appStore.getState().panes.layouts[tabId] + if (!layout) continue + const status = aggregateTerminalTabStatus(collectPaneEntries(layout).flatMap(({ content }) => ( + content.kind === 'terminal' ? [content.status] : [] + ))) + if (status) dispatch(updateTab({ id: tabId, updates: { status } })) + } const liveIds = terminals .filter((t: any) => t.status === 'running' || isRetainedManagedTerminal(t)) .map((t: any) => t.terminalId as string) diff --git a/src/lib/managed-terminal.ts b/src/lib/managed-terminal.ts index abcd05a99..a07797630 100644 --- a/src/lib/managed-terminal.ts +++ b/src/lib/managed-terminal.ts @@ -21,6 +21,14 @@ export function managedTerminalResumePaneStatus(terminal: ManagedRuntimeProjecti return managedTerminalPaneStatus(terminal, fallback) } +/** Summarize a split tab from every terminal pane in a stable severity order. */ +export function aggregateTerminalTabStatus(statuses: readonly TerminalStatus[]): TerminalStatus | undefined { + for (const status of ['error', 'recovering', 'running', 'creating', 'exited'] as const) { + if (statuses.includes(status)) return status + } + return undefined +} + export function managedTerminalRecoveryLabel(terminal: ManagedRuntimeProjectionFields): string | undefined { if (!terminal.soulId) return undefined switch (terminal.recoverySummary?.recoveryState) { diff --git a/test/unit/client/components/App.ws-bootstrap.test.tsx b/test/unit/client/components/App.ws-bootstrap.test.tsx index 6b7cbaadb..e976f5c88 100644 --- a/test/unit/client/components/App.ws-bootstrap.test.tsx +++ b/test/unit/client/components/App.ws-bootstrap.test.tsx @@ -1147,6 +1147,56 @@ describe('App WS bootstrap recovery', () => { }, ) + it('summarizes mixed managed pane outcomes independent of terminal inventory order', async () => { + const store = createStore({ + tabs: [{ id: 'inventory-mixed-tab', mode: 'opencode', status: 'running' }], + panes: { + layouts: { + 'inventory-mixed-tab': { + type: 'split', id: 'inventory-mixed-split', direction: 'horizontal', sizes: [50, 50], children: [ + { + type: 'leaf', id: 'inventory-mixed-lost-pane', content: { + kind: 'terminal', mode: 'opencode', status: 'running', terminalId: 'inventory-mixed-lost-terminal', + streamId: 'inventory-mixed-lost-stream', createRequestId: 'inventory-mixed-lost-create', + soulId: 'inventory-mixed-lost-soul', + }, + }, + { + type: 'leaf', id: 'inventory-mixed-live-pane', content: { + kind: 'terminal', mode: 'opencode', status: 'running', terminalId: 'inventory-mixed-live-terminal', + streamId: 'inventory-mixed-live-stream', createRequestId: 'inventory-mixed-live-create', + soulId: 'inventory-mixed-live-soul', + }, + }, + ], + }, + }, + activePane: { 'inventory-mixed-tab': 'inventory-mixed-live-pane' }, + }, + }) + render(<Provider store={store}><App /></Provider>) + await waitFor(() => expect(messageHandler).toBeTypeOf('function')) + + const lost = { + terminalId: 'inventory-mixed-lost-terminal', mode: 'opencode', status: 'exited', runtimeStatus: 'exited', + soulId: 'inventory-mixed-lost-soul', incarnationId: 'inventory-mixed-lost-incarnation', + recoverySummary: { + desiredState: 'running', recoveryState: 'lost', durabilityState: 'live_only', allocationState: 'degraded', + }, + } + const live = { + terminalId: 'inventory-mixed-live-terminal', mode: 'opencode', status: 'running', runtimeStatus: 'running', + soulId: 'inventory-mixed-live-soul', incarnationId: 'inventory-mixed-live-incarnation', + } + const deliver = (terminals: Array<typeof lost | typeof live>) => + act(() => messageHandler?.({ type: 'terminal.inventory', terminals })) + + deliver([lost, live]) + expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-mixed-tab')?.status).toBe('error') + deliver([live, lost]) + expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-mixed-tab')?.status).toBe('error') + }) + it('recovers an OpenCode sessionRef from inventory before clearing a stale live handle', async () => { const store = createStore({ tabs: [{ From 692455fa15f10c2df0931aee3da0730c89065f5a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:47:02 -0700 Subject: [PATCH 393/493] fix(managed): identify generated MCP by declaring project --- .../freshell-platform/src/opencode_config.rs | 18 +++++++++++------- .../src/managed_mcp_capability/source_stage.rs | 1 - .../src/managed_provider_references.rs | 1 - .../opencode_resources.rs | 1 - .../src/managed_provider_references/sources.rs | 4 +++- 5 files changed, 14 insertions(+), 11 deletions(-) diff --git a/crates/freshell-platform/src/opencode_config.rs b/crates/freshell-platform/src/opencode_config.rs index d1c033402..7f9cfd7ff 100644 --- a/crates/freshell-platform/src/opencode_config.rs +++ b/crates/freshell-platform/src/opencode_config.rs @@ -175,21 +175,25 @@ fn validate_mcp_field(value: &Value, path: &Path) -> Result<(), ConfigError> { } /// Inspect the retained, accepted raw view without reading its source again. -pub fn source_owns_freshell_mcp( - raw: &str, - path: &Path, - project_dir: &Path, -) -> Result<bool, ConfigError> { +/// The generated-entry sidecar belongs to the config file's project root, not +/// necessarily to the provider launch cwd (which may be below that root). +pub fn source_owns_freshell_mcp(raw: &str, path: &Path) -> Result<bool, ConfigError> { let value = parse_jsonc_object(raw).ok_or_else(|| ConfigError::Malformed { path: path.into() })?; validate_mcp_field(&value, path)?; + let project_dir = path + .parent() + .filter(|directory| { + path.file_name() == Some(std::ffi::OsStr::new("opencode.json")) + && directory.file_name() == Some(std::ffi::OsStr::new(".opencode")) + }) + .and_then(Path::parent); let entry = value .get("mcp") .and_then(|mcp| mcp.get("freshell")) .filter(|entry| !entry.is_null()); Ok(entry.is_some_and(|entry| { - path != project_dir.join(".opencode/opencode.json") - || !is_direct_owned_mcp_entry(project_dir, entry) + !project_dir.is_some_and(|project_dir| is_direct_owned_mcp_entry(project_dir, entry)) })) } diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 0b38a3b15..d91cf117a 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -3048,7 +3048,6 @@ pub(super) fn stage( freshell_platform::opencode_config::source_owns_freshell_mcp( source.references.unexpanded_source(), &source.source.logical_path, - cwd, ) .map_err(|error| error.to_string())?; } diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 21167c45a..cbb1cdd5e 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -659,7 +659,6 @@ async fn prepare_managed_opencode_with_typed_publisher( freshell_platform::opencode_config::source_owns_freshell_mcp( &source.view, &source.source.logical_path, - cwd, ) .map(|user_owns| owned || user_owns) }) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 934fa7d33..7f66b032f 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1765,7 +1765,6 @@ pub fn select( && !freshell_platform::opencode_config::source_owns_freshell_mcp( raw, &source.source.logical_path, - &source.instance_cwd, ) .map_err(|_| fail("OpenCode resource", "project MCP ownership check failed"))?; let mut selections = fields(raw).map_err(|cause| fail("OpenCode resource", cause))?; diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index 7e7717efc..e94a5d8a4 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1557,6 +1557,7 @@ mod tests { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let workspace = owned.path().join("workspace"); + let launch_cwd = workspace.join("packages/app"); let checkout = owned.path().join("freshell-checkout"); let config = workspace.join(".opencode/opencode.json"); let state = workspace.join(".opencode/.freshell-mcp-state.json"); @@ -1565,6 +1566,7 @@ mod tests { let user_root = owned.path().join("user-mcp"); let user_script = user_root.join("server.mjs"); std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&launch_cwd).unwrap(); std::fs::create_dir_all(config.parent().unwrap()).unwrap(); std::fs::create_dir_all(loader.parent().unwrap()).unwrap(); std::fs::create_dir_all(server.parent().unwrap()).unwrap(); @@ -1605,7 +1607,7 @@ mod tests { lane: SourceLane::Main, role: DiscoveryRole::Workspace, }], - &workspace, + &launch_cwd, &home, &|_, _, _| None, ) From 28679b64fe8aa9651fb0a95d48177d45d120c0d7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:47:04 -0700 Subject: [PATCH 394/493] fix(ui): preserve final managed inventory status --- src/App.tsx | 16 ++-- src/lib/managed-terminal.ts | 6 +- .../components/App.ws-bootstrap.test.tsx | 90 +++++++++++++++++++ 3 files changed, 102 insertions(+), 10 deletions(-) diff --git a/src/App.tsx b/src/App.tsx index 03de29836..bdd0c870f 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -1672,14 +1672,6 @@ export default function App() { } } } - for (const tabId of tabsWithManagedInventory) { - const layout = appStore.getState().panes.layouts[tabId] - if (!layout) continue - const status = aggregateTerminalTabStatus(collectPaneEntries(layout).flatMap(({ content }) => ( - content.kind === 'terminal' ? [content.status] : [] - ))) - if (status) dispatch(updateTab({ id: tabId, updates: { status } })) - } const liveIds = terminals .filter((t: any) => t.status === 'running' || isRetainedManagedTerminal(t)) .map((t: any) => t.terminalId as string) @@ -1693,6 +1685,14 @@ export default function App() { if (!paneReconcileActiveRef.current) { runDestructiveTerminalCensus(liveIds) } + for (const tabId of tabsWithManagedInventory) { + const layout = appStore.getState().panes.layouts[tabId] + if (!layout) continue + const status = aggregateTerminalTabStatus(collectPaneEntries(layout).flatMap(({ content }) => ( + content.kind === 'terminal' ? [content.status] : [] + ))) + if (status) dispatch(updateTab({ id: tabId, updates: { status } })) + } dispatch(setTerminalMetaSnapshot({ terminals: terminalMeta, requestedAt: terminalMetaRequestedAt })) dispatch(patchSessionRunningStateFromTerminalMeta({ upsert: terminalMeta, diff --git a/src/lib/managed-terminal.ts b/src/lib/managed-terminal.ts index a07797630..42b920ab8 100644 --- a/src/lib/managed-terminal.ts +++ b/src/lib/managed-terminal.ts @@ -12,8 +12,10 @@ export function managedTerminalPaneStatus(terminal: ManagedRuntimeProjectionFiel } /** Status for a pane reconstructed from a retained managed terminal projection. */ -export function managedTerminalResumePaneStatus(terminal: ManagedRuntimeProjectionFields): TerminalStatus { - const fallback = terminal.runtimeStatus === 'exited' +export function managedTerminalResumePaneStatus( + terminal: ManagedRuntimeProjectionFields & { status?: 'running' | 'exited' }, +): TerminalStatus { + const fallback = terminal.runtimeStatus === 'exited' || terminal.status === 'exited' ? 'exited' : terminal.runtimeStatus === 'recovering' ? 'recovering' diff --git a/test/unit/client/components/App.ws-bootstrap.test.tsx b/test/unit/client/components/App.ws-bootstrap.test.tsx index e976f5c88..2f964d5ca 100644 --- a/test/unit/client/components/App.ws-bootstrap.test.tsx +++ b/test/unit/client/components/App.ws-bootstrap.test.tsx @@ -1197,6 +1197,96 @@ describe('App WS bootstrap recovery', () => { expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-mixed-tab')?.status).toBe('error') }) + it('keeps an exited managed inventory row exited when runtime metadata is absent', async () => { + const store = createStore({ + tabs: [{ id: 'inventory-exited-metadata-tab', mode: 'opencode', status: 'running' }], + panes: { + layouts: { + 'inventory-exited-metadata-tab': { + type: 'leaf', id: 'inventory-exited-metadata-pane', content: { + kind: 'terminal', mode: 'opencode', status: 'running', + terminalId: 'inventory-exited-metadata-terminal', + streamId: 'inventory-exited-metadata-stream', + createRequestId: 'inventory-exited-metadata-create', + soulId: 'inventory-exited-metadata-soul', + launchFailure: { code: 'LAUNCH_FAILED', message: 'old launch failed', retryable: true }, + }, + }, + }, + activePane: { 'inventory-exited-metadata-tab': 'inventory-exited-metadata-pane' }, + }, + }) + render(<Provider store={store}><App /></Provider>) + await waitFor(() => expect(messageHandler).toBeTypeOf('function')) + + act(() => messageHandler?.({ + type: 'terminal.inventory', + terminals: [{ + terminalId: 'inventory-exited-metadata-terminal', mode: 'opencode', status: 'exited', + soulId: 'inventory-exited-metadata-soul', + }], + })) + + const layout = store.getState().panes.layouts['inventory-exited-metadata-tab'] + expect(layout.type).toBe('leaf') + if (layout.type !== 'leaf') throw new Error('expected a leaf pane') + expect(layout.content.status).toBe('exited') + expect(layout.content.launchFailure).toEqual({ code: 'LAUNCH_FAILED', message: 'old launch failed', retryable: true }) + expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-exited-metadata-tab')?.status).toBe('exited') + }) + + it('recomputes the tab summary after inventory clears a stale sibling pane', async () => { + const store = createStore({ + tabs: [{ id: 'inventory-stale-sibling-tab', mode: 'opencode', status: 'error' }], + panes: { + layouts: { + 'inventory-stale-sibling-tab': { + type: 'split', id: 'inventory-stale-sibling-split', direction: 'horizontal', sizes: [50, 50], children: [ + { + type: 'leaf', id: 'inventory-stale-managed-pane', content: { + kind: 'terminal', mode: 'opencode', status: 'error', + terminalId: 'inventory-stale-managed-terminal', + streamId: 'inventory-stale-managed-stream', + createRequestId: 'inventory-stale-managed-create', + soulId: 'inventory-stale-managed-soul', + }, + }, + { + type: 'leaf', id: 'inventory-stale-sibling-pane', content: { + kind: 'terminal', mode: 'shell', status: 'error', shell: 'system', + terminalId: 'inventory-stale-sibling-terminal', + streamId: 'inventory-stale-sibling-stream', + createRequestId: 'inventory-stale-sibling-create', + }, + }, + ], + }, + }, + activePane: { 'inventory-stale-sibling-tab': 'inventory-stale-managed-pane' }, + }, + }) + render(<Provider store={store}><App /></Provider>) + await waitFor(() => expect(messageHandler).toBeTypeOf('function')) + + act(() => messageHandler?.({ + type: 'terminal.inventory', + terminals: [{ + terminalId: 'inventory-stale-managed-terminal', mode: 'opencode', status: 'running', + runtimeStatus: 'running', soulId: 'inventory-stale-managed-soul', + incarnationId: 'inventory-stale-managed-incarnation', + }], + })) + + const layout = store.getState().panes.layouts['inventory-stale-sibling-tab'] + expect(layout.type).toBe('split') + if (layout.type !== 'split') throw new Error('expected a split pane') + const staleSibling = layout.children[1] + expect(staleSibling.type).toBe('leaf') + if (staleSibling.type !== 'leaf') throw new Error('expected a stale sibling leaf') + expect(staleSibling.content.status).toBe('creating') + expect(store.getState().tabs.tabs.find(tab => tab.id === 'inventory-stale-sibling-tab')?.status).toBe('running') + }) + it('recovers an OpenCode sessionRef from inventory before clearing a stale live handle', async () => { const store = createStore({ tabs: [{ From 185ddad1a42b84be833456e625aec927b76f91f3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:11:58 -0700 Subject: [PATCH 395/493] fix(managed): preserve OpenCode resource selection --- .../src/managed_mcp_capability.rs | 1 + .../managed_mcp_capability/source_stage.rs | 71 +++++++++++++-- .../src/managed_provider_references.rs | 11 +-- .../environment.rs | 3 + .../opencode_resources.rs | 65 ++++++++++++-- .../managed_provider_references/sources.rs | 86 ++++++++++++++++++- 6 files changed, 211 insertions(+), 26 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 94ff8d0c3..3fbb5adb1 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -10363,6 +10363,7 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ .into_iter() .map(|lane| LoadedSourceOccurrence { requires_config_bind: false, + mcp_ownership_snapshot: None, source: SourceOccurrence { logical_path: workspace.join(if lane == SourceLane::Main { "opencode.json" diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index d91cf117a..f1ab9e6e0 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -3044,12 +3044,7 @@ pub(super) fn stage( return Err("selected configuration discovery role is unrecorded".into()); } if source.source.lane == SourceLane::Main { - manifest.user_owns_freshell_mcp |= - freshell_platform::opencode_config::source_owns_freshell_mcp( - source.references.unexpanded_source(), - &source.source.logical_path, - ) - .map_err(|error| error.to_string())?; + manifest.user_owns_freshell_mcp |= source.user_owns_freshell_mcp; } view_budget .charge(source.references.unexpanded_source()) @@ -3448,6 +3443,70 @@ pub(super) fn describe( mod tests { use super::*; + #[test] + fn generated_mcp_ownership_survives_sidecar_cleanup_after_resource_selection() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let config_dir = workspace.join(".opencode"); + let config = config_dir.join("opencode.json"); + let sidecar = config_dir.join(".freshell-mcp-state.json"); + let stage_root = owned.path().join("stage"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&config_dir).unwrap(); + std::fs::create_dir(&stage_root).unwrap(); + let command = serde_json::json!(["node", "/opt/freshell-mcp/server.js"]); + std::fs::write( + &config, + serde_json::json!({"mcp":{"freshell":{"command":command}}}).to_string(), + ) + .unwrap(); + std::fs::write( + &sidecar, + serde_json::json!({"createdEntry":true,"ownedCommand":command}).to_string(), + ) + .unwrap(); + + let plan = crate::managed_provider_references::sources::plan_opencode_sources( + vec![ + crate::managed_provider_references::sources::SourceOccurrence { + logical_path: config.clone(), + canonical_source: config.clone(), + lane: crate::managed_provider_references::sources::SourceLane::Main, + role: crate::managed_provider_references::sources::DiscoveryRole::Workspace, + }, + ], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!(plan[0].configured_resources.is_empty()); + + // Ordinary OpenCode terminal cleanup can remove this after selection. + std::fs::remove_file(&sidecar).unwrap(); + stage( + &stage_root, + &home, + &workspace, + &plan, + super::super::OpencodeEphemeralInput { + config_root: Some(&home.join(".config/opencode")), + ..Default::default() + }, + ) + .unwrap(); + + let manifest: freshell_platform::opencode_recipe::SourceManifest = serde_json::from_slice( + &std::fs::read(stage_root.join(".freshell/input-manifest.json")).unwrap(), + ) + .unwrap(); + assert!( + !manifest.user_owns_freshell_mcp, + "staging must keep the generated-entry ownership decision paired with resource selection" + ); + } + #[test] fn accepted_source_view_rejects_actual_oversize_before_reading_or_prior_mutation() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index cbb1cdd5e..0605578ab 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -641,6 +641,7 @@ async fn prepare_managed_opencode_with_typed_publisher( .map_err(str::to_string)? }; let mut loaded = read_opencode_sources(selected).map_err(|error| error.to_string())?; + sources::snapshot_freshell_mcp_ownership(&mut loaded).map_err(|error| error.to_string())?; if let Some(raw) = inherited_inline { if raw.len() > 16 * 1024 * 1024 { return Err("OpenCode inherited inline source exceeds 16 MiB".into()); @@ -655,14 +656,7 @@ async fn prepare_managed_opencode_with_typed_publisher( let user_owns_freshell_mcp = loaded .iter() .filter(|source| source.source.lane == SourceLane::Main) - .try_fold(false, |owned, source| { - freshell_platform::opencode_config::source_owns_freshell_mcp( - &source.view, - &source.source.logical_path, - ) - .map(|user_owns| owned || user_owns) - }) - .map_err(|error| error.to_string())?; + .any(|source| source.mcp_ownership_snapshot == Some(true)); let view = freshell_platform::opencode_config::merge_owned_entries_bounded( Some(&raw), &freshell_platform::opencode_config::OpencodeConfigPlan { @@ -676,6 +670,7 @@ async fn prepare_managed_opencode_with_typed_publisher( .map_err(|error| error.to_string())?; let inline = LoadedSourceOccurrence { requires_config_bind: true, + mcp_ownership_snapshot: Some(false), source: SourceOccurrence { logical_path: "OPENCODE_CONFIG_CONTENT".into(), canonical_source: "OPENCODE_CONFIG_CONTENT".into(), diff --git a/crates/freshell-server/src/managed_provider_references/environment.rs b/crates/freshell-server/src/managed_provider_references/environment.rs index 3dc47c964..673a23c37 100644 --- a/crates/freshell-server/src/managed_provider_references/environment.rs +++ b/crates/freshell-server/src/managed_provider_references/environment.rs @@ -260,6 +260,7 @@ mod tests { let loaded = || { vec![LoadedSourceOccurrence { requires_config_bind: false, + mcp_ownership_snapshot: None, source: SourceOccurrence { logical_path: config.clone(), canonical_source: config.clone(), @@ -328,6 +329,7 @@ mod tests { .plan_sources( vec![LoadedSourceOccurrence { requires_config_bind: false, + mcp_ownership_snapshot: None, source: SourceOccurrence { logical_path: config.clone(), canonical_source: config, @@ -387,6 +389,7 @@ mod tests { .plan_sources( vec![LoadedSourceOccurrence { requires_config_bind: false, + mcp_ownership_snapshot: None, source: SourceOccurrence { logical_path: config.clone(), canonical_source: config, diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 7f66b032f..e8b9c63ce 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -259,19 +259,22 @@ fn nested_fields( "OpenCode mcp.script" }; for (index, argument) in command.iter().enumerate() { + let bun_run_file = is_bun_run_file_operand(&command, index); let script_file = if script_runtime { script_index.is_some_and(|script| { index > 0 && index <= script && is_script_path(argument) && !is_bun_run_bare_operand(&command, index) + && !bun_run_file }) } else { index == 0 && is_script_path(argument) }; let script_operand = script_runtime && script_index == Some(index) - && !is_bun_run_bare_operand(&command, index); + && !is_bun_run_bare_operand(&command, index) + && !bun_run_file; if index == 0 && is_explicit_command_path(argument) { if is_script_path(argument) { let field = if is_node_script_path(argument) { @@ -291,6 +294,8 @@ fn nested_fields( let script = &command[script_index]; if is_bun_run_bare_operand(&command, script_index) { result.push(("OpenCode bun run operand", script.clone())); + } else if is_bun_run_file_operand(&command, script_index) { + result.push(("OpenCode bun run file", script.clone())); } } } @@ -367,6 +372,22 @@ fn is_bun_run_bare_operand(command: &[String], index: usize) -> bool { && is_bare_bun_script_name(argument) } +fn is_bun_run_file_operand(command: &[String], index: usize) -> bool { + let Some(executable) = command + .first() + .and_then(|executable| Path::new(executable).file_name()) + else { + return false; + }; + let executable = executable.to_string_lossy().to_ascii_lowercase(); + let executable = executable.strip_suffix(".exe").unwrap_or(&executable); + executable == "bun" + && command.get(1).is_some_and(|argument| argument == "run") + && command + .get(index) + .is_some_and(|argument| is_node_script_path(argument)) +} + fn is_bare_bun_script_name(argument: &str) -> bool { !argument.is_empty() && !argument.starts_with(['-', '.']) @@ -472,6 +493,7 @@ fn is_mcp_command_path_field(field: &str) -> bool { | "OpenCode mcp.node-direct-script" | "OpenCode mcp.shebang-script" | "OpenCode mcp.shebang-node-script" + | "OpenCode bun run file" ) } @@ -1761,12 +1783,8 @@ pub fn select( let mut resource_path_bytes = 0usize; let mut variables = std::collections::BTreeSet::new(); let raw = source.references.unexpanded_source(); - let skip_owned_freshell_mcp = source.source.lane == super::sources::SourceLane::Main - && !freshell_platform::opencode_config::source_owns_freshell_mcp( - raw, - &source.source.logical_path, - ) - .map_err(|_| fail("OpenCode resource", "project MCP ownership check failed"))?; + let skip_owned_freshell_mcp = + source.source.lane == super::sources::SourceLane::Main && !source.user_owns_freshell_mcp; let mut selections = fields(raw).map_err(|cause| fail("OpenCode resource", cause))?; selections.extend( nested_fields( @@ -1877,8 +1895,10 @@ pub fn select( continue; } } - field = if is_script_path(&spelling) { - "OpenCode mcp.node-script" + field = if is_node_script_path(&spelling) { + "OpenCode bun run file" + } else if is_script_path(&spelling) { + "OpenCode mcp.script" } else { "OpenCode mcp.command" }; @@ -2055,6 +2075,33 @@ pub fn select( .parent() .ok_or_else(|| fail(field, "command has no containing scope"))?; let dependency_scope = direct_script_dependency_scope(command_parent, home); + if field == "OpenCode bun run file" { + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical.clone(), + field, + declaring, + )?; + insert_installed_package_ancestor_resources( + &canonical, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Node, + )?; + continue; + } if let Some(package_root) = command_parent .ancestors() .take_while(|ancestor| *ancestor != home && ancestor.parent().is_some()) diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index e94a5d8a4..b23cef8b0 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -311,6 +311,9 @@ pub struct PlannedSourceOccurrence { pub requires_config_bind: bool, pub instance_cwd: PathBuf, pub configured_resources: Vec<PathBuf>, + /// Captured with the retained source view so later staging does not + /// reinterpret a generated entry after ordinary-terminal cleanup. + pub user_owns_freshell_mcp: bool, } #[derive(Default)] @@ -350,10 +353,63 @@ pub fn retained_source_view_bytes( Ok(budget.bytes) } +/// Capture generated-versus-user MCP ownership only after all retained source +/// views pass the shared byte bound. The snapshot then travels with the views +/// through resource selection and staging. +pub fn snapshot_freshell_mcp_ownership( + sources: &mut [LoadedSourceOccurrence], +) -> Result<(), super::ReferenceError> { + let mut budget = SourceViewBudget::default(); + for source in sources.iter() { + budget + .charge(&source.view) + .map_err(|cause| super::ReferenceError { + preparation_failure: None, + operation: "read", + field: "OpenCode source", + source_ordinal: 0, + declaring_source: source.source.logical_path.clone(), + variables: vec![], + cause, + })?; + } + snapshot_freshell_mcp_ownership_after_budget(sources)?; + Ok(()) +} + +fn snapshot_freshell_mcp_ownership_after_budget( + sources: &mut [LoadedSourceOccurrence], +) -> Result<(), super::ReferenceError> { + for (ordinal, source) in sources.iter_mut().enumerate() { + if source.mcp_ownership_snapshot.is_some() { + continue; + } + source.mcp_ownership_snapshot = Some(if source.source.lane == SourceLane::Main { + freshell_platform::opencode_config::source_owns_freshell_mcp( + &source.view, + &source.source.logical_path, + ) + .map_err(|_| super::ReferenceError { + preparation_failure: None, + operation: "select", + field: "OpenCode MCP ownership", + source_ordinal: ordinal, + declaring_source: source.source.logical_path.clone(), + variables: vec![], + cause: "selected source MCP ownership could not be determined", + })? + } else { + false + }); + } + Ok(()) +} + pub struct LoadedSourceOccurrence { pub source: SourceOccurrence, pub view: std::sync::Arc<String>, pub requires_config_bind: bool, + pub mcp_ownership_snapshot: Option<bool>, } /// Retain one bounded canonical read while keeping every native logical origin. @@ -410,6 +466,7 @@ pub fn read_opencode_sources( source, view, requires_config_bind: false, + mcp_ownership_snapshot: None, }); } Ok(loaded) @@ -485,7 +542,7 @@ pub fn plan_loaded_opencode_sources( ) } pub fn plan_loaded_opencode_sources_relocated( - sources: Vec<LoadedSourceOccurrence>, + mut sources: Vec<LoadedSourceOccurrence>, cwd: &Path, home: &Path, lookup: &dyn Fn(SourceLane, &Path, &str) -> Option<String>, @@ -526,7 +583,9 @@ pub fn plan_loaded_opencode_sources_relocated( cause, })?; } + snapshot_freshell_mcp_ownership_after_budget(&mut sources)?; for (ordinal, loaded) in sources.into_iter().enumerate() { + let user_owns_freshell_mcp = loaded.mcp_ownership_snapshot.unwrap_or(false); let source = loaded.source; let original_form = source.lane != SourceLane::Inline && canonical_occurrences[&source.canonical_source] > 1; @@ -656,6 +715,7 @@ pub fn plan_loaded_opencode_sources_relocated( requires_config_bind: loaded.requires_config_bind, instance_cwd: cwd.into(), configured_resources: Vec::new(), + user_owns_freshell_mcp, }; let (resources, names) = super::opencode_resources::select(&planned, home, global.as_deref(), &|name| { @@ -1448,9 +1508,19 @@ mod tests { ) .expect("without a matching package script, a bare Bun file operand remains selectable"); + assert!( + plan[0] + .configured_resources + .contains(&workspace.join("mcp.js").canonicalize().unwrap()), + "selected Bun script resources were {:?}", + plan[0].configured_resources + ); assert!(plan[0] .configured_resources - .contains(&workspace.join("mcp.js").canonicalize().unwrap())); + .contains(&workspace.join("package.json").canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&workspace.canonicalize().unwrap())); } #[test] @@ -1494,9 +1564,19 @@ mod tests { ) .expect("Bun treats an explicit ./ operand as a source file"); + assert!( + plan[0] + .configured_resources + .contains(&workspace.join("mcp.js").canonicalize().unwrap()), + "selected Bun script resources were {:?}", + plan[0].configured_resources + ); assert!(plan[0] .configured_resources - .contains(&workspace.join("mcp.js").canonicalize().unwrap())); + .contains(&workspace.join("package.json").canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&workspace.canonicalize().unwrap())); } #[test] From ac37024db3df0006b62d5c85f9bd60a3079534cd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:30:02 -0700 Subject: [PATCH 396/493] fix(managed): retain Bun script resources --- .../managed_mcp_capability/source_stage.rs | 139 +++++++++++++++++- .../opencode_resources.rs | 76 +++++++++- 2 files changed, 203 insertions(+), 12 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index f1ab9e6e0..e3f215de4 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -152,12 +152,18 @@ mod configured_resource_tests { let units = resource_units(home, &home.join(".config/opencode"), workspace, &plan, None).unwrap(); let selected_resource = selected_resource.canonicalize().unwrap(); - let mount = units + let effective_destination = units .iter() - .find(|(source, _)| source == &selected_resource) - .map(|(_, mount)| mount) + .find(|(source, _)| { + source == &selected_resource + || (source.is_dir() && selected_resource.starts_with(source)) + }) + .map(|(source, mount)| { + Path::new(&mount.effective_destination) + .join(selected_resource.strip_prefix(source).unwrap()) + }) .expect("the selected MCP script must be in the resource plan"); - let effective_destination = Path::new(&mount.effective_destination); + let effective_destination = effective_destination.as_path(); let provider_relative = effective_destination .strip_prefix(owned_root) .map(Path::to_path_buf) @@ -2082,6 +2088,131 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"parent dependency loaded"); } + #[test] + fn bun_run_file_preserves_preload_module_operand_after_all_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let preload_package = owned.path().join("external/preload-package"); + let package = owned.path().join("external/server-package"); + let preload = preload_package.join("preload.mjs"); + let server = package.join("server.mjs"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&preload_package).unwrap(); + std::fs::create_dir_all(&package).unwrap(); + std::fs::write(&preload, "globalThis.fixturePreloadLoaded = true;\n").unwrap(); + std::fs::write( + &server, + "if (!globalThis.fixturePreloadLoaded) process.exit(7); process.stdout.write('preload loaded');\n", + ) + .unwrap(); + let command = vec![ + "bun".to_owned(), + "run".to_owned(), + "--import".to_owned(), + preload.to_string_lossy().into_owned(), + server.to_string_lossy().into_owned(), + ]; + + let (plan, provider_server) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command, + &server, + ); + let canonical_preload = preload.canonicalize().unwrap(); + assert!( + plan[0].configured_resources.iter().any(|selected| { + selected == &canonical_preload + || (selected.is_dir() && canonical_preload.starts_with(selected)) + }), + "Bun's --import preload module must be selected as a configured resource" + ); + + let hidden = owned.path().join("external-hidden"); + std::fs::rename(owned.path().join("external"), &hidden).unwrap(); + let provider_preload = provider_server + .parent() + .and_then(Path::parent) + .unwrap() + .join("preload-package/preload.mjs"); + assert!( + provider_preload.is_file(), + "the preload must survive both copies" + ); + let output = std::process::Command::new("node") + .arg("--import") + .arg(provider_preload) + .arg(&provider_server) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden, owned.path().join("external")).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"preload loaded"); + } + + #[test] + fn external_bun_run_file_keeps_computed_sibling_assets_after_all_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let external = owned.path().join("external"); + let workspace = external.join("project"); + let package = external.join("mcp-package"); + let server = package.join("server.mjs"); + let schema = package.join("schema.json"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&package).unwrap(); + std::fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); + std::fs::write( + &server, + "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(readFileSync(path.join(scriptDir, 'schema.json'), 'utf8'));\n", + ) + .unwrap(); + std::fs::write(&schema, "computed sibling schema loaded\n").unwrap(); + let command = vec![ + "bun".to_owned(), + "run".to_owned(), + server.to_string_lossy().into_owned(), + ]; + + let (plan, provider_server) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command, + &server, + ); + assert!( + plan[0] + .configured_resources + .contains(&package.canonicalize().unwrap()), + "an external Bun script must retain its bounded package directory for computed resources" + ); + assert!( + !plan[0] + .configured_resources + .contains(&external.canonicalize().unwrap()), + "resource closure must not widen to the unrelated external parent" + ); + + let hidden = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg(&provider_server) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden, &external).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"computed sibling schema loaded\n"); + } + #[test] fn managed_provider_nested_attention_and_literal_tilde_keep_native_origins_after_both_copies() { for scenario in ["nested", "top-replacement", "tilde"] { diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index e8b9c63ce..75b0fe01c 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -259,7 +259,8 @@ fn nested_fields( "OpenCode mcp.script" }; for (index, argument) in command.iter().enumerate() { - let bun_run_file = is_bun_run_file_operand(&command, index); + let bun_run_file = + script_index == Some(index) && is_bun_run_file_operand(&command, index); let script_file = if script_runtime { script_index.is_some_and(|script| { index > 0 @@ -388,6 +389,34 @@ fn is_bun_run_file_operand(command: &[String], index: usize) -> bool { .is_some_and(|argument| is_node_script_path(argument)) } +fn bun_run_file_resource_scope( + command_parent: &Path, + home: &Path, + instance_cwd: &Path, +) -> Option<PathBuf> { + let canonical_home = std::fs::canonicalize(home).unwrap_or_else(|_| home.to_owned()); + let canonical_cwd = + std::fs::canonicalize(instance_cwd).unwrap_or_else(|_| instance_cwd.to_owned()); + let package_root = command_parent + .ancestors() + .take_while(|ancestor| *ancestor != canonical_home && ancestor.parent().is_some()) + .find(|ancestor| ancestor.join("package.json").is_file()); + let is_bounded = |scope: &Path| { + scope != Path::new("/") + && scope != canonical_home + && !scope.starts_with(&canonical_cwd) + && !canonical_cwd.starts_with(scope) + }; + + // Prefer the package boundary, but fall back to the script's own directory + // if the package also contains the mounted project. A script inside the + // launch cwd already has its files from the workspace mount. + package_root + .filter(|scope| is_bounded(scope)) + .or_else(|| is_bounded(command_parent).then_some(command_parent)) + .map(Path::to_owned) +} + fn is_bare_bun_script_name(argument: &str) -> bool { !argument.is_empty() && !argument.starts_with(['-', '.']) @@ -2076,13 +2105,6 @@ pub fn select( .ok_or_else(|| fail(field, "command has no containing scope"))?; let dependency_scope = direct_script_dependency_scope(command_parent, home); if field == "OpenCode bun run file" { - insert_resource( - &mut result, - &mut resource_path_bytes, - canonical.clone(), - field, - declaring, - )?; insert_installed_package_ancestor_resources( &canonical, home, @@ -2100,6 +2122,44 @@ pub fn select( field, ScriptDependencyRuntime::Node, )?; + if let Some(resource_scope) = + bun_run_file_resource_scope(command_parent, home, &source.instance_cwd) + { + // Bun scripts can compute local asset paths at runtime, so + // preserve their bounded package (or containing) directory. + // Static dependencies outside that directory remain exact + // selected resources, while covered children travel once + // in the complete directory unit. + let covered = result + .iter() + .filter(|path| { + path.as_path() != resource_scope && path.starts_with(&resource_scope) + }) + .cloned() + .collect::<Vec<_>>(); + for path in covered { + result.remove(&path); + resource_path_bytes = + resource_path_bytes.saturating_sub(path.as_os_str().len()); + } + insert_resource( + &mut result, + &mut resource_path_bytes, + resource_scope, + field, + declaring, + )?; + } else { + // HOME, system roots, and the mounted project are not + // bounded external copy units. Keep exact/static closure. + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical, + field, + declaring, + )?; + } continue; } if let Some(package_root) = command_parent From 63f5492bbb2f60ad08989ac4285d35c821ae4fc9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:56:14 -0700 Subject: [PATCH 397/493] test(managed): retry transient executable busy fixture --- .../src/managed_provider_references/sources.rs | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index b23cef8b0..a315a1222 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -1218,7 +1218,19 @@ mod tests { std::fs::write(&host_command, "#!/bin/sh\nprintf 'mcp fixture\\n'\n").unwrap(); use std::os::unix::fs::PermissionsExt; std::fs::set_permissions(&host_command, std::fs::Permissions::from_mode(0o755)).unwrap(); - let direct_command = std::process::Command::new(&host_command).output(); + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); + let direct_command = loop { + let result = std::process::Command::new(&host_command).output(); + if result + .as_ref() + .is_err_and(|error| error.raw_os_error() == Some(libc::ETXTBSY)) + && std::time::Instant::now() < deadline + { + std::thread::sleep(std::time::Duration::from_millis(5)); + continue; + } + break result; + }; assert!( direct_command .as_ref() From 4cd119f88cdf42091c9d838a2e0beac1fc53a3f7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:56:22 -0700 Subject: [PATCH 398/493] fix(managed): filter workspace from package resources --- .../src/managed_mcp_capability.rs | 120 +++++++++++++++ .../managed_mcp_capability/source_stage.rs | 128 ++++++++++++++-- .../opencode_resources.rs | 18 ++- crates/freshell-session-host/src/main.rs | 145 +++++++++++++++++- 4 files changed, 387 insertions(+), 24 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 3fbb5adb1..7f27d3b5a 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1670,6 +1670,7 @@ fn stage_provider_root( .iter() .map(|reference| PathBuf::from(&reference.source_path)), ); + let mut _filtered_resource_sources = None; let mut retained_resource_sources = None; if let Some(plan) = opencode_input.source_plan { // Indexed payloads remain exclusively in the private input volume, @@ -1732,6 +1733,19 @@ fn stage_provider_root( } else { None }; + if !units.is_empty() { + let scratch = tempfile::tempdir_in(parent).map_err(|error| error.to_string())?; + source_stage::filter_workspace_from_enclosing_resource_units( + &mut units, + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + scratch.path(), + &excluded, + )?; + _filtered_resource_sources = Some(scratch); + } selections.extend(source_stage::resource_copy_selections(&units)?); let descriptors = units .into_iter() @@ -11767,6 +11781,112 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[cfg(unix)] + #[test] + fn managed_opencode_package_resource_scope_excludes_workspace_during_controller_stage() { + use crate::managed_provider_references::sources::*; + use freshell_platform::opencode_recipe::DiscoveryRole; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let external = owned.path().join("external"); + let workspace = external.join("project"); + let tools = external.join("tools"); + let server = tools.join("server.mjs"); + let config = workspace.join(".opencode/opencode.json"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(config.parent().unwrap()).unwrap(); + std::fs::write(workspace.join("workspace-data.txt"), "mounted project\n").unwrap(); + std::fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); + std::fs::write( + &server, + "import {readFileSync} from 'node:fs';import path from 'node:path';import {fileURLToPath} from 'node:url';const dir=path.dirname(fileURLToPath(import.meta.url));process.stdout.write(readFileSync(path.join(dir,'../schema.json'),'utf8'));\n", + ) + .unwrap(); + std::fs::write(external.join("schema.json"), "controller package asset\n").unwrap(); + let command = vec![ + "bun".to_owned(), + "run".to_owned(), + server.to_string_lossy().into_owned(), + ]; + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "computed-asset": { "type": "local", "command": command } }, + }) + .to_string(), + ) + .unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let capabilities = owned.path().join("capabilities"); + std::fs::create_dir(&capabilities).unwrap(); + let staged = stage_provider_root( + &capabilities, + "grant-opencode-package-resource", + "opencode", + &home, + &[], + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + ..Default::default() + }, + None, + ) + .unwrap(); + let mounts: Vec<freshell_runtime_protocol::ProviderResourceMount> = serde_json::from_slice( + &std::fs::read(staged.join(".freshell/resource-mounts.json")).unwrap(), + ) + .unwrap(); + let package_mount = mounts + .iter() + .find(|mount| mount.logical_destination == external.to_string_lossy()) + .expect("the external Bun package root must be staged"); + let package_stage = staged.join(&package_mount.stage_relative_path); + assert!(package_stage.join("tools/server.mjs").is_file()); + assert_eq!( + std::fs::read_to_string(package_stage.join("schema.json")).unwrap(), + "controller package asset\n" + ); + assert!( + !package_stage.join("project").exists(), + "the controller package resource must exclude the mounted workspace" + ); + let workspace_config = mounts + .iter() + .find(|mount| { + mount.logical_destination == workspace.join(".opencode").to_string_lossy() + }) + .expect("the selected workspace OpenCode directory must retain its own mount"); + assert!(staged + .join(&workspace_config.stage_relative_path) + .join("opencode.json") + .is_file()); + + let hidden = owned.path().join("external-hidden"); + std::fs::rename(&external, &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg(package_stage.join("tools/server.mjs")) + .output() + .unwrap(); + std::fs::rename(&hidden, &external).unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"controller package asset\n"); + } + #[test] fn managed_provider_source_stage_generated_view_replays_without_original_tree() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index e3f215de4..165bc7816 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -2161,16 +2161,21 @@ mod configured_resource_tests { let home = owned.path().join("home"); let external = owned.path().join("external"); let workspace = external.join("project"); - let package = external.join("mcp-package"); - let server = package.join("server.mjs"); - let schema = package.join("schema.json"); + let tools = external.join("tools"); + let server = tools.join("server.mjs"); + let schema = external.join("schema.json"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); - std::fs::create_dir_all(&package).unwrap(); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::write( + workspace.join("workspace-data.txt"), + "workspace stays mounted\n", + ) + .unwrap(); std::fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); std::fs::write( &server, - "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(readFileSync(path.join(scriptDir, 'schema.json'), 'utf8'));\n", + "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(readFileSync(path.join(scriptDir, '../schema.json'), 'utf8'));\n", ) .unwrap(); std::fs::write(&schema, "computed sibling schema loaded\n").unwrap(); @@ -2180,24 +2185,64 @@ mod configured_resource_tests { server.to_string_lossy().into_owned(), ]; - let (plan, provider_server) = plan_and_stage_mcp_command_for_resource( - owned.path(), - &home, + let config = workspace.join(".opencode/opencode.json"); + std::fs::create_dir_all(config.parent().unwrap()).unwrap(); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "computed-asset": { "type": "local", "command": command } }, + }) + .to_string(), + ) + .unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], &workspace, - &command, - &server, - ); + &home, + &|_, _, _| None, + ) + .unwrap(); assert!( plan[0] .configured_resources - .contains(&package.canonicalize().unwrap()), + .contains(&external.canonicalize().unwrap()), "an external Bun script must retain its bounded package directory for computed resources" ); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let filtered = tempfile::tempdir_in(owned.path()).unwrap(); + let mut units = units; + filter_workspace_from_enclosing_resource_units( + &mut units, + &workspace, + filtered.path(), + &[], + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_server = provider + .join(external.strip_prefix(owned.path()).unwrap()) + .join("tools/server.mjs"); + let provider_workspace = provider.join(workspace.strip_prefix(owned.path()).unwrap()); + assert!(provider_server.is_file()); assert!( - !plan[0] - .configured_resources - .contains(&external.canonicalize().unwrap()), - "resource closure must not widen to the unrelated external parent" + !provider_workspace.join("workspace-data.txt").exists(), + "the enclosing package copy must leave workspace data out" + ); + assert!( + provider_workspace.join(".opencode/opencode.json").is_file(), + "a separately selected workspace config must survive the package exclusion: {units:?}" ); let hidden = owned.path().join("external-hidden"); @@ -3091,6 +3136,57 @@ pub(super) fn resource_copy_selections( Ok(selections) } +/// Keep a package resource that contains the mounted workspace without +/// copying the workspace into the package unit. Each unit is filtered before +/// the aggregate staging copy, so independently selected workspace resources +/// remain eligible for their own resource mounts. +pub(super) fn filter_workspace_from_enclosing_resource_units( + units: &mut [( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + workspace: &Path, + scratch_root: &Path, + excluded_canonical_sources: &[std::path::PathBuf], +) -> Result<(), String> { + use freshell_platform::provider_environment::{copy_environment, CopySelection}; + use freshell_runtime_protocol::ProviderResourceKind; + + if units.is_empty() { + return Ok(()); + } + let canonical_workspace = std::fs::canonicalize(workspace) + .map_err(|_| "managed provider workspace is unavailable for resource staging")?; + for (index, (source, mount)) in units.iter_mut().enumerate() { + if mount.kind != ProviderResourceKind::Directory { + continue; + } + let canonical_source = std::fs::canonicalize(&*source) + .map_err(|_| "selected OpenCode resource stage source is unavailable")?; + if canonical_workspace == canonical_source + || !canonical_workspace.starts_with(&canonical_source) + { + continue; + } + + let mut exclusions = excluded_canonical_sources.to_vec(); + exclusions.push(canonical_workspace.clone()); + let destination_root = scratch_root.join(index.to_string()); + copy_environment( + &[CopySelection { + source: source.clone(), + destination_relative: "resource".into(), + }], + &destination_root, + &exclusions, + Default::default(), + ) + .map_err(|error| format!("failed to bound enclosing provider resource: {error}"))?; + *source = destination_root.join("resource"); + } + Ok(()) +} + pub(super) fn current_source_for_resource_mount( current: &[( std::path::PathBuf, diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 75b0fe01c..08b84f943 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -397,20 +397,24 @@ fn bun_run_file_resource_scope( let canonical_home = std::fs::canonicalize(home).unwrap_or_else(|_| home.to_owned()); let canonical_cwd = std::fs::canonicalize(instance_cwd).unwrap_or_else(|_| instance_cwd.to_owned()); + // Scripts that live in the mounted project already travel with that + // workspace. An external script may still be packaged beneath a directory + // that contains the project, so that ancestor package remains useful as a + // resource scope; the copy stage excludes the workspace subtree itself. + if command_parent.starts_with(&canonical_cwd) { + return None; + } let package_root = command_parent .ancestors() .take_while(|ancestor| *ancestor != canonical_home && ancestor.parent().is_some()) .find(|ancestor| ancestor.join("package.json").is_file()); let is_bounded = |scope: &Path| { - scope != Path::new("/") - && scope != canonical_home - && !scope.starts_with(&canonical_cwd) - && !canonical_cwd.starts_with(scope) + scope != Path::new("/") && scope != canonical_home && !scope.starts_with(&canonical_cwd) }; - // Prefer the package boundary, but fall back to the script's own directory - // if the package also contains the mounted project. A script inside the - // launch cwd already has its files from the workspace mount. + // Prefer the nearest package boundary, even when it contains the mounted + // project. The caller preserves computed package assets while excluding + // that separately mounted workspace subtree. package_root .filter(|scope| is_bounded(scope)) .or_else(|| is_bounded(command_parent).then_some(command_parent)) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 44b00bdc7..d17de7485 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1853,6 +1853,22 @@ fn copy_environment_with_events_for_mounts( Ok(report) } +fn workspace_suffix_under_resource(resource_root: &Path, workspace: &Path) -> Option<PathBuf> { + let suffix = workspace + .strip_prefix(resource_root) + .ok() + .map(Path::to_path_buf) + .or_else(|| { + let resource_root = std::fs::canonicalize(resource_root).ok()?; + let workspace = std::fs::canonicalize(workspace).ok()?; + workspace + .strip_prefix(resource_root) + .ok() + .map(Path::to_path_buf) + })?; + (!suffix.as_os_str().is_empty()).then(|| suffix.to_path_buf()) +} + fn copy_provider_config_references_with_ownership( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -1862,7 +1878,7 @@ fn copy_provider_config_references_with_ownership( final_home: &Path, run_as: (u32, u32), ) -> Result<(), String> { - use freshell_platform::provider_environment::CopySelection; + use freshell_platform::provider_environment::{copy_environment, CopySelection}; use freshell_runtime_protocol::ProviderConfigRoot; context.validate(provider).map_err(|error| error.message)?; let mut selections = provider_config_references(context) @@ -1935,6 +1951,54 @@ fn copy_provider_config_references_with_ownership( } } } + let mut _filtered_resource_sources = Vec::new(); + for (resource_index, unit) in context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .enumerate() + { + let resource_selection_index = config_selection_count + resource_index; + let selection = selections + .get_mut(resource_selection_index) + .ok_or("provider resource selection is incomplete")?; + if unit.kind != freshell_runtime_protocol::ProviderResourceKind::Directory { + continue; + } + let Some(workspace_suffix) = + workspace_suffix_under_resource(Path::new(&unit.logical_destination), workspace) + else { + continue; + }; + let nested_workspace = selection.source.join(workspace_suffix); + let Ok(canonical_nested_workspace) = std::fs::canonicalize(&nested_workspace) else { + // A current controller stage has already filtered this subtree. + continue; + }; + let Ok(canonical_source) = std::fs::canonicalize(&selection.source) else { + return Err("selected provider resource source is unavailable".into()); + }; + if canonical_nested_workspace == canonical_source { + continue; + } + + let mut resource_exclusions = excluded.clone(); + resource_exclusions.push(canonical_nested_workspace); + let scratch = tempfile::tempdir().map_err(|error| error.to_string())?; + let destination_root = scratch.path().join("filtered-resource"); + copy_environment( + &[CopySelection { + source: selection.source.clone(), + destination_relative: "resource".into(), + }], + &destination_root, + &resource_exclusions, + Default::default(), + ) + .map_err(|error| format!("failed to exclude workspace from provider resource: {error}"))?; + selection.source = destination_root.join("resource"); + _filtered_resource_sources.push(scratch); + } let mounted = if provider_home == final_home { resource_mount_roots(context) } else { @@ -4486,6 +4550,85 @@ mod tests { ); } + #[test] + fn provider_config_resource_projection_excludes_workspace_nested_in_package_root() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let user_root = owned.path().join("source"); + let package = owned.path().join("external"); + let workspace = package.join("project"); + let unit = user_root.join(".freshell/resource-units/root-0"); + let provider_home = owned.path().join("home"); + let provider_resource = provider_home.join(".freshell/provider-roots/root-0"); + std::fs::create_dir_all(unit.join("tools")).unwrap(); + std::fs::create_dir_all(unit.join("project")).unwrap(); + std::fs::write(unit.join("tools/server.mjs"), "export {};\n").unwrap(); + std::fs::write(unit.join("schema.json"), "package asset\n").unwrap(); + std::fs::write( + unit.join("project/workspace-data.txt"), + "mounted workspace\n", + ) + .unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + config: vec![], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + indexed_files: None, + resource_mounts: vec![ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".freshell/provider-roots/root-0".into(), + logical_destination: package.to_string_lossy().into_owned(), + effective_destination: package.to_string_lossy().into_owned(), + kind: ProviderResourceKind::Directory, + }], + }), + mcp_capability: Some(capability), + }; + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + + copy_provider_config_references( + &context, + "opencode", + &workspace, + &user_root, + &provider_home, + uid, + gid, + ) + .unwrap(); + + assert!(provider_resource.join("tools/server.mjs").is_file()); + assert_eq!( + std::fs::read_to_string(provider_resource.join("schema.json")).unwrap(), + "package asset\n" + ); + assert!( + !provider_resource.join("project").exists(), + "session-host projection must not copy workspace data from a stale package unit" + ); + } + #[test] fn provider_config_selected_resource_root_populates_and_refreshes_stable_soul_directory() { use freshell_runtime_protocol::*; From 11c995bfa6d0733f7c3cc93b90f0c68c3bfa7a1e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:26:29 -0700 Subject: [PATCH 399/493] fix(managed): preserve nested resource dependencies --- .../src/provider_environment.rs | 175 ++++++++++- .../src/managed_mcp_capability.rs | 1 + .../managed_mcp_capability/source_stage.rs | 292 +++++++++++++++++- crates/freshell-session-host/src/main.rs | 168 +++++++++- 4 files changed, 615 insertions(+), 21 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 2d115bdd3..55ee71c22 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -125,6 +125,8 @@ struct Plan { mappings: BTreeMap<PathBuf, PathBuf>, pending: VecDeque<(PathBuf, PathBuf, usize)>, exclusions: Vec<PathBuf>, + excluded_subtree: Option<PathBuf>, + included_subtree_aliases: BTreeSet<PathBuf>, counted_files: BTreeSet<PathBuf>, bytes: u64, metadata_bytes: u64, @@ -134,9 +136,44 @@ struct Plan { const MAX_METADATA_BYTES: u64 = 64 * 1024 * 1024; impl Plan { fn excluded(&self, source: &Path) -> bool { - self.exclusions - .iter() - .any(|path| source == path || source.starts_with(path)) + self.exclusions.iter().any(|path| { + if source != path && !source.starts_with(path) { + return false; + } + let included_alias = self.excluded_subtree.as_ref() == Some(path) + && self + .included_subtree_aliases + .iter() + .any(|root| source == root || source.starts_with(root)); + !included_alias + }) + } + fn is_alias_into_excluded_subtree(&self, source: &Path) -> bool { + let Some(excluded_subtree) = &self.excluded_subtree else { + return false; + }; + if !source.starts_with(excluded_subtree) { + return false; + } + !self.exclusions.iter().any(|exclusion| { + exclusion != excluded_subtree && (source == exclusion || source.starts_with(exclusion)) + }) + } + fn materialize_excluded_subtree_alias( + &mut self, + source: PathBuf, + destination: PathBuf, + depth: usize, + ) -> Result<(), CopyError> { + if let Some(mapped) = self.mapped(&source)? { + if mapped != destination { + return self.claim(destination, Node::Link(mapped)); + } + return Ok(()); + } + self.included_subtree_aliases.insert(source.clone()); + self.add_mapping(source.clone(), destination.clone())?; + self.enqueue(source, destination, depth) } fn resolve_destination(&self, path: &Path, include_leaf: bool) -> Result<PathBuf, CopyError> { let mut current = path.to_owned(); @@ -162,7 +199,19 @@ impl Plan { )) } fn mapped(&self, source: &Path) -> Result<Option<PathBuf>, CopyError> { + let inside_excluded_subtree = self + .excluded_subtree + .as_ref() + .is_some_and(|excluded| source == excluded || source.starts_with(excluded)); for root in source.ancestors() { + if inside_excluded_subtree + && !self + .excluded_subtree + .as_ref() + .is_some_and(|excluded| root == excluded || root.starts_with(excluded)) + { + continue; + } if let Some(destination) = self.mappings.get(root) { return self .resolve_destination( @@ -250,7 +299,14 @@ impl Plan { // selected-resource entry budget or cause their bytes to be read. let canonical = fs::canonicalize(&source).map_err(|cause| io_error(&path, "resolve", cause))?; - if self.excluded(&canonical) { + let workspace_alias = if self.excluded(&canonical) { + let metadata = + fs::symlink_metadata(&source).map_err(|cause| io_error(&path, "inspect", cause))?; + metadata.file_type().is_symlink() && self.is_alias_into_excluded_subtree(&canonical) + } else { + false + }; + if self.excluded(&canonical) && !workspace_alias { return Ok(()); } bound( @@ -440,6 +496,13 @@ impl Plan { let canonical = fs::canonicalize(&source).map_err(|cause| io_error(&path, "resolve", cause))?; if self.excluded(&canonical) { + if metadata.file_type().is_symlink() && self.is_alias_into_excluded_subtree(&canonical) + { + let next_depth = depth + .checked_add(1) + .ok_or_else(|| error(&path, "plan", "depth accounting overflow"))?; + return self.materialize_excluded_subtree_alias(canonical, path, next_depth); + } return Ok(()); } if !metadata.file_type().is_symlink() { @@ -585,6 +648,40 @@ fn copy_environment_with_account_home( limits, account_home, &[], + None, + ) +} + +/// Copies a selected resource while omitting one nested tree. Symlink aliases +/// inside the selected resource that target that tree are materialized at the +/// alias path, subject to the same copy limits and every other exclusion. +pub fn copy_environment_excluding_subtree( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + excluded_subtree: &Path, + limits: CopyLimits, +) -> Result<CopyReport, CopyError> { + let excluded_subtree = fs::canonicalize(excluded_subtree).map_err(|cause| { + io_error( + Path::new("excluded-subtree"), + "resolve_excluded_subtree", + cause, + ) + })?; + let mut exclusions = excluded_canonical_sources.to_vec(); + if !exclusions.contains(&excluded_subtree) { + exclusions.push(excluded_subtree.clone()); + } + let account_home = std::env::var_os("HOME").and_then(|home| fs::canonicalize(home).ok()); + copy_environment_projection( + selections, + destination_root, + &exclusions, + limits, + account_home, + &[], + Some(excluded_subtree), ) } @@ -605,6 +702,7 @@ pub fn copy_environment_for_mounts( limits, account_home, mounted_roots, + None, ) } @@ -615,6 +713,7 @@ fn copy_environment_projection( limits: CopyLimits, account_home: Option<PathBuf>, mounted_roots: &[PathBuf], + excluded_subtree: Option<PathBuf>, ) -> Result<CopyReport, CopyError> { bound( Path::new(""), @@ -636,6 +735,8 @@ fn copy_environment_projection( mappings: BTreeMap::new(), pending: VecDeque::new(), exclusions: excluded_canonical_sources.to_vec(), + excluded_subtree, + included_subtree_aliases: BTreeSet::new(), counted_files: BTreeSet::new(), bytes: 0, metadata_bytes: 0, @@ -1263,6 +1364,72 @@ mod tests { assert_eq!(fs::read_dir(&home).unwrap().count(), 2); } + #[cfg(unix)] + #[test] + fn excluding_nested_tree_materializes_only_selected_alias_targets() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + let package = source.join("external"); + let workspace = source.join("project"); + let selected_package = workspace.join("packages/local-package"); + let manifest = r#"{"name":"local-package"}"#; + let module = "export const value = 1;\n"; + fs::create_dir_all(package.join("node_modules")).unwrap(); + fs::create_dir_all(&selected_package).unwrap(); + fs::write(workspace.join("private-project-data"), "must stay excluded").unwrap(); + fs::write(selected_package.join("package.json"), manifest).unwrap(); + fs::write(selected_package.join("index.js"), module).unwrap(); + symlink( + "../../project/packages/local-package", + package.join("node_modules/local-package"), + ) + .unwrap(); + let destination = root.path().join("provider"); + + let report = copy_environment_excluding_subtree( + &[selection(&package, "package")], + &destination, + &[], + &workspace, + CopyLimits::default(), + ) + .unwrap(); + + let materialized = destination.join("package/node_modules/local-package"); + assert!(materialized.join("package.json").is_file()); + assert_eq!( + fs::read_to_string(materialized.join("index.js")).unwrap(), + module + ); + assert!( + !destination + .join("package/project/private-project-data") + .exists(), + "ordinary workspace files remain excluded" + ); + assert_eq!( + report.bytes, + (manifest.len() + module.len()) as u64, + "the enclosing workspace itself is not charged or copied" + ); + + let credential_excluded = root.path().join("credential-excluded-provider"); + copy_environment_excluding_subtree( + &[selection(&package, "package")], + &credential_excluded, + &[selected_package.canonicalize().unwrap()], + &workspace, + CopyLimits::default(), + ) + .unwrap(); + assert!( + !credential_excluded + .join("package/node_modules/local-package") + .exists(), + "the nested-tree allowance must not override another exclusion" + ); + } + #[cfg(unix)] #[test] fn copied_skill_executes_sibling_resources_after_two_stages() { diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 7f27d3b5a..171d83f87 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1737,6 +1737,7 @@ fn stage_provider_root( let scratch = tempfile::tempdir_in(parent).map_err(|error| error.to_string())?; source_stage::filter_workspace_from_enclosing_resource_units( &mut units, + provider, opencode_input .workspace .or(opencode_input.cwd) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 165bc7816..154364753 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -2155,6 +2155,7 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"preload loaded"); } + #[cfg(unix)] #[test] fn external_bun_run_file_keeps_computed_sibling_assets_after_all_copies() { let owned = tempfile::tempdir().unwrap(); @@ -2162,20 +2163,36 @@ mod configured_resource_tests { let external = owned.path().join("external"); let workspace = external.join("project"); let tools = external.join("tools"); + let local_package = workspace.join("packages/local-package"); + let local_package_link = external.join("node_modules/local-package"); let server = tools.join("server.mjs"); let schema = external.join("schema.json"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&local_package).unwrap(); + std::fs::create_dir_all(local_package_link.parent().unwrap()).unwrap(); + std::os::unix::fs::symlink("../project/packages/local-package", &local_package_link) + .unwrap(); std::fs::write( workspace.join("workspace-data.txt"), "workspace stays mounted\n", ) .unwrap(); + std::fs::write( + local_package.join("package.json"), + r#"{"type":"module","exports":"./index.js"}"#, + ) + .unwrap(); + std::fs::write( + local_package.join("index.js"), + "export const marker = 'workspace-linked-package';\n", + ) + .unwrap(); std::fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); std::fs::write( &server, - "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(readFileSync(path.join(scriptDir, '../schema.json'), 'utf8'));\n", + "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nimport { marker } from 'local-package';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(marker + ':' + readFileSync(path.join(scriptDir, '../schema.json'), 'utf8'));\n", ) .unwrap(); std::fs::write(&schema, "computed sibling schema loaded\n").unwrap(); @@ -2225,6 +2242,7 @@ mod configured_resource_tests { let mut units = units; filter_workspace_from_enclosing_resource_units( &mut units, + "opencode", &workspace, filtered.path(), &[], @@ -2255,7 +2273,201 @@ mod configured_resource_tests { std::fs::rename(&hidden, &external).unwrap(); let output = output.unwrap(); assert!(output.status.success(), "{:?}", output.stderr); - assert_eq!(output.stdout, b"computed sibling schema loaded\n"); + assert_eq!( + output.stdout, + b"workspace-linked-package:computed sibling schema loaded\n" + ); + } + + #[test] + fn filtering_shared_home_and_workspace_resource_keeps_one_copy_source() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let external = home.join("external"); + let workspace = external.join("project"); + let tools = external.join("tools"); + let server = tools.join("server.mjs"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::write(&server, "process.stdout.write('ok');\n").unwrap(); + std::fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); + let config = workspace.join(".opencode/opencode.json"); + std::fs::create_dir_all(config.parent().unwrap()).unwrap(); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "shared-package": { "type": "local", "command": [ + "bun", "run", server.to_string_lossy() + ] } }, + }) + .to_string(), + ) + .unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let mut units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let canonical_package = external.canonicalize().unwrap(); + let shared_stage_path = units + .iter() + .find(|(source, _)| source == &canonical_package) + .expect("the package resource must be selected") + .1 + .stage_relative_path + .clone(); + assert_eq!( + units + .iter() + .filter(|(_, mount)| mount.stage_relative_path == shared_stage_path) + .count(), + 2, + "the package is exposed as HOME and workspace aliases" + ); + let filtered = tempfile::tempdir_in(owned.path()).unwrap(); + filter_workspace_from_enclosing_resource_units( + &mut units, + "opencode", + &workspace, + filtered.path(), + &[], + ) + .unwrap(); + + let selections = resource_copy_selections(&units).unwrap(); + assert_eq!( + selections.len(), + 2, + "the shared package and selected workspace config are each copied once" + ); + let aliases = units + .iter() + .filter(|(_, mount)| mount.stage_relative_path == shared_stage_path) + .map(|(source, _)| source) + .collect::<Vec<_>>(); + assert_eq!( + aliases[0], aliases[1], + "both aliases share one filtered source" + ); + } + + #[cfg(unix)] + #[test] + fn workspace_filter_copy_failure_emits_structured_controller_event() { + use std::sync::{Arc, Mutex}; + use tracing_subscriber::prelude::*; + + #[derive(Clone, Default)] + struct Capture(Arc<Mutex<Vec<std::collections::BTreeMap<String, String>>>>); + impl tracing::field::Visit for Capture { + fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) { + self.0 + .lock() + .unwrap() + .last_mut() + .unwrap() + .insert(field.name().into(), format!("{value:?}")); + } + + fn record_str(&mut self, field: &tracing::field::Field, value: &str) { + self.0 + .lock() + .unwrap() + .last_mut() + .unwrap() + .insert(field.name().into(), value.into()); + } + } + impl<S: tracing::Subscriber> tracing_subscriber::Layer<S> for Capture { + fn on_event( + &self, + event: &tracing::Event<'_>, + _: tracing_subscriber::layer::Context<'_, S>, + ) { + self.0.lock().unwrap().push(Default::default()); + event.record(&mut self.clone()); + } + } + + let owned = tempfile::tempdir().unwrap(); + let resource = owned.path().join("external"); + let workspace = resource.join("project"); + std::fs::create_dir_all(&workspace).unwrap(); + std::os::unix::fs::symlink("missing-target", resource.join("dangling-resource")).unwrap(); + let stage_path = ".freshell/resource-units/root-test"; + let mut units = vec![( + resource, + freshell_runtime_protocol::ProviderResourceMount { + stage_relative_path: stage_path.into(), + provider_relative_path: ".freshell/provider-roots/root-test".into(), + logical_destination: "/tmp/external".into(), + effective_destination: "/tmp/external".into(), + kind: freshell_runtime_protocol::ProviderResourceKind::Directory, + }, + )]; + let capture = Capture::default(); + let dispatcher = + tracing::Dispatch::new(tracing_subscriber::registry().with(capture.clone())); + + let error = tracing::dispatcher::with_default(&dispatcher, || { + filter_workspace_from_enclosing_resource_units( + &mut units, + "opencode", + &workspace, + &owned.path().join("filtered"), + &[], + ) + }) + .unwrap_err(); + + assert!(error.contains("failed to bound enclosing provider resource")); + let events = capture.0.lock().unwrap(); + let failed = events + .iter() + .find(|fields| { + fields + .get("event") + .is_some_and(|event| event.contains("provider.environment.copy.failed")) + }) + .expect("filtering failures emit the standard copy failure event"); + assert!(failed + .get("stage") + .is_some_and(|value| value.contains("controller"))); + assert!(failed + .get("provider") + .is_some_and(|value| value.contains("opencode"))); + assert!(failed + .get("resource_stage_path") + .is_some_and(|value| value.contains(stage_path))); + assert!(failed + .get("resource_source_leaf") + .is_some_and(|value| value.contains("external"))); + assert!(failed + .get("relative_path") + .is_some_and(|value| value.contains("resource/dangling-resource"))); + assert!(failed.contains_key("operation")); + assert!(failed.contains_key("corrective_message")); + let serialized = format!("{events:?}"); + assert!( + !serialized.contains(&owned.path().to_string_lossy().to_string()), + "structured resource failures must not log physical source paths" + ); } #[test] @@ -3145,11 +3357,14 @@ pub(super) fn filter_workspace_from_enclosing_resource_units( std::path::PathBuf, freshell_runtime_protocol::ProviderResourceMount, )], + provider: &str, workspace: &Path, scratch_root: &Path, excluded_canonical_sources: &[std::path::PathBuf], ) -> Result<(), String> { - use freshell_platform::provider_environment::{copy_environment, CopySelection}; + use freshell_platform::provider_environment::{ + copy_environment_excluding_subtree, CopySelection, + }; use freshell_runtime_protocol::ProviderResourceKind; if units.is_empty() { @@ -3157,6 +3372,7 @@ pub(super) fn filter_workspace_from_enclosing_resource_units( } let canonical_workspace = std::fs::canonicalize(workspace) .map_err(|_| "managed provider workspace is unavailable for resource staging")?; + let mut filtered_sources = BTreeMap::<std::path::PathBuf, std::path::PathBuf>::new(); for (index, (source, mount)) in units.iter_mut().enumerate() { if mount.kind != ProviderResourceKind::Directory { continue; @@ -3168,21 +3384,77 @@ pub(super) fn filter_workspace_from_enclosing_resource_units( { continue; } + if let Some(filtered_source) = filtered_sources.get(&canonical_source) { + *source = filtered_source.clone(); + continue; + } - let mut exclusions = excluded_canonical_sources.to_vec(); - exclusions.push(canonical_workspace.clone()); let destination_root = scratch_root.join(index.to_string()); - copy_environment( + let resource_source_leaf = source + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()) + .unwrap_or_default(); + let resource_destination_leaf = Path::new(&mount.logical_destination) + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()) + .unwrap_or_default(); + let started = std::time::Instant::now(); + tracing::info!( + event = "provider.environment.copy.started", + stage = "controller", + provider, + resource_stage_path = %mount.stage_relative_path, + resource_source_leaf, + resource_destination_leaf, + "Copying bounded provider resource with nested workspace excluded" + ); + let report = copy_environment_excluding_subtree( &[CopySelection { source: source.clone(), destination_relative: "resource".into(), }], &destination_root, - &exclusions, + excluded_canonical_sources, + &canonical_workspace, Default::default(), - ) - .map_err(|error| format!("failed to bound enclosing provider resource: {error}"))?; - *source = destination_root.join("resource"); + ); + let report = match report { + Ok(report) => report, + Err(error) => { + tracing::error!( + event = "provider.environment.copy.failed", + stage = "controller", + provider, + operation = error.operation, + relative_path = %error.relative_path.display(), + resource_stage_path = %mount.stage_relative_path, + resource_source_leaf, + resource_destination_leaf, + limit_kind = error.limit_kind, + observed = error.observed, + permitted = error.permitted, + elapsed_ms = started.elapsed().as_millis() as u64, + corrective_message = %error.corrective_message + ); + return Err(format!( + "failed to bound enclosing provider resource: {error}" + )); + } + }; + tracing::info!( + event = "provider.environment.copy.completed", + stage = "controller", + provider, + resource_stage_path = %mount.stage_relative_path, + resource_source_leaf, + resource_destination_leaf, + entries = report.entries, + bytes = report.bytes, + elapsed_ms = started.elapsed().as_millis() as u64 + ); + let filtered_source = destination_root.join("resource"); + filtered_sources.insert(canonical_source, filtered_source.clone()); + *source = filtered_source; } Ok(()) } diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index d17de7485..2b283dd03 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1878,7 +1878,9 @@ fn copy_provider_config_references_with_ownership( final_home: &Path, run_as: (u32, u32), ) -> Result<(), String> { - use freshell_platform::provider_environment::{copy_environment, CopySelection}; + use freshell_platform::provider_environment::{ + copy_environment_excluding_subtree, CopySelection, + }; use freshell_runtime_protocol::ProviderConfigRoot; context.validate(provider).map_err(|error| error.message)?; let mut selections = provider_config_references(context) @@ -1952,6 +1954,7 @@ fn copy_provider_config_references_with_ownership( } } let mut _filtered_resource_sources = Vec::new(); + let mut filtered_resource_sources = std::collections::BTreeMap::<PathBuf, PathBuf>::new(); for (resource_index, unit) in context .prepared_inputs .iter() @@ -1981,22 +1984,72 @@ fn copy_provider_config_references_with_ownership( if canonical_nested_workspace == canonical_source { continue; } + if let Some(filtered_source) = filtered_resource_sources.get(&canonical_source) { + selection.source = filtered_source.clone(); + continue; + } - let mut resource_exclusions = excluded.clone(); - resource_exclusions.push(canonical_nested_workspace); let scratch = tempfile::tempdir().map_err(|error| error.to_string())?; let destination_root = scratch.path().join("filtered-resource"); - copy_environment( + let resource_source_leaf = selection + .source + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()) + .unwrap_or_default(); + let resource_destination_leaf = Path::new(&unit.logical_destination) + .file_name() + .map(|leaf| leaf.to_string_lossy().into_owned()) + .unwrap_or_default(); + let started = std::time::Instant::now(); + emit_environment_event(serde_json::json!({ + "severity": "info", "event": "provider.environment.copy.started", + "stage": "session_host", "provider": provider, + "resource_stage_path": unit.stage_relative_path, + "resource_source_leaf": resource_source_leaf, + "resource_destination_leaf": resource_destination_leaf + })); + let report = copy_environment_excluding_subtree( &[CopySelection { source: selection.source.clone(), destination_relative: "resource".into(), }], &destination_root, - &resource_exclusions, + &excluded, + &canonical_nested_workspace, Default::default(), - ) - .map_err(|error| format!("failed to exclude workspace from provider resource: {error}"))?; + ); + let report = match report { + Ok(report) => report, + Err(error) => { + emit_environment_event(serde_json::json!({ + "severity": "error", "event": "provider.environment.copy.failed", + "stage": "session_host", "provider": provider, + "operation": error.operation, + "relative_path": error.relative_path, + "resource_stage_path": unit.stage_relative_path, + "resource_source_leaf": resource_source_leaf, + "resource_destination_leaf": resource_destination_leaf, + "limit_kind": error.limit_kind, "observed": error.observed, + "permitted": error.permitted, + "elapsed_ms": started.elapsed().as_millis() as u64, + "corrective_message": error.corrective_message + })); + return Err(format!( + "failed to exclude workspace from provider resource: {error}" + )); + } + }; + emit_environment_event(serde_json::json!({ + "severity": "info", "event": "provider.environment.copy.completed", + "stage": "session_host", "provider": provider, + "resource_stage_path": unit.stage_relative_path, + "resource_source_leaf": resource_source_leaf, + "resource_destination_leaf": resource_destination_leaf, + "entries": report.entries, "bytes": report.bytes, + "elapsed_ms": started.elapsed().as_millis() as u64 + })); selection.source = destination_root.join("resource"); + filtered_resource_sources.insert(canonical_source, selection.source.clone()); _filtered_resource_sources.push(scratch); } let mounted = if provider_home == final_home { @@ -5717,6 +5770,107 @@ mod tests { .unwrap() .contains("repair")); assert!(event["elapsed_ms"].as_u64().is_some()); + + #[cfg(unix)] + { + let filtered = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "tests::provider_config_workspace_filter_failure_emits_safe_jsonl", + "--nocapture", + ]) + .output() + .unwrap(); + assert!(filtered.status.success(), "{filtered:?}"); + let event = std::str::from_utf8(&filtered.stderr) + .unwrap() + .lines() + .filter_map(|line| serde_json::from_str::<serde_json::Value>(line).ok()) + .find(|event| { + event["event"] == "provider.environment.copy.failed" + && event["stage"] == "session_host" + }) + .expect("nested-workspace filtering failures emit the standard copy failure event"); + assert_eq!(event["provider"], "opencode"); + assert_eq!(event["operation"], "resolve"); + assert_eq!(event["relative_path"], "resource/dangling-resource"); + assert_eq!( + event["resource_stage_path"], + ".freshell/resource-units/root-test" + ); + assert_eq!(event["resource_source_leaf"], "root-test"); + assert_eq!(event["resource_destination_leaf"], "external"); + assert!(event["corrective_message"] + .as_str() + .unwrap() + .contains("repair")); + assert!(event["elapsed_ms"].as_u64().is_some()); + } + } + + #[cfg(unix)] + #[test] + fn provider_config_workspace_filter_failure_emits_safe_jsonl() { + use freshell_runtime_protocol::*; + + let owned = tempfile::tempdir().unwrap(); + let external = owned.path().join("external"); + let workspace = external.join("project"); + let user_root = owned.path().join("source"); + let unit = user_root.join(".freshell/resource-units/root-test"); + std::fs::create_dir_all(unit.join("project")).unwrap(); + std::os::unix::fs::symlink("missing-target", unit.join("dangling-resource")).unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + config: vec![], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + indexed_files: None, + resource_mounts: vec![ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-test".into(), + provider_relative_path: ".freshell/provider-roots/root-test".into(), + logical_destination: external.to_string_lossy().into_owned(), + effective_destination: external.to_string_lossy().into_owned(), + kind: ProviderResourceKind::Directory, + }], + }), + mcp_capability: Some(capability), + }; + + let error = copy_provider_config_references( + &context, + "opencode", + &workspace, + &user_root, + &owned.path().join("provider-home"), + unsafe { libc::geteuid() }, + unsafe { libc::getegid() }, + ) + .unwrap_err(); + assert!( + error.contains("failed to exclude workspace from provider resource"), + "{error}" + ); } #[test] From b889a2686f8f485a4da42c910265fd78d715aa54 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:44:58 -0700 Subject: [PATCH 400/493] fix(managed): preserve file-alias sibling resources --- .../src/provider_environment.rs | 132 ++++++++++++++++-- .../managed_mcp_capability/source_stage.rs | 20 ++- crates/freshell-session-host/src/main.rs | 32 ++++- 3 files changed, 168 insertions(+), 16 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 55ee71c22..cbeda1e02 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -126,7 +126,7 @@ struct Plan { pending: VecDeque<(PathBuf, PathBuf, usize)>, exclusions: Vec<PathBuf>, excluded_subtree: Option<PathBuf>, - included_subtree_aliases: BTreeSet<PathBuf>, + included_subtree_resources: BTreeSet<PathBuf>, counted_files: BTreeSet<PathBuf>, bytes: u64, metadata_bytes: u64, @@ -142,7 +142,7 @@ impl Plan { } let included_alias = self.excluded_subtree.as_ref() == Some(path) && self - .included_subtree_aliases + .included_subtree_resources .iter() .any(|root| source == root || source.starts_with(root)); !included_alias @@ -159,21 +159,23 @@ impl Plan { exclusion != excluded_subtree && (source == exclusion || source.starts_with(exclusion)) }) } + fn include_selected_subtree_resource(&mut self, source: &Path) { + if self + .excluded_subtree + .as_ref() + .is_some_and(|excluded| source == excluded || source.starts_with(excluded)) + { + self.included_subtree_resources.insert(source.to_owned()); + } + } fn materialize_excluded_subtree_alias( &mut self, source: PathBuf, destination: PathBuf, depth: usize, ) -> Result<(), CopyError> { - if let Some(mapped) = self.mapped(&source)? { - if mapped != destination { - return self.claim(destination, Node::Link(mapped)); - } - return Ok(()); - } - self.included_subtree_aliases.insert(source.clone()); - self.add_mapping(source.clone(), destination.clone())?; - self.enqueue(source, destination, depth) + let mapped = self.external_resource(&source, &destination, depth)?; + self.claim(destination, Node::Link(mapped)) } fn resolve_destination(&self, path: &Path, include_leaf: bool) -> Result<PathBuf, CopyError> { let mut current = path.to_owned(); @@ -382,11 +384,32 @@ impl Plan { } } } + if let Some(excluded_subtree) = self + .excluded_subtree + .as_ref() + .filter(|excluded| target == *excluded || target.starts_with(*excluded)) + { + if target == excluded_subtree { + return Err(error( + path, + "select_resource_scope", + "a link to the entire excluded workspace cannot be copied as a bounded dependency; select a package or smaller resource", + )); + } + if resource == *excluded_subtree || !resource.starts_with(excluded_subtree) { + resource = target + .parent() + .filter(|parent| *parent != excluded_subtree) + .map(Path::to_owned) + .unwrap_or_else(|| target.to_owned()); + } + } if resource == Path::new("/") || self.account_home.as_deref() == Some(resource.as_path()) { return Err(error(path,"select_resource_scope","an external resource cannot infer the entire account home or filesystem root; select a complete resource/package directory with its own scope")); } let destination = Path::new(".freshell/resources/tree") .join(resource.strip_prefix(Path::new("/")).unwrap_or(&resource)); + self.include_selected_subtree_resource(&resource); if metadata.is_file() && resource == target { self.add_mapping(target.to_owned(), destination.clone())?; self.enqueue(target.to_owned(), destination, depth)?; @@ -405,6 +428,7 @@ impl Plan { .map_err(|cause| io_error(path, "resolve_lookup_context", cause))?; let destination = Path::new(".freshell/resources/tree") .join(modules.strip_prefix("/").unwrap_or(&modules)); + self.include_selected_subtree_resource(&canonical); self.directory_mapping(&canonical, &destination, depth)?; } let package = ancestor.join("package.json"); @@ -413,6 +437,7 @@ impl Plan { .map_err(|cause| io_error(path, "resolve_package_scope", cause))?; let destination = Path::new(".freshell/resources/tree") .join(package.strip_prefix("/").unwrap_or(&package)); + self.include_selected_subtree_resource(&canonical); if let Some(mapped) = self.mapped(&canonical)? { if mapped != destination { self.claim(destination, Node::Link(mapped))?; @@ -736,7 +761,7 @@ fn copy_environment_projection( pending: VecDeque::new(), exclusions: excluded_canonical_sources.to_vec(), excluded_subtree, - included_subtree_aliases: BTreeSet::new(), + included_subtree_resources: BTreeSet::new(), counted_files: BTreeSet::new(), bytes: 0, metadata_bytes: 0, @@ -1430,6 +1455,89 @@ mod tests { ); } + #[cfg(unix)] + #[test] + fn excluded_workspace_file_alias_keeps_its_sibling_package_resources() { + let root = tempfile::tempdir().unwrap(); + let external = root.path().join("external"); + let workspace = external.join("project"); + let worker_package = workspace.join("packages/worker-package"); + let tools = external.join("tools"); + fs::create_dir_all(&worker_package).unwrap(); + fs::create_dir_all(&tools).unwrap(); + fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write( + worker_package.join("schema.json"), + r#"{"nonce":"linked-file-sibling-loaded"}"#, + ) + .unwrap(); + fs::write( + worker_package.join("worker.mjs"), + "import { readFileSync } from 'node:fs';\nconst schema = JSON.parse(readFileSync(new URL('./schema.json', import.meta.url), 'utf8'));\nexport const nonce = schema.nonce;\n", + ) + .unwrap(); + symlink( + "../project/packages/worker-package/worker.mjs", + tools.join("worker-link.mjs"), + ) + .unwrap(); + fs::write( + tools.join("server.mjs"), + "import { nonce } from './worker-link.mjs';\nprocess.stdout.write(nonce + '\\n');\n", + ) + .unwrap(); + + let destination = root.path().join("provider"); + copy_environment_excluding_subtree( + &[selection(&external, "resource")], + &destination, + &[], + &workspace, + CopyLimits::default(), + ) + .unwrap(); + let copied_server = destination.join("resource/tools/server.mjs"); + let hidden = root.path().join("external-hidden"); + fs::rename(&external, &hidden).unwrap(); + let output = Command::new("node") + .arg(&copied_server) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + fs::rename(&hidden, &external).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"linked-file-sibling-loaded\n"); + } + + #[cfg(unix)] + #[test] + fn excluded_workspace_root_alias_fails_without_copying_the_workspace() { + let root = tempfile::tempdir().unwrap(); + let external = root.path().join("external"); + let workspace = external.join("project"); + fs::create_dir_all(workspace.join("packages")).unwrap(); + fs::create_dir_all(external.join("node_modules")).unwrap(); + fs::write(workspace.join("private-project-data"), "do not copy").unwrap(); + symlink("../project", external.join("node_modules/workspace")).unwrap(); + let destination = root.path().join("provider"); + + let error = copy_environment_excluding_subtree( + &[selection(&external, "resource")], + &destination, + &[], + &workspace, + CopyLimits::default(), + ) + .unwrap_err(); + + assert_eq!(error.operation, "select_resource_scope"); + assert!(error + .corrective_message + .contains("entire excluded workspace")); + assert!(!destination.exists(), "a rejected graph is never published"); + } + #[cfg(unix)] #[test] fn copied_skill_executes_sibling_resources_after_two_stages() { diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 154364753..01e2bb905 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -2166,6 +2166,7 @@ mod configured_resource_tests { let local_package = workspace.join("packages/local-package"); let local_package_link = external.join("node_modules/local-package"); let server = tools.join("server.mjs"); + let workspace_worker = tools.join("workspace-worker.mjs"); let schema = external.join("schema.json"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); @@ -2174,6 +2175,11 @@ mod configured_resource_tests { std::fs::create_dir_all(local_package_link.parent().unwrap()).unwrap(); std::os::unix::fs::symlink("../project/packages/local-package", &local_package_link) .unwrap(); + std::os::unix::fs::symlink( + "../project/packages/local-package/worker.mjs", + &workspace_worker, + ) + .unwrap(); std::fs::write( workspace.join("workspace-data.txt"), "workspace stays mounted\n", @@ -2189,10 +2195,20 @@ mod configured_resource_tests { "export const marker = 'workspace-linked-package';\n", ) .unwrap(); + std::fs::write( + local_package.join("schema.json"), + r#"{"nonce":"workspace-script-sibling-loaded"}"#, + ) + .unwrap(); + std::fs::write( + local_package.join("worker.mjs"), + "import { readFileSync } from 'node:fs';\nconst schema = JSON.parse(readFileSync(new URL('./schema.json', import.meta.url), 'utf8'));\nexport const sibling = schema.nonce;\n", + ) + .unwrap(); std::fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); std::fs::write( &server, - "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nimport { marker } from 'local-package';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(marker + ':' + readFileSync(path.join(scriptDir, '../schema.json'), 'utf8'));\n", + "import { readFileSync } from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nimport { marker } from 'local-package';\nimport { sibling } from './workspace-worker.mjs';\nconst scriptDir = path.dirname(fileURLToPath(import.meta.url));\nprocess.stdout.write(marker + ':' + sibling + ':' + readFileSync(path.join(scriptDir, '../schema.json'), 'utf8'));\n", ) .unwrap(); std::fs::write(&schema, "computed sibling schema loaded\n").unwrap(); @@ -2275,7 +2291,7 @@ mod configured_resource_tests { assert!(output.status.success(), "{:?}", output.stderr); assert_eq!( output.stdout, - b"workspace-linked-package:computed sibling schema loaded\n" + b"workspace-linked-package:workspace-script-sibling-loaded:computed sibling schema loaded\n" ); } diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 2b283dd03..02bf86fd3 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -4613,9 +4613,25 @@ mod tests { let unit = user_root.join(".freshell/resource-units/root-0"); let provider_home = owned.path().join("home"); let provider_resource = provider_home.join(".freshell/provider-roots/root-0"); + let linked_package = unit.join("project/packages/worker-package"); std::fs::create_dir_all(unit.join("tools")).unwrap(); - std::fs::create_dir_all(unit.join("project")).unwrap(); - std::fs::write(unit.join("tools/server.mjs"), "export {};\n").unwrap(); + std::fs::create_dir_all(&linked_package).unwrap(); + std::os::unix::fs::symlink( + "../project/packages/worker-package/server.mjs", + unit.join("tools/server.mjs"), + ) + .unwrap(); + std::fs::write(linked_package.join("package.json"), r#"{"type":"module"}"#).unwrap(); + std::fs::write( + linked_package.join("schema.json"), + r#"{"nonce":"session-host-linked-sibling-loaded"}"#, + ) + .unwrap(); + std::fs::write( + linked_package.join("server.mjs"), + "import { readFileSync } from 'node:fs';\nconst schema = JSON.parse(readFileSync(new URL('./schema.json', import.meta.url), 'utf8'));\nprocess.stdout.write(schema.nonce + '\\n');\n", + ) + .unwrap(); std::fs::write(unit.join("schema.json"), "package asset\n").unwrap(); std::fs::write( unit.join("project/workspace-data.txt"), @@ -4680,6 +4696,18 @@ mod tests { !provider_resource.join("project").exists(), "session-host projection must not copy workspace data from a stale package unit" ); + + let hidden = owned.path().join("source-hidden"); + std::fs::rename(&user_root, &hidden).unwrap(); + let output = std::process::Command::new("node") + .arg(provider_resource.join("tools/server.mjs")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden, &user_root).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"session-host-linked-sibling-loaded\n"); } #[test] From 2c4ef94449c6c763d7c25de08506f1274fe82843 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:57:23 -0700 Subject: [PATCH 401/493] fix(managed): reject unbounded workspace-root scripts --- .../src/provider_environment.rs | 100 ++++++++++++++++-- 1 file changed, 93 insertions(+), 7 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index cbeda1e02..9dc75e549 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -356,15 +356,16 @@ impl Plan { } let metadata = fs::metadata(target).map_err(|cause| io_error(path, "inspect_link_target", cause))?; + let is_script = matches!( + target.extension().and_then(|ext| ext.to_str()), + Some("js" | "mjs" | "cjs" | "ts" | "tsx") + ); let mut resource = target.to_owned(); if metadata.is_file() { let parent = target.parent().ok_or_else(|| { error(path, "plan", "linked resource has no containing directory") })?; - if matches!( - target.extension().and_then(|ext| ext.to_str()), - Some("js" | "mjs" | "cjs" | "ts" | "tsx") - ) { + if is_script { resource = parent.to_owned(); } for ancestor in parent.ancestors() { @@ -397,6 +398,16 @@ impl Plan { )); } if resource == *excluded_subtree || !resource.starts_with(excluded_subtree) { + if metadata.is_file() + && is_script + && target.parent() == Some(excluded_subtree.as_path()) + { + return Err(error( + path, + "select_resource_scope", + "a script linked directly from the excluded workspace root has no bounded sibling-resource scope; place the script and its required files in a package directory or select a smaller resource", + )); + } resource = target .parent() .filter(|parent| *parent != excluded_subtree) @@ -1319,6 +1330,34 @@ mod tests { } } + #[cfg(unix)] + fn execute_fixture_with_transient_text_file_busy_retry( + path: &Path, + ) -> std::io::Result<std::process::Output> { + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500); + loop { + match Command::new(path).output() { + Err(error) + if error.raw_os_error() == Some(libc::ETXTBSY) + && std::time::Instant::now() < deadline => + { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + Ok(output) + if !output.status.success() + && output + .stderr + .windows(b"Text file busy".len()) + .any(|window| window == b"Text file busy") + && std::time::Instant::now() < deadline => + { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + result => return result, + } + } + } + #[test] fn failed_stage_creation_never_removes_a_preexisting_directory() { let root = tempfile::tempdir().unwrap(); @@ -1510,6 +1549,52 @@ mod tests { assert_eq!(output.stdout, b"linked-file-sibling-loaded\n"); } + #[cfg(unix)] + #[test] + fn excluded_workspace_root_file_alias_requires_a_bounded_resource_scope() { + let root = tempfile::tempdir().unwrap(); + let external = root.path().join("external"); + let workspace = external.join("project"); + let tools = external.join("tools"); + fs::create_dir_all(&workspace).unwrap(); + fs::create_dir_all(&tools).unwrap(); + fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write( + workspace.join("schema.json"), + r#"{"nonce":"workspace-root-sibling-loaded"}"#, + ) + .unwrap(); + fs::write( + workspace.join("worker.mjs"), + "import { readFileSync } from 'node:fs';\nconst schema = JSON.parse(readFileSync(new URL('./schema.json', import.meta.url), 'utf8'));\nexport const nonce = schema.nonce;\n", + ) + .unwrap(); + symlink("../project/worker.mjs", tools.join("worker-link.mjs")).unwrap(); + fs::write( + tools.join("server.mjs"), + "import { nonce } from './worker-link.mjs';\nprocess.stdout.write(nonce + '\\n');\n", + ) + .unwrap(); + + let destination = root.path().join("provider"); + let error = copy_environment_excluding_subtree( + &[selection(&external, "resource")], + &destination, + &[], + &workspace, + CopyLimits::default(), + ) + .unwrap_err(); + assert_eq!(error.operation, "select_resource_scope"); + assert!(error + .corrective_message + .contains("bounded sibling-resource scope")); + assert!( + !destination.exists(), + "an ambiguous workspace-root resource is rejected before publication" + ); + } + #[cfg(unix)] #[test] fn excluded_workspace_root_alias_fails_without_copying_the_workspace() { @@ -1569,9 +1654,10 @@ mod tests { CopyLimits::default(), ) .unwrap(); - let output = Command::new(home.join("skills/fixture/scripts/run")) - .output() - .unwrap(); + let output = execute_fixture_with_transient_text_file_busy_retry( + &home.join("skills/fixture/scripts/run"), + ) + .unwrap(); assert!(output.status.success(), "{output:?}"); assert_eq!(output.stdout, b"resource-nonce\nasset-nonce\n"); assert_eq!( From 444fb1e9414a19abf7536788bac4d5a782debd27 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:24:19 -0700 Subject: [PATCH 402/493] fix(managed): preserve separately mounted workspace aliases --- .../src/provider_environment.rs | 168 +++++++++++++++--- .../src/managed_mcp_capability.rs | 16 +- .../managed_mcp_capability/source_stage.rs | 148 ++++++++++++++- crates/freshell-session-host/src/main.rs | 127 ++++++++++--- 4 files changed, 409 insertions(+), 50 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index 9dc75e549..b6a2d3702 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -119,6 +119,7 @@ enum Node { Directory(PathBuf), File(PathBuf, u32, u64), Link(PathBuf), + MountedLink(PathBuf), } struct Plan { nodes: BTreeMap<PathBuf, Node>, @@ -174,6 +175,17 @@ impl Plan { destination: PathBuf, depth: usize, ) -> Result<(), CopyError> { + if self.is_alias_into_excluded_subtree(&source) + && self + .excluded_subtree + .as_deref() + .is_some_and(|excluded| source == excluded || source.parent() == Some(excluded)) + { + // The managed runtime mounts the workspace at its original absolute + // path. Keep root aliases pointed at that mount instead of + // materializing an unbounded workspace as a package dependency. + return self.claim(destination, Node::MountedLink(source)); + } let mapped = self.external_resource(&source, &destination, depth)?; self.claim(destination, Node::Link(mapped)) } @@ -261,9 +273,10 @@ impl Plan { .ok_or_else(|| error(&path, "plan", "entry accounting overflow"))?; bound(&path, "entries", entries, self.limits.max_entries as u64)?; let source_bytes = match &node { - Node::Directory(source) | Node::File(source, ..) | Node::Link(source) => { - source.as_os_str().len() as u64 - } + Node::Directory(source) + | Node::File(source, ..) + | Node::Link(source) + | Node::MountedLink(source) => source.as_os_str().len() as u64, }; self.metadata_bytes = self .metadata_bytes @@ -689,14 +702,37 @@ fn copy_environment_with_account_home( } /// Copies a selected resource while omitting one nested tree. Symlink aliases -/// inside the selected resource that target that tree are materialized at the -/// alias path, subject to the same copy limits and every other exclusion. +/// into bounded subresources are materialized at the alias path; aliases to the +/// excluded tree root or one of its top-level files remain absolute links to its +/// separately mounted workspace. A child selected directly remains bounded to +/// that child. All other exclusions still apply. pub fn copy_environment_excluding_subtree( selections: &[CopySelection], destination_root: &Path, excluded_canonical_sources: &[PathBuf], excluded_subtree: &Path, limits: CopyLimits, +) -> Result<CopyReport, CopyError> { + copy_environment_excluding_subtree_for_mounts( + selections, + destination_root, + excluded_canonical_sources, + excluded_subtree, + limits, + &[], + ) +} + +/// Exclusion-aware projection for a runtime with separately mounted resource +/// roots. Aliases at the excluded workspace root remain absolute links to that +/// mount, while mounted roots retain their existing publication behavior. +pub fn copy_environment_excluding_subtree_for_mounts( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + excluded_subtree: &Path, + limits: CopyLimits, + mounted_roots: &[PathBuf], ) -> Result<CopyReport, CopyError> { let excluded_subtree = fs::canonicalize(excluded_subtree).map_err(|cause| { io_error( @@ -716,7 +752,7 @@ pub fn copy_environment_excluding_subtree( &exclusions, limits, account_home, - &[], + mounted_roots, Some(excluded_subtree), ) } @@ -809,6 +845,16 @@ fn copy_environment_projection( .map_err(|cause| io_error(path, "select", cause))?; let canonical = fs::canonicalize(&selection.source) .map_err(|cause| io_error(path, "resolve_selection", cause))?; + if plan + .excluded_subtree + .as_deref() + .is_some_and(|excluded| canonical != excluded && canonical.starts_with(excluded)) + { + // A specifically selected child remains an independent bounded + // resource even when its containing workspace is separately + // mounted and excluded from enclosing package resources. + plan.include_selected_subtree_resource(&canonical); + } if plan.excluded(&canonical) { return Err(error(path,"select","selected resource is excluded native state or a credential source; keep native state in its store and use managed references for credentials")); } @@ -974,6 +1020,17 @@ fn copy_environment_projection( "symbolic resource projection requires Unix", )); } + Node::MountedLink(target) => { + #[cfg(unix)] + std::os::unix::fs::symlink(target, &destination) + .map_err(|cause| io_error(relative, "create_link", cause))?; + #[cfg(not(unix))] + return Err(error( + relative, + "create_link", + "symbolic resource projection requires Unix", + )); + } } } if !existing_root { @@ -1551,7 +1608,7 @@ mod tests { #[cfg(unix)] #[test] - fn excluded_workspace_root_file_alias_requires_a_bounded_resource_scope() { + fn excluded_workspace_root_file_alias_uses_the_separately_mounted_workspace() { let root = tempfile::tempdir().unwrap(); let external = root.path().join("external"); let workspace = external.join("project"); @@ -1577,50 +1634,115 @@ mod tests { .unwrap(); let destination = root.path().join("provider"); - let error = copy_environment_excluding_subtree( + copy_environment_excluding_subtree( &[selection(&external, "resource")], &destination, &[], &workspace, CopyLimits::default(), ) - .unwrap_err(); - assert_eq!(error.operation, "select_resource_scope"); - assert!(error - .corrective_message - .contains("bounded sibling-resource scope")); + .unwrap(); + let copied_root = destination.join("resource"); + assert_eq!( + fs::canonicalize(copied_root.join("tools/worker-link.mjs")).unwrap(), + workspace.join("worker.mjs").canonicalize().unwrap(), + "the linked script resolves through the separately mounted workspace" + ); assert!( - !destination.exists(), - "an ambiguous workspace-root resource is rejected before publication" + !copied_root.join("project").exists(), + "the workspace is not copied under the package resource" ); + let output = Command::new("node") + .arg(copied_root.join("tools/server.mjs")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output() + .unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"workspace-root-sibling-loaded\n"); } #[cfg(unix)] #[test] - fn excluded_workspace_root_alias_fails_without_copying_the_workspace() { + fn excluded_workspace_root_alias_uses_the_separately_mounted_workspace() { let root = tempfile::tempdir().unwrap(); let external = root.path().join("external"); let workspace = external.join("project"); fs::create_dir_all(workspace.join("packages")).unwrap(); fs::create_dir_all(external.join("node_modules")).unwrap(); fs::write(workspace.join("private-project-data"), "do not copy").unwrap(); + fs::write( + workspace.join("index.mjs"), + "export const nonce = 'workspace-root-package-loaded';\n", + ) + .unwrap(); symlink("../project", external.join("node_modules/workspace")).unwrap(); + fs::write(external.join("package.json"), r#"{"type":"module"}"#).unwrap(); + fs::write( + external.join("server.mjs"), + "import { nonce } from './node_modules/workspace/index.mjs';\nprocess.stdout.write(nonce + '\\n');\n", + ) + .unwrap(); let destination = root.path().join("provider"); - let error = copy_environment_excluding_subtree( + copy_environment_excluding_subtree( &[selection(&external, "resource")], &destination, &[], &workspace, CopyLimits::default(), ) - .unwrap_err(); + .unwrap(); + + let copied_root = destination.join("resource"); + assert_eq!( + fs::canonicalize(copied_root.join("node_modules/workspace")).unwrap(), + workspace.canonicalize().unwrap(), + "the local package alias must resolve to the separately mounted workspace" + ); + assert!( + !copied_root.join("project/private-project-data").exists(), + "the excluded workspace is not copied as a package dependency" + ); + let output = Command::new("node") + .arg(copied_root.join("server.mjs")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output() + .unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"workspace-root-package-loaded\n"); + } - assert_eq!(error.operation, "select_resource_scope"); - assert!(error - .corrective_message - .contains("entire excluded workspace")); - assert!(!destination.exists(), "a rejected graph is never published"); + #[test] + fn excluded_workspace_keeps_explicitly_selected_child_resources() { + let root = tempfile::tempdir().unwrap(); + let workspace = root.path().join("workspace"); + fs::create_dir_all(&workspace).unwrap(); + fs::write(workspace.join("selected-config.json"), "selected config").unwrap(); + fs::write(workspace.join("unselected-workspace-data"), "keep out").unwrap(); + let destination = root.path().join("provider"); + + copy_environment_excluding_subtree( + &[selection( + &workspace.join("selected-config.json"), + "config.json", + )], + &destination, + &[], + &workspace, + CopyLimits::default(), + ) + .unwrap(); + + assert_eq!( + fs::read_to_string(destination.join("config.json")).unwrap(), + "selected config" + ); + assert!( + !destination.join("unselected-workspace-data").exists(), + "an explicitly selected child does not widen into its workspace" + ); } #[cfg(unix)] diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 171d83f87..5ff0b7628 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1977,7 +1977,21 @@ fn stage_provider_root( provider, selections = selections.len() ); - let report = copy_environment(&selections, &temporary, &excluded, CopyLimits::default()) + let copy_result = if provider == "opencode" { + freshell_platform::provider_environment::copy_environment_excluding_subtree( + &selections, + &temporary, + &excluded, + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + CopyLimits::default(), + ) + } else { + copy_environment(&selections, &temporary, &excluded, CopyLimits::default()) + }; + let report = copy_result .map_err(|error| { let failing_selection = selections.iter().find(|selection| { error diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 01e2bb905..9152d5111 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -2295,6 +2295,99 @@ mod configured_resource_tests { ); } + #[cfg(unix)] + #[test] + fn controller_filters_workspace_reached_through_package_symlink() { + let owned = tempfile::tempdir().unwrap(); + let package = owned.path().join("external"); + let workspace = owned.path().join("checkout/project"); + let filtered = tempfile::tempdir_in(owned.path()).unwrap(); + std::fs::create_dir_all(package.join("node_modules")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + workspace.join("index.mjs"), + "export const nonce = 'separately-mounted-workspace-loaded';\n", + ) + .unwrap(); + std::fs::write( + workspace.join("workspace-data.txt"), + "private workspace data\n", + ) + .unwrap(); + std::os::unix::fs::symlink(&workspace, package.join("project")).unwrap(); + std::os::unix::fs::symlink("../project", package.join("node_modules/project")).unwrap(); + std::fs::write(package.join("package.json"), r#"{"type":"module"}"#).unwrap(); + std::fs::write( + package.join("server.mjs"), + "import { nonce } from './node_modules/project/index.mjs';\nprocess.stdout.write(nonce + '\\n');\n", + ) + .unwrap(); + let mount = freshell_runtime_protocol::ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-symlink".into(), + provider_relative_path: ".freshell/provider-roots/root-symlink".into(), + logical_destination: package.to_string_lossy().into_owned(), + effective_destination: package.to_string_lossy().into_owned(), + kind: freshell_runtime_protocol::ProviderResourceKind::Directory, + }; + let mut units = vec![(package.clone(), mount)]; + + filter_workspace_from_enclosing_resource_units( + &mut units, + "opencode", + &workspace, + filtered.path(), + &[], + ) + .unwrap(); + + assert_ne!(units[0].0, package, "the symlinked workspace is filtered"); + let filtered_package = &units[0].0; + assert!(std::fs::symlink_metadata(filtered_package.join("project")) + .unwrap() + .file_type() + .is_symlink()); + assert_eq!( + std::fs::canonicalize(filtered_package.join("project")).unwrap(), + workspace.canonicalize().unwrap(), + "the separately mounted workspace remains reachable through its alias" + ); + let output = std::process::Command::new("node") + .arg(filtered_package.join("server.mjs")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output() + .unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"separately-mounted-workspace-loaded\n"); + + let final_stage = filtered.path().join("managed-stage"); + freshell_platform::provider_environment::copy_environment_excluding_subtree( + &[freshell_platform::provider_environment::CopySelection { + source: filtered_package.clone(), + destination_relative: "resource".into(), + }], + &final_stage, + &[], + &workspace, + Default::default(), + ) + .unwrap(); + let final_package = final_stage.join("resource"); + assert_eq!( + std::fs::canonicalize(final_package.join("project")).unwrap(), + workspace.canonicalize().unwrap(), + "the final controller stage preserves the separate workspace link" + ); + let output = std::process::Command::new("node") + .arg(final_package.join("server.mjs")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output() + .unwrap(); + assert!(output.status.success(), "{output:?}"); + assert_eq!(output.stdout, b"separately-mounted-workspace-loaded\n"); + } + #[test] fn filtering_shared_home_and_workspace_resource_keeps_one_copy_source() { let owned = tempfile::tempdir().unwrap(); @@ -3395,9 +3488,12 @@ pub(super) fn filter_workspace_from_enclosing_resource_units( } let canonical_source = std::fs::canonicalize(&*source) .map_err(|_| "selected OpenCode resource stage source is unavailable")?; - if canonical_workspace == canonical_source - || !canonical_workspace.starts_with(&canonical_source) - { + if canonical_workspace == canonical_source { + continue; + } + let contains_workspace = canonical_workspace.starts_with(&canonical_source) + || resource_contains_workspace_alias(&canonical_source, &canonical_workspace)?; + if !contains_workspace { continue; } if let Some(filtered_source) = filtered_sources.get(&canonical_source) { @@ -3475,6 +3571,52 @@ pub(super) fn filter_workspace_from_enclosing_resource_units( Ok(()) } +#[cfg(unix)] +fn resource_contains_workspace_alias(source: &Path, workspace: &Path) -> Result<bool, String> { + let limits = freshell_platform::provider_environment::CopyLimits::default(); + let mut pending = vec![(source.to_owned(), 0usize)]; + let mut entries = 0usize; + while let Some((directory, depth)) = pending.pop() { + if depth > limits.max_depth { + return Err("selected OpenCode resource exceeds the directory depth limit while checking workspace aliases".into()); + } + let children = std::fs::read_dir(&directory).map_err(|_| { + "selected OpenCode resource could not be inspected for workspace aliases" + })?; + for child in children { + let child = child.map_err(|_| { + "selected OpenCode resource could not be inspected for workspace aliases" + })?; + entries = entries + .checked_add(1) + .ok_or("selected OpenCode resource entry accounting overflow")?; + if entries > limits.max_entries { + return Err("selected OpenCode resource exceeds the entry limit while checking workspace aliases".into()); + } + let path = child.path(); + let file_type = child.file_type().map_err(|_| { + "selected OpenCode resource could not be inspected for workspace aliases" + })?; + if file_type.is_symlink() { + let target = std::fs::canonicalize(&path).map_err(|_| { + "selected OpenCode resource contains a symbolic link that cannot be resolved" + })?; + if target == workspace || target.starts_with(workspace) { + return Ok(true); + } + } else if file_type.is_dir() { + pending.push((path, depth + 1)); + } + } + } + Ok(false) +} + +#[cfg(not(unix))] +fn resource_contains_workspace_alias(_source: &Path, _workspace: &Path) -> Result<bool, String> { + Ok(false) +} + pub(super) fn current_source_for_resource_mount( current: &[( std::path::PathBuf, diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 02bf86fd3..24617467e 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1823,19 +1823,62 @@ fn copy_environment_with_events_for_mounts( freshell_platform::provider_environment::CopyReport, freshell_platform::provider_environment::CopyError, > { + copy_environment_with_events_using(stage, provider, selections, || { + freshell_platform::provider_environment::copy_environment_for_mounts( + selections, + destination, + excluded, + Default::default(), + mounted_directories, + ) + }) +} + +fn copy_environment_with_events_excluding_subtree_for_mounts( + stage: &str, + provider: &str, + selections: &[freshell_platform::provider_environment::CopySelection], + destination: &Path, + excluded: &[PathBuf], + excluded_subtree: &Path, + mounted_directories: &[PathBuf], +) -> Result< + freshell_platform::provider_environment::CopyReport, + freshell_platform::provider_environment::CopyError, +> { + copy_environment_with_events_using(stage, provider, selections, || { + freshell_platform::provider_environment::copy_environment_excluding_subtree_for_mounts( + selections, + destination, + excluded, + excluded_subtree, + Default::default(), + mounted_directories, + ) + }) +} + +fn copy_environment_with_events_using<F>( + stage: &str, + provider: &str, + selections: &[freshell_platform::provider_environment::CopySelection], + copy: F, +) -> Result< + freshell_platform::provider_environment::CopyReport, + freshell_platform::provider_environment::CopyError, +> +where + F: FnOnce() -> Result< + freshell_platform::provider_environment::CopyReport, + freshell_platform::provider_environment::CopyError, + >, +{ let started = std::time::Instant::now(); emit_environment_event(serde_json::json!({ "severity": "info", "event": "provider.environment.copy.started", "stage": stage, "provider": provider, "selections": selections.len() })); - let report = freshell_platform::provider_environment::copy_environment_for_mounts( - selections, - destination, - excluded, - Default::default(), - mounted_directories, - ) - .inspect_err(|error| { + let report = copy().inspect_err(|error| { emit_environment_event(serde_json::json!({ "severity": "error", "event": "provider.environment.copy.failed", "stage": stage, "provider": provider, @@ -2052,19 +2095,47 @@ fn copy_provider_config_references_with_ownership( filtered_resource_sources.insert(canonical_source, selection.source.clone()); _filtered_resource_sources.push(scratch); } + for (resource_index, _) in context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .enumerate() + { + let resource_selection_index = config_selection_count + resource_index; + let selection = selections + .get_mut(resource_selection_index) + .ok_or("provider resource selection is incomplete")?; + if let Ok(canonical_source) = std::fs::canonicalize(&selection.source) { + if let Some(filtered_source) = filtered_resource_sources.get(&canonical_source) { + selection.source = filtered_source.clone(); + } + } + } let mounted = if provider_home == final_home { resource_mount_roots(context) } else { Vec::new() }; - let report = copy_environment_with_events_for_mounts( - "session_host", - provider, - &selections, - provider_home, - &excluded, - &mounted, - ) + let report = if provider == "opencode" { + copy_environment_with_events_excluding_subtree_for_mounts( + "session_host", + provider, + &selections, + provider_home, + &excluded, + workspace, + &mounted, + ) + } else { + copy_environment_with_events_for_mounts( + "session_host", + provider, + &selections, + provider_home, + &excluded, + &mounted, + ) + } .map_err(|error| error.to_string())?; let projected = report .projected_paths @@ -4614,6 +4685,7 @@ mod tests { let provider_home = owned.path().join("home"); let provider_resource = provider_home.join(".freshell/provider-roots/root-0"); let linked_package = unit.join("project/packages/worker-package"); + std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(unit.join("tools")).unwrap(); std::fs::create_dir_all(&linked_package).unwrap(); std::os::unix::fs::symlink( @@ -4664,13 +4736,22 @@ mod tests { }, config_binds: vec![], indexed_files: None, - resource_mounts: vec![ProviderResourceMount { - stage_relative_path: ".freshell/resource-units/root-0".into(), - provider_relative_path: ".freshell/provider-roots/root-0".into(), - logical_destination: package.to_string_lossy().into_owned(), - effective_destination: package.to_string_lossy().into_owned(), - kind: ProviderResourceKind::Directory, - }], + resource_mounts: vec![ + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".freshell/provider-roots/root-0".into(), + logical_destination: package.to_string_lossy().into_owned(), + effective_destination: package.to_string_lossy().into_owned(), + kind: ProviderResourceKind::Directory, + }, + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-0".into(), + provider_relative_path: ".freshell/provider-roots/root-0".into(), + logical_destination: "/home/freshell/.config/opencode".into(), + effective_destination: "/home/freshell/provider/.config/opencode".into(), + kind: ProviderResourceKind::Directory, + }, + ], }), mcp_capability: Some(capability), }; From 0e8838b4183e4dafe6215732531f3e9f6aeaad62 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:43:58 -0700 Subject: [PATCH 403/493] fix(managed): stage direct OpenCode MCP command resources --- .../managed_mcp_capability/source_stage.rs | 128 +++++++++++++++ .../opencode_resources.rs | 154 ++++++++++++++++-- 2 files changed, 268 insertions(+), 14 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 9152d5111..ae1fed067 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -826,6 +826,61 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"absolute MCP executable started\n"); } + #[test] + fn absolute_mcp_symlink_command_preserves_launch_alias_after_resource_copies() { + use std::os::unix::fs::{symlink, PermissionsExt}; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.sh"); + let target = home.join("tools/start.sh"); + std::fs::create_dir_all(target.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(&target, "#!/bin/sh\nprintf 'symlink command started\\n'\n").unwrap(); + std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755)).unwrap(); + symlink("tools/start.sh", &command).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!( + plan[0].configured_resources.contains(&command), + "the absolute command alias must remain available at the configured launch path" + ); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/mcp.sh"); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"symlink command started\n"); + } + #[test] fn direct_executable_script_in_home_runs_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; @@ -1059,6 +1114,42 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"node sibling helper loaded\n"); } + #[test] + fn direct_node_script_in_home_keeps_path_join_data_after_all_resource_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let schema = home.join("schema.json"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nconst fs = require('node:fs');\nconst path = require('node:path');\nprocess.stdout.write(fs.readFileSync(path.join(__dirname, 'schema.json'), 'utf8'));\n", + ) + .unwrap(); + std::fs::write(&schema, "path.join schema loaded\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_script) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!( + plan[0] + .configured_resources + .contains(&schema.canonicalize().unwrap()), + "a statically named sibling read with path.join(__dirname, ...) must be selected" + ); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"path.join schema loaded\n"); + } + #[test] fn direct_node_script_in_home_keeps_home_node_modules_after_all_copies() { use std::os::unix::fs::PermissionsExt; @@ -1229,6 +1320,43 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"sibling-loaded:package-loaded\n"); } + #[test] + fn home_shell_wrapper_stages_node_script_after_changing_to_its_directory() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("start"); + let server = home.join("server.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\ncd \"$(dirname \"$0\")\"; exec node server.js\n", + ) + .unwrap(); + std::fs::write( + &server, + "process.stdout.write('cd-relative server started\\n');\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!( + plan[0] + .configured_resources + .contains(&server.canonicalize().unwrap()), + "a Node file run after cd to the wrapper directory must be selected" + ); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"cd-relative server started\n"); + } + #[test] fn node_command_script_in_home_keeps_home_node_modules_after_all_copies() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 08b84f943..afa7adb95 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -661,6 +661,72 @@ fn shell_directory_variables(source: &str) -> std::collections::BTreeSet<String> .collect() } +fn shell_changes_to_script_directory(line: &str) -> Option<&str> { + const CD_TO_SCRIPT_DIRECTORY: &str = "cd \"$(dirname \"$0\")\""; + let line = line.trim_start(); + let remainder = line.strip_prefix(CD_TO_SCRIPT_DIRECTORY)?; + if remainder.is_empty() { + return Some(""); + } + remainder + .trim_start() + .strip_prefix(';') + .map(str::trim_start) +} + +fn shell_executable_name(token: &str) -> String { + let token = token.trim_matches(['"', '\'']); + let name = Path::new(token) + .file_name() + .unwrap_or_default() + .to_string_lossy() + .to_ascii_lowercase(); + name.strip_suffix(".exe").unwrap_or(&name).to_owned() +} + +fn shell_node_script_operand(line: &str) -> Option<String> { + let words = line + .split_ascii_whitespace() + .map(|word| word.trim_matches(['"', '\''])) + .collect::<Vec<_>>(); + let node_index = words + .iter() + .position(|word| matches!(shell_executable_name(word).as_str(), "node" | "nodejs"))?; + let mut index = node_index + 1; + const NODE_OPTIONS_WITH_VALUE: [&str; 8] = [ + "-r", + "--require", + "--import", + "--loader", + "--experimental-loader", + "--conditions", + "--env-file", + "--inspect-port", + ]; + while let Some(argument) = words.get(index) { + if *argument == "--" { + index += 1; + break; + } + if NODE_OPTIONS_WITH_VALUE.contains(argument) { + index += 2; + continue; + } + if argument.starts_with('-') { + index += 1; + continue; + } + break; + } + let argument = words.get(index)?.trim_end_matches([';', '&']); + let path = Path::new(argument); + (!path.is_absolute() + && !argument.is_empty() + && !argument.contains(['$', '*', '?']) + && is_node_script_path(argument)) + .then(|| argument.to_owned()) +} + fn shell_source_operand(line: &str) -> Option<String> { let line = line.trim_start(); let rest = if let Some(rest) = line.strip_prefix("source") { @@ -764,16 +830,6 @@ fn shell_directory_relative_paths( } fn shell_line_invokes_node(line: &str) -> bool { - fn executable_name(token: &str) -> String { - let token = token.trim_matches(['"', '\'']); - let name = Path::new(token) - .file_name() - .unwrap_or_default() - .to_string_lossy() - .to_ascii_lowercase(); - name.strip_suffix(".exe").unwrap_or(&name).to_owned() - } - let words = line .split_ascii_whitespace() .map(|word| word.trim_matches(['"', '\''])) @@ -789,7 +845,7 @@ fn shell_line_invokes_node(line: &str) -> bool { let Some(executable) = words.get(index) else { return false; }; - match executable_name(executable).as_str() { + match shell_executable_name(executable).as_str() { "node" | "nodejs" => return true, "env" => index += 1, _ => return false, @@ -812,7 +868,7 @@ fn shell_line_invokes_node(line: &str) -> bool { index += 1; continue; } - return matches!(executable_name(argument).as_str(), "node" | "nodejs"); + return matches!(shell_executable_name(argument).as_str(), "node" | "nodejs"); } false } @@ -934,7 +990,44 @@ fn node_relative_resource_specifiers( ) -> Result<std::collections::BTreeSet<String>, &'static str> { let tokens = javascript_tokens(source)?; let mut result = std::collections::BTreeSet::new(); - for index in 0..tokens.len().saturating_sub(10) { + for index in 0..tokens.len() { + if index >= 2 + && matches!(tokens.get(index - 2), Some(JavaScriptToken::Identifier(_))) + && matches!( + tokens.get(index - 1), + Some(JavaScriptToken::Punctuation(b'.')) + ) + && matches!(tokens.get(index), Some(JavaScriptToken::Identifier(name)) if name == "join") + && matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'(')) + ) + && matches!(tokens.get(index + 2), Some(JavaScriptToken::Identifier(name)) if name == "__dirname") + && matches!( + tokens.get(index + 3), + Some(JavaScriptToken::Punctuation(b',')) + ) + { + if let Some(JavaScriptToken::String(Some(specifier))) = tokens.get(index + 4) { + let is_closed = matches!( + tokens.get(index + 5), + Some(JavaScriptToken::Punctuation(b')')) + ) || (matches!( + tokens.get(index + 5), + Some(JavaScriptToken::Punctuation(b',')) + ) && matches!( + tokens.get(index + 6), + Some(JavaScriptToken::Punctuation(b')')) + )); + if is_closed + && !specifier.is_empty() + && !Path::new(specifier).is_absolute() + && !specifier.contains(['?', '#', '\\', '$', '*']) + { + result.insert(specifier.clone()); + } + } + } let Some(JavaScriptToken::Identifier(new_keyword)) = tokens.get(index) else { continue; }; @@ -1468,10 +1561,20 @@ fn insert_shell_script_dependencies( let script_parent = canonical .parent() .ok_or_else(|| fail("direct MCP shell script has no containing directory"))?; + let mut working_directory_is_script_parent = false; for line in source .lines() .filter(|line| !line.trim_start().starts_with('#')) { + let node_line = if let Some(remainder) = shell_changes_to_script_directory(line) { + working_directory_is_script_parent = true; + remainder + } else { + if line.trim_start().starts_with("cd ") { + working_directory_is_script_parent = false; + } + line + }; let mut candidates = std::collections::BTreeMap::new(); if let Some(operand) = shell_source_operand(line) { if let Some(candidate) = @@ -1481,7 +1584,7 @@ fn insert_shell_script_dependencies( candidates.insert(candidate, (true, false)); } } - let invokes_node = shell_line_invokes_node(line); + let invokes_node = shell_line_invokes_node(node_line); for relative in shell_directory_relative_paths(line, &directory_variables) { let parse_as_node_script = invokes_node && is_node_script_path(&relative); candidates @@ -1489,6 +1592,14 @@ fn insert_shell_script_dependencies( .and_modify(|kind: &mut (bool, bool)| kind.1 |= parse_as_node_script) .or_insert((false, parse_as_node_script)); } + if working_directory_is_script_parent && invokes_node { + if let Some(relative) = shell_node_script_operand(node_line) { + candidates + .entry(script_parent.join(relative)) + .and_modify(|kind: &mut (bool, bool)| kind.1 = true) + .or_insert((false, true)); + } + } for (candidate, (parse_as_script, parse_as_node_script)) in candidates { let dependency = checked(&candidate, field, declaring)?; if !dependency.is_file() { @@ -2104,6 +2215,21 @@ pub fn select( { continue; } + if path != canonical + && std::fs::symlink_metadata(&path) + .is_ok_and(|metadata| metadata.file_type().is_symlink()) + { + // The provider must have both the resolved target used for + // dependency discovery and the original command alias used + // by OpenCode's launch argv. + insert_resource( + &mut result, + &mut resource_path_bytes, + path.clone(), + field, + declaring, + )?; + } let command_parent = canonical .parent() .ok_or_else(|| fail(field, "command has no containing scope"))?; From e75946239d2ef56d0d6c9ba5ff477ec57066a6b8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:56:01 -0700 Subject: [PATCH 404/493] fix(managed): complete Node MCP script dependency staging --- .../managed_mcp_capability/source_stage.rs | 158 ++++++++++++++++++ .../opencode_resources.rs | 87 +++++++--- 2 files changed, 219 insertions(+), 26 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index ae1fed067..9f2f3801b 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -198,6 +198,22 @@ mod configured_resource_tests { output.unwrap() } + fn execute_node_with_home_hidden( + owned_root: &Path, + home: &Path, + provider_script: &Path, + ) -> std::process::Output { + let hidden_home = owned_root.join("home-hidden"); + std::fs::rename(home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .arg(provider_script) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden_home, home).unwrap(); + output.unwrap() + } + #[test] fn managed_provider_keeps_external_indexed_payloads_out_of_persistent_generation() { fn tree_contains_bytes(path: &Path, expected: &[u8]) -> bool { @@ -881,6 +897,65 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"symlink command started\n"); } + #[test] + fn mcp_cwd_does_not_hide_absolute_command_symlink_alias() { + use std::os::unix::fs::{symlink, PermissionsExt}; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let project = workspace.join("project"); + let command = home.join("mcp.sh"); + let target = project.join("start.sh"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write( + &target, + "#!/bin/sh\nprintf 'cwd symlink command started\\n'\n", + ) + .unwrap(); + std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755)).unwrap(); + symlink("../workspace/project/start.sh", &command).unwrap(); + let config = workspace.join("opencode.json"); + let raw = format!( + r#"{{"mcp":{{"direct":{{"type":"local","cwd":{},"command":[{}]}}}}}}"#, + serde_json::to_string(project.to_str().unwrap()).unwrap(), + serde_json::to_string(command.to_str().unwrap()).unwrap(), + ); + std::fs::write(&config, raw).unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + assert!( + plan[0].configured_resources.contains(&command), + "mcp.cwd may cover the resolved target but must not remove the configured HOME alias" + ); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_command = provider.join(".provider-home/mcp.sh"); + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"cwd symlink command started\n"); + } + #[test] fn direct_executable_script_in_home_runs_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; @@ -1150,6 +1225,89 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"path.join schema loaded\n"); } + #[test] + fn direct_node_script_in_nested_home_directory_keeps_relative_module_closure() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp/bin/server.js"); + let helper = home.join("mcp/lib/helper.js"); + let nested = home.join("mcp/lib/nested.js"); + std::fs::create_dir_all(command.parent().unwrap()).unwrap(); + std::fs::create_dir_all(helper.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "process.stdout.write(require('../lib/helper.js'))\n", + ) + .unwrap(); + std::fs::write(&helper, "module.exports = require('./nested.js')\n").unwrap(); + std::fs::write(&nested, "module.exports = 'nested module loaded\\n'\n").unwrap(); + + let command_args = vec!["node".to_owned(), command.to_string_lossy().into_owned()]; + let (plan, provider_script) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command_args, + &command, + ); + for required in [&helper, &nested] { + assert!( + plan[0] + .configured_resources + .contains(&required.canonicalize().unwrap()), + "the nested Node command closure omitted {}", + required.display() + ); + } + + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"nested module loaded\n"); + } + + #[test] + fn direct_node_script_recurses_into_extensionless_imported_module() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let helper = home.join("helper"); + let nested = home.join("nested.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(&command, "process.stdout.write(require('./helper'))\n").unwrap(); + std::fs::write(&helper, "module.exports = require('./nested.js')\n").unwrap(); + std::fs::write( + &nested, + "module.exports = 'extensionless module loaded\\n'\n", + ) + .unwrap(); + + let command_args = vec!["node".to_owned(), command.to_string_lossy().into_owned()]; + let (plan, provider_script) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command_args, + &command, + ); + for required in [&helper, &nested] { + assert!( + plan[0] + .configured_resources + .contains(&required.canonicalize().unwrap()), + "the Node scanner omitted {} from the imported closure", + required.display() + ); + } + + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"extensionless module loaded\n"); + } + #[test] fn direct_node_script_in_home_keeps_home_node_modules_after_all_copies() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index afa7adb95..ddb344983 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1391,35 +1391,64 @@ enum ScriptDependencyRuntime { Ruby, } +struct ScriptDependencyPath { + path: PathBuf, + parse_as_script: bool, +} + fn direct_script_dependency_paths( parent: &Path, source: &str, runtime: ScriptDependencyRuntime, -) -> Result<Vec<PathBuf>, &'static str> { +) -> Result<Vec<ScriptDependencyPath>, &'static str> { match runtime { ScriptDependencyRuntime::Node => { - let mut result = std::collections::BTreeSet::new(); - for specifier in node_relative_module_specifiers(source)? { - let files = resolve_node_relative_module(parent, &specifier)? - .ok_or("direct MCP Node relative module is missing")?; - result.extend(files); - } - result.extend( - node_relative_resource_specifiers(source)? - .into_iter() - .map(|specifier| parent.join(specifier)), - ); - Ok(result.into_iter().collect()) + let mut result = std::collections::BTreeMap::<PathBuf, bool>::new(); + { + let mut record = |path: PathBuf, imported_module: bool| { + let parse_as_script = is_script_dependency_source(&path, runtime) + || (imported_module && path.extension().is_none()); + result + .entry(path) + .and_modify(|selected| *selected |= parse_as_script) + .or_insert(parse_as_script); + }; + for specifier in node_relative_module_specifiers(source)? { + let files = resolve_node_relative_module(parent, &specifier)? + .ok_or("direct MCP Node relative module is missing")?; + for file in files { + record(file, true); + } + } + for specifier in node_relative_resource_specifiers(source)? { + record(parent.join(specifier), false); + } + } + Ok(result + .into_iter() + .map(|(path, parse_as_script)| ScriptDependencyPath { + path, + parse_as_script, + }) + .collect()) } ScriptDependencyRuntime::Python => Ok(python_import_specifiers(source) .into_iter() .flat_map(|specifier| resolve_python_local_modules(parent, &specifier)) + .map(|path| ScriptDependencyPath { + parse_as_script: is_script_dependency_source(&path, runtime), + path, + }) .collect()), ScriptDependencyRuntime::Ruby => ruby_relative_require_specifiers(source)? .into_iter() .map(|specifier| { - resolve_ruby_relative_module(parent, &specifier) - .ok_or("direct MCP Ruby relative module is missing") + let path = resolve_ruby_relative_module(parent, &specifier) + .ok_or("direct MCP Ruby relative module is missing")?; + Ok(ScriptDependencyPath { + parse_as_script: is_script_dependency_source(&path, runtime), + path, + }) }) .collect(), } @@ -1460,9 +1489,6 @@ fn insert_direct_script_dependencies( if !canonical.is_file() || !visited.insert(canonical.clone()) { continue; } - if current != script && !is_script_dependency_source(&canonical, runtime) { - continue; - } if !parsed.insert(canonical.clone()) { continue; } @@ -1483,7 +1509,7 @@ fn insert_direct_script_dependencies( .parent() .ok_or_else(|| fail("direct MCP script has no containing directory"))?; for candidate in direct_script_dependency_paths(parent, &source, runtime).map_err(fail)? { - let dependency = checked(&candidate, field, declaring)?; + let dependency = checked(&candidate.path, field, declaring)?; let dependency_is_file = dependency.is_file(); if !dependency_is_file && !dependency.is_dir() { return Err(fail( @@ -1502,7 +1528,7 @@ fn insert_direct_script_dependencies( field, declaring, )?; - if dependency_is_file { + if dependency_is_file && candidate.parse_as_script { pending.push(dependency); } } @@ -2209,12 +2235,6 @@ pub fn select( // besides being redundant, that shape cannot be authorized by the // closed launch fixture. Canonical resolution keeps symlinks that // escape the cwd on the explicit-resource path below. - if current_mcp_working_dir - .as_ref() - .is_some_and(|working_dir| canonical.starts_with(working_dir)) - { - continue; - } if path != canonical && std::fs::symlink_metadata(&path) .is_ok_and(|metadata| metadata.file_type().is_symlink()) @@ -2230,6 +2250,12 @@ pub fn select( declaring, )?; } + if current_mcp_working_dir + .as_ref() + .is_some_and(|working_dir| canonical.starts_with(working_dir)) + { + continue; + } let command_parent = canonical .parent() .ok_or_else(|| fail(field, "command has no containing scope"))?; @@ -2455,6 +2481,15 @@ pub fn select( &mut resource_path_bytes, field, )?; + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Node, + )?; } // A standalone script may import sibling files without a // package manifest. Copy its containing directory as the From 11bbfc2f3bdfa2f06a82a8dbb1059b9cc1057371 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:12:46 -0700 Subject: [PATCH 405/493] fix(managed): complete MCP dependency staging --- .../managed_mcp_capability/source_stage.rs | 143 +++++++++++++++++- .../opencode_resources.rs | 104 ++++++++----- 2 files changed, 210 insertions(+), 37 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 9f2f3801b..a3fa9b541 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1197,12 +1197,13 @@ mod configured_resource_tests { let home = owned.path().join("home"); let workspace = owned.path().join("workspace"); let command = home.join("mcp.js"); - let schema = home.join("schema.json"); + let schema = home.join("assets/schema.json"); std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(schema.parent().unwrap()).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "#!/usr/bin/env node\nconst fs = require('node:fs');\nconst path = require('node:path');\nprocess.stdout.write(fs.readFileSync(path.join(__dirname, 'schema.json'), 'utf8'));\n", + "#!/usr/bin/env node\nconst fs = require('node:fs');\nconst path = require('node:path');\nprocess.stdout.write(fs.readFileSync(path.join(__dirname, 'assets', 'schema.json'), 'utf8'));\n", ) .unwrap(); std::fs::write(&schema, "path.join schema loaded\n").unwrap(); @@ -1308,6 +1309,142 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"extensionless module loaded\n"); } + #[test] + fn direct_node_script_keeps_imported_symlink_alias_after_all_resource_copies() { + use std::os::unix::fs::symlink; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let helper = home.join("helper.js"); + let helper_target = home.join("shared/helper.js"); + let helper_directory = home.join("linked-dir"); + let helper_directory_target = home.join("shared-dir"); + let helper_directory_index = helper_directory.join("index.js"); + let helper_directory_index_target = helper_directory_target.join("index.js"); + std::fs::create_dir_all(helper_target.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&helper_directory_target).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "process.stdout.write(require('./helper.js') + require('./linked-dir'))\n", + ) + .unwrap(); + std::fs::write( + &helper_target, + "module.exports = 'symlinked file loaded|'\n", + ) + .unwrap(); + std::fs::write( + &helper_directory_index_target, + "module.exports = 'symlinked directory loaded\\n'\n", + ) + .unwrap(); + symlink(&helper_target, &helper).unwrap(); + symlink(&helper_directory_target, &helper_directory).unwrap(); + + let command_args = vec!["node".to_owned(), command.to_string_lossy().into_owned()]; + let (plan, provider_script) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command_args, + &command, + ); + assert!( + plan[0].configured_resources.contains(&helper), + "the Node closure must preserve the imported symlink alias" + ); + assert!(plan[0] + .configured_resources + .contains(&helper_target.canonicalize().unwrap())); + assert!( + plan[0] + .configured_resources + .contains(&helper_directory_index), + "the Node closure must preserve the imported path through a symlinked directory" + ); + assert!(plan[0] + .configured_resources + .contains(&helper_directory_index_target.canonicalize().unwrap())); + + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!( + output.stdout, + b"symlinked file loaded|symlinked directory loaded\n" + ); + } + + #[test] + fn versioned_python_command_stages_script_and_sibling_after_all_resource_copies() { + use std::os::unix::fs::symlink; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.py"); + let helper = home.join("helper.py"); + let bin = owned.path().join("bin"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&bin).unwrap(); + std::fs::write(&command, "import helper\nprint(helper.VALUE)\n").unwrap(); + std::fs::write(&helper, "VALUE = 'versioned command loaded'\n").unwrap(); + + let inherited_path = std::env::var_os("PATH").unwrap_or_default(); + let interpreter = std::env::split_paths(&inherited_path) + .map(|directory| directory.join("python3")) + .find(|candidate| candidate.is_file()) + .expect("python3 is required by the existing direct Python script test"); + symlink( + std::fs::canonicalize(interpreter).unwrap(), + bin.join("python3.12"), + ) + .unwrap(); + + let command_args = vec![ + "python3.12".to_owned(), + command.to_string_lossy().into_owned(), + ]; + let (plan, provider_script) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command_args, + &command, + ); + for required in [&command, &helper] { + assert!( + plan[0] + .configured_resources + .contains(&required.canonicalize().unwrap()), + "the versioned Python command closure omitted {}", + required.display() + ); + } + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let search_path = std::iter::once(bin.clone()) + .chain(std::env::split_paths(&inherited_path)) + .collect::<Vec<_>>(); + let search_path = std::env::join_paths(search_path).unwrap(); + let output = std::process::Command::new("python3.12") + .arg(&provider_script) + .env_clear() + .env("PATH", search_path) + .output(); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"versioned command loaded\n"); + } + #[test] fn direct_node_script_in_home_keeps_home_node_modules_after_all_copies() { use std::os::unix::fs::PermissionsExt; @@ -1491,7 +1628,7 @@ mod configured_resource_tests { std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "#!/bin/sh\ncd \"$(dirname \"$0\")\"; exec node server.js\n", + "#!/bin/sh\ncd \"$(dirname \"$0\")\" && exec node server.js\n", ) .unwrap(); std::fs::write( diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index ddb344983..4012f44da 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -468,6 +468,16 @@ fn bun_package_script_root( Ok(None) } +fn is_python_interpreter_name(name: &str) -> bool { + matches!(name, "python" | "python2" | "python3") + || name.strip_prefix("python").is_some_and(|version| { + !version.is_empty() + && version.split('.').all(|component| { + !component.is_empty() && component.bytes().all(|byte| byte.is_ascii_digit()) + }) + }) +} + fn command_uses_script_runtime(executable: Option<&str>) -> bool { let Some(executable) = executable.and_then(|executable| Path::new(executable).file_name()) else { @@ -483,15 +493,13 @@ fn command_uses_script_runtime(executable: Option<&str>) -> bool { | "deno" | "tsx" | "ts-node" - | "python" - | "python3" | "ruby" | "bash" | "sh" | "zsh" | "pwsh" | "powershell" - ) + ) || is_python_interpreter_name(executable) } fn command_uses_node_module_resolution(executable: Option<&str>) -> bool { @@ -604,23 +612,13 @@ fn is_shell_script(path: &Path) -> bool { } fn is_python_script(path: &Path) -> bool { - fn is_python_interpreter(name: &str) -> bool { - matches!(name, "python" | "python2" | "python3") - || name.strip_prefix("python").is_some_and(|version| { - !version.is_empty() - && version.split('.').all(|component| { - !component.is_empty() && component.bytes().all(|byte| byte.is_ascii_digit()) - }) - }) - } - path.extension().is_some_and(|extension| { matches!( extension.to_string_lossy().to_ascii_lowercase().as_str(), "py" | "pyw" ) }) || shebang_interpreter_names(path) - .is_some_and(|names| names.iter().any(|name| is_python_interpreter(name))) + .is_some_and(|names| names.iter().any(|name| is_python_interpreter_name(name))) } fn is_ruby_script(path: &Path) -> bool { @@ -668,9 +666,10 @@ fn shell_changes_to_script_directory(line: &str) -> Option<&str> { if remainder.is_empty() { return Some(""); } + let remainder = remainder.trim_start(); remainder - .trim_start() - .strip_prefix(';') + .strip_prefix("&&") + .or_else(|| remainder.strip_prefix(';')) .map(str::trim_start) } @@ -1008,23 +1007,46 @@ fn node_relative_resource_specifiers( Some(JavaScriptToken::Punctuation(b',')) ) { - if let Some(JavaScriptToken::String(Some(specifier))) = tokens.get(index + 4) { - let is_closed = matches!( - tokens.get(index + 5), - Some(JavaScriptToken::Punctuation(b')')) - ) || (matches!( - tokens.get(index + 5), - Some(JavaScriptToken::Punctuation(b',')) - ) && matches!( - tokens.get(index + 6), - Some(JavaScriptToken::Punctuation(b')')) - )); - if is_closed - && !specifier.is_empty() - && !Path::new(specifier).is_absolute() - && !specifier.contains(['?', '#', '\\', '$', '*']) - { - result.insert(specifier.clone()); + let mut cursor = index + 4; + let mut components = Vec::new(); + let is_closed = loop { + if let Some(JavaScriptToken::String(Some(component))) = tokens.get(cursor) { + components.push(component.as_str()); + cursor += 1; + } else { + break false; + } + match tokens.get(cursor) { + Some(JavaScriptToken::Punctuation(b')')) => break true, + Some(JavaScriptToken::Punctuation(b',')) => { + cursor += 1; + if matches!(tokens.get(cursor), Some(JavaScriptToken::Punctuation(b')'))) { + break true; + } + } + _ => break false, + } + }; + if is_closed && !components.is_empty() { + let mut relative = PathBuf::new(); + let safe = components.iter().all(|component| { + if component.is_empty() || component.contains(['?', '#', '\\', '$', '*']) { + return false; + } + let path = Path::new(component); + if path.components().any(|part| { + matches!( + part, + std::path::Component::Prefix(_) | std::path::Component::RootDir + ) + }) { + return false; + } + relative.push(path); + true + }); + if safe && !relative.as_os_str().is_empty() { + result.insert(relative.to_string_lossy().into_owned()); } } } @@ -1521,6 +1543,20 @@ fn insert_direct_script_dependencies( "direct MCP dependency escapes its selected script scope", )); } + let alias = freshell_platform::opencode_config::lexical_path(&candidate.path); + if alias != dependency { + if !alias.starts_with(scope) { + return Err(fail( + "direct MCP dependency alias escapes its selected script scope", + )); + } + if checked(&alias, field, declaring)? != dependency { + return Err(fail( + "direct MCP dependency changed while resolving its alias", + )); + } + insert_resource(result, resource_path_bytes, alias, field, declaring)?; + } insert_resource( result, resource_path_bytes, @@ -1715,7 +1751,7 @@ fn script_operand_index(command: &[String]) -> Option<usize> { index = (index + 2).min(command.len()); continue; } - } else if matches!(executable, "python" | "python3") { + } else if is_python_interpreter_name(executable) { if matches!(argument, "-c" | "-m") { return None; } From 53c31c4b93730649e278329cc116599bb48b0bb1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:39:16 -0700 Subject: [PATCH 406/493] fix(managed): close recovery and resource staging races --- .../managed_mcp_capability/source_stage.rs | 112 +++++++++ .../opencode_resources.rs | 218 ++++++++++++++++-- crates/freshell-server/src/managed_runtime.rs | 60 ++++- .../managed_recovery_transition_tests.rs | 180 +++++++++++++++ 4 files changed, 547 insertions(+), 23 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index a3fa9b541..f53f4c49e 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1189,6 +1189,36 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"node sibling helper loaded\n"); } + #[test] + fn direct_node_script_scanner_ignores_quotes_inside_regex_literals() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let helper = home.join("helper.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + r#"const quote = /require\(['"]\.\/missing/; +if (false) /'/; +process.stdout.write(require('./helper.js')) +"#, + ) + .unwrap(); + std::fs::write(&helper, "module.exports = 'regex scanner loaded\\n'\n").unwrap(); + + let (plan, provider_script) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"regex scanner loaded\n"); + } + #[test] fn direct_node_script_in_home_keeps_path_join_data_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; @@ -1615,6 +1645,88 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"sibling-loaded:package-loaded\n"); } + #[test] + fn shell_mcp_wrapper_stages_node_preload_before_server_script() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("start"); + let preload = home.join("preload.js"); + let preload_helper = home.join("preload-helper.js"); + let second_preload = home.join("second-preload.js"); + let imported_preload = home.join("preload.mjs"); + let loader = home.join("loader.mjs"); + let env_file = home.join(".env"); + let server = home.join("server.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\ncd \"$(dirname \"$0\")\" && exec node -r ./preload.js --require=./second-preload.js --import ./preload.mjs --loader=./loader.mjs --env-file=.env server.js\n", + ) + .unwrap(); + std::fs::write( + &preload, + "require('./preload-helper.js');\nglobalThis.managedPreload = 'loaded';\n", + ) + .unwrap(); + std::fs::write( + &preload_helper, + "globalThis.managedPreloadHelper = 'loaded';\n", + ) + .unwrap(); + std::fs::write( + &second_preload, + "globalThis.managedSecondPreload = 'loaded';\n", + ) + .unwrap(); + std::fs::write( + &imported_preload, + "globalThis.managedImportedPreload = 'loaded';\n", + ) + .unwrap(); + std::fs::write( + &loader, + "export async function resolve(specifier, context, nextResolve) { return nextResolve(specifier, context); }\n", + ) + .unwrap(); + std::fs::write(&env_file, "MANAGED_ENV_FILE=loaded\n").unwrap(); + std::fs::write( + &server, + "if (globalThis.managedPreload !== 'loaded' || globalThis.managedPreloadHelper !== 'loaded' || globalThis.managedSecondPreload !== 'loaded' || globalThis.managedImportedPreload !== 'loaded' || process.env.MANAGED_ENV_FILE !== 'loaded') process.exit(1);\nprocess.stdout.write('preload staged\\n');\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + for required in [ + &preload, + &preload_helper, + &second_preload, + &imported_preload, + &loader, + &env_file, + ] { + assert!( + plan[0] + .configured_resources + .contains(&required.canonicalize().unwrap()), + "Node option dependency omitted {}", + required.display() + ); + } + assert!(plan[0] + .configured_resources + .contains(&server.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"preload staged\n"); + } + #[test] fn home_shell_wrapper_stages_node_script_after_changing_to_its_directory() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 4012f44da..598c98d9e 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -683,7 +683,12 @@ fn shell_executable_name(token: &str) -> String { name.strip_suffix(".exe").unwrap_or(&name).to_owned() } -fn shell_node_script_operand(line: &str) -> Option<String> { +struct ShellNodeInvocation { + script_operand: Option<String>, + file_operands: Vec<(String, bool)>, +} + +fn shell_node_invocation(line: &str) -> Option<ShellNodeInvocation> { let words = line .split_ascii_whitespace() .map(|word| word.trim_matches(['"', '\''])) @@ -702,13 +707,37 @@ fn shell_node_script_operand(line: &str) -> Option<String> { "--env-file", "--inspect-port", ]; + const NODE_SCRIPT_FILE_OPTIONS: [&str; 5] = [ + "-r", + "--require", + "--import", + "--loader", + "--experimental-loader", + ]; + let mut file_operands = Vec::new(); while let Some(argument) = words.get(index) { if *argument == "--" { index += 1; break; } - if NODE_OPTIONS_WITH_VALUE.contains(argument) { - index += 2; + let (option, inline_value) = argument + .split_once('=') + .map_or((*argument, None), |(option, value)| (option, Some(value))); + if NODE_OPTIONS_WITH_VALUE.contains(&option) { + let value = inline_value.or_else(|| words.get(index + 1).copied()); + if let Some(value) = value { + let value = value.trim_end_matches([';', '&']); + let is_script = NODE_SCRIPT_FILE_OPTIONS.contains(&option); + let is_path = (option == "--env-file" && !value.starts_with('-')) + || Path::new(value).is_absolute() + || value.starts_with("./") + || value.starts_with("../") + || value.contains(['$', '*', '?']); + if is_path && (is_script || option == "--env-file") { + file_operands.push((value.to_owned(), is_script)); + } + } + index += if inline_value.is_some() { 1 } else { 2 }; continue; } if argument.starts_with('-') { @@ -717,13 +746,20 @@ fn shell_node_script_operand(line: &str) -> Option<String> { } break; } - let argument = words.get(index)?.trim_end_matches([';', '&']); - let path = Path::new(argument); - (!path.is_absolute() - && !argument.is_empty() - && !argument.contains(['$', '*', '?']) - && is_node_script_path(argument)) - .then(|| argument.to_owned()) + let script_operand = words + .get(index) + .map(|argument| argument.trim_end_matches([';', '&'])) + .filter(|argument| { + !Path::new(argument).is_absolute() + && !argument.is_empty() + && !argument.contains(['$', '*', '?']) + && is_node_script_path(argument) + }) + .map(str::to_owned); + Some(ShellNodeInvocation { + script_operand, + file_operands, + }) } fn shell_source_operand(line: &str) -> Option<String> { @@ -875,9 +911,131 @@ fn shell_line_invokes_node(line: &str) -> bool { enum JavaScriptToken { Identifier(String), String(Option<String>), + Regex, Punctuation(u8), } +fn javascript_control_header_closes(tokens: &[JavaScriptToken]) -> bool { + if !matches!(tokens.last(), Some(JavaScriptToken::Punctuation(b')'))) { + return false; + } + let mut depth = 0usize; + for (index, token) in tokens.iter().enumerate().rev() { + match token { + JavaScriptToken::Punctuation(b')') => depth += 1, + JavaScriptToken::Punctuation(b'(') => { + depth -= 1; + if depth == 0 { + return index > 0 + && matches!( + tokens.get(index - 1), + Some(JavaScriptToken::Identifier(keyword)) + if matches!(keyword.as_str(), "for" | "if" | "while" | "with") + ); + } + } + _ => {} + } + } + false +} + +fn javascript_regex_can_start_after(tokens: &[JavaScriptToken]) -> bool { + match tokens.last() { + None => true, + Some(JavaScriptToken::Punctuation(byte)) => { + matches!( + byte, + b'(' | b'[' + | b'{' + | b',' + | b';' + | b':' + | b'=' + | b'!' + | b'?' + | b'&' + | b'|' + | b'+' + | b'-' + | b'*' + | b'%' + | b'^' + | b'~' + | b'<' + | b'>' + | b'/' + ) || matches!(byte, b')') && javascript_control_header_closes(tokens) + || matches!(byte, b'+' | b'-') + && tokens.len() >= 3 + && matches!( + (tokens.get(tokens.len() - 2), tokens.get(tokens.len() - 3)), + ( + Some(JavaScriptToken::Punctuation(previous)), + Some(JavaScriptToken::Identifier(_) + | JavaScriptToken::String(_) + | JavaScriptToken::Regex) + ) if previous == byte + ) + } + Some(JavaScriptToken::Identifier(keyword)) => { + !matches!( + tokens.get(tokens.len().saturating_sub(2)), + Some(JavaScriptToken::Punctuation(b'.')) + ) && matches!( + keyword.as_str(), + "await" + | "case" + | "delete" + | "do" + | "else" + | "in" + | "instanceof" + | "new" + | "of" + | "return" + | "throw" + | "typeof" + | "void" + | "yield" + ) + } + Some(JavaScriptToken::String(_) | JavaScriptToken::Regex) => false, + } +} + +fn javascript_regex_end(bytes: &[u8], start: usize) -> Result<Option<usize>, &'static str> { + let mut at = start + 1; + let mut escaped = false; + let mut in_character_class = false; + while at < bytes.len() { + let byte = bytes[at]; + if byte == b'\n' || byte == b'\r' { + return Err("direct MCP Node script has an unterminated regular expression"); + } + if escaped { + escaped = false; + } else if byte == b'\\' { + escaped = true; + } else if byte == b'[' { + in_character_class = true; + } else if byte == b']' { + in_character_class = false; + } else if byte == b'/' && !in_character_class { + at += 1; + while bytes + .get(at) + .is_some_and(|flag| *flag == b'_' || *flag == b'$' || flag.is_ascii_alphanumeric()) + { + at += 1; + } + return Ok(Some(at)); + } + at += 1; + } + Ok(None) +} + fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> { const MAX_TOKENS: usize = 100_000; let bytes = source.as_bytes(); @@ -940,6 +1098,14 @@ fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> } at += 1; result.push(JavaScriptToken::String(None)); + } else if byte == b'/' && javascript_regex_can_start_after(&result) { + if let Some(end) = javascript_regex_end(bytes, at)? { + at = end; + result.push(JavaScriptToken::Regex); + } else { + at += 1; + result.push(JavaScriptToken::Punctuation(byte)); + } } else if byte == b'_' || byte == b'$' || byte.is_ascii_alphabetic() { at += 1; while bytes @@ -1647,6 +1813,9 @@ fn insert_shell_script_dependencies( } } let invokes_node = shell_line_invokes_node(node_line); + let node_invocation = invokes_node + .then(|| shell_node_invocation(node_line)) + .flatten(); for relative in shell_directory_relative_paths(line, &directory_variables) { let parse_as_node_script = invokes_node && is_node_script_path(&relative); candidates @@ -1655,11 +1824,30 @@ fn insert_shell_script_dependencies( .or_insert((false, parse_as_node_script)); } if working_directory_is_script_parent && invokes_node { - if let Some(relative) = shell_node_script_operand(node_line) { - candidates - .entry(script_parent.join(relative)) - .and_modify(|kind: &mut (bool, bool)| kind.1 = true) - .or_insert((false, true)); + if let Some(invocation) = node_invocation { + if let Some(relative) = invocation.script_operand { + candidates + .entry(script_parent.join(relative)) + .and_modify(|kind: &mut (bool, bool)| kind.1 = true) + .or_insert((false, true)); + } + for (operand, parse_as_node_script) in invocation.file_operands { + if operand.contains(['$', '*', '?']) { + return Err(fail( + "direct MCP Node option has an unresolved dynamic file path", + )); + } + let path = Path::new(&operand); + let candidate = if path.is_absolute() { + path.to_owned() + } else { + script_parent.join(path) + }; + candidates + .entry(candidate) + .and_modify(|kind: &mut (bool, bool)| kind.1 |= parse_as_node_script) + .or_insert((false, parse_as_node_script)); + } } } for (candidate, (parse_as_script, parse_as_node_script)) in candidates { diff --git a/crates/freshell-server/src/managed_runtime.rs b/crates/freshell-server/src/managed_runtime.rs index ee2a74ead..2d02737e1 100644 --- a/crates/freshell-server/src/managed_runtime.rs +++ b/crates/freshell-server/src/managed_runtime.rs @@ -17,7 +17,10 @@ use std::{ collections::HashMap, path::{Path, PathBuf}, process::Command, - sync::Arc, + sync::{ + atomic::{AtomicU64, Ordering}, + Arc, + }, }; use tokio::sync::Mutex; @@ -45,6 +48,8 @@ impl RecoveryPublication { } #[derive(Debug, Clone)] struct SoulRecoverySlot { + // While recovering, this is the task token. Otherwise it is the last + // local publication or inventory-read token accepted for this soul. sequence: u64, task_in_flight: bool, // Durable supervisor generation, independent of the local task sequence. @@ -76,6 +81,26 @@ impl SoulRecoverySlot { #[derive(Default)] struct ManagedRecoveryState { slots: Mutex<HashMap<String, SoulRecoverySlot>>, + generation_clock: AtomicU64, +} +impl ManagedRecoveryState { + fn next_generation(&self) -> u64 { + self.next_generation_after(0) + } + + fn next_generation_after(&self, minimum: u64) -> u64 { + loop { + let current = self.generation_clock.load(Ordering::Relaxed); + let next = current.max(minimum).saturating_add(1); + if self + .generation_clock + .compare_exchange_weak(current, next, Ordering::Relaxed, Ordering::Relaxed) + .is_ok() + { + return next; + } + } + } } fn wire_label(value: &impl serde::Serialize) -> String { @@ -230,10 +255,17 @@ impl ServerManagedRuntimeController { .is_some_and(|slot| slot.task_in_flight) } + #[cfg(test)] async fn hydrate(&self, view: RuntimeView) { + let generation = self.recovery.next_generation(); + self.hydrate_at_generation(view, generation).await; + } + + async fn hydrate_at_generation(&self, view: RuntimeView, generation: u64) { let mut slots = self.recovery.slots.lock().await; if slots.get(view.soul_id.as_str()).is_some_and(|slot| { slot.task_in_flight + || slot.sequence > generation || slot .intent_revision .is_some_and(|revision| revision > view.intent_revision) @@ -276,6 +308,7 @@ impl ServerManagedRuntimeController { ); slot.incarnation_id = Some(view.incarnation_id.to_string()); slot.terminal_id = view.terminal_id.clone(); + slot.sequence = generation; slot.intent_revision = Some(view.intent_revision); slot.evidence_revision = Some(view.evidence_revision); slot.publication = publication; @@ -290,7 +323,7 @@ impl ServerManagedRuntimeController { slots.insert( view.soul_id.to_string(), SoulRecoverySlot { - sequence: 0, + sequence: generation, task_in_flight: false, incarnation_id: Some(view.incarnation_id.to_string()), terminal_id: view.terminal_id.clone(), @@ -326,7 +359,7 @@ impl ServerManagedRuntimeController { if !manual && lifecycle(&slot.publication).state != ManagedLifecycleState::Running { return None; } - let sequence = slot.sequence.checked_add(1)?; + let sequence = self.recovery.next_generation_after(slot.sequence); let publication = RecoveryPublication::Recovering { previous: slot.publication.result().cloned().map(Box::new), }; @@ -343,8 +376,9 @@ impl ServerManagedRuntimeController { Some(sequence) } else { let publication = RecoveryPublication::Recovering { previous: None }; + let sequence = self.recovery.next_generation(); let mut slot = SoulRecoverySlot { - sequence: 1, + sequence, task_in_flight: true, incarnation_id: None, terminal_id: None, @@ -366,7 +400,7 @@ impl ServerManagedRuntimeController { slot.incarnation_id.as_deref(), ); slots.insert(soul.to_string(), slot); - Some(1) + Some(sequence) } } @@ -386,6 +420,7 @@ impl ServerManagedRuntimeController { { return; } + let published_generation = self.recovery.next_generation_after(sequence); let publication = match result { Ok(result) => RecoveryPublication::Finished(Box::new(result.clone())), Err(error) => RecoveryPublication::Failed { @@ -408,6 +443,7 @@ impl ServerManagedRuntimeController { slot.terminal_id.as_deref(), slot.incarnation_id.as_deref(), ); + slot.sequence = published_generation; slot.publication = publication; slot.task_in_flight = false; } @@ -481,6 +517,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { create_request_id: Option<String>, ) -> ManagedTerminalFuture<'a, Result<Option<ManagedTerminalDescriptor>, String>> { Box::pin(async move { + let inventory_generation = self.recovery.next_generation(); let views = self .client .inventory() @@ -493,7 +530,8 @@ impl ManagedTerminalController for ServerManagedRuntimeController { else { return Ok(None); }; - self.hydrate(view.clone()).await; + self.hydrate_at_generation(view.clone(), inventory_generation) + .await; managed_descriptor_from_view(view, create_request_id).map(Some) }) } @@ -741,6 +779,10 @@ impl ManagedTerminalController for ServerManagedRuntimeController { }), expected_control_epoch: None, }; + // A view returned by this launch may arrive after a concurrent + // recovery publication. Reserve its ordering before dispatch so + // that a later local publication remains authoritative. + let launch_generation = self.recovery.next_generation(); let result = if request.mode == "opencode" { let home = original_provider_home(&request.env)?; let original = |name: &str| { @@ -798,7 +840,8 @@ impl ManagedTerminalController for ServerManagedRuntimeController { } } }; - self.hydrate(result.view.clone()).await; + self.hydrate_at_generation(result.view.clone(), launch_generation) + .await; Ok(ManagedTerminalDescriptor { soul_id: result.view.soul_id.to_string(), incarnation_id: result.view.incarnation_id.to_string(), @@ -873,6 +916,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { { StopOutcome::VerifiedEmpty => { crate::managed_mcp_capability::revoke_soul_capabilities(&soul)?; + let inventory_generation = self.recovery.next_generation(); let views = self .client .inventory() @@ -883,7 +927,7 @@ impl ManagedTerminalController for ServerManagedRuntimeController { .rev() .find(|view| view.soul_id.as_str() == soul_key) { - self.hydrate(view).await; + self.hydrate_at_generation(view, inventory_generation).await; } Ok(()) } diff --git a/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs b/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs index aeb4ff0b8..77f6353cd 100644 --- a/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs +++ b/crates/freshell-server/src/tests/managed_recovery_transition_tests.rs @@ -1,4 +1,5 @@ use super::*; +use freshell_runtime_protocol::*; #[derive(Clone)] pub(crate) struct CaptureLogs(pub(crate) Arc<std::sync::Mutex<Vec<serde_json::Value>>>); #[derive(Default)] @@ -21,6 +22,185 @@ impl<S: tracing::Subscriber> tracing_subscriber::Layer<S> for CaptureLogs { ); } } + +#[tokio::test] +async fn late_lookup_inventory_hydration_cannot_regress_live_recovery_or_block_input() { + let root = tempfile::tempdir().unwrap(); + let socket = root.path().join("stale-inventory.sock"); + let listener = tokio::net::UnixListener::bind(&socket).unwrap(); + let soul = SoulId::new(); + let incarnation = IncarnationId::new(); + let blocked_view: RuntimeView = serde_json::from_value(serde_json::json!({ + "soulId": soul, + "incarnationId": incarnation, + "launchState": "running", + "cleanupState": "none", + "intentRevision": 4, + "evidenceRevision": 7, + "executionGeneration": 1, + "terminalId": "stale-inventory-terminal", + "terminalStreamId": "recovered-stream", + "terminalMode": "opencode", + "terminalCwd": "/workspace", + "desiredState": "running", + "recoveryState": "blocked", + "recoveryReason": "dependency_unavailable" + })) + .unwrap(); + let mut live_view = blocked_view.clone(); + live_view.recovery_state = RecoveryState::Live; + live_view.recovery_reason = None; + + let (inventory_started_tx, inventory_started_rx) = tokio::sync::oneshot::channel(); + let (release_inventory_tx, release_inventory_rx) = tokio::sync::oneshot::channel(); + let (input_tx, mut input_rx) = tokio::sync::mpsc::unbounded_channel(); + let (stop_tx, mut stop_rx) = tokio::sync::oneshot::channel(); + let stale_inventory_view = blocked_view.clone(); + let server = tokio::spawn(async move { + let mut inventory_started_tx = Some(inventory_started_tx); + let mut release_inventory_rx = Some(release_inventory_rx); + loop { + let mut peer = tokio::select! { + _ = &mut stop_rx => break, + accepted = listener.accept() => accepted.unwrap().0, + }; + let request: Envelope<AdminCommand> = read_frame(&mut peer).await.unwrap(); + let request_id = request.request_id; + match request.body { + AdminCommand::Inventory => { + inventory_started_tx.take().unwrap().send(()).unwrap(); + let release = release_inventory_rx.take().unwrap(); + let view = stale_inventory_view.clone(); + tokio::spawn(async move { + release.await.unwrap(); + write_frame( + &mut peer, + &AdminReply { + request_id, + result: Ok(AdminResult::Inventory(vec![view])), + }, + ) + .await + .unwrap(); + }); + } + AdminCommand::Health => { + write_frame( + &mut peer, + &AdminReply { + request_id, + result: Ok(AdminResult::Health { + control_epoch: 1, + installation_id: InstallationId::new(), + }), + }, + ) + .await + .unwrap(); + } + AdminCommand::TerminalInput(request) => { + input_tx.send(request.data).unwrap(); + write_frame( + &mut peer, + &AdminReply { + request_id, + result: Ok(AdminResult::TerminalInput { + state: CommandState::Completed, + }), + }, + ) + .await + .unwrap(); + } + other => panic!("unexpected managed runtime request: {other:?}"), + } + } + }); + + let controller = Arc::new(ServerManagedRuntimeController { + client: RuntimeClient::new(&socket, "synthetic-stale-inventory-control"), + recovery: Arc::default(), + }); + controller.hydrate(blocked_view.clone()).await; + let lookup_controller = controller.clone(); + let lookup = tokio::spawn(async move { + lookup_controller + .lookup_terminal("stale-inventory-terminal", None) + .await + }); + inventory_started_rx.await.unwrap(); + + let completion = ManagedOutputCompletion { + stream_epoch: "recovered-stream".into(), + incarnation_id: incarnation.to_string(), + head_seq: 23, + exit_code: Some(42), + }; + let sequence = controller + .begin_recovery_observed( + &soul, + true, + Some(RecoveryObservation { + terminal_id: "stale-inventory-terminal".into(), + incarnation_id: incarnation.to_string(), + epoch: completion.stream_epoch.clone(), + exit_code: completion.exit_code, + completion: Some(completion.clone()), + }), + ) + .await + .unwrap(); + let recovery_probe = RecoveryProbe::ReattachReady { + handle: ReattachHandle { + incarnation_id: incarnation.clone(), + host_boot_id: HostBootId::new(), + container_id: "a".repeat(64), + }, + protocol_version: 1, + }; + controller + .publish_recovery( + &soul, + sequence, + &Ok(RecoveryResult { + outcome: RecoveryOutcome::Reattached, + view: live_view, + probe: Some(recovery_probe.clone()), + prior_incarnation_id: None, + attempt_id: None, + expected_native_session_id: None, + observed_native_session_id: None, + incident_id: None, + }), + ) + .await; + + release_inventory_tx.send(()).unwrap(); + let terminal = lookup.await.unwrap().unwrap().unwrap(); + let lifecycle = controller.lifecycle_observation(&terminal).await.unwrap(); + assert_eq!(lifecycle.state, ManagedLifecycleState::Running); + { + let slots = controller.recovery.slots.lock().await; + let slot = slots.get(soul.as_str()).unwrap(); + let published = slot.publication.result().unwrap(); + assert_eq!(published.outcome, RecoveryOutcome::Reattached); + assert_eq!(published.view.recovery_state, RecoveryState::Live); + assert_eq!(published.probe.as_ref(), Some(&recovery_probe)); + assert_eq!(slot.observed_epoch.as_deref(), Some("recovered-stream")); + assert_eq!(slot.observed_exit_code, Some(42)); + assert_eq!(slot.observed_completion.as_ref(), Some(&completion)); + } + + controller + .input(terminal, "input after recovery".into()) + .await + .unwrap(); + assert_eq!(input_rx.recv().await.as_deref(), Some("input after recovery")); + + stop_tx.send(()).unwrap(); + server.await.unwrap(); +} + #[tokio::test] async fn managed_runtime_recovery_transition_changes_cause_only_and_redacts_failed_payload() { use tracing::instrument::WithSubscriber; From d8b3461de08bdefd641e7f37058c2239997dbeb4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:53:28 -0700 Subject: [PATCH 407/493] fix(managed): scan nested imports in staged scripts --- .../managed_mcp_capability/source_stage.rs | 65 ++++++++ .../opencode_resources.rs | 147 ++++++++++++++---- 2 files changed, 184 insertions(+), 28 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index f53f4c49e..6a1c88e88 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1219,6 +1219,36 @@ process.stdout.write(require('./helper.js')) assert_eq!(output.stdout, b"regex scanner loaded\n"); } + #[test] + fn direct_node_script_scans_template_expression_relative_import_after_all_copies() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("mcp.js"); + let helper = home.join("helper.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "process.stdout.write(`raw require('./missing.js') ${`nested ${({ helper: require('./helper.js') }).helper}`}`)\n", + ) + .unwrap(); + std::fs::write(&helper, "module.exports = 'template helper loaded\\n'\n").unwrap(); + + let (plan, provider_script) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!( + output.stdout, + b"raw require('./missing.js') nested template helper loaded\n" + ); + } + #[test] fn direct_node_script_in_home_keeps_path_join_data_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; @@ -2043,6 +2073,41 @@ process.stdout.write(require('./helper.js')) assert_eq!(output.stdout, b"nested package helper loaded\n"); } + #[test] + fn direct_python_absolute_package_import_uses_entry_point_import_root() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let package = home.join("pkg"); + let command = home.join("start"); + let helper = package.join("helper.py"); + std::fs::create_dir_all(&package).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env python3\nimport pkg\nprint(pkg.helper.VALUE)\n", + ) + .unwrap(); + std::fs::write(package.join("__init__.py"), "from pkg import helper\n").unwrap(); + std::fs::write(&helper, "VALUE = 'absolute package helper loaded'\n").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(plan[0] + .configured_resources + .contains(&helper.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"absolute package helper loaded\n"); + } + #[test] fn direct_python_multiline_package_import_loads_nested_module_after_all_copies() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 598c98d9e..914ede85f 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -913,6 +913,7 @@ enum JavaScriptToken { String(Option<String>), Regex, Punctuation(u8), + TemplateBoundary, } fn javascript_control_header_closes(tokens: &[JavaScriptToken]) -> bool { @@ -1000,7 +1001,9 @@ fn javascript_regex_can_start_after(tokens: &[JavaScriptToken]) -> bool { | "yield" ) } - Some(JavaScriptToken::String(_) | JavaScriptToken::Regex) => false, + Some( + JavaScriptToken::String(_) | JavaScriptToken::Regex | JavaScriptToken::TemplateBoundary, + ) => false, } } @@ -1036,12 +1039,74 @@ fn javascript_regex_end(bytes: &[u8], start: usize) -> Result<Option<usize>, &'s Ok(None) } -fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> { - const MAX_TOKENS: usize = 100_000; +const MAX_JAVASCRIPT_TOKENS: usize = 100_000; +const MAX_JAVASCRIPT_TEMPLATE_DEPTH: usize = 64; + +fn push_javascript_token( + tokens: &mut Vec<JavaScriptToken>, + token: JavaScriptToken, + token_count: &mut usize, +) -> Result<(), &'static str> { + *token_count += 1; + if *token_count > MAX_JAVASCRIPT_TOKENS { + return Err("direct MCP Node dependency scan exceeds its token bound"); + } + tokens.push(token); + Ok(()) +} + +fn javascript_template_tokens( + source: &str, + start: usize, + depth: usize, + token_count: &mut usize, +) -> Result<(Vec<JavaScriptToken>, usize), &'static str> { + if depth >= MAX_JAVASCRIPT_TEMPLATE_DEPTH { + return Err("direct MCP Node dependency scan exceeds its template nesting bound"); + } let bytes = source.as_bytes(); - let mut at = 0usize; + let mut at = start + 1; let mut result = Vec::new(); while at < bytes.len() { + if bytes[at] == b'\\' { + at = (at + 2).min(bytes.len()); + } else if bytes[at] == b'`' { + at += 1; + push_javascript_token(&mut result, JavaScriptToken::String(None), token_count)?; + return Ok((result, at)); + } else if bytes.get(at..at + 2) == Some(b"${") { + push_javascript_token(&mut result, JavaScriptToken::TemplateBoundary, token_count)?; + let expression_start = at + 2; + let (expression, expression_end) = + javascript_tokens_inner(source, expression_start, true, depth + 1, token_count)?; + if bytes.get(expression_end) != Some(&b'}') { + return Err("direct MCP Node script has an incomplete template expression"); + } + result.extend(expression); + push_javascript_token(&mut result, JavaScriptToken::TemplateBoundary, token_count)?; + at = expression_end + 1; + } else { + at += 1; + } + } + Err("direct MCP Node script has an unterminated template string") +} + +fn javascript_tokens_inner( + source: &str, + start: usize, + stop_at_template_expression_end: bool, + template_depth: usize, + token_count: &mut usize, +) -> Result<(Vec<JavaScriptToken>, usize), &'static str> { + let bytes = source.as_bytes(); + let mut at = start; + let mut result = Vec::new(); + let mut brace_depth = 0usize; + while at < bytes.len() { + if stop_at_template_expression_end && bytes[at] == b'}' && brace_depth == 0 { + return Ok((result, at)); + } if bytes[at].is_ascii_whitespace() { at += 1; continue; @@ -1083,28 +1148,23 @@ fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> } let value = (!escaped).then(|| source[content_start..at].to_owned()); at += 1; - result.push(JavaScriptToken::String(value)); + push_javascript_token(&mut result, JavaScriptToken::String(value), token_count)?; } else if byte == b'`' { - at += 1; - while at < bytes.len() && bytes[at] != b'`' { - if bytes[at] == b'\\' { - at = (at + 2).min(bytes.len()); - } else { - at += 1; - } - } - if at == bytes.len() { - return Err("direct MCP Node script has an unterminated template string"); - } - at += 1; - result.push(JavaScriptToken::String(None)); + let (template_tokens, template_end) = + javascript_template_tokens(source, at, template_depth, token_count)?; + result.extend(template_tokens); + at = template_end; } else if byte == b'/' && javascript_regex_can_start_after(&result) { if let Some(end) = javascript_regex_end(bytes, at)? { at = end; - result.push(JavaScriptToken::Regex); + push_javascript_token(&mut result, JavaScriptToken::Regex, token_count)?; } else { at += 1; - result.push(JavaScriptToken::Punctuation(byte)); + push_javascript_token( + &mut result, + JavaScriptToken::Punctuation(byte), + token_count, + )?; } } else if byte == b'_' || byte == b'$' || byte.is_ascii_alphabetic() { at += 1; @@ -1114,16 +1174,31 @@ fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> { at += 1; } - result.push(JavaScriptToken::Identifier(source[start..at].to_owned())); + push_javascript_token( + &mut result, + JavaScriptToken::Identifier(source[start..at].to_owned()), + token_count, + )?; } else { + if byte == b'{' { + brace_depth += 1; + } else if byte == b'}' { + brace_depth = brace_depth.saturating_sub(1); + } at += 1; - result.push(JavaScriptToken::Punctuation(byte)); - } - if result.len() > MAX_TOKENS { - return Err("direct MCP Node dependency scan exceeds its token bound"); + push_javascript_token(&mut result, JavaScriptToken::Punctuation(byte), token_count)?; } } - Ok(result) + if stop_at_template_expression_end { + Err("direct MCP Node script has an incomplete template expression") + } else { + Ok((result, at)) + } +} + +fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> { + let mut token_count = 0; + javascript_tokens_inner(source, 0, false, 0, &mut token_count).map(|(tokens, _)| tokens) } fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static str> { @@ -1586,6 +1661,7 @@ struct ScriptDependencyPath { fn direct_script_dependency_paths( parent: &Path, + entry_point_import_root: &Path, source: &str, runtime: ScriptDependencyRuntime, ) -> Result<Vec<ScriptDependencyPath>, &'static str> { @@ -1622,7 +1698,14 @@ fn direct_script_dependency_paths( } ScriptDependencyRuntime::Python => Ok(python_import_specifiers(source) .into_iter() - .flat_map(|specifier| resolve_python_local_modules(parent, &specifier)) + .flat_map(|specifier| { + let import_root = if specifier.starts_with('.') { + parent + } else { + entry_point_import_root + }; + resolve_python_local_modules(import_root, &specifier) + }) .map(|path| ScriptDependencyPath { parse_as_script: is_script_dependency_source(&path, runtime), path, @@ -1668,6 +1751,11 @@ fn insert_direct_script_dependencies( const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; const MAX_SCRIPT_FILES: usize = 256; let fail = |cause| ResourceError { field, cause }; + let entry_point = checked(script, field, declaring).map_err(|error| fail(error.cause))?; + let entry_point_import_root = entry_point + .parent() + .ok_or_else(|| fail("direct MCP script has no entry-point import root"))? + .to_owned(); let mut pending = vec![script.to_owned()]; let mut visited = std::collections::BTreeSet::new(); let mut parsed = std::collections::BTreeSet::new(); @@ -1696,7 +1784,10 @@ fn insert_direct_script_dependencies( let parent = canonical .parent() .ok_or_else(|| fail("direct MCP script has no containing directory"))?; - for candidate in direct_script_dependency_paths(parent, &source, runtime).map_err(fail)? { + for candidate in + direct_script_dependency_paths(parent, &entry_point_import_root, &source, runtime) + .map_err(fail)? + { let dependency = checked(&candidate.path, field, declaring)?; let dependency_is_file = dependency.is_file(); if !dependency_is_file && !dependency.is_dir() { From b3c2eb981c468c5f04e8e564b5d0d897192a1140 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:23:53 -0700 Subject: [PATCH 408/493] fix(managed): stage direct Node options and write resources --- .../managed_mcp_capability/source_stage.rs | 384 +++++++++++++- .../opencode_resources.rs | 467 +++++++++++++++++- 2 files changed, 822 insertions(+), 29 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 6a1c88e88..83fb45181 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -127,14 +127,36 @@ mod configured_resource_tests { workspace: &Path, command: &[String], selected_resource: &Path, + ) -> (Vec<PlannedSourceOccurrence>, std::path::PathBuf) { + plan_and_stage_mcp_command_for_resource_with_cwd( + owned_root, + home, + workspace, + command, + selected_resource, + None, + ) + } + + fn plan_and_stage_mcp_command_for_resource_with_cwd( + owned_root: &Path, + home: &Path, + workspace: &Path, + command: &[String], + selected_resource: &Path, + mcp_cwd: Option<&Path>, ) -> (Vec<PlannedSourceOccurrence>, std::path::PathBuf) { let config = workspace.join("opencode.json"); + let mut mcp_entry = serde_json::json!({ + "type": "local", + "command": command, + }); + if let Some(mcp_cwd) = mcp_cwd { + mcp_entry["cwd"] = serde_json::json!(mcp_cwd.to_string_lossy()); + } std::fs::write( &config, - serde_json::json!({ - "mcp": { "direct": { "type": "local", "command": command } }, - }) - .to_string(), + serde_json::json!({ "mcp": { "direct": mcp_entry } }).to_string(), ) .unwrap(); let plan = plan_opencode_sources( @@ -214,6 +236,22 @@ mod configured_resource_tests { output.unwrap() } + fn execute_node_command_with_home_hidden( + owned_root: &Path, + home: &Path, + arguments: &[String], + ) -> std::process::Output { + let hidden_home = owned_root.join("home-hidden"); + std::fs::rename(home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .args(arguments) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden_home, home).unwrap(); + output.unwrap() + } + #[test] fn managed_provider_keeps_external_indexed_payloads_out_of_persistent_generation() { fn tree_contains_bytes(path: &Path, expected: &[u8]) -> bool { @@ -1286,6 +1324,298 @@ process.stdout.write(require('./helper.js')) assert_eq!(output.stdout, b"path.join schema loaded\n"); } + #[test] + fn direct_node_command_stages_inline_and_separated_file_options() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("server.mjs"); + let preload = home.join("preload.cjs"); + let preload_helper = home.join("preload-helper.cjs"); + let imported = home.join("imported.mjs"); + let loader = home.join("loader.mjs"); + let experimental_loader = home.join("experimental-loader.mjs"); + let env_file = home.join(".env"); + let second_env_file = home.join(".env.local"); + let optional_env_file = home.join(".env.optional"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(&command, "if (globalThis.managedPreload !== 'loaded' || globalThis.managedPreloadHelper !== 'loaded' || globalThis.managedImport !== 'loaded' || process.env.MANAGED_ENV_FILE !== 'loaded' || process.env.MANAGED_ENV_FILE_LOCAL !== 'loaded') process.exit(1);\nprocess.stdout.write('command options staged\\n');\n").unwrap(); + std::fs::write( + &preload, + "require('./preload-helper.cjs');\nglobalThis.managedPreload = 'loaded';\n", + ) + .unwrap(); + std::fs::write( + &preload_helper, + "globalThis.managedPreloadHelper = 'loaded';\n", + ) + .unwrap(); + std::fs::write(&imported, "globalThis.managedImport = 'loaded';\n").unwrap(); + std::fs::write(&loader, "export async function resolve(specifier, context, nextResolve) { return nextResolve(specifier, context); }\n").unwrap(); + std::fs::write(&experimental_loader, "export async function resolve(specifier, context, nextResolve) { return nextResolve(specifier, context); }\n").unwrap(); + std::fs::write(&env_file, "MANAGED_ENV_FILE=loaded\n").unwrap(); + std::fs::write(&second_env_file, "MANAGED_ENV_FILE_LOCAL=loaded\n").unwrap(); + + let option_arguments = vec![ + "node".to_owned(), + format!("--require={}", preload.display()), + "--import".to_owned(), + imported.to_string_lossy().into_owned(), + format!("--loader={}", loader.display()), + "--experimental-loader".to_owned(), + experimental_loader.to_string_lossy().into_owned(), + format!("--env-file={}", env_file.display()), + "--env-file".to_owned(), + second_env_file.to_string_lossy().into_owned(), + format!("--env-file-if-exists={}", optional_env_file.display()), + command.to_string_lossy().into_owned(), + ]; + let (plan, _) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &option_arguments, + &command, + ); + for required in [ + &preload, + &preload_helper, + &imported, + &loader, + &experimental_loader, + &env_file, + &second_env_file, + ] { + assert!( + plan[0] + .configured_resources + .contains(&required.canonicalize().unwrap()), + "direct Node command option resource omitted {}", + required.display() + ); + } + assert!(!plan[0] + .configured_resources + .contains(&home.canonicalize().unwrap())); + assert!(!plan[0].configured_resources.contains( + &optional_env_file + .canonicalize() + .unwrap_or(optional_env_file.clone()) + )); + + let provider = copy_selected_units_through_provider( + owned.path(), + &resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(), + ); + let staged = |path: &Path| { + provider + .join(".provider-home") + .join(path.strip_prefix(&home).unwrap()) + }; + let arguments = vec![ + format!("--require={}", staged(&preload).display()), + "--import".to_owned(), + staged(&imported).to_string_lossy().into_owned(), + format!("--loader={}", staged(&loader).display()), + "--experimental-loader".to_owned(), + staged(&experimental_loader).to_string_lossy().into_owned(), + format!("--env-file={}", staged(&env_file).display()), + "--env-file".to_owned(), + staged(&second_env_file).to_string_lossy().into_owned(), + format!( + "--env-file-if-exists={}", + staged(&optional_env_file).display() + ), + staged(&command).to_string_lossy().into_owned(), + ]; + let output = execute_node_command_with_home_hidden(owned.path(), &home, &arguments); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"command options staged\n"); + } + + #[test] + fn direct_node_command_options_inside_mcp_cwd_use_the_directory_copy() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let mcp_cwd = home.join("tools"); + let command = mcp_cwd.join("server.js"); + let preload = mcp_cwd.join("preload.cjs"); + let env_file = mcp_cwd.join(".env"); + std::fs::create_dir_all(&mcp_cwd).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "if (globalThis.cwdPreload !== 'loaded' || process.env.MANAGED_CWD_ENV !== 'loaded') process.exit(1);\nprocess.stdout.write('cwd options loaded\\n');\n", + ) + .unwrap(); + std::fs::write(&preload, "globalThis.cwdPreload = 'loaded';\n").unwrap(); + std::fs::write(&env_file, "MANAGED_CWD_ENV=loaded\n").unwrap(); + let command_arguments = vec![ + "node".to_owned(), + "--require=./preload.cjs".to_owned(), + "--env-file=.env".to_owned(), + "server.js".to_owned(), + ]; + + let (plan, _) = plan_and_stage_mcp_command_for_resource_with_cwd( + owned.path(), + &home, + &workspace, + &command_arguments, + &command, + Some(&mcp_cwd), + ); + let canonical_cwd = mcp_cwd.canonicalize().unwrap(); + assert!(plan[0].configured_resources.contains(&canonical_cwd)); + for covered in [&command, &preload, &env_file] { + assert!( + !plan[0] + .configured_resources + .contains(&covered.canonicalize().unwrap()), + "MCP cwd should supply {} without an overlapping exact-file mount", + covered.display() + ); + } + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let cwd_mount = units + .iter() + .find(|(source, _)| source == &canonical_cwd) + .map(|(_, mount)| mount) + .expect("selected MCP cwd must have a provider resource mount"); + let provider_relative = Path::new(&cwd_mount.effective_destination) + .strip_prefix(owned.path()) + .map(Path::to_path_buf) + .or_else(|_| { + Path::new(&cwd_mount.effective_destination) + .strip_prefix("/home/freshell/provider") + .map(|relative| Path::new(".provider-home").join(relative)) + }) + .expect("test MCP cwd destination must map into the provider root"); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_cwd = provider.join(provider_relative); + assert!(provider_cwd.is_dir(), "{}", provider_cwd.display()); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .args(["--require=./preload.cjs", "--env-file=.env", "server.js"]) + .current_dir(&provider_cwd) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + std::fs::rename(&hidden_home, &home).unwrap(); + let output = output.unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"cwd options loaded\n"); + } + + #[test] + fn direct_node_script_does_not_stage_absent_sibling_write_targets() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("writer.mjs"); + let path_output = home.join("generated-path.json"); + let url_output = home.join("generated-url.json"); + let append_output = home.join("existing-output.txt"); + let append_url_output = home.join("existing-url-output.txt"); + let copy_source = home.join("copy-source.txt"); + let copy_output = home.join("copy-output.txt"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", + ) + .unwrap(); + std::fs::write(&append_output, "existing").unwrap(); + std::fs::write(&append_url_output, "url existing").unwrap(); + std::fs::write(©_source, "copy input").unwrap(); + + let (plan, provider_script) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + for output in [&path_output, &url_output] { + assert!( + !plan[0] + .configured_resources + .contains(&output.canonicalize().unwrap_or_else(|_| output.clone())), + "a prospective write destination must not be a required staged input" + ); + } + assert!(plan[0] + .configured_resources + .contains(&append_output.canonicalize().unwrap())); + assert!(plan[0] + .configured_resources + .contains(&append_url_output.canonicalize().unwrap())); + assert!(plan[0] + .configured_resources + .contains(©_source.canonicalize().unwrap())); + assert!(!plan[0].configured_resources.contains( + ©_output + .canonicalize() + .unwrap_or_else(|_| copy_output.clone()) + )); + + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); + assert!(output.status.success(), "{:?}", output.stderr); + let provider_script_directory = std::fs::canonicalize(&provider_script) + .unwrap() + .parent() + .unwrap() + .to_owned(); + assert_eq!( + output.stdout, + format!("{}\n", provider_script_directory.display()).as_bytes() + ); + let staged_path_output = provider_script_directory.join("generated-path.json"); + let staged_url_output = provider_script_directory.join("generated-url.json"); + let staged_append_output = provider_script_directory.join("existing-output.txt"); + let staged_append_url_output = provider_script_directory.join("existing-url-output.txt"); + let staged_copy_output = provider_script_directory.join("copy-output.txt"); + assert_eq!( + std::fs::read(&staged_path_output).unwrap_or_else(|error| panic!( + "failed to read {} beside {}: {error}; entries: {:?}; staged script: {:?}; stdout: {:?}", + staged_path_output.display(), + provider_script.display(), + std::fs::read_dir(&provider_script_directory) + .unwrap() + .map(|entry| entry.unwrap().file_name()) + .collect::<Vec<_>>(), + std::fs::read_to_string(&provider_script), + output.stdout + )), + b"path output" + ); + assert_eq!(std::fs::read(&staged_url_output).unwrap(), b"url output"); + assert_eq!( + std::fs::read(&staged_append_output).unwrap(), + b"existing appended" + ); + assert_eq!( + std::fs::read(&staged_append_url_output).unwrap(), + b"url existing appended" + ); + assert_eq!(std::fs::read(&staged_copy_output).unwrap(), b"copy input"); + } + #[test] fn direct_node_script_in_nested_home_directory_keeps_relative_module_closure() { let owned = tempfile::tempdir().unwrap(); @@ -2275,6 +2605,52 @@ process.stdout.write(require('./helper.js')) assert_eq!(output.stdout, expected); } + #[test] + fn direct_shell_script_does_not_stage_absent_sibling_redirection_targets() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("writer"); + let generated = home.join("generated.txt"); + let existing_output = home.join("existing.txt"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\nprintf 'created by shell\\n' > \"$SCRIPT_DIR/generated.txt\"\nprintf 'appended by shell\\n' >> \"$SCRIPT_DIR/generated.txt\"\nprintf ' appended' >> \"$SCRIPT_DIR/existing.txt\"\n", + ) + .unwrap(); + std::fs::write(&existing_output, "existing").unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!( + !plan[0].configured_resources.contains( + &generated + .canonicalize() + .unwrap_or_else(|_| generated.clone()) + ), + "a shell redirection destination must not be a required staged input" + ); + assert!(plan[0] + .configured_resources + .contains(&existing_output.canonicalize().unwrap())); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!( + std::fs::read(provider_command.with_file_name("generated.txt")).unwrap(), + b"created by shell\nappended by shell\n" + ); + assert_eq!( + std::fs::read(provider_command.with_file_name("existing.txt")).unwrap(), + b"existing appended" + ); + } + #[test] fn shell_shebang_keeps_binary_support_resource_from_inline_dirname_expression() { use std::os::unix::fs::PermissionsExt; diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 914ede85f..7035e65bb 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -249,6 +249,11 @@ fn nested_fields( && command[1] == FRESHELL_IMAGE_MCP_SERVER_PATH; if !image_command { let script_index = script_operand_index(&command); + let option_resources = node_command_option_resources(&command); + let option_resource_indexes = option_resources + .iter() + .map(|(index, _, _)| *index) + .collect::<std::collections::BTreeSet<_>>(); let script_runtime = command_uses_script_runtime(command.first().map(String::as_str)); let node_script_field = @@ -266,6 +271,7 @@ fn nested_fields( index > 0 && index <= script && is_script_path(argument) + && !option_resource_indexes.contains(&index) && !is_bun_run_bare_operand(&command, index) && !bun_run_file }) @@ -291,6 +297,9 @@ fn nested_fields( result.push((node_script_field, argument.clone())); } } + for (_, field, spelling) in option_resources { + result.push((field, spelling)); + } if let Some(script_index) = script_index { let script = &command[script_index]; if is_bun_run_bare_operand(&command, script_index) { @@ -538,6 +547,31 @@ fn is_mcp_command_path_field(field: &str) -> bool { ) } +fn is_mcp_command_resource_field(field: &str) -> bool { + is_mcp_command_path_field(field) + || matches!( + field, + "OpenCode mcp.node-option-script" + | "OpenCode mcp.node-option-data" + | "OpenCode mcp.node-option-data-optional" + ) +} + +fn is_mcp_node_option_script_field(field: &str) -> bool { + field == "OpenCode mcp.node-option-script" +} + +fn is_mcp_node_option_data_field(field: &str) -> bool { + matches!( + field, + "OpenCode mcp.node-option-data" | "OpenCode mcp.node-option-data-optional" + ) +} + +fn is_mcp_node_option_optional_data_field(field: &str) -> bool { + field == "OpenCode mcp.node-option-data-optional" +} + fn is_mcp_script_field(field: &str) -> bool { matches!( field, @@ -836,6 +870,7 @@ fn shell_source_path( fn shell_directory_relative_paths( line: &str, directory_variables: &std::collections::BTreeSet<String>, + script_parent: &Path, ) -> std::collections::BTreeSet<String> { let mut result = std::collections::BTreeSet::new(); let mut prefixes = std::collections::BTreeSet::new(); @@ -845,25 +880,61 @@ fn shell_directory_relative_paths( } prefixes.insert("$(dirname \"$0\")/".to_owned()); for prefix in prefixes { - let mut remaining = line; - while let Some(start) = remaining.find(&prefix) { - let suffix = &remaining[start + prefix.len()..]; + let mut search_from = 0; + while search_from < line.len() { + let Some(relative_start) = line[search_from..].find(&prefix) else { + break; + }; + let start = search_from + relative_start; + let suffix = &line[start + prefix.len()..]; let length = suffix .find(|character: char| character.is_whitespace() || "\"'`;|&)".contains(character)) .unwrap_or(suffix.len()); let relative = &suffix[..length]; - if !relative.is_empty() && !relative.contains(['$', '*', '?']) { + if !relative.is_empty() + && !relative.contains(['$', '*', '?']) + && (!shell_directory_path_is_redirection_target(line, start) + || script_parent.join(relative).exists()) + { result.insert(relative.to_owned()); } - remaining = &suffix[length..]; - if remaining.is_empty() { - break; - } + search_from = start + prefix.len() + length; } } result } +fn shell_directory_path_is_redirection_target(line: &str, path_start: usize) -> bool { + let bytes = line.as_bytes(); + let mut at = 0; + let mut quote = None; + let mut quote_start = None; + let mut escaped = false; + while at < path_start { + let byte = bytes[at]; + if escaped { + escaped = false; + } else if byte == b'\\' && quote != Some(b'\'') { + escaped = true; + } else if let Some(active_quote) = quote { + if byte == active_quote { + quote = None; + quote_start = None; + } + } else if matches!(byte, b'\'' | b'"') { + quote = Some(byte); + quote_start = Some(at); + } + at += 1; + } + let prefix_end = if quote.is_some() { + quote_start.unwrap_or(path_start) + } else { + path_start + }; + line[..prefix_end].trim_end().ends_with('>') +} + fn shell_line_invokes_node(line: &str) -> bool { let words = line .split_ascii_whitespace() @@ -1226,9 +1297,11 @@ fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static } fn node_relative_resource_specifiers( + parent: &Path, source: &str, ) -> Result<std::collections::BTreeSet<String>, &'static str> { let tokens = javascript_tokens(source)?; + let output_destinations = node_output_destination_mask(&tokens)?; let mut result = std::collections::BTreeSet::new(); for index in 0..tokens.len() { if index >= 2 @@ -1287,7 +1360,10 @@ fn node_relative_resource_specifiers( true }); if safe && !relative.as_os_str().is_empty() { - result.insert(relative.to_string_lossy().into_owned()); + let relative = relative.to_string_lossy().into_owned(); + if !output_destinations[index - 2] || parent.join(&relative).exists() { + result.insert(relative); + } } } } @@ -1332,6 +1408,7 @@ fn node_relative_resource_specifiers( && is_import_meta_url && (specifier.starts_with("./") || specifier.starts_with("../")) && !specifier.contains(['?', '#', '\\', '$']) + && (!output_destinations[index] || parent.join(specifier).exists()) { result.insert(specifier.clone()); } @@ -1339,6 +1416,104 @@ fn node_relative_resource_specifiers( Ok(result) } +fn node_output_destination_mask(tokens: &[JavaScriptToken]) -> Result<Vec<bool>, &'static str> { + const OUTPUT_CALLS: [(&str, usize); 15] = [ + ("appendFile", 0), + ("appendFileSync", 0), + ("createWriteStream", 0), + ("mkdir", 0), + ("mkdirSync", 0), + ("writeFile", 0), + ("writeFileSync", 0), + ("copyFile", 1), + ("copyFileSync", 1), + ("link", 1), + ("linkSync", 1), + ("rename", 1), + ("renameSync", 1), + ("symlink", 1), + ("symlinkSync", 1), + ]; + let mut ranges = Vec::new(); + let mut inspection_budget = MAX_JAVASCRIPT_TOKENS; + for (index, token) in tokens.iter().enumerate() { + let JavaScriptToken::Identifier(name) = token else { + continue; + }; + let Some((_, destination_argument)) = OUTPUT_CALLS + .iter() + .find(|(output_call, _)| *output_call == name) + else { + continue; + }; + if !matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'(')) + ) { + continue; + } + let destination_argument = *destination_argument; + let mut argument_index = 0usize; + let mut start = index + 2; + let mut end = None; + let mut parentheses = 0usize; + let mut brackets = 0usize; + let mut braces = 0usize; + for (cursor, token) in tokens.iter().enumerate().skip(start) { + inspection_budget = inspection_budget.saturating_sub(1); + if inspection_budget == 0 { + return Err("direct MCP Node output scan exceeds its token bound"); + } + match token { + JavaScriptToken::Punctuation(b'(') => parentheses += 1, + JavaScriptToken::Punctuation(b')') + if parentheses == 0 && brackets == 0 && braces == 0 => + { + if argument_index == destination_argument { + end = Some(cursor); + } + break; + } + JavaScriptToken::Punctuation(b')') => parentheses = parentheses.saturating_sub(1), + JavaScriptToken::Punctuation(b'[') => brackets += 1, + JavaScriptToken::Punctuation(b']') => brackets = brackets.saturating_sub(1), + JavaScriptToken::Punctuation(b'{') => braces += 1, + JavaScriptToken::Punctuation(b'}') => braces = braces.saturating_sub(1), + JavaScriptToken::Punctuation(b',') + if parentheses == 0 && brackets == 0 && braces == 0 => + { + if argument_index == destination_argument { + end = Some(cursor); + break; + } + argument_index += 1; + start = cursor + 1; + } + _ => {} + } + } + if let Some(end) = end.filter(|end| *end > start) { + ranges.push(start..end); + } + } + ranges.sort_unstable_by_key(|range| range.start); + let mut merged = Vec::<std::ops::Range<usize>>::new(); + for range in ranges { + if let Some(previous) = merged.last_mut() { + if range.start <= previous.end { + previous.end = previous.end.max(range.end); + continue; + } + } + merged.push(range); + } + let mut result = vec![false; tokens.len()]; + for range in merged { + result[range].fill(true); + } + Ok(result) +} + fn read_node_package_manifest( directory: &Path, ) -> Result<Option<(PathBuf, Option<PathBuf>)>, &'static str> { @@ -1654,6 +1829,11 @@ enum ScriptDependencyRuntime { Ruby, } +struct ScriptDependencySelection<'a> { + runtime: ScriptDependencyRuntime, + covered_directory: Option<&'a Path>, +} + struct ScriptDependencyPath { path: PathBuf, parse_as_script: bool, @@ -1684,7 +1864,7 @@ fn direct_script_dependency_paths( record(file, true); } } - for specifier in node_relative_resource_specifiers(source)? { + for specifier in node_relative_resource_specifiers(parent, source)? { record(parent.join(specifier), false); } } @@ -1747,6 +1927,29 @@ fn insert_direct_script_dependencies( resource_path_bytes: &mut usize, field: &'static str, runtime: ScriptDependencyRuntime, +) -> Result<(), ResourceError> { + insert_direct_script_dependencies_covered( + script, + scope, + declaring, + result, + resource_path_bytes, + field, + ScriptDependencySelection { + runtime, + covered_directory: None, + }, + ) +} + +fn insert_direct_script_dependencies_covered( + script: &Path, + scope: &Path, + declaring: &Path, + result: &mut std::collections::BTreeSet<PathBuf>, + resource_path_bytes: &mut usize, + field: &'static str, + selection: ScriptDependencySelection<'_>, ) -> Result<(), ResourceError> { const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; const MAX_SCRIPT_FILES: usize = 256; @@ -1784,9 +1987,13 @@ fn insert_direct_script_dependencies( let parent = canonical .parent() .ok_or_else(|| fail("direct MCP script has no containing directory"))?; - for candidate in - direct_script_dependency_paths(parent, &entry_point_import_root, &source, runtime) - .map_err(fail)? + for candidate in direct_script_dependency_paths( + parent, + &entry_point_import_root, + &source, + selection.runtime, + ) + .map_err(fail)? { let dependency = checked(&candidate.path, field, declaring)?; let dependency_is_file = dependency.is_file(); @@ -1812,15 +2019,25 @@ fn insert_direct_script_dependencies( "direct MCP dependency changed while resolving its alias", )); } - insert_resource(result, resource_path_bytes, alias, field, declaring)?; + if !selection + .covered_directory + .is_some_and(|directory| alias.starts_with(directory)) + { + insert_resource(result, resource_path_bytes, alias, field, declaring)?; + } + } + if !selection + .covered_directory + .is_some_and(|directory| dependency.starts_with(directory)) + { + insert_resource( + result, + resource_path_bytes, + dependency.clone(), + field, + declaring, + )?; } - insert_resource( - result, - resource_path_bytes, - dependency.clone(), - field, - declaring, - )?; if dependency_is_file && candidate.parse_as_script { pending.push(dependency); } @@ -1907,7 +2124,9 @@ fn insert_shell_script_dependencies( let node_invocation = invokes_node .then(|| shell_node_invocation(node_line)) .flatten(); - for relative in shell_directory_relative_paths(line, &directory_variables) { + for relative in + shell_directory_relative_paths(line, &directory_variables, script_parent) + { let parse_as_node_script = invokes_node && is_node_script_path(&relative); candidates .entry(script_parent.join(relative)) @@ -2089,6 +2308,71 @@ fn script_operand_index(command: &[String]) -> Option<usize> { None } +fn node_command_option_resources(command: &[String]) -> Vec<(usize, &'static str, String)> { + let Some(executable) = command + .first() + .and_then(|argument| Path::new(argument).file_name()) + else { + return Vec::new(); + }; + let executable = executable.to_string_lossy().to_ascii_lowercase(); + if !matches!( + executable.strip_suffix(".exe").unwrap_or(&executable), + "node" | "nodejs" + ) { + return Vec::new(); + } + + let script_index = script_operand_index(command).unwrap_or(command.len()); + let mut result = Vec::new(); + let mut index = 1; + while index < script_index { + let argument = &command[index]; + if argument == "--" { + break; + } + let (option, inline_value) = argument + .split_once('=') + .map_or((argument.as_str(), None), |(option, value)| { + (option, Some(value)) + }); + let field = match option { + "-r" | "--require" | "--import" | "--loader" | "--experimental-loader" => { + Some("OpenCode mcp.node-option-script") + } + "--env-file" => Some("OpenCode mcp.node-option-data"), + "--env-file-if-exists" => Some("OpenCode mcp.node-option-data-optional"), + _ => None, + }; + if let Some(field) = field { + let (operand_index, value) = if let Some(value) = inline_value { + (index, value) + } else if index + 1 < script_index { + (index + 1, command[index + 1].as_str()) + } else { + index += 1; + continue; + }; + let path_like = if is_mcp_node_option_data_field(field) { + !value.is_empty() + } else { + is_explicit_command_path(value) || value.contains('\\') + }; + if path_like { + result.push((operand_index, field, value.to_owned())); + } + index = if inline_value.is_some() { + index + 1 + } else { + index + 2 + }; + continue; + } + index += 1; + } + result +} + fn resource_array( cursor: &mut Cursor<'_>, field: &'static str, @@ -2219,6 +2503,26 @@ fn insert_installed_package_ancestor_resources( result: &mut std::collections::BTreeSet<PathBuf>, resource_path_bytes: &mut usize, field: &'static str, +) -> Result<(), ResourceError> { + insert_installed_package_ancestor_resources_covered( + scope, + home, + declaring, + result, + resource_path_bytes, + field, + None, + ) +} + +fn insert_installed_package_ancestor_resources_covered( + scope: &Path, + home: &Path, + declaring: &Path, + result: &mut std::collections::BTreeSet<PathBuf>, + resource_path_bytes: &mut usize, + field: &'static str, + covered_directory: Option<&Path>, ) -> Result<(), ResourceError> { for ancestor in scope.parent().into_iter().flat_map(Path::ancestors) { if ancestor == Path::new("/") { @@ -2235,6 +2539,9 @@ fn insert_installed_package_ancestor_resources( let candidate = ancestor.join(name); match std::fs::symlink_metadata(&candidate) { Ok(_) => { + if covered_directory.is_some_and(|directory| candidate.starts_with(directory)) { + continue; + } checked(&candidate, field, declaring)?; insert_resource(result, resource_path_bytes, candidate, field, declaring)?; } @@ -2412,7 +2719,7 @@ pub fn select( } else { continue; } // Native package identity remains native. - } else if field == "OpenCode mcp.cwd" || is_mcp_command_path_field(field) { + } else if field == "OpenCode mcp.cwd" || is_mcp_command_resource_field(field) { // Native path.resolve(instance cwd, raw): no HOME or URL expansion. PathBuf::from(&spelling) } else { @@ -2489,7 +2796,7 @@ pub fn select( .join(path); } else if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { path = source.instance_cwd.join(path); - } else if is_mcp_command_path_field(field) { + } else if is_mcp_command_resource_field(field) { path = if let Some(working_dir) = ¤t_mcp_working_dir { working_dir.join(path) } else { @@ -2512,12 +2819,19 @@ pub fn select( if matches!( field, "OpenCode skills.paths" | "OpenCode mcp.cwd" | "OpenCode attention.sounds" - ) || is_mcp_command_path_field(field) + ) || is_mcp_command_resource_field(field) { // Native resolution resolution normalizes .. before deciding // whether the selected resource is supplied by the workspace. path = freshell_platform::opencode_config::lexical_path(&path); } + if is_mcp_node_option_optional_data_field(field) + && std::fs::symlink_metadata(&path) + .is_err_and(|error| error.kind() == std::io::ErrorKind::NotFound) + { + variables.extend(names.into_inner()); + continue; + } let canonical = checked(&path, field, declaring)?; if field == "OpenCode mcp.command" { if let Some(script_field) = shebang_script_field(&canonical) { @@ -2538,6 +2852,109 @@ pub fn select( )?; continue; } + if is_mcp_node_option_script_field(field) || is_mcp_node_option_data_field(field) { + if !canonical.is_file() { + return Err(fail( + field, + "selected Node option resource is not a regular file", + )); + } + if is_mcp_node_option_data_field(field) { + let canonical_is_covered = current_mcp_working_dir + .as_deref() + .filter(|working_dir| canonical.starts_with(working_dir)); + let path_is_covered = + canonical_is_covered.is_some_and(|working_dir| path.starts_with(working_dir)); + if !path_is_covered { + insert_resource( + &mut result, + &mut resource_path_bytes, + path.clone(), + field, + declaring, + )?; + } + if path != canonical && canonical_is_covered.is_none() { + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical, + field, + declaring, + )?; + } + continue; + } + + if path != canonical + && std::fs::symlink_metadata(&path) + .is_ok_and(|metadata| metadata.file_type().is_symlink()) + { + insert_resource( + &mut result, + &mut resource_path_bytes, + path, + field, + declaring, + )?; + } + let command_parent = canonical + .parent() + .ok_or_else(|| fail(field, "Node option has no containing directory"))?; + let dependency_scope = direct_script_dependency_scope(command_parent, home); + let covered_directory = current_mcp_working_dir + .as_deref() + .filter(|working_dir| canonical.starts_with(working_dir)); + if let Some(working_dir) = covered_directory { + insert_installed_package_ancestor_resources_covered( + &canonical, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + Some(working_dir), + )?; + insert_direct_script_dependencies_covered( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencySelection { + runtime: ScriptDependencyRuntime::Node, + covered_directory: Some(working_dir), + }, + )?; + } else { + insert_installed_package_ancestor_resources( + &canonical, + home, + declaring, + &mut result, + &mut resource_path_bytes, + field, + )?; + insert_direct_script_dependencies( + &canonical, + &dependency_scope, + declaring, + &mut result, + &mut resource_path_bytes, + field, + ScriptDependencyRuntime::Node, + )?; + insert_resource( + &mut result, + &mut resource_path_bytes, + canonical, + field, + declaring, + )?; + } + continue; + } if is_mcp_command_path_field(field) { if !canonical.is_file() { return Err(fail( From 2760487325212e3b09f93fc67b531cdb56e5c345 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:54:14 -0700 Subject: [PATCH 409/493] fix(e2e): retry delayed Playwright summaries --- scripts/e2e-cloud.sh | 31 ++++++++++++++++--- .../test/cloud-e2e-logging-receipt.test.sh | 17 ++++++++-- 2 files changed, 41 insertions(+), 7 deletions(-) diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index fdaa2209a..9dffeb905 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -796,13 +796,34 @@ cmd_run() { # Capture to a variable so we can print the full output AND extract a # per-shard summary, even when some shards fail. echo "[e2e-cloud] Fetching logs..." - local log_output - if ! log_output=$(gcloud beta run jobs executions logs read $(gcloud_flags) "$execution_id" 2>&1); then + local log_output="" summary_lines="" display_log_unavailable=false summary_footer_present=false + local summary_attempt + for summary_attempt in 1 2 3 4 5; do + if log_output=$(gcloud beta run jobs executions logs read $(gcloud_flags) "$execution_id" 2>&1); then + summary_lines=$(printf '%s\n' "$log_output" | node "$ROOT/scripts/testing/e2e-cloud-log-summary.mjs") + display_log_unavailable=false + if printf '%s\n' "$summary_lines" | grep -Eq '^[[:space:]]+[0-9]+ (failed|passed|skipped|interrupted)\b'; then + summary_footer_present=true + fi + if $summary_footer_present || $playwright_dry_run; then + break + fi + else + log_output="" + display_log_unavailable=true + fi + if [ "$summary_attempt" -lt 5 ]; then + echo "[e2e-cloud] Waiting for Playwright display-log summary (attempt ${summary_attempt}/5)." >&2 + sleep 3 + fi + done + if $display_log_unavailable; then # This human formatter exposes textPayload only. It remains useful for # display, but never decides whether retry evidence exists; that decision # comes from the required jsonPayload query below. - echo "[e2e-cloud] WARNING: could not read display logs: $log_output" >&2 - log_output="" + echo "[e2e-cloud] WARNING: could not read display logs after five attempts." >&2 + elif ! $summary_footer_present && ! $playwright_dry_run; then + echo "[e2e-cloud] WARNING: Playwright summary footer was not present in display logs after five attempts." >&2 fi # Print logs from every shard. Retry traces are retained as bounded base64 @@ -870,7 +891,7 @@ cmd_run() { # the footer's pass/fail/skip counts. echo "" echo "[e2e-cloud] Per-shard summary:" - printf '%s\n' "$log_output" | node "$ROOT/scripts/testing/e2e-cloud-log-summary.mjs" + printf '%s\n' "$summary_lines" # Check execution status — transient describe errors right after # `execute --wait` are a real flake class, so retry briefly; a PERMANENT diff --git a/scripts/test/cloud-e2e-logging-receipt.test.sh b/scripts/test/cloud-e2e-logging-receipt.test.sh index c17a4539e..ade3c7aa5 100644 --- a/scripts/test/cloud-e2e-logging-receipt.test.sh +++ b/scripts/test/cloud-e2e-logging-receipt.test.sh @@ -27,7 +27,15 @@ case "$*" in exit 0 ;; *"run jobs delete"*) exit 0 ;; *"run jobs execute"*) echo "Execution [exec-json-receipt] has successfully completed." ;; - *"executions logs read"*) echo " 4 passed (12.3s)" ;; + *"executions logs read"*) + count_file="$STUB_CAPTURE/logs.count" + count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" + if [ "${STUB_DELAY_SUMMARY_ONCE:-0}" = 1 ] && [ "$count" -eq 1 ]; then + echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" + else + echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" + echo " 4 passed (12.3s)" + fi ;; *"logging read"*) case "$*" in *'labels."run.googleapis.com/execution_name"="exec-json-receipt"'*) ;; @@ -40,6 +48,7 @@ case "$*" in if [ "${STUB_DELAY_ONCE:-0}" = 1 ] && [ "$count" -eq 1 ]; then echo '[]'; else printf '%s\n' "$STUB_LOGGING_JSON"; fi ;; *"executions describe"*) case "$*" in + *completionTime*) echo "2026-10-13T00:00:01Z" ;; *failedCount*) echo 0 ;; *succeededCount*) echo "${STUB_SUCCEEDED:-2}" ;; esac ;; @@ -93,7 +102,8 @@ if ! grep -Fxq ' --grep=focused smoke' "$WORK/capture/run-env.yaml" \ cat "$WORK/capture/run-env.yaml"; exit 1 fi -OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_ONCE=1 STUB_LOGGING_JSON="$records_no_retry" GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" || { +rm -f "$WORK/capture/logs.count" +OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_ONCE=1 STUB_DELAY_SUMMARY_ONCE=1 STUB_LOGGING_JSON="$records_no_retry" GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" || { echo "FAIL: delayed structured receipt run failed"; echo "$OUT"; exit 1 } if ! grep -q "All tasks completed successfully" <<< "$OUT"; then @@ -102,6 +112,9 @@ fi if [ "$(cat "$WORK/capture/logging.count")" -lt 2 ]; then echo "FAIL: wrapper did not retry delayed Cloud Logging receipt ingestion"; cat "$WORK/capture/gcloud.args"; exit 1 fi +if [ "$(cat "$WORK/capture/logs.count")" -lt 2 ] || ! grep -q "4 passed (12.3s)" <<< "$OUT"; then + echo "FAIL: wrapper did not retry its display-log read until the Playwright footer arrived"; echo "$OUT"; cat "$WORK/capture/gcloud.args"; exit 1 +fi if ! grep -Fq 'labels."run.googleapis.com/execution_name"="exec-json-receipt"' "$WORK/capture/gcloud.args" \ || grep -Fq 'labels.execution_name' "$WORK/capture/gcloud.args"; then echo "FAIL: wrapper did not issue a structured jsonPayload Cloud Logging query scoped to its execution" From b4010c38ddd52c481b20392158566607847e6ea5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:54:22 -0700 Subject: [PATCH 410/493] test(managed): cover direct HOME MCP append --- .../managed-provider-environment-target.ts | 64 ++++++++++++++++++- 1 file changed, 62 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 221031466..086e25eb3 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -150,6 +150,44 @@ process.stdin.on('data', chunk => { }) ` +function directHomeAppendMcpServerSource(outputName: string): string { + const outputUrl = `new URL(${JSON.stringify(`./${outputName}`)}, import.meta.url)` + return `import fs from 'node:fs' +const [receiptPath, expectedNonce] = process.argv.slice(2) +if (!receiptPath || !expectedNonce) process.exit(12) +let appendError +try { fs.appendFileSync(${outputUrl}, '|' + expectedNonce) } +catch (error) { appendError = error.code ?? error.message ?? String(error) } +const output = fs.readFileSync(${outputUrl}, 'utf8') +fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, appendError, output, scriptPath: process.argv[1] })) +let buffered = '' +process.stdin.setEncoding('utf8') +process.stdin.on('data', chunk => { + buffered += chunk + while (true) { + const boundary = buffered.indexOf('\\n') + if (boundary < 0) break + const line = buffered.slice(0, boundary) + buffered = buffered.slice(boundary + 1) + let request + try { request = JSON.parse(line) } catch { continue } + if (!request || !Object.hasOwn(request, 'id')) continue + let result + if (request.method === 'initialize') result = { protocolVersion: request.params?.protocolVersion ?? '2024-11-05', + capabilities: { tools: {} }, serverInfo: { name: 'freshell-home-append-fixture', version: '1.0.0' } } + else if (request.method === 'tools/list') result = { tools: [] } + else if (request.method === 'ping') result = {} + else { + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, + error: { code: -32601, message: 'Method not found' } }) + '\\n') + continue + } + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\\n') + } +}) +` +} + export function resolveManagedProviderEnvironmentTargetMode(args: string[]): ManagedProviderEnvironmentTargetMode { if (args.length === 0) return 'managed-environment' if (args.length === 1 && args[0] === '--checkpoint') return 'code-checkpoint' @@ -376,6 +414,11 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') + const homeAppendScript = path.join(home, `direct-home-${scenario}.mjs`) + const homeAppendOutput = `direct-home-${scenario}.txt` + const homeAppendReceipt = path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json') + await fs.writeFile(homeAppendScript, directHomeAppendMcpServerSource(homeAppendOutput)) + await fs.writeFile(path.join(home, homeAppendOutput), `initial-${nonce}`) await writeJson(path.join(cwd, 'opencode.json'), { agent: { fixtureRelative: { prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } }, mcp: { @@ -391,6 +434,11 @@ export default {id:'managed-environment-tui',async tui(api,options){ command: [directCommand, path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'), nonce], enabled: true, }, + direct_home_append_fixture: { + type: 'local', + command: ['node', homeAppendScript, homeAppendReceipt, nonce], + enabled: true, + }, }, }) } @@ -1305,13 +1353,19 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightDirectMcp.dependency, nonce) assert.equal(preflightDirectMcp.schemaNonce, nonce) assert(preflightDirectMcp.scriptPath.endsWith('/direct-mcp/server.mjs')) + const preflightHomeAppend = await poll('OpenCode starts the direct HOME MCP and it appends its sibling output', async () => + readJson(path.join(selected.workspace, 'preflight/.managed-fixture/direct-home-mcp-receipt.json'))) + assert.equal(preflightHomeAppend.expectedNonce, nonce) + assert.equal(preflightHomeAppend.appendError, undefined, 'the staged HOME-level append target must be writable') + assert.equal(preflightHomeAppend.output, `initial-${nonce}|${nonce}`) + assert(preflightHomeAppend.scriptPath.endsWith('/direct-home-preflight.mjs')) const preflightView = await poll('genuine managed preflight native identity', () => observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) assert.deepEqual(await webChildren(rig), [], 'Managed child must belong to the disposable session host, not the host web process') proof.preflight = { terminalId: created.terminalId, soulId: preflightView.soulId, incarnationId: preflightView.incarnationId, stage: await stageProof(rig, preflightView, nonce), child: preflightChild, mcpServer: preflightMcp, - directMcpServer: preflightDirectMcp } + directMcpServer: preflightDirectMcp, directHomeAppendMcp: preflightHomeAppend } const toolHome = path.join(root, 'tools') await fs.mkdir(toolHome, { mode: 0o700 }) const tool = path.join(toolHome, 'cloudflared') @@ -1416,9 +1470,15 @@ export async function runManagedEnvironmentTarget() { assert.equal(directMcpServer.dependency, nonce) assert.equal(directMcpServer.schemaNonce, nonce) assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.mjs')) + const directHomeAppendMcp = await poll('Cloud OpenCode appends through its direct HOME MCP', async () => + readJson(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json'))) + assert.equal(directHomeAppendMcp.expectedNonce, nonce) + assert.equal(directHomeAppendMcp.appendError, undefined, 'the managed HOME sibling mount must permit append') + assert.equal(directHomeAppendMcp.output, `initial-${nonce}|${nonce}`) + assert(directHomeAppendMcp.scriptPath.endsWith('/direct-home-healthy.mjs')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, - nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer } + nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer, directHomeAppendMcp } await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt } }, From e951fe71ca5453aa35da8418a1d92c21cb807ec1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:00:16 -0700 Subject: [PATCH 411/493] test(managed): place HOME MCP under XDG root --- scripts/testing/managed-provider-environment-target.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 086e25eb3..c92cef0dc 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -344,6 +344,7 @@ async function configure(root: string, nonce: string, candidateSha: string, runI const directMcpDependency = path.join(directMcp, 'node_modules/managed-direct-fixture-dependency') await fs.mkdir(directMcpDependency, { recursive: true, mode: 0o755 }) await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) + await fs.mkdir(path.join(xdg, 'opencode/mcp-servers'), { recursive: true, mode: 0o755 }) await fs.chmod(home, 0o700) await fs.chmod(root, 0o751) // Register a known CLI without a managed adapter through the normal HOME @@ -414,7 +415,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') - const homeAppendScript = path.join(home, `direct-home-${scenario}.mjs`) + const homeAppendScript = path.join(xdg, 'opencode/mcp-servers', `direct-home-${scenario}.mjs`) const homeAppendOutput = `direct-home-${scenario}.txt` const homeAppendReceipt = path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json') await fs.writeFile(homeAppendScript, directHomeAppendMcpServerSource(homeAppendOutput)) From fa66d4297b1a8aea9e4cd050b749eb2115fd5599 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:07:57 -0700 Subject: [PATCH 412/493] test(managed): capture safe startup log events --- scripts/testing/managed-environment-create.ts | 29 ++++++++++++++++++- ...anaged-provider-environment-target.test.ts | 14 +++++++++ 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 1ebdaf5be..061606425 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -115,12 +115,30 @@ export async function readManagedEnvironmentDaemonId(): Promise<string> { } type ManagedCreateOwner = { - web: { capturedOutput(): { stdout: string; stderr: string } } + web: { capturedOutput(): { stdout: string; stderr: string }; info: { logsDir: string } } runtime: { containerLogs(id: string): string; writeIncident(name: string, value: unknown): void; broker?: { receiptIds(): Set<string> } } supervisor: { containerId: string; runtimeRoot?: string } } +const safeLogLevels = new Set(['TRACE', 'DEBUG', 'INFO', 'WARN', 'ERROR']) + +export function summarizeRustServerLogEvents(raw: string): Array<{ level: string; event: string }> { + const boundedTail = raw.slice(-512 * 1024) + const summary: Array<{ level: string; event: string }> = [] + for (const line of boundedTail.split('\n').slice(-256)) { + if (line.length > 64 * 1024) continue + let value: unknown + try { value = JSON.parse(line) } catch { continue } + if (!value || typeof value !== 'object') continue + const record = value as Record<string, unknown> + if (typeof record.event !== 'string' || !/^[a-z][a-z0-9_.:-]{0,127}$/.test(record.event) + || typeof record.level !== 'string' || !safeLogLevels.has(record.level)) continue + summary.push({ level: record.level, event: record.event }) + } + return summary.slice(-32) +} + const failureReasons = [ 'OneCLI grant is unavailable', 'prepared private source unreadable', @@ -190,6 +208,14 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, ?? (filesystemLabel && osError !== undefined ? `${filesystemLabel} filesystem error` : 'UNKNOWN') const code = ErrorCode.safeParse(response.code) const web = owner.web.capturedOutput() + let serverLogEvents: Array<{ level: string; event: string }> = [] + try { + const logPath = path.join(owner.web.info.logsDir, 'rust-server.jsonl') + const metadata = fs.statSync(logPath) + if (metadata.isFile() && metadata.size <= 4 * 1024 * 1024) { + serverLogEvents = summarizeRustServerLogEvents(fs.readFileSync(logPath, 'utf8')) + } + } catch { /* Missing or oversized logs remain unavailable. */ } let supervisor: Record<string, string> | 'UNKNOWN' = 'UNKNOWN' let sessionHost: Record<string, unknown> | 'UNKNOWN' = 'UNKNOWN' try { @@ -239,6 +265,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, } catch { /* An absent owned supervisor leaves its observation unknown. */ } owner.runtime.writeIncident('managed-environment-create-failure', { severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), + serverLogEvents, server: { source: 'owned-server-typed-reply', capturedOutputAvailable: Boolean(web.stdout || web.stderr) }, supervisor, sessionHost, }) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 6ea1d755a..2b8e5a895 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -3,6 +3,7 @@ import fs from 'node:fs/promises' import { createConnection, createServer, type AddressInfo } from 'node:net' import path from 'node:path' import { describe, expect, it } from 'vitest' +import { summarizeRustServerLogEvents } from '../../../../scripts/testing/managed-environment-create.js' import { checkpointTcpForwarderSource, externalTargetRead, @@ -22,6 +23,19 @@ async function listenOnLoopback(server: ReturnType<typeof createServer>): Promis } describe('managed provider environment target mode', () => { + it('retains only bounded event names and levels from server logs', () => { + const logs = [ + JSON.stringify({ level: 'ERROR', event: 'provider.opencode.resource.failed', msg: 'PRIVATE_CONFIG_SENTINEL', token: 'PRIVATE_TOKEN_SENTINEL' }), + JSON.stringify({ level: 'WARN', event: 'invalid event name', msg: 'PRIVATE_MESSAGE_SENTINEL' }), + '{invalid json', + ].join('\n') + + const summary = summarizeRustServerLogEvents(logs) + + expect(summary).toEqual([{ level: 'ERROR', event: 'provider.opencode.resource.failed' }]) + expect(JSON.stringify(summary)).not.toContain('PRIVATE_') + }) + it('waits for the target Retry-After before retrying a rate-limited control read', async () => { const responses = [ new Response('rate limited', { status: 429, headers: { 'retry-after': '2' } }), From a42a32d1f7a68adbf44182ba678709c719570db4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:13:18 -0700 Subject: [PATCH 413/493] test(managed): summarize safe server log context --- scripts/testing/managed-environment-create.ts | 23 +++++++++++++++---- ...anaged-provider-environment-target.test.ts | 11 ++++++--- 2 files changed, 27 insertions(+), 7 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 061606425..5bd87b9e2 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -123,18 +123,33 @@ type ManagedCreateOwner = { const safeLogLevels = new Set(['TRACE', 'DEBUG', 'INFO', 'WARN', 'ERROR']) -export function summarizeRustServerLogEvents(raw: string): Array<{ level: string; event: string }> { +export function summarizeRustServerLogEvents(raw: string): Array<{ + level: string + event?: string + target?: string + fields: string[] +}> { const boundedTail = raw.slice(-512 * 1024) - const summary: Array<{ level: string; event: string }> = [] + const summary: Array<{ level: string; event?: string; target?: string; fields: string[] }> = [] for (const line of boundedTail.split('\n').slice(-256)) { if (line.length > 64 * 1024) continue let value: unknown try { value = JSON.parse(line) } catch { continue } if (!value || typeof value !== 'object') continue const record = value as Record<string, unknown> - if (typeof record.event !== 'string' || !/^[a-z][a-z0-9_.:-]{0,127}$/.test(record.event) + const event = typeof record.event === 'string' && /^[a-z][a-z0-9_.:-]{0,127}$/.test(record.event) + ? record.event : undefined + const target = typeof record.target === 'string' && /^[a-zA-Z0-9_:.$-]{1,160}$/.test(record.target) + ? record.target : undefined + if ((typeof record.event === 'string' && event === undefined) || (!event && !target) || typeof record.level !== 'string' || !safeLogLevels.has(record.level)) continue - summary.push({ level: record.level, event: record.event }) + const sensitiveFieldName = /token|secret|auth|prompt|content|payload|message|text|config|password|credential|key|path|url|body/i + const fields = Object.keys(record) + .filter(key => !['ts', 'level', 'target', 'app_version', 'server_pid', 'event', 'msg'].includes(key) + && /^[a-zA-Z][a-zA-Z0-9_]{0,63}$/.test(key) && !sensitiveFieldName.test(key)) + .sort() + .slice(0, 24) + summary.push({ level: record.level, ...(event ? { event } : {}), ...(target ? { target } : {}), fields }) } return summary.slice(-32) } diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 2b8e5a895..8e8bff7c0 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -25,14 +25,19 @@ async function listenOnLoopback(server: ReturnType<typeof createServer>): Promis describe('managed provider environment target mode', () => { it('retains only bounded event names and levels from server logs', () => { const logs = [ - JSON.stringify({ level: 'ERROR', event: 'provider.opencode.resource.failed', msg: 'PRIVATE_CONFIG_SENTINEL', token: 'PRIVATE_TOKEN_SENTINEL' }), - JSON.stringify({ level: 'WARN', event: 'invalid event name', msg: 'PRIVATE_MESSAGE_SENTINEL' }), + JSON.stringify({ level: 'ERROR', event: 'provider.opencode.resource.failed', target: 'freshell_server::managed_runtime', + stage: 'prepare', msg: 'PRIVATE_CONFIG_SENTINEL', token: 'PRIVATE_TOKEN_SENTINEL' }), + JSON.stringify({ level: 'ERROR', target: 'freshell_server::managed_runtime', stage: 'activate', code: 'PTY', msg: 'PRIVATE_ERROR_SENTINEL' }), + JSON.stringify({ level: 'WARN', event: 'invalid event name', target: 'freshell_server::managed_runtime', msg: 'PRIVATE_MESSAGE_SENTINEL' }), '{invalid json', ].join('\n') const summary = summarizeRustServerLogEvents(logs) - expect(summary).toEqual([{ level: 'ERROR', event: 'provider.opencode.resource.failed' }]) + expect(summary).toEqual([ + { level: 'ERROR', event: 'provider.opencode.resource.failed', target: 'freshell_server::managed_runtime', fields: ['stage'] }, + { level: 'ERROR', target: 'freshell_server::managed_runtime', fields: ['code', 'stage'] }, + ]) expect(JSON.stringify(summary)).not.toContain('PRIVATE_') }) From 05fa63af685b3170cd3304ca2bdc92a85968f198 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:21:10 -0700 Subject: [PATCH 414/493] test(managed): retain safe preparation failure cause --- scripts/testing/managed-environment-create.ts | 10 +++++ ...anaged-provider-environment-target.test.ts | 38 ++++++++++++++++++- 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 5bd87b9e2..87ec3a9aa 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -123,6 +123,13 @@ type ManagedCreateOwner = { const safeLogLevels = new Set(['TRACE', 'DEBUG', 'INFO', 'WARN', 'ERROR']) +export function summarizeProviderPreparationFailure( + value: ErrorMessage['providerPreparationFailure'], +): Pick<NonNullable<ErrorMessage['providerPreparationFailure']>, 'code' | 'stage' | 'operation' | 'cause'> | undefined { + if (!value) return undefined + return { code: value.code, stage: value.stage, operation: value.operation, cause: value.cause } +} + export function summarizeRustServerLogEvents(raw: string): Array<{ level: string event?: string @@ -281,6 +288,9 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, owner.runtime.writeIncident('managed-environment-create-failure', { severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), serverLogEvents, + ...(response.providerPreparationFailure + ? { providerPreparationFailure: summarizeProviderPreparationFailure(response.providerPreparationFailure) } + : {}), server: { source: 'owned-server-typed-reply', capturedOutputAvailable: Boolean(web.stdout || web.stderr) }, supervisor, sessionHost, }) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 8e8bff7c0..7223855cb 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -3,7 +3,10 @@ import fs from 'node:fs/promises' import { createConnection, createServer, type AddressInfo } from 'node:net' import path from 'node:path' import { describe, expect, it } from 'vitest' -import { summarizeRustServerLogEvents } from '../../../../scripts/testing/managed-environment-create.js' +import { + assertManagedTerminalCreated, + summarizeRustServerLogEvents, +} from '../../../../scripts/testing/managed-environment-create.js' import { checkpointTcpForwarderSource, externalTargetRead, @@ -23,6 +26,39 @@ async function listenOnLoopback(server: ReturnType<typeof createServer>): Promis } describe('managed provider environment target mode', () => { + it('records typed provider preparation causes without persisting paths', () => { + const incidents: unknown[] = [] + const owner = { + web: { capturedOutput: () => ({ stdout: '', stderr: '' }), info: { logsDir: '/nonexistent/managed-test' } }, + runtime: { containerLogs: () => '', writeIncident: (_name: string, value: unknown) => incidents.push(value) }, + supervisor: { containerId: 'a'.repeat(64) }, + } + const response = { + type: 'error' as const, + code: 'PTY_SPAWN_FAILED' as const, + timestamp: new Date().toISOString(), + message: 'managed runtime launch failed', + providerPreparationFailure: { + code: 'SELECTED_REFERENCE_UNAVAILABLE' as const, + stage: 'configuration' as const, + operation: 'resolve' as const, + cause: 'permission_denied' as const, + workspace: '/private/workspace', + declaringConfig: '/private/home/opencode.json', + selectedPath: '/private/home/mcp-server.mjs', + }, + } + + expect(() => assertManagedTerminalCreated(owner, response)).toThrow() + expect(incidents).toHaveLength(1) + expect(incidents[0]).toMatchObject({ + providerPreparationFailure: { + code: 'SELECTED_REFERENCE_UNAVAILABLE', stage: 'configuration', operation: 'resolve', cause: 'permission_denied', + }, + }) + expect(JSON.stringify(incidents[0])).not.toContain('/private/') + }) + it('retains only bounded event names and levels from server logs', () => { const logs = [ JSON.stringify({ level: 'ERROR', event: 'provider.opencode.resource.failed', target: 'freshell_server::managed_runtime', From 379cd4f848f8ab68d023a937d718b78efee7fc95 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:27:19 -0700 Subject: [PATCH 415/493] test(managed): classify safe runtime launch errors --- scripts/testing/managed-environment-create.ts | 21 +++++++++++++++++++ ...anaged-provider-environment-target.test.ts | 3 ++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 87ec3a9aa..7a890968a 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -130,6 +130,25 @@ export function summarizeProviderPreparationFailure( return { code: value.code, stage: value.stage, operation: value.operation, cause: value.cause } } +const runtimeErrorCodes = [ + 'InvalidRequest', 'ProviderInputsChanged', 'ProtocolVersionMismatch', 'UnauthorizedRole', 'FrameTooLarge', + 'RequestIdConflict', 'RegistryBusy', 'RegistryFailure', 'StaleControlEpoch', 'StaleIntentRevision', + 'LossCertificationBlocked', 'IncidentPersistenceFailed', 'NoticeNotFound', 'MigrationBlocked', 'RepairBlocked', + 'UnknownSoul', 'UnknownIncarnation', 'OwnershipMismatch', 'BackendUnavailable', 'TerminationUnconfirmed', + 'HostUnreachable', 'HostAuthenticationFailed', 'StaleExecutionGrant', 'InvalidRuntimeLimits', 'BlockedResource', + 'CommandAmbiguous', 'OutputCursorExpired', 'UnsupportedWorkload', 'UnsupportedOperation', + 'OperationImplementationUnavailable', 'FaultInjected', 'RecoveryBlocked', 'RecoveryImplementationUnavailable', + 'RecoveryWrongIdentity', 'RecoveryRetryBudget', 'RecoveryInProgress', 'RecoveryStopUnconfirmed', +] as const + +export function summarizeRuntimeErrorCode(message: string): typeof runtimeErrorCodes[number] | undefined { + const prefix = 'runtime supervisor rejected request: ' + const index = message.indexOf(prefix) + if (index < 0) return undefined + const rejection = message.slice(index + prefix.length) + return runtimeErrorCodes.find(code => rejection.startsWith(`${code}:`)) +} + export function summarizeRustServerLogEvents(raw: string): Array<{ level: string event?: string @@ -226,6 +245,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, const filesystemLabel = filesystemLabels.find(value => response.message.includes(`${value}:`)) const errno = response.message.match(/\(os error (\d{1,4})\)/) const osError = errno ? Number(errno[1]) : undefined + const runtimeErrorCode = summarizeRuntimeErrorCode(response.message) const reason = failureReasons.find(value => response.message.includes(value)) ?? inputReason ?? (filesystemLabel && osError !== undefined ? `${filesystemLabel} filesystem error` : 'UNKNOWN') const code = ErrorCode.safeParse(response.code) @@ -288,6 +308,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, owner.runtime.writeIncident('managed-environment-create-failure', { severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), serverLogEvents, + ...(runtimeErrorCode ? { runtimeErrorCode } : {}), ...(response.providerPreparationFailure ? { providerPreparationFailure: summarizeProviderPreparationFailure(response.providerPreparationFailure) } : {}), diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 7223855cb..6a33b21c2 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -37,7 +37,7 @@ describe('managed provider environment target mode', () => { type: 'error' as const, code: 'PTY_SPAWN_FAILED' as const, timestamp: new Date().toISOString(), - message: 'managed runtime launch failed', + message: 'managed runtime launch failed: runtime supervisor rejected request: BackendUnavailable: /private/runtime/path', providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE' as const, stage: 'configuration' as const, @@ -52,6 +52,7 @@ describe('managed provider environment target mode', () => { expect(() => assertManagedTerminalCreated(owner, response)).toThrow() expect(incidents).toHaveLength(1) expect(incidents[0]).toMatchObject({ + runtimeErrorCode: 'BackendUnavailable', providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE', stage: 'configuration', operation: 'resolve', cause: 'permission_denied', }, From 46441db2a9917c8fe083f5f1d16b021a829310f2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:32:45 -0700 Subject: [PATCH 416/493] test(managed): classify runtime client failures --- scripts/testing/managed-environment-create.ts | 11 +++++++++++ .../managed-provider-environment-target.test.ts | 1 + 2 files changed, 12 insertions(+) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 7a890968a..a8f060ae1 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -149,6 +149,15 @@ export function summarizeRuntimeErrorCode(message: string): typeof runtimeErrorC return runtimeErrorCodes.find(code => rejection.startsWith(`${code}:`)) } +export function summarizeRuntimeErrorClass(message: string): + 'supervisor_io' | 'supervisor_protocol' | 'supervisor_rejected' | 'supervisor_unexpected_reply' | undefined { + if (message.includes('runtime supervisor I/O failed: ')) return 'supervisor_io' + if (message.includes('runtime supervisor protocol failed: ')) return 'supervisor_protocol' + if (message.includes('runtime supervisor rejected request: ')) return 'supervisor_rejected' + if (message.includes('runtime supervisor returned an unexpected result')) return 'supervisor_unexpected_reply' + return undefined +} + export function summarizeRustServerLogEvents(raw: string): Array<{ level: string event?: string @@ -246,6 +255,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, const errno = response.message.match(/\(os error (\d{1,4})\)/) const osError = errno ? Number(errno[1]) : undefined const runtimeErrorCode = summarizeRuntimeErrorCode(response.message) + const runtimeErrorClass = summarizeRuntimeErrorClass(response.message) const reason = failureReasons.find(value => response.message.includes(value)) ?? inputReason ?? (filesystemLabel && osError !== undefined ? `${filesystemLabel} filesystem error` : 'UNKNOWN') const code = ErrorCode.safeParse(response.code) @@ -309,6 +319,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), serverLogEvents, ...(runtimeErrorCode ? { runtimeErrorCode } : {}), + ...(runtimeErrorClass ? { runtimeErrorClass } : {}), ...(response.providerPreparationFailure ? { providerPreparationFailure: summarizeProviderPreparationFailure(response.providerPreparationFailure) } : {}), diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 6a33b21c2..f10c0e783 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -53,6 +53,7 @@ describe('managed provider environment target mode', () => { expect(incidents).toHaveLength(1) expect(incidents[0]).toMatchObject({ runtimeErrorCode: 'BackendUnavailable', + runtimeErrorClass: 'supervisor_rejected', providerPreparationFailure: { code: 'SELECTED_REFERENCE_UNAVAILABLE', stage: 'configuration', operation: 'resolve', cause: 'permission_denied', }, From 652b2cebb939fdd9b4d72363f567972b13756990 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:39:07 -0700 Subject: [PATCH 417/493] test(managed): seed append target beside HOME MCP --- .../managed-provider-environment-target.ts | 19 +++++++++++++------ ...anaged-provider-environment-target.test.ts | 12 ++++++++++++ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index c92cef0dc..bdc126172 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -334,6 +334,15 @@ export function managedEnvironmentFixtureLayout(root: string) { return { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } } +export function managedHomeAppendFixturePaths(xdg: string, cwd: string, scenario: string) { + const script = path.join(xdg, 'opencode/mcp-servers', `direct-home-${scenario}.mjs`) + return { + script, + output: path.join(path.dirname(script), `direct-home-${scenario}.txt`), + receipt: path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json'), + } +} + async function configure(root: string, nonce: string, candidateSha: string, runId: string) { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg') const { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } = managedEnvironmentFixtureLayout(root) @@ -415,11 +424,9 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') - const homeAppendScript = path.join(xdg, 'opencode/mcp-servers', `direct-home-${scenario}.mjs`) - const homeAppendOutput = `direct-home-${scenario}.txt` - const homeAppendReceipt = path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json') - await fs.writeFile(homeAppendScript, directHomeAppendMcpServerSource(homeAppendOutput)) - await fs.writeFile(path.join(home, homeAppendOutput), `initial-${nonce}`) + const homeAppend = managedHomeAppendFixturePaths(xdg, cwd, scenario) + await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) + await fs.writeFile(homeAppend.output, `initial-${nonce}`) await writeJson(path.join(cwd, 'opencode.json'), { agent: { fixtureRelative: { prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } }, mcp: { @@ -437,7 +444,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ }, direct_home_append_fixture: { type: 'local', - command: ['node', homeAppendScript, homeAppendReceipt, nonce], + command: ['node', homeAppend.script, homeAppend.receipt, nonce], enabled: true, }, }, diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index f10c0e783..933da2933 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -10,6 +10,7 @@ import { import { checkpointTcpForwarderSource, externalTargetRead, + managedHomeAppendFixturePaths, managedEnvironmentFixtureLayout, processIdentity, resolveManagedProviderEnvironmentTargetMode, @@ -113,6 +114,17 @@ describe('managed provider environment target mode', () => { expect(fixture.mcpParentDependency).toBe(path.join(fixture.resources, 'node_modules/managed-fixture-mcp-dependency')) }) + it('seeds the direct HOME append file beside the XDG MCP script it resolves from', () => { + const xdg = '/tmp/managed-environment-fixture/home/fixture-xdg' + const cwd = '/tmp/managed-environment-fixture/workspace/preflight' + const paths = managedHomeAppendFixturePaths(xdg, cwd, 'preflight') + + expect(paths.script).toBe(path.join(xdg, 'opencode/mcp-servers/direct-home-preflight.mjs')) + expect(paths.output).toBe(path.join(path.dirname(paths.script), 'direct-home-preflight.txt')) + expect(path.dirname(paths.output)).toBe(path.dirname(paths.script)) + expect(paths.receipt).toBe(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json')) + }) + it('forwards one fixed TCP target while preserving streamed client data', async () => { const upstream = createServer(socket => { socket.on('data', chunk => socket.write(chunk)) From 9f56565e41539f6ee27d8ac0188be8f0a60a1540 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:17:44 -0700 Subject: [PATCH 418/493] fix(managed): allow exact static MCP output writes --- .../src/provider_inputs.rs | 41 ++++- .../managed_mcp_capability/amplifier_stage.rs | 1 + .../managed_mcp_capability/ordinary_stage.rs | 1 + .../managed_mcp_capability/source_stage.rs | 46 +++++ .../opencode_resources.rs | 165 +++++++++++++----- .../managed_provider_references/sources.rs | 13 +- crates/freshell-session-host/src/main.rs | 5 + .../src/backend/prepared_inputs.rs | 32 +++- .../src/backend/provider_mountpoints.rs | 1 + .../managed-provider-environment-target.ts | 8 +- scripts/testing/runtime-sandbox.ts | 6 +- ...anaged-provider-environment-target.test.ts | 10 +- .../tooling/testing/runtime-sandbox.test.ts | 3 +- 13 files changed, 269 insertions(+), 63 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 5895fed20..2f4ffeca1 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -173,6 +173,14 @@ pub struct ProviderResourceMount { pub logical_destination: String, pub effective_destination: String, pub kind: ProviderResourceKind, + /// Exact files are read-only unless a bounded static writer was selected. + /// Directories retain their existing writable resource behavior. + #[serde(default, skip_serializing_if = "is_false")] + pub writable: bool, +} + +fn is_false(value: &bool) -> bool { + !value } #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -314,7 +322,8 @@ impl PreparedProviderInputs { } for mount in &self.resource_mounts { let file = mount.kind == ProviderResourceKind::File; - if !unit_path(&mount.stage_relative_path, ".freshell/resource-units", file) + if (mount.writable && !file) + || !unit_path(&mount.stage_relative_path, ".freshell/resource-units", file) || !unit_path( &mount.provider_relative_path, ".freshell/provider-roots", @@ -710,6 +719,35 @@ mod tests { assert!(value.validate(Some(&capability())).is_err()); } #[test] + fn provider_resource_mount_write_access_is_opt_in_and_file_only() { + let mut value = serde_json::to_value(prepared()).unwrap(); + value["resourceMounts"] = serde_json::json!([{ + "stageRelativePath": ".freshell/resource-units/unit-0/source", + "providerRelativePath": ".freshell/provider-roots/unit-0/source", + "logicalDestination": "/fixture/output.txt", + "effectiveDestination": "/fixture/output.txt", + "kind": "file", + "writable": true + }]); + let writable: PreparedProviderInputs = serde_json::from_value(value.clone()) + .expect("new descriptors may opt one exact file resource into writes"); + writable.validate(Some(&capability())).unwrap(); + + let legacy = value["resourceMounts"][0] + .as_object_mut() + .unwrap() + .remove("writable"); + assert_eq!(legacy, Some(serde_json::json!(true))); + let restored: PreparedProviderInputs = serde_json::from_value(value.clone()).unwrap(); + let encoded = serde_json::to_value(restored).unwrap(); + assert!(encoded["resourceMounts"][0].get("writable").is_none()); + + value["resourceMounts"][0]["kind"] = serde_json::json!("directory"); + let writable_directory: PreparedProviderInputs = + serde_json::from_value(value).expect("write intent is structurally valid"); + assert!(writable_directory.validate(Some(&capability())).is_err()); + } + #[test] fn provider_prepared_mounts_preserve_home_and_control_parents() { for destination in [ "/home/freshell/provider", @@ -723,6 +761,7 @@ mod tests { logical_destination: destination.into(), effective_destination: destination.into(), kind: ProviderResourceKind::Directory, + writable: false, }); assert!( value.validate(Some(&capability())).is_err(), diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs index ae33cad0f..844c4ffdc 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs @@ -142,6 +142,7 @@ pub(super) fn prepare( logical_destination: "/home/freshell/provider/.freshell/resources".into(), effective_destination: "/home/freshell/provider/.freshell/resources".into(), kind: ProviderResourceKind::Directory, + writable: false, }); Ok(prepared) } diff --git a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs index 86466e0c9..b4d2f6a27 100644 --- a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs @@ -143,6 +143,7 @@ pub(super) fn prepare_selected<M: serde::Serialize>( .ok_or("Codex resource destination is not UTF-8")? .into(), kind, + writable: false, }); } } diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 83fb45181..86d42c9d9 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1574,6 +1574,39 @@ process.stdout.write(require('./helper.js')) .unwrap_or_else(|_| copy_output.clone()) )); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + for output in [&append_output, &append_url_output] { + let canonical = output.canonicalize().unwrap(); + let mounts = units + .iter() + .filter(|(source, mount)| { + source == &canonical + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!( + !mounts.is_empty(), + "existing static output must be an exact file mount" + ); + assert!(mounts.iter().all(|(_, mount)| mount.writable)); + } + let copy_source_mounts = units + .iter() + .filter(|(source, mount)| { + source == ©_source.canonicalize().unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!(!copy_source_mounts.is_empty()); + assert!(copy_source_mounts.iter().all(|(_, mount)| !mount.writable)); + let output = execute_node_with_home_hidden(owned.path(), &home, &provider_script); assert!(output.status.success(), "{:?}", output.stderr); let provider_script_directory = std::fs::canonicalize(&provider_script) @@ -3304,6 +3337,7 @@ process.stdout.write(require('./helper.js')) logical_destination: package.to_string_lossy().into_owned(), effective_destination: package.to_string_lossy().into_owned(), kind: freshell_runtime_protocol::ProviderResourceKind::Directory, + writable: false, }; let mut units = vec![(package.clone(), mount)]; @@ -3504,6 +3538,7 @@ process.stdout.write(require('./helper.js')) logical_destination: "/tmp/external".into(), effective_destination: "/tmp/external".into(), kind: freshell_runtime_protocol::ProviderResourceKind::Directory, + writable: false, }, )]; let capture = Capture::default(); @@ -4039,6 +4074,7 @@ process.stdout.write(require('./helper.js')) logical_destination: "/home/freshell/provider/.config/opencode".into(), effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, + writable: false, }; let mut current = vec![( "/synthetic/current-opencode-root".into(), @@ -4048,6 +4084,7 @@ process.stdout.write(require('./helper.js')) logical_destination: previous.logical_destination.clone(), effective_destination: previous.effective_destination.clone(), kind: ProviderResourceKind::Directory, + writable: false, }, )]; let retained = @@ -4082,6 +4119,7 @@ process.stdout.write(require('./helper.js')) logical_destination: destination.into(), effective_destination: destination.into(), kind: ProviderResourceKind::Directory, + writable: false, }; for (current_destination, previous_destination) in [ @@ -4130,6 +4168,7 @@ process.stdout.write(require('./helper.js')) logical_destination: destination.into(), effective_destination: destination.into(), kind: ProviderResourceKind::Directory, + writable: false, }; let mut aliases = vec![(source, mount("/workspace/current"))]; let prior = mount("/home/freshell/provider/.config/opencode"); @@ -4175,6 +4214,10 @@ pub(super) fn resource_units( use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; use sha2::{Digest, Sha256}; let mut roots = BTreeMap::new(); + let writable_resources = plan + .iter() + .flat_map(|source| source.configured_writable_resources.iter().cloned()) + .collect::<std::collections::BTreeSet<_>>(); for root in accepted_roots.unwrap_or_default() { if root.role == crate::managed_provider_references::sources::DiscoveryRole::Unknown { return Err("selected resource discovery role is unrecorded".into()); @@ -4248,6 +4291,8 @@ pub(super) fn resource_units( } else { ProviderResourceKind::Directory }; + let writable = + kind == ProviderResourceKind::File && writable_resources.contains(&canonical); let name = if kind == ProviderResourceKind::File { format!("{name}/source") } else { @@ -4284,6 +4329,7 @@ pub(super) fn resource_units( .ok_or("resource destination is not UTF-8")? .into(), kind, + writable, }, )); } diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 7035e65bb..abf04bead 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1299,10 +1299,10 @@ fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static fn node_relative_resource_specifiers( parent: &Path, source: &str, -) -> Result<std::collections::BTreeSet<String>, &'static str> { +) -> Result<std::collections::BTreeMap<String, bool>, &'static str> { let tokens = javascript_tokens(source)?; let output_destinations = node_output_destination_mask(&tokens)?; - let mut result = std::collections::BTreeSet::new(); + let mut result = std::collections::BTreeMap::<String, bool>::new(); for index in 0..tokens.len() { if index >= 2 && matches!(tokens.get(index - 2), Some(JavaScriptToken::Identifier(_))) @@ -1362,7 +1362,10 @@ fn node_relative_resource_specifiers( if safe && !relative.as_os_str().is_empty() { let relative = relative.to_string_lossy().into_owned(); if !output_destinations[index - 2] || parent.join(&relative).exists() { - result.insert(relative); + result + .entry(relative) + .and_modify(|writable| *writable |= output_destinations[index - 2]) + .or_insert(output_destinations[index - 2]); } } } @@ -1410,7 +1413,10 @@ fn node_relative_resource_specifiers( && !specifier.contains(['?', '#', '\\', '$']) && (!output_destinations[index] || parent.join(specifier).exists()) { - result.insert(specifier.clone()); + result + .entry(specifier.clone()) + .and_modify(|writable| *writable |= output_destinations[index]) + .or_insert(output_destinations[index]); } } Ok(result) @@ -1837,6 +1843,13 @@ struct ScriptDependencySelection<'a> { struct ScriptDependencyPath { path: PathBuf, parse_as_script: bool, + writable: bool, +} + +struct DirectScriptResources<'a> { + paths: &'a mut std::collections::BTreeSet<PathBuf>, + path_bytes: &'a mut usize, + writable_files: &'a mut std::collections::BTreeSet<PathBuf>, } fn direct_script_dependency_paths( @@ -1847,32 +1860,36 @@ fn direct_script_dependency_paths( ) -> Result<Vec<ScriptDependencyPath>, &'static str> { match runtime { ScriptDependencyRuntime::Node => { - let mut result = std::collections::BTreeMap::<PathBuf, bool>::new(); + let mut result = std::collections::BTreeMap::<PathBuf, (bool, bool)>::new(); { - let mut record = |path: PathBuf, imported_module: bool| { + let mut record = |path: PathBuf, imported_module: bool, writable: bool| { let parse_as_script = is_script_dependency_source(&path, runtime) || (imported_module && path.extension().is_none()); result .entry(path) - .and_modify(|selected| *selected |= parse_as_script) - .or_insert(parse_as_script); + .and_modify(|selected| { + selected.0 |= parse_as_script; + selected.1 |= writable; + }) + .or_insert((parse_as_script, writable)); }; for specifier in node_relative_module_specifiers(source)? { let files = resolve_node_relative_module(parent, &specifier)? .ok_or("direct MCP Node relative module is missing")?; for file in files { - record(file, true); + record(file, true, false); } } - for specifier in node_relative_resource_specifiers(parent, source)? { - record(parent.join(specifier), false); + for (specifier, writable) in node_relative_resource_specifiers(parent, source)? { + record(parent.join(specifier), false, writable); } } Ok(result .into_iter() - .map(|(path, parse_as_script)| ScriptDependencyPath { + .map(|(path, (parse_as_script, writable))| ScriptDependencyPath { path, parse_as_script, + writable, }) .collect()) } @@ -1889,6 +1906,7 @@ fn direct_script_dependency_paths( .map(|path| ScriptDependencyPath { parse_as_script: is_script_dependency_source(&path, runtime), path, + writable: false, }) .collect()), ScriptDependencyRuntime::Ruby => ruby_relative_require_specifiers(source)? @@ -1899,6 +1917,7 @@ fn direct_script_dependency_paths( Ok(ScriptDependencyPath { parse_as_script: is_script_dependency_source(&path, runtime), path, + writable: false, }) }) .collect(), @@ -1923,8 +1942,7 @@ fn insert_direct_script_dependencies( script: &Path, scope: &Path, declaring: &Path, - result: &mut std::collections::BTreeSet<PathBuf>, - resource_path_bytes: &mut usize, + resources: DirectScriptResources<'_>, field: &'static str, runtime: ScriptDependencyRuntime, ) -> Result<(), ResourceError> { @@ -1932,8 +1950,7 @@ fn insert_direct_script_dependencies( script, scope, declaring, - result, - resource_path_bytes, + resources, field, ScriptDependencySelection { runtime, @@ -1946,8 +1963,7 @@ fn insert_direct_script_dependencies_covered( script: &Path, scope: &Path, declaring: &Path, - result: &mut std::collections::BTreeSet<PathBuf>, - resource_path_bytes: &mut usize, + resources: DirectScriptResources<'_>, field: &'static str, selection: ScriptDependencySelection<'_>, ) -> Result<(), ResourceError> { @@ -2007,6 +2023,9 @@ fn insert_direct_script_dependencies_covered( "direct MCP dependency escapes its selected script scope", )); } + if candidate.writable && dependency_is_file { + resources.writable_files.insert(dependency.clone()); + } let alias = freshell_platform::opencode_config::lexical_path(&candidate.path); if alias != dependency { if !alias.starts_with(scope) { @@ -2023,7 +2042,13 @@ fn insert_direct_script_dependencies_covered( .covered_directory .is_some_and(|directory| alias.starts_with(directory)) { - insert_resource(result, resource_path_bytes, alias, field, declaring)?; + insert_resource( + &mut *resources.paths, + &mut *resources.path_bytes, + alias, + field, + declaring, + )?; } } if !selection @@ -2031,8 +2056,8 @@ fn insert_direct_script_dependencies_covered( .is_some_and(|directory| dependency.starts_with(directory)) { insert_resource( - result, - resource_path_bytes, + &mut *resources.paths, + &mut *resources.path_bytes, dependency.clone(), field, declaring, @@ -2052,6 +2077,7 @@ fn insert_shell_script_dependencies( declaring: &Path, result: &mut std::collections::BTreeSet<PathBuf>, resource_path_bytes: &mut usize, + writable_resources: &mut std::collections::BTreeSet<PathBuf>, field: &'static str, ) -> Result<(), ResourceError> { const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; @@ -2190,8 +2216,11 @@ fn insert_shell_script_dependencies( &dependency, scope, declaring, - result, - resource_path_bytes, + DirectScriptResources { + paths: result, + path_bytes: resource_path_bytes, + writable_files: writable_resources, + }, field, ScriptDependencyRuntime::Node, )?; @@ -2425,6 +2454,12 @@ pub struct ResourceError { pub field: &'static str, pub cause: &'static str, } + +pub struct ResourceSelection { + pub resources: Vec<PathBuf>, + pub variables: Vec<String>, + pub writable_resources: std::collections::BTreeSet<PathBuf>, +} fn checked(path: &Path, field: &'static str, _source: &Path) -> Result<PathBuf, ResourceError> { let fail = |cause| ResourceError { field, cause }; let canonical = std::fs::canonicalize(path) @@ -2568,11 +2603,12 @@ pub fn select( home: &Path, global: Option<&Path>, lookup: &dyn Fn(&str) -> Option<String>, -) -> Result<(Vec<PathBuf>, Vec<String>), ResourceError> { +) -> Result<ResourceSelection, ResourceError> { let declaring = &source.source.logical_path; let fail = |field: &'static str, cause: &'static str| ResourceError { field, cause }; let mut result = std::collections::BTreeSet::new(); let mut resource_path_bytes = 0usize; + let mut writable_resources = std::collections::BTreeSet::new(); let mut variables = std::collections::BTreeSet::new(); let raw = source.references.unexpanded_source(); let skip_owned_freshell_mcp = @@ -2919,8 +2955,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencySelection { runtime: ScriptDependencyRuntime::Node, @@ -2940,8 +2979,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Node, )?; @@ -3005,8 +3047,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Node, )?; @@ -3094,8 +3139,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Node, )?; @@ -3107,6 +3155,7 @@ pub fn select( declaring, &mut result, &mut resource_path_bytes, + &mut writable_resources, field, )?; } @@ -3115,8 +3164,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Python, )?; @@ -3126,8 +3178,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Ruby, )?; @@ -3162,8 +3217,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Node, )?; @@ -3177,6 +3235,7 @@ pub fn select( declaring, &mut result, &mut resource_path_bytes, + &mut writable_resources, field, )?; } @@ -3185,8 +3244,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Python, )?; @@ -3196,8 +3258,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Ruby, )?; @@ -3217,8 +3282,11 @@ pub fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, + DirectScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_files: &mut writable_resources, + }, field, ScriptDependencyRuntime::Node, )?; @@ -3311,8 +3379,9 @@ pub fn select( field, )?; } - Ok(( - result.into_iter().collect(), - variables.into_iter().collect(), - )) + Ok(ResourceSelection { + resources: result.into_iter().collect(), + variables: variables.into_iter().collect(), + writable_resources, + }) } diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index a315a1222..a8b87a724 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -311,6 +311,8 @@ pub struct PlannedSourceOccurrence { pub requires_config_bind: bool, pub instance_cwd: PathBuf, pub configured_resources: Vec<PathBuf>, + /// Canonical exact files with statically identified Node write targets. + pub configured_writable_resources: std::collections::BTreeSet<PathBuf>, /// Captured with the retained source view so later staging does not /// reinterpret a generated entry after ordinary-terminal cleanup. pub user_owns_freshell_mcp: bool, @@ -715,9 +717,10 @@ pub fn plan_loaded_opencode_sources_relocated( requires_config_bind: loaded.requires_config_bind, instance_cwd: cwd.into(), configured_resources: Vec::new(), + configured_writable_resources: Default::default(), user_owns_freshell_mcp, }; - let (resources, names) = + let selected_resources = super::opencode_resources::select(&planned, home, global.as_deref(), &|name| { lookup(planned.source.lane, &planned.source.logical_path, name) }) @@ -730,8 +733,12 @@ pub fn plan_loaded_opencode_sources_relocated( variables: vec![], cause: error.cause, })?; - planned.configured_resources = resources; - planned.references.path_variables.extend(names); + planned.configured_resources = selected_resources.resources; + planned.configured_writable_resources = selected_resources.writable_resources; + planned + .references + .path_variables + .extend(selected_resources.variables); planned.references.path_variables.sort(); planned.references.path_variables.dedup(); result.push(planned); diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 24617467e..841459665 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -4628,6 +4628,7 @@ mod tests { logical_destination: native.to_str().unwrap().into(), effective_destination: native.to_str().unwrap().into(), kind: ProviderResourceKind::Directory, + writable: false, }], }), mcp_capability: Some(capability), @@ -4743,6 +4744,7 @@ mod tests { logical_destination: package.to_string_lossy().into_owned(), effective_destination: package.to_string_lossy().into_owned(), kind: ProviderResourceKind::Directory, + writable: false, }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-0".into(), @@ -4750,6 +4752,7 @@ mod tests { logical_destination: "/home/freshell/.config/opencode".into(), effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, + writable: false, }, ], }), @@ -4852,6 +4855,7 @@ mod tests { logical_destination: "/home/freshell/provider/.config/opencode".into(), effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, + writable: false, }], indexed_files: None, }), @@ -5961,6 +5965,7 @@ mod tests { logical_destination: external.to_string_lossy().into_owned(), effective_destination: external.to_string_lossy().into_owned(), kind: ProviderResourceKind::Directory, + writable: false, }], }), mcp_capability: Some(capability), diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs index 875b6ba05..e5e81c59c 100644 --- a/crates/freshell-supervisor/src/backend/prepared_inputs.rs +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -29,7 +29,7 @@ pub(super) fn mount_descriptors( add( &root.effective_destination, json!({"Type":"volume","Source":provider_volume,"Target":root.effective_destination, - "ReadOnly":root.kind==freshell_runtime_protocol::ProviderResourceKind::File, + "ReadOnly":root.kind==freshell_runtime_protocol::ProviderResourceKind::File && !root.writable, "VolumeOptions":{"NoCopy":true,"Subpath":root.provider_relative_path}}), )?; } @@ -175,6 +175,7 @@ mod tests { logical_destination: "/outside/plugin".into(), effective_destination: "/outside/plugin".into(), kind: ProviderResourceKind::Directory, + writable: false, }], indexed_files: Some(IndexedProviderFiles { volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), @@ -228,6 +229,35 @@ mod tests { assert!(verify(&actual, &mounts, "owned-provider", &volumes).is_err()); } + #[test] + fn provider_prepared_file_mounts_are_readonly_unless_explicitly_writable() { + let (mut mounts, _, _) = fixture(); + for (name, destination, writable) in [ + ("readonly", "/outside/input.txt", false), + ("writable", "/outside/output.txt", true), + ] { + mounts.resource_mounts.push(ProviderResourceMount { + stage_relative_path: format!(".freshell/resource-units/{name}/source"), + provider_relative_path: format!(".freshell/provider-roots/{name}/source"), + logical_destination: destination.into(), + effective_destination: destination.into(), + kind: ProviderResourceKind::File, + writable, + }); + } + + let descriptors = mount_descriptors(&mounts, "owned-provider").unwrap(); + let descriptor = |destination: &str| { + descriptors + .iter() + .find(|mount| mount["Target"] == destination) + .expect("selected exact file mount") + }; + assert_eq!(descriptor("/outside/input.txt")["ReadOnly"], true); + assert_eq!(descriptor("/outside/output.txt")["ReadOnly"], false); + assert_eq!(descriptor("/outside/plugin")["ReadOnly"], false); + } + #[test] fn provider_prepared_indexed_secret_read_uses_the_published_generation() { let (mut mounts, mut actual, volumes) = fixture(); diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index a090af2bc..b174e7c16 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -736,6 +736,7 @@ mod tests { logical_destination: "/synthetic/workspace/.opencode".into(), effective_destination: "/synthetic/workspace/.opencode".into(), kind: ProviderResourceKind::Directory, + writable: false, }], }; let selected = selections(&mounts).unwrap(); diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index bdc126172..872168bd5 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -334,8 +334,8 @@ export function managedEnvironmentFixtureLayout(root: string) { return { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } } -export function managedHomeAppendFixturePaths(xdg: string, cwd: string, scenario: string) { - const script = path.join(xdg, 'opencode/mcp-servers', `direct-home-${scenario}.mjs`) +export function managedHomeAppendFixturePaths(home: string, cwd: string, scenario: string) { + const script = path.join(home, `direct-home-${scenario}.mjs`) return { script, output: path.join(path.dirname(script), `direct-home-${scenario}.txt`), @@ -424,7 +424,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') - const homeAppend = managedHomeAppendFixturePaths(xdg, cwd, scenario) + const homeAppend = managedHomeAppendFixturePaths(home, cwd, scenario) await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) await fs.writeFile(homeAppend.output, `initial-${nonce}`) await writeJson(path.join(cwd, 'opencode.json'), { agent: { fixtureRelative: { @@ -1300,7 +1300,7 @@ export async function runManagedEnvironmentTarget() { FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE: '', FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE: '' } rig = new ManagedRuntimeBrowserRig(repoRoot, 2, { ...environment, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY: '1' }, environment, 'release', { enabledProviders: ['opencode'] }, home, - { workspaceRoots: [workspace], providerInputRoots: [managedEnvironmentFixtureLayout(root).resources], + { workspaceRoots: [workspace], providerInputRoots: [managedEnvironmentFixtureLayout(root).resources, home], bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) const selected = await configure(root, nonce, rig.runtime.candidateSha, rig.runtime.runId) await rig.start() diff --git a/scripts/testing/runtime-sandbox.ts b/scripts/testing/runtime-sandbox.ts index b0c06ddb7..e586169dd 100644 --- a/scripts/testing/runtime-sandbox.ts +++ b/scripts/testing/runtime-sandbox.ts @@ -485,7 +485,11 @@ export class RuntimeHarness { args.push('-v', `${root}:${root}:ro`) } for (const root of this.phase2ProviderInputRoots()) { - if (root === this.testRoot || root === this.buildDir || this.phase2WorkspaceRoots().includes(root)) continue + // The explicitly selected HOME is already mounted above; avoid + // publishing a duplicate Docker mount when it is also an approved + // provider-input root. + if (root === this.providerFixtureHome || root === this.testRoot || root === this.buildDir + || this.phase2WorkspaceRoots().includes(root)) continue args.push('-v', `${root}:${root}:ro`) } } diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 933da2933..cba3756aa 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -114,12 +114,14 @@ describe('managed provider environment target mode', () => { expect(fixture.mcpParentDependency).toBe(path.join(fixture.resources, 'node_modules/managed-fixture-mcp-dependency')) }) - it('seeds the direct HOME append file beside the XDG MCP script it resolves from', () => { - const xdg = '/tmp/managed-environment-fixture/home/fixture-xdg' + it('seeds the direct HOME append file at the HOME root outside XDG resource roots', () => { + const home = '/tmp/managed-environment-fixture/home' const cwd = '/tmp/managed-environment-fixture/workspace/preflight' - const paths = managedHomeAppendFixturePaths(xdg, cwd, 'preflight') + const paths = managedHomeAppendFixturePaths(home, cwd, 'preflight') - expect(paths.script).toBe(path.join(xdg, 'opencode/mcp-servers/direct-home-preflight.mjs')) + expect(paths.script).toBe(path.join(home, 'direct-home-preflight.mjs')) + expect(path.dirname(paths.script)).toBe(home) + expect(paths.script.startsWith(path.join(home, 'fixture-xdg') + path.sep)).toBe(false) expect(paths.output).toBe(path.join(path.dirname(paths.script), 'direct-home-preflight.txt')) expect(path.dirname(paths.output)).toBe(path.dirname(paths.script)) expect(paths.receipt).toBe(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json')) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index d08214c3e..6cff3d662 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -598,7 +598,7 @@ describe('closed managed environment fixture inputs', () => { process.env.DOCKER_HOST = `unix://${socket}` const rig = new ManagedRuntimeBrowserRig(repository, 2, { HOME: home, XDG_CONFIG_HOME: xdg }, { HOME: home, XDG_CONFIG_HOME: xdg }, 'test', { enabledProviders: ['opencode'] }, home, - { workspaceRoots: [workspace], providerInputRoots: [providerInputs], bootstrapFiles: [grant], defaultCwd: workspace }) + { workspaceRoots: [workspace], providerInputRoots: [providerInputs, home], bootstrapFiles: [grant], defaultCwd: workspace }) harness = rig.runtime useOwnedHarnessArtifacts(harness, host) await harness.prepare() @@ -613,6 +613,7 @@ describe('closed managed environment fixture inputs', () => { expect(dispatched).toContain(`HOME=${home}`) expect(dispatched).toContain(`XDG_CONFIG_HOME=${xdg}`) expect(dispatched).toContain(`${home}:${home}:ro`) + expect(dispatched.filter(argument => argument === `${home}:${home}:ro`)).toHaveLength(1) expect(dispatched).not.toContain(`${repository}:${repository}:ro`) expect((await fs.stat(home)).mode & 0o777).toBe(0o700) } finally { From d882a1ab74fbc8d1c21d48163179b4f3c281938f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:50:15 -0700 Subject: [PATCH 419/493] fix(managed): preserve writable MCP output state --- .../src/provider_environment.rs | 208 ++++++++++++++++- .../managed_mcp_capability/source_stage.rs | 24 +- .../opencode_resources.rs | 29 ++- crates/freshell-session-host/src/main.rs | 220 +++++++++++++++++- 4 files changed, 448 insertions(+), 33 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index b6a2d3702..c6b046217 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -695,9 +695,10 @@ fn copy_environment_with_account_home( destination_root, excluded_canonical_sources, limits, - account_home, - &[], - None, + CopyProjectionOptions { + account_home, + ..Default::default() + }, ) } @@ -733,6 +734,28 @@ pub fn copy_environment_excluding_subtree_for_mounts( excluded_subtree: &Path, limits: CopyLimits, mounted_roots: &[PathBuf], +) -> Result<CopyReport, CopyError> { + copy_environment_excluding_subtree_for_mounts_preserving_files( + selections, + destination_root, + excluded_canonical_sources, + excluded_subtree, + limits, + mounted_roots, + &[], + ) +} + +/// Exclusion-aware projection that can retain an existing writable exact-file +/// mount's contents during recovery while still refreshing other mounted inputs. +pub fn copy_environment_excluding_subtree_for_mounts_preserving_files( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + excluded_subtree: &Path, + limits: CopyLimits, + mounted_roots: &[PathBuf], + preserve_existing_mounted_files: &[PathBuf], ) -> Result<CopyReport, CopyError> { let excluded_subtree = fs::canonicalize(excluded_subtree).map_err(|cause| { io_error( @@ -751,9 +774,12 @@ pub fn copy_environment_excluding_subtree_for_mounts( destination_root, &exclusions, limits, - account_home, - mounted_roots, - Some(excluded_subtree), + CopyProjectionOptions { + account_home, + mounted_roots, + preserve_existing_mounted_files, + excluded_subtree: Some(excluded_subtree), + }, ) } @@ -766,27 +792,72 @@ pub fn copy_environment_for_mounts( limits: CopyLimits, mounted_roots: &[PathBuf], ) -> Result<CopyReport, CopyError> { - let account_home = std::env::var_os("HOME").and_then(|home| fs::canonicalize(home).ok()); - copy_environment_projection( + copy_environment_for_mounts_preserving_files( selections, destination_root, excluded_canonical_sources, limits, - account_home, mounted_roots, - None, + &[], ) } -fn copy_environment_projection( +/// Projects mounted resources while retaining selected existing writable file +/// contents. Missing preserved files still follow normal mount publication. +pub fn copy_environment_for_mounts_preserving_files( selections: &[CopySelection], destination_root: &Path, excluded_canonical_sources: &[PathBuf], limits: CopyLimits, - account_home: Option<PathBuf>, mounted_roots: &[PathBuf], + preserve_existing_mounted_files: &[PathBuf], +) -> Result<CopyReport, CopyError> { + let account_home = std::env::var_os("HOME").and_then(|home| fs::canonicalize(home).ok()); + copy_environment_projection( + selections, + destination_root, + excluded_canonical_sources, + limits, + CopyProjectionOptions { + account_home, + mounted_roots, + preserve_existing_mounted_files, + ..Default::default() + }, + ) +} + +#[derive(Default)] +struct CopyProjectionOptions<'a> { + account_home: Option<PathBuf>, + mounted_roots: &'a [PathBuf], + preserve_existing_mounted_files: &'a [PathBuf], excluded_subtree: Option<PathBuf>, +} + +fn copy_environment_projection( + selections: &[CopySelection], + destination_root: &Path, + excluded_canonical_sources: &[PathBuf], + limits: CopyLimits, + options: CopyProjectionOptions<'_>, ) -> Result<CopyReport, CopyError> { + let CopyProjectionOptions { + account_home, + mounted_roots, + preserve_existing_mounted_files, + excluded_subtree, + } = options; + if preserve_existing_mounted_files + .iter() + .any(|path| !mounted_roots.contains(path)) + { + return Err(error( + Path::new("mounted-resource"), + "validate_preserved_mounts", + "preserved exact files must also be declared mounted roots", + )); + } bound( Path::new(""), "entries", @@ -1116,6 +1187,11 @@ fn copy_environment_projection( } mode(&destination, 0o700, relative)?; } else if mounted_roots.contains(relative) && matches!(node, Node::File(..)) { + if preserve_existing_mounted_files.contains(relative) + && preserve_existing_mounted_file(&destination, relative, limits)? + { + continue; + } let Node::File(_, source_mode, _) = node else { unreachable!() }; @@ -1144,6 +1220,56 @@ fn copy_environment_projection( }) } +fn preserve_existing_mounted_file( + destination: &Path, + relative: &Path, + limits: CopyLimits, +) -> Result<bool, CopyError> { + #[cfg(unix)] + use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; + let metadata = match fs::symlink_metadata(destination) { + Ok(metadata) => metadata, + Err(cause) if cause.kind() == std::io::ErrorKind::NotFound => return Ok(false), + Err(cause) => return Err(io_error(relative, "inspect_preserved_mount", cause)), + }; + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err(error( + relative, + "inspect_preserved_mount", + "preserved mounted resource must be a physical regular file", + )); + } + bound( + relative, + "file_bytes", + metadata.len(), + limits.max_file_bytes, + )?; + let mut options = fs::OpenOptions::new(); + options.read(true); + #[cfg(unix)] + options.custom_flags(libc::O_NOFOLLOW); + let opened = options + .open(destination) + .map_err(|cause| io_error(relative, "open_preserved_mount", cause))?; + let opened_metadata = opened + .metadata() + .map_err(|cause| io_error(relative, "inspect_preserved_mount", cause))?; + #[cfg(unix)] + let inode_changed = + metadata.dev() != opened_metadata.dev() || metadata.ino() != opened_metadata.ino(); + #[cfg(not(unix))] + let inode_changed = false; + if inode_changed || metadata.len() != opened_metadata.len() { + return Err(error( + relative, + "inspect_preserved_mount", + "mounted resource changed while recovery inspected its inode", + )); + } + Ok(true) +} + fn publish_mounted_file( source: &Path, destination: &Path, @@ -1304,6 +1430,64 @@ mod tests { } } + #[test] + fn provider_environment_recovery_keeps_existing_writable_mount_and_refreshes_read_only_mount() { + use std::io::{Read, Seek, SeekFrom, Write}; + let fixture = tempfile::tempdir().unwrap(); + let writable_source = fixture.path().join("writable-source"); + let readonly_source = fixture.path().join("readonly-source"); + fs::write(&writable_source, b"writable seed").unwrap(); + fs::write(&readonly_source, b"readonly seed").unwrap(); + let home = fixture.path().join("provider-home"); + let writable = PathBuf::from(".freshell/provider-roots/writable/source"); + let readonly = PathBuf::from(".freshell/provider-roots/readonly/source"); + fs::create_dir_all(home.join(writable.parent().unwrap())).unwrap(); + fs::create_dir_all(home.join(readonly.parent().unwrap())).unwrap(); + fs::write(home.join(&writable), b"").unwrap(); + fs::write(home.join(&readonly), b"").unwrap(); + // The provider's native volume alias points at this same exact-file + // inode and writes to it while the runtime is alive. + let mut native_alias = fs::OpenOptions::new() + .read(true) + .write(true) + .open(home.join(&writable)) + .unwrap(); + let selections = [ + selection(&writable_source, writable.to_str().unwrap()), + selection(&readonly_source, readonly.to_str().unwrap()), + ]; + let mounted = [writable.clone(), readonly.clone()]; + copy_environment_for_mounts(&selections, &home, &[], CopyLimits::default(), &mounted) + .unwrap(); + native_alias.seek(SeekFrom::End(0)).unwrap(); + native_alias.write_all(b" + runtime state").unwrap(); + fs::write(&writable_source, b"refreshed writable seed").unwrap(); + fs::write(&readonly_source, b"refreshed readonly seed").unwrap(); + + copy_environment_for_mounts_preserving_files( + &selections, + &home, + &[], + CopyLimits::default(), + &mounted, + std::slice::from_ref(&writable), + ) + .unwrap(); + + native_alias.seek(SeekFrom::Start(0)).unwrap(); + let mut runtime_state = Vec::new(); + native_alias.read_to_end(&mut runtime_state).unwrap(); + assert_eq!( + runtime_state, b"writable seed + runtime state", + "recovery must retain runtime data in an existing writable mount" + ); + assert_eq!( + fs::read(home.join(&readonly)).unwrap(), + b"refreshed readonly seed", + "ordinary read-only mounted inputs must still refresh" + ); + } + #[cfg(unix)] #[test] fn provider_environment_mounted_file_invalid_target_and_profile_failure_preserve_prior_bytes() { diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 86d42c9d9..ee8f23839 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1536,17 +1536,19 @@ process.stdout.write(require('./helper.js')) let url_output = home.join("generated-url.json"); let append_output = home.join("existing-output.txt"); let append_url_output = home.join("existing-url-output.txt"); + let append_variable_url_output = home.join("existing-url-variable-output.txt"); let copy_source = home.join("copy-source.txt"); let copy_output = home.join("copy-output.txt"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", + "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nconst stateUrl = new URL('./existing-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(stateUrl, ' appended');\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", ) .unwrap(); std::fs::write(&append_output, "existing").unwrap(); std::fs::write(&append_url_output, "url existing").unwrap(); + std::fs::write(&append_variable_url_output, "url variable existing").unwrap(); std::fs::write(©_source, "copy input").unwrap(); let (plan, provider_script) = @@ -1562,9 +1564,11 @@ process.stdout.write(require('./helper.js')) assert!(plan[0] .configured_resources .contains(&append_output.canonicalize().unwrap())); - assert!(plan[0] - .configured_resources - .contains(&append_url_output.canonicalize().unwrap())); + for output in [&append_url_output, &append_variable_url_output] { + assert!(plan[0] + .configured_resources + .contains(&output.canonicalize().unwrap())); + } assert!(plan[0] .configured_resources .contains(©_source.canonicalize().unwrap())); @@ -1582,7 +1586,11 @@ process.stdout.write(require('./helper.js')) None, ) .unwrap(); - for output in [&append_output, &append_url_output] { + for output in [ + &append_output, + &append_url_output, + &append_variable_url_output, + ] { let canonical = output.canonicalize().unwrap(); let mounts = units .iter() @@ -1622,6 +1630,8 @@ process.stdout.write(require('./helper.js')) let staged_url_output = provider_script_directory.join("generated-url.json"); let staged_append_output = provider_script_directory.join("existing-output.txt"); let staged_append_url_output = provider_script_directory.join("existing-url-output.txt"); + let staged_append_variable_url_output = + provider_script_directory.join("existing-url-variable-output.txt"); let staged_copy_output = provider_script_directory.join("copy-output.txt"); assert_eq!( std::fs::read(&staged_path_output).unwrap_or_else(|error| panic!( @@ -1646,6 +1656,10 @@ process.stdout.write(require('./helper.js')) std::fs::read(&staged_append_url_output).unwrap(), b"url existing appended" ); + assert_eq!( + std::fs::read(&staged_append_variable_url_output).unwrap(), + b"url variable existing appended" + ); assert_eq!(std::fs::read(&staged_copy_output).unwrap(), b"copy input"); } diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index abf04bead..ac047b5d6 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1301,7 +1301,8 @@ fn node_relative_resource_specifiers( source: &str, ) -> Result<std::collections::BTreeMap<String, bool>, &'static str> { let tokens = javascript_tokens(source)?; - let output_destinations = node_output_destination_mask(&tokens)?; + let (output_destinations, output_destination_variables) = + node_output_destination_mask(&tokens)?; let mut result = std::collections::BTreeMap::<String, bool>::new(); for index in 0..tokens.len() { if index >= 2 @@ -1406,23 +1407,33 @@ fn node_relative_resource_specifiers( tokens.get(index + 11), Some(JavaScriptToken::Punctuation(b')')) ))); + let is_bound_output_destination = index >= 3 + && matches!( + tokens.get(index - 1), + Some(JavaScriptToken::Punctuation(b'=')) + ) + && matches!(tokens.get(index - 2), Some(JavaScriptToken::Identifier(name)) if output_destination_variables.contains(name)) + && matches!(tokens.get(index - 3), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")); + let is_output_destination = output_destinations[index] || is_bound_output_destination; if new_keyword == "new" && url_constructor == "URL" && is_import_meta_url && (specifier.starts_with("./") || specifier.starts_with("../")) && !specifier.contains(['?', '#', '\\', '$']) - && (!output_destinations[index] || parent.join(specifier).exists()) + && (!is_output_destination || parent.join(specifier).exists()) { result .entry(specifier.clone()) - .and_modify(|writable| *writable |= output_destinations[index]) - .or_insert(output_destinations[index]); + .and_modify(|writable| *writable |= is_output_destination) + .or_insert(is_output_destination); } } Ok(result) } -fn node_output_destination_mask(tokens: &[JavaScriptToken]) -> Result<Vec<bool>, &'static str> { +fn node_output_destination_mask( + tokens: &[JavaScriptToken], +) -> Result<(Vec<bool>, std::collections::BTreeSet<String>), &'static str> { const OUTPUT_CALLS: [(&str, usize); 15] = [ ("appendFile", 0), ("appendFileSync", 0), @@ -1441,6 +1452,7 @@ fn node_output_destination_mask(tokens: &[JavaScriptToken]) -> Result<Vec<bool>, ("symlinkSync", 1), ]; let mut ranges = Vec::new(); + let mut direct_variables = std::collections::BTreeSet::new(); let mut inspection_budget = MAX_JAVASCRIPT_TOKENS; for (index, token) in tokens.iter().enumerate() { let JavaScriptToken::Identifier(name) = token else { @@ -1499,6 +1511,11 @@ fn node_output_destination_mask(tokens: &[JavaScriptToken]) -> Result<Vec<bool>, } } if let Some(end) = end.filter(|end| *end > start) { + if end == start + 1 { + if let Some(JavaScriptToken::Identifier(name)) = tokens.get(start) { + direct_variables.insert(name.clone()); + } + } ranges.push(start..end); } } @@ -1517,7 +1534,7 @@ fn node_output_destination_mask(tokens: &[JavaScriptToken]) -> Result<Vec<bool>, for range in merged { result[range].fill(true); } - Ok(result) + Ok((result, direct_variables)) } fn read_node_package_manifest( diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 841459665..2a4483445 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1809,7 +1809,23 @@ fn copy_environment_with_events( freshell_platform::provider_environment::CopyReport, freshell_platform::provider_environment::CopyError, > { - copy_environment_with_events_for_mounts(stage, provider, selections, destination, excluded, &[]) + copy_environment_with_events_for_mounts( + stage, + provider, + selections, + destination, + excluded, + MountedResourceOptions { + mounted_roots: &[], + preserve_existing_mounted_files: &[], + }, + ) +} + +#[derive(Clone, Copy)] +struct MountedResourceOptions<'a> { + mounted_roots: &'a [PathBuf], + preserve_existing_mounted_files: &'a [PathBuf], } fn copy_environment_with_events_for_mounts( @@ -1818,18 +1834,19 @@ fn copy_environment_with_events_for_mounts( selections: &[freshell_platform::provider_environment::CopySelection], destination: &Path, excluded: &[PathBuf], - mounted_directories: &[PathBuf], + resource_options: MountedResourceOptions<'_>, ) -> Result< freshell_platform::provider_environment::CopyReport, freshell_platform::provider_environment::CopyError, > { copy_environment_with_events_using(stage, provider, selections, || { - freshell_platform::provider_environment::copy_environment_for_mounts( + freshell_platform::provider_environment::copy_environment_for_mounts_preserving_files( selections, destination, excluded, Default::default(), - mounted_directories, + resource_options.mounted_roots, + resource_options.preserve_existing_mounted_files, ) }) } @@ -1841,19 +1858,20 @@ fn copy_environment_with_events_excluding_subtree_for_mounts( destination: &Path, excluded: &[PathBuf], excluded_subtree: &Path, - mounted_directories: &[PathBuf], + resource_options: MountedResourceOptions<'_>, ) -> Result< freshell_platform::provider_environment::CopyReport, freshell_platform::provider_environment::CopyError, > { copy_environment_with_events_using(stage, provider, selections, || { - freshell_platform::provider_environment::copy_environment_excluding_subtree_for_mounts( + freshell_platform::provider_environment::copy_environment_excluding_subtree_for_mounts_preserving_files( selections, destination, excluded, excluded_subtree, Default::default(), - mounted_directories, + resource_options.mounted_roots, + resource_options.preserve_existing_mounted_files, ) }) } @@ -2116,6 +2134,18 @@ fn copy_provider_config_references_with_ownership( } else { Vec::new() }; + let preserve_existing_mounted_files = if provider_home == final_home { + let previous = read_projection_manifest(final_home)?; + writable_resource_mount_roots(context) + .into_iter() + .filter(|path| { + path.to_str() + .is_some_and(|relative| previous.contains(relative)) + }) + .collect::<Vec<_>>() + } else { + Vec::new() + }; let report = if provider == "opencode" { copy_environment_with_events_excluding_subtree_for_mounts( "session_host", @@ -2124,7 +2154,10 @@ fn copy_provider_config_references_with_ownership( provider_home, &excluded, workspace, - &mounted, + MountedResourceOptions { + mounted_roots: &mounted, + preserve_existing_mounted_files: &preserve_existing_mounted_files, + }, ) } else { copy_environment_with_events_for_mounts( @@ -2133,7 +2166,10 @@ fn copy_provider_config_references_with_ownership( &selections, provider_home, &excluded, - &mounted, + MountedResourceOptions { + mounted_roots: &mounted, + preserve_existing_mounted_files: &preserve_existing_mounted_files, + }, ) } .map_err(|error| error.to_string())?; @@ -2167,6 +2203,20 @@ fn resource_mount_roots( .collect() } +fn writable_resource_mount_roots( + context: &freshell_runtime_protocol::ProviderLaunchContext, +) -> Vec<PathBuf> { + context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .filter(|mount| { + mount.kind == freshell_runtime_protocol::ProviderResourceKind::File && mount.writable + }) + .map(|mount| PathBuf::from(&mount.provider_relative_path)) + .collect() +} + // A resource's native HOME alias is a separate volume-subpath mount. Publish // through its declared physical HOME path, where the transaction and journal // share a filesystem. Every selected source still goes through the copier. @@ -2503,6 +2553,14 @@ fn refresh_provider_config_in_stage( let mut protected = private_bound_destinations(context); let previous = read_projection_manifest(provider_home)?; let current = read_projection_manifest(&projection)?; + let preserved_writable_files = writable_resource_mount_roots(context) + .into_iter() + .filter(|path| { + path.to_str() + .is_some_and(|relative| previous.contains(relative)) + }) + .map(|path| path.to_string_lossy().into_owned()) + .collect::<Vec<_>>(); protected.extend(protected_resource_alias_entries(context, &previous)); protected.extend(protected_resource_alias_entries(context, ¤t)); std::fs::write( @@ -2518,6 +2576,16 @@ fn refresh_provider_config_in_stage( .map_err(|_| "resource mountpoint metadata profile exceeded")?; std::fs::write(stage.path().join("resource-mountpoints.json"), mounted) .map_err(|_| "resource mountpoint metadata write failed")?; + let preserved = freshell_runtime_protocol::bounded_json_bytes( + &preserved_writable_files, + MAX_PROJECTION_METADATA_BYTES, + ) + .map_err(|_| "preserved writable resource metadata profile exceeded")?; + std::fs::write( + stage.path().join("preserved-writable-resource-files.json"), + preserved, + ) + .map_err(|_| "preserved writable resource metadata write failed")?; allow_provider_read_stage(stage.path(), unsafe { libc::getegid() })?; apply_staged_provider_config(stage.path(), provider_home) } @@ -2874,6 +2942,24 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() }) { return Err("resource mountpoint metadata has an invalid owned root".into()); } + let preserve_existing_mounted_files: Vec<PathBuf> = + match read_projection_metadata(stage, "preserved-writable-resource-files.json")? { + Some(bytes) => serde_json::from_slice::<Vec<String>>(&bytes) + .map_err(|_| "preserved writable resource metadata invalid")? + .into_iter() + .map(PathBuf::from) + .collect(), + None => Vec::new(), + }; + if preserve_existing_mounted_files.iter().any(|path| { + !path.starts_with(".freshell/provider-roots") + || path + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + || !mounted.contains(path) + }) { + return Err("preserved writable resource metadata has an invalid exact file".into()); + } publish_provider_projection(ProviderProjection { provider: &provider, selections: &selections, @@ -2882,6 +2968,7 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() current: ¤t, protected: &protected, mounted: &mounted, + preserve_existing_mounted_files: &preserve_existing_mounted_files, }) } @@ -2893,6 +2980,7 @@ struct ProviderProjection<'a> { current: &'a std::collections::BTreeSet<String>, protected: &'a std::collections::BTreeSet<String>, mounted: &'a [PathBuf], + preserve_existing_mounted_files: &'a [PathBuf], } // One provider-owned transaction covers replacement, stale removal and manifest @@ -2916,6 +3004,7 @@ fn publish_provider_projection_with_manifest( current, protected, mounted, + preserve_existing_mounted_files, } = projection; use std::{collections::BTreeMap, fs, os::unix::fs::PermissionsExt}; emit_environment_event( @@ -3027,7 +3116,10 @@ fn publish_provider_projection_with_manifest( let limits = freshell_platform::provider_environment::CopyLimits::default(); let mut bytes = 0u64; for (relative, metadata) in &old { - if mounted.contains(&PathBuf::from(relative)) && metadata.is_file() { + if mounted.contains(&PathBuf::from(relative)) + && metadata.is_file() + && !preserve_existing_mounted_files.contains(&PathBuf::from(relative)) + { bytes = bytes .checked_add(metadata.len()) .ok_or("provider mounted backup byte accounting overflow")?; @@ -3120,6 +3212,11 @@ fn publish_provider_projection_with_manifest( "provider transaction mounted file {relative} must retain a regular inode" )); } + if preserve_existing_mounted_files.contains(&PathBuf::from(relative)) + && previous.contains(relative) + { + continue; + } touched_mounts.push(relative.clone()); replace_mounted_projection_file(&desired.join(relative), &destination)?; } else { @@ -4412,6 +4509,7 @@ mod tests { current: ¤t, protected: &BTreeSet::new(), mounted: &[], + preserve_existing_mounted_files: &[], }, |source, _destination| { // This callback occupies only the final publication boundary. The @@ -4675,6 +4773,108 @@ mod tests { ); } + #[test] + fn provider_config_recovery_preserves_written_writable_file_and_refreshes_read_only_file() { + use freshell_runtime_protocol::*; + let owned = tempfile::tempdir().unwrap(); + let source_root = owned.path().join("source"); + let user_root = source_root.join("provider"); + let workspace = owned.path().join("workspace"); + let home = owned.path().join("home"); + let writable_source = user_root.join(".freshell/resource-units/writable/source"); + let readonly_source = user_root.join(".freshell/resource-units/readonly/source"); + let writable_relative = PathBuf::from(".freshell/provider-roots/writable/source"); + let readonly_relative = PathBuf::from(".freshell/provider-roots/readonly/source"); + std::fs::create_dir_all(writable_source.parent().unwrap()).unwrap(); + std::fs::create_dir_all(readonly_source.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(home.join(writable_relative.parent().unwrap())).unwrap(); + std::fs::create_dir_all(home.join(readonly_relative.parent().unwrap())).unwrap(); + std::fs::write(&writable_source, "writable seed").unwrap(); + std::fs::write(&readonly_source, "readonly seed").unwrap(); + // Exact-file mounts exist as physical placeholders before initial + // publication, just as their Docker volume subpath mounts do. + std::fs::write(home.join(&writable_relative), "").unwrap(); + std::fs::write(home.join(&readonly_relative), "").unwrap(); + let capability = McpCapabilityReference { + grant_id: format!("grant-{}", uuid::Uuid::new_v4()), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + let context = ProviderLaunchContext { + config: vec![], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + indexed_files: None, + resource_mounts: vec![ + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/writable/source".into(), + provider_relative_path: writable_relative.to_string_lossy().into_owned(), + logical_destination: "/home/freshell/provider/mcp/state.txt".into(), + effective_destination: "/home/freshell/provider/mcp/state.txt".into(), + kind: ProviderResourceKind::File, + writable: true, + }, + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/readonly/source".into(), + provider_relative_path: readonly_relative.to_string_lossy().into_owned(), + logical_destination: "/home/freshell/provider/mcp/reference.txt".into(), + effective_destination: "/home/freshell/provider/mcp/reference.txt".into(), + kind: ProviderResourceKind::File, + writable: false, + }, + ], + }), + mcp_capability: Some(capability), + }; + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + copy_provider_config_references( + &context, "opencode", &workspace, &user_root, &home, uid, gid, + ) + .unwrap(); + use std::io::Write; + std::fs::OpenOptions::new() + .append(true) + .open(home.join(&writable_relative)) + .unwrap() + .write_all(b" + runtime state") + .unwrap(); + std::fs::write(&writable_source, "refreshed writable seed").unwrap(); + std::fs::write(&readonly_source, "refreshed readonly seed").unwrap(); + + refresh_provider_config_references( + &context, "opencode", &workspace, &user_root, &home, uid, gid, + ) + .unwrap(); + + assert_eq!( + std::fs::read(home.join(&writable_relative)).unwrap(), + b"writable seed + runtime state", + "replacement must keep the existing writable mount's runtime data" + ); + assert_eq!( + std::fs::read(home.join(&readonly_relative)).unwrap(), + b"refreshed readonly seed", + "read-only inputs still refresh from their selected source" + ); + } + #[test] fn provider_config_resource_projection_excludes_workspace_nested_in_package_root() { use freshell_runtime_protocol::*; From 810415b31fb56fecb63252f9052b52adb6ddc4b4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:50:22 -0700 Subject: [PATCH 420/493] fix(testing): retain complete managed run diagnostics --- scripts/e2e-cloud.sh | 16 +++++---- .../test/cloud-e2e-logging-receipt.test.sh | 23 +++++++++++- scripts/testing/managed-environment-create.ts | 36 ++++++++++++++----- ...anaged-provider-environment-target.test.ts | 26 ++++++++++++++ 4 files changed, 86 insertions(+), 15 deletions(-) diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index 9dffeb905..8e93100aa 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -796,16 +796,20 @@ cmd_run() { # Capture to a variable so we can print the full output AND extract a # per-shard summary, even when some shards fail. echo "[e2e-cloud] Fetching logs..." - local log_output="" summary_lines="" display_log_unavailable=false summary_footer_present=false + local log_output="" summary_lines="" display_log_unavailable=false summary_footers_complete=false + local summary_footer_count local summary_attempt for summary_attempt in 1 2 3 4 5; do if log_output=$(gcloud beta run jobs executions logs read $(gcloud_flags) "$execution_id" 2>&1); then summary_lines=$(printf '%s\n' "$log_output" | node "$ROOT/scripts/testing/e2e-cloud-log-summary.mjs") display_log_unavailable=false - if printf '%s\n' "$summary_lines" | grep -Eq '^[[:space:]]+[0-9]+ (failed|passed|skipped|interrupted)\b'; then - summary_footer_present=true + summary_footer_count=$(printf '%s\n' "$summary_lines" \ + | grep -Ec '^[[:space:]]+[0-9]+ (failed|passed|skipped|interrupted)\b' || true) + summary_footers_complete=false + if [ "$summary_footer_count" -ge "$shards" ]; then + summary_footers_complete=true fi - if $summary_footer_present || $playwright_dry_run; then + if $summary_footers_complete || $playwright_dry_run; then break fi else @@ -822,8 +826,8 @@ cmd_run() { # display, but never decides whether retry evidence exists; that decision # comes from the required jsonPayload query below. echo "[e2e-cloud] WARNING: could not read display logs after five attempts." >&2 - elif ! $summary_footer_present && ! $playwright_dry_run; then - echo "[e2e-cloud] WARNING: Playwright summary footer was not present in display logs after five attempts." >&2 + elif ! $summary_footers_complete && ! $playwright_dry_run; then + echo "[e2e-cloud] WARNING: Playwright summary footers for all ${shards} shard(s) were not present in display logs after five attempts." >&2 fi # Print logs from every shard. Retry traces are retained as bounded base64 diff --git a/scripts/test/cloud-e2e-logging-receipt.test.sh b/scripts/test/cloud-e2e-logging-receipt.test.sh index ade3c7aa5..2fbefeb56 100644 --- a/scripts/test/cloud-e2e-logging-receipt.test.sh +++ b/scripts/test/cloud-e2e-logging-receipt.test.sh @@ -30,7 +30,14 @@ case "$*" in *"executions logs read"*) count_file="$STUB_CAPTURE/logs.count" count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" - if [ "${STUB_DELAY_SUMMARY_ONCE:-0}" = 1 ] && [ "$count" -eq 1 ]; then + if [ "${STUB_DELAY_SECOND_SHARD_FOOTER_ONCE:-0}" = 1 ]; then + echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" + echo "[e2e-entrypoint] Shard 2/2 assignment: focused fixture" + echo " 2 passed (5.0s)" + if [ "$count" -gt 1 ]; then + echo " 2 passed (7.0s)" + fi + elif [ "${STUB_DELAY_SUMMARY_ONCE:-0}" = 1 ] && [ "$count" -eq 1 ]; then echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" else echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" @@ -121,6 +128,20 @@ if ! grep -Fq 'labels."run.googleapis.com/execution_name"="exec-json-receipt"' " cat "$WORK/capture/gcloud.args"; exit 1 fi +# A footer from the first task is not a complete display summary when the +# requested run has multiple shards. Keep reading until every shard footer is +# visible, even if one task's footer arrives before the others. +rm -f "$WORK/capture/logs.count" "$WORK/capture/logging.count" "$WORK/capture/gcloud.args" +PARTIAL_SHARD_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_SECOND_SHARD_FOOTER_ONCE=1 STUB_LOGGING_JSON="$records_no_retry" GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" || { + echo "FAIL: two-shard run with delayed display footer failed"; echo "$PARTIAL_SHARD_OUT"; exit 1 +} +if [ "$(cat "$WORK/capture/logs.count")" -lt 2 ] \ + || ! grep -q "2 passed (5.0s)" <<< "$PARTIAL_SHARD_OUT" \ + || ! grep -q "2 passed (7.0s)" <<< "$PARTIAL_SHARD_OUT"; then + echo "FAIL: wrapper stopped reading display logs before every shard footer arrived" + echo "$PARTIAL_SHARD_OUT"; cat "$WORK/capture/gcloud.args"; exit 1 +fi + records_empty_shards='[ {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":0,"taskCount":4,"recoveredRetryCount":0}}, {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":1,"taskCount":4,"recoveredRetryCount":0}}, diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index a8f060ae1..5fd1ef309 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -189,6 +189,33 @@ export function summarizeRustServerLogEvents(raw: string): Array<{ return summary.slice(-32) } +const maxRustServerLogTailBytes = 512 * 1024 + +function readRustServerLogEvents(logPath: string): ReturnType<typeof summarizeRustServerLogEvents> { + let descriptor: number | undefined + try { + descriptor = fs.openSync(logPath, 'r') + const metadata = fs.fstatSync(descriptor) + if (!metadata.isFile()) return [] + const byteLength = Math.min(metadata.size, maxRustServerLogTailBytes) + const buffer = Buffer.allocUnsafe(byteLength) + const position = metadata.size - byteLength + let bytesRead = 0 + while (bytesRead < byteLength) { + const count = fs.readSync(descriptor, buffer, bytesRead, byteLength - bytesRead, position + bytesRead) + if (count === 0) break + bytesRead += count + } + return summarizeRustServerLogEvents(buffer.subarray(0, bytesRead).toString('utf8')) + } catch { + return [] + } finally { + if (descriptor !== undefined) { + try { fs.closeSync(descriptor) } catch { /* A diagnostic descriptor may already be closed. */ } + } + } +} + const failureReasons = [ 'OneCLI grant is unavailable', 'prepared private source unreadable', @@ -260,14 +287,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, ?? (filesystemLabel && osError !== undefined ? `${filesystemLabel} filesystem error` : 'UNKNOWN') const code = ErrorCode.safeParse(response.code) const web = owner.web.capturedOutput() - let serverLogEvents: Array<{ level: string; event: string }> = [] - try { - const logPath = path.join(owner.web.info.logsDir, 'rust-server.jsonl') - const metadata = fs.statSync(logPath) - if (metadata.isFile() && metadata.size <= 4 * 1024 * 1024) { - serverLogEvents = summarizeRustServerLogEvents(fs.readFileSync(logPath, 'utf8')) - } - } catch { /* Missing or oversized logs remain unavailable. */ } + const serverLogEvents = readRustServerLogEvents(path.join(owner.web.info.logsDir, 'rust-server.jsonl')) let supervisor: Record<string, string> | 'UNKNOWN' = 'UNKNOWN' let sessionHost: Record<string, unknown> | 'UNKNOWN' = 'UNKNOWN' try { diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index cba3756aa..a0e6ab1e5 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -80,6 +80,32 @@ describe('managed provider environment target mode', () => { expect(JSON.stringify(summary)).not.toContain('PRIVATE_') }) + it('keeps the bounded tail of a large active server log in failure evidence', async () => { + const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-server-log-')) + const incidents: Array<Record<string, unknown>> = [] + try { + const logPath = path.join(directory, 'rust-server.jsonl') + const tail = { level: 'INFO', target: 'freshell_server::managed_runtime', stage: 'activate' } + await fs.writeFile(logPath, `${'x'.repeat(5 * 1024 * 1024)}\n${JSON.stringify(tail)}\n`) + expect((await fs.stat(logPath)).size).toBeGreaterThan(4 * 1024 * 1024) + + const owner = { + web: { capturedOutput: () => ({ stdout: '', stderr: '' }), info: { logsDir: directory } }, + runtime: { containerLogs: () => '', writeIncident: (_name: string, value: unknown) => incidents.push(value as Record<string, unknown>) }, + supervisor: { containerId: 'a'.repeat(64) }, + } + const response = { type: 'error' as const, code: 'PTY_SPAWN_FAILED' as const, + timestamp: new Date().toISOString(), message: 'managed runtime launch failed' } + + expect(() => assertManagedTerminalCreated(owner, response)).toThrow() + expect(incidents[0]?.serverLogEvents).toEqual([ + { level: 'INFO', target: tail.target, fields: ['stage'] }, + ]) + } finally { + await fs.rm(directory, { recursive: true, force: true }) + } + }) + it('waits for the target Retry-After before retrying a rate-limited control read', async () => { const responses = [ new Response('rate limited', { status: 429, headers: { 'retry-after': '2' } }), From 50d12f0a4499a9a131e2fa7c8e725c554933b591 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:50:27 -0700 Subject: [PATCH 421/493] fix(testing): authorize prepared writable resource files --- scripts/testing/runtime-test-broker.ts | 13 +++-- .../testing/runtime-test-broker.test.ts | 52 +++++++++++++++++-- 2 files changed, 58 insertions(+), 7 deletions(-) diff --git a/scripts/testing/runtime-test-broker.ts b/scripts/testing/runtime-test-broker.ts index fa288fd86..565d97b20 100644 --- a/scripts/testing/runtime-test-broker.ts +++ b/scripts/testing/runtime-test-broker.ts @@ -71,7 +71,8 @@ function preparedMountsMatch(actual: unknown[], input: PreparedInputAuthorizatio Type: 'bind', Source: path.join(input.stageRoot, bind.sourceRelativePath), Target: bind.effectiveDestination, ReadOnly: true, })) expected.push(...input.resourceMounts.map((mount) => ({ - Type: 'volume', Source: input.providerVolumeName, Target: mount.effectiveDestination, ReadOnly: mount.kind === 'file', + Type: 'volume', Source: input.providerVolumeName, Target: mount.effectiveDestination, + ReadOnly: mount.kind === 'file' && mount.writable !== true, VolumeOptions: { NoCopy: true, Subpath: mount.providerRelativePath }, }))) if (input.indexedFiles) { @@ -161,7 +162,7 @@ export type PreparedInputAuthorization = { stageRoot: string generationId: string configurationBinds: { sourceRelativePath: string; effectiveDestination: string }[] - resourceMounts: { providerRelativePath: string; effectiveDestination: string; kind: 'file' | 'directory' }[] + resourceMounts: { providerRelativePath: string; effectiveDestination: string; kind: 'file' | 'directory'; writable?: boolean }[] indexedFiles?: { volumeName: string; aliases: { sourceId: number; effectiveDestination: string }[] } indexedSecretSources: { grantIndex: number; sourceId: number }[] } @@ -335,7 +336,10 @@ export class RestrictedDockerBroker { } for (const mount of input.resourceMounts) { const relative=mount.kind==='file' && mount.providerRelativePath.endsWith('/source') ? mount.providerRelativePath.slice(0,-'/source'.length) : mount.providerRelativePath; - if (!unit(relative, '.freshell/provider-roots/') || (mount.kind==='file' && !mount.providerRelativePath.endsWith('/source')) || !['file', 'directory'].includes(mount.kind)) throw new Error('prepared resource source is invalid') + if (!unit(relative, '.freshell/provider-roots/') || (mount.kind==='file' && !mount.providerRelativePath.endsWith('/source')) + || !['file', 'directory'].includes(mount.kind) + || (mount.writable !== undefined && typeof mount.writable !== 'boolean') + || (mount.kind !== 'file' && mount.writable === true)) throw new Error('prepared resource source is invalid') claim(mount.effectiveDestination) } if (input.indexedFiles) { @@ -1368,7 +1372,8 @@ export class RestrictedDockerBroker { configurationBinds: (inputs.configBinds ?? []).map((bind: Record<string, any>) => ({ sourceRelativePath: bind.sourceRelativePath, effectiveDestination: bind.effectiveDestination, })), resourceMounts: (inputs.resourceMounts ?? []).map((mount: Record<string, any>) => ({ - providerRelativePath: mount.providerRelativePath, effectiveDestination: mount.effectiveDestination, kind: mount.kind, + providerRelativePath: mount.providerRelativePath, effectiveDestination: mount.effectiveDestination, + kind: mount.kind, writable: mount.writable, })), indexedFiles: inputs.indexedFiles, indexedSecretSources, }) } diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 62ae42c74..71d21e410 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -305,10 +305,14 @@ describe('managed runtime broker actor storage', () => { stageRoot:`/tmp/owned/scenarios/scenario/control/provider-roots/${generationId}`, configurationBinds:[{sourceRelativePath:'.freshell/templates/source-0',effectiveDestination:'/workspace/opencode.json'}], resourceMounts:[{providerRelativePath:'.freshell/provider-roots/plugin-0',effectiveDestination:'/workspace/plugin',kind:'directory'}, - {providerRelativePath:'.freshell/provider-roots/instructions-0/source',effectiveDestination:'/workspace/instructions.md',kind:'file'}], + {providerRelativePath:'.freshell/provider-roots/instructions-0/source',effectiveDestination:'/workspace/instructions.md',kind:'file'}, + {providerRelativePath:'.freshell/provider-roots/output-0/source',effectiveDestination:'/workspace/generated.txt',kind:'file',writable:true}], indexedFiles:{volumeName,aliases:[{sourceId:0,effectiveDestination:'/workspace/selected-input'}]}, indexedSecretSources:[{grantIndex:1,sourceId:0}], }; + const directoryWrite = structuredClone(authorization); + directoryWrite.resourceMounts[0].writable = true; + expect(() => instance.authorizePreparedInputs(directoryWrite)).toThrow('prepared resource source is invalid'); instance.authorizePreparedInputs(authorization); const body=JSON.parse(createBody([...ordinaryBinds,actorBind]).toString()); body.HostConfig.Mounts=[ @@ -317,14 +321,18 @@ describe('managed runtime broker actor storage', () => { {Type:'volume',Source:volumeName,Target:'/workspace/selected-input',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'files/source-0'}}, {Type:'volume',Source:volumeName,Target:'/run/freshell-secrets/provider-1',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'files/source-0'}}, {Type:'volume',Source:providerVolume,Target:'/workspace/instructions.md',ReadOnly:true,VolumeOptions:{NoCopy:true,Subpath:'.freshell/provider-roots/instructions-0/source'}}, + {Type:'volume',Source:providerVolume,Target:'/workspace/generated.txt',ReadOnly:false,VolumeOptions:{NoCopy:true,Subpath:'.freshell/provider-roots/output-0/source'}}, ]; expect((await request('POST','/v1.47/containers/create?name=owned-prepared',body)).status).toBe(201); expect(forwarded).toHaveLength(1); - for (const mutation of ['source','target','writable','subpath','nocopy','missing','empty','duplicate','owner','image','volume']) { + for (const mutation of ['source','target','writable','prepared-writable-file-readonly','directory-readonly','ordinary-file-writable','subpath','nocopy','missing','empty','duplicate','owner','image','volume']) { const changed=structuredClone(body); if (mutation==='source') changed.HostConfig.Mounts[0].Source='/var/run/docker.sock'; if (mutation==='target') changed.HostConfig.Mounts[0].Target='/run/freshell'; if (mutation==='writable') changed.HostConfig.Mounts[2].ReadOnly=false; + if (mutation==='prepared-writable-file-readonly') changed.HostConfig.Mounts[5].ReadOnly=true; + if (mutation==='directory-readonly') changed.HostConfig.Mounts[1].ReadOnly=true; + if (mutation==='ordinary-file-writable') changed.HostConfig.Mounts[4].ReadOnly=false; if (mutation==='subpath') changed.HostConfig.Mounts[2].VolumeOptions.Subpath='files/source-1'; if (mutation==='nocopy') changed.HostConfig.Mounts[1].VolumeOptions.NoCopy=false; if (mutation==='missing') changed.HostConfig.Mounts.pop(); @@ -1332,7 +1340,11 @@ describe('fixed indexed publisher broker',()=>{ const input = { ...publisherAuthorization(), soulId: owner.soulId, generationId, hostBinaryPath, providerVolumeName: owner.providerVolumeName, stageRoot: path.join(directory, 'scenarios/scenario/control/provider-roots', generationId), configurationBinds: [{ sourceRelativePath: '.freshell/templates/codex-0', effectiveDestination: '/home/freshell/provider/.codex/config.toml' }], - resourceMounts: [{ providerRelativePath: '.freshell/provider-roots/root-0', effectiveDestination: '/home/freshell/provider/.codex/skills', kind: 'directory' as const }], + resourceMounts: [ + { providerRelativePath: '.freshell/provider-roots/root-0', effectiveDestination: '/home/freshell/provider/.codex/skills', kind: 'directory' as const }, + { providerRelativePath: '.freshell/provider-roots/root-1/source', effectiveDestination: '/home/freshell/provider/.codex/README.md', kind: 'file' as const }, + { providerRelativePath: '.freshell/provider-roots/root-2/source', effectiveDestination: '/home/freshell/provider/.codex/generated.txt', kind: 'file' as const, writable: true }, + ], indexedFiles: { volumeName: `freshell-inputs-${createHash('sha256').update(`${owner.soulId}\0${generationId}`).digest('hex').slice(0, 24)}`, aliases: [] }, indexedSecretSources: [] } const { metadata, capabilities, context } = await publishControllerPreparation(directory, input, '11111111-1111-4111-8111-111111111111', 'codex') @@ -1392,6 +1404,40 @@ describe('fixed indexed publisher broker',()=>{ expect(() => instance.authorizePreparedInputs(changed), destination).toThrow() } expect((await request('POST', '/v1.47/volumes/create', volume)).status, JSON.stringify(instance.eventsSnapshot().slice(-4))).toBe(201) + const registered = [...(instance as unknown as { preparedInputs: Map<string, PreparedInputAuthorization> }).preparedInputs.values()] + .find(prepared => prepared.generationId === generationId) + expect(registered?.resourceMounts).toEqual(input.resourceMounts) + const runtimeDir = path.join(directory, 'r/scenario/x/incarnation-one') + const runtimeBinds = ordinaryBinds.map(bind => bind.replace('/tmp/owned/freshell-session-host', owner.hostBinaryPath) + .replace('/tmp/owned/r/scenario/x/incarnation-one', runtimeDir).replace(providerVolume, owner.providerVolumeName)) + const runtimeActorKey = createHash('sha256').update(`${owner.installationId}\0${owner.soulId}`).digest('hex') + const runtimeActorBind = `${path.join(directory, 'r/scenario/x/souls', runtimeActorKey, 'actor')}:/run/freshell-host-actor:rw` + const runtime = JSON.parse(createBody([...runtimeBinds, runtimeActorBind]).toString()) + runtime.Labels['com.freshell.soul-id'] = owner.soulId + runtime.HostConfig.Mounts = [ + ...registered!.configurationBinds.map(bind => ({ Type: 'bind', Source: path.join(input.stageRoot, bind.sourceRelativePath), + Target: bind.effectiveDestination, ReadOnly: true })), + ...registered!.resourceMounts.map(mount => ({ Type: 'volume', Source: owner.providerVolumeName, + Target: mount.effectiveDestination, ReadOnly: mount.kind === 'file' && mount.writable !== true, + VolumeOptions: { NoCopy: true, Subpath: mount.providerRelativePath } })), + ] + const writableIndex = runtime.HostConfig.Mounts.findIndex((mount: any) => mount.Target === '/home/freshell/provider/.codex/generated.txt') + const directoryIndex = runtime.HostConfig.Mounts.findIndex((mount: any) => mount.Target === '/home/freshell/provider/.codex/skills') + const ordinaryFileIndex = runtime.HostConfig.Mounts.findIndex((mount: any) => mount.Target === '/home/freshell/provider/.codex/README.md') + expect([writableIndex, directoryIndex, ordinaryFileIndex].every((index: number) => index >= 0)).toBe(true) + expect((await request('POST', '/v1.47/containers/create?name=registered-writable-resource', runtime)).status, + JSON.stringify(instance.eventsSnapshot().slice(-4))).toBe(201) + for (const [name, index, readOnly] of [ + ['writable-file-made-readonly', writableIndex, true], + ['directory-made-readonly', directoryIndex, true], + ['ordinary-file-made-writable', ordinaryFileIndex, false], + ] as const) { + const changed = structuredClone(runtime) + changed.HostConfig.Mounts[index].ReadOnly = readOnly + const before = forwarded.length + expect((await request('POST', `/v1.47/containers/create?name=${name}`, changed)).status, name).toBe(403) + expect(forwarded).toHaveLength(before) + } const before = forwarded.length for (const mutation of ['owner', 'generation', 'permissions', 'volume']) { const changed = structuredClone(volume) From 97b7d602a8f1e93bb9a86c58e86a4407aef3a9ef Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 19:25:23 -0700 Subject: [PATCH 422/493] fix(managed): scope URL output bindings --- .../managed_mcp_capability/source_stage.rs | 43 +++++++- .../opencode_resources.rs | 97 ++++++++++++++++++- 2 files changed, 134 insertions(+), 6 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index ee8f23839..b4cfad0ce 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1537,18 +1537,26 @@ process.stdout.write(require('./helper.js')) let append_output = home.join("existing-output.txt"); let append_url_output = home.join("existing-url-output.txt"); let append_variable_url_output = home.join("existing-url-variable-output.txt"); + let append_let_variable_url_output = home.join("existing-let-url-variable-output.txt"); + let append_var_variable_url_output = home.join("existing-var-url-variable-output.txt"); + let reassigned_url_output = home.join("existing-reassigned-url-output.txt"); + let unrelated_url_output = home.join("existing-unrelated-url-output.txt"); let copy_source = home.join("copy-source.txt"); let copy_output = home.join("copy-output.txt"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nconst stateUrl = new URL('./existing-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(stateUrl, ' appended');\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", + "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nconst stateUrl = new URL('./existing-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(stateUrl, ' appended');\nlet letUrl = new URL('./existing-let-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(letUrl, ' appended');\nvar varUrl = new URL('./existing-var-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(varUrl, ' appended');\nlet reassignedUrl = new URL('./existing-reassigned-url-output.txt', import.meta.url);\nreassignedUrl = new URL('./existing-reassigned-url-output.txt', import.meta.url);\nfs.appendFileSync(reassignedUrl, ' appended');\nconst url = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nfunction writeParameter(url) { fs.appendFileSync(url, ' appended'); }\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", ) .unwrap(); std::fs::write(&append_output, "existing").unwrap(); std::fs::write(&append_url_output, "url existing").unwrap(); std::fs::write(&append_variable_url_output, "url variable existing").unwrap(); + std::fs::write(&append_let_variable_url_output, "let url variable existing").unwrap(); + std::fs::write(&append_var_variable_url_output, "var url variable existing").unwrap(); + std::fs::write(&reassigned_url_output, "reassigned url existing").unwrap(); + std::fs::write(&unrelated_url_output, "unrelated url existing").unwrap(); std::fs::write(©_source, "copy input").unwrap(); let (plan, provider_script) = @@ -1564,7 +1572,14 @@ process.stdout.write(require('./helper.js')) assert!(plan[0] .configured_resources .contains(&append_output.canonicalize().unwrap())); - for output in [&append_url_output, &append_variable_url_output] { + for output in [ + &append_url_output, + &append_variable_url_output, + &append_let_variable_url_output, + &append_var_variable_url_output, + &reassigned_url_output, + &unrelated_url_output, + ] { assert!(plan[0] .configured_resources .contains(&output.canonicalize().unwrap())); @@ -1590,6 +1605,8 @@ process.stdout.write(require('./helper.js')) &append_output, &append_url_output, &append_variable_url_output, + &append_let_variable_url_output, + &append_var_variable_url_output, ] { let canonical = output.canonicalize().unwrap(); let mounts = units @@ -1605,6 +1622,28 @@ process.stdout.write(require('./helper.js')) ); assert!(mounts.iter().all(|(_, mount)| mount.writable)); } + let unrelated_url_mounts = units + .iter() + .filter(|(source, mount)| { + source == &unrelated_url_output.canonicalize().unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!(!unrelated_url_mounts.is_empty()); + assert!(unrelated_url_mounts + .iter() + .all(|(_, mount)| !mount.writable)); + let reassigned_url_mounts = units + .iter() + .filter(|(source, mount)| { + source == &reassigned_url_output.canonicalize().unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!(!reassigned_url_mounts.is_empty()); + assert!(reassigned_url_mounts + .iter() + .all(|(_, mount)| !mount.writable)); let copy_source_mounts = units .iter() .filter(|(source, mount)| { diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index ac047b5d6..cf91e66ee 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1301,8 +1301,16 @@ fn node_relative_resource_specifiers( source: &str, ) -> Result<std::collections::BTreeMap<String, bool>, &'static str> { let tokens = javascript_tokens(source)?; + let scope_ids = javascript_token_scope_ids(&tokens)?; let (output_destinations, output_destination_variables) = - node_output_destination_mask(&tokens)?; + node_output_destination_mask(&tokens, &scope_ids)?; + let binding_declarations = node_variable_binding_declaration_counts(&tokens, &scope_ids); + let ambiguous_output_variables = node_ambiguous_output_variables( + &tokens, + &scope_ids, + &output_destinations, + &output_destination_variables, + ); let mut result = std::collections::BTreeMap::<String, bool>::new(); for index in 0..tokens.len() { if index >= 2 @@ -1412,7 +1420,10 @@ fn node_relative_resource_specifiers( tokens.get(index - 1), Some(JavaScriptToken::Punctuation(b'=')) ) - && matches!(tokens.get(index - 2), Some(JavaScriptToken::Identifier(name)) if output_destination_variables.contains(name)) + && matches!(tokens.get(index - 2), Some(JavaScriptToken::Identifier(name)) + if output_destination_variables.contains(&(name.clone(), scope_ids[index])) + && binding_declarations.get(&(name.clone(), scope_ids[index])) == Some(&1) + && !ambiguous_output_variables.contains(&(name.clone(), scope_ids[index]))) && matches!(tokens.get(index - 3), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")); let is_output_destination = output_destinations[index] || is_bound_output_destination; if new_keyword == "new" @@ -1431,9 +1442,87 @@ fn node_relative_resource_specifiers( Ok(result) } +fn javascript_token_scope_ids(tokens: &[JavaScriptToken]) -> Result<Vec<usize>, &'static str> { + let mut scope_stack = Vec::new(); + let mut scope_ids = Vec::with_capacity(tokens.len()); + for (index, token) in tokens.iter().enumerate() { + if matches!(token, JavaScriptToken::Punctuation(b'}')) && scope_stack.pop().is_none() { + return Err("direct MCP Node script has an unmatched closing brace"); + } + scope_ids.push(scope_stack.last().copied().unwrap_or(0)); + if matches!(token, JavaScriptToken::Punctuation(b'{')) { + // Reserve zero for the top-level scope and give each brace scope a + // unique token-derived identity. This deliberately treats nested + // blocks as separate scopes, so uncertain captures remain read-only. + scope_stack.push(index + 1); + } + } + if !scope_stack.is_empty() { + return Err("direct MCP Node script has an unmatched opening brace"); + } + Ok(scope_ids) +} + +fn node_variable_binding_declaration_counts( + tokens: &[JavaScriptToken], + scope_ids: &[usize], +) -> std::collections::BTreeMap<(String, usize), usize> { + let mut declarations = std::collections::BTreeMap::new(); + for index in 0..tokens.len().saturating_sub(1) { + if !matches!(tokens.get(index), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")) + { + continue; + } + let Some(JavaScriptToken::Identifier(name)) = tokens.get(index + 1) else { + continue; + }; + let key = (name.clone(), scope_ids[index]); + *declarations.entry(key).or_insert(0) += 1; + } + declarations +} + +fn node_ambiguous_output_variables( + tokens: &[JavaScriptToken], + scope_ids: &[usize], + output_destinations: &[bool], + output_destination_variables: &std::collections::BTreeSet<(String, usize)>, +) -> std::collections::BTreeSet<(String, usize)> { + let mut ambiguous = std::collections::BTreeSet::new(); + for (index, token) in tokens.iter().enumerate() { + let JavaScriptToken::Identifier(name) = token else { + continue; + }; + let key = (name.clone(), scope_ids[index]); + if !output_destination_variables.contains(&key) || output_destinations[index] { + continue; + } + let is_url_binding_declaration = index >= 1 + && matches!(tokens.get(index - 1), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")) + && matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'=')) + ) + && matches!(tokens.get(index + 2), Some(JavaScriptToken::Identifier(keyword)) if keyword == "new") + && matches!(tokens.get(index + 3), Some(JavaScriptToken::Identifier(constructor)) if constructor == "URL") + && matches!( + tokens.get(index + 4), + Some(JavaScriptToken::Punctuation(b'(')) + ); + if !is_url_binding_declaration { + // A second use outside a recognized output argument may reassign + // or mutate the URL. Keep the selected input read-only when this + // bounded scanner cannot prove the binding stays stable. + ambiguous.insert(key); + } + } + ambiguous +} + fn node_output_destination_mask( tokens: &[JavaScriptToken], -) -> Result<(Vec<bool>, std::collections::BTreeSet<String>), &'static str> { + scope_ids: &[usize], +) -> Result<(Vec<bool>, std::collections::BTreeSet<(String, usize)>), &'static str> { const OUTPUT_CALLS: [(&str, usize); 15] = [ ("appendFile", 0), ("appendFileSync", 0), @@ -1513,7 +1602,7 @@ fn node_output_destination_mask( if let Some(end) = end.filter(|end| *end > start) { if end == start + 1 { if let Some(JavaScriptToken::Identifier(name)) = tokens.get(start) { - direct_variables.insert(name.clone()); + direct_variables.insert((name.clone(), scope_ids[start])); } } ranges.push(start..end); From 376ac5aaeda5a710ceb08993c5c82e6903306e35 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 19:40:38 -0700 Subject: [PATCH 423/493] fix(testing): attribute cloud summaries by task --- scripts/e2e-cloud-task-summaries.mjs | 100 +++++++++++++++ scripts/e2e-cloud.sh | 64 ++++++---- .../test/cloud-e2e-logging-receipt.test.sh | 118 +++++++++++++++--- 3 files changed, 244 insertions(+), 38 deletions(-) create mode 100644 scripts/e2e-cloud-task-summaries.mjs diff --git a/scripts/e2e-cloud-task-summaries.mjs b/scripts/e2e-cloud-task-summaries.mjs new file mode 100644 index 000000000..a5f9e9997 --- /dev/null +++ b/scripts/e2e-cloud-task-summaries.mjs @@ -0,0 +1,100 @@ +#!/usr/bin/env node +/** Validate per-task Playwright footer or explicit empty-shard log evidence. */ +import { readFileSync } from 'node:fs' + +const [execution, expectedTaskCountText] = process.argv.slice(2) +const expectedTaskCount = Number.parseInt(expectedTaskCountText ?? '', 10) + +if (!execution || !Number.isInteger(expectedTaskCount) || expectedTaskCount < 1) { + console.error('usage: e2e-cloud-task-summaries.mjs <execution> <expected-task-count>') + process.exit(2) +} + +let entries +try { + entries = JSON.parse(readFileSync(0, 'utf8')) +} catch (error) { + console.error(`Cloud Logging response is not valid JSON: ${error instanceof Error ? error.message : String(error)}`) + process.exit(2) +} +if (!Array.isArray(entries)) { + console.error('Cloud Logging response must be a JSON array') + process.exit(2) +} + +const ansiEscape = /\u001B\[[0-?]*[ -/]*[@-~]/g +const footerPattern = /^\s*[0-9]+\s+(?:failed|passed|skipped|interrupted)\b/ +const emptyShardMarker = '[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt.' +const footerTaskIndexes = new Set() +const emptyTaskIndexes = new Set() +const footerLinesByTask = new Map() + +for (const [entryIndex, entry] of entries.entries()) { + const labels = entry?.labels + const textPayload = entry?.textPayload + if (typeof textPayload !== 'string') { + console.error(`Cloud Logging task-summary entry ${entryIndex} has no textPayload`) + process.exit(2) + } + const normalizedTextPayload = textPayload.replace(ansiEscape, '') + const isFooter = footerPattern.test(normalizedTextPayload) + const isEmptyShard = normalizedTextPayload.trim() === emptyShardMarker + if (!isFooter && !isEmptyShard) continue + + const entryExecution = labels?.['run.googleapis.com/execution_name'] ?? labels?.execution_name + const taskIndexValue = labels?.['run.googleapis.com/task_index'] ?? labels?.task_index + const taskIndexText = typeof taskIndexValue === 'number' && Number.isInteger(taskIndexValue) + ? String(taskIndexValue) + : taskIndexValue + + if (entryExecution !== execution || typeof taskIndexText !== 'string' || !/^\d+$/.test(taskIndexText)) { + console.error(`Cloud Logging task-summary entry ${entryIndex} has invalid execution or task-index labels`) + process.exit(2) + } + const taskIndex = Number(taskIndexText) + if (!Number.isInteger(taskIndex) || taskIndex < 0 || taskIndex >= expectedTaskCount) { + console.error(`Cloud Logging task-summary entry ${entryIndex} has out-of-range task index ${taskIndexText}`) + process.exit(2) + } + if (isFooter) { + footerTaskIndexes.add(taskIndex) + const lines = footerLinesByTask.get(taskIndex) ?? new Set() + lines.add(normalizedTextPayload.trim()) + footerLinesByTask.set(taskIndex, lines) + } else if (isEmptyShard) { + emptyTaskIndexes.add(taskIndex) + } +} + +for (const taskIndex of footerTaskIndexes) { + if (emptyTaskIndexes.has(taskIndex)) { + console.error(`task ${taskIndex} has both a Playwright footer and an empty-shard marker`) + process.exit(2) + } +} + +const accountedTaskIndexes = new Set([...footerTaskIndexes, ...emptyTaskIndexes]) +const missingTaskIndexes = Array.from({ length: expectedTaskCount }, (_, taskIndex) => taskIndex) + .filter((taskIndex) => !accountedTaskIndexes.has(taskIndex)) +if (missingTaskIndexes.length > 0) { + console.error(`missing task-indexed Playwright summary evidence for task(s): ${missingTaskIndexes.join(', ')}`) + process.exit(1) +} + +const summaries = [...accountedTaskIndexes] + .sort((left, right) => left - right) + .flatMap((taskIndex) => { + if (emptyTaskIndexes.has(taskIndex)) { + return [{ taskIndex, kind: 'empty', text: emptyShardMarker }] + } + return [...(footerLinesByTask.get(taskIndex) ?? [])] + .map((text) => ({ taskIndex, kind: 'footer', text })) + }) + +process.stdout.write(`${JSON.stringify({ + execution, + taskCount: expectedTaskCount, + footerTaskIndexes: [...footerTaskIndexes].sort((left, right) => left - right), + emptyTaskIndexes: [...emptyTaskIndexes].sort((left, right) => left - right), + summaries, +})}\n`) diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index 8e93100aa..7452de4b7 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -796,28 +796,47 @@ cmd_run() { # Capture to a variable so we can print the full output AND extract a # per-shard summary, even when some shards fail. echo "[e2e-cloud] Fetching logs..." - local log_output="" summary_lines="" display_log_unavailable=false summary_footers_complete=false - local summary_footer_count + local log_output="" summary_lines="" display_log_unavailable=false task_summaries_complete=false + local task_summary_query task_summary_logs task_summary_receipts="" task_summary_error="" + local -a task_summary_account_args=() local summary_attempt + if [ -n "${GCP_ACCOUNT:-}" ]; then + task_summary_account_args=(--account="$GCP_ACCOUNT") + fi + # Cloud Run's raw textPayload can begin with cursor-control ANSI sequences, + # so the Logging filter intentionally matches the status phrase unanchored; + # the parser strips ANSI and confirms that the normalized line is a footer. + task_summary_query="resource.type=\"cloud_run_job\" AND labels.\"run.googleapis.com/execution_name\"=\"${execution_id}\" AND (textPayload =~ \"[0-9]+ (failed|passed|skipped|interrupted)\" OR textPayload=\"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt.\")" for summary_attempt in 1 2 3 4 5; do if log_output=$(gcloud beta run jobs executions logs read $(gcloud_flags) "$execution_id" 2>&1); then summary_lines=$(printf '%s\n' "$log_output" | node "$ROOT/scripts/testing/e2e-cloud-log-summary.mjs") display_log_unavailable=false - summary_footer_count=$(printf '%s\n' "$summary_lines" \ - | grep -Ec '^[[:space:]]+[0-9]+ (failed|passed|skipped|interrupted)\b' || true) - summary_footers_complete=false - if [ "$summary_footer_count" -ge "$shards" ]; then - summary_footers_complete=true - fi - if $summary_footers_complete || $playwright_dry_run; then - break - fi else log_output="" display_log_unavailable=true fi + + task_summaries_complete=false + task_summary_error="" + if $playwright_dry_run; then + break + fi + if task_summary_logs=$(gcloud logging read "$task_summary_query" "${task_summary_account_args[@]}" \ + --project="$GCP_PROJECT" --format=json --limit=1000 2>&1); then + if task_summary_receipts=$(printf '%s' "$task_summary_logs" \ + | node "$ROOT/scripts/e2e-cloud-task-summaries.mjs" "$execution_id" "$shards" 2>&1); then + task_summaries_complete=true + else + task_summary_error="$task_summary_receipts" + fi + else + task_summary_error="$task_summary_logs" + fi + if $task_summaries_complete; then + break + fi if [ "$summary_attempt" -lt 5 ]; then - echo "[e2e-cloud] Waiting for Playwright display-log summary (attempt ${summary_attempt}/5)." >&2 + echo "[e2e-cloud] Waiting for task-indexed Playwright summary evidence (attempt ${summary_attempt}/5): ${task_summary_error}" >&2 sleep 3 fi done @@ -826,8 +845,9 @@ cmd_run() { # display, but never decides whether retry evidence exists; that decision # comes from the required jsonPayload query below. echo "[e2e-cloud] WARNING: could not read display logs after five attempts." >&2 - elif ! $summary_footers_complete && ! $playwright_dry_run; then - echo "[e2e-cloud] WARNING: Playwright summary footers for all ${shards} shard(s) were not present in display logs after five attempts." >&2 + fi + if ! $task_summaries_complete && ! $playwright_dry_run; then + echo "[e2e-cloud] WARNING: task-indexed Playwright footer or empty-shard evidence for all ${shards} task(s) was not present in Cloud Logging after five attempts." >&2 fi # Print logs from every shard. Retry traces are retained as bounded base64 @@ -886,16 +906,16 @@ cmd_run() { fi fi - # Extract and display a per-shard summary from the Playwright output. - # Each shard's entrypoint prints "Shard X/Y assignment" and Playwright's - # line reporter prints a final " N passed (duration)" or - # " N failed, M passed (duration)" summary line. Cloud Logging prefixes - # each returned line with its timestamp and can preserve terminal cursor - # controls from Playwright, so normalize those before the target validates - # the footer's pass/fail/skip counts. + # Display only the summary rows whose raw textPayload entries were validated + # against this execution's Cloud Run task indexes. The formatted logs above + # can lag independently, so they are not a reliable source for this section. echo "" echo "[e2e-cloud] Per-shard summary:" - printf '%s\n' "$summary_lines" + if $playwright_dry_run; then + printf '%s\n' "$summary_lines" + elif $task_summaries_complete; then + jq -r '.summaries[] | "[Cloud Run task \(.taskIndex)]\n\(.text)"' <<< "$task_summary_receipts" + fi # Check execution status — transient describe errors right after # `execute --wait` are a real flake class, so retry briefly; a PERMANENT diff --git a/scripts/test/cloud-e2e-logging-receipt.test.sh b/scripts/test/cloud-e2e-logging-receipt.test.sh index 2fbefeb56..031f93b10 100644 --- a/scripts/test/cloud-e2e-logging-receipt.test.sh +++ b/scripts/test/cloud-e2e-logging-receipt.test.sh @@ -30,12 +30,24 @@ case "$*" in *"executions logs read"*) count_file="$STUB_CAPTURE/logs.count" count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" - if [ "${STUB_DELAY_SECOND_SHARD_FOOTER_ONCE:-0}" = 1 ]; then + if [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then + echo "[e2e-entrypoint] Shard 1/4 assignment: focused fixture" + echo "[e2e-entrypoint] Shard 2/4 assignment: focused fixture" + echo "[e2e-entrypoint] Shard 3/4 assignment: focused fixture" + echo "[e2e-entrypoint] Shard 4/4 assignment: focused fixture" + echo "[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt." + elif [ "${STUB_DELAY_TASK_FOOTER_ONCE:-0}" = 1 ]; then echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" echo "[e2e-entrypoint] Shard 2/2 assignment: focused fixture" echo " 2 passed (5.0s)" + echo " 1 skipped (1.0s)" if [ "$count" -gt 1 ]; then - echo " 2 passed (7.0s)" + echo " 3 passed (7.0s)" + fi + elif [ "${STUB_DELAY_INDEXED_DISPLAY_ONCE:-0}" = 1 ]; then + echo "[e2e-entrypoint] Shard 1/1 assignment: focused fixture" + if [ "$count" -gt 1 ]; then + echo " 3 passed (42.9s)" fi elif [ "${STUB_DELAY_SUMMARY_ONCE:-0}" = 1 ] && [ "$count" -eq 1 ]; then echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" @@ -49,6 +61,57 @@ case "$*" in *'labels.execution_name'*) echo "obsolete execution label" >&2; exit 9 ;; *) echo "missing or incorrect Cloud Run Jobs execution label" >&2; exit 9 ;; esac + if [[ "$*" == *"textPayload =~"* ]]; then + count_file="$STUB_CAPTURE/task-footers.count" + count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" + if [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"1"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"2"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"3"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."} +] +JSON + elif [ "${STUB_DELAY_TASK_FOOTER_ONCE:-0}" = 1 ]; then + if [[ "$*" == *'textPayload =~ "^[[:space:]]*'* ]]; then + # Cloud Logging applies its filter to the raw payload, before ANSI + # controls are stripped. The old anchored query misses this footer. + echo '[]' + elif [ "$count" -eq 1 ]; then + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"\u001b[1A\u001b[2K 2 passed (5.0s)"}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"\u001b[1A\u001b[2K 1 skipped (1.0s)"} +] +JSON + else + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"\u001b[1A\u001b[2K 2 passed (5.0s)"}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"\u001b[1A\u001b[2K 1 skipped (1.0s)"}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"1"},"textPayload":"\u001b[1A\u001b[2K 3 passed (7.0s)"} +] +JSON + fi + elif [ "${STUB_DELAY_INDEXED_DISPLAY_ONCE:-0}" = 1 ]; then + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"\u001b[1A\u001b[2K 3 passed (42.9s)"} +] +JSON + elif [ "${STUB_DELAY_SUMMARY_ONCE:-0}" = 1 ] && [ "$count" -eq 1 ]; then + echo '[]' + else + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":" 2 passed (5.0s)"}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"1"},"textPayload":" 2 passed (7.0s)"} +] +JSON + fi + exit 0 + fi if [ "${STUB_LOGGING_FAIL:-0}" = 1 ]; then echo "logging unavailable" >&2; exit 7; fi count_file="$STUB_CAPTURE/logging.count" count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" @@ -68,6 +131,9 @@ records_no_retry='[ {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":0,"taskCount":2,"recoveredRetryCount":0}}, {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":1,"taskCount":2,"recoveredRetryCount":0}} ]' +records_one_task='[ + {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":0,"taskCount":1,"recoveredRetryCount":0}} +]' # --dry-run is an entrypoint diagnostic: it deliberately does not run # Playwright and therefore cannot emit a completion receipt. The wrapper must @@ -128,18 +194,36 @@ if ! grep -Fq 'labels."run.googleapis.com/execution_name"="exec-json-receipt"' " cat "$WORK/capture/gcloud.args"; exit 1 fi -# A footer from the first task is not a complete display summary when the -# requested run has multiple shards. Keep reading until every shard footer is -# visible, even if one task's footer arrives before the others. -rm -f "$WORK/capture/logs.count" "$WORK/capture/logging.count" "$WORK/capture/gcloud.args" -PARTIAL_SHARD_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_SECOND_SHARD_FOOTER_ONCE=1 STUB_LOGGING_JSON="$records_no_retry" GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" || { - echo "FAIL: two-shard run with delayed display footer failed"; echo "$PARTIAL_SHARD_OUT"; exit 1 +# Multiple summary lines can come from one Cloud Run task. The per-task JSON +# records below attribute both task 0 statuses to index 0; task 1 is absent on +# the first query and appears on the next query. The merged display output has +# no task identity, so it must not satisfy the two-task requirement by count. +rm -f "$WORK/capture/logs.count" "$WORK/capture/logging.count" "$WORK/capture/task-footers.count" "$WORK/capture/gcloud.args" +TASK_SCOPED_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_TASK_FOOTER_ONCE=1 STUB_LOGGING_JSON="$records_no_retry" GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" || { + echo "FAIL: two-task run with a delayed second task footer failed"; echo "$TASK_SCOPED_OUT"; exit 1 +} +TASK_FOOTER_READS="$(cat "$WORK/capture/task-footers.count" 2>/dev/null || echo 0)" +if [ "$TASK_FOOTER_READS" -ne 2 ] \ + || [ "$(cat "$WORK/capture/logs.count")" -lt 2 ] \ + || grep -q "task-indexed Playwright footer or empty-shard evidence" <<< "$TASK_SCOPED_OUT" \ + || ! grep -q "1 skipped (1.0s)" <<< "$TASK_SCOPED_OUT" \ + || ! grep -q "3 passed (7.0s)" <<< "$TASK_SCOPED_OUT"; then + echo "FAIL: multiple task 0 footer lines incorrectly satisfied the two-task summary wait" + echo "$TASK_SCOPED_OUT"; cat "$WORK/capture/gcloud.args"; exit 1 +fi + +# Task-indexed Cloud Logging can have the footer before the human-formatted +# executions-log read catches up. The accepted per-shard summary must come +# from the indexed payload so that a stale display read cannot erase it. +rm -f "$WORK/capture/logs.count" "$WORK/capture/logging.count" "$WORK/capture/task-footers.count" "$WORK/capture/gcloud.args" +INDEXED_EARLY_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_INDEXED_DISPLAY_ONCE=1 STUB_LOGGING_JSON="$records_one_task" STUB_SUCCEEDED=1 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=1 2>&1)" || { + echo "FAIL: one-task run with an early indexed footer failed"; echo "$INDEXED_EARLY_OUT"; exit 1 } -if [ "$(cat "$WORK/capture/logs.count")" -lt 2 ] \ - || ! grep -q "2 passed (5.0s)" <<< "$PARTIAL_SHARD_OUT" \ - || ! grep -q "2 passed (7.0s)" <<< "$PARTIAL_SHARD_OUT"; then - echo "FAIL: wrapper stopped reading display logs before every shard footer arrived" - echo "$PARTIAL_SHARD_OUT"; cat "$WORK/capture/gcloud.args"; exit 1 +INDEXED_SUMMARY="$(sed -n '/^\[e2e-cloud\] Per-shard summary:$/,/^\[e2e-cloud\] Succeeded tasks:/p' <<< "$INDEXED_EARLY_OUT")" +if [ "$(cat "$WORK/capture/logs.count" 2>/dev/null || echo 0)" -ne 1 ] \ + || ! grep -Fq '[Cloud Run task 0]' <<< "$INDEXED_SUMMARY" \ + || ! grep -Fq '3 passed (42.9s)' <<< "$INDEXED_SUMMARY"; then + echo "FAIL: wrapper summary omitted a task-attributed footer already present in Cloud Logging"; echo "$INDEXED_EARLY_OUT"; cat "$WORK/capture/gcloud.args"; exit 1 fi records_empty_shards='[ @@ -148,11 +232,13 @@ records_empty_shards='[ {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":2,"taskCount":4,"recoveredRetryCount":0}}, {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":3,"taskCount":4,"recoveredRetryCount":0}} ]' -EMPTY_SHARDS_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_LOGGING_JSON="$records_empty_shards" STUB_SUCCEEDED=4 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=4 2>&1)" || { +EMPTY_SHARDS_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_EMPTY_SHARDS=1 STUB_LOGGING_JSON="$records_empty_shards" STUB_SUCCEEDED=4 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=4 2>&1)" || { echo "FAIL: a narrow multi-task run with empty shards did not accept one zero-retry receipt for each task"; echo "$EMPTY_SHARDS_OUT"; exit 1 } -if ! grep -q "All tasks completed successfully" <<< "$EMPTY_SHARDS_OUT"; then - echo "FAIL: complete zero-retry receipts from empty shards did not allow success"; echo "$EMPTY_SHARDS_OUT"; exit 1 +if ! grep -q "All tasks completed successfully" <<< "$EMPTY_SHARDS_OUT" \ + || grep -Eq '^[[:space:]]+[0-9]+ (failed|passed|skipped|interrupted)\b' <<< "$EMPTY_SHARDS_OUT" \ + || grep -q "task-indexed Playwright footer or empty-shard evidence" <<< "$EMPTY_SHARDS_OUT"; then + echo "FAIL: empty shards without Playwright footers did not retain zero-test completion semantics"; echo "$EMPTY_SHARDS_OUT"; exit 1 fi records_retry='[ From 24b8ac32cc34a1e8860f8755d159df674c56ac80 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 20:03:16 -0700 Subject: [PATCH 424/493] fix(managed): resolve nested URL output bindings --- .../managed_mcp_capability/source_stage.rs | 61 ++- .../opencode_resources.rs | 464 +++++++++++++++--- 2 files changed, 468 insertions(+), 57 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index b4cfad0ce..8299840f2 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1537,6 +1537,11 @@ process.stdout.write(require('./helper.js')) let append_output = home.join("existing-output.txt"); let append_url_output = home.join("existing-url-output.txt"); let append_variable_url_output = home.join("existing-url-variable-output.txt"); + let append_nested_url_output = home.join("existing-nested-url-variable-output.txt"); + let append_deleted_property_url_output = + home.join("existing-deleted-property-url-output.txt"); + let append_parenthesized_property_url_output = + home.join("existing-parenthesized-property-url-output.txt"); let append_let_variable_url_output = home.join("existing-let-url-variable-output.txt"); let append_var_variable_url_output = home.join("existing-var-url-variable-output.txt"); let reassigned_url_output = home.join("existing-reassigned-url-output.txt"); @@ -1547,12 +1552,32 @@ process.stdout.write(require('./helper.js')) std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nconst stateUrl = new URL('./existing-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(stateUrl, ' appended');\nlet letUrl = new URL('./existing-let-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(letUrl, ' appended');\nvar varUrl = new URL('./existing-var-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(varUrl, ' appended');\nlet reassignedUrl = new URL('./existing-reassigned-url-output.txt', import.meta.url);\nreassignedUrl = new URL('./existing-reassigned-url-output.txt', import.meta.url);\nfs.appendFileSync(reassignedUrl, ' appended');\nconst url = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nfunction writeParameter(url) { fs.appendFileSync(url, ' appended'); }\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", + "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.writeFileSync(path.join(__dirname, 'generated-path.json'), 'path output');\nfs.writeFileSync(new URL('./generated-url.json', import.meta.url), 'url output');\nfs.appendFileSync(path.join(__dirname, 'existing-output.txt'), ' appended');\nfs.appendFileSync(new URL('./existing-url-output.txt', import.meta.url), ' appended');\nconst stateUrl = new URL('./existing-url-variable-output.txt', import.meta.url);\nconsole.error(stateUrl.href);\nfs.appendFileSync(stateUrl, ' appended');\nconst nestedStateUrl = new URL('./existing-nested-url-variable-output.txt', import.meta.url);\nif (true) { fs.appendFileSync(nestedStateUrl, ' appended'); }\nconst deletedPropertyUrl = new URL('./existing-deleted-property-url-output.txt', import.meta.url);\ndelete deletedPropertyUrl.href;\nfs.appendFileSync(deletedPropertyUrl, ' appended');\nlet letUrl = new URL('./existing-let-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(letUrl, ' appended');\nvar varUrl = new URL('./existing-var-url-variable-output.txt', import.meta.url);\nfs.appendFileSync(varUrl, ' appended');\nlet reassignedUrl = new URL('./existing-reassigned-url-output.txt', import.meta.url);\nreassignedUrl = new URL('./existing-reassigned-url-output.txt', import.meta.url);\nfs.appendFileSync(reassignedUrl, ' appended');\nconst url = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nfunction writeParameter(url) { fs.appendFileSync(url, ' appended'); }\nfs.copyFileSync(path.join(__dirname, 'copy-source.txt'), path.join(__dirname, 'copy-output.txt'));\nprocess.stdout.write(__dirname + '\\n');\n", + ) + .unwrap(); + let mut command_file = std::fs::OpenOptions::new() + .append(true) + .open(&command) + .unwrap(); + std::io::Write::write_all( + &mut command_file, + b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\n", ) .unwrap(); std::fs::write(&append_output, "existing").unwrap(); std::fs::write(&append_url_output, "url existing").unwrap(); std::fs::write(&append_variable_url_output, "url variable existing").unwrap(); + std::fs::write(&append_nested_url_output, "nested url variable existing").unwrap(); + std::fs::write( + &append_deleted_property_url_output, + "deleted property URL existing", + ) + .unwrap(); + std::fs::write( + &append_parenthesized_property_url_output, + "parenthesized property URL existing", + ) + .unwrap(); std::fs::write(&append_let_variable_url_output, "let url variable existing").unwrap(); std::fs::write(&append_var_variable_url_output, "var url variable existing").unwrap(); std::fs::write(&reassigned_url_output, "reassigned url existing").unwrap(); @@ -1575,6 +1600,8 @@ process.stdout.write(require('./helper.js')) for output in [ &append_url_output, &append_variable_url_output, + &append_nested_url_output, + &append_parenthesized_property_url_output, &append_let_variable_url_output, &append_var_variable_url_output, &reassigned_url_output, @@ -1605,6 +1632,7 @@ process.stdout.write(require('./helper.js')) &append_output, &append_url_output, &append_variable_url_output, + &append_nested_url_output, &append_let_variable_url_output, &append_var_variable_url_output, ] { @@ -1620,7 +1648,11 @@ process.stdout.write(require('./helper.js')) !mounts.is_empty(), "existing static output must be an exact file mount" ); - assert!(mounts.iter().all(|(_, mount)| mount.writable)); + assert!( + mounts.iter().all(|(_, mount)| mount.writable), + "{} should be writable for a direct URL output", + output.display() + ); } let unrelated_url_mounts = units .iter() @@ -1633,6 +1665,31 @@ process.stdout.write(require('./helper.js')) assert!(unrelated_url_mounts .iter() .all(|(_, mount)| !mount.writable)); + let deleted_property_url_mounts = units + .iter() + .filter(|(source, mount)| { + source == &append_deleted_property_url_output.canonicalize().unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!(!deleted_property_url_mounts.is_empty()); + assert!(deleted_property_url_mounts + .iter() + .all(|(_, mount)| !mount.writable)); + let parenthesized_property_url_mounts = units + .iter() + .filter(|(source, mount)| { + source + == &append_parenthesized_property_url_output + .canonicalize() + .unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!(!parenthesized_property_url_mounts.is_empty()); + assert!(parenthesized_property_url_mounts + .iter() + .all(|(_, mount)| !mount.writable)); let reassigned_url_mounts = units .iter() .filter(|(source, mount)| { diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index cf91e66ee..650f03d28 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1301,16 +1301,23 @@ fn node_relative_resource_specifiers( source: &str, ) -> Result<std::collections::BTreeMap<String, bool>, &'static str> { let tokens = javascript_tokens(source)?; - let scope_ids = javascript_token_scope_ids(&tokens)?; + let scopes = javascript_lexical_scopes(&tokens)?; let (output_destinations, output_destination_variables) = - node_output_destination_mask(&tokens, &scope_ids)?; - let binding_declarations = node_variable_binding_declaration_counts(&tokens, &scope_ids); - let ambiguous_output_variables = node_ambiguous_output_variables( + node_output_destination_mask(&tokens, &scopes.token_scope_ids)?; + let binding_declarations = node_variable_binding_declaration_counts(&tokens, &scopes); + let mut binding_resolver = JavaScriptBindingResolver::new(&scopes, &binding_declarations); + let mut output_destination_bindings = std::collections::BTreeSet::new(); + for (name, reference_scope) in &output_destination_variables { + if let Some(binding_scope) = binding_resolver.resolve(name, *reference_scope)? { + output_destination_bindings.insert((name.clone(), binding_scope)); + } + } + let ambiguous_output_bindings = node_ambiguous_output_bindings( &tokens, - &scope_ids, + &scopes, &output_destinations, - &output_destination_variables, - ); + &mut binding_resolver, + )?; let mut result = std::collections::BTreeMap::<String, bool>::new(); for index in 0..tokens.len() { if index >= 2 @@ -1415,16 +1422,14 @@ fn node_relative_resource_specifiers( tokens.get(index + 11), Some(JavaScriptToken::Punctuation(b')')) ))); - let is_bound_output_destination = index >= 3 - && matches!( - tokens.get(index - 1), - Some(JavaScriptToken::Punctuation(b'=')) - ) - && matches!(tokens.get(index - 2), Some(JavaScriptToken::Identifier(name)) - if output_destination_variables.contains(&(name.clone(), scope_ids[index])) - && binding_declarations.get(&(name.clone(), scope_ids[index])) == Some(&1) - && !ambiguous_output_variables.contains(&(name.clone(), scope_ids[index]))) - && matches!(tokens.get(index - 3), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")); + let is_bound_output_destination = node_simple_url_binding_declaration( + &tokens, index, &scopes, + ) + .is_some_and(|(name, binding_scope)| { + output_destination_bindings.contains(&(name.clone(), binding_scope)) + && binding_declarations.get(&(name.clone(), binding_scope)) == Some(&1) + && !ambiguous_output_bindings.contains(&(name, binding_scope)) + }); let is_output_destination = output_destinations[index] || is_bound_output_destination; if new_keyword == "new" && url_constructor == "URL" @@ -1442,87 +1447,436 @@ fn node_relative_resource_specifiers( Ok(result) } -fn javascript_token_scope_ids(tokens: &[JavaScriptToken]) -> Result<Vec<usize>, &'static str> { +struct JavaScriptScopes { + token_scope_ids: Vec<usize>, + parent_scope_ids: Vec<Option<usize>>, + function_scope_ids: std::collections::BTreeSet<usize>, + parameter_names: std::collections::BTreeMap<usize, std::collections::BTreeSet<String>>, + parameter_token_indices: std::collections::BTreeSet<usize>, +} + +impl JavaScriptScopes { + fn function_scope_for(&self, mut scope_id: usize) -> usize { + loop { + if self.function_scope_ids.contains(&scope_id) { + return scope_id; + } + scope_id = self.parent_scope_ids[scope_id].unwrap_or(0); + } + } +} + +fn javascript_matching_open_parentheses(tokens: &[JavaScriptToken]) -> Vec<Option<usize>> { + let mut openings = Vec::new(); + let mut matching_openings = vec![None; tokens.len()]; + for (index, token) in tokens.iter().enumerate() { + match token { + JavaScriptToken::Punctuation(b'(') => openings.push(index), + JavaScriptToken::Punctuation(b')') => { + if let Some(opening) = openings.pop() { + matching_openings[index] = Some(opening); + } + } + _ => {} + } + } + matching_openings +} + +fn javascript_arrow_parameter_range( + tokens: &[JavaScriptToken], + arrow_index: usize, + matching_openings: &[Option<usize>], +) -> Option<std::ops::Range<usize>> { + let previous_index = arrow_index.checked_sub(1)?; + match tokens.get(previous_index) { + Some(JavaScriptToken::Identifier(_)) => Some(previous_index..arrow_index), + Some(JavaScriptToken::Punctuation(b')')) => { + let opening = matching_openings.get(previous_index).copied().flatten()?; + Some(opening + 1..previous_index) + } + _ => None, + } +} + +fn javascript_scope_header( + tokens: &[JavaScriptToken], + brace_index: usize, + matching_openings: &[Option<usize>], +) -> (bool, Option<std::ops::Range<usize>>) { + if brace_index >= 2 + && matches!( + tokens.get(brace_index - 1), + Some(JavaScriptToken::Punctuation(b'>')) + ) + && matches!( + tokens.get(brace_index - 2), + Some(JavaScriptToken::Punctuation(b'=')) + ) + { + return ( + true, + javascript_arrow_parameter_range(tokens, brace_index - 2, matching_openings), + ); + } + + if brace_index == 0 + || !matches!( + tokens.get(brace_index - 1), + Some(JavaScriptToken::Punctuation(b')')) + ) + { + return (false, None); + } + let close = brace_index - 1; + let Some(open) = matching_openings.get(close).copied().flatten() else { + return (false, None); + }; + let parameters = Some(open + 1..close); + match open.checked_sub(1).and_then(|index| tokens.get(index)) { + Some(JavaScriptToken::Identifier(keyword)) if keyword == "catch" => (false, parameters), + Some(JavaScriptToken::Identifier(keyword)) + if matches!(keyword.as_str(), "if" | "for" | "while" | "switch" | "with") => + { + (false, None) + } + Some(JavaScriptToken::Identifier(_)) | Some(JavaScriptToken::String(_)) => { + (true, parameters) + } + Some(JavaScriptToken::Punctuation(b']' | b')')) => (true, parameters), + _ => (false, None), + } +} + +fn javascript_add_parameter_names( + tokens: &[JavaScriptToken], + scope_id: usize, + parameter_range: std::ops::Range<usize>, + parameter_names: &mut std::collections::BTreeMap<usize, std::collections::BTreeSet<String>>, + parameter_token_indices: &mut std::collections::BTreeSet<usize>, + inspection_budget: &mut usize, +) -> Result<(), &'static str> { + for index in parameter_range { + *inspection_budget = inspection_budget.saturating_sub(1); + if *inspection_budget == 0 { + return Err("direct MCP Node parameter scan exceeds its token bound"); + } + if let Some(JavaScriptToken::Identifier(name)) = tokens.get(index) { + parameter_names + .entry(scope_id) + .or_default() + .insert(name.clone()); + parameter_token_indices.insert(index); + } + } + Ok(()) +} + +fn javascript_lexical_scopes(tokens: &[JavaScriptToken]) -> Result<JavaScriptScopes, &'static str> { + let matching_openings = javascript_matching_open_parentheses(tokens); + let mut token_scope_ids = Vec::with_capacity(tokens.len()); + let mut parent_scope_ids = vec![None]; + let mut function_scope_ids = std::collections::BTreeSet::from([0]); + let mut parameter_names = std::collections::BTreeMap::new(); + let mut parameter_token_indices = std::collections::BTreeSet::new(); let mut scope_stack = Vec::new(); - let mut scope_ids = Vec::with_capacity(tokens.len()); + let mut parameter_inspection_budget = MAX_JAVASCRIPT_TOKENS; + for (index, token) in tokens.iter().enumerate() { if matches!(token, JavaScriptToken::Punctuation(b'}')) && scope_stack.pop().is_none() { return Err("direct MCP Node script has an unmatched closing brace"); } - scope_ids.push(scope_stack.last().copied().unwrap_or(0)); + token_scope_ids.push(scope_stack.last().copied().unwrap_or(0)); if matches!(token, JavaScriptToken::Punctuation(b'{')) { - // Reserve zero for the top-level scope and give each brace scope a - // unique token-derived identity. This deliberately treats nested - // blocks as separate scopes, so uncertain captures remain read-only. - scope_stack.push(index + 1); + let (is_function_scope, parameters) = + javascript_scope_header(tokens, index, &matching_openings); + let parent_scope = scope_stack.last().copied().unwrap_or(0); + let new_scope = parent_scope_ids.len(); + parent_scope_ids.push(Some(parent_scope)); + if is_function_scope { + function_scope_ids.insert(new_scope); + } + if let Some(parameters) = parameters { + javascript_add_parameter_names( + tokens, + new_scope, + parameters, + &mut parameter_names, + &mut parameter_token_indices, + &mut parameter_inspection_budget, + )?; + } + scope_stack.push(new_scope); } } if !scope_stack.is_empty() { return Err("direct MCP Node script has an unmatched opening brace"); } - Ok(scope_ids) + + // A concise arrow expression has no brace scope in the token stream. Add + // its parameters to the containing scope so a same-name output is treated + // as shadowed (conservatively for other expressions in that scope too). + for (index, scope_id) in token_scope_ids + .iter() + .enumerate() + .skip(1) + .take(tokens.len().saturating_sub(2)) + { + if !matches!(tokens.get(index), Some(JavaScriptToken::Punctuation(b'='))) + || !matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'>')) + ) + || matches!( + tokens.get(index + 2), + Some(JavaScriptToken::Punctuation(b'{')) + ) + { + continue; + } + if let Some(parameters) = + javascript_arrow_parameter_range(tokens, index, &matching_openings) + { + let scope_id = *scope_id; + javascript_add_parameter_names( + tokens, + scope_id, + parameters, + &mut parameter_names, + &mut parameter_token_indices, + &mut parameter_inspection_budget, + )?; + } + } + + Ok(JavaScriptScopes { + token_scope_ids, + parent_scope_ids, + function_scope_ids, + parameter_names, + parameter_token_indices, + }) +} + +fn node_variable_binding_scope(kind: &str, scope_id: usize, scopes: &JavaScriptScopes) -> usize { + if kind == "var" { + scopes.function_scope_for(scope_id) + } else { + scope_id + } } fn node_variable_binding_declaration_counts( tokens: &[JavaScriptToken], - scope_ids: &[usize], + scopes: &JavaScriptScopes, ) -> std::collections::BTreeMap<(String, usize), usize> { let mut declarations = std::collections::BTreeMap::new(); for index in 0..tokens.len().saturating_sub(1) { - if !matches!(tokens.get(index), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")) - { + let Some(JavaScriptToken::Identifier(kind)) = tokens.get(index) else { + continue; + }; + if !matches!(kind.as_str(), "const" | "let" | "var") { continue; } let Some(JavaScriptToken::Identifier(name)) = tokens.get(index + 1) else { continue; }; - let key = (name.clone(), scope_ids[index]); - *declarations.entry(key).or_insert(0) += 1; + let scope_id = node_variable_binding_scope(kind, scopes.token_scope_ids[index], scopes); + *declarations.entry((name.clone(), scope_id)).or_insert(0) += 1; + } + for (scope_id, names) in &scopes.parameter_names { + for name in names { + *declarations.entry((name.clone(), *scope_id)).or_insert(0) += 1; + } } declarations } -fn node_ambiguous_output_variables( +struct JavaScriptBindingResolver<'a> { + scopes: &'a JavaScriptScopes, + declarations: &'a std::collections::BTreeMap<(String, usize), usize>, + resolved: std::collections::BTreeMap<(String, usize), Option<usize>>, + inspection_budget: usize, +} + +impl<'a> JavaScriptBindingResolver<'a> { + fn new( + scopes: &'a JavaScriptScopes, + declarations: &'a std::collections::BTreeMap<(String, usize), usize>, + ) -> Self { + Self { + scopes, + declarations, + resolved: std::collections::BTreeMap::new(), + inspection_budget: MAX_JAVASCRIPT_TOKENS, + } + } + + fn resolve(&mut self, name: &str, mut scope_id: usize) -> Result<Option<usize>, &'static str> { + let cache_key = (name.to_owned(), scope_id); + if let Some(resolved) = self.resolved.get(&cache_key) { + return Ok(*resolved); + } + let result = loop { + self.inspection_budget = self.inspection_budget.saturating_sub(1); + if self.inspection_budget == 0 { + return Err("direct MCP Node binding resolution exceeds its token bound"); + } + if self.declarations.contains_key(&(name.to_owned(), scope_id)) { + break Some(scope_id); + } + let Some(parent_scope) = self.scopes.parent_scope_ids[scope_id] else { + break None; + }; + scope_id = parent_scope; + }; + self.resolved.insert(cache_key, result); + Ok(result) + } +} + +fn node_simple_url_binding_declaration( tokens: &[JavaScriptToken], - scope_ids: &[usize], + index: usize, + scopes: &JavaScriptScopes, +) -> Option<(String, usize)> { + if index < 3 + || !matches!( + tokens.get(index - 1), + Some(JavaScriptToken::Punctuation(b'=')) + ) + { + return None; + } + let Some(JavaScriptToken::Identifier(name)) = tokens.get(index - 2) else { + return None; + }; + let Some(JavaScriptToken::Identifier(kind)) = tokens.get(index - 3) else { + return None; + }; + if !matches!(kind.as_str(), "const" | "let" | "var") { + return None; + } + Some(( + name.clone(), + node_variable_binding_scope(kind, scopes.token_scope_ids[index], scopes), + )) +} + +fn node_ambiguous_output_bindings( + tokens: &[JavaScriptToken], + scopes: &JavaScriptScopes, output_destinations: &[bool], - output_destination_variables: &std::collections::BTreeSet<(String, usize)>, -) -> std::collections::BTreeSet<(String, usize)> { + resolver: &mut JavaScriptBindingResolver<'_>, +) -> Result<std::collections::BTreeSet<(String, usize)>, &'static str> { let mut ambiguous = std::collections::BTreeSet::new(); for (index, token) in tokens.iter().enumerate() { let JavaScriptToken::Identifier(name) = token else { continue; }; - let key = (name.clone(), scope_ids[index]); - if !output_destination_variables.contains(&key) || output_destinations[index] { - continue; - } - let is_url_binding_declaration = index >= 1 - && matches!(tokens.get(index - 1), Some(JavaScriptToken::Identifier(keyword)) if matches!(keyword.as_str(), "const" | "let" | "var")) - && matches!( + if output_destinations[index] + || scopes.parameter_token_indices.contains(&index) + || is_javascript_variable_declaration_name(tokens, index) + || matches!( + tokens.get(index.checked_sub(1).unwrap_or(usize::MAX)), + Some(JavaScriptToken::Punctuation(b'.')) + ) + || matches!( tokens.get(index + 1), - Some(JavaScriptToken::Punctuation(b'=')) + Some(JavaScriptToken::Punctuation(b':')) ) - && matches!(tokens.get(index + 2), Some(JavaScriptToken::Identifier(keyword)) if keyword == "new") - && matches!(tokens.get(index + 3), Some(JavaScriptToken::Identifier(constructor)) if constructor == "URL") - && matches!( - tokens.get(index + 4), - Some(JavaScriptToken::Punctuation(b'(')) - ); - if !is_url_binding_declaration { - // A second use outside a recognized output argument may reassign - // or mutate the URL. Keep the selected input read-only when this - // bounded scanner cannot prove the binding stays stable. - ambiguous.insert(key); + || javascript_is_safe_property_read(tokens, index) + { + continue; + } + if let Some(binding_scope) = resolver.resolve(name, scopes.token_scope_ids[index])? { + ambiguous.insert((name.clone(), binding_scope)); } } - ambiguous + Ok(ambiguous) } +fn is_javascript_variable_declaration_name(tokens: &[JavaScriptToken], index: usize) -> bool { + index > 0 + && matches!( + tokens.get(index - 1), + Some(JavaScriptToken::Identifier(keyword)) + if matches!(keyword.as_str(), "const" | "let" | "var") + ) +} + +fn javascript_is_safe_property_read(tokens: &[JavaScriptToken], index: usize) -> bool { + if !matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'.')) + ) || !matches!(tokens.get(index + 2), Some(JavaScriptToken::Identifier(_))) + { + return false; + } + + let mut expression_start = index; + while expression_start > 0 + && matches!( + tokens.get(expression_start - 1), + Some(JavaScriptToken::Punctuation(b'(')) + ) + { + expression_start -= 1; + } + if matches!( + tokens.get(expression_start.checked_sub(1).unwrap_or(usize::MAX)), + Some(JavaScriptToken::Identifier(keyword)) if keyword == "delete" + ) || matches!( + ( + tokens.get(expression_start.checked_sub(2).unwrap_or(usize::MAX)), + tokens.get(expression_start.checked_sub(1).unwrap_or(usize::MAX)), + ), + ( + Some(JavaScriptToken::Punctuation(b'+' | b'-')), + Some(JavaScriptToken::Punctuation(b'+' | b'-')), + ) + ) { + return false; + } + + let mut expression_end = index + 3; + while matches!( + tokens.get(expression_end), + Some(JavaScriptToken::Punctuation(b')')) + ) { + expression_end += 1; + } + if matches!( + tokens.get(expression_end), + Some(JavaScriptToken::Punctuation( + b'(' | b'[' + | b'.' + | b'+' + | b'-' + | b'*' + | b'/' + | b'%' + | b'&' + | b'|' + | b'^' + | b'<' + | b'>' + | b'=' + )) + ) { + return false; + } + true +} + +type JavaScriptBindingKey = (String, usize); +type JavaScriptOutputDestinationAnalysis = + (Vec<bool>, std::collections::BTreeSet<JavaScriptBindingKey>); + fn node_output_destination_mask( tokens: &[JavaScriptToken], scope_ids: &[usize], -) -> Result<(Vec<bool>, std::collections::BTreeSet<(String, usize)>), &'static str> { +) -> Result<JavaScriptOutputDestinationAnalysis, &'static str> { const OUTPUT_CALLS: [(&str, usize); 15] = [ ("appendFile", 0), ("appendFileSync", 0), From 726375bb6bd88888c40d1da93cb0d5b6585d2eab Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 20:33:35 -0700 Subject: [PATCH 425/493] fix(testing): require passing Cloud browser evidence --- scripts/e2e-cloud-task-summaries.mjs | 3 ++ scripts/e2e-cloud.sh | 11 +++++++ .../test/cloud-e2e-logging-receipt.test.sh | 33 ++++++++++++++++--- scripts/test/cloud-exec-id-parse.test.sh | 4 +++ scripts/test/cloud-gcp-identity.test.sh | 4 +++ scripts/test/cloud-run-wrapper.test.sh | 16 ++++++++- 6 files changed, 65 insertions(+), 6 deletions(-) diff --git a/scripts/e2e-cloud-task-summaries.mjs b/scripts/e2e-cloud-task-summaries.mjs index a5f9e9997..c3ba94d0d 100644 --- a/scripts/e2e-cloud-task-summaries.mjs +++ b/scripts/e2e-cloud-task-summaries.mjs @@ -28,6 +28,7 @@ const emptyShardMarker = '[e2e-entrypoint] No specs assigned to this shard. Expo const footerTaskIndexes = new Set() const emptyTaskIndexes = new Set() const footerLinesByTask = new Map() +let hasPassedTest = false for (const [entryIndex, entry] of entries.entries()) { const labels = entry?.labels @@ -58,6 +59,7 @@ for (const [entryIndex, entry] of entries.entries()) { } if (isFooter) { footerTaskIndexes.add(taskIndex) + hasPassedTest ||= /\b[1-9][0-9]*\s+passed\b/.test(normalizedTextPayload) const lines = footerLinesByTask.get(taskIndex) ?? new Set() lines.add(normalizedTextPayload.trim()) footerLinesByTask.set(taskIndex, lines) @@ -94,6 +96,7 @@ const summaries = [...accountedTaskIndexes] process.stdout.write(`${JSON.stringify({ execution, taskCount: expectedTaskCount, + hasPassedTest, footerTaskIndexes: [...footerTaskIndexes].sort((left, right) => left - right), emptyTaskIndexes: [...emptyTaskIndexes].sort((left, right) => left - right), summaries, diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index 7452de4b7..a0e074913 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -960,6 +960,17 @@ cmd_run() { exit 1 fi + if ! $playwright_dry_run; then + if ! $task_summaries_complete; then + echo "[e2e-cloud] ERROR: required task-indexed Playwright summary evidence was not present after five attempts: ${task_summary_error}" >&2 + exit 1 + fi + if [ "$(jq -r '.hasPassedTest' <<< "$task_summary_receipts")" != "true" ]; then + echo "[e2e-cloud] ERROR: Cloud execution completed with no passed Playwright tests; skipped tests and empty shards do not count." >&2 + exit 1 + fi + fi + # Playwright retries are useful diagnostics, but a task that passes only on # retry is not a zero-flake release receipt. The structured evidence above # names the exact first failure and durable trace artifact before failing the diff --git a/scripts/test/cloud-e2e-logging-receipt.test.sh b/scripts/test/cloud-e2e-logging-receipt.test.sh index 031f93b10..7b3947732 100644 --- a/scripts/test/cloud-e2e-logging-receipt.test.sh +++ b/scripts/test/cloud-e2e-logging-receipt.test.sh @@ -30,12 +30,16 @@ case "$*" in *"executions logs read"*) count_file="$STUB_CAPTURE/logs.count" count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" - if [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then + if [ "${STUB_SKIPPED_ONLY:-0}" = 1 ]; then + echo "[e2e-entrypoint] Shard 1/1 assignment: focused fixture" + echo " 3 skipped (1.0s)" + elif [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then echo "[e2e-entrypoint] Shard 1/4 assignment: focused fixture" echo "[e2e-entrypoint] Shard 2/4 assignment: focused fixture" echo "[e2e-entrypoint] Shard 3/4 assignment: focused fixture" echo "[e2e-entrypoint] Shard 4/4 assignment: focused fixture" echo "[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt." + echo " 3 passed (2.0s)" elif [ "${STUB_DELAY_TASK_FOOTER_ONCE:-0}" = 1 ]; then echo "[e2e-entrypoint] Shard 1/2 assignment: focused fixture" echo "[e2e-entrypoint] Shard 2/2 assignment: focused fixture" @@ -64,10 +68,18 @@ case "$*" in if [[ "$*" == *"textPayload =~"* ]]; then count_file="$STUB_CAPTURE/task-footers.count" count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" - if [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then + if [ "${STUB_NO_TASK_SUMMARY:-0}" = 1 ]; then + echo '[]' + elif [ "${STUB_SKIPPED_ONLY:-0}" = 1 ]; then + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":" 3 skipped (1.0s)"} +] +JSON + elif [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then cat <<'JSON' [ - {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":" 3 passed (2.0s)"}, {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"1"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."}, {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"2"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."}, {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"3"},"textPayload":"[e2e-entrypoint] No specs assigned to this shard. Exporting zero-test receipt."} @@ -236,9 +248,20 @@ EMPTY_SHARDS_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB echo "FAIL: a narrow multi-task run with empty shards did not accept one zero-retry receipt for each task"; echo "$EMPTY_SHARDS_OUT"; exit 1 } if ! grep -q "All tasks completed successfully" <<< "$EMPTY_SHARDS_OUT" \ - || grep -Eq '^[[:space:]]+[0-9]+ (failed|passed|skipped|interrupted)\b' <<< "$EMPTY_SHARDS_OUT" \ + || ! grep -q "3 passed (2.0s)" <<< "$EMPTY_SHARDS_OUT" \ + || ! grep -q "No specs assigned to this shard" <<< "$EMPTY_SHARDS_OUT" \ || grep -q "task-indexed Playwright footer or empty-shard evidence" <<< "$EMPTY_SHARDS_OUT"; then - echo "FAIL: empty shards without Playwright footers did not retain zero-test completion semantics"; echo "$EMPTY_SHARDS_OUT"; exit 1 + echo "FAIL: empty shards beside a passing task did not retain completion semantics"; echo "$EMPTY_SHARDS_OUT"; exit 1 +fi + +SKIPPED_ONLY_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_SKIPPED_ONLY=1 STUB_LOGGING_JSON="$records_one_task" STUB_SUCCEEDED=1 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=1 2>&1)" && SKIPPED_ONLY_RC=0 || SKIPPED_ONLY_RC=$? +if [ "$SKIPPED_ONLY_RC" -eq 0 ] || ! grep -q "no passed Playwright tests" <<< "$SKIPPED_ONLY_OUT"; then + echo "FAIL: skipped-only Playwright evidence was accepted as a passing Cloud run"; echo "$SKIPPED_ONLY_OUT"; exit 1 +fi + +NO_TASK_SUMMARY_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_NO_TASK_SUMMARY=1 STUB_LOGGING_JSON="$records_no_retry" STUB_SUCCEEDED=2 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" && NO_TASK_SUMMARY_RC=0 || NO_TASK_SUMMARY_RC=$? +if [ "$NO_TASK_SUMMARY_RC" -eq 0 ] || ! grep -q "required task-indexed Playwright summary evidence was not present" <<< "$NO_TASK_SUMMARY_OUT"; then + echo "FAIL: a Cloud run without complete task-indexed Playwright summaries was accepted"; echo "$NO_TASK_SUMMARY_OUT"; exit 1 fi records_retry='[ diff --git a/scripts/test/cloud-exec-id-parse.test.sh b/scripts/test/cloud-exec-id-parse.test.sh index 771d0d15f..e969ce62d 100755 --- a/scripts/test/cloud-exec-id-parse.test.sh +++ b/scripts/test/cloud-exec-id-parse.test.sh @@ -78,6 +78,10 @@ fi if [[ "$*" == *"executions list"* ]]; then echo "test-exec-123"; exit 0; fi if [[ "$*" == *"logs read"* ]]; then echo "Test Files 1 passed (1)"; exit 0; fi if [[ "$*" == *"logging read"* ]]; then + if [[ "$*" == *"textPayload =~"* ]]; then + printf '[{"labels":{"run.googleapis.com/execution_name":"test-exec-123","run.googleapis.com/task_index":"0"},"textPayload":" 1 passed (1.0s)"}]\n' + exit 0 + fi # One zero-retry completion receipt per task, as Cloud Logging would return # for the single-task execution this stub reports above. Without this # branch the e2e wrapper's structured-receipt reconciliation fails closed diff --git a/scripts/test/cloud-gcp-identity.test.sh b/scripts/test/cloud-gcp-identity.test.sh index 1c9d02e3a..29baafcea 100644 --- a/scripts/test/cloud-gcp-identity.test.sh +++ b/scripts/test/cloud-gcp-identity.test.sh @@ -409,6 +409,10 @@ if [[ "$*" == *"auth print-access-token"* ]]; then echo stub-token; exit 0; fi if [[ "$*" == *"executions list"* ]]; then echo "green-exec-1"; exit 0; fi if [[ "$*" == *"logs read"* ]]; then echo " 1 passed (1.0s)"; exit 0; fi if [[ "$*" == *"logging read"* ]]; then + if [[ "$*" == *"textPayload =~"* ]]; then + printf '[{"labels":{"run.googleapis.com/execution_name":"green-exec-1","run.googleapis.com/task_index":"0"},"textPayload":" 1 passed (1.0s)"}]\n' + exit 0 + fi # Structured retry-receipt reconciliation for the green single-task # execution this stub reports. Without this branch the e2e wrapper fails # closed on receipt evidence before the identity-ladder behavior under diff --git a/scripts/test/cloud-run-wrapper.test.sh b/scripts/test/cloud-run-wrapper.test.sh index 366105667..8f4eadb88 100644 --- a/scripts/test/cloud-run-wrapper.test.sh +++ b/scripts/test/cloud-run-wrapper.test.sh @@ -149,7 +149,11 @@ case "$args" in exit 0 ;; *"logs read"*) echo " 6 passed (4.2s)"; exit 0 ;; *"logging read"*) - echo '[{"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-stub","taskIndex":0,"taskCount":1,"recoveredRetryCount":0}}]' + if [[ "$*" == *"textPayload =~"* ]]; then + echo '[{"labels":{"run.googleapis.com/execution_name":"exec-stub","run.googleapis.com/task_index":"0"},"textPayload":" 6 passed (4.2s)"}]' + else + echo '[{"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-stub","taskIndex":0,"taskCount":1,"recoveredRetryCount":0}}]' + fi exit 0 ;; *) exit 0 ;; esac @@ -373,6 +377,16 @@ case "$*" in if [ -n "${STUB2_LOGS:-}" ]; then printf '%s\n' "$STUB2_LOGS"; else echo " 6 passed (4.2s)"; fi exit 0 ;; *"logging read"*) + if [[ "$*" == *"textPayload =~"* ]]; then + tasks=$(cat "$STUB2_CAPTURE/tasks" 2>/dev/null || echo 1) + printf '[' + for ((task = 0; task < tasks; task++)); do + [ "$task" -gt 0 ] && printf ',' + printf '{"labels":{"run.googleapis.com/execution_name":"exec-stub-7","run.googleapis.com/task_index":"%s"},"textPayload":" 6 passed (4.2s)"}' "$task" + done + printf ']\n' + exit 0 + fi if [ -n "${STUB2_STRUCTURED_LOGS:-}" ]; then printf '%s\n' "$STUB2_STRUCTURED_LOGS" exit 0 From 2204ff46984aae171a88f1321f470335b2f72c5d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 20:55:11 -0700 Subject: [PATCH 426/493] fix(managed): detect conditional URL aliases --- .../managed_mcp_capability/source_stage.rs | 21 ++++++++++++++++++- .../opencode_resources.rs | 15 +++++++++---- 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 8299840f2..20d325086 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1540,6 +1540,8 @@ process.stdout.write(require('./helper.js')) let append_nested_url_output = home.join("existing-nested-url-variable-output.txt"); let append_deleted_property_url_output = home.join("existing-deleted-property-url-output.txt"); + let append_conditional_alias_url_output = + home.join("existing-conditional-alias-url-output.txt"); let append_parenthesized_property_url_output = home.join("existing-parenthesized-property-url-output.txt"); let append_let_variable_url_output = home.join("existing-let-url-variable-output.txt"); @@ -1561,7 +1563,7 @@ process.stdout.write(require('./helper.js')) .unwrap(); std::io::Write::write_all( &mut command_file, - b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\n", + b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\nconst conditionalOutputUrl = new URL('./existing-conditional-alias-url-output.txt', import.meta.url);\nconst fallbackUrl = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nconst selectedUrl = true ? conditionalOutputUrl : fallbackUrl;\nselectedUrl.href = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nif (false) fs.appendFileSync(conditionalOutputUrl, ' appended');\n", ) .unwrap(); std::fs::write(&append_output, "existing").unwrap(); @@ -1573,6 +1575,11 @@ process.stdout.write(require('./helper.js')) "deleted property URL existing", ) .unwrap(); + std::fs::write( + &append_conditional_alias_url_output, + "conditional alias URL existing", + ) + .unwrap(); std::fs::write( &append_parenthesized_property_url_output, "parenthesized property URL existing", @@ -1606,6 +1613,7 @@ process.stdout.write(require('./helper.js')) &append_var_variable_url_output, &reassigned_url_output, &unrelated_url_output, + &append_conditional_alias_url_output, ] { assert!(plan[0] .configured_resources @@ -1665,6 +1673,17 @@ process.stdout.write(require('./helper.js')) assert!(unrelated_url_mounts .iter() .all(|(_, mount)| !mount.writable)); + let conditional_alias_url_mounts = units + .iter() + .filter(|(source, mount)| { + source == &append_conditional_alias_url_output.canonicalize().unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .collect::<Vec<_>>(); + assert!(!conditional_alias_url_mounts.is_empty()); + assert!(conditional_alias_url_mounts + .iter() + .all(|(_, mount)| !mount.writable)); let deleted_property_url_mounts = units .iter() .filter(|(source, mount)| { diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 650f03d28..7cc3b109b 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1781,10 +1781,7 @@ fn node_ambiguous_output_bindings( tokens.get(index.checked_sub(1).unwrap_or(usize::MAX)), Some(JavaScriptToken::Punctuation(b'.')) ) - || matches!( - tokens.get(index + 1), - Some(JavaScriptToken::Punctuation(b':')) - ) + || javascript_is_object_property_key(tokens, index) || javascript_is_safe_property_read(tokens, index) { continue; @@ -1805,6 +1802,16 @@ fn is_javascript_variable_declaration_name(tokens: &[JavaScriptToken], index: us ) } +fn javascript_is_object_property_key(tokens: &[JavaScriptToken], index: usize) -> bool { + matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b':')) + ) && matches!( + tokens.get(index.checked_sub(1).unwrap_or(usize::MAX)), + Some(JavaScriptToken::Punctuation(b'{' | b',')) + ) +} + fn javascript_is_safe_property_read(tokens: &[JavaScriptToken], index: usize) -> bool { if !matches!( tokens.get(index + 1), From bf60fcf954be92e26713936d820f341e7d05f8b4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 20:55:14 -0700 Subject: [PATCH 427/493] fix(testing): wait for passed cloud footer --- scripts/e2e-cloud.sh | 5 ++- .../test/cloud-e2e-logging-receipt.test.sh | 39 ++++++++++++++++++- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/scripts/e2e-cloud.sh b/scripts/e2e-cloud.sh index a0e074913..d65f9f23d 100755 --- a/scripts/e2e-cloud.sh +++ b/scripts/e2e-cloud.sh @@ -833,7 +833,10 @@ cmd_run() { task_summary_error="$task_summary_logs" fi if $task_summaries_complete; then - break + if [ "$(jq -r '.hasPassedTest' <<< "$task_summary_receipts")" = "true" ]; then + break + fi + task_summary_error="task-indexed summaries are complete, but no passed Playwright footer has arrived yet" fi if [ "$summary_attempt" -lt 5 ]; then echo "[e2e-cloud] Waiting for task-indexed Playwright summary evidence (attempt ${summary_attempt}/5): ${task_summary_error}" >&2 diff --git a/scripts/test/cloud-e2e-logging-receipt.test.sh b/scripts/test/cloud-e2e-logging-receipt.test.sh index 7b3947732..623af27ba 100644 --- a/scripts/test/cloud-e2e-logging-receipt.test.sh +++ b/scripts/test/cloud-e2e-logging-receipt.test.sh @@ -30,7 +30,13 @@ case "$*" in *"executions logs read"*) count_file="$STUB_CAPTURE/logs.count" count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" - if [ "${STUB_SKIPPED_ONLY:-0}" = 1 ]; then + if [ "${STUB_DELAY_PASSED_AFTER_SKIPPED_ONCE:-0}" = 1 ]; then + echo "[e2e-entrypoint] Shard 1/1 assignment: focused fixture" + echo " 3 skipped (1.0s)" + if [ "$count" -gt 1 ]; then + echo " 2 passed (5.0s)" + fi + elif [ "${STUB_SKIPPED_ONLY:-0}" = 1 ]; then echo "[e2e-entrypoint] Shard 1/1 assignment: focused fixture" echo " 3 skipped (1.0s)" elif [ "${STUB_EMPTY_SHARDS:-0}" = 1 ]; then @@ -70,6 +76,21 @@ case "$*" in count="$(cat "$count_file" 2>/dev/null || echo 0)"; count=$((count + 1)); echo "$count" > "$count_file" if [ "${STUB_NO_TASK_SUMMARY:-0}" = 1 ]; then echo '[]' + elif [ "${STUB_DELAY_PASSED_AFTER_SKIPPED_ONCE:-0}" = 1 ]; then + if [ "$count" -eq 1 ]; then + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":" 3 skipped (1.0s)"} +] +JSON + else + cat <<'JSON' +[ + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":" 3 skipped (1.0s)"}, + {"labels":{"run.googleapis.com/execution_name":"exec-json-receipt","run.googleapis.com/task_index":"0"},"textPayload":" 2 passed (5.0s)"} +] +JSON + fi elif [ "${STUB_SKIPPED_ONLY:-0}" = 1 ]; then cat <<'JSON' [ @@ -138,6 +159,12 @@ JSON esac GCLOUD chmod +x "$WORK/bin/gcloud" +cat > "$WORK/bin/sleep" <<'SLEEP' +#!/usr/bin/env bash +# Poll retries are driven by fixture responses; real waiting adds no coverage. +exit 0 +SLEEP +chmod +x "$WORK/bin/sleep" records_no_retry='[ {"jsonPayload":{"event":"e2e_playwright_task_complete","execution":"exec-json-receipt","taskIndex":0,"taskCount":2,"recoveredRetryCount":0}}, @@ -259,6 +286,16 @@ if [ "$SKIPPED_ONLY_RC" -eq 0 ] || ! grep -q "no passed Playwright tests" <<< "$ echo "FAIL: skipped-only Playwright evidence was accepted as a passing Cloud run"; echo "$SKIPPED_ONLY_OUT"; exit 1 fi +rm -f "$WORK/capture/logs.count" "$WORK/capture/task-footers.count" +DELAYED_PASS_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_DELAY_PASSED_AFTER_SKIPPED_ONCE=1 STUB_LOGGING_JSON="$records_one_task" STUB_SUCCEEDED=1 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=1 2>&1)" && DELAYED_PASS_RC=0 || DELAYED_PASS_RC=$? +if [ "$DELAYED_PASS_RC" -ne 0 ] \ + || [ "$(cat "$WORK/capture/task-footers.count" 2>/dev/null || echo 0)" -ne 2 ] \ + || ! grep -q "3 skipped (1.0s)" <<< "$DELAYED_PASS_OUT" \ + || ! grep -q "2 passed (5.0s)" <<< "$DELAYED_PASS_OUT" \ + || ! grep -q "All tasks completed successfully" <<< "$DELAYED_PASS_OUT"; then + echo "FAIL: wrapper stopped polling after a skipped footer instead of waiting for the same task's later passed footer"; echo "$DELAYED_PASS_OUT"; exit 1 +fi + NO_TASK_SUMMARY_OUT="$(env PATH="$WORK/bin:$PATH" STUB_CAPTURE="$WORK/capture" STUB_NO_TASK_SUMMARY=1 STUB_LOGGING_JSON="$records_no_retry" STUB_SUCCEEDED=2 GCLOUD_IDENT=stub@example.invalid "$SCRIPT" run --cloud --shards=2 2>&1)" && NO_TASK_SUMMARY_RC=0 || NO_TASK_SUMMARY_RC=$? if [ "$NO_TASK_SUMMARY_RC" -eq 0 ] || ! grep -q "required task-indexed Playwright summary evidence was not present" <<< "$NO_TASK_SUMMARY_OUT"; then echo "FAIL: a Cloud run without complete task-indexed Playwright summaries was accepted"; echo "$NO_TASK_SUMMARY_OUT"; exit 1 From 8f6325e7b59a3679ea21d18d95a08f1b6c129b26 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:15:08 -0700 Subject: [PATCH 428/493] test(runtime): provide managed failure log directory --- test/unit/tooling/testing/runtime-sandbox.test.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 6cff3d662..afffad205 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -300,7 +300,8 @@ describe('managed environment create failure retention', () => { await fs.writeFile(lifecycle, `${JSON.stringify(failureEvent)}\n`) } const logs = vi.spyOn(rig.runtime, 'containerLogs').mockReturnValue(fileEvidence ? secretMarkers.join(' ') : JSON.stringify(failureEvent)) - const web = { capturedOutput: vi.fn(() => ({ stdout: JSON.stringify(failureEvent), stderr: secretMarkers.join(' ') })), + const web = { info: { logsDir: directory }, + capturedOutput: vi.fn(() => ({ stdout: JSON.stringify(failureEvent), stderr: secretMarkers.join(' ') })), stop: async () => fs.rm(home, { recursive: true }) } rig.web = web as unknown as typeof rig.web rig.supervisor = { containerId: 'd'.repeat(64), runtimeRoot } as typeof rig.supervisor @@ -374,7 +375,7 @@ describe('managed environment create failure retention', () => { const logs = vi.spyOn(rig.runtime, 'containerLogs').mockImplementation(id => id === hostId ? childLogs : '') const receipts = vi.spyOn(rig.runtime.broker, 'receiptIds').mockReturnValue(new Set(kind === 'foreign-preparation' ? [] : [hostId])) restoreReceipts = () => receipts.mockRestore() - rig.web = { capturedOutput: () => ({ stdout: '', stderr: markers.join(' ') }), + rig.web = { info: { logsDir: directory }, capturedOutput: () => ({ stdout: '', stderr: markers.join(' ') }), stop: async () => fs.rm(home, { recursive: true }) } as unknown as typeof rig.web rig.supervisor = { containerId: 'd'.repeat(64), runtimeRoot } as typeof rig.supervisor const owner = rig From 4e49f258722028e08ffdbc009291d11eec1aa676 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:15:12 -0700 Subject: [PATCH 429/493] fix(managed): preserve harmless conditional URL outputs --- .../managed_mcp_capability/source_stage.rs | 11 +- .../opencode_resources.rs | 119 ++++++++++++++++++ 2 files changed, 129 insertions(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 20d325086..4fee35629 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1542,6 +1542,8 @@ process.stdout.write(require('./helper.js')) home.join("existing-deleted-property-url-output.txt"); let append_conditional_alias_url_output = home.join("existing-conditional-alias-url-output.txt"); + let append_harmless_conditional_alias_url_output = + home.join("existing-harmless-conditional-alias-url-output.txt"); let append_parenthesized_property_url_output = home.join("existing-parenthesized-property-url-output.txt"); let append_let_variable_url_output = home.join("existing-let-url-variable-output.txt"); @@ -1563,7 +1565,7 @@ process.stdout.write(require('./helper.js')) .unwrap(); std::io::Write::write_all( &mut command_file, - b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\nconst conditionalOutputUrl = new URL('./existing-conditional-alias-url-output.txt', import.meta.url);\nconst fallbackUrl = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nconst selectedUrl = true ? conditionalOutputUrl : fallbackUrl;\nselectedUrl.href = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nif (false) fs.appendFileSync(conditionalOutputUrl, ' appended');\n", + b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\nconst conditionalOutputUrl = new URL('./existing-conditional-alias-url-output.txt', import.meta.url);\nconst fallbackUrl = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nconst selectedUrl = true ? conditionalOutputUrl : fallbackUrl;\nselectedUrl.href = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nif (false) fs.appendFileSync(conditionalOutputUrl, ' appended');\nconst harmlessConditionalOutputUrl = new URL('./existing-harmless-conditional-alias-url-output.txt', import.meta.url);\nconst harmlessSelectedUrl = true ? harmlessConditionalOutputUrl : fallbackUrl;\nconsole.error(harmlessSelectedUrl.href);\nif (false) fs.appendFileSync(harmlessConditionalOutputUrl, ' appended');\n", ) .unwrap(); std::fs::write(&append_output, "existing").unwrap(); @@ -1580,6 +1582,11 @@ process.stdout.write(require('./helper.js')) "conditional alias URL existing", ) .unwrap(); + std::fs::write( + &append_harmless_conditional_alias_url_output, + "harmless conditional alias URL existing", + ) + .unwrap(); std::fs::write( &append_parenthesized_property_url_output, "parenthesized property URL existing", @@ -1614,6 +1621,7 @@ process.stdout.write(require('./helper.js')) &reassigned_url_output, &unrelated_url_output, &append_conditional_alias_url_output, + &append_harmless_conditional_alias_url_output, ] { assert!(plan[0] .configured_resources @@ -1643,6 +1651,7 @@ process.stdout.write(require('./helper.js')) &append_nested_url_output, &append_let_variable_url_output, &append_var_variable_url_output, + &append_harmless_conditional_alias_url_output, ] { let canonical = output.canonicalize().unwrap(); let mounts = units diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 7cc3b109b..a3a39c80f 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1312,12 +1312,21 @@ fn node_relative_resource_specifiers( output_destination_bindings.insert((name.clone(), binding_scope)); } } + let conditional_aliases = node_conditional_alias_analysis( + &tokens, + &scopes, + &binding_declarations, + &mut binding_resolver, + )?; let ambiguous_output_bindings = node_ambiguous_output_bindings( &tokens, &scopes, &output_destinations, &mut binding_resolver, + &conditional_aliases.safe_reference_indices, )?; + let mut ambiguous_output_bindings = ambiguous_output_bindings; + conditional_aliases.propagate_ambiguity(&mut ambiguous_output_bindings); let mut result = std::collections::BTreeMap::<String, bool>::new(); for index in 0..tokens.len() { if index >= 2 @@ -1763,11 +1772,120 @@ fn node_simple_url_binding_declaration( )) } +#[derive(Default)] +struct JavaScriptConditionalAliasAnalysis { + sources_by_alias: std::collections::BTreeMap< + JavaScriptBindingKey, + std::collections::BTreeSet<JavaScriptBindingKey>, + >, + safe_reference_indices: std::collections::BTreeSet<usize>, +} + +impl JavaScriptConditionalAliasAnalysis { + fn propagate_ambiguity( + &self, + ambiguous: &mut std::collections::BTreeSet<JavaScriptBindingKey>, + ) { + let mut pending = ambiguous.iter().cloned().collect::<Vec<_>>(); + while let Some(alias) = pending.pop() { + let Some(sources) = self.sources_by_alias.get(&alias) else { + continue; + }; + for source in sources { + if ambiguous.insert(source.clone()) { + pending.push(source.clone()); + } + } + } + } +} + +fn node_conditional_alias_analysis( + tokens: &[JavaScriptToken], + scopes: &JavaScriptScopes, + binding_declarations: &std::collections::BTreeMap<JavaScriptBindingKey, usize>, + resolver: &mut JavaScriptBindingResolver<'_>, +) -> Result<JavaScriptConditionalAliasAnalysis, &'static str> { + let mut analysis = JavaScriptConditionalAliasAnalysis::default(); + for index in 0..tokens.len().saturating_sub(7) { + let Some(JavaScriptToken::Identifier(kind)) = tokens.get(index) else { + continue; + }; + if !matches!(kind.as_str(), "const" | "let" | "var") { + continue; + } + let Some(JavaScriptToken::Identifier(alias_name)) = tokens.get(index + 1) else { + continue; + }; + if !matches!( + tokens.get(index + 2), + Some(JavaScriptToken::Punctuation(b'=')) + ) || !matches!(tokens.get(index + 3), Some(JavaScriptToken::Identifier(_))) + || !matches!( + tokens.get(index + 4), + Some(JavaScriptToken::Punctuation(b'?')) + ) + || !matches!(tokens.get(index + 5), Some(JavaScriptToken::Identifier(_))) + || !matches!( + tokens.get(index + 6), + Some(JavaScriptToken::Punctuation(b':')) + ) + || !matches!(tokens.get(index + 7), Some(JavaScriptToken::Identifier(_))) + || !matches!( + tokens.get(index + 8), + None | Some(JavaScriptToken::Punctuation(b';' | b'}')) + ) + { + continue; + } + + let alias_scope = node_variable_binding_scope(kind, scopes.token_scope_ids[index], scopes); + let alias_key = (alias_name.clone(), alias_scope); + if binding_declarations.get(&alias_key) != Some(&1) { + continue; + } + let Some(JavaScriptToken::Identifier(source_a_name)) = tokens.get(index + 5) else { + continue; + }; + let Some(JavaScriptToken::Identifier(source_b_name)) = tokens.get(index + 7) else { + continue; + }; + let Some(source_a_scope) = + resolver.resolve(source_a_name, scopes.token_scope_ids[index + 5])? + else { + continue; + }; + let Some(source_b_scope) = + resolver.resolve(source_b_name, scopes.token_scope_ids[index + 7])? + else { + continue; + }; + let source_a_key = (source_a_name.clone(), source_a_scope); + let source_b_key = (source_b_name.clone(), source_b_scope); + if binding_declarations.get(&source_a_key) != Some(&1) + || binding_declarations.get(&source_b_key) != Some(&1) + { + continue; + } + + analysis + .sources_by_alias + .entry(alias_key) + .or_default() + .extend([source_a_key, source_b_key]); + analysis + .safe_reference_indices + .extend([index + 3, index + 5, index + 7]); + } + Ok(analysis) +} + fn node_ambiguous_output_bindings( tokens: &[JavaScriptToken], scopes: &JavaScriptScopes, output_destinations: &[bool], resolver: &mut JavaScriptBindingResolver<'_>, + conditional_alias_references: &std::collections::BTreeSet<usize>, ) -> Result<std::collections::BTreeSet<(String, usize)>, &'static str> { let mut ambiguous = std::collections::BTreeSet::new(); for (index, token) in tokens.iter().enumerate() { @@ -1775,6 +1893,7 @@ fn node_ambiguous_output_bindings( continue; }; if output_destinations[index] + || conditional_alias_references.contains(&index) || scopes.parameter_token_indices.contains(&index) || is_javascript_variable_declaration_name(tokens, index) || matches!( From 4cea612bea4e08562e2112efa30a289af29c26db Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:26:16 -0700 Subject: [PATCH 430/493] fix(managed): write through conditional URL aliases --- .../managed_mcp_capability/source_stage.rs | 20 ++++++++++- .../opencode_resources.rs | 33 ++++++++++++++----- 2 files changed, 43 insertions(+), 10 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 4fee35629..16e66e48e 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1544,6 +1544,10 @@ process.stdout.write(require('./helper.js')) home.join("existing-conditional-alias-url-output.txt"); let append_harmless_conditional_alias_url_output = home.join("existing-harmless-conditional-alias-url-output.txt"); + let append_conditional_alias_write_source_a = + home.join("existing-conditional-alias-write-a.txt"); + let append_conditional_alias_write_source_b = + home.join("existing-conditional-alias-write-b.txt"); let append_parenthesized_property_url_output = home.join("existing-parenthesized-property-url-output.txt"); let append_let_variable_url_output = home.join("existing-let-url-variable-output.txt"); @@ -1565,7 +1569,7 @@ process.stdout.write(require('./helper.js')) .unwrap(); std::io::Write::write_all( &mut command_file, - b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\nconst conditionalOutputUrl = new URL('./existing-conditional-alias-url-output.txt', import.meta.url);\nconst fallbackUrl = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nconst selectedUrl = true ? conditionalOutputUrl : fallbackUrl;\nselectedUrl.href = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nif (false) fs.appendFileSync(conditionalOutputUrl, ' appended');\nconst harmlessConditionalOutputUrl = new URL('./existing-harmless-conditional-alias-url-output.txt', import.meta.url);\nconst harmlessSelectedUrl = true ? harmlessConditionalOutputUrl : fallbackUrl;\nconsole.error(harmlessSelectedUrl.href);\nif (false) fs.appendFileSync(harmlessConditionalOutputUrl, ' appended');\n", + b"\nconst parenthesizedPropertyUrl = new URL('./existing-parenthesized-property-url-output.txt', import.meta.url);\n(parenthesizedPropertyUrl.href) = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nfs.appendFileSync(parenthesizedPropertyUrl, ' appended');\nconst conditionalOutputUrl = new URL('./existing-conditional-alias-url-output.txt', import.meta.url);\nconst fallbackUrl = new URL('./existing-unrelated-url-output.txt', import.meta.url);\nconst selectedUrl = true ? conditionalOutputUrl : fallbackUrl;\nselectedUrl.href = new URL('./existing-unrelated-url-output.txt', import.meta.url).href;\nif (false) fs.appendFileSync(conditionalOutputUrl, ' appended');\nconst harmlessConditionalOutputUrl = new URL('./existing-harmless-conditional-alias-url-output.txt', import.meta.url);\nconst harmlessSelectedUrl = true ? harmlessConditionalOutputUrl : fallbackUrl\nconsole.error(harmlessSelectedUrl.href);\nif (false) fs.appendFileSync(harmlessConditionalOutputUrl, ' appended');\nconst conditionalWriteSourceA = new URL('./existing-conditional-alias-write-a.txt', import.meta.url);\nconst conditionalWriteSourceB = new URL('./existing-conditional-alias-write-b.txt', import.meta.url);\nconst conditionalWriteUrl = true ? conditionalWriteSourceA : conditionalWriteSourceB;\nfs.appendFileSync(conditionalWriteUrl, ' appended');\n", ) .unwrap(); std::fs::write(&append_output, "existing").unwrap(); @@ -1587,6 +1591,16 @@ process.stdout.write(require('./helper.js')) "harmless conditional alias URL existing", ) .unwrap(); + std::fs::write( + &append_conditional_alias_write_source_a, + "conditional alias write source A existing", + ) + .unwrap(); + std::fs::write( + &append_conditional_alias_write_source_b, + "conditional alias write source B existing", + ) + .unwrap(); std::fs::write( &append_parenthesized_property_url_output, "parenthesized property URL existing", @@ -1622,6 +1636,8 @@ process.stdout.write(require('./helper.js')) &unrelated_url_output, &append_conditional_alias_url_output, &append_harmless_conditional_alias_url_output, + &append_conditional_alias_write_source_a, + &append_conditional_alias_write_source_b, ] { assert!(plan[0] .configured_resources @@ -1652,6 +1668,8 @@ process.stdout.write(require('./helper.js')) &append_let_variable_url_output, &append_var_variable_url_output, &append_harmless_conditional_alias_url_output, + &append_conditional_alias_write_source_a, + &append_conditional_alias_write_source_b, ] { let canonical = output.canonicalize().unwrap(); let mounts = units diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index a3a39c80f..72c2c6a45 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1318,6 +1318,7 @@ fn node_relative_resource_specifiers( &binding_declarations, &mut binding_resolver, )?; + conditional_aliases.propagate_to_sources(&mut output_destination_bindings); let ambiguous_output_bindings = node_ambiguous_output_bindings( &tokens, &scopes, @@ -1326,7 +1327,7 @@ fn node_relative_resource_specifiers( &conditional_aliases.safe_reference_indices, )?; let mut ambiguous_output_bindings = ambiguous_output_bindings; - conditional_aliases.propagate_ambiguity(&mut ambiguous_output_bindings); + conditional_aliases.propagate_to_sources(&mut ambiguous_output_bindings); let mut result = std::collections::BTreeMap::<String, bool>::new(); for index in 0..tokens.len() { if index >= 2 @@ -1782,17 +1783,17 @@ struct JavaScriptConditionalAliasAnalysis { } impl JavaScriptConditionalAliasAnalysis { - fn propagate_ambiguity( + fn propagate_to_sources( &self, - ambiguous: &mut std::collections::BTreeSet<JavaScriptBindingKey>, + bindings: &mut std::collections::BTreeSet<JavaScriptBindingKey>, ) { - let mut pending = ambiguous.iter().cloned().collect::<Vec<_>>(); + let mut pending = bindings.iter().cloned().collect::<Vec<_>>(); while let Some(alias) = pending.pop() { let Some(sources) = self.sources_by_alias.get(&alias) else { continue; }; for source in sources { - if ambiguous.insert(source.clone()) { + if bindings.insert(source.clone()) { pending.push(source.clone()); } } @@ -1831,10 +1832,7 @@ fn node_conditional_alias_analysis( Some(JavaScriptToken::Punctuation(b':')) ) || !matches!(tokens.get(index + 7), Some(JavaScriptToken::Identifier(_))) - || !matches!( - tokens.get(index + 8), - None | Some(JavaScriptToken::Punctuation(b';' | b'}')) - ) + || !javascript_conditional_alias_initializer_terminates(tokens, index + 8) { continue; } @@ -1880,6 +1878,23 @@ fn node_conditional_alias_analysis( Ok(analysis) } +fn javascript_conditional_alias_initializer_terminates( + tokens: &[JavaScriptToken], + next_index: usize, +) -> bool { + // Tokenization drops whitespace, so a following statement's identifier is + // the bounded ASI signal for this simple initializer. `in` and + // `instanceof` can still continue the expression; member/call/operator + // continuations stay fail-closed below. + match tokens.get(next_index) { + None | Some(JavaScriptToken::Punctuation(b';' | b',' | b'}')) => true, + Some(JavaScriptToken::Identifier(keyword)) => { + !matches!(keyword.as_str(), "in" | "instanceof") + } + _ => false, + } +} + fn node_ambiguous_output_bindings( tokens: &[JavaScriptToken], scopes: &JavaScriptScopes, From 80a7d8c1f548f2731fb7ac2f92591fa3d915943b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:59:46 -0700 Subject: [PATCH 431/493] fix(managed): preserve nested writable resources --- .../src/provider_mountpoints.rs | 33 +++++ .../managed_mcp_capability/source_stage.rs | 128 ++++++++++++++++++ crates/freshell-session-host/src/main.rs | 29 +++- .../src/backend/provider_mountpoints.rs | 45 ++++++ 4 files changed, 234 insertions(+), 1 deletion(-) diff --git a/crates/freshell-platform/src/provider_mountpoints.rs b/crates/freshell-platform/src/provider_mountpoints.rs index 1e65a6b8d..03ab9b586 100644 --- a/crates/freshell-platform/src/provider_mountpoints.rs +++ b/crates/freshell-platform/src/provider_mountpoints.rs @@ -490,6 +490,39 @@ mod tests { ); } + #[test] + fn provider_mountpoints_create_a_file_target_inside_a_selected_directory() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + std::fs::create_dir(&home).unwrap(); + let selected = [ + Mountpoint { + relative_path: ".freshell/provider-roots/directory".into(), + directory: true, + }, + Mountpoint { + relative_path: ".freshell/provider-roots/state/source".into(), + directory: false, + }, + Mountpoint { + relative_path: "mcp".into(), + directory: true, + }, + Mountpoint { + relative_path: "mcp/state.json".into(), + directory: false, + }, + ]; + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + + prepare_for_identity(&home, &selected, uid, gid).unwrap(); + + assert!(home.join(".freshell/provider-roots/directory").is_dir()); + assert!(home.join(".freshell/provider-roots/state/source").is_file()); + assert!(home.join("mcp").is_dir()); + assert!(home.join("mcp/state.json").is_file()); + } + #[test] fn provider_mountpoints_remove_only_retired_empty_resource_placeholders() { let owned = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 16e66e48e..6e0c5a05f 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1526,6 +1526,111 @@ process.stdout.write(require('./helper.js')) assert_eq!(output.stdout, b"cwd options loaded\n"); } + #[test] + fn nested_direct_script_keeps_writable_sibling_mount_under_selected_directory() { + use freshell_platform::opencode_recipe::{DiscoveryRole, DiscoveryRoot}; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let script_directory = home.join("mcp"); + let command = script_directory.join("server.mjs"); + let state_file = script_directory.join("state.json"); + std::fs::create_dir_all(&script_directory).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.appendFileSync(path.join(__dirname, 'state.json'), ' + runtime state');\nprocess.stdout.write('nested state updated\\n');\n", + ) + .unwrap(); + std::fs::write(&state_file, "initial state").unwrap(); + + let (plan, _) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + let canonical_state = state_file.canonicalize().unwrap(); + assert!(plan[0] + .configured_writable_resources + .contains(&canonical_state)); + let previous_units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let previous_state_mount = previous_units + .iter() + .find(|(source, mount)| { + source == &canonical_state + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + && mount.effective_destination == "/home/freshell/provider/mcp/state.json" + }) + .map(|(_, mount)| mount.clone()) + .expect("the initial generation needs a writable state mount"); + let selected_directory = DiscoveryRoot { + logical_path: script_directory.clone(), + role: DiscoveryRole::Home, + }; + let mut units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + Some(std::slice::from_ref(&selected_directory)), + ) + .unwrap(); + merge_recovery_resource_mounts( + &mut units, + &previous_units + .iter() + .map(|(_, mount)| mount.clone()) + .collect::<Vec<_>>(), + ) + .unwrap(); + let state_mount = units + .iter() + .find(|(source, mount)| { + source == &canonical_state + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .expect("writable state beneath a selected directory needs an exact file mount"); + assert!(state_mount.1.writable); + assert_eq!( + state_mount.1.provider_relative_path, previous_state_mount.provider_relative_path, + "recovery must reuse the exact writable file's provider subpath" + ); + + let directory_mount = units + .iter() + .find(|(source, mount)| { + source == &script_directory.canonicalize().unwrap() + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::Directory + && mount + .effective_destination + .starts_with("/home/freshell/provider") + }) + .map(|(_, mount)| mount) + .expect("the selected script directory needs a directory mount"); + let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider_script_directory = provider.join(".provider-home").join( + Path::new(&directory_mount.effective_destination) + .strip_prefix("/home/freshell/provider") + .unwrap(), + ); + let output = execute_node_with_home_hidden( + owned.path(), + &home, + &provider_script_directory.join("server.mjs"), + ); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"nested state updated\n"); + assert_eq!( + std::fs::read(provider_script_directory.join("state.json")).unwrap(), + b"initial state + runtime state" + ); + } + #[test] fn direct_node_script_does_not_stage_absent_sibling_write_targets() { let owned = tempfile::tempdir().unwrap(); @@ -4429,6 +4534,29 @@ pub(super) fn resource_units( } } } + // A containing directory supplies ordinary child inputs, but its writable + // projection cannot preserve an exact file across provider refresh. Keep + // every existing static write target as its own file root even when a + // selected directory already covers that path. + for writable in &writable_resources { + if writable.starts_with(workspace) { + continue; + } + let canonical = std::fs::canonicalize(writable) + .map_err(|_| "selected OpenCode writable resource is unavailable")?; + if canonical != *writable || !canonical.is_file() { + return Err("selected OpenCode writable resource changed during preparation".into()); + } + let mut roles = vec![crate::managed_provider_references::sources::DiscoveryRole::Workspace]; + if writable.starts_with(global) { + roles.push(crate::managed_provider_references::sources::DiscoveryRole::Global); + } else if writable.starts_with(home) { + roles.push(crate::managed_provider_references::sources::DiscoveryRole::Home); + } + for role in roles { + roots.insert((writable.clone(), role), canonical.clone()); + } + } let mut units = Vec::new(); let mut identities = BTreeMap::<std::path::PathBuf, String>::new(); for ((logical, role), canonical) in roots { diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 2a4483445..7676f95e1 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -4781,18 +4781,23 @@ mod tests { let user_root = source_root.join("provider"); let workspace = owned.path().join("workspace"); let home = owned.path().join("home"); + let directory_source = user_root.join(".freshell/resource-units/directory"); + let directory_relative = PathBuf::from(".freshell/provider-roots/directory"); let writable_source = user_root.join(".freshell/resource-units/writable/source"); let readonly_source = user_root.join(".freshell/resource-units/readonly/source"); let writable_relative = PathBuf::from(".freshell/provider-roots/writable/source"); let readonly_relative = PathBuf::from(".freshell/provider-roots/readonly/source"); + std::fs::create_dir_all(&directory_source).unwrap(); std::fs::create_dir_all(writable_source.parent().unwrap()).unwrap(); std::fs::create_dir_all(readonly_source.parent().unwrap()).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(home.join(&directory_relative)).unwrap(); std::fs::create_dir_all(home.join(writable_relative.parent().unwrap())).unwrap(); std::fs::create_dir_all(home.join(readonly_relative.parent().unwrap())).unwrap(); + std::fs::write(directory_source.join("state.txt"), "directory seed").unwrap(); std::fs::write(&writable_source, "writable seed").unwrap(); std::fs::write(&readonly_source, "readonly seed").unwrap(); - // Exact-file mounts exist as physical placeholders before initial + // Resource roots exist as physical placeholders before initial // publication, just as their Docker volume subpath mounts do. std::fs::write(home.join(&writable_relative), "").unwrap(); std::fs::write(home.join(&readonly_relative), "").unwrap(); @@ -4823,6 +4828,14 @@ mod tests { config_binds: vec![], indexed_files: None, resource_mounts: vec![ + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/directory".into(), + provider_relative_path: directory_relative.to_string_lossy().into_owned(), + logical_destination: "/home/freshell/provider/mcp".into(), + effective_destination: "/home/freshell/provider/mcp".into(), + kind: ProviderResourceKind::Directory, + writable: false, + }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/writable/source".into(), provider_relative_path: writable_relative.to_string_lossy().into_owned(), @@ -4848,6 +4861,10 @@ mod tests { &context, "opencode", &workspace, &user_root, &home, uid, gid, ) .unwrap(); + assert_eq!( + std::fs::read(home.join(&directory_relative).join("state.txt")).unwrap(), + b"directory seed" + ); use std::io::Write; std::fs::OpenOptions::new() .append(true) @@ -4855,6 +4872,11 @@ mod tests { .unwrap() .write_all(b" + runtime state") .unwrap(); + std::fs::write( + directory_source.join("state.txt"), + "refreshed directory seed", + ) + .unwrap(); std::fs::write(&writable_source, "refreshed writable seed").unwrap(); std::fs::write(&readonly_source, "refreshed readonly seed").unwrap(); @@ -4863,6 +4885,11 @@ mod tests { ) .unwrap(); + assert_eq!( + std::fs::read(home.join(&directory_relative).join("state.txt")).unwrap(), + b"refreshed directory seed", + "the selected directory's ordinary children must still refresh" + ); assert_eq!( std::fs::read(home.join(&writable_relative)).unwrap(), b"writable seed + runtime state", diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index b174e7c16..1b2ebdd36 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -743,4 +743,49 @@ mod tests { assert!(selected.iter().any(|point| point.relative_path == ".freshell/provider-roots/root-0/opencode.json" && !point.directory), "exact private leaves must exist in the owned soul subpath before Docker mounts the external root"); } + + #[test] + fn provider_mountpoints_include_nested_writable_file_mounts() { + let mounts = docker::TerminalMounts { + workspace: "/synthetic/workspace".into(), + git_common_dir: None, + provider_bootstrap_files: vec![], + provider_secret_files: vec![], + indexed_secret_sources: BTreeMap::new(), + provider_user_root: None, + mcp_capability_file: None, + input_manifest: None, + indexed_files: None, + config_binds: vec![], + resource_mounts: vec![ + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-directory".into(), + provider_relative_path: ".freshell/provider-roots/root-directory".into(), + logical_destination: "/home/freshell/provider/mcp".into(), + effective_destination: "/home/freshell/provider/mcp".into(), + kind: ProviderResourceKind::Directory, + writable: false, + }, + ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/root-state/source".into(), + provider_relative_path: ".freshell/provider-roots/root-state/source".into(), + logical_destination: "/home/freshell/provider/mcp/state.json".into(), + effective_destination: "/home/freshell/provider/mcp/state.json".into(), + kind: ProviderResourceKind::File, + writable: true, + }, + ], + }; + + let selected = selections(&mounts).unwrap(); + let has = |path: &str, directory| { + selected + .iter() + .any(|point| point.relative_path == path && point.directory == directory) + }; + assert!(has(".freshell/provider-roots/root-directory", true)); + assert!(has(".freshell/provider-roots/root-state/source", false)); + assert!(has("mcp", true)); + assert!(has("mcp/state.json", false)); + } } From 29fd605fcda669a05b9cfca73c5f8f3d224d8abd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 22:31:07 -0700 Subject: [PATCH 432/493] Fix nested writable file mount staging --- .../src/provider_environment.rs | 114 +++++++-- .../src/managed_mcp_capability.rs | 121 +++++++++- .../managed_mcp_capability/source_stage.rs | 65 ++++- crates/freshell-session-host/src/main.rs | 224 ++++++++++++++++-- 4 files changed, 480 insertions(+), 44 deletions(-) diff --git a/crates/freshell-platform/src/provider_environment.rs b/crates/freshell-platform/src/provider_environment.rs index c6b046217..2c861c224 100644 --- a/crates/freshell-platform/src/provider_environment.rs +++ b/crates/freshell-platform/src/provider_environment.rs @@ -124,6 +124,7 @@ enum Node { struct Plan { nodes: BTreeMap<PathBuf, Node>, mappings: BTreeMap<PathBuf, PathBuf>, + independent_file_roots: BTreeSet<PathBuf>, pending: VecDeque<(PathBuf, PathBuf, usize)>, exclusions: Vec<PathBuf>, excluded_subtree: Option<PathBuf>, @@ -554,7 +555,7 @@ impl Plan { } return Ok(()); } - if !metadata.file_type().is_symlink() { + if !metadata.file_type().is_symlink() && !self.independent_file_roots.contains(&path) { if let Some(mapped) = self.mapped(&canonical)? { if mapped != path { return self.claim(path, Node::Link(mapped)); @@ -873,19 +874,6 @@ fn copy_environment_projection( )); } } - let mut plan = Plan { - nodes: BTreeMap::new(), - mappings: BTreeMap::new(), - pending: VecDeque::new(), - exclusions: excluded_canonical_sources.to_vec(), - excluded_subtree, - included_subtree_resources: BTreeSet::new(), - counted_files: BTreeSet::new(), - bytes: 0, - metadata_bytes: 0, - limits, - account_home, - }; let mut selections = selections.iter().collect::<Vec<_>>(); selections.sort_by_key(|selection| { ( @@ -903,6 +891,29 @@ fn copy_environment_projection( )); } } + let independent_file_roots = selections + .iter() + .filter(|selection| mounted_roots.contains(&selection.destination_relative)) + .filter_map(|selection| { + let metadata = fs::symlink_metadata(&selection.source).ok()?; + (metadata.is_file() && !metadata.file_type().is_symlink()) + .then(|| selection.destination_relative.clone()) + }) + .collect(); + let mut plan = Plan { + nodes: BTreeMap::new(), + mappings: BTreeMap::new(), + independent_file_roots, + pending: VecDeque::new(), + exclusions: excluded_canonical_sources.to_vec(), + excluded_subtree, + included_subtree_resources: BTreeSet::new(), + counted_files: BTreeSet::new(), + bytes: 0, + metadata_bytes: 0, + limits, + account_home, + }; for selection in &selections { let path = &selection.destination_relative; if !valid_relative(path) { @@ -932,6 +943,12 @@ fn copy_environment_projection( plan.select_linked_ancestors(selection, &selections, &canonical)?; if metadata.file_type().is_symlink() { plan.enqueue(selection.source.clone(), path.clone(), 0)?; + } else if plan.independent_file_roots.contains(path) { + // A separately mounted exact file needs its own inode even when + // the same source file was already copied through a selected + // directory. Canonical-path accounting still charges its bytes + // once. + plan.enqueue(selection.source.clone(), path.clone(), 0)?; } else if let Some(mapped) = plan.mapped(&canonical)? { if mapped != *path { plan.claim(path.clone(), Node::Link(mapped))?; @@ -1018,6 +1035,7 @@ fn copy_environment_projection( NEXT_PROJECTION.fetch_add(1, Ordering::Relaxed) )))?; let mut actual_bytes = 0u64; + let mut counted_sources = BTreeSet::new(); for (relative, node) in &plan.nodes { let destination = temporary.0.join(relative); match node { @@ -1027,6 +1045,7 @@ fn copy_environment_projection( mode(&destination, 0o700, relative)?; } Node::File(source, source_mode, expected) => { + let count_source_bytes = counted_sources.insert(source.clone()); let mut input = File::open(source).map_err(|cause| io_error(relative, "open", cause))?; let mut options = OpenOptions::new(); @@ -1051,16 +1070,20 @@ fn copy_environment_projection( file_bytes = file_bytes .checked_add(read as u64) .ok_or_else(|| error(relative, "read", "file byte accounting overflow"))?; - actual_bytes = actual_bytes - .checked_add(read as u64) - .ok_or_else(|| error(relative, "read", "total byte accounting overflow"))?; + if count_source_bytes { + actual_bytes = actual_bytes.checked_add(read as u64).ok_or_else(|| { + error(relative, "read", "total byte accounting overflow") + })?; + } bound(relative, "file_bytes", file_bytes, limits.max_file_bytes)?; - bound( - relative, - "total_bytes", - actual_bytes, - limits.max_total_bytes, - )?; + if count_source_bytes { + bound( + relative, + "total_bytes", + actual_bytes, + limits.max_total_bytes, + )?; + } output .write_all(&buffer[..read]) .map_err(|cause| io_error(relative, "write", cause))?; @@ -2744,6 +2767,51 @@ mod tests { assert_eq!(fs::read(destination.join("skill/data")).unwrap(), b"nonce"); } + #[cfg(unix)] + #[test] + fn mounted_nested_file_root_keeps_an_independent_inode_and_single_byte_charge() { + let root = tempfile::tempdir().unwrap(); + let source = root.path().join("source"); + fs::create_dir_all(source.join("mcp")).unwrap(); + fs::write(source.join("mcp/state.json"), "nonce").unwrap(); + let destination = root.path().join("out"); + fs::create_dir_all(destination.join("exact")).unwrap(); + fs::write(destination.join("exact/source"), []).unwrap(); + + let report = copy_environment_for_mounts( + &[ + selection(&source.join("mcp"), "directory"), + selection(&source.join("mcp/state.json"), "exact/source"), + ], + &destination, + &[], + CopyLimits { + max_total_bytes: 5, + ..CopyLimits::default() + }, + &[PathBuf::from("exact/source")], + ) + .unwrap(); + + assert_eq!( + report.bytes, 5, + "duplicate physical copies share one byte charge" + ); + for path in [ + destination.join("directory/state.json"), + destination.join("exact/source"), + ] { + let metadata = fs::symlink_metadata(&path).unwrap(); + assert!(metadata.is_file() && !metadata.file_type().is_symlink()); + } + fs::write(destination.join("exact/source"), "runtime state").unwrap(); + assert_eq!( + fs::read(destination.join("directory/state.json")).unwrap(), + b"nonce", + "the exact mounted file must not alias its selected directory copy" + ); + } + #[cfg(unix)] #[test] fn overlapping_root_and_external_canonical_unit_preserve_the_logical_link() { diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 5ff0b7628..96b1937d5 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1672,6 +1672,7 @@ fn stage_provider_root( ); let mut _filtered_resource_sources = None; let mut retained_resource_sources = None; + let mut mounted_resource_file_roots = Vec::new(); if let Some(plan) = opencode_input.source_plan { // Indexed payloads remain exclusively in the private input volume, // even when an ordinary package scope also contains their files. @@ -1752,6 +1753,22 @@ fn stage_provider_root( .into_iter() .map(|(_, mount)| mount) .collect::<Vec<_>>(); + mounted_resource_file_roots = descriptors + .iter() + .filter(|mount| mount.kind == freshell_runtime_protocol::ProviderResourceKind::File) + .map(|mount| PathBuf::from(&mount.stage_relative_path)) + .collect::<std::collections::BTreeSet<_>>() + .into_iter() + .collect(); + for root in &mounted_resource_file_roots { + let destination = temporary.join(root); + private_directory( + destination + .parent() + .ok_or("mounted resource file stage has no parent")?, + )?; + write_private_bytes(&destination, &[])?; + } let metadata = freshell_runtime_protocol::bounded_json_bytes( &descriptors, freshell_runtime_protocol::MAX_CONTROL_FRAME_BYTES, @@ -1978,7 +1995,7 @@ fn stage_provider_root( selections = selections.len() ); let copy_result = if provider == "opencode" { - freshell_platform::provider_environment::copy_environment_excluding_subtree( + freshell_platform::provider_environment::copy_environment_excluding_subtree_for_mounts( &selections, &temporary, &excluded, @@ -1987,6 +2004,7 @@ fn stage_provider_root( .or(opencode_input.cwd) .unwrap_or(home), CopyLimits::default(), + &mounted_resource_file_roots, ) } else { copy_environment(&selections, &temporary, &excluded, CopyLimits::default()) @@ -10058,6 +10076,107 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ ); } + #[cfg(unix)] + #[test] + fn managed_opencode_controller_stages_nested_writable_file_as_regular_mount_source() { + use crate::managed_provider_references::sources::*; + use freshell_platform::opencode_recipe::DiscoveryRoot; + use freshell_runtime_protocol::*; + use std::os::unix::fs::MetadataExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let config_root = home.join(".config/opencode"); + let project_config = workspace.join(".opencode/opencode.json"); + let mcp_directory = home.join("mcp"); + let script = mcp_directory.join("server.mjs"); + let state = mcp_directory.join("state.json"); + std::fs::create_dir_all(&config_root).unwrap(); + std::fs::create_dir_all(project_config.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&mcp_directory).unwrap(); + std::fs::write( + &project_config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [script] } } + }) + .to_string(), + ) + .unwrap(); + std::fs::write( + &script, + "import fs from 'node:fs';\nimport path from 'node:path';\nimport { fileURLToPath } from 'node:url';\nconst __dirname = path.dirname(fileURLToPath(import.meta.url));\nfs.appendFileSync(path.join(__dirname, 'state.json'), ' + runtime state');\n", + ) + .unwrap(); + std::fs::write(&state, "initial state").unwrap(); + + let sources = discover_opencode(OpencodeDiscovery { + home: &home, + cwd: &workspace, + worktree: &workspace, + config_root: &config_root, + env: &|_| None, + }) + .unwrap(); + let plan = plan_opencode_sources(sources, &workspace, &home, &|_, _, _| None).unwrap(); + let canonical_state = state.canonicalize().unwrap(); + assert!(plan.iter().any(|source| { + source + .configured_writable_resources + .contains(&canonical_state) + })); + + let selected_roots = [DiscoveryRoot { + logical_path: mcp_directory, + role: DiscoveryRole::Home, + }]; + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let capability = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&workspace), + workspace: Some(&workspace), + source_plan: Some(&plan), + secret_references: Some(&[]), + resource_roots: Some(&selected_roots), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let writable_files = inputs + .resource_mounts + .iter() + .filter(|mount| { + mount.kind == ProviderResourceKind::File + && mount.writable + && mount.effective_destination.ends_with("/mcp/state.json") + }) + .collect::<Vec<_>>(); + assert!( + !writable_files.is_empty(), + "controller staging must include the exact writable file mount" + ); + for mount in writable_files { + let staged_file = stage.join(&mount.stage_relative_path); + let metadata = std::fs::symlink_metadata(&staged_file).unwrap(); + assert!(metadata.is_file() && !metadata.file_type().is_symlink()); + assert_eq!(metadata.nlink(), 1, "mount sources require an owned inode"); + assert_eq!(std::fs::read(&staged_file).unwrap(), b"initial state"); + } + } + #[tokio::test] async fn managed_provider_selected_directory_without_config_retains_discovery_resources() { use freshell_runtime_protocol::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 6e0c5a05f..7c84a0b2b 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -11,7 +11,9 @@ use std::{collections::BTreeMap, path::Path}; mod configured_resource_tests { use super::*; use crate::managed_provider_references::sources::*; - use freshell_platform::provider_environment::{copy_environment, CopySelection}; + use freshell_platform::provider_environment::{ + copy_environment, copy_environment_for_mounts, CopySelection, + }; fn execute_selected_resource_with_path( path: &Path, @@ -74,7 +76,28 @@ mod configured_resource_tests { destination_relative: mount.stage_relative_path.clone().into(), }) .collect::<Vec<_>>(); - copy_environment(&stage_selections, &stage, &[], Default::default()).unwrap(); + let mounted_file_roots = units + .iter() + .filter(|(_, mount)| { + mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + }) + .map(|(_, mount)| Path::new(&mount.stage_relative_path).to_owned()) + .collect::<std::collections::BTreeSet<_>>() + .into_iter() + .collect::<Vec<_>>(); + for root in &mounted_file_roots { + let placeholder = stage.join(root); + std::fs::create_dir_all(placeholder.parent().unwrap()).unwrap(); + std::fs::write(placeholder, []).unwrap(); + } + copy_environment_for_mounts( + &stage_selections, + &stage, + &[], + Default::default(), + &mounted_file_roots, + ) + .unwrap(); let host = owned_root.join("host"); let host_selections = units @@ -89,6 +112,15 @@ mod configured_resource_tests { let provider = owned_root.join("provider"); let provider_selections = units .iter() + .filter(|(_, mount)| { + !units.iter().any(|(_, directory)| { + directory.kind == freshell_runtime_protocol::ProviderResourceKind::Directory + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + && Path::new(&mount.effective_destination) + .strip_prefix(&directory.effective_destination) + .is_ok_and(|suffix| !suffix.as_os_str().is_empty()) + }) + }) .map(|(_, mount)| CopySelection { source: host.join(&mount.provider_relative_path), destination_relative: Path::new(&mount.effective_destination) @@ -1613,6 +1645,35 @@ process.stdout.write(require('./helper.js')) .map(|(_, mount)| mount) .expect("the selected script directory needs a directory mount"); let provider = copy_selected_units_through_provider(owned.path(), &units); + let staged_state = owned + .path() + .join("stage") + .join(&state_mount.1.stage_relative_path); + let staged_state_metadata = std::fs::symlink_metadata(&staged_state).unwrap(); + assert!( + staged_state_metadata.is_file() && !staged_state_metadata.file_type().is_symlink(), + "a file mount must have a physical regular inode in the controller stage" + ); + let host_state = owned + .path() + .join("host") + .join(&state_mount.1.provider_relative_path); + let host_directory_state = owned + .path() + .join("host") + .join(&directory_mount.provider_relative_path) + .join("state.json"); + let host_state_metadata = std::fs::symlink_metadata(&host_state).unwrap(); + assert!( + host_state_metadata.is_file() && !host_state_metadata.file_type().is_symlink(), + "the provider-volume file subpath must be a physical regular inode" + ); + std::fs::write(&host_state, "mount-only runtime state").unwrap(); + assert_eq!( + std::fs::read(&host_directory_state).unwrap(), + b"initial state", + "writes through the exact mount must not alias the enclosing directory copy" + ); let provider_script_directory = provider.join(".provider-home").join( Path::new(&directory_mount.effective_destination) .strip_prefix("/home/freshell/provider") diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 7676f95e1..1b6b16d2f 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1944,6 +1944,11 @@ fn copy_provider_config_references_with_ownership( }; use freshell_runtime_protocol::ProviderConfigRoot; context.validate(provider).map_err(|error| error.message)?; + let previous_projection = if provider == "opencode" && provider_home != final_home { + Some(read_projection_manifest(final_home)?) + } else { + None + }; let mut selections = provider_config_references(context) .into_iter() .map(|reference| { @@ -1962,16 +1967,25 @@ fn copy_provider_config_references_with_ownership( }) .collect::<Vec<_>>(); let config_selection_count = selections.len(); - selections.extend( - context - .prepared_inputs - .iter() - .flat_map(|inputs| &inputs.resource_mounts) - .map(|unit| CopySelection { - source: user_provider_root.join(&unit.stage_relative_path), - destination_relative: unit.provider_relative_path.clone().into(), - }), - ); + let mut resource_selections = Vec::new(); + for unit in context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + { + let staged_source = user_provider_root.join(&unit.stage_relative_path); + let source = if let Some(previous) = previous_projection.as_ref() { + nested_writable_file_recovery_source(context, unit, previous, final_home)? + .unwrap_or(staged_source) + } else { + staged_source + }; + resource_selections.push(CopySelection { + source, + destination_relative: unit.provider_relative_path.clone().into(), + }); + } + selections.extend(resource_selections); let protected = private_bound_destinations(context); let mut excluded = provider_credential_sources(user_provider_root); // A prepared config leaf is the selected view of the raw file still held @@ -2217,6 +2231,108 @@ fn writable_resource_mount_roots( .collect() } +fn nested_writable_file_recovery_source( + context: &freshell_runtime_protocol::ProviderLaunchContext, + mount: &freshell_runtime_protocol::ProviderResourceMount, + previous: &std::collections::BTreeSet<String>, + previous_home: &Path, +) -> Result<Option<PathBuf>, String> { + use freshell_runtime_protocol::ProviderResourceKind; + if mount.kind != ProviderResourceKind::File || !mount.writable { + return Ok(None); + } + let mount_relative = Path::new(&mount.provider_relative_path); + if mount_relative.as_os_str().is_empty() || previous.contains(&mount.provider_relative_path) { + return Ok(None); + } + let Some(mount_alias) = Path::new(&mount.effective_destination) + .strip_prefix("/home/freshell/provider") + .ok() + else { + return Ok(None); + }; + let canonical_home = std::fs::canonicalize(previous_home) + .map_err(|_| "prior provider state is unavailable for writable-file recovery")?; + let mut candidates = Vec::<(usize, PathBuf)>::new(); + for directory in context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + .filter(|candidate| candidate.kind == ProviderResourceKind::Directory) + { + let Some(directory_alias) = Path::new(&directory.effective_destination) + .strip_prefix("/home/freshell/provider") + .ok() + else { + continue; + }; + if directory_alias.as_os_str().is_empty() { + continue; + } + let Ok(suffix) = mount_alias.strip_prefix(directory_alias) else { + continue; + }; + if suffix.as_os_str().is_empty() { + continue; + } + let prior_relative = Path::new(&directory.provider_relative_path).join(suffix); + let prior_text = prior_relative.to_string_lossy().into_owned(); + if !previous.contains(&prior_text) { + continue; + } + let prior_path = previous_home.join(&prior_relative); + let canonical = match std::fs::canonicalize(&prior_path) { + Ok(path) => path, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return Err( + "prior nested writable resource was deleted; retain inputs and repair".into(), + ) + } + Err(_) => { + return Err( + "prior nested writable resource is unavailable; retain inputs and repair" + .into(), + ) + } + }; + let canonical_relative = canonical.strip_prefix(&canonical_home).map_err(|_| { + "prior nested writable resource escaped provider state; retain inputs and repair" + })?; + if !previous.contains(&canonical_relative.to_string_lossy().into_owned()) { + return Err( + "prior nested writable resource target is not owned; retain inputs and repair" + .into(), + ); + } + let metadata = std::fs::metadata(&canonical).map_err(|_| { + "prior nested writable resource is unavailable; retain inputs and repair" + })?; + if !metadata.is_file() { + return Err( + "prior nested writable resource is not a regular file; retain inputs and repair" + .into(), + ); + } + candidates.push((directory_alias.components().count(), canonical)); + } + candidates.sort_by_key(|(depth, path)| (std::cmp::Reverse(*depth), path.clone())); + let Some((depth, source)) = candidates.first().cloned() else { + return Ok(None); + }; + if candidates + .iter() + .skip(1) + .take_while(|(candidate_depth, _)| *candidate_depth == depth) + .any(|(_, candidate)| candidate != &source) + { + return Err( + "prior nested writable resource has ambiguous directory owners; retain inputs and repair" + .into(), + ); + } + Ok(Some(source)) +} + // A resource's native HOME alias is a separate volume-subpath mount. Publish // through its declared physical HOME path, where the transaction and journal // share a filesystem. Every selected source still goes through the copier. @@ -4783,22 +4899,34 @@ mod tests { let home = owned.path().join("home"); let directory_source = user_root.join(".freshell/resource-units/directory"); let directory_relative = PathBuf::from(".freshell/provider-roots/directory"); + let preserved_source = user_root.join(".freshell/resource-units/preserved/source"); let writable_source = user_root.join(".freshell/resource-units/writable/source"); let readonly_source = user_root.join(".freshell/resource-units/readonly/source"); + let deleted_source = user_root.join(".freshell/resource-units/deleted/source"); + let preserved_relative = PathBuf::from(".freshell/provider-roots/preserved/source"); let writable_relative = PathBuf::from(".freshell/provider-roots/writable/source"); let readonly_relative = PathBuf::from(".freshell/provider-roots/readonly/source"); std::fs::create_dir_all(&directory_source).unwrap(); + std::fs::create_dir_all(preserved_source.parent().unwrap()).unwrap(); std::fs::create_dir_all(writable_source.parent().unwrap()).unwrap(); std::fs::create_dir_all(readonly_source.parent().unwrap()).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(home.join(&directory_relative)).unwrap(); + std::fs::create_dir_all(home.join(preserved_relative.parent().unwrap())).unwrap(); std::fs::create_dir_all(home.join(writable_relative.parent().unwrap())).unwrap(); std::fs::create_dir_all(home.join(readonly_relative.parent().unwrap())).unwrap(); std::fs::write(directory_source.join("state.txt"), "directory seed").unwrap(); + std::fs::write( + directory_source.join("removed.txt"), + "deleted directory seed", + ) + .unwrap(); + std::fs::write(&preserved_source, "preserved seed").unwrap(); std::fs::write(&writable_source, "writable seed").unwrap(); std::fs::write(&readonly_source, "readonly seed").unwrap(); // Resource roots exist as physical placeholders before initial // publication, just as their Docker volume subpath mounts do. + std::fs::write(home.join(&preserved_relative), "").unwrap(); std::fs::write(home.join(&writable_relative), "").unwrap(); std::fs::write(home.join(&readonly_relative), "").unwrap(); let capability = McpCapabilityReference { @@ -4807,7 +4935,7 @@ mod tests { provider_relative_path: ".freshell/mcp-capability.json".into(), host_gateway_address: None, }; - let context = ProviderLaunchContext { + let mut context = ProviderLaunchContext { config: vec![], preparation: ProviderPreparation::Opencode { project_config: vec![], @@ -4837,10 +4965,10 @@ mod tests { writable: false, }, ProviderResourceMount { - stage_relative_path: ".freshell/resource-units/writable/source".into(), - provider_relative_path: writable_relative.to_string_lossy().into_owned(), - logical_destination: "/home/freshell/provider/mcp/state.txt".into(), - effective_destination: "/home/freshell/provider/mcp/state.txt".into(), + stage_relative_path: ".freshell/resource-units/preserved/source".into(), + provider_relative_path: preserved_relative.to_string_lossy().into_owned(), + logical_destination: "/home/freshell/provider/mcp/preserved.txt".into(), + effective_destination: "/home/freshell/provider/mcp/preserved.txt".into(), kind: ProviderResourceKind::File, writable: true, }, @@ -4868,7 +4996,13 @@ mod tests { use std::io::Write; std::fs::OpenOptions::new() .append(true) - .open(home.join(&writable_relative)) + .open(home.join(&preserved_relative)) + .unwrap() + .write_all(b" + exact runtime state") + .unwrap(); + std::fs::OpenOptions::new() + .append(true) + .open(home.join(&directory_relative).join("state.txt")) .unwrap() .write_all(b" + runtime state") .unwrap(); @@ -4877,8 +5011,22 @@ mod tests { "refreshed directory seed", ) .unwrap(); + std::fs::write(&preserved_source, "refreshed preserved seed").unwrap(); std::fs::write(&writable_source, "refreshed writable seed").unwrap(); std::fs::write(&readonly_source, "refreshed readonly seed").unwrap(); + context + .prepared_inputs + .as_mut() + .unwrap() + .resource_mounts + .push(ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/writable/source".into(), + provider_relative_path: writable_relative.to_string_lossy().into_owned(), + logical_destination: "/home/freshell/provider/mcp/state.txt".into(), + effective_destination: "/home/freshell/provider/mcp/state.txt".into(), + kind: ProviderResourceKind::File, + writable: true, + }); refresh_provider_config_references( &context, "opencode", &workspace, &user_root, &home, uid, gid, @@ -4892,14 +5040,54 @@ mod tests { ); assert_eq!( std::fs::read(home.join(&writable_relative)).unwrap(), - b"writable seed + runtime state", - "replacement must keep the existing writable mount's runtime data" + b"directory seed + runtime state", + "adding an exact writable mount must migrate the prior directory-backed runtime data" + ); + assert_eq!( + std::fs::read(home.join(&preserved_relative)).unwrap(), + b"preserved seed + exact runtime state", + "an existing exact writable mount must keep its runtime data across refresh" ); assert_eq!( std::fs::read(home.join(&readonly_relative)).unwrap(), b"refreshed readonly seed", "read-only inputs still refresh from their selected source" ); + + // If a new exact writable mount replaces a directory child that the + // provider deleted, recovery must not silently restore stale staged + // contents from the original host source. + std::fs::create_dir_all(deleted_source.parent().unwrap()).unwrap(); + std::fs::write(&deleted_source, "stale host seed").unwrap(); + let deleted_relative = PathBuf::from(".freshell/provider-roots/deleted/source"); + std::fs::create_dir_all(home.join(deleted_relative.parent().unwrap())).unwrap(); + std::fs::write(home.join(&deleted_relative), "").unwrap(); + std::fs::remove_file(home.join(&directory_relative).join("removed.txt")).unwrap(); + context + .prepared_inputs + .as_mut() + .unwrap() + .resource_mounts + .push(ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/deleted/source".into(), + provider_relative_path: deleted_relative.to_string_lossy().into_owned(), + logical_destination: "/home/freshell/provider/mcp/removed.txt".into(), + effective_destination: "/home/freshell/provider/mcp/removed.txt".into(), + kind: ProviderResourceKind::File, + writable: true, + }); + let recovery = refresh_provider_config_references( + &context, "opencode", &workspace, &user_root, &home, uid, gid, + ); + assert!( + matches!(recovery, Err(ref error) if error.contains("was deleted")), + "deleted prior writable state must block recovery instead of restoring stale source" + ); + assert_eq!( + std::fs::read(home.join(&deleted_relative)).unwrap(), + b"", + "blocked recovery must not restore stale host content into the mount placeholder" + ); } #[test] From b4ba96788584f019d61d51d01b2f06a680db979e Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Tue, 13 Oct 2026 22:48:09 -0700 Subject: [PATCH 433/493] fix(managed): recover aliased external writable mounts --- .../managed_mcp_capability/source_stage.rs | 62 ++- crates/freshell-session-host/src/main.rs | 443 ++++++++++++++++-- 2 files changed, 445 insertions(+), 60 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 7c84a0b2b..85244812d 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -110,16 +110,24 @@ mod configured_resource_tests { copy_environment(&host_selections, &host, &[], Default::default()).unwrap(); let provider = owned_root.join("provider"); + let nested_file_mounts = units + .iter() + .filter(|(_, mount)| { + mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + && units.iter().any(|(_, directory)| { + directory.kind == freshell_runtime_protocol::ProviderResourceKind::Directory + && Path::new(&mount.effective_destination) + .strip_prefix(&directory.effective_destination) + .is_ok_and(|suffix| !suffix.as_os_str().is_empty()) + }) + }) + .collect::<Vec<_>>(); let provider_selections = units .iter() .filter(|(_, mount)| { - !units.iter().any(|(_, directory)| { - directory.kind == freshell_runtime_protocol::ProviderResourceKind::Directory - && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File - && Path::new(&mount.effective_destination) - .strip_prefix(&directory.effective_destination) - .is_ok_and(|suffix| !suffix.as_os_str().is_empty()) - }) + !nested_file_mounts + .iter() + .any(|(_, nested_mount)| *nested_mount == *mount) }) .map(|(_, mount)| CopySelection { source: host.join(&mount.provider_relative_path), @@ -135,6 +143,34 @@ mod configured_resource_tests { }) .collect::<Vec<_>>(); copy_environment(&provider_selections, &provider, &[], Default::default()).unwrap(); + + // The filesystem helper cannot create Docker volume subpath mounts. + // Overlay each exact nested file as a symlink to its independent host + // volume inode, after projecting the enclosing directory. The script + // then exercises the same path that OpenCode sees at runtime. + for (_, mount) in nested_file_mounts { + let destination_relative = Path::new(&mount.effective_destination) + .strip_prefix(owned_root) + .map(Path::to_path_buf) + .or_else(|_| { + Path::new(&mount.effective_destination) + .strip_prefix("/home/freshell/provider") + .map(|relative| Path::new(".provider-home").join(relative)) + }) + .expect("test provider destinations must map into the temporary root"); + let destination = provider.join(destination_relative); + let host_mount = host.join(&mount.provider_relative_path); + std::fs::create_dir_all(destination.parent().unwrap()).unwrap(); + match std::fs::symlink_metadata(&destination) { + Ok(metadata) => { + assert!(metadata.is_file() || metadata.file_type().is_symlink()); + std::fs::remove_file(&destination).unwrap(); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => panic!("cannot inspect nested provider mount: {error}"), + } + std::os::unix::fs::symlink(host_mount, &destination).unwrap(); + } provider } @@ -1686,9 +1722,19 @@ process.stdout.write(require('./helper.js')) ); assert!(output.status.success(), "{:?}", output.stderr); assert_eq!(output.stdout, b"nested state updated\n"); + assert_eq!( + std::fs::read(&host_state).unwrap(), + b"mount-only runtime state + runtime state", + "the executed provider script must write through the exact host-volume file mount" + ); + assert_eq!( + std::fs::read(&host_directory_state).unwrap(), + b"initial state", + "the exact mount write must not mutate the enclosing directory volume" + ); assert_eq!( std::fs::read(provider_script_directory.join("state.json")).unwrap(), - b"initial state + runtime state" + b"mount-only runtime state + runtime state" ); } diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 1b6b16d2f..9cc0af1cc 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1967,19 +1967,54 @@ fn copy_provider_config_references_with_ownership( }) .collect::<Vec<_>>(); let config_selection_count = selections.len(); - let mut resource_selections = Vec::new(); - for unit in context + let resource_mounts = context .prepared_inputs .iter() .flat_map(|inputs| &inputs.resource_mounts) - { + .collect::<Vec<_>>(); + let mut recovered_writable_files = std::collections::BTreeMap::<PathBuf, PathBuf>::new(); + if let Some(previous) = previous_projection.as_ref() { + for unit in &resource_mounts { + let recovered = + match nested_writable_file_recovery_source(context, unit, previous, final_home) { + Ok(recovered) => recovered, + Err(error) => { + emit_nested_writable_file_recovery_failure(provider, unit, error); + return Err(format!( + "provider resource {} ({}) could not be recovered: {}", + unit.provider_relative_path, + unit.stage_relative_path, + error.message() + )); + } + }; + let Some(source) = recovered else { + continue; + }; + let identity = PathBuf::from(&unit.provider_relative_path); + if recovered_writable_files + .get(&identity) + .is_some_and(|existing| existing != &source) + { + let error = NestedWritableFileRecoveryFailure::AmbiguousAliasSources; + emit_nested_writable_file_recovery_failure(provider, unit, error); + return Err(format!( + "provider resource {} ({}) could not be recovered: {}", + unit.provider_relative_path, + unit.stage_relative_path, + error.message() + )); + } + recovered_writable_files.insert(identity, source); + } + } + let mut resource_selections = Vec::new(); + for unit in resource_mounts { let staged_source = user_provider_root.join(&unit.stage_relative_path); - let source = if let Some(previous) = previous_projection.as_ref() { - nested_writable_file_recovery_source(context, unit, previous, final_home)? - .unwrap_or(staged_source) - } else { - staged_source - }; + let source = recovered_writable_files + .get(Path::new(&unit.provider_relative_path)) + .cloned() + .unwrap_or(staged_source); resource_selections.push(CopySelection { source, destination_relative: unit.provider_relative_path.clone().into(), @@ -2231,12 +2266,76 @@ fn writable_resource_mount_roots( .collect() } +#[derive(Clone, Copy)] +enum NestedWritableFileRecoveryFailure { + PriorStateUnavailable, + PriorResourceDeleted, + PriorResourceUnavailable, + PriorResourceEscapedProjection, + PriorResourceNotOwned, + PriorResourceNotFile, + AmbiguousDirectoryOwners, + AmbiguousAliasSources, +} + +impl NestedWritableFileRecoveryFailure { + fn code(self) -> &'static str { + match self { + Self::PriorStateUnavailable => "prior_provider_state_unavailable", + Self::PriorResourceDeleted => "prior_resource_deleted", + Self::PriorResourceUnavailable => "prior_resource_unavailable", + Self::PriorResourceEscapedProjection => "target_outside_projection", + Self::PriorResourceNotOwned => "target_not_owned", + Self::PriorResourceNotFile => "target_not_file", + Self::AmbiguousDirectoryOwners => "ambiguous_directory_owners", + Self::AmbiguousAliasSources => "ambiguous_alias_sources", + } + } + + fn message(self) -> &'static str { + match self { + Self::PriorStateUnavailable => "prior provider state is unavailable", + Self::PriorResourceDeleted => "prior nested writable resource was deleted", + Self::PriorResourceUnavailable => "prior nested writable resource is unavailable", + Self::PriorResourceEscapedProjection => { + "prior nested writable resource escaped provider state" + } + Self::PriorResourceNotOwned => "prior nested writable resource target is not owned", + Self::PriorResourceNotFile => "prior nested writable resource is not a regular file", + Self::AmbiguousDirectoryOwners => { + "prior nested writable resource has ambiguous directory owners" + } + Self::AmbiguousAliasSources => { + "resource aliases resolve to different prior runtime files" + } + } + } +} + +fn emit_nested_writable_file_recovery_failure( + provider: &str, + mount: &freshell_runtime_protocol::ProviderResourceMount, + error: NestedWritableFileRecoveryFailure, +) { + emit_environment_event(serde_json::json!({ + "severity": "error", + "event": "provider.inputs.recovery.failed", + "stage": "session_host", + "provider": provider, + "operation": "recover_nested_writable_file", + "reason": error.code(), + "resource_stage_path": mount.stage_relative_path, + "provider_relative_path": mount.provider_relative_path, + "corrective_message": "retain provider inputs and repair the resource selection before retrying" + })); +} + fn nested_writable_file_recovery_source( context: &freshell_runtime_protocol::ProviderLaunchContext, mount: &freshell_runtime_protocol::ProviderResourceMount, previous: &std::collections::BTreeSet<String>, previous_home: &Path, -) -> Result<Option<PathBuf>, String> { +) -> Result<Option<PathBuf>, NestedWritableFileRecoveryFailure> { use freshell_runtime_protocol::ProviderResourceKind; if mount.kind != ProviderResourceKind::File || !mount.writable { return Ok(None); @@ -2245,14 +2344,12 @@ fn nested_writable_file_recovery_source( if mount_relative.as_os_str().is_empty() || previous.contains(&mount.provider_relative_path) { return Ok(None); } - let Some(mount_alias) = Path::new(&mount.effective_destination) - .strip_prefix("/home/freshell/provider") - .ok() - else { + let mount_destination = Path::new(&mount.effective_destination); + if !mount_destination.is_absolute() { return Ok(None); - }; + } let canonical_home = std::fs::canonicalize(previous_home) - .map_err(|_| "prior provider state is unavailable for writable-file recovery")?; + .map_err(|_| NestedWritableFileRecoveryFailure::PriorStateUnavailable)?; let mut candidates = Vec::<(usize, PathBuf)>::new(); for directory in context .prepared_inputs @@ -2260,16 +2357,11 @@ fn nested_writable_file_recovery_source( .flat_map(|inputs| &inputs.resource_mounts) .filter(|candidate| candidate.kind == ProviderResourceKind::Directory) { - let Some(directory_alias) = Path::new(&directory.effective_destination) - .strip_prefix("/home/freshell/provider") - .ok() - else { - continue; - }; - if directory_alias.as_os_str().is_empty() { + let directory_destination = Path::new(&directory.effective_destination); + if !directory_destination.is_absolute() { continue; } - let Ok(suffix) = mount_alias.strip_prefix(directory_alias) else { + let Ok(suffix) = mount_destination.strip_prefix(directory_destination) else { continue; }; if suffix.as_os_str().is_empty() { @@ -2284,36 +2376,22 @@ fn nested_writable_file_recovery_source( let canonical = match std::fs::canonicalize(&prior_path) { Ok(path) => path, Err(error) if error.kind() == std::io::ErrorKind::NotFound => { - return Err( - "prior nested writable resource was deleted; retain inputs and repair".into(), - ) - } - Err(_) => { - return Err( - "prior nested writable resource is unavailable; retain inputs and repair" - .into(), - ) + return Err(NestedWritableFileRecoveryFailure::PriorResourceDeleted) } + Err(_) => return Err(NestedWritableFileRecoveryFailure::PriorResourceUnavailable), }; - let canonical_relative = canonical.strip_prefix(&canonical_home).map_err(|_| { - "prior nested writable resource escaped provider state; retain inputs and repair" - })?; + let canonical_relative = canonical + .strip_prefix(&canonical_home) + .map_err(|_| NestedWritableFileRecoveryFailure::PriorResourceEscapedProjection)?; if !previous.contains(&canonical_relative.to_string_lossy().into_owned()) { - return Err( - "prior nested writable resource target is not owned; retain inputs and repair" - .into(), - ); + return Err(NestedWritableFileRecoveryFailure::PriorResourceNotOwned); } - let metadata = std::fs::metadata(&canonical).map_err(|_| { - "prior nested writable resource is unavailable; retain inputs and repair" - })?; + let metadata = std::fs::metadata(&canonical) + .map_err(|_| NestedWritableFileRecoveryFailure::PriorResourceUnavailable)?; if !metadata.is_file() { - return Err( - "prior nested writable resource is not a regular file; retain inputs and repair" - .into(), - ); + return Err(NestedWritableFileRecoveryFailure::PriorResourceNotFile); } - candidates.push((directory_alias.components().count(), canonical)); + candidates.push((directory_destination.components().count(), canonical)); } candidates.sort_by_key(|(depth, path)| (std::cmp::Reverse(*depth), path.clone())); let Some((depth, source)) = candidates.first().cloned() else { @@ -2325,10 +2403,7 @@ fn nested_writable_file_recovery_source( .take_while(|(candidate_depth, _)| *candidate_depth == depth) .any(|(_, candidate)| candidate != &source) { - return Err( - "prior nested writable resource has ambiguous directory owners; retain inputs and repair" - .into(), - ); + return Err(NestedWritableFileRecoveryFailure::AmbiguousDirectoryOwners); } Ok(Some(source)) } @@ -4494,6 +4569,219 @@ mod tests { ) } + fn opencode_resource_mount( + id: &str, + logical_destination: &str, + effective_destination: &str, + kind: freshell_runtime_protocol::ProviderResourceKind, + writable: bool, + ) -> freshell_runtime_protocol::ProviderResourceMount { + let file = kind == freshell_runtime_protocol::ProviderResourceKind::File; + freshell_runtime_protocol::ProviderResourceMount { + stage_relative_path: if file { + format!(".freshell/resource-units/{id}/source") + } else { + format!(".freshell/resource-units/{id}") + }, + provider_relative_path: if file { + format!(".freshell/provider-roots/{id}/source") + } else { + format!(".freshell/provider-roots/{id}") + }, + logical_destination: logical_destination.into(), + effective_destination: effective_destination.into(), + kind, + writable, + } + } + + fn opencode_context_with_mounts( + resource_mounts: Vec<freshell_runtime_protocol::ProviderResourceMount>, + ) -> freshell_runtime_protocol::ProviderLaunchContext { + use freshell_runtime_protocol::*; + let capability = McpCapabilityReference { + grant_id: "grant-managed-provider-recovery-test".into(), + endpoint: "http://host.docker.internal:1234".into(), + provider_relative_path: ".freshell/mcp-capability.json".into(), + host_gateway_address: None, + }; + ProviderLaunchContext { + config: vec![], + preparation: ProviderPreparation::Opencode { + project_config: vec![], + tui_config: None, + tui_source: None, + inline_config: false, + }, + prepared_inputs: Some(PreparedProviderInputs { + generation_id: capability.grant_id.clone(), + initial_auth_reservation: None, + manifest: ProviderInputManifestReference { + relative_path: ".freshell/input-manifest.json".into(), + sha256: "a".repeat(64), + source_count: 0, + alias_count: 0, + recipe_count: 0, + }, + config_binds: vec![], + indexed_files: None, + resource_mounts, + }), + mcp_capability: Some(capability), + } + } + + #[test] + fn provider_config_recovery_migrates_nested_writable_file_under_external_directory() { + use freshell_runtime_protocol::ProviderResourceKind; + + let owned = tempfile::tempdir().unwrap(); + let user_root = owned.path().join("provider"); + let workspace = owned.path().join("workspace"); + let home = owned.path().join("home"); + let directory_source = user_root.join(".freshell/resource-units/external-directory"); + let file_source = user_root.join(".freshell/resource-units/external-state/source"); + let directory_relative = PathBuf::from(".freshell/provider-roots/external-directory"); + let file_relative = PathBuf::from(".freshell/provider-roots/external-state/source"); + std::fs::create_dir_all(&directory_source).unwrap(); + std::fs::create_dir_all(file_source.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::write(directory_source.join("state.json"), "external seed").unwrap(); + std::fs::write(directory_source.join("ordinary.json"), "ordinary seed").unwrap(); + std::fs::write(&file_source, "stale exact seed").unwrap(); + + let directory = opencode_resource_mount( + "external-directory", + "/opt/provider-mcp", + "/opt/provider-mcp", + ProviderResourceKind::Directory, + false, + ); + let initial = opencode_context_with_mounts(vec![directory.clone()]); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + copy_provider_config_references( + &initial, "opencode", &workspace, &user_root, &home, uid, gid, + ) + .unwrap(); + use std::io::Write; + std::fs::OpenOptions::new() + .append(true) + .open(home.join(&directory_relative).join("state.json")) + .unwrap() + .write_all(b" + runtime state") + .unwrap(); + std::fs::write( + directory_source.join("state.json"), + "external refreshed seed", + ) + .unwrap(); + std::fs::write( + directory_source.join("ordinary.json"), + "ordinary refreshed seed", + ) + .unwrap(); + std::fs::create_dir_all(home.join(file_relative.parent().unwrap())).unwrap(); + std::fs::write(home.join(&file_relative), "").unwrap(); + + let writable = opencode_resource_mount( + "external-state", + "/opt/provider-mcp/state.json", + "/opt/provider-mcp/state.json", + ProviderResourceKind::File, + true, + ); + let refresh = opencode_context_with_mounts(vec![directory, writable]); + refresh_provider_config_references( + &refresh, "opencode", &workspace, &user_root, &home, uid, gid, + ) + .unwrap(); + + assert_eq!( + std::fs::read(home.join(&file_relative)).unwrap(), + b"external seed + runtime state", + "new exact writable mounts under external directories must inherit prior runtime data" + ); + assert_eq!( + std::fs::read(home.join(&directory_relative).join("state.json")).unwrap(), + b"external refreshed seed", + "the directory projection must still refresh independently" + ); + assert_eq!( + std::fs::read(home.join(&directory_relative).join("ordinary.json")).unwrap(), + b"ordinary refreshed seed" + ); + } + + #[test] + fn provider_config_recovery_uses_one_source_for_resource_aliases() { + use freshell_runtime_protocol::ProviderResourceKind; + + let owned = tempfile::tempdir().unwrap(); + let user_root = owned.path().join("provider"); + let workspace = owned.path().join("workspace"); + let home = owned.path().join("home"); + let directory_source = user_root.join(".freshell/resource-units/aliased-directory"); + let file_source = user_root.join(".freshell/resource-units/aliased-state/source"); + let directory_relative = PathBuf::from(".freshell/provider-roots/aliased-directory"); + let file_relative = PathBuf::from(".freshell/provider-roots/aliased-state/source"); + std::fs::create_dir_all(&directory_source).unwrap(); + std::fs::create_dir_all(file_source.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::write(directory_source.join("state.json"), "alias seed").unwrap(); + std::fs::write(&file_source, "stale alias seed").unwrap(); + let directory = opencode_resource_mount( + "aliased-directory", + "/workspace/mcp", + "/home/freshell/provider/mcp", + ProviderResourceKind::Directory, + false, + ); + let initial = opencode_context_with_mounts(vec![directory.clone()]); + let (uid, gid) = unsafe { (libc::geteuid(), libc::getegid()) }; + copy_provider_config_references( + &initial, "opencode", &workspace, &user_root, &home, uid, gid, + ) + .unwrap(); + use std::io::Write; + std::fs::OpenOptions::new() + .append(true) + .open(home.join(&directory_relative).join("state.json")) + .unwrap() + .write_all(b" + runtime state") + .unwrap(); + std::fs::create_dir_all(home.join(file_relative.parent().unwrap())).unwrap(); + std::fs::write(home.join(&file_relative), "").unwrap(); + + let primary_alias = opencode_resource_mount( + "aliased-state", + "/workspace/mcp/state.json", + "/home/freshell/provider/mcp/state.json", + ProviderResourceKind::File, + true, + ); + let original_path_alias = opencode_resource_mount( + "aliased-state", + "/external/input/state.json", + "/external/input/state.json", + ProviderResourceKind::File, + true, + ); + let refresh = + opencode_context_with_mounts(vec![directory, primary_alias, original_path_alias]); + refresh_provider_config_references( + &refresh, "opencode", &workspace, &user_root, &home, uid, gid, + ) + .unwrap(); + + assert_eq!( + std::fs::read(home.join(&file_relative)).unwrap(), + b"alias seed + runtime state", + "aliases of one physical writable root must all use its recovered runtime source" + ); + } + fn executable_fixture() -> tempfile::TempDir { match std::env::var_os("FRESHELL_TEST_EXECUTABLE_TMP_DIR") { Some(directory) => tempfile::tempdir_in(directory), @@ -6299,6 +6587,57 @@ mod tests { .contains("repair")); assert!(event["elapsed_ms"].as_u64().is_some()); + let redaction_root = tempfile::tempdir().unwrap(); + let redaction_root_text = redaction_root.path().to_string_lossy().into_owned(); + let recovery = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "tests::provider_config_recovery_preserves_written_writable_file_and_refreshes_read_only_file", + "--nocapture", + ]) + .env("TMPDIR", redaction_root.path()) + .env("TMP", redaction_root.path()) + .env("TEMP", redaction_root.path()) + .output() + .unwrap(); + assert!(recovery.status.success(), "{recovery:?}"); + let recovery_stderr = std::str::from_utf8(&recovery.stderr).unwrap(); + let recovery_events = recovery_stderr + .lines() + .map(|line| serde_json::from_str::<serde_json::Value>(line).unwrap()) + .collect::<Vec<_>>(); + let recovery_failures = recovery_events + .iter() + .filter(|event| event["event"] == "provider.inputs.recovery.failed") + .collect::<Vec<_>>(); + assert_eq!( + recovery_failures.len(), + 1, + "a blocked writable-file recovery must produce one resource-specific diagnostic" + ); + let recovery_failure = recovery_failures[0]; + assert_eq!(recovery_failure["severity"], "error"); + assert_eq!(recovery_failure["stage"], "session_host"); + assert_eq!(recovery_failure["provider"], "opencode"); + assert_eq!( + recovery_failure["operation"], + "recover_nested_writable_file" + ); + assert_eq!(recovery_failure["reason"], "prior_resource_deleted"); + assert_eq!( + recovery_failure["resource_stage_path"], + ".freshell/resource-units/deleted/source" + ); + assert_eq!( + recovery_failure["provider_relative_path"], + ".freshell/provider-roots/deleted/source" + ); + assert!( + !recovery_stderr.contains("stale host seed") + && !recovery_stderr.contains(&redaction_root_text), + "recovery diagnostics must not expose fixture contents or temporary host paths" + ); + #[cfg(unix)] { let filtered = std::process::Command::new(std::env::current_exe().unwrap()) From 8cfc59d2bf89af44cac47e4483ba215bc1738a17 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 09:09:05 -0700 Subject: [PATCH 434/493] fix(test): unblock managed runtime browser retries --- scripts/testing/managed-environment-create.ts | 60 +++++++++ .../managed-provider-environment-target.ts | 126 ++++++++++++------ ...aged-provider-environment-external.spec.ts | 28 +++- .../tooling/testing/runtime-sandbox.test.ts | 54 +++++++- 4 files changed, 219 insertions(+), 49 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 5fd1ef309..61797a849 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -17,6 +17,66 @@ export type ManagedEnvironmentAttempt = ManagedEnvironmentFixture & { export type ManagedEnvironmentCompletion = ManagedEnvironmentAttempt & { status: string willRetry: boolean + failureEvidence?: unknown +} + +export type ManagedEnvironmentReadyOutcome<T> = + | { kind: 'ready'; value: T } + | { kind: 'browser_attempt_failed' } + +export function writeManagedEnvironmentBrowserFailureIncident( + request: ManagedEnvironmentAttempt, + evidence: Record<string, unknown>, + writeIncident: (name: string, value: unknown) => void, +): boolean { + try { + writeIncident(`managed-environment-browser-${request.scenario}-${request.attempt}`, { + ...evidence, + severity: 'error', event: 'managed_environment.browser_runtime_capture', + scenario: request.scenario, attempt: request.attempt, + }) + return true + } catch { + return false + } +} + +/** + * Wait for a healthy OpenCode child without trapping a failed browser attempt + * behind the longer child-startup deadline. Playwright writes the matching + * completion from afterEach; a retry cannot be authorized until this wait + * returns. + */ +export async function waitForManagedEnvironmentReadyOrBrowserFailure<T>( + request: ManagedEnvironmentAttempt, + observeReady: () => Promise<T | undefined>, + observeCompletion: () => Promise<ManagedEnvironmentCompletion | undefined>, + options: { timeoutMs?: number; pollIntervalMs?: number } = {}, +): Promise<ManagedEnvironmentReadyOutcome<T>> { + const timeoutMs = options.timeoutMs ?? 180_000 + const pollIntervalMs = options.pollIntervalMs ?? 250 + const deadline = Date.now() + timeoutMs + while (true) { + const ready = await observeReady() + if (ready !== undefined) return { kind: 'ready', value: ready } + + const completion = await observeCompletion() + if (completion + && completion.candidateSha === request.candidateSha + && completion.runId === request.runId + && completion.remoteWorkspacePath === request.remoteWorkspacePath + && completion.expectedNonce === request.expectedNonce + && completion.scenario === request.scenario + && completion.attempt === request.attempt + && (completion.createRequestId === undefined || completion.createRequestId === request.createRequestId) + && completion.status !== 'passed') { + return { kind: 'browser_attempt_failed' } + } + if (Date.now() >= deadline) { + throw new Error(`Managed environment timed out: actual selected Bun child for attempt ${request.attempt}`) + } + await new Promise<void>(resolve => setTimeout(resolve, pollIntervalMs)) + } } export async function serviceManagedEnvironmentCases( diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 872168bd5..c8e4d9c80 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -10,11 +10,13 @@ import WebSocket from 'ws' import { WS_PROTOCOL_VERSION, type ClientMessage } from '../../shared/ws-protocol.js' import { ManagedRuntimeBrowserRig, type ManagedRuntimeView } from '../../test/e2e-browser/helpers/managed-runtime.js' import { authorizeProviderParityRequest, captureProviderParityVolumes, cleanupProviderParityVolumes } from '../../test/helpers/provider-parity-broker.js' -import { managedEnvironmentOpencodeSelection } from '../../test/helpers/provider-parity-environment.js' +import { managedEnvironmentOpencodeSelection, providerParityFailureDiagnostics } from '../../test/helpers/provider-parity-environment.js' import { fakeOnecliGrants } from '../../test/helpers/provider-parity-onecli.js' import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.js' import { resolvePairedExternalFixture } from './e2e-external-target.js' -import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentCases } from './managed-environment-create.js' +import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, + serviceManagedEnvironmentCases, summarizeRustServerLogEvents, waitForManagedEnvironmentReadyOrBrowserFailure, + writeManagedEnvironmentBrowserFailureIncident } from './managed-environment-create.js' import { waitForCloudExecutionDispatch } from './cloud-execution-dispatch.js' import { ManagedEnvironmentBrowserSummaryCapture, type ManagedEnvironmentBrowserSummary } from './managed-environment-browser-summary.js' @@ -1449,47 +1451,85 @@ export async function runManagedEnvironmentTarget() { assert.equal(browser!.exitCode, null, 'Cloud browser exited without its attempt completion') return undefined }, scenario === 'healthy' ? 360_000 : 300_000), - onAuthorized: async request => { if (scenario === 'healthy') { - const { child, view } = await poll('actual selected Bun child and current request', async () => { - const child = await readJson(path.join(cwd, '.managed-fixture/child-receipt.json')) - if (!child) return undefined - const view = (await rig!.inventory()).find(row => row.terminalId === child.terminalId - && row.terminalCreateRequestId === request.createRequestId && row.nativeSessionId === child.nativeSessionId && row.launchState === 'running') - return view ? { child, view } : undefined - }) - assert.equal(child.expectedNonce, nonce); assert.equal(child.candidateSha, fixture.candidateSha); assert.equal(child.runId, fixture.runId) - assert.equal(child.bun, true); assert.equal(child.opencodeVersion, '1.18.21'); assert.match(child.bunExecutable, /opencode/) - assert.equal(child.executableMode, '0711') - const mcpServer = await poll('Cloud OpenCode starts the nested Bun package-script MCP', async () => - readJson(path.join(cwd, '.managed-fixture/mcp-receipt.json'))) - assert.equal(mcpServer.expectedNonce, nonce) - assert.equal(mcpServer.dependency, nonce, 'OpenCode MCP must load the staged parent dependency') - assert.equal(mcpServer.bun, true, 'OpenCode must run the local MCP package script under Bun') - assert.equal(mcpServer.bunVersion, '1.4.3', 'Managed OpenCode MCP must use the pinned Bun runtime') - assert.equal(mcpServer.launchCwd, selected.mcpPackage, 'Bun runs a package script from its package root') - assert.equal(mcpServer.bunCommandCwd, selected.mcpNested, 'OpenCode must launch Bun from the configured nested MCP cwd') - assert.equal(mcpServer.configuredCwd, selected.mcpNested) - assert(mcpServer.scriptPath.endsWith('/fixture-mcp/server.mjs')) - const directMcpServer = await poll('Cloud OpenCode starts the direct extensionless MCP through its sibling helper', async () => - readJson(path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'))) - assert.equal(directMcpServer.expectedNonce, nonce) - assert.equal(directMcpServer.helper, nonce) - assert.equal(directMcpServer.sibling, nonce) - assert.equal(directMcpServer.dependency, nonce) - assert.equal(directMcpServer.schemaNonce, nonce) - assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.mjs')) - const directHomeAppendMcp = await poll('Cloud OpenCode appends through its direct HOME MCP', async () => - readJson(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json'))) - assert.equal(directHomeAppendMcp.expectedNonce, nonce) - assert.equal(directHomeAppendMcp.appendError, undefined, 'the managed HOME sibling mount must permit append') - assert.equal(directHomeAppendMcp.output, `initial-${nonce}|${nonce}`) - assert(directHomeAppendMcp.scriptPath.endsWith('/direct-home-healthy.mjs')) - const stage = await stageProof(rig!, view, nonce) - const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, - nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer, directHomeAppendMcp } - await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) - proof.healthy = receipt - } }, + onAuthorized: async request => { + if (scenario !== 'healthy') return + const completionPath = path.join(cwd, '.managed-fixture/completion.json') + const outcome = await waitForManagedEnvironmentReadyOrBrowserFailure(request, async () => { + const child = await readJson(path.join(cwd, '.managed-fixture/child-receipt.json')) + if (!child) return undefined + const view = (await rig!.inventory()).find(row => row.terminalId === child.terminalId + && row.terminalCreateRequestId === request.createRequestId && row.nativeSessionId === child.nativeSessionId + && row.launchState === 'running') + return view ? { child, view } : undefined + }, () => readJson(completionPath)) + if (outcome.kind === 'browser_attempt_failed') { + const completion = await readJson(completionPath) + let rows: ManagedRuntimeView[] = [] + try { rows = (await rig!.inventory()).filter(row => row.terminalCreateRequestId === request.createRequestId) } + catch { /* Keep browser failure evidence even when a fresh inventory cannot be read. */ } + const view = rows[0] + const summarizeOwnedLogs = (containerId: string) => { + try { return providerParityFailureDiagnostics(rig!.runtime.containerLogs(containerId).slice(-512 * 1024)) } + catch { return 'UNKNOWN' } + } + const supervisorLogs = summarizeOwnedLogs(rig!.supervisor.containerId) + const sessionHostLogs = view?.containerId ? summarizeOwnedLogs(view.containerId) : undefined + let serverLogEvents: ReturnType<typeof summarizeRustServerLogEvents> = [] + try { serverLogEvents = summarizeRustServerLogEvents(Object.values(rig!.web.capturedOutput()).join('\n')) } + catch { /* Keep browser and runtime evidence if the scratch server log is unavailable. */ } + const captured = writeManagedEnvironmentBrowserFailureIncident(request, { + browser: completion?.failureEvidence ?? 'UNKNOWN', + managedTerminal: view ? { + launchState: view.launchState, desiredState: view.desiredState, recoveryState: view.recoveryState, + durabilityState: view.durabilityState, allocationState: view.allocationState, + provider: view.provider, terminalMode: view.terminalMode, + nativeIdentityPresent: Boolean(view.nativeSessionId), sessionHostPresent: Boolean(view.containerId), + } : { matchingTerminalCount: rows.length }, + ...(sessionHostLogs ? { sessionHostLogs } : {}), supervisorLogs, serverLogEvents, + }, (name, value) => rig!.runtime.writeIncident(name, value)) + if (!captured) { + console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_runtime_capture_persist_failed', scenario, attempt: request.attempt })) + } + return + } + const { child, view } = outcome.value + assert.equal(child.expectedNonce, nonce) + assert.equal(child.candidateSha, fixture.candidateSha) + assert.equal(child.runId, fixture.runId) + assert.equal(child.bun, true) + assert.equal(child.opencodeVersion, '1.18.21') + assert.match(child.bunExecutable, /opencode/) + assert.equal(child.executableMode, '0711') + const mcpServer = await poll('Cloud OpenCode starts the nested Bun package-script MCP', async () => + readJson(path.join(cwd, '.managed-fixture/mcp-receipt.json'))) + assert.equal(mcpServer.expectedNonce, nonce) + assert.equal(mcpServer.dependency, nonce, 'OpenCode MCP must load the staged parent dependency') + assert.equal(mcpServer.bun, true, 'OpenCode must run the local MCP package script under Bun') + assert.equal(mcpServer.bunVersion, '1.4.3', 'Managed OpenCode MCP must use the pinned Bun runtime') + assert.equal(mcpServer.launchCwd, selected.mcpPackage, 'Bun runs a package script from its package root') + assert.equal(mcpServer.bunCommandCwd, selected.mcpNested, 'OpenCode must launch Bun from the configured nested MCP cwd') + assert.equal(mcpServer.configuredCwd, selected.mcpNested) + assert(mcpServer.scriptPath.endsWith('/fixture-mcp/server.mjs')) + const directMcpServer = await poll('Cloud OpenCode starts the direct extensionless MCP through its sibling helper', async () => + readJson(path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'))) + assert.equal(directMcpServer.expectedNonce, nonce) + assert.equal(directMcpServer.helper, nonce) + assert.equal(directMcpServer.sibling, nonce) + assert.equal(directMcpServer.dependency, nonce) + assert.equal(directMcpServer.schemaNonce, nonce) + assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.mjs')) + const directHomeAppendMcp = await poll('Cloud OpenCode appends through its direct HOME MCP', async () => + readJson(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json'))) + assert.equal(directHomeAppendMcp.expectedNonce, nonce) + assert.equal(directHomeAppendMcp.appendError, undefined, 'the managed HOME sibling mount must permit append') + assert.equal(directHomeAppendMcp.output, `initial-${nonce}|${nonce}`) + assert(directHomeAppendMcp.scriptPath.endsWith('/direct-home-healthy.mjs')) + const stage = await stageProof(rig!, view, nonce) + const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, + nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer, directHomeAppendMcp } + await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) + proof.healthy = receipt + }, } }, async () => { const browserExit = await bounded(browserJoin!, 120_000, 'configured Cloud browser final join') diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 6227910ff..6b0d99ea2 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -4,6 +4,7 @@ import { test, expect } from '../helpers/fixtures.js' import { openPanePicker } from '../helpers/pane-picker.js' import { TestHarness } from '../helpers/test-harness.js' import type { ProviderPreparationFailure } from '../../../shared/ws-protocol.js' +import { summarizeProviderPreparationFailure, summarizeRuntimeErrorClass, summarizeRuntimeErrorCode } from '../../../scripts/testing/managed-environment-create.js' const paired = resolvePairedExternalFixture(process.env, true) if (paired?.kind !== 'managed-environment') throw new Error('Managed environment requires its paired owned target') @@ -17,7 +18,7 @@ const caseScenarios: Record<string, string> = { } async function terminalDiagnostic(page: Page, terminalId?: string) { - return page.evaluate(({ terminalId, marker }) => { + const diagnostic = await page.evaluate(({ terminalId, marker }) => { const harness = window.__FRESHELL_TEST_HARNESS__ const buffer = terminalId && harness ? harness.getTerminalBuffer(terminalId) : null const received = (harness?.getReceivedWsMessages?.() ?? []) as any[] @@ -42,7 +43,9 @@ async function terminalDiagnostic(page: Page, terminalId?: string) { const attachErrors = received.filter(message => message.type === 'error' && (message.terminalId === terminalId || typeof message.requestId === 'string' && attachRequests.has(message.requestId))) .slice(-16).map(message => ({ type: 'error', code: ['INVALID_REQUEST', 'TERMINAL_NOT_FOUND', 'PTY_SPAWN_FAILED', - 'SESSION_RESERVED', 'INTERNAL_ERROR', 'AUTH_FAILED'].includes(message.code) ? message.code : 'UNKNOWN' })) + 'SESSION_RESERVED', 'INTERNAL_ERROR', 'AUTH_FAILED'].includes(message.code) ? message.code : 'UNKNOWN', + message: typeof message.message === 'string' ? message.message.slice(0, 4096) : undefined, + providerPreparationFailure: message.providerPreparationFailure })) const metadata = (messages: any[]) => messages.filter(message => message.terminalId === terminalId && [ 'terminal.attach', 'terminal.replay.credit', 'terminal.attach.ready', 'terminal.output.gap', 'terminal.stream.changed', 'terminal.modes.sync', 'terminal.turn.complete', @@ -73,6 +76,16 @@ async function terminalDiagnostic(page: Page, terminalId?: string) { // These getters retain bounded history, not a complete transport trace. receivedHistoryAtBound: received.length >= 500, writeHistoryAtBound: writes.length >= 1000 } }, { terminalId, marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) + if (!('attachErrors' in diagnostic) || !Array.isArray(diagnostic.attachErrors)) return diagnostic + return { ...diagnostic, attachErrors: diagnostic.attachErrors.map((error: any) => { + const runtimeErrorClass = summarizeRuntimeErrorClass(error.message ?? '') + const runtimeErrorCode = summarizeRuntimeErrorCode(error.message ?? '') + const providerPreparationFailure = summarizeProviderPreparationFailure(error.providerPreparationFailure) + return { type: error.type, code: error.code, + ...(runtimeErrorClass ? { runtimeErrorClass } : {}), + ...(runtimeErrorCode ? { runtimeErrorCode } : {}), + ...(providerPreparationFailure ? { providerPreparationFailure } : {}) } + }) } } async function hasCompletedOutputWrite(page: Page, terminalId: string, marker: string, after = 0) { @@ -94,20 +107,25 @@ test.afterEach(async ({ page }, info) => { if (!scenario) throw new Error('Unknown managed environment case') const cwd = `${fixture.remoteWorkspacePath}/${scenario}` const request = await readReceipt(page, `${cwd}/.managed-fixture/request.json`) + let failureEvidence: Record<string, unknown> | undefined if (info.status !== info.expectedStatus) { const pane = await activePane(page).catch(() => null) const diagnostic = await terminalDiagnostic(page, pane?.content.kind === 'terminal' ? pane.content.terminalId : undefined) - console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_attempt_failed', - scenario, attempt: info.retry, status: info.status, + failureEvidence = { stage: info.annotations.filter(value => value.type === 'managed-stage').at(-1)?.description ?? 'UNKNOWN', terminalPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.terminalId), soulPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.soulId), nativeIdentityPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.sessionRef), - diagnostic })) + diagnostic, + } + console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_attempt_failed', + scenario, attempt: info.retry, status: info.status, + ...failureEvidence })) } const response = await page.request.post(`${paired!.targetUrl}/api/files/write`, { headers, timeout: 30_000, data: { path: `${cwd}/.managed-fixture/completion.json`, content: JSON.stringify({ ...fixture, scenario, attempt: info.retry, status: info.status, willRetry: info.status !== info.expectedStatus && info.retry < info.project.retries, + ...(failureEvidence ? { failureEvidence } : {}), ...(request?.attempt === info.retry ? { createRequestId: request.createRequestId } : {}) }) } }) expect(response.ok()).toBe(true) }) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index afffad205..93f47fb9f 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -35,7 +35,17 @@ import { RuntimeHarness, dockerSocketPath } from '../../../../scripts/testing/ru import { RestrictedDockerBroker, type PreparedInputAuthorization } from '../../../../scripts/testing/runtime-test-broker.js' import { ManagedRuntimeBrowserRig } from '../../../e2e-browser/helpers/managed-runtime.js' import { prepareProviderParityUserRoots } from '../../../helpers/provider-parity-environment.js' -import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentScenario, serviceManagedEnvironmentCases, type ManagedEnvironmentAttempt, type ManagedEnvironmentCompletion } from '../../../../scripts/testing/managed-environment-create.js' +import { + assertManagedTerminalCreated, + observeManagedEnvironmentNativeIdentity, + readManagedEnvironmentDaemonId, + serviceManagedEnvironmentScenario, + serviceManagedEnvironmentCases, + waitForManagedEnvironmentReadyOrBrowserFailure, + writeManagedEnvironmentBrowserFailureIncident, + type ManagedEnvironmentAttempt, + type ManagedEnvironmentCompletion, +} from '../../../../scripts/testing/managed-environment-create.js' import type { ErrorMessage } from '../../../../shared/ws-protocol.js' describe('managed environment browser attempt lifetime', () => { @@ -69,6 +79,48 @@ describe('managed environment browser attempt lifetime', () => { }) }) + it('stops waiting for a healthy child when the matching browser attempt has already failed', async () => { + const current = request(0) + let completionReady = false + let childReads = 0 + const outcome = waitForManagedEnvironmentReadyOrBrowserFailure( + current, + async () => { childReads += 1; return undefined }, + async () => completionReady ? completion(0, true) : undefined, + { timeoutMs: 500, pollIntervalMs: 1 }, + ) + setTimeout(() => { completionReady = true }, 20) + + await expect(outcome).resolves.toMatchObject({ kind: 'browser_attempt_failed' }) + expect(childReads).toBeGreaterThan(0) + }) + + it('ignores stale retry completions while waiting for the current child', async () => { + let childReads = 0 + const outcome = await waitForManagedEnvironmentReadyOrBrowserFailure( + request(1), + async () => { childReads += 1; return childReads === 2 ? 'owned-child' : undefined }, + async () => completion(0, true), + { timeoutMs: 500, pollIntervalMs: 1 }, + ) + + expect(outcome).toEqual({ kind: 'ready', value: 'owned-child' }) + expect(childReads).toBe(2) + }) + + it('keeps browser failure evidence best effort so incident-write errors cannot block retries', () => { + const requestValue = request(1) + const writeIncident = vi.fn(() => { throw new Error('incident storage unavailable') }) + + expect(writeManagedEnvironmentBrowserFailureIncident(requestValue, { browser: 'safe diagnostic' }, writeIncident)).toBe(false) + expect(writeIncident).toHaveBeenCalledWith('managed-environment-browser-healthy-1', expect.objectContaining({ + event: 'managed_environment.browser_runtime_capture', + scenario: 'healthy', + attempt: 1, + browser: 'safe diagnostic', + })) + }) + it('keeps the owned target alive until the browser reports attempt completion', async () => { await ownedFixture(async directory => { await fs.writeFile(path.join(directory, 'request.json'), JSON.stringify(request(0))) From c273fb72ea695307c40e61d1b195b4254edf42f1 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 10:23:38 -0700 Subject: [PATCH 435/493] fix(managed): protect OpenCode MCP env files --- .../src/managed_mcp_capability.rs | 4 + .../managed_mcp_capability/source_stage.rs | 338 +++++++++++++++--- .../src/managed_provider_references.rs | 70 ++-- .../opencode_resources.rs | 231 +++++++----- .../managed_provider_references/sources.rs | 136 ++++++- 5 files changed, 616 insertions(+), 163 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 96b1937d5..29d96c60f 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1683,6 +1683,10 @@ fn stage_provider_root( .iter() .map(|selection| selection.canonical_source.clone()) })); + excluded.extend( + plan.iter() + .flat_map(|source| source.configured_resource_exclusions.iter().cloned()), + ); let mut units = source_stage::resource_units( home, &source, diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 85244812d..2f73f1fbd 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -67,6 +67,17 @@ mod configured_resource_tests { std::path::PathBuf, freshell_runtime_protocol::ProviderResourceMount, )], + ) -> std::path::PathBuf { + copy_selected_units_through_provider_excluding(owned_root, units, &[]) + } + + fn copy_selected_units_through_provider_excluding( + owned_root: &Path, + units: &[( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + excluded: &[std::path::PathBuf], ) -> std::path::PathBuf { let stage = owned_root.join("stage"); let stage_selections = units @@ -93,7 +104,7 @@ mod configured_resource_tests { copy_environment_for_mounts( &stage_selections, &stage, - &[], + excluded, Default::default(), &mounted_file_roots, ) @@ -174,6 +185,25 @@ mod configured_resource_tests { provider } + fn tree_contains_bytes(root: &Path, needle: &[u8]) -> bool { + let Ok(entries) = std::fs::read_dir(root) else { + return false; + }; + entries.flatten().any(|entry| { + let Ok(metadata) = std::fs::symlink_metadata(entry.path()) else { + return false; + }; + if metadata.is_dir() { + tree_contains_bytes(&entry.path(), needle) + } else if metadata.is_file() { + std::fs::read(entry.path()) + .is_ok_and(|bytes| bytes.windows(needle.len()).any(|window| window == needle)) + } else { + false + } + }) + } + fn plan_and_stage_direct_mcp_command( owned_root: &Path, home: &Path, @@ -263,7 +293,21 @@ mod configured_resource_tests { .map(|relative| Path::new(".provider-home").join(relative)) }) .expect("the test provider destination must map into the temporary root"); - let provider = copy_selected_units_through_provider(owned_root, &units); + let excluded = plan + .iter() + .flat_map(|source| { + source + .references + .selections + .iter() + .map(|selection| selection.canonical_source.clone()) + .chain(source.configured_resource_exclusions.iter().cloned()) + }) + .collect::<std::collections::BTreeSet<_>>() + .into_iter() + .collect::<Vec<_>>(); + let provider = + copy_selected_units_through_provider_excluding(owned_root, &units, &excluded); (plan, provider.join(provider_relative)) } @@ -1233,7 +1277,7 @@ mod configured_resource_tests { std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "#!/usr/bin/env node\nprocess.stdout.write(require('./helper.js'))\n", + "#!/usr/bin/env node\nconst helperPath = require.resolve('./helper.js');\nprocess.stdout.write(require(helperPath))\n", ) .unwrap(); std::fs::write( @@ -1406,9 +1450,10 @@ process.stdout.write(require('./helper.js')) let env_file = home.join(".env"); let second_env_file = home.join(".env.local"); let optional_env_file = home.join(".env.optional"); + let missing_optional_env_file = home.join(".env.missing-optional"); std::fs::create_dir_all(&home).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); - std::fs::write(&command, "if (globalThis.managedPreload !== 'loaded' || globalThis.managedPreloadHelper !== 'loaded' || globalThis.managedImport !== 'loaded' || process.env.MANAGED_ENV_FILE !== 'loaded' || process.env.MANAGED_ENV_FILE_LOCAL !== 'loaded') process.exit(1);\nprocess.stdout.write('command options staged\\n');\n").unwrap(); + std::fs::write(&command, "if (globalThis.managedPreload !== 'loaded' || globalThis.managedPreloadHelper !== 'loaded' || globalThis.managedImport !== 'loaded' || !process.env.MANAGED_ENV_FILE?.startsWith('MCP_ENV_SECRET_') || !process.env.MANAGED_ENV_FILE_LOCAL?.startsWith('MCP_ENV_SECRET_') || !process.env.MANAGED_ENV_FILE_OPTIONAL?.startsWith('MCP_ENV_SECRET_')) process.exit(1);\nprocess.stdout.write('command options staged\\n');\n").unwrap(); std::fs::write( &preload, "require('./preload-helper.cjs');\nglobalThis.managedPreload = 'loaded';\n", @@ -1422,8 +1467,24 @@ process.stdout.write(require('./helper.js')) std::fs::write(&imported, "globalThis.managedImport = 'loaded';\n").unwrap(); std::fs::write(&loader, "export async function resolve(specifier, context, nextResolve) { return nextResolve(specifier, context); }\n").unwrap(); std::fs::write(&experimental_loader, "export async function resolve(specifier, context, nextResolve) { return nextResolve(specifier, context); }\n").unwrap(); - std::fs::write(&env_file, "MANAGED_ENV_FILE=loaded\n").unwrap(); - std::fs::write(&second_env_file, "MANAGED_ENV_FILE_LOCAL=loaded\n").unwrap(); + let first_secret = b"MCP_ENV_SECRET_ALPHA_4f7c5d91\n"; + let second_secret = b"MCP_ENV_SECRET_BETA_92b16a03\n"; + let optional_secret = b"MCP_ENV_SECRET_OPTIONAL_3468ef22\n"; + std::fs::write( + &env_file, + [b"MANAGED_ENV_FILE=".as_slice(), first_secret].concat(), + ) + .unwrap(); + std::fs::write( + &second_env_file, + [b"MANAGED_ENV_FILE_LOCAL=".as_slice(), second_secret].concat(), + ) + .unwrap(); + std::fs::write( + &optional_env_file, + [b"MANAGED_ENV_FILE_OPTIONAL=".as_slice(), optional_secret].concat(), + ) + .unwrap(); let option_arguments = vec![ "node".to_owned(), @@ -1437,6 +1498,8 @@ process.stdout.write(require('./helper.js')) "--env-file".to_owned(), second_env_file.to_string_lossy().into_owned(), format!("--env-file-if-exists={}", optional_env_file.display()), + "--env-file-if-exists".to_owned(), + missing_optional_env_file.to_string_lossy().into_owned(), command.to_string_lossy().into_owned(), ]; let (plan, _) = plan_and_stage_mcp_command_for_resource( @@ -1452,8 +1515,6 @@ process.stdout.write(require('./helper.js')) &imported, &loader, &experimental_loader, - &env_file, - &second_env_file, ] { assert!( plan[0] @@ -1466,51 +1527,142 @@ process.stdout.write(require('./helper.js')) assert!(!plan[0] .configured_resources .contains(&home.canonicalize().unwrap())); + for secret_file in [&env_file, &second_env_file, &optional_env_file] { + let canonical = secret_file.canonicalize().unwrap(); + assert!( + !plan[0].configured_resources.contains(&canonical), + "Node env file {} must use the managed secret path, not ordinary resources", + secret_file.display() + ); + assert!(plan[0].configured_resource_exclusions.contains(&canonical)); + } assert!(!plan[0].configured_resources.contains( &optional_env_file .canonicalize() .unwrap_or(optional_env_file.clone()) )); - let provider = copy_selected_units_through_provider( + assert_eq!(plan[0].references.selections.len(), 3); + let prepared_config: serde_json::Value = + serde_json::from_str(plan[0].references.template.as_str()).unwrap(); + let prepared_arguments = prepared_config["mcp"]["direct"]["command"] + .as_array() + .unwrap() + .iter() + .map(|argument| argument.as_str().unwrap().to_owned()) + .collect::<Vec<_>>(); + assert_eq!( + prepared_arguments[7], + "--env-file=/home/freshell/provider/.freshell/references/0" + ); + assert_eq!(prepared_arguments[8], "--env-file"); + assert_eq!( + prepared_arguments[9], + "/home/freshell/provider/.freshell/references/1" + ); + assert_eq!( + prepared_arguments[10], + "--env-file-if-exists=/home/freshell/provider/.freshell/references/2" + ); + assert_eq!(prepared_arguments[11], "--env-file-if-exists"); + assert_eq!( + prepared_arguments[12], + missing_optional_env_file.display().to_string(), + "an absent separated optional file keeps Node's native optional-file behavior" + ); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + assert!(units.iter().all(|(source, mount)| { + ![&env_file, &second_env_file, &optional_env_file] + .iter() + .any(|secret_file| { + let secret = secret_file.canonicalize().unwrap(); + source == &secret + || (mount.kind + == freshell_runtime_protocol::ProviderResourceKind::Directory + && secret.starts_with(source)) + }) + })); + let provider = copy_selected_units_through_provider_excluding( owned.path(), - &resource_units( - &home, - &home.join(".config/opencode"), - &workspace, - &plan, - None, - ) - .unwrap(), + &units, + &plan[0] + .configured_resource_exclusions + .iter() + .cloned() + .collect::<Vec<_>>(), ); - let staged = |path: &Path| { - provider - .join(".provider-home") - .join(path.strip_prefix(&home).unwrap()) + for root in ["stage", "host", "provider"] { + assert!(!tree_contains_bytes( + &owned.path().join(root), + first_secret.strip_suffix(b"\n").unwrap() + )); + assert!(!tree_contains_bytes( + &owned.path().join(root), + second_secret.strip_suffix(b"\n").unwrap() + )); + assert!(!tree_contains_bytes( + &owned.path().join(root), + optional_secret.strip_suffix(b"\n").unwrap() + )); + } + let secret_root = provider.join(".provider-home/.freshell/references"); + std::fs::create_dir_all(&secret_root).unwrap(); + for selection in &plan[0].references.selections { + let id = selection.runtime_destination.rsplit('/').next().unwrap(); + std::fs::copy(&selection.canonical_source, secret_root.join(id)).unwrap(); + } + let provider_argument = |argument: &str| { + if let Some((option, value)) = argument.split_once('=') { + let path = Path::new(value); + if let Some(relative) = value.strip_prefix("/home/freshell/provider/") { + return format!( + "{option}={}", + provider.join(".provider-home").join(relative).display() + ); + } + if let Ok(relative) = path.strip_prefix(&home) { + return format!( + "{option}={}", + provider.join(".provider-home").join(relative).display() + ); + } + } + if let Some(relative) = argument.strip_prefix("/home/freshell/provider/") { + return provider + .join(".provider-home") + .join(relative) + .to_string_lossy() + .into_owned(); + } + let path = Path::new(argument); + if let Ok(relative) = path.strip_prefix(&home) { + return provider + .join(".provider-home") + .join(relative) + .to_string_lossy() + .into_owned(); + } + argument.to_owned() }; - let arguments = vec![ - format!("--require={}", staged(&preload).display()), - "--import".to_owned(), - staged(&imported).to_string_lossy().into_owned(), - format!("--loader={}", staged(&loader).display()), - "--experimental-loader".to_owned(), - staged(&experimental_loader).to_string_lossy().into_owned(), - format!("--env-file={}", staged(&env_file).display()), - "--env-file".to_owned(), - staged(&second_env_file).to_string_lossy().into_owned(), - format!( - "--env-file-if-exists={}", - staged(&optional_env_file).display() - ), - staged(&command).to_string_lossy().into_owned(), - ]; + let arguments = prepared_arguments[1..] + .iter() + .map(|argument| provider_argument(argument)) + .collect::<Vec<_>>(); let output = execute_node_command_with_home_hidden(owned.path(), &home, &arguments); assert!(output.status.success(), "{:?}", output.stderr); assert_eq!(output.stdout, b"command options staged\n"); } #[test] - fn direct_node_command_options_inside_mcp_cwd_use_the_directory_copy() { + fn direct_node_env_file_inside_selected_mcp_cwd_uses_a_secret_mount() { let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let workspace = owned.path().join("workspace"); @@ -1522,11 +1674,12 @@ process.stdout.write(require('./helper.js')) std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "if (globalThis.cwdPreload !== 'loaded' || process.env.MANAGED_CWD_ENV !== 'loaded') process.exit(1);\nprocess.stdout.write('cwd options loaded\\n');\n", + "if (globalThis.cwdPreload !== 'loaded' || !process.env.MANAGED_CWD_ENV?.startsWith('MCP_ENV_SECRET_')) process.exit(1);\nprocess.stdout.write('cwd options loaded\\n');\n", ) .unwrap(); std::fs::write(&preload, "globalThis.cwdPreload = 'loaded';\n").unwrap(); - std::fs::write(&env_file, "MANAGED_CWD_ENV=loaded\n").unwrap(); + let secret = b"MCP_ENV_SECRET_CWD_20ac671d\n"; + std::fs::write(&env_file, [b"MANAGED_CWD_ENV=".as_slice(), secret].concat()).unwrap(); let command_arguments = vec![ "node".to_owned(), "--require=./preload.cjs".to_owned(), @@ -1553,6 +1706,17 @@ process.stdout.write(require('./helper.js')) covered.display() ); } + let canonical_env_file = env_file.canonicalize().unwrap(); + assert!(plan[0] + .configured_resource_exclusions + .contains(&canonical_env_file)); + assert_eq!(plan[0].references.selections.len(), 1); + let prepared_config: serde_json::Value = + serde_json::from_str(plan[0].references.template.as_str()).unwrap(); + assert_eq!( + prepared_config["mcp"]["direct"]["command"][2], + "--env-file=/home/freshell/provider/.freshell/references/0" + ); let units = resource_units( &home, @@ -1562,6 +1726,53 @@ process.stdout.write(require('./helper.js')) None, ) .unwrap(); + let unfiltered = tempfile::tempdir().unwrap(); + let unfiltered_stage = unfiltered.path().join("stage"); + copy_environment_for_mounts( + &[CopySelection { + source: canonical_cwd.clone(), + destination_relative: "tools".into(), + }], + &unfiltered_stage, + &[], + Default::default(), + &[], + ) + .unwrap(); + assert!(tree_contains_bytes( + &unfiltered_stage, + secret.strip_suffix(b"\n").unwrap() + )); + let unfiltered_host = unfiltered.path().join("host"); + copy_environment( + &[CopySelection { + source: unfiltered_stage.join("tools"), + destination_relative: "tools".into(), + }], + &unfiltered_host, + &[], + Default::default(), + ) + .unwrap(); + assert!(tree_contains_bytes( + &unfiltered_host, + secret.strip_suffix(b"\n").unwrap() + )); + let unfiltered_provider = unfiltered.path().join("provider"); + copy_environment( + &[CopySelection { + source: unfiltered_host.join("tools"), + destination_relative: "tools".into(), + }], + &unfiltered_provider, + &[], + Default::default(), + ) + .unwrap(); + assert!(tree_contains_bytes( + &unfiltered_provider, + secret.strip_suffix(b"\n").unwrap() + )); let cwd_mount = units .iter() .find(|(source, _)| source == &canonical_cwd) @@ -1576,14 +1787,57 @@ process.stdout.write(require('./helper.js')) .map(|relative| Path::new(".provider-home").join(relative)) }) .expect("test MCP cwd destination must map into the provider root"); - let provider = copy_selected_units_through_provider(owned.path(), &units); + let provider = copy_selected_units_through_provider_excluding( + owned.path(), + &units, + &plan[0] + .configured_resource_exclusions + .iter() + .cloned() + .collect::<Vec<_>>(), + ); let provider_cwd = provider.join(provider_relative); assert!(provider_cwd.is_dir(), "{}", provider_cwd.display()); + let staged_cwd = owned + .path() + .join("stage") + .join(&cwd_mount.stage_relative_path); + assert!( + !staged_cwd.join(".env").exists(), + "the stage copy retained the selected env file: exclusions={:?}, units={:?}, stage={}", + plan[0].configured_resource_exclusions, + units, + staged_cwd.display() + ); + let host_cwd = owned + .path() + .join("host") + .join(&cwd_mount.provider_relative_path); + assert!( + !host_cwd.join(".env").exists(), + "the host copy retained the selected env file: {}", + host_cwd.display() + ); + assert!( + !provider_cwd.join(".env").exists(), + "the selected cwd copy must omit its env file" + ); + for root in ["stage", "host", "provider"] { + assert!(!tree_contains_bytes( + &owned.path().join(root), + secret.strip_suffix(b"\n").unwrap() + )); + } + let secret_target = provider.join(".provider-home/.freshell/references/0"); + std::fs::create_dir_all(secret_target.parent().unwrap()).unwrap(); + std::fs::copy(&canonical_env_file, &secret_target).unwrap(); let hidden_home = owned.path().join("home-hidden"); std::fs::rename(&home, &hidden_home).unwrap(); let output = std::process::Command::new("node") - .args(["--require=./preload.cjs", "--env-file=.env", "server.js"]) + .arg("--require=./preload.cjs") + .arg(format!("--env-file={}", secret_target.display())) + .arg("server.js") .current_dir(&provider_cwd) .env_clear() .env("PATH", std::env::var_os("PATH").unwrap_or_default()) diff --git a/crates/freshell-server/src/managed_provider_references.rs b/crates/freshell-server/src/managed_provider_references.rs index 0605578ab..de3cb0690 100644 --- a/crates/freshell-server/src/managed_provider_references.rs +++ b/crates/freshell-server/src/managed_provider_references.rs @@ -153,6 +153,7 @@ pub(super) fn resolve_file_references_from_view( relocated_files: &std::collections::BTreeMap::new(), original_form: false, available_view_bytes: (64 * 1024 * 1024usize).saturating_sub(raw.len()), + additional_rewrites: &[], }, ) } @@ -160,6 +161,44 @@ pub(super) struct ReferenceProjection<'a> { pub relocated_files: &'a std::collections::BTreeMap<PathBuf, PathBuf>, pub original_form: bool, pub available_view_bytes: usize, + pub additional_rewrites: &'a [(std::ops::Range<usize>, String)], +} + +fn apply_source_edits( + raw: &str, + edits: &[(std::ops::Range<usize>, String)], +) -> Result<String, &'static str> { + let mut edits = edits.to_vec(); + edits.sort_by_key(|(span, _)| span.start); + let mut previous_end = 0; + let mut output_bytes = raw.len(); + for (span, replacement) in &edits { + if span.start < previous_end + || span.start > span.end + || span.end > raw.len() + || !raw.is_char_boundary(span.start) + || !raw.is_char_boundary(span.end) + { + return Err("selected configuration edits overlap or exceed their source"); + } + output_bytes = output_bytes + .checked_sub(span.len()) + .and_then(|bytes| bytes.checked_add(replacement.len())) + .ok_or("source-view size overflow")?; + previous_end = span.end; + } + if output_bytes > 16 * 1024 * 1024 { + return Err("source view exceeds 16 MiB"); + } + let mut output = String::with_capacity(output_bytes); + let mut cursor = 0; + for (span, replacement) in edits { + output.push_str(&raw[cursor..span.start]); + output.push_str(&replacement); + cursor = span.end; + } + output.push_str(&raw[cursor..]); + Ok(output) } pub(super) fn resolve_file_references_from_view_relocated( raw: &str, @@ -400,8 +439,11 @@ pub(super) fn resolve_file_references_from_view_relocated( "generated configuration name collides with an original child input", )); } + let recipe_source = apply_source_edits(raw, projection.additional_rewrites) + .map_err(|cause| fail(0, Vec::new(), cause))?; check_new_views( - raw.len() + recipe_source + .len() .checked_add(generated_name.len() + 6) .ok_or_else(|| fail(0, Vec::new(), "source-view size overflow"))?, )?; @@ -409,7 +451,7 @@ pub(super) fn resolve_file_references_from_view_relocated( format!("{{env:{generated_name}}}"), vec![GeneratedFileRecipe { generated_name, - unexpanded_source: raw.into(), + unexpanded_source: recipe_source, declaring_config: origin.declaring_config.into(), cwd: origin.cwd.into(), original_home: origin.original_home.into(), @@ -417,7 +459,7 @@ pub(super) fn resolve_file_references_from_view_relocated( }], ) } else { - if rewritten.is_empty() { + if rewritten.is_empty() && projection.additional_rewrites.is_empty() { let source = source_view.unwrap_or_else(|| std::sync::Arc::new(raw.to_owned())); return Ok(ResolvedFileReferencePlan { template: source.clone(), @@ -431,25 +473,11 @@ pub(super) fn resolve_file_references_from_view_relocated( native_alias_kinds, }); } - let mut template_bytes = raw.len(); - for (span, replacement) in &rewritten { - template_bytes = template_bytes - .checked_sub(span.len()) - .and_then(|bytes| bytes.checked_add(replacement.len())) - .ok_or_else(|| fail(0, Vec::new(), "source-view size overflow"))?; - } - if template_bytes > 16 * 1024 * 1024 { - return Err(fail(0, Vec::new(), "source view exceeds 16 MiB")); - } + rewritten.extend(projection.additional_rewrites.iter().cloned()); + let template = + apply_source_edits(raw, &rewritten).map_err(|cause| fail(0, Vec::new(), cause))?; + let template_bytes = template.len(); check_new_views(template_bytes)?; - let mut template = String::new(); - let mut cursor = 0; - for (span, replacement) in rewritten { - template.push_str(&raw[cursor..span.start]); - template.push_str(&replacement); - cursor = span.end; - } - template.push_str(&raw[cursor..]); (template, Vec::new()) }; let unexpanded_direct_source = (!generated || projection.original_form) diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 72c2c6a45..8c15ec7c1 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -1,6 +1,5 @@ //! Finite native ordinary resource fields. Keep raw documents and every //! unrelated string intact; these selections are paths, never private payloads. -use super::sources::PlannedSourceOccurrence; use std::io::Read; use std::path::{Path, PathBuf}; @@ -68,7 +67,7 @@ impl Cursor<'_> { // A finite field visitor rather than decoded/serialized document rewriting. // Plugin tuple options (and nested unrelated fields) are never path schemas. -fn fields(raw: &str) -> Result<Vec<(&'static str, String)>, &'static str> { +fn fields(raw: &str) -> Result<Vec<SelectedField>, &'static str> { // Native JSONC parsing assigns each member in order: only the final // spelling owns a configured capability, including escaped duplicate keys. // Reuse raw member spans so plugin tuple options and source origins survive. @@ -101,6 +100,27 @@ fn fields(raw: &str) -> Result<Vec<(&'static str, String)>, &'static str> { // an entire source document, nor interpret unrelated command arguments as paths. type ObjectMember = (std::ops::Range<usize>, std::ops::Range<usize>); +struct SelectedField { + field: &'static str, + spelling: String, + node_env_file_argument: Option<NodeEnvFileArgument>, +} + +struct NodeEnvFileArgument { + raw_span: std::ops::Range<usize>, + option_prefix: String, +} + +impl SelectedField { + fn new(field: &'static str, spelling: String) -> Self { + Self { + field, + spelling, + node_env_file_argument: None, + } + } +} + fn object_members(raw: &str, start: usize) -> Result<Vec<ObjectMember>, &'static str> { let mut cursor = Cursor { raw, at: start }; if !matches!(cursor.next(), Some(Token::Punctuation(b'{'))) { @@ -146,19 +166,22 @@ fn member<'a>( members: &[(std::ops::Range<usize>, std::ops::Range<usize>)], name: &str, ) -> Option<&'a str> { - members.iter().rev().find_map(|(key, value)| { + member_range(raw, members, name).map(|(_, value)| &raw[value.clone()]) +} + +fn member_range<'a>( + raw: &'a str, + members: &'a [ObjectMember], + name: &str, +) -> Option<&'a ObjectMember> { + members.iter().rev().find(|(key, _)| { // Every ASCII name character can use a six-byte Unicode escape. // Bound the finite name before decoding, without excluding valid keys. - if key.len() <= 2 + 6 * name.len() + key.len() <= 2 + 6 * name.len() && serde_json::from_str::<String>(&raw[key.clone()]) .ok() .as_deref() == Some(name) - { - Some(&raw[value.clone()]) - } else { - None - } }) } fn selected_string(raw: &str) -> Result<Option<String>, &'static str> { @@ -176,7 +199,7 @@ fn nested_fields( raw: &str, tui: bool, skip_owned_freshell_mcp: bool, -) -> Result<Vec<(&'static str, String)>, &'static str> { +) -> Result<Vec<SelectedField>, &'static str> { let top = object_members(raw, 0)?; let mut result = Vec::new(); if tui { @@ -197,14 +220,16 @@ fn nested_fields( if let Some(value) = member(sounds, &entries, name) { if let Some(path) = selected_string(value)?.filter(|path| !path.is_empty()) { - result.push(("OpenCode attention.sounds", path)); + result.push(SelectedField::new("OpenCode attention.sounds", path)); } } } } } } - if let Some(mcp) = member(raw, &top, "mcp") { + if let Some((_, mcp_range)) = member_range(raw, &top, "mcp") { + let mcp_base = mcp_range.start; + let mcp = &raw[mcp_range.clone()]; // Native Object.entries sees only the last value for each decoded // server name. Discard overwritten entries before selecting their cwd. let mut names = std::collections::BTreeSet::new(); @@ -221,6 +246,7 @@ fn nested_fields( } drop(names); for (name, range) in winners.into_iter().rev() { + let entry_base = mcp_base + range.start; let entry = &mcp[range]; let entries = object_members(entry, 0)?; if member(entry, &entries, "type") @@ -233,14 +259,19 @@ fn nested_fields( } // Keep each server's cwd adjacent to its command references so a // different server's cwd cannot accidentally satisfy a missing path. - result.push(("OpenCode mcp.entry", String::new())); + result.push(SelectedField::new("OpenCode mcp.entry", String::new())); if let Some(value) = member(entry, &entries, "cwd") { if let Some(path) = selected_string(value)? { - result.push(("OpenCode mcp.cwd", path)); + result.push(SelectedField::new("OpenCode mcp.cwd", path)); } } - if let Some(value) = member(entry, &entries, "command") { - let command = string_array(value)?; + if let Some((_, command_range)) = member_range(entry, &entries, "command") { + let command_raw = &entry[command_range.clone()]; + let command_spans = string_array_with_spans(command_raw)?; + let command = command_spans + .iter() + .map(|(argument, _)| argument.clone()) + .collect::<Vec<_>>(); // Freshell's exact server command is supplied by the managed // runtime image. Its /opt path is not a host resource to copy. let image_command = name == "freshell" @@ -289,23 +320,45 @@ fn nested_fields( } else { "OpenCode mcp.direct-script" }; - result.push((field, argument.clone())); + result.push(SelectedField::new(field, argument.clone())); } else { - result.push(("OpenCode mcp.command", argument.clone())); + result.push(SelectedField::new( + "OpenCode mcp.command", + argument.clone(), + )); } } else if script_file || script_operand { - result.push((node_script_field, argument.clone())); + result.push(SelectedField::new(node_script_field, argument.clone())); } } - for (_, field, spelling) in option_resources { - result.push((field, spelling)); + for (index, field, spelling) in option_resources { + let mut selected = SelectedField::new(field, spelling.clone()); + if is_mcp_node_option_data_field(field) { + let (argument, span) = command_spans + .get(index) + .ok_or("selected Node env-file argument span unavailable")?; + let option_prefix = argument + .strip_suffix(&spelling) + .ok_or("selected Node env-file option changed during parsing")? + .to_owned(); + selected.node_env_file_argument = Some(NodeEnvFileArgument { + raw_span: command_range.start + entry_base + span.start + ..command_range.start + entry_base + span.end, + option_prefix, + }); + } + result.push(selected); } if let Some(script_index) = script_index { let script = &command[script_index]; if is_bun_run_bare_operand(&command, script_index) { - result.push(("OpenCode bun run operand", script.clone())); + result.push(SelectedField::new( + "OpenCode bun run operand", + script.clone(), + )); } else if is_bun_run_file_operand(&command, script_index) { - result.push(("OpenCode bun run file", script.clone())); + result + .push(SelectedField::new("OpenCode bun run file", script.clone())); } } } @@ -315,7 +368,9 @@ fn nested_fields( Ok(result) } -fn string_array(raw: &str) -> Result<Vec<String>, &'static str> { +fn string_array_with_spans( + raw: &str, +) -> Result<Vec<(String, std::ops::Range<usize>)>, &'static str> { let mut cursor = Cursor { raw, at: 0 }; if !matches!(cursor.next(), Some(Token::Punctuation(b'['))) { return Ok(Vec::new()); @@ -335,10 +390,11 @@ fn string_array(raw: &str) -> Result<Vec<String>, &'static str> { if result.len() >= 100_000 || end - start > 64 * 1024 { return Err("configured resource selection profile exceeded"); } - result.push( + result.push(( serde_json::from_str(&cursor.raw[start..end]) .map_err(|_| "invalid selected command argument")?, - ); + start..end, + )); } _ => {} } @@ -1278,10 +1334,29 @@ fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static for (index, token) in tokens.iter().enumerate() { let module = match token { JavaScriptToken::Identifier(name) if name == "require" || name == "import" => { - match tokens.get(index + 1) { - Some(JavaScriptToken::Punctuation(b'(')) => tokens.get(index + 2), - Some(JavaScriptToken::String(_)) if name == "import" => tokens.get(index + 1), - _ => None, + let require_resolve = name == "require" + && matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'.')) + ) + && matches!( + tokens.get(index + 2), + Some(JavaScriptToken::Identifier(method)) if method == "resolve" + ) + && matches!( + tokens.get(index + 3), + Some(JavaScriptToken::Punctuation(b'(')) + ); + if require_resolve { + tokens.get(index + 4) + } else { + match tokens.get(index + 1) { + Some(JavaScriptToken::Punctuation(b'(')) => tokens.get(index + 2), + Some(JavaScriptToken::String(_)) if name == "import" => { + tokens.get(index + 1) + } + _ => None, + } } } JavaScriptToken::Identifier(name) if name == "from" => tokens.get(index + 1), @@ -3006,7 +3081,7 @@ fn node_command_option_resources(command: &[String]) -> Vec<(usize, &'static str fn resource_array( cursor: &mut Cursor<'_>, field: &'static str, - result: &mut Vec<(&'static str, String)>, + result: &mut Vec<SelectedField>, ) -> Result<(), &'static str> { let mut depth = 1usize; let mut tuple_first = false; @@ -3031,7 +3106,7 @@ fn resource_array( if result.len() >= 100_000 || end - start > 64 * 1024 { return Err("configured resource selection profile exceeded"); } - result.push(( + result.push(SelectedField::new( field, serde_json::from_str(&cursor.raw[start..end]) .map_err(|_| "invalid selected resource path")?, @@ -3056,10 +3131,18 @@ pub struct ResourceError { pub cause: &'static str, } -pub struct ResourceSelection { +pub(super) struct ResourceSelection { pub resources: Vec<PathBuf>, pub variables: Vec<String>, pub writable_resources: std::collections::BTreeSet<PathBuf>, + pub node_env_files: Vec<NodeEnvFileSelection>, +} + +pub(super) struct NodeEnvFileSelection { + pub canonical_source: PathBuf, + pub selected_alias: PathBuf, + pub argument_span: std::ops::Range<usize>, + pub option_prefix: String, } fn checked(path: &Path, field: &'static str, _source: &Path) -> Result<PathBuf, ResourceError> { let fail = |cause| ResourceError { field, cause }; @@ -3199,34 +3282,41 @@ fn insert_installed_package_ancestor_resources_covered( /// Return complete ordinary units and native installed ancestor lookup units. /// Exact mounts retain native field spellings and their declaring origins. -pub fn select( - source: &PlannedSourceOccurrence, +pub(super) fn select( + source: &super::sources::SourceOccurrence, + raw: &str, + instance_cwd: &Path, + user_owns_freshell_mcp: bool, home: &Path, global: Option<&Path>, lookup: &dyn Fn(&str) -> Option<String>, ) -> Result<ResourceSelection, ResourceError> { - let declaring = &source.source.logical_path; + let declaring = &source.logical_path; let fail = |field: &'static str, cause: &'static str| ResourceError { field, cause }; let mut result = std::collections::BTreeSet::new(); let mut resource_path_bytes = 0usize; let mut writable_resources = std::collections::BTreeSet::new(); + let mut node_env_files = Vec::new(); let mut variables = std::collections::BTreeSet::new(); - let raw = source.references.unexpanded_source(); let skip_owned_freshell_mcp = - source.source.lane == super::sources::SourceLane::Main && !source.user_owns_freshell_mcp; + source.lane == super::sources::SourceLane::Main && !user_owns_freshell_mcp; let mut selections = fields(raw).map_err(|cause| fail("OpenCode resource", cause))?; selections.extend( nested_fields( raw, - source.source.lane == super::sources::SourceLane::Tui, + source.lane == super::sources::SourceLane::Tui, skip_owned_freshell_mcp, ) .map_err(|cause| fail("OpenCode resource", cause))?, ); let mut current_mcp_working_dir: Option<PathBuf> = None; let mut current_mcp_resource_path: Option<PathBuf> = None; - for (field, spelling) in selections { - let mut field = field; + for selected in selections { + let SelectedField { + mut field, + spelling, + node_env_file_argument, + } = selected; if field == "OpenCode mcp.entry" { current_mcp_working_dir = None; current_mcp_resource_path = None; @@ -3268,9 +3358,7 @@ pub fn select( continue; } if field == "OpenCode bun run operand" { - let working_dir = current_mcp_resource_path - .as_deref() - .unwrap_or(&source.instance_cwd); + let working_dir = current_mcp_resource_path.as_deref().unwrap_or(instance_cwd); if is_bare_bun_script_name(&spelling) { if let Some(package_root) = bun_package_script_root(working_dir, &spelling) .map_err(|cause| fail("OpenCode mcp.command", cause))? @@ -3383,8 +3471,7 @@ pub fn select( let candidates = if path.is_absolute() { vec![path] } else { - source - .instance_cwd + instance_cwd .ancestors() .map(|base| base.join(&path)) .collect() @@ -3427,30 +3514,25 @@ pub fn select( } if !path.is_absolute() { if field == "OpenCode plugin" || field == "OpenCode attention.sounds" { - path = declaring - .parent() - .unwrap_or(&source.instance_cwd) - .join(path); + path = declaring.parent().unwrap_or(instance_cwd).join(path); } else if field == "OpenCode skills.paths" || field == "OpenCode mcp.cwd" { - path = source.instance_cwd.join(path); + path = instance_cwd.join(path); } else if is_mcp_command_resource_field(field) { path = if let Some(working_dir) = ¤t_mcp_working_dir { working_dir.join(path) } else { - source - .instance_cwd + instance_cwd .ancestors() .map(|base| base.join(&path)) .find(|candidate| std::fs::symlink_metadata(candidate).is_ok()) - .unwrap_or_else(|| source.instance_cwd.join(path)) + .unwrap_or_else(|| instance_cwd.join(path)) }; } else { - path = source - .instance_cwd + path = instance_cwd .ancestors() .map(|base| base.join(&path)) .find(|candidate| std::fs::symlink_metadata(candidate).is_ok()) - .unwrap_or_else(|| source.instance_cwd.join(path)); + .unwrap_or_else(|| instance_cwd.join(path)); } } if matches!( @@ -3497,29 +3579,15 @@ pub fn select( )); } if is_mcp_node_option_data_field(field) { - let canonical_is_covered = current_mcp_working_dir - .as_deref() - .filter(|working_dir| canonical.starts_with(working_dir)); - let path_is_covered = - canonical_is_covered.is_some_and(|working_dir| path.starts_with(working_dir)); - if !path_is_covered { - insert_resource( - &mut result, - &mut resource_path_bytes, - path.clone(), - field, - declaring, - )?; - } - if path != canonical && canonical_is_covered.is_none() { - insert_resource( - &mut result, - &mut resource_path_bytes, - canonical, - field, - declaring, - )?; - } + let argument = node_env_file_argument.ok_or_else(|| { + fail(field, "selected Node env-file argument span unavailable") + })?; + node_env_files.push(NodeEnvFileSelection { + canonical_source: canonical, + selected_alias: path, + argument_span: argument.raw_span, + option_prefix: argument.option_prefix, + }); continue; } @@ -3657,7 +3725,7 @@ pub fn select( ScriptDependencyRuntime::Node, )?; if let Some(resource_scope) = - bun_run_file_resource_scope(command_parent, home, &source.instance_cwd) + bun_run_file_resource_scope(command_parent, home, instance_cwd) { // Bun scripts can compute local asset paths at runtime, so // preserve their bounded package (or containing) directory. @@ -3984,5 +4052,6 @@ pub fn select( resources: result.into_iter().collect(), variables: variables.into_iter().collect(), writable_resources, + node_env_files, }) } diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index a8b87a724..e9bd21b98 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -309,10 +309,11 @@ pub struct PlannedSourceOccurrence { pub references: super::ResolvedFileReferencePlan, pub original_form: bool, pub requires_config_bind: bool, - pub instance_cwd: PathBuf, pub configured_resources: Vec<PathBuf>, /// Canonical exact files with statically identified Node write targets. pub configured_writable_resources: std::collections::BTreeSet<PathBuf>, + /// Selected Node env-file aliases excluded from ordinary resource copies. + pub configured_resource_exclusions: std::collections::BTreeSet<PathBuf>, /// Captured with the retained source view so later staging does not /// reinterpret a generated entry after ordinary-terminal cleanup. pub user_owns_freshell_mcp: bool, @@ -637,6 +638,32 @@ pub fn plan_loaded_opencode_sources_relocated( selected.insert(canonical.to_owned(), destination.clone()); Ok(destination) }; + let selected_resources = super::opencode_resources::select( + &source, + &loaded.view, + cwd, + user_owns_freshell_mcp, + home, + global.as_deref(), + &|name| lookup(source.lane, &source.logical_path, name), + ) + .map_err(|error| super::ReferenceError { + preparation_failure: None, + operation: "select", + field: error.field, + source_ordinal: ordinal, + declaring_source: source.logical_path.clone(), + variables: vec![], + cause: error.cause, + })?; + let mut node_env_file_edits = Vec::with_capacity(selected_resources.node_env_files.len()); + for env_file in &selected_resources.node_env_files { + let destination = allocate(&env_file.canonical_source).map_err(fail)?; + let replacement = format!("{}{destination}", env_file.option_prefix); + let replacement = serde_json::to_string(&replacement) + .map_err(|_| fail("selected Node env-file path could not be encoded"))?; + node_env_file_edits.push((env_file.argument_span.clone(), replacement)); + } let references = super::resolve_file_references_from_view_relocated( &loaded.view, Some(loaded.view.clone()), @@ -653,8 +680,26 @@ pub fn plan_loaded_opencode_sources_relocated( original_form, available_view_bytes: MAX_SIMULTANEOUS_SOURCE_VIEW_BYTES as usize - view_budget.bytes, + additional_rewrites: &node_env_file_edits, }, )?; + let mut references = references; + for env_file in &selected_resources.node_env_files { + let runtime_destination = selected + .get(&env_file.canonical_source) + .cloned() + .ok_or_else(|| fail("selected Node env-file reference was not allocated"))?; + if !references + .selections + .iter() + .any(|selection| selection.canonical_source == env_file.canonical_source) + { + references.selections.push(super::FileReferenceSelection { + canonical_source: env_file.canonical_source.clone(), + runtime_destination, + }); + } + } if original_form { if let Some(prior) = result .iter() @@ -710,31 +755,26 @@ pub fn plan_loaded_opencode_sources_relocated( "combined source recipe, alias, variable or event limit exceeded", )); } + let configured_resource_exclusions = selected_resources + .node_env_files + .iter() + .flat_map(|env_file| { + [ + env_file.selected_alias.clone(), + env_file.canonical_source.clone(), + ] + }) + .collect(); let mut planned = PlannedSourceOccurrence { source, references, original_form, requires_config_bind: loaded.requires_config_bind, - instance_cwd: cwd.into(), - configured_resources: Vec::new(), - configured_writable_resources: Default::default(), + configured_resources: selected_resources.resources, + configured_writable_resources: selected_resources.writable_resources, + configured_resource_exclusions, user_owns_freshell_mcp, }; - let selected_resources = - super::opencode_resources::select(&planned, home, global.as_deref(), &|name| { - lookup(planned.source.lane, &planned.source.logical_path, name) - }) - .map_err(|error| super::ReferenceError { - preparation_failure: None, - operation: "select", - field: error.field, - source_ordinal: ordinal, - declaring_source: planned.source.logical_path.clone(), - variables: vec![], - cause: error.cause, - })?; - planned.configured_resources = selected_resources.resources; - planned.configured_writable_resources = selected_resources.writable_resources; planned .references .path_variables @@ -751,6 +791,64 @@ mod tests { use super::*; use sha2::{Digest, Sha256}; + #[test] + fn managed_provider_env_file_rewrite_survives_generated_config_recipe() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let env_file = home.join("mcp.env"); + let script = workspace.join("server.js"); + let payload = workspace.join("rules"); + std::fs::write(&env_file, "SYNTHETIC_MCP_VALUE=fixture\n").unwrap(); + std::fs::write(&script, "process.exit(0);\n").unwrap(); + std::fs::write(&payload, "fixture prompt payload\n").unwrap(); + let config = workspace.join("opencode.json"); + let raw = serde_json::json!({ + "agent": { "fixture": { "prompt": "{env:INSERT}" } }, + "mcp": { + "direct": { + "type": "local", + "command": ["node", format!("--env-file={}", env_file.display()), script], + } + } + }) + .to_string(); + std::fs::write(&config, &raw).unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, name| (name == "INSERT").then(|| "prefix {file:./rules} suffix".into()), + ) + .unwrap(); + let planned = &plan[0]; + assert_eq!(planned.references.recipes.len(), 1); + let env_selection = planned + .references + .selections + .iter() + .find(|selection| selection.canonical_source == env_file.canonicalize().unwrap()) + .expect("Node env files use an approved managed secret reference"); + assert!(!planned + .configured_resources + .contains(&env_file.canonicalize().unwrap())); + let recipe: serde_json::Value = + serde_json::from_str(&planned.references.recipes[0].unexpanded_source).unwrap(); + assert_eq!( + recipe["mcp"]["direct"]["command"][1], + format!("--env-file={}", env_selection.runtime_destination) + ); + assert!(planned.configured_resource_exclusions.contains(&env_file)); + } + #[test] fn managed_provider_references_actual_source_file_growth_rejects_before_reading() { let owned = tempfile::tempdir().unwrap(); From b193f42a8339f07c11151fed9c42f7c8253e9b8b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 11:11:32 -0700 Subject: [PATCH 436/493] fix(managed): protect shell MCP env references --- .../src/managed_mcp_capability.rs | 3 + .../managed_mcp_capability/source_stage.rs | 505 ++++++++++++++- .../opencode_resources.rs | 608 +++++++++++++++--- .../managed_provider_references/sources.rs | 36 +- .../managed-provider-environment-target.ts | 26 +- ...aged-provider-environment-external.spec.ts | 4 +- 6 files changed, 1087 insertions(+), 95 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 29d96c60f..47e97a174 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1673,6 +1673,7 @@ fn stage_provider_root( let mut _filtered_resource_sources = None; let mut retained_resource_sources = None; let mut mounted_resource_file_roots = Vec::new(); + let mut configured_resource_edits = Vec::new(); if let Some(plan) = opencode_input.source_plan { // Indexed payloads remain exclusively in the private input volume, // even when an ordinary package scope also contains their files. @@ -1752,6 +1753,7 @@ fn stage_provider_root( )?; _filtered_resource_sources = Some(scratch); } + configured_resource_edits = source_stage::configured_resource_edits(&units, plan)?; selections.extend(source_stage::resource_copy_selections(&units)?); let descriptors = units .into_iter() @@ -2160,6 +2162,7 @@ fn stage_provider_root( bytes = report.bytes, elapsed_ms = started.elapsed().as_millis() as u64 ); + source_stage::apply_configured_resource_edits(&temporary, &configured_resource_edits)?; if let Some(resources) = amplifier_resources { if let Some(settings) = resources.rewritten_settings { let relative = if let Some(prepared) = &ordinary_resources { diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 2f73f1fbd..b9a1ef2fe 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -7,6 +7,137 @@ use freshell_platform::{ }; use std::{collections::BTreeMap, path::Path}; +pub(super) struct StagedResourceEdit { + stage_relative_path: std::path::PathBuf, + span: std::ops::Range<usize>, + expected: String, + replacement: String, +} + +pub(super) fn configured_resource_edits( + units: &[( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + plan: &[PlannedSourceOccurrence], +) -> Result<Vec<StagedResourceEdit>, String> { + let mut edits = Vec::new(); + for edit in plan + .iter() + .flat_map(|source| source.configured_resource_edits.iter()) + { + let mut mapped = false; + for (resource_source, mount) in units { + let suffix = if edit.source == *resource_source { + std::path::PathBuf::new() + } else if resource_source.is_dir() { + let Ok(suffix) = edit.source.strip_prefix(resource_source) else { + continue; + }; + suffix.to_owned() + } else { + continue; + }; + let stage_relative_path = if suffix.as_os_str().is_empty() { + std::path::PathBuf::from(&mount.stage_relative_path) + } else { + Path::new(&mount.stage_relative_path).join(suffix) + }; + if !edits.iter().any(|prior: &StagedResourceEdit| { + prior.stage_relative_path == stage_relative_path + && prior.span == edit.span + && prior.expected == edit.expected + && prior.replacement == edit.replacement + }) { + edits.push(StagedResourceEdit { + stage_relative_path, + span: edit.span.clone(), + expected: edit.expected.clone(), + replacement: edit.replacement.clone(), + }); + } + mapped = true; + } + if !mapped { + return Err( + "selected shell wrapper env-file operand has no staged resource copy to rewrite" + .into(), + ); + } + } + Ok(edits) +} + +pub(super) fn apply_configured_resource_edits( + staged_root: &Path, + edits: &[StagedResourceEdit], +) -> Result<(), String> { + let mut by_path = BTreeMap::<std::path::PathBuf, Vec<&StagedResourceEdit>>::new(); + for edit in edits { + by_path + .entry(edit.stage_relative_path.clone()) + .or_default() + .push(edit); + } + for (relative, mut file_edits) in by_path { + if relative.is_absolute() + || relative + .components() + .any(|component| !matches!(component, std::path::Component::Normal(_))) + { + return Err("selected shell wrapper stage path is unsafe".into()); + } + let target = staged_root.join(relative); + let metadata = std::fs::symlink_metadata(&target).map_err(|error| { + format!( + "selected shell wrapper was not copied into its stage at {}: {error}", + target.display() + ) + })?; + if !metadata.is_file() { + return Err("selected shell wrapper stage target is not a regular file".into()); + } + let mut content = std::fs::read_to_string(&target) + .map_err(|_| "selected shell wrapper stage target is not UTF-8")?; + file_edits.sort_by_key(|edit| std::cmp::Reverse(edit.span.start)); + for (index, edit) in file_edits.iter().enumerate() { + if edit.span.start > edit.span.end + || edit.span.end > content.len() + || !content.is_char_boundary(edit.span.start) + || !content.is_char_boundary(edit.span.end) + { + return Err( + "selected shell wrapper env-file span changed during preparation".into(), + ); + } + if content.get(edit.span.clone()) != Some(edit.expected.as_str()) { + return Err( + "selected shell wrapper env-file operand changed during preparation".into(), + ); + } + if file_edits[..index] + .iter() + .any(|prior| prior.span.start < edit.span.end && edit.span.start < prior.span.end) + { + return Err("selected shell wrapper env-file operands overlap".into()); + } + content.replace_range(edit.span.clone(), &edit.replacement); + } + let parent = target + .parent() + .ok_or("selected shell wrapper stage parent unavailable")?; + let temporary = parent.join(format!(".env-rewrite-{}", uuid::Uuid::new_v4())); + std::fs::write(&temporary, content.as_bytes()).map_err(|error| error.to_string())?; + std::fs::set_permissions(&temporary, metadata.permissions()) + .map_err(|error| error.to_string())?; + if let Err(error) = std::fs::rename(&temporary, &target) { + let _ = std::fs::remove_file(&temporary); + return Err(error.to_string()); + } + } + Ok(()) +} + #[cfg(test)] mod configured_resource_tests { use super::*; @@ -18,6 +149,14 @@ mod configured_resource_tests { fn execute_selected_resource_with_path( path: &Path, path_prefix: Option<&Path>, + ) -> std::io::Result<std::process::Output> { + execute_selected_resource_with_context(path, path_prefix, None) + } + + fn execute_selected_resource_with_context( + path: &Path, + path_prefix: Option<&Path>, + working_directory: Option<&Path>, ) -> std::io::Result<std::process::Output> { let inherited_path = std::env::var_os("PATH").unwrap_or_default(); let search_path = if let Some(prefix) = path_prefix { @@ -30,6 +169,9 @@ mod configured_resource_tests { }; let mut command = std::process::Command::new(path); command.env_clear().env("PATH", search_path); + if let Some(working_directory) = working_directory { + command.current_dir(working_directory); + } // Retain this consumer's isolated environment while retrying a brief // executable-copy race. A shebang's `/usr/bin/env` can report ETXTBSY // in stderr after the script process itself has already started. @@ -78,8 +220,21 @@ mod configured_resource_tests { freshell_runtime_protocol::ProviderResourceMount, )], excluded: &[std::path::PathBuf], + ) -> std::path::PathBuf { + copy_selected_units_through_provider_excluding_with_plan(owned_root, units, excluded, &[]) + } + + fn copy_selected_units_through_provider_excluding_with_plan( + owned_root: &Path, + units: &[( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + excluded: &[std::path::PathBuf], + plan: &[PlannedSourceOccurrence], ) -> std::path::PathBuf { let stage = owned_root.join("stage"); + let resource_edits = configured_resource_edits(units, plan).unwrap(); let stage_selections = units .iter() .map(|(source, mount)| CopySelection { @@ -109,6 +264,7 @@ mod configured_resource_tests { &mounted_file_roots, ) .unwrap(); + apply_configured_resource_edits(&stage, &resource_edits).unwrap(); let host = owned_root.join("host"); let host_selections = units @@ -306,8 +462,9 @@ mod configured_resource_tests { .collect::<std::collections::BTreeSet<_>>() .into_iter() .collect::<Vec<_>>(); - let provider = - copy_selected_units_through_provider_excluding(owned_root, &units, &excluded); + let provider = copy_selected_units_through_provider_excluding_with_plan( + owned_root, &units, &excluded, &plan, + ); (plan, provider.join(provider_relative)) } @@ -1339,6 +1496,162 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"node sibling helper loaded\n"); } + #[test] + fn direct_node_script_allows_missing_optional_require_resolve_after_all_resource_copies() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("start.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\ntry { require.resolve('./optional.js'); (() => require.resolve('./optional-immediate.js'))(); process.exit(12); } catch (error) { if (error.code !== 'MODULE_NOT_FOUND') throw error; process.stdout.write('optional fallback works\\n'); }\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let (plan, provider_command) = + plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert!(!plan[0].configured_resources.iter().any(|resource| resource + .file_name() + .is_some_and(|name| matches!( + name.to_str(), + Some("optional.js" | "optional-immediate.js") + )))); + + let output = execute_with_home_hidden(owned.path(), &home, &provider_command); + assert!( + output.status.success(), + "a missing module caught by the configured optional probe must reach its fallback: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"optional fallback works\n"); + } + + #[test] + fn shell_wrapper_inside_mcp_cwd_uses_protected_node_env_reference() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let mcp_cwd = home.join("mcp"); + let wrapper = mcp_cwd.join("bin/start"); + let server = mcp_cwd.join("server.mjs"); + let env_file = mcp_cwd.join(".env"); + std::fs::create_dir_all(wrapper.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &wrapper, + "#!/bin/sh\nexec node --env-file=.env server.mjs\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write( + &server, + "if (process.env.MANAGED_ENV_FILE !== ('MCP_ENV_SECRET_' + 'CWD_8a5e')) process.exit(11);\nprocess.stdout.write('cwd env loaded\\n');\n", + ) + .unwrap(); + let secret = b"MCP_ENV_SECRET_CWD_8a5e\n"; + std::fs::write( + &env_file, + [b"MANAGED_ENV_FILE=".as_slice(), secret].concat(), + ) + .unwrap(); + + let (plan, provider_wrapper) = plan_and_stage_mcp_command_for_resource_with_cwd( + owned.path(), + &home, + &workspace, + &[wrapper.to_string_lossy().into_owned()], + &wrapper, + Some(&mcp_cwd), + ); + let canonical_env = env_file.canonicalize().unwrap(); + assert!(plan[0] + .configured_resource_exclusions + .contains(&canonical_env)); + assert!(plan[0] + .configured_resources + .contains(&mcp_cwd.canonicalize().unwrap())); + assert_eq!(plan[0].references.selections.len(), 1); + for root in ["stage", "host", "provider"] { + assert!(!tree_contains_bytes( + &owned.path().join(root), + secret.strip_suffix(b"\n").unwrap() + )); + } + + let reference = owned + .path() + .join("provider/.provider-home/.freshell/references/0"); + std::fs::create_dir_all(reference.parent().unwrap()).unwrap(); + std::fs::copy(&canonical_env, &reference).unwrap(); + let wrapper_permissions = std::fs::metadata(&provider_wrapper).unwrap().permissions(); + let runtime_reference = "/home/freshell/provider/.freshell/references/0"; + let wrapper_source = std::fs::read_to_string(&provider_wrapper).unwrap(); + assert!(wrapper_source.contains(runtime_reference)); + std::fs::write( + &provider_wrapper, + wrapper_source.replace(runtime_reference, &reference.to_string_lossy()), + ) + .unwrap(); + std::fs::set_permissions(&provider_wrapper, wrapper_permissions).unwrap(); + + let provider_cwd = provider_wrapper.parent().unwrap().parent().unwrap(); + let output = + execute_selected_resource_with_context(&provider_wrapper, None, Some(provider_cwd)) + .unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"cwd env loaded\n"); + } + + #[test] + fn direct_node_script_keeps_deferred_require_resolve_required_inside_try() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let command = home.join("start.js"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &command, + "#!/usr/bin/env node\nlet resolveLater;\ntry { resolveLater = () => require.resolve('./required.js'); } catch {}\nresolveLater();\n", + ) + .unwrap(); + std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "direct": { "type": "local", "command": [command] } }, + }) + .to_string(), + ) + .unwrap(); + + let result = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ); + assert!( + result.is_err(), + "a deferred probe that runs outside its try block must remain a required dependency" + ); + } + #[test] fn direct_node_script_scanner_ignores_quotes_inside_regex_literals() { let owned = tempfile::tempdir().unwrap(); @@ -1848,6 +2161,155 @@ process.stdout.write(require('./helper.js')) assert_eq!(output.stdout, b"cwd options loaded\n"); } + #[test] + fn shared_node_env_file_keeps_direct_and_shell_mcp_consumers_working() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let tools = home.join("tools"); + let env_file = tools.join(".env"); + let direct_script = tools.join("direct.js"); + let wrapper_script = tools.join("wrapper-server.js"); + let wrapper = tools.join("start"); + std::fs::create_dir_all(&tools).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + let sentinel = b"MCP_ENV_SECRET_SHARED_2d8a\n"; + std::fs::write( + &env_file, + [b"MANAGED_SHARED_VALUE=".as_slice(), sentinel].concat(), + ) + .unwrap(); + for script in [&direct_script, &wrapper_script] { + std::fs::write( + script, + "if (process.env.MANAGED_SHARED_VALUE !== ('MCP_ENV_SECRET_' + 'SHARED_2d8a')) process.exit(21);\nprocess.stdout.write('shared env loaded\\n');\n", + ) + .unwrap(); + } + std::fs::write( + &wrapper, + "#!/bin/sh\ncd \"$(dirname \"$0\")\" && exec node --env-file=.env wrapper-server.js\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { + "direct": { + "type": "local", + "command": ["node", "--env-file=.env", "direct.js"], + "cwd": tools, + }, + "wrapped": { + "type": "local", + "command": [wrapper], + }, + }, + }) + .to_string(), + ) + .unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: freshell_platform::opencode_recipe::DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let canonical_env = env_file.canonicalize().unwrap(); + assert!(!plan[0].configured_resources.contains(&canonical_env)); + assert!(plan[0] + .configured_resource_exclusions + .contains(&canonical_env)); + assert_eq!(plan[0].references.selections.len(), 1); + let prepared: serde_json::Value = + serde_json::from_str(&plan[0].references.template).unwrap(); + let reference_argument = prepared["mcp"]["direct"]["command"][1].as_str().unwrap(); + assert_eq!( + reference_argument, + "--env-file=/home/freshell/provider/.freshell/references/0" + ); + + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let provider = copy_selected_units_through_provider_excluding_with_plan( + owned.path(), + &units, + &plan[0] + .configured_resource_exclusions + .iter() + .cloned() + .collect::<Vec<_>>(), + &plan, + ); + let provider_tools = provider.join(".provider-home/tools"); + let reference = provider.join(".provider-home/.freshell/references/0"); + for root in ["stage", "host", "provider"] { + assert!( + !tree_contains_bytes( + &owned.path().join(root), + sentinel.strip_suffix(b"\n").unwrap() + ), + "the shared secret must not enter ordinary {root} copies" + ); + } + std::fs::create_dir_all(reference.parent().unwrap()).unwrap(); + std::fs::copy(&canonical_env, &reference).unwrap(); + assert!(std::fs::read(&reference) + .unwrap() + .windows(sentinel.strip_suffix(b"\n").unwrap().len()) + .any(|window| window == sentinel.strip_suffix(b"\n").unwrap())); + + let provider_wrapper = provider_tools.join("start"); + let wrapper_permissions = std::fs::metadata(&provider_wrapper).unwrap().permissions(); + let runtime_reference = "/home/freshell/provider/.freshell/references/0"; + let wrapper_source = std::fs::read_to_string(&provider_wrapper).unwrap(); + assert!(wrapper_source.contains(runtime_reference)); + std::fs::write( + &provider_wrapper, + wrapper_source.replace(runtime_reference, &reference.to_string_lossy()), + ) + .unwrap(); + std::fs::set_permissions(&provider_wrapper, wrapper_permissions).unwrap(); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let direct = std::process::Command::new("node") + .arg(format!("--env-file={}", reference.display())) + .arg(direct_script.file_name().unwrap()) + .current_dir(&provider_tools) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output(); + let wrapped = execute_selected_resource(&provider_tools.join("start")); + std::fs::rename(&hidden_home, &home).unwrap(); + let direct = direct.unwrap(); + assert!(direct.status.success(), "{:?}", direct.stderr); + assert_eq!(direct.stdout, b"shared env loaded\n"); + let wrapped = wrapped.unwrap(); + assert!( + wrapped.status.success(), + "the shell wrapper must use the same protected env reference: {:?}", + wrapped.stderr + ); + assert_eq!(wrapped.stdout, b"shared env loaded\n"); + } + #[test] fn nested_direct_script_keeps_writable_sibling_mount_under_selected_directory() { use freshell_platform::opencode_recipe::{DiscoveryRole, DiscoveryRoot}; @@ -2679,7 +3141,7 @@ process.stdout.write(require('./helper.js')) std::fs::create_dir_all(&workspace).unwrap(); std::fs::write( &command, - "#!/bin/sh\ncd \"$(dirname \"$0\")\" && exec node -r ./preload.js --require=./second-preload.js --import ./preload.mjs --loader=./loader.mjs --env-file=.env server.js\n", + "#!/bin/sh\ncd \"$(dirname \"$0\")\" && exec node -r ./preload.js --require=./second-preload.js --import ./preload.mjs --loader=./loader.mjs --env-file=.env --env-file-if-exists=.env.missing server.js\n", ) .unwrap(); std::fs::write( @@ -2707,23 +3169,28 @@ process.stdout.write(require('./helper.js')) "export async function resolve(specifier, context, nextResolve) { return nextResolve(specifier, context); }\n", ) .unwrap(); - std::fs::write(&env_file, "MANAGED_ENV_FILE=loaded\n").unwrap(); + let env_secret = b"MCP_ENV_SECRET_SHELL_7c3a\n"; + std::fs::write( + &env_file, + [b"MANAGED_ENV_FILE=".as_slice(), env_secret].concat(), + ) + .unwrap(); std::fs::write( &server, - "if (globalThis.managedPreload !== 'loaded' || globalThis.managedPreloadHelper !== 'loaded' || globalThis.managedSecondPreload !== 'loaded' || globalThis.managedImportedPreload !== 'loaded' || process.env.MANAGED_ENV_FILE !== 'loaded') process.exit(1);\nprocess.stdout.write('preload staged\\n');\n", + "if (globalThis.managedPreload !== 'loaded' || globalThis.managedPreloadHelper !== 'loaded' || globalThis.managedSecondPreload !== 'loaded' || globalThis.managedImportedPreload !== 'loaded' || process.env.MANAGED_ENV_FILE !== ('MCP_ENV_SECRET_' + 'SHELL_7c3a')) process.exit(1);\nprocess.stdout.write('preload staged\\n');\n", ) .unwrap(); std::fs::set_permissions(&command, std::fs::Permissions::from_mode(0o755)).unwrap(); let (plan, provider_command) = plan_and_stage_direct_mcp_command(owned.path(), &home, &workspace, &command); + assert_eq!(plan[0].references.selections.len(), 1); for required in [ &preload, &preload_helper, &second_preload, &imported_preload, &loader, - &env_file, ] { assert!( plan[0] @@ -2736,6 +3203,32 @@ process.stdout.write(require('./helper.js')) assert!(plan[0] .configured_resources .contains(&server.canonicalize().unwrap())); + let canonical_env = env_file.canonicalize().unwrap(); + assert!(!plan[0].configured_resources.contains(&canonical_env)); + assert!(plan[0] + .configured_resource_exclusions + .contains(&canonical_env)); + for root in ["stage", "host", "provider"] { + assert!(!tree_contains_bytes( + &owned.path().join(root), + env_secret.strip_suffix(b"\n").unwrap() + )); + } + let secret_target = owned + .path() + .join("provider/.provider-home/.freshell/references/0"); + std::fs::create_dir_all(secret_target.parent().unwrap()).unwrap(); + std::fs::copy(&canonical_env, &secret_target).unwrap(); + let wrapper_permissions = std::fs::metadata(&provider_command).unwrap().permissions(); + let runtime_reference = "/home/freshell/provider/.freshell/references/0"; + let wrapper_source = std::fs::read_to_string(&provider_command).unwrap(); + assert!(wrapper_source.contains(runtime_reference)); + std::fs::write( + &provider_command, + wrapper_source.replace(runtime_reference, &secret_target.to_string_lossy()), + ) + .unwrap(); + std::fs::set_permissions(&provider_command, wrapper_permissions).unwrap(); let output = execute_with_home_hidden(owned.path(), &home, &provider_command); assert!(output.status.success(), "{:?}", output.stderr); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index 8c15ec7c1..f7d239300 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -776,18 +776,52 @@ fn shell_executable_name(token: &str) -> String { struct ShellNodeInvocation { script_operand: Option<String>, file_operands: Vec<(String, bool)>, + env_file_operands: Vec<(String, std::ops::Range<usize>, bool)>, +} + +fn shell_argument_span( + line: &str, + span: std::ops::Range<usize>, +) -> (String, std::ops::Range<usize>) { + let bytes = line.as_bytes(); + let mut start = span.start; + let mut end = span.end; + while start < end && matches!(bytes[start], b'"' | b'\'') { + start += 1; + } + while start < end && matches!(bytes[end - 1], b'"' | b'\'') { + end -= 1; + } + while start < end && matches!(bytes[end - 1], b';' | b'&') { + end -= 1; + } + while start < end && matches!(bytes[start], b'"' | b'\'') { + start += 1; + } + while start < end && matches!(bytes[end - 1], b'"' | b'\'') { + end -= 1; + } + (line[start..end].to_owned(), start..end) } fn shell_node_invocation(line: &str) -> Option<ShellNodeInvocation> { - let words = line - .split_ascii_whitespace() - .map(|word| word.trim_matches(['"', '\''])) + let mut word_spans = Vec::new(); + let mut offset = 0; + for word in line.split_ascii_whitespace() { + let start = line[offset..].find(word)? + offset; + let span = start..start + word.len(); + offset = span.end; + word_spans.push(span); + } + let words = word_spans + .iter() + .map(|span| line[span.clone()].trim_matches(['"', '\''])) .collect::<Vec<_>>(); let node_index = words .iter() .position(|word| matches!(shell_executable_name(word).as_str(), "node" | "nodejs"))?; let mut index = node_index + 1; - const NODE_OPTIONS_WITH_VALUE: [&str; 8] = [ + const NODE_OPTIONS_WITH_VALUE: [&str; 9] = [ "-r", "--require", "--import", @@ -795,6 +829,7 @@ fn shell_node_invocation(line: &str) -> Option<ShellNodeInvocation> { "--experimental-loader", "--conditions", "--env-file", + "--env-file-if-exists", "--inspect-port", ]; const NODE_SCRIPT_FILE_OPTIONS: [&str; 5] = [ @@ -805,6 +840,7 @@ fn shell_node_invocation(line: &str) -> Option<ShellNodeInvocation> { "--experimental-loader", ]; let mut file_operands = Vec::new(); + let mut env_file_operands = Vec::new(); while let Some(argument) = words.get(index) { if *argument == "--" { index += 1; @@ -814,18 +850,39 @@ fn shell_node_invocation(line: &str) -> Option<ShellNodeInvocation> { .split_once('=') .map_or((*argument, None), |(option, value)| (option, Some(value))); if NODE_OPTIONS_WITH_VALUE.contains(&option) { - let value = inline_value.or_else(|| words.get(index + 1).copied()); - if let Some(value) = value { - let value = value.trim_end_matches([';', '&']); - let is_script = NODE_SCRIPT_FILE_OPTIONS.contains(&option); - let is_path = (option == "--env-file" && !value.starts_with('-')) - || Path::new(value).is_absolute() - || value.starts_with("./") - || value.starts_with("../") - || value.contains(['$', '*', '?']); - if is_path && (is_script || option == "--env-file") { - file_operands.push((value.to_owned(), is_script)); - } + let operand_index = if inline_value.is_some() { + index + } else { + index + 1 + }; + let raw_span = word_spans.get(operand_index)?.clone(); + let operand_span = if inline_value.is_some() { + let equals = line[raw_span.clone()].find('=')? + raw_span.start + 1; + equals..raw_span.end + } else { + raw_span + }; + let (raw_value, value_span) = shell_argument_span(line, operand_span); + let value = if raw_value.is_empty() { + inline_value + .or_else(|| words.get(index + 1).copied())? + .trim_matches(['"', '\'']) + .to_owned() + } else { + raw_value + }; + let is_script = NODE_SCRIPT_FILE_OPTIONS.contains(&option); + let is_env_file = matches!(option, "--env-file" | "--env-file-if-exists"); + let is_path = (is_env_file && !value.starts_with('-')) + || Path::new(&value).is_absolute() + || value.starts_with("./") + || value.starts_with("../") + || value.contains(['$', '*', '?']); + if is_path && is_script { + file_operands.push((value.clone(), is_script)); + } + if is_env_file && is_path { + env_file_operands.push((value, value_span, option == "--env-file-if-exists")); } index += if inline_value.is_some() { 1 } else { 2 }; continue; @@ -849,6 +906,7 @@ fn shell_node_invocation(line: &str) -> Option<ShellNodeInvocation> { Some(ShellNodeInvocation { script_operand, file_operands, + env_file_operands, }) } @@ -1328,11 +1386,222 @@ fn javascript_tokens(source: &str) -> Result<Vec<JavaScriptToken>, &'static str> javascript_tokens_inner(source, 0, false, 0, &mut token_count).map(|(tokens, _)| tokens) } -fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static str> { - let tokens = javascript_tokens(source)?; +fn javascript_deferred_function_tokens( + tokens: &[JavaScriptToken], + matching_braces: &[Option<usize>], +) -> std::collections::BTreeSet<usize> { + let mut matching_parentheses = vec![None; tokens.len()]; + let mut opening_parentheses = vec![None; tokens.len()]; + let mut open_parentheses = Vec::new(); + for (index, token) in tokens.iter().enumerate() { + match token { + JavaScriptToken::Punctuation(b'(') => open_parentheses.push(index), + JavaScriptToken::Punctuation(b')') => { + if let Some(open) = open_parentheses.pop() { + matching_parentheses[open] = Some(index); + opening_parentheses[index] = Some(open); + } + } + _ => {} + } + } + + let immediately_invoked_group = |function_start: usize, body_end: usize| { + matches!( + tokens.get(body_end + 1), + Some(JavaScriptToken::Punctuation(b')')) + ) && opening_parentheses + .get(body_end + 1) + .and_then(|open| *open) + .is_some_and(|open| { + open < function_start + && matches!( + tokens.get(body_end + 2), + Some(JavaScriptToken::Punctuation(b'(')) + ) + }) + }; + + let mut deferred_ranges = Vec::new(); + for (index, token) in tokens.iter().enumerate() { + if matches!(token, JavaScriptToken::Identifier(name) if name == "function") { + let mut cursor = index + 1; + if matches!(tokens.get(cursor), Some(JavaScriptToken::Punctuation(b'*'))) { + cursor += 1; + } + if matches!(tokens.get(cursor), Some(JavaScriptToken::Identifier(_))) { + cursor += 1; + } + if matches!(tokens.get(cursor), Some(JavaScriptToken::Punctuation(b'('))) { + if let Some(parameters_end) = matching_parentheses[cursor] { + let body = parameters_end + 1; + if matches!(tokens.get(body), Some(JavaScriptToken::Punctuation(b'{'))) { + if let Some(body_end) = matching_braces[body] { + if !immediately_invoked_group(index, body_end) { + deferred_ranges.push(body + 1..body_end); + } + } + } + } + } + } + + if matches!( + (token, tokens.get(index + 1)), + ( + JavaScriptToken::Punctuation(b'='), + Some(JavaScriptToken::Punctuation(b'>')) + ) + ) { + let body = index + 2; + if matches!(tokens.get(body), Some(JavaScriptToken::Punctuation(b'{'))) { + if let Some(body_end) = matching_braces[body] { + if !immediately_invoked_group(index, body_end) { + deferred_ranges.push(body + 1..body_end); + } + } + } else { + let (mut parentheses, mut brackets, mut braces) = (0usize, 0usize, 0usize); + let mut end = body; + while end < tokens.len() { + match tokens.get(end) { + Some(JavaScriptToken::Punctuation(b'(')) => parentheses += 1, + Some(JavaScriptToken::Punctuation(b')')) if parentheses == 0 => break, + Some(JavaScriptToken::Punctuation(b')')) => parentheses -= 1, + Some(JavaScriptToken::Punctuation(b'[')) => brackets += 1, + Some(JavaScriptToken::Punctuation(b']')) if brackets == 0 => break, + Some(JavaScriptToken::Punctuation(b']')) => brackets -= 1, + Some(JavaScriptToken::Punctuation(b'{')) => braces += 1, + Some(JavaScriptToken::Punctuation(b'}')) if braces == 0 => break, + Some(JavaScriptToken::Punctuation(b'}')) => braces -= 1, + Some(JavaScriptToken::Punctuation(b';' | b',')) + if parentheses == 0 && brackets == 0 && braces == 0 => + { + break + } + _ => {} + } + end += 1; + } + let immediately_invoked = + matches!(tokens.get(end), Some(JavaScriptToken::Punctuation(b')'))) + && opening_parentheses + .get(end) + .and_then(|open| *open) + .is_some_and(|open| { + open < index + && matches!( + tokens.get(end + 1), + Some(JavaScriptToken::Punctuation(b'(')) + ) + }); + if !immediately_invoked { + deferred_ranges.push(body..end); + } + } + } + + if matches!(token, JavaScriptToken::Punctuation(b')')) + && matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'{')) + ) + { + let Some(open) = opening_parentheses[index] else { + continue; + }; + let is_control_body = matches!( + tokens.get(open.wrapping_sub(1)), + Some(JavaScriptToken::Identifier(keyword)) + if matches!(keyword.as_str(), "if" | "for" | "while" | "switch" | "catch" | "with") + ); + if !is_control_body { + if let Some(body_end) = matching_braces[index + 1] { + deferred_ranges.push(index + 2..body_end); + } + } + } + } + deferred_ranges.into_iter().flatten().collect() +} + +fn javascript_try_catch_bodies(tokens: &[JavaScriptToken]) -> std::collections::BTreeSet<usize> { + let mut matching_braces = vec![None; tokens.len()]; + let mut open_braces = Vec::new(); + for (index, token) in tokens.iter().enumerate() { + match token { + JavaScriptToken::Punctuation(b'{') => open_braces.push(index), + JavaScriptToken::Punctuation(b'}') => { + if let Some(open) = open_braces.pop() { + matching_braces[open] = Some(index); + } + } + _ => {} + } + } + let mut result = std::collections::BTreeSet::new(); for (index, token) in tokens.iter().enumerate() { - let module = match token { + if !matches!(token, JavaScriptToken::Identifier(keyword) if keyword == "try") + || !matches!( + tokens.get(index + 1), + Some(JavaScriptToken::Punctuation(b'{')) + ) + { + continue; + } + let Some(body_end) = matching_braces[index + 1] else { + continue; + }; + if !matches!(tokens.get(body_end + 1), Some(JavaScriptToken::Identifier(keyword)) if keyword == "catch") + { + continue; + } + let mut catch_body = body_end + 2; + if matches!( + tokens.get(catch_body), + Some(JavaScriptToken::Punctuation(b'(')) + ) { + let mut depth = 0usize; + while catch_body < tokens.len() { + match tokens.get(catch_body) { + Some(JavaScriptToken::Punctuation(b'(')) => depth += 1, + Some(JavaScriptToken::Punctuation(b')')) => { + depth = depth.saturating_sub(1); + if depth == 0 { + catch_body += 1; + break; + } + } + _ => {} + } + catch_body += 1; + } + } + if matches!( + tokens.get(catch_body), + Some(JavaScriptToken::Punctuation(b'{')) + ) && matching_braces[catch_body].is_some() + { + result.extend(index + 2..body_end); + } + } + let deferred_function_tokens = javascript_deferred_function_tokens(tokens, &matching_braces); + result.retain(|index| !deferred_function_tokens.contains(index)); + result +} + +struct NodeModuleSpecifier { + value: String, + missing_is_optional: bool, +} + +fn node_relative_module_specifiers(source: &str) -> Result<Vec<NodeModuleSpecifier>, &'static str> { + let tokens = javascript_tokens(source)?; + let optional_try_tokens = javascript_try_catch_bodies(&tokens); + let mut result = std::collections::BTreeMap::<String, bool>::new(); + for (index, token) in tokens.iter().enumerate() { + let (module, require_resolve) = match token { JavaScriptToken::Identifier(name) if name == "require" || name == "import" => { let require_resolve = name == "require" && matches!( @@ -1348,27 +1617,40 @@ fn node_relative_module_specifiers(source: &str) -> Result<Vec<String>, &'static Some(JavaScriptToken::Punctuation(b'(')) ); if require_resolve { - tokens.get(index + 4) + (tokens.get(index + 4), true) } else { - match tokens.get(index + 1) { - Some(JavaScriptToken::Punctuation(b'(')) => tokens.get(index + 2), - Some(JavaScriptToken::String(_)) if name == "import" => { - tokens.get(index + 1) - } - _ => None, - } + ( + match tokens.get(index + 1) { + Some(JavaScriptToken::Punctuation(b'(')) => tokens.get(index + 2), + Some(JavaScriptToken::String(_)) if name == "import" => { + tokens.get(index + 1) + } + _ => None, + }, + false, + ) } } - JavaScriptToken::Identifier(name) if name == "from" => tokens.get(index + 1), - _ => None, + JavaScriptToken::Identifier(name) if name == "from" => (tokens.get(index + 1), false), + _ => (None, false), }; if let Some(JavaScriptToken::String(Some(specifier))) = module { if specifier.starts_with("./") || specifier.starts_with("../") { - result.insert(specifier.clone()); + let missing_is_optional = require_resolve && optional_try_tokens.contains(&index); + result + .entry(specifier.clone()) + .and_modify(|prior| *prior &= missing_is_optional) + .or_insert(missing_is_optional); } } } - Ok(result.into_iter().collect()) + Ok(result + .into_iter() + .map(|(value, missing_is_optional)| NodeModuleSpecifier { + value, + missing_is_optional, + }) + .collect()) } fn node_relative_resource_specifiers( @@ -2528,6 +2810,18 @@ struct DirectScriptResources<'a> { writable_files: &'a mut std::collections::BTreeSet<PathBuf>, } +struct ShellScriptSelection<'a> { + default_working_directory: &'a Path, + covered_directory: Option<&'a Path>, +} + +struct ShellScriptResources<'a> { + paths: &'a mut std::collections::BTreeSet<PathBuf>, + path_bytes: &'a mut usize, + writable_resources: &'a mut std::collections::BTreeSet<PathBuf>, + node_env_files: &'a mut Vec<NodeEnvFileSelection>, +} + fn direct_script_dependency_paths( parent: &Path, entry_point_import_root: &Path, @@ -2550,8 +2844,13 @@ fn direct_script_dependency_paths( .or_insert((parse_as_script, writable)); }; for specifier in node_relative_module_specifiers(source)? { - let files = resolve_node_relative_module(parent, &specifier)? - .ok_or("direct MCP Node relative module is missing")?; + let Some(files) = resolve_node_relative_module(parent, &specifier.value)? + else { + if specifier.missing_is_optional { + continue; + } + return Err("direct MCP Node relative module is missing"); + }; for file in files { record(file, true, false); } @@ -2751,9 +3050,8 @@ fn insert_shell_script_dependencies( script: &Path, scope: &Path, declaring: &Path, - result: &mut std::collections::BTreeSet<PathBuf>, - resource_path_bytes: &mut usize, - writable_resources: &mut std::collections::BTreeSet<PathBuf>, + selection: ShellScriptSelection<'_>, + resources: ShellScriptResources<'_>, field: &'static str, ) -> Result<(), ResourceError> { const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; @@ -2799,21 +3097,31 @@ fn insert_shell_script_dependencies( let script_parent = canonical .parent() .ok_or_else(|| fail("direct MCP shell script has no containing directory"))?; - let mut working_directory_is_script_parent = false; - for line in source - .lines() - .filter(|line| !line.trim_start().starts_with('#')) - { + let mut working_directory = Some(selection.default_working_directory.to_owned()); + let mut source_offset = 0usize; + for full_line in source.split_inclusive('\n') { + let line = full_line + .strip_suffix('\n') + .unwrap_or(full_line) + .strip_suffix('\r') + .unwrap_or(full_line.strip_suffix('\n').unwrap_or(full_line)); + if line.trim_start().starts_with('#') { + source_offset += full_line.len(); + continue; + } let node_line = if let Some(remainder) = shell_changes_to_script_directory(line) { - working_directory_is_script_parent = true; + working_directory = Some(script_parent.to_owned()); remainder } else { if line.trim_start().starts_with("cd ") { - working_directory_is_script_parent = false; + working_directory = None; } line }; + let node_line_offset = source_offset + line.len().saturating_sub(node_line.len()); let mut candidates = std::collections::BTreeMap::new(); + let mut env_file_candidates = + std::collections::BTreeMap::<PathBuf, Vec<(String, std::ops::Range<usize>)>>::new(); if let Some(operand) = shell_source_operand(line) { if let Some(candidate) = shell_source_path(script_parent, &operand, &directory_variables) @@ -2835,15 +3143,53 @@ fn insert_shell_script_dependencies( .and_modify(|kind: &mut (bool, bool)| kind.1 |= parse_as_node_script) .or_insert((false, parse_as_node_script)); } - if working_directory_is_script_parent && invokes_node { + if invokes_node { if let Some(invocation) = node_invocation { - if let Some(relative) = invocation.script_operand { - candidates - .entry(script_parent.join(relative)) - .and_modify(|kind: &mut (bool, bool)| kind.1 = true) - .or_insert((false, true)); + let ShellNodeInvocation { + script_operand, + file_operands, + env_file_operands, + } = invocation; + if let Some(relative) = script_operand { + let path = Path::new(&relative); + let candidate = if path.is_absolute() { + Some(path.to_owned()) + } else { + working_directory + .as_deref() + .map(|directory| directory.join(path)) + }; + if let Some(candidate) = candidate { + candidates + .entry(candidate) + .and_modify(|kind: &mut (bool, bool)| kind.1 = true) + .or_insert((false, true)); + } } - for (operand, parse_as_node_script) in invocation.file_operands { + for (operand, parse_as_node_script) in file_operands { + if operand.contains(['$', '*', '?']) { + return Err(fail( + "direct MCP Node option has an unresolved dynamic file path", + )); + } + let path = Path::new(&operand); + let candidate = if path.is_absolute() { + Some(path.to_owned()) + } else { + working_directory + .as_deref() + .map(|directory| directory.join(path)) + }; + if let Some(candidate) = candidate { + candidates + .entry(candidate) + .and_modify(|kind: &mut (bool, bool)| { + kind.1 |= parse_as_node_script + }) + .or_insert((false, parse_as_node_script)); + } + } + for (operand, argument_span, optional) in env_file_operands { if operand.contains(['$', '*', '?']) { return Err(fail( "direct MCP Node option has an unresolved dynamic file path", @@ -2853,12 +3199,27 @@ fn insert_shell_script_dependencies( let candidate = if path.is_absolute() { path.to_owned() } else { - script_parent.join(path) + working_directory + .as_deref() + .ok_or_else(|| { + fail("direct MCP Node env-file has an unresolved working directory") + })? + .join(path) }; - candidates - .entry(candidate) - .and_modify(|kind: &mut (bool, bool)| kind.1 |= parse_as_node_script) - .or_insert((false, parse_as_node_script)); + if optional + && std::fs::symlink_metadata(&candidate) + .is_err_and(|error| error.kind() == std::io::ErrorKind::NotFound) + { + continue; + } + env_file_candidates + .entry(candidate.clone()) + .or_default() + .push(( + operand, + node_line_offset + argument_span.start + ..node_line_offset + argument_span.end, + )); } } } @@ -2872,37 +3233,69 @@ fn insert_shell_script_dependencies( "direct MCP shell source escapes its selected script scope", )); } - insert_resource( - result, - resource_path_bytes, - dependency.clone(), - field, - declaring, - )?; + if !selection + .covered_directory + .is_some_and(|directory| dependency.starts_with(directory)) + { + insert_resource( + resources.paths, + resources.path_bytes, + dependency.clone(), + field, + declaring, + )?; + } if parse_as_node_script { - insert_installed_package_ancestor_resources( + insert_installed_package_ancestor_resources_covered( &dependency, scope, declaring, - result, - resource_path_bytes, + resources.paths, + resources.path_bytes, field, + selection.covered_directory, )?; - insert_direct_script_dependencies( + insert_direct_script_dependencies_covered( &dependency, scope, declaring, DirectScriptResources { - paths: result, - path_bytes: resource_path_bytes, - writable_files: writable_resources, + paths: &mut *resources.paths, + path_bytes: &mut *resources.path_bytes, + writable_files: &mut *resources.writable_resources, }, field, - ScriptDependencyRuntime::Node, + ScriptDependencySelection { + runtime: ScriptDependencyRuntime::Node, + covered_directory: selection.covered_directory, + }, )?; } pending.push((dependency, parse_as_script)); } + for (candidate, arguments) in env_file_candidates { + let dependency = checked(&candidate, field, declaring)?; + if !dependency.is_file() { + return Err(fail("direct MCP Node env file is not a regular file")); + } + if !dependency.starts_with(scope) { + return Err(fail( + "direct MCP Node env file escapes its selected script scope", + )); + } + for (expected_operand, argument_span) in arguments { + resources.node_env_files.push(NodeEnvFileSelection { + canonical_source: dependency.clone(), + selected_alias: candidate.clone(), + rewrite: NodeEnvFileRewrite::ShellScriptArgument { + script_source: canonical.clone(), + argument_span, + expected_operand, + }, + }); + } + } + source_offset += full_line.len(); } } Ok(()) @@ -3138,11 +3531,22 @@ pub(super) struct ResourceSelection { pub node_env_files: Vec<NodeEnvFileSelection>, } +pub(super) enum NodeEnvFileRewrite { + ConfigArgument { + argument_span: std::ops::Range<usize>, + option_prefix: String, + }, + ShellScriptArgument { + script_source: PathBuf, + argument_span: std::ops::Range<usize>, + expected_operand: String, + }, +} + pub(super) struct NodeEnvFileSelection { pub canonical_source: PathBuf, pub selected_alias: PathBuf, - pub argument_span: std::ops::Range<usize>, - pub option_prefix: String, + pub rewrite: NodeEnvFileRewrite, } fn checked(path: &Path, field: &'static str, _source: &Path) -> Result<PathBuf, ResourceError> { let fail = |cause| ResourceError { field, cause }; @@ -3585,8 +3989,10 @@ pub(super) fn select( node_env_files.push(NodeEnvFileSelection { canonical_source: canonical, selected_alias: path, - argument_span: argument.raw_span, - option_prefix: argument.option_prefix, + rewrite: NodeEnvFileRewrite::ConfigArgument { + argument_span: argument.raw_span, + option_prefix: argument.option_prefix, + }, }); continue; } @@ -3697,6 +4103,30 @@ pub(super) fn select( .as_ref() .is_some_and(|working_dir| canonical.starts_with(working_dir)) { + if is_shell_script(&canonical) { + let command_parent = canonical + .parent() + .ok_or_else(|| fail(field, "command has no containing scope"))?; + let dependency_scope = direct_script_dependency_scope(command_parent, home); + let working_directory = + current_mcp_working_dir.as_deref().unwrap_or(instance_cwd); + insert_shell_script_dependencies( + &canonical, + &dependency_scope, + declaring, + ShellScriptSelection { + default_working_directory: working_directory, + covered_directory: current_mcp_working_dir.as_deref(), + }, + ShellScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_resources: &mut writable_resources, + node_env_files: &mut node_env_files, + }, + field, + )?; + } continue; } let command_parent = canonical @@ -3822,9 +4252,18 @@ pub(super) fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, - &mut writable_resources, + ShellScriptSelection { + default_working_directory: current_mcp_working_dir + .as_deref() + .unwrap_or(instance_cwd), + covered_directory: None, + }, + ShellScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_resources: &mut writable_resources, + node_env_files: &mut node_env_files, + }, field, )?; } @@ -3902,9 +4341,18 @@ pub(super) fn select( &canonical, &dependency_scope, declaring, - &mut result, - &mut resource_path_bytes, - &mut writable_resources, + ShellScriptSelection { + default_working_directory: current_mcp_working_dir + .as_deref() + .unwrap_or(instance_cwd), + covered_directory: None, + }, + ShellScriptResources { + paths: &mut result, + path_bytes: &mut resource_path_bytes, + writable_resources: &mut writable_resources, + node_env_files: &mut node_env_files, + }, field, )?; } diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index e9bd21b98..e4c9ecb42 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -314,11 +314,20 @@ pub struct PlannedSourceOccurrence { pub configured_writable_resources: std::collections::BTreeSet<PathBuf>, /// Selected Node env-file aliases excluded from ordinary resource copies. pub configured_resource_exclusions: std::collections::BTreeSet<PathBuf>, + /// Shell wrapper Node env-file operands rewritten in their staged copies. + pub configured_resource_edits: Vec<ConfiguredResourceEdit>, /// Captured with the retained source view so later staging does not /// reinterpret a generated entry after ordinary-terminal cleanup. pub user_owns_freshell_mcp: bool, } +pub struct ConfiguredResourceEdit { + pub source: PathBuf, + pub span: std::ops::Range<usize>, + pub expected: String, + pub replacement: String, +} + #[derive(Default)] pub(crate) struct SourceViewBudget { views: std::collections::BTreeSet<(usize, usize)>, @@ -657,12 +666,30 @@ pub fn plan_loaded_opencode_sources_relocated( cause: error.cause, })?; let mut node_env_file_edits = Vec::with_capacity(selected_resources.node_env_files.len()); + let mut configured_resource_edits = Vec::new(); for env_file in &selected_resources.node_env_files { let destination = allocate(&env_file.canonical_source).map_err(fail)?; - let replacement = format!("{}{destination}", env_file.option_prefix); - let replacement = serde_json::to_string(&replacement) - .map_err(|_| fail("selected Node env-file path could not be encoded"))?; - node_env_file_edits.push((env_file.argument_span.clone(), replacement)); + match &env_file.rewrite { + super::opencode_resources::NodeEnvFileRewrite::ConfigArgument { + argument_span, + option_prefix, + } => { + let replacement = format!("{option_prefix}{destination}"); + let replacement = serde_json::to_string(&replacement) + .map_err(|_| fail("selected Node env-file path could not be encoded"))?; + node_env_file_edits.push((argument_span.clone(), replacement)); + } + super::opencode_resources::NodeEnvFileRewrite::ShellScriptArgument { + script_source, + argument_span, + expected_operand, + } => configured_resource_edits.push(ConfiguredResourceEdit { + source: script_source.clone(), + span: argument_span.clone(), + expected: expected_operand.clone(), + replacement: destination, + }), + } } let references = super::resolve_file_references_from_view_relocated( &loaded.view, @@ -773,6 +800,7 @@ pub fn plan_loaded_opencode_sources_relocated( configured_resources: selected_resources.resources, configured_writable_resources: selected_resources.writable_resources, configured_resource_exclusions, + configured_resource_edits, user_owns_freshell_mcp, }; planned diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index c8e4d9c80..edfd183c8 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -123,8 +123,12 @@ import dependency from 'managed-direct-fixture-dependency' const schema = JSON.parse(fs.readFileSync(new URL('./schema.json', import.meta.url), 'utf8')) const [receiptPath, expectedNonce] = process.argv.slice(2) if (!receiptPath || !expectedNonce) process.exit(12) +const envFileOperandRewritten = process.execArgv.some(value => + /^--env-file=\/home\/freshell\/provider\/\.freshell\/references\/\d+$/.test(value)) +const envValueLoaded = process.env.FRESHELL_MANAGED_SHARED_ENV === expectedNonce fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper: process.env.MANAGED_DIRECT_MCP_HELPER, - sibling, dependency, schemaNonce: schema.nonce, cwd: process.cwd(), scriptPath: process.argv[1] })) + sibling, dependency, schemaNonce: schema.nonce, cwd: process.cwd(), scriptPath: process.argv[1], + envFileOperandRewritten, envValueLoaded })) let buffered = '' process.stdin.setEncoding('utf8') process.stdin.on('data', chunk => { @@ -161,7 +165,11 @@ let appendError try { fs.appendFileSync(${outputUrl}, '|' + expectedNonce) } catch (error) { appendError = error.code ?? error.message ?? String(error) } const output = fs.readFileSync(${outputUrl}, 'utf8') -fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, appendError, output, scriptPath: process.argv[1] })) +const envFileOperandRewritten = process.execArgv.some(value => + /^--env-file=\/home\/freshell\/provider\/\.freshell\/references\/\d+$/.test(value)) +const envValueLoaded = process.env.FRESHELL_MANAGED_SHARED_ENV === expectedNonce +fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, appendError, output, scriptPath: process.argv[1], + envFileOperandRewritten, envValueLoaded })) let buffered = '' process.stdin.setEncoding('utf8') process.stdin.on('data', chunk => { @@ -352,6 +360,8 @@ async function configure(root: string, nonce: string, candidateSha: string, runI await fs.mkdir(mcpNested, { recursive: true, mode: 0o755 }) await fs.mkdir(directMcp, { recursive: true, mode: 0o755 }) await fs.mkdir(mcpParentDependency, { recursive: true, mode: 0o755 }) + const sharedEnvFile = path.join(directMcp, '.env') + await fs.writeFile(sharedEnvFile, `FRESHELL_MANAGED_SHARED_ENV=${nonce}\n`, { mode: 0o600 }) const directMcpDependency = path.join(directMcp, 'node_modules/managed-direct-fixture-dependency') await fs.mkdir(directMcpDependency, { recursive: true, mode: 0o755 }) await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) @@ -394,7 +404,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.writeFile(path.join(mcpPackage, 'server.mjs'), managedMcpServerSource) const directCommand = path.join(directMcp, 'start') await fs.writeFile(directCommand, - '#!/bin/sh\nSCRIPT_DIR=$(dirname "$0")\n. "$SCRIPT_DIR/helper.sh"\nexec node "$SCRIPT_DIR/server.mjs" "$@"\n', + '#!/bin/sh\ncd "$(dirname "$0")"\nSCRIPT_DIR=$(dirname "$0")\n. "$SCRIPT_DIR/helper.sh"\nexec node --env-file=.env "$SCRIPT_DIR/server.mjs" "$@"\n', { mode: 0o751 }) await fs.chmod(directCommand, 0o751) await fs.writeFile(path.join(directMcp, 'helper.sh'), @@ -446,7 +456,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ }, direct_home_append_fixture: { type: 'local', - command: ['node', homeAppend.script, homeAppend.receipt, nonce], + command: ['node', `--env-file=${sharedEnvFile}`, homeAppend.script, homeAppend.receipt, nonce], enabled: true, }, }, @@ -1362,12 +1372,16 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightDirectMcp.sibling, nonce) assert.equal(preflightDirectMcp.dependency, nonce) assert.equal(preflightDirectMcp.schemaNonce, nonce) + assert.equal(preflightDirectMcp.envFileOperandRewritten, true) + assert.equal(preflightDirectMcp.envValueLoaded, true) assert(preflightDirectMcp.scriptPath.endsWith('/direct-mcp/server.mjs')) const preflightHomeAppend = await poll('OpenCode starts the direct HOME MCP and it appends its sibling output', async () => readJson(path.join(selected.workspace, 'preflight/.managed-fixture/direct-home-mcp-receipt.json'))) assert.equal(preflightHomeAppend.expectedNonce, nonce) assert.equal(preflightHomeAppend.appendError, undefined, 'the staged HOME-level append target must be writable') assert.equal(preflightHomeAppend.output, `initial-${nonce}|${nonce}`) + assert.equal(preflightHomeAppend.envFileOperandRewritten, true) + assert.equal(preflightHomeAppend.envValueLoaded, true) assert(preflightHomeAppend.scriptPath.endsWith('/direct-home-preflight.mjs')) const preflightView = await poll('genuine managed preflight native identity', () => observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) @@ -1517,12 +1531,16 @@ export async function runManagedEnvironmentTarget() { assert.equal(directMcpServer.sibling, nonce) assert.equal(directMcpServer.dependency, nonce) assert.equal(directMcpServer.schemaNonce, nonce) + assert.equal(directMcpServer.envFileOperandRewritten, true) + assert.equal(directMcpServer.envValueLoaded, true) assert(directMcpServer.scriptPath.endsWith('/direct-mcp/server.mjs')) const directHomeAppendMcp = await poll('Cloud OpenCode appends through its direct HOME MCP', async () => readJson(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json'))) assert.equal(directHomeAppendMcp.expectedNonce, nonce) assert.equal(directHomeAppendMcp.appendError, undefined, 'the managed HOME sibling mount must permit append') assert.equal(directHomeAppendMcp.output, `initial-${nonce}|${nonce}`) + assert.equal(directHomeAppendMcp.envFileOperandRewritten, true) + assert.equal(directHomeAppendMcp.envValueLoaded, true) assert(directHomeAppendMcp.scriptPath.endsWith('/direct-home-healthy.mjs')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 6b0d99ea2..12cf56f2a 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -236,8 +236,10 @@ test(healthyTitle, async ({ page }, info) => { return directMcpReceipt }, { timeout: 120_000 }).toMatchObject({ expectedNonce: fixture.expectedNonce, helper: fixture.expectedNonce, cwd: pane.cwd }) expect(directMcpReceipt).toMatchObject({ sibling: fixture.expectedNonce, - dependency: fixture.expectedNonce, schemaNonce: fixture.expectedNonce }) + dependency: fixture.expectedNonce, schemaNonce: fixture.expectedNonce, + envFileOperandRewritten: true, envValueLoaded: true }) expect(directMcpReceipt.scriptPath).toMatch(/\/direct-mcp\/server\.mjs$/) + expect(receipt.directHomeAppendMcp).toMatchObject({ envFileOperandRewritten: true, envValueLoaded: true }) expect(receipt.soulId).toBeTruthy() expect(receipt.incarnationId).toBeTruthy() expect(receipt.nativeSessionId).toBeTruthy() From 261e05a57f088407553a6de945f44a9c3486f50d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 11:41:24 -0700 Subject: [PATCH 437/493] fix(managed): stage OpenCode MCP environment references --- .../src/managed_mcp_capability.rs | 25 +- .../managed_mcp_capability/source_stage.rs | 465 ++++++++++++++++++ .../opencode_resources.rs | 67 ++- 3 files changed, 540 insertions(+), 17 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 47e97a174..9da0b96cf 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1741,19 +1741,22 @@ fn stage_provider_root( }; if !units.is_empty() { let scratch = tempfile::tempdir_in(parent).map_err(|error| error.to_string())?; - source_stage::filter_workspace_from_enclosing_resource_units( - &mut units, - provider, - opencode_input - .workspace - .or(opencode_input.cwd) - .unwrap_or(home), - scratch.path(), - &excluded, - )?; + configured_resource_edits = + source_stage::filter_workspace_from_enclosing_resource_units_with_edits( + &mut units, + provider, + opencode_input + .workspace + .or(opencode_input.cwd) + .unwrap_or(home), + scratch.path(), + &excluded, + plan, + )?; _filtered_resource_sources = Some(scratch); + } else { + configured_resource_edits = source_stage::configured_resource_edits(&units, plan)?; } - configured_resource_edits = source_stage::configured_resource_edits(&units, plan)?; selections.extend(source_stage::resource_copy_selections(&units)?); let descriptors = units .into_iter() diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index b9a1ef2fe..a6c9ed8c6 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1496,6 +1496,145 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"node sibling helper loaded\n"); } + #[test] + fn direct_home_mcp_appends_to_a_selected_writable_sibling_with_env_reference() { + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let mcp = home.join("direct-mcp"); + let script = home.join("direct-home-preflight.mjs"); + let output_file = home.join("direct-home-preflight.txt"); + let env_file = mcp.join(".env"); + let receipt = workspace.join(".managed-fixture/direct-home-receipt.json"); + let nonce = "direct-home-mcp-fixture"; + std::fs::create_dir_all(&mcp).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(receipt.parent().unwrap()).unwrap(); + std::fs::write(&output_file, format!("initial-{nonce}")).unwrap(); + std::fs::write(&env_file, format!("FRESHELL_MANAGED_SHARED_ENV={nonce}\n")).unwrap(); + std::fs::write( + &script, + format!( + "import fs from 'node:fs'\nconst [receiptPath, expectedNonce] = process.argv.slice(2)\nlet appendError\ntry {{ fs.appendFileSync(new URL('./{}', import.meta.url), '|' + expectedNonce) }} catch (error) {{ appendError = error.code ?? error.message }}\nconst output = fs.readFileSync(new URL('./{}', import.meta.url), 'utf8')\nconst envFileOperandRewritten = process.execArgv.some(value => /^--env-file=.*\\/\\.freshell\\/references\\/\\d+$/.test(value))\nconst envValueLoaded = process.env.FRESHELL_MANAGED_SHARED_ENV === expectedNonce\nfs.writeFileSync(receiptPath, JSON.stringify({{ expectedNonce, appendError, output, envFileOperandRewritten, envValueLoaded }}))\n", + output_file.file_name().unwrap().to_string_lossy(), + output_file.file_name().unwrap().to_string_lossy(), + ), + ) + .unwrap(); + let command = vec![ + "node".to_owned(), + format!("--env-file={}", env_file.display()), + script.to_string_lossy().into_owned(), + receipt.to_string_lossy().into_owned(), + nonce.to_owned(), + ]; + let (plan, provider_script) = plan_and_stage_mcp_command_for_resource( + owned.path(), + &home, + &workspace, + &command, + &script, + ); + assert!(plan[0] + .configured_writable_resources + .contains(&output_file.canonicalize().unwrap())); + assert!(!plan[0] + .configured_resources + .contains(&env_file.canonicalize().unwrap())); + let selected = &plan[0].references.selections[0]; + let prepared_config: serde_json::Value = + serde_json::from_str(&plan[0].references.template).unwrap(); + let prepared_arguments = prepared_config["mcp"]["direct"]["command"] + .as_array() + .unwrap() + .iter() + .map(|argument| argument.as_str().unwrap().to_owned()) + .collect::<Vec<_>>(); + assert_eq!( + prepared_arguments[1], + format!("--env-file={}", selected.runtime_destination) + ); + let map_provider_path = |value: &str| { + let runtime_path = value + .strip_prefix("/home/freshell/provider/") + .map(|relative| { + owned + .path() + .join("provider/.provider-home") + .join(relative) + .to_string_lossy() + .into_owned() + }); + runtime_path + .or_else(|| { + Path::new(value) + .starts_with(&workspace) + .then(|| value.to_owned()) + }) + .or_else(|| { + Path::new(value) + .strip_prefix(owned.path()) + .ok() + .map(|relative| { + owned + .path() + .join("provider") + .join(relative) + .to_string_lossy() + .into_owned() + }) + }) + .unwrap_or_else(|| value.to_owned()) + }; + let map_provider_argument = |value: &str| { + value + .split_once('=') + .map(|(option, path)| format!("{option}={}", map_provider_path(path))) + .unwrap_or_else(|| map_provider_path(value)) + }; + let prepared_provider_arguments = prepared_arguments + .iter() + .map(|argument| map_provider_argument(argument)) + .collect::<Vec<_>>(); + assert_eq!( + Path::new(&prepared_provider_arguments[2]), + provider_script, + "the prepared direct HOME MCP command must point at the staged script" + ); + let reference_relative = selected + .runtime_destination + .strip_prefix("/home/freshell/provider/") + .unwrap(); + let provider_reference = owned + .path() + .join("provider/.provider-home") + .join(reference_relative); + std::fs::create_dir_all(provider_reference.parent().unwrap()).unwrap(); + std::fs::copy(&env_file, &provider_reference).unwrap(); + + let hidden_home = owned.path().join("home-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + let output = std::process::Command::new("node") + .args(&prepared_provider_arguments[1..]) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .output() + .unwrap(); + std::fs::rename(&hidden_home, &home).unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert!( + receipt.is_file(), + "prepared arguments did not write the expected provider receipt: {prepared_provider_arguments:?}" + ); + let receipt: serde_json::Value = + serde_json::from_slice(&std::fs::read(receipt).unwrap()).unwrap(); + assert_eq!(receipt["expectedNonce"], nonce); + assert!(receipt["appendError"].is_null()); + assert_eq!(receipt["output"], format!("initial-{nonce}|{nonce}")); + assert_eq!(receipt["envFileOperandRewritten"], true); + assert_eq!(receipt["envValueLoaded"], true); + } + #[test] fn direct_node_script_allows_missing_optional_require_resolve_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; @@ -1609,6 +1748,310 @@ mod configured_resource_tests { assert_eq!(output.stdout, b"cwd env loaded\n"); } + #[test] + fn shell_wrapper_inside_mounted_workspace_does_not_need_a_staged_rewrite() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let mcp_cwd = workspace.join("mcp"); + let wrapper = mcp_cwd.join("bin/start"); + let env_file = mcp_cwd.join(".env"); + std::fs::create_dir_all(wrapper.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &wrapper, + "#!/bin/sh\nexec node --env-file=.env server.mjs\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write( + mcp_cwd.join("server.mjs"), + "process.stdout.write(process.env.MANAGED_WORKSPACE_ENV || 'missing env');\n", + ) + .unwrap(); + std::fs::write(&env_file, "MANAGED_WORKSPACE_ENV=workspace-mounted\n").unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "workspace": { + "type": "local", + "command": [wrapper], + "cwd": mcp_cwd, + } }, + }) + .to_string(), + ) + .unwrap(); + + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let edits = configured_resource_edits(&units, &plan); + assert!( + edits.is_ok(), + "a workspace-mounted wrapper has no separate resource copy to rewrite: {:?}", + edits.as_ref().err() + ); + assert!(edits.unwrap().is_empty()); + assert!(!plan[0] + .configured_resources + .contains(&env_file.canonicalize().unwrap())); + } + + #[test] + fn shell_wrapper_can_cd_to_a_script_directory_variable_before_using_env_file() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let mcp_cwd = home.join("mcp"); + let script_dir = mcp_cwd.join("bin"); + let wrapper = script_dir.join("start"); + std::fs::create_dir_all(&script_dir).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write( + &wrapper, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\ncd \"$SCRIPT_DIR\"\nexec node --env-file=.env server.mjs\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write( + script_dir.join("server.mjs"), + "if (process.env.MANAGED_SCRIPT_DIR_ENV !== 'script-dir-secret') process.exit(11);\nprocess.stdout.write('script directory env loaded\\n');\n", + ) + .unwrap(); + std::fs::write( + script_dir.join(".env"), + "MANAGED_SCRIPT_DIR_ENV=script-dir-secret\n", + ) + .unwrap(); + + let (plan, provider_wrapper) = plan_and_stage_mcp_command_for_resource_with_cwd( + owned.path(), + &home, + &workspace, + &[wrapper.to_string_lossy().into_owned()], + &wrapper, + Some(&mcp_cwd), + ); + assert_eq!(plan[0].references.selections.len(), 1); + let reference = owned + .path() + .join("provider/.provider-home/.freshell/references/0"); + std::fs::create_dir_all(reference.parent().unwrap()).unwrap(); + std::fs::copy(script_dir.join(".env"), &reference).unwrap(); + let wrapper_permissions = std::fs::metadata(&provider_wrapper).unwrap().permissions(); + let wrapper_source = std::fs::read_to_string(&provider_wrapper).unwrap(); + std::fs::write( + &provider_wrapper, + wrapper_source.replace( + "/home/freshell/provider/.freshell/references/0", + &reference.to_string_lossy(), + ), + ) + .unwrap(); + std::fs::set_permissions(&provider_wrapper, wrapper_permissions).unwrap(); + let provider_cwd = provider_wrapper.parent().unwrap().parent().unwrap(); + let output = + execute_selected_resource_with_context(&provider_wrapper, None, Some(provider_cwd)) + .unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"script directory env loaded\n"); + } + + #[test] + fn shell_wrapper_can_use_script_and_env_files_from_its_configured_project_cwd() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let project = owned.path().join("project"); + let wrapper = home.join("bin/start"); + std::fs::create_dir_all(wrapper.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&project).unwrap(); + std::fs::write( + &wrapper, + "#!/bin/sh\nexec node --env-file=.env server.mjs\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write( + project.join("server.mjs"), + "if (process.env.MANAGED_PROJECT_ENV !== 'project-secret') process.exit(11);\nprocess.stdout.write('project cwd env loaded\\n');\n", + ) + .unwrap(); + std::fs::write(project.join(".env"), "MANAGED_PROJECT_ENV=project-secret\n").unwrap(); + + let (plan, provider_wrapper) = plan_and_stage_mcp_command_for_resource_with_cwd( + owned.path(), + &home, + &workspace, + &[wrapper.to_string_lossy().into_owned()], + &wrapper, + Some(&project), + ); + assert_eq!(plan[0].references.selections.len(), 1); + let reference = owned + .path() + .join("provider/.provider-home/.freshell/references/0"); + std::fs::create_dir_all(reference.parent().unwrap()).unwrap(); + std::fs::copy(project.join(".env"), &reference).unwrap(); + let wrapper_permissions = std::fs::metadata(&provider_wrapper).unwrap().permissions(); + let wrapper_source = std::fs::read_to_string(&provider_wrapper).unwrap(); + std::fs::write( + &provider_wrapper, + wrapper_source.replace( + "/home/freshell/provider/.freshell/references/0", + &reference.to_string_lossy(), + ), + ) + .unwrap(); + std::fs::set_permissions(&provider_wrapper, wrapper_permissions).unwrap(); + let units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let project_mount = units + .iter() + .find(|(source, _)| source == &project.canonicalize().unwrap()) + .expect("the configured MCP project cwd must be staged"); + let provider_cwd_relative = Path::new(&project_mount.1.effective_destination) + .strip_prefix(owned.path()) + .map(Path::to_path_buf) + .or_else(|_| { + Path::new(&project_mount.1.effective_destination) + .strip_prefix("/home/freshell/provider") + .map(|relative| Path::new(".provider-home").join(relative)) + }) + .expect("the configured project cwd must map into the provider fixture"); + let provider_cwd = owned.path().join("provider").join(provider_cwd_relative); + let hidden_home = owned.path().join("home-hidden"); + let hidden_project = owned.path().join("project-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + std::fs::rename(&project, &hidden_project).unwrap(); + let output = + execute_selected_resource_with_context(&provider_wrapper, None, Some(&provider_cwd)) + .unwrap(); + std::fs::rename(&hidden_project, &project).unwrap(); + std::fs::rename(&hidden_home, &home).unwrap(); + assert!( + output.status.success(), + "the home wrapper must launch from its configured project cwd: {:?}", + output.stderr + ); + assert_eq!(output.stdout, b"project cwd env loaded\n"); + } + + #[test] + fn shell_wrapper_edits_map_to_a_resource_filtered_around_the_workspace() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let external = owned.path().join("external"); + let workspace = external.join("workspace"); + let wrapper = external.join("bin/start"); + std::fs::create_dir_all(wrapper.parent().unwrap()).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + std::fs::write( + &wrapper, + "#!/bin/sh\nexec node --env-file=.env server.mjs\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write( + external.join("server.mjs"), + "process.stdout.write(process.env.MANAGED_FILTERED_ENV || 'missing env');\n", + ) + .unwrap(); + std::fs::write( + external.join(".env"), + "MANAGED_FILTERED_ENV=filtered-resource-secret\n", + ) + .unwrap(); + std::fs::write( + workspace.join("workspace-only.txt"), + "must stay mounted separately", + ) + .unwrap(); + let config = workspace.join("opencode.json"); + std::fs::write( + &config, + serde_json::json!({ + "mcp": { "filtered": { + "type": "local", + "command": [wrapper], + "cwd": external, + } }, + }) + .to_string(), + ) + .unwrap(); + let plan = plan_opencode_sources( + vec![SourceOccurrence { + logical_path: config.clone(), + canonical_source: config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let mut units = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap(); + let scratch = owned.path().join("scratch"); + std::fs::create_dir_all(&scratch).unwrap(); + let edits = filter_workspace_from_enclosing_resource_units_with_edits( + &mut units, + "opencode", + &workspace, + &scratch, + &[], + &plan, + ) + .unwrap_or_else(|error| { + panic!("filtering a containing resource must preserve the original-to-stage edit mapping: {error}") + }); + assert_eq!(edits.len(), 1); + } + #[test] fn direct_node_script_keeps_deferred_require_resolve_required_inside_try() { use std::os::unix::fs::PermissionsExt; @@ -5731,6 +6174,28 @@ pub(super) fn filter_workspace_from_enclosing_resource_units( Ok(()) } +pub(super) fn filter_workspace_from_enclosing_resource_units_with_edits( + units: &mut [( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + provider: &str, + workspace: &Path, + scratch_root: &Path, + excluded_canonical_sources: &[std::path::PathBuf], + plan: &[PlannedSourceOccurrence], +) -> Result<Vec<StagedResourceEdit>, String> { + let edits = configured_resource_edits(units, plan)?; + filter_workspace_from_enclosing_resource_units( + units, + provider, + workspace, + scratch_root, + excluded_canonical_sources, + )?; + Ok(edits) +} + #[cfg(unix)] fn resource_contains_workspace_alias(source: &Path, workspace: &Path) -> Result<bool, String> { let limits = freshell_platform::provider_environment::CopyLimits::default(); diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index f7d239300..b465f3f02 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -749,11 +749,38 @@ fn shell_directory_variables(source: &str) -> std::collections::BTreeSet<String> .collect() } -fn shell_changes_to_script_directory(line: &str) -> Option<&str> { +fn shell_changes_to_script_directory<'a>( + line: &'a str, + directory_variables: &std::collections::BTreeSet<String>, +) -> Option<&'a str> { const CD_TO_SCRIPT_DIRECTORY: &str = "cd \"$(dirname \"$0\")\""; let line = line.trim_start(); - let remainder = line.strip_prefix(CD_TO_SCRIPT_DIRECTORY)?; - if remainder.is_empty() { + if let Some(remainder) = line.strip_prefix(CD_TO_SCRIPT_DIRECTORY) { + return shell_command_after_directory_change(remainder); + } + + let remainder = line.strip_prefix("cd ")?.trim_start(); + let (operand, remainder) = if let Some(quoted) = remainder.strip_prefix('"') { + let end = quoted.find('"')?; + ("ed[..end], "ed[end + 1..]) + } else { + let end = remainder + .find(|character: char| character.is_whitespace() || ";|&".contains(character)) + .unwrap_or(remainder.len()); + (&remainder[..end], &remainder[end..]) + }; + let variable = operand + .strip_prefix("${") + .and_then(|value| value.strip_suffix('}')) + .or_else(|| operand.strip_prefix('$'))?; + if !directory_variables.contains(variable) { + return None; + } + shell_command_after_directory_change(remainder) +} + +fn shell_command_after_directory_change(remainder: &str) -> Option<&str> { + if remainder.trim().is_empty() { return Some(""); } let remainder = remainder.trim_start(); @@ -2813,6 +2840,7 @@ struct DirectScriptResources<'a> { struct ShellScriptSelection<'a> { default_working_directory: &'a Path, covered_directory: Option<&'a Path>, + workspace_directory: Option<&'a Path>, } struct ShellScriptResources<'a> { @@ -3057,6 +3085,8 @@ fn insert_shell_script_dependencies( const MAX_SCRIPT_BYTES: u64 = 1024 * 1024; const MAX_SCRIPT_FILES: usize = 256; let fail = |cause| ResourceError { field, cause }; + let default_working_directory_scope = + std::fs::canonicalize(selection.default_working_directory).ok(); let mut pending = vec![(script.to_owned(), true)]; let mut visited = std::collections::BTreeSet::new(); let mut parsed = std::collections::BTreeSet::new(); @@ -3109,7 +3139,9 @@ fn insert_shell_script_dependencies( source_offset += full_line.len(); continue; } - let node_line = if let Some(remainder) = shell_changes_to_script_directory(line) { + let node_line = if let Some(remainder) = + shell_changes_to_script_directory(line, &directory_variables) + { working_directory = Some(script_parent.to_owned()); remainder } else { @@ -3228,7 +3260,14 @@ fn insert_shell_script_dependencies( if !dependency.is_file() { return Err(fail("direct MCP shell source is not a regular file")); } - if !dependency.starts_with(scope) { + if !dependency.starts_with(scope) + && !default_working_directory_scope + .as_ref() + .is_some_and(|directory| dependency.starts_with(directory)) + && !selection + .workspace_directory + .is_some_and(|directory| dependency.starts_with(directory)) + { return Err(fail( "direct MCP shell source escapes its selected script scope", )); @@ -3278,11 +3317,23 @@ fn insert_shell_script_dependencies( if !dependency.is_file() { return Err(fail("direct MCP Node env file is not a regular file")); } - if !dependency.starts_with(scope) { + if !dependency.starts_with(scope) + && !default_working_directory_scope + .as_ref() + .is_some_and(|directory| dependency.starts_with(directory)) + && !selection + .workspace_directory + .is_some_and(|directory| dependency.starts_with(directory)) + { return Err(fail( "direct MCP Node env file escapes its selected script scope", )); } + if selection.workspace_directory.is_some_and(|workspace| { + canonical.starts_with(workspace) && dependency.starts_with(workspace) + }) { + continue; + } for (expected_operand, argument_span) in arguments { resources.node_env_files.push(NodeEnvFileSelection { canonical_source: dependency.clone(), @@ -3697,6 +3748,7 @@ pub(super) fn select( ) -> Result<ResourceSelection, ResourceError> { let declaring = &source.logical_path; let fail = |field: &'static str, cause: &'static str| ResourceError { field, cause }; + let workspace_directory = std::fs::canonicalize(instance_cwd).ok(); let mut result = std::collections::BTreeSet::new(); let mut resource_path_bytes = 0usize; let mut writable_resources = std::collections::BTreeSet::new(); @@ -4117,6 +4169,7 @@ pub(super) fn select( ShellScriptSelection { default_working_directory: working_directory, covered_directory: current_mcp_working_dir.as_deref(), + workspace_directory: workspace_directory.as_deref(), }, ShellScriptResources { paths: &mut result, @@ -4257,6 +4310,7 @@ pub(super) fn select( .as_deref() .unwrap_or(instance_cwd), covered_directory: None, + workspace_directory: workspace_directory.as_deref(), }, ShellScriptResources { paths: &mut result, @@ -4346,6 +4400,7 @@ pub(super) fn select( .as_deref() .unwrap_or(instance_cwd), covered_directory: None, + workspace_directory: workspace_directory.as_deref(), }, ShellScriptResources { paths: &mut result, From 6bf5b18d73f15edc3689553e2323d0ee5ee89d35 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 12:26:52 -0700 Subject: [PATCH 438/493] fix(managed): preserve OpenCode MCP env-file references --- .../src/provider_inputs.rs | 96 ++++++ .../src/managed_mcp_capability.rs | 1 + .../managed_mcp_capability/amplifier_stage.rs | 1 + .../managed_mcp_capability/ordinary_stage.rs | 1 + .../managed_mcp_capability/source_stage.rs | 291 +++++++++++++++- .../opencode_resources.rs | 178 +++++++--- .../managed_provider_references/sources.rs | 21 ++ crates/freshell-session-host/src/main.rs | 321 +++++++++++++++++- .../src/backend/prepared_inputs.rs | 2 + .../src/backend/provider_mountpoints.rs | 3 + .../managed-provider-environment-target.ts | 79 ++++- 11 files changed, 939 insertions(+), 55 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 2f4ffeca1..88119a3ea 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -3,6 +3,7 @@ use crate::{McpCapabilityReference, RuntimeError, RuntimeErrorCode, SoulId}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; +use std::path::Path; /// One immutable preparation result for every launch and replacement route. /// Source documents and selected payloads stay in their private filesystem. @@ -177,6 +178,17 @@ pub struct ProviderResourceMount { /// Directories retain their existing writable resource behavior. #[serde(default, skip_serializing_if = "is_false")] pub writable: bool, + /// Safe in-root aliases to private file references, created after the + /// resource tree has been copied into the provider HOME. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub reference_aliases: Vec<ProviderResourceReferenceAlias>, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ProviderResourceReferenceAlias { + pub relative_path: String, + pub reference_index: u16, } fn is_false(value: &bool) -> bool { @@ -320,9 +332,16 @@ impl PreparedProviderInputs { ProviderResourceKind::File, )?; } + let mut total_resource_reference_aliases = 0usize; for mount in &self.resource_mounts { let file = mount.kind == ProviderResourceKind::File; + total_resource_reference_aliases = total_resource_reference_aliases + .checked_add(mount.reference_aliases.len()) + .filter(|count| *count <= 256) + .ok_or_else(|| invalid("resource reference alias count"))?; if (mount.writable && !file) + || (!mount.reference_aliases.is_empty() && file) + || mount.reference_aliases.len() > 256 || !unit_path(&mount.stage_relative_path, ".freshell/resource-units", file) || !unit_path( &mount.provider_relative_path, @@ -344,6 +363,32 @@ impl PreparedProviderInputs { format!("resource:{}", mount.provider_relative_path), mount.kind, )?; + let mut aliases = BTreeMap::<String, u16>::new(); + for alias in &mount.reference_aliases { + let relative = Path::new(&alias.relative_path); + if alias.relative_path.len() > 64 * 1024 + || alias.relative_path.is_empty() + || relative.is_absolute() + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + || alias.reference_index >= self.manifest.source_count + || aliases + .insert(alias.relative_path.clone(), alias.reference_index) + .is_some() + { + return Err(invalid("resource reference alias")); + } + } + for path in aliases.keys() { + let path = Path::new(path); + if aliases + .keys() + .any(|other| other != path && Path::new(other).starts_with(path)) + { + return Err(invalid("overlapping resource reference aliases")); + } + } } if let Some(files) = &self.indexed_files { if !files @@ -747,6 +792,56 @@ mod tests { serde_json::from_value(value).expect("write intent is structurally valid"); assert!(writable_directory.validate(Some(&capability())).is_err()); } + + #[test] + fn provider_resource_reference_aliases_are_safe_and_reference_bounded() { + let mut value = prepared(); + value.resource_mounts.push(ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/project".into(), + provider_relative_path: ".freshell/provider-roots/project".into(), + logical_destination: "/fixture/project".into(), + effective_destination: "/fixture/project".into(), + kind: ProviderResourceKind::Directory, + writable: false, + reference_aliases: vec![ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 1, + }], + }); + value.validate(Some(&capability())).unwrap(); + let wire = serde_json::to_vec(&value).unwrap(); + let received: PreparedProviderInputs = serde_json::from_slice(&wire).unwrap(); + assert_eq!(received, value); + + for unsafe_path in ["../outside", "nested/../../outside", "/absolute", ""] { + let mut invalid = value.clone(); + invalid.resource_mounts[0].reference_aliases[0].relative_path = unsafe_path.into(); + assert!( + invalid.validate(Some(&capability())).is_err(), + "accepted {unsafe_path:?}" + ); + } + let mut invalid_index = value.clone(); + invalid_index.resource_mounts[0].reference_aliases[0].reference_index = 2; + assert!(invalid_index.validate(Some(&capability())).is_err()); + + let mut file_mount = value.clone(); + file_mount.resource_mounts[0].kind = ProviderResourceKind::File; + file_mount.resource_mounts[0].stage_relative_path = + ".freshell/resource-units/project/source".into(); + file_mount.resource_mounts[0].provider_relative_path = + ".freshell/provider-roots/project/source".into(); + assert!(file_mount.validate(Some(&capability())).is_err()); + + let mut overlapping = value; + overlapping.resource_mounts[0] + .reference_aliases + .push(ProviderResourceReferenceAlias { + relative_path: ".env/child".into(), + reference_index: 0, + }); + assert!(overlapping.validate(Some(&capability())).is_err()); + } #[test] fn provider_prepared_mounts_preserve_home_and_control_parents() { for destination in [ @@ -762,6 +857,7 @@ mod tests { effective_destination: destination.into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }); assert!( value.validate(Some(&capability())).is_err(), diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 9da0b96cf..65c336e43 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1756,6 +1756,7 @@ fn stage_provider_root( _filtered_resource_sources = Some(scratch); } else { configured_resource_edits = source_stage::configured_resource_edits(&units, plan)?; + source_stage::attach_configured_resource_reference_aliases(&mut units, plan)?; } selections.extend(source_stage::resource_copy_selections(&units)?); let descriptors = units diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs index 844c4ffdc..e8fcde128 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs @@ -143,6 +143,7 @@ pub(super) fn prepare( effective_destination: "/home/freshell/provider/.freshell/resources".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }); Ok(prepared) } diff --git a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs index b4d2f6a27..0341749d5 100644 --- a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs @@ -144,6 +144,7 @@ pub(super) fn prepare_selected<M: serde::Serialize>( .into(), kind, writable: false, + reference_aliases: Vec::new(), }); } } diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index a6c9ed8c6..6c72c4d45 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -68,6 +68,124 @@ pub(super) fn configured_resource_edits( Ok(edits) } +pub(super) fn attach_configured_resource_reference_aliases( + units: &mut [( + std::path::PathBuf, + freshell_runtime_protocol::ProviderResourceMount, + )], + plan: &[PlannedSourceOccurrence], +) -> Result<(), String> { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceReferenceAlias}; + + let mut added = 0usize; + for source in plan { + for alias in &source.configured_resource_reference_aliases { + let reference_destination = source + .references + .selections + .iter() + .find(|selection| { + selection.canonical_source == alias.source + && selection.runtime_destination == alias.reference_destination + }) + .map(|selection| selection.runtime_destination.as_str()) + .ok_or("selected MCP env-file alias has no matching private reference")?; + let reference_index = reference_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .and_then(|value| value.parse::<u16>().ok()) + .ok_or("selected MCP env-file reference has an invalid runtime path")?; + let canonical_working_directory = std::fs::canonicalize(&alias.working_directory) + .map_err(|_| "selected MCP env-file working directory is unavailable")?; + let canonical_source = std::fs::canonicalize(&alias.source) + .map_err(|_| "selected MCP env-file reference source is unavailable")?; + let mut relative_alias = std::path::PathBuf::new(); + for component in alias.relative_path.components() { + match component { + std::path::Component::Normal(part) => relative_alias.push(part), + _ => return Err("selected MCP env-file alias path is unsafe".into()), + } + } + let destination = canonical_working_directory.join(&relative_alias); + let canonical_alias_parent = destination + .parent() + .and_then(|parent| std::fs::canonicalize(parent).ok()) + .ok_or("selected MCP env-file alias parent is unavailable")?; + let destination_source = match std::fs::symlink_metadata(&destination) { + Ok(_) => std::fs::canonicalize(&destination) + .map_err(|_| "selected MCP env-file alias path is unavailable")?, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + canonical_source.clone() + } + Err(_) => { + return Err("selected MCP env-file alias path could not be inspected".into()) + } + }; + if destination_source != canonical_source { + return Err( + "selected MCP env-file alias would replace a different resource".into(), + ); + } + + let mut matching_mounts = units + .iter_mut() + .filter_map(|(resource_source, mount)| { + if mount.kind != ProviderResourceKind::Directory { + return None; + } + let canonical_root = std::fs::canonicalize(&*resource_source).ok()?; + let cwd_suffix = canonical_working_directory + .strip_prefix(&canonical_root) + .ok()?; + canonical_alias_parent.strip_prefix(&canonical_root).ok()?; + Some(( + canonical_root.components().count(), + cwd_suffix.to_path_buf(), + mount, + )) + }) + .collect::<Vec<_>>(); + matching_mounts.sort_by_key(|(depth, _, _)| *depth); + let (_, cwd_suffix, mount) = matching_mounts + .pop() + .ok_or("selected MCP env-file alias has no containing resource directory")?; + let relative_path = cwd_suffix.join(&relative_alias); + if relative_path.as_os_str().is_empty() + || relative_path + .components() + .any(|component| !matches!(component, std::path::Component::Normal(_))) + { + return Err( + "selected MCP env-file alias path escapes its resource directory".into(), + ); + } + if mount.reference_aliases.iter().any(|existing| { + existing.relative_path == relative_path.to_string_lossy() + && existing.reference_index != reference_index + }) { + return Err( + "selected MCP env-file aliases conflict within one resource directory".into(), + ); + } + if !mount.reference_aliases.iter().any(|existing| { + existing.relative_path == relative_path.to_string_lossy() + && existing.reference_index == reference_index + }) { + added += 1; + if added > 256 { + return Err("selected MCP env-file aliases exceed their bound".into()); + } + mount + .reference_aliases + .push(ProviderResourceReferenceAlias { + relative_path: relative_path.to_string_lossy().into_owned(), + reference_index, + }); + } + } + } + Ok(()) +} + pub(super) fn apply_configured_resource_edits( staged_root: &Path, edits: &[StagedResourceEdit], @@ -1901,7 +2019,12 @@ mod configured_resource_tests { std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); std::fs::write( project.join("server.mjs"), - "if (process.env.MANAGED_PROJECT_ENV !== 'project-secret') process.exit(11);\nprocess.stdout.write('project cwd env loaded\\n');\n", + "import helper from './helper.mjs';\nif (helper !== 'project-secret:helper') process.exit(11);\nprocess.stdout.write('project cwd env loaded\\n');\n", + ) + .unwrap(); + std::fs::write( + project.join("helper.mjs"), + "export default `${process.env.MANAGED_PROJECT_ENV}:helper`;\n", ) .unwrap(); std::fs::write(project.join(".env"), "MANAGED_PROJECT_ENV=project-secret\n").unwrap(); @@ -1971,14 +2094,116 @@ mod configured_resource_tests { } #[test] - fn shell_wrapper_edits_map_to_a_resource_filtered_around_the_workspace() { + fn shell_wrapper_resolves_env_file_after_cd_to_script_directory_parent() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let project = owned.path().join("project"); + let script_directory = project.join("bin"); + let wrapper = script_directory.join("start"); + std::fs::create_dir_all(&home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(&script_directory).unwrap(); + std::fs::write( + &wrapper, + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")/..\ncd \"$SCRIPT_DIR\"\nexec node --env-file=.env server.mjs\n", + ) + .unwrap(); + std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap(); + std::fs::write( + project.join("server.mjs"), + "if (process.env.MANAGED_PARENT_ENV !== 'project-root') process.exit(11);\nprocess.stdout.write('parent env loaded\\n');\n", + ) + .unwrap(); + std::fs::write( + script_directory.join("server.mjs"), + "process.stdout.write('wrong script directory selected\\n');\n", + ) + .unwrap(); + std::fs::write(project.join(".env"), "MANAGED_PARENT_ENV=project-root\n").unwrap(); + std::fs::write( + script_directory.join(".env"), + "MANAGED_PARENT_ENV=script-directory-decoy\n", + ) + .unwrap(); + + let (plan, provider_wrapper) = plan_and_stage_mcp_command_for_resource_with_cwd( + owned.path(), + &home, + &workspace, + &[wrapper.to_string_lossy().into_owned()], + &wrapper, + Some(&project), + ); + let selected_env = plan[0] + .references + .selections + .iter() + .find(|selection| { + selection.canonical_source == project.join(".env").canonicalize().unwrap() + }) + .expect("relative Node env-file must resolve from the parent directory selected by the wrapper"); + let reference = owned + .path() + .join("provider/.provider-home/.freshell/references/0"); + std::fs::create_dir_all(reference.parent().unwrap()).unwrap(); + std::fs::copy(&selected_env.canonical_source, &reference).unwrap(); + let wrapper_permissions = std::fs::metadata(&provider_wrapper).unwrap().permissions(); + let wrapper_source = std::fs::read_to_string(&provider_wrapper).unwrap(); + std::fs::write( + &provider_wrapper, + wrapper_source.replace( + "/home/freshell/provider/.freshell/references/0", + &reference.to_string_lossy(), + ), + ) + .unwrap(); + std::fs::set_permissions(&provider_wrapper, wrapper_permissions).unwrap(); + let project_mount = resource_units( + &home, + &home.join(".config/opencode"), + &workspace, + &plan, + None, + ) + .unwrap() + .into_iter() + .find(|(source, _)| source == &project.canonicalize().unwrap()) + .expect("the configured MCP project cwd must be staged"); + let provider_cwd_relative = Path::new(&project_mount.1.effective_destination) + .strip_prefix(owned.path()) + .map(Path::to_path_buf) + .or_else(|_| { + Path::new(&project_mount.1.effective_destination) + .strip_prefix("/home/freshell/provider") + .map(|relative| Path::new(".provider-home").join(relative)) + }) + .expect("the configured project cwd must map into the provider fixture"); + let provider_cwd = owned.path().join("provider").join(provider_cwd_relative); + let hidden_home = owned.path().join("home-hidden"); + let hidden_project = owned.path().join("project-hidden"); + std::fs::rename(&home, &hidden_home).unwrap(); + std::fs::rename(&project, &hidden_project).unwrap(); + let output = + execute_selected_resource_with_context(&provider_wrapper, None, Some(&provider_cwd)) + .unwrap(); + std::fs::rename(&hidden_project, &project).unwrap(); + std::fs::rename(&hidden_home, &home).unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"parent env loaded\n"); + } + + #[test] + fn shell_wrapper_workspace_env_file_uses_private_cwd_alias_after_filter() { use std::os::unix::fs::PermissionsExt; let owned = tempfile::tempdir().unwrap(); let home = owned.path().join("home"); let external = owned.path().join("external"); let workspace = external.join("workspace"); - let wrapper = external.join("bin/start"); + let wrapper = workspace.join("bin/start"); std::fs::create_dir_all(wrapper.parent().unwrap()).unwrap(); std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(&home).unwrap(); @@ -2038,18 +2263,66 @@ mod configured_resource_tests { .unwrap(); let scratch = owned.path().join("scratch"); std::fs::create_dir_all(&scratch).unwrap(); + let excluded = plan + .iter() + .flat_map(|source| { + source + .references + .selections + .iter() + .map(|selection| selection.canonical_source.clone()) + }) + .collect::<Vec<_>>(); let edits = filter_workspace_from_enclosing_resource_units_with_edits( &mut units, "opencode", &workspace, &scratch, - &[], + &excluded, &plan, ) .unwrap_or_else(|error| { panic!("filtering a containing resource must preserve the original-to-stage edit mapping: {error}") }); - assert_eq!(edits.len(), 1); + assert!( + edits.is_empty(), + "the mounted workspace wrapper stays unchanged" + ); + let resource_mount = units + .iter() + .find(|(_, mount)| mount.logical_destination == external.to_string_lossy()) + .expect("the configured MCP cwd must remain a selected resource"); + assert_eq!( + resource_mount.1.reference_aliases, + vec![freshell_runtime_protocol::ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 0, + }] + ); + + let staged_root = owned.path().join("filtered-stage"); + let selections = resource_copy_selections(&units).unwrap(); + freshell_platform::provider_environment::copy_environment( + &selections, + &staged_root, + &excluded, + Default::default(), + ) + .unwrap(); + let filtered_resource = staged_root.join(&resource_mount.1.stage_relative_path); + assert!(!filtered_resource.join("workspace").exists()); + assert!(!filtered_resource.join(".env").exists()); + assert!(filtered_resource.join("server.mjs").is_file()); + + let private_reference = owned.path().join("private/references/0"); + std::fs::create_dir_all(private_reference.parent().unwrap()).unwrap(); + std::fs::copy(external.join(".env"), &private_reference).unwrap(); + std::os::unix::fs::symlink(&private_reference, filtered_resource.join(".env")).unwrap(); + let output = + execute_selected_resource_with_context(&wrapper, None, Some(&filtered_resource)) + .unwrap(); + assert!(output.status.success(), "{:?}", output.stderr); + assert_eq!(output.stdout, b"filtered-resource-secret"); } #[test] @@ -4896,6 +5169,7 @@ process.stdout.write(require('./helper.js')) effective_destination: package.to_string_lossy().into_owned(), kind: freshell_runtime_protocol::ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }; let mut units = vec![(package.clone(), mount)]; @@ -5097,6 +5371,7 @@ process.stdout.write(require('./helper.js')) effective_destination: "/tmp/external".into(), kind: freshell_runtime_protocol::ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }, )]; let capture = Capture::default(); @@ -5633,6 +5908,7 @@ process.stdout.write(require('./helper.js')) effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }; let mut current = vec![( "/synthetic/current-opencode-root".into(), @@ -5643,6 +5919,7 @@ process.stdout.write(require('./helper.js')) effective_destination: previous.effective_destination.clone(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }, )]; let retained = @@ -5678,6 +5955,7 @@ process.stdout.write(require('./helper.js')) effective_destination: destination.into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }; for (current_destination, previous_destination) in [ @@ -5727,6 +6005,7 @@ process.stdout.write(require('./helper.js')) effective_destination: destination.into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: Vec::new(), }; let mut aliases = vec![(source, mount("/workspace/current"))]; let prior = mount("/home/freshell/provider/.config/opencode"); @@ -5911,6 +6190,7 @@ pub(super) fn resource_units( .into(), kind, writable, + reference_aliases: Vec::new(), }, )); } @@ -6186,6 +6466,7 @@ pub(super) fn filter_workspace_from_enclosing_resource_units_with_edits( plan: &[PlannedSourceOccurrence], ) -> Result<Vec<StagedResourceEdit>, String> { let edits = configured_resource_edits(units, plan)?; + attach_configured_resource_reference_aliases(units, plan)?; filter_workspace_from_enclosing_resource_units( units, provider, diff --git a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs index b465f3f02..44b522404 100644 --- a/crates/freshell-server/src/managed_provider_references/opencode_resources.rs +++ b/crates/freshell-server/src/managed_provider_references/opencode_resources.rs @@ -730,7 +730,10 @@ fn direct_script_dependency_scope(command_parent: &Path, home: &Path) -> PathBuf .to_owned() } -fn shell_directory_variables(source: &str) -> std::collections::BTreeSet<String> { +fn shell_directory_variables( + source: &str, + script_parent: &Path, +) -> std::collections::BTreeMap<String, PathBuf> { source .lines() .filter_map(|line| { @@ -743,20 +746,49 @@ fn shell_directory_variables(source: &str) -> std::collections::BTreeSet<String> && name .bytes() .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_'); - (valid_name && value.contains("dirname") && value.contains("$0")) - .then(|| name.to_owned()) + if !valid_name { + return None; + } + let value = value.trim().trim_matches(['"', '\'']); + let prefix = ["$(dirname \"$0\")", "$(dirname -- \"$0\")"] + .into_iter() + .find(|prefix| value.starts_with(prefix))?; + let suffix = value.strip_prefix(prefix)?; + if suffix.contains(['$', '*', '?', ';', '&', '|', '\n', '\r']) { + return None; + } + let relative = suffix.strip_prefix('/').unwrap_or(suffix); + let relative_path = Path::new(relative); + if relative_path.is_absolute() + || relative_path.components().any(|component| { + !matches!( + component, + std::path::Component::Normal(_) + | std::path::Component::CurDir + | std::path::Component::ParentDir + ) + }) + { + return None; + } + let directory = script_parent.join(relative_path).canonicalize().ok()?; + directory.is_dir().then(|| (name.to_owned(), directory)) }) .collect() } fn shell_changes_to_script_directory<'a>( line: &'a str, - directory_variables: &std::collections::BTreeSet<String>, -) -> Option<&'a str> { + directory_variables: &std::collections::BTreeMap<String, PathBuf>, + script_parent: &Path, +) -> Option<(PathBuf, &'a str)> { const CD_TO_SCRIPT_DIRECTORY: &str = "cd \"$(dirname \"$0\")\""; let line = line.trim_start(); if let Some(remainder) = line.strip_prefix(CD_TO_SCRIPT_DIRECTORY) { - return shell_command_after_directory_change(remainder); + return Some(( + script_parent.to_owned(), + shell_command_after_directory_change(remainder)?, + )); } let remainder = line.strip_prefix("cd ")?.trim_start(); @@ -773,10 +805,10 @@ fn shell_changes_to_script_directory<'a>( .strip_prefix("${") .and_then(|value| value.strip_suffix('}')) .or_else(|| operand.strip_prefix('$'))?; - if !directory_variables.contains(variable) { - return None; - } - shell_command_after_directory_change(remainder) + Some(( + directory_variables.get(variable)?.clone(), + shell_command_after_directory_change(remainder)?, + )) } fn shell_command_after_directory_change(remainder: &str) -> Option<&str> { @@ -972,7 +1004,7 @@ fn shell_source_operand(line: &str) -> Option<String> { fn shell_source_path( script_parent: &Path, operand: &str, - directory_variables: &std::collections::BTreeSet<String>, + directory_variables: &std::collections::BTreeMap<String, PathBuf>, ) -> Result<Option<PathBuf>, &'static str> { if operand.starts_with('/') { return Ok(None); @@ -989,18 +1021,26 @@ fn shell_source_path( let Some((name, suffix)) = variable.split_once('}') else { return Ok(None); }; - if !directory_variables.contains(name) { + let base = directory_variables + .get(name) + .ok_or("direct MCP shell script has an unresolved dynamic source path")?; + relative = suffix.strip_prefix('/').unwrap_or(suffix); + if relative.is_empty() || relative.contains(['$', '*', '?']) { return Err("direct MCP shell script has an unresolved dynamic source path"); } - relative = suffix.strip_prefix('/').unwrap_or(suffix); + return Ok(Some(base.join(relative))); } else if let Some(variable) = operand.strip_prefix('$') { let (name, suffix) = variable .split_once('/') .ok_or("direct MCP shell script has an unresolved dynamic source path")?; - if !directory_variables.contains(name) { + let base = directory_variables + .get(name) + .ok_or("direct MCP shell script has an unresolved dynamic source path")?; + relative = suffix; + if relative.is_empty() || relative.contains(['$', '*', '?']) { return Err("direct MCP shell script has an unresolved dynamic source path"); } - relative = suffix; + return Ok(Some(base.join(relative))); } if relative.is_empty() || relative.contains(['$', '*', '?']) { return Err("direct MCP shell script has an unresolved dynamic source path"); @@ -1010,17 +1050,17 @@ fn shell_source_path( fn shell_directory_relative_paths( line: &str, - directory_variables: &std::collections::BTreeSet<String>, + directory_variables: &std::collections::BTreeMap<String, PathBuf>, script_parent: &Path, -) -> std::collections::BTreeSet<String> { +) -> std::collections::BTreeSet<PathBuf> { let mut result = std::collections::BTreeSet::new(); - let mut prefixes = std::collections::BTreeSet::new(); - for variable in directory_variables { - prefixes.insert(format!("${variable}/")); - prefixes.insert(format!("${{{variable}}}/")); + let mut prefixes = std::collections::BTreeMap::new(); + for (variable, directory) in directory_variables { + prefixes.insert(format!("${variable}/"), directory.clone()); + prefixes.insert(format!("${{{variable}}}/"), directory.clone()); } - prefixes.insert("$(dirname \"$0\")/".to_owned()); - for prefix in prefixes { + prefixes.insert("$(dirname \"$0\")/".to_owned(), script_parent.to_owned()); + for (prefix, directory) in prefixes { let mut search_from = 0; while search_from < line.len() { let Some(relative_start) = line[search_from..].find(&prefix) else { @@ -1032,12 +1072,13 @@ fn shell_directory_relative_paths( .find(|character: char| character.is_whitespace() || "\"'`;|&)".contains(character)) .unwrap_or(suffix.len()); let relative = &suffix[..length]; + let candidate = directory.join(relative); if !relative.is_empty() && !relative.contains(['$', '*', '?']) - && (!shell_directory_path_is_redirection_target(line, start) - || script_parent.join(relative).exists()) + && candidate.is_file() + && (!shell_directory_path_is_redirection_target(line, start) || candidate.exists()) { - result.insert(relative.to_owned()); + result.insert(candidate); } search_from = start + prefix.len() + length; } @@ -3118,15 +3159,15 @@ fn insert_shell_script_dependencies( } let source = std::fs::read_to_string(&canonical) .map_err(|_| fail("direct MCP shell script is not readable UTF-8"))?; - let directory_variables = shell_directory_variables(&source); + let script_parent = canonical + .parent() + .ok_or_else(|| fail("direct MCP shell script has no containing directory"))?; + let directory_variables = shell_directory_variables(&source, script_parent); if directory_variables.len() > 128 { return Err(fail( "direct MCP shell dependency closure exceeds its variable bound", )); } - let script_parent = canonical - .parent() - .ok_or_else(|| fail("direct MCP shell script has no containing directory"))?; let mut working_directory = Some(selection.default_working_directory.to_owned()); let mut source_offset = 0usize; for full_line in source.split_inclusive('\n') { @@ -3139,10 +3180,10 @@ fn insert_shell_script_dependencies( source_offset += full_line.len(); continue; } - let node_line = if let Some(remainder) = - shell_changes_to_script_directory(line, &directory_variables) + let node_line = if let Some((directory, remainder)) = + shell_changes_to_script_directory(line, &directory_variables, script_parent) { - working_directory = Some(script_parent.to_owned()); + working_directory = Some(directory); remainder } else { if line.trim_start().starts_with("cd ") { @@ -3152,8 +3193,10 @@ fn insert_shell_script_dependencies( }; let node_line_offset = source_offset + line.len().saturating_sub(node_line.len()); let mut candidates = std::collections::BTreeMap::new(); - let mut env_file_candidates = - std::collections::BTreeMap::<PathBuf, Vec<(String, std::ops::Range<usize>)>>::new(); + let mut env_file_candidates = std::collections::BTreeMap::< + PathBuf, + Vec<(String, std::ops::Range<usize>, Option<PathBuf>)>, + >::new(); if let Some(operand) = shell_source_operand(line) { if let Some(candidate) = shell_source_path(script_parent, &operand, &directory_variables) @@ -3166,12 +3209,13 @@ fn insert_shell_script_dependencies( let node_invocation = invokes_node .then(|| shell_node_invocation(node_line)) .flatten(); - for relative in + for candidate in shell_directory_relative_paths(line, &directory_variables, script_parent) { - let parse_as_node_script = invokes_node && is_node_script_path(&relative); + let parse_as_node_script = + invokes_node && is_node_script_path(&candidate.to_string_lossy()); candidates - .entry(script_parent.join(relative)) + .entry(candidate) .and_modify(|kind: &mut (bool, bool)| kind.1 |= parse_as_node_script) .or_insert((false, parse_as_node_script)); } @@ -3251,6 +3295,7 @@ fn insert_shell_script_dependencies( operand, node_line_offset + argument_span.start ..node_line_offset + argument_span.end, + working_directory.clone(), )); } } @@ -3285,6 +3330,12 @@ fn insert_shell_script_dependencies( )?; } if parse_as_node_script { + let dependency_scope = std::iter::once(scope) + .chain(default_working_directory_scope.as_deref()) + .chain(selection.workspace_directory) + .filter(|directory| dependency.starts_with(directory)) + .max_by_key(|directory| directory.components().count()) + .unwrap_or(scope); insert_installed_package_ancestor_resources_covered( &dependency, scope, @@ -3296,7 +3347,7 @@ fn insert_shell_script_dependencies( )?; insert_direct_script_dependencies_covered( &dependency, - scope, + dependency_scope, declaring, DirectScriptResources { paths: &mut *resources.paths, @@ -3334,15 +3385,48 @@ fn insert_shell_script_dependencies( }) { continue; } - for (expected_operand, argument_span) in arguments { + for (expected_operand, argument_span, working_directory) in arguments { + let alias = selection + .workspace_directory + .filter(|workspace| canonical.starts_with(workspace)) + .filter(|workspace| !dependency.starts_with(workspace)) + .and_then(|_| { + let directory = working_directory?.canonicalize().ok()?; + if !dependency.starts_with(&directory) { + return None; + } + let operand = Path::new(&expected_operand); + if operand.is_absolute() { + return None; + } + let mut relative_path = PathBuf::new(); + for component in operand.components() { + match component { + std::path::Component::Normal(part) => relative_path.push(part), + std::path::Component::CurDir => {} + _ => return None, + } + } + (!relative_path.as_os_str().is_empty()) + .then_some((directory, relative_path)) + }); + let rewrite = alias.map_or_else( + || NodeEnvFileRewrite::ShellScriptArgument { + script_source: canonical.clone(), + argument_span: argument_span.clone(), + expected_operand: expected_operand.clone(), + }, + |(working_directory, relative_path)| { + NodeEnvFileRewrite::ShellWorkingDirectoryAlias { + working_directory, + relative_path, + } + }, + ); resources.node_env_files.push(NodeEnvFileSelection { canonical_source: dependency.clone(), selected_alias: candidate.clone(), - rewrite: NodeEnvFileRewrite::ShellScriptArgument { - script_source: canonical.clone(), - argument_span, - expected_operand, - }, + rewrite, }); } } @@ -3592,6 +3676,10 @@ pub(super) enum NodeEnvFileRewrite { argument_span: std::ops::Range<usize>, expected_operand: String, }, + ShellWorkingDirectoryAlias { + working_directory: PathBuf, + relative_path: PathBuf, + }, } pub(super) struct NodeEnvFileSelection { diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index e4c9ecb42..e104ec66e 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -316,6 +316,9 @@ pub struct PlannedSourceOccurrence { pub configured_resource_exclusions: std::collections::BTreeSet<PathBuf>, /// Shell wrapper Node env-file operands rewritten in their staged copies. pub configured_resource_edits: Vec<ConfiguredResourceEdit>, + /// Shell wrappers in the workspace can keep their relative env-file + /// operands when the MCP cwd exposes a private reference through a link. + pub configured_resource_reference_aliases: Vec<ConfiguredResourceReferenceAlias>, /// Captured with the retained source view so later staging does not /// reinterpret a generated entry after ordinary-terminal cleanup. pub user_owns_freshell_mcp: bool, @@ -328,6 +331,13 @@ pub struct ConfiguredResourceEdit { pub replacement: String, } +pub struct ConfiguredResourceReferenceAlias { + pub source: PathBuf, + pub working_directory: PathBuf, + pub relative_path: PathBuf, + pub reference_destination: String, +} + #[derive(Default)] pub(crate) struct SourceViewBudget { views: std::collections::BTreeSet<(usize, usize)>, @@ -667,6 +677,7 @@ pub fn plan_loaded_opencode_sources_relocated( })?; let mut node_env_file_edits = Vec::with_capacity(selected_resources.node_env_files.len()); let mut configured_resource_edits = Vec::new(); + let mut configured_resource_reference_aliases = Vec::new(); for env_file in &selected_resources.node_env_files { let destination = allocate(&env_file.canonical_source).map_err(fail)?; match &env_file.rewrite { @@ -689,6 +700,15 @@ pub fn plan_loaded_opencode_sources_relocated( expected: expected_operand.clone(), replacement: destination, }), + super::opencode_resources::NodeEnvFileRewrite::ShellWorkingDirectoryAlias { + working_directory, + relative_path, + } => configured_resource_reference_aliases.push(ConfiguredResourceReferenceAlias { + source: env_file.canonical_source.clone(), + working_directory: working_directory.clone(), + relative_path: relative_path.clone(), + reference_destination: destination, + }), } } let references = super::resolve_file_references_from_view_relocated( @@ -801,6 +821,7 @@ pub fn plan_loaded_opencode_sources_relocated( configured_writable_resources: selected_resources.writable_resources, configured_resource_exclusions, configured_resource_edits, + configured_resource_reference_aliases, user_owns_freshell_mcp, }; planned diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 9cc0af1cc..6d45661ff 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1930,6 +1930,138 @@ fn workspace_suffix_under_resource(resource_root: &Path, workspace: &Path) -> Op (!suffix.as_os_str().is_empty()).then(|| suffix.to_path_buf()) } +fn install_provider_resource_reference_aliases( + context: &freshell_runtime_protocol::ProviderLaunchContext, + provider_home: &Path, +) -> Result<Vec<PathBuf>, String> { + use freshell_runtime_protocol::ProviderResourceKind; + + let mut installed = Vec::new(); + for mount in context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + { + if mount.reference_aliases.is_empty() { + continue; + } + if mount.kind != ProviderResourceKind::Directory { + return Err("provider reference aliases require a directory resource".into()); + } + let mount_relative = Path::new(&mount.provider_relative_path); + if mount_relative.is_absolute() + || mount_relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("provider reference alias mount path is unsafe".into()); + } + let mount_root = provider_home.join(mount_relative); + if !std::fs::metadata(&mount_root).is_ok_and(|metadata| metadata.is_dir()) { + return Err("provider reference alias resource directory is unavailable".into()); + } + let canonical_provider_home = std::fs::canonicalize(provider_home) + .map_err(|_| "provider reference alias provider HOME is unavailable")?; + let canonical_mount_root = std::fs::canonicalize(&mount_root) + .map_err(|_| "provider reference alias resource directory is unavailable")?; + if !canonical_mount_root.starts_with(&canonical_provider_home) { + return Err("provider reference alias resource directory escapes provider HOME".into()); + } + for alias in &mount.reference_aliases { + let relative = Path::new(&alias.relative_path); + if relative.as_os_str().is_empty() + || relative.is_absolute() + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("provider reference alias path is unsafe".into()); + } + let reference_relative = + PathBuf::from(format!(".freshell/references/{}", alias.reference_index)); + let reference = provider_home.join(&reference_relative); + if !std::fs::metadata(&reference).is_ok_and(|metadata| metadata.is_file()) { + return Err("provider reference alias target is unavailable".into()); + } + let destination = mount_root.join(relative); + let parent = destination + .parent() + .ok_or("provider reference alias has no parent directory")?; + if !std::fs::metadata(parent).is_ok_and(|metadata| metadata.is_dir()) { + return Err("provider reference alias parent directory is unavailable".into()); + } + let canonical_parent = std::fs::canonicalize(parent) + .map_err(|_| "provider reference alias parent directory is unavailable")?; + if !canonical_parent.starts_with(&canonical_mount_root) { + return Err( + "provider reference alias parent directory escapes its resource".into(), + ); + } + let resolved_reference = std::fs::canonicalize(&reference) + .map_err(|_| "provider reference alias target is unavailable")?; + match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + let link = std::fs::read_link(&destination) + .map_err(|_| "provider reference alias could not be inspected")?; + let resolved_link = if link.is_absolute() { + link + } else { + parent.join(link) + }; + if std::fs::canonicalize(resolved_link).ok().as_ref() + != Some(&resolved_reference) + { + return Err( + "provider reference alias conflicts with an existing link".into() + ); + } + installed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); + continue; + } + Ok(_) => { + return Err( + "provider reference alias would replace an existing resource".into(), + ) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("provider reference alias path could not be inspected".into()), + } + + let from_components = Path::new(&mount.provider_relative_path) + .join(relative.parent().unwrap_or_else(|| Path::new(""))) + .components() + .map(|part| match part { + std::path::Component::Normal(value) => Ok(value.to_owned()), + _ => Err("provider reference alias source path is unsafe"), + }) + .collect::<Result<Vec<_>, _>>()?; + let to_components = reference_relative + .components() + .map(|part| match part { + std::path::Component::Normal(value) => Ok(value.to_owned()), + _ => Err("provider reference alias target path is unsafe"), + }) + .collect::<Result<Vec<_>, _>>()?; + let common = from_components + .iter() + .zip(&to_components) + .take_while(|(left, right)| left == right) + .count(); + let mut link = PathBuf::new(); + for _ in common..from_components.len() { + link.push(".."); + } + for component in &to_components[common..] { + link.push(component); + } + std::os::unix::fs::symlink(link, &destination) + .map_err(|_| "provider reference alias could not be created")?; + installed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); + } + } + Ok(installed) +} + fn copy_provider_config_references_with_ownership( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -2222,10 +2354,20 @@ fn copy_provider_config_references_with_ownership( ) } .map_err(|error| error.to_string())?; + let reference_aliases = if provider_home == final_home { + install_provider_resource_reference_aliases(context, provider_home)? + } else { + Vec::new() + }; let projected = report .projected_paths .iter() .map(|path| path.to_string_lossy().into_owned()) + .chain( + reference_aliases + .iter() + .map(|path| path.to_string_lossy().into_owned()), + ) .collect::<std::collections::BTreeSet<_>>(); let mut ownership_paths = report .projected_paths @@ -2778,7 +2920,29 @@ fn refresh_provider_config_in_stage( ) .map_err(|_| "preserved writable resource metadata write failed")?; allow_provider_read_stage(stage.path(), unsafe { libc::getegid() })?; - apply_staged_provider_config(stage.path(), provider_home) + apply_staged_provider_config(stage.path(), provider_home)?; + record_provider_resource_reference_aliases(context, provider_home) +} + +fn record_provider_resource_reference_aliases( + context: &freshell_runtime_protocol::ProviderLaunchContext, + provider_home: &Path, +) -> Result<(), String> { + let aliases = install_provider_resource_reference_aliases(context, provider_home)?; + if aliases.is_empty() { + return Ok(()); + } + let mut projected = read_projection_manifest(provider_home)?; + projected.extend( + aliases + .into_iter() + .map(|path| path.to_string_lossy().into_owned()), + ); + let manifest = write_projection_manifest(provider_home, &projected)?; + set_owner(&manifest, unsafe { libc::geteuid() }, unsafe { + libc::getegid() + })?; + Ok(()) } fn refresh_provider_config_as_provider( @@ -4592,6 +4756,7 @@ mod tests { effective_destination: effective_destination.into(), kind, writable, + reference_aliases: vec![], } } @@ -4631,6 +4796,150 @@ mod tests { } } + #[cfg(unix)] + #[test] + fn provider_resource_reference_alias_links_to_private_reference_without_copying_it() { + use freshell_runtime_protocol::{ + IndexedProviderFileAlias, IndexedProviderFiles, ProviderResourceKind, + ProviderResourceReferenceAlias, + }; + + let owned = tempfile::tempdir().unwrap(); + let provider_home = owned.path().join("provider"); + let user_provider_root = owned.path().join("prepared"); + let resource_stage = user_provider_root.join(".freshell/resource-units/mcp-project"); + let resource_root = provider_home.join(".freshell/provider-roots/mcp-project"); + let workspace = owned.path().join("workspace"); + std::fs::create_dir_all(&resource_stage).unwrap(); + std::fs::create_dir_all(&provider_home).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); + std::fs::write(resource_stage.join("server.mjs"), "console.log('ready')\n").unwrap(); + std::fs::write( + provider_home.join(".freshell/references/0"), + "MANAGED_ALIAS_SECRET=private-value\n", + ) + .unwrap(); + let mut mount = opencode_resource_mount( + "mcp-project", + "/opt/mcp-project", + "/opt/mcp-project", + ProviderResourceKind::Directory, + false, + ); + mount.reference_aliases = vec![ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 0, + }]; + let mut context = opencode_context_with_mounts(vec![mount]); + let inputs = context.prepared_inputs.as_mut().unwrap(); + inputs.manifest.source_count = 1; + inputs.manifest.alias_count = 1; + inputs.indexed_files = Some(IndexedProviderFiles { + volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), + aliases: vec![IndexedProviderFileAlias { + source_id: 0, + logical_destination: "/fixture/private-reference".into(), + effective_destination: "/fixture/private-reference".into(), + }], + staged_source_paths: None, + }); + + let stage = tempfile::tempdir_in(&provider_home).unwrap(); + super::refresh_provider_config_in_stage( + &context, + "opencode", + &workspace, + &user_provider_root, + &provider_home, + stage, + ) + .unwrap(); + let alias = resource_root.join(".env"); + assert!(std::fs::symlink_metadata(&alias) + .unwrap() + .file_type() + .is_symlink()); + assert_eq!( + std::fs::read_to_string(&alias).unwrap(), + "MANAGED_ALIAS_SECRET=private-value\n" + ); + assert_eq!( + std::fs::read_link(&alias).unwrap(), + PathBuf::from("../../references/0") + ); + assert!(super::read_projection_manifest(&provider_home) + .unwrap() + .contains(".freshell/provider-roots/mcp-project/.env")); + } + + #[cfg(unix)] + #[test] + fn provider_resource_reference_alias_refuses_to_replace_resource_content() { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceReferenceAlias}; + + let owned = tempfile::tempdir().unwrap(); + let provider_home = owned.path().join("provider"); + let resource_root = provider_home.join(".freshell/provider-roots/mcp-project"); + std::fs::create_dir_all(&resource_root).unwrap(); + std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); + std::fs::write(provider_home.join(".freshell/references/0"), "secret\n").unwrap(); + std::fs::write(resource_root.join(".env"), "ordinary resource\n").unwrap(); + let mut mount = opencode_resource_mount( + "mcp-project", + "/opt/mcp-project", + "/opt/mcp-project", + ProviderResourceKind::Directory, + false, + ); + mount.reference_aliases = vec![ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 0, + }]; + let context = opencode_context_with_mounts(vec![mount]); + + let error = super::install_provider_resource_reference_aliases(&context, &provider_home) + .expect_err("an alias cannot overwrite an unrelated resource file"); + assert!(error.contains("replace an existing resource")); + assert_eq!( + std::fs::read_to_string(resource_root.join(".env")).unwrap(), + "ordinary resource\n" + ); + } + + #[cfg(unix)] + #[test] + fn provider_resource_reference_alias_cannot_escape_through_parent_symlink() { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceReferenceAlias}; + + let owned = tempfile::tempdir().unwrap(); + let provider_home = owned.path().join("provider"); + let resource_root = provider_home.join(".freshell/provider-roots/mcp-project"); + let outside = owned.path().join("outside"); + std::fs::create_dir_all(&resource_root).unwrap(); + std::fs::create_dir_all(&outside).unwrap(); + std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); + std::fs::write(provider_home.join(".freshell/references/0"), "secret\n").unwrap(); + std::os::unix::fs::symlink(&outside, resource_root.join("nested")).unwrap(); + let mut mount = opencode_resource_mount( + "mcp-project", + "/opt/mcp-project", + "/opt/mcp-project", + ProviderResourceKind::Directory, + false, + ); + mount.reference_aliases = vec![ProviderResourceReferenceAlias { + relative_path: "nested/.env".into(), + reference_index: 0, + }]; + let context = opencode_context_with_mounts(vec![mount]); + + let error = super::install_provider_resource_reference_aliases(&context, &provider_home) + .expect_err("an alias parent must remain inside its managed resource directory"); + assert!(error.contains("parent directory")); + assert!(!outside.join(".env").exists()); + } + #[test] fn provider_config_recovery_migrates_nested_writable_file_under_external_directory() { use freshell_runtime_protocol::ProviderResourceKind; @@ -5131,6 +5440,7 @@ mod tests { effective_destination: native.to_str().unwrap().into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }], }), mcp_capability: Some(capability), @@ -5251,6 +5561,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/preserved/source".into(), @@ -5259,6 +5570,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp/preserved.txt".into(), kind: ProviderResourceKind::File, writable: true, + reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/readonly/source".into(), @@ -5267,6 +5579,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp/reference.txt".into(), kind: ProviderResourceKind::File, writable: false, + reference_aliases: vec![], }, ], }), @@ -5314,6 +5627,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp/state.txt".into(), kind: ProviderResourceKind::File, writable: true, + reference_aliases: vec![], }); refresh_provider_config_references( @@ -5363,6 +5677,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp/removed.txt".into(), kind: ProviderResourceKind::File, writable: true, + reference_aliases: vec![], }); let recovery = refresh_provider_config_references( &context, "opencode", &workspace, &user_root, &home, uid, gid, @@ -5448,6 +5763,7 @@ mod tests { effective_destination: package.to_string_lossy().into_owned(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-0".into(), @@ -5456,6 +5772,7 @@ mod tests { effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }, ], }), @@ -5559,6 +5876,7 @@ mod tests { effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }], indexed_files: None, }), @@ -6720,6 +7038,7 @@ mod tests { effective_destination: external.to_string_lossy().into_owned(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }], }), mcp_capability: Some(capability), diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs index e5e81c59c..b4c1512e9 100644 --- a/crates/freshell-supervisor/src/backend/prepared_inputs.rs +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -176,6 +176,7 @@ mod tests { effective_destination: "/outside/plugin".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }], indexed_files: Some(IndexedProviderFiles { volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), @@ -243,6 +244,7 @@ mod tests { effective_destination: destination.into(), kind: ProviderResourceKind::File, writable, + reference_aliases: vec![], }); } diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index 1b2ebdd36..904c57ad3 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -737,6 +737,7 @@ mod tests { effective_destination: "/synthetic/workspace/.opencode".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }], }; let selected = selections(&mounts).unwrap(); @@ -765,6 +766,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp".into(), kind: ProviderResourceKind::Directory, writable: false, + reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-state/source".into(), @@ -773,6 +775,7 @@ mod tests { effective_destination: "/home/freshell/provider/mcp/state.json".into(), kind: ProviderResourceKind::File, writable: true, + reference_aliases: vec![], }, ], }; diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index edfd183c8..d24468919 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -156,6 +156,41 @@ process.stdin.on('data', chunk => { }) ` +const workspaceCwdAliasMcpServerSource = String.raw`import fs from 'node:fs' +import helper from './helper.mjs' +const [receiptPath, expectedNonce] = process.argv.slice(2) +if (!receiptPath || !expectedNonce) process.exit(12) +fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper, + envValueLoaded: process.env.FRESHELL_MANAGED_CWD_ALIAS_ENV === expectedNonce, + envFileOperand: process.execArgv.find(value => value.startsWith('--env-file=')), + cwd: process.cwd(), scriptPath: process.argv[1] })) +let buffered = '' +process.stdin.setEncoding('utf8') +process.stdin.on('data', chunk => { + buffered += chunk + while (true) { + const boundary = buffered.indexOf('\n') + if (boundary < 0) break + const line = buffered.slice(0, boundary) + buffered = buffered.slice(boundary + 1) + let request + try { request = JSON.parse(line) } catch { continue } + if (!request || !Object.hasOwn(request, 'id')) continue + let result + if (request.method === 'initialize') result = { protocolVersion: request.params?.protocolVersion ?? '2024-11-05', + capabilities: { tools: {} }, serverInfo: { name: 'freshell-workspace-cwd-alias-fixture', version: '1.0.0' } } + else if (request.method === 'tools/list') result = { tools: [] } + else if (request.method === 'ping') result = {} + else { + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, + error: { code: -32601, message: 'Method not found' } }) + '\n') + continue + } + process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n') + } +}) +` + function directHomeAppendMcpServerSource(outputName: string): string { const outputUrl = `new URL(${JSON.stringify(`./${outputName}`)}, import.meta.url)` return `import fs from 'node:fs' @@ -339,9 +374,10 @@ export function managedEnvironmentFixtureLayout(root: string) { const plugin = path.join(resources, 'fixture-plugin') const mcpPackage = path.join(resources, 'fixture-mcp') const directMcp = path.join(resources, 'direct-mcp') + const cwdAliasProject = path.join(resources, 'cwd-alias-project') const mcpNested = path.join(mcpPackage, 'nested') const mcpParentDependency = path.join(resources, 'node_modules/managed-fixture-mcp-dependency') - return { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } + return { workspace, resources, plugin, mcpPackage, directMcp, cwdAliasProject, mcpNested, mcpParentDependency } } export function managedHomeAppendFixturePaths(home: string, cwd: string, scenario: string) { @@ -355,13 +391,18 @@ export function managedHomeAppendFixturePaths(home: string, cwd: string, scenari async function configure(root: string, nonce: string, candidateSha: string, runId: string) { const home = path.join(root, 'home'), xdg = path.join(home, 'fixture-xdg') - const { workspace, resources, plugin, mcpPackage, directMcp, mcpNested, mcpParentDependency } = managedEnvironmentFixtureLayout(root) + const { workspace, resources, plugin, mcpPackage, directMcp, cwdAliasProject, mcpNested, mcpParentDependency } = managedEnvironmentFixtureLayout(root) await fs.mkdir(path.join(plugin, 'node_modules/managed-fixture-dependency'), { recursive: true, mode: 0o755 }) await fs.mkdir(mcpNested, { recursive: true, mode: 0o755 }) await fs.mkdir(directMcp, { recursive: true, mode: 0o755 }) + await fs.mkdir(cwdAliasProject, { recursive: true, mode: 0o755 }) + await fs.symlink(workspace, path.join(cwdAliasProject, 'workspace')) await fs.mkdir(mcpParentDependency, { recursive: true, mode: 0o755 }) const sharedEnvFile = path.join(directMcp, '.env') await fs.writeFile(sharedEnvFile, `FRESHELL_MANAGED_SHARED_ENV=${nonce}\n`, { mode: 0o600 }) + await fs.writeFile(path.join(cwdAliasProject, '.env'), `FRESHELL_MANAGED_CWD_ALIAS_ENV=${nonce}\n`, { mode: 0o600 }) + await fs.writeFile(path.join(cwdAliasProject, 'helper.mjs'), `export default ${JSON.stringify(nonce)};\n`) + await fs.writeFile(path.join(cwdAliasProject, 'server.mjs'), workspaceCwdAliasMcpServerSource) const directMcpDependency = path.join(directMcp, 'node_modules/managed-direct-fixture-dependency') await fs.mkdir(directMcpDependency, { recursive: true, mode: 0o755 }) await fs.mkdir(path.join(xdg, 'opencode'), { recursive: true, mode: 0o755 }) @@ -436,6 +477,12 @@ export default {id:'managed-environment-tui',async tui(api,options){ await fs.chmod(path.join(cwd, '.managed-fixture'), 0o777) await fs.writeFile(path.join(cwd, 'rules.txt'), nonce, { mode: 0o600 }) const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') + const cwdAliasWrapper = path.join(cwd, '.managed-fixture/mcp-cwd-alias/start') + const cwdAliasReceipt = path.join(cwd, '.managed-fixture/mcp-cwd-alias-receipt.json') + await fs.mkdir(path.dirname(cwdAliasWrapper), { recursive: true, mode: 0o755 }) + await fs.writeFile(cwdAliasWrapper, + '#!/bin/sh\nexec node --env-file=.env server.mjs "$@"\n', { mode: 0o751 }) + await fs.chmod(cwdAliasWrapper, 0o751) const homeAppend = managedHomeAppendFixturePaths(home, cwd, scenario) await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) await fs.writeFile(homeAppend.output, `initial-${nonce}`) @@ -454,6 +501,12 @@ export default {id:'managed-environment-tui',async tui(api,options){ command: [directCommand, path.join(cwd, '.managed-fixture/direct-mcp-receipt.json'), nonce], enabled: true, }, + workspace_cwd_alias_fixture: { + type: 'local', + command: [cwdAliasWrapper, cwdAliasReceipt, nonce], + cwd: cwdAliasProject, + enabled: true, + }, direct_home_append_fixture: { type: 'local', command: ['node', `--env-file=${sharedEnvFile}`, homeAppend.script, homeAppend.receipt, nonce], @@ -462,7 +515,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ }, }) } - return { home, xdg, workspace, command, mcpPackage, mcpNested } + return { home, xdg, workspace, command, mcpPackage, mcpNested, cwdAliasProject } } async function socketCreate(rig: ManagedRuntimeBrowserRig, requestId: string, mode: 'shell' | 'opencode', cwd: string, @@ -1383,12 +1436,21 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightHomeAppend.envFileOperandRewritten, true) assert.equal(preflightHomeAppend.envValueLoaded, true) assert(preflightHomeAppend.scriptPath.endsWith('/direct-home-preflight.mjs')) + const preflightCwdAliasMcp = await poll('OpenCode starts a workspace wrapper with its external MCP cwd env file', async () => + readJson(path.join(selected.workspace, 'preflight/.managed-fixture/mcp-cwd-alias-receipt.json'))) + assert.equal(preflightCwdAliasMcp.expectedNonce, nonce) + assert.equal(preflightCwdAliasMcp.helper, nonce) + assert.equal(preflightCwdAliasMcp.envValueLoaded, true) + assert.equal(preflightCwdAliasMcp.envFileOperand, '--env-file=.env', 'the workspace wrapper must retain its relative operand') + assert.equal(preflightCwdAliasMcp.cwd, selected.cwdAliasProject) + assert(preflightCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) const preflightView = await poll('genuine managed preflight native identity', () => observeManagedEnvironmentNativeIdentity(rig!, created.terminalId, preflightChild.nativeSessionId)) assert((await terminalDirectory(rig)).some(row => row.terminalId === created.terminalId)) assert.deepEqual(await webChildren(rig), [], 'Managed child must belong to the disposable session host, not the host web process') proof.preflight = { terminalId: created.terminalId, soulId: preflightView.soulId, incarnationId: preflightView.incarnationId, stage: await stageProof(rig, preflightView, nonce), child: preflightChild, mcpServer: preflightMcp, + workspaceCwdAliasMcp: preflightCwdAliasMcp, directMcpServer: preflightDirectMcp, directHomeAppendMcp: preflightHomeAppend } const toolHome = path.join(root, 'tools') await fs.mkdir(toolHome, { mode: 0o700 }) @@ -1542,9 +1604,18 @@ export async function runManagedEnvironmentTarget() { assert.equal(directHomeAppendMcp.envFileOperandRewritten, true) assert.equal(directHomeAppendMcp.envValueLoaded, true) assert(directHomeAppendMcp.scriptPath.endsWith('/direct-home-healthy.mjs')) + const workspaceCwdAliasMcp = await poll('Cloud OpenCode starts a workspace wrapper with its external MCP cwd env file', async () => + readJson(path.join(cwd, '.managed-fixture/mcp-cwd-alias-receipt.json'))) + assert.equal(workspaceCwdAliasMcp.expectedNonce, nonce) + assert.equal(workspaceCwdAliasMcp.helper, nonce) + assert.equal(workspaceCwdAliasMcp.envValueLoaded, true) + assert.equal(workspaceCwdAliasMcp.envFileOperand, '--env-file=.env', 'the workspace wrapper must retain its relative operand') + assert.equal(workspaceCwdAliasMcp.cwd, selected.cwdAliasProject) + assert(workspaceCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) const stage = await stageProof(rig!, view, nonce) const receipt = { ...fixture, terminalId: view.terminalId, soulId: view.soulId, incarnationId: view.incarnationId, - nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer, directHomeAppendMcp } + nativeSessionId: view.nativeSessionId, stage, child, mcpServer, directMcpServer, directHomeAppendMcp, + workspaceCwdAliasMcp } await writeJson(path.join(cwd, '.managed-fixture/target-receipt.json'), receipt) proof.healthy = receipt }, From b264b9deca159c5c4daa8a2d567f99aa0a15571b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 12:45:34 -0700 Subject: [PATCH 439/493] fix(managed): retire stale OpenCode env aliases --- .../src/provider_inputs.rs | 55 +++- .../managed_mcp_capability/amplifier_stage.rs | 1 + .../managed_mcp_capability/ordinary_stage.rs | 1 + .../managed_mcp_capability/source_stage.rs | 48 ++- crates/freshell-session-host/src/main.rs | 285 ++++++++++++++---- .../src/backend/prepared_inputs.rs | 2 + .../src/backend/provider_mountpoints.rs | 3 + 7 files changed, 335 insertions(+), 60 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/provider_inputs.rs b/crates/freshell-runtime-protocol/src/provider_inputs.rs index 88119a3ea..53b934bbc 100644 --- a/crates/freshell-runtime-protocol/src/provider_inputs.rs +++ b/crates/freshell-runtime-protocol/src/provider_inputs.rs @@ -182,6 +182,9 @@ pub struct ProviderResourceMount { /// resource tree has been copied into the provider HOME. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub reference_aliases: Vec<ProviderResourceReferenceAlias>, + /// Prior private-reference links to remove from this retained resource. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub retired_reference_aliases: Vec<String>, } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -337,11 +340,15 @@ impl PreparedProviderInputs { let file = mount.kind == ProviderResourceKind::File; total_resource_reference_aliases = total_resource_reference_aliases .checked_add(mount.reference_aliases.len()) + .and_then(|count| count.checked_add(mount.retired_reference_aliases.len())) .filter(|count| *count <= 256) .ok_or_else(|| invalid("resource reference alias count"))?; if (mount.writable && !file) - || (!mount.reference_aliases.is_empty() && file) + || ((!mount.reference_aliases.is_empty() + || !mount.retired_reference_aliases.is_empty()) + && file) || mount.reference_aliases.len() > 256 + || mount.retired_reference_aliases.len() > 256 || !unit_path(&mount.stage_relative_path, ".freshell/resource-units", file) || !unit_path( &mount.provider_relative_path, @@ -389,6 +396,38 @@ impl PreparedProviderInputs { return Err(invalid("overlapping resource reference aliases")); } } + let mut retired_aliases = std::collections::BTreeSet::new(); + for alias in &mount.retired_reference_aliases { + let relative = Path::new(alias); + if alias.len() > 64 * 1024 + || alias.is_empty() + || relative.is_absolute() + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + || !retired_aliases.insert(alias.as_str()) + { + return Err(invalid("retired resource reference alias")); + } + } + for path in &retired_aliases { + let path = Path::new(path); + if retired_aliases + .iter() + .any(|other| Path::new(other) != path && Path::new(other).starts_with(path)) + { + return Err(invalid("overlapping retired resource reference aliases")); + } + } + if aliases.keys().any(|live| { + retired_aliases.iter().any(|retired| { + let live = Path::new(live); + let retired = Path::new(retired); + live != retired && (live.starts_with(retired) || retired.starts_with(live)) + }) + }) { + return Err(invalid("overlapping live and retired resource aliases")); + } } if let Some(files) = &self.indexed_files { if !files @@ -807,12 +846,25 @@ mod tests { relative_path: ".env".into(), reference_index: 1, }], + retired_reference_aliases: vec![], }); value.validate(Some(&capability())).unwrap(); let wire = serde_json::to_vec(&value).unwrap(); let received: PreparedProviderInputs = serde_json::from_slice(&wire).unwrap(); assert_eq!(received, value); + let mut replaced_alias = value.clone(); + replaced_alias.resource_mounts[0].retired_reference_aliases = vec![".env".into()]; + replaced_alias.validate(Some(&capability())).unwrap(); + let mut unsafe_retired_alias = value.clone(); + unsafe_retired_alias.resource_mounts[0].retired_reference_aliases = + vec!["nested/../../outside".into()]; + assert!(unsafe_retired_alias.validate(Some(&capability())).is_err()); + let mut overlapping_aliases = value.clone(); + overlapping_aliases.resource_mounts[0].retired_reference_aliases = + vec![".env/child".into()]; + assert!(overlapping_aliases.validate(Some(&capability())).is_err()); + for unsafe_path in ["../outside", "nested/../../outside", "/absolute", ""] { let mut invalid = value.clone(); invalid.resource_mounts[0].reference_aliases[0].relative_path = unsafe_path.into(); @@ -858,6 +910,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }); assert!( value.validate(Some(&capability())).is_err(), diff --git a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs index e8fcde128..4bdaaa8c0 100644 --- a/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/amplifier_stage.rs @@ -144,6 +144,7 @@ pub(super) fn prepare( kind: ProviderResourceKind::Directory, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }); Ok(prepared) } diff --git a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs index 0341749d5..abfdf8116 100644 --- a/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/ordinary_stage.rs @@ -145,6 +145,7 @@ pub(super) fn prepare_selected<M: serde::Serialize>( kind, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }); } } diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 6c72c4d45..d8d32c794 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -5170,6 +5170,7 @@ process.stdout.write(require('./helper.js')) kind: freshell_runtime_protocol::ProviderResourceKind::Directory, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }; let mut units = vec![(package.clone(), mount)]; @@ -5372,6 +5373,7 @@ process.stdout.write(require('./helper.js')) kind: freshell_runtime_protocol::ProviderResourceKind::Directory, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }, )]; let capture = Capture::default(); @@ -5899,7 +5901,9 @@ process.stdout.write(require('./helper.js')) #[test] fn recovery_reuses_matching_home_mount_and_retains_dropped_home_directories() { - use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; + use freshell_runtime_protocol::{ + ProviderResourceKind, ProviderResourceMount, ProviderResourceReferenceAlias, + }; let previous = ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-legacy".into(), @@ -5908,7 +5912,11 @@ process.stdout.write(require('./helper.js')) effective_destination: "/home/freshell/provider/.config/opencode".into(), kind: ProviderResourceKind::Directory, writable: false, - reference_aliases: Vec::new(), + reference_aliases: vec![ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 0, + }], + retired_reference_aliases: Vec::new(), }; let mut current = vec![( "/synthetic/current-opencode-root".into(), @@ -5920,6 +5928,7 @@ process.stdout.write(require('./helper.js')) kind: ProviderResourceKind::Directory, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }, )]; let retained = @@ -5932,6 +5941,8 @@ process.stdout.write(require('./helper.js')) current[0].1.provider_relative_path, previous.provider_relative_path ); + assert_eq!(current[0].1.reference_aliases, Vec::new()); + assert_eq!(current[0].1.retired_reference_aliases, vec![".env"]); assert!( retained.is_empty(), "the current alias already retains this root" @@ -5941,7 +5952,12 @@ process.stdout.write(require('./helper.js')) let retained = merge_recovery_resource_mounts(&mut no_current_roots, std::slice::from_ref(&previous)) .unwrap(); - assert_eq!(retained, vec![previous]); + assert_eq!(retained.len(), 1); + assert!( + retained[0].reference_aliases.is_empty(), + "a retained resource cannot keep a previous generation's private reference index" + ); + assert_eq!(retained[0].retired_reference_aliases, vec![".env"]); } #[test] @@ -5956,6 +5972,7 @@ process.stdout.write(require('./helper.js')) kind: ProviderResourceKind::Directory, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }; for (current_destination, previous_destination) in [ @@ -6006,6 +6023,7 @@ process.stdout.write(require('./helper.js')) kind: ProviderResourceKind::Directory, writable: false, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }; let mut aliases = vec![(source, mount("/workspace/current"))]; let prior = mount("/home/freshell/provider/.config/opencode"); @@ -6191,6 +6209,7 @@ pub(super) fn resource_units( kind, writable, reference_aliases: Vec::new(), + retired_reference_aliases: vec![], }, )); } @@ -6251,6 +6270,17 @@ pub(super) fn merge_recovery_resource_mounts( } mount.stage_relative_path = prior.stage_relative_path.clone(); mount.provider_relative_path = prior.provider_relative_path.clone(); + mount + .retired_reference_aliases + .extend(prior.retired_reference_aliases.iter().cloned()); + mount.retired_reference_aliases.extend( + prior + .reference_aliases + .iter() + .map(|alias| alias.relative_path.clone()), + ); + mount.retired_reference_aliases.sort(); + mount.retired_reference_aliases.dedup(); } } @@ -6274,7 +6304,17 @@ pub(super) fn merge_recovery_resource_mounts( ); } if retained_destinations.insert(prior.effective_destination.clone()) { - retained.push(prior.clone()); + let mut retained_mount = prior.clone(); + retained_mount.retired_reference_aliases.extend( + prior + .reference_aliases + .iter() + .map(|alias| alias.relative_path.clone()), + ); + retained_mount.reference_aliases.clear(); + retained_mount.retired_reference_aliases.sort(); + retained_mount.retired_reference_aliases.dedup(); + retained.push(retained_mount); } } let merged = current diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 6d45661ff..ffddd5a46 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -1930,19 +1930,70 @@ fn workspace_suffix_under_resource(resource_root: &Path, workspace: &Path) -> Op (!suffix.as_os_str().is_empty()).then(|| suffix.to_path_buf()) } +fn managed_provider_reference_alias_index( + link: &Path, + destination: &Path, + provider_home: &Path, +) -> Option<u16> { + fn normalized(path: &Path) -> Option<PathBuf> { + if !path.is_absolute() { + return None; + } + let mut result = PathBuf::new(); + for component in path.components() { + match component { + std::path::Component::RootDir => result.push("/"), + std::path::Component::Normal(part) => result.push(part), + std::path::Component::CurDir => {} + std::path::Component::ParentDir => { + if !result.pop() || result.as_os_str().is_empty() { + return None; + } + } + std::path::Component::Prefix(_) => return None, + } + } + Some(result) + } + + fn index_under(path: &Path, root: &Path) -> Option<u16> { + let suffix = path.strip_prefix(root).ok()?; + let mut components = suffix.components(); + let index = match components.next()? { + std::path::Component::Normal(value) => value.to_str()?.parse().ok()?, + _ => return None, + }; + components.next().is_none().then_some(index) + } + + let target = if link.is_absolute() { + normalized(link)? + } else { + let parent = std::fs::canonicalize(destination.parent()?).ok()?; + normalized(&parent.join(link))? + }; + let runtime_references = Path::new("/home/freshell/provider/.freshell/references"); + if let Some(index) = index_under(&target, runtime_references) { + return Some(index); + } + let provider_home = std::fs::canonicalize(provider_home).ok()?; + index_under(&target, &provider_home.join(".freshell/references")) +} + fn install_provider_resource_reference_aliases( context: &freshell_runtime_protocol::ProviderLaunchContext, provider_home: &Path, -) -> Result<Vec<PathBuf>, String> { +) -> Result<(Vec<PathBuf>, Vec<PathBuf>), String> { use freshell_runtime_protocol::ProviderResourceKind; let mut installed = Vec::new(); + let mut removed = Vec::new(); for mount in context .prepared_inputs .iter() .flat_map(|inputs| &inputs.resource_mounts) { - if mount.reference_aliases.is_empty() { + if mount.reference_aliases.is_empty() && mount.retired_reference_aliases.is_empty() { continue; } if mount.kind != ProviderResourceKind::Directory { @@ -1967,6 +2018,55 @@ fn install_provider_resource_reference_aliases( if !canonical_mount_root.starts_with(&canonical_provider_home) { return Err("provider reference alias resource directory escapes provider HOME".into()); } + for retired in &mount.retired_reference_aliases { + let relative = Path::new(retired); + if relative.as_os_str().is_empty() + || relative.is_absolute() + || relative + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("retired provider reference alias path is unsafe".into()); + } + let destination = mount_root.join(relative); + let parent = destination + .parent() + .ok_or("retired provider reference alias has no parent directory")?; + let canonical_parent = match std::fs::canonicalize(parent) { + Ok(parent) => parent, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => { + return Err( + "retired provider reference alias parent directory is unavailable".into(), + ) + } + }; + if !canonical_parent.starts_with(&canonical_mount_root) { + return Err( + "retired provider reference alias parent directory escapes its resource".into(), + ); + } + match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + let link = std::fs::read_link(&destination) + .map_err(|_| "retired provider reference alias could not be inspected")?; + if managed_provider_reference_alias_index(&link, &destination, provider_home) + .is_some() + { + std::fs::remove_file(&destination) + .map_err(|_| "retired provider reference alias could not be removed")?; + removed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); + } + } + Ok(_) => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => { + return Err( + "retired provider reference alias path could not be inspected".into(), + ) + } + } + } for alias in &mount.reference_aliases { let relative = Path::new(&alias.relative_path); if relative.as_os_str().is_empty() @@ -1997,26 +2097,29 @@ fn install_provider_resource_reference_aliases( "provider reference alias parent directory escapes its resource".into(), ); } - let resolved_reference = std::fs::canonicalize(&reference) - .map_err(|_| "provider reference alias target is unavailable")?; + let runtime_reference = PathBuf::from(format!( + "/home/freshell/provider/.freshell/references/{}", + alias.reference_index + )); match std::fs::symlink_metadata(&destination) { Ok(metadata) if metadata.file_type().is_symlink() => { let link = std::fs::read_link(&destination) .map_err(|_| "provider reference alias could not be inspected")?; - let resolved_link = if link.is_absolute() { - link - } else { - parent.join(link) - }; - if std::fs::canonicalize(resolved_link).ok().as_ref() - != Some(&resolved_reference) + if link == runtime_reference { + installed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); + continue; + } + if managed_provider_reference_alias_index(&link, &destination, provider_home) + .is_some() { + std::fs::remove_file(&destination) + .map_err(|_| "provider reference alias could not be replaced")?; + removed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); + } else { return Err( "provider reference alias conflicts with an existing link".into() ); } - installed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); - continue; } Ok(_) => { return Err( @@ -2027,39 +2130,12 @@ fn install_provider_resource_reference_aliases( Err(_) => return Err("provider reference alias path could not be inspected".into()), } - let from_components = Path::new(&mount.provider_relative_path) - .join(relative.parent().unwrap_or_else(|| Path::new(""))) - .components() - .map(|part| match part { - std::path::Component::Normal(value) => Ok(value.to_owned()), - _ => Err("provider reference alias source path is unsafe"), - }) - .collect::<Result<Vec<_>, _>>()?; - let to_components = reference_relative - .components() - .map(|part| match part { - std::path::Component::Normal(value) => Ok(value.to_owned()), - _ => Err("provider reference alias target path is unsafe"), - }) - .collect::<Result<Vec<_>, _>>()?; - let common = from_components - .iter() - .zip(&to_components) - .take_while(|(left, right)| left == right) - .count(); - let mut link = PathBuf::new(); - for _ in common..from_components.len() { - link.push(".."); - } - for component in &to_components[common..] { - link.push(component); - } - std::os::unix::fs::symlink(link, &destination) + std::os::unix::fs::symlink(&runtime_reference, &destination) .map_err(|_| "provider reference alias could not be created")?; installed.push(PathBuf::from(&mount.provider_relative_path).join(relative)); } } - Ok(installed) + Ok((installed, removed)) } fn copy_provider_config_references_with_ownership( @@ -2354,21 +2430,24 @@ fn copy_provider_config_references_with_ownership( ) } .map_err(|error| error.to_string())?; - let reference_aliases = if provider_home == final_home { + let (reference_aliases, removed_reference_aliases) = if provider_home == final_home { install_provider_resource_reference_aliases(context, provider_home)? } else { - Vec::new() + (Vec::new(), Vec::new()) }; - let projected = report + let mut projected = report .projected_paths .iter() .map(|path| path.to_string_lossy().into_owned()) - .chain( - reference_aliases - .iter() - .map(|path| path.to_string_lossy().into_owned()), - ) .collect::<std::collections::BTreeSet<_>>(); + for path in removed_reference_aliases { + projected.remove(&path.to_string_lossy().into_owned()); + } + projected.extend( + reference_aliases + .iter() + .map(|path| path.to_string_lossy().into_owned()), + ); let mut ownership_paths = report .projected_paths .into_iter() @@ -2928,11 +3007,15 @@ fn record_provider_resource_reference_aliases( context: &freshell_runtime_protocol::ProviderLaunchContext, provider_home: &Path, ) -> Result<(), String> { - let aliases = install_provider_resource_reference_aliases(context, provider_home)?; - if aliases.is_empty() { + let (aliases, removed_aliases) = + install_provider_resource_reference_aliases(context, provider_home)?; + if aliases.is_empty() && removed_aliases.is_empty() { return Ok(()); } let mut projected = read_projection_manifest(provider_home)?; + for path in removed_aliases { + projected.remove(&path.to_string_lossy().into_owned()); + } projected.extend( aliases .into_iter() @@ -4757,6 +4840,7 @@ mod tests { kind, writable, reference_aliases: vec![], + retired_reference_aliases: vec![], } } @@ -4861,18 +4945,93 @@ mod tests { .file_type() .is_symlink()); assert_eq!( - std::fs::read_to_string(&alias).unwrap(), + std::fs::read_to_string(provider_home.join(".freshell/references/0")).unwrap(), "MANAGED_ALIAS_SECRET=private-value\n" ); assert_eq!( std::fs::read_link(&alias).unwrap(), - PathBuf::from("../../references/0") + PathBuf::from("/home/freshell/provider/.freshell/references/0") ); assert!(super::read_projection_manifest(&provider_home) .unwrap() .contains(".freshell/provider-roots/mcp-project/.env")); } + #[cfg(unix)] + #[test] + fn provider_resource_reference_alias_rebinds_a_reused_reference_index() { + use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceReferenceAlias}; + + let owned = tempfile::tempdir().unwrap(); + let provider_home = owned.path().join("provider"); + let resource_root = provider_home.join(".freshell/provider-roots/mcp-project"); + std::fs::create_dir_all(&resource_root).unwrap(); + std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); + std::fs::write(provider_home.join(".freshell/references/0"), "old secret\n").unwrap(); + std::fs::write(provider_home.join(".freshell/references/1"), "new secret\n").unwrap(); + let mut mount = opencode_resource_mount( + "mcp-project", + "/opt/mcp-project", + "/opt/mcp-project", + ProviderResourceKind::Directory, + false, + ); + mount.reference_aliases = vec![ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 0, + }]; + let first_context = opencode_context_with_mounts(vec![mount.clone()]); + super::install_provider_resource_reference_aliases(&first_context, &provider_home).unwrap(); + + mount.reference_aliases[0].reference_index = 1; + let second_context = opencode_context_with_mounts(vec![mount]); + super::install_provider_resource_reference_aliases(&second_context, &provider_home) + .expect("an old managed reference alias must be replaceable when its index changes"); + + assert_eq!( + std::fs::read_link(resource_root.join(".env")).unwrap(), + PathBuf::from("/home/freshell/provider/.freshell/references/1") + ); + } + + #[cfg(unix)] + #[test] + fn provider_resource_reference_alias_retirement_removes_stale_managed_link() { + use freshell_runtime_protocol::ProviderResourceKind; + + let owned = tempfile::tempdir().unwrap(); + let provider_home = owned.path().join("provider"); + let resource_root = provider_home.join(".freshell/provider-roots/mcp-project"); + std::fs::create_dir_all(&resource_root).unwrap(); + let alias = resource_root.join(".env"); + std::os::unix::fs::symlink("/home/freshell/provider/.freshell/references/7", &alias) + .unwrap(); + let relative_alias = ".freshell/provider-roots/mcp-project/.env".to_owned(); + super::write_projection_manifest( + &provider_home, + &std::collections::BTreeSet::from([relative_alias.clone()]), + ) + .unwrap(); + let mut mount = opencode_resource_mount( + "mcp-project", + "/opt/mcp-project", + "/opt/mcp-project", + ProviderResourceKind::Directory, + false, + ); + mount.retired_reference_aliases = vec![".env".into()]; + let context = opencode_context_with_mounts(vec![mount]); + + super::record_provider_resource_reference_aliases(&context, &provider_home).unwrap(); + + assert!( + std::fs::symlink_metadata(&alias).unwrap_err().kind() == std::io::ErrorKind::NotFound + ); + assert!(!super::read_projection_manifest(&provider_home) + .unwrap() + .contains(&relative_alias)); + } + #[cfg(unix)] #[test] fn provider_resource_reference_alias_refuses_to_replace_resource_content() { @@ -4921,6 +5080,11 @@ mod tests { std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); std::fs::write(provider_home.join(".freshell/references/0"), "secret\n").unwrap(); std::os::unix::fs::symlink(&outside, resource_root.join("nested")).unwrap(); + std::os::unix::fs::symlink( + "/home/freshell/provider/.freshell/references/0", + outside.join(".env"), + ) + .unwrap(); let mut mount = opencode_resource_mount( "mcp-project", "/opt/mcp-project", @@ -4932,12 +5096,13 @@ mod tests { relative_path: "nested/.env".into(), reference_index: 0, }]; + mount.retired_reference_aliases = vec!["nested/.env".into()]; let context = opencode_context_with_mounts(vec![mount]); let error = super::install_provider_resource_reference_aliases(&context, &provider_home) .expect_err("an alias parent must remain inside its managed resource directory"); assert!(error.contains("parent directory")); - assert!(!outside.join(".env").exists()); + assert!(std::fs::symlink_metadata(outside.join(".env")).is_ok()); } #[test] @@ -5441,6 +5606,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }], }), mcp_capability: Some(capability), @@ -5562,6 +5728,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/preserved/source".into(), @@ -5571,6 +5738,7 @@ mod tests { kind: ProviderResourceKind::File, writable: true, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/readonly/source".into(), @@ -5580,6 +5748,7 @@ mod tests { kind: ProviderResourceKind::File, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ], }), @@ -5628,6 +5797,7 @@ mod tests { kind: ProviderResourceKind::File, writable: true, reference_aliases: vec![], + retired_reference_aliases: vec![], }); refresh_provider_config_references( @@ -5678,6 +5848,7 @@ mod tests { kind: ProviderResourceKind::File, writable: true, reference_aliases: vec![], + retired_reference_aliases: vec![], }); let recovery = refresh_provider_config_references( &context, "opencode", &workspace, &user_root, &home, uid, gid, @@ -5764,6 +5935,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-0".into(), @@ -5773,6 +5945,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ], }), @@ -5877,6 +6050,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }], indexed_files: None, }), @@ -7039,6 +7213,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }], }), mcp_capability: Some(capability), diff --git a/crates/freshell-supervisor/src/backend/prepared_inputs.rs b/crates/freshell-supervisor/src/backend/prepared_inputs.rs index b4c1512e9..70a10cbdc 100644 --- a/crates/freshell-supervisor/src/backend/prepared_inputs.rs +++ b/crates/freshell-supervisor/src/backend/prepared_inputs.rs @@ -177,6 +177,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }], indexed_files: Some(IndexedProviderFiles { volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), @@ -245,6 +246,7 @@ mod tests { kind: ProviderResourceKind::File, writable, reference_aliases: vec![], + retired_reference_aliases: vec![], }); } diff --git a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs index 904c57ad3..2ff4828cd 100644 --- a/crates/freshell-supervisor/src/backend/provider_mountpoints.rs +++ b/crates/freshell-supervisor/src/backend/provider_mountpoints.rs @@ -738,6 +738,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }], }; let selected = selections(&mounts).unwrap(); @@ -767,6 +768,7 @@ mod tests { kind: ProviderResourceKind::Directory, writable: false, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ProviderResourceMount { stage_relative_path: ".freshell/resource-units/root-state/source".into(), @@ -776,6 +778,7 @@ mod tests { kind: ProviderResourceKind::File, writable: true, reference_aliases: vec![], + retired_reference_aliases: vec![], }, ], }; From e3cbc7eb2d3d33ea97cba3525aefaaaed22dcb6a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:03:14 -0700 Subject: [PATCH 440/493] fix(managed): keep recovery env aliases bounded --- .../managed_mcp_capability/source_stage.rs | 112 +++++++++++++++--- .../managed-provider-environment-target.ts | 13 +- 2 files changed, 106 insertions(+), 19 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index d8d32c794..9dd6285b3 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -5916,7 +5916,7 @@ process.stdout.write(require('./helper.js')) relative_path: ".env".into(), reference_index: 0, }], - retired_reference_aliases: Vec::new(), + retired_reference_aliases: vec![".previously-retired-env".into()], }; let mut current = vec![( "/synthetic/current-opencode-root".into(), @@ -5960,6 +5960,83 @@ process.stdout.write(require('./helper.js')) assert_eq!(retained[0].retired_reference_aliases, vec![".env"]); } + #[test] + fn recovery_keeps_live_reference_aliases_and_expires_old_retirement_instructions() { + use freshell_runtime_protocol::{ + ProviderResourceKind, ProviderResourceMount, ProviderResourceReferenceAlias, + }; + + let aliases = |index_offset: u16| { + (0u16..129) + .map(|index| ProviderResourceReferenceAlias { + relative_path: format!(".env-{index}"), + reference_index: index + index_offset, + }) + .collect::<Vec<_>>() + }; + let mount = |aliases, retired_aliases| ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/opencode-home".into(), + provider_relative_path: ".freshell/provider-roots/opencode-home".into(), + logical_destination: "/home/freshell/provider/.config/opencode".into(), + effective_destination: "/home/freshell/provider/.config/opencode".into(), + kind: ProviderResourceKind::Directory, + writable: false, + reference_aliases: aliases, + retired_reference_aliases: retired_aliases, + }; + + let previous = mount(aliases(0), vec![".expired-env".into()]); + let mut current = vec![( + "/synthetic/current-opencode-root".into(), + mount(aliases(1), Vec::new()), + )]; + + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)).unwrap(); + + assert_eq!(current[0].1.reference_aliases, aliases(1)); + assert!( + current[0].1.retired_reference_aliases.is_empty(), + "live aliases must not also be scheduled for retirement, and old retirement instructions expire" + ); + } + + #[test] + fn recovery_retires_only_previously_live_aliases_that_are_now_absent() { + use freshell_runtime_protocol::{ + ProviderResourceKind, ProviderResourceMount, ProviderResourceReferenceAlias, + }; + + let mount = |aliases, retired_aliases| ProviderResourceMount { + stage_relative_path: ".freshell/resource-units/opencode-home".into(), + provider_relative_path: ".freshell/provider-roots/opencode-home".into(), + logical_destination: "/home/freshell/provider/.config/opencode".into(), + effective_destination: "/home/freshell/provider/.config/opencode".into(), + kind: ProviderResourceKind::Directory, + writable: false, + reference_aliases: aliases, + retired_reference_aliases: retired_aliases, + }; + let previous = mount( + vec![ProviderResourceReferenceAlias { + relative_path: ".removed-env".into(), + reference_index: 0, + }], + vec![".already-removed-env".into()], + ); + let mut current = vec![( + "/synthetic/current-opencode-root".into(), + mount(Vec::new(), Vec::new()), + )]; + + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)).unwrap(); + + assert_eq!( + current[0].1.retired_reference_aliases, + vec![".removed-env"], + "only live aliases from the previous generation need cleanup" + ); + } + #[test] fn recovery_rejects_nested_retained_and_current_directory_mounts() { use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; @@ -6257,6 +6334,8 @@ pub(super) fn merge_recovery_resource_mounts( }) } + // Previous-generation retirement entries have already served their cleanup + // purpose. Recompute this generation's list from the still-live prior links. for (_, mount) in current.iter_mut() { if let Some(prior) = previous .iter() @@ -6270,15 +6349,17 @@ pub(super) fn merge_recovery_resource_mounts( } mount.stage_relative_path = prior.stage_relative_path.clone(); mount.provider_relative_path = prior.provider_relative_path.clone(); - mount - .retired_reference_aliases - .extend(prior.retired_reference_aliases.iter().cloned()); - mount.retired_reference_aliases.extend( - prior - .reference_aliases - .iter() - .map(|alias| alias.relative_path.clone()), - ); + let live_aliases = mount + .reference_aliases + .iter() + .map(|alias| alias.relative_path.as_str()) + .collect::<std::collections::HashSet<_>>(); + mount.retired_reference_aliases = prior + .reference_aliases + .iter() + .filter(|alias| !live_aliases.contains(alias.relative_path.as_str())) + .map(|alias| alias.relative_path.clone()) + .collect(); mount.retired_reference_aliases.sort(); mount.retired_reference_aliases.dedup(); } @@ -6305,12 +6386,11 @@ pub(super) fn merge_recovery_resource_mounts( } if retained_destinations.insert(prior.effective_destination.clone()) { let mut retained_mount = prior.clone(); - retained_mount.retired_reference_aliases.extend( - prior - .reference_aliases - .iter() - .map(|alias| alias.relative_path.clone()), - ); + retained_mount.retired_reference_aliases = prior + .reference_aliases + .iter() + .map(|alias| alias.relative_path.clone()) + .collect(); retained_mount.reference_aliases.clear(); retained_mount.retired_reference_aliases.sort(); retained_mount.retired_reference_aliases.dedup(); diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index d24468919..2271ba53b 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -24,6 +24,7 @@ const cloudflaredVersion = '2026.10.0' const cloudflaredSha = 'd33ff2d14475178d2012c2c56beba87389ac5ded27649519f198a7d3134a99db' const scenarios = ['healthy', 'missing-reference', 'startup-before-identity'] as const const ordinaryFixtureMode = 'managed-fixture-ordinary' +const managedEnvironmentFixtureModel = 'opencode/big-pickle' const checkpointImage = 'freshell-sandbox:latest' const checkpointContainerPort = 3477 const checkpointForwarderPort = 3478 @@ -162,6 +163,7 @@ const [receiptPath, expectedNonce] = process.argv.slice(2) if (!receiptPath || !expectedNonce) process.exit(12) fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper, envValueLoaded: process.env.FRESHELL_MANAGED_CWD_ALIAS_ENV === expectedNonce, + envFileTarget: fs.realpathSync('.env'), envFileOperand: process.execArgv.find(value => value.startsWith('--env-file=')), cwd: process.cwd(), scriptPath: process.argv[1] })) let buffered = '' @@ -435,7 +437,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); - const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; + const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) await writeJson(path.join(mcpPackage, 'package.json'), { @@ -468,7 +470,8 @@ export default {id:'managed-environment-tui',async tui(api,options){ const absoluteRule = path.join(resources, 'absolute-rule.txt') await fs.writeFile(absoluteRule, nonce, { mode: 0o600 }) const options = { expectedNonce: nonce, candidateSha, runId } - await writeJson(path.join(xdg, 'opencode/opencode.json'), { plugin: [[plugin, options]], agent: { fixtureAbsolute: { prompt: `{file:${absoluteRule}}` } } }) + await writeJson(path.join(xdg, 'opencode/opencode.json'), { model: managedEnvironmentFixtureModel, + plugin: [[plugin, options]], agent: { fixtureAbsolute: { prompt: `{file:${absoluteRule}}` } } }) await writeJson(path.join(xdg, 'opencode/tui.json'), { plugin: [[plugin, options]] }) for (const scenario of [...scenarios, 'preflight']) { const cwd = path.join(workspace, scenario) @@ -486,7 +489,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ const homeAppend = managedHomeAppendFixturePaths(home, cwd, scenario) await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) await fs.writeFile(homeAppend.output, `initial-${nonce}`) - await writeJson(path.join(cwd, 'opencode.json'), { agent: { fixtureRelative: { + await writeJson(path.join(cwd, 'opencode.json'), { model: managedEnvironmentFixtureModel, agent: { fixtureRelative: { prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } }, mcp: { managed_fixture: { @@ -1406,6 +1409,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightChild.terminalId, created.terminalId) assert.equal(preflightChild.candidateSha, rig.runtime.candidateSha) assert.equal(preflightChild.runId, rig.runtime.runId) + assert.equal(preflightChild.model, managedEnvironmentFixtureModel) assert.equal(preflightChild.expectedNonce, nonce) assert.equal(preflightChild.bun, true) const preflightMcp = await poll('OpenCode starts the nested Bun package-script MCP', async () => @@ -1441,6 +1445,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightCwdAliasMcp.expectedNonce, nonce) assert.equal(preflightCwdAliasMcp.helper, nonce) assert.equal(preflightCwdAliasMcp.envValueLoaded, true) + assert.match(preflightCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) assert.equal(preflightCwdAliasMcp.envFileOperand, '--env-file=.env', 'the workspace wrapper must retain its relative operand') assert.equal(preflightCwdAliasMcp.cwd, selected.cwdAliasProject) assert(preflightCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) @@ -1574,6 +1579,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(child.runId, fixture.runId) assert.equal(child.bun, true) assert.equal(child.opencodeVersion, '1.18.21') + assert.equal(child.model, managedEnvironmentFixtureModel) assert.match(child.bunExecutable, /opencode/) assert.equal(child.executableMode, '0711') const mcpServer = await poll('Cloud OpenCode starts the nested Bun package-script MCP', async () => @@ -1609,6 +1615,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(workspaceCwdAliasMcp.expectedNonce, nonce) assert.equal(workspaceCwdAliasMcp.helper, nonce) assert.equal(workspaceCwdAliasMcp.envValueLoaded, true) + assert.match(workspaceCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) assert.equal(workspaceCwdAliasMcp.envFileOperand, '--env-file=.env', 'the workspace wrapper must retain its relative operand') assert.equal(workspaceCwdAliasMcp.cwd, selected.cwdAliasProject) assert(workspaceCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) From e26cc0ba3f2c6c90831f6e5344cb24967e50231a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:12:59 -0700 Subject: [PATCH 441/493] test(managed): select a valid OpenCode fixture model --- scripts/testing/managed-provider-environment-target.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 2271ba53b..4d7671ff2 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1367,7 +1367,10 @@ export async function runManagedEnvironmentTarget() { const environment = { ...grants.serverEnv, ...managedEnvironmentOpencodeSelection(), HOME: home, XDG_CONFIG_HOME: xdg, FRESHELL_MANAGED_OPENCODE_ONECLI_AUTH_FILE: '', FRESHELL_MANAGED_OPENCODE_ONECLI_CA_FILE: '' } rig = new ManagedRuntimeBrowserRig(repoRoot, 2, { ...environment, FRESHELL_MANAGED_ENVIRONMENT_FIXTURE_ONLY: '1' }, - environment, 'release', { enabledProviders: ['opencode'] }, home, + environment, 'release', { + enabledProviders: ['opencode'], + providerSettings: { opencode: { model: managedEnvironmentFixtureModel } }, + }, home, { workspaceRoots: [workspace], providerInputRoots: [managedEnvironmentFixtureLayout(root).resources, home], bootstrapFiles: [grants.grants.opencode], defaultCwd: workspace }) const selected = await configure(root, nonce, rig.runtime.candidateSha, rig.runtime.runId) From d50680477e1813da067baa989c93c0f6ec4f0891 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:27:15 -0700 Subject: [PATCH 442/493] test(managed): diagnose HOME MCP sibling mounts --- .../managed_mcp_capability/source_stage.rs | 148 ++++++++++++++++++ .../managed-provider-environment-target.ts | 15 +- 2 files changed, 160 insertions(+), 3 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 9dd6285b3..d1e3be637 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1753,6 +1753,154 @@ mod configured_resource_tests { assert_eq!(receipt["envValueLoaded"], true); } + #[test] + fn direct_home_mcp_sibling_survives_a_multi_entry_resource_plan() { + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let resources = owned.path().join("selected-resources"); + let global = home.join(".config/opencode"); + let package = resources.join("fixture-mcp"); + let package_cwd = package.join("nested"); + let direct = resources.join("direct-mcp"); + let cwd_alias = resources.join("cwd-alias-project"); + let workspace_cwd = workspace.join("preflight/.managed-fixture/mcp-cwd-alias"); + let home_script = home.join("direct-home-preflight.mjs"); + let home_output = home.join("direct-home-preflight.txt"); + let shared_env = direct.join(".env"); + let workspace_config = workspace.join("preflight/opencode.json"); + std::fs::create_dir_all(&global).unwrap(); + std::fs::create_dir_all(&package_cwd).unwrap(); + std::fs::create_dir_all(&direct).unwrap(); + std::fs::create_dir_all(&cwd_alias).unwrap(); + std::fs::create_dir_all(&workspace_cwd).unwrap(); + std::fs::create_dir_all(workspace_config.parent().unwrap()).unwrap(); + std::fs::write(global.join("opencode.json"), r#"{"model":"fixture/model"}"#).unwrap(); + std::fs::write(package.join("package.json"), r#"{"type":"module","scripts":{"mcp.js":"bun ./server.mjs"}}"#).unwrap(); + std::fs::write(package.join("server.mjs"), "process.stdin.resume();\n").unwrap(); + std::fs::write( + direct.join("start"), + "#!/bin/sh\ncd \"$(dirname \"$0\")\"\nSCRIPT_DIR=$(dirname \"$0\")\n. \"$SCRIPT_DIR/helper.sh\"\nexec node --env-file=.env \"$SCRIPT_DIR/server.mjs\" \"$@\"\n", + ) + .unwrap(); + std::fs::set_permissions( + direct.join("start"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + std::fs::write(direct.join("helper.sh"), "export FIXTURE_HELPER=loaded\n").unwrap(); + std::fs::write(direct.join("server.mjs"), "process.stdin.resume();\n").unwrap(); + std::fs::write(&shared_env, "FIXTURE_SHARED_ENV=loaded\n").unwrap(); + std::fs::write(cwd_alias.join("server.mjs"), "process.stdin.resume();\n").unwrap(); + std::fs::write(cwd_alias.join(".env"), "FIXTURE_CWD_ENV=loaded\n").unwrap(); + std::os::unix::fs::symlink(&workspace, cwd_alias.join("workspace")).unwrap(); + std::fs::write( + workspace_cwd.join("start"), + "#!/bin/sh\nexec node --env-file=.env server.mjs \"$@\"\n", + ) + .unwrap(); + std::fs::set_permissions( + workspace_cwd.join("start"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + std::fs::write(&home_output, "initial").unwrap(); + std::fs::write( + &home_script, + "import fs from 'node:fs'\nconst output = new URL('./direct-home-preflight.txt', import.meta.url)\nfs.appendFileSync(output, '|managed')\n", + ) + .unwrap(); + let command = |values: Vec<String>, cwd: Option<&Path>| { + let mut entry = serde_json::json!({ "type": "local", "command": values }); + if let Some(cwd) = cwd { + entry["cwd"] = serde_json::json!(cwd.to_string_lossy()); + } + entry + }; + std::fs::write( + &workspace_config, + serde_json::json!({ + "mcp": { + "managed_fixture": command(vec!["bun".into(), "run".into(), "mcp.js".into()], Some(&package_cwd)), + "direct_script_fixture": command(vec![direct.join("start").to_string_lossy().into_owned()], None), + "workspace_cwd_alias_fixture": command(vec![workspace_cwd.join("start").to_string_lossy().into_owned()], Some(&cwd_alias)), + "direct_home_append_fixture": command(vec![ + "node".into(), + format!("--env-file={}", shared_env.display()), + home_script.to_string_lossy().into_owned(), + ], None), + } + }) + .to_string(), + ) + .unwrap(); + + let global_config = global.join("opencode.json"); + let plan = plan_opencode_sources( + vec![ + SourceOccurrence { + logical_path: global_config.clone(), + canonical_source: global_config, + lane: SourceLane::Main, + role: DiscoveryRole::Global, + }, + SourceOccurrence { + logical_path: workspace_config.clone(), + canonical_source: workspace_config, + lane: SourceLane::Main, + role: DiscoveryRole::Workspace, + }, + ], + &workspace, + &home, + &|_, _, _| None, + ) + .unwrap(); + let selected_home_output = home_output.canonicalize().unwrap(); + assert!( + plan.iter().any(|source| source + .configured_writable_resources + .contains(&selected_home_output)), + "the full MCP config must mark the direct HOME append target writable" + ); + let units = resource_units(&home, &global, &workspace, &plan, None).unwrap(); + let home_output_mount = units + .iter() + .find(|(source, mount)| { + source == &selected_home_output + && mount.kind == freshell_runtime_protocol::ProviderResourceKind::File + && mount.writable + && mount.effective_destination + == "/home/freshell/provider/direct-home-preflight.txt" + }) + .expect("the direct HOME sibling needs its own writable provider mount"); + assert_eq!( + home_output_mount.1.effective_destination, + "/home/freshell/provider/direct-home-preflight.txt" + ); + let excluded = plan + .iter() + .flat_map(|source| { + source + .references + .selections + .iter() + .map(|selection| selection.canonical_source.clone()) + .chain(source.configured_resource_exclusions.iter().cloned()) + }) + .collect::<std::collections::BTreeSet<_>>() + .into_iter() + .collect::<Vec<_>>(); + let provider = copy_selected_units_through_provider_excluding_with_plan( + owned.path(), &units, &excluded, &plan, + ); + let provider_output = provider.join(".provider-home/direct-home-preflight.txt"); + assert!(provider_output.is_file(), "the HOME sibling must survive the complete selected-resource copy"); + assert_eq!(std::fs::read_to_string(provider_output).unwrap(), "initial"); + } + #[test] fn direct_node_script_allows_missing_optional_require_resolve_after_all_resource_copies() { use std::os::unix::fs::PermissionsExt; diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 4d7671ff2..2e63c758b 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -196,17 +196,23 @@ process.stdin.on('data', chunk => { function directHomeAppendMcpServerSource(outputName: string): string { const outputUrl = `new URL(${JSON.stringify(`./${outputName}`)}, import.meta.url)` return `import fs from 'node:fs' +import { fileURLToPath } from 'node:url' const [receiptPath, expectedNonce] = process.argv.slice(2) if (!receiptPath || !expectedNonce) process.exit(12) let appendError try { fs.appendFileSync(${outputUrl}, '|' + expectedNonce) } catch (error) { appendError = error.code ?? error.message ?? String(error) } -const output = fs.readFileSync(${outputUrl}, 'utf8') +let output, readError +try { output = fs.readFileSync(${outputUrl}, 'utf8') } +catch (error) { readError = error.code ?? error.message ?? String(error) } +const outputPath = fileURLToPath(${outputUrl}) +const outputExists = fs.existsSync(outputPath) +const siblingMountPresent = (() => { try { return fs.readFileSync('/proc/self/mountinfo', 'utf8').split('\n').some(line => line.includes(outputPath)) } catch { return false } })() const envFileOperandRewritten = process.execArgv.some(value => /^--env-file=\/home\/freshell\/provider\/\.freshell\/references\/\d+$/.test(value)) const envValueLoaded = process.env.FRESHELL_MANAGED_SHARED_ENV === expectedNonce -fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, appendError, output, scriptPath: process.argv[1], - envFileOperandRewritten, envValueLoaded })) +fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, appendError, readError, output, outputExists, + siblingMountPresent, scriptPath: process.argv[1], envFileOperandRewritten, envValueLoaded })) let buffered = '' process.stdin.setEncoding('utf8') process.stdin.on('data', chunk => { @@ -1439,6 +1445,9 @@ export async function runManagedEnvironmentTarget() { readJson(path.join(selected.workspace, 'preflight/.managed-fixture/direct-home-mcp-receipt.json'))) assert.equal(preflightHomeAppend.expectedNonce, nonce) assert.equal(preflightHomeAppend.appendError, undefined, 'the staged HOME-level append target must be writable') + assert.equal(preflightHomeAppend.readError, undefined, 'the staged HOME-level append target must be readable') + assert.equal(preflightHomeAppend.outputExists, true, 'the sibling file must exist inside the provider container') + assert.equal(preflightHomeAppend.siblingMountPresent, true, 'the sibling file must have its own provider-volume mount') assert.equal(preflightHomeAppend.output, `initial-${nonce}|${nonce}`) assert.equal(preflightHomeAppend.envFileOperandRewritten, true) assert.equal(preflightHomeAppend.envValueLoaded, true) From c089a437c9fc55b8fd2d21b28fa1bfe8dccd4509 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:46:09 -0700 Subject: [PATCH 443/493] test(managed): trace HOME MCP startup and mounts --- .../src/managed_mcp_capability.rs | 136 ++++++++++++++++++ .../managed_mcp_capability/source_stage.rs | 23 +-- .../managed-provider-environment-target.ts | 53 ++++++- 3 files changed, 199 insertions(+), 13 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index 65c336e43..e1fa610d4 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -10826,6 +10826,142 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ } } + #[tokio::test] + async fn managed_opencode_controller_publishes_direct_home_sibling_file_mount() { + use crate::managed_provider_references::sources::*; + use freshell_runtime_protocol::*; + use std::os::unix::fs::PermissionsExt; + + let owned = tempfile::tempdir().unwrap(); + let home = owned.path().join("home"); + let workspace = owned.path().join("workspace"); + let cwd = workspace.join("preflight"); + let config_root = home.join(".config/opencode"); + let resources = owned.path().join("selected-resources"); + let direct_home_script = home.join("direct-home-preflight.mjs"); + let direct_home_wrapper = home.join("direct-home-preflight-start"); + let direct_home_output = home.join("direct-home-preflight.txt"); + let direct_home_env = home.join("direct-home-preflight.env"); + let workspace_config = workspace.join("preflight/opencode.json"); + std::fs::create_dir_all(&config_root).unwrap(); + std::fs::create_dir_all(&resources).unwrap(); + std::fs::create_dir_all(workspace_config.parent().unwrap()).unwrap(); + std::fs::write( + &direct_home_env, + "FRESHELL_MANAGED_SHARED_ENV=fixture-value\n", + ) + .unwrap(); + std::fs::write(&direct_home_output, "initial").unwrap(); + std::fs::write( + &direct_home_script, + "import fs from 'node:fs'\nconst output = new URL('./direct-home-preflight.txt', import.meta.url)\nfs.appendFileSync(output, '|managed')\n", + ) + .unwrap(); + std::fs::write( + &direct_home_wrapper, + format!( + "#!/bin/sh\nSCRIPT_DIR=$(dirname \"$0\")\nnode --env-file={} \"$SCRIPT_DIR/direct-home-preflight.mjs\" \"$@\"\n", + direct_home_env.display() + ), + ) + .unwrap(); + std::fs::set_permissions(&direct_home_wrapper, std::fs::Permissions::from_mode(0o755)) + .unwrap(); + let mut mcp = serde_json::Map::new(); + for name in ["nested_package", "direct_script", "workspace_alias"] { + let script = resources.join(format!("{name}.mjs")); + std::fs::write(&script, "process.stdin.resume()\n").unwrap(); + mcp.insert( + name.into(), + serde_json::json!({ + "type":"local", + "command":["node", script.to_string_lossy()], + "enabled":true, + }), + ); + } + mcp.insert( + "direct_home_append_fixture".into(), + serde_json::json!({ + "type":"local", + "command":[direct_home_wrapper, "receipt.json", "fixture-value"], + "enabled":true, + }), + ); + std::fs::write( + &workspace_config, + serde_json::json!({ "mcp": mcp }).to_string(), + ) + .unwrap(); + + let sources = discover_opencode(OpencodeDiscovery { + home: &home, + cwd: &cwd, + worktree: &workspace, + config_root: &config_root, + env: &|_| None, + }) + .unwrap(); + let plan = plan_opencode_sources(sources, &cwd, &home, &|_, _, _| None).unwrap(); + let direct_home_output = direct_home_output.canonicalize().unwrap(); + assert!(plan.iter().any(|source| source + .configured_writable_resources + .contains(&direct_home_output))); + let secret_references = plan + .iter() + .flat_map(|source| &source.references.selections) + .map(|selection| { + let index = selection + .runtime_destination + .strip_prefix("/home/freshell/provider/.freshell/references/") + .unwrap(); + ProviderSecretReference { + source_path: selection.canonical_source.to_string_lossy().into_owned(), + profile: ProviderSecretProfile::OpencodeConfigurationFile, + provider_relative_path: Some(format!(".freshell/references/{index}")), + } + }) + .collect::<Vec<_>>(); + let directory = owned.path().join("control/mcp-capabilities"); + std::fs::create_dir_all(&directory).unwrap(); + let store = CapabilityStore::default(); + let capability = store + .issue( + SoulId::new(), + &directory, + "http://host.docker.internal:4000".into(), + Some(("opencode", &home, &[])), + OpencodeEphemeralInput { + cwd: Some(&cwd), + workspace: Some(&workspace), + source_plan: Some(&plan), + secret_references: Some(&secret_references), + ..Default::default() + }, + None, + ) + .unwrap(); + let stage = staged_provider_root_path(&directory, &capability.grant_id).unwrap(); + let inputs: PreparedProviderInputs = serde_json::from_slice( + &std::fs::read(stage.join(".freshell/prepared-inputs.json")).unwrap(), + ) + .unwrap(); + let mount = inputs + .resource_mounts + .iter() + .find(|mount| { + mount.kind == ProviderResourceKind::File + && mount.writable + && mount.effective_destination + == "/home/freshell/provider/direct-home-preflight.txt" + }) + .expect("prepared input descriptor must publish the exact writable HOME sibling mount"); + let staged = stage.join(&mount.stage_relative_path); + let metadata = std::fs::symlink_metadata(&staged).unwrap(); + assert!(metadata.is_file() && !metadata.file_type().is_symlink()); + assert_eq!(std::fs::read(staged).unwrap(), b"initial"); + } + #[test] fn managed_provider_ancestor_root_has_its_own_native_mount_location() { use crate::managed_provider_references::sources::*; diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index d1e3be637..129ef94c0 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -1778,18 +1778,19 @@ mod configured_resource_tests { std::fs::create_dir_all(&workspace_cwd).unwrap(); std::fs::create_dir_all(workspace_config.parent().unwrap()).unwrap(); std::fs::write(global.join("opencode.json"), r#"{"model":"fixture/model"}"#).unwrap(); - std::fs::write(package.join("package.json"), r#"{"type":"module","scripts":{"mcp.js":"bun ./server.mjs"}}"#).unwrap(); - std::fs::write(package.join("server.mjs"), "process.stdin.resume();\n").unwrap(); std::fs::write( - direct.join("start"), - "#!/bin/sh\ncd \"$(dirname \"$0\")\"\nSCRIPT_DIR=$(dirname \"$0\")\n. \"$SCRIPT_DIR/helper.sh\"\nexec node --env-file=.env \"$SCRIPT_DIR/server.mjs\" \"$@\"\n", + package.join("package.json"), + r#"{"type":"module","scripts":{"mcp.js":"bun ./server.mjs"}}"#, ) .unwrap(); - std::fs::set_permissions( + std::fs::write(package.join("server.mjs"), "process.stdin.resume();\n").unwrap(); + std::fs::write( direct.join("start"), - std::fs::Permissions::from_mode(0o755), + "#!/bin/sh\ncd \"$(dirname \"$0\")\"\nSCRIPT_DIR=$(dirname \"$0\")\n. \"$SCRIPT_DIR/helper.sh\"\nexec node --env-file=.env \"$SCRIPT_DIR/server.mjs\" \"$@\"\n", ) .unwrap(); + std::fs::set_permissions(direct.join("start"), std::fs::Permissions::from_mode(0o755)) + .unwrap(); std::fs::write(direct.join("helper.sh"), "export FIXTURE_HELPER=loaded\n").unwrap(); std::fs::write(direct.join("server.mjs"), "process.stdin.resume();\n").unwrap(); std::fs::write(&shared_env, "FIXTURE_SHARED_ENV=loaded\n").unwrap(); @@ -1894,10 +1895,16 @@ mod configured_resource_tests { .into_iter() .collect::<Vec<_>>(); let provider = copy_selected_units_through_provider_excluding_with_plan( - owned.path(), &units, &excluded, &plan, + owned.path(), + &units, + &excluded, + &plan, ); let provider_output = provider.join(".provider-home/direct-home-preflight.txt"); - assert!(provider_output.is_file(), "the HOME sibling must survive the complete selected-resource copy"); + assert!( + provider_output.is_file(), + "the HOME sibling must survive the complete selected-resource copy" + ); assert_eq!(std::fs::read_to_string(provider_output).unwrap(), "initial"); } diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 2e63c758b..5f3806be9 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -390,10 +390,14 @@ export function managedEnvironmentFixtureLayout(root: string) { export function managedHomeAppendFixturePaths(home: string, cwd: string, scenario: string) { const script = path.join(home, `direct-home-${scenario}.mjs`) + const receipt = path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json') return { script, + wrapper: path.join(home, `direct-home-${scenario}-start`), + envFile: path.join(home, `direct-home-${scenario}.env`), output: path.join(path.dirname(script), `direct-home-${scenario}.txt`), - receipt: path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json'), + receipt, + started: `${receipt}.started`, } } @@ -495,6 +499,31 @@ export default {id:'managed-environment-tui',async tui(api,options){ const homeAppend = managedHomeAppendFixturePaths(home, cwd, scenario) await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) await fs.writeFile(homeAppend.output, `initial-${nonce}`) + await fs.writeFile(homeAppend.envFile, `FRESHELL_MANAGED_SHARED_ENV=${nonce}\n`, { mode: 0o600 }) + const shellQuote = (value: string) => `'${value.replaceAll("'", "'\\''")}'` + await fs.writeFile(homeAppend.wrapper, `#!/bin/sh +set +e +SCRIPT_DIR=$(dirname "$0") +receipt=$1 +expected=$2 +started="${'${receipt}'}.started" +printf '%s' wrapper-started > "$started" +diagnostic=$(mktemp) +node --env-file=${shellQuote(homeAppend.envFile)} "$SCRIPT_DIR/${path.basename(homeAppend.script)}" "$receipt" "$expected" 2>"$diagnostic" +status=$? +if [ "$status" -ne 0 ]; then + has_enoent=false + has_eacces=false + has_env_file_error=false + grep -q 'ENOENT' "$diagnostic" && has_enoent=true + grep -q 'EACCES' "$diagnostic" && has_eacces=true + grep -qi 'env file' "$diagnostic" && has_env_file_error=true + printf '{"wrapperStarted":true,"nodeExitCode":%s,"stderrHasENOENT":%s,"stderrHasEACCES":%s,"stderrHasEnvFileError":%s}' "$status" "$has_enoent" "$has_eacces" "$has_env_file_error" > "$receipt" +fi +rm -f "$diagnostic" +exit "$status" +`, { mode: 0o751 }) + await fs.chmod(homeAppend.wrapper, 0o751) await writeJson(path.join(cwd, 'opencode.json'), { model: managedEnvironmentFixtureModel, agent: { fixtureRelative: { prompt: `{file:./${scenario === 'missing-reference' ? 'missing-selected.txt' : 'rules.txt'}}` } }, mcp: { @@ -518,7 +547,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ }, direct_home_append_fixture: { type: 'local', - command: ['node', `--env-file=${sharedEnvFile}`, homeAppend.script, homeAppend.receipt, nonce], + command: [homeAppend.wrapper, homeAppend.receipt, nonce], enabled: true, }, }, @@ -1441,9 +1470,23 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightDirectMcp.envFileOperandRewritten, true) assert.equal(preflightDirectMcp.envValueLoaded, true) assert(preflightDirectMcp.scriptPath.endsWith('/direct-mcp/server.mjs')) - const preflightHomeAppend = await poll('OpenCode starts the direct HOME MCP and it appends its sibling output', async () => - readJson(path.join(selected.workspace, 'preflight/.managed-fixture/direct-home-mcp-receipt.json'))) - assert.equal(preflightHomeAppend.expectedNonce, nonce) + const preflightHomeAppendPaths = managedHomeAppendFixturePaths( + selected.home, path.join(selected.workspace, 'preflight'), 'preflight') + const preflightHomeAppend = await poll('OpenCode starts the direct HOME MCP and it appends its sibling output', async () => { + const receipt = await readJson(preflightHomeAppendPaths.receipt) + if (receipt) return receipt + try { + const [started, metadata] = await Promise.all([ + fs.readFile(preflightHomeAppendPaths.started, 'utf8'), fs.stat(preflightHomeAppendPaths.started), + ]) + if (started === 'wrapper-started' && Date.now() - metadata.mtimeMs > 15_000) { + return { wrapperStarted: true, nodeStartupTimedOut: true } + } + } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + return undefined + }) + assert.equal(preflightHomeAppend.expectedNonce, nonce, + `direct HOME MCP did not complete its receipt: ${JSON.stringify(preflightHomeAppend)}`) assert.equal(preflightHomeAppend.appendError, undefined, 'the staged HOME-level append target must be writable') assert.equal(preflightHomeAppend.readError, undefined, 'the staged HOME-level append target must be readable') assert.equal(preflightHomeAppend.outputExists, true, 'the sibling file must exist inside the provider container') From 2b84451544a29df047dce1fe39e3a4e588fd4fad Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:54:41 -0700 Subject: [PATCH 444/493] test(managed): validate generated HOME MCP script --- .../managed-provider-environment-target.ts | 10 +++--- ...anaged-provider-environment-target.test.ts | 34 +++++++++++++++++++ 2 files changed, 40 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 5f3806be9..475e347fb 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -193,7 +193,7 @@ process.stdin.on('data', chunk => { }) ` -function directHomeAppendMcpServerSource(outputName: string): string { +export function directHomeAppendMcpServerSource(outputName: string): string { const outputUrl = `new URL(${JSON.stringify(`./${outputName}`)}, import.meta.url)` return `import fs from 'node:fs' import { fileURLToPath } from 'node:url' @@ -207,9 +207,9 @@ try { output = fs.readFileSync(${outputUrl}, 'utf8') } catch (error) { readError = error.code ?? error.message ?? String(error) } const outputPath = fileURLToPath(${outputUrl}) const outputExists = fs.existsSync(outputPath) -const siblingMountPresent = (() => { try { return fs.readFileSync('/proc/self/mountinfo', 'utf8').split('\n').some(line => line.includes(outputPath)) } catch { return false } })() +const siblingMountPresent = (() => { try { return fs.readFileSync('/proc/self/mountinfo', 'utf8').split('\\n').some(line => line.includes(outputPath)) } catch { return false } })() const envFileOperandRewritten = process.execArgv.some(value => - /^--env-file=\/home\/freshell\/provider\/\.freshell\/references\/\d+$/.test(value)) + /^--env-file=\\/home\\/freshell\\/provider\\/\\.freshell\\/references\\/\\d+$/.test(value)) const envValueLoaded = process.env.FRESHELL_MANAGED_SHARED_ENV === expectedNonce fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, appendError, readError, output, outputExists, siblingMountPresent, scriptPath: process.argv[1], envFileOperandRewritten, envValueLoaded })) @@ -515,10 +515,12 @@ if [ "$status" -ne 0 ]; then has_enoent=false has_eacces=false has_env_file_error=false + has_syntax_error=false grep -q 'ENOENT' "$diagnostic" && has_enoent=true grep -q 'EACCES' "$diagnostic" && has_eacces=true grep -qi 'env file' "$diagnostic" && has_env_file_error=true - printf '{"wrapperStarted":true,"nodeExitCode":%s,"stderrHasENOENT":%s,"stderrHasEACCES":%s,"stderrHasEnvFileError":%s}' "$status" "$has_enoent" "$has_eacces" "$has_env_file_error" > "$receipt" + grep -q 'SyntaxError' "$diagnostic" && has_syntax_error=true + printf '{"wrapperStarted":true,"nodeExitCode":%s,"stderrHasENOENT":%s,"stderrHasEACCES":%s,"stderrHasEnvFileError":%s,"stderrHasSyntaxError":%s}' "$status" "$has_enoent" "$has_eacces" "$has_env_file_error" "$has_syntax_error" > "$receipt" fi rm -f "$diagnostic" exit "$status" diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index a0e6ab1e5..60cf78352 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -9,6 +9,7 @@ import { } from '../../../../scripts/testing/managed-environment-create.js' import { checkpointTcpForwarderSource, + directHomeAppendMcpServerSource, externalTargetRead, managedHomeAppendFixturePaths, managedEnvironmentFixtureLayout, @@ -153,6 +154,39 @@ describe('managed provider environment target mode', () => { expect(paths.receipt).toBe(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json')) }) + it('generates a runnable direct HOME MCP script that appends to its sibling file', async () => { + const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-home-mcp-')) + try { + const script = path.join(directory, 'server.mjs') + const output = path.join(directory, 'sibling.txt') + const receipt = path.join(directory, 'receipt.json') + await fs.writeFile(script, directHomeAppendMcpServerSource(path.basename(output))) + await fs.writeFile(output, 'before') + + const child = spawn(process.execPath, [script, receipt, 'expected-nonce'], { + cwd: directory, + stdio: ['ignore', 'ignore', 'pipe'], + }) + let stderr = '' + child.stderr?.setEncoding('utf8') + child.stderr?.on('data', chunk => { stderr += chunk }) + const exit = await new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => { + child.once('error', reject) + child.once('close', (code, signal) => resolve({ code, signal })) + }) + + expect(exit, stderr).toEqual({ code: 0, signal: null }) + expect(await fs.readFile(output, 'utf8')).toBe('before|expected-nonce') + expect(JSON.parse(await fs.readFile(receipt, 'utf8'))).toMatchObject({ + expectedNonce: 'expected-nonce', + output: 'before|expected-nonce', + outputExists: true, + }) + } finally { + await fs.rm(directory, { recursive: true, force: true }) + } + }) + it('forwards one fixed TCP target while preserving streamed client data', async () => { const upstream = createServer(socket => { socket.on('data', chunk => socket.write(chunk)) From 60a1fc30beaa2c70779057adf7a7c4a1bd02bc8a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:04:18 -0700 Subject: [PATCH 445/493] test(managed): diagnose workspace MCP startup --- .../managed-provider-environment-target.ts | 50 ++++++++++++++-- ...anaged-provider-environment-target.test.ts | 58 +++++++++++++++++++ 2 files changed, 104 insertions(+), 4 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 475e347fb..eb8f760ff 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -193,6 +193,35 @@ process.stdin.on('data', chunk => { }) ` +export function workspaceCwdAliasMcpWrapperSource(): string { + const receiptVariable = '${receipt}' + return `#!/bin/sh +set +e +receipt=$1 +started="${receiptVariable}.started" +printf '%s' wrapper-started > "$started" +diagnostic=$(mktemp) +node --env-file=.env server.mjs "$@" 2>"$diagnostic" +status=$? +if [ "$status" -ne 0 ]; then + has_enoent=false + has_eacces=false + has_env_file_error=false + has_syntax_error=false + has_module_error=false + grep -q 'ENOENT' "$diagnostic" && has_enoent=true + grep -q 'EACCES' "$diagnostic" && has_eacces=true + grep -qi 'env file' "$diagnostic" && has_env_file_error=true + grep -q 'SyntaxError' "$diagnostic" && has_syntax_error=true + if grep -q 'ERR_MODULE_NOT_FOUND' "$diagnostic" || grep -qi 'Cannot find module' "$diagnostic"; then has_module_error=true; fi + printf '{"wrapperStarted":true,"nodeExitCode":%s,"stderrHasENOENT":%s,"stderrHasEACCES":%s,"stderrHasEnvFileError":%s,"stderrHasSyntaxError":%s,"stderrHasModuleError":%s}' \ + "$status" "$has_enoent" "$has_eacces" "$has_env_file_error" "$has_syntax_error" "$has_module_error" > "$receipt" +fi +rm -f "$diagnostic" +exit "$status" +` +} + export function directHomeAppendMcpServerSource(outputName: string): string { const outputUrl = `new URL(${JSON.stringify(`./${outputName}`)}, import.meta.url)` return `import fs from 'node:fs' @@ -493,8 +522,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ const cwdAliasWrapper = path.join(cwd, '.managed-fixture/mcp-cwd-alias/start') const cwdAliasReceipt = path.join(cwd, '.managed-fixture/mcp-cwd-alias-receipt.json') await fs.mkdir(path.dirname(cwdAliasWrapper), { recursive: true, mode: 0o755 }) - await fs.writeFile(cwdAliasWrapper, - '#!/bin/sh\nexec node --env-file=.env server.mjs "$@"\n', { mode: 0o751 }) + await fs.writeFile(cwdAliasWrapper, workspaceCwdAliasMcpWrapperSource(), { mode: 0o751 }) await fs.chmod(cwdAliasWrapper, 0o751) const homeAppend = managedHomeAppendFixturePaths(home, cwd, scenario) await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) @@ -1497,8 +1525,22 @@ export async function runManagedEnvironmentTarget() { assert.equal(preflightHomeAppend.envFileOperandRewritten, true) assert.equal(preflightHomeAppend.envValueLoaded, true) assert(preflightHomeAppend.scriptPath.endsWith('/direct-home-preflight.mjs')) - const preflightCwdAliasMcp = await poll('OpenCode starts a workspace wrapper with its external MCP cwd env file', async () => - readJson(path.join(selected.workspace, 'preflight/.managed-fixture/mcp-cwd-alias-receipt.json'))) + const preflightCwdAliasReceipt = path.join(selected.workspace, 'preflight/.managed-fixture/mcp-cwd-alias-receipt.json') + const aliasWaitStartedAt = Date.now() + const preflightCwdAliasMcp = await poll('OpenCode starts a workspace wrapper with its external MCP cwd env file', async () => { + const receipt = await readJson(preflightCwdAliasReceipt) + if (receipt) return receipt + try { + const [started, metadata] = await Promise.all([ + fs.readFile(`${preflightCwdAliasReceipt}.started`, 'utf8'), fs.stat(`${preflightCwdAliasReceipt}.started`), + ]) + if (started === 'wrapper-started' && Date.now() - metadata.mtimeMs > 15_000) { + return { wrapperStarted: true, nodeStartupTimedOut: true } + } + } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error } + if (Date.now() - aliasWaitStartedAt > 30_000) return { wrapperStarted: false, nodeStartupTimedOut: true } + return undefined + }, 45_000) assert.equal(preflightCwdAliasMcp.expectedNonce, nonce) assert.equal(preflightCwdAliasMcp.helper, nonce) assert.equal(preflightCwdAliasMcp.envValueLoaded, true) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 60cf78352..9e4e8eaaf 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -17,6 +17,7 @@ import { resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, stopExact, + workspaceCwdAliasMcpWrapperSource, } from '../../../../scripts/testing/managed-provider-environment-target.js' async function listenOnLoopback(server: ReturnType<typeof createServer>): Promise<number> { @@ -154,6 +155,63 @@ describe('managed provider environment target mode', () => { expect(paths.receipt).toBe(path.join(cwd, '.managed-fixture/direct-home-mcp-receipt.json')) }) + it('runs an external-CWD MCP wrapper and reports a Node startup syntax error', async () => { + const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-cwd-alias-mcp-')) + try { + const workspace = path.join(directory, 'workspace') + const resource = path.join(directory, 'selected-resources/cwd-alias-project') + const wrapper = path.join(workspace, '.managed-fixture/mcp-cwd-alias/start') + const receipt = path.join(workspace, '.managed-fixture/mcp-cwd-alias-receipt.json') + await fs.mkdir(path.dirname(wrapper), { recursive: true }) + await fs.mkdir(resource, { recursive: true }) + await fs.writeFile(wrapper, workspaceCwdAliasMcpWrapperSource(), { mode: 0o751 }) + await fs.writeFile(path.join(resource, '.env'), 'FRESHELL_MANAGED_CWD_ALIAS_ENV=expected-nonce\n') + await fs.writeFile(path.join(resource, 'server.mjs'), `import fs from 'node:fs' +const [receiptPath, expectedNonce] = process.argv.slice(2) +fs.writeFileSync(receiptPath, JSON.stringify({ envValueLoaded: process.env.FRESHELL_MANAGED_CWD_ALIAS_ENV === expectedNonce, + envFileTarget: fs.realpathSync('.env'), envFileOperand: process.execArgv.find(value => value.startsWith('--env-file=')), + cwd: process.cwd() })) +`) + + const launch = async (expectedReceipt: string) => { + const child = spawn(wrapper, [expectedReceipt, 'expected-nonce'], { + cwd: resource, + stdio: ['ignore', 'ignore', 'pipe'], + }) + let stderr = '' + child.stderr?.setEncoding('utf8') + child.stderr?.on('data', chunk => { stderr += chunk }) + const exit = await new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => { + child.once('error', reject) + child.once('close', (code, signal) => resolve({ code, signal })) + }) + return { exit, stderr } + } + + const success = await launch(receipt) + expect(success).toMatchObject({ exit: { code: 0, signal: null }, stderr: '' }) + expect(await fs.readFile(`${receipt}.started`, 'utf8')).toBe('wrapper-started') + expect(JSON.parse(await fs.readFile(receipt, 'utf8'))).toEqual({ + envValueLoaded: true, + envFileTarget: path.join(resource, '.env'), + envFileOperand: '--env-file=.env', + cwd: resource, + }) + + await fs.writeFile(path.join(resource, 'server.mjs'), 'const = invalid\n') + const failureReceipt = path.join(workspace, '.managed-fixture/syntax-error-receipt.json') + const failure = await launch(failureReceipt) + expect(failure.exit).toEqual({ code: 1, signal: null }) + expect(JSON.parse(await fs.readFile(failureReceipt, 'utf8'))).toMatchObject({ + wrapperStarted: true, + nodeExitCode: 1, + stderrHasSyntaxError: true, + }) + } finally { + await fs.rm(directory, { recursive: true, force: true }) + } + }) + it('generates a runnable direct HOME MCP script that appends to its sibling file', async () => { const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-home-mcp-')) try { From 43e7bd2cb9ecef32d481650a64d3477dc1b7c834 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:09:58 -0700 Subject: [PATCH 446/493] test(managed): include MCP startup failure receipt --- scripts/testing/managed-provider-environment-target.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index eb8f760ff..c204e3f4c 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1541,7 +1541,8 @@ export async function runManagedEnvironmentTarget() { if (Date.now() - aliasWaitStartedAt > 30_000) return { wrapperStarted: false, nodeStartupTimedOut: true } return undefined }, 45_000) - assert.equal(preflightCwdAliasMcp.expectedNonce, nonce) + assert.equal(preflightCwdAliasMcp.expectedNonce, nonce, + `workspace external-CWD MCP did not complete its receipt: ${JSON.stringify(preflightCwdAliasMcp)}`) assert.equal(preflightCwdAliasMcp.helper, nonce) assert.equal(preflightCwdAliasMcp.envValueLoaded, true) assert.match(preflightCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) From a0e8aa683d513987f4c02668b3cfe72809d8d4f0 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:20:21 -0700 Subject: [PATCH 447/493] test(managed): report OpenCode MCP preflight status --- scripts/testing/managed-provider-environment-target.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index c204e3f4c..8d3caa690 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -476,7 +476,8 @@ export default {id:'managed-environment-tui',async tui(api,options){ const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); - const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),pid:process.pid,createdAt:new Date().toISOString()}; + let mcpStatuses={};if(cwd.endsWith('/preflight')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}} + const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),mcpStatuses,pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) await writeJson(path.join(mcpPackage, 'package.json'), { @@ -1542,7 +1543,7 @@ export async function runManagedEnvironmentTarget() { return undefined }, 45_000) assert.equal(preflightCwdAliasMcp.expectedNonce, nonce, - `workspace external-CWD MCP did not complete its receipt: ${JSON.stringify(preflightCwdAliasMcp)}`) + `workspace external-CWD MCP did not complete its receipt: ${JSON.stringify({ receipt: preflightCwdAliasMcp, mcpStatuses: preflightChild.mcpStatuses })}`) assert.equal(preflightCwdAliasMcp.helper, nonce) assert.equal(preflightCwdAliasMcp.envValueLoaded, true) assert.match(preflightCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) From 1ad261048704949ff77eb290051d09b1b8331ff9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:27:27 -0700 Subject: [PATCH 448/493] test(managed): capture MCP command mount diagnostics --- scripts/testing/managed-provider-environment-target.ts | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 8d3caa690..0ea7678d4 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -476,8 +476,8 @@ export default {id:'managed-environment-tui',async tui(api,options){ const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); - let mcpStatuses={};if(cwd.endsWith('/preflight')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}} - const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),mcpStatuses,pid:process.pid,createdAt:new Date().toISOString()}; + let mcpStatuses={},wrapperProbe={};if(cwd.endsWith('/preflight')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}try{const wrapperPath=path.join(cwd,'.managed-fixture/mcp-cwd-alias/start'),metadata=fs.statSync(wrapperPath);wrapperProbe.mode=(metadata.mode&511).toString(8).padStart(4,'0');wrapperProbe.uid=metadata.uid;wrapperProbe.gid=metadata.gid;fs.accessSync(wrapperPath,fs.constants.X_OK);wrapperProbe.executableAccess='ok'}catch(error){wrapperProbe.executableAccessError={code:error?.code??'unknown',message:String(error).slice(0,160)}}try{const wrapperPath=path.join(cwd,'.managed-fixture/mcp-cwd-alias/start'),mount=fs.readFileSync('/proc/self/mountinfo','utf8').split('\\n').flatMap(line=>{const [left,right]=line.split(' - ');if(!right)return[];const fields=left.split(' '),mountpoint=fields[4]??'';if(wrapperPath!==mountpoint&&!wrapperPath.startsWith(mountpoint==='/'?'/':mountpoint+'/'))return[];const detail=right.split(' ');return[{mountpoint,mountOptions:fields[5],filesystem:detail[0],source:detail[1],superOptions:detail[2]}]}).sort((a,b)=>b.mountpoint.length-a.mountpoint.length)[0];wrapperProbe.mount=mount??null}catch(error){wrapperProbe.mountProbeError=String(error).slice(0,160)}} + const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),mcpStatuses,wrapperProbe,pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) await writeJson(path.join(mcpPackage, 'package.json'), { @@ -1543,7 +1543,7 @@ export async function runManagedEnvironmentTarget() { return undefined }, 45_000) assert.equal(preflightCwdAliasMcp.expectedNonce, nonce, - `workspace external-CWD MCP did not complete its receipt: ${JSON.stringify({ receipt: preflightCwdAliasMcp, mcpStatuses: preflightChild.mcpStatuses })}`) + `workspace external-CWD MCP did not complete its receipt: ${JSON.stringify({ receipt: preflightCwdAliasMcp, mcpStatuses: preflightChild.mcpStatuses, wrapperProbe: preflightChild.wrapperProbe })}`) assert.equal(preflightCwdAliasMcp.helper, nonce) assert.equal(preflightCwdAliasMcp.envValueLoaded, true) assert.match(preflightCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) From 6b46139585936bd744cb0af54e0beb0ddd74e491 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:33:24 -0700 Subject: [PATCH 449/493] test(managed): report workspace MCP path permissions --- scripts/testing/managed-provider-environment-target.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 0ea7678d4..ea59f9d5d 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -476,7 +476,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ const command=new URL('./command',import.meta.url), resource=fs.readFileSync(new URL('./selected-resource',import.meta.url),'utf8'); const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); - let mcpStatuses={},wrapperProbe={};if(cwd.endsWith('/preflight')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}try{const wrapperPath=path.join(cwd,'.managed-fixture/mcp-cwd-alias/start'),metadata=fs.statSync(wrapperPath);wrapperProbe.mode=(metadata.mode&511).toString(8).padStart(4,'0');wrapperProbe.uid=metadata.uid;wrapperProbe.gid=metadata.gid;fs.accessSync(wrapperPath,fs.constants.X_OK);wrapperProbe.executableAccess='ok'}catch(error){wrapperProbe.executableAccessError={code:error?.code??'unknown',message:String(error).slice(0,160)}}try{const wrapperPath=path.join(cwd,'.managed-fixture/mcp-cwd-alias/start'),mount=fs.readFileSync('/proc/self/mountinfo','utf8').split('\\n').flatMap(line=>{const [left,right]=line.split(' - ');if(!right)return[];const fields=left.split(' '),mountpoint=fields[4]??'';if(wrapperPath!==mountpoint&&!wrapperPath.startsWith(mountpoint==='/'?'/':mountpoint+'/'))return[];const detail=right.split(' ');return[{mountpoint,mountOptions:fields[5],filesystem:detail[0],source:detail[1],superOptions:detail[2]}]}).sort((a,b)=>b.mountpoint.length-a.mountpoint.length)[0];wrapperProbe.mount=mount??null}catch(error){wrapperProbe.mountProbeError=String(error).slice(0,160)}} + let mcpStatuses={},wrapperProbe={};if(cwd.endsWith('/preflight')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}const wrapperPath=path.join(cwd,'.managed-fixture/mcp-cwd-alias/start');wrapperProbe.paths=[cwd,path.join(cwd,'.managed-fixture'),path.dirname(wrapperPath),wrapperPath].map(candidate=>{const relative=path.relative(cwd,candidate)||'.';try{const metadata=fs.statSync(candidate);fs.accessSync(candidate,fs.constants.X_OK);return{path:relative,mode:(metadata.mode&511).toString(8).padStart(4,'0'),uid:metadata.uid,gid:metadata.gid,searchAccess:'ok'}}catch(error){return{path:relative,code:error?.code??'unknown',message:String(error).slice(0,120)}}});try{const mount=fs.readFileSync('/proc/self/mountinfo','utf8').split('\\n').flatMap(line=>{const [left,right]=line.split(' - ');if(!right)return[];const fields=left.split(' '),mountpoint=fields[4]??'';if(wrapperPath!==mountpoint&&!wrapperPath.startsWith(mountpoint==='/'?'/':mountpoint+'/'))return[];const detail=right.split(' ');return[{mountRoot:fields[3],mountpoint,mountOptions:fields[5],filesystem:detail[0],source:detail[1],superOptions:detail[2]}]}).sort((a,b)=>b.mountpoint.length-a.mountpoint.length)[0];wrapperProbe.mount=mount??null}catch(error){wrapperProbe.mountProbeError=String(error).slice(0,160)}} const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),mcpStatuses,wrapperProbe,pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) From cf7cda70f64d4badb7b7f2a60c70d797276d538c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:40:57 -0700 Subject: [PATCH 450/493] test(managed): make MCP fixture wrapper searchable --- .../managed-provider-environment-target.ts | 11 ++++-- ...anaged-provider-environment-target.test.ts | 38 +++++++++++++++++++ 2 files changed, 46 insertions(+), 3 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index ea59f9d5d..de0f2d0e0 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -222,6 +222,13 @@ exit "$status" ` } +export async function writeWorkspaceCwdAliasMcpWrapper(wrapperPath: string): Promise<void> { + await fs.mkdir(path.dirname(wrapperPath), { recursive: true, mode: 0o755 }) + await fs.chmod(path.dirname(wrapperPath), 0o755) + await fs.writeFile(wrapperPath, workspaceCwdAliasMcpWrapperSource(), { mode: 0o751 }) + await fs.chmod(wrapperPath, 0o751) +} + export function directHomeAppendMcpServerSource(outputName: string): string { const outputUrl = `new URL(${JSON.stringify(`./${outputName}`)}, import.meta.url)` return `import fs from 'node:fs' @@ -522,9 +529,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ const mcpReceipt = path.join(cwd, '.managed-fixture/mcp-receipt.json') const cwdAliasWrapper = path.join(cwd, '.managed-fixture/mcp-cwd-alias/start') const cwdAliasReceipt = path.join(cwd, '.managed-fixture/mcp-cwd-alias-receipt.json') - await fs.mkdir(path.dirname(cwdAliasWrapper), { recursive: true, mode: 0o755 }) - await fs.writeFile(cwdAliasWrapper, workspaceCwdAliasMcpWrapperSource(), { mode: 0o751 }) - await fs.chmod(cwdAliasWrapper, 0o751) + await writeWorkspaceCwdAliasMcpWrapper(cwdAliasWrapper) const homeAppend = managedHomeAppendFixturePaths(home, cwd, scenario) await fs.writeFile(homeAppend.script, directHomeAppendMcpServerSource(path.basename(homeAppend.output))) await fs.writeFile(homeAppend.output, `initial-${nonce}`) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 9e4e8eaaf..67b1e87c3 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -2,6 +2,7 @@ import { spawn } from 'node:child_process' import fs from 'node:fs/promises' import { createConnection, createServer, type AddressInfo } from 'node:net' import path from 'node:path' +import { pathToFileURL } from 'node:url' import { describe, expect, it } from 'vitest' import { assertManagedTerminalCreated, @@ -18,6 +19,7 @@ import { runManagedProviderEnvironmentTarget, stopExact, workspaceCwdAliasMcpWrapperSource, + writeWorkspaceCwdAliasMcpWrapper, } from '../../../../scripts/testing/managed-provider-environment-target.js' async function listenOnLoopback(server: ReturnType<typeof createServer>): Promise<number> { @@ -212,6 +214,42 @@ fs.writeFileSync(receiptPath, JSON.stringify({ envValueLoaded: process.env.FRESH } }) + it('keeps a workspace MCP wrapper searchable under a private process umask', async () => { + const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-wrapper-mode-')) + try { + const fixtureDirectory = path.join(directory, '.managed-fixture') + await fs.mkdir(fixtureDirectory, { recursive: true, mode: 0o777 }) + await fs.chmod(fixtureDirectory, 0o777) + const wrapper = path.join(fixtureDirectory, 'mcp-cwd-alias/start') + const moduleUrl = pathToFileURL(path.resolve('scripts/testing/managed-provider-environment-target.ts')).href + const script = `import fs from 'node:fs/promises' +import path from 'node:path' +import { writeWorkspaceCwdAliasMcpWrapper } from ${JSON.stringify(moduleUrl)} +const wrapper = ${JSON.stringify(wrapper)} +process.umask(0o077) +await writeWorkspaceCwdAliasMcpWrapper(wrapper) +const directory = await fs.stat(path.dirname(wrapper)) +const executable = await fs.stat(wrapper) +process.stdout.write(JSON.stringify({ directoryMode: directory.mode & 0o777, executableMode: executable.mode & 0o777 }))` + const child = spawn(process.execPath, ['--import', 'tsx/esm', '--input-type=module', '-e', script], { + cwd: path.resolve('.'), stdio: ['ignore', 'pipe', 'pipe'], + }) + let stdout = '', stderr = '' + child.stdout?.setEncoding('utf8') + child.stderr?.setEncoding('utf8') + child.stdout?.on('data', chunk => { stdout += chunk }) + child.stderr?.on('data', chunk => { stderr += chunk }) + const exit = await new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => { + child.once('error', reject) + child.once('close', (code, signal) => resolve({ code, signal })) + }) + expect(exit, stderr).toEqual({ code: 0, signal: null }) + expect(JSON.parse(stdout)).toEqual({ directoryMode: 0o755, executableMode: 0o751 }) + } finally { + await fs.rm(directory, { recursive: true, force: true }) + } + }) + it('generates a runnable direct HOME MCP script that appends to its sibling file', async () => { const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-home-mcp-')) try { From 6940c48166c4526566cd104a4e10cad8e1702774 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 14:46:09 -0700 Subject: [PATCH 451/493] test(managed): assert private env mount target --- scripts/testing/managed-provider-environment-target.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index de0f2d0e0..08de9461f 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1551,7 +1551,9 @@ export async function runManagedEnvironmentTarget() { `workspace external-CWD MCP did not complete its receipt: ${JSON.stringify({ receipt: preflightCwdAliasMcp, mcpStatuses: preflightChild.mcpStatuses, wrapperProbe: preflightChild.wrapperProbe })}`) assert.equal(preflightCwdAliasMcp.helper, nonce) assert.equal(preflightCwdAliasMcp.envValueLoaded, true) - assert.match(preflightCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) + // The path alias resolves through the private indexed source mount, so + // realpath returns its read-only secret-mount destination. + assert.match(preflightCwdAliasMcp.envFileTarget, /^\/run\/freshell-secrets\/provider-\d+$/) assert.equal(preflightCwdAliasMcp.envFileOperand, '--env-file=.env', 'the workspace wrapper must retain its relative operand') assert.equal(preflightCwdAliasMcp.cwd, selected.cwdAliasProject) assert(preflightCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) @@ -1721,7 +1723,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(workspaceCwdAliasMcp.expectedNonce, nonce) assert.equal(workspaceCwdAliasMcp.helper, nonce) assert.equal(workspaceCwdAliasMcp.envValueLoaded, true) - assert.match(workspaceCwdAliasMcp.envFileTarget, /^\/home\/freshell\/provider\/\.freshell\/references\/\d+$/) + assert.match(workspaceCwdAliasMcp.envFileTarget, /^\/run\/freshell-secrets\/provider-\d+$/) assert.equal(workspaceCwdAliasMcp.envFileOperand, '--env-file=.env', 'the workspace wrapper must retain its relative operand') assert.equal(workspaceCwdAliasMcp.cwd, selected.cwdAliasProject) assert(workspaceCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) From e41edb1c970a11fd7cd79ea949cb87d0689029fd Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 15:11:49 -0700 Subject: [PATCH 452/493] test(managed): assert direct MCP wrapper cwd --- .../specs/managed-provider-environment-external.spec.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 12cf56f2a..5b941c716 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -1,4 +1,5 @@ import type { Page } from '@playwright/test' +import path from 'node:path' import { resolvePairedExternalFixture } from '../../../scripts/testing/e2e-external-target.js' import { test, expect } from '../helpers/fixtures.js' import { openPanePicker } from '../helpers/pane-picker.js' @@ -234,7 +235,9 @@ test(healthyTitle, async ({ page }, info) => { await expect.poll(async () => { directMcpReceipt = await readReceipt(page, `${pane.cwd}/.managed-fixture/direct-mcp-receipt.json`) return directMcpReceipt - }, { timeout: 120_000 }).toMatchObject({ expectedNonce: fixture.expectedNonce, helper: fixture.expectedNonce, cwd: pane.cwd }) + }, { timeout: 120_000 }).toMatchObject({ expectedNonce: fixture.expectedNonce, helper: fixture.expectedNonce }) + // The shell wrapper changes into its own MCP directory before launching Node. + expect(directMcpReceipt.cwd).toBe(path.dirname(directMcpReceipt.scriptPath)) expect(directMcpReceipt).toMatchObject({ sibling: fixture.expectedNonce, dependency: fixture.expectedNonce, schemaNonce: fixture.expectedNonce, envFileOperandRewritten: true, envValueLoaded: true }) From c7a0ae23a16eb3e8a39a075a4f1ec980b39b30d5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 15:29:29 -0700 Subject: [PATCH 453/493] test(managed): tolerate retry terminals in reconnect check --- .../managed-provider-environment-external.spec.ts | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 5b941c716..386469d55 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -338,6 +338,11 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { const diagnostic = page.locator('[data-testid="terminal-work-area"]') await expect(diagnostic).toContainText(/blocked|unable|failed/i) await expect(diagnostic).toContainText(scenario === 'missing-reference' ? /missing-selected/i : /startup-fixture/i) + const terminalIdsBeforeReconnect = (await json(page, '/api/terminals')) + .filter((item: any) => item.cwd === pane.cwd) + .map((item: any) => item.terminalId) + .sort() + expect(terminalIdsBeforeReconnect).toContain(terminalId) const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length) await new TestHarness(page).forceDisconnect() @@ -345,6 +350,9 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { expect((await activePane(page)).content).toMatchObject({ terminalId, status: 'error' }) expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) - expect((await json(page, '/api/terminals')).filter((item: any) => item.cwd === pane.cwd)).toHaveLength(1) + expect((await json(page, '/api/terminals')) + .filter((item: any) => item.cwd === pane.cwd) + .map((item: any) => item.terminalId) + .sort()).toEqual(terminalIdsBeforeReconnect) }) } From d9bc4d7a61f880ff4b51fd031256a479bc8c49e7 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 16:56:30 -0700 Subject: [PATCH 454/493] fix: scope OpenCode resources to selected worktree --- .../freshell-server/src/managed_provider_references/sources.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/freshell-server/src/managed_provider_references/sources.rs b/crates/freshell-server/src/managed_provider_references/sources.rs index e104ec66e..0a4cd7dc8 100644 --- a/crates/freshell-server/src/managed_provider_references/sources.rs +++ b/crates/freshell-server/src/managed_provider_references/sources.rs @@ -237,7 +237,7 @@ pub fn select_opencode_resource_roots( ) -> Result<Vec<DiscoveryRoot>, &'static str> { let disabled = (request.env)("OPENCODE_DISABLE_PROJECT_CONFIG") .is_some_and(|value| matches!(value.as_str(), "1" | "true")); - let mut roots = config_directories(request, None, disabled)?; + let mut roots = config_directories(request, Some(request.worktree), disabled)?; for (directory, role) in std::iter::once((request.home.to_path_buf(), DiscoveryRole::Home)) .chain( ancestors(request.cwd, Some(request.worktree)) From fd68d144f4349dba9b025bfc9f4217261bd2a451 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 16:56:33 -0700 Subject: [PATCH 455/493] test: cover checkpoint timeout and cooldown policy --- crates/freshell-server/src/checkpoints.rs | 682 +++++++++++++++++++++- 1 file changed, 675 insertions(+), 7 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 08d1248ba..903acd33a 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -267,6 +267,20 @@ pub struct CheckpointService { volatile_outcomes: StdMutex<BTreeMap<String, StoredCheckpointOutcome>>, #[cfg(test)] test_capture_attempt_override: StdMutex<Option<CaptureAttempt>>, + #[cfg(test)] + test_auto_budget_override: StdMutex<Option<Duration>>, + #[cfg(test)] + test_explicit_budget_override: StdMutex<Option<Duration>>, + #[cfg(test)] + test_cooldown_override: StdMutex<Option<Duration>>, + #[cfg(test)] + test_clock_advance: StdMutex<Duration>, + #[cfg(test)] + test_capture_delays: StdMutex<std::collections::VecDeque<Duration>>, + #[cfg(test)] + test_capture_calls: std::sync::atomic::AtomicUsize, + #[cfg(test)] + test_capture_entered: tokio::sync::Notify, } tokio::task_local! { @@ -282,6 +296,8 @@ struct CheckpointCaptureControl { jobs: StdMutex<Vec<Arc<WindowsJob>>>, #[cfg(test)] test_settlement_gate: StdMutex<Option<Arc<TestSettlementGate>>>, + #[cfg(test)] + test_cancel_notify: tokio::sync::Notify, } #[cfg(test)] @@ -377,6 +393,8 @@ impl CheckpointCaptureControl { async fn cancel_processes(&self) -> std::io::Result<()> { self.cancelled.store(true, Ordering::SeqCst); + #[cfg(test)] + self.test_cancel_notify.notify_one(); #[cfg(unix)] { let groups = self @@ -793,9 +811,72 @@ impl CheckpointService { volatile_outcomes: StdMutex::new(BTreeMap::new()), #[cfg(test)] test_capture_attempt_override: StdMutex::new(None), + #[cfg(test)] + test_auto_budget_override: StdMutex::new(None), + #[cfg(test)] + test_explicit_budget_override: StdMutex::new(None), + #[cfg(test)] + test_cooldown_override: StdMutex::new(None), + #[cfg(test)] + test_clock_advance: StdMutex::new(Duration::ZERO), + #[cfg(test)] + test_capture_delays: StdMutex::new(std::collections::VecDeque::new()), + #[cfg(test)] + test_capture_calls: std::sync::atomic::AtomicUsize::new(0), + #[cfg(test)] + test_capture_entered: tokio::sync::Notify::new(), } } + fn automatic_capture_budget(&self) -> Duration { + #[cfg(test)] + if let Some(budget) = *self + .test_auto_budget_override + .lock() + .expect("automatic checkpoint budget mutex poisoned") + { + return budget; + } + CHECKPOINT_AUTO_BUDGET + } + + fn explicit_capture_budget(&self) -> Duration { + #[cfg(test)] + if let Some(budget) = *self + .test_explicit_budget_override + .lock() + .expect("explicit checkpoint budget mutex poisoned") + { + return budget; + } + CHECKPOINT_EXPLICIT_BUDGET + } + + fn timeout_cooldown(&self) -> Duration { + #[cfg(test)] + if let Some(cooldown) = *self + .test_cooldown_override + .lock() + .expect("checkpoint cooldown mutex poisoned") + { + return cooldown; + } + CHECKPOINT_TIMEOUT_COOLDOWN + } + + fn checkpoint_monotonic_now(&self) -> Instant { + #[cfg(test)] + { + return Instant::now() + + *self + .test_clock_advance + .lock() + .expect("checkpoint test clock mutex poisoned"); + } + #[cfg(not(test))] + Instant::now() + } + pub fn start_observer(self: &Arc<Self>) { let mut receiver = self.broadcast_tx.subscribe(); let service = Arc::downgrade(self); @@ -1525,7 +1606,7 @@ impl CheckpointService { } fn active_cooldown(&self, workspace: &Path) -> Option<Duration> { - let now = Instant::now(); + let now = self.checkpoint_monotonic_now(); let mut cooldowns = self .cooldowns .lock() @@ -1537,7 +1618,7 @@ impl CheckpointService { } fn establish_cooldown(&self, workspace: &Path) { - let now = Instant::now(); + let now = self.checkpoint_monotonic_now(); let mut cooldowns = self .cooldowns .lock() @@ -1546,7 +1627,7 @@ impl CheckpointService { cooldowns.insert( workspace.to_path_buf(), CheckpointCooldown { - expires_at: now + CHECKPOINT_TIMEOUT_COOLDOWN, + expires_at: now + self.timeout_cooldown(), }, ); while cooldowns.len() > CHECKPOINT_COOLDOWN_LIMIT { @@ -1691,7 +1772,8 @@ impl CheckpointService { .await }, )); - let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); + let automatic_budget = self.automatic_capture_budget(); + let remaining = automatic_budget.saturating_sub(started_at.elapsed()); let workspace = match tokio::time::timeout(remaining, &mut resolve_task).await { Ok(Ok(Ok(workspace))) => workspace, Ok(Ok(Err(error))) => { @@ -1792,7 +1874,7 @@ impl CheckpointService { ); } - let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); + let remaining = automatic_budget.saturating_sub(started_at.elapsed()); if remaining.is_zero() { log_automatic_checkpoint_failure( Some(request_id), @@ -1830,7 +1912,7 @@ impl CheckpointService { ); } }; - let remaining = CHECKPOINT_AUTO_BUDGET.saturating_sub(started_at.elapsed()); + let remaining = automatic_budget.saturating_sub(started_at.elapsed()); if remaining.is_zero() { drop(lock); log_automatic_checkpoint_failure( @@ -2041,7 +2123,7 @@ impl CheckpointService { label: String, request_id: Option<String>, ) -> Result<(Value, ResolvedCheckpointWorkspace), ExplicitCaptureError> { - let deadline = Instant::now() + CHECKPOINT_EXPLICIT_BUDGET; + let deadline = Instant::now() + self.explicit_capture_budget(); let (cancel_tx, cancel_rx) = watch::channel(false); let cancellation_guard = CaptureRequestCancellation { sender: cancel_tx }; let service = Arc::clone(self); @@ -2237,6 +2319,12 @@ impl CheckpointService { budget: Duration, mut cancellation: Option<watch::Receiver<bool>>, ) -> CaptureAttempt { + #[cfg(test)] + self.test_capture_calls + .fetch_add(1, std::sync::atomic::Ordering::SeqCst); + #[cfg(test)] + self.test_capture_entered.notify_one(); + #[cfg(test)] if let Some(attempt) = self .test_capture_attempt_override @@ -2256,7 +2344,30 @@ impl CheckpointService { let label = label.to_string(); let request_id_owned = request_id.to_string(); let task_control = Arc::clone(&control); + #[cfg(test)] + let task_control_for_delay = Arc::clone(&task_control); + #[cfg(test)] + let test_delay = self + .test_capture_delays + .lock() + .expect("checkpoint test capture delay queue poisoned") + .pop_front() + .unwrap_or(Duration::ZERO); let mut task = tokio::spawn(CHECKPOINT_CAPTURE_CONTEXT.scope(task_control, async move { + #[cfg(test)] + if !test_delay.is_zero() { + let cancelled = task_control_for_delay.test_cancel_notify.notified(); + tokio::pin!(cancelled); + if task_control_for_delay.cancelled.load(Ordering::SeqCst) { + return Err("checkpoint test capture delay was cancelled".to_string()); + } + tokio::select! { + _ = tokio::time::sleep(test_delay) => {} + _ = &mut cancelled => { + return Err("checkpoint test capture delay was cancelled".to_string()); + } + } + } create_checkpoint( &home, &workspace_path, @@ -5388,6 +5499,75 @@ mod tests { } } + fn checkpoint_policy_send( + workspace: &Path, + request_id: &str, + attempt_id: &str, + ) -> FreshAgentSend { + let path = workspace.to_string_lossy().into_owned(); + let mut send = review_r6_send(Some(path), Some(workspace), request_id); + send.checkpoint_intent + .as_mut() + .expect("test send has checkpoint intent") + .logical_attempt_id = attempt_id.to_string(); + send + } + + fn checkpoint_test_service(home: &Path) -> Arc<CheckpointService> { + let (broadcast_tx, _) = tokio::sync::broadcast::channel(32); + Arc::new(CheckpointService::new( + Arc::new(home.to_path_buf()), + Arc::new(broadcast_tx), + )) + } + + async fn initialize_checkpoint_workspace(workspace: &Path) { + tokio::fs::create_dir_all(workspace).await.unwrap(); + tokio::fs::write(workspace.join("source.txt"), "checkpoint source\n") + .await + .unwrap(); + run_git(&["init", "-q"], Some(workspace)).await.unwrap(); + } + + async fn run_checkpoint_policy_send( + service: Arc<CheckpointService>, + send: FreshAgentSend, + ) -> (Value, usize, Duration) { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let mut outcomes = service.broadcast_tx.subscribe(); + let dispatches = Arc::new(AtomicUsize::new(0)); + let dispatched = Arc::clone(&dispatches); + let started = Instant::now(); + service + .clone() + .barrier_and_forward( + send, + Box::pin(async move { + dispatched.fetch_add(1, Ordering::SeqCst); + }), + ) + .await; + let elapsed = started.elapsed(); + let frame = tokio::time::timeout(Duration::from_secs(2), outcomes.recv()) + .await + .expect("checkpoint outcome should be broadcast promptly") + .expect("checkpoint outcome should not be lost"); + ( + serde_json::from_str(&frame).unwrap(), + dispatches.load(Ordering::SeqCst), + elapsed, + ) + } + + async fn published_checkpoint_entries( + service: &CheckpointService, + workspace: &Path, + ) -> Vec<Value> { + list_checkpoints(&service.home, workspace).await.unwrap() + } + fn headers_with_token(token: &str) -> HeaderMap { let mut headers = HeaderMap::new(); headers.insert("x-auth-token", token.parse().unwrap()); @@ -7451,6 +7631,494 @@ mod tests { .contains("private prompt marker")); } + #[tokio::test] + async fn capture_timeout_establishes_cooldown() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(40)); + service + .test_capture_delays + .lock() + .unwrap() + .push_back(Duration::from_secs(5)); + + let (outcome, dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send(workspace.path(), "timeout-send", "timeout-attempt"), + ) + .await; + + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "capture_timeout"); + assert_eq!(dispatches, 1, "the failed snapshot must not block the send"); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 1); + assert!(service.active_cooldown(workspace.path()).is_some()); + assert!(published_checkpoint_entries(&service, workspace.path()) + .await + .is_empty()); + } + + #[tokio::test] + async fn cooldown_sends_immediately_without_capture() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let service = checkpoint_test_service(home.path()); + service.establish_cooldown(workspace.path()); + + let (outcome, dispatches, elapsed) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send(workspace.path(), "cooldown-send", "cooldown-attempt"), + ) + .await; + + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "capture_timeout_cooldown"); + assert_eq!(dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 0); + assert!( + elapsed < Duration::from_secs(1), + "cooldown should not repeat the capture wait: {elapsed:?}" + ); + } + + #[tokio::test] + async fn cooldown_scope_change_attempts_capture() { + let home = tempfile::tempdir().unwrap(); + let timed_out_workspace = tempfile::tempdir().unwrap(); + let changed_workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(changed_workspace.path()).await; + let service = checkpoint_test_service(home.path()); + service.establish_cooldown(timed_out_workspace.path()); + + let (outcome, dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send( + changed_workspace.path(), + "changed-scope-send", + "changed-scope-attempt", + ), + ) + .await; + + assert_eq!(outcome["status"], "available"); + assert_eq!(dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 1); + assert!(service + .active_cooldown(timed_out_workspace.path()) + .is_some()); + assert!(service.active_cooldown(changed_workspace.path()).is_none()); + assert_eq!( + published_checkpoint_entries(&service, changed_workspace.path()) + .await + .len(), + 1 + ); + } + + #[tokio::test] + async fn explicit_capture_bypasses_cooldown() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + service.establish_cooldown(workspace.path()); + + let intent = json!({ + "path": workspace.path().to_string_lossy(), + "provenance": "directory-picker" + }); + let result = service + .capture_explicit( + workspace.path().to_path_buf(), + Some(intent), + "explicit retry".into(), + Some("explicit-cooldown-retry".into()), + ) + .await; + + let (entry, _) = match result { + Ok(value) => value, + Err(_) => panic!("explicit capture should attempt and publish during cooldown"), + }; + assert!(entry["id"].as_str().is_some()); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 1); + assert!(service.active_cooldown(workspace.path()).is_none()); + assert_eq!( + published_checkpoint_entries(&service, workspace.path()) + .await + .len(), + 1 + ); + } + + #[tokio::test] + async fn repaired_scope_needs_published_success() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(40)); + service + .test_capture_delays + .lock() + .unwrap() + .push_back(Duration::from_secs(5)); + + let failed_send = + checkpoint_policy_send(workspace.path(), "repair-timeout", "repair-attempt-1"); + let (first, first_dispatches, _) = + run_checkpoint_policy_send(Arc::clone(&service), failed_send.clone()).await; + assert_eq!(first["code"], "capture_timeout"); + assert_eq!(first_dispatches, 1); + + tokio::fs::write(workspace.path().join("source.txt"), "repaired source\n") + .await + .unwrap(); + let (during_cooldown, second_dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send( + workspace.path(), + "repair-during-cooldown", + "repair-attempt-2", + ), + ) + .await; + assert_eq!(during_cooldown["code"], "capture_timeout_cooldown"); + assert_eq!(second_dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 1); + assert!(published_checkpoint_entries(&service, workspace.path()) + .await + .is_empty()); + + let intent = json!({ + "path": workspace.path().to_string_lossy(), + "provenance": "directory-picker" + }); + let explicit = service + .capture_explicit( + workspace.path().to_path_buf(), + Some(intent), + "repaired retry".into(), + Some("repair-explicit".into()), + ) + .await; + assert!( + explicit.is_ok(), + "a successful explicit retry publishes a complete target" + ); + assert!(service.active_cooldown(workspace.path()).is_none()); + assert_eq!( + published_checkpoint_entries(&service, workspace.path()) + .await + .len(), + 1 + ); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(3)); + + let (after_publish, third_dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send(workspace.path(), "repair-after-publish", "repair-attempt-3"), + ) + .await; + assert_eq!(after_publish["status"], "available"); + assert_eq!(third_dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 3); + let prior = service + .find_outcome(&checkpoint_attempt_key(&failed_send, "repair-attempt-1")) + .await + .unwrap() + .unwrap(); + assert_eq!( + prior.outcome.status, + FreshAgentCheckpointStatus::Unavailable + ); + assert_eq!(prior.outcome.code.as_deref(), Some("capture_timeout")); + } + + #[tokio::test] + async fn cooldown_expiry_retries() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + service.establish_cooldown(workspace.path()); + *service.test_clock_advance.lock().unwrap() += Duration::from_secs(61); + + let (outcome, dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send( + workspace.path(), + "cooldown-expired", + "cooldown-expired-attempt", + ), + ) + .await; + + assert_eq!(outcome["status"], "available"); + assert_eq!(dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 1); + assert!(service.active_cooldown(workspace.path()).is_none()); + assert_eq!( + published_checkpoint_entries(&service, workspace.path()) + .await + .len(), + 1 + ); + } + + #[tokio::test] + async fn explicit_longer_budget_bootstraps_complete_index() { + assert_eq!(CHECKPOINT_AUTO_BUDGET, Duration::from_secs(15)); + assert_eq!(CHECKPOINT_EXPLICIT_BUDGET, Duration::from_secs(120)); + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + service + .test_capture_delays + .lock() + .unwrap() + .extend([Duration::from_secs(20), Duration::from_secs(20)]); + + let (automatic, dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send( + workspace.path(), + "bootstrap-auto-timeout", + "bootstrap-attempt-1", + ), + ) + .await; + assert_eq!(automatic["code"], "capture_timeout"); + assert_eq!(dispatches, 1); + assert!(published_checkpoint_entries(&service, workspace.path()) + .await + .is_empty()); + assert!(!checkpoint_git_dir(&service.home, workspace.path()) + .join("index") + .exists()); + + let intent = json!({ + "path": workspace.path().to_string_lossy(), + "provenance": "directory-picker" + }); + let (first_entry, _) = match service + .capture_explicit( + workspace.path().to_path_buf(), + Some(intent), + "longer first-index capture".into(), + Some("bootstrap-explicit".into()), + ) + .await + { + Ok(value) => value, + Err(_) => { + panic!("the explicit 120-second budget should finish the delayed first capture") + } + }; + let first_id = first_entry["id"].as_str().unwrap().to_string(); + let index_path = checkpoint_git_dir(&service.home, workspace.path()).join("index"); + assert!(tokio::fs::metadata(&index_path).await.is_ok()); + assert_eq!( + published_checkpoint_entries(&service, workspace.path()) + .await + .len(), + 1 + ); + + tokio::fs::write( + workspace.path().join("source.txt"), + "mutated after capture\n", + ) + .await + .unwrap(); + restore_checkpoint(&service.home, workspace.path(), &first_id) + .await + .unwrap(); + assert_eq!( + tokio::fs::read_to_string(workspace.path().join("source.txt")) + .await + .unwrap(), + "checkpoint source\n" + ); + + let (after_bootstrap, next_dispatches, elapsed) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send( + workspace.path(), + "bootstrap-auto-success", + "bootstrap-attempt-2", + ), + ) + .await; + assert_eq!(after_bootstrap["status"], "available"); + assert_eq!(next_dispatches, 1); + assert!( + elapsed < CHECKPOINT_AUTO_BUDGET, + "the published index should permit a fast automatic capture: {elapsed:?}" + ); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 3); + assert_eq!( + published_checkpoint_entries(&service, workspace.path()) + .await + .len(), + 2 + ); + } + + #[tokio::test] + async fn lock_wait_timeout_does_not_establish_cooldown() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(50)); + let holder = service + .lock_workspace(workspace.path(), Duration::from_secs(1)) + .await + .unwrap(); + + let (timed_out, dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send(workspace.path(), "lock-wait-timeout", "lock-attempt-1"), + ) + .await; + assert_eq!(timed_out["code"], "workspace_lock_timeout"); + assert_eq!(dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 0); + assert!(service.active_cooldown(workspace.path()).is_none()); + drop(holder); + + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(3)); + let (after_wait, next_dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send(workspace.path(), "lock-released", "lock-attempt-2"), + ) + .await; + assert_eq!(after_wait["status"], "available"); + assert_eq!(next_dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 1); + } + + #[tokio::test] + async fn explicit_lock_wait_timeout_does_not_renew_cooldown() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + let service = checkpoint_test_service(home.path()); + *service.test_explicit_budget_override.lock().unwrap() = Some(Duration::from_millis(50)); + service.establish_cooldown(workspace.path()); + let before = service.cooldowns.lock().unwrap()[workspace.path()].expires_at; + let holder = service + .lock_workspace(workspace.path(), Duration::from_secs(1)) + .await + .unwrap(); + let intent = json!({ + "path": workspace.path().to_string_lossy(), + "provenance": "directory-picker" + }); + + let error = service + .capture_explicit( + workspace.path().to_path_buf(), + Some(intent), + "blocked explicit retry".into(), + Some("explicit-lock-timeout".into()), + ) + .await + .expect_err("explicit lock wait should time out"); + assert!(matches!(error, ExplicitCaptureError::WorkspaceBusy(_))); + assert_eq!( + service.cooldowns.lock().unwrap()[workspace.path()].expires_at, + before + ); + assert!(service.active_cooldown(workspace.path()).is_some()); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 0); + drop(holder); + } + + #[tokio::test] + async fn healthy_holder_and_next_pane_capture_after_waiter_timeout() { + let home = tempfile::tempdir().unwrap(); + let workspace = tempfile::tempdir().unwrap(); + initialize_checkpoint_workspace(workspace.path()).await; + let service = checkpoint_test_service(home.path()); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(50)); + service + .test_capture_delays + .lock() + .unwrap() + .push_back(Duration::from_millis(250)); + let holder_workspace = ResolvedCheckpointWorkspace { + path: canonical_workspace(workspace.path()).await.unwrap(), + provenance: "directory-picker".to_string(), + }; + let holder_service = Arc::clone(&service); + let holder_path = workspace.path().to_path_buf(); + let capture_entered = service.test_capture_entered.notified(); + tokio::pin!(capture_entered); + let holder = tokio::spawn(async move { + let lock = holder_service + .lock_workspace(&holder_path, Duration::from_secs(1)) + .await + .unwrap(); + let result = holder_service + .capture_with_budget( + &holder_workspace, + "healthy holder", + "healthy-holder-request", + Duration::from_secs(2), + None, + ) + .await; + drop(lock); + result + }); + tokio::time::timeout(Duration::from_secs(1), &mut capture_entered) + .await + .expect("healthy holder should enter capture while retaining its lock"); + + let (waiter, waiter_dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send(workspace.path(), "waiter-times-out", "waiter-attempt"), + ) + .await; + assert_eq!(waiter["code"], "workspace_lock_timeout"); + assert_eq!(waiter_dispatches, 1); + assert!(service.active_cooldown(workspace.path()).is_none()); + + match tokio::time::timeout(Duration::from_secs(3), holder) + .await + .expect("healthy capture should finish after the independent waiter timed out") + .expect("healthy holder task should complete") + { + CaptureAttempt::Published(_) => {} + _ => panic!("healthy holder must publish its complete target"), + } + assert_eq!( + published_checkpoint_entries(&service, workspace.path()) + .await + .len(), + 1 + ); + + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(3)); + let (next_pane, next_dispatches, _) = run_checkpoint_policy_send( + Arc::clone(&service), + checkpoint_policy_send( + workspace.path(), + "next-pane-after-waiter", + "next-pane-attempt", + ), + ) + .await; + assert_eq!(next_pane["status"], "available"); + assert_eq!(next_dispatches, 1); + assert_eq!(service.test_capture_calls.load(Ordering::SeqCst), 2); + assert!(service.active_cooldown(workspace.path()).is_none()); + } + #[tokio::test] async fn home_launch_without_workspace_does_not_publish_checkpoint() { let home_dir = tempfile::tempdir().unwrap(); From 1ed7d8f11cc49b067ebf3d7ac503d569a102295a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 16:56:35 -0700 Subject: [PATCH 456/493] fix: give strict sandbox installs a writable pnpm store --- docker/sandbox/Dockerfile | 11 ++++++++--- docker/sandbox/entrypoint.sh | 1 + docs/development/test-sandbox.md | 6 ++++-- scripts/sandbox-test.sh | 1 + 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/docker/sandbox/Dockerfile b/docker/sandbox/Dockerfile index e36ae88c8..066ffe11b 100644 --- a/docker/sandbox/Dockerfile +++ b/docker/sandbox/Dockerfile @@ -60,9 +60,9 @@ RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \ # the image for legacy npm-era branches: docker/sandbox/entrypoint.sh # selects the manager per tree (packageManager field, else lock presence), # so old package-lock.json-only branches keep their `npm ci` path. pnpm's -# per-user store and state land under the sandbox user's HOME -# (/home/sandbox, sandbox-owned by construction), so no extra chown or -# VOLUME entries are needed for it. +# per-user store and state use an explicit sandbox-owned volume. Runtime-suite +# containers have a read-only root, so the store cannot live only in the +# container filesystem. RUN npm install -g pnpm@10.34.5 # pnpm's default store-path resolution probes the project root's @@ -94,6 +94,11 @@ RUN if getent passwd "${UID}" >/dev/null 2>&1; then \ fi \ && chown -R "${UID}:${GID}" /usr/local/cargo /usr/local/rustup +# This writable target must exist in the image before the named pnpm store is +# mounted, because runtime-suite containers have a read-only root filesystem. +RUN mkdir -p /home/sandbox/.local/share/pnpm/store \ + && chown -R "${UID}:${GID}" /home/sandbox/.local + # Playwright's own installer needs its system dependency list resolved once # a browser version is known (that happens per-run from package.json at # runtime), so we only pre-install the OS libraries here as a cache-friendly diff --git a/docker/sandbox/entrypoint.sh b/docker/sandbox/entrypoint.sh index 0ed75c6f6..6d100f94f 100755 --- a/docker/sandbox/entrypoint.sh +++ b/docker/sandbox/entrypoint.sh @@ -71,6 +71,7 @@ chown "${TARGET_UID}:${TARGET_GID}" /home/sandbox/.cache VOLUME_DIRS=( "/usr/local/cargo/registry" "/usr/local/cargo/git" + "/home/sandbox/.local/share/pnpm/store" "/workspace/target" "/workspace/node_modules" "/home/sandbox/.cache/ms-playwright" diff --git a/docs/development/test-sandbox.md b/docs/development/test-sandbox.md index b451b008b..247e7f158 100644 --- a/docs/development/test-sandbox.md +++ b/docs/development/test-sandbox.md @@ -113,8 +113,10 @@ lock presence, npm's `package-lock.json` first so legacy branches keep their explicit `--store-dir /home/sandbox/.local/share/pnpm/store`. The explicit store dir is load-bearing: pnpm's default store-path resolution probes the project root's writability, and the bind-mounted repo is not writable by the - sandbox user, so the default probe would fail every install. The store lives - inside the container at the same per-user location pnpm would pick itself. + sandbox user, so the default probe would fail every install. The store uses + the sandbox-owned `freshell-sandbox-pnpm-store` volume. This keeps it writable + under `--runtime-suite`'s read-only container root and lets its offline + dependency install reuse packages prepared by an ordinary sandbox run. - Install success is recorded in `node_modules/.sandbox-deps-state` — a fingerprint over the manager and its version, the install policy flags, the root manifest, the selected manager's lock/config files, and the workspace diff --git a/scripts/sandbox-test.sh b/scripts/sandbox-test.sh index 80c3ea65e..f75d9e2a3 100755 --- a/scripts/sandbox-test.sh +++ b/scripts/sandbox-test.sh @@ -344,6 +344,7 @@ DOCKER_ARGS=( -v "${REPO_ROOT}:/workspace" -v freshell-sandbox-cargo-registry:/usr/local/cargo/registry -v freshell-sandbox-cargo-git:/usr/local/cargo/git + -v freshell-sandbox-pnpm-store:/home/sandbox/.local/share/pnpm/store -v freshell-sandbox-cargo-target:/workspace/target -v freshell-sandbox-node-modules:/workspace/node_modules # pnpm workspace members with their own dependencies get their own From da3ef36dc86a60eec6d23176c77a7815f72d92b4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 16:56:38 -0700 Subject: [PATCH 457/493] docs: document managed references and copy limits --- docs/development/managed-runtime.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/development/managed-runtime.md b/docs/development/managed-runtime.md index d3f072a9c..8aa27e8a0 100644 --- a/docs/development/managed-runtime.md +++ b/docs/development/managed-runtime.md @@ -357,6 +357,30 @@ being silently removed. Native conversation databases and credential sources are excluded from ordinary resource copying and continue to use their dedicated store or secret-reference paths. +The shared resource copier accepts at most 100,000 entries, 512 MiB of file +contents in total, 128 MiB for any one file, and 64 directory levels; copied +metadata is capped at 64 MiB. A selected configuration source is limited to +16 MiB, with 64 MiB total for simultaneously retained source views. When a +selected file reference cannot be resolved, preparation reports +`SELECTED_REFERENCE_UNAVAILABLE` at stage `configuration`, operation +`resolve`, with the selected field and path in the user-facing detail. Repair +the named file or its declaring configuration rather than relying on Freshell +to omit that capability. + +For OpenCode, `--env-file` and `--env-file-if-exists` paths selected from an +enabled local MCP command are handled as private file references. Freshell +resolves them from the MCP working directory or the selected wrapper's actual +script directory, excludes the source file from the ordinary resource copy, +and exposes it inside the provider home under +`/home/freshell/provider/.freshell/references/<index>`. Direct MCP arguments +and supported copied shell wrappers point Node at that private file; a wrapper +that runs from the mounted workspace can use a managed alias in its MCP +working directory. On recovery, Freshell removes only aliases recorded as its +own and no longer used by the current configuration. Node reads the file when +the MCP child starts, so the resulting variables belong to that child. The +file contents are not copied into ordinary resources, launch configuration, +registry metadata, or logs. + Modern fresh-agent sends may include a checkpoint attempt ID. The server takes and verifies the pre-turn code snapshot for the selected workspace, then records an outcome tied to that exact send before forwarding it to the provider. From b83c308782488db56d74ba0fb6324c6b8cae4843 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:16:49 -0700 Subject: [PATCH 458/493] test: wait for retained pane state after reconnect --- .../specs/managed-provider-environment-external.spec.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 386469d55..6f77b701e 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -310,7 +310,8 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { pane.createRequestId)).toBe(1) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) - expect((await activePane(page)).content).toMatchObject(retained) + await expect.poll(async () => (await activePane(page)).content, { timeout: 30_000 }) + .toMatchObject(retained) expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) await assertNoHandle() @@ -347,7 +348,8 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { .filter((message: any) => message.type === 'terminal.create').length) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) - expect((await activePane(page)).content).toMatchObject({ terminalId, status: 'error' }) + await expect.poll(async () => (await activePane(page)).content, { timeout: 30_000 }) + .toMatchObject({ terminalId, status: 'error' }) expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) expect((await json(page, '/api/terminals')) From 8ecd4bfcb2c281b163ba5e91bd6b7805d38c4037 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:45:05 -0700 Subject: [PATCH 459/493] test: stabilize checkpoint cooldown log test --- crates/freshell-server/src/checkpoints.rs | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 903acd33a..a8f36eaec 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -7592,6 +7592,7 @@ mod tests { let (broadcast_tx, _) = tokio::sync::broadcast::channel(8); let service = CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(60)); service.establish_cooldown(workspace.path()); let events = Arc::new(std::sync::Mutex::new(Vec::new())); let subscriber = tracing_subscriber::registry().with(ReviewR6CaptureLogs(events.clone())); @@ -7611,14 +7612,16 @@ mod tests { FreshAgentCheckpointStatus::Unavailable ); let events = events.lock().unwrap(); - let failure = events - .iter() - .find(|event| { - event - .get("failure_code") - .is_some_and(|value| value == "capture_timeout_cooldown") - }) - .expect("cooldown unavailable outcome must emit a structured failure event"); + let failure = events.iter().find(|event| { + event + .get("failure_code") + .is_some_and(|value| value == "capture_timeout_cooldown") + }); + assert!( + failure.is_some(), + "cooldown unavailable outcome must emit a structured failure event; captured={events:#?}" + ); + let failure = failure.expect("cooldown failure event was checked above"); assert_eq!(failure["send_request_id"], "review-r6-cooldown"); assert_eq!(failure["stage"], "capture"); assert_eq!(failure["operation"], "automatic-pre-turn-snapshot"); From 1a652377d781c0161e5c9573b77cf18fcab2a70a Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:16:48 -0700 Subject: [PATCH 460/493] test: delay checkpoint fixture turn completion --- scripts/testing/managed-provider-environment-target.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 08de9461f..43d0295b5 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1110,7 +1110,10 @@ export async function runCodeCheckpointTarget() { `FRESHELL_LOG_DIR=${checkpointFixtureRoot}/storage/logs`, 'FRESHELL_CLIENT_DIR=/checkpoint-target/client', 'CODEX_CMD=/checkpoint-target/fake-codex', - 'FAKE_CODEX_APP_SERVER_BEHAVIOR={"recordTurns":true,"uniqueThreadIds":true,"persistRolloutUserMessages":true}', + // Keep the fake turn active long enough for the backend to record the + // turn/start result before the completion notifications arrive. The + // shorter default can race that state update on the remote browser path. + 'FAKE_CODEX_APP_SERVER_BEHAVIOR={"recordTurns":true,"uniqueThreadIds":true,"persistRolloutUserMessages":true,"turnCompleteDelayMs":1000}', 'FAKE_CODEX_APP_SERVER_ALLOW_DURABLE_WRITES=1', `FRESHELL_CHECKPOINT_FIXTURE_ROOT=${checkpointFixtureRoot}`, `FRESHELL_CHECKPOINT_FIXTURE_UID=${identity.uid}`, From 2ac3055001fd9891aef66e46cc7925dc12cf4260 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 12:32:53 -0700 Subject: [PATCH 461/493] Index retained output spool replay --- .../src/terminal_spool.rs | 410 +++++++++++++++++- .../src/output_journal.rs | 215 +++++++-- .../src/retained_output.rs | 406 +++++++++++++---- 3 files changed, 898 insertions(+), 133 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/terminal_spool.rs b/crates/freshell-runtime-protocol/src/terminal_spool.rs index 730512a11..96449e479 100644 --- a/crates/freshell-runtime-protocol/src/terminal_spool.rs +++ b/crates/freshell-runtime-protocol/src/terminal_spool.rs @@ -2,7 +2,7 @@ //! belong to the session host and supervisor, never to this codec. use crate::{IncarnationId, RuntimeOutputBatch, RuntimeOutputFrame}; use serde::{Deserialize, Serialize}; -use std::io::{BufRead, Read}; +use std::io::{BufRead, Read, Seek, SeekFrom, Write}; pub const MAX_OUTPUT_FRAME_BYTES: usize = 64 * 1024; pub const MAX_SPOOL_SEGMENT_BYTES: u64 = 32 * 1024 * 1024; @@ -11,6 +11,10 @@ pub const SPOOL_DESCRIPTOR_FILE: &str = "terminal-spool-descriptor.json"; pub const SPOOL_COMPLETION_FILE: &str = "terminal-spool-completion.json"; pub const SPOOL_CURRENT_FILE: &str = "terminal-spool-current.jsonl"; pub const SPOOL_PREVIOUS_FILE: &str = "terminal-spool-previous.jsonl"; +pub const SPOOL_CURRENT_INDEX_FILE: &str = "terminal-spool-current.index.jsonl"; +pub const SPOOL_PREVIOUS_INDEX_FILE: &str = "terminal-spool-previous.index.jsonl"; +pub const SPOOL_INDEX_STRIDE_BYTES: u64 = 64 * 1024; +pub const MAX_SPOOL_INDEX_BYTES: usize = 256 * 1024; #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] @@ -74,6 +78,178 @@ pub struct TerminalSpoolFrame { pub frame: RuntimeOutputFrame, } +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "record", rename_all = "snake_case")] +pub enum TerminalSpoolIndexRecord { + Header { descriptor: TerminalSpoolDescriptor }, + Checkpoint { seq: u64, offset: u64 }, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct TerminalSpoolIndexEntry { + pub seq: u64, + pub offset: u64, +} + +pub struct IndexedSpoolSegment<R> { + pub reader: R, + pub file_bytes: u64, + pub checkpoints: Vec<TerminalSpoolIndexEntry>, +} + +pub struct IndexedSpoolRead { + pub batch: RuntimeOutputBatch, + pub bytes_scanned: u64, +} + +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct SpoolScanSummary { + pub torn_tail: bool, + pub bytes_scanned: u64, + pub stopped_early: bool, +} + +pub fn read_spool_index<R: BufRead>( + mut reader: R, + source: &TerminalSpoolDescriptor, + spool_bytes: u64, +) -> Result<Vec<TerminalSpoolIndexEntry>, String> { + source.validate()?; + let mut bytes = Vec::new(); + Read::by_ref(&mut reader) + .take((MAX_SPOOL_INDEX_BYTES + 1) as u64) + .read_to_end(&mut bytes) + .map_err(|_| "retained output index read failed")?; + if bytes.is_empty() || bytes.len() > MAX_SPOOL_INDEX_BYTES || bytes.last() != Some(&b'\n') { + return Err("invalid retained output index size or framing".into()); + } + + let mut lines = bytes.split(|byte| *byte == b'\n'); + let header_line = lines + .next() + .filter(|line| !line.is_empty()) + .ok_or("retained output index header missing")?; + let header: TerminalSpoolIndexRecord = + serde_json::from_slice(header_line).map_err(|_| "invalid retained output index header")?; + if header + != (TerminalSpoolIndexRecord::Header { + descriptor: source.clone(), + }) + { + return Err("retained output index source mismatch".into()); + } + + let mut entries = Vec::new(); + for line in lines { + if line.is_empty() { + continue; + } + let record: TerminalSpoolIndexRecord = + serde_json::from_slice(line).map_err(|_| "invalid retained output index checkpoint")?; + let TerminalSpoolIndexRecord::Checkpoint { seq, offset } = record else { + return Err("unexpected retained output index header".into()); + }; + entries.push(TerminalSpoolIndexEntry { seq, offset }); + } + validate_spool_index_entries(&entries, spool_bytes)?; + Ok(entries) +} + +pub fn validate_spool_index_entries( + entries: &[TerminalSpoolIndexEntry], + spool_bytes: u64, +) -> Result<(), String> { + let maximum_entries = spool_bytes / SPOOL_INDEX_STRIDE_BYTES + 2; + if entries.len() as u64 > maximum_entries { + return Err("retained output index exceeds checkpoint bound".into()); + } + let mut previous = None; + for entry in entries { + if entry.seq == 0 + || entry.seq >= i64::MAX as u64 + || entry.offset >= spool_bytes + || previous.is_some_and(|(seq, offset)| entry.seq <= seq || entry.offset <= offset) + { + return Err("invalid retained output index checkpoint bounds".into()); + } + previous = Some((entry.seq, entry.offset)); + } + Ok(()) +} + +pub fn build_spool_index<R: BufRead>( + reader: R, + source: &TerminalSpoolDescriptor, + start_offset: u64, +) -> Result<(Vec<TerminalSpoolIndexEntry>, u64, SpoolScanSummary), String> { + let mut checkpoints = Vec::new(); + let mut head = 0; + let summary = scan_spool_from(reader, source, start_offset, |offset, frame| { + if checkpoints + .last() + .is_none_or(|last: &TerminalSpoolIndexEntry| { + offset.saturating_sub(last.offset) >= SPOOL_INDEX_STRIDE_BYTES + }) + { + checkpoints.push(TerminalSpoolIndexEntry { + seq: frame.seq_start, + offset, + }); + } + head = frame.seq_end; + Ok(false) + })?; + Ok((checkpoints, head, summary)) +} + +pub fn encode_spool_index( + source: &TerminalSpoolDescriptor, + checkpoints: &[TerminalSpoolIndexEntry], + spool_bytes: u64, +) -> Result<Vec<u8>, String> { + validate_spool_index_entries(checkpoints, spool_bytes)?; + let mut bytes = Vec::new(); + write_spool_index_header(&mut bytes, source)?; + for checkpoint in checkpoints { + write_spool_index_checkpoint(&mut bytes, *checkpoint)?; + } + if bytes.len() > MAX_SPOOL_INDEX_BYTES { + return Err("retained output index exceeds byte bound".into()); + } + Ok(bytes) +} + +pub fn write_spool_index_header<W: Write>( + writer: &mut W, + source: &TerminalSpoolDescriptor, +) -> Result<(), String> { + let record = TerminalSpoolIndexRecord::Header { + descriptor: source.clone(), + }; + serde_json::to_writer(&mut *writer, &record) + .map_err(|_| "retained output index encoding failed")?; + writer + .write_all(b"\n") + .map_err(|_| "retained output index write failed")?; + Ok(()) +} + +pub fn write_spool_index_checkpoint<W: Write>( + writer: &mut W, + entry: TerminalSpoolIndexEntry, +) -> Result<(), String> { + let record = TerminalSpoolIndexRecord::Checkpoint { + seq: entry.seq, + offset: entry.offset, + }; + serde_json::to_writer(&mut *writer, &record) + .map_err(|_| "retained output index encoding failed")?; + writer + .write_all(b"\n") + .map_err(|_| "retained output index write failed")?; + Ok(()) +} + /// Keeps one bounded JSON line in memory. A non-newline final record is an /// uncommitted torn tail, including valid JSON without its commit newline. pub fn scan_spool<R: BufRead>( @@ -81,8 +257,27 @@ pub fn scan_spool<R: BufRead>( source: &TerminalSpoolDescriptor, mut accept: impl FnMut(RuntimeOutputFrame) -> Result<(), String>, ) -> Result<bool, String> { + Ok(scan_spool_from(&mut reader, source, 0, |_, frame| { + accept(frame)?; + Ok(false) + })? + .torn_tail) +} + +/// Scan a spool starting at a known record boundary. The callback returns +/// `true` to stop after the current frame. `start_offset` is the byte offset +/// of the first record in `reader`; callers seeking through a sparse index +/// use it to preserve absolute offsets in the result. +pub fn scan_spool_from<R: BufRead>( + mut reader: R, + source: &TerminalSpoolDescriptor, + start_offset: u64, + mut accept: impl FnMut(u64, RuntimeOutputFrame) -> Result<bool, String>, +) -> Result<SpoolScanSummary, String> { source.validate()?; let mut bytes = Vec::new(); + let mut offset = start_offset; + let mut summary = SpoolScanSummary::default(); loop { bytes.clear(); let length = Read::by_ref(&mut reader) @@ -90,13 +285,17 @@ pub fn scan_spool<R: BufRead>( .read_until(b'\n', &mut bytes) .map_err(|_| "retained output read failed")?; if length == 0 { - return Ok(false); + return Ok(summary); } + summary.bytes_scanned = summary.bytes_scanned.saturating_add(length as u64); + let frame_offset = offset; + offset = offset.saturating_add(length as u64); if length > MAX_SPOOL_LINE_BYTES { return Err("retained output frame exceeds codec bound".into()); } if bytes.last() != Some(&b'\n') { - return Ok(true); + summary.torn_tail = true; + return Ok(summary); } let row: TerminalSpoolFrame = serde_json::from_slice(&bytes) .map_err(|_| "invalid committed retained output frame")?; @@ -113,7 +312,10 @@ pub fn scan_spool<R: BufRead>( { return Err("retained output frame ownership or bounds mismatch".into()); } - accept(frame)?; + if accept(frame_offset, frame)? { + summary.stopped_early = true; + return Ok(summary); + } } } @@ -164,22 +366,41 @@ impl RetainedOutputPage { } Ok(()) } + + pub fn should_stop_scanning(&self) -> bool { + self.more + } + pub fn finish( self, torn: bool, completion: Option<TerminalSpoolCompletion>, ) -> Result<RuntimeOutputBatch, String> { + let retained = self.first.unwrap_or(self.head.saturating_add(1)); + let head = self.head; + self.finish_with_bounds(torn, completion, retained, head) + } + + pub fn finish_with_bounds( + self, + torn: bool, + completion: Option<TerminalSpoolCompletion>, + retained: u64, + head: u64, + ) -> Result<RuntimeOutputBatch, String> { + if retained == 0 || retained > head.saturating_add(1) { + return Err("invalid retained output bounds".into()); + } if let Some(c) = &completion { - c.validate(&self.source, self.head)?; + c.validate(&self.source, head)?; } - let retained = self.first.unwrap_or(self.head.saturating_add(1)); - let reset = self.after > self.head || self.after.saturating_add(1) < retained; + let reset = self.after > head || self.after.saturating_add(1) < retained; Ok(RuntimeOutputBatch { incarnation_id: self.source.incarnation_id, terminal_id: self.source.terminal_id, stream_epoch: self.source.stream_epoch, retained_from_seq: retained, - head_seq: self.head, + head_seq: head, reset_required: reset, truncated: reset || torn || self.more, exited: completion.is_some(), @@ -194,6 +415,134 @@ impl RetainedOutputPage { } } +pub fn read_indexed_spool<R: BufRead + Seek>( + source: TerminalSpoolDescriptor, + mut segments: Vec<IndexedSpoolSegment<R>>, + after: u64, + budget: u64, + completion: Option<TerminalSpoolCompletion>, +) -> Result<IndexedSpoolRead, String> { + source.validate()?; + let mut segment_bounds = Vec::with_capacity(segments.len()); + let mut torn = false; + let mut bytes_scanned = 0_u64; + + for segment in &mut segments { + validate_spool_index_entries(&segment.checkpoints, segment.file_bytes)?; + let Some(last_checkpoint) = segment.checkpoints.last().copied() else { + segment_bounds.push(None); + continue; + }; + + segment + .reader + .seek(SeekFrom::Start(last_checkpoint.offset)) + .map_err(|_| "retained output seek failed")?; + let mut first = true; + let mut head = None; + let summary = scan_spool_from( + &mut segment.reader, + &source, + last_checkpoint.offset, + |_, frame| { + if first { + first = false; + if frame.seq_start != last_checkpoint.seq { + return Err("retained output index does not match its spool".into()); + } + } else if head.is_some_and(|previous| frame.seq_start != previous + 1) { + return Err("retained output sequences are not contiguous".into()); + } + head = Some(frame.seq_end); + Ok(false) + }, + )?; + if first { + return Err("retained output index checkpoint is missing from its spool".into()); + } + torn |= summary.torn_tail; + bytes_scanned = bytes_scanned.saturating_add(summary.bytes_scanned); + segment_bounds.push(Some((segment.checkpoints[0].seq, head.unwrap_or(0)))); + } + + let mut previous_head: Option<u64> = None; + for bounds in segment_bounds.iter().flatten() { + if let Some(head) = previous_head { + if bounds.0 != head.saturating_add(1) { + return Err("retained output segments are not contiguous".into()); + } + } + previous_head = Some(bounds.1); + } + let retained = segment_bounds + .iter() + .flatten() + .next() + .map(|bounds| bounds.0) + .unwrap_or(1); + let head = previous_head.unwrap_or(0); + let reset = after > head || after.saturating_add(1) < retained; + let effective = if reset { + retained.saturating_sub(1) + } else { + after + }; + let mut page = RetainedOutputPage::new(source.clone(), effective, budget)?; + + for (segment, bounds) in segments.iter_mut().zip(segment_bounds.iter()) { + let Some((_, segment_head)) = bounds else { + continue; + }; + if *segment_head <= effective { + continue; + } + let checkpoint = segment + .checkpoints + .iter() + .take_while(|checkpoint| checkpoint.seq <= effective) + .last() + .or_else(|| segment.checkpoints.first()) + .copied() + .ok_or("retained output index checkpoint missing")?; + segment + .reader + .seek(SeekFrom::Start(checkpoint.offset)) + .map_err(|_| "retained output seek failed")?; + let mut first = true; + let summary = scan_spool_from( + &mut segment.reader, + &source, + checkpoint.offset, + |_, frame| { + if first { + first = false; + if frame.seq_start != checkpoint.seq { + return Err("retained output index does not match its spool".into()); + } + } + page.accept(frame)?; + Ok(page.should_stop_scanning()) + }, + )?; + if first { + return Err("retained output index checkpoint is missing from its spool".into()); + } + torn |= summary.torn_tail; + bytes_scanned = bytes_scanned.saturating_add(summary.bytes_scanned); + if summary.stopped_early { + break; + } + } + + let mut batch = page.finish_with_bounds(torn, completion, retained, head)?; + batch.reset_required |= reset; + batch.truncated |= reset; + Ok(IndexedSpoolRead { + batch, + bytes_scanned, + }) +} + #[cfg(test)] mod tests { use super::*; @@ -220,6 +569,51 @@ mod tests { bytes.push(b'\n'); bytes } + #[test] + fn indexed_spool_reads_a_near_tail_page_without_scanning_its_prefix() { + let source = source(); + let mut bytes = Vec::new(); + let mut checkpoints: Vec<TerminalSpoolIndexEntry> = Vec::new(); + for seq in 1..=128 { + let offset = bytes.len() as u64; + if checkpoints.is_empty() + || offset.saturating_sub(checkpoints.last().unwrap().offset) + >= SPOOL_INDEX_STRIDE_BYTES + { + checkpoints.push(TerminalSpoolIndexEntry { seq, offset }); + } + bytes.extend(line(frame( + &source, + seq, + &"x".repeat(MAX_OUTPUT_FRAME_BYTES), + ))); + } + let file_bytes = bytes.len() as u64; + + let read = read_indexed_spool( + source, + vec![IndexedSpoolSegment { + reader: std::io::Cursor::new(bytes), + file_bytes, + checkpoints, + }], + 126, + MAX_OUTPUT_FRAME_BYTES as u64, + None, + ) + .unwrap(); + + assert_eq!(read.batch.frames.len(), 1); + assert_eq!(read.batch.frames[0].seq_start, 127); + assert_eq!(read.batch.head_seq, 128); + assert!( + read.bytes_scanned < file_bytes / 2, + "near-tail page scanned {} of {} spool bytes", + read.bytes_scanned, + file_bytes + ); + } + #[test] fn retained_output_codec_preserves_utf8_full_pages_for_two_readers() { let source = source(); diff --git a/crates/freshell-session-host/src/output_journal.rs b/crates/freshell-session-host/src/output_journal.rs index d47d5a39d..0ca2be0ca 100644 --- a/crates/freshell-session-host/src/output_journal.rs +++ b/crates/freshell-session-host/src/output_journal.rs @@ -1,7 +1,9 @@ use freshell_runtime_protocol::{ terminal_spool::{ - scan_spool, RetainedOutputPage, TerminalSpoolFrame as SpoolFrame, SPOOL_COMPLETION_FILE, - SPOOL_DESCRIPTOR_FILE, + build_spool_index, encode_spool_index, read_indexed_spool, write_spool_index_checkpoint, + IndexedSpoolSegment, TerminalSpoolFrame as SpoolFrame, TerminalSpoolIndexEntry, + SPOOL_COMPLETION_FILE, SPOOL_CURRENT_INDEX_FILE, SPOOL_DESCRIPTOR_FILE, + SPOOL_INDEX_STRIDE_BYTES, SPOOL_PREVIOUS_INDEX_FILE, }, IncarnationId, RuntimeOutputBatch, RuntimeOutputFrame, TerminalExitObservation, TerminalSpoolCompletion, TerminalSpoolDescriptor, @@ -27,8 +29,12 @@ pub struct OutputJournal { ring_limit: usize, spool_limit: u64, next_seq: u64, + previous_index: Vec<TerminalSpoolIndexEntry>, + current_index: Vec<TerminalSpoolIndexEntry>, completion: Option<TerminalSpoolCompletion>, failure: Option<String>, + #[cfg(test)] + last_read_bytes_scanned: std::cell::Cell<u64>, } impl OutputJournal { @@ -78,8 +84,12 @@ impl OutputJournal { spool_limit: spool_limit .clamp((MAX_TRANSPORT_CHUNK_BYTES * 4) as u64, DEFAULT_SPOOL_BYTES), next_seq: 1, + previous_index: Vec::new(), + current_index: Vec::new(), completion: None, failure: None, + #[cfg(test)] + last_read_bytes_scanned: std::cell::Cell::new(0), }; atomic_json( &journal.state_dir.join(SPOOL_DESCRIPTOR_FILE), @@ -157,56 +167,49 @@ impl OutputJournal { Ok(()) } - fn scan( - &self, - mut accept: impl FnMut(RuntimeOutputFrame) -> Result<(), String>, - ) -> Result<bool, String> { + pub fn read(&self, after_seq: u64, max_bytes: u64) -> Result<RuntimeOutputBatch, String> { + if let Some(error) = &self.failure { + return Err(error.clone()); + } let source = self.descriptor(); - let mut torn = false; - for (path, required) in [(self.previous_path(), false), (self.current_path(), true)] { + let mut segments = Vec::with_capacity(2); + for (path, checkpoints, required) in [ + (self.previous_path(), &self.previous_index, false), + (self.current_path(), &self.current_index, true), + ] { let file = match std::fs::File::open(path) { Ok(file) => file, Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => continue, Err(_) => return Err("retained output spool unavailable".into()), }; - if file + let file_bytes = file .metadata() .map_err(|_| "retained spool metadata unavailable")? - .len() - > source.segment_limit_bytes - { + .len(); + if file_bytes > source.segment_limit_bytes { return Err("retained spool exceeds segment bound".into()); } - torn |= scan_spool(BufReader::new(file), &source, &mut accept)?; + segments.push(IndexedSpoolSegment { + reader: BufReader::new(file), + file_bytes, + checkpoints: checkpoints.clone(), + }); } - Ok(torn) + let indexed = read_indexed_spool( + source, + segments, + after_seq, + max_bytes, + self.completion.clone(), + )?; + #[cfg(test)] + self.last_read_bytes_scanned.set(indexed.bytes_scanned); + Ok(indexed.batch) } - pub fn read(&self, after_seq: u64, max_bytes: u64) -> Result<RuntimeOutputBatch, String> { - if let Some(error) = &self.failure { - return Err(error.clone()); - } - // First scan establishes bounds without retaining the spool in memory. - let mut first = None; - let mut head = 0; - self.scan(|frame| { - first.get_or_insert(frame.seq_start); - head = frame.seq_end; - Ok(()) - })?; - let retained = first.unwrap_or(head.saturating_add(1)); - let reset = after_seq > head || after_seq.saturating_add(1) < retained; - let effective = if reset { - retained.saturating_sub(1) - } else { - after_seq - }; - let mut page = RetainedOutputPage::new(self.descriptor(), effective, max_bytes)?; - let torn = self.scan(|frame| page.accept(frame))?; - let mut batch = page.finish(torn, self.completion.clone())?; - batch.reset_required |= reset; - batch.truncated |= reset; - Ok(batch) + #[cfg(test)] + fn last_read_bytes_scanned(&self) -> u64 { + self.last_read_bytes_scanned.get() } pub fn head_seq(&self) -> u64 { @@ -223,7 +226,7 @@ impl OutputJournal { .sum() } - fn append_spool(&self, frame: &RuntimeOutputFrame) -> Result<(), String> { + fn append_spool(&mut self, frame: &RuntimeOutputFrame) -> Result<(), String> { let line = serde_json::to_vec(&SpoolFrame { frame: frame.clone(), }) @@ -233,10 +236,22 @@ impl OutputJournal { let current_len = current.metadata().map(|m| m.len()).unwrap_or(0); if current_len > 0 && current_len.saturating_add(line.len() as u64 + 1) > segment_limit { let previous = self.previous_path(); + let previous_index_path = self.previous_index_path(); + let current_index_path = self.current_index_path(); let _ = std::fs::remove_file(&previous); + let _ = std::fs::remove_file(&previous_index_path); std::fs::rename(¤t, &previous).map_err(|e| e.to_string())?; + std::fs::rename(¤t_index_path, &previous_index_path) + .map_err(|e| e.to_string())?; sync_parent(¤t)?; + self.previous_index = std::mem::take(&mut self.current_index); + write_index_atomic(¤t_index_path, &self.descriptor(), &[], 0)?; } + let current_len = current.metadata().map(|m| m.len()).unwrap_or(0); + let checkpoint_needed = self + .current_index + .last() + .is_none_or(|last| current_len.saturating_sub(last.offset) >= SPOOL_INDEX_STRIDE_BYTES); let mut file = OpenOptions::new() .create(true) .append(true) @@ -246,15 +261,41 @@ impl OutputJournal { .and_then(|_| file.write_all(b"\n")) .map_err(|e| e.to_string())?; file.sync_data().map_err(|e| e.to_string())?; + if checkpoint_needed { + let checkpoint = TerminalSpoolIndexEntry { + seq: frame.seq_start, + offset: current_len, + }; + let mut index = OpenOptions::new() + .append(true) + .open(self.current_index_path()) + .map_err(|_| "retained output index unavailable")?; + write_spool_index_checkpoint(&mut index, checkpoint)?; + index + .sync_data() + .map_err(|_| "retained output index sync failed")?; + self.current_index.push(checkpoint); + } Ok(()) } fn seed_sequence_from_spool(&mut self) -> Result<(), String> { - let mut head = 0; - self.scan(|frame| { - head = frame.seq_end; - Ok(()) - })?; + let source = self.descriptor(); + let (previous_index, previous_head) = rebuild_spool_index( + &self.previous_path(), + &self.previous_index_path(), + false, + &source, + )?; + let (current_index, current_head) = rebuild_spool_index( + &self.current_path(), + &self.current_index_path(), + true, + &source, + )?; + self.previous_index = previous_index; + self.current_index = current_index; + let head = current_head.max(previous_head); self.next_seq = head.checked_add(1).ok_or("output sequence exhausted")?; Ok(()) } @@ -265,6 +306,59 @@ impl OutputJournal { fn previous_path(&self) -> PathBuf { self.state_dir.join("terminal-spool-previous.jsonl") } + fn current_index_path(&self) -> PathBuf { + self.state_dir.join(SPOOL_CURRENT_INDEX_FILE) + } + fn previous_index_path(&self) -> PathBuf { + self.state_dir.join(SPOOL_PREVIOUS_INDEX_FILE) + } +} + +fn rebuild_spool_index( + spool_path: &Path, + index_path: &Path, + required: bool, + source: &TerminalSpoolDescriptor, +) -> Result<(Vec<TerminalSpoolIndexEntry>, u64), String> { + let file = match std::fs::File::open(spool_path) { + Ok(file) => file, + Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => { + write_index_atomic(index_path, source, &[], 0)?; + return Ok((Vec::new(), 0)); + } + Err(_) => return Err("retained output spool unavailable".into()), + }; + let file_bytes = file + .metadata() + .map_err(|_| "retained spool metadata unavailable")? + .len(); + if file_bytes > source.segment_limit_bytes { + return Err("retained spool exceeds segment bound".into()); + } + let (checkpoints, head, _) = build_spool_index(BufReader::new(file), source, 0)?; + write_index_atomic(index_path, source, &checkpoints, file_bytes)?; + Ok((checkpoints, head)) +} + +fn write_index_atomic( + path: &Path, + source: &TerminalSpoolDescriptor, + checkpoints: &[TerminalSpoolIndexEntry], + spool_bytes: u64, +) -> Result<(), String> { + let bytes = encode_spool_index(source, checkpoints, spool_bytes)?; + let temporary = path.with_extension("tmp"); + let mut file = OpenOptions::new() + .create(true) + .truncate(true) + .write(true) + .open(&temporary) + .map_err(|_| "retained output index unavailable")?; + file.write_all(&bytes) + .and_then(|_| file.sync_all()) + .map_err(|_| "retained output index sync failed")?; + std::fs::rename(&temporary, path).map_err(|_| "retained output index commit failed")?; + sync_parent(path) } pub fn atomic_json(path: &Path, value: &impl serde::Serialize) -> Result<(), String> { @@ -435,6 +529,37 @@ mod tests { assert!(journal.read(1, 64 * 1024).unwrap().frames.is_empty()); } + #[test] + fn later_replay_page_does_not_rescan_the_entire_retained_spool() { + let dir = tempfile::tempdir().unwrap(); + let mut journal = OutputJournal::with_limits( + dir.path(), + IncarnationId::new(), + "t".into(), + "epoch".into(), + 64 * 1024, + 16 * 1024 * 1024, + ) + .unwrap(); + let payload = "x".repeat(MAX_TRANSPORT_CHUNK_BYTES); + let mut last_seq = 0; + for _ in 0..96 { + last_seq = journal.append(&payload).unwrap()[0].seq_end; + } + let spool_bytes = journal.spool_bytes(); + + let page = journal.read(last_seq - 1, 64 * 1024).unwrap(); + + assert_eq!(page.frames.len(), 1); + assert_eq!(page.frames[0].seq_start, last_seq); + assert!( + journal.last_read_bytes_scanned() < spool_bytes / 2, + "a near-tail page should scan only its indexed neighborhood, not the full spool twice; scanned {} of {} bytes", + journal.last_read_bytes_scanned(), + spool_bytes + ); + } + #[test] fn transport_frames_never_split_utf8_or_exceed_64k() { let dir = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/retained_output.rs b/crates/freshell-supervisor/src/retained_output.rs index dd45d0922..2bf5b352a 100644 --- a/crates/freshell-supervisor/src/retained_output.rs +++ b/crates/freshell-supervisor/src/retained_output.rs @@ -1,14 +1,17 @@ use crate::registry::RetainedOutputSource; use freshell_runtime_protocol::{ terminal_spool::{ - scan_spool, RetainedOutputPage, SPOOL_COMPLETION_FILE, SPOOL_CURRENT_FILE, - SPOOL_DESCRIPTOR_FILE, SPOOL_PREVIOUS_FILE, + build_spool_index, encode_spool_index, read_indexed_spool, read_spool_index, + scan_spool_from, write_spool_index_checkpoint, IndexedSpoolRead, IndexedSpoolSegment, + TerminalSpoolIndexEntry, MAX_SPOOL_INDEX_BYTES, MAX_SPOOL_LINE_BYTES, + SPOOL_COMPLETION_FILE, SPOOL_CURRENT_FILE, SPOOL_CURRENT_INDEX_FILE, SPOOL_DESCRIPTOR_FILE, + SPOOL_INDEX_STRIDE_BYTES, SPOOL_PREVIOUS_FILE, SPOOL_PREVIOUS_INDEX_FILE, }, RuntimeOutputBatch, TerminalExitObservation, TerminalSpoolCompletion, TerminalSpoolDescriptor, }; use std::{ fs::{File, OpenOptions}, - io::{BufReader, Read, Write}, + io::{BufReader, Read, Seek, SeekFrom, Write}, path::Path, }; @@ -26,6 +29,8 @@ pub(crate) fn validate_source( SPOOL_COMPLETION_FILE, SPOOL_CURRENT_FILE, SPOOL_PREVIOUS_FILE, + SPOOL_CURRENT_INDEX_FILE, + SPOOL_PREVIOUS_INDEX_FILE, ] { match std::fs::symlink_metadata(directory.join(name)) { Ok(metadata) if !metadata.file_type().is_file() => { @@ -50,31 +55,6 @@ fn metadata<T: serde::de::DeserializeOwned>(path: &Path) -> Result<T, String> { } serde_json::from_slice(&bytes).map_err(|_| "invalid retained output metadata".into()) } -fn scan( - source: &RetainedOutputSource, - descriptor: &TerminalSpoolDescriptor, - mut accept: impl FnMut(freshell_runtime_protocol::RuntimeOutputFrame) -> Result<(), String>, -) -> Result<bool, String> { - let mut torn = false; - for (name, required) in [(SPOOL_PREVIOUS_FILE, false), (SPOOL_CURRENT_FILE, true)] { - let file = match File::open(source.runtime_dir.join(name)) { - Ok(file) => file, - Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => continue, - Err(_) => return Err("retained output spool unavailable".into()), - }; - if file - .metadata() - .map_err(|_| "retained spool metadata unavailable")? - .len() - > descriptor.segment_limit_bytes - { - return Err("retained spool exceeds segment bound".into()); - } - torn |= scan_spool(BufReader::new(file), descriptor, &mut accept)?; - } - Ok(torn) -} - pub(crate) fn read( source: RetainedOutputSource, after: u64, @@ -93,63 +73,329 @@ pub(crate) fn read( { return Err("retained output source association mismatch".into()); } - let mut first = None; - let mut head = 0; - scan(&source, &descriptor, |frame| { - first.get_or_insert(frame.seq_start); - head = frame.seq_end; - Ok(()) - })?; - let retained = first.unwrap_or(head.saturating_add(1)); - let changed = expected_epoch - .as_deref() - .is_some_and(|epoch| epoch != descriptor.stream_epoch); - let reset = changed || after > head || after.saturating_add(1) < retained; - let effective = if reset { - retained.saturating_sub(1) - } else { - after - }; let completion_path = source.runtime_dir.join(SPOOL_COMPLETION_FILE); let completion = match metadata::<TerminalSpoolCompletion>(&completion_path) { Ok(completion) => Some(completion), - Err(_) if !completion_path.exists() && !verified_empty => None, - Err(_) if !completion_path.exists() && verified_empty => { - // An exact backend empty proof settles a writer whose host could - // not commit EOF. Freeze its actual retained head; code is unknown. - let completion = TerminalSpoolCompletion { - incarnation_id: descriptor.incarnation_id.clone(), - terminal_id: descriptor.terminal_id.clone(), - stream_epoch: descriptor.stream_epoch.clone(), - retained_from_seq: retained, - head_seq: head, - exit: TerminalExitObservation::Unavailable, - }; - let temporary = completion_path.with_extension("tmp"); - let mut file = OpenOptions::new() - .create(true) - .truncate(true) - .write(true) - .open(&temporary) - .map_err(|_| "completion unavailable")?; - let bytes = - serde_json::to_vec(&completion).map_err(|_| "completion encoding failed")?; - file.write_all(&bytes) - .and_then(|_| file.sync_all()) - .map_err(|_| "completion write failed")?; - std::fs::rename(temporary, &completion_path).map_err(|_| "completion commit failed")?; - File::open(&source.runtime_dir) - .and_then(|dir| dir.sync_all()) - .map_err(|_| "completion parent sync failed")?; - Some(completion) - } + Err(_) if !completion_path.exists() => None, Err(error) => return Err(error), }; - let mut page = RetainedOutputPage::new(descriptor.clone(), effective, budget)?; - let torn = scan(&source, &descriptor, |frame| page.accept(frame))?; - let mut batch = page.finish(torn, completion)?; - batch.reset_required |= reset; - batch.truncated |= reset; + + let changed = expected_epoch + .as_deref() + .is_some_and(|epoch| epoch != descriptor.stream_epoch); + let read_after = if changed { 0 } else { after }; + let mut indexed = read_indexed( + &source, + &descriptor, + read_after, + budget, + completion.clone(), + false, + ); + if indexed.is_err() { + // A missing index is expected for older owned hosts. Rebuild it from + // the authoritative spool once, then use indexed paging. Invalid or + // stale indexes take the same repair path; malformed spool data still + // fails the rebuild rather than being hidden. + indexed = read_indexed(&source, &descriptor, read_after, budget, completion, true); + } + let mut batch = indexed?.batch; + if changed { + batch.reset_required = true; + batch.truncated = true; + } + + if batch.completion.is_none() && verified_empty { + let completed = TerminalSpoolCompletion { + incarnation_id: descriptor.incarnation_id.clone(), + terminal_id: descriptor.terminal_id.clone(), + stream_epoch: descriptor.stream_epoch.clone(), + retained_from_seq: batch.retained_from_seq, + head_seq: batch.head_seq, + exit: TerminalExitObservation::Unavailable, + }; + completed.validate(&descriptor, batch.head_seq)?; + let temporary = completion_path.with_extension("tmp"); + let mut file = OpenOptions::new() + .create(true) + .truncate(true) + .write(true) + .open(&temporary) + .map_err(|_| "completion unavailable")?; + let bytes = serde_json::to_vec(&completed).map_err(|_| "completion encoding failed")?; + file.write_all(&bytes) + .and_then(|_| file.sync_all()) + .map_err(|_| "completion write failed")?; + std::fs::rename(temporary, &completion_path).map_err(|_| "completion commit failed")?; + File::open(&source.runtime_dir) + .and_then(|dir| dir.sync_all()) + .map_err(|_| "completion parent sync failed")?; + batch.completion = Some(completed); + batch.exited = true; + batch.exit_code = None; + } batch.native_session_id = source.native_session_id; Ok(batch) } + +fn read_indexed( + source: &RetainedOutputSource, + descriptor: &TerminalSpoolDescriptor, + after: u64, + budget: u64, + completion: Option<TerminalSpoolCompletion>, + rebuild: bool, +) -> Result<IndexedSpoolRead, String> { + let mut segments = Vec::with_capacity(2); + for (spool_name, index_name, required) in [ + (SPOOL_PREVIOUS_FILE, SPOOL_PREVIOUS_INDEX_FILE, false), + (SPOOL_CURRENT_FILE, SPOOL_CURRENT_INDEX_FILE, true), + ] { + let spool_path = source.runtime_dir.join(spool_name); + let file = match File::open(&spool_path) { + Ok(file) => file, + Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => return Err("retained output spool unavailable".into()), + }; + let file_bytes = file + .metadata() + .map_err(|_| "retained spool metadata unavailable")? + .len(); + if file_bytes > descriptor.segment_limit_bytes { + return Err("retained spool exceeds segment bound".into()); + } + + let index_path = source.runtime_dir.join(index_name); + let checkpoints = if !rebuild { + match File::open(&index_path) { + Ok(index_file) => { + if index_file + .metadata() + .map_err(|_| "retained output index metadata unavailable")? + .len() + > MAX_SPOOL_INDEX_BYTES as u64 + { + return Err("retained output index exceeds byte bound".into()); + } + match read_spool_index(BufReader::new(index_file), descriptor, file_bytes) { + Ok(mut checkpoints) => { + extend_index_if_needed( + &file, + &index_path, + descriptor, + file_bytes, + &mut checkpoints, + )?; + Some(checkpoints) + } + Err(_) => None, + } + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => None, + Err(_) => return Err("retained output index unavailable".into()), + } + } else { + None + }; + let checkpoints = match checkpoints { + Some(checkpoints) => checkpoints, + None => { + let (checkpoints, _, _) = build_spool_index( + BufReader::new( + file.try_clone() + .map_err(|_| "retained output spool unavailable")?, + ), + descriptor, + 0, + )?; + write_index_atomic(&index_path, descriptor, &checkpoints, file_bytes)?; + checkpoints + } + }; + segments.push(IndexedSpoolSegment { + reader: BufReader::new(file), + file_bytes, + checkpoints, + }); + } + read_indexed_spool(descriptor.clone(), segments, after, budget, completion) +} + +fn extend_index_if_needed( + spool: &File, + index_path: &Path, + descriptor: &TerminalSpoolDescriptor, + file_bytes: u64, + checkpoints: &mut Vec<TerminalSpoolIndexEntry>, +) -> Result<(), String> { + let Some(mut last_checkpoint) = checkpoints.last().copied() else { + return Ok(()); + }; + if file_bytes.saturating_sub(last_checkpoint.offset) + <= SPOOL_INDEX_STRIDE_BYTES + MAX_SPOOL_LINE_BYTES as u64 + { + return Ok(()); + } + + let mut reader = BufReader::new( + spool + .try_clone() + .map_err(|_| "retained output spool unavailable")?, + ); + reader + .seek(SeekFrom::Start(last_checkpoint.offset)) + .map_err(|_| "retained output seek failed")?; + let mut first = true; + let mut previous_seq = 0_u64; + let mut appended = Vec::new(); + scan_spool_from( + &mut reader, + descriptor, + last_checkpoint.offset, + |offset, frame| { + if first { + first = false; + if frame.seq_start != last_checkpoint.seq { + return Err("retained output index does not match its spool".into()); + } + } else if frame.seq_start != previous_seq.saturating_add(1) { + return Err("retained output sequences are not contiguous".into()); + } + previous_seq = frame.seq_end; + if !first && offset.saturating_sub(last_checkpoint.offset) >= SPOOL_INDEX_STRIDE_BYTES { + let checkpoint = TerminalSpoolIndexEntry { + seq: frame.seq_start, + offset, + }; + appended.push(checkpoint); + last_checkpoint = checkpoint; + } + Ok(false) + }, + )?; + if first { + return Err("retained output index checkpoint is missing from its spool".into()); + } + if !appended.is_empty() { + let mut index = OpenOptions::new() + .append(true) + .open(index_path) + .map_err(|_| "retained output index unavailable")?; + for checkpoint in &appended { + write_spool_index_checkpoint(&mut index, *checkpoint)?; + } + index + .sync_data() + .map_err(|_| "retained output index sync failed")?; + checkpoints.extend(appended); + } + Ok(()) +} + +fn write_index_atomic( + path: &Path, + descriptor: &TerminalSpoolDescriptor, + checkpoints: &[freshell_runtime_protocol::terminal_spool::TerminalSpoolIndexEntry], + spool_bytes: u64, +) -> Result<(), String> { + let bytes = encode_spool_index(descriptor, checkpoints, spool_bytes)?; + let temporary = path.with_extension("tmp"); + let mut file = OpenOptions::new() + .create(true) + .truncate(true) + .write(true) + .open(&temporary) + .map_err(|_| "retained output index unavailable")?; + file.write_all(&bytes) + .and_then(|_| file.sync_all()) + .map_err(|_| "retained output index write failed")?; + std::fs::rename(&temporary, path).map_err(|_| "retained output index commit failed")?; + File::open( + path.parent() + .ok_or("retained output directory unavailable")?, + ) + .and_then(|directory| directory.sync_all()) + .map_err(|_| "retained output index parent sync failed")?; + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use freshell_runtime_protocol::{ + terminal_spool::{TerminalSpoolFrame, MAX_OUTPUT_FRAME_BYTES}, + IncarnationId, TerminalLaunchSpec, + }; + + #[test] + fn legacy_supervisor_fallback_builds_and_reuses_sparse_spool_index() { + let directory = tempfile::tempdir().unwrap(); + let incarnation_id = IncarnationId::new(); + let terminal_id = "retained-terminal"; + let stream_epoch = "launch-prefix-hostboot-indexed"; + let source = RetainedOutputSource { + incarnation_id: incarnation_id.clone(), + runtime_dir: directory.path().to_path_buf(), + terminal: serde_json::from_value::<TerminalLaunchSpec>(serde_json::json!({ + "terminalId":terminal_id,"streamId":"launch-prefix","mode":"shell", + "program":"/bin/sh","cwd":"/workspace","runAsUid":65534, + "runAsGid":0,"cols":120,"rows":30,"projectKey":"fixture", + "workspacePath":"/workspace" + })) + .unwrap(), + native_session_id: None, + }; + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id, + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + segment_limit_bytes: 32 * 1024 * 1024, + }; + let mut spool = File::create(directory.path().join(SPOOL_CURRENT_FILE)).unwrap(); + let payload = "x".repeat(MAX_OUTPUT_FRAME_BYTES); + for seq in 1..=96 { + let frame = freshell_runtime_protocol::RuntimeOutputFrame { + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + seq_start: seq, + seq_end: seq, + data: payload.clone(), + }; + serde_json::to_writer(&mut spool, &TerminalSpoolFrame { frame }).unwrap(); + spool.write_all(b"\n").unwrap(); + } + spool.sync_all().unwrap(); + let spool_bytes = spool.metadata().unwrap().len(); + drop(spool); + + let first = read_indexed( + &source, + &descriptor, + 94, + MAX_OUTPUT_FRAME_BYTES as u64, + None, + false, + ) + .unwrap(); + assert_eq!(first.batch.frames[0].seq_start, 95); + assert!(directory.path().join(SPOOL_CURRENT_INDEX_FILE).is_file()); + + let second = read_indexed( + &source, + &descriptor, + 95, + MAX_OUTPUT_FRAME_BYTES as u64, + None, + false, + ) + .unwrap(); + assert_eq!(second.batch.frames[0].seq_start, 96); + assert!( + second.bytes_scanned < spool_bytes / 2, + "supervisor replay scanned {} of {} spool bytes after indexing", + second.bytes_scanned, + spool_bytes + ); + } +} From e887f7193d46faed790fdb965fe2f2df9a39b4a8 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 12:51:04 -0700 Subject: [PATCH 462/493] Harden retained output index recovery --- .../src/terminal_spool.rs | 174 +++++++- .../src/output_journal.rs | 68 +++- .../src/retained_output.rs | 373 +++++++++++++++--- 3 files changed, 523 insertions(+), 92 deletions(-) diff --git a/crates/freshell-runtime-protocol/src/terminal_spool.rs b/crates/freshell-runtime-protocol/src/terminal_spool.rs index 96449e479..4a5077fd4 100644 --- a/crates/freshell-runtime-protocol/src/terminal_spool.rs +++ b/crates/freshell-runtime-protocol/src/terminal_spool.rs @@ -81,8 +81,17 @@ pub struct TerminalSpoolFrame { #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(tag = "record", rename_all = "snake_case")] pub enum TerminalSpoolIndexRecord { - Header { descriptor: TerminalSpoolDescriptor }, - Checkpoint { seq: u64, offset: u64 }, + Header { + descriptor: TerminalSpoolDescriptor, + #[serde(default)] + scanned_through_offset: u64, + #[serde(default)] + torn_tail: bool, + }, + Checkpoint { + seq: u64, + offset: u64, + }, } #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -91,10 +100,48 @@ pub struct TerminalSpoolIndexEntry { pub offset: u64, } +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TerminalSpoolIndex { + pub checkpoints: Vec<TerminalSpoolIndexEntry>, + /// With no matching-epoch checkpoints, this is the last complete record + /// boundary already scanned, including records from older epochs. + pub scanned_through_offset: u64, + pub torn_tail: bool, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum IndexedSpoolReadError { + /// The sidecar points to the wrong spool records and may be rebuilt. + InvalidIndex(String), + /// The source or requested page failed; retrying with a full scan is not useful. + Other(String), +} + +impl From<String> for IndexedSpoolReadError { + fn from(message: String) -> Self { + Self::Other(message) + } +} + +impl From<&str> for IndexedSpoolReadError { + fn from(message: &str) -> Self { + Self::Other(message.into()) + } +} + +impl std::fmt::Display for IndexedSpoolReadError { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::InvalidIndex(message) | Self::Other(message) => formatter.write_str(message), + } + } +} + pub struct IndexedSpoolSegment<R> { pub reader: R, pub file_bytes: u64, pub checkpoints: Vec<TerminalSpoolIndexEntry>, + pub torn_tail: bool, } pub struct IndexedSpoolRead { @@ -107,13 +154,14 @@ pub struct SpoolScanSummary { pub torn_tail: bool, pub bytes_scanned: u64, pub stopped_early: bool, + pub last_complete_offset: u64, } pub fn read_spool_index<R: BufRead>( mut reader: R, source: &TerminalSpoolDescriptor, spool_bytes: u64, -) -> Result<Vec<TerminalSpoolIndexEntry>, String> { +) -> Result<TerminalSpoolIndex, String> { source.validate()?; let mut bytes = Vec::new(); Read::by_ref(&mut reader) @@ -131,11 +179,15 @@ pub fn read_spool_index<R: BufRead>( .ok_or("retained output index header missing")?; let header: TerminalSpoolIndexRecord = serde_json::from_slice(header_line).map_err(|_| "invalid retained output index header")?; - if header - != (TerminalSpoolIndexRecord::Header { - descriptor: source.clone(), - }) - { + let TerminalSpoolIndexRecord::Header { + descriptor, + scanned_through_offset, + torn_tail, + } = header + else { + return Err("retained output index header missing".into()); + }; + if descriptor != *source || scanned_through_offset > spool_bytes { return Err("retained output index source mismatch".into()); } @@ -152,7 +204,11 @@ pub fn read_spool_index<R: BufRead>( entries.push(TerminalSpoolIndexEntry { seq, offset }); } validate_spool_index_entries(&entries, spool_bytes)?; - Ok(entries) + Ok(TerminalSpoolIndex { + checkpoints: entries, + scanned_through_offset, + torn_tail, + }) } pub fn validate_spool_index_entries( @@ -206,10 +262,15 @@ pub fn encode_spool_index( source: &TerminalSpoolDescriptor, checkpoints: &[TerminalSpoolIndexEntry], spool_bytes: u64, + scanned_through_offset: u64, + torn_tail: bool, ) -> Result<Vec<u8>, String> { validate_spool_index_entries(checkpoints, spool_bytes)?; + if scanned_through_offset > spool_bytes { + return Err("retained output index scan bound exceeds spool".into()); + } let mut bytes = Vec::new(); - write_spool_index_header(&mut bytes, source)?; + write_spool_index_header(&mut bytes, source, scanned_through_offset, torn_tail)?; for checkpoint in checkpoints { write_spool_index_checkpoint(&mut bytes, *checkpoint)?; } @@ -222,9 +283,13 @@ pub fn encode_spool_index( pub fn write_spool_index_header<W: Write>( writer: &mut W, source: &TerminalSpoolDescriptor, + scanned_through_offset: u64, + torn_tail: bool, ) -> Result<(), String> { let record = TerminalSpoolIndexRecord::Header { descriptor: source.clone(), + scanned_through_offset, + torn_tail, }; serde_json::to_writer(&mut *writer, &record) .map_err(|_| "retained output index encoding failed")?; @@ -277,7 +342,10 @@ pub fn scan_spool_from<R: BufRead>( source.validate()?; let mut bytes = Vec::new(); let mut offset = start_offset; - let mut summary = SpoolScanSummary::default(); + let mut summary = SpoolScanSummary { + last_complete_offset: start_offset, + ..SpoolScanSummary::default() + }; loop { bytes.clear(); let length = Read::by_ref(&mut reader) @@ -299,6 +367,7 @@ pub fn scan_spool_from<R: BufRead>( } let row: TerminalSpoolFrame = serde_json::from_slice(&bytes) .map_err(|_| "invalid committed retained output frame")?; + summary.last_complete_offset = offset; let frame = row.frame; // A restarted host can leave its prior epoch in the bounded segments. if frame.stream_epoch != source.stream_epoch { @@ -421,14 +490,15 @@ pub fn read_indexed_spool<R: BufRead + Seek>( after: u64, budget: u64, completion: Option<TerminalSpoolCompletion>, -) -> Result<IndexedSpoolRead, String> { +) -> Result<IndexedSpoolRead, IndexedSpoolReadError> { source.validate()?; let mut segment_bounds = Vec::with_capacity(segments.len()); - let mut torn = false; + let mut torn = segments.iter().any(|segment| segment.torn_tail); let mut bytes_scanned = 0_u64; for segment in &mut segments { - validate_spool_index_entries(&segment.checkpoints, segment.file_bytes)?; + validate_spool_index_entries(&segment.checkpoints, segment.file_bytes) + .map_err(IndexedSpoolReadError::InvalidIndex)?; let Some(last_checkpoint) = segment.checkpoints.last().copied() else { segment_bounds.push(None); continue; @@ -440,6 +510,7 @@ pub fn read_indexed_spool<R: BufRead + Seek>( .map_err(|_| "retained output seek failed")?; let mut first = true; let mut head = None; + let mut invalid_index = false; let summary = scan_spool_from( &mut segment.reader, &source, @@ -448,6 +519,7 @@ pub fn read_indexed_spool<R: BufRead + Seek>( if first { first = false; if frame.seq_start != last_checkpoint.seq { + invalid_index = true; return Err("retained output index does not match its spool".into()); } } else if head.is_some_and(|previous| frame.seq_start != previous + 1) { @@ -456,9 +528,18 @@ pub fn read_indexed_spool<R: BufRead + Seek>( head = Some(frame.seq_end); Ok(false) }, - )?; + ) + .map_err(|error| { + if invalid_index { + IndexedSpoolReadError::InvalidIndex(error) + } else { + IndexedSpoolReadError::Other(error) + } + })?; if first { - return Err("retained output index checkpoint is missing from its spool".into()); + return Err(IndexedSpoolReadError::InvalidIndex( + "retained output index checkpoint is missing from its spool".into(), + )); } torn |= summary.torn_tail; bytes_scanned = bytes_scanned.saturating_add(summary.bytes_scanned); @@ -469,7 +550,9 @@ pub fn read_indexed_spool<R: BufRead + Seek>( for bounds in segment_bounds.iter().flatten() { if let Some(head) = previous_head { if bounds.0 != head.saturating_add(1) { - return Err("retained output segments are not contiguous".into()); + return Err(IndexedSpoolReadError::InvalidIndex( + "retained output segments are not contiguous".into(), + )); } } previous_head = Some(bounds.1); @@ -509,6 +592,7 @@ pub fn read_indexed_spool<R: BufRead + Seek>( .seek(SeekFrom::Start(checkpoint.offset)) .map_err(|_| "retained output seek failed")?; let mut first = true; + let mut invalid_index = false; let summary = scan_spool_from( &mut segment.reader, &source, @@ -517,15 +601,25 @@ pub fn read_indexed_spool<R: BufRead + Seek>( if first { first = false; if frame.seq_start != checkpoint.seq { + invalid_index = true; return Err("retained output index does not match its spool".into()); } } page.accept(frame)?; Ok(page.should_stop_scanning()) }, - )?; + ) + .map_err(|error| { + if invalid_index { + IndexedSpoolReadError::InvalidIndex(error) + } else { + IndexedSpoolReadError::Other(error) + } + })?; if first { - return Err("retained output index checkpoint is missing from its spool".into()); + return Err(IndexedSpoolReadError::InvalidIndex( + "retained output index checkpoint is missing from its spool".into(), + )); } torn |= summary.torn_tail; bytes_scanned = bytes_scanned.saturating_add(summary.bytes_scanned); @@ -596,6 +690,7 @@ mod tests { reader: std::io::Cursor::new(bytes), file_bytes, checkpoints, + torn_tail: false, }], 126, MAX_OUTPUT_FRAME_BYTES as u64, @@ -614,6 +709,47 @@ mod tests { ); } + #[test] + fn empty_sparse_index_preserves_a_torn_tail_marker() { + let source = source(); + let bytes = b"partial committed frame".to_vec(); + let (checkpoints, _, summary) = + build_spool_index(std::io::Cursor::new(bytes.clone()), &source, 0).unwrap(); + assert!(checkpoints.is_empty()); + assert!(summary.torn_tail); + + let index_bytes = encode_spool_index( + &source, + &checkpoints, + bytes.len() as u64, + summary.last_complete_offset, + summary.torn_tail, + ) + .unwrap(); + let index = read_spool_index( + std::io::Cursor::new(index_bytes), + &source, + bytes.len() as u64, + ) + .unwrap(); + assert!(index.torn_tail); + + let read = read_indexed_spool( + source, + vec![IndexedSpoolSegment { + reader: std::io::Cursor::new(bytes.clone()), + file_bytes: bytes.len() as u64, + checkpoints: index.checkpoints, + torn_tail: index.torn_tail, + }], + 0, + MAX_OUTPUT_FRAME_BYTES as u64, + None, + ) + .unwrap(); + assert!(read.batch.truncated); + } + #[test] fn retained_output_codec_preserves_utf8_full_pages_for_two_readers() { let source = source(); diff --git a/crates/freshell-session-host/src/output_journal.rs b/crates/freshell-session-host/src/output_journal.rs index 0ca2be0ca..bbaf62f36 100644 --- a/crates/freshell-session-host/src/output_journal.rs +++ b/crates/freshell-session-host/src/output_journal.rs @@ -31,6 +31,8 @@ pub struct OutputJournal { next_seq: u64, previous_index: Vec<TerminalSpoolIndexEntry>, current_index: Vec<TerminalSpoolIndexEntry>, + previous_torn_tail: bool, + current_torn_tail: bool, completion: Option<TerminalSpoolCompletion>, failure: Option<String>, #[cfg(test)] @@ -86,6 +88,8 @@ impl OutputJournal { next_seq: 1, previous_index: Vec::new(), current_index: Vec::new(), + previous_torn_tail: false, + current_torn_tail: false, completion: None, failure: None, #[cfg(test)] @@ -173,9 +177,19 @@ impl OutputJournal { } let source = self.descriptor(); let mut segments = Vec::with_capacity(2); - for (path, checkpoints, required) in [ - (self.previous_path(), &self.previous_index, false), - (self.current_path(), &self.current_index, true), + for (path, checkpoints, torn_tail, required) in [ + ( + self.previous_path(), + &self.previous_index, + self.previous_torn_tail, + false, + ), + ( + self.current_path(), + &self.current_index, + self.current_torn_tail, + true, + ), ] { let file = match std::fs::File::open(path) { Ok(file) => file, @@ -193,6 +207,7 @@ impl OutputJournal { reader: BufReader::new(file), file_bytes, checkpoints: checkpoints.clone(), + torn_tail, }); } let indexed = read_indexed_spool( @@ -201,7 +216,8 @@ impl OutputJournal { after_seq, max_bytes, self.completion.clone(), - )?; + ) + .map_err(|error| error.to_string())?; #[cfg(test)] self.last_read_bytes_scanned.set(indexed.bytes_scanned); Ok(indexed.batch) @@ -245,7 +261,9 @@ impl OutputJournal { .map_err(|e| e.to_string())?; sync_parent(¤t)?; self.previous_index = std::mem::take(&mut self.current_index); - write_index_atomic(¤t_index_path, &self.descriptor(), &[], 0)?; + self.previous_torn_tail = self.current_torn_tail; + self.current_torn_tail = false; + write_index_atomic(¤t_index_path, &self.descriptor(), &[], 0, 0, false)?; } let current_len = current.metadata().map(|m| m.len()).unwrap_or(0); let checkpoint_needed = self @@ -281,13 +299,13 @@ impl OutputJournal { fn seed_sequence_from_spool(&mut self) -> Result<(), String> { let source = self.descriptor(); - let (previous_index, previous_head) = rebuild_spool_index( + let (previous_index, previous_head, previous_torn_tail) = rebuild_spool_index( &self.previous_path(), &self.previous_index_path(), false, &source, )?; - let (current_index, current_head) = rebuild_spool_index( + let (current_index, current_head, current_torn_tail) = rebuild_spool_index( &self.current_path(), &self.current_index_path(), true, @@ -295,6 +313,8 @@ impl OutputJournal { )?; self.previous_index = previous_index; self.current_index = current_index; + self.previous_torn_tail = previous_torn_tail; + self.current_torn_tail = current_torn_tail; let head = current_head.max(previous_head); self.next_seq = head.checked_add(1).ok_or("output sequence exhausted")?; Ok(()) @@ -319,12 +339,12 @@ fn rebuild_spool_index( index_path: &Path, required: bool, source: &TerminalSpoolDescriptor, -) -> Result<(Vec<TerminalSpoolIndexEntry>, u64), String> { +) -> Result<(Vec<TerminalSpoolIndexEntry>, u64, bool), String> { let file = match std::fs::File::open(spool_path) { Ok(file) => file, Err(error) if !required && error.kind() == std::io::ErrorKind::NotFound => { - write_index_atomic(index_path, source, &[], 0)?; - return Ok((Vec::new(), 0)); + write_index_atomic(index_path, source, &[], 0, 0, false)?; + return Ok((Vec::new(), 0, false)); } Err(_) => return Err("retained output spool unavailable".into()), }; @@ -335,9 +355,21 @@ fn rebuild_spool_index( if file_bytes > source.segment_limit_bytes { return Err("retained spool exceeds segment bound".into()); } - let (checkpoints, head, _) = build_spool_index(BufReader::new(file), source, 0)?; - write_index_atomic(index_path, source, &checkpoints, file_bytes)?; - Ok((checkpoints, head)) + let (checkpoints, head, summary) = build_spool_index(BufReader::new(file), source, 0)?; + let scanned_through_offset = if checkpoints.is_empty() { + summary.last_complete_offset + } else { + 0 + }; + write_index_atomic( + index_path, + source, + &checkpoints, + file_bytes, + scanned_through_offset, + summary.torn_tail, + )?; + Ok((checkpoints, head, summary.torn_tail)) } fn write_index_atomic( @@ -345,8 +377,16 @@ fn write_index_atomic( source: &TerminalSpoolDescriptor, checkpoints: &[TerminalSpoolIndexEntry], spool_bytes: u64, + scanned_through_offset: u64, + torn_tail: bool, ) -> Result<(), String> { - let bytes = encode_spool_index(source, checkpoints, spool_bytes)?; + let bytes = encode_spool_index( + source, + checkpoints, + spool_bytes, + scanned_through_offset, + torn_tail, + )?; let temporary = path.with_extension("tmp"); let mut file = OpenOptions::new() .create(true) diff --git a/crates/freshell-supervisor/src/retained_output.rs b/crates/freshell-supervisor/src/retained_output.rs index 2bf5b352a..2c1692892 100644 --- a/crates/freshell-supervisor/src/retained_output.rs +++ b/crates/freshell-supervisor/src/retained_output.rs @@ -2,10 +2,11 @@ use crate::registry::RetainedOutputSource; use freshell_runtime_protocol::{ terminal_spool::{ build_spool_index, encode_spool_index, read_indexed_spool, read_spool_index, - scan_spool_from, write_spool_index_checkpoint, IndexedSpoolRead, IndexedSpoolSegment, - TerminalSpoolIndexEntry, MAX_SPOOL_INDEX_BYTES, MAX_SPOOL_LINE_BYTES, - SPOOL_COMPLETION_FILE, SPOOL_CURRENT_FILE, SPOOL_CURRENT_INDEX_FILE, SPOOL_DESCRIPTOR_FILE, - SPOOL_INDEX_STRIDE_BYTES, SPOOL_PREVIOUS_FILE, SPOOL_PREVIOUS_INDEX_FILE, + scan_spool_from, write_spool_index_checkpoint, IndexedSpoolRead, IndexedSpoolReadError, + IndexedSpoolSegment, TerminalSpoolIndex, TerminalSpoolIndexEntry, MAX_SPOOL_INDEX_BYTES, + MAX_SPOOL_LINE_BYTES, SPOOL_COMPLETION_FILE, SPOOL_CURRENT_FILE, SPOOL_CURRENT_INDEX_FILE, + SPOOL_DESCRIPTOR_FILE, SPOOL_INDEX_STRIDE_BYTES, SPOOL_PREVIOUS_FILE, + SPOOL_PREVIOUS_INDEX_FILE, }, RuntimeOutputBatch, TerminalExitObservation, TerminalSpoolCompletion, TerminalSpoolDescriptor, }; @@ -15,6 +16,11 @@ use std::{ path::Path, }; +#[cfg(test)] +std::thread_local! { + static FORCED_INDEX_REBUILDS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) }; +} + pub(crate) fn validate_source( source: &RetainedOutputSource, runtime_root: &Path, @@ -84,7 +90,7 @@ pub(crate) fn read( .as_deref() .is_some_and(|epoch| epoch != descriptor.stream_epoch); let read_after = if changed { 0 } else { after }; - let mut indexed = read_indexed( + let indexed = read_indexed( &source, &descriptor, read_after, @@ -92,14 +98,16 @@ pub(crate) fn read( completion.clone(), false, ); - if indexed.is_err() { - // A missing index is expected for older owned hosts. Rebuild it from - // the authoritative spool once, then use indexed paging. Invalid or - // stale indexes take the same repair path; malformed spool data still - // fails the rebuild rather than being hidden. - indexed = read_indexed(&source, &descriptor, read_after, budget, completion, true); - } - let mut batch = indexed?.batch; + let indexed = match indexed { + Err(IndexedSpoolReadError::InvalidIndex(_)) => { + // Rebuild only when the sidecar's offsets or bounds disagree with + // the authoritative spool. Page-budget and spool errors must not + // trigger another full scan. + read_indexed(&source, &descriptor, read_after, budget, completion, true) + } + result => result, + }; + let mut batch = indexed.map_err(|error| error.to_string())?.batch; if changed { batch.reset_required = true; batch.truncated = true; @@ -145,7 +153,12 @@ fn read_indexed( budget: u64, completion: Option<TerminalSpoolCompletion>, rebuild: bool, -) -> Result<IndexedSpoolRead, String> { +) -> Result<IndexedSpoolRead, IndexedSpoolReadError> { + #[cfg(test)] + if rebuild { + FORCED_INDEX_REBUILDS.with(|count| count.set(count.get() + 1)); + } + let mut segments = Vec::with_capacity(2); for (spool_name, index_name, required) in [ (SPOOL_PREVIOUS_FILE, SPOOL_PREVIOUS_INDEX_FILE, false), @@ -166,29 +179,29 @@ fn read_indexed( } let index_path = source.runtime_dir.join(index_name); - let checkpoints = if !rebuild { + let indexed = if !rebuild { match File::open(&index_path) { Ok(index_file) => { - if index_file + let index_bytes = index_file .metadata() .map_err(|_| "retained output index metadata unavailable")? - .len() - > MAX_SPOOL_INDEX_BYTES as u64 - { - return Err("retained output index exceeds byte bound".into()); - } - match read_spool_index(BufReader::new(index_file), descriptor, file_bytes) { - Ok(mut checkpoints) => { - extend_index_if_needed( - &file, - &index_path, - descriptor, - file_bytes, - &mut checkpoints, - )?; - Some(checkpoints) + .len(); + if index_bytes > MAX_SPOOL_INDEX_BYTES as u64 { + None + } else { + match read_spool_index(BufReader::new(index_file), descriptor, file_bytes) { + Ok(mut indexed) => { + extend_index_if_needed( + &file, + &index_path, + descriptor, + file_bytes, + &mut indexed, + )?; + Some(indexed) + } + Err(_) => None, } - Err(_) => None, } } Err(error) if error.kind() == std::io::ErrorKind::NotFound => None, @@ -197,10 +210,10 @@ fn read_indexed( } else { None }; - let checkpoints = match checkpoints { - Some(checkpoints) => checkpoints, + let indexed = match indexed { + Some(indexed) => indexed, None => { - let (checkpoints, _, _) = build_spool_index( + let (checkpoints, _, summary) = build_spool_index( BufReader::new( file.try_clone() .map_err(|_| "retained output spool unavailable")?, @@ -208,14 +221,32 @@ fn read_indexed( descriptor, 0, )?; - write_index_atomic(&index_path, descriptor, &checkpoints, file_bytes)?; - checkpoints + let scanned_through_offset = if checkpoints.is_empty() { + summary.last_complete_offset + } else { + 0 + }; + let torn_tail = summary.torn_tail; + write_index_atomic( + &index_path, + descriptor, + &checkpoints, + file_bytes, + scanned_through_offset, + torn_tail, + )?; + TerminalSpoolIndex { + checkpoints, + scanned_through_offset, + torn_tail, + } } }; segments.push(IndexedSpoolSegment { reader: BufReader::new(file), file_bytes, - checkpoints, + checkpoints: indexed.checkpoints, + torn_tail: indexed.torn_tail, }); } read_indexed_spool(descriptor.clone(), segments, after, budget, completion) @@ -226,11 +257,49 @@ fn extend_index_if_needed( index_path: &Path, descriptor: &TerminalSpoolDescriptor, file_bytes: u64, - checkpoints: &mut Vec<TerminalSpoolIndexEntry>, -) -> Result<(), String> { - let Some(mut last_checkpoint) = checkpoints.last().copied() else { + indexed: &mut TerminalSpoolIndex, +) -> Result<(), IndexedSpoolReadError> { + if indexed.checkpoints.is_empty() { + if file_bytes <= indexed.scanned_through_offset { + return Ok(()); + } + let mut reader = BufReader::new( + spool + .try_clone() + .map_err(|_| "retained output spool unavailable")?, + ); + reader + .seek(SeekFrom::Start(indexed.scanned_through_offset)) + .map_err(|_| "retained output seek failed")?; + let (new_checkpoints, _, summary) = + build_spool_index(&mut reader, descriptor, indexed.scanned_through_offset)?; + indexed.checkpoints.extend(new_checkpoints); + let scanned_through_offset = if indexed.checkpoints.is_empty() { + summary.last_complete_offset + } else { + 0 + }; + indexed.torn_tail |= summary.torn_tail; + let spool_bytes = spool + .metadata() + .map_err(|_| "retained spool metadata unavailable")? + .len(); + if spool_bytes > descriptor.segment_limit_bytes { + return Err("retained spool exceeds segment bound".into()); + } + write_index_atomic( + index_path, + descriptor, + &indexed.checkpoints, + spool_bytes, + scanned_through_offset, + indexed.torn_tail, + )?; + indexed.scanned_through_offset = scanned_through_offset; return Ok(()); - }; + } + + let mut last_checkpoint = indexed.checkpoints.last().copied().unwrap(); if file_bytes.saturating_sub(last_checkpoint.offset) <= SPOOL_INDEX_STRIDE_BYTES + MAX_SPOOL_LINE_BYTES as u64 { @@ -248,7 +317,8 @@ fn extend_index_if_needed( let mut first = true; let mut previous_seq = 0_u64; let mut appended = Vec::new(); - scan_spool_from( + let mut invalid_index = false; + let scan = scan_spool_from( &mut reader, descriptor, last_checkpoint.offset, @@ -256,6 +326,7 @@ fn extend_index_if_needed( if first { first = false; if frame.seq_start != last_checkpoint.seq { + invalid_index = true; return Err("retained output index does not match its spool".into()); } } else if frame.seq_start != previous_seq.saturating_add(1) { @@ -272,9 +343,19 @@ fn extend_index_if_needed( } Ok(false) }, - )?; + ); + let summary = scan.map_err(|error| { + if invalid_index { + IndexedSpoolReadError::InvalidIndex(error) + } else { + IndexedSpoolReadError::Other(error) + } + })?; + indexed.torn_tail |= summary.torn_tail; if first { - return Err("retained output index checkpoint is missing from its spool".into()); + return Err(IndexedSpoolReadError::InvalidIndex( + "retained output index checkpoint is missing from its spool".into(), + )); } if !appended.is_empty() { let mut index = OpenOptions::new() @@ -287,7 +368,7 @@ fn extend_index_if_needed( index .sync_data() .map_err(|_| "retained output index sync failed")?; - checkpoints.extend(appended); + indexed.checkpoints.extend(appended); } Ok(()) } @@ -297,8 +378,16 @@ fn write_index_atomic( descriptor: &TerminalSpoolDescriptor, checkpoints: &[freshell_runtime_protocol::terminal_spool::TerminalSpoolIndexEntry], spool_bytes: u64, + scanned_through_offset: u64, + torn_tail: bool, ) -> Result<(), String> { - let bytes = encode_spool_index(descriptor, checkpoints, spool_bytes)?; + let bytes = encode_spool_index( + descriptor, + checkpoints, + spool_bytes, + scanned_through_offset, + torn_tail, + )?; let temporary = path.with_extension("tmp"); let mut file = OpenOptions::new() .create(true) @@ -323,7 +412,7 @@ fn write_index_atomic( mod tests { use super::*; use freshell_runtime_protocol::{ - terminal_spool::{TerminalSpoolFrame, MAX_OUTPUT_FRAME_BYTES}, + terminal_spool::{write_spool_index_header, TerminalSpoolFrame, MAX_OUTPUT_FRAME_BYTES}, IncarnationId, TerminalLaunchSpec, }; @@ -333,8 +422,15 @@ mod tests { let incarnation_id = IncarnationId::new(); let terminal_id = "retained-terminal"; let stream_epoch = "launch-prefix-hostboot-indexed"; - let source = RetainedOutputSource { - incarnation_id: incarnation_id.clone(), + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id, + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + segment_limit_bytes: 32 * 1024 * 1024, + }; + let make_source = || RetainedOutputSource { + incarnation_id: descriptor.incarnation_id.clone(), runtime_dir: directory.path().to_path_buf(), terminal: serde_json::from_value::<TerminalLaunchSpec>(serde_json::json!({ "terminalId":terminal_id,"streamId":"launch-prefix","mode":"shell", @@ -345,13 +441,6 @@ mod tests { .unwrap(), native_session_id: None, }; - let descriptor = TerminalSpoolDescriptor { - schema_version: 1, - incarnation_id, - terminal_id: terminal_id.into(), - stream_epoch: stream_epoch.into(), - segment_limit_bytes: 32 * 1024 * 1024, - }; let mut spool = File::create(directory.path().join(SPOOL_CURRENT_FILE)).unwrap(); let payload = "x".repeat(MAX_OUTPUT_FRAME_BYTES); for seq in 1..=96 { @@ -370,7 +459,7 @@ mod tests { drop(spool); let first = read_indexed( - &source, + &make_source(), &descriptor, 94, MAX_OUTPUT_FRAME_BYTES as u64, @@ -382,7 +471,7 @@ mod tests { assert!(directory.path().join(SPOOL_CURRENT_INDEX_FILE).is_file()); let second = read_indexed( - &source, + &make_source(), &descriptor, 95, MAX_OUTPUT_FRAME_BYTES as u64, @@ -398,4 +487,170 @@ mod tests { spool_bytes ); } + + #[test] + fn legacy_supervisor_recovers_new_epoch_output_after_indexing_old_epoch_spool() { + let directory = tempfile::tempdir().unwrap(); + let incarnation_id = IncarnationId::new(); + let terminal_id = "retained-terminal"; + let stream_epoch = "launch-prefix-hostboot-indexed"; + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id, + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + segment_limit_bytes: 32 * 1024 * 1024, + }; + let make_source = || RetainedOutputSource { + incarnation_id: descriptor.incarnation_id.clone(), + runtime_dir: directory.path().to_path_buf(), + terminal: serde_json::from_value::<TerminalLaunchSpec>(serde_json::json!({ + "terminalId":terminal_id,"streamId":"launch-prefix","mode":"shell", + "program":"/bin/sh","cwd":"/workspace","runAsUid":65534, + "runAsGid":0,"cols":120,"rows":30,"projectKey":"fixture", + "workspacePath":"/workspace" + })) + .unwrap(), + native_session_id: None, + }; + std::fs::write( + directory.path().join(SPOOL_DESCRIPTOR_FILE), + serde_json::to_vec(&descriptor).unwrap(), + ) + .unwrap(); + let old_frame = freshell_runtime_protocol::RuntimeOutputFrame { + terminal_id: terminal_id.into(), + stream_epoch: "previous-host-epoch".into(), + seq_start: 11, + seq_end: 11, + data: "old epoch output".into(), + }; + let mut spool = File::create(directory.path().join(SPOOL_CURRENT_FILE)).unwrap(); + serde_json::to_writer(&mut spool, &TerminalSpoolFrame { frame: old_frame }).unwrap(); + spool.write_all(b"\n").unwrap(); + spool.sync_all().unwrap(); + drop(spool); + + let empty = read(make_source(), 0, MAX_OUTPUT_FRAME_BYTES as u64, None, false).unwrap(); + assert_eq!(empty.head_seq, 0); + assert!(directory.path().join(SPOOL_CURRENT_INDEX_FILE).is_file()); + + let frame = freshell_runtime_protocol::RuntimeOutputFrame { + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + seq_start: 1, + seq_end: 1, + data: "durable before index checkpoint".into(), + }; + let mut spool = OpenOptions::new() + .append(true) + .open(directory.path().join(SPOOL_CURRENT_FILE)) + .unwrap(); + serde_json::to_writer(&mut spool, &TerminalSpoolFrame { frame }).unwrap(); + spool.write_all(b"\n").unwrap(); + spool.sync_all().unwrap(); + + let recovered = read(make_source(), 0, MAX_OUTPUT_FRAME_BYTES as u64, None, true).unwrap(); + assert_eq!(recovered.frames.len(), 1); + assert_eq!(recovered.frames[0].seq_start, 1); + assert_eq!(recovered.head_seq, 1); + assert_eq!(recovered.completion.unwrap().head_seq, 1); + } + + #[test] + fn too_small_page_budget_does_not_force_a_full_spool_rebuild() { + let directory = tempfile::tempdir().unwrap(); + let incarnation_id = IncarnationId::new(); + let terminal_id = "retained-terminal"; + let stream_epoch = "launch-prefix-hostboot-indexed"; + let source = RetainedOutputSource { + incarnation_id: incarnation_id.clone(), + runtime_dir: directory.path().to_path_buf(), + terminal: serde_json::from_value::<TerminalLaunchSpec>(serde_json::json!({ + "terminalId":terminal_id,"streamId":"launch-prefix","mode":"shell", + "program":"/bin/sh","cwd":"/workspace","runAsUid":65534, + "runAsGid":0,"cols":120,"rows":30,"projectKey":"fixture", + "workspacePath":"/workspace" + })) + .unwrap(), + native_session_id: None, + }; + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id, + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + segment_limit_bytes: 32 * 1024 * 1024, + }; + std::fs::write( + directory.path().join(SPOOL_DESCRIPTOR_FILE), + serde_json::to_vec(&descriptor).unwrap(), + ) + .unwrap(); + + let mut spool = File::create(directory.path().join(SPOOL_CURRENT_FILE)).unwrap(); + let make_frame = |seq| freshell_runtime_protocol::RuntimeOutputFrame { + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + seq_start: seq, + seq_end: seq, + data: "x".repeat(MAX_OUTPUT_FRAME_BYTES), + }; + let first_offset = 0; + serde_json::to_writer( + &mut spool, + &TerminalSpoolFrame { + frame: make_frame(1), + }, + ) + .unwrap(); + spool.write_all(b"\n").unwrap(); + let broken_offset = spool.metadata().unwrap().len(); + spool.write_all(b"not-json\n").unwrap(); + let second_offset = spool.metadata().unwrap().len(); + serde_json::to_writer( + &mut spool, + &TerminalSpoolFrame { + frame: make_frame(2), + }, + ) + .unwrap(); + spool.write_all(b"\n").unwrap(); + spool.sync_all().unwrap(); + let spool_bytes = spool.metadata().unwrap().len(); + drop(spool); + + let mut index = File::create(directory.path().join(SPOOL_CURRENT_INDEX_FILE)).unwrap(); + write_spool_index_header(&mut index, &descriptor, 0, false).unwrap(); + write_spool_index_checkpoint( + &mut index, + TerminalSpoolIndexEntry { + seq: 1, + offset: first_offset, + }, + ) + .unwrap(); + write_spool_index_checkpoint( + &mut index, + TerminalSpoolIndexEntry { + seq: 2, + offset: second_offset, + }, + ) + .unwrap(); + index.sync_all().unwrap(); + assert!(broken_offset < second_offset && second_offset < spool_bytes); + + let rebuilds_before = FORCED_INDEX_REBUILDS.with(std::cell::Cell::get); + let error = read(source, 0, 1, None, false).unwrap_err(); + let rebuilds_after = FORCED_INDEX_REBUILDS.with(std::cell::Cell::get); + assert!( + error.contains("output budget cannot hold"), + "a page-budget error should be returned without rescanning the spool: {error}" + ); + assert_eq!( + rebuilds_after, rebuilds_before, + "a page-budget error must not force an index rebuild" + ); + } } From 634ca389c01e3cce1cde0aff2ea5b1068399072f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 13:02:10 -0700 Subject: [PATCH 463/493] Restore durable checkpoint rewind availability --- src/components/fresh-agent/FreshAgentView.tsx | 144 +++++++------ src/store/paneTypes.ts | 42 +++- src/store/panesSlice.ts | 5 +- src/store/persistedState.ts | 2 + .../fresh-agent/FreshAgentView.test.tsx | 190 +++++++++++++----- .../panesSlice.checkpoint-outcomes.test.ts | 47 +++++ 6 files changed, 321 insertions(+), 109 deletions(-) diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index aec12c76a..4776900e8 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -10,6 +10,7 @@ import { } from 'react' import { nanoid } from 'nanoid' import { + freshAgentCheckpointConversationKeyForContent, normalizeFreshAgentCheckpointWorkspace, type FreshAgentCheckpointOutcome, type FreshAgentPaneContent, @@ -310,9 +311,8 @@ function checkpointTurnAvailabilityKey(scope: string, id: string): string { } function checkpointAvailabilityScope(content: FreshAgentPaneContent): string { - // A provider may replace its provisional session id when it materializes - // the durable conversation. The pane create request is the stable identity - // for that conversation generation across the transition. + // Request keys fence in-flight sends within this pane. Persisted outcomes + // also carry the durable session key so another pane can recover them. return content.createRequestId } @@ -337,9 +337,14 @@ function checkpointOutcomeAvailability(outcome: FreshAgentCheckpointOutcome): Ch function checkpointAvailabilityMap(content: FreshAgentPaneContent): Map<string, CheckpointTurnAvailability> { const scope = checkpointAvailabilityScope(content) + const sessionKey = freshAgentCheckpointConversationKeyForContent(content) const availability = new Map<string, CheckpointTurnAvailability>() for (const outcome of content.checkpointOutcomes ?? []) { - if (outcome.conversationRequestId && outcome.conversationRequestId !== content.createRequestId) continue + if ( + outcome.conversationRequestId + && outcome.conversationRequestId !== content.createRequestId + && (!sessionKey || outcome.conversationSessionKey !== sessionKey) + ) continue const value = checkpointOutcomeAvailability(outcome) availability.set(checkpointAvailabilityKey(scope, 'request', outcome.requestId), value) if (outcome.displayTurnId) { @@ -1118,10 +1123,16 @@ export function FreshAgentView({ }) if (!requestId || availability.status === 'pending') return + const state = appStore.getState() + const paneRoot = state.panes.layouts[tabId] + const latestPaneContent = paneRoot ? findPaneContent(paneRoot, paneId) : null + if (latestPaneContent?.kind !== 'fresh-agent' || latestPaneContent.createRequestId !== scope) return + const conversationSessionKey = freshAgentCheckpointConversationKeyForContent(latestPaneContent) const outcome: FreshAgentCheckpointOutcome = { version: 1, requestId, conversationRequestId: scope, + ...(conversationSessionKey ? { conversationSessionKey } : {}), ...(availability.displayTurnId ? { displayTurnId: availability.displayTurnId } : {}), ...(availability.nativeTurnId ? { nativeTurnId: availability.nativeTurnId } : {}), status: availability.status, @@ -1136,10 +1147,6 @@ export function FreshAgentView({ } : {}), } - const state = appStore.getState() - const paneRoot = state.panes.layouts[tabId] - const latestPaneContent = paneRoot ? findPaneContent(paneRoot, paneId) : null - if (latestPaneContent?.kind !== 'fresh-agent' || latestPaneContent.createRequestId !== scope) return const priorOutcomes = latestPaneContent.checkpointOutcomes ?? [] const checkpointOutcomes = [ ...priorOutcomes.filter((item) => item.requestId !== requestId), @@ -2749,6 +2756,21 @@ export function FreshAgentView({ ) { const current = paneContentRef.current const sessionRef = message.sessionRef ?? { provider: message.provider, sessionId: message.sessionId } + const nextContent = { + ...current, + sessionId: message.sessionId, + sessionRef, + resumeSessionId: message.sessionId, + restoreError: undefined, + } + const conversationSessionKey = freshAgentCheckpointConversationKeyForContent(nextContent) + const checkpointOutcomes = conversationSessionKey + ? current.checkpointOutcomes?.map((outcome) => ( + !outcome.conversationRequestId || outcome.conversationRequestId === current.createRequestId + ? { ...outcome, conversationSessionKey } + : outcome + )) + : current.checkpointOutcomes persistDurableFreshAgentFlavor({ provider: message.provider, sessionId: message.sessionId, @@ -2761,11 +2783,8 @@ export function FreshAgentView({ tabId, paneId, content: { - ...current, - sessionId: message.sessionId, - sessionRef, - resumeSessionId: message.sessionId, - restoreError: undefined, + ...nextContent, + ...(checkpointOutcomes ? { checkpointOutcomes } : {}), }, })) } @@ -4067,6 +4086,55 @@ export function FreshAgentView({ setNotice(`Rewind unavailable for this message: ${reason}`) return } + + const restoreCheckpoint = ( + checkpointId: string, + label: string, + restoreWorkspace: typeof workspace, + ) => Promise + .resolve(api.post('/api/fresh-agent/checkpoints/restore', { + cwd, + ...(restoreWorkspace ? { workspace: restoreWorkspace } : {}), + id: checkpointId, + })) + .then(() => setNotice(`Code rewound to before: "${label}"`)) + .catch((error: unknown) => { + if (error && typeof error === 'object') { + const details = (error as { details?: unknown }).details + if (details && typeof details === 'object' && (details as { code?: unknown }).code === 'workspace_unavailable') { + const reason = checkpointFailureReason(error) + const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId + publishCheckpointAvailability( + checkpointAvailabilityScope(current), + typeof requestId === 'string' ? requestId : undefined, + getFreshAgentDisplayTurnKey(turn), + { + status: 'unavailable', + reason, + code: checkpointFailureCode(error) ?? 'checkpoint_restore_failed', + ...(workspace ? { workspace } : {}), + }, + ) + setNotice(`Rewind unavailable for this message: ${reason}`) + return + } + } + setNotice(error instanceof Error ? `Rewind failed: ${error.message}` : 'Rewind failed') + }) + + // An exact checkpoint ID came from the settled outcome, which is the + // authoritative association to this provider turn. Restore accepts any + // still-published ancestor, including commits outside the capped list. + if (availability?.checkpointId) { + const label = turn.summary.trim() || 'this message' + const confirmed = typeof window === 'undefined' || window.confirm( + `Rewind code to the state before "${label}"?\n\nTracked files changed since will be overwritten. Files created since are left in place. The conversation is not affected.`, + ) + if (!confirmed) return + void restoreCheckpoint(availability.checkpointId, label, workspace) + return + } + const query = new URLSearchParams({ cwd }) if (workspace) { query.set('workspacePath', workspace.path) @@ -4082,20 +4150,13 @@ export function FreshAgentView({ return } const checkpoints = result.checkpoints - // Settled outcomes persist the exact create response across reloads. - // This is authoritative when metadata association failed: duplicate - // prompt labels cannot safely identify which snapshot belongs to the - // selected user turn. Label matching remains only for legacy turns - // without a persisted checkpoint ID. - const checkpoint = availability?.checkpointId - ? checkpoints.find((entry) => entry.id === availability.checkpointId) ?? null - : isModernCheckpointTurn(turn) - ? null - : pickCheckpointForTurn(checkpoints, snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null + // Legacy turns do not have a durable outcome or provider identity, so + // keep their label-based lookup inside the bounded list window. + const checkpoint = isModernCheckpointTurn(turn) + ? null + : pickCheckpointForTurn(checkpoints, snapshot?.turns ?? [], turn) as CheckpointCreationEntry | null if (!checkpoint) { - const reason = availability?.checkpointId - ? 'The saved code checkpoint for this message is no longer available.' - : 'No code checkpoint was saved for this message.' + const reason = 'No code checkpoint was saved for this message.' const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId publishCheckpointAvailability( checkpointAvailabilityScope(current), @@ -4111,36 +4172,7 @@ export function FreshAgentView({ ) if (!confirmed) return const restoreWorkspace = normalizeFreshAgentCheckpointWorkspace(checkpoint.workspace) ?? workspace - return Promise - .resolve(api.post('/api/fresh-agent/checkpoints/restore', { - cwd, - ...(restoreWorkspace ? { workspace: restoreWorkspace } : {}), - id: checkpoint.id, - })) - .then(() => setNotice(`Code rewound to before: "${checkpoint.label}"`)) - .catch((error: unknown) => { - if (error && typeof error === 'object') { - const details = (error as { details?: unknown }).details - if (details && typeof details === 'object' && (details as { code?: unknown }).code === 'workspace_unavailable') { - const reason = checkpointFailureReason(error) - const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId - publishCheckpointAvailability( - checkpointAvailabilityScope(current), - typeof requestId === 'string' ? requestId : undefined, - getFreshAgentDisplayTurnKey(turn), - { - status: 'unavailable', - reason, - code: checkpointFailureCode(error) ?? 'checkpoint_restore_failed', - ...(workspace ? { workspace } : {}), - }, - ) - setNotice(`Rewind unavailable for this message: ${reason}`) - return - } - } - setNotice(error instanceof Error ? `Rewind failed: ${error.message}` : 'Rewind failed') - }) + return restoreCheckpoint(checkpoint.id, checkpoint.label, restoreWorkspace) }) .catch(() => { // A failed or malformed list read is uncertain. Keep the durable diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index dc4770a19..604b674c2 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -72,6 +72,8 @@ export type FreshAgentCheckpointOutcome = { version: 1 /** Conversation generation; changes only when the pane is deliberately replaced. */ conversationRequestId?: string + /** Durable provider session identity, allowing a reopened pane to reuse saved outcomes. */ + conversationSessionKey?: string requestId: string displayTurnId?: string nativeTurnId?: string @@ -90,6 +92,38 @@ function checkpointOutcomeString(value: unknown, maxLength: number): string | un return value } +export function freshAgentCheckpointConversationKey( + provider: unknown, + sessionType: unknown, + sessionId: unknown, +): string | undefined { + if (provider !== 'claude' && provider !== 'codex' && provider !== 'opencode') return undefined + if (sessionType !== 'freshclaude' && sessionType !== 'freshcodex' + && sessionType !== 'kilroy' && sessionType !== 'freshopencode') return undefined + const normalizedSessionId = checkpointOutcomeString(sessionId, 256) + if (!normalizedSessionId) return undefined + return JSON.stringify([provider, sessionType, normalizedSessionId]) +} + +export function freshAgentCheckpointConversationKeyForContent( + content: { + provider?: unknown + sessionType?: unknown + sessionId?: unknown + resumeSessionId?: unknown + sessionRef?: unknown + }, +): string | undefined { + const sessionRef = content.sessionRef && typeof content.sessionRef === 'object' && !Array.isArray(content.sessionRef) + ? content.sessionRef as Record<string, unknown> + : undefined + const provider = content.provider + const sessionId = sessionRef && sessionRef.provider === provider + ? sessionRef.sessionId + : (content.sessionId ?? content.resumeSessionId) + return freshAgentCheckpointConversationKey(provider, content.sessionType, sessionId) +} + function checkpointOutcomeReason(value: unknown): string | undefined { if (typeof value !== 'string') return undefined const normalized = value @@ -119,6 +153,7 @@ export function normalizeFreshAgentCheckpointWorkspace(value: unknown): FreshAge export function normalizeFreshAgentCheckpointOutcomes( value: unknown, conversationRequestId?: string, + conversationSessionKey?: string, ): FreshAgentCheckpointOutcome[] | undefined { if (!Array.isArray(value)) return undefined const byRequest = new Map<string, FreshAgentCheckpointOutcome>() @@ -131,10 +166,14 @@ export function normalizeFreshAgentCheckpointOutcomes( const recordConversationRequestId = record.conversationRequestId === undefined ? undefined : checkpointOutcomeString(record.conversationRequestId, 128) + const recordConversationSessionKey = record.conversationSessionKey === undefined + ? undefined + : checkpointOutcomeString(record.conversationSessionKey, 512) if (record.conversationRequestId !== undefined && (!recordConversationRequestId || !/^[A-Za-z0-9_.:-]+$/.test(recordConversationRequestId))) continue if (conversationRequestId && recordConversationRequestId - && recordConversationRequestId !== conversationRequestId) continue + && recordConversationRequestId !== conversationRequestId + && (!conversationSessionKey || recordConversationSessionKey !== conversationSessionKey)) continue if (record.status !== 'available' && record.status !== 'unavailable') continue const displayTurnId = checkpointOutcomeString(record.displayTurnId, 256) @@ -156,6 +195,7 @@ export function normalizeFreshAgentCheckpointOutcomes( version: 1, requestId, ...(recordConversationRequestId ? { conversationRequestId: recordConversationRequestId } : {}), + ...(recordConversationSessionKey ? { conversationSessionKey: recordConversationSessionKey } : {}), ...(displayTurnId ? { displayTurnId } : {}), ...(nativeTurnId ? { nativeTurnId } : {}), status: record.status, diff --git a/src/store/panesSlice.ts b/src/store/panesSlice.ts index a56cba339..8db8d28e9 100644 --- a/src/store/panesSlice.ts +++ b/src/store/panesSlice.ts @@ -2,6 +2,7 @@ import { createSlice, PayloadAction } from '@reduxjs/toolkit' import { nanoid } from 'nanoid' import { normalizeFreshAgentEffortOverride, + freshAgentCheckpointConversationKeyForContent, normalizeFreshAgentCheckpointOutcomes, normalizeFreshAgentModelEffortLevels, normalizeFreshAgentModelSelection, @@ -88,8 +89,9 @@ function normalizeCheckpointWorkspace(value: unknown): FreshAgentCheckpointWorks function normalizeCheckpointOutcomes( value: unknown, conversationRequestId?: string, + conversationSessionKey?: string, ): FreshAgentCheckpointOutcome[] | undefined { - return normalizeFreshAgentCheckpointOutcomes(value, conversationRequestId) + return normalizeFreshAgentCheckpointOutcomes(value, conversationRequestId, conversationSessionKey) } function buildPreservedSessionRef( @@ -204,6 +206,7 @@ function normalizePaneContent( const checkpointOutcomes = normalizeCheckpointOutcomes( rawFreshAgent.checkpointOutcomes, outcomeConversationRequestId, + freshAgentCheckpointConversationKeyForContent(rawFreshAgent), ) const closeError = typeof input.closeError === 'string' ? input.closeError : undefined const rawModelLabel = rawFreshAgent.modelLabel diff --git a/src/store/persistedState.ts b/src/store/persistedState.ts index a80bfd3d9..473356cc6 100644 --- a/src/store/persistedState.ts +++ b/src/store/persistedState.ts @@ -21,6 +21,7 @@ import { stripSessionOwnedTabFreezeFlags, } from '@/lib/session-name-migration' import { + freshAgentCheckpointConversationKeyForContent, normalizeFreshAgentCheckpointWorkspace, normalizeFreshAgentCheckpointOutcomes, normalizeFreshAgentPaneModelSelection, @@ -352,6 +353,7 @@ function normalizeFreshAgentContent(content: Record<string, unknown>): Record<st const checkpointOutcomes = normalizeFreshAgentCheckpointOutcomes( content.checkpointOutcomes, typeof content.createRequestId === 'string' ? content.createRequestId : undefined, + freshAgentCheckpointConversationKeyForContent(content), ) const existingRestoreError = readRestoreError(content.restoreError) if (existingRestoreError) { diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index 8539b6c74..6651104df 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3419,7 +3419,6 @@ describe('FreshAgentView', () => { const store = createStore() const checkpointWorkspace = { path: '/repo/project', provenance: 'directory-picker' as const } const resolvedCheckpointWorkspace = { path: '/repo/project-root', provenance: 'directory-picker' as const } - const listedCheckpointWorkspace = resolvedCheckpointWorkspace let onMessage: ((message: Record<string, unknown>) => void) | undefined wsMock.onMessage.mockImplementation((handler: (message: Record<string, unknown>) => void) => { onMessage = handler @@ -3477,16 +3476,6 @@ describe('FreshAgentView', () => { ], }) apiMock.post.mockResolvedValue({ ok: true }) - apiMock.get.mockResolvedValue({ - checkpoints: [{ - id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', - ts: 1, - label: 'Ship it', - requestId: 'req-normalized-send', - turnId: 'display-user-1', - workspace: listedCheckpointWorkspace, - }], - }) store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', @@ -3611,6 +3600,7 @@ describe('FreshAgentView', () => { expect(persistedOutcomes).toEqual(expect.arrayContaining([expect.objectContaining({ requestId, conversationRequestId: 'req-normalized-send', + conversationSessionKey: JSON.stringify(['codex', 'freshcodex', 'thread-materialized-send']), displayTurnId: 'codex-turn-17:row-0', nativeTurnId: 'codex-turn-17', status: 'available', @@ -3629,18 +3619,14 @@ describe('FreshAgentView', () => { vi.spyOn(window, 'confirm').mockReturnValue(true) fireEvent.click(screen.getByRole('button', { name: 'Rewind code to here' })) - await waitFor(() => { - expect(apiMock.get).toHaveBeenCalledWith( - '/api/fresh-agent/checkpoints?cwd=%2Frepo&workspacePath=%2Frepo%2Fproject-root&workspaceProvenance=directory-picker', - ) - }) await waitFor(() => { expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { cwd: '/repo', - workspace: listedCheckpointWorkspace, + workspace: resolvedCheckpointWorkspace, id: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', }) }) + expect(apiMock.get).not.toHaveBeenCalledWith(expect.stringContaining('/api/fresh-agent/checkpoints?')) }) it('accepts a replayed checkpoint outcome after reload when it carries an exact provider row identity', async () => { @@ -3830,6 +3816,129 @@ describe('FreshAgentView', () => { expect(apiMock.post).not.toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', expect.anything()) }) + it('restores saved rewind availability when the same durable session opens with a new pane request', async () => { + const sessionId = 'thread-reopened-checkpoints' + const checkpointId = 'dddddddddddddddddddddddddddddddd' + const sessionKey = JSON.stringify(['codex', 'freshcodex', sessionId]) + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'reopened conversation', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: `${sessionId}:row-0`, + turnId: `${sessionId}:row-0`, + nativeTurnId: sessionId, + role: 'user', + summary: 'A saved turn in this session', + items: [{ id: 'reopened-user-text', kind: 'text', text: 'A saved turn in this session' }], + }], + }) + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + sessionId, + sessionRef: { provider: 'codex', sessionId }, + createRequestId: 'new-pane-request', + status: 'idle', + initialCwd: '/repo', + checkpointOutcomes: [{ + version: 1, + conversationRequestId: 'old-pane-request', + conversationSessionKey: sessionKey, + requestId: 'saved-turn-request', + displayTurnId: `${sessionId}:row-0`, + nativeTurnId: sessionId, + status: 'available', + checkpointId, + workspace: { path: '/repo', provenance: 'directory-picker' }, + }], + } as never, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const userRow = await screen.findByRole('article', { name: 'You transcript turn' }) + expect(within(userRow).getByRole('status', { name: 'Code rewind available for this message' })) + .toHaveTextContent('Code rewind is available for this message.') + expect(within(userRow).getByRole('button', { name: 'Rewind code to here' })).toBeEnabled() + }) + + it('restores a saved checkpoint by ID even when it is outside the newest checkpoint page', async () => { + const checkpointId = 'eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee' + const workspace = { path: '/repo/older-checkpoint', provenance: 'directory-picker' as const } + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'older checkpoint conversation', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: 'older-checkpoint-row', + turnId: 'older-checkpoint-row', + nativeTurnId: 'older-checkpoint-native-turn', + role: 'user', + summary: 'Restore a checkpoint older than the list window', + items: [{ id: 'older-checkpoint-user-text', kind: 'text', text: 'Restore a checkpoint older than the list window' }], + }], + }) + apiMock.get.mockResolvedValue({ checkpoints: [{ + id: 'ffffffffffffffffffffffffffffffff', + ts: 100, + label: 'A newer checkpoint from the capped list', + workspace, + }] }) + apiMock.post.mockResolvedValue({ ok: true }) + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-older-checkpoint', + sessionId: 'thread-older-checkpoint', + status: 'idle', + initialCwd: '/repo', + checkpointOutcomes: [{ + version: 1, + conversationRequestId: 'req-older-checkpoint', + requestId: 'request-older-checkpoint', + displayTurnId: 'older-checkpoint-row', + nativeTurnId: 'older-checkpoint-native-turn', + status: 'available', + checkpointId, + workspace, + }], + } as never, + })) + + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + vi.spyOn(window, 'confirm').mockReturnValue(true) + const userRow = await screen.findByRole('article', { name: 'You transcript turn' }) + fireEvent.click(within(userRow).getByRole('button', { name: 'Rewind code to here' })) + + await waitFor(() => { + expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: '/repo', + workspace, + id: checkpointId, + }) + }) + expect(apiMock.get).not.toHaveBeenCalledWith(expect.stringContaining('/api/fresh-agent/checkpoints?')) + }) + it('uses the selected checkpoint workspace to rewind when a resumed pane has no launch cwd', async () => { const checkpointId = 'cccccccccccccccccccccccccccccccc' const workspace = { path: '/repo/selected-without-launch-cwd', provenance: 'directory-picker' as const } @@ -3847,12 +3956,6 @@ describe('FreshAgentView', () => { items: [{ id: 'resume-user-text', kind: 'text', text: 'Restore the selected workspace' }], }], }) - apiMock.get.mockResolvedValue({ checkpoints: [{ - id: checkpointId, - ts: 1, - label: 'Restore the selected workspace', - workspace, - }] }) const store = createStore() store.dispatch(initLayout({ tabId: 'tab-1', @@ -3891,15 +3994,13 @@ describe('FreshAgentView', () => { fireEvent.click(within(article as HTMLElement).getByRole('button', { name: 'Rewind code to here' })) await waitFor(() => { - expect(apiMock.get).toHaveBeenCalledWith( - '/api/fresh-agent/checkpoints?cwd=%2Frepo%2Fselected-without-launch-cwd&workspacePath=%2Frepo%2Fselected-without-launch-cwd&workspaceProvenance=directory-picker', - ) expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { cwd: '/repo/selected-without-launch-cwd', workspace, id: checkpointId, }) }) + expect(apiMock.get).not.toHaveBeenCalledWith(expect.stringContaining('/api/fresh-agent/checkpoints?')) }) it('rewinds to the persisted checkpoint ID after reload when duplicate prompts have no metadata links', async () => { @@ -3928,12 +4029,8 @@ describe('FreshAgentView', () => { capabilities: { send: true, interrupt: true, fork: true }, turns, }) - // Metadata writes failed before reload, so the list has no request/turn - // association. The durable outcomes still identify each exact checkpoint. - apiMock.get.mockResolvedValue({ checkpoints: [ - { id: secondCheckpointId, ts: 2, label: 'Repeat this prompt' }, - { id: firstCheckpointId, ts: 1, label: 'Repeat this prompt' }, - ] }) + // The list can be missing metadata links; settled outcomes still identify + // the exact checkpoint for each duplicate prompt. const persistedContent = JSON.parse(JSON.stringify({ kind: 'fresh-agent', sessionType: 'freshcodex', @@ -3995,24 +4092,14 @@ describe('FreshAgentView', () => { }) }) - it('preserves an exact available checkpoint after a transient list failure and allows retry', async () => { - const checkpointId = '33333333333333333333333333333333' + it('keeps legacy label-based rewind retryable after a transient checkpoint list failure', async () => { const checkpointWorkspace = { path: '/repo/selected', provenance: 'directory-picker' as const } - const persistedOutcome = { - version: 1 as const, - conversationRequestId: 'req-checkpoint-list-retry', - requestId: 'request-checkpoint-list-retry', - displayTurnId: 'display-retry-turn', - status: 'available' as const, - checkpointId, - workspace: checkpointWorkspace, - } apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ status: 'idle', summary: 'saved turn', capabilities: { send: true, interrupt: true, fork: true }, turns: [{ - id: 'native-retry-turn', + id: 'display-retry-turn', turnId: 'display-retry-turn', role: 'user', summary: 'Retry checkpoint lookup', @@ -4024,7 +4111,12 @@ describe('FreshAgentView', () => { code: 'workspace_unavailable', details: 'The selected project workspace is temporarily unavailable.', })) - .mockResolvedValue({ checkpoints: [{ id: checkpointId, ts: 1, label: 'Retry checkpoint lookup', workspace: checkpointWorkspace }] }) + .mockResolvedValue({ checkpoints: [{ + id: '33333333333333333333333333333333', + ts: 1, + label: 'Retry checkpoint lookup', + workspace: checkpointWorkspace, + }] }) const persistedContent = JSON.parse(JSON.stringify({ kind: 'fresh-agent', sessionType: 'freshcodex', @@ -4034,7 +4126,6 @@ describe('FreshAgentView', () => { status: 'idle', initialCwd: '/repo', checkpointWorkspace, - checkpointOutcomes: [persistedOutcome], })) as FreshAgentPaneContent const store = createStore() store.dispatch(initLayout({ tabId: 'tab-1', paneId: 'pane-1', content: persistedContent })) @@ -4050,9 +4141,6 @@ describe('FreshAgentView', () => { fireEvent.click(rewind) expect(await screen.findByText('Could not check the saved code checkpoint list. Try again.')) .toBeInTheDocument() - expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual([persistedOutcome]) - expect(screen.getByRole('status', { name: 'Code rewind available for this message' })) - .toHaveTextContent('Code rewind is available for this message.') expect(apiMock.post).not.toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', expect.anything()) fireEvent.click(rewind) @@ -4060,10 +4148,10 @@ describe('FreshAgentView', () => { expect(apiMock.post).toHaveBeenCalledWith('/api/fresh-agent/checkpoints/restore', { cwd: '/repo', workspace: checkpointWorkspace, - id: checkpointId, + id: '33333333333333333333333333333333', }) }) - expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual([persistedOutcome]) + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toBeUndefined() }) it('does not promote a live provider cwd into an explicit checkpoint workspace', async () => { diff --git a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts index 134a5dcd8..bf626bedb 100644 --- a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts +++ b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts @@ -134,4 +134,51 @@ describe('FreshAgent checkpoint outcomes', () => { code: 'late_failure', }) }) + + it('retains outcomes from an earlier pane create when the durable session is the same', () => { + const sessionKey = JSON.stringify(['codex', 'freshcodex', 'thread-reopened-checkpoints']) + const state = panesReducer(undefined, initLayout({ + tabId: 'tab-checkpoints', + paneId: 'pane-checkpoints', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + sessionId: 'thread-reopened-checkpoints', + sessionRef: { provider: 'codex', sessionId: 'thread-reopened-checkpoints' }, + createRequestId: 'new-pane-create', + status: 'idle', + checkpointOutcomes: [ + { + requestId: 'saved-turn', + conversationRequestId: 'old-pane-create', + conversationSessionKey: sessionKey, + displayTurnId: 'thread-reopened-checkpoints:row-0', + nativeTurnId: 'thread-reopened-checkpoints', + status: 'available', + checkpointId: 'saved-checkpoint', + }, + { + requestId: 'other-session-turn', + conversationRequestId: 'old-pane-create', + conversationSessionKey: JSON.stringify(['codex', 'freshcodex', 'different-thread']), + displayTurnId: 'different-thread:row-0', + status: 'available', + checkpointId: 'other-session-checkpoint', + }, + ], + } as never, + })) + + const leaf = state.layouts['tab-checkpoints'] as Extract<PaneNode, { type: 'leaf' }> + expect(leaf.content).toMatchObject({ + kind: 'fresh-agent', + checkpointOutcomes: [{ + requestId: 'saved-turn', + conversationRequestId: 'old-pane-create', + conversationSessionKey: sessionKey, + checkpointId: 'saved-checkpoint', + }], + }) + }) }) From 5d76df4332dfa8697ca7072eab01ac7f841cf2be Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 13:18:01 -0700 Subject: [PATCH 464/493] Publish provider reference aliases transactionally --- crates/freshell-session-host/src/main.rs | 668 ++++++++++++++++++++++- 1 file changed, 662 insertions(+), 6 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index ffddd5a46..6f0512651 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -2138,6 +2138,170 @@ fn install_provider_resource_reference_aliases( Ok((installed, removed)) } +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ProviderResourceReferenceAliasPlan { + active: Vec<ProviderResourceReferenceAliasInstall>, + retired: Vec<ProviderResourceReferenceAliasRetirement>, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ProviderResourceReferenceAliasInstall { + mount_root: String, + relative_path: String, + reference_index: u16, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ProviderResourceReferenceAliasRetirement { + mount_root: String, + relative_path: String, +} + +impl ProviderResourceReferenceAliasInstall { + fn provider_path(&self) -> PathBuf { + Path::new(&self.mount_root).join(&self.relative_path) + } +} + +impl ProviderResourceReferenceAliasRetirement { + fn provider_path(&self) -> PathBuf { + Path::new(&self.mount_root).join(&self.relative_path) + } +} + +fn provider_resource_reference_alias_plan( + context: &freshell_runtime_protocol::ProviderLaunchContext, +) -> Result<ProviderResourceReferenceAliasPlan, String> { + use freshell_runtime_protocol::ProviderResourceKind; + + let mut plan = ProviderResourceReferenceAliasPlan::default(); + let mut paths = std::collections::BTreeSet::new(); + for mount in context + .prepared_inputs + .iter() + .flat_map(|inputs| &inputs.resource_mounts) + { + if mount.reference_aliases.is_empty() && mount.retired_reference_aliases.is_empty() { + continue; + } + if mount.kind != ProviderResourceKind::Directory { + return Err("provider reference aliases require a directory resource".into()); + } + for alias in &mount.reference_aliases { + let path = Path::new(&alias.relative_path); + if path.as_os_str().is_empty() + || path.is_absolute() + || path + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("provider reference alias path is unsafe".into()); + } + let provider_path = Path::new(&mount.provider_relative_path).join(path); + if !paths.insert(provider_path.to_string_lossy().into_owned()) { + return Err("provider reference alias paths overlap".into()); + } + plan.active.push(ProviderResourceReferenceAliasInstall { + mount_root: mount.provider_relative_path.clone(), + relative_path: alias.relative_path.clone(), + reference_index: alias.reference_index, + }); + } + for relative in &mount.retired_reference_aliases { + let path = Path::new(relative); + if path.as_os_str().is_empty() + || path.is_absolute() + || path + .components() + .any(|part| !matches!(part, std::path::Component::Normal(_))) + { + return Err("provider reference alias path is unsafe".into()); + } + let provider_path = Path::new(&mount.provider_relative_path).join(path); + if !paths.insert(provider_path.to_string_lossy().into_owned()) { + return Err("provider reference alias paths overlap".into()); + } + plan.retired.push(ProviderResourceReferenceAliasRetirement { + mount_root: mount.provider_relative_path.clone(), + relative_path: relative.clone(), + }); + } + } + plan.active.sort_by_key(|alias| alias.provider_path()); + plan.retired.sort_by_key(|alias| alias.provider_path()); + provider_resource_reference_alias_paths(&plan)?; + Ok(plan) +} + +fn provider_resource_reference_alias_paths( + plan: &ProviderResourceReferenceAliasPlan, +) -> Result<std::collections::BTreeSet<String>, String> { + let resource_roots = Path::new(".freshell/provider-roots"); + let mut paths = std::collections::BTreeSet::new(); + let mut parsed = Vec::new(); + for (mount_root, relative_path) in plan + .active + .iter() + .map(|alias| (alias.mount_root.as_str(), alias.relative_path.as_str())) + .chain( + plan.retired + .iter() + .map(|alias| (alias.mount_root.as_str(), alias.relative_path.as_str())), + ) + { + let mount_root = Path::new(mount_root); + let relative_path = Path::new(relative_path); + let safe_relative = |path: &Path| { + !path.as_os_str().is_empty() + && !path.is_absolute() + && path + .components() + .all(|part| matches!(part, std::path::Component::Normal(_))) + }; + if !safe_relative(mount_root) + || !mount_root.starts_with(resource_roots) + || mount_root.components().count() <= resource_roots.components().count() + || !safe_relative(relative_path) + { + return Err("provider reference alias metadata has an invalid path".into()); + } + let path = mount_root.join(relative_path); + let path = path + .to_str() + .ok_or("provider reference alias path is not UTF-8")? + .to_owned(); + if !paths.insert(path.clone()) { + return Err("provider reference alias paths overlap".into()); + } + parsed.push(PathBuf::from(path)); + } + for (index, path) in parsed.iter().enumerate() { + if parsed + .iter() + .skip(index + 1) + .any(|other| path != other && (path.starts_with(other) || other.starts_with(path))) + { + return Err("provider reference alias paths overlap".into()); + } + } + Ok(paths) +} + +fn provider_reference_alias_overlaps_selected_paths( + plan: &ProviderResourceReferenceAliasPlan, + selected: &std::collections::BTreeSet<String>, +) -> bool { + plan.active.iter().any(|alias| { + let alias_path = alias.provider_path(); + selected + .iter() + .any(|path| Path::new(path).starts_with(&alias_path)) + }) +} + fn copy_provider_config_references_with_ownership( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -2998,11 +3162,22 @@ fn refresh_provider_config_in_stage( preserved, ) .map_err(|_| "preserved writable resource metadata write failed")?; + let reference_aliases = provider_resource_reference_alias_plan(context)?; + let reference_alias_metadata = freshell_runtime_protocol::bounded_json_bytes( + &reference_aliases, + MAX_PROJECTION_METADATA_BYTES, + ) + .map_err(|_| "provider reference alias metadata profile exceeded")?; + std::fs::write( + stage.path().join("resource-reference-aliases.json"), + reference_alias_metadata, + ) + .map_err(|_| "provider reference alias metadata write failed")?; allow_provider_read_stage(stage.path(), unsafe { libc::getegid() })?; - apply_staged_provider_config(stage.path(), provider_home)?; - record_provider_resource_reference_aliases(context, provider_home) + apply_staged_provider_config(stage.path(), provider_home) } +#[cfg(test)] fn record_provider_resource_reference_aliases( context: &freshell_runtime_protocol::ProviderLaunchContext, provider_home: &Path, @@ -3287,13 +3462,24 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() let projection = stage.join("projection"); let previous = read_projection_manifest(provider_home)?; let current = read_projection_manifest(&projection)?; - let protected: std::collections::BTreeSet<String> = + let mut protected: std::collections::BTreeSet<String> = match read_projection_metadata(stage, "private-mountpoints.json")? { Some(bytes) => { serde_json::from_slice(&bytes).map_err(|_| "private mountpoint metadata invalid")? } None => Default::default(), }; + let reference_aliases: ProviderResourceReferenceAliasPlan = + match read_projection_metadata(stage, "resource-reference-aliases.json")? { + Some(bytes) => serde_json::from_slice(&bytes) + .map_err(|_| "provider reference alias metadata invalid")?, + None => Default::default(), + }; + let reference_alias_paths = provider_resource_reference_alias_paths(&reference_aliases)?; + if provider_reference_alias_overlaps_selected_paths(&reference_aliases, ¤t) { + return Err("provider reference alias overlaps a selected resource file".into()); + } + protected.retain(|relative| !reference_alias_paths.contains(relative)); if protected.iter().any(|relative| { Path::new(relative).as_os_str().is_empty() || Path::new(relative) @@ -3380,6 +3566,20 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() }) { return Err("resource mountpoint metadata has an invalid owned root".into()); } + if reference_aliases + .active + .iter() + .map(|alias| Path::new(&alias.mount_root)) + .chain( + reference_aliases + .retired + .iter() + .map(|alias| Path::new(&alias.mount_root)), + ) + .any(|root| !mounted.iter().any(|mounted_root| mounted_root == root)) + { + return Err("provider reference alias is outside its mounted resource".into()); + } let preserve_existing_mounted_files: Vec<PathBuf> = match read_projection_metadata(stage, "preserved-writable-resource-files.json")? { Some(bytes) => serde_json::from_slice::<Vec<String>>(&bytes) @@ -3407,6 +3607,7 @@ fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<() protected: &protected, mounted: &mounted, preserve_existing_mounted_files: &preserve_existing_mounted_files, + reference_aliases, }) } @@ -3419,6 +3620,7 @@ struct ProviderProjection<'a> { protected: &'a std::collections::BTreeSet<String>, mounted: &'a [PathBuf], preserve_existing_mounted_files: &'a [PathBuf], + reference_aliases: ProviderResourceReferenceAliasPlan, } // One provider-owned transaction covers replacement, stale removal and manifest @@ -3443,10 +3645,38 @@ fn publish_provider_projection_with_manifest( protected, mounted, preserve_existing_mounted_files, + reference_aliases, } = projection; use std::{collections::BTreeMap, fs, os::unix::fs::PermissionsExt}; + let reference_alias_paths = provider_resource_reference_alias_paths(&reference_aliases)?; + if provider_reference_alias_overlaps_selected_paths(&reference_aliases, current) { + return Err("provider reference alias overlaps a selected resource file".into()); + } + preflight_provider_reference_aliases(home, &reference_aliases, current)?; + let active_alias_paths = reference_aliases + .active + .iter() + .map(|alias| alias.provider_path().to_string_lossy().into_owned()) + .collect::<std::collections::BTreeSet<_>>(); + let retired_alias_paths = reference_aliases + .retired + .iter() + .map(|alias| alias.provider_path().to_string_lossy().into_owned()) + .collect::<std::collections::BTreeSet<_>>(); + let mut transactional_alias_paths = reference_alias_paths; + transactional_alias_paths.retain(|relative| { + active_alias_paths.contains(relative) + || (retired_alias_paths.contains(relative) && !current.contains(relative)) + }); + let mut manifest_current = current.clone(); + manifest_current.extend(active_alias_paths.iter().cloned()); + for relative in &retired_alias_paths { + if !current.contains(relative) { + manifest_current.remove(relative); + } + } emit_environment_event( - serde_json::json!({"severity":"info","event":"provider.projection.transaction.started","provider":provider,"stage":"stage","previous_entries":previous.len(),"desired_entries":current.len()}), + serde_json::json!({"severity":"info","event":"provider.projection.transaction.started","provider":provider,"stage":"stage","previous_entries":previous.len(),"desired_entries":manifest_current.len()}), ); let transaction = tempfile::Builder::new() .prefix(".freshell-provider-transaction-") @@ -3476,7 +3706,7 @@ fn publish_provider_projection_with_manifest( ); } // Serialize, write and sync the complete manifest before changing HOME. - let manifest = write_projection_manifest(transaction.path(), current)?; + let manifest = write_projection_manifest(transaction.path(), &manifest_current)?; // Links crossing mounted resource boundaries must still resolve in the // final HOME graph, not relative to the independent alias mount. for relative in current { @@ -3519,7 +3749,7 @@ fn publish_provider_projection_with_manifest( let mut incoming = BTreeMap::new(); let prepared: Result<(), String> = (|| { for relative in previous.union(current) { - if protected.contains(relative) { + if protected.contains(relative) || transactional_alias_paths.contains(relative) { continue; } match fs::symlink_metadata(home.join(relative)) { @@ -3663,6 +3893,14 @@ fn publish_provider_projection_with_manifest( installed.push(relative.clone()); } } + publish_provider_reference_aliases( + home, + current, + &reference_aliases, + &backup, + &mut moved, + &mut installed, + )?; publish_manifest(&manifest, &home.join(PROVIDER_PROJECTION_MANIFEST)) .map_err(|error| format!("provider transaction publish manifest: {error}"))?; Ok(()) @@ -3717,6 +3955,177 @@ fn publish_provider_projection_with_manifest( Ok(()) } +fn preflight_provider_reference_aliases( + home: &Path, + plan: &ProviderResourceReferenceAliasPlan, + current: &std::collections::BTreeSet<String>, +) -> Result<(), String> { + for alias in &plan.active { + let relative = alias.provider_path(); + let reference = home.join(format!(".freshell/references/{}", alias.reference_index)); + for parent in [home.join(".freshell"), home.join(".freshell/references")] { + match std::fs::symlink_metadata(&parent) { + Ok(metadata) if metadata.is_dir() => {} + Ok(_) => return Err("provider reference alias target parent is unsafe".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return Err("provider reference alias target is unavailable".into()) + } + Err(_) => return Err("provider reference alias target is unavailable".into()), + } + } + if !std::fs::symlink_metadata(&reference).is_ok_and(|metadata| metadata.is_file()) { + return Err("provider reference alias target is unavailable".into()); + } + if !provider_reference_alias_parent_available(home, &relative, current)? { + return Err("provider reference alias parent directory is unavailable".into()); + } + let destination = home.join(&relative); + match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + let link = std::fs::read_link(&destination) + .map_err(|_| "provider reference alias could not be inspected")?; + let runtime_reference = PathBuf::from(format!( + "/home/freshell/provider/.freshell/references/{}", + alias.reference_index + )); + if link != runtime_reference + && managed_provider_reference_alias_index(&link, &destination, home).is_none() + { + return Err("provider reference alias conflicts with an existing link".into()); + } + } + Ok(_) => { + return Err(format!( + "provider-created resource blocks provider reference alias {}; retain it and repair the selected path", + relative.display() + )) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("provider reference alias path could not be inspected".into()), + } + } + for alias in &plan.retired { + let relative = alias.provider_path(); + let _ = provider_reference_alias_parent_available(home, &relative, current)?; + } + Ok(()) +} + +fn provider_reference_alias_parent_available( + home: &Path, + relative: &Path, + current: &std::collections::BTreeSet<String>, +) -> Result<bool, String> { + let parent = relative + .parent() + .ok_or("provider reference alias has no parent directory")?; + let mut prefix = PathBuf::new(); + for component in parent.components() { + prefix.push(component.as_os_str()); + let absolute = home.join(&prefix); + match std::fs::symlink_metadata(&absolute) { + Ok(metadata) if metadata.is_dir() => {} + Ok(_) => return Err("provider reference alias parent directory is unsafe".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + let prefix = prefix + .to_str() + .ok_or("provider reference alias path is not UTF-8")?; + return Ok(current.contains(prefix)); + } + Err(_) => return Err("provider reference alias parent directory is unavailable".into()), + } + } + Ok(true) +} + +fn publish_provider_reference_aliases( + home: &Path, + current: &std::collections::BTreeSet<String>, + plan: &ProviderResourceReferenceAliasPlan, + backup: &Path, + moved: &mut Vec<String>, + installed: &mut Vec<String>, +) -> Result<(), String> { + for alias in &plan.active { + let relative = alias.provider_path(); + let relative_string = relative + .to_str() + .ok_or("provider reference alias path is not UTF-8")? + .to_owned(); + if !provider_reference_alias_parent_available(home, &relative, current)? { + return Err("provider reference alias parent directory is unavailable".into()); + } + let destination = home.join(&relative); + let target = PathBuf::from(format!( + "/home/freshell/provider/.freshell/references/{}", + alias.reference_index + )); + match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + let link = std::fs::read_link(&destination) + .map_err(|_| "provider reference alias could not be inspected")?; + if link == target { + continue; + } + if managed_provider_reference_alias_index(&link, &destination, home).is_none() { + return Err("provider reference alias conflicts with an existing link".into()); + } + let saved = backup.join(&relative); + std::fs::create_dir_all(saved.parent().ok_or("backup parent unavailable")?) + .map_err(|error| error.to_string())?; + std::fs::rename(&destination, saved) + .map_err(|error| format!("provider reference alias backup: {error}"))?; + moved.push(relative_string.clone()); + } + Ok(_) => { + return Err(format!( + "provider-created resource blocks provider reference alias {}; retain it and repair the selected path", + relative.display() + )) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("provider reference alias path could not be inspected".into()), + } + std::os::unix::fs::symlink(&target, &destination) + .map_err(|error| format!("provider reference alias could not be created: {error}"))?; + installed.push(relative_string); + } + for alias in &plan.retired { + let relative = alias.provider_path(); + let relative_string = relative + .to_str() + .ok_or("provider reference alias path is not UTF-8")? + .to_owned(); + if current.contains(&relative_string) + || !provider_reference_alias_parent_available(home, &relative, current)? + { + continue; + } + let destination = home.join(&relative); + let link = match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + std::fs::read_link(&destination) + .map_err(|_| "retired provider reference alias could not be inspected")? + } + Ok(_) => continue, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => { + return Err("retired provider reference alias path could not be inspected".into()) + } + }; + if managed_provider_reference_alias_index(&link, &destination, home).is_none() { + continue; + } + let saved = backup.join(&relative); + std::fs::create_dir_all(saved.parent().ok_or("backup parent unavailable")?) + .map_err(|error| error.to_string())?; + std::fs::rename(&destination, saved) + .map_err(|error| format!("retired provider reference alias backup: {error}"))?; + moved.push(relative_string); + } + Ok(()) +} + fn validate_owned_projection_descendants( home: &Path, relative: &Path, @@ -4957,6 +5366,128 @@ mod tests { .contains(".freshell/provider-roots/mcp-project/.env")); } + #[cfg(unix)] + #[test] + fn failed_reference_alias_refresh_preserves_the_previous_provider_projection() { + use freshell_runtime_protocol::{ + ProviderConfigReference, ProviderConfigRoot, ProviderResourceKind, + ProviderResourceReferenceAlias, + }; + + let owned = tempfile::tempdir().unwrap(); + let provider_home = owned.path().join("provider"); + let user_provider_root = owned.path().join("prepared"); + let workspace = owned.path().join("workspace"); + let resource_stage = user_provider_root.join(".freshell/resource-units/mcp-project"); + let resource_root = provider_home.join(".freshell/provider-roots/mcp-project"); + std::fs::create_dir_all(&resource_stage).unwrap(); + std::fs::create_dir_all(&resource_root).unwrap(); + std::fs::create_dir_all(provider_home.join(".opencode")).unwrap(); + std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); + std::fs::create_dir_all(user_provider_root.join("references")).unwrap(); + std::fs::create_dir_all(&workspace).unwrap(); + std::fs::write(user_provider_root.join("settings.json"), b"new config").unwrap(); + std::fs::write(user_provider_root.join("references/0"), b"new secret").unwrap(); + std::fs::write(resource_stage.join("server.mjs"), b"new resource").unwrap(); + std::fs::write( + provider_home.join(".opencode/settings.json"), + b"prior config", + ) + .unwrap(); + std::fs::write( + provider_home.join(".freshell/references/0"), + b"prior secret", + ) + .unwrap(); + std::fs::write(resource_root.join("server.mjs"), b"prior resource").unwrap(); + // A provider-created file blocks the requested managed alias. The + // refresh must fail without publishing its config or resource updates. + std::fs::write(resource_root.join(".env"), b"provider-owned file").unwrap(); + super::write_projection_manifest( + &provider_home, + &[ + ".opencode/settings.json".to_owned(), + ".freshell/references/0".to_owned(), + ".freshell/provider-roots/mcp-project".to_owned(), + ".freshell/provider-roots/mcp-project/server.mjs".to_owned(), + ] + .into_iter() + .collect(), + ) + .unwrap(); + + let mut mount = opencode_resource_mount( + "mcp-project", + "/opt/mcp-project", + "/opt/mcp-project", + ProviderResourceKind::Directory, + false, + ); + mount.reference_aliases = vec![ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 0, + }]; + let mut context = opencode_context_with_mounts(vec![mount]); + context.config = vec![ + ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "settings.json".into(), + provider_relative_path: ".opencode/settings.json".into(), + format: "json".into(), + }, + ProviderConfigReference { + root: ProviderConfigRoot::UserProvider, + relative_path: "references/0".into(), + provider_relative_path: ".freshell/references/0".into(), + format: "text".into(), + }, + ]; + let inputs = context.prepared_inputs.as_mut().unwrap(); + inputs.manifest.source_count = 1; + inputs.manifest.alias_count = 1; + inputs.indexed_files = Some(freshell_runtime_protocol::IndexedProviderFiles { + volume_name: "freshell-inputs-aaaaaaaaaaaaaaaaaaaaaaaa".into(), + aliases: vec![freshell_runtime_protocol::IndexedProviderFileAlias { + source_id: 0, + logical_destination: "/fixture/private-reference".into(), + effective_destination: "/fixture/private-reference".into(), + }], + staged_source_paths: None, + }); + + let stage = tempfile::tempdir_in(&provider_home).unwrap(); + let error = super::refresh_provider_config_in_stage( + &context, + "opencode", + &workspace, + &user_provider_root, + &provider_home, + stage, + ) + .expect_err("a provider-owned alias collision must reject the full refresh"); + + assert!(error.contains("provider-created"), "{error}"); + assert_eq!( + std::fs::read(provider_home.join(".opencode/settings.json")).unwrap(), + b"prior config" + ); + assert_eq!( + std::fs::read(provider_home.join(".freshell/references/0")).unwrap(), + b"prior secret" + ); + assert_eq!( + std::fs::read(resource_root.join("server.mjs")).unwrap(), + b"prior resource" + ); + assert_eq!( + std::fs::read(resource_root.join(".env")).unwrap(), + b"provider-owned file" + ); + assert!(super::read_projection_manifest(&provider_home) + .unwrap() + .contains(".freshell/provider-roots/mcp-project/server.mjs")); + } + #[cfg(unix)] #[test] fn provider_resource_reference_alias_rebinds_a_reused_reference_index() { @@ -5388,6 +5919,7 @@ mod tests { protected: &BTreeSet::new(), mounted: &[], preserve_existing_mounted_files: &[], + reference_aliases: Default::default(), }, |source, _destination| { // This callback occupies only the final publication boundary. The @@ -5431,6 +5963,130 @@ mod tests { .starts_with(".freshell-provider-transaction-"))); } + #[cfg(unix)] + #[test] + fn provider_reference_aliases_roll_back_with_projection_when_manifest_publish_fails() { + use std::{collections::BTreeSet, fs, os::unix::fs::symlink}; + + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let selected = root.path().join("selected"); + let resource_relative = Path::new(".freshell/provider-roots/mcp-project"); + let resource_root = home.join(resource_relative); + fs::create_dir_all(&resource_root).unwrap(); + fs::create_dir_all(home.join(".freshell/references")).unwrap(); + fs::create_dir(&selected).unwrap(); + fs::write(home.join(".freshell/references/0"), "new reference").unwrap(); + fs::write(home.join(".freshell/references/1"), "prior reference").unwrap(); + fs::write(resource_root.join("server.mjs"), "prior resource").unwrap(); + symlink( + "/home/freshell/provider/.freshell/references/1", + resource_root.join(".env"), + ) + .unwrap(); + symlink( + "/home/freshell/provider/.freshell/references/1", + resource_root.join(".oldenv"), + ) + .unwrap(); + fs::write(selected.join("server.mjs"), "new resource").unwrap(); + + let alias = resource_relative + .join(".env") + .to_string_lossy() + .into_owned(); + let retired = resource_relative + .join(".oldenv") + .to_string_lossy() + .into_owned(); + let previous = BTreeSet::from([ + ".freshell".into(), + ".freshell/provider-roots".into(), + resource_relative.to_string_lossy().into_owned(), + resource_relative + .join("server.mjs") + .to_string_lossy() + .into_owned(), + alias.clone(), + retired.clone(), + ]); + let current = BTreeSet::from([ + ".freshell".into(), + ".freshell/provider-roots".into(), + resource_relative.to_string_lossy().into_owned(), + resource_relative + .join("server.mjs") + .to_string_lossy() + .into_owned(), + ]); + write_projection_manifest(&home, &previous).unwrap(); + let prior_manifest = fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(); + let selections = [freshell_platform::provider_environment::CopySelection { + source: selected, + destination_relative: resource_relative.to_path_buf(), + }]; + let reference_aliases = ProviderResourceReferenceAliasPlan { + active: vec![ProviderResourceReferenceAliasInstall { + mount_root: resource_relative.to_string_lossy().into_owned(), + relative_path: ".env".into(), + reference_index: 0, + }], + retired: vec![ProviderResourceReferenceAliasRetirement { + mount_root: resource_relative.to_string_lossy().into_owned(), + relative_path: ".oldenv".into(), + }], + }; + + let result = publish_provider_projection_with_manifest( + ProviderProjection { + provider: "opencode", + selections: &selections, + home: &home, + previous: &previous, + current: ¤t, + protected: &BTreeSet::new(), + mounted: &[], + preserve_existing_mounted_files: &[], + reference_aliases, + }, + |_source, _destination| { + assert_eq!( + fs::read(resource_root.join("server.mjs")).unwrap(), + b"new resource" + ); + assert_eq!( + fs::read_link(resource_root.join(".env")).unwrap(), + Path::new("/home/freshell/provider/.freshell/references/0") + ); + assert!(!resource_root.join(".oldenv").exists()); + Err(std::io::Error::other("injected manifest publish failure")) + }, + ); + + assert!(result.unwrap_err().contains("publish manifest")); + assert_eq!( + fs::read(resource_root.join("server.mjs")).unwrap(), + b"prior resource" + ); + assert_eq!( + fs::read_link(resource_root.join(".env")).unwrap(), + Path::new("/home/freshell/provider/.freshell/references/1") + ); + assert_eq!( + fs::read_link(resource_root.join(".oldenv")).unwrap(), + Path::new("/home/freshell/provider/.freshell/references/1") + ); + assert_eq!( + fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(), + prior_manifest + ); + assert!(!fs::read_dir(&home).unwrap().any(|entry| entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with(".freshell-provider-transaction-"))); + } + #[test] fn provider_native_auth_failed_replacement_retains_prior_usable_destination() { use std::{fs, os::unix::fs::PermissionsExt}; From 0e36e8764f03f779319e749df7bf5aff24fc56ea Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 13:29:35 -0700 Subject: [PATCH 465/493] fix: retain older replayed checkpoint outcomes --- src/components/fresh-agent/FreshAgentView.tsx | 3 ++- src/store/paneTypes.ts | 5 ++-- .../fresh-agent/FreshAgentView.test.tsx | 26 +++++++++++++++++++ .../panesSlice.checkpoint-outcomes.test.ts | 18 ++++++------- 4 files changed, 40 insertions(+), 12 deletions(-) diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 4776900e8..7f5b61942 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -11,6 +11,7 @@ import { import { nanoid } from 'nanoid' import { freshAgentCheckpointConversationKeyForContent, + FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT, normalizeFreshAgentCheckpointWorkspace, type FreshAgentCheckpointOutcome, type FreshAgentPaneContent, @@ -1151,7 +1152,7 @@ export function FreshAgentView({ const checkpointOutcomes = [ ...priorOutcomes.filter((item) => item.requestId !== requestId), outcome, - ].slice(-200) + ].slice(-FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) dispatch(mergePaneContent({ tabId, paneId, updates: { checkpointOutcomes } })) }, [appStore, dispatch, paneId, tabId]) const getCheckpointAvailability = useCallback((turn: FreshAgentTurn): CheckpointTurnAvailability | undefined => { diff --git a/src/store/paneTypes.ts b/src/store/paneTypes.ts index 604b674c2..04f928875 100644 --- a/src/store/paneTypes.ts +++ b/src/store/paneTypes.ts @@ -84,7 +84,8 @@ export type FreshAgentCheckpointOutcome = { reason?: string } -const FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT = 200 +/** Keep the same maximum history window as the durable server outcome store. */ +export const FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT = 8192 function checkpointOutcomeString(value: unknown, maxLength: number): string | undefined { if (typeof value !== 'string' || value.length === 0 || value.length > maxLength) return undefined @@ -147,7 +148,7 @@ export function normalizeFreshAgentCheckpointWorkspace(value: unknown): FreshAge } /** - * Persist only a bounded record of settled checkpoint availability. The + * Retain a bounded record of settled checkpoint availability. The * whitelist deliberately excludes the submitted prompt and provider payloads. */ export function normalizeFreshAgentCheckpointOutcomes( diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index 6651104df..89bcbc1df 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3745,6 +3745,32 @@ describe('FreshAgentView', () => { }), ])) + // The server may replay many durable outcomes after attach. A local + // retention limit must not discard an older exact checkpoint from that + // replay before the pane is reloaded again. + act(() => { + for (let index = 0; index < 200; index += 1) { + onMessage?.({ + type: 'freshAgent.checkpoint.outcome', + requestId: `request-replayed-extra-${index}`, + logicalAttemptId: `attempt-replayed-extra-${index}`, + sessionId: 'thread-replayed-conversation', + sessionType: 'freshcodex', + provider: 'codex', + status: 'available', + checkpointId: `checkpoint-extra-${index}`, + nativeTurnId: `codex-replayed-extra-${index}`, + displayTurnId: `codex-replayed-extra-${index}:row-0`, + }) + } + }) + const retainedOutcomes = getFreshAgentPaneContent(store).checkpointOutcomes ?? [] + expect(retainedOutcomes).toHaveLength(201) + expect(retainedOutcomes[0]).toMatchObject({ + requestId: 'request-replayed-turn', + checkpointId, + }) + view.unmount() render( <Provider store={store}> diff --git a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts index bf626bedb..f895409a1 100644 --- a/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts +++ b/test/unit/client/store/panesSlice.checkpoint-outcomes.test.ts @@ -88,24 +88,24 @@ describe('FreshAgent checkpoint outcomes', () => { expect(JSON.stringify(leaf.content)).not.toContain('must never be retained') }) - it('keeps only the newest 200 settled outcomes and deduplicates by request id', () => { - const outcomes = Array.from({ length: 202 }, (_, index) => ({ + it('retains the latest server replay window and deduplicates by request id', () => { + const outcomes = Array.from({ length: 8194 }, (_, index) => ({ requestId: `send-${index}`, displayTurnId: `turn-${index}`, status: 'unavailable', code: 'capture_failed', reason: `failure ${index}`, })) - outcomes[201] = { - requestId: 'send-201', - displayTurnId: 'turn-201', + outcomes[8193] = { + requestId: 'send-8193', + displayTurnId: 'turn-8193', status: 'available', checkpointId: 'checkpoint-latest', workspace: { path: '/work/project', provenance: 'resolved-project' }, } as typeof outcomes[number] outcomes.push({ - requestId: 'send-201', - displayTurnId: 'turn-201', + requestId: 'send-8193', + displayTurnId: 'turn-8193', status: 'unavailable', code: 'late_failure', reason: 'later result for the same request', @@ -126,10 +126,10 @@ describe('FreshAgent checkpoint outcomes', () => { const leaf = state.layouts['tab-checkpoints'] as Extract<PaneNode, { type: 'leaf' }> const stored = (leaf.content as { checkpointOutcomes?: Array<Record<string, unknown>> }).checkpointOutcomes! - expect(stored).toHaveLength(200) + expect(stored).toHaveLength(8192) expect(stored[0].requestId).toBe('send-2') expect(stored.at(-1)).toMatchObject({ - requestId: 'send-201', + requestId: 'send-8193', status: 'unavailable', code: 'late_failure', }) From 41adfc5e86e07355f91f4e0d429abacde59219d9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 13:52:23 -0700 Subject: [PATCH 466/493] fix: paginate checkpoint recovery and isolate index rebuilds --- crates/freshell-server/src/checkpoints.rs | 494 ++++++++++++++++-- .../freshell-server/src/fresh_agent_proxy.rs | 9 - .../src/retained_output.rs | 101 +++- crates/freshell-ws/src/checkpoint_barrier.rs | 12 +- crates/freshell-ws/src/terminal.rs | 57 +- src/components/fresh-agent/FreshAgentView.tsx | 331 ++++++++---- src/lib/api.ts | 42 +- .../fresh-agent/FreshAgentView.test.tsx | 95 +++- test/unit/client/lib/api.test.ts | 29 + 9 files changed, 944 insertions(+), 226 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index a8f36eaec..3c536cb92 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -39,6 +39,7 @@ use axum::{ routing::post, Json, Router, }; +use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine as _}; use serde::{Deserialize, Serialize}; use serde_json::{json, Map, Value}; use sha1::{Digest, Sha1}; @@ -97,6 +98,8 @@ const CHECKPOINT_TIMEOUT_COOLDOWN: Duration = Duration::from_secs(60); const CHECKPOINT_COOLDOWN_LIMIT: usize = 128; const CHECKPOINT_OUTCOME_STORE_VERSION: u8 = 1; const CHECKPOINT_OUTCOME_STORE_LIMIT: usize = 8192; +const CHECKPOINT_OUTCOME_REPLAY_LIMIT: usize = 8192; +const CHECKPOINT_OUTCOME_REPLAY_PAGE_LIMIT: usize = 200; const CHECKPOINT_DISPATCH_FENCE_LIMIT: usize = CHECKPOINT_OUTCOME_STORE_LIMIT; const CHECKPOINT_VOLATILE_FENCE_LIMIT: usize = 1024; @@ -170,6 +173,35 @@ struct StoredCheckpointOutcome { created_at_ms: u64, } +#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +struct CheckpointOutcomeReplayPosition { + created_at_ms: u64, + key: String, + request_id: String, +} + +#[derive(Clone, Debug, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +struct CheckpointOutcomeReplayCursor { + provider: AgentProvider, + session_id: String, + session_type: SessionType, + window_start: CheckpointOutcomeReplayPosition, + snapshot_end: CheckpointOutcomeReplayPosition, + after: CheckpointOutcomeReplayPosition, +} + +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +struct CheckpointOutcomeReplayQuery { + provider: String, + session_id: String, + session_type: String, + cursor: Option<String>, + limit: Option<usize>, +} + fn capture_settled_by_default() -> bool { true } @@ -1312,12 +1344,14 @@ impl CheckpointService { Ok(store.outcomes.get(key).cloned()) } - async fn replay_for_identity( + async fn replay_outcome_page( &self, provider: AgentProvider, session_id: &str, session_type: SessionType, - ) { + cursor: Option<&str>, + limit: usize, + ) -> Result<(Vec<FreshAgentCheckpointOutcome>, Option<String>), String> { let records = { let _guard = self.outcome_gate.lock().await; match self.read_outcome_store().await { @@ -1326,7 +1360,7 @@ impl CheckpointService { tracing::error!( event = "code_checkpoint.outcome_store_read_failed", cause = %error, - "durable checkpoint outcomes could not be replayed" + "durable checkpoint outcomes could not be read for recovery" ); let mut cached = self .durable_outcome_cache @@ -1339,11 +1373,28 @@ impl CheckpointService { .expect("volatile checkpoint outcomes mutex poisoned") .clone(), ); + if cached.is_empty() { + return Err(error); + } cached } } }; - for record in records.values().filter(|record| { + let cursor = cursor + .map(|value| { + if value.len() > 4096 { + return Err("checkpoint outcome cursor exceeds bound".to_string()); + } + let bytes = URL_SAFE_NO_PAD + .decode(value) + .map_err(|_| "invalid checkpoint outcome cursor".to_string())?; + serde_json::from_slice::<CheckpointOutcomeReplayCursor>(&bytes) + .map_err(|_| "invalid checkpoint outcome cursor".to_string()) + }) + .transpose()?; + + let mut entries = Vec::new(); + for (key, record) in records.iter().filter(|(_, record)| { record.outcome.provider == provider && record.outcome.session_type == session_type && record @@ -1352,15 +1403,92 @@ impl CheckpointService { .any(|alias| alias == session_id) }) { if record.request_ids.is_empty() { - self.broadcast_outcome(record.outcome.clone()); + entries.push(( + CheckpointOutcomeReplayPosition { + created_at_ms: record.created_at_ms, + key: key.clone(), + request_id: record.outcome.request_id.clone(), + }, + &record.outcome, + )); } else { for request_id in &record.request_ids { - let mut outcome = record.outcome.clone(); - outcome.request_id = request_id.clone(); - self.broadcast_outcome(outcome); + entries.push(( + CheckpointOutcomeReplayPosition { + created_at_ms: record.created_at_ms, + key: key.clone(), + request_id: request_id.clone(), + }, + &record.outcome, + )); } } } + entries.sort_by(|left, right| left.0.cmp(&right.0)); + + let (window_start, snapshot_end) = if let Some(cursor) = &cursor { + if cursor.provider != provider + || cursor.session_id != session_id + || cursor.session_type != session_type + || cursor.after < cursor.window_start + || cursor.after > cursor.snapshot_end + { + return Err("invalid checkpoint outcome cursor".to_string()); + } + (cursor.window_start.clone(), cursor.snapshot_end.clone()) + } else { + if entries.len() > CHECKPOINT_OUTCOME_REPLAY_LIMIT { + entries.drain(..entries.len() - CHECKPOINT_OUTCOME_REPLAY_LIMIT); + } + let Some((first, _)) = entries.first() else { + return Ok((Vec::new(), None)); + }; + let Some((last, _)) = entries.last() else { + return Ok((Vec::new(), None)); + }; + (first.clone(), last.clone()) + }; + + let entries = entries + .into_iter() + .filter(|(position, _)| { + position >= &window_start + && position <= &snapshot_end + && cursor + .as_ref() + .is_none_or(|cursor| position > &cursor.after) + }) + .collect::<Vec<_>>(); + let page_limit = limit.clamp(1, CHECKPOINT_OUTCOME_REPLAY_PAGE_LIMIT); + let has_more = entries.len() > page_limit; + let mut outcomes = Vec::with_capacity(page_limit.min(entries.len())); + let mut last_position = None; + for (position, stored) in entries.into_iter().take(page_limit) { + let mut outcome = stored.clone(); + outcome.request_id = position.request_id.clone(); + // Materialized sessions retain the provisional id as an alias in + // the store. The pane needs the id it attached with on this page. + outcome.session_id = session_id.to_string(); + last_position = Some(position); + outcomes.push(outcome); + } + let next_cursor = if has_more { + let after = last_position.ok_or("checkpoint outcome page lost its cursor")?; + let cursor = CheckpointOutcomeReplayCursor { + provider, + session_id: session_id.to_string(), + session_type, + window_start, + snapshot_end, + after, + }; + let encoded = serde_json::to_vec(&cursor) + .map_err(|_| "checkpoint outcome cursor encoding failed")?; + Some(URL_SAFE_NO_PAD.encode(encoded)) + } else { + None + }; + Ok((outcomes, next_cursor)) } fn broadcast_outcome(&self, outcome: FreshAgentCheckpointOutcome) { @@ -2788,18 +2916,6 @@ impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for Checkpoint } }) } - - fn replay_for_session( - self: Arc<Self>, - provider: AgentProvider, - session_id: String, - session_type: SessionType, - ) -> futures_util::future::BoxFuture<'static, ()> { - Box::pin(async move { - self.replay_for_identity(provider, &session_id, session_type) - .await; - }) - } } #[derive(Clone, Debug, PartialEq, Eq)] @@ -3673,6 +3789,10 @@ pub fn router(state: CheckpointsApiState) -> Router { "/api/fresh-agent/checkpoints", post(post_checkpoint).get(get_method(get_checkpoints)), ) + .route( + "/api/fresh-agent/checkpoints/outcomes", + get_method(get_checkpoint_outcomes), + ) .route( "/api/fresh-agent/checkpoints/restore", post(post_checkpoint_restore), @@ -5380,6 +5500,84 @@ async fn post_checkpoint_metadata( } } +async fn get_checkpoint_outcomes( + State(state): State<CheckpointsApiState>, + headers: HeaderMap, + Query(query): Query<CheckpointOutcomeReplayQuery>, +) -> Response { + if !is_authed(&headers, &state.auth_token) { + return unauthorized(); + } + let Some(provider) = parse_agent_provider(&query.provider) else { + return bad_request("provider is invalid"); + }; + let Some(session_type) = parse_session_type(&query.session_type) else { + return bad_request("sessionType is invalid"); + }; + if query.session_id.is_empty() || query.session_id.len() > 1024 { + return bad_request("sessionId is invalid"); + } + + match state + .service + .replay_outcome_page( + provider, + &query.session_id, + session_type, + query.cursor.as_deref(), + query.limit.unwrap_or(CHECKPOINT_OUTCOME_REPLAY_PAGE_LIMIT), + ) + .await + { + Ok((outcomes, next_cursor)) => { + let outcomes = outcomes + .into_iter() + .map(|outcome| { + serde_json::to_value( + freshell_protocol::ServerMessage::FreshAgentCheckpointOutcome(outcome), + ) + }) + .collect::<Result<Vec<_>, _>>(); + let outcomes = match outcomes { + Ok(outcomes) => outcomes, + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_replay_serialize_failed", + cause = %error, + "checkpoint outcomes could not be serialized for session recovery" + ); + return ( + StatusCode::INTERNAL_SERVER_ERROR, + Json(json!({ "error": "checkpoint outcomes could not be loaded" })), + ) + .into_response(); + } + }; + let mut body = json!({ "outcomes": outcomes }); + if let Some(next_cursor) = next_cursor { + body["nextCursor"] = json!(next_cursor); + } + Json(body).into_response() + } + Err(error) if error.contains("cursor") => bad_request(&error), + Err(error) => { + tracing::error!( + event = "code_checkpoint.outcome_replay_read_failed", + provider = %query.provider, + session_id = %query.session_id, + session_type = %query.session_type, + cause = %error, + "checkpoint outcomes could not be loaded for session recovery" + ); + ( + StatusCode::INTERNAL_SERVER_ERROR, + Json(json!({ "error": "checkpoint outcomes could not be loaded" })), + ) + .into_response() + } + } +} + /// `POST /checkpoints/restore` (`fresh-agent-extras-router.ts:410-428`). async fn post_checkpoint_restore( State(state): State<CheckpointsApiState>, @@ -6121,7 +6319,7 @@ mod tests { } #[tokio::test] - async fn cached_durable_outcome_replays_when_store_becomes_unreadable() { + async fn cached_durable_outcome_page_survives_when_store_becomes_unreadable() { use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; use std::sync::atomic::AtomicUsize; @@ -6167,17 +6365,25 @@ mod tests { tokio::fs::remove_file(&store_path).await.unwrap(); tokio::fs::create_dir(&store_path).await.unwrap(); - service - .replay_for_identity( + let (outcomes, next_cursor) = service + .replay_outcome_page( AgentProvider::Opencode, "session-cache-replay", SessionType::Freshopencode, + None, + 200, ) - .await; + .await + .unwrap(); - let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); - assert_eq!(outcome["requestId"], "request-cache-replay"); - assert_eq!(outcome["logicalAttemptId"], "attempt-cache-replay"); + let outcome = outcomes.first().unwrap(); + assert_eq!(outcome.request_id, "request-cache-replay"); + assert_eq!(outcome.logical_attempt_id, "attempt-cache-replay"); + assert!(next_cursor.is_none()); + assert!( + outcome_rx.try_recv().is_ok(), + "the live outcome stays on the broadcast bus" + ); assert_eq!(dispatches.load(Ordering::SeqCst), 1); } @@ -6758,9 +6964,7 @@ mod tests { } #[tokio::test] - async fn codex_identity_receipt_is_durable_and_replayed_after_service_reload() { - use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; - + async fn codex_identity_receipt_is_durable_and_loaded_after_service_reload() { let home = tempfile::tempdir().unwrap(); let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); let service = @@ -6830,25 +7034,233 @@ mod tests { "durable checkpoint identity state never stores the sent prompt" ); - let (replay_tx, mut replay_rx) = tokio::sync::broadcast::channel(8); + let (replay_tx, _) = tokio::sync::broadcast::channel(8); let reloaded = Arc::new(CheckpointService::new( Arc::new(home.path().to_path_buf()), Arc::new(replay_tx), )); - reloaded - .clone() - .replay_for_session( + let (outcomes, next_cursor) = reloaded + .replay_outcome_page( AgentProvider::Codex, - "codex-session".to_string(), + "codex-session", SessionType::Freshcodex, + None, + 200, ) + .await + .unwrap(); + let replayed = outcomes.first().unwrap(); + assert!(next_cursor.is_none()); + assert_eq!(replayed.logical_attempt_id, "codex-logical-attempt-1"); + assert_eq!( + replayed.native_turn_id.as_deref(), + Some("codex-native-turn-1") + ); + assert_eq!( + replayed.display_turn_id.as_deref(), + Some("codex-native-turn-1:row-0") + ); + } + + #[tokio::test] + async fn replayed_checkpoint_uses_the_requested_materialized_session_alias() { + use tower::ServiceExt; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + let workspace = ResolvedCheckpointWorkspace { + path: PathBuf::from("/work/codex-project"), + provenance: "directory-picker".to_string(), + }; + let send = FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "provisional-codex-session".to_string(), + session_type: SessionType::Freshcodex, + text: "turn text is not stored".to_string(), + cwd: Some("/work/codex-project".to_string()), + images: None, + request_id: Some("codex-materialized-request".to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "codex-materialized-attempt".to_string(), + workspace: None, + }), + }; + let key = checkpoint_attempt_key(&send, "codex-materialized-attempt"); + let mut record = service.new_record( + &send, + "codex-materialized-attempt", + "codex-materialized-request", + Some((&workspace, "")), + Ok("checkpoint-materialized".to_string()), + ); + record.dispatch_started = true; + record.outcome.native_turn_id = Some("codex-native-materialized-turn".to_string()); + record.outcome.display_turn_id = Some("codex-native-materialized-turn:row-0".to_string()); + service.persist_outcome(&key, record).await.unwrap(); + + service + .observe_materialized(&json!({ + "provider": "codex", + "sessionType": "freshcodex", + "previousSessionId": "provisional-codex-session", + "sessionId": "durable-codex-session" + })) .await; - let frame = replay_rx.recv().await.unwrap(); - let replayed: Value = serde_json::from_str(&frame).unwrap(); - assert_eq!(replayed["type"], "freshAgent.checkpoint.outcome"); - assert_eq!(replayed["logicalAttemptId"], "codex-logical-attempt-1"); - assert_eq!(replayed["nativeTurnId"], "codex-native-turn-1"); - assert_eq!(replayed["displayTurnId"], "codex-native-turn-1:row-0"); + + let (replay_tx, mut replay_rx) = tokio::sync::broadcast::channel(8); + let reloaded = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(replay_tx), + )); + let (outcomes, next_cursor) = reloaded + .replay_outcome_page( + AgentProvider::Codex, + "durable-codex-session", + SessionType::Freshcodex, + None, + 200, + ) + .await + .unwrap(); + + let replayed = outcomes.first().unwrap(); + assert!(next_cursor.is_none()); + assert_eq!(replayed.session_id, "durable-codex-session"); + assert_eq!( + replayed.native_turn_id.as_deref(), + Some("codex-native-materialized-turn") + ); + assert_eq!( + replayed.checkpoint_id.as_deref(), + Some("checkpoint-materialized") + ); + assert!(matches!( + replay_rx.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + )); + + let api_state = CheckpointsApiState { + auth_token: Arc::new("tok".to_string()), + home: Arc::new(home.path().to_path_buf()), + service: Arc::clone(&reloaded), + }; + let app = router(api_state); + let unauthorized_request = axum::http::Request::builder() + .uri("/api/fresh-agent/checkpoints/outcomes?provider=codex&sessionId=durable-codex-session&sessionType=freshcodex&limit=200") + .body(axum::body::Body::empty()) + .unwrap(); + let unauthorized_response = app.clone().oneshot(unauthorized_request).await.unwrap(); + assert_eq!(unauthorized_response.status(), StatusCode::UNAUTHORIZED); + let request = axum::http::Request::builder() + .uri("/api/fresh-agent/checkpoints/outcomes?provider=codex&sessionId=durable-codex-session&sessionType=freshcodex&limit=200") + .header("x-auth-token", "tok") + .body(axum::body::Body::empty()) + .unwrap(); + let response = app.oneshot(request).await.unwrap(); + assert_eq!(response.status(), StatusCode::OK); + let response = body_json(response).await; + assert_eq!( + response["outcomes"][0]["sessionId"], + "durable-codex-session" + ); + assert_eq!( + response["outcomes"][0]["type"], + "freshAgent.checkpoint.outcome" + ); + } + + #[tokio::test] + async fn checkpoint_outcome_recovery_paginates_large_histories_without_broadcasting() { + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, mut broadcast_rx) = tokio::sync::broadcast::channel(1024); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + let mut outcomes = BTreeMap::new(); + for index in 0..1100 { + let request_id = format!("replay-request-{index:04}"); + let record = StoredCheckpointOutcome { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcome: FreshAgentCheckpointOutcome { + logical_attempt_id: format!("replay-attempt-{index:04}"), + request_id: request_id.clone(), + provider: AgentProvider::Codex, + session_id: "replay-session".to_string(), + session_type: SessionType::Freshcodex, + status: FreshAgentCheckpointStatus::Available, + checkpoint_id: Some(format!("replay-checkpoint-{index:04}")), + workspace: None, + code: None, + reason: None, + native_turn_id: Some(format!("native-turn-{index:04}")), + display_turn_id: Some(format!("native-turn-{index:04}:row-0")), + }, + request_ids: vec![request_id], + session_aliases: vec!["replay-session".to_string()], + dispatch_started: true, + capture_settled: true, + submitted_turn_id: None, + created_at_ms: index, + }; + outcomes.insert(format!("replay-key-{index:04}"), record); + } + service + .write_outcome_store(&CheckpointOutcomeStore { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcomes, + }) + .await + .unwrap(); + + let mut recovered = Vec::new(); + let mut cursor = None; + loop { + let (page, next_cursor) = service + .replay_outcome_page( + AgentProvider::Codex, + "replay-session", + SessionType::Freshcodex, + cursor.as_deref(), + 200, + ) + .await + .unwrap(); + assert!(page.len() <= 200); + recovered.extend(page.into_iter().map(|outcome| outcome.request_id)); + let Some(next_cursor) = next_cursor else { + break; + }; + if recovered.len() == 200 { + let mismatched_page = service + .replay_outcome_page( + AgentProvider::Codex, + "another-session", + SessionType::Freshcodex, + Some(&next_cursor), + 200, + ) + .await; + assert!(mismatched_page.is_err()); + } + cursor = Some(next_cursor); + } + assert_eq!(recovered.len(), 1100); + assert_eq!( + recovered.first().map(String::as_str), + Some("replay-request-0000") + ); + assert_eq!( + recovered.last().map(String::as_str), + Some("replay-request-1099") + ); + assert!(matches!( + broadcast_rx.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + )); } #[tokio::test] diff --git a/crates/freshell-server/src/fresh_agent_proxy.rs b/crates/freshell-server/src/fresh_agent_proxy.rs index 1d0b5fd98..7aff20185 100644 --- a/crates/freshell-server/src/fresh_agent_proxy.rs +++ b/crates/freshell-server/src/fresh_agent_proxy.rs @@ -210,15 +210,6 @@ impl HostedFreshAgentProxy { } } HostedFreshAgentCommand::Attach(message) => { - if let Some(barrier) = self.checkpoint_barrier.get() { - Arc::clone(barrier) - .replay_for_session( - message.provider, - message.session_id.clone(), - message.session_type, - ) - .await; - } if let Some(soul) = self .resolve_soul(&message.provider, message.session_type, &message.session_id) .await diff --git a/crates/freshell-supervisor/src/retained_output.rs b/crates/freshell-supervisor/src/retained_output.rs index 2c1692892..ea05249c9 100644 --- a/crates/freshell-supervisor/src/retained_output.rs +++ b/crates/freshell-supervisor/src/retained_output.rs @@ -13,9 +13,12 @@ use freshell_runtime_protocol::{ use std::{ fs::{File, OpenOptions}, io::{BufReader, Read, Seek, SeekFrom, Write}, - path::Path, + path::{Path, PathBuf}, + sync::atomic::{AtomicU64, Ordering}, }; +static INDEX_TEMP_SEQUENCE: AtomicU64 = AtomicU64::new(0); + #[cfg(test)] std::thread_local! { static FORCED_INDEX_REBUILDS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) }; @@ -388,17 +391,19 @@ fn write_index_atomic( scanned_through_offset, torn_tail, )?; - let temporary = path.with_extension("tmp"); - let mut file = OpenOptions::new() - .create(true) - .truncate(true) - .write(true) - .open(&temporary) - .map_err(|_| "retained output index unavailable")?; + let (temporary, mut file) = create_index_temporary(path)?; + let _temporary_cleanup = RemoveIndexTemporary(temporary.clone()); file.write_all(&bytes) .and_then(|_| file.sync_all()) .map_err(|_| "retained output index write failed")?; - std::fs::rename(&temporary, path).map_err(|_| "retained output index commit failed")?; + if std::fs::rename(&temporary, path).is_err() { + // Windows may reject replacing an index another reader just committed. + // A complete, valid index for this spool is already enough to proceed; + // never report a read failure merely because the concurrent writer won. + if !index_is_valid_for_spool(path, descriptor, spool_bytes) { + return Err("retained output index commit failed".into()); + } + } File::open( path.parent() .ok_or("retained output directory unavailable")?, @@ -408,6 +413,49 @@ fn write_index_atomic( Ok(()) } +fn create_index_temporary(path: &Path) -> Result<(PathBuf, File), String> { + let parent = path + .parent() + .ok_or("retained output directory unavailable")?; + let name = path + .file_name() + .ok_or("retained output index unavailable")? + .to_string_lossy(); + for _ in 0..128 { + let sequence = INDEX_TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed); + let temporary = parent.join(format!(".{name}.tmp-{}-{sequence}", std::process::id())); + match OpenOptions::new() + .create_new(true) + .write(true) + .open(&temporary) + { + Ok(file) => return Ok((temporary, file)), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue, + Err(_) => return Err("retained output index unavailable".into()), + } + } + Err("retained output index temporary namespace exhausted".into()) +} + +struct RemoveIndexTemporary(PathBuf); + +impl Drop for RemoveIndexTemporary { + fn drop(&mut self) { + let _ = std::fs::remove_file(&self.0); + } +} + +fn index_is_valid_for_spool( + path: &Path, + descriptor: &TerminalSpoolDescriptor, + spool_bytes: u64, +) -> bool { + File::open(path) + .ok() + .and_then(|file| read_spool_index(BufReader::new(file), descriptor, spool_bytes).ok()) + .is_some() +} + #[cfg(test)] mod tests { use super::*; @@ -416,6 +464,41 @@ mod tests { IncarnationId, TerminalLaunchSpec, }; + #[test] + fn concurrent_cold_index_rebuilds_do_not_collide_on_temporary_files() { + let directory = tempfile::tempdir().unwrap(); + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id: IncarnationId::new(), + terminal_id: "concurrent-index-terminal".into(), + stream_epoch: "launch-prefix-hostboot-concurrent".into(), + segment_limit_bytes: 32 * 1024 * 1024, + }; + let path = directory.path().join(SPOOL_CURRENT_INDEX_FILE); + let ready = std::sync::Arc::new(std::sync::Barrier::new(16)); + let workers = (0..16) + .map(|_| { + let path = path.clone(); + let descriptor = descriptor.clone(); + let ready = std::sync::Arc::clone(&ready); + std::thread::spawn(move || { + ready.wait(); + write_index_atomic(&path, &descriptor, &[], 0, 0, false) + }) + }) + .collect::<Vec<_>>(); + + for worker in workers { + worker.join().unwrap().unwrap(); + } + assert!(index_is_valid_for_spool(&path, &descriptor, 0)); + assert_eq!( + std::fs::read_dir(directory.path()).unwrap().count(), + 1, + "all unique temporary files are removed after the atomic commits" + ); + } + #[test] fn legacy_supervisor_fallback_builds_and_reuses_sparse_spool_index() { let directory = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-ws/src/checkpoint_barrier.rs b/crates/freshell-ws/src/checkpoint_barrier.rs index aaa8b9976..d4c21055b 100644 --- a/crates/freshell-ws/src/checkpoint_barrier.rs +++ b/crates/freshell-ws/src/checkpoint_barrier.rs @@ -7,7 +7,7 @@ use std::sync::Arc; -use freshell_protocol::{AgentProvider, FreshAgentSend, SessionType}; +use freshell_protocol::FreshAgentSend; use futures::future::BoxFuture; pub trait FreshAgentCheckpointBarrier: Send + Sync { @@ -16,16 +16,6 @@ pub trait FreshAgentCheckpointBarrier: Send + Sync { send: FreshAgentSend, forward: BoxFuture<'static, ()>, ) -> BoxFuture<'static, ()>; - - /// Replays durable per-turn results when a pane reattaches after missing - /// the pre-forward outcome broadcast. Implementations filter by provider - /// session identity and never infer an outcome from transcript text. - fn replay_for_session( - self: Arc<Self>, - provider: AgentProvider, - session_id: String, - session_type: SessionType, - ) -> BoxFuture<'static, ()>; } #[derive(Clone, Default)] diff --git a/crates/freshell-ws/src/terminal.rs b/crates/freshell-ws/src/terminal.rs index ede548503..fe247e4dc 100644 --- a/crates/freshell-ws/src/terminal.rs +++ b/crates/freshell-ws/src/terminal.rs @@ -1963,62 +1963,26 @@ async fn handle_client_text( // same pattern as the other `freshAgent.*` arms. `_` keeps swallowing only // `Amplifier` (no fresh-agent runtime, same as the `FreshAgentSend` arm). ClientMessage::FreshAgentAttach(attach) => { - let checkpoint_replay = state.checkpoint_send.service.clone(); match attach.provider { freshell_protocol::AgentProvider::Codex => { let fresh_codex = state.fresh_codex.clone(); tokio::spawn( - async move { - if let Some(service) = checkpoint_replay { - Arc::clone(&service) - .replay_for_session( - attach.provider, - attach.session_id.clone(), - attach.session_type, - ) - .await; - } - fresh_codex.handle_attach(attach).await - } - .instrument(tracing::Span::current()), + async move { fresh_codex.handle_attach(attach).await } + .instrument(tracing::Span::current()), ); } freshell_protocol::AgentProvider::Claude => { let fresh_claude = state.fresh_claude.clone(); - let checkpoint_replay = state.checkpoint_send.service.clone(); tokio::spawn( - async move { - if let Some(service) = checkpoint_replay { - Arc::clone(&service) - .replay_for_session( - attach.provider, - attach.session_id.clone(), - attach.session_type, - ) - .await; - } - fresh_claude.handle_attach(attach).await - } - .instrument(tracing::Span::current()), + async move { fresh_claude.handle_attach(attach).await } + .instrument(tracing::Span::current()), ); } freshell_protocol::AgentProvider::Opencode => { let fresh_opencode = state.fresh_opencode.clone(); - let checkpoint_replay = state.checkpoint_send.service.clone(); tokio::spawn( - async move { - if let Some(service) = checkpoint_replay { - Arc::clone(&service) - .replay_for_session( - attach.provider, - attach.session_id.clone(), - attach.session_type, - ) - .await; - } - fresh_opencode.handle_attach(attach).await - } - .instrument(tracing::Span::current()), + async move { fresh_opencode.handle_attach(attach).await } + .instrument(tracing::Span::current()), ); } _ => {} @@ -13052,15 +13016,6 @@ mod pane_reconcile_gate_tests { forward.await; }) } - - fn replay_for_session( - self: Arc<Self>, - _provider: freshell_protocol::AgentProvider, - _session_id: String, - _session_type: freshell_protocol::SessionType, - ) -> futures::future::BoxFuture<'static, ()> { - Box::pin(async {}) - } } #[tokio::test] diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 7f5b61942..8403966e9 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -26,7 +26,14 @@ import { sendSuppressedAwareFreshAgentFrame } from '@/lib/fresh-agent-configure' import { KILL_ACK_TIMEOUT_MESSAGE, KILL_FAILED_MESSAGE, sendFreshAgentKillAndAwait, sendFreshAgentRecoveryStopAndAwait } from '@/lib/kill-ack' import { createLogger } from '@/lib/client-logger' import { getInstalledPerfAuditBridge } from '@/lib/perf-audit-bridge' -import { ApiError, api, getFreshAgentModelCapabilities, getFreshAgentThreadSnapshot, setSessionMetadata } from '@/lib/api' +import { + ApiError, + api, + getFreshAgentCheckpointOutcomePage, + getFreshAgentModelCapabilities, + getFreshAgentThreadSnapshot, + setSessionMetadata, +} from '@/lib/api' import { clearPaneCloseError, clearReconcilePendingPane, consumePaneRefreshRequest, mergePaneContent, startNewManagedRuntimeConversation, updatePaneContent } from '@/store/panesSlice' import { retryManagedConversation } from '@/lib/managed-runtime-retry' import { isManagedRuntimeRecoveryDecision, ManagedRuntimeRecoveryCard } from '@/components/ManagedRuntimeRecoveryCard' @@ -261,6 +268,12 @@ type CheckpointTurnAvailability = FreshAgentTurnRewindAvailability & { displayTurnId?: string } +type CheckpointAvailabilityPublication = { + requestId?: string + submittedTurnId?: string + availability: CheckpointTurnAvailability +} + const CHECKPOINT_FAILURE_FALLBACK = 'The project workspace could not be saved before this message was sent.' function sanitizeCheckpointReason(value: string): string { @@ -650,6 +663,61 @@ function getQuestionAgentLabel(paneContent: FreshAgentPaneContent, descriptorLab } } +function checkpointOutcomeMessageAvailability( + message: Record<string, unknown>, +): CheckpointTurnAvailability { + const safeTurnId = (value: unknown): string | undefined => ( + typeof value === 'string' + && value.length > 0 + && value.length <= 256 + && !/[\u0000-\u001f\u007f-\u009f]/.test(value) + ? value + : undefined + ) + const nativeTurnId = safeTurnId(message.nativeTurnId) + const displayTurnId = safeTurnId(message.displayTurnId) + const workspace = normalizeFreshAgentCheckpointWorkspace(message.workspace) + if ( + message.status === 'available' + && typeof message.checkpointId === 'string' + && message.checkpointId.length > 0 + && message.checkpointId.length <= 256 + && !/[\u0000-\u001f\u007f-\u009f]/.test(message.checkpointId) + ) { + return { + status: 'available', + checkpointId: message.checkpointId, + ...(workspace ? { workspace } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), + } + } + if (message.status === 'unavailable') { + const reason = typeof message.reason === 'string' + ? sanitizeCheckpointReason(message.reason) + : '' + const code = typeof message.code === 'string' && /^[a-z0-9_.:-]{1,96}$/.test(message.code) + ? message.code + : undefined + return { + status: 'unavailable', + reason: reason || CHECKPOINT_FAILURE_FALLBACK, + ...(code ? { code } : {}), + ...(workspace ? { workspace } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), + } + } + return { + status: 'unavailable', + reason: 'The server did not return a saved code checkpoint.', + code: 'checkpoint_capture_failed', + ...(workspace ? { workspace } : {}), + ...(nativeTurnId ? { nativeTurnId } : {}), + ...(displayTurnId ? { displayTurnId } : {}), + } +} + function isUnmaterializedCodexThreadError(error: unknown): boolean { return !!error && typeof error === 'object' @@ -1100,61 +1168,180 @@ export function FreshAgentView({ const [checkpointAvailabilityByTurn, setCheckpointAvailabilityByTurn] = useState<Map<string, CheckpointTurnAvailability>>( () => checkpointAvailabilityMap(paneContent), ) - const publishCheckpointAvailability = useCallback(( + const publishCheckpointAvailabilities = useCallback(( scope: string, - requestId: string | undefined, - submittedTurnId: string | undefined, - availability: CheckpointTurnAvailability, + publications: CheckpointAvailabilityPublication[], ) => { setCheckpointAvailabilityByTurn((current) => { const next = new Map(current) - if (requestId) { - next.set(checkpointAvailabilityKey(scope, 'request', requestId), availability) - } - if (submittedTurnId) { - next.set(checkpointTurnAvailabilityKey(scope, submittedTurnId), availability) - } - if (availability.displayTurnId) { - next.set(checkpointTurnAvailabilityKey(scope, availability.displayTurnId), availability) - } - if (availability.nativeTurnId) { - next.set(checkpointTurnAvailabilityKey(scope, availability.nativeTurnId), availability) + for (const { requestId, submittedTurnId, availability } of publications) { + if (requestId) { + next.set(checkpointAvailabilityKey(scope, 'request', requestId), availability) + } + if (submittedTurnId) { + next.set(checkpointTurnAvailabilityKey(scope, submittedTurnId), availability) + } + if (availability.displayTurnId) { + next.set(checkpointTurnAvailabilityKey(scope, availability.displayTurnId), availability) + } + if (availability.nativeTurnId) { + next.set(checkpointTurnAvailabilityKey(scope, availability.nativeTurnId), availability) + } } return next }) - if (!requestId || availability.status === 'pending') return + if (!publications.some(({ requestId, availability }) => ( + requestId && availability.status !== 'pending' + ))) return const state = appStore.getState() const paneRoot = state.panes.layouts[tabId] const latestPaneContent = paneRoot ? findPaneContent(paneRoot, paneId) : null if (latestPaneContent?.kind !== 'fresh-agent' || latestPaneContent.createRequestId !== scope) return const conversationSessionKey = freshAgentCheckpointConversationKeyForContent(latestPaneContent) - const outcome: FreshAgentCheckpointOutcome = { - version: 1, - requestId, - conversationRequestId: scope, - ...(conversationSessionKey ? { conversationSessionKey } : {}), - ...(availability.displayTurnId ? { displayTurnId: availability.displayTurnId } : {}), - ...(availability.nativeTurnId ? { nativeTurnId: availability.nativeTurnId } : {}), - status: availability.status, - ...(availability.status === 'available' && availability.checkpointId - ? { checkpointId: availability.checkpointId } - : {}), - ...(availability.workspace ? { workspace: availability.workspace } : {}), - ...(availability.status === 'unavailable' - ? { - ...(availability.code ? { code: availability.code } : {}), - reason: availability.reason, - } - : {}), + const outcomesByRequestId = new Map( + (latestPaneContent.checkpointOutcomes ?? []).map((outcome) => [outcome.requestId, outcome]), + ) + for (const { requestId, availability } of publications) { + if (!requestId || availability.status === 'pending') continue + const outcome: FreshAgentCheckpointOutcome = { + version: 1, + requestId, + conversationRequestId: scope, + ...(conversationSessionKey ? { conversationSessionKey } : {}), + ...(availability.displayTurnId ? { displayTurnId: availability.displayTurnId } : {}), + ...(availability.nativeTurnId ? { nativeTurnId: availability.nativeTurnId } : {}), + status: availability.status, + ...(availability.status === 'available' && availability.checkpointId + ? { checkpointId: availability.checkpointId } + : {}), + ...(availability.workspace ? { workspace: availability.workspace } : {}), + ...(availability.status === 'unavailable' + ? { + ...(availability.code ? { code: availability.code } : {}), + reason: availability.reason, + } + : {}), + } + outcomesByRequestId.delete(requestId) + outcomesByRequestId.set(requestId, outcome) + } + while (outcomesByRequestId.size > FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) { + const oldestRequestId = outcomesByRequestId.keys().next().value + if (oldestRequestId === undefined) break + outcomesByRequestId.delete(oldestRequestId) } - const priorOutcomes = latestPaneContent.checkpointOutcomes ?? [] - const checkpointOutcomes = [ - ...priorOutcomes.filter((item) => item.requestId !== requestId), - outcome, - ].slice(-FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) + const checkpointOutcomes = [...outcomesByRequestId.values()] dispatch(mergePaneContent({ tabId, paneId, updates: { checkpointOutcomes } })) }, [appStore, dispatch, paneId, tabId]) + const publishCheckpointAvailability = useCallback(( + scope: string, + requestId: string | undefined, + submittedTurnId: string | undefined, + availability: CheckpointTurnAvailability, + ) => { + publishCheckpointAvailabilities(scope, [{ requestId, submittedTurnId, availability }]) + }, [publishCheckpointAvailabilities]) + const checkpointOutcomeHistoryFull = (paneContent.checkpointOutcomes?.length ?? 0) + >= FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT + useEffect(() => { + const sessionId = paneContent.sessionId + ?? (paneContent.sessionRef?.provider === paneContent.provider + ? paneContent.sessionRef.sessionId + : undefined) + ?? paneContent.resumeSessionId + if ( + connectionStatus !== 'ready' + || !sessionId + || checkpointOutcomeHistoryFull + ) return + + const controller = new AbortController() + let cancelled = false + const provider = paneContent.provider + const sessionType = paneContent.sessionType + const scope = paneContent.createRequestId + const loadCheckpointOutcomes = async () => { + let cursor: string | undefined + let total = 0 + const seenCursors = new Set<string>() + do { + const page = await getFreshAgentCheckpointOutcomePage( + { provider, sessionId, sessionType, cursor }, + { signal: controller.signal }, + ) + if (cancelled) return + const pagePublications: CheckpointAvailabilityPublication[] = [] + const latestState = appStore.getState() + const latestLayout = latestState.panes.layouts[tabId] + const latestContent = latestLayout ? findPaneContent(latestLayout, paneId) : null + const existingOutcomes = new Map( + (latestContent?.kind === 'fresh-agent' ? latestContent.checkpointOutcomes ?? [] : []) + .map((saved) => [saved.requestId, saved]), + ) + for (const outcome of page.outcomes) { + if ( + outcome.provider !== provider + || outcome.sessionType !== sessionType + || outcome.sessionId !== sessionId + ) continue + const fetched = checkpointOutcomeMessageAvailability(outcome as unknown as Record<string, unknown>) + const existing = existingOutcomes.get(outcome.requestId) + const availability = existing + ? (() => { + const saved = checkpointOutcomeAvailability(existing) + return { + ...fetched, + ...saved, + nativeTurnId: fetched.nativeTurnId ?? saved.nativeTurnId, + displayTurnId: fetched.displayTurnId ?? saved.displayTurnId, + workspace: fetched.workspace ?? saved.workspace, + } + })() + : fetched + pagePublications.push({ requestId: outcome.requestId, availability }) + total += 1 + if (total >= FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) break + } + publishCheckpointAvailabilities(scope, pagePublications) + if (total >= FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) return + cursor = page.nextCursor + if (cursor && seenCursors.has(cursor)) { + throw new Error('Checkpoint outcome history returned a repeated page cursor.') + } + if (cursor) seenCursors.add(cursor) + } while (cursor) + } + + void loadCheckpointOutcomes().catch((err: unknown) => { + if (cancelled || controller.signal.aborted) return + log.warn({ + event: 'fresh_agent.checkpoint_outcomes_fetch_failed', + provider, + sessionId, + sessionType, + err, + }) + }) + return () => { + cancelled = true + controller.abort() + } + }, [ + appStore, + checkpointOutcomeHistoryFull, + connectionStatus, + paneContent.createRequestId, + paneContent.provider, + paneContent.resumeSessionId, + paneContent.sessionId, + paneContent.sessionRef?.provider, + paneContent.sessionRef?.sessionId, + paneContent.sessionType, + paneId, + publishCheckpointAvailabilities, + tabId, + ]) const getCheckpointAvailability = useCallback((turn: FreshAgentTurn): CheckpointTurnAvailability | undefined => { if (turn.role !== 'user') return undefined const scope = checkpointAvailabilityScope(paneContent) @@ -2805,20 +2992,10 @@ export function FreshAgentView({ const requestId = message.requestId const pending = pendingSendMetadataRef.current.get(requestId) const current = paneContentRef.current - const checkpointTurnId = (value: unknown): string | undefined => ( - typeof value === 'string' - && value.length > 0 - && value.length <= 256 - && !/[\u0000-\u001f\u007f-\u009f]/.test(value) - ? value - : undefined - ) - const nativeTurnId = checkpointTurnId(message.nativeTurnId) - const displayTurnId = checkpointTurnId(message.displayTurnId) - const hasProviderTurnIdentity = Boolean(nativeTurnId || displayTurnId) + const fetchedOutcome = checkpointOutcomeMessageAvailability(message) if ( (pending && message.logicalAttemptId !== pending.logicalAttemptId) - || (!pending && !hasProviderTurnIdentity) + || (!pending && !fetchedOutcome.nativeTurnId && !fetchedOutcome.displayTurnId) || !locatorMatchesPane( message, current, @@ -2827,57 +3004,15 @@ export function FreshAgentView({ ) ) return - const workspace = normalizeFreshAgentCheckpointWorkspace(message.workspace) + const workspace = fetchedOutcome.workspace ?? pending?.checkpointWorkspace ?? current.checkpointWorkspace - let checkpointOutcome: CheckpointTurnAvailability - if ( - message.status === 'available' - && typeof message.checkpointId === 'string' - && message.checkpointId.length > 0 - && message.checkpointId.length <= 256 - && !/[\u0000-\u001f\u007f-\u009f]/.test(message.checkpointId) - ) { - checkpointOutcome = { - status: 'available', - checkpointId: message.checkpointId, - ...(workspace ? { workspace } : {}), - ...(nativeTurnId ? { nativeTurnId } : {}), - ...(displayTurnId ? { displayTurnId } : {}), - } - } else if (message.status === 'unavailable') { - const suppliedReason = typeof message.reason === 'string' - ? sanitizeCheckpointReason(message.reason) - : '' - const suppliedCode = typeof message.code === 'string' - && /^[a-z0-9_.:-]{1,96}$/.test(message.code) - ? message.code - : undefined - checkpointOutcome = { - status: 'unavailable', - reason: suppliedReason || CHECKPOINT_FAILURE_FALLBACK, - ...(suppliedCode ? { code: suppliedCode } : {}), - ...(workspace ? { workspace } : {}), - ...(nativeTurnId ? { nativeTurnId } : {}), - ...(displayTurnId ? { displayTurnId } : {}), - } - } else { - checkpointOutcome = { - status: 'unavailable', - reason: 'The server did not return a saved code checkpoint.', - code: 'checkpoint_capture_failed', - ...(workspace ? { workspace } : {}), - ...(nativeTurnId ? { nativeTurnId } : {}), - ...(displayTurnId ? { displayTurnId } : {}), - } - } + const checkpointOutcome = { ...fetchedOutcome, ...(workspace ? { workspace } : {}) } if (pending) { recordPendingSendMetadata(requestId, { checkpointOutcome }) } else { - // Replayed outcomes arrive after remount, when the transient send - // map is empty. The provider IDs are stamped only after observing - // the exact persisted user row, so they safely bind the durable - // result to its transcript row without prompt or ordinal matching. + // Late outcome events have no transient send metadata. Provider row + // IDs bind them to the exact transcript row without prompt matching. publishCheckpointAvailability( checkpointAvailabilityScope(current), requestId, diff --git a/src/lib/api.ts b/src/lib/api.ts index dfa9c6142..e253c73db 100644 --- a/src/lib/api.ts +++ b/src/lib/api.ts @@ -1,5 +1,8 @@ import type { CodingCliProviderName } from './coding-cli-types' -import type { TokenSummary } from '@shared/ws-protocol' +import { + FreshAgentCheckpointOutcomeSchema, + type TokenSummary, +} from '@shared/ws-protocol' import type { TitleSource } from '@shared/title-source' import { FreshAgentSnapshotSchema, @@ -626,6 +629,43 @@ export async function getFreshAgentModelCapabilities( } } +const FreshAgentCheckpointOutcomePageSchema = z.object({ + outcomes: z.array(FreshAgentCheckpointOutcomeSchema), + nextCursor: z.string().min(1).optional(), +}).strict() + +export type FreshAgentCheckpointOutcomePage = z.infer<typeof FreshAgentCheckpointOutcomePageSchema> + +export async function getFreshAgentCheckpointOutcomePage( + identity: { + provider: string + sessionId: string + sessionType: string + cursor?: string + }, + options: ApiRequestOptions = {}, +): Promise<FreshAgentCheckpointOutcomePage> { + const query = new URLSearchParams({ + provider: identity.provider, + sessionId: identity.sessionId, + sessionType: identity.sessionType, + limit: '200', + }) + if (identity.cursor) query.set('cursor', identity.cursor) + const raw = await with429Retry( + () => api.get(`/api/fresh-agent/checkpoints/outcomes?${query.toString()}`, options), + options, + ) + const parsed = FreshAgentCheckpointOutcomePageSchema.safeParse(raw) + if (!parsed.success) { + throw new FreshAgentApiContractError( + 'Fresh-agent checkpoint outcome response did not match the shared contract.', + parsed.error.issues, + ) + } + return parsed.data +} + export async function refreshFreshAgentModelCapabilities( sessionType: string, options: ApiRequestOptions & { cwd?: string } = {}, diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index 89bcbc1df..f66dca56d 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -298,6 +298,7 @@ const wsMock = vi.hoisted(() => ({ const apiMock = vi.hoisted(() => ({ get: vi.fn(), getFreshAgentThreadSnapshot: vi.fn(), + getFreshAgentCheckpointOutcomePage: vi.fn(), getFreshAgentModelCapabilities: vi.fn(), post: vi.fn(), requestSessionHandoff: vi.fn(), @@ -324,6 +325,7 @@ vi.mock('@/lib/api', async () => { ...actual, api: { ...actual.api, get: apiMock.get, post: apiMock.post }, getFreshAgentThreadSnapshot: apiMock.getFreshAgentThreadSnapshot, + getFreshAgentCheckpointOutcomePage: apiMock.getFreshAgentCheckpointOutcomePage, getFreshAgentModelCapabilities: apiMock.getFreshAgentModelCapabilities, requestSessionHandoff: apiMock.requestSessionHandoff, setSessionMetadata: apiMock.setSessionMetadata, @@ -554,9 +556,11 @@ beforeEach(() => { window.localStorage.removeItem('fresh-agent-prompt-history:freshcodex') window.localStorage.removeItem('fresh-agent-prompt-history:freshclaude') apiMock.getFreshAgentThreadSnapshot.mockReset() + apiMock.getFreshAgentCheckpointOutcomePage.mockReset() apiMock.getFreshAgentModelCapabilities.mockReset() apiMock.get.mockReset() apiMock.get.mockResolvedValue({ checkpoints: [] }) + apiMock.getFreshAgentCheckpointOutcomePage.mockResolvedValue({ outcomes: [] }) apiMock.post.mockReset() apiMock.requestSessionHandoff.mockReset() apiMock.setSessionMetadata.mockReset() @@ -3629,7 +3633,7 @@ describe('FreshAgentView', () => { expect(apiMock.get).not.toHaveBeenCalledWith(expect.stringContaining('/api/fresh-agent/checkpoints?')) }) - it('accepts a replayed checkpoint outcome after reload when it carries an exact provider row identity', async () => { + it('accepts a late checkpoint outcome event when it carries an exact provider row identity', async () => { const checkpointId = 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' const workspace = { path: '/repo/selected', provenance: 'directory-picker' as const } const nativeTurnId = 'codex-replayed-turn' @@ -3706,8 +3710,8 @@ describe('FreshAgentView', () => { expect(getFreshAgentPaneContent(store).checkpointOutcomes ?? []).toHaveLength(0) expect(screen.queryByRole('status', { name: 'Code rewind available for this message' })).not.toBeInTheDocument() - // The first broadcast was missed while this pane was detached. The server - // replay has no in-memory pending send metadata to rely on after remount. + // A later identity-enriched event can arrive after the pane has already + // processed the checkpoint result, so it must still bind only the exact row. act(() => { onMessage?.({ type: 'freshAgent.checkpoint.outcome', @@ -3745,9 +3749,8 @@ describe('FreshAgentView', () => { }), ])) - // The server may replay many durable outcomes after attach. A local - // retention limit must not discard an older exact checkpoint from that - // replay before the pane is reloaded again. + // Recovery can return many durable outcomes in pages. A local retention + // limit must not discard an older exact checkpoint before the pane reloads. act(() => { for (let index = 0; index < 200; index += 1) { onMessage?.({ @@ -3790,6 +3793,86 @@ describe('FreshAgentView', () => { .toBeDisabled() }) + it('restores durable checkpoint outcomes from paged session history after reopening a pane', async () => { + const sessionId = 'thread-reopened-checkpoints' + const requestId = 'request-reopened-checkpoints' + const nativeTurnId = 'native-reopened-turn' + const checkpointId = 'dddddddddddddddddddddddddddddddd' + const workspace = { path: '/repo/selected', provenance: 'directory-picker' as const } + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'reopened session', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: `${nativeTurnId}:row-0`, + turnId: `${nativeTurnId}:row-0`, + nativeTurnId, + source: 'durable', + role: 'user', + summary: 'A turn from the reopened session', + items: [{ id: 'reopened-turn-text', kind: 'text', text: 'A turn from the reopened session' }], + }], + }) + apiMock.getFreshAgentCheckpointOutcomePage + .mockResolvedValueOnce({ outcomes: [], nextCursor: 'checkpoint-page-2' }) + .mockResolvedValueOnce({ outcomes: [{ + type: 'freshAgent.checkpoint.outcome', + logicalAttemptId: 'attempt-reopened-checkpoints', + requestId, + provider: 'codex', + sessionId, + sessionType: 'freshcodex', + status: 'available', + checkpointId, + workspace, + nativeTurnId, + displayTurnId: `${nativeTurnId}:row-0`, + }] }) + + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'new-pane-after-reopen', + sessionId, + sessionRef: { provider: 'codex', sessionId }, + status: 'idle', + }, + })) + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + const rewind = await screen.findByRole('button', { name: 'Rewind code to here' }) + expect(rewind).toBeEnabled() + expect(apiMock.getFreshAgentCheckpointOutcomePage).toHaveBeenNthCalledWith( + 1, + { provider: 'codex', sessionId, sessionType: 'freshcodex', cursor: undefined }, + expect.objectContaining({ signal: expect.any(AbortSignal) }), + ) + expect(apiMock.getFreshAgentCheckpointOutcomePage).toHaveBeenNthCalledWith( + 2, + { provider: 'codex', sessionId, sessionType: 'freshcodex', cursor: 'checkpoint-page-2' }, + expect.objectContaining({ signal: expect.any(AbortSignal) }), + ) + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ + requestId, + conversationRequestId: 'new-pane-after-reopen', + conversationSessionKey: JSON.stringify(['codex', 'freshcodex', sessionId]), + checkpointId, + nativeTurnId, + displayTurnId: `${nativeTurnId}:row-0`, + }), + ])) + }) + it('keeps an unbound modern turn unavailable instead of matching its prompt to another checkpoint', async () => { const duplicatePrompt = 'Apply this repeated prompt' apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ diff --git a/test/unit/client/lib/api.test.ts b/test/unit/client/lib/api.test.ts index 55c88f983..a06aef188 100644 --- a/test/unit/client/lib/api.test.ts +++ b/test/unit/client/lib/api.test.ts @@ -1,6 +1,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { api, + getFreshAgentCheckpointOutcomePage, getFreshAgentModelCapabilities, refreshFreshAgentModelCapabilities, getFreshAgentThreadSnapshot, @@ -222,6 +223,34 @@ describe('visible-first read-model helpers', () => { ) }) + it('loads a typed checkpoint outcome page for one provider session', async () => { + const signal = new AbortController().signal + const outcome = { + type: 'freshAgent.checkpoint.outcome', + logicalAttemptId: 'attempt-reopened', + requestId: 'request-reopened', + provider: 'codex', + sessionId: 'thread/one', + sessionType: 'freshcodex', + status: 'available', + checkpointId: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + nativeTurnId: 'native-turn-1', + displayTurnId: 'native-turn-1:row-0', + } + mockFetch.mockResolvedValueOnce(mockJson({ outcomes: [outcome], nextCursor: 'cursor+/' })) + + await expect(getFreshAgentCheckpointOutcomePage({ + provider: 'codex', + sessionId: 'thread/one', + sessionType: 'freshcodex', + cursor: 'cursor+/', + }, { signal })).resolves.toEqual({ outcomes: [outcome], nextCursor: 'cursor+/' }) + expect(mockFetch).toHaveBeenCalledWith( + '/api/fresh-agent/checkpoints/outcomes?provider=codex&sessionId=thread%2Fone&sessionType=freshcodex&limit=200&cursor=cursor%2B%2F', + expect.objectContaining({ signal, headers: expect.any(Headers) }), + ) + }) + it('getTerminalDirectoryPage encodes cursor, priority, revision, and limit consistently', async () => { const signal = new AbortController().signal mockFetch.mockResolvedValueOnce(mockJson({ items: [] })) From 128ec671d0b01c79321a56578674b7b48d9de1cb Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 14:00:06 -0700 Subject: [PATCH 467/493] fix: preserve checkpoint recovery and provider exit --- crates/freshell-session-host/src/pty.rs | 31 ++++++- src/components/fresh-agent/FreshAgentView.tsx | 9 +-- .../fresh-agent/FreshAgentView.test.tsx | 81 ++++++++++++++++++- 3 files changed, 109 insertions(+), 12 deletions(-) diff --git a/crates/freshell-session-host/src/pty.rs b/crates/freshell-session-host/src/pty.rs index 6e78f094b..5a0c52c8c 100644 --- a/crates/freshell-session-host/src/pty.rs +++ b/crates/freshell-session-host/src/pty.rs @@ -93,16 +93,17 @@ impl HostedPty { .lock() .map(|mut journal| journal.complete(code).is_ok()) .unwrap_or(false); + // Process exit is independent of whether retained output settled; + // supervision must never reattach a process that has already exited. + exit_value.store(code, Ordering::SeqCst); + exit_flag.store(true, Ordering::SeqCst); if !settled { tracing::error!( event = "managed_runtime.output_completion_failed", cause = "spool_settlement", "Output completion could not be committed" ); - return; } - exit_value.store(code, Ordering::SeqCst); - exit_flag.store(true, Ordering::SeqCst); if let (Some(handle), Some(sidecar)) = (runtime_handle.as_ref(), codex_sidecar_on_exit.as_ref()) { @@ -550,6 +551,30 @@ mod output_epoch_tests { assert_eq!(completion["exit"]["code"], 17); } + #[tokio::test] + async fn retained_output_failure_does_not_hide_provider_exit() { + let root = tempfile::tempdir().unwrap(); + let (hosted, incarnation) = retained_pty(root.path(), "sleep 0.1; exit 23").await; + hosted.output.lock().unwrap().record_failure(); + + tokio::time::timeout(std::time::Duration::from_secs(2), async { + while !hosted.exited() { + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + }) + .await + .expect("journal completion failure must not leave an exited provider reported live"); + + assert_eq!(hosted.exit_code(), 23); + let output_error = hosted + .read_output(incarnation, 0, 65536, Some(hosted.stream_epoch())) + .expect_err("the failed retained-output journal remains unavailable"); + assert_eq!( + output_error.code, + freshell_runtime_protocol::RuntimeErrorCode::HostUnreachable + ); + } + #[tokio::test] async fn retained_output_successful_stop_joins_reader_before_reply() { let root = tempfile::tempdir().unwrap(); diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 8403966e9..7e2464361 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -1242,19 +1242,13 @@ export function FreshAgentView({ ) => { publishCheckpointAvailabilities(scope, [{ requestId, submittedTurnId, availability }]) }, [publishCheckpointAvailabilities]) - const checkpointOutcomeHistoryFull = (paneContent.checkpointOutcomes?.length ?? 0) - >= FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT useEffect(() => { const sessionId = paneContent.sessionId ?? (paneContent.sessionRef?.provider === paneContent.provider ? paneContent.sessionRef.sessionId : undefined) ?? paneContent.resumeSessionId - if ( - connectionStatus !== 'ready' - || !sessionId - || checkpointOutcomeHistoryFull - ) return + if (connectionStatus !== 'ready' || !sessionId) return const controller = new AbortController() let cancelled = false @@ -1329,7 +1323,6 @@ export function FreshAgentView({ } }, [ appStore, - checkpointOutcomeHistoryFull, connectionStatus, paneContent.createRequestId, paneContent.provider, diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index f66dca56d..04f487571 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -35,7 +35,7 @@ import { handleFreshAgentMessage } from '@/lib/fresh-agent-ws' import { buildReconcileRequest, foldVerdicts, setFreshAgentReconcileActive } from '@/lib/pane-reconcile' import type { AppDispatch, RootState } from '@/store/store' import { getFreshAgentSnapshotThreadId, getManagedBootstrapHistoryThreadId } from '@/lib/fresh-agent-snapshot-thread' -import type { FreshAgentPaneContent } from '@/store/paneTypes' +import { FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT, type FreshAgentPaneContent } from '@/store/paneTypes' import { ApiError } from '@/lib/api' import { resetSnapshotSchedulerForTests, SNAPSHOT_DEBOUNCE_MS } from '@/lib/fresh-agent-snapshot-scheduler' import { SESSION_HANDOFF_RETRY_BACKOFF_MS } from '@/lib/session-handoff' @@ -3873,6 +3873,85 @@ describe('FreshAgentView', () => { ])) }) + it('refreshes checkpoint history when the local outcome list is already at its limit', async () => { + const sessionId = 'thread-full-checkpoint-cache' + const nativeTurnId = 'native-late-checkpoint-turn' + const requestId = 'request-late-checkpoint' + const sessionKey = JSON.stringify(['codex', 'freshcodex', sessionId]) + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'full local checkpoint cache', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: `${nativeTurnId}:row-0`, + turnId: `${nativeTurnId}:row-0`, + nativeTurnId, + source: 'durable', + role: 'user', + summary: 'A turn whose checkpoint arrived on another device', + items: [{ id: 'late-checkpoint-turn-text', kind: 'text', text: 'A turn whose checkpoint arrived on another device' }], + }], + }) + apiMock.getFreshAgentCheckpointOutcomePage.mockResolvedValue({ outcomes: [{ + type: 'freshAgent.checkpoint.outcome', + logicalAttemptId: 'attempt-late-checkpoint', + requestId, + provider: 'codex', + sessionId, + sessionType: 'freshcodex', + status: 'available', + checkpointId: 'ffffffffffffffffffffffffffffffff', + nativeTurnId, + displayTurnId: `${nativeTurnId}:row-0`, + }] }) + + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'new-pane-with-full-cache', + sessionId, + sessionRef: { provider: 'codex', sessionId }, + status: 'idle', + checkpointOutcomes: Array.from({ length: FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT }, (_, index) => ({ + version: 1 as const, + conversationRequestId: 'old-pane-scope', + conversationSessionKey: sessionKey, + requestId: `saved-outcome-${index}`, + status: 'unavailable' as const, + reason: 'No old checkpoint is available for this turn.', + })), + } as never, + })) + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + expect(await screen.findByRole('button', { name: 'Rewind code to here' })).toBeEnabled() + expect(apiMock.getFreshAgentCheckpointOutcomePage).toHaveBeenCalledWith( + { provider: 'codex', sessionId, sessionType: 'freshcodex', cursor: undefined }, + expect.objectContaining({ signal: expect.any(AbortSignal) }), + ) + const outcomes = getFreshAgentPaneContent(store).checkpointOutcomes ?? [] + expect(outcomes).toHaveLength(FRESH_AGENT_CHECKPOINT_OUTCOME_LIMIT) + expect(outcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ + requestId, + conversationRequestId: 'new-pane-with-full-cache', + conversationSessionKey: sessionKey, + nativeTurnId, + displayTurnId: `${nativeTurnId}:row-0`, + }), + ])) + expect(outcomes.some((outcome) => outcome.requestId === 'saved-outcome-0')).toBe(false) + }) + it('keeps an unbound modern turn unavailable instead of matching its prompt to another checkpoint', async () => { const duplicatePrompt = 'Apply this repeated prompt' apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ From 6c67ed7adb44f9aff615598d93a84c12e0808630 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 14:14:13 -0700 Subject: [PATCH 468/493] fix: preserve rewind outcomes and completion writes --- .../src/retained_output.rs | 158 +++++++++++++----- .../src/retained_output_tests.rs | 70 ++++++++ src/components/fresh-agent/FreshAgentView.tsx | 27 +-- .../fresh-agent/FreshAgentView.test.tsx | 153 +++++++++++++++++ 4 files changed, 346 insertions(+), 62 deletions(-) diff --git a/crates/freshell-supervisor/src/retained_output.rs b/crates/freshell-supervisor/src/retained_output.rs index ea05249c9..2f22f41ec 100644 --- a/crates/freshell-supervisor/src/retained_output.rs +++ b/crates/freshell-supervisor/src/retained_output.rs @@ -17,7 +17,7 @@ use std::{ sync::atomic::{AtomicU64, Ordering}, }; -static INDEX_TEMP_SEQUENCE: AtomicU64 = AtomicU64::new(0); +static TEMP_FILE_SEQUENCE: AtomicU64 = AtomicU64::new(0); #[cfg(test)] std::thread_local! { @@ -126,29 +126,84 @@ pub(crate) fn read( exit: TerminalExitObservation::Unavailable, }; completed.validate(&descriptor, batch.head_seq)?; - let temporary = completion_path.with_extension("tmp"); - let mut file = OpenOptions::new() - .create(true) - .truncate(true) - .write(true) - .open(&temporary) - .map_err(|_| "completion unavailable")?; - let bytes = serde_json::to_vec(&completed).map_err(|_| "completion encoding failed")?; - file.write_all(&bytes) - .and_then(|_| file.sync_all()) - .map_err(|_| "completion write failed")?; - std::fs::rename(temporary, &completion_path).map_err(|_| "completion commit failed")?; - File::open(&source.runtime_dir) - .and_then(|dir| dir.sync_all()) - .map_err(|_| "completion parent sync failed")?; + let completed = commit_completion_if_absent(&completion_path, &descriptor, completed)?; + batch.exit_code = match &completed.exit { + TerminalExitObservation::Known { code } => Some(*code), + TerminalExitObservation::Unavailable => None, + }; batch.completion = Some(completed); batch.exited = true; - batch.exit_code = None; } batch.native_session_id = source.native_session_id; Ok(batch) } +fn commit_completion_if_absent( + path: &Path, + descriptor: &TerminalSpoolDescriptor, + completion: TerminalSpoolCompletion, +) -> Result<TerminalSpoolCompletion, String> { + completion.validate(descriptor, completion.head_seq)?; + let bytes = serde_json::to_vec(&completion).map_err(|_| "completion encoding failed")?; + let (temporary, mut file) = create_completion_temporary(path)?; + let _temporary_cleanup = RemoveTemporaryFile(temporary.clone()); + file.write_all(&bytes) + .and_then(|_| file.sync_all()) + .map_err(|_| "completion write failed")?; + drop(file); + + match std::fs::hard_link(&temporary, path) { + Ok(()) => {} + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + let existing: TerminalSpoolCompletion = metadata(path)?; + existing.validate(descriptor, completion.head_seq)?; + return Ok(existing); + } + Err(_) => return Err("completion commit failed".into()), + } + + File::open( + path.parent() + .ok_or("retained output directory unavailable")?, + ) + .and_then(|directory| directory.sync_all()) + .map_err(|_| "completion parent sync failed")?; + Ok(completion) +} + +fn create_completion_temporary(path: &Path) -> Result<(PathBuf, File), String> { + create_unique_temporary( + path, + "completion unavailable", + "completion temporary namespace exhausted", + ) +} + +fn create_unique_temporary( + path: &Path, + unavailable: &'static str, + namespace_exhausted: &'static str, +) -> Result<(PathBuf, File), String> { + let parent = path + .parent() + .ok_or("retained output directory unavailable")?; + let name = path.file_name().ok_or(unavailable)?.to_string_lossy(); + for _ in 0..128 { + let sequence = TEMP_FILE_SEQUENCE.fetch_add(1, Ordering::Relaxed); + let temporary = parent.join(format!(".{name}.tmp-{}-{sequence}", std::process::id())); + match OpenOptions::new() + .create_new(true) + .write(true) + .open(&temporary) + { + Ok(file) => return Ok((temporary, file)), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue, + Err(_) => return Err(unavailable.into()), + } + } + Err(namespace_exhausted.into()) +} + fn read_indexed( source: &RetainedOutputSource, descriptor: &TerminalSpoolDescriptor, @@ -392,7 +447,7 @@ fn write_index_atomic( torn_tail, )?; let (temporary, mut file) = create_index_temporary(path)?; - let _temporary_cleanup = RemoveIndexTemporary(temporary.clone()); + let _temporary_cleanup = RemoveTemporaryFile(temporary.clone()); file.write_all(&bytes) .and_then(|_| file.sync_all()) .map_err(|_| "retained output index write failed")?; @@ -414,32 +469,16 @@ fn write_index_atomic( } fn create_index_temporary(path: &Path) -> Result<(PathBuf, File), String> { - let parent = path - .parent() - .ok_or("retained output directory unavailable")?; - let name = path - .file_name() - .ok_or("retained output index unavailable")? - .to_string_lossy(); - for _ in 0..128 { - let sequence = INDEX_TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed); - let temporary = parent.join(format!(".{name}.tmp-{}-{sequence}", std::process::id())); - match OpenOptions::new() - .create_new(true) - .write(true) - .open(&temporary) - { - Ok(file) => return Ok((temporary, file)), - Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue, - Err(_) => return Err("retained output index unavailable".into()), - } - } - Err("retained output index temporary namespace exhausted".into()) + create_unique_temporary( + path, + "retained output index unavailable", + "retained output index temporary namespace exhausted", + ) } -struct RemoveIndexTemporary(PathBuf); +struct RemoveTemporaryFile(PathBuf); -impl Drop for RemoveIndexTemporary { +impl Drop for RemoveTemporaryFile { fn drop(&mut self) { let _ = std::fs::remove_file(&self.0); } @@ -464,6 +503,43 @@ mod tests { IncarnationId, TerminalLaunchSpec, }; + #[test] + fn completion_commit_preserves_a_concurrent_known_exit_observation() { + let directory = tempfile::tempdir().unwrap(); + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id: IncarnationId::new(), + terminal_id: "completion-terminal".into(), + stream_epoch: "launch-prefix-hostboot-completion".into(), + segment_limit_bytes: 32 * 1024 * 1024, + }; + let path = directory.path().join(SPOOL_COMPLETION_FILE); + let known = TerminalSpoolCompletion { + incarnation_id: descriptor.incarnation_id.clone(), + terminal_id: descriptor.terminal_id.clone(), + stream_epoch: descriptor.stream_epoch.clone(), + retained_from_seq: 1, + head_seq: 3, + exit: TerminalExitObservation::Known { code: 17 }, + }; + std::fs::write(&path, serde_json::to_vec(&known).unwrap()).unwrap(); + let fallback = TerminalSpoolCompletion { + exit: TerminalExitObservation::Unavailable, + ..known.clone() + }; + + let committed = commit_completion_if_absent(&path, &descriptor, fallback).unwrap(); + let on_disk: TerminalSpoolCompletion = metadata(&path).unwrap(); + + assert_eq!(committed, known); + assert_eq!(on_disk, known); + assert_eq!( + std::fs::read_dir(directory.path()).unwrap().count(), + 1, + "the temporary completion file is removed after another writer wins" + ); + } + #[test] fn concurrent_cold_index_rebuilds_do_not_collide_on_temporary_files() { let directory = tempfile::tempdir().unwrap(); diff --git a/crates/freshell-supervisor/src/retained_output_tests.rs b/crates/freshell-supervisor/src/retained_output_tests.rs index c47e682ac..27e596000 100644 --- a/crates/freshell-supervisor/src/retained_output_tests.rs +++ b/crates/freshell-supervisor/src/retained_output_tests.rs @@ -442,6 +442,76 @@ async fn retained_output_verified_end_survives_cleanup_and_cold_supervisor() { assert_eq!(windows[0], "x".repeat(131072) + "final diagnostic"); } +#[tokio::test] +async fn concurrent_verified_empty_reads_share_one_completion_commit() { + let root = tempfile::tempdir().unwrap(); + let (_supervisor, context) = + retained_output_fixture(root.path(), BackendRuntimeState::Exited).await; + let incarnation_id = context.prior_handle.incarnation_id().clone(); + let runtime_dir = context.prior_handle.runtime_dir().to_path_buf(); + let terminal = context.prior_handle.terminal().unwrap().clone(); + let gate = std::sync::Arc::new(std::sync::Barrier::new(32)); + let readers = (0..32) + .map(|_| { + let gate = std::sync::Arc::clone(&gate); + let source = crate::registry::RetainedOutputSource { + incarnation_id: incarnation_id.clone(), + runtime_dir: runtime_dir.clone(), + terminal: terminal.clone(), + native_session_id: None, + }; + tokio::task::spawn_blocking(move || { + gate.wait(); + crate::retained_output::read( + source, + 0, + 65536, + Some("launch-prefix-hostboot-exact".into()), + true, + ) + }) + }) + .collect::<Vec<_>>(); + + for reader in readers { + let output = reader.await.unwrap().unwrap(); + assert!(output.exited); + assert_eq!(output.head_seq, 3); + assert!(matches!( + output.completion.unwrap().exit, + freshell_runtime_protocol::TerminalExitObservation::Unavailable + )); + } + let completion: freshell_runtime_protocol::TerminalSpoolCompletion = serde_json::from_slice( + &std::fs::read(runtime_dir.join("terminal-spool-completion.json")).unwrap(), + ) + .unwrap(); + completion + .validate( + &freshell_runtime_protocol::TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id, + terminal_id: terminal.terminal_id, + stream_epoch: "launch-prefix-hostboot-exact".into(), + segment_limit_bytes: 33554432, + }, + 3, + ) + .unwrap(); + assert_eq!( + std::fs::read_dir(&runtime_dir) + .unwrap() + .filter_map(Result::ok) + .filter(|entry| entry + .file_name() + .to_string_lossy() + .starts_with(".terminal-spool-completion.json.tmp-")) + .count(), + 0, + "concurrent completion temporaries are cleaned up" + ); +} + #[tokio::test] async fn retained_output_missing_spool_reports_real_failure() { let root = tempfile::tempdir().unwrap(); diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 7e2464361..6106c74b9 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -1284,9 +1284,13 @@ export function FreshAgentView({ const availability = existing ? (() => { const saved = checkpointOutcomeAvailability(existing) + const savedRestoreFailure = saved.status === 'unavailable' + && saved.code === 'workspace_unavailable' + const merged = savedRestoreFailure && fetched.status === 'available' + ? fetched + : { ...fetched, ...saved } return { - ...fetched, - ...saved, + ...merged, nativeTurnId: fetched.nativeTurnId ?? saved.nativeTurnId, displayTurnId: fetched.displayTurnId ?? saved.displayTurnId, workspace: fetched.workspace ?? saved.workspace, @@ -4205,13 +4209,6 @@ export function FreshAgentView({ const cwd = agentSessionCwdRef.current ?? current.initialCwd ?? workspace?.path if (!cwd) { const reason = 'No project working directory is available for this message.' - const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId - publishCheckpointAvailability( - checkpointAvailabilityScope(current), - typeof requestId === 'string' ? requestId : undefined, - getFreshAgentDisplayTurnKey(turn), - { status: 'unavailable', reason, code: 'workspace_unavailable', ...(workspace ? { workspace } : {}) }, - ) setNotice(`Rewind unavailable for this message: ${reason}`) return } @@ -4232,18 +4229,6 @@ export function FreshAgentView({ const details = (error as { details?: unknown }).details if (details && typeof details === 'object' && (details as { code?: unknown }).code === 'workspace_unavailable') { const reason = checkpointFailureReason(error) - const requestId = (turn as FreshAgentTurn & { requestId?: unknown }).requestId - publishCheckpointAvailability( - checkpointAvailabilityScope(current), - typeof requestId === 'string' ? requestId : undefined, - getFreshAgentDisplayTurnKey(turn), - { - status: 'unavailable', - reason, - code: checkpointFailureCode(error) ?? 'checkpoint_restore_failed', - ...(workspace ? { workspace } : {}), - }, - ) setNotice(`Rewind unavailable for this message: ${reason}`) return } diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index 04f487571..c17dfe1bb 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3873,6 +3873,80 @@ describe('FreshAgentView', () => { ])) }) + it('recovers a stale workspace restore error from the authoritative checkpoint history', async () => { + const sessionId = 'thread-recovered-checkpoint-error' + const requestId = 'request-stale-workspace-error' + const nativeTurnId = 'native-recovered-checkpoint-error' + const displayTurnId = `${nativeTurnId}:row-0` + const checkpointId = 'abababababababababababababababab' + const workspace = { path: '/repo/recovered', provenance: 'directory-picker' as const } + const sessionKey = JSON.stringify(['codex', 'freshcodex', sessionId]) + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'recovered checkpoint', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: displayTurnId, + turnId: displayTurnId, + nativeTurnId, + source: 'durable', + role: 'user', + summary: 'A checkpoint whose workspace is available again', + items: [{ id: 'recovered-checkpoint-text', kind: 'text', text: 'A checkpoint whose workspace is available again' }], + }], + }) + apiMock.getFreshAgentCheckpointOutcomePage.mockResolvedValue({ outcomes: [{ + type: 'freshAgent.checkpoint.outcome', + logicalAttemptId: 'attempt-recovered-checkpoint-error', + requestId, + provider: 'codex', + sessionId, + sessionType: 'freshcodex', + status: 'available', + checkpointId, + workspace, + nativeTurnId, + displayTurnId, + }] }) + + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'new-pane-after-restore-error', + sessionId, + sessionRef: { provider: 'codex', sessionId }, + status: 'idle', + checkpointOutcomes: [{ + version: 1, + conversationRequestId: 'old-pane-scope', + conversationSessionKey: sessionKey, + requestId, + nativeTurnId, + displayTurnId, + status: 'unavailable', + code: 'workspace_unavailable', + reason: 'The selected project workspace is temporarily unavailable.', + workspace, + }], + } as never, + })) + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + + expect(await screen.findByRole('button', { name: 'Rewind code to here' })).toBeEnabled() + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ requestId, status: 'available', checkpointId, workspace }), + ])) + }) + it('refreshes checkpoint history when the local outcome list is already at its limit', async () => { const sessionId = 'thread-full-checkpoint-cache' const nativeTurnId = 'native-late-checkpoint-turn' @@ -4191,6 +4265,85 @@ describe('FreshAgentView', () => { expect(apiMock.get).not.toHaveBeenCalledWith(expect.stringContaining('/api/fresh-agent/checkpoints?')) }) + it('keeps a checkpoint available and retryable after a temporary workspace restore failure', async () => { + const sessionId = 'thread-temporary-restore-failure' + const requestId = 'request-temporary-restore-failure' + const nativeTurnId = 'native-temporary-restore-failure' + const displayTurnId = `${nativeTurnId}:row-0` + const checkpointId = 'cdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcd' + const workspace = { path: '/repo/temporarily-unavailable', provenance: 'directory-picker' as const } + const reason = 'The selected project workspace is temporarily unavailable.' + apiMock.getFreshAgentThreadSnapshot.mockResolvedValue({ + status: 'idle', + summary: 'checkpoint retry after workspace recovery', + capabilities: { send: true, interrupt: true, fork: true }, + turns: [{ + id: displayTurnId, + turnId: displayTurnId, + nativeTurnId, + requestId, + role: 'user', + summary: 'Retry this rewind after the workspace returns', + items: [{ id: 'temporary-restore-failure-text', kind: 'text', text: 'Retry this rewind after the workspace returns' }], + }], + }) + apiMock.post + .mockRejectedValueOnce(new ApiError(422, 'workspace unavailable', { + code: 'workspace_unavailable', + details: reason, + })) + .mockResolvedValue({ ok: true }) + + const store = createStore() + store.dispatch(initLayout({ + tabId: 'tab-1', + paneId: 'pane-1', + content: { + kind: 'fresh-agent', + sessionType: 'freshcodex', + provider: 'codex', + createRequestId: 'req-temporary-restore-failure', + sessionId, + status: 'idle', + initialCwd: workspace.path, + checkpointOutcomes: [{ + version: 1, + conversationRequestId: 'req-temporary-restore-failure', + requestId, + nativeTurnId, + displayTurnId, + status: 'available', + checkpointId, + workspace, + }], + } as never, + })) + render( + <Provider store={store}> + <StoreBackedFreshAgentView tabId="tab-1" paneId="pane-1" /> + </Provider>, + ) + vi.spyOn(window, 'confirm').mockReturnValue(true) + const turn = await screen.findByText('Retry this rewind after the workspace returns') + const article = turn.closest('[data-turn-role="user"]') as HTMLElement + fireEvent.click(within(article).getByRole('button', { name: 'Rewind code to here' })) + + expect(await screen.findByText(`Rewind unavailable for this message: ${reason}`)).toBeInTheDocument() + expect(getFreshAgentPaneContent(store).checkpointOutcomes).toEqual(expect.arrayContaining([ + expect.objectContaining({ requestId, status: 'available', checkpointId, workspace }), + ])) + const retry = within(article).getByRole('button', { name: 'Rewind code to here' }) + expect(retry).toBeEnabled() + fireEvent.click(retry) + + await waitFor(() => expect(apiMock.post).toHaveBeenCalledTimes(2)) + expect(apiMock.post).toHaveBeenLastCalledWith('/api/fresh-agent/checkpoints/restore', { + cwd: workspace.path, + workspace, + id: checkpointId, + }) + }) + it('rewinds to the persisted checkpoint ID after reload when duplicate prompts have no metadata links', async () => { const checkpointWorkspace = { path: '/repo/selected', provenance: 'directory-picker' as const } const firstCheckpointId = '11111111111111111111111111111111' From 841eece9ee56c9c0ef4272facaabf1265b71e73d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 14:44:00 -0700 Subject: [PATCH 469/493] test: assert selected checkpoint rewind target --- .../specs/code-checkpoint-workspace-external.spec.ts | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts index 8fe5f7641..94f50eec9 100644 --- a/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts +++ b/test/e2e-browser/specs/code-checkpoint-workspace-external.spec.ts @@ -317,7 +317,8 @@ async function verifyPickerWorkspaceReopensForRewind( ) const beforeTurn = `${label} before reopened turn\n` await targetWrite(page, file, beforeTurn) - const turn = await sendThroughComposer(page, reopenedPane, `${label} reopened ${fixture.expectedNonce}`, selectedWorkspace) + const prompt = `${label} reopened ${fixture.expectedNonce}` + const turn = await sendThroughComposer(page, reopenedPane, prompt, selectedWorkspace) expect(turn.outcome).toMatchObject({ status: 'available', workspace: selectedWorkspace }) await targetWrite(page, file, `${label} changed after reopened turn\n`) @@ -325,7 +326,7 @@ async function verifyPickerWorkspaceReopensForRewind( const rewind = turn.row.getByRole('button', { name: 'Rewind code to here' }) await expect(rewind).toBeEnabled() const rewindNotice = expect(reopenedPane.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) - .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + .toContainText(`Code rewound to before: "${prompt}"`, { timeout: 30_000 }) const dialogAccepted = page.waitForEvent('dialog').then((dialog) => dialog.accept()) await rewind.click() await Promise.all([dialogAccepted, rewindNotice]) @@ -409,7 +410,7 @@ test('Git workspace sends retain per-turn success and failure, then restore the const rewind = duplicateRows.nth(0).getByRole('button', { name: 'Rewind code to here' }) await expect(rewind).toBeEnabled() const rewindNotice = expect(restoredPane.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) - .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + .toContainText(`Code rewound to before: "${samePrompt}"`, { timeout: 30_000 }) const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) await rewind.click() await Promise.all([dialogAccepted, rewindNotice]) @@ -509,7 +510,7 @@ test('opening a linked-worktree session from History preserves picker scope and const rewind = turn.row.getByRole('button', { name: 'Rewind code to here' }) await expect(rewind).toBeEnabled() const rewindNotice = expect(reopenedPane.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) - .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + .toContainText(`Code rewound to before: "${turnText}"`, { timeout: 30_000 }) const dialogAccepted = page.waitForEvent('dialog').then((dialog) => dialog.accept()) await rewind.click() await Promise.all([dialogAccepted, rewindNotice]) @@ -542,7 +543,7 @@ test('picker-confirmed non-Git workspace captures and restores all selected file await targetWrite(page, `${fixture.nonGitWorkspacePath}/two.txt`, 'two changed after send\n') await turn.row.hover() const rewindNotice = expect(paneRoot.getByRole('alert').filter({ hasText: /Code rewound to before:/ })) - .toContainText('Automatic pre-turn snapshot', { timeout: 30_000 }) + .toContainText(`Code rewound to before: "${marker}"`, { timeout: 30_000 }) const dialogAccepted = page.waitForEvent('dialog').then(dialog => dialog.accept()) await turn.row.getByRole('button', { name: 'Rewind code to here' }).click() await Promise.all([dialogAccepted, rewindNotice]) From a5c6f51548a7bcaf02e3f2017dad987764ab6264 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 15:08:21 -0700 Subject: [PATCH 470/493] test: classify managed runtime launch operation --- scripts/testing/managed-environment-create.ts | 30 +++++++++++++++++-- ...anaged-provider-environment-target.test.ts | 20 +++++++++++++ 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 61797a849..9864e6794 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -201,12 +201,36 @@ const runtimeErrorCodes = [ 'RecoveryWrongIdentity', 'RecoveryRetryBudget', 'RecoveryInProgress', 'RecoveryStopUnconfirmed', ] as const -export function summarizeRuntimeErrorCode(message: string): typeof runtimeErrorCodes[number] | undefined { +function summarizeRuntimeRejection(message: string): { code: typeof runtimeErrorCodes[number]; detail: string } | undefined { const prefix = 'runtime supervisor rejected request: ' const index = message.indexOf(prefix) if (index < 0) return undefined const rejection = message.slice(index + prefix.length) - return runtimeErrorCodes.find(code => rejection.startsWith(`${code}:`)) + const code = runtimeErrorCodes.find(candidate => rejection.startsWith(`${candidate}:`)) + return code ? { code, detail: rejection.slice(code.length + 2) } : undefined +} + +export function summarizeRuntimeErrorCode(message: string): typeof runtimeErrorCodes[number] | undefined { + return summarizeRuntimeRejection(message)?.code +} + +const runtimeErrorOperations = [ + ['resolve managed provider credentials:', 'credential_resolution'], + ['prepare OneCLI provider files:', 'provider_files'], + ['prepare hosted OneCLI provider files:', 'provider_files'], + ['prepare managed provider:', 'provider_launch'], + ['spawn managed terminal:', 'pty_spawn'], + ['prepare provider config:', 'provider_config'], + ['prepare hosted provider config:', 'provider_config'], + ['prepare managed provider state:', 'provider_state'], + ['prepare managed Amplifier state:', 'provider_state'], + ['prepare hosted fresh-agent provider state failed:', 'provider_state'], + ['transfer managed provider state:', 'provider_state_transfer'], +] as const + +export function summarizeRuntimeErrorOperation(message: string): typeof runtimeErrorOperations[number][1] | undefined { + const detail = summarizeRuntimeRejection(message)?.detail + return detail === undefined ? undefined : runtimeErrorOperations.find(([operation]) => detail.startsWith(operation))?.[1] } export function summarizeRuntimeErrorClass(message: string): @@ -342,6 +366,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, const errno = response.message.match(/\(os error (\d{1,4})\)/) const osError = errno ? Number(errno[1]) : undefined const runtimeErrorCode = summarizeRuntimeErrorCode(response.message) + const runtimeErrorOperation = summarizeRuntimeErrorOperation(response.message) const runtimeErrorClass = summarizeRuntimeErrorClass(response.message) const reason = failureReasons.find(value => response.message.includes(value)) ?? inputReason ?? (filesystemLabel && osError !== undefined ? `${filesystemLabel} filesystem error` : 'UNKNOWN') @@ -399,6 +424,7 @@ export function assertManagedTerminalCreated(owner: ManagedCreateOwner, severity: 'error', event: 'managed_environment.create_failed', code: code.success ? code.data : 'UNKNOWN', reason, ...(osError === undefined ? {} : { osError }), serverLogEvents, ...(runtimeErrorCode ? { runtimeErrorCode } : {}), + ...(runtimeErrorOperation ? { runtimeErrorOperation } : {}), ...(runtimeErrorClass ? { runtimeErrorClass } : {}), ...(response.providerPreparationFailure ? { providerPreparationFailure: summarizeProviderPreparationFailure(response.providerPreparationFailure) } diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 67b1e87c3..2c309018b 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -66,6 +66,26 @@ describe('managed provider environment target mode', () => { expect(JSON.stringify(incidents[0])).not.toContain('/private/') }) + it('records only a fixed runtime launch operation for managed host failures', () => { + const incidents: unknown[] = [] + const owner = { + web: { capturedOutput: () => ({ stdout: '', stderr: '' }), info: { logsDir: '/nonexistent/managed-test' } }, + runtime: { containerLogs: () => '', writeIncident: (_name: string, value: unknown) => incidents.push(value) }, + supervisor: { containerId: 'a'.repeat(64) }, + } + const response = { + type: 'error' as const, + code: 'PTY_SPAWN_FAILED' as const, + timestamp: new Date().toISOString(), + message: 'managed runtime launch failed: runtime supervisor rejected request: HostUnreachable: prepare provider config: /private/provider/opencode.json TOKEN_SENTINEL', + } + + expect(() => assertManagedTerminalCreated(owner, response)).toThrow() + expect(incidents[0]).toMatchObject({ runtimeErrorOperation: 'provider_config' }) + expect(JSON.stringify(incidents[0])).not.toContain('/private/') + expect(JSON.stringify(incidents[0])).not.toContain('TOKEN_SENTINEL') + }) + it('retains only bounded event names and levels from server logs', () => { const logs = [ JSON.stringify({ level: 'ERROR', event: 'provider.opencode.resource.failed', target: 'freshell_server::managed_runtime', From 99d51364f87c165b9c6561643fdd7ce85590b414 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 15:13:39 -0700 Subject: [PATCH 471/493] test: classify managed host failure operation --- scripts/testing/managed-environment-create.ts | 20 +++----------- test/helpers/provider-parity-environment.ts | 27 ++++++++++++++----- .../testing/runtime-test-broker.test.ts | 7 ++++- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 9864e6794..91e642d7f 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -4,7 +4,7 @@ import http from 'node:http' import path from 'node:path' import { ErrorCode, type ErrorMessage } from '../../shared/ws-protocol.js' import { dockerSocketPath } from './runtime-sandbox.js' -import { providerParityFailureDiagnostics } from '../../test/helpers/provider-parity-environment.js' +import { providerParityFailureDiagnostics, summarizeProviderFailureOperation } from '../../test/helpers/provider-parity-environment.js' import type { ManagedRuntimeBrowserRig } from '../../test/e2e-browser/helpers/managed-runtime.js' import type { ManagedEnvironmentFixture } from './e2e-external-target.js' @@ -214,23 +214,9 @@ export function summarizeRuntimeErrorCode(message: string): typeof runtimeErrorC return summarizeRuntimeRejection(message)?.code } -const runtimeErrorOperations = [ - ['resolve managed provider credentials:', 'credential_resolution'], - ['prepare OneCLI provider files:', 'provider_files'], - ['prepare hosted OneCLI provider files:', 'provider_files'], - ['prepare managed provider:', 'provider_launch'], - ['spawn managed terminal:', 'pty_spawn'], - ['prepare provider config:', 'provider_config'], - ['prepare hosted provider config:', 'provider_config'], - ['prepare managed provider state:', 'provider_state'], - ['prepare managed Amplifier state:', 'provider_state'], - ['prepare hosted fresh-agent provider state failed:', 'provider_state'], - ['transfer managed provider state:', 'provider_state_transfer'], -] as const - -export function summarizeRuntimeErrorOperation(message: string): typeof runtimeErrorOperations[number][1] | undefined { +export function summarizeRuntimeErrorOperation(message: string) { const detail = summarizeRuntimeRejection(message)?.detail - return detail === undefined ? undefined : runtimeErrorOperations.find(([operation]) => detail.startsWith(operation))?.[1] + return detail === undefined ? undefined : summarizeProviderFailureOperation(detail) } export function summarizeRuntimeErrorClass(message: string): diff --git a/test/helpers/provider-parity-environment.ts b/test/helpers/provider-parity-environment.ts index f3e041c20..995e65073 100644 --- a/test/helpers/provider-parity-environment.ts +++ b/test/helpers/provider-parity-environment.ts @@ -19,6 +19,25 @@ export function prepareProviderParityUserRoots(home: string, xdgConfigHome: stri fs.mkdirSync(path.join(xdgConfigHome, 'opencode'), { recursive: true }) } +const providerFailureOperations = [ + ['provider transaction backup ', 'backup'], ['provider transaction install ', 'install'], + ['provider transaction create ', 'create'], ['provider transaction remove directory ', 'remove_directory'], + ['provider transaction publish manifest:', 'manifest'], ['provider transaction mounted file ', 'mounted_file'], + ['provider refresh control ', 'control'], ['provider refresh transaction creation ', 'create_stage'], + ['resolve managed provider credentials:', 'credential_resolution'], + ['prepare OneCLI provider files:', 'provider_files'], ['prepare hosted OneCLI provider files:', 'provider_files'], + ['prepare managed provider:', 'provider_launch'], ['spawn managed terminal:', 'pty_spawn'], + ['prepare provider config:', 'provider_config'], ['prepare hosted provider config:', 'provider_config'], + ['prepare managed provider state:', 'provider_state'], ['prepare managed Amplifier state:', 'provider_state'], + ['prepare hosted fresh-agent provider state failed:', 'provider_state'], + ['transfer managed provider state:', 'provider_state_transfer'], +] as const + +export function summarizeProviderFailureOperation(message: string) { + return providerFailureOperations.find(([prefix]) => + message.startsWith(prefix) || message.includes(`: ${prefix}`))?.[1] +} + /** Retain only fixed classifications from the owned host's existing logs. */ export function providerParityFailureDiagnostics(logs: string) { const events = logs.split('\n').flatMap(line => { @@ -29,19 +48,13 @@ export function providerParityFailureDiagnostics(logs: string) { }) const classify = (cause: unknown) => { const message = typeof cause === 'string' ? cause : '' - const operations = [ - ['provider transaction backup ', 'backup'], ['provider transaction install ', 'install'], - ['provider transaction create ', 'create'], ['provider transaction remove directory ', 'remove_directory'], - ['provider transaction publish manifest:', 'manifest'], ['provider transaction mounted file ', 'mounted_file'], - ['provider refresh control ', 'control'], ['provider refresh transaction creation ', 'create_stage'], - ] as const const errors = [ ['(os error 18)', 'cross_device'], ['(os error 13)', 'permission_denied'], ['(os error 2)', 'missing'], ['(os error 30)', 'read_only'], ['(os error 16)', 'busy'], ['(os error 1)', 'operation_not_permitted'], ] as const return { - operation: operations.find(([prefix]) => message.startsWith(prefix))?.[1] ?? 'unknown', + operation: summarizeProviderFailureOperation(message) ?? 'unknown', errno: errors.find(([marker]) => message.includes(marker))?.[1] ?? 'unknown', } } diff --git a/test/unit/tooling/testing/runtime-test-broker.test.ts b/test/unit/tooling/testing/runtime-test-broker.test.ts index 71d21e410..aeaaabfa7 100644 --- a/test/unit/tooling/testing/runtime-test-broker.test.ts +++ b/test/unit/tooling/testing/runtime-test-broker.test.ts @@ -80,12 +80,17 @@ it('classifies owned provider publication failures without retaining error bodie resource_source_config_scope: 'provider_config', limit_kind: 'entries', observed: 100_001, permitted: 100_000 }), JSON.stringify({ event: 'session_host.fatal', error: `provider refresh control ${secret}` }), + JSON.stringify({ event: 'session_host.fatal', error: `runtime: HostUnreachable: prepare provider config: /private/${secret}.json: Permission denied (os error 13)` }), JSON.stringify({ event: 'session_host.fatal', error: secret }), ].join('\n') const diagnostic = providerParityFailureDiagnostics(logs) expect(diagnostic).toMatchObject({ transactionFailures: [{ stage: 'publish', operation: 'install', errno: 'cross_device' }], - workerFailures: [{ operation: 'control', errno: 'unknown' }, { operation: 'unknown', errno: 'unknown' }], + workerFailures: [ + { operation: 'control', errno: 'unknown' }, + { operation: 'provider_config', errno: 'permission_denied' }, + { operation: 'unknown', errno: 'unknown' }, + ], }) expect(diagnostic.transactionEvents).toEqual([ { event: 'provider.projection.transaction.started', stage: 'stage' }, From 7875ec310daed3819eaf183d2f17f0724fbcfd81 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 15:20:32 -0700 Subject: [PATCH 472/493] fix: follow managed provider reference links --- crates/freshell-session-host/src/main.rs | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index 6f0512651..cf336a63b 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -3973,7 +3973,7 @@ fn preflight_provider_reference_aliases( Err(_) => return Err("provider reference alias target is unavailable".into()), } } - if !std::fs::symlink_metadata(&reference).is_ok_and(|metadata| metadata.is_file()) { + if !std::fs::metadata(&reference).is_ok_and(|metadata| metadata.is_file()) { return Err("provider reference alias target is unavailable".into()); } if !provider_reference_alias_parent_available(home, &relative, current)? { @@ -5308,9 +5308,12 @@ mod tests { std::fs::create_dir_all(&workspace).unwrap(); std::fs::create_dir_all(provider_home.join(".freshell/references")).unwrap(); std::fs::write(resource_stage.join("server.mjs"), "console.log('ready')\n").unwrap(); - std::fs::write( + let private_reference = owned.path().join("private-reference-0"); + std::fs::write(&private_reference, "MANAGED_ALIAS_SECRET=private-value\n").unwrap(); + // Indexed configuration references point into the read-only secret mount at runtime. + std::os::unix::fs::symlink( + &private_reference, provider_home.join(".freshell/references/0"), - "MANAGED_ALIAS_SECRET=private-value\n", ) .unwrap(); let mut mount = opencode_resource_mount( @@ -5357,6 +5360,12 @@ mod tests { std::fs::read_to_string(provider_home.join(".freshell/references/0")).unwrap(), "MANAGED_ALIAS_SECRET=private-value\n" ); + assert!( + std::fs::symlink_metadata(provider_home.join(".freshell/references/0")) + .unwrap() + .file_type() + .is_symlink() + ); assert_eq!( std::fs::read_link(&alias).unwrap(), PathBuf::from("/home/freshell/provider/.freshell/references/0") From 95bb1ad4b8fb1ad173257a9784732e873d57c484 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 15:50:17 -0700 Subject: [PATCH 473/493] test: capture managed recovery failure diagnostics --- scripts/testing/managed-environment-create.ts | 65 +++++++++++++++++++ .../managed-provider-environment-target.ts | 26 +++++++- .../tooling/testing/runtime-sandbox.test.ts | 39 +++++++++++ 3 files changed, 127 insertions(+), 3 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 91e642d7f..86bf500a3 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -41,6 +41,71 @@ export function writeManagedEnvironmentBrowserFailureIncident( } } +/** Select fixed lifecycle fields for a failed replacement without retaining raw host errors. */ +export function summarizeManagedEnvironmentRecoveryFailure( + lifecycle: string, + soulId: string, + priorIncarnationId: string, +) { + const containerByIncarnation = new Map<string, string>() + let recovery: Record<string, string> | undefined + const activations = new Map<string, Record<string, string>>() + const identity = (value: unknown) => typeof value === 'string' + && /^[a-z0-9][a-z0-9-]{0,127}$/i.test(value) ? value : undefined + const fixed = (value: unknown, allowed: readonly string[]) => + typeof value === 'string' && allowed.includes(value) ? value : 'UNKNOWN' + + for (const line of lifecycle.split('\n')) { + try { + const entry = JSON.parse(line) + const raw = entry?.event === 'supervisor.lifecycle' ? entry.data?.raw : undefined + const data = raw?.data + if (!raw || !data || typeof data !== 'object') continue + + if (raw.event === 'supervisor.host_started') { + const incarnationId = identity(data.incarnationId) + if (incarnationId && typeof data.containerId === 'string' && /^[a-f0-9]{64}$/.test(data.containerId)) { + containerByIncarnation.set(incarnationId, data.containerId) + } + } else if (data.soulId === soulId && raw.event === 'supervisor.activation_failed') { + const incarnationId = identity(data.incarnationId) + if (!incarnationId || incarnationId === priorIncarnationId) continue + activations.set(incarnationId, { + incarnationId, + stage: fixed(data.stage, ['authenticate_started_host', 'deliver_execution_grant', 'commit_execution_grant', + 'enable_long_lived_runtime', 'commit_running_state', 'commit_provider_inputs', 'record_native_identity']), + errorCode: fixed(data.errorCode, ['HOST_UNREACHABLE', 'HOST_AUTHENTICATION_FAILED', 'INVALID_REQUEST', + 'PROVIDER_INPUTS_CHANGED', 'BACKEND_UNAVAILABLE', 'REGISTRY_FAILURE', 'OWNERSHIP_MISMATCH']), + errorClass: fixed(data.errorClass, ['runtime_activation_failed', 'provider_activation_failed']), + launchState: fixed(data.launchState, ['created', 'starting', 'running', 'failed', 'stopping', 'stopped']), + }) + } else if (data.soulId === soulId && raw.event === 'supervisor.recovery.activation_failed' + && data.priorIncarnationId === priorIncarnationId) { + const replacementIncarnationId = identity(data.replacementIncarnationId) + if (!replacementIncarnationId) continue + recovery = { + priorIncarnationId, + replacementIncarnationId, + errorCode: fixed(data.errorCode, ['HOST_UNREACHABLE', 'HOST_AUTHENTICATION_FAILED', 'INVALID_REQUEST', + 'PROVIDER_INPUTS_CHANGED', 'BACKEND_UNAVAILABLE', 'REGISTRY_FAILURE', 'OWNERSHIP_MISMATCH']), + errorClass: fixed(data.errorClass, ['runtime_activation_failed', 'provider_activation_failed']), + blockReason: fixed(data.blockReason, ['PROVIDER_UNAVAILABLE', 'RECOVERY_UNAVAILABLE', 'CHECKPOINT_UNAVAILABLE']), + } + } + } catch { /* Ignore malformed lifecycle records and retain only valid evidence. */ } + } + + const replacementIncarnationId = recovery?.replacementIncarnationId + ?? [...activations.keys()].at(-1) + const activation = replacementIncarnationId ? activations.get(replacementIncarnationId) : undefined + const hostContainerId = replacementIncarnationId ? containerByIncarnation.get(replacementIncarnationId) : undefined + return { + ...(recovery ? { recovery } : {}), + ...(activation ? { activation } : {}), + ...(hostContainerId ? { hostContainerId } : {}), + } +} + /** * Wait for a healthy OpenCode child without trapping a failed browser attempt * behind the longer child-startup deadline. Playwright writes the matching diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 43d0295b5..60a37cea4 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -16,7 +16,7 @@ import type { InitialProviderVolumeAuthorization } from './runtime-test-broker.j import { resolvePairedExternalFixture } from './e2e-external-target.js' import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, serviceManagedEnvironmentCases, summarizeRustServerLogEvents, waitForManagedEnvironmentReadyOrBrowserFailure, - writeManagedEnvironmentBrowserFailureIncident } from './managed-environment-create.js' + summarizeManagedEnvironmentRecoveryFailure, writeManagedEnvironmentBrowserFailureIncident } from './managed-environment-create.js' import { waitForCloudExecutionDispatch } from './cloud-execution-dispatch.js' import { ManagedEnvironmentBrowserSummaryCapture, type ManagedEnvironmentBrowserSummary } from './managed-environment-browser-summary.js' @@ -1754,8 +1754,28 @@ export async function runManagedEnvironmentTarget() { assert(original?.containerId) await fs.rename(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'), path.join(root, 'initial-child-receipt.json')) rig.runtime.stopOwnedRuntimeExact(original.containerId) - const replacement = await poll('current-owner replacement incarnation', async () => (await rig!.inventory()).find(row => - row.soulId === original.soulId && row.incarnationId !== original.incarnationId && row.launchState === 'running'), 300_000) + let replacement: ManagedRuntimeView + try { + replacement = await poll('current-owner replacement incarnation', async () => (await rig!.inventory()).find(row => + row.soulId === original.soulId && row.incarnationId !== original.incarnationId && row.launchState === 'running'), 300_000) + } catch (error) { + try { + const lifecycle = await fs.readFile(path.join(rig.runtime.evidenceDir, 'lifecycle.jsonl'), 'utf8') + const { hostContainerId, ...safeFailure } = summarizeManagedEnvironmentRecoveryFailure( + lifecycle, original.soulId, original.incarnationId) + const supervisorLogs = providerParityFailureDiagnostics( + rig.runtime.containerLogs(rig.supervisor.containerId).slice(-512 * 1024)) + const sessionHostLogs = hostContainerId && rig.runtime.broker?.receiptIds().has(hostContainerId) + ? providerParityFailureDiagnostics(rig.runtime.containerLogs(hostContainerId).slice(-512 * 1024)) + : 'UNKNOWN' + rig.runtime.writeIncident('managed-environment-replacement-failure', { + severity: 'error', event: 'managed_environment.replacement_failed', + soulId: original.soulId, priorIncarnationId: original.incarnationId, + failure: safeFailure, supervisorLogs, sessionHostLogs, + }) + } catch { /* Diagnostic capture must not hide the acceptance failure. */ } + throw error + } const replacementChild = await poll('replacement Bun child', async () => readJson(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'))) const replacementStage = await stageProof(rig, replacement, nonce) assert.equal(replacement.nativeSessionId, original.nativeSessionId) diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index 93f47fb9f..d92c10e75 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -39,6 +39,7 @@ import { assertManagedTerminalCreated, observeManagedEnvironmentNativeIdentity, readManagedEnvironmentDaemonId, + summarizeManagedEnvironmentRecoveryFailure, serviceManagedEnvironmentScenario, serviceManagedEnvironmentCases, waitForManagedEnvironmentReadyOrBrowserFailure, @@ -121,6 +122,44 @@ describe('managed environment browser attempt lifetime', () => { })) }) + it('extracts only safe replacement activation evidence and its owned host id', () => { + const secretPath = '/provider/.config/opencode/private.json' + const lifecycle = [ + { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.host_started', data: { + incarnationId: 'replacement-incarnation', containerId: 'a'.repeat(64), + } } } }, + { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.activation_failed', data: { + soulId: 'owned-soul', incarnationId: 'replacement-incarnation', stage: 'deliver_execution_grant', + errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'starting', + errorMessage: `provider config failed at ${secretPath}`, + } } } }, + { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.recovery.activation_failed', data: { + soulId: 'owned-soul', priorIncarnationId: 'prior-incarnation', + replacementIncarnationId: 'replacement-incarnation', errorCode: 'HOST_UNREACHABLE', + errorClass: 'provider_activation_failed', blockReason: 'PROVIDER_UNAVAILABLE', + } } } }, + { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.activation_failed', data: { + soulId: 'foreign-soul', incarnationId: 'foreign-incarnation', stage: secretPath, + errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'starting', + } } } }, + ].map(entry => JSON.stringify(entry)).join('\n') + + const evidence = summarizeManagedEnvironmentRecoveryFailure(lifecycle, 'owned-soul', 'prior-incarnation') + + expect(evidence).toEqual({ + recovery: { + priorIncarnationId: 'prior-incarnation', replacementIncarnationId: 'replacement-incarnation', + errorCode: 'HOST_UNREACHABLE', errorClass: 'provider_activation_failed', blockReason: 'PROVIDER_UNAVAILABLE', + }, + activation: { + incarnationId: 'replacement-incarnation', stage: 'deliver_execution_grant', + errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'starting', + }, + hostContainerId: 'a'.repeat(64), + }) + expect(JSON.stringify(evidence)).not.toContain(secretPath) + }) + it('keeps the owned target alive until the browser reports attempt completion', async () => { await ownedFixture(async directory => { await fs.writeFile(path.join(directory, 'request.json'), JSON.stringify(request(0))) From 5d18279a097dcbe6a75e988d8e7922a042c3bdc4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 16:16:27 -0700 Subject: [PATCH 474/493] test: capture managed recovery failure details --- scripts/testing/managed-environment-create.ts | 13 ++++++++----- .../testing/managed-provider-environment-target.ts | 2 +- test/unit/tooling/testing/runtime-sandbox.test.ts | 14 +++++++------- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/scripts/testing/managed-environment-create.ts b/scripts/testing/managed-environment-create.ts index 86bf500a3..ffb66f402 100644 --- a/scripts/testing/managed-environment-create.ts +++ b/scripts/testing/managed-environment-create.ts @@ -58,7 +58,7 @@ export function summarizeManagedEnvironmentRecoveryFailure( for (const line of lifecycle.split('\n')) { try { const entry = JSON.parse(line) - const raw = entry?.event === 'supervisor.lifecycle' ? entry.data?.raw : undefined + const raw = entry?.event === 'supervisor.lifecycle' ? entry.data?.raw : entry const data = raw?.data if (!raw || !data || typeof data !== 'object') continue @@ -72,12 +72,11 @@ export function summarizeManagedEnvironmentRecoveryFailure( if (!incarnationId || incarnationId === priorIncarnationId) continue activations.set(incarnationId, { incarnationId, - stage: fixed(data.stage, ['authenticate_started_host', 'deliver_execution_grant', 'commit_execution_grant', - 'enable_long_lived_runtime', 'commit_running_state', 'commit_provider_inputs', 'record_native_identity']), + stage: fixed(data.stage, activationStages), errorCode: fixed(data.errorCode, ['HOST_UNREACHABLE', 'HOST_AUTHENTICATION_FAILED', 'INVALID_REQUEST', 'PROVIDER_INPUTS_CHANGED', 'BACKEND_UNAVAILABLE', 'REGISTRY_FAILURE', 'OWNERSHIP_MISMATCH']), errorClass: fixed(data.errorClass, ['runtime_activation_failed', 'provider_activation_failed']), - launchState: fixed(data.launchState, ['created', 'starting', 'running', 'failed', 'stopping', 'stopped']), + launchState: fixed(data.launchState, ['prepared', 'created', 'starting', 'running', 'failed', 'stopping', 'stopped']), }) } else if (data.soulId === soulId && raw.event === 'supervisor.recovery.activation_failed' && data.priorIncarnationId === priorIncarnationId) { @@ -89,7 +88,11 @@ export function summarizeManagedEnvironmentRecoveryFailure( errorCode: fixed(data.errorCode, ['HOST_UNREACHABLE', 'HOST_AUTHENTICATION_FAILED', 'INVALID_REQUEST', 'PROVIDER_INPUTS_CHANGED', 'BACKEND_UNAVAILABLE', 'REGISTRY_FAILURE', 'OWNERSHIP_MISMATCH']), errorClass: fixed(data.errorClass, ['runtime_activation_failed', 'provider_activation_failed']), - blockReason: fixed(data.blockReason, ['PROVIDER_UNAVAILABLE', 'RECOVERY_UNAVAILABLE', 'CHECKPOINT_UNAVAILABLE']), + blockReason: fixed(data.blockReason, ['CAPABILITY_PENDING', 'CREDENTIALS_EXPIRED', 'RATE_LIMITED', + 'PROVIDER_UNAVAILABLE', 'STORE_UNREADABLE', 'STORE_MISSING', 'WORKSPACE_UNAVAILABLE', + 'INCOMPATIBLE_BINARY', 'UNSUPPORTED_PROTOCOL', 'AMBIGUOUS_IDENTITY', 'IMPLEMENTATION_UNAVAILABLE', + 'INSUFFICIENT_RESOURCES', 'RETRY_BUDGET', 'STOP_INTENT', 'OLD_RUNTIME_NOT_EMPTY', + 'WRONG_NATIVE_IDENTITY', 'COMMAND_AMBIGUOUS']), } } } catch { /* Ignore malformed lifecycle records and retain only valid evidence. */ } diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 60a37cea4..5af44951f 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1760,7 +1760,7 @@ export async function runManagedEnvironmentTarget() { row.soulId === original.soulId && row.incarnationId !== original.incarnationId && row.launchState === 'running'), 300_000) } catch (error) { try { - const lifecycle = await fs.readFile(path.join(rig.runtime.evidenceDir, 'lifecycle.jsonl'), 'utf8') + const lifecycle = await fs.readFile(path.join(path.dirname(rig.supervisor.runtimeRoot), 'evidence', 'lifecycle.jsonl'), 'utf8') const { hostContainerId, ...safeFailure } = summarizeManagedEnvironmentRecoveryFailure( lifecycle, original.soulId, original.incarnationId) const supervisorLogs = providerParityFailureDiagnostics( diff --git a/test/unit/tooling/testing/runtime-sandbox.test.ts b/test/unit/tooling/testing/runtime-sandbox.test.ts index d92c10e75..440c35416 100644 --- a/test/unit/tooling/testing/runtime-sandbox.test.ts +++ b/test/unit/tooling/testing/runtime-sandbox.test.ts @@ -129,31 +129,31 @@ describe('managed environment browser attempt lifetime', () => { incarnationId: 'replacement-incarnation', containerId: 'a'.repeat(64), } } } }, { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.activation_failed', data: { - soulId: 'owned-soul', incarnationId: 'replacement-incarnation', stage: 'deliver_execution_grant', - errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'starting', + soulId: 'owned-soul', incarnationId: 'replacement-incarnation', stage: 'create_stopped_runtime', + errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'prepared', errorMessage: `provider config failed at ${secretPath}`, } } } }, { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.recovery.activation_failed', data: { soulId: 'owned-soul', priorIncarnationId: 'prior-incarnation', replacementIncarnationId: 'replacement-incarnation', errorCode: 'HOST_UNREACHABLE', - errorClass: 'provider_activation_failed', blockReason: 'PROVIDER_UNAVAILABLE', + errorClass: 'provider_activation_failed', blockReason: 'RETRY_BUDGET', } } } }, { event: 'supervisor.lifecycle', data: { raw: { event: 'supervisor.activation_failed', data: { soulId: 'foreign-soul', incarnationId: 'foreign-incarnation', stage: secretPath, errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'starting', } } } }, - ].map(entry => JSON.stringify(entry)).join('\n') + ].map(entry => JSON.stringify(entry.data.raw)).join('\n') const evidence = summarizeManagedEnvironmentRecoveryFailure(lifecycle, 'owned-soul', 'prior-incarnation') expect(evidence).toEqual({ recovery: { priorIncarnationId: 'prior-incarnation', replacementIncarnationId: 'replacement-incarnation', - errorCode: 'HOST_UNREACHABLE', errorClass: 'provider_activation_failed', blockReason: 'PROVIDER_UNAVAILABLE', + errorCode: 'HOST_UNREACHABLE', errorClass: 'provider_activation_failed', blockReason: 'RETRY_BUDGET', }, activation: { - incarnationId: 'replacement-incarnation', stage: 'deliver_execution_grant', - errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'starting', + incarnationId: 'replacement-incarnation', stage: 'create_stopped_runtime', + errorCode: 'HOST_UNREACHABLE', errorClass: 'runtime_activation_failed', launchState: 'prepared', }, hostContainerId: 'a'.repeat(64), }) From 434632dcec69ccd47cfb46d4dac8fd1345e0f5ca Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 16:16:32 -0700 Subject: [PATCH 475/493] fix: trust durable checkpoint history on reload --- src/components/fresh-agent/FreshAgentView.tsx | 5 ++--- .../components/fresh-agent/FreshAgentView.test.tsx | 10 +++++++--- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/src/components/fresh-agent/FreshAgentView.tsx b/src/components/fresh-agent/FreshAgentView.tsx index 6106c74b9..487cd15c4 100644 --- a/src/components/fresh-agent/FreshAgentView.tsx +++ b/src/components/fresh-agent/FreshAgentView.tsx @@ -1284,9 +1284,8 @@ export function FreshAgentView({ const availability = existing ? (() => { const saved = checkpointOutcomeAvailability(existing) - const savedRestoreFailure = saved.status === 'unavailable' - && saved.code === 'workspace_unavailable' - const merged = savedRestoreFailure && fetched.status === 'available' + const savedFailure = saved.status === 'unavailable' + const merged = savedFailure && fetched.status === 'available' ? fetched : { ...fetched, ...saved } return { diff --git a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx index c17dfe1bb..46758a526 100644 --- a/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx +++ b/test/unit/client/components/fresh-agent/FreshAgentView.test.tsx @@ -3873,7 +3873,11 @@ describe('FreshAgentView', () => { ])) }) - it('recovers a stale workspace restore error from the authoritative checkpoint history', async () => { + it.each([ + ['workspace_unavailable', 'The selected project workspace is temporarily unavailable.'], + ['checkpoint_outcome_missing', 'The server did not confirm a code checkpoint for this message.'], + ['checkpoint_not_found', 'The server could not find the requested code checkpoint.'], + ] as const)('recovers a stale %s error from authoritative checkpoint history', async (staleCode, staleReason) => { const sessionId = 'thread-recovered-checkpoint-error' const requestId = 'request-stale-workspace-error' const nativeTurnId = 'native-recovered-checkpoint-error' @@ -3929,8 +3933,8 @@ describe('FreshAgentView', () => { nativeTurnId, displayTurnId, status: 'unavailable', - code: 'workspace_unavailable', - reason: 'The selected project workspace is temporarily unavailable.', + code: staleCode, + reason: staleReason, workspace, }], } as never, From 98de86243ff580d0bc1b828132a52b4f24e913d2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 16:16:38 -0700 Subject: [PATCH 476/493] fix: preserve configured aliases during recovery --- .../src/managed_mcp_capability.rs | 1 + .../managed_mcp_capability/source_stage.rs | 52 +++++++++++++++---- 2 files changed, 42 insertions(+), 11 deletions(-) diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index e1fa610d4..f62991920 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1702,6 +1702,7 @@ fn stage_provider_root( let retained = source_stage::merge_recovery_resource_mounts( &mut units, opencode_input.prior_resource_mounts.unwrap_or_default(), + plan, )?; if retained.is_empty() { None diff --git a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs index 129ef94c0..522d653f7 100644 --- a/crates/freshell-server/src/managed_mcp_capability/source_stage.rs +++ b/crates/freshell-server/src/managed_mcp_capability/source_stage.rs @@ -2416,6 +2416,18 @@ mod configured_resource_tests { None, ) .unwrap(); + let mut previous_mount = units + .iter() + .find(|(_, mount)| mount.logical_destination == external.to_string_lossy()) + .expect("the prior generation must contain the configured MCP cwd") + .1 + .clone(); + previous_mount.reference_aliases = + vec![freshell_runtime_protocol::ProviderResourceReferenceAlias { + relative_path: ".env".into(), + reference_index: 1, + }]; + merge_recovery_resource_mounts(&mut units, &[previous_mount], &plan).unwrap(); let scratch = owned.path().join("scratch"); std::fs::create_dir_all(&scratch).unwrap(); let excluded = plan @@ -2454,6 +2466,10 @@ mod configured_resource_tests { reference_index: 0, }] ); + assert!( + resource_mount.1.retired_reference_aliases.is_empty(), + "a currently configured alias must not also be retired during recovery" + ); let staged_root = owned.path().join("filtered-stage"); let selections = resource_copy_selections(&units).unwrap(); @@ -3241,6 +3257,7 @@ process.stdout.write(require('./helper.js')) .iter() .map(|(_, mount)| mount.clone()) .collect::<Vec<_>>(), + &[], ) .unwrap(); let state_mount = units @@ -6087,7 +6104,8 @@ process.stdout.write(require('./helper.js')) }, )]; let retained = - merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)).unwrap(); + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous), &[]) + .unwrap(); assert_eq!( current[0].1.stage_relative_path, previous.stage_relative_path @@ -6104,9 +6122,12 @@ process.stdout.write(require('./helper.js')) ); let mut no_current_roots = Vec::new(); - let retained = - merge_recovery_resource_mounts(&mut no_current_roots, std::slice::from_ref(&previous)) - .unwrap(); + let retained = merge_recovery_resource_mounts( + &mut no_current_roots, + std::slice::from_ref(&previous), + &[], + ) + .unwrap(); assert_eq!(retained.len(), 1); assert!( retained[0].reference_aliases.is_empty(), @@ -6146,7 +6167,7 @@ process.stdout.write(require('./helper.js')) mount(aliases(1), Vec::new()), )]; - merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)).unwrap(); + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous), &[]).unwrap(); assert_eq!(current[0].1.reference_aliases, aliases(1)); assert!( @@ -6183,7 +6204,7 @@ process.stdout.write(require('./helper.js')) mount(Vec::new(), Vec::new()), )]; - merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)).unwrap(); + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous), &[]).unwrap(); assert_eq!( current[0].1.retired_reference_aliases, @@ -6223,7 +6244,7 @@ process.stdout.write(require('./helper.js')) mount("current", current_destination), )]; let error = - merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous)) + merge_recovery_resource_mounts(&mut current, std::slice::from_ref(&previous), &[]) .expect_err("nested directory mount roots must preserve the prior generation"); assert!(error.contains("overlapping") && error.contains("reprepare")); } @@ -6233,9 +6254,12 @@ process.stdout.write(require('./helper.js')) "/synthetic/current-resource".into(), mount("current", "/home/freshell/provider/.config/opencode-extra"), )]; - let retained = - merge_recovery_resource_mounts(&mut prefix_sibling, std::slice::from_ref(&previous)) - .unwrap(); + let retained = merge_recovery_resource_mounts( + &mut prefix_sibling, + std::slice::from_ref(&previous), + &[], + ) + .unwrap(); assert_eq!(retained, vec![previous]); } @@ -6260,7 +6284,8 @@ process.stdout.write(require('./helper.js')) let mut aliases = vec![(source, mount("/workspace/current"))]; let prior = mount("/home/freshell/provider/.config/opencode"); let retained = - merge_recovery_resource_mounts(&mut aliases, std::slice::from_ref(&prior)).unwrap(); + merge_recovery_resource_mounts(&mut aliases, std::slice::from_ref(&prior), &[]) + .unwrap(); assert_eq!(retained, vec![prior]); let source = current_source_for_resource_mount(&aliases, &retained[0]) .expect("the selected alias already supplies this physical subpath"); @@ -6457,10 +6482,15 @@ pub(super) fn merge_recovery_resource_mounts( freshell_runtime_protocol::ProviderResourceMount, )], previous: &[freshell_runtime_protocol::ProviderResourceMount], + plan: &[PlannedSourceOccurrence], ) -> Result<Vec<freshell_runtime_protocol::ProviderResourceMount>, String> { use freshell_runtime_protocol::{ProviderResourceKind, ProviderResourceMount}; const PROVIDER_HOME: &str = "/home/freshell/provider"; + // Retirement must be computed against this generation's configured + // aliases, before it is compared with aliases from the previous grant. + attach_configured_resource_reference_aliases(current, plan)?; + fn paired_resource_paths(mount: &ProviderResourceMount) -> bool { let Some(stage) = mount .stage_relative_path From 5bc5f207c51bfb93624cb1888c42fd416698764b Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 18:56:19 -0700 Subject: [PATCH 477/493] test: stabilize checkpoint lock timeout --- crates/freshell-server/src/checkpoints.rs | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 3c536cb92..e02f8992f 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -233,6 +233,7 @@ impl Drop for CaptureRequestCancellation { } } +#[derive(Debug)] enum ExplicitCaptureError { WorkspaceUnavailable(CheckpointWorkspaceUnavailable), ResolutionTimeout(String), @@ -8422,7 +8423,10 @@ mod tests { let home = tempfile::tempdir().unwrap(); let workspace = tempfile::tempdir().unwrap(); let service = checkpoint_test_service(home.path()); - *service.test_explicit_budget_override.lock().unwrap() = Some(Duration::from_millis(50)); + // The explicit budget includes workspace resolution as well as lock + // acquisition. Leave headroom for resolution under parallel test load + // so this test exercises the held lock, not scheduler delay. + *service.test_explicit_budget_override.lock().unwrap() = Some(Duration::from_secs(2)); service.establish_cooldown(workspace.path()); let before = service.cooldowns.lock().unwrap()[workspace.path()].expires_at; let holder = service @@ -8443,7 +8447,10 @@ mod tests { ) .await .expect_err("explicit lock wait should time out"); - assert!(matches!(error, ExplicitCaptureError::WorkspaceBusy(_))); + assert!( + matches!(error, ExplicitCaptureError::WorkspaceBusy(_)), + "expected the held workspace lock to time out, got {error:?}" + ); assert_eq!( service.cooldowns.lock().unwrap()[workspace.path()].expires_at, before From f490d893c7a76ce1550a8231769b95d5014fcaec Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 18:56:23 -0700 Subject: [PATCH 478/493] test: verify provider aliases after recovery --- .../managed-provider-environment-target.ts | 24 +++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 5af44951f..255bf03f7 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -161,7 +161,8 @@ const workspaceCwdAliasMcpServerSource = String.raw`import fs from 'node:fs' import helper from './helper.mjs' const [receiptPath, expectedNonce] = process.argv.slice(2) if (!receiptPath || !expectedNonce) process.exit(12) -fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper, +fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper, terminalId: process.env.FRESHELL_TERMINAL_ID, + pid: process.pid, createdAt: new Date().toISOString(), envValueLoaded: process.env.FRESHELL_MANAGED_CWD_ALIAS_ENV === expectedNonce, envFileTarget: fs.realpathSync('.env'), envFileOperand: process.execArgv.find(value => value.startsWith('--env-file=')), @@ -1753,6 +1754,11 @@ export async function runManagedEnvironmentTarget() { const original = (await rig.inventory()).find(row => row.soulId === healthy.soulId) assert(original?.containerId) await fs.rename(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'), path.join(root, 'initial-child-receipt.json')) + const cwdAliasReceiptPath = path.join(selected.workspace, 'healthy/.managed-fixture/mcp-cwd-alias-receipt.json') + const initialCwdAliasReceipt = await readJson(cwdAliasReceiptPath) + assert.equal(initialCwdAliasReceipt?.expectedNonce, nonce, 'the original host must start the workspace-CWD MCP alias') + await fs.rename(cwdAliasReceiptPath, path.join(root, 'initial-cwd-alias-mcp-receipt.json')) + await fs.rm(`${cwdAliasReceiptPath}.started`, { force: true }) rig.runtime.stopOwnedRuntimeExact(original.containerId) let replacement: ManagedRuntimeView try { @@ -1777,6 +1783,8 @@ export async function runManagedEnvironmentTarget() { throw error } const replacementChild = await poll('replacement Bun child', async () => readJson(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'))) + const replacementCwdAliasMcp = await poll('replacement OpenCode starts the workspace-CWD MCP alias', async () => + readJson(cwdAliasReceiptPath)) const replacementStage = await stageProof(rig, replacement, nonce) assert.equal(replacement.nativeSessionId, original.nativeSessionId) assert.equal(replacementChild.bun, true) @@ -1786,8 +1794,20 @@ export async function runManagedEnvironmentTarget() { assert.equal(replacementChild.runId, rig.runtime.runId) assert.equal(replacementChild.expectedNonce, nonce) assert.equal(replacementChild.executableMode, '0711') + assert.equal(replacementCwdAliasMcp.expectedNonce, nonce) + assert.equal(replacementCwdAliasMcp.helper, nonce) + assert.equal(replacementCwdAliasMcp.terminalId, replacement.terminalId) + assert(Number.isSafeInteger(replacementCwdAliasMcp.pid) && replacementCwdAliasMcp.pid > 1) + assert.equal(replacementCwdAliasMcp.envValueLoaded, true) + assert.match(replacementCwdAliasMcp.envFileTarget, /^\/run\/freshell-secrets\/provider-\d+$/) + assert.equal(replacementCwdAliasMcp.envFileOperand, '--env-file=.env') + assert.equal(replacementCwdAliasMcp.cwd, selected.cwdAliasProject) + assert(replacementCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) + assert.equal(await fs.readFile(`${cwdAliasReceiptPath}.started`, 'utf8'), 'wrapper-started') proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, - soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, child: replacementChild } + soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, + child: replacementChild, workspaceCwdAliasMcp: { ...replacementCwdAliasMcp, + associatedIncarnationId: replacement.incarnationId } } await writeJson(path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json'), proof) const browserSummary = proof.cloudBrowser as ManagedEnvironmentBrowserSummary console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.acceptance_passed', candidateSha: fixture.candidateSha, From 39acdcd520ac2093d07526ffbf47801e7bbcbf33 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 19:10:40 -0700 Subject: [PATCH 479/493] test: bind recovery MCP receipt to replacement process --- .../managed-provider-environment-target.ts | 42 +++++++++++++- ...anaged-provider-environment-target.test.ts | 58 ++++++++++++++++++- 2 files changed, 97 insertions(+), 3 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 255bf03f7..59a9441ab 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -161,8 +161,10 @@ const workspaceCwdAliasMcpServerSource = String.raw`import fs from 'node:fs' import helper from './helper.mjs' const [receiptPath, expectedNonce] = process.argv.slice(2) if (!receiptPath || !expectedNonce) process.exit(12) +const processStat = fs.readFileSync('/proc/self/stat', 'utf8') +const processStartTicks = processStat.slice(processStat.lastIndexOf(')') + 2).trim().split(/\s+/)[19] fs.writeFileSync(receiptPath, JSON.stringify({ expectedNonce, helper, terminalId: process.env.FRESHELL_TERMINAL_ID, - pid: process.pid, createdAt: new Date().toISOString(), + pid: process.pid, processStartTicks, createdAt: new Date().toISOString(), envValueLoaded: process.env.FRESHELL_MANAGED_CWD_ALIAS_ENV === expectedNonce, envFileTarget: fs.realpathSync('.env'), envFileOperand: process.execArgv.find(value => value.startsWith('--env-file=')), @@ -194,6 +196,30 @@ process.stdin.on('data', chunk => { }) ` +export function workspaceCwdAliasReplacementProcessProbeSource(): string { + return String.raw`const fs = require('node:fs') +const path = require('node:path') +const [pidText, receiptPath, expectedNonce, expectedCwd, expectedStartTicks] = process.argv.slice(1) +const procPath = '/proc/' + Number(pidText) +const result = { exists: false, isNode: false, envFileOperand: false, fixtureArguments: false, cwdMatches: false, + processStartTicksMatches: false } +try { + result.exists = fs.existsSync(procPath + '/stat') + const command = fs.readFileSync(procPath + '/cmdline', 'utf8').split(String.fromCharCode(0)).filter(Boolean) + const stat = fs.readFileSync(procPath + '/stat', 'utf8') + const startTicks = stat.slice(stat.lastIndexOf(')') + 2).trim().split(/\s+/)[19] + const scriptIndex = command.indexOf('server.mjs') + result.isNode = path.basename(fs.readlinkSync(procPath + '/exe').replace(/ \(deleted\)$/, '')) === 'node' + result.envFileOperand = command.includes('--env-file=.env') + result.fixtureArguments = scriptIndex >= 0 && command[scriptIndex + 1] === receiptPath + && command[scriptIndex + 2] === expectedNonce + result.cwdMatches = fs.readlinkSync(procPath + '/cwd') === expectedCwd + result.processStartTicksMatches = startTicks === expectedStartTicks +} catch { /* Emit false checks so the host-side assertion reports the failed evidence. */ } +process.stdout.write(JSON.stringify(result)) +` +} + export function workspaceCwdAliasMcpWrapperSource(): string { const receiptVariable = '${receipt}' return `#!/bin/sh @@ -1798,16 +1824,28 @@ export async function runManagedEnvironmentTarget() { assert.equal(replacementCwdAliasMcp.helper, nonce) assert.equal(replacementCwdAliasMcp.terminalId, replacement.terminalId) assert(Number.isSafeInteger(replacementCwdAliasMcp.pid) && replacementCwdAliasMcp.pid > 1) + assert.match(replacementCwdAliasMcp.processStartTicks, /^\d+$/) assert.equal(replacementCwdAliasMcp.envValueLoaded, true) assert.match(replacementCwdAliasMcp.envFileTarget, /^\/run\/freshell-secrets\/provider-\d+$/) assert.equal(replacementCwdAliasMcp.envFileOperand, '--env-file=.env') assert.equal(replacementCwdAliasMcp.cwd, selected.cwdAliasProject) assert(replacementCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) assert.equal(await fs.readFile(`${cwdAliasReceiptPath}.started`, 'utf8'), 'wrapper-started') + assert(replacement.containerId) + const replacementMcpProcessProbe = JSON.parse(rig.runtime.execOwnedContainerExact(replacement.containerId, [ + 'node', '-e', workspaceCwdAliasReplacementProcessProbeSource(), + String(replacementCwdAliasMcp.pid), cwdAliasReceiptPath, nonce, selected.cwdAliasProject, + replacementCwdAliasMcp.processStartTicks, + ])) + assert.deepEqual(replacementMcpProcessProbe, { + exists: true, isNode: true, envFileOperand: true, fixtureArguments: true, cwdMatches: true, + processStartTicksMatches: true, + }, 'the .env MCP receipt must belong to a live process in the replacement container') proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, child: replacementChild, workspaceCwdAliasMcp: { ...replacementCwdAliasMcp, - associatedIncarnationId: replacement.incarnationId } } + verifiedInsideContainerId: replacement.containerId, verifiedIncarnationId: replacement.incarnationId, + processProbe: replacementMcpProcessProbe } } await writeJson(path.join(rig.runtime.evidenceDir, 'managed-environment-acceptance.json'), proof) const browserSummary = proof.cloudBrowser as ManagedEnvironmentBrowserSummary console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.acceptance_passed', candidateSha: fixture.candidateSha, diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 2c309018b..0436ce3d9 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -1,4 +1,4 @@ -import { spawn } from 'node:child_process' +import { execFileSync, spawn } from 'node:child_process' import fs from 'node:fs/promises' import { createConnection, createServer, type AddressInfo } from 'node:net' import path from 'node:path' @@ -19,6 +19,7 @@ import { runManagedProviderEnvironmentTarget, stopExact, workspaceCwdAliasMcpWrapperSource, + workspaceCwdAliasReplacementProcessProbeSource, writeWorkspaceCwdAliasMcpWrapper, } from '../../../../scripts/testing/managed-provider-environment-target.js' @@ -234,6 +235,61 @@ fs.writeFileSync(receiptPath, JSON.stringify({ envValueLoaded: process.env.FRESH } }) + it('binds a generated replacement-process probe to the live MCP process identity', async () => { + const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-cwd-alias-probe-')) + const receiptPath = path.join(directory, 'mcp-cwd-alias-receipt.json') + const nonce = 'replacement-probe-nonce' + let child: ReturnType<typeof spawn> | undefined + try { + await fs.writeFile(path.join(directory, '.env'), 'PROBE_ENV=ready\n') + await fs.writeFile(path.join(directory, 'server.mjs'), `import fs from 'node:fs' +const [receiptPath] = process.argv.slice(2) +const stat = fs.readFileSync('/proc/self/stat', 'utf8') +const processStartTicks = stat.slice(stat.lastIndexOf(')') + 2).trim().split(/\\s+/)[19] +fs.writeFileSync(receiptPath, JSON.stringify({ pid: process.pid, processStartTicks })) +process.stdout.write(JSON.stringify({ pid: process.pid, processStartTicks }) + '\\n') +setInterval(() => {}, 1000) +`) + child = spawn(process.execPath, ['--env-file=.env', 'server.mjs', receiptPath, nonce], { + cwd: directory, stdio: ['ignore', 'pipe', 'pipe'], + }) + let buffered = '' + const receipt = await new Promise<{ pid: number; processStartTicks: string }>((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error('MCP fixture did not report ready')), 5_000) + child!.once('error', error => { clearTimeout(timeout); reject(error) }) + child!.once('exit', (code, signal) => { + clearTimeout(timeout) + reject(new Error(`MCP fixture exited before ready (${code ?? signal})`)) + }) + child!.stdout!.setEncoding('utf8') + child!.stdout!.on('data', chunk => { + buffered += chunk + const boundary = buffered.indexOf('\n') + if (boundary < 0) return + clearTimeout(timeout) + try { resolve(JSON.parse(buffered.slice(0, boundary))) } + catch (error) { reject(error) } + }) + }) + const runProbe = (startTicks: string) => JSON.parse(execFileSync(process.execPath, [ + '-e', workspaceCwdAliasReplacementProcessProbeSource(), String(receipt.pid), receiptPath, nonce, directory, startTicks, + ], { cwd: directory, encoding: 'utf8' })) + + expect(runProbe(receipt.processStartTicks)).toEqual({ + exists: true, isNode: true, envFileOperand: true, fixtureArguments: true, cwdMatches: true, + processStartTicksMatches: true, + }) + expect(runProbe(`${receipt.processStartTicks}-stale`).processStartTicksMatches).toBe(false) + } finally { + if (child && child.exitCode === null && child.signalCode === null) { + const closed = new Promise<void>(resolve => child!.once('close', () => resolve())) + child.kill('SIGTERM') + await closed + } + await fs.rm(directory, { recursive: true, force: true }) + } + }) + it('keeps a workspace MCP wrapper searchable under a private process umask', async () => { const directory = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-managed-wrapper-mode-')) try { From 736de268258baa33068389ade6b08e86d5645507 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 22:24:06 -0700 Subject: [PATCH 480/493] fix managed pane reconcile before identity --- src/lib/pane-reconcile.ts | 9 ++++++ .../App.reconcile-adoption.test.tsx | 30 ++++++++++++++++--- .../lib/pane-reconcile.fresh-agent.test.ts | 17 +++++++++++ 3 files changed, 52 insertions(+), 4 deletions(-) diff --git a/src/lib/pane-reconcile.ts b/src/lib/pane-reconcile.ts index 543956e95..8e72d4c04 100644 --- a/src/lib/pane-reconcile.ts +++ b/src/lib/pane-reconcile.ts @@ -153,6 +153,11 @@ function toReconcilePane(tabId: string, paneId: string, content: TerminalPaneCon // same createRequestId would classify it as fresh and silently retry it. if (retainedManagedPreparationFailure(content)) return null const sessionRef = effectiveReconcileSessionRef(content.sessionRef, content.resumeSessionId, content.mode) + // A managed runtime may still be starting or blocked before its native + // provider session exists. In that state, generic reconciliation would + // label the pane `fresh` and clear its retained terminal handle. Managed + // inventory owns that lifecycle until a durable provider identity exists. + if (content.soulId && !sessionRef) return null return { paneKey: paneKeyFor(tabId, paneId), kind: 'terminal', @@ -178,6 +183,10 @@ function toFreshAgentReconcilePane( // Panes without a createRequestId cannot be reconciled — skip. if (!content.createRequestId) return null const sessionRef = effectiveReconcileSessionRef(content.sessionRef, content.resumeSessionId, content.provider) + // As with managed terminals, a soul without a native session identity has + // no safe generic reconcile verdict. Keep its supervisor-owned recovery + // projection intact and let managed inventory decide its state. + if (content.soulId && !sessionRef) return null return { paneKey: paneKeyFor(tabId, paneId), kind: 'fresh-agent', diff --git a/test/unit/client/components/App.reconcile-adoption.test.tsx b/test/unit/client/components/App.reconcile-adoption.test.tsx index 353de85d1..4835ef4ed 100644 --- a/test/unit/client/components/App.reconcile-adoption.test.tsx +++ b/test/unit/client/components/App.reconcile-adoption.test.tsx @@ -155,10 +155,10 @@ const sentFrames: any[] = [] const dispatchedTypes: string[] = [] const dispatched = { types: dispatchedTypes } -let seededPane: { createRequestId: string } | null = null +let seededPane: { createRequestId: string; mode?: string; soulId?: string; runtimeStatus?: string; status?: string } | null = null let seededFreshAgentPane: { createRequestId: string } | null = null -function seedPersistedTerminalPane(opts: { createRequestId: string }): void { +function seedPersistedTerminalPane(opts: { createRequestId: string; mode?: string; soulId?: string; runtimeStatus?: string; status?: string }): void { seededPane = opts } @@ -185,11 +185,13 @@ function createStore() { id: 'pane-1', content: { kind: 'terminal', - mode: 'shell', + mode: seededPane.mode ?? 'shell', shell: 'system', - status: 'running', + status: seededPane.status ?? 'running', createRequestId: seededPane.createRequestId, terminalId: 'term-old', + ...(seededPane.soulId ? { soulId: seededPane.soulId } : {}), + ...(seededPane.runtimeStatus ? { runtimeStatus: seededPane.runtimeStatus } : {}), }, } : null @@ -462,6 +464,26 @@ describe('App pane.reconcile adoption', () => { expect(dispatched.types).toContain(setLiveTerminalIds.type) // non-destructive part stays }) + it('leaves managed soul panes to terminal inventory instead of generic reconcile', async () => { + seedPersistedTerminalPane({ + createRequestId: 'cr-managed-blocked', + mode: 'opencode', + soulId: 'soul-blocked', + runtimeStatus: 'blocked', + status: 'error', + }) + const { sentFrames, store } = await bootAppWithReady({ capabilities: { paneReconcileV1: true } }) + + expect(sentFrames.some((frame) => frame.type === 'pane.reconcile.request')).toBe(false) + expect(wsMocks.clearReconcileCreateHold).toHaveBeenCalled() + expect(terminalPaneContentOf(store)).toMatchObject({ + terminalId: 'term-old', + soulId: 'soul-blocked', + runtimeStatus: 'blocked', + status: 'error', + }) + }) + it('runs the legacy census when the server does not ack the capability', async () => { seedPersistedTerminalPane({ createRequestId: 'cr-1' }) const { sentFrames, dispatched } = await bootAppWithReady({ /* no capabilities */ }) diff --git a/test/unit/client/lib/pane-reconcile.fresh-agent.test.ts b/test/unit/client/lib/pane-reconcile.fresh-agent.test.ts index 638153a70..f040e370f 100644 --- a/test/unit/client/lib/pane-reconcile.fresh-agent.test.ts +++ b/test/unit/client/lib/pane-reconcile.fresh-agent.test.ts @@ -224,6 +224,23 @@ describe('buildReconcileRequest with fresh-agent panes', () => { expect(fa.sessionRef).toEqual({ provider: 'claude', sessionId: DURABLE }) }) + it('leaves managed panes without a native identity to managed inventory', () => { + let panes = emptyPanesState() + panes = addTerminalPane(panes, 'managed-terminal-tab', 'managed-terminal', { + mode: 'opencode', + terminalId: 'managed-terminal-id', + soulId: 'managed-terminal-soul', + runtimeStatus: 'blocked', + }) + panes = addFreshAgentPane(panes, 'managed-agent-tab', 'managed-agent', { + soulId: 'managed-agent-soul', + }) + + const req = buildReconcileRequest(asRootState(panes), { includeFreshAgent: true }) + + expect(req).toBeNull() + }) + it('skips fresh-agent panes without createRequestId', () => { // Built by hand: the initLayout reducer would mint a createRequestId. const panes = emptyPanesState() From 729ec91cfba0f0b26d38704eff6d283eb075483c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 22:24:08 -0700 Subject: [PATCH 481/493] tighten managed browser reconnect checks --- ...aged-provider-environment-external.spec.ts | 115 +++++++++++++----- 1 file changed, 86 insertions(+), 29 deletions(-) diff --git a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts index 6f77b701e..fa4cb2457 100644 --- a/test/e2e-browser/specs/managed-provider-environment-external.spec.ts +++ b/test/e2e-browser/specs/managed-provider-environment-external.spec.ts @@ -18,12 +18,14 @@ const caseScenarios: Record<string, string> = { 'startup-before-identity retains an actionable blocked result without a replacement conversation': 'startup-before-identity', } -async function terminalDiagnostic(page: Page, terminalId?: string) { - const diagnostic = await page.evaluate(({ terminalId, marker }) => { +async function terminalDiagnostic(page: Page, terminalId?: string, paneId?: string, createRequestId?: string) { + const diagnostic = await page.evaluate(({ terminalId, paneId, createRequestId, marker }) => { const harness = window.__FRESHELL_TEST_HARNESS__ const buffer = terminalId && harness ? harness.getTerminalBuffer(terminalId) : null const received = (harness?.getReceivedWsMessages?.() ?? []) as any[] const sent = (harness?.getSentWsMessages?.() ?? []) as any[] + const paneCreates = sent.filter(message => message.type === 'terminal.create' && message.paneId === paneId) + const paneCreateRequestIds = new Set(paneCreates.map(message => message.requestId).filter(value => typeof value === 'string')) const writes = (harness?.getTerminalWriteEvents?.() ?? []).filter(event => event.terminalId === terminalId) const output = received.filter(message => message.terminalId === terminalId && ['terminal.output', 'terminal.output.batch'].includes(message.type)) @@ -39,12 +41,20 @@ async function terminalDiagnostic(page: Page, terminalId?: string) { } return { chunks: chunks.length, markerPresent, reset, alternateEntered, alternateExited } } - const attachRequests = new Set(sent.filter(message => message.terminalId === terminalId + const attachRequests = new Set(sent.filter(message => terminalId && message.terminalId === terminalId && message.type === 'terminal.attach').map(message => message.attachRequestId)) - const attachErrors = received.filter(message => message.type === 'error' - && (message.terminalId === terminalId || typeof message.requestId === 'string' && attachRequests.has(message.requestId))) + const attachErrors = received.filter(message => message.type === 'error' && ( + (Boolean(terminalId) && message.terminalId === terminalId) + || typeof message.requestId === 'string' && ( + message.requestId === createRequestId + || paneCreateRequestIds.has(message.requestId) + || attachRequests.has(message.requestId) + ) + )) .slice(-16).map(message => ({ type: 'error', code: ['INVALID_REQUEST', 'TERMINAL_NOT_FOUND', 'PTY_SPAWN_FAILED', 'SESSION_RESERVED', 'INTERNAL_ERROR', 'AUTH_FAILED'].includes(message.code) ? message.code : 'UNKNOWN', + ...(typeof message.terminalId === 'string' ? { terminalId: message.terminalId } : {}), + ...(typeof message.requestId === 'string' ? { requestId: message.requestId } : {}), message: typeof message.message === 'string' ? message.message.slice(0, 4096) : undefined, providerPreparationFailure: message.providerPreparationFailure })) const metadata = (messages: any[]) => messages.filter(message => message.terminalId === terminalId && [ @@ -66,6 +76,31 @@ async function terminalDiagnostic(page: Page, terminalId?: string) { 'new_pty_session', 'codex_pty_recovery', 'retention_lost', 'server_restart_incompatible_retention'].includes(message.reason)) value.reason = message.reason return value }) + const relevantMessages = received.filter(message => [ + 'terminal.created', 'terminal.status', 'terminal.exit', 'terminal.replaced', 'error', + ].includes(message.type) && ( + (Boolean(terminalId) && (message.terminalId === terminalId || message.oldTerminalId === terminalId || message.newTerminalId === terminalId)) + || typeof message.requestId === 'string' && ( + message.requestId === createRequestId || paneCreateRequestIds.has(message.requestId) + ) + )).slice(-24).map(message => { + const value: Record<string, string | number | boolean> = { type: message.type } + for (const key of ['code', 'requestId', 'terminalId', 'oldTerminalId', 'newTerminalId', 'status', 'exitCode', 'mode']) { + if (typeof message[key] === 'string' || Number.isSafeInteger(message[key])) value[key] = message[key] + } + if (message.recoverySummary && typeof message.recoverySummary === 'object') { + const summary = message.recoverySummary as Record<string, unknown> + for (const key of ['recoveryState', 'desiredState', 'reason']) { + if (typeof summary[key] === 'string') value[`recovery${key[0]!.toUpperCase()}${key.slice(1)}`] = summary[key] as string + } + } + return value + }) + const recentPaneCreates = paneCreates.slice(-16).map(message => ({ + requestId: typeof message.requestId === 'string' ? message.requestId : 'UNKNOWN', + mode: typeof message.mode === 'string' ? message.mode : 'UNKNOWN', + restore: message.restore === true, + })) return { harnessPresent: Boolean(harness), bufferType: typeof buffer, bufferNull: buffer === null, bufferLength: typeof buffer === 'string' ? buffer.length : null, markerPresent: typeof buffer === 'string' ? buffer.includes(marker) : false, @@ -74,15 +109,18 @@ async function terminalDiagnostic(page: Page, terminalId?: string) { submittedWrites: classify(writes.filter(event => event.phase === 'submitted').map(event => event.data)), completedWrites: classify(writes.filter(event => event.phase === 'written').map(event => event.data)), sentMetadata: metadata(sent), receivedMetadata: metadata(received), attachErrors, + paneCreates: recentPaneCreates, relevantMessages, // These getters retain bounded history, not a complete transport trace. receivedHistoryAtBound: received.length >= 500, writeHistoryAtBound: writes.length >= 1000 } - }, { terminalId, marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) + }, { terminalId, paneId, createRequestId, marker: `MANAGED_ENV_OUTPUT ${fixture.expectedNonce}` }).catch(() => ({ observation: 'UNKNOWN' })) if (!('attachErrors' in diagnostic) || !Array.isArray(diagnostic.attachErrors)) return diagnostic return { ...diagnostic, attachErrors: diagnostic.attachErrors.map((error: any) => { const runtimeErrorClass = summarizeRuntimeErrorClass(error.message ?? '') const runtimeErrorCode = summarizeRuntimeErrorCode(error.message ?? '') const providerPreparationFailure = summarizeProviderPreparationFailure(error.providerPreparationFailure) return { type: error.type, code: error.code, + ...(typeof error.terminalId === 'string' ? { terminalId: error.terminalId } : {}), + ...(typeof error.requestId === 'string' ? { requestId: error.requestId } : {}), ...(runtimeErrorClass ? { runtimeErrorClass } : {}), ...(runtimeErrorCode ? { runtimeErrorCode } : {}), ...(providerPreparationFailure ? { providerPreparationFailure } : {}) } @@ -111,12 +149,20 @@ test.afterEach(async ({ page }, info) => { let failureEvidence: Record<string, unknown> | undefined if (info.status !== info.expectedStatus) { const pane = await activePane(page).catch(() => null) - const diagnostic = await terminalDiagnostic(page, pane?.content.kind === 'terminal' ? pane.content.terminalId : undefined) + const content = pane?.content.kind === 'terminal' ? pane.content : undefined + const diagnostic = await terminalDiagnostic(page, content?.terminalId, pane?.paneId, content?.createRequestId) failureEvidence = { stage: info.annotations.filter(value => value.type === 'managed-stage').at(-1)?.description ?? 'UNKNOWN', - terminalPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.terminalId), - soulPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.soulId), - nativeIdentityPresent: pane?.content.kind === 'terminal' && Boolean(pane.content.sessionRef), + paneId: pane?.paneId, + terminalId: content?.terminalId, + createRequestId: content?.createRequestId, + paneStatus: content?.status, + runtimeStatus: content?.runtimeStatus, + recoveryState: content?.recoverySummary?.recoveryState, + recoveryReason: content?.recoverySummary?.reason, + terminalPresent: Boolean(content?.terminalId), + soulPresent: Boolean(content?.soulId), + nativeIdentityPresent: Boolean(content?.sessionRef), diagnostic, } console.error(JSON.stringify({ severity: 'error', event: 'managed_environment.browser_attempt_failed', @@ -246,10 +292,14 @@ test(healthyTitle, async ({ page }, info) => { expect(receipt.soulId).toBeTruthy() expect(receipt.incarnationId).toBeTruthy() expect(receipt.nativeSessionId).toBeTruthy() - const beforeCreates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length) + const replacementCreateRequestsBeforeReconnect = await page.evaluate(({ paneId, createRequestId }) => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId) + .map((message: any) => message.requestId) + .filter((requestId: string) => requestId !== createRequestId), + { paneId: pane.paneId, createRequestId: pane.createRequestId }) console.log(JSON.stringify({ severity: 'info', event: 'managed_environment.browser_before_reconnect', - scenario: 'healthy', attempt: info.retry, diagnostic: await terminalDiagnostic(page, terminalId) })) + scenario: 'healthy', attempt: info.retry, + diagnostic: await terminalDiagnostic(page, terminalId, pane.paneId, pane.createRequestId) })) const reconnectStartedAt = await page.evaluate(() => performance.now()) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) @@ -258,9 +308,14 @@ test(healthyTitle, async ({ page }, info) => { info.annotations.push({ type: 'managed-stage', description: 'healthy-post-reconnect-output' }) await expect.poll(() => hasCompletedOutputWrite(page, terminalId, marker, reconnectStartedAt), { timeout: 120_000 }).toBe(true) - expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length)).toBe(beforeCreates) + expect(await page.evaluate(({ paneId, createRequestId }) => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId) + .map((message: any) => message.requestId) + .filter((requestId: string) => requestId !== createRequestId), + { paneId: pane.paneId, createRequestId: pane.createRequestId })) + .toEqual(replacementCreateRequestsBeforeReconnect) const rows = await json(page, '/api/terminals') + expect(rows.filter((row: any) => row.cwd === pane.cwd)).toHaveLength(1) expect(rows.find((row: any) => row.terminalId === terminalId)).toMatchObject({ terminalId, soulId: receipt.soulId, incarnationId: receipt.incarnationId, runtimeStatus: 'running' }) }) @@ -303,25 +358,25 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { await expect(diagnostic.getByRole('button', { name: 'Retry launch' })).toBeVisible() await expect(diagnostic.getByRole('button', { name: /^Attach/ })).toHaveCount(0) await assertNoHandle() - const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length) - expect(await page.evaluate(requestId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create' && message.requestId === requestId).length, - pane.createRequestId)).toBe(1) + const paneCreateRequests = await page.evaluate(paneId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId) + .map((message: any) => message.requestId), pane.paneId) + expect(paneCreateRequests).toEqual([pane.createRequestId]) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) await expect.poll(async () => (await activePane(page)).content, { timeout: 30_000 }) .toMatchObject(retained) - expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) + expect(await page.evaluate(paneId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId) + .map((message: any) => message.requestId), pane.paneId)).toEqual(paneCreateRequests) await assertNoHandle() await page.reload() await page.waitForFunction(() => Boolean(window.__FRESHELL_TEST_HARNESS__), undefined, { timeout: 60_000 }) await new TestHarness(page).waitForConnection(120_000) expect((await activePane(page)).content).toMatchObject(retained) - expect(await page.evaluate(requestId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create' && message.requestId === requestId).length, - pane.createRequestId)).toBe(0) + expect(await page.evaluate(({ paneId, requestId }) => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId && message.requestId === requestId).length, + { paneId: pane.paneId, requestId: pane.createRequestId })).toBe(0) await expect(diagnostic).toContainText(selectedPath) await expect(diagnostic).toContainText(/not found/i) await assertNoHandle() @@ -344,14 +399,16 @@ for (const scenario of ['missing-reference', 'startup-before-identity']) { .map((item: any) => item.terminalId) .sort() expect(terminalIdsBeforeReconnect).toContain(terminalId) - const creates = await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length) + const paneCreateRequests = await page.evaluate(paneId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId) + .map((message: any) => message.requestId), pane.paneId) await new TestHarness(page).forceDisconnect() await new TestHarness(page).waitForConnection(120_000) await expect.poll(async () => (await activePane(page)).content, { timeout: 30_000 }) .toMatchObject({ terminalId, status: 'error' }) - expect(await page.evaluate(() => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() - .filter((message: any) => message.type === 'terminal.create').length)).toBe(creates) + expect(await page.evaluate(paneId => window.__FRESHELL_TEST_HARNESS__!.getSentWsMessages?.() + .filter((message: any) => message.type === 'terminal.create' && message.paneId === paneId) + .map((message: any) => message.requestId), pane.paneId)).toEqual(paneCreateRequests) expect((await json(page, '/api/terminals')) .filter((item: any) => item.cwd === pane.cwd) .map((item: any) => item.terminalId) From e400e3eb127231c2922e4f19be10cd3bd6e315c2 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Thu, 15 Oct 2026 22:24:10 -0700 Subject: [PATCH 482/493] preserve complete cloud retry evidence --- scripts/e2e-cloud-retry-receipt.mjs | 4 +++- scripts/test/cloud-e2e-retry-receipt.test.sh | 11 ++++++++++- .../managed-environment-browser-summary.ts | 11 ++++++----- .../managed-provider-environment-target.ts | 2 ++ ...managed-environment-browser-summary.test.ts | 18 ++++++++++++++++++ 5 files changed, 39 insertions(+), 7 deletions(-) diff --git a/scripts/e2e-cloud-retry-receipt.mjs b/scripts/e2e-cloud-retry-receipt.mjs index 43bc93c3b..b0055cbd7 100644 --- a/scripts/e2e-cloud-retry-receipt.mjs +++ b/scripts/e2e-cloud-retry-receipt.mjs @@ -10,7 +10,9 @@ import { createHash } from 'node:crypto' import { readFileSync } from 'node:fs' const MAX_TRACE_BYTES = 8 * 1024 * 1024 -const TRACE_CHUNK_BYTES = 120 * 1024 +// Keep each serialized base64 JSONL record below Cloud Logging's 102,400-byte +// entry limit (48 KiB raw becomes 64 KiB base64 plus a small fixed envelope). +const TRACE_CHUNK_BYTES = 48 * 1024 const MAX_ERROR_TEXT_BYTES = 64 * 1024 const PLAYWRIGHT_OUTCOMES = ['expected', 'skipped', 'unexpected', 'flaky'] diff --git a/scripts/test/cloud-e2e-retry-receipt.test.sh b/scripts/test/cloud-e2e-retry-receipt.test.sh index e331b4190..63538af78 100644 --- a/scripts/test/cloud-e2e-retry-receipt.test.sh +++ b/scripts/test/cloud-e2e-retry-receipt.test.sh @@ -14,7 +14,7 @@ TRACE="$WORK/first-retry-trace.zip" UNRELATED_TRACE="$WORK/retry-success-trace.zip" REPORT="$WORK/report.json" OUT="$WORK/receipt.jsonl" -printf 'synthetic first retry trace\n' > "$TRACE" +head -c 196608 /dev/urandom > "$TRACE" printf 'unrelated successful retry trace\n' > "$UNRELATED_TRACE" cat > "$REPORT" <<JSON @@ -99,6 +99,15 @@ if [ -z "$CHUNKS" ]; then exit 1 fi +if ! jq -s -e ' + [.[] | select(.event == "e2e_playwright_retry_trace_chunk")] as $chunks + | ($chunks | length) > 1 + and all($chunks[]; ((tojson | length) < 100000) and ((.data | length) <= 65536)) +' "$OUT" >/dev/null; then + echo "FAIL: retry trace JSONL chunks exceed Cloud Logging's safe entry size" + exit 1 +fi + RECONSTRUCTED="$WORK/reconstructed-trace.zip" jq -r 'select(.event == "e2e_playwright_retry_trace_chunk") | .data' "$OUT" \ | base64 --decode > "$RECONSTRUCTED" diff --git a/scripts/testing/managed-environment-browser-summary.ts b/scripts/testing/managed-environment-browser-summary.ts index af1f24922..40a54e0af 100644 --- a/scripts/testing/managed-environment-browser-summary.ts +++ b/scripts/testing/managed-environment-browser-summary.ts @@ -11,21 +11,22 @@ export type ManagedEnvironmentBrowserSummary = { failedCases: number skippedCases: number interruptedCases: number + flakyCases: number } type Counts = Omit<ManagedEnvironmentBrowserSummary, 'footer' | 'executedCases'> function parseFooter(line: string): ManagedEnvironmentBrowserSummary | undefined { const normalized = line.replace(ansiEscape, '').trim() - if (!/\b(?:failed|passed|skipped|interrupted)\b/.test(normalized)) return undefined + if (!/\b(?:failed|passed|skipped|interrupted|flaky)\b/.test(normalized)) return undefined const withoutDuration = normalized.replace(/\s+\([^()]*\)$/, '') - const tokens = [...withoutDuration.matchAll(/(\d+)\s+(failed|passed|skipped|interrupted)\b/g)] + const tokens = [...withoutDuration.matchAll(/(\d+)\s+(failed|passed|skipped|interrupted|flaky)\b/g)] if (tokens.length === 0) return undefined - const remainder = withoutDuration.replace(/\d+\s+(?:failed|passed|skipped|interrupted)\b/g, '').replace(/[\s,]/g, '') + const remainder = withoutDuration.replace(/\d+\s+(?:failed|passed|skipped|interrupted|flaky)\b/g, '').replace(/[\s,]/g, '') if (remainder.length > 0) return undefined - const counts: Counts = { passedCases: 0, failedCases: 0, skippedCases: 0, interruptedCases: 0 } + const counts: Counts = { passedCases: 0, failedCases: 0, skippedCases: 0, interruptedCases: 0, flakyCases: 0 } const seen = new Set<string>() for (const token of tokens) { const count = Number(token[1]) @@ -35,7 +36,7 @@ function parseFooter(line: string): ManagedEnvironmentBrowserSummary | undefined counts[`${category}Cases` as keyof Counts] = count } - const executedCases = counts.passedCases + counts.failedCases + counts.skippedCases + counts.interruptedCases + const executedCases = counts.passedCases + counts.failedCases + counts.skippedCases + counts.interruptedCases + counts.flakyCases if (!Number.isSafeInteger(executedCases) || executedCases < 1) return undefined return { footer: normalized, executedCases, ...counts } } diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 59a9441ab..9dccef4b4 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1328,6 +1328,7 @@ export async function runCodeCheckpointTarget() { assert.equal(browserSummary.failedCases, 0) assert.equal(browserSummary.skippedCases, 0) assert.equal(browserSummary.interruptedCases, 0) + assert.equal(browserSummary.flakyCases, 0, 'Configured Cloud browser cases passed only after retry') proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, ...browserSummary } await fs.mkdir(evidenceDir, { recursive: true, mode: 0o700 }) await writeJson(receiptPath, proof) @@ -1774,6 +1775,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(browserSummary.failedCases, 0, 'Configured managed-environment browser cases failed') assert.equal(browserSummary.skippedCases, 0, 'Configured managed-environment browser cases were skipped') assert.equal(browserSummary.interruptedCases, 0, 'Configured managed-environment browser cases were interrupted') + assert.equal(browserSummary.flakyCases, 0, 'Configured managed-environment browser cases passed only after retry') proof.cloudBrowser = { actualNativeExit: browserExit, identity: browserIdentity, selectedScenarios: [...scenarios], ...browserSummary } }) const healthy = proof.healthy as any diff --git a/test/unit/tooling/testing/managed-environment-browser-summary.test.ts b/test/unit/tooling/testing/managed-environment-browser-summary.test.ts index 52a607feb..ebd8b2151 100644 --- a/test/unit/tooling/testing/managed-environment-browser-summary.test.ts +++ b/test/unit/tooling/testing/managed-environment-browser-summary.test.ts @@ -16,6 +16,7 @@ describe('managed environment configured-browser summary', () => { failedCases: 0, skippedCases: 0, interruptedCases: 0, + flakyCases: 0, }) }) @@ -31,6 +32,23 @@ describe('managed environment configured-browser summary', () => { failedCases: 1, skippedCases: 1, interruptedCases: 0, + flakyCases: 0, + }) + }) + + it('counts flaky cases in the Playwright footer instead of treating them as missing tests', () => { + const capture = new ManagedEnvironmentBrowserSummaryCapture() + capture.write('[e2e-cloud] Per-shard summary:\n 1 flaky, 2 passed (1.9m)\n') + capture.write('[e2e-cloud] Succeeded tasks: 1\n') + + expect(capture.finish()).toEqual({ + footer: '1 flaky, 2 passed (1.9m)', + executedCases: 3, + passedCases: 2, + failedCases: 0, + skippedCases: 0, + interruptedCases: 0, + flakyCases: 1, }) }) From 92c45cd25b93c647a6c44dc1da26880df20fecf9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 11:57:39 -0700 Subject: [PATCH 483/493] fix managed provider recovery and checkpoints --- crates/freshell-freshagent/src/claude.rs | 69 +- crates/freshell-freshagent/src/codex.rs | 441 ++++- crates/freshell-freshagent/src/lib.rs | 106 + crates/freshell-freshagent/src/opencode_ws.rs | 50 +- crates/freshell-server/Cargo.toml | 1 + crates/freshell-server/src/checkpoints.rs | 1570 ++++++++++++--- crates/freshell-server/src/main.rs | 20 + .../src/managed_mcp_capability.rs | 30 + crates/freshell-session-host/src/main.rs | 1746 ++++++++++++++++- .../src/output_journal.rs | 168 +- .../src/retained_output.rs | 75 +- .../managed-provider-environment-target.ts | 29 +- ...anaged-provider-environment-target.test.ts | 19 + 13 files changed, 3886 insertions(+), 438 deletions(-) diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index 82ce23230..970fbc34a 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -120,6 +120,8 @@ pub struct FreshClaudeState { /// `freshell-ws` connection so the oracle's capture socket records /// `freshAgent.created` / `freshAgent.send.accepted` / `freshAgent.event`. broadcast_tx: Arc<tokio::sync::broadcast::Sender<String>>, + /// Server-owned durability edge for checkpoint-related provider frames. + checkpoint_event_sink: crate::CheckpointEventSink, /// placeholder-nanoid → live claude session (sidecar stdin + owned child + consumer). sessions: Arc<TokioMutex<HashMap<String, ClaudeSession>>>, /// durable Claude UUID (`cliSessionId` from `sdk.session.init`) -> sessions-map key. @@ -781,6 +783,7 @@ impl FreshClaudeState { Self { hosted_mcp: false, broadcast_tx, + checkpoint_event_sink: crate::CheckpointEventSink::default(), sessions: Arc::new(TokioMutex::new(HashMap::new())), cli_index: Arc::new(TokioMutex::new(HashMap::new())), create_dedup: Arc::new(FreshAgentCreateDedup::new()), @@ -818,6 +821,13 @@ impl FreshClaudeState { self } + pub fn set_checkpoint_event_persister( + &self, + persister: crate::CheckpointEventPersister, + ) -> Result<(), &'static str> { + self.checkpoint_event_sink.set_persister(persister) + } + /// b8ke focused episode-2 round-3 F3: crate-visible session-key resolver /// for in-crate cross-module tests. #[cfg(test)] @@ -1843,6 +1853,16 @@ impl FreshClaudeState { } } + async fn broadcast_checkpoint_frame( + &self, + frame: String, + require_match: bool, + ) -> Result<(), String> { + self.checkpoint_event_sink + .broadcast(&self.broadcast_tx, frame, require_match) + .await + } + // ── freshAgent.create (WS) ─────────────────────────────────────────────────────── /// Handle a `freshAgent.create` for claude/kilroy: spawn the Node sidecar, drive the @@ -5258,20 +5278,31 @@ impl FreshClaudeState { }; let name_ref = projection.as_ref().map(|(r, _)| r.clone()); let session_name = projection.map(|(_, record)| record); - self.broadcast(&ServerMessage::FreshAgentSessionMaterialized( - FreshAgentSessionMaterialized { - previous_session_id: old_id.to_string(), + let message = ServerMessage::FreshAgentSessionMaterialized(FreshAgentSessionMaterialized { + previous_session_id: old_id.to_string(), + provider: PROVIDER.to_string(), + session_id: new_id.to_string(), + session_type: session_type.to_string(), + session_ref: Some(freshell_protocol::SessionLocator { provider: PROVIDER.to_string(), session_id: new_id.to_string(), - session_type: session_type.to_string(), - session_ref: Some(freshell_protocol::SessionLocator { - provider: PROVIDER.to_string(), - session_id: new_id.to_string(), - }), - name_ref, - session_name, - }, - )); + }), + name_ref, + session_name, + }); + if let Ok(frame) = serde_json::to_string(&message) { + if let Err(error) = self + .checkpoint_event_sink + .persist_before_broadcast(&frame, false) + .await + { + tracing::error!(target: "freshell_freshagent::claude", + cause = %error, + "freshagent.claude.checkpoint_alias_persist_failed" + ); + } + } + self.broadcast(&message); } /// Handle a `freshAgent.undo` / `freshAgent.redo` for claude/kilroy (kata 1wxv @@ -8636,7 +8667,19 @@ impl FreshClaudeState { // map key) so an attach-by-durable rebind flips live event routing. let stamp = broadcast_id.lock().expect("broadcast id lock").clone(); if let Some(frame) = sdk_line_to_frame(&value, &stamp, &session_type) { - let _ = broadcast_tx.send(frame); + if value.get("type").and_then(Value::as_str) + == Some("sdk.checkpointTurnIdentity") + { + if let Err(error) = state.broadcast_checkpoint_frame(frame, true).await { + tracing::error!(target: "freshell_freshagent::claude", + session_id = %stamp, + cause = %error, + "freshagent.claude.checkpoint_identity_persist_failed" + ); + } + } else { + let _ = broadcast_tx.send(frame); + } } } // Task 5 (unified needs-attention signal): capture the armed state diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index c5d87348a..0f5f256b5 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -188,11 +188,13 @@ fn locate_thread_rollout( const SESSION_TYPE: &str = "freshcodex"; /// The runtime provider (`AGENT_SESSION_TYPES.codex.provider`). const PROVIDER: &str = "codex"; -/// Each checkpoint identity read uses the app-server's existing 30-second -/// snapshot bound; two attempts keep delayed persistence and one transient read -/// failure recoverable without leaving an unbounded background poller. -const CHECKPOINT_IDENTITY_PROBE_ATTEMPTS: usize = 2; -const CHECKPOINT_IDENTITY_PROBE_RETRY_DELAY: Duration = Duration::from_millis(100); +/// Codex history is sometimes behind the turn-completion notification. Keep +/// reconciling for a bounded settlement window after the target turn retires; +/// each read is capped by the same deadline even though the app-server's normal +/// snapshot timeout is longer. +const CHECKPOINT_IDENTITY_SETTLE_WINDOW: Duration = Duration::from_secs(30); +const CHECKPOINT_IDENTITY_PROBE_RETRY_DELAY: Duration = Duration::from_millis(250); +const CHECKPOINT_IDENTITY_COMPLETION_POLL_DELAY: Duration = Duration::from_millis(100); enum CodexStopMode { Close, @@ -367,6 +369,10 @@ pub struct FreshCodexState { /// post-construction by `freshell-server` (precedent: /// `TerminalRegistry::set_activity_observer`). identity_sink: Arc<std::sync::OnceLock<SharedPaneIdentitySink>>, + /// Server-owned durable path for checkpoint events. When present, accepted + /// turn ids and final history identities are persisted before their public + /// broadcast so a server exit cannot strand a one-shot receipt in memory. + checkpoint_event_sink: crate::CheckpointEventSink, /// The per-sessionRef create/resume lease (D8 for fresh agents, Task 12) — /// ALWAYS ON at this runtime seam (never capability-gated). `main.rs` replaces the /// default with the ONE server-wide shared map via [`Self::set_session_leases`]. @@ -559,6 +565,9 @@ struct CodexSession { /// `ensureRuntime` lazy-restart invariant, adapter.ts:935-946). Cleared back to `false` /// once a respawn succeeds. exited: Arc<AtomicBool>, + /// Cancels checkpoint identity probes when this session is explicitly stopped. A live + /// turn may never emit its normal completion notification after teardown. + checkpoint_probe_cancelled: Arc<AtomicBool>, /// b8ke focused round-2 review R2-1: the sidecar child's pid + `/proc` /// ownership tag — the recorded identity `kill_for_handoff` condemns /// BEFORE its first cancellable await, so the handoff watcher's @@ -645,6 +654,17 @@ struct CodexSession { provenance: Option<crate::BindProvenance>, } +struct CodexCheckpointIdentityProbe { + client: Arc<CodexAppServerClient>, + session_id: String, + submitted_turn_id: String, + request_id: String, + logical_attempt_id: String, + active_turn: Arc<StdMutex<Option<String>>>, + exited: Arc<AtomicBool>, + cancelled: Arc<AtomicBool>, +} + #[derive(Debug, Clone, Copy, PartialEq, Eq)] enum CodexTeardownSettlement { Confirmed, @@ -785,6 +805,7 @@ impl FreshCodexState { dead_thread_ttl: DEAD_THREAD_CACHE_TTL, create_dedup: Arc::new(FreshAgentCreateDedup::new()), identity_sink: Arc::new(std::sync::OnceLock::new()), + checkpoint_event_sink: crate::CheckpointEventSink::default(), leases: Arc::new(crate::session_lease::FreshAgentSessionLeases::new()), ownership: None, ownership_stamps: Arc::new(std::sync::Mutex::new(HashMap::new())), @@ -806,6 +827,15 @@ impl FreshCodexState { self } + /// Wire the server-owned durable checkpoint-event path before this state is + /// cloned into routers and session consumers. + pub fn set_checkpoint_event_persister( + &mut self, + persister: crate::CheckpointEventPersister, + ) -> Result<(), &'static str> { + self.checkpoint_event_sink.set_persister(persister) + } + /// The configured wedged-sidecar quiet window, in milliseconds. fn codex_quiet_window_ms(&self) -> u64 { self.codex_quiet_window_ms.load(Ordering::SeqCst) @@ -1813,6 +1843,32 @@ impl FreshCodexState { } } + async fn persist_checkpoint_frame( + &self, + msg: &ServerMessage, + require_match: bool, + ) -> Result<String, String> { + let Ok(frame) = serde_json::to_string(msg) else { + return Err("checkpoint event could not be serialized".to_string()); + }; + self.checkpoint_event_sink + .persist_before_broadcast(&frame, require_match) + .await?; + Ok(frame) + } + + async fn broadcast_checkpoint_event( + &self, + msg: &ServerMessage, + require_match: bool, + ) -> Result<(), String> { + let frame = serde_json::to_string(msg) + .map_err(|_| "checkpoint event could not be serialized".to_string())?; + self.checkpoint_event_sink + .broadcast(&self.broadcast_tx, frame, require_match) + .await + } + // ── freshAgent.create (WS) ─────────────────────────────────────────────── /// Handle a `freshAgent.create` for codex: spawn the app-server sidecar, start a thread, @@ -2834,6 +2890,7 @@ impl FreshCodexState { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -3361,6 +3418,7 @@ impl FreshCodexState { s.event_emission_gate.clone(), s.quiet_deadman.clone(), s.exited.clone(), + s.checkpoint_probe_cancelled.clone(), ) }) }; @@ -3373,6 +3431,7 @@ impl FreshCodexState { event_emission_gate, quiet_deadman, exited, + checkpoint_probe_cancelled, )) = looked_up else { self.send_error(&request_id, "SESSION_NOT_FOUND", "codex session not found"); @@ -3649,48 +3708,95 @@ impl FreshCodexState { ); // Broadcast freshAgent.send.accepted (ws-handler.ts:3487). turnAccepted edge. - self.broadcast(&ServerMessage::FreshAgentSendAccepted( - freshell_protocol::FreshAgentSendAccepted { + let accepted = + ServerMessage::FreshAgentSendAccepted(freshell_protocol::FreshAgentSendAccepted { provider: PROVIDER.to_string(), request_id: request_id.clone().unwrap_or_default(), session_id: session_id.clone(), session_type: SESSION_TYPE.to_string(), cwd: turn_cwd, submitted_turn_id: Some(submitted_turn_id.clone()), - }, - )); + }); + if checkpoint_identity_request.is_some() { + if let Err(error) = self.broadcast_checkpoint_event(&accepted, true).await { + tracing::error!(target: "freshell_freshagent::codex", + session_id = %session_id, + request_id = request_id.as_deref().unwrap_or_default(), + cause = %error, + "freshagent.codex.checkpoint_accepted_persist_failed" + ); + // The provider accepted the turn, so keep the acknowledgement + // flowing, but withhold the unpersisted native ID. The exact + // identity probe below will publish only if its own durable + // write succeeds. + if let ServerMessage::FreshAgentSendAccepted(mut fallback) = accepted { + fallback.submitted_turn_id = None; + self.broadcast(&ServerMessage::FreshAgentSendAccepted(fallback)); + } + } + } else { + self.broadcast(&accepted); + } if let Some((request_id, logical_attempt_id)) = checkpoint_identity_request { - self.spawn_checkpoint_identity_probe( - client, - session_id, - submitted_turn_id, - request_id, - logical_attempt_id, + drop( + self.spawn_checkpoint_identity_probe(CodexCheckpointIdentityProbe { + client, + session_id, + submitted_turn_id, + request_id, + logical_attempt_id, + active_turn, + exited, + cancelled: checkpoint_probe_cancelled, + }), ); } } fn spawn_checkpoint_identity_probe( &self, - client: Arc<CodexAppServerClient>, - session_id: String, - submitted_turn_id: String, - request_id: String, - logical_attempt_id: String, - ) { + probe: CodexCheckpointIdentityProbe, + ) -> tokio::task::JoinHandle<()> { let state = self.clone(); tokio::spawn(async move { - // `turn/start` acceptance is not itself proof that the native user row - // is persisted. Read the provider history and emit a receipt only when - // the exact submitted turn projects to one unique user row. Reads - // are bounded individually by the app-server client's existing - // 30-second snapshot timeout; the fixed attempt count bounds this - // service-owned probe to at most two reads. + let CodexCheckpointIdentityProbe { + client, + session_id, + submitted_turn_id, + request_id, + logical_attempt_id, + active_turn, + exited, + cancelled, + } = probe; + // Wait for the accepted turn to retire before treating its projected + // row ids as final. A live turn may contain only the user row today, + // then gain assistant rows that change its display id. After retirement, + // the bounded read/retry window covers the provider-history persistence + // lag without abandoning the receipt after a handful of fast empty reads. + while active_turn.lock().expect("active_turn mutex").as_deref() + == Some(submitted_turn_id.as_str()) + && !exited.load(Ordering::SeqCst) + && !cancelled.load(Ordering::SeqCst) + { + tokio::time::sleep(CHECKPOINT_IDENTITY_COMPLETION_POLL_DELAY).await; + } + if cancelled.load(Ordering::SeqCst) { + return; + } + + let deadline = tokio::time::Instant::now() + CHECKPOINT_IDENTITY_SETTLE_WINDOW; let mut identity = None; - for attempt in 1..=CHECKPOINT_IDENTITY_PROBE_ATTEMPTS { - match client.read_thread(&session_id, true).await { - Ok(raw_thread) => { + let mut attempt = 0; + loop { + if cancelled.load(Ordering::SeqCst) { + return; + } + attempt += 1; + match tokio::time::timeout_at(deadline, client.read_thread(&session_id, true)).await + { + Ok(Ok(raw_thread)) => { identity = checkpoint_codex_turn_identity(&raw_thread, &submitted_turn_id); if identity.is_some() { break; @@ -3703,7 +3809,7 @@ impl FreshCodexState { "freshagent.codex.checkpoint_identity_row_not_yet_observed" ); } - Err(error) => { + Ok(Err(error)) => { tracing::warn!(target: "freshell_freshagent::codex", session_id = %session_id, submitted_turn_id = %submitted_turn_id, @@ -3713,34 +3819,57 @@ impl FreshCodexState { "freshagent.codex.checkpoint_identity_read_failed" ); } + Err(_) => break, } - if attempt < CHECKPOINT_IDENTITY_PROBE_ATTEMPTS { - tokio::time::sleep(CHECKPOINT_IDENTITY_PROBE_RETRY_DELAY).await; + let now = tokio::time::Instant::now(); + if now >= deadline { + break; } + tokio::time::sleep( + CHECKPOINT_IDENTITY_PROBE_RETRY_DELAY + .min(deadline.saturating_duration_since(now)), + ) + .await; } let Some((native_turn_id, display_turn_id)) = identity else { tracing::warn!(target: "freshell_freshagent::codex", session_id = %session_id, submitted_turn_id = %submitted_turn_id, request_id = %request_id, - "freshagent.codex.checkpoint_identity_probe_exhausted" + attempts = attempt, + "freshagent.codex.checkpoint_identity_settlement_expired" ); return; }; - state.broadcast(&ServerMessage::FreshAgentEvent(FreshAgentEvent { - event: json!({ - "type": "checkpointTurnIdentity", - "logicalAttemptId": logical_attempt_id, - "requestId": request_id, - "sessionId": session_id, - "nativeTurnId": native_turn_id, - "displayTurnId": display_turn_id, - }), - provider: PROVIDER.to_string(), - session_id, - session_type: SESSION_TYPE.to_string(), - })); - }); + if cancelled.load(Ordering::SeqCst) { + return; + } + if let Err(error) = state + .broadcast_checkpoint_event( + &ServerMessage::FreshAgentEvent(FreshAgentEvent { + event: json!({ + "type": "checkpointTurnIdentity", + "logicalAttemptId": logical_attempt_id, + "requestId": request_id, + "sessionId": session_id, + "nativeTurnId": native_turn_id, + "displayTurnId": display_turn_id, + }), + provider: PROVIDER.to_string(), + session_id, + session_type: SESSION_TYPE.to_string(), + }), + true, + ) + .await + { + tracing::error!(target: "freshell_freshagent::codex", + request_id = %request_id, + cause = %error, + "freshagent.codex.checkpoint_identity_persist_failed" + ); + } + }) } fn send_error(&self, request_id: &Option<String>, code: &str, message: &str) { @@ -5648,6 +5777,14 @@ impl FreshCodexState { self.clear_controls(&session_id).await; let removed = self.sessions.lock().await.remove(&session_id); + if let Some(session) = removed.as_ref() { + // Requested teardown ends the probe's wait even when the provider never + // emits turn/completed. Signal before closing the client so an in-flight + // history read can settle and the probe exits promptly. + session + .checkpoint_probe_cancelled + .store(true, Ordering::SeqCst); + } let removed_ownership_id = removed .as_ref() .map(|session| session.sidecar_ownership_id.clone()); @@ -6707,6 +6844,7 @@ impl FreshCodexState { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -7007,6 +7145,7 @@ impl FreshCodexState { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -7141,8 +7280,8 @@ impl FreshCodexState { let name_ref = naming_projection.as_ref().map(|(r, _)| r.clone()); let session_name = naming_projection.map(|(_, record)| record); - self.broadcast(&ServerMessage::FreshAgentSessionMaterialized( - FreshAgentSessionMaterialized { + let materialized = + ServerMessage::FreshAgentSessionMaterialized(FreshAgentSessionMaterialized { previous_session_id: old_session_id.to_string(), provider: PROVIDER.to_string(), session_id: new_thread_id.clone(), @@ -7153,8 +7292,16 @@ impl FreshCodexState { }), name_ref, session_name, - }, - )); + }); + if let Err(error) = self.persist_checkpoint_frame(&materialized, false).await { + tracing::error!(target: "freshell_freshagent::codex", + old_session_id = %old_session_id, + new_session_id = %new_thread_id, + cause = %error, + "freshagent.codex.checkpoint_alias_persist_failed" + ); + } + self.broadcast(&materialized); // P1.13 §2.6b (Task 6): memory loss must be user-visible, not server-log-only. // Emitted AFTER the materialized broadcast above -- the frozen client re-keys @@ -8767,6 +8914,7 @@ impl FreshCodexState { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -8860,6 +9008,7 @@ impl FreshCodexState { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -9695,6 +9844,12 @@ fn checkpoint_codex_turn_identity( if matching.next().is_some() { return None; } + if !matches!( + raw_turn.get("status").and_then(Value::as_str), + Some("completed" | "interrupted" | "failed") + ) { + return None; + } let projected = build_codex_turn_json(raw_turn, 0).ok()?; let mut user_rows = projected .iter() @@ -11913,6 +12068,7 @@ pub(crate) mod tests { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -12021,6 +12177,7 @@ pub(crate) mod tests { kill_tx: Some(kill_tx), watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), sidecar_pid, sidecar_ownership_id, quiet_deadman, @@ -14032,6 +14189,7 @@ pub(crate) mod tests { kill_tx: None, watcher, exited, + checkpoint_probe_cancelled: Arc::new(AtomicBool::new(false)), quiet_deadman: QuietDeadman::new_shared(), turn_in_flight: Arc::new(AtomicBool::new(false)), retired_turn_id: Arc::new(StdMutex::new(None)), @@ -26805,6 +26963,16 @@ pub(crate) mod tests { None, "a snapshot row from another send cannot satisfy this request" ); + let mut in_progress_turn = raw_turn.clone(); + in_progress_turn["status"] = json!("inProgress"); + assert_eq!( + checkpoint_codex_turn_identity( + &json!({"thread":{"turns":[in_progress_turn]}}), + "turn-native-1" + ), + None, + "an in-progress turn's row id can still change when Codex adds its assistant row" + ); let ambiguous = json!({ "id": "turn-native-ambiguous", @@ -26836,13 +27004,27 @@ pub(crate) mod tests { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, _notifs) = CodexAppServerClient::connect(transport); let (state, mut rx) = state_with_bus(); - state.spawn_checkpoint_identity_probe( - Arc::new(client), - "thread-checkpoint-receipt".to_string(), - "turn-persisted-user".to_string(), - "request-checkpoint-1".to_string(), - "logical-checkpoint-1".to_string(), + let active_turn = Arc::new(StdMutex::new(Some("turn-persisted-user".to_string()))); + drop( + state.spawn_checkpoint_identity_probe(CodexCheckpointIdentityProbe { + client: Arc::new(client), + session_id: "thread-checkpoint-receipt".to_string(), + submitted_turn_id: "turn-persisted-user".to_string(), + request_id: "request-checkpoint-1".to_string(), + logical_attempt_id: "logical-checkpoint-1".to_string(), + active_turn: active_turn.clone(), + exited: Arc::new(AtomicBool::new(false)), + cancelled: Arc::new(AtomicBool::new(false)), + }), + ); + + assert!( + tokio::time::timeout(Duration::from_millis(50), peer.expect_request()) + .await + .is_err(), + "identity reads wait until the accepted turn has retired" ); + *active_turn.lock().expect("active turn mutex") = None; let (initialize_id, initialize_method, _) = peer.expect_request().await; assert_eq!(initialize_method, "initialize"); @@ -26851,27 +27033,38 @@ pub(crate) mod tests { json!({ "userAgent": "test", "codexHome": "/h", "platformFamily": "u", "platformOs": "l" }), ); let _ = peer.expect_notification().await; - let (first_read_id, first_read_method, first_read_params) = peer.expect_request().await; - assert_eq!(first_read_method, "thread/read"); - assert_eq!(first_read_params["threadId"], "thread-checkpoint-receipt"); - assert_eq!(first_read_params["includeTurns"], true); - peer.respond( - &first_read_id, - json!({ - "thread": { - "id": "thread-checkpoint-receipt", - "status": "running", - "turns": [] - } - }), - ); + for attempt in 1..=12 { + let (read_id, read_method, read_params) = + tokio::time::timeout(Duration::from_secs(2), peer.expect_request()) + .await + .expect("probe should continue while the user row is not persisted"); + assert_eq!(read_method, "thread/read", "attempt {attempt}"); + assert_eq!(read_params["threadId"], "thread-checkpoint-receipt"); + assert_eq!(read_params["includeTurns"], true); + peer.respond( + &read_id, + json!({ + "thread": { + "id": "thread-checkpoint-receipt", + "status": "running", + "turns": [] + } + }), + ); + } - let (read_id, read_method, read_params) = peer.expect_request().await; - assert_eq!(read_method, "thread/read"); - assert_eq!(read_params["threadId"], "thread-checkpoint-receipt"); - assert_eq!(read_params["includeTurns"], true); + let (persisted_read_id, persisted_read_method, persisted_read_params) = + tokio::time::timeout(Duration::from_secs(2), peer.expect_request()) + .await + .expect("probe should continue after twelve empty history reads"); + assert_eq!(persisted_read_method, "thread/read"); + assert_eq!( + persisted_read_params["threadId"], + "thread-checkpoint-receipt" + ); + assert_eq!(persisted_read_params["includeTurns"], true); peer.respond( - &read_id, + &persisted_read_id, json!({ "thread": { "id": "thread-checkpoint-receipt", @@ -26914,12 +27107,17 @@ pub(crate) mod tests { let (transport, peer) = freshell_codex::new_channel_transport(); let (client, _notifs) = CodexAppServerClient::connect(transport); let (state, mut rx) = state_with_bus(); - state.spawn_checkpoint_identity_probe( - Arc::new(client), - "thread-checkpoint-read-retry".to_string(), - "turn-persisted-after-retry".to_string(), - "request-checkpoint-retry".to_string(), - "logical-checkpoint-retry".to_string(), + drop( + state.spawn_checkpoint_identity_probe(CodexCheckpointIdentityProbe { + client: Arc::new(client), + session_id: "thread-checkpoint-read-retry".to_string(), + submitted_turn_id: "turn-persisted-after-retry".to_string(), + request_id: "request-checkpoint-retry".to_string(), + logical_attempt_id: "logical-checkpoint-retry".to_string(), + active_turn: Arc::new(StdMutex::new(None)), + exited: Arc::new(AtomicBool::new(false)), + cancelled: Arc::new(AtomicBool::new(false)), + }), ); let (initialize_id, initialize_method, _) = peer.expect_request().await; @@ -26969,6 +27167,83 @@ pub(crate) mod tests { assert_eq!(frame["event"]["nativeTurnId"], "turn-persisted-after-retry"); } + #[tokio::test] + async fn checkpoint_identity_probe_stops_when_its_session_is_killed_mid_turn() { + let (transport, peer) = freshell_codex::new_channel_transport(); + let (client, _notifs) = CodexAppServerClient::connect(transport); + let client = Arc::new(client); + let (state, _rx) = state_with_bus(); + let active_turn = Arc::new(StdMutex::new(Some("turn-stopped".to_string()))); + insert_fake_session( + &state, + "thread-stopped-checkpoint", + client.clone(), + active_turn.clone(), + spawn_sleeper(), + "codex-sidecar-checkpoint-stop", + ) + .await; + let (cancelled, exited) = { + let sessions = state.sessions.lock().await; + let session = sessions + .get("thread-stopped-checkpoint") + .expect("fixture session inserted"); + ( + session.checkpoint_probe_cancelled.clone(), + session.exited.clone(), + ) + }; + let mut probe = state.spawn_checkpoint_identity_probe(CodexCheckpointIdentityProbe { + client, + session_id: "thread-stopped-checkpoint".to_string(), + submitted_turn_id: "turn-stopped".to_string(), + request_id: "request-stopped".to_string(), + logical_attempt_id: "logical-stopped".to_string(), + active_turn, + exited, + cancelled: cancelled.clone(), + }); + + assert!( + tokio::time::timeout(Duration::from_millis(50), peer.expect_request()) + .await + .is_err(), + "the probe should wait for the in-flight turn before reading history" + ); + // Closing the client drains its reader task; dropping the fake provider lets its + // receive future finish so the fixture's transport close can complete. + drop(peer); + + state + .handle_kill(FreshAgentKill { + observed_epoch: None, + observed_generation: None, + provider: freshell_protocol::AgentProvider::Codex, + session_id: "thread-stopped-checkpoint".to_string(), + session_type: freshell_protocol::SessionType::Freshcodex, + cwd: None, + }) + .await; + + let probe_stopped = tokio::time::timeout(Duration::from_secs(1), &mut probe) + .await + .is_ok(); + if !probe_stopped { + // Keep this regression leak-free while it is red: the pre-fix kill path + // leaves the probe polling, so explicitly release it after the assertion + // window and then verify that it settles. + cancelled.store(true, Ordering::SeqCst); + tokio::time::timeout(Duration::from_secs(1), &mut probe) + .await + .expect("test cleanup releases the probe") + .expect("probe task joins"); + } + assert!( + probe_stopped, + "killing a session must cancel its checkpoint probe instead of retaining it forever" + ); + } + #[test] fn build_codex_turn_json_empty_response_synthetic_row_also_carries_assistant_role() { // A completed turn with ONLY user-role items and no assistant output gets a synthetic diff --git a/crates/freshell-freshagent/src/lib.rs b/crates/freshell-freshagent/src/lib.rs index 56df42889..f73e21e4d 100644 --- a/crates/freshell-freshagent/src/lib.rs +++ b/crates/freshell-freshagent/src/lib.rs @@ -102,6 +102,52 @@ pub use naming::{ /// Defaults to "always false" (behavior-preserving for constructors that don't wire it). pub type TerminalLivenessProbe = std::sync::Arc<dyn Fn(&str, &str) -> bool + Send + Sync>; +/// Server-owned durability hook for checkpoint-relevant provider frames. The +/// fresh-agent crate sends the frame through this hook before broadcasting it, +/// while the concrete checkpoint store remains owned by `freshell-server`. +pub type CheckpointEventPersister = std::sync::Arc< + dyn Fn(String, bool) -> futures_util::future::BoxFuture<'static, Result<(), String>> + + Send + + Sync, +>; + +/// Clone-shared, set-once bridge from provider producers to the server-owned +/// checkpoint store. A required match means this frame carries an identity for +/// a checkpointed request; persistence must refuse a missing association rather +/// than report success and expose an ID that will disappear after restart. +#[derive(Clone, Default)] +pub struct CheckpointEventSink(Arc<std::sync::OnceLock<CheckpointEventPersister>>); + +impl CheckpointEventSink { + pub fn set_persister(&self, persister: CheckpointEventPersister) -> Result<(), &'static str> { + self.0 + .set(persister) + .map_err(|_| "checkpoint event persister is already installed") + } + + pub async fn persist_before_broadcast( + &self, + frame: &str, + require_match: bool, + ) -> Result<(), String> { + if let Some(persist) = self.0.get() { + persist(frame.to_string(), require_match).await?; + } + Ok(()) + } + + pub async fn broadcast( + &self, + broadcast_tx: &tokio::sync::broadcast::Sender<String>, + frame: String, + require_match: bool, + ) -> Result<(), String> { + self.persist_before_broadcast(&frame, require_match).await?; + let _ = broadcast_tx.send(frame); + Ok(()) + } +} + /// Door 3 (resume-validation): the gate-fired callback shape -- /// `(provider, stale_session_id)`. In production, `freshell-server`'s main.rs /// implements it as pane-ledger `retire_missing` + `tracing::warn!`. @@ -1975,6 +2021,7 @@ pub struct FreshAgentState { /// `freshell-ws` connection. `ui.command` / `freshAgent.session.materialized` / /// `sessions.changed` are pushed here. broadcast_tx: Arc<tokio::sync::broadcast::Sender<String>>, + checkpoint_event_sink: CheckpointEventSink, /// paneId → pane record (placeholder id, cwd, model/effort, durable id). panes: Arc<Mutex<HashMap<String, PaneEntry>>>, /// The single lazily-started `opencode serve` client for this server process. @@ -2297,6 +2344,7 @@ impl FreshAgentState { Self { auth_token, broadcast_tx, + checkpoint_event_sink: CheckpointEventSink::default(), panes: Arc::new(Mutex::new(HashMap::new())), opencode: Arc::new(tokio::sync::Mutex::new(None)), hosted_rest: Arc::new(std::sync::OnceLock::new()), @@ -2346,6 +2394,15 @@ impl FreshAgentState { self } + /// Wire the server-owned checkpoint store before provider states clone + /// this shared REST state. + pub fn set_checkpoint_event_persister( + &self, + persister: CheckpointEventPersister, + ) -> Result<(), &'static str> { + self.checkpoint_event_sink.set_persister(persister) + } + /// Side-effect-free coordinator read for `(provider, session_id)` /// (kata b8ke Task 3): the lane's snapshot of who owns the key. /// `Vacant`/0 when the registry is unwired (every pre-existing test). @@ -2776,6 +2833,26 @@ impl FreshAgentState { } } + pub(crate) async fn persist_checkpoint_frame( + &self, + frame: &str, + require_match: bool, + ) -> Result<(), String> { + self.checkpoint_event_sink + .persist_before_broadcast(frame, require_match) + .await + } + + pub(crate) async fn broadcast_checkpoint_frame( + &self, + frame: String, + require_match: bool, + ) -> Result<(), String> { + self.checkpoint_event_sink + .broadcast(&self.broadcast_tx, frame, require_match) + .await + } + /// b8ke focused round-3 review R3-1: register the REST-driven turn /// witness for a canonical durable `ses_*` id, replacing any leftover /// entry an earlier timed-out drive left (the witness tracks the @@ -7251,6 +7328,35 @@ mod in_flight_registry_tests { } } +#[cfg(test)] +mod checkpoint_event_sink_tests { + use super::*; + + #[tokio::test] + async fn persistence_failure_withholds_provider_identity_frame() { + let (tx, mut rx) = tokio::sync::broadcast::channel(4); + let sink = CheckpointEventSink::default(); + sink.set_persister(Arc::new(|_, _| { + Box::pin(async { Err("checkpoint store is unavailable".to_string()) }) + })) + .unwrap(); + + let result = sink + .broadcast(&tx, "checkpoint identity with native id".to_string(), true) + .await; + + assert_eq!( + result.unwrap_err(), + "checkpoint store is unavailable", + "the producer must learn that its identity receipt was not made durable" + ); + assert!(matches!( + rx.try_recv(), + Err(tokio::sync::broadcast::error::TryRecvError::Empty) + )); + } +} + /// RAII holder for a claimed fresh-agent session lease (Task 12, mirror of /// `SessionRefLeaseGuard` in `freshell-ws/src/terminal.rs`). `Drop` calls `fail()` ONLY /// while armed AND no kill handle was recorded: once a live child exists (kill handle diff --git a/crates/freshell-freshagent/src/opencode_ws.rs b/crates/freshell-freshagent/src/opencode_ws.rs index d2a0dea6b..819d02974 100644 --- a/crates/freshell-freshagent/src/opencode_ws.rs +++ b/crates/freshell-freshagent/src/opencode_ws.rs @@ -1224,6 +1224,41 @@ impl FreshOpencodeState { self.fresh_agent.broadcast(msg); } + async fn broadcast_materialized(&self, msg: &ServerMessage) { + if let Ok(frame) = serde_json::to_string(msg) { + if let Err(error) = self + .fresh_agent + .persist_checkpoint_frame(&frame, false) + .await + { + tracing::error!(target: "freshell_freshagent::opencode", + cause = %error, + "freshagent.opencode.checkpoint_alias_persist_failed" + ); + } + } + self.broadcast(msg); + } + + async fn broadcast_checkpoint_frame(&self, msg: &ServerMessage) { + let Ok(frame) = serde_json::to_string(msg) else { + tracing::error!(target: "freshell_freshagent::opencode", + "freshagent.opencode.checkpoint_identity_serialize_failed" + ); + return; + }; + if let Err(error) = self + .fresh_agent + .broadcast_checkpoint_frame(frame, true) + .await + { + tracing::error!(target: "freshell_freshagent::opencode", + cause = %error, + "freshagent.opencode.checkpoint_identity_persist_failed" + ); + } + } + /// The shared P1.13 binding-row write (materialization row, settings refresh, /// resume refresh, forked child): AWAITED (durable-before-answer), and a write /// failure surfaces as a user-visible `LEDGER_WRITE_FAILED` `freshAgent.error` @@ -2548,12 +2583,13 @@ impl FreshOpencodeState { // `freshAgent.session.materialized` (ws-handler.ts:3477-3484): placeholder -> // durable, emitted EXACTLY ONCE (a later send never re-enters this branch). - self.broadcast(&materialized_frame( + self.broadcast_materialized(&materialized_frame( &session.placeholder_id, &durable_id, name_ref, session_name, - )); + )) + .await; // PR-3: `bindServeStream(state)` (adapter.ts:349) -- start the persistent // serve-SSE bridge ONCE, right after materialization. A later send never @@ -2803,7 +2839,7 @@ impl FreshOpencodeState { if let Some(native_id) = observe_checkpoint_turn_identity(&manager, &real_id, &route, identity).await { - this.broadcast(&event_frame( + this.broadcast_checkpoint_frame(&event_frame( &real_id, json!({ "type": "checkpointTurnIdentity", @@ -2813,7 +2849,8 @@ impl FreshOpencodeState { "nativeTurnId": native_id, "displayTurnId": identity.native_message_id, }), - )); + )) + .await; } } } @@ -7106,12 +7143,13 @@ impl FreshOpencodeState { let projection = self.created_projection_for(real_id).await; let name_ref = projection.as_ref().map(|(r, _)| r.clone()); let session_name = projection.map(|(_, record)| record); - self.broadcast(&materialized_frame( + self.broadcast_materialized(&materialized_frame( &msg.session_id, real_id, name_ref, session_name, - )); + )) + .await; } } diff --git a/crates/freshell-server/Cargo.toml b/crates/freshell-server/Cargo.toml index a2fa29b8c..eeee40c92 100644 --- a/crates/freshell-server/Cargo.toml +++ b/crates/freshell-server/Cargo.toml @@ -166,6 +166,7 @@ windows-sys = { version = "=0.61.2", features = [ "Win32_System_Diagnostics_ToolHelp", "Win32_System_JobObjects", "Win32_System_Threading", + "Win32_Storage_FileSystem", ] } [target.'cfg(unix)'.dev-dependencies] diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index e02f8992f..fcece9976 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -28,7 +28,7 @@ use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; #[cfg(unix)] use std::collections::HashSet; -use std::collections::{BTreeMap, HashMap}; +use std::collections::{BTreeMap, BTreeSet, HashMap}; use std::sync::atomic::{AtomicBool, Ordering}; use axum::routing::get as get_method; @@ -96,12 +96,14 @@ const CHECKPOINT_AUTO_BUDGET: Duration = Duration::from_secs(15); const CHECKPOINT_EXPLICIT_BUDGET: Duration = Duration::from_secs(120); const CHECKPOINT_TIMEOUT_COOLDOWN: Duration = Duration::from_secs(60); const CHECKPOINT_COOLDOWN_LIMIT: usize = 128; -const CHECKPOINT_OUTCOME_STORE_VERSION: u8 = 1; +const CHECKPOINT_OUTCOME_STORE_VERSION: u8 = 2; +const CHECKPOINT_OUTCOME_STORE_LEGACY_VERSION: u8 = 1; const CHECKPOINT_OUTCOME_STORE_LIMIT: usize = 8192; const CHECKPOINT_OUTCOME_REPLAY_LIMIT: usize = 8192; const CHECKPOINT_OUTCOME_REPLAY_PAGE_LIMIT: usize = 200; const CHECKPOINT_DISPATCH_FENCE_LIMIT: usize = CHECKPOINT_OUTCOME_STORE_LIMIT; const CHECKPOINT_VOLATILE_FENCE_LIMIT: usize = 1024; +const CHECKPOINT_DISPATCH_FENCE_DIRECTORY: &str = "attempt-dispatch-fences-v1"; #[derive(Clone, Copy)] struct CheckpointAttemptFence { @@ -112,6 +114,8 @@ struct CheckpointAttemptFence { #[derive(Default)] struct CheckpointOutcomeCache { outcomes: HashMap<String, StoredCheckpointOutcome>, + dispatch_fences: BTreeSet<String>, + has_pruned_dispatch_fences: bool, snapshot_loaded: bool, store_file_observed: bool, snapshot_complete: bool, @@ -121,7 +125,22 @@ impl CheckpointOutcomeCache { fn replace_from_store(&mut self, store: &CheckpointOutcomeStore) { self.snapshot_loaded = true; self.store_file_observed = true; - self.snapshot_complete = store.outcomes.len() <= CHECKPOINT_DISPATCH_FENCE_LIMIT; + let mut dispatch_fences = store.dispatch_fences.clone(); + dispatch_fences.extend( + store + .outcomes + .iter() + .filter(|(_, record)| record.dispatch_started) + .map(|(key, _)| checkpoint_attempt_fence_key(key)), + ); + self.snapshot_complete = store.outcomes.len() <= CHECKPOINT_DISPATCH_FENCE_LIMIT + && dispatch_fences.len() <= CHECKPOINT_DISPATCH_FENCE_LIMIT + && !store.has_pruned_dispatch_fences; + self.has_pruned_dispatch_fences = store.has_pruned_dispatch_fences; + self.dispatch_fences = dispatch_fences + .into_iter() + .take(CHECKPOINT_DISPATCH_FENCE_LIMIT) + .collect(); self.outcomes = store .outcomes .iter() @@ -136,16 +155,30 @@ impl CheckpointOutcomeCache { } fn known_attempt(&self, key: &str) -> Option<CheckpointAttemptFence> { - self.outcomes.get(key).map(|record| CheckpointAttemptFence { - dispatch_started: record.dispatch_started, - capture_settled: record.capture_settled, - }) + self.outcomes + .get(key) + .map(|record| CheckpointAttemptFence { + dispatch_started: record.dispatch_started, + capture_settled: record.capture_settled, + }) + .or_else(|| { + self.cached_dispatch_fence(key) + .then_some(CheckpointAttemptFence { + dispatch_started: true, + capture_settled: true, + }) + }) } fn cached_outcome(&self, key: &str) -> Option<StoredCheckpointOutcome> { self.outcomes.get(key).cloned() } + fn cached_dispatch_fence(&self, key: &str) -> bool { + self.dispatch_fences + .contains(&checkpoint_attempt_fence_key(key)) + } + fn cached_outcomes(&self) -> BTreeMap<String, StoredCheckpointOutcome> { self.outcomes .iter() @@ -206,11 +239,155 @@ fn capture_settled_by_default() -> bool { true } -#[derive(Default, Serialize, Deserialize)] +#[derive(Clone, Default, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] struct CheckpointOutcomeStore { version: u8, outcomes: BTreeMap<String, StoredCheckpointOutcome>, + #[serde(default)] + dispatch_fences: BTreeSet<String>, + #[serde(default)] + has_pruned_dispatch_fences: bool, +} + +enum CheckpointAttemptLookup { + Outcome(Box<StoredCheckpointOutcome>), + DispatchFence, +} + +async fn publish_outcome_store_file_with_parent_sync<F, Fut>( + temporary: &Path, + destination: &Path, + sync_parent: F, +) -> Result<(), String> +where + F: FnOnce(PathBuf) -> Fut, + Fut: std::future::Future<Output = Result<(), String>>, +{ + publish_checkpoint_file(temporary, destination) + .await + .map_err(|error| format!("failed to publish checkpoint outcomes: {error}"))?; + let parent = destination + .parent() + .ok_or_else(|| "checkpoint outcome store has no parent directory".to_string())?; + sync_parent(parent.to_path_buf()).await?; + Ok(()) +} + +async fn publish_checkpoint_file(temporary: &Path, destination: &Path) -> std::io::Result<()> { + #[cfg(windows)] + { + use std::os::windows::ffi::OsStrExt; + + let source = temporary + .as_os_str() + .encode_wide() + .chain(std::iter::once(0)) + .collect::<Vec<_>>(); + let target = destination + .as_os_str() + .encode_wide() + .chain(std::iter::once(0)) + .collect::<Vec<_>>(); + let result = unsafe { + windows_sys::Win32::Storage::FileSystem::MoveFileExW( + source.as_ptr(), + target.as_ptr(), + windows_sys::Win32::Storage::FileSystem::MOVEFILE_REPLACE_EXISTING + | windows_sys::Win32::Storage::FileSystem::MOVEFILE_WRITE_THROUGH, + ) + }; + if result == 0 { + Err(std::io::Error::last_os_error()) + } else { + Ok(()) + } + } + #[cfg(not(windows))] + { + tokio::fs::rename(temporary, destination).await + } +} + +async fn sync_checkpoint_outcome_directory_chain( + parent: PathBuf, + home: PathBuf, +) -> Result<(), String> { + #[cfg(unix)] + { + let directories = checkpoint_outcome_directory_chain(&parent, &home)?; + for directory in directories { + tokio::fs::File::open(&directory) + .await + .map_err(|error| { + format!( + "failed to open checkpoint outcome directory {} for flush: {error}", + directory.display() + ) + })? + .sync_all() + .await + .map_err(|error| { + format!( + "failed to flush checkpoint outcome directory {}: {error}", + directory.display() + ) + })?; + } + Ok(()) + } + #[cfg(windows)] + { + // Windows publication uses MoveFileExW with MOVEFILE_WRITE_THROUGH, + // and each staged file is flushed before the move. Windows does not + // expose a portable equivalent of POSIX fsync(directory). + let _ = (parent, home); + Ok(()) + } + #[cfg(not(any(unix, windows)))] + { + let _ = (parent, home); + Err("durable checkpoint directory publication is unsupported on this platform".into()) + } +} + +#[cfg(any(unix, test))] +fn checkpoint_outcome_directory_chain(parent: &Path, home: &Path) -> Result<Vec<PathBuf>, String> { + if !parent.starts_with(home) { + return Err("checkpoint outcome directory is outside the configured home".into()); + } + let mut directories = Vec::new(); + let mut current = parent; + loop { + directories.push(current.to_path_buf()); + if current == home { + break; + } + current = current.parent().ok_or_else(|| { + "checkpoint outcome directory does not reach the configured home".to_string() + })?; + if !current.starts_with(home) { + return Err("checkpoint outcome directory does not reach the configured home".into()); + } + } + Ok(directories) +} + +async fn write_synced_checkpoint_file(path: &Path, contents: &[u8]) -> Result<(), String> { + let mut options = tokio::fs::OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + options.mode(0o600); + let mut file = options + .open(path) + .await + .map_err(|error| format!("failed to create checkpoint outcome file: {error}"))?; + file.write_all(contents) + .await + .map_err(|error| format!("failed to write checkpoint outcome file: {error}"))?; + file.sync_all() + .await + .map_err(|error| format!("failed to flush checkpoint outcome file: {error}")) } struct CheckpointAttemptJob { @@ -913,15 +1090,17 @@ impl CheckpointService { pub fn start_observer(self: &Arc<Self>) { let mut receiver = self.broadcast_tx.subscribe(); let service = Arc::downgrade(self); + let (observer_tx, mut observer_rx) = tokio::sync::mpsc::unbounded_channel::<String>(); + let processing_service = service.clone(); tokio::spawn(async move { loop { match receiver.recv().await { - Ok(frame) => { - let Some(service) = service.upgrade() else { + Ok(frame) if is_checkpoint_outcome_observer_frame(&frame) => { + if observer_tx.send(frame).is_err() { break; - }; - service.observe_broadcast_frame(&frame).await; + } } + Ok(_) => {} Err(tokio::sync::broadcast::error::RecvError::Lagged(count)) => { tracing::warn!( event = "code_checkpoint.outcome_observer_lagged", @@ -933,6 +1112,14 @@ impl CheckpointService { } } }); + tokio::spawn(async move { + while let Some(frame) = observer_rx.recv().await { + let Some(service) = processing_service.upgrade() else { + break; + }; + service.observe_broadcast_frame(&frame).await; + } + }); } fn outcome_store_path(&self) -> PathBuf { @@ -949,54 +1136,161 @@ impl CheckpointService { .join("attempt-outcomes-v1.initialized") } + fn dispatch_fence_root(&self) -> PathBuf { + self.home + .join(".freshell") + .join("checkpoints") + .join(CHECKPOINT_DISPATCH_FENCE_DIRECTORY) + } + + fn dispatch_fence_path(&self, fence: &str) -> Result<PathBuf, String> { + validate_checkpoint_dispatch_fence(fence)?; + Ok(self.dispatch_fence_root().join(&fence[..2]).join(fence)) + } + + async fn persist_dispatch_fence_hash(&self, fence: &str) -> Result<(), String> { + let destination = self.dispatch_fence_path(fence)?; + let parent = destination + .parent() + .ok_or_else(|| "checkpoint dispatch fence has no parent directory".to_string())?; + tokio::fs::create_dir_all(parent).await.map_err(|error| { + format!("failed to create checkpoint dispatch fence directory: {error}") + })?; + sync_checkpoint_outcome_directory_chain(parent.to_path_buf(), self.home.as_ref().clone()) + .await?; + + if let Ok(metadata) = tokio::fs::symlink_metadata(&destination).await { + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err("checkpoint dispatch fence has an unsafe file type".into()); + } + let contents = tokio::fs::read(&destination) + .await + .map_err(|error| format!("failed to read checkpoint dispatch fence: {error}"))?; + if contents != fence.as_bytes() { + return Err("checkpoint dispatch fence contents do not match its key".into()); + } + tokio::fs::File::open(&destination) + .await + .map_err(|error| format!("failed to open checkpoint dispatch fence: {error}"))? + .sync_all() + .await + .map_err(|error| format!("failed to flush checkpoint dispatch fence: {error}"))?; + return sync_checkpoint_outcome_directory_chain( + parent.to_path_buf(), + self.home.as_ref().clone(), + ) + .await; + } + + let temporary = parent.join(format!("{fence}.tmp-{}", uuid::Uuid::new_v4())); + if let Err(error) = write_synced_checkpoint_file(&temporary, fence.as_bytes()).await { + let _ = tokio::fs::remove_file(&temporary).await; + return Err(error); + } + if let Err(error) = publish_checkpoint_file(&temporary, &destination).await { + let _ = tokio::fs::remove_file(&temporary).await; + return Err(format!( + "failed to publish checkpoint dispatch fence: {error}" + )); + } + sync_checkpoint_outcome_directory_chain(parent.to_path_buf(), self.home.as_ref().clone()) + .await + } + + async fn dispatch_fence_exists(&self, fence: &str, expected: bool) -> Result<bool, String> { + let root = self.dispatch_fence_root(); + match tokio::fs::symlink_metadata(&root).await { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + Ok(_) => return Err("checkpoint dispatch fence directory has an unsafe type".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound && !expected => { + return Ok(false) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + return Err( + "checkpoint dispatch fence directory is missing after initialization".into(), + ) + } + Err(error) => { + return Err(format!( + "failed to inspect checkpoint dispatch fence directory: {error}" + )) + } + } + let path = self.dispatch_fence_path(fence)?; + match tokio::fs::symlink_metadata(&path).await { + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => { + let contents = tokio::fs::read(&path).await.map_err(|error| { + format!("failed to read checkpoint dispatch fence: {error}") + })?; + if contents != fence.as_bytes() { + return Err("checkpoint dispatch fence contents do not match its key".into()); + } + Ok(true) + } + Ok(_) => Err("checkpoint dispatch fence has an unsafe file type".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(error) => Err(format!( + "failed to inspect checkpoint dispatch fence: {error}" + )), + } + } + async fn ensure_outcome_store_presence_marker(&self) -> Result<(), String> { let path = self.outcome_store_presence_marker_path(); - #[cfg(unix)] - let open_result = { - let mut options = tokio::fs::OpenOptions::new(); - options.write(true).create_new(true).mode(0o600); - options.open(&path).await - }; - #[cfg(not(unix))] - let open_result = tokio::fs::OpenOptions::new() - .write(true) - .create_new(true) - .open(&path) - .await; + let parent = path.parent().ok_or_else(|| { + "checkpoint outcome presence marker has no parent directory".to_owned() + })?; + tokio::fs::create_dir_all(parent) + .await + .map_err(|error| format!("failed to create checkpoint outcome directory: {error}"))?; + sync_checkpoint_outcome_directory_chain(parent.to_path_buf(), self.home.as_ref().clone()) + .await?; - let mut marker = match open_result { - Ok(marker) => marker, - Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => return Ok(()), + match tokio::fs::symlink_metadata(&path).await { + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => { + let marker = tokio::fs::read(&path).await.map_err(|error| { + format!("failed to read checkpoint outcome presence marker: {error}") + })?; + if marker == b"freshell checkpoint outcomes initialized v1\n" { + return sync_checkpoint_outcome_directory_chain( + parent.to_path_buf(), + self.home.as_ref().clone(), + ) + .await; + } + } + Ok(_) => { + return Err("checkpoint outcome presence marker has an unsafe file type".into()) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} Err(error) => { return Err(format!( - "failed to create checkpoint outcome presence marker: {error}" - )); + "failed to inspect checkpoint outcome presence marker: {error}" + )) } - }; - marker - .write_all(b"freshell checkpoint outcomes initialized v1\n") + } + + let temporary = parent.join(format!( + "attempt-outcomes-v1.initialized.tmp-{}", + uuid::Uuid::new_v4() + )); + if let Err(error) = write_synced_checkpoint_file( + &temporary, + b"freshell checkpoint outcomes initialized v1\n", + ) + .await + { + let _ = tokio::fs::remove_file(&temporary).await; + return Err(error); + } + if let Err(error) = + publish_outcome_store_file_with_parent_sync(&temporary, &path, |parent| { + sync_checkpoint_outcome_directory_chain(parent, self.home.as_ref().clone()) + }) .await - .map_err(|error| { - format!("failed to write checkpoint outcome presence marker: {error}") - })?; - marker.sync_all().await.map_err(|error| { - format!("failed to flush checkpoint outcome presence marker: {error}") - })?; - #[cfg(unix)] { - let parent = path.parent().ok_or_else(|| { - "checkpoint outcome presence marker has no parent directory".to_owned() - })?; - tokio::fs::File::open(parent) - .await - .map_err(|error| { - format!("failed to open checkpoint outcome directory for flush: {error}") - })? - .sync_all() - .await - .map_err(|error| { - format!("failed to flush checkpoint outcome directory: {error}") - })?; + let _ = tokio::fs::remove_file(&temporary).await; + return Err(error); } Ok(()) } @@ -1134,6 +1428,8 @@ impl CheckpointService { return Ok(CheckpointOutcomeStore { version: CHECKPOINT_OUTCOME_STORE_VERSION, outcomes: BTreeMap::new(), + dispatch_fences: BTreeSet::new(), + has_pruned_dispatch_fences: false, }); } Err(error) => { @@ -1157,11 +1453,24 @@ impl CheckpointService { "checkpoint outcome store was readable but its durable presence marker could not be written" ); } - let store: CheckpointOutcomeStore = serde_json::from_slice(&raw) + let mut store: CheckpointOutcomeStore = serde_json::from_slice(&raw) .map_err(|error| format!("invalid checkpoint attempt outcome store: {error}"))?; - if store.version != CHECKPOINT_OUTCOME_STORE_VERSION { + if store.version != CHECKPOINT_OUTCOME_STORE_VERSION + && store.version != CHECKPOINT_OUTCOME_STORE_LEGACY_VERSION + { return Err("unsupported checkpoint attempt outcome store version".to_string()); } + if store.version == CHECKPOINT_OUTCOME_STORE_LEGACY_VERSION + || !store.dispatch_fences.is_empty() + { + let legacy_fences = std::mem::take(&mut store.dispatch_fences); + for fence in &legacy_fences { + self.persist_dispatch_fence_hash(fence).await?; + } + store.has_pruned_dispatch_fences |= !legacy_fences.is_empty(); + store.version = CHECKPOINT_OUTCOME_STORE_VERSION; + self.write_outcome_store(&store).await?; + } self.durable_outcome_cache .lock() .expect("checkpoint dispatch fences mutex poisoned") @@ -1170,6 +1479,7 @@ impl CheckpointService { } async fn write_outcome_store(&self, store: &CheckpointOutcomeStore) -> Result<(), String> { + let mut store = store.clone(); let path = self.outcome_store_path(); let parent = path .parent() @@ -1177,44 +1487,55 @@ impl CheckpointService { tokio::fs::create_dir_all(parent) .await .map_err(|error| format!("failed to create checkpoint outcome directory: {error}"))?; + sync_checkpoint_outcome_directory_chain(parent.to_path_buf(), self.home.as_ref().clone()) + .await?; self.ensure_outcome_store_presence_marker().await?; + + for fence in std::mem::take(&mut store.dispatch_fences) { + self.persist_dispatch_fence_hash(&fence).await?; + store.has_pruned_dispatch_fences = true; + } + while store.outcomes.len() > CHECKPOINT_OUTCOME_STORE_LIMIT { + let oldest = store + .outcomes + .iter() + .min_by_key(|(_, item)| item.created_at_ms) + .map(|(key, item)| (key.clone(), item.dispatch_started)); + let Some((oldest, dispatch_started)) = oldest else { + break; + }; + if dispatch_started { + self.persist_dispatch_fence_hash(&checkpoint_attempt_fence_key(&oldest)) + .await?; + store.has_pruned_dispatch_fences = true; + } + store.outcomes.remove(&oldest); + } + store.version = CHECKPOINT_OUTCOME_STORE_VERSION; + store.dispatch_fences.clear(); let temporary = parent.join(format!( "attempt-outcomes-v1.json.tmp-{}", uuid::Uuid::new_v4() )); - let serialized = serde_json::to_vec_pretty(store) + let serialized = serde_json::to_vec_pretty(&store) .map_err(|error| format!("failed to serialize checkpoint outcomes: {error}"))?; - #[cfg(unix)] - let write_result = { - let mut options = tokio::fs::OpenOptions::new(); - options.write(true).create_new(true).mode(0o600); - match options.open(&temporary).await { - Ok(mut file) => match file.write_all(&serialized).await { - Ok(()) => file - .sync_all() - .await - .map_err(|error| format!("failed to flush checkpoint outcomes: {error}")), - Err(error) => Err(format!("failed to write checkpoint outcomes: {error}")), - }, - Err(error) => Err(format!("failed to create checkpoint outcome file: {error}")), - } - }; - #[cfg(not(unix))] - let write_result = tokio::fs::write(&temporary, &serialized) - .await - .map_err(|error| format!("failed to write checkpoint outcomes: {error}")); - if let Err(error) = write_result { + if let Err(error) = write_synced_checkpoint_file(&temporary, &serialized).await { let _ = tokio::fs::remove_file(&temporary).await; return Err(error); } - if let Err(error) = tokio::fs::rename(&temporary, &path).await { + if let Err(error) = + publish_outcome_store_file_with_parent_sync(&temporary, &path, |parent| { + sync_checkpoint_outcome_directory_chain(parent, self.home.as_ref().clone()) + }) + .await + { let _ = tokio::fs::remove_file(&temporary).await; - return Err(format!("failed to publish checkpoint outcomes: {error}")); + return Err(error); } self.durable_outcome_cache .lock() .expect("checkpoint dispatch fences mutex poisoned") - .replace_from_store(store); + .replace_from_store(&store); Ok(()) } @@ -1226,15 +1547,6 @@ impl CheckpointService { let _guard = self.outcome_gate.lock().await; let mut store = self.read_outcome_store().await?; store.outcomes.insert(key.to_string(), outcome.clone()); - while store.outcomes.len() > CHECKPOINT_OUTCOME_STORE_LIMIT { - let oldest = store - .outcomes - .iter() - .min_by_key(|(_, item)| item.created_at_ms) - .map(|(key, _)| key.clone()); - let Some(oldest) = oldest else { break }; - store.outcomes.remove(&oldest); - } self.write_outcome_store(&store).await?; self.volatile_outcomes .lock() @@ -1322,7 +1634,7 @@ impl CheckpointService { ); } - async fn find_outcome(&self, key: &str) -> Result<Option<StoredCheckpointOutcome>, String> { + async fn find_attempt(&self, key: &str) -> Result<Option<CheckpointAttemptLookup>, String> { if let Some(outcome) = self .volatile_outcomes .lock() @@ -1330,19 +1642,63 @@ impl CheckpointService { .get(key) .cloned() { - return Ok(Some(outcome)); + return Ok(Some(CheckpointAttemptLookup::Outcome(Box::new(outcome)))); } - if let Some(outcome) = self - .durable_outcome_cache - .lock() - .expect("durable checkpoint outcome cache mutex poisoned") - .cached_outcome(key) { - return Ok(Some(outcome)); + let cache = self + .durable_outcome_cache + .lock() + .expect("durable checkpoint outcome cache mutex poisoned"); + if let Some(outcome) = cache.cached_outcome(key) { + return Ok(Some(CheckpointAttemptLookup::Outcome(Box::new(outcome)))); + } + if cache.cached_dispatch_fence(key) { + return Ok(Some(CheckpointAttemptLookup::DispatchFence)); + } } let _guard = self.outcome_gate.lock().await; - let store = self.read_outcome_store().await?; - Ok(store.outcomes.get(key).cloned()) + let store = match self.read_outcome_store().await { + Ok(store) => store, + Err(error) => { + let has_pruned_fences = self + .durable_outcome_cache + .lock() + .expect("durable checkpoint outcome cache mutex poisoned") + .has_pruned_dispatch_fences; + if has_pruned_fences { + match self + .dispatch_fence_exists(&checkpoint_attempt_fence_key(key), true) + .await + { + Ok(true) => return Ok(Some(CheckpointAttemptLookup::DispatchFence)), + Ok(false) => {} + Err(fence_error) => { + return Err(format!("{error}; {fence_error}")); + } + } + } + return Err(error); + } + }; + if let Some(outcome) = store.outcomes.get(key).cloned() { + return Ok(Some(CheckpointAttemptLookup::Outcome(Box::new(outcome)))); + } + let fence = checkpoint_attempt_fence_key(key); + if store.dispatch_fences.contains(&fence) + || (store.has_pruned_dispatch_fences + && self.dispatch_fence_exists(&fence, true).await?) + { + return Ok(Some(CheckpointAttemptLookup::DispatchFence)); + } + Ok(None) + } + + #[cfg(test)] + async fn find_outcome(&self, key: &str) -> Result<Option<StoredCheckpointOutcome>, String> { + Ok(match self.find_attempt(key).await? { + Some(CheckpointAttemptLookup::Outcome(outcome)) => Some(*outcome), + Some(CheckpointAttemptLookup::DispatchFence) | None => None, + }) } async fn replay_outcome_page( @@ -1500,15 +1856,44 @@ impl CheckpointService { } async fn observe_broadcast_frame(&self, frame: &str) { - let Ok(message) = serde_json::from_str::<Value>(frame) else { - return; - }; + if let Err(error) = self.persist_event_before_broadcast(frame).await { + tracing::error!( + event = "code_checkpoint.outcome_observer_persist_failed", + cause = %error, + "checkpoint outcome observer could not persist a provider event" + ); + } + } + + /// Persist checkpoint-relevant provider events before a producer forwards them + /// to the shared broadcast bus. `require_match` is used for frames carrying an + /// identity promised by a checkpointed request; returning success without an + /// associated outcome would falsely claim that identity is durable. + pub async fn persist_event_before_broadcast(&self, frame: &str) -> Result<(), String> { + self.persist_event_before_broadcast_with_match(frame, false) + .await + } + + pub async fn persist_event_before_broadcast_with_match( + &self, + frame: &str, + require_match: bool, + ) -> Result<(), String> { + let message = serde_json::from_str::<Value>(frame) + .map_err(|error| format!("checkpoint event was not valid JSON: {error}"))?; match message.get("type").and_then(Value::as_str) { - Some("freshAgent.send.accepted") => self.observe_accepted(&message).await, - Some("freshAgent.session.materialized") => self.observe_materialized(&message).await, - Some("freshAgent.event") => self.observe_turn_identity(&message).await, + Some("freshAgent.send.accepted") => { + self.observe_accepted_with_match(&message, require_match) + .await? + } + Some("freshAgent.session.materialized") => self.observe_materialized(&message).await?, + Some("freshAgent.event") => { + self.observe_turn_identity_with_match(&message, require_match) + .await? + } _ => {} } + Ok(()) } async fn update_matching_outcomes<Matches, Update>( @@ -1535,32 +1920,54 @@ impl CheckpointService { Ok(changed) } - async fn observe_accepted(&self, message: &Value) { + #[cfg(test)] + async fn observe_accepted(&self, message: &Value) -> Result<(), String> { + self.observe_accepted_with_match(message, false).await + } + + async fn observe_accepted_with_match( + &self, + message: &Value, + require_match: bool, + ) -> Result<(), String> { + let missing_field = |field: &str| { + if require_match { + Err(format!( + "required checkpoint accepted frame is missing {field}" + )) + } else { + Ok(()) + } + }; let Some(request_id) = message.get("requestId").and_then(Value::as_str) else { - return; + missing_field("requestId")?; + return Ok(()); }; let Some(provider) = message .get("provider") .and_then(Value::as_str) .and_then(parse_agent_provider) else { - return; + missing_field("provider")?; + return Ok(()); }; let Some(session_type) = message .get("sessionType") .and_then(Value::as_str) .and_then(parse_session_type) else { - return; + missing_field("sessionType")?; + return Ok(()); }; let Some(session_id) = message.get("sessionId").and_then(Value::as_str) else { - return; + missing_field("sessionId")?; + return Ok(()); }; let submitted_turn_id = message .get("submittedTurnId") .and_then(Value::as_str) .map(str::to_string); - let updated = self + let changed = self .update_matching_outcomes( |record| { record.outcome.provider == provider @@ -1570,6 +1977,13 @@ impl CheckpointService { .session_aliases .iter() .any(|alias| alias == session_id) + && submitted_turn_id.as_deref().is_some_and(|submitted| { + record.submitted_turn_id.as_deref() != Some(submitted) + || (provider == AgentProvider::Codex + && (record.outcome.native_turn_id.as_deref() + != Some(submitted) + || record.outcome.display_turn_id.is_none())) + }) && submitted_turn_id.as_deref().is_none_or(|submitted| { record .outcome @@ -1582,88 +1996,150 @@ impl CheckpointService { if submitted_turn_id.is_some() { record.submitted_turn_id = submitted_turn_id.clone(); } + if provider == AgentProvider::Codex && record.outcome.native_turn_id.is_none() { + record.outcome.native_turn_id = submitted_turn_id.clone(); + if record.outcome.display_turn_id.is_none() { + // The client explicitly treats this as the provisional id + // for a Codex turn before transcript projection splits it + // into the final user row `:row-0`. + record.outcome.display_turn_id = submitted_turn_id.clone(); + } + } }, ) - .await; - if let Err(error) = updated { - tracing::error!( - event = "code_checkpoint.accepted_identity_persist_failed", - request_id, - cause = %error, - "accepted turn identity could not be persisted" - ); + .await + .map_err(|error| format!("accepted turn identity could not be persisted: {error}"))?; + if require_match && changed.is_empty() { + let already_durable = self + .accepted_identity_is_durable( + provider, + session_type, + request_id, + session_id, + submitted_turn_id.as_deref(), + ) + .await?; + if !already_durable { + return Err("accepted turn identity did not match a checkpoint outcome".into()); + } } + Ok(()) + } + + async fn accepted_identity_is_durable( + &self, + provider: AgentProvider, + session_type: SessionType, + request_id: &str, + session_id: &str, + submitted_turn_id: Option<&str>, + ) -> Result<bool, String> { + let _guard = self.outcome_gate.lock().await; + let store = self.read_outcome_store().await?; + Ok(store.outcomes.values().any(|record| { + record.outcome.provider == provider + && record.outcome.session_type == session_type + && record.request_ids.iter().any(|alias| alias == request_id) + && record + .session_aliases + .iter() + .any(|alias| alias == session_id) + && submitted_turn_id.is_none_or(|submitted| { + record.submitted_turn_id.as_deref() == Some(submitted) + && record.outcome.native_turn_id.as_deref() == Some(submitted) + }) + })) } - async fn observe_materialized(&self, message: &Value) { + async fn observe_materialized(&self, message: &Value) -> Result<(), String> { let Some(previous) = message.get("previousSessionId").and_then(Value::as_str) else { - return; + return Ok(()); }; let Some(session_id) = message.get("sessionId").and_then(Value::as_str) else { - return; + return Ok(()); }; let Some(provider) = message .get("provider") .and_then(Value::as_str) .and_then(parse_agent_provider) else { - return; + return Ok(()); }; let Some(session_type) = message .get("sessionType") .and_then(Value::as_str) .and_then(parse_session_type) else { - return; + return Ok(()); }; - let updated = self - .update_matching_outcomes( - |record| { - record.outcome.provider == provider - && record.outcome.session_type == session_type - && record.session_aliases.iter().any(|alias| alias == previous) - }, - |record| { - if !record - .session_aliases - .iter() - .any(|alias| alias == session_id) - { - record.session_aliases.push(session_id.to_string()); - } - }, - ) - .await; - if let Err(error) = updated { - tracing::error!( - event = "code_checkpoint.session_alias_persist_failed", - cause = %error, - "checkpoint session materialization alias could not be persisted" - ); - } + self.update_matching_outcomes( + |record| { + record.outcome.provider == provider + && record.outcome.session_type == session_type + && record.session_aliases.iter().any(|alias| alias == previous) + }, + |record| { + if !record + .session_aliases + .iter() + .any(|alias| alias == session_id) + { + record.session_aliases.push(session_id.to_string()); + } + }, + ) + .await + .map(|_| ()) + .map_err(|error| format!("checkpoint session alias could not be persisted: {error}")) + } + + #[cfg(test)] + async fn observe_turn_identity(&self, message: &Value) -> Result<(), String> { + self.observe_turn_identity_with_match(message, false).await } - async fn observe_turn_identity(&self, message: &Value) { + async fn observe_turn_identity_with_match( + &self, + message: &Value, + require_match: bool, + ) -> Result<(), String> { + let missing_field = |field: &str| { + if require_match { + Err(format!( + "required checkpoint identity frame is missing {field}" + )) + } else { + Ok(()) + } + }; let Some(event) = message.get("event") else { - return; + missing_field("event")?; + return Ok(()); }; if event.get("type").and_then(Value::as_str) != Some("checkpointTurnIdentity") { - return; + missing_field("event.type=checkpointTurnIdentity")?; + return Ok(()); } let Some(logical_attempt_id) = event.get("logicalAttemptId").and_then(Value::as_str) else { - return; + missing_field("event.logicalAttemptId")?; + return Ok(()); }; let Some(request_id) = event.get("requestId").and_then(Value::as_str) else { - return; + missing_field("event.requestId")?; + return Ok(()); }; let Some(native_turn_id) = event.get("nativeTurnId").and_then(Value::as_str) else { - return; + missing_field("event.nativeTurnId")?; + return Ok(()); }; let session_id = event .get("sessionId") .and_then(Value::as_str) .or_else(|| message.get("sessionId").and_then(Value::as_str)); - let Some(session_id) = session_id else { return }; + let Some(session_id) = session_id else { + missing_field("event.sessionId")?; + return Ok(()); + }; let display_turn_id = event .get("displayTurnId") .and_then(Value::as_str) @@ -1673,16 +2149,18 @@ impl CheckpointService { .and_then(Value::as_str) .and_then(parse_agent_provider) else { - return; + missing_field("provider")?; + return Ok(()); }; let Some(session_type) = message .get("sessionType") .and_then(Value::as_str) .and_then(parse_session_type) else { - return; + missing_field("sessionType")?; + return Ok(()); }; - let updated = self + let records = self .update_matching_outcomes( |record| { record.outcome.logical_attempt_id == logical_attempt_id @@ -1702,36 +2180,67 @@ impl CheckpointService { .native_turn_id .as_deref() .is_none_or(|existing| existing == native_turn_id) + && (record.outcome.native_turn_id.as_deref() != Some(native_turn_id) + || display_turn_id.as_deref().is_some_and(|display| { + record.outcome.display_turn_id.as_deref() != Some(display) + }) + || record.submitted_turn_id.as_deref() != Some(native_turn_id)) }, |record| { record.outcome.native_turn_id = Some(native_turn_id.to_string()); + // The provider receipt is emitted only after the native history + // row was matched to the exact turn id accepted for this request. + // Persist that accepted id with the receipt so a restart does not + // depend on the separately queued send.accepted frame. + if record.submitted_turn_id.is_none() { + record.submitted_turn_id = Some(native_turn_id.to_string()); + } if display_turn_id.is_some() { record.outcome.display_turn_id = display_turn_id.clone(); } }, ) - .await; - match updated { - Ok(records) => { - for record in records { - if record.request_ids.is_empty() { - self.broadcast_outcome(record.outcome.clone()); - } else { - for request_id in &record.request_ids { - let mut outcome = record.outcome.clone(); - outcome.request_id = request_id.clone(); - self.broadcast_outcome(outcome); - } - } + .await + .map_err(|error| { + format!("provider checkpoint turn identity could not be persisted: {error}") + })?; + if require_match && records.is_empty() { + let _guard = self.outcome_gate.lock().await; + let store = self.read_outcome_store().await?; + let already_durable = store.outcomes.values().any(|record| { + record.outcome.logical_attempt_id == logical_attempt_id + && record.outcome.provider == provider + && record.outcome.session_type == session_type + && record.request_ids.iter().any(|alias| alias == request_id) + && record + .session_aliases + .iter() + .any(|alias| alias == session_id) + && record.outcome.native_turn_id.as_deref() == Some(native_turn_id) + && record + .submitted_turn_id + .as_deref() + .is_none_or(|submitted| submitted == native_turn_id) + && display_turn_id.as_deref().is_none_or(|display| { + record.outcome.display_turn_id.as_deref() == Some(display) + }) + }); + if !already_durable { + return Err("provider turn identity did not match a checkpoint outcome".into()); + } + } + for record in records { + if record.request_ids.is_empty() { + self.broadcast_outcome(record.outcome.clone()); + } else { + for request_id in &record.request_ids { + let mut outcome = record.outcome.clone(); + outcome.request_id = request_id.clone(); + self.broadcast_outcome(outcome); } } - Err(error) => tracing::error!( - event = "code_checkpoint.turn_identity_persist_failed", - logical_attempt_id, - cause = %error, - "provider checkpoint turn identity could not be persisted" - ), } + Ok(()) } fn active_cooldown(&self, workspace: &Path) -> Option<Duration> { @@ -2613,6 +3122,23 @@ impl CheckpointService { } } +fn is_checkpoint_outcome_observer_frame(frame: &str) -> bool { + let Ok(message) = serde_json::from_str::<Value>(frame) else { + return false; + }; + match message.get("type").and_then(Value::as_str) { + Some("freshAgent.send.accepted") | Some("freshAgent.session.materialized") => true, + Some("freshAgent.event") => { + message + .get("event") + .and_then(|event| event.get("type")) + .and_then(Value::as_str) + == Some("checkpointTurnIdentity") + } + _ => false, + } +} + enum CaptureAttempt { Published(String), Failed(String), @@ -2652,6 +3178,20 @@ fn checkpoint_attempt_key(send: &FreshAgentSend, logical_attempt_id: &str) -> St ) } +fn checkpoint_attempt_fence_key(key: &str) -> String { + URL_SAFE_NO_PAD.encode(sha2::Sha256::digest(key.as_bytes())) +} + +fn validate_checkpoint_dispatch_fence(fence: &str) -> Result<(), String> { + let decoded = URL_SAFE_NO_PAD + .decode(fence) + .map_err(|_| "checkpoint dispatch fence key is invalid".to_string())?; + if decoded.len() != 32 || URL_SAFE_NO_PAD.encode(decoded) != fence { + return Err("checkpoint dispatch fence key is invalid".into()); + } + Ok(()) +} + fn parse_agent_provider(value: &str) -> Option<AgentProvider> { match value { "claude" => Some(AgentProvider::Claude), @@ -2748,8 +3288,9 @@ impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for Checkpoint } let mut volatile_fallback = false; - let (record, mut allow_forward) = match self.find_outcome(&key).await { - Ok(Some(mut record)) => { + let (record, mut allow_forward) = match self.find_attempt(&key).await { + Ok(Some(CheckpointAttemptLookup::Outcome(record))) => { + let mut record = *record; let added_alias = !record.request_ids.iter().any(|alias| alias == &request_id) && record.request_ids.len() < 16; if added_alias { @@ -2810,6 +3351,26 @@ impl freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier for Checkpoint } } } + Ok(Some(CheckpointAttemptLookup::DispatchFence)) => { + tracing::warn!( + event = "code_checkpoint.pruned_attempt_retry_blocked", + send_request_id = %request_id, + logical_attempt_id = %logical_attempt_id, + "retry was withheld because its earlier dispatch record aged out of outcome history" + ); + let mut record = self.new_record( + &send, + &logical_attempt_id, + &request_id, + None, + Err(( + "attempt_history_expired", + "This logical attempt was already sent, but its detailed rewind result has expired. Retry with a new logical attempt ID.".into(), + )), + ); + record.dispatch_started = true; + (record, false) + } Ok(None) => { let mut record = self .capture_automatic(&send, &request_id, &logical_attempt_id) @@ -5638,6 +6199,58 @@ async fn post_checkpoint_restore( mod tests { use super::*; + #[tokio::test] + async fn outcome_store_publish_requires_parent_directory_flush() { + use std::sync::atomic::{AtomicBool, Ordering}; + + let directory = tempfile::tempdir().unwrap(); + let temporary = directory.path().join("attempt-outcomes.tmp"); + let destination = directory.path().join("attempt-outcomes.json"); + tokio::fs::write(&temporary, b"durable dispatch fence") + .await + .unwrap(); + let sync_attempted = Arc::new(AtomicBool::new(false)); + let sync_attempted_in_callback = Arc::clone(&sync_attempted); + let expected_parent = directory.path().to_path_buf(); + let published_path = destination.clone(); + + let result = publish_outcome_store_file_with_parent_sync( + &temporary, + &destination, + move |parent| async move { + assert_eq!(parent, expected_parent); + assert_eq!( + tokio::fs::read(&published_path).await.unwrap(), + b"durable dispatch fence", + "the directory flush must follow the atomic rename" + ); + sync_attempted_in_callback.store(true, Ordering::SeqCst); + Err("injected parent directory sync failure".to_string()) + }, + ) + .await; + + assert!(sync_attempted.load(Ordering::SeqCst)); + assert!( + result + .unwrap_err() + .contains("injected parent directory sync failure"), + "a failed directory flush must prevent durable publication success" + ); + } + + #[test] + fn checkpoint_outcome_directory_flush_includes_new_home_ancestors() { + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let checkpoint_directory = home.join(".freshell/checkpoints"); + + assert_eq!( + checkpoint_outcome_directory_chain(&checkpoint_directory, &home).unwrap(), + vec![checkpoint_directory, home.join(".freshell"), home.clone(),] + ); + } + #[derive(Clone)] struct ReviewR6CaptureLogs( Arc<std::sync::Mutex<Vec<std::collections::BTreeMap<String, String>>>>, @@ -5920,67 +6533,231 @@ mod tests { use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; use std::sync::atomic::AtomicUsize; - let home = tempfile::tempdir().unwrap(); - let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); - let service = Arc::new(CheckpointService::new( + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let dispatches = Arc::new(AtomicUsize::new(0)); + let make_send = |request_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-1".into(), + session_type: SessionType::Freshopencode, + text: "the request body is never stored in checkpoint outcomes".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "logical-attempt-1".into(), + workspace: None, + }), + }; + + let dispatches_for_first = Arc::clone(&dispatches); + service + .clone() + .barrier_and_forward( + make_send("request-1"), + Box::pin(async move { + dispatches_for_first.fetch_add(1, Ordering::SeqCst); + }), + ) + .await; + service + .observe_accepted(&json!({ + "requestId": "request-1", + "provider": "opencode", + "sessionId": "session-1", + "sessionType": "freshopencode", + "submittedTurnId": "provider-turn-1" + })) + .await + .unwrap(); + + let dispatches_for_second = Arc::clone(&dispatches); + service + .clone() + .barrier_and_forward( + make_send("request-2"), + Box::pin(async move { + dispatches_for_second.fetch_add(1, Ordering::SeqCst); + }), + ) + .await; + + assert_eq!(dispatches.load(Ordering::SeqCst), 1); + let key = "\"opencode\":\"freshopencode\":session-1:logical-attempt-1".to_string(); + let record = service.find_outcome(&key).await.unwrap().unwrap(); + assert!(record.dispatch_started); + assert_eq!(record.submitted_turn_id.as_deref(), Some("provider-turn-1")); + assert!(record.request_ids.contains(&"request-2".to_string())); + } + + #[tokio::test] + async fn pruned_dispatch_outcome_keeps_a_durable_retry_fence() { + use freshell_ws::checkpoint_barrier::FreshAgentCheckpointBarrier; + use std::sync::atomic::AtomicUsize; + + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx.clone()), + )); + let send_for_key = |request_id: &str, logical_attempt_id: &str| FreshAgentSend { + provider: AgentProvider::Opencode, + session_id: "session-pruned-dispatch".into(), + session_type: SessionType::Freshopencode, + text: "retrying this attempt must not send it twice".into(), + cwd: None, + images: None, + request_id: Some(request_id.into()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: logical_attempt_id.into(), + workspace: None, + }), + }; + + let old_logical_attempt = "attempt-pruned-before-retry"; + let old_key = checkpoint_attempt_key( + &send_for_key("request-original", old_logical_attempt), + old_logical_attempt, + ); + let mut outcomes = BTreeMap::new(); + for index in 0..CHECKPOINT_OUTCOME_STORE_LIMIT { + let logical_attempt_id = if index == 0 { + old_logical_attempt.to_string() + } else { + format!("filler-attempt-{index}") + }; + let send = send_for_key(&format!("filler-request-{index}"), &logical_attempt_id); + let mut record = service.new_record( + &send, + &logical_attempt_id, + send.request_id.as_deref().unwrap(), + None, + Err(("workspace_unavailable", "No workspace".into())), + ); + record.created_at_ms = index as u64; + record.dispatch_started = true; + outcomes.insert(checkpoint_attempt_key(&send, &logical_attempt_id), record); + } + service + .write_outcome_store(&CheckpointOutcomeStore { + version: CHECKPOINT_OUTCOME_STORE_VERSION, + outcomes, + dispatch_fences: BTreeSet::new(), + has_pruned_dispatch_fences: false, + }) + .await + .unwrap(); + + let newer_logical_attempt = "newer-attempt-after-prune"; + let newer_send = send_for_key("newer-request", newer_logical_attempt); + let mut newer_record = service.new_record( + &newer_send, + newer_logical_attempt, + "newer-request", + None, + Err(("workspace_unavailable", "No workspace".into())), + ); + newer_record.created_at_ms = u64::MAX; + newer_record.dispatch_started = true; + service + .persist_outcome( + &checkpoint_attempt_key(&newer_send, newer_logical_attempt), + newer_record, + ) + .await + .unwrap(); + + let pruned = service.read_outcome_store().await.unwrap(); + assert!(!pruned.outcomes.contains_key(&old_key)); + assert!( + pruned.dispatch_fences.len() <= CHECKPOINT_DISPATCH_FENCE_LIMIT, + "the checkpoint JSON store must not retain an unbounded fence set" + ); + assert!(pruned.has_pruned_dispatch_fences); + assert!(service + .dispatch_fence_path(&checkpoint_attempt_fence_key(&old_key)) + .unwrap() + .exists()); + assert!(pruned.outcomes.len() <= CHECKPOINT_OUTCOME_STORE_LIMIT); + + let (retry_broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let mut outcome_rx = retry_broadcast_tx.subscribe(); + let recovered = Arc::new(CheckpointService::new( Arc::new(home.path().to_path_buf()), - Arc::new(broadcast_tx), + Arc::new(retry_broadcast_tx), )); let dispatches = Arc::new(AtomicUsize::new(0)); - let make_send = |request_id: &str| FreshAgentSend { - provider: AgentProvider::Opencode, - session_id: "session-1".into(), - session_type: SessionType::Freshopencode, - text: "the request body is never stored in checkpoint outcomes".into(), - cwd: None, - images: None, - request_id: Some(request_id.into()), - settings: None, - observed_epoch: None, - observed_generation: None, - checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { - logical_attempt_id: "logical-attempt-1".into(), - workspace: None, - }), - }; - - let dispatches_for_first = Arc::clone(&dispatches); - service + let retry_send = send_for_key("request-retry", old_logical_attempt); + let dispatches_for_retry = Arc::clone(&dispatches); + recovered .clone() .barrier_and_forward( - make_send("request-1"), + retry_send, Box::pin(async move { - dispatches_for_first.fetch_add(1, Ordering::SeqCst); + dispatches_for_retry.fetch_add(1, Ordering::SeqCst); }), ) .await; - service - .observe_accepted(&json!({ - "requestId": "request-1", - "provider": "opencode", - "sessionId": "session-1", - "sessionType": "freshopencode", - "submittedTurnId": "provider-turn-1" + + assert_eq!(dispatches.load(Ordering::SeqCst), 0); + let outcome: Value = serde_json::from_str(&outcome_rx.recv().await.unwrap()).unwrap(); + assert_eq!(outcome["status"], "unavailable"); + assert_eq!(outcome["code"], "attempt_history_expired"); + assert!(outcome["reason"] + .as_str() + .unwrap() + .contains("new logical attempt ID")); + } + + #[tokio::test] + async fn legacy_dispatch_fences_migrate_to_addressable_durable_files() { + let home = tempfile::tempdir().unwrap(); + let store_dir = home.path().join(".freshell/checkpoints"); + tokio::fs::create_dir_all(&store_dir).await.unwrap(); + let store_path = store_dir.join("attempt-outcomes-v1.json"); + let old_key = "legacy-pruned-checkpoint-attempt"; + let legacy_fence = checkpoint_attempt_fence_key(old_key); + tokio::fs::write( + &store_path, + serde_json::to_vec(&serde_json::json!({ + "version": CHECKPOINT_OUTCOME_STORE_LEGACY_VERSION, + "outcomes": {}, + "dispatchFences": [legacy_fence], })) - .await; + .unwrap(), + ) + .await + .unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(4); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); - let dispatches_for_second = Arc::clone(&dispatches); - service - .clone() - .barrier_and_forward( - make_send("request-2"), - Box::pin(async move { - dispatches_for_second.fetch_add(1, Ordering::SeqCst); - }), - ) - .await; + let migrated = service.read_outcome_store().await.unwrap(); - assert_eq!(dispatches.load(Ordering::SeqCst), 1); - let key = "\"opencode\":\"freshopencode\":session-1:logical-attempt-1".to_string(); - let record = service.find_outcome(&key).await.unwrap().unwrap(); - assert!(record.dispatch_started); - assert_eq!(record.submitted_turn_id.as_deref(), Some("provider-turn-1")); - assert!(record.request_ids.contains(&"request-2".to_string())); + assert_eq!(migrated.version, CHECKPOINT_OUTCOME_STORE_VERSION); + assert!(migrated.dispatch_fences.is_empty()); + assert!(migrated.has_pruned_dispatch_fences); + assert!(matches!( + service.find_attempt(old_key).await.unwrap(), + Some(CheckpointAttemptLookup::DispatchFence) + )); + assert_eq!( + tokio::fs::read(service.dispatch_fence_path(&legacy_fence).unwrap()) + .await + .unwrap(), + legacy_fence.as_bytes() + ); } #[tokio::test] @@ -6484,6 +7261,8 @@ mod tests { let empty_store = CheckpointOutcomeStore { version: CHECKPOINT_OUTCOME_STORE_VERSION, outcomes: BTreeMap::new(), + dispatch_fences: BTreeSet::new(), + has_pruned_dispatch_fences: false, }; tokio::fs::write(&store_path, serde_json::to_vec(&empty_store).unwrap()) .await @@ -6570,6 +7349,8 @@ mod tests { let empty_store = CheckpointOutcomeStore { version: CHECKPOINT_OUTCOME_STORE_VERSION, outcomes: BTreeMap::new(), + dispatch_fences: BTreeSet::new(), + has_pruned_dispatch_fences: false, }; tokio::fs::write(&store_path, serde_json::to_vec(&empty_store).unwrap()) .await @@ -6713,6 +7494,8 @@ mod tests { .write_outcome_store(&CheckpointOutcomeStore { version: CHECKPOINT_OUTCOME_STORE_VERSION, outcomes, + dispatch_fences: BTreeSet::new(), + has_pruned_dispatch_fences: false, }) .await .unwrap(); @@ -6724,7 +7507,8 @@ mod tests { "sessionType": "freshopencode", "submittedTurnId": "durable-turn-0" })) - .await; + .await + .unwrap(); let durable_count = service.read_outcome_store().await.unwrap().outcomes.len(); assert_eq!(durable_count, CHECKPOINT_VOLATILE_FENCE_LIMIT); @@ -7016,7 +7800,8 @@ mod tests { "displayTurnId": "codex-native-turn-1:row-0" } })) - .await; + .await + .unwrap(); let persisted = service.read_outcome_store().await.unwrap(); let saved = persisted.outcomes.get(&key).unwrap(); @@ -7111,7 +7896,8 @@ mod tests { "previousSessionId": "provisional-codex-session", "sessionId": "durable-codex-session" })) - .await; + .await + .unwrap(); let (replay_tx, mut replay_rx) = tokio::sync::broadcast::channel(8); let reloaded = Arc::new(CheckpointService::new( @@ -7213,6 +7999,8 @@ mod tests { .write_outcome_store(&CheckpointOutcomeStore { version: CHECKPOINT_OUTCOME_STORE_VERSION, outcomes, + dispatch_fences: BTreeSet::new(), + has_pruned_dispatch_fences: false, }) .await .unwrap(); @@ -7326,7 +8114,8 @@ mod tests { "attempt-identity-first", "different-turn-first", )) - .await; + .await + .unwrap(); let persisted = service.read_outcome_store().await.unwrap(); let first_saved = persisted.outcomes.get(&first_key).unwrap(); assert_eq!(first_saved.outcome.native_turn_id, None); @@ -7338,7 +8127,8 @@ mod tests { "attempt-identity-first", "accepted-turn-first", )) - .await; + .await + .unwrap(); let persisted = service.read_outcome_store().await.unwrap(); assert_eq!( persisted @@ -7368,7 +8158,8 @@ mod tests { "attempt-identity-second", "identity-before-acceptance", )) - .await; + .await + .unwrap(); service .observe_accepted(&json!({ "requestId": "request-identity-second", @@ -7377,7 +8168,8 @@ mod tests { "sessionType": "freshcodex", "submittedTurnId": "different-accepted-turn" })) - .await; + .await + .unwrap(); let persisted = service.read_outcome_store().await.unwrap(); let second_saved = persisted.outcomes.get(&second_key).unwrap(); @@ -7386,8 +8178,9 @@ mod tests { Some("identity-before-acceptance") ); assert_eq!( - second_saved.submitted_turn_id, None, - "a conflicting later acceptance cannot be joined to this checkpoint" + second_saved.submitted_turn_id.as_deref(), + Some("identity-before-acceptance"), + "the persisted identity receipt proves the submitted turn before a conflicting later acceptance" ); let third_send = send("request-identity-third", "attempt-identity-third"); @@ -7409,7 +8202,8 @@ mod tests { "sessionId": "codex-session", "sessionType": "freshcodex" })) - .await; + .await + .unwrap(); let persisted = service.read_outcome_store().await.unwrap(); assert_eq!( @@ -7424,6 +8218,308 @@ mod tests { ); } + #[tokio::test] + async fn checkpoint_identity_is_durable_for_each_provider_before_forwarding() { + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = + CheckpointService::new(Arc::new(home.path().to_path_buf()), Arc::new(broadcast_tx)); + + for (provider, session_type) in [ + ("codex", "freshcodex"), + ("claude", "freshclaude"), + ("opencode", "freshopencode"), + ] { + let request_id = format!("{provider}-checkpoint-request"); + let logical_attempt_id = format!("{provider}-checkpoint-attempt"); + let session_id = format!("{provider}-checkpoint-session"); + let native_turn_id = format!("{provider}-native-turn"); + let send = FreshAgentSend { + provider: parse_agent_provider(provider).unwrap(), + session_id: session_id.clone(), + session_type: parse_session_type(session_type).unwrap(), + text: "private prompt is never stored".to_string(), + cwd: None, + images: None, + request_id: Some(request_id.clone()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: logical_attempt_id.clone(), + workspace: None, + }), + }; + let key = checkpoint_attempt_key(&send, &logical_attempt_id); + let mut record = service.new_record( + &send, + &logical_attempt_id, + &request_id, + None, + Err(("workspace_unavailable", "No workspace".into())), + ); + record.dispatch_started = true; + service.persist_outcome(&key, record).await.unwrap(); + + let frame = json!({ + "type": "freshAgent.event", + "provider": provider, + "sessionId": session_id.clone(), + "sessionType": session_type, + "event": { + "type": "checkpointTurnIdentity", + "logicalAttemptId": logical_attempt_id.clone(), + "requestId": request_id.clone(), + "sessionId": session_id.clone(), + "nativeTurnId": native_turn_id.clone(), + "displayTurnId": format!("{provider}-display-row") + } + }) + .to_string(); + service + .persist_event_before_broadcast_with_match(&frame, true) + .await + .unwrap_or_else(|error| { + panic!("{provider} identity was not durably recorded: {error}") + }); + + let store = service.read_outcome_store().await.unwrap(); + let outcome = &store.outcomes.get(&key).unwrap().outcome; + assert_eq!( + outcome.native_turn_id.as_deref(), + Some(native_turn_id.as_str()) + ); + let expected_display_turn_id = format!("{provider}-display-row"); + assert_eq!( + outcome.display_turn_id.as_deref(), + Some(expected_display_turn_id.as_str()) + ); + } + } + + #[tokio::test] + async fn codex_checkpoint_identity_is_durable_before_its_event_is_forwarded() { + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(16); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx), + )); + let send = FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "codex-placeholder-session".to_string(), + session_type: SessionType::Freshcodex, + text: "private prompt is not stored".to_string(), + cwd: None, + images: None, + request_id: Some("durable-codex-request".to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "durable-codex-attempt".to_string(), + workspace: None, + }), + }; + let key = checkpoint_attempt_key(&send, "durable-codex-attempt"); + let mut record = service.new_record( + &send, + "durable-codex-attempt", + "durable-codex-request", + None, + Err(("workspace_unavailable", "No workspace".into())), + ); + record.dispatch_started = true; + service.persist_outcome(&key, record).await.unwrap(); + + // Model the Codex respawn path while the asynchronous observer is + // still behind: the materialized alias must reach disk before the + // accepted frame for the replacement session can save its turn id. + service + .persist_event_before_broadcast_with_match( + &json!({ + "type": "freshAgent.session.materialized", + "provider": "codex", + "sessionType": "freshcodex", + "previousSessionId": "codex-placeholder-session", + "sessionId": "durable-codex-session" + }) + .to_string(), + false, + ) + .await + .unwrap(); + service + .persist_event_before_broadcast_with_match( + &json!({ + "type": "freshAgent.send.accepted", + "provider": "codex", + "sessionId": "durable-codex-session", + "sessionType": "freshcodex", + "requestId": "durable-codex-request", + "submittedTurnId": "durable-codex-turn" + }) + .to_string(), + true, + ) + .await + .unwrap(); + let accepted_store = service.read_outcome_store().await.unwrap(); + let accepted_outcome = &accepted_store.outcomes.get(&key).unwrap().outcome; + assert_eq!( + accepted_outcome.native_turn_id.as_deref(), + Some("durable-codex-turn") + ); + assert_eq!( + accepted_outcome.display_turn_id.as_deref(), + Some("durable-codex-turn"), + "the Codex UI recognizes the submitted id as a provisional row key" + ); + + service + .persist_event_before_broadcast_with_match( + &json!({ + "type": "freshAgent.event", + "provider": "codex", + "sessionId": "durable-codex-session", + "sessionType": "freshcodex", + "event": { + "type": "checkpointTurnIdentity", + "logicalAttemptId": "durable-codex-attempt", + "requestId": "durable-codex-request", + "sessionId": "durable-codex-session", + "nativeTurnId": "durable-codex-turn", + "displayTurnId": "durable-codex-turn:row-0" + } + }) + .to_string(), + true, + ) + .await + .unwrap(); + + let reloaded = CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(tokio::sync::broadcast::channel(16).0), + ); + let stored = reloaded.read_outcome_store().await.unwrap(); + let outcome = &stored.outcomes.get(&key).unwrap().outcome; + assert_eq!( + outcome.native_turn_id.as_deref(), + Some("durable-codex-turn") + ); + assert_eq!( + outcome.display_turn_id.as_deref(), + Some("durable-codex-turn:row-0") + ); + assert_eq!( + stored + .outcomes + .get(&key) + .unwrap() + .submitted_turn_id + .as_deref(), + Some("durable-codex-turn"), + "the durable identity receipt also proves its exact accepted turn" + ); + } + + #[tokio::test] + async fn checkpoint_identity_survives_a_broadcast_burst_during_outcome_persistence() { + let home = tempfile::tempdir().unwrap(); + let (broadcast_tx, _) = tokio::sync::broadcast::channel(8); + let service = Arc::new(CheckpointService::new( + Arc::new(home.path().to_path_buf()), + Arc::new(broadcast_tx.clone()), + )); + let send = FreshAgentSend { + provider: AgentProvider::Codex, + session_id: "queued-codex-session".to_string(), + session_type: SessionType::Freshcodex, + text: "private prompt is not stored".to_string(), + cwd: None, + images: None, + request_id: Some("queued-codex-request".to_string()), + settings: None, + observed_epoch: None, + observed_generation: None, + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "queued-codex-attempt".to_string(), + workspace: None, + }), + }; + let key = checkpoint_attempt_key(&send, "queued-codex-attempt"); + let mut record = service.new_record( + &send, + "queued-codex-attempt", + "queued-codex-request", + None, + Err(("workspace_unavailable", "No workspace".into())), + ); + record.dispatch_started = true; + service.persist_outcome(&key, record).await.unwrap(); + service.start_observer(); + + let outcome_gate = service.outcome_gate.lock().await; + broadcast_tx + .send( + json!({ + "type": "freshAgent.send.accepted", + "requestId": "queued-codex-request", + "provider": "codex", + "sessionId": "queued-codex-session", + "sessionType": "freshcodex", + "submittedTurnId": "queued-native-turn" + }) + .to_string(), + ) + .unwrap(); + tokio::time::sleep(Duration::from_millis(10)).await; + broadcast_tx + .send( + json!({ + "type": "freshAgent.event", + "provider": "codex", + "sessionId": "queued-codex-session", + "sessionType": "freshcodex", + "event": { + "type": "checkpointTurnIdentity", + "logicalAttemptId": "queued-codex-attempt", + "requestId": "queued-codex-request", + "sessionId": "queued-codex-session", + "nativeTurnId": "queued-native-turn", + "displayTurnId": "queued-native-turn:row-0" + } + }) + .to_string(), + ) + .unwrap(); + for index in 0..32 { + broadcast_tx + .send(json!({"type":"terminal.output","seq":index}).to_string()) + .unwrap(); + tokio::task::yield_now().await; + } + drop(outcome_gate); + + tokio::time::timeout(Duration::from_secs(3), async { + loop { + let stored = service.read_outcome_store().await.unwrap(); + if stored + .outcomes + .get(&key) + .and_then(|record| record.outcome.native_turn_id.as_deref()) + == Some("queued-native-turn") + { + break; + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .expect("queued identity is persisted after unrelated broadcast traffic"); + } + async fn body_json(resp: Response) -> Value { let body = axum::body::to_bytes(resp.into_body(), usize::MAX) .await diff --git a/crates/freshell-server/src/main.rs b/crates/freshell-server/src/main.rs index ee6fefdb5..2ed6cae21 100644 --- a/crates/freshell-server/src/main.rs +++ b/crates/freshell-server/src/main.rs @@ -1032,6 +1032,20 @@ async fn main() -> ExitCode { Arc::clone(&broadcast_tx), serde_json::to_value(settings.as_ref()).unwrap_or_else(|_| serde_json::json!({})), ); + let checkpoint_event_persister: freshell_freshagent::CheckpointEventPersister = { + let checkpoint_service = Arc::clone(&checkpoint_service); + Arc::new(move |frame, require_match| { + let service = Arc::clone(&checkpoint_service); + Box::pin(async move { + service + .persist_event_before_broadcast_with_match(&frame, require_match) + .await + }) + }) + }; + fresh_codex_state + .set_checkpoint_event_persister(Arc::clone(&checkpoint_event_persister)) + .expect("Codex checkpoint event persister is installed once"); // The freshclaude WS fresh-agent slice: shares the broadcast bus so its // freshAgent.created/send.accepted/event frames reach every WS client (incl. the @@ -1039,6 +1053,9 @@ async fn main() -> ExitCode { // create gate is the SHARED settings.freshAgent.enabled flag (owned by fresh_codex). let mut fresh_claude_state = freshell_freshagent::FreshClaudeState::new(Arc::clone(&broadcast_tx)); + fresh_claude_state + .set_checkpoint_event_persister(Arc::clone(&checkpoint_event_persister)) + .expect("Claude checkpoint event persister is installed once"); // Task 12 (D8 for fresh agents): the ONE server-wide per-sessionRef create/resume // lease map, shared by every fresh-agent runtime (keys are provider-namespaced). @@ -1112,6 +1129,9 @@ async fn main() -> ExitCode { .with_shared_sessions_revision(Arc::clone(&sessions_revision)) .with_layout(layout_store.clone()) .with_ownership(Arc::clone(&ownership)); + fresh_agent_state + .set_checkpoint_event_persister(Arc::clone(&checkpoint_event_persister)) + .expect("OpenCode checkpoint event persister is installed once"); // The freshopencode WS fresh-agent slice: the post-handshake loop dispatches // `freshAgent.create`/`send`/`kill`/`interrupt` (opencode) here. let mut fresh_opencode_state = diff --git a/crates/freshell-server/src/managed_mcp_capability.rs b/crates/freshell-server/src/managed_mcp_capability.rs index f62991920..61e83bfad 100644 --- a/crates/freshell-server/src/managed_mcp_capability.rs +++ b/crates/freshell-server/src/managed_mcp_capability.rs @@ -1549,6 +1549,26 @@ fn cleanup_orphaned_provider_roots(directory: &Path) -> Result<(), String> { let Some(grant_id) = entry.file_name().to_str().map(str::to_owned) else { continue; }; + if let Some((staged_grant_id, temporary_id_str)) = grant_id.split_once(".tmp-") { + let valid_stage = grant_path(directory, staged_grant_id).is_ok() + && uuid::Uuid::parse_str(temporary_id_str).is_ok_and(|temporary_id| { + temporary_id.get_version_num() == 4 + && temporary_id.to_string() == temporary_id_str + }); + if valid_stage + && !protected.contains(staged_grant_id) + && !grant_path(directory, staged_grant_id)?.exists() + { + let metadata = + std::fs::symlink_metadata(entry.path()).map_err(|error| error.to_string())?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + std::fs::remove_dir_all(entry.path()).map_err(|error| error.to_string())?; + } else { + return Err("managed provider staging path has an unsafe type".into()); + } + continue; + } + } let Ok(grant_file) = grant_path(directory, &grant_id) else { continue; }; @@ -6054,8 +6074,14 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ std::fs::create_dir(&capabilities).unwrap(); let active = staged_provider_root_path(&capabilities, "grant-active").unwrap(); let orphan = staged_provider_root_path(&capabilities, "grant-orphan").unwrap(); + let interrupted_copy = control + .path() + .join("provider-roots") + .join(format!("grant-interrupted.tmp-{}", uuid::Uuid::new_v4())); std::fs::create_dir_all(&active).unwrap(); std::fs::create_dir_all(&orphan).unwrap(); + std::fs::create_dir_all(&interrupted_copy).unwrap(); + std::fs::write(interrupted_copy.join("private-config.json"), "partial copy").unwrap(); let path = grant_path(&capabilities, "grant-active").unwrap(); write_capability( &path, @@ -6076,6 +6102,10 @@ process.stdout.write(readFileSync(new URL('./resource',import.meta.url),'utf8')+ cleanup_orphaned_provider_roots(&capabilities).unwrap(); assert!(active.is_dir()); assert!(!orphan.exists()); + assert!( + !interrupted_copy.exists(), + "restart removes an incomplete private provider copy" + ); } #[test] diff --git a/crates/freshell-session-host/src/main.rs b/crates/freshell-session-host/src/main.rs index cf336a63b..fdd14fff8 100644 --- a/crates/freshell-session-host/src/main.rs +++ b/crates/freshell-session-host/src/main.rs @@ -2928,10 +2928,844 @@ fn private_bound_destinations( } const PROVIDER_PROJECTION_MANIFEST: &str = ".freshell-config-projection.json"; +const PROVIDER_PROJECTION_TRANSACTION_PREFIX: &str = ".freshell-provider-transaction-"; +const PROVIDER_PROJECTION_TRANSACTION_JOURNAL: &str = "transaction.json"; +const PROVIDER_PROJECTION_TRANSACTION_JOURNAL_TEMP: &str = "transaction.json.tmp"; +const PROVIDER_PROJECTION_TRANSACTION_MANIFEST_BACKUP: &str = "manifest.backup"; // Ordinary disk metadata shares the resource-manifest profile. The control // pipe's 1 MiB limit does not reduce these provider-filesystem views. const MAX_PROJECTION_METADATA_BYTES: usize = 64 * 1024 * 1024; +#[derive(Debug, Serialize, Deserialize, Clone, Copy, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +enum ProviderProjectionTransactionPhase { + Preparing, + Publishing, + Committed, + RolledBack, +} + +#[derive(Debug, Serialize, Deserialize, Clone, Copy)] +#[serde(tag = "kind", rename_all = "snake_case")] +enum ProviderProjectionOriginalNode { + Missing, + File { in_place: bool }, + Directory { mode: u32 }, + Symlink, + Other, +} + +#[derive(Debug, Serialize, Deserialize, Clone)] +#[serde(rename_all = "camelCase")] +struct ProviderProjectionTransactionEntry { + relative: String, + original: ProviderProjectionOriginalNode, +} + +#[derive(Debug, Serialize, Deserialize, Clone)] +#[serde(rename_all = "camelCase")] +struct ProviderProjectionTransactionJournal { + version: u32, + phase: ProviderProjectionTransactionPhase, + manifest_existed: bool, + entries: Vec<ProviderProjectionTransactionEntry>, +} + +fn lock_provider_projection(provider_home: &Path) -> Result<std::fs::File, String> { + use std::os::unix::fs::OpenOptionsExt; + use std::os::unix::io::AsRawFd; + + let metadata = std::fs::symlink_metadata(provider_home) + .map_err(|_| "provider projection HOME unavailable; retain inputs and repair")?; + if !metadata.is_dir() || metadata.file_type().is_symlink() { + return Err("provider projection HOME must be a physical directory".into()); + } + let file = std::fs::OpenOptions::new() + .read(true) + .custom_flags(libc::O_DIRECTORY | libc::O_NOFOLLOW) + .open(provider_home) + .map_err(|_| "provider projection lock unavailable; retain inputs and repair")?; + loop { + let result = unsafe { libc::flock(file.as_raw_fd(), libc::LOCK_EX) }; + if result == 0 { + return Ok(file); + } + let error = std::io::Error::last_os_error(); + if error.kind() != std::io::ErrorKind::Interrupted { + return Err(format!( + "provider projection lock unavailable; retain inputs and repair: {error}" + )); + } + } +} + +#[derive(Debug)] +enum ProviderProjectionJournalWriteError { + BeforePublish(String), + PublishedButUnconfirmed(String), +} + +impl std::fmt::Display for ProviderProjectionJournalWriteError { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::BeforePublish(cause) | Self::PublishedButUnconfirmed(cause) => { + formatter.write_str(cause) + } + } + } +} + +fn write_provider_projection_transaction_journal( + transaction: &Path, + journal: &ProviderProjectionTransactionJournal, +) -> Result<(), ProviderProjectionJournalWriteError> { + write_provider_projection_transaction_journal_with_sync(transaction, journal, |path| { + std::fs::File::open(path)?.sync_all() + }) +} + +fn write_provider_projection_transaction_journal_with_sync( + transaction: &Path, + journal: &ProviderProjectionTransactionJournal, + sync_transaction_directory: impl FnOnce(&Path) -> std::io::Result<()>, +) -> Result<(), ProviderProjectionJournalWriteError> { + use std::io::Write; + use std::os::unix::fs::OpenOptionsExt; + + let bytes = + freshell_runtime_protocol::bounded_json_bytes(journal, MAX_PROJECTION_METADATA_BYTES) + .map_err(|_| { + ProviderProjectionJournalWriteError::BeforePublish( + "provider transaction journal exceeds 64 MiB".into(), + ) + })?; + let temporary = transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL_TEMP); + let mut file = std::fs::OpenOptions::new() + .create(true) + .truncate(true) + .write(true) + .mode(0o600) + .open(&temporary) + .map_err(|error| { + ProviderProjectionJournalWriteError::BeforePublish(format!( + "provider transaction journal stage unavailable: {error}" + )) + })?; + file.write_all(&bytes) + .and_then(|()| file.sync_all()) + .map_err(|error| { + ProviderProjectionJournalWriteError::BeforePublish(format!( + "provider transaction journal write unavailable: {error}" + )) + })?; + std::fs::rename( + &temporary, + transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL), + ) + .map_err(|error| { + ProviderProjectionJournalWriteError::BeforePublish(format!( + "provider transaction journal publish unavailable: {error}" + )) + })?; + sync_transaction_directory(transaction).map_err(|error| { + ProviderProjectionJournalWriteError::PublishedButUnconfirmed(format!( + "provider transaction journal was published but its directory sync failed: {error}" + )) + }) +} + +fn provider_projection_relative_path(relative: &str) -> Result<&Path, String> { + let path = Path::new(relative); + if relative.is_empty() + || path.is_absolute() + || path + .components() + .any(|component| !matches!(component, std::path::Component::Normal(_))) + { + return Err("provider transaction contains an unsafe destination".into()); + } + Ok(path) +} + +fn provider_projection_original_node( + home: &Path, + relative: &str, + in_place: bool, +) -> Result<ProviderProjectionOriginalNode, String> { + use std::os::unix::fs::PermissionsExt; + + let path = provider_projection_relative_path(relative)?; + match std::fs::symlink_metadata(home.join(path)) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + Ok(ProviderProjectionOriginalNode::Missing) + } + Err(_) => Err(format!( + "provider transaction destination {relative} could not be inspected" + )), + Ok(metadata) if metadata.file_type().is_symlink() => { + Ok(ProviderProjectionOriginalNode::Symlink) + } + Ok(metadata) if metadata.is_file() => Ok(ProviderProjectionOriginalNode::File { in_place }), + Ok(metadata) if metadata.is_dir() => Ok(ProviderProjectionOriginalNode::Directory { + mode: metadata.permissions().mode() & 0o7777, + }), + Ok(_) => Ok(ProviderProjectionOriginalNode::Other), + } +} + +fn projection_transaction_path(transaction: &Path, relative: &str) -> Result<PathBuf, String> { + Ok(transaction + .join("backup") + .join(provider_projection_relative_path(relative)?)) +} + +fn projection_path_parent_is_physical(home: &Path, relative: &Path) -> Result<(), String> { + let mut parent = home.to_path_buf(); + for component in relative.parent().into_iter().flat_map(Path::components) { + parent.push(component.as_os_str()); + match std::fs::symlink_metadata(&parent) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + Ok(_) => { + return Err("provider transaction destination parent is not a physical directory; retain inputs and repair".into()) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => break, + Err(_) => { + return Err("provider transaction destination parent unavailable; retain inputs and repair".into()) + } + } + } + Ok(()) +} + +fn remove_provider_projection_node(path: &Path) -> Result<(), String> { + match std::fs::symlink_metadata(path) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(_) => Err("provider transaction recovery destination unavailable".into()), + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => { + std::fs::remove_dir(path).map_err(|error| { + format!("provider transaction recovery directory is not empty: {error}") + }) + } + Ok(_) => std::fs::remove_file(path) + .map_err(|_| "provider transaction recovery destination could not be removed".into()), + } +} + +fn provider_projection_backup_exists(transaction: &Path, relative: &str) -> Result<bool, String> { + let backup = projection_transaction_path(transaction, relative)?; + match std::fs::symlink_metadata(&backup) { + Ok(_) => Ok(true), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(_) => Err("provider transaction backup unavailable; retain inputs and repair".into()), + } +} + +fn sync_provider_projection_directory(path: &Path) -> Result<(), String> { + std::fs::File::open(path) + .and_then(|directory| directory.sync_all()) + .map_err(|error| { + format!( + "provider projection directory flush failed for {}: {error}", + path.display() + ) + }) +} + +fn sync_provider_projection_tree(root: &Path) -> Result<(), String> { + sync_provider_projection_tree_with(root, &mut |path| std::fs::File::open(path)?.sync_all()) +} + +fn sync_provider_projection_tree_with( + root: &Path, + sync_node: &mut impl FnMut(&Path) -> std::io::Result<()>, +) -> Result<(), String> { + let metadata = std::fs::symlink_metadata(root).map_err(|error| { + format!( + "provider projection sync inspect {}: {error}", + root.display() + ) + })?; + if metadata.file_type().is_symlink() { + return Ok(()); + } + if metadata.is_dir() { + for entry in std::fs::read_dir(root) + .map_err(|error| format!("provider projection sync read {}: {error}", root.display()))? + { + let entry = entry.map_err(|error| { + format!("provider projection sync read {}: {error}", root.display()) + })?; + sync_provider_projection_tree_with(&entry.path(), sync_node)?; + } + } + sync_node(root).map_err(|error| { + format!( + "provider projection flush failed for {}: {error}", + root.display() + ) + }) +} + +fn sync_provider_projection_file(path: &Path) -> Result<(), String> { + let metadata = std::fs::symlink_metadata(path).map_err(|error| { + format!( + "provider projection sync inspect {}: {error}", + path.display() + ) + })?; + if metadata.is_file() && !metadata.file_type().is_symlink() { + std::fs::File::open(path) + .and_then(|file| file.sync_all()) + .map_err(|error| { + format!( + "provider projection file flush failed for {}: {error}", + path.display() + ) + })?; + } + Ok(()) +} + +fn sync_provider_projection_directories( + home: &Path, + entries: &[ProviderProjectionTransactionEntry], +) -> Result<(), String> { + use std::collections::BTreeSet; + + let mut directories = BTreeSet::new(); + directories.insert(home.to_path_buf()); + for entry in entries { + let relative = provider_projection_relative_path(&entry.relative)?; + let mut candidate = home.join(relative); + loop { + if let Ok(metadata) = std::fs::symlink_metadata(&candidate) { + if metadata.is_dir() && !metadata.file_type().is_symlink() { + directories.insert(candidate.clone()); + } + } + if candidate == home { + break; + } + let Some(parent) = candidate.parent() else { + break; + }; + if !parent.starts_with(home) { + break; + } + candidate = parent.to_path_buf(); + } + } + for directory in directories { + sync_provider_projection_directory(&directory)?; + } + Ok(()) +} + +fn restore_provider_projection_manifest( + transaction: &Path, + home: &Path, + manifest_existed: bool, +) -> Result<(), String> { + use std::io::Write; + + let destination = home.join(PROVIDER_PROJECTION_MANIFEST); + if !manifest_existed { + match std::fs::symlink_metadata(&destination) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => { + std::fs::remove_file(destination) + .map_err(|_| "provider projection manifest rollback failed; retain inputs and repair".to_owned())?; + sync_provider_projection_directory(home) + .map_err(|_| "provider projection manifest rollback sync unavailable".to_owned())?; + return Ok(()); + } + _ => return Err("provider projection manifest rollback found an unsafe path; retain inputs and repair".into()), + } + } + let bytes = crate::provider_input_preparation::read_private( + transaction, + PROVIDER_PROJECTION_TRANSACTION_MANIFEST_BACKUP, + MAX_PROJECTION_METADATA_BYTES, + ) + .map_err(|_| "provider projection manifest backup unavailable; retain inputs and repair")?; + if let Ok(metadata) = std::fs::symlink_metadata(&destination) { + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err("provider projection manifest rollback found an unsafe path; retain inputs and repair".into()); + } + } + let mut temporary = tempfile::Builder::new() + .prefix(".freshell-provider-manifest-restore-") + .tempfile_in(home) + .map_err(|_| "provider projection manifest rollback stage unavailable")?; + temporary + .write_all(&bytes) + .and_then(|()| temporary.as_file().sync_all()) + .map_err(|_| "provider projection manifest rollback write unavailable")?; + temporary + .persist(&destination) + .map_err(|_| "provider projection manifest rollback publish unavailable")?; + sync_provider_projection_directory(home) + .map_err(|_| "provider projection manifest rollback sync unavailable")?; + Ok(()) +} + +fn recover_provider_projection_transaction( + transaction: &Path, + home: &Path, + journal: &ProviderProjectionTransactionJournal, +) -> Result<(), String> { + use std::{collections::BTreeSet, fs, os::unix::fs::PermissionsExt}; + + if journal.version != 1 { + return Err( + "provider transaction journal version is unsupported; retain inputs and repair".into(), + ); + } + let mut seen = BTreeSet::new(); + for entry in &journal.entries { + provider_projection_relative_path(&entry.relative)?; + if !seen.insert(entry.relative.as_str()) { + return Err( + "provider transaction journal has duplicate destinations; retain inputs and repair" + .into(), + ); + } + } + let mut entries = journal.entries.iter().collect::<Vec<_>>(); + entries.sort_by_key(|entry| { + std::cmp::Reverse( + provider_projection_relative_path(&entry.relative) + .map(|path| path.components().count()) + .unwrap_or(0), + ) + }); + + // A prior directory can now be an installed file. Remove those replacement + // leaves before walking deeper entries, since their former children have a + // non-directory parent until the old directory is rebuilt below. + let mut replaced_directories = journal + .entries + .iter() + .filter(|entry| { + matches!( + entry.original, + ProviderProjectionOriginalNode::Directory { .. } + ) + }) + .collect::<Vec<_>>(); + replaced_directories.sort_by_key(|entry| { + provider_projection_relative_path(&entry.relative) + .map(|path| path.components().count()) + .unwrap_or(usize::MAX) + }); + for entry in replaced_directories { + let relative = provider_projection_relative_path(&entry.relative)?; + projection_path_parent_is_physical(home, relative)?; + let destination = home.join(relative); + if let Ok(metadata) = fs::symlink_metadata(&destination) { + if !metadata.is_dir() || metadata.file_type().is_symlink() { + remove_provider_projection_node(&destination)?; + } + } + } + + // Remove paths that did not exist before publication and clear new file + // nodes that replaced old directories. Only empty directories are removed; + // an unexpected provider-created descendant leaves recovery fail-closed. + for entry in &entries { + let relative = provider_projection_relative_path(&entry.relative)?; + projection_path_parent_is_physical(home, relative)?; + let path = home.join(relative); + match entry.original { + ProviderProjectionOriginalNode::Missing => remove_provider_projection_node(&path)?, + ProviderProjectionOriginalNode::Directory { .. } => {} + ProviderProjectionOriginalNode::File { .. } + | ProviderProjectionOriginalNode::Symlink + | ProviderProjectionOriginalNode::Other => { + if provider_projection_backup_exists(transaction, &entry.relative)? + && fs::symlink_metadata(&path).is_ok_and(|metadata| { + metadata.is_dir() && !metadata.file_type().is_symlink() + }) + { + remove_provider_projection_node(&path)?; + } + } + } + } + + // Parent directories must exist before backed-up files are restored. Their + // recorded modes also undo the publication's temporary 0700 directory mode. + let mut directories = journal + .entries + .iter() + .filter_map(|entry| { + if let ProviderProjectionOriginalNode::Directory { mode } = entry.original { + Some((entry, mode)) + } else { + None + } + }) + .collect::<Vec<_>>(); + directories.sort_by_key(|(entry, _)| { + provider_projection_relative_path(&entry.relative) + .map(|path| path.components().count()) + .unwrap_or(usize::MAX) + }); + for (entry, mode) in directories { + let relative = provider_projection_relative_path(&entry.relative)?; + projection_path_parent_is_physical(home, relative)?; + let destination = home.join(relative); + match fs::symlink_metadata(&destination) { + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + fs::create_dir(&destination) + .map_err(|_| "provider transaction recovery directory could not be restored")?; + } + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + _ => return Err("provider transaction recovery directory has an unsafe type; retain inputs and repair".into()), + } + fs::set_permissions(destination, fs::Permissions::from_mode(mode)) + .map_err(|_| "provider transaction recovery directory mode could not be restored")?; + } + + for entry in &journal.entries { + let relative = provider_projection_relative_path(&entry.relative)?; + projection_path_parent_is_physical(home, relative)?; + let destination = home.join(relative); + let backup = projection_transaction_path(transaction, &entry.relative)?; + let has_backup = provider_projection_backup_exists(transaction, &entry.relative)?; + match entry.original { + ProviderProjectionOriginalNode::Missing => {} + ProviderProjectionOriginalNode::Directory { .. } => {} + ProviderProjectionOriginalNode::File { in_place } if has_backup => { + if in_place { + let metadata = fs::symlink_metadata(&destination).map_err(|_| { + "provider mounted projection file disappeared during recovery" + })?; + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err( + "provider mounted projection file has an unsafe type during recovery" + .into(), + ); + } + replace_mounted_projection_file(&backup, &destination)?; + } else { + if fs::symlink_metadata(&destination).is_ok_and(|metadata| { + metadata.is_dir() && !metadata.file_type().is_symlink() + }) { + remove_provider_projection_node(&destination)?; + } + fs::rename(&backup, &destination) + .map_err(|_| "provider transaction file backup could not be restored; retain inputs and repair")?; + } + } + ProviderProjectionOriginalNode::File { in_place: true } => { + return Err( + "provider mounted projection backup is missing; retain inputs and repair" + .into(), + ) + } + ProviderProjectionOriginalNode::File { in_place: false } => { + let metadata = fs::symlink_metadata(&destination).map_err(|_| { + "provider transaction original file is missing; retain inputs and repair" + })?; + if !metadata.is_file() || metadata.file_type().is_symlink() { + return Err("provider transaction original file has an unsafe type; retain inputs and repair".into()); + } + } + ProviderProjectionOriginalNode::Symlink if has_backup => { + fs::rename(&backup, &destination) + .map_err(|_| "provider transaction link backup could not be restored; retain inputs and repair")?; + } + ProviderProjectionOriginalNode::Symlink => { + if !fs::symlink_metadata(&destination) + .is_ok_and(|metadata| metadata.file_type().is_symlink()) + { + return Err( + "provider transaction original link is missing; retain inputs and repair" + .into(), + ); + } + } + ProviderProjectionOriginalNode::Other if has_backup => { + fs::rename(&backup, &destination) + .map_err(|_| "provider transaction node backup could not be restored; retain inputs and repair")?; + } + ProviderProjectionOriginalNode::Other => { + if fs::symlink_metadata(&destination).is_err() { + return Err( + "provider transaction original node is missing; retain inputs and repair" + .into(), + ); + } + } + } + } + restore_provider_projection_manifest(transaction, home, journal.manifest_existed)?; + sync_provider_projection_directories(home, &journal.entries)?; + sync_provider_projection_tree(&transaction.join("backup"))?; + let mut rolled_back = journal.clone(); + rolled_back.phase = ProviderProjectionTransactionPhase::RolledBack; + write_provider_projection_transaction_journal(transaction, &rolled_back) + .map_err(|error| format!("provider rollback completion could not be recorded: {error}"))?; + Ok(()) +} + +fn cleanup_provider_projection_transaction( + transaction: &Path, + provider_home: &Path, + expected_phase: ProviderProjectionTransactionPhase, +) -> Result<(), String> { + use std::fs; + + let journal_path = transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL); + let bytes = crate::provider_input_preparation::read_private( + transaction, + PROVIDER_PROJECTION_TRANSACTION_JOURNAL, + MAX_PROJECTION_METADATA_BYTES, + ) + .map_err(|_| "provider transaction cleanup journal unavailable; retain inputs and repair")?; + let journal: ProviderProjectionTransactionJournal = serde_json::from_slice(&bytes) + .map_err(|_| "provider transaction cleanup journal invalid; retain inputs and repair")?; + if journal.phase != expected_phase + || !matches!( + journal.phase, + ProviderProjectionTransactionPhase::Preparing + | ProviderProjectionTransactionPhase::Committed + | ProviderProjectionTransactionPhase::RolledBack + ) + { + return Err( + "provider transaction cleanup phase is not durable; retain inputs and repair".into(), + ); + } + + for entry in fs::read_dir(transaction).map_err(|_| { + "provider transaction cleanup inventory unavailable; retain inputs and repair" + })? { + let entry = entry.map_err(|_| { + "provider transaction cleanup inventory unavailable; retain inputs and repair" + })?; + let name = entry.file_name(); + let name = name.to_string_lossy(); + if name == PROVIDER_PROJECTION_TRANSACTION_JOURNAL { + continue; + } + if name != "desired" + && name != "backup" + && name != PROVIDER_PROJECTION_TRANSACTION_MANIFEST_BACKUP + && name != PROVIDER_PROJECTION_MANIFEST + && name != PROVIDER_PROJECTION_TRANSACTION_JOURNAL_TEMP + && !name.starts_with(".freshell-config-projection-") + { + return Err(format!( + "unexpected provider transaction cleanup entry {}; retain inputs and repair", + entry.path().display() + )); + } + let path = entry.path(); + let metadata = fs::symlink_metadata(&path).map_err(|_| { + "provider transaction cleanup entry could not be inspected; retain inputs and repair" + })?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + fs::remove_dir_all(&path).map_err(|_| { + "provider transaction payload cleanup failed; retain inputs and repair" + })?; + } else { + fs::remove_file(&path).map_err(|_| { + "provider transaction payload cleanup failed; retain inputs and repair" + })?; + } + } + + // Keep the phase record until every backup and staged input deletion is + // durable. Recovery can then repeat cleanup without trying to replay it. + sync_provider_projection_directory(transaction)?; + fs::remove_file(&journal_path) + .map_err(|_| "provider transaction journal cleanup failed; retain inputs and repair")?; + sync_provider_projection_directory(transaction)?; + fs::remove_dir(transaction) + .map_err(|_| "provider transaction directory cleanup failed; retain inputs and repair")?; + sync_provider_projection_directory(provider_home)?; + Ok(()) +} + +fn cleanup_orphaned_provider_refresh_stages( + provider_home: &Path, + preserved_stage: Option<&Path>, +) -> Result<(), String> { + use std::fs; + + let mut removed = false; + for entry in fs::read_dir(provider_home) + .map_err(|_| "provider refresh stage inventory unavailable; retain inputs and repair")? + { + let entry = entry.map_err(|_| { + "provider refresh stage inventory unavailable; retain inputs and repair" + })?; + if !entry + .file_name() + .to_string_lossy() + .starts_with(".freshell-provider-refresh-") + || preserved_stage.is_some_and(|stage| stage == entry.path()) + { + continue; + } + let metadata = fs::symlink_metadata(entry.path()).map_err(|_| { + "provider refresh stage could not be inspected; retain inputs and repair" + })?; + if !metadata.is_dir() || metadata.file_type().is_symlink() { + return Err( + "provider refresh stage has an unsafe type; retain inputs and repair".into(), + ); + } + fs::remove_dir_all(entry.path()).map_err(|_| { + "orphaned provider refresh stage cleanup failed; retain inputs and repair" + })?; + removed = true; + } + if removed { + sync_provider_projection_directory(provider_home)?; + } + Ok(()) +} + +fn recover_provider_projection_transactions(provider_home: &Path) -> Result<(), String> { + let result = recover_provider_projection_transactions_inner(provider_home); + if let Err(cause) = &result { + emit_environment_event(serde_json::json!({ + "severity": "error", + "event": "provider.projection.transaction.recovery_failed", + "stage": "recovery", + "cause": cause, + })); + } + result +} + +fn recover_provider_projection_transactions_inner(provider_home: &Path) -> Result<(), String> { + let mut transactions = Vec::new(); + for entry in std::fs::read_dir(provider_home).map_err(|_| { + "provider projection HOME unavailable during recovery; retain inputs and repair" + })? { + let entry = entry + .map_err(|_| "provider transaction inventory unavailable; retain inputs and repair")?; + let name = entry.file_name(); + if !name + .to_string_lossy() + .starts_with(PROVIDER_PROJECTION_TRANSACTION_PREFIX) + { + continue; + } + let metadata = std::fs::symlink_metadata(entry.path()) + .map_err(|_| "provider transaction could not be inspected; retain inputs and repair")?; + if !metadata.is_dir() || metadata.file_type().is_symlink() { + return Err( + "provider transaction path has an unsafe type; retain inputs and repair".into(), + ); + } + transactions.push(entry.path()); + } + transactions.sort(); + + let mut publishing = Vec::new(); + for transaction in &transactions { + let journal_path = transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL); + let bytes = match crate::provider_input_preparation::read_private( + transaction, + PROVIDER_PROJECTION_TRANSACTION_JOURNAL, + MAX_PROJECTION_METADATA_BYTES, + ) { + Ok(bytes) => Some(bytes), + Err(_) if !journal_path.exists() => None, + Err(_) => { + return Err("provider transaction journal invalid; retain inputs and repair".into()) + } + }; + let Some(bytes) = bytes else { + let mut contents = std::fs::read_dir(transaction).map_err(|_| { + "provider transaction inventory unavailable; retain inputs and repair" + })?; + let safe_initial_stage = contents.by_ref().all(|entry| { + let Ok(entry) = entry else { return false }; + if entry.file_name() != PROVIDER_PROJECTION_TRANSACTION_JOURNAL_TEMP { + return false; + } + std::fs::symlink_metadata(entry.path()) + .is_ok_and(|metadata| metadata.is_file() && !metadata.file_type().is_symlink()) + }); + if !safe_initial_stage { + return Err( + "unrecognized provider transaction remains; retain inputs and repair".into(), + ); + } + if transaction + .join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL_TEMP) + .exists() + { + std::fs::remove_file( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL_TEMP), + ) + .map_err(|_| { + "provider initial transaction stage cleanup failed; retain inputs and repair" + })?; + } + std::fs::remove_dir(transaction).map_err(|_| { + "provider initial transaction stage cleanup failed; retain inputs and repair" + })?; + sync_provider_projection_directory(provider_home)?; + continue; + }; + let journal: ProviderProjectionTransactionJournal = serde_json::from_slice(&bytes) + .map_err(|_| "provider transaction journal invalid; retain inputs and repair")?; + if journal.version != 1 { + return Err( + "provider transaction journal version is unsupported; retain inputs and repair" + .into(), + ); + } + match journal.phase { + ProviderProjectionTransactionPhase::Preparing => { + cleanup_provider_projection_transaction( + transaction, + provider_home, + ProviderProjectionTransactionPhase::Preparing, + )?; + } + ProviderProjectionTransactionPhase::Committed + | ProviderProjectionTransactionPhase::RolledBack => { + // A visible commit/rollback journal is not a confirmed state + // until its containing directory flush succeeds. Retry that + // flush before deleting the backups that make the phase safe. + sync_provider_projection_directory(transaction)?; + cleanup_provider_projection_transaction(transaction, provider_home, journal.phase)?; + } + ProviderProjectionTransactionPhase::Publishing => { + publishing.push((transaction, journal)) + } + } + } + if publishing.len() > 1 { + return Err( + "multiple interrupted provider publications require repair; retain inputs".into(), + ); + } + if let Some((transaction, journal)) = publishing.pop() { + emit_environment_event( + serde_json::json!({"severity":"warn","event":"provider.projection.transaction.recovery_started","stage":"recovery","transaction":transaction.file_name().and_then(|name| name.to_str())}), + ); + recover_provider_projection_transaction(transaction, provider_home, &journal)?; + cleanup_provider_projection_transaction( + transaction, + provider_home, + ProviderProjectionTransactionPhase::RolledBack, + )?; + emit_environment_event( + serde_json::json!({"severity":"info","event":"provider.projection.transaction.recovered","stage":"recovery"}), + ); + } + Ok(()) +} + fn read_projection_metadata(root: &Path, relative: &str) -> Result<Option<Vec<u8>>, String> { match std::fs::symlink_metadata(root.join(relative)) { Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), @@ -3063,11 +3897,14 @@ fn refresh_provider_config_references( run_as_gid, ); } + let _projection_lock = lock_provider_projection(provider_home)?; + recover_provider_projection_transactions(provider_home)?; + cleanup_orphaned_provider_refresh_stages(provider_home, None)?; let stage = tempfile::Builder::new() .prefix(".freshell-provider-refresh-") .tempdir_in(provider_home) .map_err(|error| error.to_string())?; - refresh_provider_config_in_stage( + refresh_provider_config_in_stage_locked( context, provider, workspace, @@ -3077,6 +3914,7 @@ fn refresh_provider_config_references( ) } +#[cfg(test)] fn refresh_provider_config_in_stage( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -3084,6 +3922,27 @@ fn refresh_provider_config_in_stage( user_provider_root: &Path, provider_home: &Path, stage: tempfile::TempDir, +) -> Result<(), String> { + let _projection_lock = lock_provider_projection(provider_home)?; + recover_provider_projection_transactions(provider_home)?; + cleanup_orphaned_provider_refresh_stages(provider_home, Some(stage.path()))?; + refresh_provider_config_in_stage_locked( + context, + provider, + workspace, + user_provider_root, + provider_home, + stage, + ) +} + +fn refresh_provider_config_in_stage_locked( + context: &freshell_runtime_protocol::ProviderLaunchContext, + provider: &str, + workspace: &Path, + user_provider_root: &Path, + provider_home: &Path, + stage: tempfile::TempDir, ) -> Result<(), String> { let projection = stage.path().join("projection"); std::fs::create_dir(&projection).map_err(|error| error.to_string())?; @@ -3174,7 +4033,7 @@ fn refresh_provider_config_in_stage( ) .map_err(|_| "provider reference alias metadata write failed")?; allow_provider_read_stage(stage.path(), unsafe { libc::getegid() })?; - apply_staged_provider_config(stage.path(), provider_home) + apply_staged_provider_config_locked(stage.path(), provider_home) } #[cfg(test)] @@ -3251,12 +4110,56 @@ fn refresh_provider_config_as_provider( gid, std::time::Duration::from_secs(25), &stage, - )?; + ); + match status { + Ok(status) if status.success() => Ok(()), + Ok(status) => { + let cause = format!("provider-owned config refresh failed with status {status}"); + recover_provider_projection_as_provider(provider_home, uid, gid).map_err( + |recovery| format!("{cause}; interrupted projection recovery failed: {recovery}"), + )?; + Err(cause) + } + Err(cause) if cause.contains("provider-owned config refresh timed out") => { + recover_provider_projection_as_provider(provider_home, uid, gid).map_err( + |recovery| format!("{cause}; interrupted projection recovery failed: {recovery}"), + )?; + Err(cause) + } + Err(cause) => Err(cause), + } +} + +fn recover_provider_projection_as_provider( + provider_home: &Path, + uid: u32, + gid: u32, +) -> Result<(), String> { + let executable = std::env::current_exe() + .map_err(|_| "provider projection recovery executable unavailable")?; + let status = std::process::Command::new("/usr/bin/setpriv") + .args(provider_identity_args(uid, gid)) + .args(["/usr/bin/timeout", "--kill-after=2s", "30s"]) + .arg(executable) + .args(["refresh-provider-config", "--recover"]) + .arg(provider_home) + .env_clear() + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()) + .status() + .map_err(|_| "provider projection recovery failed to start")?; if status.success() { + emit_environment_event( + serde_json::json!({"severity":"info","event":"provider.projection.transaction.timeout_recovered","stage":"recovery"}), + ); Ok(()) } else { + emit_environment_event( + serde_json::json!({"severity":"error","event":"provider.projection.transaction.timeout_recovery_failed","stage":"recovery","status":status.to_string()}), + ); Err(format!( - "provider-owned config refresh failed with status {status}" + "provider projection recovery exited with status {status}" )) } } @@ -3394,6 +4297,12 @@ fn allow_provider_read_stage(path: &Path, gid: u32) -> Result<(), String> { } fn refresh_provider_config_worker(args: &[String]) -> Result<(), String> { + if args.len() == 2 && args[0] == "--recover" { + let provider_home = Path::new(&args[1]); + let _projection_lock = lock_provider_projection(provider_home)?; + recover_provider_projection_transactions(provider_home)?; + return cleanup_orphaned_provider_refresh_stages(provider_home, None); + } if args.len() == 6 && args[0] == "--prepare" { use std::io::Read; let mut input = std::io::stdin().lock(); @@ -3430,15 +4339,19 @@ fn refresh_provider_config_worker(args: &[String]) -> Result<(), String> { if transaction.get_version_num() != 4 || transaction.to_string() != args[5] { return Err("provider refresh transaction invalid".into()); } + let provider_home = Path::new(&args[4]); + let _projection_lock = lock_provider_projection(provider_home)?; + recover_provider_projection_transactions(provider_home)?; + cleanup_orphaned_provider_refresh_stages(provider_home, None)?; let name = format!(".freshell-provider-refresh-{transaction}"); let stage = tempfile::Builder::new() .prefix(&name) .rand_bytes(0) - .tempdir_in(&args[4]) + .tempdir_in(provider_home) .map_err(|error| { format!("provider refresh transaction creation unavailable: {error}") })?; - return refresh_provider_config_in_stage( + return refresh_provider_config_in_stage_locked( &context, &args[1], Path::new(&args[2]), @@ -3454,6 +4367,13 @@ fn refresh_provider_config_worker(args: &[String]) -> Result<(), String> { } fn apply_staged_provider_config(stage: &Path, provider_home: &Path) -> Result<(), String> { + let _projection_lock = lock_provider_projection(provider_home)?; + recover_provider_projection_transactions(provider_home)?; + cleanup_orphaned_provider_refresh_stages(provider_home, None)?; + apply_staged_provider_config_locked(stage, provider_home) +} + +fn apply_staged_provider_config_locked(stage: &Path, provider_home: &Path) -> Result<(), String> { let destinations: Vec<String> = serde_json::from_slice( &read_projection_metadata(stage, "manifest.json")? .ok_or("provider projection destinations unavailable")?, @@ -3678,12 +4598,23 @@ fn publish_provider_projection_with_manifest( emit_environment_event( serde_json::json!({"severity":"info","event":"provider.projection.transaction.started","provider":provider,"stage":"stage","previous_entries":previous.len(),"desired_entries":manifest_current.len()}), ); + let previous_manifest = read_projection_metadata(home, PROVIDER_PROJECTION_MANIFEST)?; let transaction = tempfile::Builder::new() - .prefix(".freshell-provider-transaction-") + .prefix(PROVIDER_PROJECTION_TRANSACTION_PREFIX) .tempdir_in(home) .map_err(|error| format!("provider transaction stage: {error}"))?; - let desired = transaction.path().join("desired"); - let backup = transaction.path().join("backup"); + let transaction_path = transaction.keep(); + sync_provider_projection_directory(home)?; + let mut journal = ProviderProjectionTransactionJournal { + version: 1, + phase: ProviderProjectionTransactionPhase::Preparing, + manifest_existed: previous_manifest.is_some(), + entries: Vec::new(), + }; + write_provider_projection_transaction_journal(&transaction_path, &journal) + .map_err(|error| error.to_string())?; + let desired = transaction_path.join("desired"); + let backup = transaction_path.join("backup"); fs::create_dir(&backup).map_err(|error| error.to_string())?; let staged = (|| { let report = copy_environment_with_events( @@ -3706,7 +4637,7 @@ fn publish_provider_projection_with_manifest( ); } // Serialize, write and sync the complete manifest before changing HOME. - let manifest = write_projection_manifest(transaction.path(), &manifest_current)?; + let manifest = write_projection_manifest(&transaction_path, &manifest_current)?; // Links crossing mounted resource boundaries must still resolve in the // final HOME graph, not relative to the independent alias mount. for relative in current { @@ -3742,6 +4673,16 @@ fn publish_provider_projection_with_manifest( emit_environment_event( serde_json::json!({"severity":"error","event":"provider.projection.transaction.failed","provider":provider,"stage":"stage","cause":cause}), ); + if let Err(cleanup_cause) = cleanup_provider_projection_transaction( + &transaction_path, + home, + ProviderProjectionTransactionPhase::Preparing, + ) { + return Err(format!( + "{cause}; staged transaction cleanup failed: {cleanup_cause}; retain backup {}", + transaction_path.display() + )); + } return Err(cause); } }; @@ -3781,6 +4722,11 @@ fn publish_provider_projection_with_manifest( } // Mounted files keep their inode. Back up their complete prior contents // before any mutation, with the same shared copy byte profile. + for (relative, metadata) in &old { + if metadata.is_file() { + sync_provider_projection_file(&home.join(relative))?; + } + } let limits = freshell_platform::provider_environment::CopyLimits::default(); let mut bytes = 0u64; for (relative, metadata) in &old { @@ -3801,6 +4747,11 @@ fn publish_provider_projection_with_manifest( .map_err(|error| error.to_string())?; fs::copy(home.join(relative), &saved) .map_err(|error| format!("provider transaction backup {relative}: {error}"))?; + fs::File::open(&saved) + .and_then(|file| file.sync_all()) + .map_err(|error| { + format!("provider transaction backup sync {relative}: {error}") + })?; } } Ok(()) @@ -3809,13 +4760,67 @@ fn publish_provider_projection_with_manifest( emit_environment_event( serde_json::json!({"severity":"error","event":"provider.projection.transaction.failed","provider":provider,"stage":"validate","cause":cause}), ); + if let Err(cleanup_cause) = cleanup_provider_projection_transaction( + &transaction_path, + home, + ProviderProjectionTransactionPhase::Preparing, + ) { + return Err(format!( + "{cause}; staged transaction cleanup failed: {cleanup_cause}; retain backup {}", + transaction_path.display() + )); + } return Err(cause); } + if let Some(bytes) = previous_manifest.as_deref() { + use std::io::Write; + use std::os::unix::fs::OpenOptionsExt; + let saved_manifest = transaction_path.join(PROVIDER_PROJECTION_TRANSACTION_MANIFEST_BACKUP); + let mut file = fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(saved_manifest) + .map_err(|_| "provider transaction manifest backup unavailable")?; + file.write_all(bytes) + .and_then(|()| file.sync_all()) + .map_err(|_| "provider transaction manifest backup write unavailable")?; + } + let mut touched_paths = std::collections::BTreeSet::new(); + touched_paths.extend( + previous + .union(current) + .filter(|relative| { + !protected.contains(*relative) && !transactional_alias_paths.contains(*relative) + }) + .cloned(), + ); + touched_paths.extend(transactional_alias_paths.iter().cloned()); + journal.entries = touched_paths + .into_iter() + .map(|relative| { + let in_place = mounted.contains(&PathBuf::from(&relative)) + && incoming + .get(&relative) + .is_some_and(|metadata| metadata.is_file()) + && old + .get(&relative) + .is_some_and(|metadata| metadata.is_file()) + && !(preserve_existing_mounted_files.contains(&PathBuf::from(&relative)) + && previous.contains(&relative)); + Ok(ProviderProjectionTransactionEntry { + original: provider_projection_original_node(home, &relative, in_place)?, + relative, + }) + }) + .collect::<Result<Vec<_>, String>>()?; + sync_provider_projection_tree(&transaction_path)?; + sync_provider_projection_directory(home)?; + journal.phase = ProviderProjectionTransactionPhase::Publishing; + write_provider_projection_transaction_journal(&transaction_path, &journal) + .map_err(|error| error.to_string())?; let mut moved = Vec::<String>::new(); let mut installed = Vec::<String>::new(); - let mut created = Vec::<String>::new(); - let mut removed = Vec::<(String, u32)>::new(); - let mut touched_mounts = Vec::<String>::new(); let publication: Result<(), String> = (|| { for (relative, metadata) in &old { if metadata.is_dir() || !previous.contains(relative) || protected.contains(relative) { @@ -3831,13 +4836,19 @@ fn publish_provider_projection_with_manifest( .map_err(|error| format!("provider transaction backup {relative}: {error}"))?; moved.push(relative.clone()); } + backup_provider_reference_aliases(home, current, &reference_aliases, &backup, &mut moved)?; + // The old nodes and their backups must be durable before any new node + // is installed. Recovery can then restore the prior projection even + // after a machine crash during publication. + sync_provider_projection_tree(&backup)?; + sync_provider_projection_directories(home, &journal.entries)?; let mut directories = old .iter() .filter(|(_, metadata)| metadata.is_dir()) .collect::<Vec<_>>(); directories .sort_by_key(|(path, _)| std::cmp::Reverse(Path::new(path).components().count())); - for (relative, metadata) in directories { + for (relative, _) in directories { if !previous.contains(relative) || protected.contains(relative) || incoming @@ -3848,7 +4859,7 @@ fn publish_provider_projection_with_manifest( continue; } match fs::remove_dir(home.join(relative)) { - Ok(()) => removed.push((relative.clone(), metadata.permissions().mode())), + Ok(()) => {} Err(error) if error.kind() == std::io::ErrorKind::DirectoryNotEmpty && !incoming.contains_key(relative) => {} @@ -3868,7 +4879,6 @@ fn publish_provider_projection_with_manifest( fs::create_dir(&destination).map_err(|error| { format!("provider transaction create {relative}: {error}") })?; - created.push(relative.clone()); fs::set_permissions(&destination, fs::Permissions::from_mode(0o700)) .map_err(|error| error.to_string())?; } @@ -3885,7 +4895,6 @@ fn publish_provider_projection_with_manifest( { continue; } - touched_mounts.push(relative.clone()); replace_mounted_projection_file(&desired.join(relative), &destination)?; } else { fs::rename(desired.join(relative), &destination) @@ -3903,30 +4912,12 @@ fn publish_provider_projection_with_manifest( )?; publish_manifest(&manifest, &home.join(PROVIDER_PROJECTION_MANIFEST)) .map_err(|error| format!("provider transaction publish manifest: {error}"))?; + sync_provider_projection_directories(home, &journal.entries)?; + sync_provider_projection_tree(&backup)?; Ok(()) })(); if let Err(cause) = publication { - let rollback: Result<(), String> = (|| { - for relative in installed.iter().rev() { - fs::remove_file(home.join(relative)).map_err(|error| error.to_string())?; - } - for relative in created.iter().rev() { - fs::remove_dir(home.join(relative)).map_err(|error| error.to_string())?; - } - for (relative, mode) in removed.iter().rev() { - fs::create_dir(home.join(relative)).map_err(|error| error.to_string())?; - fs::set_permissions(home.join(relative), fs::Permissions::from_mode(*mode)) - .map_err(|error| error.to_string())?; - } - for relative in moved.iter().rev() { - fs::rename(backup.join(relative), home.join(relative)) - .map_err(|error| error.to_string())?; - } - for relative in touched_mounts.iter().rev() { - replace_mounted_projection_file(&backup.join(relative), &home.join(relative))?; - } - Ok(()) - })(); + let rollback = recover_provider_projection_transaction(&transaction_path, home, &journal); emit_environment_event( serde_json::json!({"severity":if rollback.is_ok(){"info"}else{"error"},"event":"provider.projection.transaction.rollback","provider":provider,"stage":"rollback","restored":moved.len(),"removed":installed.len(),"successful":rollback.is_ok()}), ); @@ -3934,18 +4925,72 @@ fn publish_provider_projection_with_manifest( serde_json::json!({"severity":"error","event":"provider.projection.transaction.failed","provider":provider,"stage":"publish","cause":cause}), ); if let Err(rollback_cause) = rollback { - let retained = transaction.keep(); return Err(format!( "{cause}; rollback failed: {rollback_cause}; retain backup {} and repair", - retained.display() + transaction_path.display() + )); + } + if let Err(cleanup_cause) = cleanup_provider_projection_transaction( + &transaction_path, + home, + ProviderProjectionTransactionPhase::RolledBack, + ) { + return Err(format!( + "{cause}; prior projection restored but backup cleanup failed: {cleanup_cause}; retain backup {}", + transaction_path.display() )); } return Err(cause); } + journal.phase = ProviderProjectionTransactionPhase::Committed; + match write_provider_projection_transaction_journal(&transaction_path, &journal) { + Ok(()) => {} + Err(ProviderProjectionJournalWriteError::PublishedButUnconfirmed(cause)) => { + emit_environment_event(serde_json::json!({ + "severity": "error", + "event": "provider.projection.transaction.commit_uncertain", + "provider": provider, + "stage": "commit", + "cause": cause, + "transaction": transaction_path.display().to_string(), + })); + return Err(format!( + "{cause}; provider projection is installed but commit durability is unconfirmed; retain backup {} and repair", + transaction_path.display() + )); + } + Err(ProviderProjectionJournalWriteError::BeforePublish(cause)) => { + let rollback = + recover_provider_projection_transaction(&transaction_path, home, &journal); + if let Err(rollback_cause) = rollback { + return Err(format!( + "{cause}; provider projection recovery failed: {rollback_cause}; retain backup {} and repair", + transaction_path.display() + )); + } + if let Err(cleanup_cause) = cleanup_provider_projection_transaction( + &transaction_path, + home, + ProviderProjectionTransactionPhase::RolledBack, + ) { + return Err(format!( + "{cause}; prior projection restored but backup cleanup failed: {cleanup_cause}; retain backup {}", + transaction_path.display() + )); + } + return Err(format!( + "{cause}; prior provider projection restored; retain inputs and retry" + )); + } + } emit_environment_event( serde_json::json!({"severity":"info","event":"provider.projection.transaction.committed","provider":provider,"stage":"commit","installed":installed.len(),"retired":moved.len()}), ); - if let Err(error) = transaction.close() { + if let Err(error) = cleanup_provider_projection_transaction( + &transaction_path, + home, + ProviderProjectionTransactionPhase::Committed, + ) { emit_environment_event( serde_json::json!({"severity":"error","event":"provider.projection.transaction.cleanup_failed","provider":provider,"stage":"cleanup","cause":error.to_string()}), ); @@ -3994,50 +5039,136 @@ fn preflight_provider_reference_aliases( return Err("provider reference alias conflicts with an existing link".into()); } } - Ok(_) => { - return Err(format!( - "provider-created resource blocks provider reference alias {}; retain it and repair the selected path", - relative.display() - )) + Ok(_) => { + return Err(format!( + "provider-created resource blocks provider reference alias {}; retain it and repair the selected path", + relative.display() + )) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err("provider reference alias path could not be inspected".into()), + } + } + for alias in &plan.retired { + let relative = alias.provider_path(); + let _ = provider_reference_alias_parent_available(home, &relative, current)?; + } + Ok(()) +} + +fn provider_reference_alias_parent_available( + home: &Path, + relative: &Path, + current: &std::collections::BTreeSet<String>, +) -> Result<bool, String> { + let parent = relative + .parent() + .ok_or("provider reference alias has no parent directory")?; + let mut prefix = PathBuf::new(); + for component in parent.components() { + prefix.push(component.as_os_str()); + let absolute = home.join(&prefix); + match std::fs::symlink_metadata(&absolute) { + Ok(metadata) if metadata.is_dir() => {} + Ok(_) => return Err("provider reference alias parent directory is unsafe".into()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + let prefix = prefix + .to_str() + .ok_or("provider reference alias path is not UTF-8")?; + return Ok(current.contains(prefix)); + } + Err(_) => return Err("provider reference alias parent directory is unavailable".into()), + } + } + Ok(true) +} + +fn backup_provider_reference_aliases( + home: &Path, + current: &std::collections::BTreeSet<String>, + plan: &ProviderResourceReferenceAliasPlan, + backup: &Path, + moved: &mut Vec<String>, +) -> Result<(), String> { + for alias in &plan.active { + let relative = alias.provider_path(); + if !provider_reference_alias_parent_available(home, &relative, current)? { + continue; + } + let destination = home.join(&relative); + let existing = match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + Some(std::fs::read_link(&destination).map_err(|_| { + "provider reference alias could not be inspected" + })?) + } + Ok(_) => { + return Err(format!( + "provider-created resource blocks provider reference alias {}; retain it and repair the selected path", + relative.display() + )) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => None, + Err(_) => return Err("provider reference alias path could not be inspected".into()), + }; + let Some(link) = existing else { continue }; + let target = PathBuf::from(format!( + "/home/freshell/provider/.freshell/references/{}", + alias.reference_index + )); + if link == target { + continue; + } + if managed_provider_reference_alias_index(&link, &destination, home).is_none() { + return Err("provider reference alias conflicts with an existing link".into()); + } + let relative_string = relative + .to_str() + .ok_or("provider reference alias path is not UTF-8")? + .to_owned(); + let saved = backup.join(&relative); + std::fs::create_dir_all(saved.parent().ok_or("backup parent unavailable")?) + .map_err(|error| error.to_string())?; + std::fs::rename(&destination, saved) + .map_err(|error| format!("provider reference alias backup: {error}"))?; + moved.push(relative_string); + } + for alias in &plan.retired { + let relative = alias.provider_path(); + let relative_string = relative + .to_str() + .ok_or("provider reference alias path is not UTF-8")? + .to_owned(); + if current.contains(&relative_string) + || !provider_reference_alias_parent_available(home, &relative, current)? + { + continue; + } + let destination = home.join(&relative); + let link = match std::fs::symlink_metadata(&destination) { + Ok(metadata) if metadata.file_type().is_symlink() => { + std::fs::read_link(&destination) + .map_err(|_| "retired provider reference alias could not be inspected")? + } + Ok(_) => continue, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => { + return Err("retired provider reference alias path could not be inspected".into()) } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} - Err(_) => return Err("provider reference alias path could not be inspected".into()), + }; + if managed_provider_reference_alias_index(&link, &destination, home).is_none() { + continue; } - } - for alias in &plan.retired { - let relative = alias.provider_path(); - let _ = provider_reference_alias_parent_available(home, &relative, current)?; + let saved = backup.join(&relative); + std::fs::create_dir_all(saved.parent().ok_or("backup parent unavailable")?) + .map_err(|error| error.to_string())?; + std::fs::rename(&destination, saved) + .map_err(|error| format!("retired provider reference alias backup: {error}"))?; + moved.push(relative_string); } Ok(()) } -fn provider_reference_alias_parent_available( - home: &Path, - relative: &Path, - current: &std::collections::BTreeSet<String>, -) -> Result<bool, String> { - let parent = relative - .parent() - .ok_or("provider reference alias has no parent directory")?; - let mut prefix = PathBuf::new(); - for component in parent.components() { - prefix.push(component.as_os_str()); - let absolute = home.join(&prefix); - match std::fs::symlink_metadata(&absolute) { - Ok(metadata) if metadata.is_dir() => {} - Ok(_) => return Err("provider reference alias parent directory is unsafe".into()), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => { - let prefix = prefix - .to_str() - .ok_or("provider reference alias path is not UTF-8")?; - return Ok(current.contains(prefix)); - } - Err(_) => return Err("provider reference alias parent directory is unavailable".into()), - } - } - Ok(true) -} - fn publish_provider_reference_aliases( home: &Path, current: &std::collections::BTreeSet<String>, @@ -4152,6 +5283,14 @@ fn validate_owned_projection_descendants( } fn replace_mounted_projection_file(source: &Path, destination: &Path) -> Result<(), String> { + replace_mounted_projection_file_with_sync(source, destination, |file| file.sync_all()) +} + +fn replace_mounted_projection_file_with_sync( + source: &Path, + destination: &Path, + mut sync_file: impl FnMut(&std::fs::File) -> std::io::Result<()>, +) -> Result<(), String> { use std::{fs, io::Write}; let mut input = fs::File::open(source).map_err(|error| error.to_string())?; let mut output = fs::OpenOptions::new() @@ -4161,14 +5300,15 @@ fn replace_mounted_projection_file(source: &Path, destination: &Path) -> Result< .map_err(|error| error.to_string())?; std::io::copy(&mut input, &mut output).map_err(|error| error.to_string())?; output.flush().map_err(|error| error.to_string())?; - output.sync_all().map_err(|error| error.to_string())?; + sync_file(&output).map_err(|error| error.to_string())?; fs::set_permissions( destination, fs::metadata(source) .map_err(|error| error.to_string())? .permissions(), ) - .map_err(|error| error.to_string()) + .map_err(|error| error.to_string())?; + sync_file(&output).map_err(|error| format!("failed to flush mounted projection mode: {error}")) } fn prepare_provider_context( @@ -5205,6 +6345,103 @@ fn now_millis() -> i64 { mod tests { #[path = "probe_recovery_tests.rs"] mod probe_recovery; + + #[test] + fn committed_journal_directory_sync_failure_is_not_treated_as_pre_publish() { + use super::*; + use std::io; + + let transaction = tempfile::tempdir().unwrap(); + let journal = ProviderProjectionTransactionJournal { + version: 1, + phase: ProviderProjectionTransactionPhase::Committed, + manifest_existed: false, + entries: Vec::new(), + }; + + let error = write_provider_projection_transaction_journal_with_sync( + transaction.path(), + &journal, + |_| Err(io::Error::other("injected directory flush failure")), + ) + .unwrap_err(); + + assert!(matches!( + error, + ProviderProjectionJournalWriteError::PublishedButUnconfirmed(_) + )); + let published: ProviderProjectionTransactionJournal = serde_json::from_slice( + &std::fs::read( + transaction + .path() + .join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL), + ) + .unwrap(), + ) + .unwrap(); + assert_eq!( + published.phase, + ProviderProjectionTransactionPhase::Committed + ); + } + + #[cfg(unix)] + #[test] + fn mounted_projection_mode_is_flushed_after_permissions_change() { + use super::*; + use std::os::unix::fs::PermissionsExt; + + let directory = tempfile::tempdir().unwrap(); + let source = directory.path().join("source"); + let destination = directory.path().join("mounted"); + std::fs::write(&source, b"provider state").unwrap(); + std::fs::write(&destination, b"old state").unwrap(); + std::fs::set_permissions(&source, std::fs::Permissions::from_mode(0o600)).unwrap(); + std::fs::set_permissions(&destination, std::fs::Permissions::from_mode(0o644)).unwrap(); + let mut observed_modes = Vec::new(); + + replace_mounted_projection_file_with_sync(&source, &destination, |file| { + file.sync_all()?; + observed_modes.push(std::fs::metadata(&destination)?.permissions().mode() & 0o777); + Ok(()) + }) + .unwrap(); + + assert_eq!(std::fs::read(&destination).unwrap(), b"provider state"); + assert_eq!(observed_modes, vec![0o644, 0o600]); + } + + #[cfg(unix)] + #[test] + fn provider_projection_sync_walk_flushes_files_and_directories_without_following_links() { + use std::os::unix::fs::symlink; + + let tree = tempfile::tempdir().unwrap(); + let outside = tempfile::tempdir().unwrap(); + let nested = tree.path().join("nested"); + std::fs::create_dir(&nested).unwrap(); + let file = nested.join("provider.json"); + std::fs::write(&file, "projected provider state").unwrap(); + let outside_file = outside.path().join("outside.json"); + std::fs::write(&outside_file, "outside state").unwrap(); + symlink(&outside_file, nested.join("outside-link.json")).unwrap(); + + let mut synced = Vec::new(); + super::sync_provider_projection_tree_with(tree.path(), &mut |path| { + synced.push(path.to_path_buf()); + Ok(()) + }) + .unwrap(); + + assert!(synced.contains(&file)); + assert!(synced.contains(&nested)); + assert!(synced.contains(&tree.path().to_path_buf())); + assert!( + !synced.contains(&outside_file), + "sync traversal must not follow provider-controlled symlinks" + ); + } + fn copy_provider_config_references( context: &freshell_runtime_protocol::ProviderLaunchContext, provider: &str, @@ -5877,6 +7114,350 @@ mod tests { .is_symlink()); } + #[test] + fn provider_config_refresh_recovers_interrupted_projection_before_next_refresh() { + use super::*; + use std::{fs, os::unix::fs::PermissionsExt}; + + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let transaction = home.join(".freshell-provider-transaction-interrupted"); + let backup = transaction.join("backup"); + let stage = root.path().join("stage"); + fs::create_dir_all(home.join("skills")).unwrap(); + fs::create_dir_all(backup.join("skills")).unwrap(); + fs::create_dir_all(stage.join("projection/skills")).unwrap(); + fs::set_permissions(home.join("skills"), fs::Permissions::from_mode(0o751)).unwrap(); + fs::write(home.join("skills/command"), b"new-resource").unwrap(); + fs::write(home.join("skills/new"), b"new-owned").unwrap(); + fs::write(home.join("skills/provider-created"), b"native-state").unwrap(); + fs::write(backup.join("skills/command"), b"old-resource").unwrap(); + fs::write(backup.join("skills/stale"), b"prior-owned").unwrap(); + let previous = std::collections::BTreeSet::from([ + "skills".to_owned(), + "skills/command".to_owned(), + "skills/stale".to_owned(), + ]); + let current = std::collections::BTreeSet::from([ + "skills".to_owned(), + "skills/command".to_owned(), + "skills/new".to_owned(), + ]); + let previous_manifest = + freshell_runtime_protocol::bounded_json_bytes(&previous, MAX_PROJECTION_METADATA_BYTES) + .unwrap(); + let current_manifest = + freshell_runtime_protocol::bounded_json_bytes(¤t, MAX_PROJECTION_METADATA_BYTES) + .unwrap(); + fs::write(transaction.join("manifest.backup"), &previous_manifest).unwrap(); + fs::write(home.join(PROVIDER_PROJECTION_MANIFEST), ¤t_manifest).unwrap(); + let entries = serde_json::json!([ + { "relative": "skills", "original": { "kind": "directory", "mode": 0o751 } }, + { "relative": "skills/command", "original": { "kind": "file", "in_place": false } }, + { "relative": "skills/stale", "original": { "kind": "file", "in_place": false } }, + { "relative": "skills/new", "original": { "kind": "missing" } } + ]); + fs::write( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL), + serde_json::to_vec(&serde_json::json!({ + "version": 1, + "phase": "publishing", + "manifestExisted": true, + "entries": entries, + })) + .unwrap(), + ) + .unwrap(); + fs::write(stage.join("projection/skills/command"), b"latest-resource").unwrap(); + fs::write(stage.join("projection/skills/new"), b"latest-new-owned").unwrap(); + write_projection_manifest(&stage.join("projection"), ¤t).unwrap(); + fs::write( + stage.join("manifest.json"), + serde_json::to_vec(&["skills"]).unwrap(), + ) + .unwrap(); + + apply_staged_provider_config(&stage, &home).unwrap(); + + assert_eq!( + fs::read(home.join("skills/command")).unwrap(), + b"latest-resource" + ); + assert!(!home.join("skills/stale").exists()); + assert_eq!( + fs::read(home.join("skills/provider-created")).unwrap(), + b"native-state" + ); + assert_eq!( + fs::read(home.join("skills/new")).unwrap(), + b"latest-new-owned" + ); + assert_eq!( + fs::metadata(home.join("skills")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o751 + ); + assert_eq!( + fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(), + current_manifest + ); + assert!(!transaction.exists()); + } + + #[cfg(unix)] + #[test] + fn provider_config_refresh_recovery_restores_mounted_inode_and_reference_alias() { + use super::*; + use std::{ + fs, + os::unix::fs::{symlink, MetadataExt}, + }; + + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let transaction = home.join(".freshell-provider-transaction-mounted"); + let backup = transaction.join("backup/.freshell/provider-roots/mcp-project"); + let resource = home.join(".freshell/provider-roots/mcp-project"); + let references = home.join(".freshell/references"); + fs::create_dir_all(&resource).unwrap(); + fs::create_dir_all(&references).unwrap(); + fs::create_dir_all(&backup).unwrap(); + fs::write(references.join("0"), b"new reference").unwrap(); + fs::write(references.join("1"), b"old reference").unwrap(); + let mounted_file = resource.join("state.json"); + fs::write(&mounted_file, b"new mounted state").unwrap(); + let original_inode = fs::metadata(&mounted_file).unwrap().ino(); + fs::write(backup.join("state.json"), b"old mounted state").unwrap(); + symlink( + "/home/freshell/provider/.freshell/references/0", + resource.join(".env"), + ) + .unwrap(); + symlink( + "/home/freshell/provider/.freshell/references/1", + backup.join(".env"), + ) + .unwrap(); + fs::write(resource.join("new-file"), b"uncommitted").unwrap(); + let previous = std::collections::BTreeSet::from([ + ".freshell".to_owned(), + ".freshell/provider-roots".to_owned(), + ".freshell/provider-roots/mcp-project".to_owned(), + ".freshell/provider-roots/mcp-project/state.json".to_owned(), + ".freshell/provider-roots/mcp-project/.env".to_owned(), + ]); + let current = std::collections::BTreeSet::from([ + ".freshell".to_owned(), + ".freshell/provider-roots".to_owned(), + ".freshell/provider-roots/mcp-project".to_owned(), + ".freshell/provider-roots/mcp-project/state.json".to_owned(), + ".freshell/provider-roots/mcp-project/.env".to_owned(), + ".freshell/provider-roots/mcp-project/new-file".to_owned(), + ]); + let previous_manifest = + freshell_runtime_protocol::bounded_json_bytes(&previous, MAX_PROJECTION_METADATA_BYTES) + .unwrap(); + let current_manifest = + freshell_runtime_protocol::bounded_json_bytes(¤t, MAX_PROJECTION_METADATA_BYTES) + .unwrap(); + fs::write( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_MANIFEST_BACKUP), + &previous_manifest, + ) + .unwrap(); + fs::write(home.join(PROVIDER_PROJECTION_MANIFEST), current_manifest).unwrap(); + let entries = serde_json::json!([ + { "relative": ".freshell", "original": { "kind": "directory", "mode": 0o700 } }, + { "relative": ".freshell/provider-roots", "original": { "kind": "directory", "mode": 0o700 } }, + { "relative": ".freshell/provider-roots/mcp-project", "original": { "kind": "directory", "mode": 0o700 } }, + { "relative": ".freshell/provider-roots/mcp-project/state.json", "original": { "kind": "file", "in_place": true } }, + { "relative": ".freshell/provider-roots/mcp-project/.env", "original": { "kind": "symlink" } }, + { "relative": ".freshell/provider-roots/mcp-project/new-file", "original": { "kind": "missing" } } + ]); + fs::write( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL), + serde_json::to_vec(&serde_json::json!({ + "version": 1, + "phase": "publishing", + "manifestExisted": true, + "entries": entries, + })) + .unwrap(), + ) + .unwrap(); + + recover_provider_projection_transactions(&home).unwrap(); + + assert_eq!(fs::read(&mounted_file).unwrap(), b"old mounted state"); + assert_eq!(fs::metadata(&mounted_file).unwrap().ino(), original_inode); + assert_eq!( + fs::read_link(resource.join(".env")).unwrap(), + Path::new("/home/freshell/provider/.freshell/references/1") + ); + assert!(!resource.join("new-file").exists()); + assert_eq!( + fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(), + previous_manifest + ); + assert!(!transaction.exists()); + } + + #[test] + fn provider_projection_recovery_cleans_a_rolled_back_transaction_without_replaying_it() { + use super::*; + use std::fs; + + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let transaction = home.join(format!( + "{PROVIDER_PROJECTION_TRANSACTION_PREFIX}rolled-back" + )); + fs::create_dir_all(transaction.join("desired/partial")).unwrap(); + fs::create_dir_all(&home).unwrap(); + fs::write(transaction.join("desired/partial/file"), b"staged residue").unwrap(); + fs::write( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL), + serde_json::to_vec(&serde_json::json!({ + "version": 1, + "phase": "rolled_back", + "manifestExisted": false, + "entries": [ + { "relative": "mounted/state.json", "original": { "kind": "file", "in_place": true } } + ], + })) + .unwrap(), + ) + .unwrap(); + + recover_provider_projection_transactions(&home).unwrap(); + + assert!( + !transaction.exists(), + "a rollback-complete journal makes leftover stage data safe to remove" + ); + } + + #[test] + fn provider_refresh_recovery_removes_an_interrupted_private_stage() { + use super::*; + use std::fs; + + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let abandoned = home.join(".freshell-provider-refresh-interrupted"); + fs::create_dir_all(&abandoned).unwrap(); + fs::write( + abandoned.join("private-config.json"), + b"partial selected config", + ) + .unwrap(); + fs::create_dir_all(home.join("unrelated-state")).unwrap(); + + let _projection_lock = lock_provider_projection(&home).unwrap(); + recover_provider_projection_transactions(&home).unwrap(); + cleanup_orphaned_provider_refresh_stages(&home, None).unwrap(); + + assert!( + !abandoned.exists(), + "a later refresh removes private staging data left by a killed worker" + ); + assert!(home.join("unrelated-state").is_dir()); + } + + #[test] + fn provider_projection_recovery_restores_children_after_directory_becomes_file() { + use super::*; + use std::{fs, os::unix::fs::PermissionsExt}; + + let root = tempfile::tempdir().unwrap(); + let home = root.path().join("home"); + let transaction = home.join(".freshell-provider-transaction-directory-change"); + let backup = transaction.join("backup/skills"); + fs::create_dir_all(home.join("skills")).unwrap(); + fs::create_dir_all(&backup).unwrap(); + fs::set_permissions(home.join("skills"), fs::Permissions::from_mode(0o751)).unwrap(); + fs::write(backup.join("command"), b"prior executable").unwrap(); + fs::remove_dir(home.join("skills")).unwrap(); + fs::write(home.join("skills"), b"replacement file").unwrap(); + let previous = + std::collections::BTreeSet::from(["skills".to_owned(), "skills/command".to_owned()]); + let previous_manifest = + freshell_runtime_protocol::bounded_json_bytes(&previous, MAX_PROJECTION_METADATA_BYTES) + .unwrap(); + fs::write( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_MANIFEST_BACKUP), + &previous_manifest, + ) + .unwrap(); + fs::write(home.join(PROVIDER_PROJECTION_MANIFEST), b"[\"skills\"]").unwrap(); + fs::write( + transaction.join(PROVIDER_PROJECTION_TRANSACTION_JOURNAL), + serde_json::to_vec(&serde_json::json!({ + "version": 1, + "phase": "publishing", + "manifestExisted": true, + "entries": [ + { "relative": "skills", "original": { "kind": "directory", "mode": 0o751 } }, + { "relative": "skills/command", "original": { "kind": "file", "in_place": false } } + ], + })) + .unwrap(), + ) + .unwrap(); + + recover_provider_projection_transactions(&home).unwrap(); + + assert_eq!( + fs::read(home.join("skills/command")).unwrap(), + b"prior executable" + ); + assert_eq!( + fs::metadata(home.join("skills")) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o751 + ); + assert_eq!( + fs::read(home.join(PROVIDER_PROJECTION_MANIFEST)).unwrap(), + previous_manifest + ); + assert!(!transaction.exists()); + } + + #[test] + fn provider_projection_lock_serializes_refreshes_for_one_home() { + use super::*; + use std::sync::mpsc; + + let home = tempfile::tempdir().unwrap(); + let first = lock_provider_projection(home.path()).unwrap(); + let (started_tx, started_rx) = mpsc::channel(); + let (acquired_tx, acquired_rx) = mpsc::channel(); + let second_home = home.path().to_path_buf(); + let second = std::thread::spawn(move || { + started_tx.send(()).unwrap(); + let _lock = lock_provider_projection(&second_home).unwrap(); + acquired_tx.send(()).unwrap(); + }); + + started_rx + .recv_timeout(std::time::Duration::from_secs(1)) + .expect("second refresh attempted to acquire the shared lock"); + assert!(acquired_rx + .recv_timeout(std::time::Duration::from_millis(100)) + .is_err()); + drop(first); + acquired_rx + .recv_timeout(std::time::Duration::from_secs(1)) + .expect("second refresh proceeds after the first releases the lock"); + second.join().unwrap(); + } + #[test] fn provider_config_manifest_rename_failure_restores_every_published_leaf() { use std::{collections::BTreeSet, fs, os::unix::fs::PermissionsExt}; @@ -5942,7 +7523,8 @@ mod tests { fs::rename(source, &occupied) }, ); - assert!(result.unwrap_err().contains("publish manifest")); + let error = result.unwrap_err(); + assert!(error.contains("publish manifest"), "{error}"); assert_eq!( fs::read(home.join("skills/command")).unwrap(), b"#!/bin/sh\nprintf old-resource" diff --git a/crates/freshell-session-host/src/output_journal.rs b/crates/freshell-session-host/src/output_journal.rs index bbaf62f36..0566ee3c1 100644 --- a/crates/freshell-session-host/src/output_journal.rs +++ b/crates/freshell-session-host/src/output_journal.rs @@ -18,6 +18,10 @@ use std::{ pub const DEFAULT_RING_BYTES: usize = 1024 * 1024; pub const DEFAULT_SPOOL_BYTES: u64 = 64 * 1024 * 1024; pub const MAX_TRANSPORT_CHUNK_BYTES: usize = 64 * 1024; +// JSON encodes a control byte such as NUL as six bytes (for example, `\u0000`). +// At the minimum 128 KiB segment size, 16 KiB of payload plus the largest +// descriptor-bounded terminal and epoch identifiers still fits in one line. +const MAX_SPOOL_FRAME_DATA_BYTES: usize = 16 * 1024; pub struct OutputJournal { incarnation_id: IncarnationId, @@ -112,7 +116,7 @@ impl OutputJournal { pub fn append(&mut self, data: &str) -> Result<Vec<RuntimeOutputFrame>, String> { let mut written = Vec::new(); - for chunk in split_utf8(data, MAX_TRANSPORT_CHUNK_BYTES) { + for chunk in split_utf8(data, MAX_SPOOL_FRAME_DATA_BYTES) { let seq = self.next_seq; let frame = RuntimeOutputFrame { terminal_id: self.terminal_id.clone(), @@ -188,7 +192,7 @@ impl OutputJournal { self.current_path(), &self.current_index, self.current_torn_tail, - true, + false, ), ] { let file = match std::fs::File::open(path) { @@ -210,6 +214,9 @@ impl OutputJournal { torn_tail, }); } + if segments.is_empty() { + return Err("retained output spool unavailable".into()); + } let indexed = read_indexed_spool( source, segments, @@ -248,6 +255,9 @@ impl OutputJournal { }) .map_err(|e| e.to_string())?; let segment_limit = self.spool_limit / 2; + if line.len() as u64 + 1 > segment_limit { + return Err("retained output frame exceeds segment bound".into()); + } let current = self.current_path(); let current_len = current.metadata().map(|m| m.len()).unwrap_or(0); if current_len > 0 && current_len.saturating_add(line.len() as u64 + 1) > segment_limit { @@ -356,6 +366,27 @@ fn rebuild_spool_index( return Err("retained spool exceeds segment bound".into()); } let (checkpoints, head, summary) = build_spool_index(BufReader::new(file), source, 0)?; + let (file_bytes, torn_tail) = if summary.torn_tail { + // This segment is about to become appendable again. Drop only the + // uncommitted final record so the next JSONL append starts on a fresh + // line and preserves the complete sequence already in the file. + let spool = OpenOptions::new() + .write(true) + .open(spool_path) + .map_err(|_| "retained output spool unavailable")?; + spool + .set_len(summary.last_complete_offset) + .and_then(|()| spool.sync_all()) + .map_err(|_| "retained output torn tail recovery failed")?; + tracing::warn!( + event = "runtime.output_journal.torn_tail_recovered", + discarded_bytes = file_bytes.saturating_sub(summary.last_complete_offset), + "discarded one incomplete retained-output record" + ); + (summary.last_complete_offset, false) + } else { + (file_bytes, false) + }; let scanned_through_offset = if checkpoints.is_empty() { summary.last_complete_offset } else { @@ -367,9 +398,9 @@ fn rebuild_spool_index( &checkpoints, file_bytes, scanned_through_offset, - summary.torn_tail, + torn_tail, )?; - Ok((checkpoints, head, summary.torn_tail)) + Ok((checkpoints, head, torn_tail)) } fn write_index_atomic( @@ -497,6 +528,55 @@ mod tests { assert!(read.truncated, "torn tail must be visible"); } + #[test] + fn reopening_truncates_a_torn_tail_before_the_next_jsonl_append() { + let dir = tempfile::tempdir().unwrap(); + let incarnation_id = IncarnationId::new(); + let mut journal = OutputJournal::new( + dir.path(), + incarnation_id.clone(), + "torn-restart".into(), + "epoch".into(), + ) + .unwrap(); + journal.append("complete before restart").unwrap(); + let spool_path = journal.current_path(); + drop(journal); + + let mut file = OpenOptions::new().append(true).open(&spool_path).unwrap(); + file.write_all(b"{\"frame\":{\"data\":\"partial").unwrap(); + file.sync_all().unwrap(); + drop(file); + + let mut recovered = OutputJournal::new( + dir.path(), + incarnation_id.clone(), + "torn-restart".into(), + "epoch".into(), + ) + .expect("a torn final frame is recoverable on reopen"); + assert!( + std::fs::read(&spool_path).unwrap().ends_with(b"\n"), + "reopen removes the incomplete bytes before the next append" + ); + let appended = recovered.append("complete after restart").unwrap(); + assert_eq!(appended[0].seq_start, 2); + drop(recovered); + + let reopened = OutputJournal::new( + dir.path(), + incarnation_id, + "torn-restart".into(), + "epoch".into(), + ) + .unwrap(); + let read = reopened.read(0, 65536).unwrap(); + assert_eq!(read.frames.len(), 2); + assert_eq!(read.frames[0].data, "complete before restart"); + assert_eq!(read.frames[1].data, "complete after restart"); + assert!(!read.truncated); + } + #[test] fn retained_output_rejects_foreign_terminal_frames_in_same_epoch() { let dir = tempfile::tempdir().unwrap(); @@ -547,6 +627,84 @@ mod tests { assert!(!batch.frames.is_empty()); } + #[test] + fn escaped_output_frames_fit_the_minimum_spool_segment_and_remain_readable() { + let dir = tempfile::tempdir().unwrap(); + let mut journal = OutputJournal::with_limits( + dir.path(), + IncarnationId::new(), + "t".into(), + "epoch".into(), + 64 * 1024, + 256 * 1024, + ) + .unwrap(); + let segment_limit = journal.descriptor().segment_limit_bytes; + + let frames = journal.append(&"\0".repeat(64 * 1024)).unwrap(); + + assert!( + frames.len() > 1, + "escaped data must be split before spooling" + ); + assert!( + [journal.previous_path(), journal.current_path()] + .into_iter() + .filter_map(|path| path.metadata().ok().map(|metadata| metadata.len())) + .all(|length| length <= segment_limit), + "every retained spool segment must stay within the descriptor bound" + ); + let replay = journal + .read(0, MAX_TRANSPORT_CHUNK_BYTES as u64) + .expect("valid escaped output must not be rejected as an oversized segment"); + assert!(!replay.frames.is_empty()); + assert!(replay + .frames + .iter() + .all(|frame| frame.data.chars().all(|c| c == '\0'))); + journal + .complete(0) + .expect("completion must be able to read the retained escaped output"); + } + + #[test] + fn previous_only_spool_remains_readable_after_rotation_interruption() { + let dir = tempfile::tempdir().unwrap(); + let mut journal = OutputJournal::with_limits( + dir.path(), + IncarnationId::new(), + "t".into(), + "epoch".into(), + 64 * 1024, + 256 * 1024, + ) + .unwrap(); + journal + .append(&"x".repeat(MAX_TRANSPORT_CHUNK_BYTES)) + .unwrap(); + journal + .append(&"y".repeat(MAX_TRANSPORT_CHUNK_BYTES)) + .unwrap(); + assert!(journal.previous_path().is_file()); + std::fs::remove_file(journal.current_path()).unwrap(); + + let replay = journal.read(0, MAX_TRANSPORT_CHUNK_BYTES as u64).expect( + "a committed previous segment is valid if rotation stopped before current creation", + ); + + assert_eq!(replay.frames.len(), 4); + assert_eq!(replay.frames.first().unwrap().seq_start, 1); + assert_eq!(replay.frames.last().unwrap().seq_end, 4); + assert_eq!( + replay + .frames + .iter() + .map(|frame| frame.data.as_str()) + .collect::<String>(), + "x".repeat(MAX_TRANSPORT_CHUNK_BYTES) + ); + } + #[test] fn cursor_from_a_previous_host_cannot_hide_a_new_epochs_output() { let dir = tempfile::tempdir().unwrap(); @@ -584,7 +742,7 @@ mod tests { let payload = "x".repeat(MAX_TRANSPORT_CHUNK_BYTES); let mut last_seq = 0; for _ in 0..96 { - last_seq = journal.append(&payload).unwrap()[0].seq_end; + last_seq = journal.append(&payload).unwrap().last().unwrap().seq_end; } let spool_bytes = journal.spool_bytes(); diff --git a/crates/freshell-supervisor/src/retained_output.rs b/crates/freshell-supervisor/src/retained_output.rs index 2f22f41ec..d9b5de9ed 100644 --- a/crates/freshell-supervisor/src/retained_output.rs +++ b/crates/freshell-supervisor/src/retained_output.rs @@ -220,7 +220,7 @@ fn read_indexed( let mut segments = Vec::with_capacity(2); for (spool_name, index_name, required) in [ (SPOOL_PREVIOUS_FILE, SPOOL_PREVIOUS_INDEX_FILE, false), - (SPOOL_CURRENT_FILE, SPOOL_CURRENT_INDEX_FILE, true), + (SPOOL_CURRENT_FILE, SPOOL_CURRENT_INDEX_FILE, false), ] { let spool_path = source.runtime_dir.join(spool_name); let file = match File::open(&spool_path) { @@ -307,6 +307,9 @@ fn read_indexed( torn_tail: indexed.torn_tail, }); } + if segments.is_empty() { + return Err("retained output spool unavailable".into()); + } read_indexed_spool(descriptor.clone(), segments, after, budget, completion) } @@ -647,6 +650,76 @@ mod tests { ); } + #[test] + fn supervisor_reads_previous_only_spool_after_rotation_interruption() { + let directory = tempfile::tempdir().unwrap(); + let incarnation_id = IncarnationId::new(); + let terminal_id = "retained-terminal"; + let stream_epoch = "launch-prefix-hostboot-previous-only"; + let descriptor = TerminalSpoolDescriptor { + schema_version: 1, + incarnation_id, + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + segment_limit_bytes: 128 * 1024, + }; + let source = RetainedOutputSource { + incarnation_id: descriptor.incarnation_id.clone(), + runtime_dir: directory.path().to_path_buf(), + terminal: serde_json::from_value::<TerminalLaunchSpec>(serde_json::json!({ + "terminalId":terminal_id,"streamId":"launch-prefix","mode":"shell", + "program":"/bin/sh","cwd":"/workspace","runAsUid":65534, + "runAsGid":0,"cols":120,"rows":30,"projectKey":"fixture", + "workspacePath":"/workspace" + })) + .unwrap(), + native_session_id: None, + }; + let frame = freshell_runtime_protocol::RuntimeOutputFrame { + terminal_id: terminal_id.into(), + stream_epoch: stream_epoch.into(), + seq_start: 7, + seq_end: 7, + data: "committed before interrupted rotation".into(), + }; + let mut previous = File::create(directory.path().join(SPOOL_PREVIOUS_FILE)).unwrap(); + serde_json::to_writer(&mut previous, &TerminalSpoolFrame { frame }).unwrap(); + previous.write_all(b"\n").unwrap(); + previous.sync_all().unwrap(); + + let replay = read_indexed( + &source, + &descriptor, + 0, + MAX_OUTPUT_FRAME_BYTES as u64, + None, + false, + ) + .expect("previous-only spool is recoverable when current creation was interrupted"); + + assert_eq!(replay.batch.frames.len(), 1); + assert_eq!(replay.batch.frames[0].seq_start, 7); + assert_eq!( + replay.batch.frames[0].data, + "committed before interrupted rotation" + ); + assert!(directory.path().join(SPOOL_PREVIOUS_INDEX_FILE).is_file()); + + std::fs::remove_file(directory.path().join(SPOOL_PREVIOUS_FILE)).unwrap(); + assert!( + read_indexed( + &source, + &descriptor, + 0, + MAX_OUTPUT_FRAME_BYTES as u64, + None, + false + ) + .is_err(), + "missing both spool segments must not masquerade as empty retained output" + ); + } + #[test] fn legacy_supervisor_recovers_new_epoch_output_after_indexing_old_epoch_spool() { let directory = tempfile::tempdir().unwrap(); diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 9dccef4b4..8fbf60be1 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -37,6 +37,18 @@ export type ManagedProviderEnvironmentTargetMode = 'managed-environment' | 'code const delay = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)) const digest = (bytes: Buffer | string) => createHash('sha256').update(bytes).digest('hex') +export function committedAcceptanceSha(repoRoot: string): string { + const candidateSha = execFileSync('git', ['rev-parse', 'HEAD'], { + cwd: repoRoot, encoding: 'utf8', stdio: ['ignore', 'pipe', 'inherit'], timeout: 5_000, + }).trim() + assert.match(candidateSha, /^[a-f0-9]{40}$/, 'Cloud acceptance requires a known worktree commit') + const dirty = execFileSync('git', ['status', '--porcelain'], { + cwd: repoRoot, encoding: 'utf8', stdio: ['ignore', 'pipe', 'inherit'], timeout: 5_000, + }).trim() + assert.equal(dirty, '', 'Cloud acceptance must build an exact committed candidate') + return candidateSha +} + export const checkpointTcpForwarderSource = String.raw` const net = require('node:net') const listenHost = process.env.FRESHELL_CHECKPOINT_FORWARDER_HOST @@ -1070,10 +1082,7 @@ export async function runCodeCheckpointTarget() { assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Checkpoint acceptance requires the robot identity') assert.equal(process.env.GCLOUD_ROBOT_ACCOUNT, 'gcloud-robot@misc-puttering-project.iam.gserviceaccount.com') const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') - const candidateSha = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: repoRoot, encoding: 'utf8', timeout: 5_000 }).trim() - assert.match(candidateSha, /^[a-f0-9]{40}$/) - const dirty = execFileSync('git', ['status', '--porcelain'], { cwd: repoRoot, encoding: 'utf8', timeout: 5_000 }).trim() - assert.equal(dirty, '', 'Checkpoint Cloud acceptance must build an exact committed candidate') + const candidateSha = committedAcceptanceSha(repoRoot) execFileSync('mise', ['exec', 'rust@1.96', '--', 'cargo', 'build', '-p', 'freshell-server', '--features', 'managed-runtime-v1', '--locked'], { cwd: repoRoot, env: { ...process.env, FRESHELL_BUILD_COMMIT: candidateSha }, stdio: 'inherit', @@ -1442,12 +1451,7 @@ export async function runManagedEnvironmentTarget() { assert.equal(process.env.GCLOUD_ROBOT_REQUIRE, '1', 'Managed acceptance requires the robot identity') assert.equal(process.env.GCLOUD_ROBOT_ACCOUNT, 'gcloud-robot@misc-puttering-project.iam.gserviceaccount.com') const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') - const candidateSha = execFileSync('git', ['rev-parse', 'HEAD'], { - cwd: repoRoot, - encoding: 'utf8', - stdio: ['ignore', 'pipe', 'inherit'], - timeout: 5_000, - }).trim() + const candidateSha = committedAcceptanceSha(repoRoot) buildManagedEnvironmentClient(repoRoot, candidateSha) const root = await fs.mkdtemp(path.join(os.tmpdir(), 'frs-managed-environment-')) const grants = fakeOnecliGrants(['opencode']) @@ -1813,7 +1817,6 @@ export async function runManagedEnvironmentTarget() { const replacementChild = await poll('replacement Bun child', async () => readJson(path.join(selected.workspace, 'healthy/.managed-fixture/child-receipt.json'))) const replacementCwdAliasMcp = await poll('replacement OpenCode starts the workspace-CWD MCP alias', async () => readJson(cwdAliasReceiptPath)) - const replacementStage = await stageProof(rig, replacement, nonce) assert.equal(replacement.nativeSessionId, original.nativeSessionId) assert.equal(replacementChild.bun, true) assert.equal(replacementChild.terminalId, replacement.terminalId) @@ -1843,6 +1846,10 @@ export async function runManagedEnvironmentTarget() { exists: true, isNode: true, envFileOperand: true, fixtureArguments: true, cwdMatches: true, processStartTicksMatches: true, }, 'the .env MCP receipt must belong to a live process in the replacement container') + // Probe the receipt as soon as it appears. The provider-stage walk below can + // be sizeable, and the MCP child is owned by OpenCode; delaying this check + // until after that unrelated scan can observe a child that has since exited. + const replacementStage = await stageProof(rig, replacement, nonce) proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, child: replacementChild, workspaceCwdAliasMcp: { ...replacementCwdAliasMcp, diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 0436ce3d9..dfcc94b51 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -10,6 +10,7 @@ import { } from '../../../../scripts/testing/managed-environment-create.js' import { checkpointTcpForwarderSource, + committedAcceptanceSha, directHomeAppendMcpServerSource, externalTargetRead, managedHomeAppendFixturePaths, @@ -32,6 +33,24 @@ async function listenOnLoopback(server: ReturnType<typeof createServer>): Promis } describe('managed provider environment target mode', () => { + it('rejects acceptance evidence from a dirty worktree', async () => { + const repository = await fs.mkdtemp(path.join(process.env.TMPDIR ?? '/tmp', 'frs-acceptance-provenance-')) + try { + await fs.writeFile(path.join(repository, 'source.txt'), 'committed candidate\n') + execFileSync('git', ['init', '-q'], { cwd: repository }) + execFileSync('git', ['add', 'source.txt'], { cwd: repository }) + execFileSync('git', ['-c', 'user.name=Acceptance Test', '-c', 'user.email=acceptance@example.invalid', + 'commit', '-q', '-m', 'candidate'], { cwd: repository }) + + const candidate = committedAcceptanceSha(repository) + expect(candidate).toMatch(/^[a-f0-9]{40}$/) + await fs.writeFile(path.join(repository, 'uncommitted.txt'), 'not part of HEAD\n') + expect(() => committedAcceptanceSha(repository)).toThrow(/exact committed candidate/) + } finally { + await fs.rm(repository, { recursive: true, force: true }) + } + }) + it('records typed provider preparation causes without persisting paths', () => { const incidents: unknown[] = [] const owner = { From 917570a45a46b1db52dd0d5dc9694dac0d0e2606 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 12:14:52 -0700 Subject: [PATCH 484/493] test: tolerate loaded Claude candidate fixtures --- .../src/backend/claude_source_observation_tests.rs | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs index 105dd847e..cdf6ffc86 100644 --- a/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs +++ b/crates/freshell-supervisor/src/backend/claude_source_observation_tests.rs @@ -1169,13 +1169,18 @@ async fn qualify_candidate_helper_transport( } }); let backend = DockerEngineBackend::new(owned.path().join("docker.sock")); + let budget = if mode == "projection-timeout" { + Duration::from_millis(250) + } else { + Duration::from_secs(3) + }; let result = backend .read_claude_candidate_helper( owner, request, stage, "synthetic-source-run", - Duration::from_millis(250), + budget, retained_name.as_ref().map(|_| prior_stage.as_path()), ) .await; From 2f1aa8b8073e4c556e117382e0cda7c2c1d308d4 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 12:24:20 -0700 Subject: [PATCH 485/493] test: assert checkpoint cooldown outcome code --- crates/freshell-server/src/checkpoints.rs | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index fcece9976..63bf1dd3d 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -9120,6 +9120,10 @@ mod tests { outcome.outcome.status, FreshAgentCheckpointStatus::Unavailable ); + assert_eq!( + outcome.outcome.code.as_deref(), + Some("capture_timeout_cooldown") + ); let events = events.lock().unwrap(); let failure = events.iter().find(|event| { event From c0ac484321a0b89124e1043a4905830cf23aaa9d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 13:10:41 -0700 Subject: [PATCH 486/493] test: diagnose managed replacement MCP liveness --- .../managed-provider-environment-target.ts | 120 ++++++++++++++---- ...anaged-provider-environment-target.test.ts | 19 ++- 2 files changed, 114 insertions(+), 25 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 8fbf60be1..5d02e99ba 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -214,20 +214,42 @@ const path = require('node:path') const [pidText, receiptPath, expectedNonce, expectedCwd, expectedStartTicks] = process.argv.slice(1) const procPath = '/proc/' + Number(pidText) const result = { exists: false, isNode: false, envFileOperand: false, fixtureArguments: false, cwdMatches: false, - processStartTicksMatches: false } -try { - result.exists = fs.existsSync(procPath + '/stat') - const command = fs.readFileSync(procPath + '/cmdline', 'utf8').split(String.fromCharCode(0)).filter(Boolean) - const stat = fs.readFileSync(procPath + '/stat', 'utf8') - const startTicks = stat.slice(stat.lastIndexOf(')') + 2).trim().split(/\s+/)[19] + processStartTicksMatches: false, processState: null, processName: null, parentPid: null, executableName: null, + readErrors: [] } +const readProc = (field, read) => { + try { return read() } + catch (error) { + result.readErrors.push({ field, code: error && typeof error === 'object' && 'code' in error ? String(error.code) : 'UNKNOWN' }) + return null + } +} +result.exists = fs.existsSync(procPath + '/stat') +const stat = readProc('stat', () => fs.readFileSync(procPath + '/stat', 'utf8')) +if (stat !== null) { + const open = stat.indexOf('('), close = stat.lastIndexOf(')') + const fields = close >= 0 ? stat.slice(close + 2).trim().split(/\s+/) : [] + result.processName = open >= 0 && close > open ? stat.slice(open + 1, close) : null + result.processState = fields[0] ?? null + result.parentPid = Number.isSafeInteger(Number(fields[1])) ? Number(fields[1]) : null + result.processStartTicksMatches = fields[19] === expectedStartTicks +} +const commandLine = readProc('cmdline', () => fs.readFileSync(procPath + '/cmdline', 'utf8')) +if (commandLine !== null) { + const command = commandLine.split(String.fromCharCode(0)).filter(Boolean) const scriptIndex = command.indexOf('server.mjs') - result.isNode = path.basename(fs.readlinkSync(procPath + '/exe').replace(/ \(deleted\)$/, '')) === 'node' result.envFileOperand = command.includes('--env-file=.env') result.fixtureArguments = scriptIndex >= 0 && command[scriptIndex + 1] === receiptPath && command[scriptIndex + 2] === expectedNonce - result.cwdMatches = fs.readlinkSync(procPath + '/cwd') === expectedCwd - result.processStartTicksMatches = startTicks === expectedStartTicks -} catch { /* Emit false checks so the host-side assertion reports the failed evidence. */ } +} +const executable = readProc('exe', () => fs.readlinkSync(procPath + '/exe')) +if (executable !== null) { + result.executableName = path.basename(executable.replace(/ \(deleted\)$/, '')) + result.isNode = result.executableName === 'node' +} +const cwd = readProc('cwd', () => fs.readlinkSync(procPath + '/cwd')) +if (cwd !== null) { + result.cwdMatches = cwd === expectedCwd +} process.stdout.write(JSON.stringify(result)) ` } @@ -523,6 +545,7 @@ export default {id:'managed-environment-tui',async tui(api,options){ const child=Bun.spawn([command.pathname],{stdout:'pipe',stderr:'pipe'});const output=JSON.parse(await new Response(child.stdout).text());if(await child.exited!==0)throw new Error('selected executable failed'); const configuration=await api.client.config.get({directory:cwd});if(!configuration.data?.agent)throw new Error('resolved provider configuration was not available');const sessionFlag=process.argv.findIndex(value=>value==='--session'||value==='-s');const resumed=sessionFlag>=0?process.argv[sessionFlag+1]:undefined;const result=resumed?await api.client.session.get({directory:cwd,sessionID:resumed}):await api.client.session.create({directory:cwd,title:'Managed fixture'});if(!result.data?.id||(resumed&&result.data.id!==resumed))throw new Error('native session creation or resume failed');api.route.navigate('session',{sessionID:result.data.id}); let mcpStatuses={},wrapperProbe={};if(cwd.endsWith('/preflight')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}const wrapperPath=path.join(cwd,'.managed-fixture/mcp-cwd-alias/start');wrapperProbe.paths=[cwd,path.join(cwd,'.managed-fixture'),path.dirname(wrapperPath),wrapperPath].map(candidate=>{const relative=path.relative(cwd,candidate)||'.';try{const metadata=fs.statSync(candidate);fs.accessSync(candidate,fs.constants.X_OK);return{path:relative,mode:(metadata.mode&511).toString(8).padStart(4,'0'),uid:metadata.uid,gid:metadata.gid,searchAccess:'ok'}}catch(error){return{path:relative,code:error?.code??'unknown',message:String(error).slice(0,120)}}});try{const mount=fs.readFileSync('/proc/self/mountinfo','utf8').split('\\n').flatMap(line=>{const [left,right]=line.split(' - ');if(!right)return[];const fields=left.split(' '),mountpoint=fields[4]??'';if(wrapperPath!==mountpoint&&!wrapperPath.startsWith(mountpoint==='/'?'/':mountpoint+'/'))return[];const detail=right.split(' ');return[{mountRoot:fields[3],mountpoint,mountOptions:fields[5],filesystem:detail[0],source:detail[1],superOptions:detail[2]}]}).sort((a,b)=>b.mountpoint.length-a.mountpoint.length)[0];wrapperProbe.mount=mount??null}catch(error){wrapperProbe.mountProbeError=String(error).slice(0,160)}} + if(cwd.endsWith('/healthy')){await new Promise(resolve=>setTimeout(resolve,3000));try{const response=await api.client.mcp.status({directory:cwd});mcpStatuses=Object.fromEntries(Object.entries(response.data??{}).map(([name,status])=>[name,{status:status?.status??'unknown',...(status?.status==='failed'?{error:String(status.error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}:{})}]))}catch(error){mcpStatuses={queryError:String(error).replaceAll(options.expectedNonce,'[fixture-nonce]').slice(0,240)}}} const receipt={...options,terminalId:process.env.FRESHELL_TERMINAL_ID,nativeSessionId:result.data.id,bun:typeof Bun==='object',bunVersion:Bun.version,bunExecutable:process.execPath,opencodeVersion:api.app.version,model:configuration.data.model,dependency,resource,absoluteReference:configuration.data.agent.fixtureAbsolute.prompt,relativeReference:configuration.data.agent.fixtureRelative.prompt,executableMode:(fs.statSync(command).mode&511).toString(8).padStart(4,'0'),mcpStatuses,wrapperProbe,pid:process.pid,createdAt:new Date().toISOString()}; fs.writeFileSync(path.join(cwd,'.managed-fixture/child-receipt.json'),JSON.stringify(receipt));const marker='MANAGED_ENV_OUTPUT '+output.resource;process.stdout.write('\\n'+marker+'\\n');api.ui.toast.show({message:marker,duration:180000}); }};`) @@ -1722,6 +1745,8 @@ export async function runManagedEnvironmentTarget() { assert.equal(child.runId, fixture.runId) assert.equal(child.bun, true) assert.equal(child.opencodeVersion, '1.18.21') + assert.equal(child.mcpStatuses?.workspace_cwd_alias_fixture?.status, 'connected', + 'OpenCode must report the external-CWD MCP connected while its healthy TUI is alive') assert.equal(child.model, managedEnvironmentFixtureModel) assert.match(child.bunExecutable, /opencode/) assert.equal(child.executableMode, '0711') @@ -1825,6 +1850,8 @@ export async function runManagedEnvironmentTarget() { assert.equal(replacementChild.runId, rig.runtime.runId) assert.equal(replacementChild.expectedNonce, nonce) assert.equal(replacementChild.executableMode, '0711') + assert.equal(replacementChild.mcpStatuses?.workspace_cwd_alias_fixture?.status, 'connected', + 'replacement OpenCode must report the external-CWD MCP connected') assert.equal(replacementCwdAliasMcp.expectedNonce, nonce) assert.equal(replacementCwdAliasMcp.helper, nonce) assert.equal(replacementCwdAliasMcp.terminalId, replacement.terminalId) @@ -1837,21 +1864,70 @@ export async function runManagedEnvironmentTarget() { assert(replacementCwdAliasMcp.scriptPath.endsWith('/cwd-alias-project/server.mjs')) assert.equal(await fs.readFile(`${cwdAliasReceiptPath}.started`, 'utf8'), 'wrapper-started') assert(replacement.containerId) - const replacementMcpProcessProbe = JSON.parse(rig.runtime.execOwnedContainerExact(replacement.containerId, [ - 'node', '-e', workspaceCwdAliasReplacementProcessProbeSource(), - String(replacementCwdAliasMcp.pid), cwdAliasReceiptPath, nonce, selected.cwdAliasProject, - replacementCwdAliasMcp.processStartTicks, - ])) - assert.deepEqual(replacementMcpProcessProbe, { - exists: true, isNode: true, envFileOperand: true, fixtureArguments: true, cwdMatches: true, - processStartTicksMatches: true, - }, 'the .env MCP receipt must belong to a live process in the replacement container') - // Probe the receipt as soon as it appears. The provider-stage walk below can - // be sizeable, and the MCP child is owned by OpenCode; delaying this check - // until after that unrelated scan can observe a child that has since exited. + const processProbeObservations: Array<{ attempt: number; result: Record<string, unknown> }> = [] + let consecutiveLiveIdentityMatches = 0 + const liveIdentityFields = ['exists', 'isNode', 'envFileOperand', 'fixtureArguments', 'cwdMatches', + 'processStartTicksMatches'] + for (let attempt = 0; attempt < 12; attempt++) { + let result: Record<string, unknown> + try { + result = JSON.parse(rig.runtime.execOwnedContainerExact(replacement.containerId, [ + 'node', '-e', workspaceCwdAliasReplacementProcessProbeSource(), + String(replacementCwdAliasMcp.pid), cwdAliasReceiptPath, nonce, selected.cwdAliasProject, + replacementCwdAliasMcp.processStartTicks, + ])) as Record<string, unknown> + } catch (error) { + const probeError = error as NodeJS.ErrnoException + result = { probeExecutionError: probeError.code ?? probeError.name ?? 'UNKNOWN' } + } + processProbeObservations.push({ attempt, result }) + const matchesReceipt = liveIdentityFields.every(field => result[field] === true) + && result.processName === 'node' && result.executableName === 'node' + && result.processState !== 'Z' && Array.isArray(result.readErrors) && result.readErrors.length === 0 + consecutiveLiveIdentityMatches = matchesReceipt ? consecutiveLiveIdentityMatches + 1 : 0 + if (consecutiveLiveIdentityMatches >= 2) break + if (attempt < 11) await delay(250) + } + const replacementTerminalRow = (await terminalDirectory(rig)).find(row => row.terminalId === replacement.terminalId) + const terminalStatus = { + status: typeof replacementTerminalRow?.status === 'string' ? replacementTerminalRow.status : 'missing', + runtimeStatus: typeof replacementTerminalRow?.runtimeStatus === 'string' ? replacementTerminalRow.runtimeStatus : null, + hasClients: replacementTerminalRow?.hasClients === true, + } + const replacementView = (await rig.inventory()).find(row => row.incarnationId === replacement.incarnationId) + const runtimeStatus = replacementView ? { + launchState: replacementView.launchState, + desiredState: replacementView.desiredState ?? null, + recoveryState: replacementView.recoveryState ?? null, + durabilityState: replacementView.durabilityState ?? null, + allocationState: replacementView.allocationState ?? null, + } : null + const mcpStatusAtTuiStartup = replacementChild.mcpStatuses?.workspace_cwd_alias_fixture?.status ?? 'missing' + const replacementMcpProcessProbe = processProbeObservations.at(-1)?.result ?? { probeExecutionError: 'NO_OBSERVATION' } + const liveProcessProved = consecutiveLiveIdentityMatches >= 2 + const terminalStillRunning = terminalStatus.status === 'running' + if (!liveProcessProved || !terminalStillRunning || mcpStatusAtTuiStartup !== 'connected') { + let processTable: string | 'UNAVAILABLE' + try { + processTable = rig.runtime.topOwnedContainerExact(replacement.containerId, ['-eo', 'pid,ppid,stat,comm']) + .split('\n').slice(0, 80).join('\n') + } catch { processTable = 'UNAVAILABLE' } + rig.runtime.writeIncident('managed-environment-replacement-mcp-process-failure', { + severity: 'error', event: 'managed_environment.replacement_mcp_process_failed', + replacement: { terminalId: replacement.terminalId, incarnationId: replacement.incarnationId, + containerId: replacement.containerId, soulId: replacement.soulId }, + mcpStatusAtTuiStartup, terminalStatus, runtimeStatus, consecutiveLiveIdentityMatches, + probeObservations: processProbeObservations, processTable, + }) + throw new Error(`replacement MCP liveness proof failed: ${JSON.stringify({ + mcpStatusAtTuiStartup, terminalStatus, runtimeStatus, consecutiveLiveIdentityMatches, + latestProbe: replacementMcpProcessProbe, + })}`) + } const replacementStage = await stageProof(rig, replacement, nonce) proof.nativeReplacement = { priorIncarnationId: original.incarnationId, incarnationId: replacement.incarnationId, soulId: replacement.soulId, nativeSessionId: replacement.nativeSessionId, stage: replacementStage, + terminalStatus, mcpStatusAtTuiStartup, consecutiveLiveIdentityMatches, child: replacementChild, workspaceCwdAliasMcp: { ...replacementCwdAliasMcp, verifiedInsideContainerId: replacement.containerId, verifiedIncarnationId: replacement.incarnationId, processProbe: replacementMcpProcessProbe } } diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index dfcc94b51..64f8c0262 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -294,11 +294,24 @@ setInterval(() => {}, 1000) '-e', workspaceCwdAliasReplacementProcessProbeSource(), String(receipt.pid), receiptPath, nonce, directory, startTicks, ], { cwd: directory, encoding: 'utf8' })) - expect(runProbe(receipt.processStartTicks)).toEqual({ + const liveProbe = runProbe(receipt.processStartTicks) + expect(liveProbe).toMatchObject({ exists: true, isNode: true, envFileOperand: true, fixtureArguments: true, cwdMatches: true, - processStartTicksMatches: true, + processStartTicksMatches: true, processState: 'S', processName: 'node', executableName: 'node', + parentPid: expect.any(Number), readErrors: [], }) - expect(runProbe(`${receipt.processStartTicks}-stale`).processStartTicksMatches).toBe(false) + const staleIdentityProbe = runProbe(`${receipt.processStartTicks}-stale`) + expect(staleIdentityProbe.processStartTicksMatches).toBe(false) + expect(staleIdentityProbe.processName).toBe('node') + + const missingProbe = JSON.parse(execFileSync(process.execPath, [ + '-e', workspaceCwdAliasReplacementProcessProbeSource(), String(Number.MAX_SAFE_INTEGER), + receiptPath, nonce, directory, receipt.processStartTicks, + ], { cwd: directory, encoding: 'utf8' })) + expect(missingProbe).toMatchObject({ exists: false, processState: null, processName: null, executableName: null }) + expect(missingProbe.readErrors).toContainEqual({ field: 'stat', code: 'ENOENT' }) + expect(JSON.stringify(missingProbe)).not.toContain(nonce) + expect(JSON.stringify(missingProbe)).not.toContain(receiptPath) } finally { if (child && child.exitCode === null && child.signalCode === null) { const closed = new Promise<void>(resolve => child!.once('close', () => resolve())) From e72e02e29508487dffb1987acb352bce0e17f8fc Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 13:27:49 -0700 Subject: [PATCH 487/493] test: inspect managed MCP as provider user --- scripts/testing/managed-provider-environment-target.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index 5d02e99ba..a5c25b0ed 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -1871,7 +1871,7 @@ export async function runManagedEnvironmentTarget() { for (let attempt = 0; attempt < 12; attempt++) { let result: Record<string, unknown> try { - result = JSON.parse(rig.runtime.execOwnedContainerExact(replacement.containerId, [ + result = JSON.parse(rig.runtime.execOwnedContainerAsExact(replacement.containerId, '65534:0', [ 'node', '-e', workspaceCwdAliasReplacementProcessProbeSource(), String(replacementCwdAliasMcp.pid), cwdAliasReceiptPath, nonce, selected.cwdAliasProject, replacementCwdAliasMcp.processStartTicks, From 15598fbfa02804046f2171b7fd01e7abfd369618 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 14:19:59 -0700 Subject: [PATCH 488/493] fix managed recovery and checkpoint ownership races --- crates/freshell-freshagent/src/claude.rs | 162 +++++++++++++ crates/freshell-freshagent/src/codex.rs | 98 ++++++++ crates/freshell-freshagent/src/opencode_ws.rs | 223 +++++++++++++++--- .../src/provider_preparation_tests.rs | 72 +++++- crates/freshell-supervisor/src/recovery.rs | 27 ++- crates/freshell-supervisor/src/registry.rs | 178 +++++++++++++- 6 files changed, 719 insertions(+), 41 deletions(-) diff --git a/crates/freshell-freshagent/src/claude.rs b/crates/freshell-freshagent/src/claude.rs index 970fbc34a..79c103dbc 100644 --- a/crates/freshell-freshagent/src/claude.rs +++ b/crates/freshell-freshagent/src/claude.rs @@ -4321,6 +4321,19 @@ impl FreshClaudeState { let session_id = msg.session_id.clone(); let session_type = session_type_str(msg.session_type); + // The checkpoint barrier can park this send while the captured + // workspace is saved. Parse and retain the pane's ownership fence so + // that a close, handoff, or reattach during that wait cannot redirect + // the delayed prompt to a different owner of the same session id. + let send_fence = + match crate::ownership_lane::wire_fence(msg.observed_epoch, msg.observed_generation) { + Ok(fence) => fence, + Err(err) => { + self.send_error(&request_id, err.code(), err.message()); + return; + } + }; + // Task 10b: a durable claude UUID resolves through `cli_index` to the live // session -- no more SESSION_NOT_FOUND for a durable id the index maps. // @@ -4368,6 +4381,35 @@ impl FreshClaudeState { // resolved before the teardown window serializes identically to one that // parked through it. let _turn = turn_lock.lock().await; + // Arm the coordinator guard only after the send has waited behind any + // earlier turn. A lifecycle transition during that wait advances the + // generation, so the captured fence is checked against the owner that + // will actually receive this prompt. Checkpointed sends always require + // the fence when the coordinator is wired; ordinary legacy sends keep + // their existing unfenced behavior, while any supplied fence is honored. + let ownership_guard = if msg.checkpoint_intent.is_some() || send_fence.is_some() { + let owner_key = self.resolve_ownership_key(&session_id); + match crate::ownership_lane::arm_reclaimless_op_guard( + &self.ownership, + PROVIDER, + &owner_key, + &format!("send-{}", uuid::Uuid::new_v4()), + send_fence, + "freshclaude/send", + ) { + crate::ownership_lane::LaneOpGuard::Armed(guard) => Some(guard), + crate::ownership_lane::LaneOpGuard::Unwired => None, + crate::ownership_lane::LaneOpGuard::Refused { message } => { + tracing::warn!(target: "freshell_freshagent::claude", + session_id = %session_id, request_id = ?request_id, + "fresh_agent_send_refused: ownership guard refused"); + self.send_error(&request_id, "SESSION_RESERVED", &message); + return; + } + } + } else { + None + }; if let Some(settings) = msg.settings.as_ref() { // kata z7j7: sandbox has NO claude-side concept — when it arrives on // a send, log observably (id + boolean flag only, never the prompt @@ -4501,6 +4543,7 @@ impl FreshClaudeState { return; } drop(guard); + drop(ownership_guard); // Unified agent names (Task 4): the shared accepted-input callback — // the send reached the sidecar, so the user input is ACCEPTED. The @@ -11596,6 +11639,125 @@ rl.on('line', (line) => { assert!(checkpoint_attempt_for_send(&send_msg("legacy", "prompt")).is_none()); } + #[tokio::test(flavor = "multi_thread")] + async fn checkpoint_delayed_send_with_stale_owner_fence_is_refused_before_dispatch() { + let (mut st, mut rx) = state_with_bus(); + let session_id = "claude-send-stale-owner"; + let registry = Arc::new(freshell_ownership::RuntimeOwnershipRegistry::new()); + st.set_ownership(registry.clone()); + insert_fake_claude_session(&st, session_id).await; + + let mut first_ticket = match crate::ownership_lane::begin_lane_claim( + &st.ownership, + PROVIDER, + session_id, + "test-send-owner-first", + None, + "test", + crate::session_lease::now_epoch_ms(), + ) { + crate::ownership_lane::LaneClaim::Granted(ticket) => Some(ticket), + _ => panic!("fixture: first send owner claim must be granted"), + }; + assert!(crate::ownership_lane::commit_lane_claim( + &st.ownership, + &st.ownership_stamps, + Some(&st.broadcast_tx), + PROVIDER, + session_id, + &mut first_ticket, + session_id, + None, + ) + .is_ok()); + let old = registry.observe(PROVIDER, session_id); + let old_generation = old.generation; + let old_owner = match old.state { + freshell_ownership::OwnershipState::Live { owner, .. } => owner, + other => panic!("fixture: first owner must be live: {other:?}"), + }; + let old_operation = old_owner + .ownership_id + .clone() + .expect("the committed owner has an operation id"); + registry.release( + PROVIDER, + session_id, + &freshell_ownership::ReleaseClaim { + operation_id: old_operation, + generation: old_generation, + runtime: Some(old_owner), + }, + "test", + ); + + let mut replacement_ticket = match crate::ownership_lane::begin_lane_claim( + &st.ownership, + PROVIDER, + session_id, + "test-send-owner-replacement", + None, + "test", + crate::session_lease::now_epoch_ms(), + ) { + crate::ownership_lane::LaneClaim::Granted(ticket) => Some(ticket), + _ => panic!("fixture: replacement send owner claim must be granted"), + }; + assert!(crate::ownership_lane::commit_lane_claim( + &st.ownership, + &st.ownership_stamps, + Some(&st.broadcast_tx), + PROVIDER, + session_id, + &mut replacement_ticket, + session_id, + None, + ) + .is_ok()); + let current = registry.observe(PROVIDER, session_id); + assert!(current.generation > old_generation); + + let mut send = send_msg(session_id, "delayed checkpoint prompt"); + send.request_id = Some("stale-checkpoint-send".into()); + send.checkpoint_intent = Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "stale-checkpoint-attempt".into(), + workspace: None, + }); + send.observed_epoch = Some(old.epoch); + send.observed_generation = Some(old_generation); + st.handle_send(send).await; + + let mut frames: Vec<Value> = Vec::new(); + while let Ok(raw) = rx.try_recv() { + if let Ok(frame) = serde_json::from_str(&raw) { + frames.push(frame); + } + } + assert!( + frames.iter().any(|frame| { + frame.to_string().contains("SESSION_RESERVED") + && frame.to_string().contains("stale") + }), + "the delayed send reports the stale owner fence: {frames:?}" + ); + let sessions = st.sessions.lock().await; + let session = sessions + .get(session_id) + .expect("the old session stays mapped"); + assert!( + !session.in_turn.load(std::sync::atomic::Ordering::SeqCst), + "a stale delayed prompt never arms a turn or reaches the sidecar" + ); + assert!( + !session + .turn_tracker + .lock() + .expect("turn tracker mutex") + .busy(), + "a refused stale prompt leaves the turn tracker idle" + ); + } + /// kata z7j7 (claude scope honesty): a settings-bearing send whose payload /// includes `sandbox` — a knob claude has NO wire concept for (the configure /// frame carries only model/effort/permissionMode/cwd) — must surface the diff --git a/crates/freshell-freshagent/src/codex.rs b/crates/freshell-freshagent/src/codex.rs index 0f5f256b5..f575c92eb 100644 --- a/crates/freshell-freshagent/src/codex.rs +++ b/crates/freshell-freshagent/src/codex.rs @@ -3449,6 +3449,36 @@ impl FreshCodexState { // ("send waits, rollback wins, then destroys"). let _turn = turn_lock.lock().await; + // The checkpoint workspace barrier can park a send outside this + // handler while ownership changes. Recheck the captured wire fence + // after the turn lock so a delayed send cannot reach a replacement + // owner of the same thread id. Keep the coordinator guard through the + // turn/start request; ordinary legacy sends without a fence retain + // their existing behavior. + let ownership_guard = if msg.checkpoint_intent.is_some() || send_fence.is_some() { + let owner_key = self.resolve_ownership_key(&session_id); + match crate::ownership_lane::arm_reclaimless_op_guard( + &self.ownership, + PROVIDER, + &owner_key, + &format!("send-{}", uuid::Uuid::new_v4()), + send_fence, + "freshcodex/send", + ) { + crate::ownership_lane::LaneOpGuard::Armed(guard) => Some(guard), + crate::ownership_lane::LaneOpGuard::Unwired => None, + crate::ownership_lane::LaneOpGuard::Refused { message } => { + tracing::warn!(target: "freshell_freshagent::codex", + session_id = %session_id, request_id = ?request_id, + "fresh_agent_send_refused: ownership guard refused"); + self.send_error(&request_id, "SESSION_RESERVED", &message); + return; + } + } + } else { + None + }; + // Read defaults after acquiring the turn lock: a queued send inherits the // last accepted settings, while this message's explicit choices win. let stored = { @@ -3594,6 +3624,7 @@ impl FreshCodexState { return; } }; + drop(ownership_guard); // Only persist choices once the provider has accepted them. Failed sends // keep the previous working settings for resume and retry. @@ -27960,6 +27991,73 @@ pub(crate) mod tests { } } + #[tokio::test(flavor = "multi_thread")] + async fn checkpoint_send_with_stale_fence_is_refused_for_an_already_running_codex_session() { + let _guard = ENV_LOCK.lock().await; + let (mut st, mut rx) = state_with_bus(); + let registry = Arc::new(freshell_ownership::RuntimeOwnershipRegistry::new()); + st.set_ownership(Arc::clone(®istry)); + configure_fake_codex_cmd("{}"); + let thread_id = create_real_fake_session(&st, &mut rx).await; + let before = registry.observe(PROVIDER, &thread_id); + let freshell_ownership::BeginOutcome::Granted { generation } = registry.begin_handoff( + PROVIDER, + &thread_id, + freshell_ownership::RuntimeOwnerKind::Terminal, + "op-checkpoint-send-generation-bump", + None, + "test", + freshell_ownership::now_epoch_ms(), + ) else { + panic!("the generation-bump handoff must grant") + }; + let _ = registry.fail( + PROVIDER, + &thread_id, + "op-checkpoint-send-generation-bump", + generation, + true, + ); + let after = registry.observe(PROVIDER, &thread_id); + assert!(after.generation > before.generation); + assert!(matches!( + after.state, + freshell_ownership::OwnershipState::Live { .. } + )); + while rx.try_recv().is_ok() {} + + st.handle_send(FreshAgentSend { + request_id: Some("req-stale-checkpoint-codex".into()), + provider: freshell_protocol::AgentProvider::Codex, + session_id: thread_id, + session_type: freshell_protocol::SessionType::Freshcodex, + text: "delayed checkpoint prompt".into(), + images: None, + cwd: None, + settings: None, + observed_epoch: Some(before.epoch), + observed_generation: Some(before.generation), + checkpoint_intent: Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "stale-codex-attempt".into(), + workspace: None, + }), + }) + .await; + + let mut saw_reserved = false; + let mut saw_accepted = false; + while let Ok(raw) = rx.try_recv() { + let frame: Value = serde_json::from_str(&raw).expect("event frame JSON"); + saw_reserved |= frame.to_string().contains("SESSION_RESERVED"); + saw_accepted |= frame.to_string().contains("freshAgent.send.accepted"); + } + assert!(saw_reserved, "the stale owner fence is refused typed"); + assert!( + !saw_accepted, + "the stale prompt is never accepted for dispatch" + ); + } + /// b8ke ext r21 F2: a reconnect-replayed stale-fence COMPACT arriving /// after a crash advanced the generation (the key VACATED — the fence-less /// claim's one accepted case pre-r21) is typed-refused (SESSION_RESERVED) diff --git a/crates/freshell-freshagent/src/opencode_ws.rs b/crates/freshell-freshagent/src/opencode_ws.rs index 819d02974..7d5069822 100644 --- a/crates/freshell-freshagent/src/opencode_ws.rs +++ b/crates/freshell-freshagent/src/opencode_ws.rs @@ -2134,6 +2134,39 @@ impl FreshOpencodeState { } }; + // The checkpoint workspace barrier can park this request before it + // reaches the provider lane. For an already-materialized session, arm + // against its captured owner pair only after this session lock is held, + // so an old send cannot be redirected to a reattached owner of the same + // durable id. A first send to a placeholder still uses the materialize + // claim below; a queued send releases this guard here and the drain + // re-arms it immediately before dispatch. + let mut checkpoint_send_guard = + if msg.checkpoint_intent.is_some() && session.real_session_id.is_some() { + let owner_key = self.resolve_ownership_key(&session_id); + match crate::ownership_lane::arm_reclaimless_op_guard( + &self.fresh_agent.ownership, + PROVIDER, + &owner_key, + &format!("send-{}", uuid::Uuid::new_v4()), + send_fence, + "freshopencode/send", + ) { + crate::ownership_lane::LaneOpGuard::Armed(guard) => Some(guard), + crate::ownership_lane::LaneOpGuard::Unwired => None, + crate::ownership_lane::LaneOpGuard::Refused { message } => { + drop(session); + tracing::warn!(target: "freshell_freshagent::opencode", + session_id = %session_id, request_id = ?request_id, + "fresh_agent_send_refused: checkpoint ownership guard refused"); + self.send_error(&request_id, "SESSION_RESERVED", &message); + return; + } + } + } else { + None + }; + // send-during-compact queue: a send arriving while a COMPACT is in // flight — including the compact's settling tail (the tail is // still the compact's FIFO turn; Task 3's settling-flag gate @@ -2159,6 +2192,7 @@ impl FreshOpencodeState { .real_session_id .clone() .unwrap_or_else(|| session.placeholder_id.clone()); + drop(checkpoint_send_guard.take()); self.broadcast_send_accepted(&real_id, &msg.request_id, &session.cwd); session.pending_sends.push_back(msg.clone()); tracing::warn!(target: "freshell_freshagent::opencode", @@ -2178,13 +2212,14 @@ impl FreshOpencodeState { return; } + let op_guard = checkpoint_send_guard.take(); self.send_locked( &session_arc, &mut session, msg, session_id, send_fence, - None, + op_guard, false, ) .await; @@ -2975,38 +3010,37 @@ impl FreshOpencodeState { // Drain-time fence re-validation (the user constraint): the // entry's captured pair is validated ATOMICALLY via the // lane's op guard — never a bare unlocked observe-then-act - // (the exact TOCTOU the guard's docs exist to close). FENCED - // entries only: an unfenced entry keeps the direct send + // (the exact TOCTOU the guard's docs exist to close). Fenced + // entries and checkpoint-intent entries are guarded; an ordinary unfenced entry keeps the direct send // path's parse-only tolerance (the queue re-enters the send // path; it is not stricter than the path it re-enters — the // no-laundering discipline binds the history-restructuring // lanes, not the append-a-turn send lane). - let op_guard = match send_fence { - None => None, // unfenced: direct-path tolerance - Some(fence) => { - let op_id = format!("send-drain-{}", uuid::Uuid::new_v4()); - match crate::ownership_lane::arm_reclaimless_op_guard( - &self.fresh_agent.ownership, - PROVIDER, - &real_id, - &op_id, - Some(fence), - "freshopencode/send-drain", - ) { - crate::ownership_lane::LaneOpGuard::Armed(guard) => Some(guard), - crate::ownership_lane::LaneOpGuard::Unwired => None, - crate::ownership_lane::LaneOpGuard::Refused { message } => { - session.pending_sends.pop_front(); - tracing::warn!(target: "freshell_freshagent::opencode", - session_id = %lookup_id, request_id = ?msg.request_id, - observed_epoch = fence.epoch, - observed_generation = fence.generation, - "fresh_agent_send_drain_refused: ownership guard refused"); - self.send_error(&msg.request_id, "SESSION_RESERVED", &message); - continue; - } + let op_guard = if send_fence.is_some() || msg.checkpoint_intent.is_some() { + let op_id = format!("send-drain-{}", uuid::Uuid::new_v4()); + match crate::ownership_lane::arm_reclaimless_op_guard( + &self.fresh_agent.ownership, + PROVIDER, + &real_id, + &op_id, + send_fence, + "freshopencode/send-drain", + ) { + crate::ownership_lane::LaneOpGuard::Armed(guard) => Some(guard), + crate::ownership_lane::LaneOpGuard::Unwired => None, + crate::ownership_lane::LaneOpGuard::Refused { message } => { + session.pending_sends.pop_front(); + tracing::warn!(target: "freshell_freshagent::opencode", + session_id = %lookup_id, request_id = ?msg.request_id, + observed_epoch = send_fence.map(|fence| fence.epoch), + observed_generation = send_fence.map(|fence| fence.generation), + "fresh_agent_send_drain_refused: ownership guard refused"); + self.send_error(&msg.request_id, "SESSION_RESERVED", &message); + continue; } } + } else { + None // ordinary unfenced sends retain their direct-path tolerance }; session.pending_sends.pop_front(); self.send_locked( @@ -10617,6 +10651,141 @@ mod tests { ); } + #[tokio::test] + async fn checkpoint_send_with_stale_fence_is_refused_for_an_already_materialized_session() { + let (mut st, mut rx) = state_with_durable_serve_session().await; + let registry = Arc::new(freshell_ownership::RuntimeOwnershipRegistry::new()); + st.set_ownership(Arc::clone(®istry)); + + let initial = registry.observe(PROVIDER, DURABLE_ID); + let mut attach = attach_msg(DURABLE_ID); + attach.observed_epoch = Some(initial.epoch); + attach.observed_generation = Some(initial.generation); + st.handle_attach(attach).await; + let before = registry.observe(PROVIDER, DURABLE_ID); + assert!(matches!( + before.state, + freshell_ownership::OwnershipState::Live { .. } + )); + + let freshell_ownership::BeginOutcome::Granted { generation } = registry.begin_handoff( + PROVIDER, + DURABLE_ID, + freshell_ownership::RuntimeOwnerKind::Terminal, + "op-checkpoint-send-generation-bump", + None, + "test", + freshell_ownership::now_epoch_ms(), + ) else { + panic!("the generation-bump handoff must grant") + }; + let _ = registry.fail( + PROVIDER, + DURABLE_ID, + "op-checkpoint-send-generation-bump", + generation, + true, + ); + let after = registry.observe(PROVIDER, DURABLE_ID); + assert!(after.generation > before.generation); + assert!(matches!( + after.state, + freshell_ownership::OwnershipState::Live { .. } + )); + while rx.try_recv().is_ok() {} + + let mut send = send_msg_fenced( + DURABLE_ID, + "delayed checkpoint prompt", + Some(before.epoch), + Some(before.generation), + ); + send.request_id = Some("req-stale-checkpoint-opencode".into()); + send.checkpoint_intent = Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "stale-opencode-attempt".into(), + workspace: None, + }); + st.handle_send(send).await; + + let mut saw_reserved = false; + let mut saw_accepted = false; + while let Ok(raw) = rx.try_recv() { + let frame: serde_json::Value = serde_json::from_str(&raw).expect("event frame JSON"); + saw_reserved |= frame.to_string().contains("SESSION_RESERVED"); + saw_accepted |= frame.to_string().contains("freshAgent.send.accepted"); + } + assert!(saw_reserved, "the stale owner fence is refused typed"); + assert!( + !saw_accepted, + "the stale prompt is never accepted for dispatch" + ); + let session_arc = st + .sessions + .lock() + .await + .get(DURABLE_ID) + .cloned() + .expect("the resumed session remains mapped"); + assert!( + session_arc.lock().await.turn_task.is_none(), + "a stale checkpoint prompt never starts a turn" + ); + } + + #[tokio::test] + async fn checkpoint_send_can_materialize_a_new_opencode_session() { + let (tx, mut rx) = tokio::sync::broadcast::channel::<String>(64); + let fresh_agent = FreshAgentState::new(Arc::new("tok".to_string()), Arc::new(tx)); + let (manager, _killed) = started_manager().await; + fresh_agent.set_manager_for_test(manager).await; + let mut st = FreshOpencodeState::new(fresh_agent); + let registry = Arc::new(freshell_ownership::RuntimeOwnershipRegistry::new()); + st.set_ownership(Arc::clone(®istry)); + + st.handle_create(create_msg("req-checkpoint-first-send"), None) + .await; + let placeholder = "freshopencode-req-checkpoint-first-send"; + let observation = registry.observe(PROVIDER, placeholder); + let mut send = send_msg_fenced( + placeholder, + "first checkpoint prompt", + Some(observation.epoch), + Some(observation.generation), + ); + send.request_id = Some("req-checkpoint-first-send-prompt".into()); + send.checkpoint_intent = Some(freshell_protocol::FreshAgentCheckpointIntent { + logical_attempt_id: "first-opencode-checkpoint-attempt".into(), + workspace: None, + }); + st.handle_send(send).await; + + let sessions = st.sessions.lock().await; + let session = sessions + .get(placeholder) + .expect("placeholder remains mapped after materialization") + .lock() + .await; + let durable_id = session + .real_session_id + .as_deref() + .expect("the checkpoint send materializes the provider session"); + assert!(matches!( + registry.observe(PROVIDER, durable_id).state, + freshell_ownership::OwnershipState::Live { .. } + )); + assert!( + session.turn_task.is_some(), + "the first prompt is dispatched" + ); + let frames = std::iter::from_fn(|| rx.try_recv().ok()).collect::<Vec<_>>(); + assert!( + frames + .iter() + .any(|frame| frame.contains("freshAgent.session.materialized")), + "the normal materialization event is emitted" + ); + } + /// Retire-on-kill (delta-review round 5, restore-open-sessions-only): an /// explicit kill is an intentional session END. Killing a MATERIALIZED /// session must (a) retire its durable row `Closed` through the identity diff --git a/crates/freshell-supervisor/src/provider_preparation_tests.rs b/crates/freshell-supervisor/src/provider_preparation_tests.rs index 525ca8dd1..f63072841 100644 --- a/crates/freshell-supervisor/src/provider_preparation_tests.rs +++ b/crates/freshell-supervisor/src/provider_preparation_tests.rs @@ -22,6 +22,12 @@ struct Daemon { consumed_rejection: bool, recovery_probes: Vec<String>, recovery_probe_available: bool, + recovery_probe_gate: Option<Arc<RecoveryProbeGate>>, +} +#[derive(Default)] +struct RecoveryProbeGate { + entered: tokio::sync::Notify, + release: tokio::sync::Notify, } struct Backend(Arc<Mutex<Daemon>>, std::path::PathBuf); @@ -215,11 +221,21 @@ impl RuntimeBackend for Backend { resume_spec: Some(spec), .. } => { - let mut state = state.lock().unwrap(); - state - .recovery_probes - .push(spec.provider_session.native_session_id.clone()); - if state.recovery_probe_available { + let (available, gate) = { + let mut state = state.lock().unwrap(); + state + .recovery_probes + .push(spec.provider_session.native_session_id.clone()); + ( + state.recovery_probe_available, + state.recovery_probe_gate.take(), + ) + }; + if let Some(gate) = gate { + gate.entered.notify_one(); + gate.release.notified().await; + } + if available { HostResult::RecoveryProbe(RecoveryProbe::ResumeReady { evidence_revision: spec.evidence_revision, resume_spec: spec, @@ -529,8 +545,42 @@ async fn live_opencode_probe_coalesces_failed_evidence_reads_then_recovers_befor crate::recovery::LIVE_RESUME_EVIDENCE_RETRY_INTERVAL + Duration::from_millis(25), ) .await; - daemon.lock().unwrap().recovery_probe_available = true; - let retried_live = supervisor.probe_recovery(soul.clone()).await.unwrap(); + let gate = Arc::new(RecoveryProbeGate::default()); + { + let mut daemon = daemon.lock().unwrap(); + daemon.recovery_probe_available = true; + daemon.recovery_probe_gate = Some(gate.clone()); + } + let entered = gate.entered.notified(); + let delayed_probe_supervisor = supervisor.clone(); + let delayed_probe_soul = soul.clone(); + let delayed_probe = tokio::spawn(async move { + delayed_probe_supervisor + .probe_recovery(delayed_probe_soul) + .await + .unwrap() + }); + tokio::time::timeout(Duration::from_secs(5), entered) + .await + .expect("the successful provider-store read reaches its gate"); + + // A newer store observation wins while the older provider read is still + // in flight. Releasing the delayed result must leave this position intact. + let mut newer = + crate::resume_catalog::exact_resume_candidate(&after_first_probe, Some("ses_exact")) + .unwrap() + .expect("the observed native session has an exact candidate"); + newer.durable_position.provider_cursor = Some("position-after-delayed-probe".into()); + let newer = registry + .store_verified_resume_spec(soul.clone(), newer) + .await + .unwrap(); + gate.release.notify_one(); + + let retried_live = tokio::time::timeout(Duration::from_secs(5), delayed_probe) + .await + .expect("the delayed provider-store read completes") + .unwrap(); assert!(matches!(retried_live, RecoveryProbe::ReattachReady { .. })); let durable = registry.recovery_context(soul.clone()).await.unwrap(); assert_eq!( @@ -539,7 +589,13 @@ async fn live_opencode_probe_coalesces_failed_evidence_reads_then_recovers_befor .as_ref() .map(|spec| spec.provider_session.native_session_id.as_str()), Some("ses_exact"), - "a later verified exact-session read must become durable resume evidence" + "a newer exact-session proof stays durable while an older read finishes" + ); + let retained = durable.resume_spec.as_ref().expect("newer proof is saved"); + assert_eq!(retained.evidence_revision, newer.evidence_revision); + assert_eq!( + retained.durable_position.provider_cursor.as_deref(), + Some("position-after-delayed-probe") ); assert_eq!( daemon.lock().unwrap().recovery_probes, diff --git a/crates/freshell-supervisor/src/recovery.rs b/crates/freshell-supervisor/src/recovery.rs index cf0fbe08c..e91184183 100644 --- a/crates/freshell-supervisor/src/recovery.rs +++ b/crates/freshell-supervisor/src/recovery.rs @@ -146,9 +146,22 @@ impl Supervisor { RecoveryProbe::ResumeReady { resume_spec, .. } => { let stored = self .registry - .store_verified_resume_spec(context.soul_id.clone(), *resume_spec) + .store_verified_resume_spec_for_live_incarnation( + context.soul_id.clone(), + context.prior_handle.incarnation_id().clone(), + context.evidence_revision, + *resume_spec, + ) .await .map_err(map_registry)?; + let Some(stored) = stored else { + return Ok(blocked( + RecoveryPath::NativeResume, + RecoveryBlockReason::ProviderUnavailable, + "provider-store evidence was superseded by a newer incarnation or recovery state; retry the probe", + Some(2_000), + )); + }; RecoveryProbe::ResumeReady { evidence_revision: stored.evidence_revision, resume_spec: Box::new(stored), @@ -212,13 +225,21 @@ impl Supervisor { Ok(RecoveryProbe::ResumeReady { resume_spec, .. }) => { match self .registry - .store_verified_resume_spec(context.soul_id.clone(), *resume_spec) + .store_verified_resume_spec_for_live_incarnation( + context.soul_id.clone(), + context.prior_handle.incarnation_id().clone(), + context.evidence_revision, + *resume_spec, + ) .await { - Ok(stored) => serde_json::json!({ + Ok(Some(stored)) => serde_json::json!({ "result": "captured", "evidenceRevision": stored.evidence_revision, }), + Ok(None) => serde_json::json!({ + "result": "superseded", + }), Err(error) => serde_json::json!({ "result": "persist_failed", "error": error.to_string(), diff --git a/crates/freshell-supervisor/src/registry.rs b/crates/freshell-supervisor/src/registry.rs index 4b15cc38e..2b0cc57ef 100644 --- a/crates/freshell-supervisor/src/registry.rs +++ b/crates/freshell-supervisor/src/registry.rs @@ -1354,8 +1354,42 @@ impl Registry { pub async fn store_verified_resume_spec( &self, soul_id: SoulId, - mut resume_spec: ResumeSpec, + resume_spec: ResumeSpec, ) -> Result<ResumeSpec, RegistryError> { + self.store_verified_resume_spec_inner(soul_id, resume_spec, None) + .await? + .ok_or_else(|| { + RegistryError::Integrity( + "unfenced resume evidence update was unexpectedly superseded".into(), + ) + }) + } + + /// Store provider-store evidence only if the incarnation and evidence + /// revision observed before the read are still current. An incarnation may + /// remain usable for a retry after its soul is blocked, including when a + /// failed cleanup left it stopping; `None` means its source or state changed. + pub async fn store_verified_resume_spec_for_live_incarnation( + &self, + soul_id: SoulId, + incarnation_id: IncarnationId, + expected_evidence_revision: u64, + resume_spec: ResumeSpec, + ) -> Result<Option<ResumeSpec>, RegistryError> { + self.store_verified_resume_spec_inner( + soul_id, + resume_spec, + Some((incarnation_id, expected_evidence_revision)), + ) + .await + } + + async fn store_verified_resume_spec_inner( + &self, + soul_id: SoulId, + mut resume_spec: ResumeSpec, + expected_live_source: Option<(IncarnationId, u64)>, + ) -> Result<Option<ResumeSpec>, RegistryError> { self.run_blocking(move |mut conn| { let tx = conn.transaction_with_behavior(TransactionBehavior::Immediate)?; let (provider, store, native, evidence, existing): ( @@ -1380,6 +1414,17 @@ impl Registry { ) .optional()? .ok_or_else(|| RegistryError::UnknownSoul(soul_id.clone()))?; + if let Some((incarnation_id, expected_revision)) = expected_live_source.as_ref() { + let current: bool = tx.query_row( + "SELECT EXISTS(SELECT 1 FROM incarnations i JOIN souls s ON s.soul_id=i.soul_id WHERE s.soul_id=?1 AND i.incarnation_id=?2 AND (i.launch_state='running' OR (i.launch_state='stopping' AND s.recovery_state='blocked')) AND s.desired_state='running' AND s.recovery_state IN ('live','blocked') AND s.recovery_attempt_id IS NULL AND s.evidence_revision=?3)", + params![soul_id.as_str(), incarnation_id.as_str(), expected_revision], + |row| row.get(0), + )?; + if !current { + tx.commit()?; + return Ok(None); + } + } if provider != resume_spec.provider_session.provider || store != resume_spec.provider_session.provider_store_id || native.as_deref() != Some(resume_spec.provider_session.native_session_id.as_str()) @@ -1397,7 +1442,7 @@ impl Registry { && existing.durable_position == resume_spec.durable_position { tx.commit()?; - return Ok(existing); + return Ok(Some(existing)); } } let next_evidence = evidence @@ -1423,7 +1468,7 @@ impl Registry { ], )?; tx.commit()?; - Ok(resume_spec) + Ok(Some(resume_spec)) }) .await } @@ -4514,6 +4559,133 @@ mod tests { assert_eq!(native, None); } + #[tokio::test] + async fn live_resume_evidence_rejects_superseded_incarnation_and_revision() { + let dir = tempfile::tempdir().unwrap(); + let registry = Registry::open(dir.path(), None).unwrap(); + let soul = SoulId::new(); + let prepared = materialize_test_runtime(®istry, soul.clone()).await; + let conn = open_connection(®istry.inner.db_path).unwrap(); + conn.execute( + "UPDATE souls SET provider='opencode',native_session_id='exact-session' WHERE soul_id=?1", + params![soul.as_str()], + ) + .unwrap(); + drop(conn); + + let mut delayed = crate::checkpoints::tests::spec("opencode"); + delayed.durable_position.provider_cursor = Some("older-position".into()); + let mut newer = delayed.clone(); + newer.durable_position.provider_cursor = Some("newer-position".into()); + + let first = registry + .store_verified_resume_spec_for_live_incarnation( + soul.clone(), + prepared.incarnation_id.clone(), + 0, + delayed.clone(), + ) + .await + .unwrap() + .expect("the initial live incarnation and evidence revision are current"); + assert_eq!(first.evidence_revision, 1); + + let wrong_incarnation = registry + .store_verified_resume_spec_for_live_incarnation( + soul.clone(), + IncarnationId::new(), + first.evidence_revision, + delayed.clone(), + ) + .await + .unwrap(); + assert!( + wrong_incarnation.is_none(), + "a probe from another incarnation cannot publish evidence" + ); + + registry + .mark_stop_outcome( + prepared.incarnation_id.clone(), + StopOutcome::TerminationUnconfirmed, + ) + .await + .unwrap(); + registry + .mark_recovery_blocked( + soul.clone(), + None, + RecoveryBlockReason::ProviderUnavailable, + vec!["synthetic provider store outage".into()], + ) + .await + .unwrap(); + let newer = registry + .store_verified_resume_spec_for_live_incarnation( + soul.clone(), + prepared.incarnation_id.clone(), + first.evidence_revision, + newer, + ) + .await + .unwrap() + .expect("a retry can publish evidence after blocked cleanup of the same incarnation"); + assert_eq!( + newer.durable_position.provider_cursor.as_deref(), + Some("newer-position") + ); + + let late = registry + .store_verified_resume_spec_for_live_incarnation( + soul.clone(), + prepared.incarnation_id.clone(), + 0, + delayed.clone(), + ) + .await + .unwrap(); + assert!(late.is_none(), "a late probe cannot replace newer evidence"); + + let context = registry.recovery_context(soul.clone()).await.unwrap(); + let retained = context.resume_spec.expect("newer evidence remains saved"); + assert_eq!(retained.evidence_revision, newer.evidence_revision); + assert_eq!( + retained.durable_position.provider_cursor.as_deref(), + Some("newer-position") + ); + + let start = registry + .begin_recovery( + soul.clone(), + RecoveryTrigger::ProviderExit, + RecoveryPath::NativeResume, + ) + .await + .unwrap(); + let during_recovery = registry + .store_verified_resume_spec_for_live_incarnation( + soul.clone(), + start.context.prior_handle.incarnation_id().clone(), + newer.evidence_revision, + delayed, + ) + .await + .unwrap(); + assert!( + during_recovery.is_none(), + "an active recovery attempt cannot publish proof" + ); + let final_context = registry.recovery_context(soul).await.unwrap(); + assert_eq!(final_context.recovery_state, RecoveryState::Recovering); + assert_eq!( + final_context + .resume_spec + .as_ref() + .and_then(|spec| spec.durable_position.provider_cursor.as_deref()), + Some("newer-position") + ); + } + #[tokio::test] async fn stop_intent_fences_a_recovery_attempt_before_replacement() { let dir = tempfile::tempdir().unwrap(); From 8ab2802e41ca519e4161eff7614a70f3e4704e1d Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 14:35:46 -0700 Subject: [PATCH 489/493] fix checkpoint acceptance cleanup race --- .../managed-provider-environment-target.ts | 39 ++++++++++++++++++- ...anaged-provider-environment-target.test.ts | 17 ++++++++ 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index a5c25b0ed..f590ab651 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -854,6 +854,33 @@ function dockerMaybe(args: string[], timeout = 10_000): string | undefined { return result.stdout.trim() } +export async function settleExactDockerContainerRemoval( + containerId: string, + remove: () => void, + listMatching: () => string[], + wait: (milliseconds: number) => Promise<void> = delay, + timeoutMs = 15_000, +): Promise<void> { + try { + remove() + } catch (error) { + const message = error instanceof Error ? error.message : String(error) + if (!message.includes(`removal of container ${containerId} is already in progress`)) throw error + } + + const deadline = Date.now() + timeoutMs + while (true) { + const matching = listMatching() + assert(matching.length === 0 || (matching.length === 1 && matching[0] === containerId), + 'Only the exact owned Docker container may remain during removal') + if (matching.length === 0) return + const remainingMs = deadline - Date.now() + if (remainingMs <= 0) break + await wait(Math.min(250, remainingMs)) + } + throw new Error(`Owned Docker container ${containerId} did not disappear after removal`) +} + function cancelOwnedCloudExecution(executionName: string, candidateSha: string): Record<string, unknown> { assert.match(executionName, new RegExp(`^freshell-e2e-${candidateSha.slice(0, 12)}-[a-z0-9-]+$`), 'Refusing to cancel a Cloud execution not created for this checkpoint candidate') @@ -1396,7 +1423,11 @@ export async function runCodeCheckpointTarget() { docker(['container', 'stop', '--timeout', '10', forwarderContainerId], 20_000) } const remaining = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) - if (remaining.length) docker(['container', 'rm', '--force', forwarderContainerId], 20_000) + if (remaining.length) await settleExactDockerContainerRemoval( + forwarderContainerId, + () => { docker(['container', 'rm', '--force', forwarderContainerId], 20_000) }, + () => docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean), + ) assert.deepEqual(docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean), []) return { containerId: forwarderContainerId, settled: true } }) @@ -1409,7 +1440,11 @@ export async function runCodeCheckpointTarget() { docker(['container', 'stop', '--time', '10', containerId], 20_000) } const remaining = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) - if (remaining.length) docker(['container', 'rm', '--force', containerId], 20_000) + if (remaining.length) await settleExactDockerContainerRemoval( + containerId, + () => { docker(['container', 'rm', '--force', containerId], 20_000) }, + () => docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean), + ) assert.deepEqual(docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean), []) return { containerId, settled: true } }) diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index 64f8c0262..cf9b55f92 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -18,6 +18,7 @@ import { processIdentity, resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, + settleExactDockerContainerRemoval, stopExact, workspaceCwdAliasMcpWrapperSource, workspaceCwdAliasReplacementProcessProbeSource, @@ -177,6 +178,22 @@ describe('managed provider environment target mode', () => { expect(resolveManagedProviderEnvironmentTargetMode(['--checkpoint'])).toBe('code-checkpoint') }) + it('waits for an exact container whose automatic Docker removal is already in progress', async () => { + const containerId = 'a'.repeat(64) + const snapshots = [[containerId], [containerId], []] + const waits: number[] = [] + + await expect(settleExactDockerContainerRemoval( + containerId, + () => { throw new Error(`Error response from daemon: removal of container ${containerId} is already in progress`) }, + () => snapshots.shift() ?? [], + async milliseconds => { waits.push(milliseconds) }, + 1_000, + )).resolves.toBeUndefined() + + expect(waits).toEqual([250, 250]) + }) + it('keeps the OpenCode MCP package and parent dependency outside the mounted workspace', () => { const fixture = managedEnvironmentFixtureLayout('/tmp/managed-environment-fixture') const packageRelativeToWorkspace = path.relative(fixture.workspace, fixture.mcpPackage) From fa44c434039d95f309cb715c6475054fc18cb359 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 14:51:35 -0700 Subject: [PATCH 490/493] fix checkpoint target auto-removal races --- .../managed-provider-environment-target.ts | 106 +++++++++++------- ...anaged-provider-environment-target.test.ts | 34 ++++++ 2 files changed, 102 insertions(+), 38 deletions(-) diff --git a/scripts/testing/managed-provider-environment-target.ts b/scripts/testing/managed-provider-environment-target.ts index f590ab651..16157f2d2 100644 --- a/scripts/testing/managed-provider-environment-target.ts +++ b/scripts/testing/managed-provider-environment-target.ts @@ -861,24 +861,84 @@ export async function settleExactDockerContainerRemoval( wait: (milliseconds: number) => Promise<void> = delay, timeoutMs = 15_000, ): Promise<void> { + const result = await runExactDockerContainerOperation(containerId, remove, listMatching, wait, timeoutMs) + if (result.kind === 'gone') return + await waitForExactDockerContainerAbsence(containerId, listMatching, wait, timeoutMs) +} + +type ExactDockerContainerOperationResult<T> = { kind: 'completed'; value: T } | { kind: 'gone' } + +export async function runExactDockerContainerOperation<T>( + containerId: string, + operation: () => T, + listMatching: () => string[], + wait: (milliseconds: number) => Promise<void> = delay, + timeoutMs = 15_000, +): Promise<ExactDockerContainerOperationResult<T>> { try { - remove() + return { kind: 'completed', value: operation() } } catch (error) { const message = error instanceof Error ? error.message : String(error) - if (!message.includes(`removal of container ${containerId} is already in progress`)) throw error + if (message.includes(`removal of container ${containerId} is already in progress`)) { + await waitForExactDockerContainerAbsence(containerId, listMatching, wait, timeoutMs) + return { kind: 'gone' } + } + + const matching = listMatching() + assert(matching.length === 0 || (matching.length === 1 && matching[0] === containerId), + 'Only the exact owned Docker container may remain during cleanup') + if (matching.length === 0) return { kind: 'gone' } + throw error } +} +async function waitForExactDockerContainerAbsence( + containerId: string, + listMatching: () => string[], + wait: (milliseconds: number) => Promise<void>, + timeoutMs: number, +): Promise<void> { const deadline = Date.now() + timeoutMs while (true) { const matching = listMatching() assert(matching.length === 0 || (matching.length === 1 && matching[0] === containerId), - 'Only the exact owned Docker container may remain during removal') + 'Only the exact owned Docker container may remain during cleanup') if (matching.length === 0) return const remainingMs = deadline - Date.now() if (remainingMs <= 0) break await wait(Math.min(250, remainingMs)) } - throw new Error(`Owned Docker container ${containerId} did not disappear after removal`) + throw new Error(`Owned Docker container ${containerId} did not disappear after cleanup`) +} + +async function cleanupOwnedDockerContainer( + containerId: string, + runId: string, + stopArgs: string[], +): Promise<{ containerId: string; settled: true }> { + const listMatching = () => docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) + const matching = listMatching() + if (matching.length) { + assert.deepEqual(matching, [containerId], 'Only the exact owned checkpoint container may be stopped') + const inspected = await runExactDockerContainerOperation(containerId, + () => docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', containerId]), + listMatching) + if (inspected.kind === 'gone') return { containerId, settled: true } + assert.equal(inspected.value, `${containerId} ${runId}`, 'Refusing to clean up an unowned checkpoint container') + + const stopped = await runExactDockerContainerOperation(containerId, + () => docker([...stopArgs, containerId], 20_000), listMatching) + if (stopped.kind === 'gone') return { containerId, settled: true } + } + + const remaining = listMatching() + if (remaining.length) await settleExactDockerContainerRemoval( + containerId, + () => { docker(['container', 'rm', '--force', containerId], 20_000) }, + listMatching, + ) + assert.deepEqual(listMatching(), []) + return { containerId, settled: true } } function cancelOwnedCloudExecution(executionName: string, candidateSha: string): Record<string, unknown> { @@ -1414,40 +1474,10 @@ export async function runCodeCheckpointTarget() { } await attempt('browserJoin', () => stopExact(browser, browserIdentity, browserJoin, true)) await attempt('tunnelJoin', () => stopExact(tunnel, tunnelIdentity, tunnelJoin, true)) - if (forwarderContainerId) await attempt('forwarderContainer', async () => { - const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) - if (matching.length) { - assert.deepEqual(matching, [forwarderContainerId], 'Only the exact owned checkpoint forwarder may be stopped') - const details = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', forwarderContainerId]) - assert.equal(details, `${forwarderContainerId} ${runId}`, 'Refusing to clean up an unowned checkpoint forwarder') - docker(['container', 'stop', '--timeout', '10', forwarderContainerId], 20_000) - } - const remaining = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean) - if (remaining.length) await settleExactDockerContainerRemoval( - forwarderContainerId, - () => { docker(['container', 'rm', '--force', forwarderContainerId], 20_000) }, - () => docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean), - ) - assert.deepEqual(docker(['ps', '-aq', '--no-trunc', '--filter', `id=${forwarderContainerId}`]).split('\n').filter(Boolean), []) - return { containerId: forwarderContainerId, settled: true } - }) - if (containerId) await attempt('container', async () => { - const matching = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) - if (matching.length) { - assert.deepEqual(matching, [containerId], 'Only the exact owned checkpoint container may be stopped') - const details = docker(['container', 'inspect', '--format', '{{.Id}} {{index .Config.Labels "freshell.run-id"}}', containerId]) - assert.equal(details, `${containerId} ${runId}`, 'Refusing to clean up an unowned checkpoint container') - docker(['container', 'stop', '--time', '10', containerId], 20_000) - } - const remaining = docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean) - if (remaining.length) await settleExactDockerContainerRemoval( - containerId, - () => { docker(['container', 'rm', '--force', containerId], 20_000) }, - () => docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean), - ) - assert.deepEqual(docker(['ps', '-aq', '--no-trunc', '--filter', `id=${containerId}`]).split('\n').filter(Boolean), []) - return { containerId, settled: true } - }) + if (forwarderContainerId) await attempt('forwarderContainer', () => + cleanupOwnedDockerContainer(forwarderContainerId, runId, ['container', 'stop', '--timeout', '10'])) + if (containerId) await attempt('container', () => + cleanupOwnedDockerContainer(containerId, runId, ['container', 'stop', '--time', '10'])) else if (dockerRun && dockerRunIdentity && dockerRunJoin) await attempt('containerIdentity', async () => { const named = docker(['ps', '-aq', '--no-trunc', '--filter', `name=^/${containerName}$`]).split('\n').filter(Boolean) assert.deepEqual(named, [], 'Container without a published owned ID remains unknown; retain its fixture') diff --git a/test/unit/tooling/testing/managed-provider-environment-target.test.ts b/test/unit/tooling/testing/managed-provider-environment-target.test.ts index cf9b55f92..d5e4b831a 100644 --- a/test/unit/tooling/testing/managed-provider-environment-target.test.ts +++ b/test/unit/tooling/testing/managed-provider-environment-target.test.ts @@ -18,6 +18,7 @@ import { processIdentity, resolveManagedProviderEnvironmentTargetMode, runManagedProviderEnvironmentTarget, + runExactDockerContainerOperation, settleExactDockerContainerRemoval, stopExact, workspaceCwdAliasMcpWrapperSource, @@ -194,6 +195,39 @@ describe('managed provider environment target mode', () => { expect(waits).toEqual([250, 250]) }) + it('accepts a failed cleanup command only after the exact Docker container is gone', async () => { + const containerId = 'b'.repeat(64) + const error = new Error(`No such container: ${containerId}`) + + await expect(runExactDockerContainerOperation( + containerId, + () => { throw error }, + () => [], + )).resolves.toEqual({ kind: 'gone' }) + + await expect(runExactDockerContainerOperation( + containerId, + () => { throw new Error('Docker daemon is unavailable') }, + () => [containerId], + )).rejects.toThrow('Docker daemon is unavailable') + }) + + it('waits for the exact ID to disappear when a cleanup command races automatic removal', async () => { + const containerId = 'c'.repeat(64) + const snapshots = [[containerId], [containerId], []] + const waits: number[] = [] + + await expect(runExactDockerContainerOperation( + containerId, + () => { throw new Error(`Error response from daemon: removal of container ${containerId} is already in progress`) }, + () => snapshots.shift() ?? [], + async milliseconds => { waits.push(milliseconds) }, + 1_000, + )).resolves.toEqual({ kind: 'gone' }) + + expect(waits).toEqual([250, 250]) + }) + it('keeps the OpenCode MCP package and parent dependency outside the mounted workspace', () => { const fixture = managedEnvironmentFixtureLayout('/tmp/managed-environment-fixture') const packageRelativeToWorkspace = path.relative(fixture.workspace, fixture.mcpPackage) From 0bfcc0b4dd2b17597c1a2b01bded03008685f44c Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 15:22:43 -0700 Subject: [PATCH 491/493] stabilize checkpoint timeout tests --- crates/freshell-server/src/checkpoints.rs | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index 63bf1dd3d..faef7cefc 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -9153,7 +9153,9 @@ mod tests { let workspace = tempfile::tempdir().unwrap(); initialize_checkpoint_workspace(workspace.path()).await; let service = checkpoint_test_service(home.path()); - *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(40)); + // Leave headroom for workspace resolution under parallel test load so + // the injected capture delay is the phase that times out. + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(2)); service .test_capture_delays .lock() @@ -9275,7 +9277,9 @@ mod tests { let workspace = tempfile::tempdir().unwrap(); initialize_checkpoint_workspace(workspace.path()).await; let service = checkpoint_test_service(home.path()); - *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(40)); + // Leave headroom for workspace resolution under parallel test load so + // the injected capture delay is the phase that times out. + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(2)); service .test_capture_delays .lock() @@ -9490,7 +9494,8 @@ mod tests { let workspace = tempfile::tempdir().unwrap(); initialize_checkpoint_workspace(workspace.path()).await; let service = checkpoint_test_service(home.path()); - *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(50)); + // The total budget includes workspace resolution before lock waiting. + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(2)); let holder = service .lock_workspace(workspace.path(), Duration::from_secs(1)) .await @@ -9566,12 +9571,12 @@ mod tests { let workspace = tempfile::tempdir().unwrap(); initialize_checkpoint_workspace(workspace.path()).await; let service = checkpoint_test_service(home.path()); - *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_millis(50)); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(2)); service .test_capture_delays .lock() .unwrap() - .push_back(Duration::from_millis(250)); + .push_back(Duration::from_secs(3)); let holder_workspace = ResolvedCheckpointWorkspace { path: canonical_workspace(workspace.path()).await.unwrap(), provenance: "directory-picker".to_string(), @@ -9590,7 +9595,7 @@ mod tests { &holder_workspace, "healthy holder", "healthy-holder-request", - Duration::from_secs(2), + Duration::from_secs(4), None, ) .await; @@ -9610,7 +9615,7 @@ mod tests { assert_eq!(waiter_dispatches, 1); assert!(service.active_cooldown(workspace.path()).is_none()); - match tokio::time::timeout(Duration::from_secs(3), holder) + match tokio::time::timeout(Duration::from_secs(5), holder) .await .expect("healthy capture should finish after the independent waiter timed out") .expect("healthy holder task should complete") From a3b93606afbf184599e9847c03c38b7a2f98473f Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 16:02:39 -0700 Subject: [PATCH 492/493] add headroom to checkpoint timeout test --- crates/freshell-server/src/checkpoints.rs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/crates/freshell-server/src/checkpoints.rs b/crates/freshell-server/src/checkpoints.rs index faef7cefc..96a320aef 100644 --- a/crates/freshell-server/src/checkpoints.rs +++ b/crates/freshell-server/src/checkpoints.rs @@ -9595,7 +9595,7 @@ mod tests { &holder_workspace, "healthy holder", "healthy-holder-request", - Duration::from_secs(4), + Duration::from_secs(15), None, ) .await; @@ -9615,7 +9615,7 @@ mod tests { assert_eq!(waiter_dispatches, 1); assert!(service.active_cooldown(workspace.path()).is_none()); - match tokio::time::timeout(Duration::from_secs(5), holder) + match tokio::time::timeout(Duration::from_secs(20), holder) .await .expect("healthy capture should finish after the independent waiter timed out") .expect("healthy holder task should complete") @@ -9630,7 +9630,7 @@ mod tests { 1 ); - *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(3)); + *service.test_auto_budget_override.lock().unwrap() = Some(Duration::from_secs(15)); let (next_pane, next_dispatches, _) = run_checkpoint_policy_send( Arc::clone(&service), checkpoint_policy_send( From d1b64e18d3b9b3dfee071e75b489f57bb04dca40 Mon Sep 17 00:00:00 2001 From: Dan Shapiro <3732858+danshapiro@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:08:47 -0700 Subject: [PATCH 493/493] test: report managed fresh-agent create failures --- .../runtime-opencode-home-file-reference-rust.spec.ts | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/test/e2e-browser/specs/runtime-opencode-home-file-reference-rust.spec.ts b/test/e2e-browser/specs/runtime-opencode-home-file-reference-rust.spec.ts index 1f7819bf2..a67d78c4a 100644 --- a/test/e2e-browser/specs/runtime-opencode-home-file-reference-rust.spec.ts +++ b/test/e2e-browser/specs/runtime-opencode-home-file-reference-rust.spec.ts @@ -138,7 +138,13 @@ test.describe.serial('freshopencode operator-home {file:} references', () => { cwd: workspace, model: 'openai/gpt-5.6-luna', })) - if (!created?.paneId || !created?.sessionId) throw new Error('MCP new-tab did not create a fresh-agent pane') + if (!created?.paneId || !created?.sessionId) { + throw new Error(`MCP new-tab did not create a fresh-agent pane: ${JSON.stringify({ + status: created?.status, + error: created?.error, + message: created?.message, + })}`) + } const paneId = created.paneId as string const sessionId = created.sessionId as string const tabId = created.tabId as string